2023-09-27T23:00:19.269 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-27T23:00:19.269 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi --verbose --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461 -- /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/orig.config.yaml 2023-09-27T23:00:19.312 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:00:19.392 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461 branch: wip-yuri-testing-2023-09-27-0906-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7403461' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri-testing-2023-09-27-0906-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-09-27-0906-quincy sha1: 471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3538 sha1: 471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri-testing-2023-09-27-0906-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-27_19:23:53 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-27T23:00:19.393 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/qa; will attempt to use it 2023-09-27T23:00:19.393 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/qa/tasks 2023-09-27T23:00:19.393 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-27T23:00:19.395 INFO:teuthology.task.internal:Checking packages... 2023-09-27T23:00:19.414 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28' 2023-09-27T23:00:19.414 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-27T23:00:19.415 INFO:teuthology.packaging:ref: None 2023-09-27T23:00:19.415 INFO:teuthology.packaging:tag: None 2023-09-27T23:00:19.415 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-09-27-0906-quincy 2023-09-27T23:00:19.415 INFO:teuthology.packaging:sha1: 471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 2023-09-27T23:00:19.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri-testing-2023-09-27-0906-quincy 2023-09-27T23:00:19.636 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:00:19.636 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-27T23:00:19.644 INFO:teuthology.task.internal:no buildpackages task found 2023-09-27T23:00:19.644 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-27T23:00:19.664 INFO:teuthology.task.internal:Saving configuration 2023-09-27T23:00:19.678 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-27T23:00:19.684 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-27T23:00:19.708 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-27 22:53:49.707654', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-27T23:00:19.730 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-27 22:53:49.708687', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-27T23:00:19.730 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-27T23:00:19.737 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-09-27T23:00:19.737 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-09-27T23:00:19.737 INFO:teuthology.run_tasks:Running task console_log... 2023-09-27T23:00:19.806 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1ebcaff040>, signals=[15]) 2023-09-27T23:00:19.807 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-27T23:00:19.813 INFO:teuthology.task.internal:Opening connections... 2023-09-27T23:00:19.813 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:00:19.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:00:19.862 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:00:19.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:00:19.906 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-27T23:00:19.913 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-09-27T23:00:19.942 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-09-27T23:00:19.942 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-09-27T23:00:19.989 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu" 2023-09-27T23:00:19.989 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="20.04" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=focal 2023-09-27T23:00:19.990 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=focal 2023-09-27T23:00:19.991 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-09-27T23:00:20.020 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-09-27T23:00:20.026 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-09-27T23:00:20.026 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Ubuntu" 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:ID=ubuntu 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE=debian 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="20.04" 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-27T23:00:20.073 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-27T23:00:20.074 INFO:teuthology.orchestra.run.smithi123.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-27T23:00:20.074 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_CODENAME=focal 2023-09-27T23:00:20.074 INFO:teuthology.orchestra.run.smithi123.stdout:UBUNTU_CODENAME=focal 2023-09-27T23:00:20.074 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-09-27T23:00:20.095 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-27T23:00:20.104 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-27T23:00:20.132 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-27T23:00:20.132 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-09-27T23:00:20.134 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-09-27T23:00:20.140 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-27T23:00:20.146 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-27T23:00:20.146 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-09-27T23:00:20.148 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-09-27T23:00:20.163 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-27T23:00:20.219 INFO:teuthology.run_tasks:Running task kernel... 2023-09-27T23:00:20.232 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-27T23:00:20.232 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-27T23:00:20.232 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-27T23:00:20.233 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-27T23:00:20.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:00:20.238 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-09-27T23:00:20.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:00:20.239 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2023-09-27T23:00:20.245 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-124-generic 2023-09-27T23:00:20.245 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.4.0-124-generic 2023-09-27T23:00:20.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2023-09-27T23:00:20.246 INFO:teuthology.orchestra.run.smithi123.stdout:5.4.0-124-generic 2023-09-27T23:00:20.247 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.4.0-124-generic 2023-09-27T23:00:20.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get clean 2023-09-27T23:00:20.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-09-27T23:00:20.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get update 2023-09-27T23:00:20.512 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-27T23:00:20.520 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-27T23:00:20.555 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-27T23:00:20.555 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-27T23:00:20.555 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-27T23:00:20.593 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-27T23:00:20.614 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-27T23:00:20.632 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-27T23:00:20.879 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-27T23:00:20.926 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-27T23:00:21.004 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-27T23:00:21.005 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-27T23:00:21.015 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [468 kB] 2023-09-27T23:00:21.019 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-27T23:00:21.019 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.2 kB] 2023-09-27T23:00:21.020 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,334 kB] 2023-09-27T23:00:21.039 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-27T23:00:21.042 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [568 B] 2023-09-27T23:00:21.042 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-27T23:00:21.048 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-27T23:00:21.050 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-27T23:00:21.050 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-27T23:00:21.050 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,211 kB] 2023-09-27T23:00:21.060 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-27T23:00:21.063 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-27T23:00:21.065 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-27T23:00:21.065 INFO:teuthology.orchestra.run.smithi123.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [32.8 kB] 2023-09-27T23:00:21.066 INFO:teuthology.orchestra.run.smithi123.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-27T23:00:21.066 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-27T23:00:21.067 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-27T23:00:21.067 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-27T23:00:21.067 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-27T23:00:21.067 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-27T23:00:21.068 INFO:teuthology.orchestra.run.smithi123.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-27T23:00:21.068 INFO:teuthology.orchestra.run.smithi123.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-27T23:00:21.068 INFO:teuthology.orchestra.run.smithi123.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-27T23:00:21.072 INFO:teuthology.orchestra.run.smithi123.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-27T23:00:21.078 INFO:teuthology.orchestra.run.smithi123.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-27T23:00:21.080 INFO:teuthology.orchestra.run.smithi123.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-27T23:00:21.081 INFO:teuthology.orchestra.run.smithi123.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.5 kB] 2023-09-27T23:00:21.081 INFO:teuthology.orchestra.run.smithi123.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-27T23:00:21.085 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-27T23:00:21.085 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-27T23:00:21.086 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-27T23:00:21.086 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-27T23:00:21.098 INFO:teuthology.orchestra.run.smithi123.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-27T23:00:21.098 INFO:teuthology.orchestra.run.smithi123.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-27T23:00:21.186 INFO:teuthology.orchestra.run.smithi123.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-27T23:00:21.205 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-27T23:00:21.231 INFO:teuthology.orchestra.run.smithi123.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [468 kB] 2023-09-27T23:00:21.235 INFO:teuthology.orchestra.run.smithi123.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-27T23:00:21.235 INFO:teuthology.orchestra.run.smithi123.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,334 kB] 2023-09-27T23:00:21.250 INFO:teuthology.orchestra.run.smithi123.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.2 kB] 2023-09-27T23:00:21.250 INFO:teuthology.orchestra.run.smithi123.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-27T23:00:21.252 INFO:teuthology.orchestra.run.smithi123.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [568 B] 2023-09-27T23:00:21.253 INFO:teuthology.orchestra.run.smithi123.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-27T23:00:21.262 INFO:teuthology.orchestra.run.smithi123.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-27T23:00:21.298 INFO:teuthology.orchestra.run.smithi123.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-27T23:00:21.302 INFO:teuthology.orchestra.run.smithi123.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-27T23:00:21.303 INFO:teuthology.orchestra.run.smithi123.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-27T23:00:21.304 INFO:teuthology.orchestra.run.smithi123.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-27T23:00:21.304 INFO:teuthology.orchestra.run.smithi123.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-27T23:00:21.305 INFO:teuthology.orchestra.run.smithi123.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-27T23:00:21.305 INFO:teuthology.orchestra.run.smithi123.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-27T23:00:21.306 INFO:teuthology.orchestra.run.smithi123.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-27T23:00:21.312 INFO:teuthology.orchestra.run.smithi123.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-27T23:00:21.319 INFO:teuthology.orchestra.run.smithi123.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-27T23:00:21.339 INFO:teuthology.orchestra.run.smithi123.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-27T23:00:21.339 INFO:teuthology.orchestra.run.smithi123.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-27T23:00:21.529 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-27T23:00:21.717 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-27T23:00:21.731 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-27T23:00:21.732 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,211 kB] 2023-09-27T23:00:21.799 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [32.8 kB] 2023-09-27T23:00:21.800 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-27T23:00:21.806 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-27T23:00:21.806 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-27T23:00:21.817 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-27T23:00:22.019 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-27T23:00:22.435 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-27T23:00:22.441 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-27T23:00:22.443 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.5 kB] 2023-09-27T23:00:22.451 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-27T23:00:22.453 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-27T23:00:24.885 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 17.4 MB in 2s (7,091 kB/s) 2023-09-27T23:00:25.112 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 17.4 MB in 3s (6,662 kB/s) 2023-09-27T23:00:26.298 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:00:26.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-27T23:00:26.399 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:00:26.537 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:00:26.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-27T23:00:26.577 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2023-09-27T23:00:26.577 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2023-09-27T23:00:26.633 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:00:26.748 INFO:teuthology.orchestra.run.smithi123.stdout:The following additional packages will be installed: 2023-09-27T23:00:26.749 INFO:teuthology.orchestra.run.smithi123.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-27T23:00:26.749 INFO:teuthology.orchestra.run.smithi123.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-27T23:00:26.749 INFO:teuthology.orchestra.run.smithi123.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-27T23:00:26.750 INFO:teuthology.orchestra.run.smithi123.stdout:Suggested packages: 2023-09-27T23:00:26.751 INFO:teuthology.orchestra.run.smithi123.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-27T23:00:26.751 INFO:teuthology.orchestra.run.smithi123.stdout:Recommended packages: 2023-09-27T23:00:26.751 INFO:teuthology.orchestra.run.smithi123.stdout: thermald 2023-09-27T23:00:26.809 INFO:teuthology.orchestra.run.smithi123.stdout:The following NEW packages will be installed: 2023-09-27T23:00:26.809 INFO:teuthology.orchestra.run.smithi123.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-27T23:00:26.809 INFO:teuthology.orchestra.run.smithi123.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-27T23:00:26.809 INFO:teuthology.orchestra.run.smithi123.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-27T23:00:26.811 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages will be upgraded: 2023-09-27T23:00:26.811 INFO:teuthology.orchestra.run.smithi123.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-27T23:00:26.836 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-09-27T23:00:26.837 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-09-27T23:00:26.868 INFO:teuthology.orchestra.run.smithi123.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-27T23:00:26.869 INFO:teuthology.orchestra.run.smithi123.stdout:Need to get 77.1 MB of archives. 2023-09-27T23:00:26.869 INFO:teuthology.orchestra.run.smithi123.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-27T23:00:26.869 INFO:teuthology.orchestra.run.smithi123.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-27T23:00:27.023 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-09-27T23:00:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-27T23:00:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-27T23:00:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-27T23:00:27.024 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-09-27T23:00:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-27T23:00:27.024 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2023-09-27T23:00:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: thermald 2023-09-27T23:00:27.080 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-09-27T23:00:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-27T23:00:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-27T23:00:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-27T23:00:27.081 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2023-09-27T23:00:27.082 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-27T23:00:27.137 INFO:teuthology.orchestra.run.smithi064.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-27T23:00:27.137 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 77.1 MB of archives. 2023-09-27T23:00:27.137 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-27T23:00:27.137 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-27T23:00:27.233 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-27T23:00:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-27T23:00:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-27T23:00:27.534 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-27T23:00:28.033 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-27T23:00:28.033 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-27T23:00:28.033 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-27T23:00:28.098 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-27T23:00:28.098 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-27T23:00:28.099 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-27T23:00:28.128 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-27T23:00:28.139 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-27T23:00:28.217 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-27T23:00:28.230 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-27T23:00:28.933 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 77.1 MB in 1s (73.4 MB/s) 2023-09-27T23:00:28.942 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 77.1 MB in 1s (54.6 MB/s) 2023-09-27T23:00:29.085 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-27T23:00:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-27T23:00:30.801 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-27T23:00:30.806 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:30.847 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-27T23:00:30.852 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:30.899 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:30.922 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:32.899 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-27T23:00:32.905 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-27T23:00:32.913 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:32.917 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:33.010 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:33.013 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:34.107 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-27T23:00:34.110 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-27T23:00:34.120 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:34.123 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:34.157 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:34.160 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:38.777 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-27T23:00:38.871 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-27T23:00:38.902 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-27T23:00:38.997 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-27T23:00:39.251 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-27T23:00:39.324 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-27T23:00:39.372 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-27T23:00:39.451 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-27T23:00:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-27T23:00:39.732 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-27T23:00:39.775 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-27T23:00:39.804 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-27T23:00:39.820 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-27T23:00:39.866 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-27T23:00:45.009 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-27T23:00:45.032 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:45.067 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:45.645 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-27T23:00:45.666 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-27T23:00:45.703 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:48.368 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-27T23:00:48.659 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-27T23:00:48.675 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-27T23:00:48.804 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-27T23:00:48.830 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:49.138 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:50.325 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-27T23:00:50.343 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-27T23:00:50.460 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:50.469 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:51.625 INFO:teuthology.orchestra.run.smithi123.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-27T23:00:51.625 INFO:teuthology.orchestra.run.smithi123.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-27T23:00:51.665 INFO:teuthology.orchestra.run.smithi064.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-27T23:00:51.665 INFO:teuthology.orchestra.run.smithi064.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-27T23:00:51.754 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:51.837 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:51.880 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:51.988 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:53.206 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-27T23:00:53.332 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-27T23:00:53.458 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-27T23:00:53.584 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-27T23:00:53.670 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-27T23:00:53.670 INFO:teuthology.orchestra.run.smithi064.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-27T23:00:53.792 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-27T23:00:53.917 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-27T23:00:54.044 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-27T23:00:54.118 INFO:teuthology.orchestra.run.smithi123.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-27T23:00:54.118 INFO:teuthology.orchestra.run.smithi123.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-27T23:01:16.196 INFO:teuthology.orchestra.run.smithi123.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-27T23:01:16.196 INFO:teuthology.orchestra.run.smithi123.stdout:Sourcing file `/etc/default/grub' 2023-09-27T23:01:16.207 INFO:teuthology.orchestra.run.smithi123.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-27T23:01:16.219 INFO:teuthology.orchestra.run.smithi123.stdout:Generating grub configuration file ... 2023-09-27T23:01:16.287 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-27T23:01:16.287 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub' 2023-09-27T23:01:16.293 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-27T23:01:16.321 INFO:teuthology.orchestra.run.smithi064.stdout:Generating grub configuration file ... 2023-09-27T23:01:16.549 INFO:teuthology.orchestra.run.smithi123.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-27T23:01:16.573 INFO:teuthology.orchestra.run.smithi123.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-27T23:01:16.629 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-27T23:01:16.656 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-27T23:01:16.929 INFO:teuthology.orchestra.run.smithi123.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-27T23:01:16.941 INFO:teuthology.orchestra.run.smithi123.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-27T23:01:17.004 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-27T23:01:17.016 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-27T23:01:19.656 INFO:teuthology.orchestra.run.smithi123.stdout:done 2023-09-27T23:01:19.733 INFO:teuthology.orchestra.run.smithi064.stdout:done 2023-09-27T23:01:20.057 DEBUG:teuthology.orchestra.run.smithi123:> dpkg -s linux-image-generic 2023-09-27T23:01:20.108 INFO:teuthology.orchestra.run.smithi123.stdout:Package: linux-image-generic 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Status: install ok installed 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Priority: optional 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Section: kernel 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Installed-Size: 20 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Maintainer: Ubuntu Kernel Team 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: amd64 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Source: linux-meta 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Version: 5.4.0.163.160 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-27T23:01:20.109 INFO:teuthology.orchestra.run.smithi123.stdout:Recommends: thermald 2023-09-27T23:01:20.110 INFO:teuthology.orchestra.run.smithi123.stdout:Description: Generic Linux kernel image 2023-09-27T23:01:20.110 INFO:teuthology.orchestra.run.smithi123.stdout: This package will always depend on the latest generic kernel image 2023-09-27T23:01:20.110 INFO:teuthology.orchestra.run.smithi123.stdout: available. 2023-09-27T23:01:20.110 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-27T23:01:20.110 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-27T23:01:20.110 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-27T23:01:20.110 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-09-27T23:01:20.110 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2023-09-27T23:01:20.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get clean 2023-09-27T23:01:20.137 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2023-09-27T23:01:20.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get update 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.163.160 2023-09-27T23:01:20.206 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-27T23:01:20.207 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-27T23:01:20.207 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2023-09-27T23:01:20.207 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2023-09-27T23:01:20.207 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2023-09-27T23:01:20.207 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2023-09-27T23:01:20.207 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-27T23:01:20.207 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-27T23:01:20.207 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-27T23:01:20.207 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-27T23:01:20.207 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2023-09-27T23:01:20.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2023-09-27T23:01:20.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-09-27T23:01:20.418 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-27T23:01:20.423 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-27T23:01:20.423 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-27T23:01:20.430 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-27T23:01:20.440 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-27T23:01:20.493 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-27T23:01:20.534 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-27T23:01:20.575 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-27T23:01:22.160 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:01:22.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-27T23:01:22.205 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:01:22.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-27T23:01:22.254 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:01:22.299 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:01:22.451 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2023-09-27T23:01:22.451 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2023-09-27T23:01:22.482 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-09-27T23:01:22.483 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-09-27T23:01:22.655 INFO:teuthology.orchestra.run.smithi123.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-27T23:01:22.655 INFO:teuthology.orchestra.run.smithi123.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-27T23:01:22.658 DEBUG:teuthology.orchestra.run.smithi123:> dpkg -s linux-image-generic 2023-09-27T23:01:22.678 INFO:teuthology.orchestra.run.smithi123.stdout:Package: linux-image-generic 2023-09-27T23:01:22.678 INFO:teuthology.orchestra.run.smithi123.stdout:Status: install ok installed 2023-09-27T23:01:22.678 INFO:teuthology.orchestra.run.smithi123.stdout:Priority: optional 2023-09-27T23:01:22.678 INFO:teuthology.orchestra.run.smithi123.stdout:Section: kernel 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Installed-Size: 20 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Maintainer: Ubuntu Kernel Team 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: amd64 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Source: linux-meta 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Version: 5.4.0.163.160 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Recommends: thermald 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout:Description: Generic Linux kernel image 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout: This package will always depend on the latest generic kernel image 2023-09-27T23:01:22.679 INFO:teuthology.orchestra.run.smithi123.stdout: available. 2023-09-27T23:01:22.680 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-09-27T23:01:22.684 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.ZhgbO2vlQO 2023-09-27T23:01:22.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ZhgbO2vlQO 2023-09-27T23:01:22.696 INFO:teuthology.orchestra.run.smithi064.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-27T23:01:22.696 INFO:teuthology.orchestra.run.smithi064.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-27T23:01:22.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.ZhgbO2vlQO 2023-09-27T23:01:22.698 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2023-09-27T23:01:22.716 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2023-09-27T23:01:22.716 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2023-09-27T23:01:22.716 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2023-09-27T23:01:22.716 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2023-09-27T23:01:22.716 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.163.160 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2023-09-27T23:01:22.717 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2023-09-27T23:01:22.717 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-09-27T23:01:22.724 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.abzgsgFr4J 2023-09-27T23:01:22.724 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub/grub.cfg /tmp/tmp.abzgsgFr4J 2023-09-27T23:01:22.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.abzgsgFr4J 2023-09-27T23:01:22.780 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.ZhgbO2vlQO is 9KB 2023-09-27T23:01:22.794 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.ZhgbO2vlQO 2023-09-27T23:01:22.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-27T23:01:22.828 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.abzgsgFr4J is 9KB 2023-09-27T23:01:22.841 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.abzgsgFr4J 2023-09-27T23:01:22.847 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-27T23:01:22.860 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:01:22.860 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-27T23:01:22.860 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-27T23:01:22.862 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:01:22.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-27T23:01:22.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-27T23:01:22.890 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-27T23:01:22.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo update-grub 2023-09-27T23:01:22.937 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-27T23:01:22.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo update-grub 2023-09-27T23:01:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:Sourcing file `/etc/default/grub' 2023-09-27T23:01:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-27T23:01:24.013 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2023-09-27T23:01:24.241 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub' 2023-09-27T23:01:24.252 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-27T23:01:24.262 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2023-09-27T23:01:24.267 INFO:teuthology.orchestra.run.smithi123.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-27T23:01:24.279 INFO:teuthology.orchestra.run.smithi123.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-27T23:01:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-27T23:01:24.511 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-27T23:01:24.633 INFO:teuthology.orchestra.run.smithi123.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-27T23:01:24.646 INFO:teuthology.orchestra.run.smithi123.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-27T23:01:24.907 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-27T23:01:24.919 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-27T23:01:26.304 INFO:teuthology.orchestra.run.smithi123.stderr:done 2023-09-27T23:01:26.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2023-09-27T23:01:26.616 INFO:teuthology.orchestra.run.smithi064.stderr:done 2023-09-27T23:01:26.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2023-09-27T23:01:56.332 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-09-27T23:01:56.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:01:56.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:01:56.620 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-27T23:01:56.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:01:56.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:02:14.650 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-09-27T23:02:15.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-09-27T23:02:23.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:02:23.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:02:24.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:02:24.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:02:26.714 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-09-27T23:02:27.226 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-09-27T23:02:38.726 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:02:38.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:02:39.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:02:39.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:02:57.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-09-27T23:02:57.658 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-09-27T23:03:12.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:03:12.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:03:12.357 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-09-27T23:03:12.659 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:03:12.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:03:12.867 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-09-27T23:03:12.931 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:03:12.932 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-163-generic"... 2023-09-27T23:03:12.932 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2023-09-27T23:03:12.938 INFO:teuthology.orchestra.run.smithi123.stdout:5.4.0-163-generic 2023-09-27T23:03:12.938 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-27T23:03:12.938 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-27T23:03:12.938 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-27T23:03:13.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:03:13.448 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-163-generic"... 2023-09-27T23:03:13.448 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-09-27T23:03:13.454 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-163-generic 2023-09-27T23:03:13.454 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-27T23:03:13.454 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-27T23:03:13.455 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-27T23:03:13.939 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-27T23:03:13.940 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-27T23:03:14.027 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2023-09-27T23:03:14.050 DEBUG:teuthology.parallel:result is None 2023-09-27T23:03:14.455 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-27T23:03:14.456 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-27T23:03:14.539 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2023-09-27T23:03:14.571 DEBUG:teuthology.parallel:result is None 2023-09-27T23:03:14.572 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-27T23:03:14.579 INFO:teuthology.task.internal:Creating test directory... 2023-09-27T23:03:14.579 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-27T23:03:14.581 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-27T23:03:14.588 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-27T23:03:14.684 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-27T23:03:14.696 INFO:teuthology.task.internal:Creating archive directory... 2023-09-27T23:03:14.697 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-27T23:03:14.699 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-27T23:03:14.722 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-27T23:03:14.731 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-27T23:03:14.731 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-27T23:03:14.734 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-27T23:03:14.749 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-27T23:03:14.759 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-27T23:03:14.772 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-27T23:03:14.781 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-27T23:03:14.784 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-27T23:03:14.791 INFO:teuthology.task.internal:Configuring sudo... 2023-09-27T23:03:14.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-27T23:03:14.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-27T23:03:14.823 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-27T23:03:14.832 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-27T23:03:14.832 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-27T23:03:14.834 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-27T23:03:14.870 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-27T23:03:14.887 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-27T23:03:14.894 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:03:14.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-27T23:03:14.919 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-27T23:03:14.927 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-27T23:03:14.933 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:03:14.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-27T23:03:14.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-09-27T23:03:15.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-09-27T23:03:15.045 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-27T23:03:15.054 INFO:teuthology.task.internal:Starting timer... 2023-09-27T23:03:15.054 INFO:teuthology.run_tasks:Running task pcp... 2023-09-27T23:03:15.067 INFO:teuthology.run_tasks:Running task selinux... 2023-09-27T23:03:15.075 DEBUG:teuthology.task.selinux:Excluding smithi064: OS 'ubuntu' does not support SELinux 2023-09-27T23:03:15.076 DEBUG:teuthology.task.selinux:Excluding smithi123: OS 'ubuntu' does not support SELinux 2023-09-27T23:03:15.076 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-27T23:03:15.076 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-09-27T23:03:15.076 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-27T23:03:15.076 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-27T23:03:15.085 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-27T23:03:15.090 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-27T23:03:15.220 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-27T23:03:15.235 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-27T23:03:15.236 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-27T23:09:10.311 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2023-09-27T23:09:10.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:09:10.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:09:10.361 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-09-27T23:09:10.479 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-27T23:09:10.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:09:10.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T23:09:10.529 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-09-27T23:09:10.655 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-27T23:09:10.656 INFO:teuthology.run_tasks:Running task clock... 2023-09-27T23:09:10.668 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-27T23:09:10.668 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-27T23:09:10.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-27T23:09:10.670 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-27T23:09:10.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-27T23:09:10.702 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-27T23:09:10.702 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Command line: ntpd -gq 2023-09-27T23:09:10.703 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: proto: precision = 0.068 usec (-24) 2023-09-27T23:09:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-27T23:09:10.703 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-27T23:09:10.704 INFO:teuthology.orchestra.run.smithi123.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: restrict ::: KOD does nothing without LIMITED. 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listen and drop on 0 v6wildcard [::]:123 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listen normally on 2 lo 127.0.0.1:123 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listen normally on 3 enp3s0f1 172.21.15.123:123 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listen normally on 4 lo [::1]:123 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3f1%5]:123 2023-09-27T23:09:10.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:10 ntpd[22710]: Listening on routing socket on fd #22 for interface updates 2023-09-27T23:09:10.706 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-27T23:09:10.706 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Command line: ntpd -gq 2023-09-27T23:09:10.706 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: proto: precision = 0.092 usec (-23) 2023-09-27T23:09:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-27T23:09:10.707 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-27T23:09:10.707 INFO:teuthology.orchestra.run.smithi064.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: restrict ::: KOD does nothing without LIMITED. 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listen and drop on 0 v6wildcard [::]:123 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listen normally on 2 lo 127.0.0.1:123 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listen normally on 3 enp3s0f1 172.21.15.64:123 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listen normally on 4 lo [::1]:123 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8df%5]:123 2023-09-27T23:09:10.708 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:10 ntpd[22767]: Listening on routing socket on fd #22 for interface updates 2023-09-27T23:09:11.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:11.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:11 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:11.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:11.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:11.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:11.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:11.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:11 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:12.705 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:12.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:12 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:13.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:13.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:13.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:13.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:13 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:13.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:13.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:13 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:14.705 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:14 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:14.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:14 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:14.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:14 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:14.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:14 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:14.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:14 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:14.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:14 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:15.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:15.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:15.706 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:15 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:15 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:16.705 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:16 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:16.705 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:16 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:16.705 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:16 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:16.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:16 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:16.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:16 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:16.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:16 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:17.703 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:17.704 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:17.705 INFO:teuthology.orchestra.run.smithi123.stdout:27 Sep 23:09:17 ntpd[22710]: ntpd: time slew -0.003311 s 2023-09-27T23:09:17.705 INFO:teuthology.orchestra.run.smithi123.stdout:ntpd: time slew -0.003311s 2023-09-27T23:09:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-27T23:09:17.705 INFO:teuthology.orchestra.run.smithi123.stderr:27 Sep 23:09:17 ntpd[22710]: can't open /var/log/ntpstats/loopstats.20230927: Permission denied 2023-09-27T23:09:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: can't open /var/log/ntpstats/rawstats.20230927: Permission denied 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: can't open /var/log/ntpstats/peerstats.20230927: Permission denied 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stdout:27 Sep 23:09:17 ntpd[22767]: ntpd: time slew +0.000214 s 2023-09-27T23:09:17.707 INFO:teuthology.orchestra.run.smithi064.stdout:ntpd: time slew +0.000214s 2023-09-27T23:09:17.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-27T23:09:17.708 INFO:teuthology.orchestra.run.smithi064.stderr:27 Sep 23:09:17 ntpd[22767]: can't open /var/log/ntpstats/loopstats.20230927: Permission denied 2023-09-27T23:09:17.758 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2023-09-27T23:09:17.759 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2023-09-27T23:09:17.759 INFO:teuthology.orchestra.run.smithi064.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.759 INFO:teuthology.orchestra.run.smithi064.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.759 INFO:teuthology.orchestra.run.smithi064.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.759 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: remote refid st t when poll reach delay offset jitter 2023-09-27T23:09:17.773 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================== 2023-09-27T23:09:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.773 INFO:teuthology.orchestra.run.smithi123.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-27T23:09:17.774 INFO:teuthology.run_tasks:Running task install... 2023-09-27T23:09:17.784 DEBUG:teuthology.task.install:project ceph 2023-09-27T23:09:17.784 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28'}} 2023-09-27T23:09:17.784 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28'} 2023-09-27T23:09:17.784 INFO:teuthology.task.install:Using flavor: default 2023-09-27T23:09:17.790 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-27T23:09:17.790 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-27T23:09:17.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-key list | grep Ceph 2023-09-27T23:09:17.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-key list | grep Ceph 2023-09-27T23:09:17.907 INFO:teuthology.orchestra.run.smithi064.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-27T23:09:17.918 INFO:teuthology.orchestra.run.smithi123.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-27T23:09:17.969 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-27T23:09:17.970 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph.com (release key) 2023-09-27T23:09:17.971 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-09-27T23:09:17.971 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-09-27T23:09:17.971 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 2023-09-27T23:09:17.980 INFO:teuthology.orchestra.run.smithi123.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-27T23:09:17.980 INFO:teuthology.orchestra.run.smithi123.stdout:uid [ unknown] Ceph.com (release key) 2023-09-27T23:09:17.981 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-09-27T23:09:17.981 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-09-27T23:09:17.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 2023-09-27T23:09:18.116 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default/ 2023-09-27T23:09:18.117 INFO:teuthology.task.install.deb:Package version is 17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:09:18.121 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default/ 2023-09-27T23:09:18.121 INFO:teuthology.task.install.deb:Package version is 17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:09:18.284 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:09:18.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-27T23:09:18.289 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:09:18.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-27T23:09:18.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-09-27T23:09:18.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo apt-get update 2023-09-27T23:09:18.451 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-27T23:09:18.455 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-27T23:09:18.458 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-27T23:09:18.466 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-27T23:09:18.475 INFO:teuthology.orchestra.run.smithi123.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-27T23:09:18.562 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-27T23:09:18.644 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-27T23:09:18.726 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-27T23:09:18.741 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal InRelease 2023-09-27T23:09:18.751 INFO:teuthology.orchestra.run.smithi123.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal InRelease 2023-09-27T23:09:18.828 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-27T23:09:18.839 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-27T23:09:18.918 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal Release.gpg 2023-09-27T23:09:18.930 INFO:teuthology.orchestra.run.smithi123.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal Release.gpg 2023-09-27T23:09:19.006 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-27T23:09:19.019 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main i386 Packages [3,370 B] 2023-09-27T23:09:19.128 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-27T23:09:19.204 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main i386 Packages [3,370 B] 2023-09-27T23:09:19.232 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 28.4 kB in 1s (37.4 kB/s) 2023-09-27T23:09:19.306 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 28.4 kB in 1s (34.1 kB/s) 2023-09-27T23:09:20.264 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:09:20.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1161-g471b6dfd-1focal cephadm=17.2.6-1161-g471b6dfd-1focal ceph-mds=17.2.6-1161-g471b6dfd-1focal ceph-mgr=17.2.6-1161-g471b6dfd-1focal ceph-common=17.2.6-1161-g471b6dfd-1focal ceph-fuse=17.2.6-1161-g471b6dfd-1focal ceph-test=17.2.6-1161-g471b6dfd-1focal ceph-volume=17.2.6-1161-g471b6dfd-1focal radosgw=17.2.6-1161-g471b6dfd-1focal python3-rados=17.2.6-1161-g471b6dfd-1focal python3-rgw=17.2.6-1161-g471b6dfd-1focal python3-cephfs=17.2.6-1161-g471b6dfd-1focal python3-rbd=17.2.6-1161-g471b6dfd-1focal libcephfs2=17.2.6-1161-g471b6dfd-1focal libcephfs-dev=17.2.6-1161-g471b6dfd-1focal librados2=17.2.6-1161-g471b6dfd-1focal librbd1=17.2.6-1161-g471b6dfd-1focal rbd-fuse=17.2.6-1161-g471b6dfd-1focal python3-cephfs=17.2.6-1161-g471b6dfd-1focal cephfs-shell=17.2.6-1161-g471b6dfd-1focal cephfs-top=17.2.6-1161-g471b6dfd-1focal cephfs-mirror=17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:09:20.356 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:09:20.373 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:09:20.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1161-g471b6dfd-1focal cephadm=17.2.6-1161-g471b6dfd-1focal ceph-mds=17.2.6-1161-g471b6dfd-1focal ceph-mgr=17.2.6-1161-g471b6dfd-1focal ceph-common=17.2.6-1161-g471b6dfd-1focal ceph-fuse=17.2.6-1161-g471b6dfd-1focal ceph-test=17.2.6-1161-g471b6dfd-1focal ceph-volume=17.2.6-1161-g471b6dfd-1focal radosgw=17.2.6-1161-g471b6dfd-1focal python3-rados=17.2.6-1161-g471b6dfd-1focal python3-rgw=17.2.6-1161-g471b6dfd-1focal python3-cephfs=17.2.6-1161-g471b6dfd-1focal python3-rbd=17.2.6-1161-g471b6dfd-1focal libcephfs2=17.2.6-1161-g471b6dfd-1focal libcephfs-dev=17.2.6-1161-g471b6dfd-1focal librados2=17.2.6-1161-g471b6dfd-1focal librbd1=17.2.6-1161-g471b6dfd-1focal rbd-fuse=17.2.6-1161-g471b6dfd-1focal python3-cephfs=17.2.6-1161-g471b6dfd-1focal cephfs-shell=17.2.6-1161-g471b6dfd-1focal cephfs-top=17.2.6-1161-g471b6dfd-1focal cephfs-mirror=17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:09:20.468 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:09:20.540 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2023-09-27T23:09:20.541 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2023-09-27T23:09:20.672 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-09-27T23:09:20.673 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-09-27T23:09:20.716 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages were automatically installed and are no longer required: 2023-09-27T23:09:20.717 INFO:teuthology.orchestra.run.smithi123.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-27T23:09:20.717 INFO:teuthology.orchestra.run.smithi123.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-27T23:09:20.717 INFO:teuthology.orchestra.run.smithi123.stdout:The following additional packages will be installed: 2023-09-27T23:09:20.718 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-27T23:09:20.718 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-27T23:09:20.718 INFO:teuthology.orchestra.run.smithi123.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-27T23:09:20.718 INFO:teuthology.orchestra.run.smithi123.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-27T23:09:20.718 INFO:teuthology.orchestra.run.smithi123.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-09-27T23:09:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: qttranslations5-l10n socat xmlstarlet 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout:Suggested packages: 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-27T23:09:20.720 INFO:teuthology.orchestra.run.smithi123.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout:Recommended packages: 2023-09-27T23:09:20.721 INFO:teuthology.orchestra.run.smithi123.stdout: btrfs-tools 2023-09-27T23:09:20.843 INFO:teuthology.orchestra.run.smithi123.stdout:The following NEW packages will be installed: 2023-09-27T23:09:20.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-27T23:09:20.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-27T23:09:20.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-27T23:09:20.844 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-09-27T23:09:20.844 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-09-27T23:09:20.844 INFO:teuthology.orchestra.run.smithi123.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-09-27T23:09:20.844 INFO:teuthology.orchestra.run.smithi123.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-09-27T23:09:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-09-27T23:09:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-09-27T23:09:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-27T23:09:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-09-27T23:09:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-09-27T23:09:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-27T23:09:20.846 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages will be upgraded: 2023-09-27T23:09:20.847 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 librbd1 2023-09-27T23:09:20.860 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2023-09-27T23:09:20.861 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-27T23:09:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-09-27T23:09:20.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-27T23:09:20.864 INFO:teuthology.orchestra.run.smithi064.stdout: qttranslations5-l10n socat xmlstarlet 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-27T23:09:20.865 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2023-09-27T23:09:20.866 INFO:teuthology.orchestra.run.smithi064.stdout: btrfs-tools 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-09-27T23:09:20.988 INFO:teuthology.orchestra.run.smithi064.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-09-27T23:09:20.989 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-09-27T23:09:20.990 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2023-09-27T23:09:20.991 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 librbd1 2023-09-27T23:09:21.183 INFO:teuthology.orchestra.run.smithi064.stdout:2 upgraded, 114 newly installed, 0 to remove and 245 not upgraded. 2023-09-27T23:09:21.183 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 210 MB of archives. 2023-09-27T23:09:21.184 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 981 MB of additional disk space will be used. 2023-09-27T23:09:21.184 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-09-27T23:09:21.267 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-27T23:09:21.296 INFO:teuthology.orchestra.run.smithi123.stdout:2 upgraded, 114 newly installed, 0 to remove and 245 not upgraded. 2023-09-27T23:09:21.296 INFO:teuthology.orchestra.run.smithi123.stdout:Need to get 210 MB of archives. 2023-09-27T23:09:21.296 INFO:teuthology.orchestra.run.smithi123.stdout:After this operation, 981 MB of additional disk space will be used. 2023-09-27T23:09:21.296 INFO:teuthology.orchestra.run.smithi123.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1161-g471b6dfd-1focal [3,500 kB] 2023-09-27T23:09:21.301 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-09-27T23:09:21.415 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-09-27T23:09:21.425 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-27T23:09:21.437 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-27T23:09:21.446 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-09-27T23:09:21.475 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-27T23:09:21.509 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-27T23:09:21.511 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-27T23:09:21.522 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-27T23:09:21.525 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-27T23:09:21.546 INFO:teuthology.orchestra.run.smithi123.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-27T23:09:21.546 INFO:teuthology.orchestra.run.smithi123.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-27T23:09:21.552 INFO:teuthology.orchestra.run.smithi123.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-27T23:09:21.563 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-27T23:09:21.573 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-27T23:09:21.583 INFO:teuthology.orchestra.run.smithi123.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-27T23:09:21.586 INFO:teuthology.orchestra.run.smithi123.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-27T23:09:21.592 INFO:teuthology.orchestra.run.smithi123.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-27T23:09:21.593 INFO:teuthology.orchestra.run.smithi123.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-27T23:09:21.594 INFO:teuthology.orchestra.run.smithi123.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-27T23:09:21.596 INFO:teuthology.orchestra.run.smithi123.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-27T23:09:21.597 INFO:teuthology.orchestra.run.smithi123.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-27T23:09:21.598 INFO:teuthology.orchestra.run.smithi123.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-27T23:09:21.599 INFO:teuthology.orchestra.run.smithi123.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-27T23:09:21.610 INFO:teuthology.orchestra.run.smithi123.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-27T23:09:21.610 INFO:teuthology.orchestra.run.smithi123.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-27T23:09:21.618 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-27T23:09:21.629 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-27T23:09:21.629 INFO:teuthology.orchestra.run.smithi123.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-27T23:09:21.630 INFO:teuthology.orchestra.run.smithi123.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-27T23:09:21.631 INFO:teuthology.orchestra.run.smithi123.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-27T23:09:21.632 INFO:teuthology.orchestra.run.smithi123.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-27T23:09:21.632 INFO:teuthology.orchestra.run.smithi123.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-27T23:09:21.634 INFO:teuthology.orchestra.run.smithi123.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-27T23:09:21.634 INFO:teuthology.orchestra.run.smithi123.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-27T23:09:21.636 INFO:teuthology.orchestra.run.smithi123.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-27T23:09:21.649 INFO:teuthology.orchestra.run.smithi123.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-27T23:09:21.650 INFO:teuthology.orchestra.run.smithi123.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-27T23:09:21.669 INFO:teuthology.orchestra.run.smithi123.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-27T23:09:21.669 INFO:teuthology.orchestra.run.smithi123.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-27T23:09:21.669 INFO:teuthology.orchestra.run.smithi123.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-27T23:09:21.670 INFO:teuthology.orchestra.run.smithi123.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-27T23:09:21.671 INFO:teuthology.orchestra.run.smithi123.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-27T23:09:21.674 INFO:teuthology.orchestra.run.smithi123.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-27T23:09:21.675 INFO:teuthology.orchestra.run.smithi123.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-27T23:09:21.676 INFO:teuthology.orchestra.run.smithi123.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-27T23:09:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-27T23:09:21.689 INFO:teuthology.orchestra.run.smithi123.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-27T23:09:21.708 INFO:teuthology.orchestra.run.smithi123.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-27T23:09:21.708 INFO:teuthology.orchestra.run.smithi123.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-27T23:09:21.709 INFO:teuthology.orchestra.run.smithi123.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-27T23:09:21.711 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-27T23:09:21.717 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-27T23:09:21.739 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-27T23:09:21.805 INFO:teuthology.orchestra.run.smithi123.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-27T23:09:21.870 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1161-g471b6dfd-1focal [3,500 kB] 2023-09-27T23:09:21.875 INFO:teuthology.orchestra.run.smithi123.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-27T23:09:21.878 INFO:teuthology.orchestra.run.smithi123.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-27T23:09:21.893 INFO:teuthology.orchestra.run.smithi123.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-27T23:09:21.893 INFO:teuthology.orchestra.run.smithi123.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-27T23:09:21.894 INFO:teuthology.orchestra.run.smithi123.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-27T23:09:21.894 INFO:teuthology.orchestra.run.smithi123.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-27T23:09:21.894 INFO:teuthology.orchestra.run.smithi123.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-27T23:09:21.895 INFO:teuthology.orchestra.run.smithi123.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-27T23:09:21.895 INFO:teuthology.orchestra.run.smithi123.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-27T23:09:21.896 INFO:teuthology.orchestra.run.smithi123.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-27T23:09:21.916 INFO:teuthology.orchestra.run.smithi123.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-27T23:09:21.917 INFO:teuthology.orchestra.run.smithi123.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-27T23:09:21.922 INFO:teuthology.orchestra.run.smithi123.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-27T23:09:21.925 INFO:teuthology.orchestra.run.smithi123.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-27T23:09:21.927 INFO:teuthology.orchestra.run.smithi123.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-27T23:09:21.928 INFO:teuthology.orchestra.run.smithi123.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-27T23:09:21.930 INFO:teuthology.orchestra.run.smithi123.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-27T23:09:21.935 INFO:teuthology.orchestra.run.smithi123.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-09-27T23:09:21.936 INFO:teuthology.orchestra.run.smithi123.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-27T23:09:21.954 INFO:teuthology.orchestra.run.smithi123.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-27T23:09:21.957 INFO:teuthology.orchestra.run.smithi123.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-27T23:09:22.006 INFO:teuthology.orchestra.run.smithi123.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-27T23:09:22.007 INFO:teuthology.orchestra.run.smithi123.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-27T23:09:22.007 INFO:teuthology.orchestra.run.smithi123.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-27T23:09:22.008 INFO:teuthology.orchestra.run.smithi123.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-27T23:09:22.008 INFO:teuthology.orchestra.run.smithi123.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-27T23:09:22.009 INFO:teuthology.orchestra.run.smithi123.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-27T23:09:22.020 INFO:teuthology.orchestra.run.smithi123.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-27T23:09:22.020 INFO:teuthology.orchestra.run.smithi123.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-27T23:09:22.021 INFO:teuthology.orchestra.run.smithi123.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-27T23:09:22.029 INFO:teuthology.orchestra.run.smithi123.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-27T23:09:22.029 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-27T23:09:22.030 INFO:teuthology.orchestra.run.smithi123.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-27T23:09:22.031 INFO:teuthology.orchestra.run.smithi123.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-27T23:09:22.031 INFO:teuthology.orchestra.run.smithi123.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-27T23:09:22.032 INFO:teuthology.orchestra.run.smithi123.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-27T23:09:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-27T23:09:22.462 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-27T23:09:22.644 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-27T23:09:22.676 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-27T23:09:22.681 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-27T23:09:22.706 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-27T23:09:22.711 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-27T23:09:22.714 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-27T23:09:22.726 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-27T23:09:22.729 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-27T23:09:22.730 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-27T23:09:22.895 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-27T23:09:22.982 INFO:teuthology.orchestra.run.smithi123.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1161-g471b6dfd-1focal [3,553 kB] 2023-09-27T23:09:23.059 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-27T23:09:23.200 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-27T23:09:23.218 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-27T23:09:23.230 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-27T23:09:23.306 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-27T23:09:23.328 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-27T23:09:23.369 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-27T23:09:23.398 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-27T23:09:23.402 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-27T23:09:23.497 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-27T23:09:23.661 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-27T23:09:23.845 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-27T23:09:23.894 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-27T23:09:24.026 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-27T23:09:24.130 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-27T23:09:24.160 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-27T23:09:24.163 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-27T23:09:24.187 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-27T23:09:24.208 INFO:teuthology.orchestra.run.smithi064.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-27T23:09:24.225 INFO:teuthology.orchestra.run.smithi064.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-27T23:09:24.227 INFO:teuthology.orchestra.run.smithi064.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-27T23:09:24.395 INFO:teuthology.orchestra.run.smithi064.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-27T23:09:24.539 INFO:teuthology.orchestra.run.smithi123.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1161-g471b6dfd-1focal [758 kB] 2023-09-27T23:09:24.686 INFO:teuthology.orchestra.run.smithi123.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1161-g471b6dfd-1focal [413 kB] 2023-09-27T23:09:25.405 INFO:teuthology.orchestra.run.smithi064.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-27T23:09:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-27T23:09:25.430 INFO:teuthology.orchestra.run.smithi064.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-27T23:09:25.503 INFO:teuthology.orchestra.run.smithi064.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-27T23:09:25.504 INFO:teuthology.orchestra.run.smithi064.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-27T23:09:25.504 INFO:teuthology.orchestra.run.smithi064.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-27T23:09:25.504 INFO:teuthology.orchestra.run.smithi064.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-27T23:09:25.504 INFO:teuthology.orchestra.run.smithi064.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-27T23:09:25.504 INFO:teuthology.orchestra.run.smithi064.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-27T23:09:25.505 INFO:teuthology.orchestra.run.smithi064.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-27T23:09:25.669 INFO:teuthology.orchestra.run.smithi064.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-27T23:09:25.836 INFO:teuthology.orchestra.run.smithi064.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-27T23:09:25.918 INFO:teuthology.orchestra.run.smithi064.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-27T23:09:26.017 INFO:teuthology.orchestra.run.smithi123.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1161-g471b6dfd-1focal [22.5 kB] 2023-09-27T23:09:26.018 INFO:teuthology.orchestra.run.smithi123.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1161-g471b6dfd-1focal [200 kB] 2023-09-27T23:09:26.080 INFO:teuthology.orchestra.run.smithi123.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1161-g471b6dfd-1focal [47.1 kB] 2023-09-27T23:09:26.082 INFO:teuthology.orchestra.run.smithi123.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1161-g471b6dfd-1focal [386 kB] 2023-09-27T23:09:26.148 INFO:teuthology.orchestra.run.smithi064.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-27T23:09:26.165 INFO:teuthology.orchestra.run.smithi123.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1161-g471b6dfd-1focal [5,565 kB] 2023-09-27T23:09:26.192 INFO:teuthology.orchestra.run.smithi064.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-27T23:09:26.221 INFO:teuthology.orchestra.run.smithi064.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-27T23:09:26.232 INFO:teuthology.orchestra.run.smithi064.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-27T23:09:26.280 INFO:teuthology.orchestra.run.smithi064.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-27T23:09:26.308 INFO:teuthology.orchestra.run.smithi064.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-09-27T23:09:26.311 INFO:teuthology.orchestra.run.smithi064.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-27T23:09:26.430 INFO:teuthology.orchestra.run.smithi064.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-27T23:09:26.526 INFO:teuthology.orchestra.run.smithi064.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1161-g471b6dfd-1focal [3,553 kB] 2023-09-27T23:09:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-27T23:09:27.244 INFO:teuthology.orchestra.run.smithi123.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1161-g471b6dfd-1focal [116 kB] 2023-09-27T23:09:27.362 INFO:teuthology.orchestra.run.smithi123.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1161-g471b6dfd-1focal [480 kB] 2023-09-27T23:09:27.432 INFO:teuthology.orchestra.run.smithi064.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-27T23:09:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-27T23:09:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-27T23:09:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-27T23:09:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-27T23:09:27.439 INFO:teuthology.orchestra.run.smithi064.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-27T23:09:27.445 INFO:teuthology.orchestra.run.smithi123.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1161-g471b6dfd-1focal [27.7 MB] 2023-09-27T23:09:27.456 INFO:teuthology.orchestra.run.smithi064.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-27T23:09:27.456 INFO:teuthology.orchestra.run.smithi064.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-27T23:09:27.456 INFO:teuthology.orchestra.run.smithi064.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-27T23:09:27.514 INFO:teuthology.orchestra.run.smithi064.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-27T23:09:27.680 INFO:teuthology.orchestra.run.smithi064.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-27T23:09:27.966 INFO:teuthology.orchestra.run.smithi064.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-27T23:09:27.988 INFO:teuthology.orchestra.run.smithi064.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-27T23:09:28.139 INFO:teuthology.orchestra.run.smithi064.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-27T23:09:28.157 INFO:teuthology.orchestra.run.smithi064.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-27T23:09:29.184 INFO:teuthology.orchestra.run.smithi064.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1161-g471b6dfd-1focal [758 kB] 2023-09-27T23:09:29.689 INFO:teuthology.orchestra.run.smithi064.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1161-g471b6dfd-1focal [413 kB] 2023-09-27T23:09:29.920 INFO:teuthology.orchestra.run.smithi064.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1161-g471b6dfd-1focal [22.5 kB] 2023-09-27T23:09:29.943 INFO:teuthology.orchestra.run.smithi064.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1161-g471b6dfd-1focal [200 kB] 2023-09-27T23:09:30.068 INFO:teuthology.orchestra.run.smithi064.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1161-g471b6dfd-1focal [47.1 kB] 2023-09-27T23:09:30.095 INFO:teuthology.orchestra.run.smithi064.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1161-g471b6dfd-1focal [386 kB] 2023-09-27T23:09:30.358 INFO:teuthology.orchestra.run.smithi064.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1161-g471b6dfd-1focal [5,565 kB] 2023-09-27T23:09:33.273 INFO:teuthology.orchestra.run.smithi064.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1161-g471b6dfd-1focal [116 kB] 2023-09-27T23:09:33.333 INFO:teuthology.orchestra.run.smithi064.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1161-g471b6dfd-1focal [480 kB] 2023-09-27T23:09:33.464 INFO:teuthology.orchestra.run.smithi123.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1161-g471b6dfd-1focal [6,452 kB] 2023-09-27T23:09:33.708 INFO:teuthology.orchestra.run.smithi064.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1161-g471b6dfd-1focal [27.7 MB] 2023-09-27T23:09:34.730 INFO:teuthology.orchestra.run.smithi123.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1161-g471b6dfd-1focal [221 kB] 2023-09-27T23:09:34.801 INFO:teuthology.orchestra.run.smithi123.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1161-g471b6dfd-1focal [152 kB] 2023-09-27T23:09:34.807 INFO:teuthology.orchestra.run.smithi123.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1161-g471b6dfd-1focal [1,589 kB] 2023-09-27T23:09:35.522 INFO:teuthology.orchestra.run.smithi123.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1161-g471b6dfd-1focal [7,844 kB] 2023-09-27T23:09:37.229 INFO:teuthology.orchestra.run.smithi123.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1161-g471b6dfd-1focal [28.7 MB] 2023-09-27T23:09:44.246 INFO:teuthology.orchestra.run.smithi064.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1161-g471b6dfd-1focal [6,452 kB] 2023-09-27T23:09:45.647 INFO:teuthology.orchestra.run.smithi123.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1161-g471b6dfd-1focal [3,928 B] 2023-09-27T23:09:45.715 INFO:teuthology.orchestra.run.smithi123.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1161-g471b6dfd-1focal [868 kB] 2023-09-27T23:09:46.094 INFO:teuthology.orchestra.run.smithi123.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1161-g471b6dfd-1focal [2,407 kB] 2023-09-27T23:09:47.022 INFO:teuthology.orchestra.run.smithi123.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1161-g471b6dfd-1focal [78.9 kB] 2023-09-27T23:09:47.024 INFO:teuthology.orchestra.run.smithi123.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1161-g471b6dfd-1focal [117 kB] 2023-09-27T23:09:47.031 INFO:teuthology.orchestra.run.smithi123.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1161-g471b6dfd-1focal [1,760 kB] 2023-09-27T23:09:47.267 INFO:teuthology.orchestra.run.smithi064.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1161-g471b6dfd-1focal [221 kB] 2023-09-27T23:09:47.345 INFO:teuthology.orchestra.run.smithi064.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1161-g471b6dfd-1focal [152 kB] 2023-09-27T23:09:47.422 INFO:teuthology.orchestra.run.smithi064.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1161-g471b6dfd-1focal [1,589 kB] 2023-09-27T23:09:47.573 INFO:teuthology.orchestra.run.smithi123.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1161-g471b6dfd-1focal [7,879 kB] 2023-09-27T23:09:48.054 INFO:teuthology.orchestra.run.smithi064.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1161-g471b6dfd-1focal [7,844 kB] 2023-09-27T23:09:49.554 INFO:teuthology.orchestra.run.smithi123.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1161-g471b6dfd-1focal [4,040 B] 2023-09-27T23:09:49.555 INFO:teuthology.orchestra.run.smithi123.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1161-g471b6dfd-1focal [56.2 MB] 2023-09-27T23:09:50.623 INFO:teuthology.orchestra.run.smithi064.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1161-g471b6dfd-1focal [28.7 MB] 2023-09-27T23:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1161-g471b6dfd-1focal [3,928 B] 2023-09-27T23:10:00.799 INFO:teuthology.orchestra.run.smithi064.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1161-g471b6dfd-1focal [868 kB] 2023-09-27T23:10:00.875 INFO:teuthology.orchestra.run.smithi064.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1161-g471b6dfd-1focal [2,407 kB] 2023-09-27T23:10:01.494 INFO:teuthology.orchestra.run.smithi064.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1161-g471b6dfd-1focal [78.9 kB] 2023-09-27T23:10:01.505 INFO:teuthology.orchestra.run.smithi064.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1161-g471b6dfd-1focal [117 kB] 2023-09-27T23:10:01.545 INFO:teuthology.orchestra.run.smithi064.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1161-g471b6dfd-1focal [1,760 kB] 2023-09-27T23:10:02.021 INFO:teuthology.orchestra.run.smithi064.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1161-g471b6dfd-1focal [7,879 kB] 2023-09-27T23:10:04.585 INFO:teuthology.orchestra.run.smithi064.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1161-g471b6dfd-1focal [4,040 B] 2023-09-27T23:10:04.586 INFO:teuthology.orchestra.run.smithi064.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1161-g471b6dfd-1focal [56.2 MB] 2023-09-27T23:10:04.596 INFO:teuthology.orchestra.run.smithi123.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1161-g471b6dfd-1focal [229 kB] 2023-09-27T23:10:04.600 INFO:teuthology.orchestra.run.smithi123.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1161-g471b6dfd-1focal [240 kB] 2023-09-27T23:10:04.605 INFO:teuthology.orchestra.run.smithi123.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1161-g471b6dfd-1focal [22.0 kB] 2023-09-27T23:10:04.606 INFO:teuthology.orchestra.run.smithi123.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1161-g471b6dfd-1focal [13.1 kB] 2023-09-27T23:10:04.606 INFO:teuthology.orchestra.run.smithi123.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1161-g471b6dfd-1focal [21.9 kB] 2023-09-27T23:10:04.607 INFO:teuthology.orchestra.run.smithi123.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1161-g471b6dfd-1focal [17.0 MB] 2023-09-27T23:10:06.432 INFO:teuthology.orchestra.run.smithi123.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1161-g471b6dfd-1focal [79.5 kB] 2023-09-27T23:10:07.069 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 210 MB in 46s (4,603 kB/s) 2023-09-27T23:10:07.242 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyperclip. 2023-09-27T23:10:07.298 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-27T23:10:07.302 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-09-27T23:10:07.349 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-09-27T23:10:07.619 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-27T23:10:07.637 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-27T23:10:07.695 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-27T23:10:07.965 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cmd2. 2023-09-27T23:10:07.981 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-09-27T23:10:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-09-27T23:10:08.393 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-27T23:10:08.410 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-27T23:10:08.493 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-27T23:10:08.871 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../004-librbd1_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:08.980 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librbd1 (17.2.6-1161-g471b6dfd-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-27T23:10:09.669 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../005-librados2_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:09.801 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librados2 (17.2.6-1161-g471b6dfd-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-27T23:10:10.505 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libcephfs2. 2023-09-27T23:10:10.522 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:10.584 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libcephfs2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:10.891 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rados. 2023-09-27T23:10:10.909 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:10.949 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rados (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:11.335 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-27T23:10:11.351 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:11.393 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-ceph-argparse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:11.737 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cephfs. 2023-09-27T23:10:11.753 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:11.795 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cephfs (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:12.115 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-27T23:10:12.133 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:12.164 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:12.443 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-prettytable. 2023-09-27T23:10:12.459 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-09-27T23:10:12.534 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-27T23:10:12.787 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rbd. 2023-09-27T23:10:12.802 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:12.845 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rbd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:13.198 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-27T23:10:13.213 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-27T23:10:13.255 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-27T23:10:13.618 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-27T23:10:13.634 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-27T23:10:13.668 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-27T23:10:14.046 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-27T23:10:14.062 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-27T23:10:14.105 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-27T23:10:14.499 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-27T23:10:14.514 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-27T23:10:14.558 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-27T23:10:14.924 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-27T23:10:14.940 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-27T23:10:14.982 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-27T23:10:15.410 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-27T23:10:15.427 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-27T23:10:15.479 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:10:16.005 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-27T23:10:16.022 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-27T23:10:16.063 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:10:16.417 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-27T23:10:16.436 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-27T23:10:16.474 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:10:16.861 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-27T23:10:16.877 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-27T23:10:16.919 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-27T23:10:17.264 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package librgw2. 2023-09-27T23:10:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../022-librgw2_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:17.339 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking librgw2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:18.125 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rgw. 2023-09-27T23:10:18.142 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:18.183 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:18.544 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liboath0. 2023-09-27T23:10:18.560 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-27T23:10:18.602 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-27T23:10:18.930 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libradosstriper1. 2023-09-27T23:10:18.949 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:18.988 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libradosstriper1 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:19.475 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-common. 2023-09-27T23:10:19.491 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:19.533 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:20.957 INFO:teuthology.orchestra.run.smithi064.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1161-g471b6dfd-1focal [229 kB] 2023-09-27T23:10:20.961 INFO:teuthology.orchestra.run.smithi064.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1161-g471b6dfd-1focal [240 kB] 2023-09-27T23:10:20.966 INFO:teuthology.orchestra.run.smithi064.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1161-g471b6dfd-1focal [22.0 kB] 2023-09-27T23:10:20.967 INFO:teuthology.orchestra.run.smithi064.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1161-g471b6dfd-1focal [13.1 kB] 2023-09-27T23:10:20.968 INFO:teuthology.orchestra.run.smithi064.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1161-g471b6dfd-1focal [21.9 kB] 2023-09-27T23:10:20.968 INFO:teuthology.orchestra.run.smithi064.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1161-g471b6dfd-1focal [17.0 MB] 2023-09-27T23:10:22.188 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-base. 2023-09-27T23:10:22.207 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:22.281 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-base (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:23.285 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cryptography. 2023-09-27T23:10:23.305 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-27T23:10:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-27T23:10:23.730 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-openssl. 2023-09-27T23:10:23.748 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-27T23:10:23.788 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-27T23:10:24.066 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-27T23:10:24.083 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:24.124 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:24.546 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-27T23:10:24.563 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-27T23:10:24.604 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-27T23:10:24.931 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-27T23:10:24.950 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-27T23:10:24.990 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-27T23:10:25.368 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-natsort. 2023-09-27T23:10:25.389 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-09-27T23:10:25.426 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-27T23:10:25.622 INFO:teuthology.orchestra.run.smithi064.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-09-27-0906-quincy/471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1161-g471b6dfd-1focal [79.5 kB] 2023-09-27T23:10:25.729 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-jwt. 2023-09-27T23:10:25.745 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-27T23:10:25.780 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-27T23:10:26.116 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-logutils. 2023-09-27T23:10:26.137 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-09-27T23:10:26.174 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-27T23:10:26.275 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 210 MB in 1min 5s (3,246 kB/s) 2023-09-27T23:10:26.423 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyperclip. 2023-09-27T23:10:26.477 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-27T23:10:26.480 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-27T23:10:26.484 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-09-27T23:10:26.493 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-27T23:10:26.529 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-09-27T23:10:26.535 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-27T23:10:26.808 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-27T23:10:26.825 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-27T23:10:26.838 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-mako. 2023-09-27T23:10:26.855 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-27T23:10:26.866 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-27T23:10:26.896 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-27T23:10:27.170 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cmd2. 2023-09-27T23:10:27.186 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-09-27T23:10:27.199 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-27T23:10:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-27T23:10:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-09-27T23:10:27.249 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-27T23:10:27.569 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-27T23:10:27.585 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-27T23:10:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-27T23:10:27.608 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-27T23:10:27.627 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-27T23:10:27.689 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-27T23:10:27.913 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-webob. 2023-09-27T23:10:27.933 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-27T23:10:27.971 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-27T23:10:28.026 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../004-librbd1_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:28.127 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librbd1 (17.2.6-1161-g471b6dfd-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-27T23:10:28.315 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-27T23:10:28.332 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-27T23:10:28.374 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-27T23:10:28.693 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-bs4. 2023-09-27T23:10:28.710 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-09-27T23:10:28.760 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-27T23:10:28.863 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../005-librados2_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:28.998 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librados2 (17.2.6-1161-g471b6dfd-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-27T23:10:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-waitress. 2023-09-27T23:10:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-27T23:10:29.199 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-27T23:10:29.549 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-tempita. 2023-09-27T23:10:29.566 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-09-27T23:10:29.607 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-27T23:10:29.651 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs2. 2023-09-27T23:10:29.668 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:29.710 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:29.868 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-paste. 2023-09-27T23:10:29.884 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-27T23:10:29.935 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-27T23:10:30.079 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rados. 2023-09-27T23:10:30.095 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:30.137 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rados (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-27T23:10:30.282 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-27T23:10:30.321 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-27T23:10:30.465 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-27T23:10:30.481 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:30.540 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-argparse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:30.640 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-27T23:10:30.656 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-27T23:10:30.699 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-27T23:10:30.750 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cephfs. 2023-09-27T23:10:30.769 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:30.800 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cephfs (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:31.018 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-webtest. 2023-09-27T23:10:31.035 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-09-27T23:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-27T23:10:31.161 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-27T23:10:31.180 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:31.219 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:31.421 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pecan. 2023-09-27T23:10:31.438 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-27T23:10:31.479 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-27T23:10:31.539 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-prettytable. 2023-09-27T23:10:31.557 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-09-27T23:10:31.598 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-27T23:10:31.863 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-27T23:10:31.882 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-27T23:10:31.883 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rbd. 2023-09-27T23:10:31.901 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:31.925 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-27T23:10:31.942 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rbd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:32.253 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-27T23:10:32.271 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:32.294 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-27T23:10:32.311 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-27T23:10:32.311 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:32.353 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-27T23:10:32.622 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr. 2023-09-27T23:10:32.641 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:32.680 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:32.753 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-27T23:10:32.772 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-27T23:10:32.806 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-27T23:10:33.109 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mon. 2023-09-27T23:10:33.124 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:33.158 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mon (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:33.168 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-27T23:10:33.185 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-27T23:10:33.227 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-27T23:10:33.585 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-27T23:10:33.604 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-27T23:10:33.643 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-27T23:10:33.996 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-27T23:10:34.013 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-27T23:10:34.054 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-27T23:10:34.329 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-osd. 2023-09-27T23:10:34.349 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:34.387 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-osd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:34.475 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-27T23:10:34.493 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-27T23:10:34.534 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:10:35.061 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-27T23:10:35.078 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-27T23:10:35.119 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:10:35.481 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-27T23:10:35.498 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-27T23:10:35.539 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:10:35.900 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-27T23:10:35.917 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-27T23:10:35.950 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-27T23:10:36.329 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librgw2. 2023-09-27T23:10:36.348 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../022-librgw2_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:36.395 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librgw2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:37.123 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph. 2023-09-27T23:10:37.141 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../055-ceph_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:37.181 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:37.206 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rgw. 2023-09-27T23:10:37.224 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:37.264 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:37.467 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-fuse. 2023-09-27T23:10:37.485 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:37.516 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:37.583 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liboath0. 2023-09-27T23:10:37.601 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-27T23:10:37.650 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-27T23:10:37.936 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libradosstriper1. 2023-09-27T23:10:37.953 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:37.955 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mds. 2023-09-27T23:10:37.971 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:37.994 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libradosstriper1 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:38.013 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mds (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:38.490 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-common. 2023-09-27T23:10:38.506 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:38.524 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephadm. 2023-09-27T23:10:38.540 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../058-cephadm_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:38.547 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:38.574 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:38.894 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-jinja2. 2023-09-27T23:10:38.910 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-09-27T23:10:38.951 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-27T23:10:39.280 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-27T23:10:39.295 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:39.329 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-27T23:10:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-09-27T23:10:39.774 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-27T23:10:40.026 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjs-underscore. 2023-09-27T23:10:40.042 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-27T23:10:40.084 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-27T23:10:40.395 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-27T23:10:40.413 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-27T23:10:40.765 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-27T23:10:41.181 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-routes. 2023-09-27T23:10:41.198 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-09-27T23:10:41.239 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-27T23:10:41.286 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-base. 2023-09-27T23:10:41.302 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:41.346 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-base (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:41.584 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-27T23:10:41.600 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:41.642 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:42.138 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-decorator. 2023-09-27T23:10:42.155 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-27T23:10:42.195 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-27T23:10:42.308 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cryptography. 2023-09-27T23:10:42.328 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-27T23:10:42.366 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-27T23:10:42.523 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-27T23:10:42.542 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-27T23:10:42.581 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-27T23:10:42.687 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-openssl. 2023-09-27T23:10:42.705 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-27T23:10:42.745 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-27T23:10:42.959 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-scipy. 2023-09-27T23:10:42.975 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-27T23:10:43.017 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-27T23:10:43.047 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-27T23:10:43.063 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:43.106 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:43.652 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-27T23:10:43.668 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-27T23:10:43.710 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-27T23:10:43.997 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-27T23:10:44.016 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-27T23:10:44.054 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-27T23:10:44.259 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-27T23:10:44.278 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-27T23:10:44.317 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-27T23:10:44.416 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-natsort. 2023-09-27T23:10:44.436 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-09-27T23:10:44.474 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-27T23:10:44.820 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jwt. 2023-09-27T23:10:44.840 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-27T23:10:44.854 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-joblib. 2023-09-27T23:10:44.870 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-09-27T23:10:44.878 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-27T23:10:44.920 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-27T23:10:45.190 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-logutils. 2023-09-27T23:10:45.206 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-09-27T23:10:45.247 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-27T23:10:45.307 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-sklearn. 2023-09-27T23:10:45.323 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-27T23:10:45.365 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-27T23:10:45.567 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-27T23:10:45.583 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-27T23:10:45.625 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-27T23:10:45.912 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-mako. 2023-09-27T23:10:45.928 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-27T23:10:45.970 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-27T23:10:46.006 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-27T23:10:46.023 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:46.064 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:46.323 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-27T23:10:46.341 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-27T23:10:46.381 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-27T23:10:46.717 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-27T23:10:46.736 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-27T23:10:46.775 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-27T23:10:47.078 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webob. 2023-09-27T23:10:47.097 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-27T23:10:47.136 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-27T23:10:47.184 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-cachetools. 2023-09-27T23:10:47.200 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-09-27T23:10:47.242 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-27T23:10:47.486 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-27T23:10:47.489 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-27T23:10:47.503 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-27T23:10:47.506 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-27T23:10:47.536 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-27T23:10:47.547 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-27T23:10:47.851 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bs4. 2023-09-27T23:10:47.867 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-09-27T23:10:47.873 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-27T23:10:47.890 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-27T23:10:47.909 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-27T23:10:47.931 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-27T23:10:48.275 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-rsa. 2023-09-27T23:10:48.292 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-09-27T23:10:48.295 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-waitress. 2023-09-27T23:10:48.313 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-27T23:10:48.333 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-27T23:10:48.356 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-27T23:10:48.653 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-google-auth. 2023-09-27T23:10:48.673 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-09-27T23:10:48.711 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-27T23:10:48.906 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-tempita. 2023-09-27T23:10:48.922 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-09-27T23:10:48.964 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-27T23:10:49.056 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-blinker. 2023-09-27T23:10:49.072 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-27T23:10:49.114 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-27T23:10:49.301 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-paste. 2023-09-27T23:10:49.320 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-27T23:10:49.367 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-27T23:10:49.417 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-27T23:10:49.433 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-27T23:10:49.498 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-27T23:10:49.696 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-27T23:10:49.712 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-27T23:10:49.753 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-27T23:10:49.828 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-27T23:10:49.849 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-27T23:10:49.886 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-27T23:10:50.090 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-27T23:10:50.108 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-27T23:10:50.148 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-27T23:10:50.325 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-websocket. 2023-09-27T23:10:50.341 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-27T23:10:50.383 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-27T23:10:50.451 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webtest. 2023-09-27T23:10:50.471 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-09-27T23:10:50.509 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-27T23:10:50.736 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-27T23:10:50.757 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-27T23:10:50.794 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-27T23:10:50.845 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pecan. 2023-09-27T23:10:50.862 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-27T23:10:50.895 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-27T23:10:51.267 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-27T23:10:51.286 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-27T23:10:51.316 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-27T23:10:51.342 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-27T23:10:51.359 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:51.416 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:51.637 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-27T23:10:51.656 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:51.694 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:51.736 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-27T23:10:51.754 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-09-27T23:10:51.786 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-27T23:10:52.064 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr. 2023-09-27T23:10:52.081 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:52.122 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:52.181 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-27T23:10:52.197 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-27T23:10:52.239 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:10:52.533 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package jq. 2023-09-27T23:10:52.551 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-27T23:10:52.591 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:10:52.634 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mon. 2023-09-27T23:10:52.650 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:52.692 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mon (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:52.869 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package socat. 2023-09-27T23:10:52.889 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-09-27T23:10:52.927 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-27T23:10:53.247 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package xmlstarlet. 2023-09-27T23:10:53.264 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-27T23:10:53.305 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-27T23:10:53.701 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-test. 2023-09-27T23:10:53.717 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:53.759 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-test (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:53.862 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-osd. 2023-09-27T23:10:53.880 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:53.920 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-osd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:56.681 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph. 2023-09-27T23:10:56.698 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../055-ceph_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:56.739 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:57.084 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-fuse. 2023-09-27T23:10:57.104 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:57.142 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:57.553 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mds. 2023-09-27T23:10:57.572 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:57.611 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mds (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:58.148 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephadm. 2023-09-27T23:10:58.164 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../058-cephadm_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:58.197 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:58.551 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jinja2. 2023-09-27T23:10:58.567 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-09-27T23:10:58.580 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ceph-volume. 2023-09-27T23:10:58.598 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:58.608 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-27T23:10:58.638 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ceph-volume (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:58.993 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-27T23:10:59.010 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:59.053 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:59.073 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephfs-mirror. 2023-09-27T23:10:59.089 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:10:59.131 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephfs-mirror (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:10:59.373 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-27T23:10:59.392 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-09-27T23:10:59.430 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-27T23:10:59.484 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-colorama. 2023-09-27T23:10:59.500 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-09-27T23:10:59.542 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-09-27T23:10:59.717 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-underscore. 2023-09-27T23:10:59.734 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-27T23:10:59.775 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-27T23:10:59.811 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephfs-shell. 2023-09-27T23:10:59.830 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:10:59.869 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephfs-shell (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:00.052 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-27T23:11:00.070 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-27T23:11:00.105 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package cephfs-top. 2023-09-27T23:11:00.110 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-27T23:11:00.123 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:00.155 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking cephfs-top (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:00.440 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-27T23:11:00.455 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-routes. 2023-09-27T23:11:00.457 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-27T23:11:00.471 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-09-27T23:11:00.498 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:00.512 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-27T23:11:00.826 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-27T23:11:00.845 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:00.909 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:01.027 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-27T23:11:01.044 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:01.085 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libcephfs-dev (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:01.379 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-decorator. 2023-09-27T23:11:01.398 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-27T23:11:01.437 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-27T23:11:01.480 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package nvme-cli. 2023-09-27T23:11:01.500 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-27T23:11:01.556 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-27T23:11:01.774 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-27T23:11:01.790 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-27T23:11:01.832 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-27T23:11:01.905 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-27T23:11:01.922 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-27T23:11:01.963 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:02.160 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-scipy. 2023-09-27T23:11:02.176 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-27T23:11:02.226 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-27T23:11:03.477 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-27T23:11:03.496 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-27T23:11:03.535 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-27T23:11:03.672 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-27T23:11:03.692 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-27T23:11:03.730 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-27T23:11:04.055 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-joblib. 2023-09-27T23:11:04.073 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-09-27T23:11:04.085 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-27T23:11:04.102 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-27T23:11:04.138 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-27T23:11:04.143 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-27T23:11:04.412 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-attr. 2023-09-27T23:11:04.432 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-09-27T23:11:04.470 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-27T23:11:04.500 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn. 2023-09-27T23:11:04.520 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-27T23:11:04.558 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-27T23:11:04.823 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-webencodings. 2023-09-27T23:11:04.840 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-27T23:11:04.881 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-27T23:11:05.226 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-html5lib. 2023-09-27T23:11:05.243 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-09-27T23:11:05.283 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-27T23:11:05.349 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-27T23:11:05.366 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:05.424 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:05.645 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-27T23:11:05.663 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-27T23:11:05.703 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-27T23:11:06.165 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-packaging. 2023-09-27T23:11:06.185 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-09-27T23:11:06.222 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-27T23:11:06.519 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cachetools. 2023-09-27T23:11:06.535 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-09-27T23:11:06.550 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pastescript. 2023-09-27T23:11:06.568 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-09-27T23:11:06.577 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-27T23:11:06.608 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-27T23:11:06.905 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-27T23:11:06.924 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-27T23:11:06.937 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pluggy. 2023-09-27T23:11:06.954 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-09-27T23:11:06.963 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-27T23:11:07.003 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-27T23:11:07.283 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-27T23:11:07.302 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-27T23:11:07.323 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-psutil. 2023-09-27T23:11:07.340 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-27T23:11:07.349 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-27T23:11:07.381 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-27T23:11:07.644 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rsa. 2023-09-27T23:11:07.662 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-09-27T23:11:07.702 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-27T23:11:07.743 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-py. 2023-09-27T23:11:07.760 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-27T23:11:07.801 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-27T23:11:08.038 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-google-auth. 2023-09-27T23:11:08.055 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-09-27T23:11:08.097 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-27T23:11:08.137 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-27T23:11:08.158 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-27T23:11:08.195 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-27T23:11:08.400 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-blinker. 2023-09-27T23:11:08.420 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-27T23:11:08.458 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-27T23:11:08.523 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-pytest. 2023-09-27T23:11:08.540 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-09-27T23:11:08.581 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-27T23:11:08.777 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-27T23:11:08.794 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-27T23:11:08.835 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-27T23:11:08.960 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package python3-simplejson. 2023-09-27T23:11:08.977 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-27T23:11:09.018 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-27T23:11:09.139 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-27T23:11:09.155 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-27T23:11:09.197 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-27T23:11:09.304 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-27T23:11:09.325 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-27T23:11:09.362 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-27T23:11:09.611 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-websocket. 2023-09-27T23:11:09.627 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-27T23:11:09.669 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-27T23:11:10.022 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-27T23:11:10.041 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-27T23:11:10.080 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-27T23:11:10.143 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package radosgw. 2023-09-27T23:11:10.160 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../114-radosgw_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:10.203 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking radosgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:10.670 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-27T23:11:10.688 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:10.728 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:11.031 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-27T23:11:11.047 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-09-27T23:11:11.089 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-27T23:11:11.459 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-27T23:11:11.475 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-27T23:11:11.516 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:11:11.820 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package jq. 2023-09-27T23:11:11.836 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-27T23:11:11.878 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:11:12.089 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package rbd-fuse. 2023-09-27T23:11:12.109 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:12.114 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package socat. 2023-09-27T23:11:12.130 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-09-27T23:11:12.147 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking rbd-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:12.172 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-27T23:11:12.475 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package xmlstarlet. 2023-09-27T23:11:12.476 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-27T23:11:12.493 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-27T23:11:12.533 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-27T23:11:12.769 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-27T23:11:12.871 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-test. 2023-09-27T23:11:12.889 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:12.920 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-27T23:11:12.929 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-test (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:13.046 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-27T23:11:13.255 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:13.867 INFO:teuthology.orchestra.run.smithi123.stdout:Adding system user cephadm....done 2023-09-27T23:11:13.995 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:14.121 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-27T23:11:14.468 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-27T23:11:14.769 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:14.895 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-27T23:11:15.021 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-27T23:11:15.356 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-27T23:11:15.783 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-27T23:11:16.053 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-27T23:11:16.346 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-09-27T23:11:16.657 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-27T23:11:16.948 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-27T23:11:17.249 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-ceph-argparse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:17.525 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-27T23:11:17.667 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-09-27T23:11:17.916 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-volume. 2023-09-27T23:11:17.933 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:17.943 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-27T23:11:17.974 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-volume (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:18.236 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-27T23:11:18.369 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-mirror. 2023-09-27T23:11:18.386 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:18.427 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-mirror (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:18.521 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-27T23:11:18.672 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-27T23:11:18.764 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-colorama. 2023-09-27T23:11:18.780 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-09-27T23:11:18.822 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-09-27T23:11:19.062 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-27T23:11:19.247 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-shell. 2023-09-27T23:11:19.268 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:19.366 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-shell (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:19.366 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-27T23:11:19.618 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-top. 2023-09-27T23:11:19.633 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-27T23:11:19.635 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1161-g471b6dfd-1focal_all.deb ... 2023-09-27T23:11:19.676 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-top (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:19.912 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-27T23:11:19.929 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-27T23:11:19.960 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-27T23:11:19.970 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:20.278 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-27T23:11:20.491 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-27T23:11:20.509 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:20.549 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs-dev (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:20.620 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-27T23:11:20.921 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-27T23:11:20.944 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package nvme-cli. 2023-09-27T23:11:20.964 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-27T23:11:21.020 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-27T23:11:21.239 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-27T23:11:21.336 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-27T23:11:21.356 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-27T23:11:21.365 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-27T23:11:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:21.524 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-27T23:11:21.801 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-27T23:11:22.178 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-09-27T23:11:22.487 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-27T23:11:22.873 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-27T23:11:23.189 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-27T23:11:23.210 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-27T23:11:23.247 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-27T23:11:23.260 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-27T23:11:23.402 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:23.592 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-27T23:11:23.613 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-27T23:11:23.650 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-27T23:11:23.712 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:11:23.871 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-27T23:11:23.928 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-attr. 2023-09-27T23:11:23.949 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-09-27T23:11:23.986 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-27T23:11:24.206 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-27T23:11:24.347 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webencodings. 2023-09-27T23:11:24.351 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-27T23:11:24.364 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-27T23:11:24.405 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-27T23:11:24.478 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-27T23:11:24.733 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-html5lib. 2023-09-27T23:11:24.750 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-09-27T23:11:24.762 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-27T23:11:24.791 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-27T23:11:25.105 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:11:25.119 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-27T23:11:25.138 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-27T23:11:25.177 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-27T23:11:25.231 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-27T23:11:25.382 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-27T23:11:25.673 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-packaging. 2023-09-27T23:11:25.690 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-09-27T23:11:25.709 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-27T23:11:25.730 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-27T23:11:26.037 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-27T23:11:26.041 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastescript. 2023-09-27T23:11:26.060 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-09-27T23:11:26.100 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-27T23:11:26.338 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-27T23:11:26.420 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pluggy. 2023-09-27T23:11:26.438 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-09-27T23:11:26.464 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-27T23:11:26.477 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-27T23:11:26.782 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-27T23:11:26.807 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-psutil. 2023-09-27T23:11:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-27T23:11:26.865 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-27T23:11:27.066 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-27T23:11:27.176 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-py. 2023-09-27T23:11:27.197 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-27T23:11:27.250 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-27T23:11:27.350 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-27T23:11:27.571 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-27T23:11:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-27T23:11:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-27T23:11:27.743 INFO:teuthology.orchestra.run.smithi123.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-27T23:11:27.942 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pytest. 2023-09-27T23:11:27.960 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-09-27T23:11:28.000 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-27T23:11:28.020 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-27T23:11:28.146 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-27T23:11:28.371 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplejson. 2023-09-27T23:11:28.390 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-27T23:11:28.429 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-27T23:11:28.505 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-27T23:11:28.723 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-27T23:11:28.743 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-27T23:11:28.773 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-27T23:11:28.865 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-27T23:11:28.991 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-27T23:11:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-27T23:11:29.468 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-27T23:11:29.521 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package radosgw. 2023-09-27T23:11:29.539 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../114-radosgw_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:29.580 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking radosgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librados2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:29.871 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-27T23:11:31.326 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package rbd-fuse. 2023-09-27T23:11:31.334 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:31.343 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1161-g471b6dfd-1focal_amd64.deb ... 2023-09-27T23:11:31.384 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking rbd-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:31.460 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:11:31.619 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-27T23:11:31.686 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-27T23:11:32.013 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-27T23:11:32.038 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-27T23:11:32.139 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-27T23:11:32.266 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-27T23:11:32.347 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-27T23:11:32.501 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:32.656 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:11:32.765 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-27T23:11:33.108 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libcephfs2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:33.121 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user cephadm....done 2023-09-27T23:11:33.243 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-27T23:11:33.291 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:33.417 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-27T23:11:33.578 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libradosstriper1 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:33.704 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-27T23:11:33.764 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-27T23:11:34.047 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-27T23:11:34.057 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-27T23:11:34.192 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-27T23:11:34.334 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-27T23:11:34.669 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-27T23:11:35.071 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-27T23:11:35.110 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librbd1 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:35.236 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-27T23:11:35.350 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-27T23:11:35.384 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-27T23:11:35.513 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:11:35.639 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-27T23:11:35.642 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-09-27T23:11:35.932 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-27T23:11:35.943 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-27T23:11:36.041 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-27T23:11:36.236 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-27T23:11:36.419 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-modules-core (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:36.512 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-argparse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:36.545 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:36.772 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-27T23:11:36.773 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-27T23:11:36.796 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-27T23:11:36.956 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-09-27T23:11:37.230 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libcephfs-dev (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:37.240 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-27T23:11:37.381 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-27T23:11:37.525 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-27T23:11:37.666 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-27T23:11:37.809 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-27T23:11:37.935 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-27T23:11:37.967 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-27T23:11:38.320 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rados (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:38.320 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-27T23:11:38.445 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up librgw2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:38.571 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephfs-top (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:38.612 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-27T23:11:38.697 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rbd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:38.848 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up rbd-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:38.897 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-27T23:11:38.966 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-27T23:11:39.248 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-27T23:11:39.250 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-27T23:11:39.549 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-27T23:11:39.552 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-rgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-cephfs (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:39.829 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephfs-shell (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:39.900 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-27T23:11:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-27T23:11:40.210 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-27T23:11:40.511 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-27T23:11:40.637 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-27T23:11:40.779 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-27T23:11:40.861 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:41.056 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-27T23:11:41.429 INFO:teuthology.orchestra.run.smithi123.stdout:Adding group ceph....done 2023-09-27T23:11:41.465 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-09-27T23:11:41.775 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-27T23:11:41.823 INFO:teuthology.orchestra.run.smithi123.stdout:Adding system user ceph....done 2023-09-27T23:11:41.999 INFO:teuthology.orchestra.run.smithi123.stdout:Setting system user ceph properties....done 2023-09-27T23:11:42.020 INFO:teuthology.orchestra.run.smithi123.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-27T23:11:42.135 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-27T23:11:42.164 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-27T23:11:42.474 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-27T23:11:42.489 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-27T23:11:42.632 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:42.920 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up cephfs-mirror (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:42.967 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:11:43.101 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-27T23:11:43.141 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-27T23:11:43.141 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-27T23:11:43.461 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-27T23:11:43.580 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-test (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:43.587 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-27T23:11:43.697 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up radosgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:43.704 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-27T23:11:43.964 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-27T23:11:43.964 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-27T23:11:44.001 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-27T23:11:44.327 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:11:44.453 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-27T23:11:44.582 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-base (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:44.604 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-27T23:11:44.930 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-27T23:11:44.971 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-27T23:11:45.240 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-27T23:11:45.412 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mds (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:45.541 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-27T23:11:45.669 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-27T23:11:45.669 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-27T23:11:45.669 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-27T23:11:45.962 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-27T23:11:46.105 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:46.255 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-27T23:11:46.331 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-27T23:11:46.331 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-27T23:11:46.556 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-27T23:11:46.789 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-osd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:46.997 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-27T23:11:47.267 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-27T23:11:47.295 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-27T23:11:47.296 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-27T23:11:47.393 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-27T23:11:47.742 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:47.744 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-27T23:11:47.886 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:48.004 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mon (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:48.120 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-27T23:11:48.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-27T23:11:48.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-27T23:11:48.246 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-27T23:11:48.389 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-27T23:11:48.639 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:48.740 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-27T23:11:48.783 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:48.934 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-mgr-dashboard (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:49.041 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librados2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:49.076 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ceph-volume (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:49.167 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-27T23:11:49.212 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-27T23:11:49.388 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-27T23:11:49.656 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-27T23:11:50.673 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:11:50.958 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-27T23:11:51.343 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-27T23:11:51.644 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-27T23:11:51.937 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-27T23:11:52.063 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-27T23:11:52.397 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:52.523 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-27T23:11:52.593 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-27T23:11:52.875 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libradosstriper1 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-27T23:11:53.196 INFO:teuthology.orchestra.run.smithi123.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-27T23:11:53.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-09-27T23:11:53.267 INFO:teuthology.orchestra.run.smithi123.stdout:Reading package lists... 2023-09-27T23:11:53.327 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-27T23:11:53.468 INFO:teuthology.orchestra.run.smithi123.stdout:Building dependency tree... 2023-09-27T23:11:53.469 INFO:teuthology.orchestra.run.smithi123.stdout:Reading state information... 2023-09-27T23:11:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-09-27T23:11:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:flex is already the newest version (2.6.4-6.2). 2023-09-27T23:11:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-09-27T23:11:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-09-27T23:11:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-09-27T23:11:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages were automatically installed and are no longer required: 2023-09-27T23:11:53.668 INFO:teuthology.orchestra.run.smithi123.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-27T23:11:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-27T23:11:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:The following additional packages will be installed: 2023-09-27T23:11:53.669 INFO:teuthology.orchestra.run.smithi123.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-09-27T23:11:53.669 INFO:teuthology.orchestra.run.smithi123.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-09-27T23:11:53.669 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-09-27T23:11:53.669 INFO:teuthology.orchestra.run.smithi123.stdout: usb-modeswitch-data wpasupplicant 2023-09-27T23:11:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:Suggested packages: 2023-09-27T23:11:53.670 INFO:teuthology.orchestra.run.smithi123.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-09-27T23:11:53.671 INFO:teuthology.orchestra.run.smithi123.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-09-27T23:11:53.718 INFO:teuthology.orchestra.run.smithi123.stdout:The following NEW packages will be installed: 2023-09-27T23:11:53.719 INFO:teuthology.orchestra.run.smithi123.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-09-27T23:11:53.719 INFO:teuthology.orchestra.run.smithi123.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-09-27T23:11:53.720 INFO:teuthology.orchestra.run.smithi123.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-09-27T23:11:53.720 INFO:teuthology.orchestra.run.smithi123.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-09-27T23:11:53.721 INFO:teuthology.orchestra.run.smithi123.stdout:The following packages will be upgraded: 2023-09-27T23:11:53.722 INFO:teuthology.orchestra.run.smithi123.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-09-27T23:11:53.776 INFO:teuthology.orchestra.run.smithi123.stdout:6 upgraded, 23 newly installed, 0 to remove and 239 not upgraded. 2023-09-27T23:11:53.776 INFO:teuthology.orchestra.run.smithi123.stdout:Need to get 10.4 MB of archives. 2023-09-27T23:11:53.776 INFO:teuthology.orchestra.run.smithi123.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-09-27T23:11:53.776 INFO:teuthology.orchestra.run.smithi123.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-09-27T23:11:53.906 INFO:teuthology.orchestra.run.smithi123.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-09-27T23:11:53.907 INFO:teuthology.orchestra.run.smithi123.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-09-27T23:11:53.908 INFO:teuthology.orchestra.run.smithi123.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-09-27T23:11:53.914 INFO:teuthology.orchestra.run.smithi123.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-09-27T23:11:53.915 INFO:teuthology.orchestra.run.smithi123.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-09-27T23:11:53.930 INFO:teuthology.orchestra.run.smithi123.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-09-27T23:11:53.931 INFO:teuthology.orchestra.run.smithi123.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-09-27T23:11:53.932 INFO:teuthology.orchestra.run.smithi123.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-09-27T23:11:53.933 INFO:teuthology.orchestra.run.smithi123.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-09-27T23:11:53.933 INFO:teuthology.orchestra.run.smithi123.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-09-27T23:11:53.939 INFO:teuthology.orchestra.run.smithi123.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-09-27T23:11:53.940 INFO:teuthology.orchestra.run.smithi123.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-09-27T23:11:53.942 INFO:teuthology.orchestra.run.smithi123.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-09-27T23:11:53.942 INFO:teuthology.orchestra.run.smithi123.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-09-27T23:11:53.957 INFO:teuthology.orchestra.run.smithi123.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-09-27T23:11:53.957 INFO:teuthology.orchestra.run.smithi123.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-09-27T23:11:53.962 INFO:teuthology.orchestra.run.smithi123.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-09-27T23:11:53.969 INFO:teuthology.orchestra.run.smithi123.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-09-27T23:11:53.969 INFO:teuthology.orchestra.run.smithi123.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-09-27T23:11:53.983 INFO:teuthology.orchestra.run.smithi123.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-09-27T23:11:53.984 INFO:teuthology.orchestra.run.smithi123.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-09-27T23:11:53.991 INFO:teuthology.orchestra.run.smithi123.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-09-27T23:11:54.001 INFO:teuthology.orchestra.run.smithi123.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-09-27T23:11:54.016 INFO:teuthology.orchestra.run.smithi123.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-09-27T23:11:54.019 INFO:teuthology.orchestra.run.smithi123.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-09-27T23:11:54.019 INFO:teuthology.orchestra.run.smithi123.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-09-27T23:11:54.020 INFO:teuthology.orchestra.run.smithi123.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-09-27T23:11:54.020 INFO:teuthology.orchestra.run.smithi123.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-09-27T23:11:54.328 INFO:teuthology.orchestra.run.smithi123.stdout:Preconfiguring packages ... 2023-09-27T23:11:54.399 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librbd1 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:54.464 INFO:teuthology.orchestra.run.smithi123.stdout:Fetched 10.4 MB in 0s (35.9 MB/s) 2023-09-27T23:11:54.533 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-27T23:11:54.660 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-27T23:11:54.762 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152686 files and directories currently installed.) 2023-09-27T23:11:54.767 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-09-27T23:11:54.810 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-27T23:11:54.911 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-27T23:11:54.919 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-09-27T23:11:55.196 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-27T23:11:55.305 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-27T23:11:55.734 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-modules-core (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:55.860 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:56.067 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-09-27T23:11:56.084 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-27T23:11:56.084 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-27T23:11:56.545 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs-dev (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:56.696 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-27T23:11:56.972 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-27T23:11:57.282 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-27T23:11:57.461 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152686 files and directories currently installed.) 2023-09-27T23:11:57.466 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-09-27T23:11:57.683 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-09-27T23:11:57.685 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rados (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:57.802 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librgw2 (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:57.928 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-top (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:58.054 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rbd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:58.123 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-09-27T23:11:58.155 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up rbd-fuse (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:58.264 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-27T23:11:58.297 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-09-27T23:11:58.574 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-27T23:11:58.651 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-09-27T23:11:58.926 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:59.052 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cephfs (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:59.108 INFO:teuthology.orchestra.run.smithi123.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152686 files and directories currently installed.) 2023-09-27T23:11:59.112 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-09-27T23:11:59.178 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-shell (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:11:59.246 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-09-27T23:11:59.320 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-27T23:11:59.743 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-09-27T23:11:59.910 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-09-27T23:12:00.249 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-common (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:00.432 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-09-27T23:12:00.573 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-09-27T23:12:00.796 INFO:teuthology.orchestra.run.smithi064.stdout:Adding group ceph....done 2023-09-27T23:12:01.110 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-09-27T23:12:01.128 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-09-27T23:12:01.140 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user ceph....done 2023-09-27T23:12:01.168 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-09-27T23:12:01.350 INFO:teuthology.orchestra.run.smithi064.stdout:Setting system user ceph properties....done 2023-09-27T23:12:01.369 INFO:teuthology.orchestra.run.smithi064.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-27T23:12:01.523 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-27T23:12:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package dump. 2023-09-27T23:12:01.556 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-09-27T23:12:01.596 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking dump (0.4b46-6) ... 2023-09-27T23:12:01.857 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-27T23:12:01.908 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package indent. 2023-09-27T23:12:01.926 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-09-27T23:12:01.966 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-09-27T23:12:02.304 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-mirror (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:02.310 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-09-27T23:12:02.328 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-09-27T23:12:02.368 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-09-27T23:12:02.525 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-27T23:12:02.525 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-09-27T23:12:02.972 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-test (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:03.014 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-09-27T23:12:03.034 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-09-27T23:12:03.123 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up radosgw (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:03.198 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-09-27T23:12:03.400 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-27T23:12:03.400 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-27T23:12:03.509 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-09-27T23:12:03.530 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-09-27T23:12:03.567 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:03.946 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-09-27T23:12:03.963 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:04.004 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:04.042 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-base (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:04.306 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libmbim-proxy. 2023-09-27T23:12:04.324 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:04.364 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:04.427 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-27T23:12:04.692 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-09-27T23:12:04.709 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:04.767 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:04.886 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mds (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:05.087 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libndp0:amd64. 2023-09-27T23:12:05.106 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-09-27T23:12:05.137 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-27T23:12:05.137 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-27T23:12:05.145 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-09-27T23:12:05.490 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libnm0:amd64. 2023-09-27T23:12:05.510 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-09-27T23:12:05.548 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:05.724 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:05.942 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-09-27T23:12:05.957 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-27T23:12:05.957 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-27T23:12:05.962 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:06.000 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:06.353 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libqmi-proxy. 2023-09-27T23:12:06.371 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:06.411 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:06.417 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-osd (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:06.680 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-09-27T23:12:06.701 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-09-27T23:12:06.738 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-09-27T23:12:06.920 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-27T23:12:06.920 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-27T23:12:07.250 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-09-27T23:12:07.270 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-09-27T23:12:07.308 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-09-27T23:12:07.360 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:07.486 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:07.645 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mon (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:07.720 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package modemmanager. 2023-09-27T23:12:07.739 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:07.789 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:07.865 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-27T23:12:07.865 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-27T23:12:08.305 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-cephadm (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:08.448 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:08.574 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-dashboard (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:08.611 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package wpasupplicant. 2023-09-27T23:12:08.629 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-09-27T23:12:08.669 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-09-27T23:12:08.709 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-volume (17.2.6-1161-g471b6dfd-1focal) ... 2023-09-27T23:12:08.836 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-27T23:12:08.996 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-27T23:12:09.232 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package network-manager. 2023-09-27T23:12:09.253 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-09-27T23:12:09.289 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-27T23:12:09.315 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:10.056 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package ppp. 2023-09-27T23:12:10.075 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-09-27T23:12:10.323 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-09-27T23:12:10.686 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package pptp-linux. 2023-09-27T23:12:10.703 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-09-27T23:12:10.743 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-09-27T23:12:11.131 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package network-manager-pptp. 2023-09-27T23:12:11.147 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-09-27T23:12:11.198 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-09-27T23:12:11.483 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-09-27T23:12:11.500 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-09-27T23:12:11.541 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-09-27T23:12:11.945 INFO:teuthology.orchestra.run.smithi123.stdout:Selecting previously unselected package usb-modeswitch. 2023-09-27T23:12:11.964 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-09-27T23:12:12.013 INFO:teuthology.orchestra.run.smithi123.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-09-27T23:12:12.250 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-27T23:12:12.315 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-09-27T23:12:12.641 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-09-27T23:12:12.767 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-09-27T23:12:12.831 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-27T23:12:12.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-09-27T23:12:12.907 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-09-27T23:12:12.927 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-09-27T23:12:13.115 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-09-27T23:12:13.116 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-09-27T23:12:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-09-27T23:12:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:flex is already the newest version (2.6.4-6.2). 2023-09-27T23:12:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-09-27T23:12:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-09-27T23:12:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-09-27T23:12:13.316 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2023-09-27T23:12:13.317 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-27T23:12:13.317 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-27T23:12:13.317 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-09-27T23:12:13.317 INFO:teuthology.orchestra.run.smithi064.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-09-27T23:12:13.318 INFO:teuthology.orchestra.run.smithi064.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-09-27T23:12:13.318 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-09-27T23:12:13.318 INFO:teuthology.orchestra.run.smithi064.stdout: usb-modeswitch-data wpasupplicant 2023-09-27T23:12:13.319 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-09-27T23:12:13.319 INFO:teuthology.orchestra.run.smithi064.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-09-27T23:12:13.319 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-09-27T23:12:13.327 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-27T23:12:13.328 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-27T23:12:13.364 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-09-27T23:12:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-09-27T23:12:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-09-27T23:12:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-09-27T23:12:13.366 INFO:teuthology.orchestra.run.smithi064.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-09-27T23:12:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2023-09-27T23:12:13.367 INFO:teuthology.orchestra.run.smithi064.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-09-27T23:12:13.420 INFO:teuthology.orchestra.run.smithi064.stdout:6 upgraded, 23 newly installed, 0 to remove and 239 not upgraded. 2023-09-27T23:12:13.421 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 10.4 MB of archives. 2023-09-27T23:12:13.421 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-09-27T23:12:13.421 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-09-27T23:12:13.548 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-09-27T23:12:13.549 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-09-27T23:12:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-09-27T23:12:13.555 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-09-27T23:12:13.555 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-09-27T23:12:13.572 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-09-27T23:12:13.573 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-09-27T23:12:13.574 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-09-27T23:12:13.575 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-09-27T23:12:13.576 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-09-27T23:12:13.581 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-09-27T23:12:13.582 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-09-27T23:12:13.583 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-09-27T23:12:13.584 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-09-27T23:12:13.598 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-09-27T23:12:13.599 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-09-27T23:12:13.609 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-09-27T23:12:13.616 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-09-27T23:12:13.617 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-09-27T23:12:13.630 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-09-27T23:12:13.630 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-09-27T23:12:13.638 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-09-27T23:12:13.648 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-09-27T23:12:13.664 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-09-27T23:12:13.667 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-09-27T23:12:13.667 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-09-27T23:12:13.667 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-09-27T23:12:13.668 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-09-27T23:12:13.987 INFO:teuthology.orchestra.run.smithi064.stdout:Preconfiguring packages ... 2023-09-27T23:12:14.144 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 10.4 MB in 0s (35.5 MB/s) 2023-09-27T23:12:14.229 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-09-27T23:12:14.368 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152686 files and directories currently installed.) 2023-09-27T23:12:14.372 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-09-27T23:12:14.382 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-09-27T23:12:14.502 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-09-27T23:12:14.508 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-09-27T23:12:14.634 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-09-27T23:12:14.760 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:14.919 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-09-27T23:12:15.627 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-09-27T23:12:15.896 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-09-27T23:12:16.133 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up dump (0.4b46-6) ... 2023-09-27T23:12:16.229 INFO:teuthology.orchestra.run.smithi123.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-09-27T23:12:16.318 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:16.444 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-09-27T23:12:16.579 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-09-27T23:12:16.705 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-09-27T23:12:16.906 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-09-27T23:12:17.032 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:17.158 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:17.334 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-09-27T23:12:17.424 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152686 files and directories currently installed.) 2023-09-27T23:12:17.428 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-09-27T23:12:17.502 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:17.544 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-09-27T23:12:17.628 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:17.953 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-09-27T23:12:18.125 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout:The following network interfaces were found in /etc/network/interfaces 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout:which means they are currently configured by ifupdown: 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout:- enp3s0f1 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout:If you want to manage those interfaces with NetworkManager instead 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout:remove their configuration from /etc/network/interfaces. 2023-09-27T23:12:18.353 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:18.457 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-09-27T23:12:18.509 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-09-27T23:12:18.827 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-09-27T23:12:18.898 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152686 files and directories currently installed.) 2023-09-27T23:12:18.903 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-09-27T23:12:19.041 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-09-27T23:12:19.363 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-09-27T23:12:19.505 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-09-27T23:12:19.638 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-09-27T23:12:19.873 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:20.016 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-09-27T23:12:20.135 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-09-27T23:12:20.183 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:20.250 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-09-27T23:12:20.309 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:20.436 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:20.562 INFO:teuthology.orchestra.run.smithi123.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:20.754 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-09-27T23:12:20.772 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-09-27T23:12:20.813 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-09-27T23:12:20.856 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-09-27T23:12:20.856 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-09-27T23:12:21.182 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package dump. 2023-09-27T23:12:21.199 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-09-27T23:12:21.257 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking dump (0.4b46-6) ... 2023-09-27T23:12:21.398 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-09-27T23:12:21.498 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-09-27T23:12:21.716 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-27T23:12:21.867 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-27T23:12:22.129 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package indent. 2023-09-27T23:12:22.149 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-09-27T23:12:22.152 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-27T23:12:22.178 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-09-27T23:12:22.514 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-09-27T23:12:22.534 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-09-27T23:12:22.573 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-09-27T23:12:22.901 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-09-27T23:12:22.918 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-09-27T23:12:22.959 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-09-27T23:12:23.237 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-09-27T23:12:23.255 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-09-27T23:12:23.287 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:23.598 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-09-27T23:12:23.615 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:23.656 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:23.959 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmbim-proxy. 2023-09-27T23:12:23.979 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:24.017 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:24.329 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-09-27T23:12:24.349 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:24.412 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:24.706 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libndp0:amd64. 2023-09-27T23:12:24.726 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-09-27T23:12:24.756 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-09-27T23:12:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libnm0:amd64. 2023-09-27T23:12:25.127 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-09-27T23:12:25.167 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:25.537 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-09-27T23:12:25.557 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:25.595 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:25.939 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqmi-proxy. 2023-09-27T23:12:25.959 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:25.980 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-09-27T23:12:25.989 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:26.072 INFO:teuthology.orchestra.run.smithi123.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-09-27T23:12:26.208 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-09-27T23:12:26.227 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-09-27T23:12:26.266 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-09-27T23:12:26.804 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-09-27T23:12:26.824 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-09-27T23:12:26.862 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-09-27T23:12:26.894 INFO:teuthology.orchestra.run.smithi123.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-27T23:12:26.897 DEBUG:teuthology.parallel:result is None 2023-09-27T23:12:27.366 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package modemmanager. 2023-09-27T23:12:27.383 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-09-27T23:12:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:28.081 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package wpasupplicant. 2023-09-27T23:12:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-09-27T23:12:28.138 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-09-27T23:12:28.710 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package network-manager. 2023-09-27T23:12:28.727 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-09-27T23:12:28.785 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:29.509 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ppp. 2023-09-27T23:12:29.527 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-09-27T23:12:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-09-27T23:12:30.139 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package pptp-linux. 2023-09-27T23:12:30.161 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-09-27T23:12:30.214 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-09-27T23:12:30.551 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package network-manager-pptp. 2023-09-27T23:12:30.570 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-09-27T23:12:30.626 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-09-27T23:12:30.929 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-09-27T23:12:30.947 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-09-27T23:12:30.987 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-09-27T23:12:31.357 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usb-modeswitch. 2023-09-27T23:12:31.374 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-09-27T23:12:31.426 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-09-27T23:12:31.759 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-09-27T23:12:32.077 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-09-27T23:12:32.457 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-09-27T23:12:32.583 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-09-27T23:12:33.000 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-27T23:12:33.001 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-09-27T23:12:33.478 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-09-27T23:12:33.604 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-09-27T23:12:33.713 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-09-27T23:12:33.831 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-09-27T23:12:33.957 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:34.108 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-09-27T23:12:35.090 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-09-27T23:12:35.339 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up dump (0.4b46-6) ... 2023-09-27T23:12:35.434 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-09-27T23:12:35.498 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:35.624 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-09-27T23:12:35.750 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-09-27T23:12:35.868 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-09-27T23:12:36.036 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-09-27T23:12:36.154 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:36.280 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:36.406 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-09-27T23:12:36.607 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:36.733 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-09-27T23:12:37.456 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:37.456 INFO:teuthology.orchestra.run.smithi064.stdout:The following network interfaces were found in /etc/network/interfaces 2023-09-27T23:12:37.457 INFO:teuthology.orchestra.run.smithi064.stdout:which means they are currently configured by ifupdown: 2023-09-27T23:12:37.457 INFO:teuthology.orchestra.run.smithi064.stdout:- enp3s0f1 2023-09-27T23:12:37.457 INFO:teuthology.orchestra.run.smithi064.stdout:If you want to manage those interfaces with NetworkManager instead 2023-09-27T23:12:37.457 INFO:teuthology.orchestra.run.smithi064.stdout:remove their configuration from /etc/network/interfaces. 2023-09-27T23:12:37.457 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-09-27T23:12:37.954 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-09-27T23:12:38.478 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-09-27T23:12:38.962 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-09-27T23:12:39.105 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-09-27T23:12:39.281 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-09-27T23:12:39.407 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:39.533 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-09-27T23:12:39.659 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-09-27T23:12:39.960 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-09-27T23:12:39.960 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-09-27T23:12:40.503 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-09-27T23:12:40.604 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-09-27T23:12:40.722 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-27T23:12:40.865 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-27T23:12:41.158 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-27T23:12:45.077 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-09-27T23:12:45.186 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-09-27T23:12:46.025 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-27T23:12:46.035 DEBUG:teuthology.parallel:result is None 2023-09-27T23:12:46.036 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 2023-09-27T23:12:46.268 DEBUG:teuthology.orchestra.run.smithi064:> dpkg-query -W -f '${Version}' ceph 2023-09-27T23:12:46.293 INFO:teuthology.orchestra.run.smithi064.stdout:17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:12:46.293 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:12:46.293 INFO:teuthology.task.install:The correct ceph version 17.2.6-1161-g471b6dfd-1focal is installed. 2023-09-27T23:12:46.294 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=471b6dfd2d990acf7f03e5d073b84a1b39d1ad28 2023-09-27T23:12:46.507 DEBUG:teuthology.orchestra.run.smithi123:> dpkg-query -W -f '${Version}' ceph 2023-09-27T23:12:46.530 INFO:teuthology.orchestra.run.smithi123.stdout:17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:12:46.530 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1161-g471b6dfd-1focal 2023-09-27T23:12:46.530 INFO:teuthology.task.install:The correct ceph version 17.2.6-1161-g471b6dfd-1focal is installed. 2023-09-27T23:12:46.530 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-27T23:12:46.530 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:46.530 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-27T23:12:46.547 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:46.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-27T23:12:46.561 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-27T23:12:46.561 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:46.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2023-09-27T23:12:46.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-27T23:12:46.586 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:46.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-09-27T23:12:46.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-27T23:12:46.658 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-27T23:12:46.659 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:46.659 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-27T23:12:46.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-27T23:12:46.734 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:46.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-27T23:12:46.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-27T23:12:46.761 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-27T23:12:46.761 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:46.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2023-09-27T23:12:46.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-27T23:12:46.789 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:46.790 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-09-27T23:12:46.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-27T23:12:46.862 INFO:teuthology.run_tasks:Running task ceph... 2023-09-27T23:12:46.920 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-27T23:12:46.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2023-09-27T23:12:46.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2023-09-27T23:12:46.938 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-27T23:12:46.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-27T23:12:46.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-27T23:12:46.996 INFO:tasks.ceph:Creating extra log directories... 2023-09-27T23:12:46.996 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-27T23:12:46.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-27T23:12:47.061 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-09-27T23:12:47.061 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-09-27T23:12:47.061 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-09-27T23:12:47.061 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:47.061 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-27T23:12:47.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-27T23:12:47.149 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:47.149 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-27T23:12:47.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-27T23:12:47.242 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-27T23:12:47.242 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-27T23:12:47.243 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461', 'branch': 'wip-yuri-testing-2023-09-27-0906-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7403461', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-09-27-0906-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-09-27-0906-quincy', 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3538, 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-09-27-0906-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_471b6dfd2d990acf7f03e5d073b84a1b39d1ad28/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28', 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '471b6dfd2d990acf7f03e5d073b84a1b39d1ad28'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-27_19:23:53', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-27T23:12:47.243 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-27T23:12:47.246 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-27T23:12:47.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2023-09-27T23:12:47.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2023-09-27T23:12:47.313 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:47.313 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-09-27T23:12:47.318 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-27T23:12:47.318 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 360 Links: 1 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-27 23:07:18.617682787 +0000 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-27 23:07:18.345689047 +0000 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-27 23:07:18.345689047 +0000 2023-09-27T23:12:47.324 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-09-27T23:12:47.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-27T23:12:47.381 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-27T23:12:47.381 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-27T23:12:47.381 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000406975 s, 1.3 MB/s 2023-09-27T23:12:47.382 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-27T23:12:47.390 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 364 Links: 1 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-27 23:07:18.845677542 +0000 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-27 23:07:18.841677633 +0000 2023-09-27T23:12:47.396 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-27 23:07:18.841677633 +0000 2023-09-27T23:12:47.397 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-09-27T23:12:47.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-27T23:12:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-27T23:12:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-27T23:12:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000404119 s, 1.3 MB/s 2023-09-27T23:12:47.455 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-27T23:12:47.464 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-27 23:07:19.613659868 +0000 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-27 23:07:19.349665942 +0000 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-27 23:07:19.349665942 +0000 2023-09-27T23:12:47.471 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-09-27T23:12:47.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-27T23:12:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-27T23:12:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-27T23:12:47.484 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000366286 s, 1.4 MB/s 2023-09-27T23:12:47.485 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-27T23:12:47.535 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-09-27T23:12:47.543 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-27T23:12:47.543 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.543 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 377 Links: 1 2023-09-27T23:12:47.544 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.544 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-27 23:07:20.105648546 +0000 2023-09-27T23:12:47.544 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-27 23:07:19.841654621 +0000 2023-09-27T23:12:47.544 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-27 23:07:19.841654621 +0000 2023-09-27T23:12:47.544 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-09-27T23:12:47.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-27T23:12:47.557 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-27T23:12:47.557 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-27T23:12:47.557 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000425271 s, 1.2 MB/s 2023-09-27T23:12:47.558 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-27T23:12:47.607 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-27T23:12:47.608 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:47.608 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-09-27T23:12:47.615 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-27T23:12:47.615 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-09-27T23:12:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-27T23:12:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.662 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 361 Links: 1 2023-09-27T23:12:47.662 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.662 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-27 23:07:25.441743463 +0000 2023-09-27T23:12:47.662 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-27 23:07:25.437743556 +0000 2023-09-27T23:12:47.663 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-27 23:07:25.437743556 +0000 2023-09-27T23:12:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-27T23:12:47.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-27T23:12:47.673 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-27T23:12:47.673 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-27T23:12:47.673 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000390052 s, 1.3 MB/s 2023-09-27T23:12:47.674 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-27T23:12:47.681 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-09-27T23:12:47.729 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 367 Links: 1 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-27 23:07:26.197726038 +0000 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-27 23:07:25.945731846 +0000 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-27 23:07:25.945731846 +0000 2023-09-27T23:12:47.730 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-27T23:12:47.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-27T23:12:47.743 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-27T23:12:47.744 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-27T23:12:47.744 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000390588 s, 1.3 MB/s 2023-09-27T23:12:47.745 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-27T23:12:47.753 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-09-27T23:12:47.802 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-27T23:12:47.802 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.802 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-09-27T23:12:47.802 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.802 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-27 23:07:26.689714698 +0000 2023-09-27T23:12:47.803 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-27 23:07:26.441720414 +0000 2023-09-27T23:12:47.803 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-27 23:07:26.441720414 +0000 2023-09-27T23:12:47.803 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-27T23:12:47.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-27T23:12:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-27T23:12:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-27T23:12:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000408274 s, 1.3 MB/s 2023-09-27T23:12:47.818 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-27T23:12:47.827 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-09-27T23:12:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-27T23:12:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-27T23:12:47.882 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 379 Links: 1 2023-09-27T23:12:47.882 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-27T23:12:47.882 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-27 23:07:27.177703449 +0000 2023-09-27T23:12:47.882 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-27 23:07:26.913709534 +0000 2023-09-27T23:12:47.883 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-27 23:07:26.913709534 +0000 2023-09-27T23:12:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-27T23:12:47.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-27T23:12:47.894 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-27T23:12:47.894 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-27T23:12:47.894 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000417755 s, 1.2 MB/s 2023-09-27T23:12:47.896 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-27T23:12:47.904 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-27T23:12:47.904 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-27T23:12:47.904 INFO:tasks.ceph:Generating config... 2023-09-27T23:12:47.906 INFO:tasks.ceph:[client] client mount timeout = 600 2023-09-27T23:12:47.906 INFO:tasks.ceph:[client] debug client = 20 2023-09-27T23:12:47.906 INFO:tasks.ceph:[client] debug ms = 1 2023-09-27T23:12:47.906 INFO:tasks.ceph:[client] pid file = 2023-09-27T23:12:47.906 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-09-27T23:12:47.906 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-09-27T23:12:47.906 INFO:tasks.ceph:[mds] debug mds = 20 2023-09-27T23:12:47.906 INFO:tasks.ceph:[mds] debug ms = 1 2023-09-27T23:12:47.906 INFO:tasks.ceph:[mds] mds debug frag = True 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mgr] debug client = 20 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-27T23:12:47.907 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-27T23:12:47.908 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-27T23:12:47.908 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-27T23:12:47.909 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-09-27T23:12:47.909 INFO:tasks.ceph:Setting up mon.a... 2023-09-27T23:12:47.909 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-27T23:12:47.944 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2023-09-27T23:12:47.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-27T23:12:48.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-27T23:12:48.034 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.123'), ('mon.c', '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]')] 2023-09-27T23:12:48.037 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-09-27T23:12:48.038 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:48.038 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-27T23:12:48.046 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.123 --addv c '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid 9497c172-4ccf-4119-8910-f606cb59c9f0 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = octopus 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 9497c172-4ccf-4119-8910-f606cb59c9f0 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2023-09-27T23:12:48.087294+0000 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:created 2023-09-27T23:12:48.087294+0000 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 15 (octopus) 2023-09-27T23:12:48.087 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2023-09-27T23:12:48.088 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2023-09-27T23:12:48.088 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2023-09-27T23:12:48.088 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] mon.c 2023-09-27T23:12:48.088 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-27T23:12:48.089 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-27T23:12:48.137 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9497c172-4ccf-4119-8910-f606cb59c9f0... 2023-09-27T23:12:48.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-27T23:12:48.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-27T23:12:48.172 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-09-27T23:12:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-09-27T23:12:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.64,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 9497c172-4ccf-4119-8910-f606cb59c9f0 2023-09-27T23:12:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-09-27T23:12:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2023-09-27T23:12:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-09-27T23:12:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: debug finisher = 20 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-09-27T23:12:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-27T23:12:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-27T23:12:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-09-27T23:12:48.184 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-09-27T23:12:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2023-09-27T23:12:48.187 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2023-09-27T23:12:48.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-27T23:12:48.189 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-09-27T23:12:48.190 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 9497c172-4ccf-4119-8910-f606cb59c9f0 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.191 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2023-09-27T23:12:48.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 20 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 20 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 10 2023-09-27T23:12:48.193 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.194 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-09-27T23:12:48.195 INFO:teuthology.orchestra.run.smithi064.stdout: debug finisher = 20 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2023-09-27T23:12:48.196 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2023-09-27T23:12:48.197 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2023-09-27T23:12:48.198 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-09-27T23:12:48.199 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-09-27T23:12:48.200 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2023-09-27T23:12:48.201 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2023-09-27T23:12:48.201 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:12:48.201 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2023-09-27T23:12:48.201 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2023-09-27T23:12:48.201 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2023-09-27T23:12:48.201 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2023-09-27T23:12:48.202 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-09-27T23:12:48.202 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-09-27T23:12:48.202 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-09-27T23:12:48.202 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-27T23:12:48.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-27T23:12:48.279 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-27T23:12:48.280 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:48.280 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-27T23:12:48.286 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:48.287 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-27T23:12:48.293 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:12:48.293 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:48.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-27T23:12:48.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-27T23:12:48.361 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:12:48.361 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:48.366 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:12:48.366 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:48.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-27T23:12:48.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-27T23:12:48.392 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:12:48.392 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:48.399 INFO:tasks.ceph:Setting up mon nodes... 2023-09-27T23:12:48.400 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-27T23:12:48.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-27T23:12:48.445 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-27T23:12:48.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-27T23:12:48.492 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-27T23:12:48.495 INFO:tasks.ceph:Setting up mds nodes... 2023-09-27T23:12:48.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-27T23:12:48.541 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-27T23:12:48.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-27T23:12:48.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-09-27T23:12:48.595 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-09-27T23:12:48.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-09-27T23:12:48.608 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-09-27T23:12:48.653 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-09-27T23:12:48.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-09-27T23:12:48.666 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-09-27T23:12:48.705 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-09-27T23:12:48.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-09-27T23:12:48.718 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-27T23:12:48.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-27T23:12:48.756 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-27T23:12:48.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-09-27T23:12:48.846 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-09-27T23:12:48.859 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-27T23:12:48.896 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-27T23:12:48.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-09-27T23:12:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-09-27T23:12:49.001 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-27T23:12:49.001 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-27T23:12:49.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-27T23:12:49.018 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-27T23:12:49.018 INFO:tasks.ceph:role: osd.0 2023-09-27T23:12:49.018 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:12:49.018 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:12:49.556 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:12:49.557 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:12:49.557 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:12:49.557 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:12:49.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-09-27T23:12:49.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-27T23:12:49.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-27T23:12:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:12:49.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:12:49.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-27T23:12:49.680 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-27T23:12:49.680 INFO:tasks.ceph:role: osd.1 2023-09-27T23:12:49.680 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:12:49.681 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-27T23:12:50.188 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:12:50.189 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:12:50.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-09-27T23:12:50.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-27T23:12:50.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-27T23:12:50.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:12:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:12:50.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-27T23:12:50.265 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-27T23:12:50.265 INFO:tasks.ceph:role: osd.2 2023-09-27T23:12:50.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:12:50.266 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-27T23:12:50.727 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:12:50.727 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:12:50.727 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:12:50.727 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-09-27T23:12:50.727 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:12:50.728 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:12:50.728 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:12:50.728 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:12:50.728 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:12:50.728 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:12:50.729 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-09-27T23:12:50.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-27T23:12:50.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-27T23:12:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:12:50.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:12:50.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-27T23:12:50.803 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-27T23:12:50.803 INFO:tasks.ceph:role: osd.3 2023-09-27T23:12:50.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:12:50.803 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-27T23:12:51.311 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:12:51.312 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:12:51.313 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:12:51.314 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-09-27T23:12:51.314 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-27T23:12:51.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-27T23:12:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:12:51.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:12:51.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:51.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:51.477+0000 7f60c6c253c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-27T23:12:51.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:51.477+0000 7f60c6c253c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-27T23:12:51.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:51.477+0000 7f60c6c253c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-27T23:12:51.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:51.477+0000 7f60c6c253c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-27T23:12:51.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:51.477+0000 7f60c6c253c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-27T23:12:51.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:51.481+0000 7f60c6c253c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-27T23:12:52.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:52.289+0000 7f60c6c253c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:53.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:53.065+0000 7f60c6c253c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:53.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:53.849+0000 7f60c6c253c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:54.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-27T23:12:54.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:54.784+0000 7f4a1b93b3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-27T23:12:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:54.784+0000 7f4a1b93b3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-27T23:12:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:54.784+0000 7f4a1b93b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-27T23:12:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:54.784+0000 7f4a1b93b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-27T23:12:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:54.784+0000 7f4a1b93b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-27T23:12:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:54.784+0000 7f4a1b93b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-27T23:12:55.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:55.344+0000 7f4a1b93b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:56.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:56.124+0000 7f4a1b93b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:56.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:56.908+0000 7f4a1b93b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:57.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-27T23:12:57.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:12:57.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:57.808+0000 7f809950a3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-27T23:12:57.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:57.808+0000 7f809950a3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-27T23:12:57.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:57.808+0000 7f809950a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-27T23:12:57.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:57.808+0000 7f809950a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-27T23:12:57.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:57.808+0000 7f809950a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-27T23:12:57.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:57.812+0000 7f809950a3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-27T23:12:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:58.368+0000 7f809950a3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:59.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:59.156+0000 7f809950a3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:12:59.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:12:59.932+0000 7f809950a3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:00.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-27T23:13:00.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:00.908+0000 7fc19908f3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-27T23:13:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:00.912+0000 7fc19908f3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-27T23:13:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:00.912+0000 7fc19908f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-27T23:13:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:00.912+0000 7fc19908f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-27T23:13:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:00.912+0000 7fc19908f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-27T23:13:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:00.912+0000 7fc19908f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-27T23:13:01.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:01.476+0000 7fc19908f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:02.260+0000 7fc19908f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:03.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:03.048+0000 7fc19908f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:03.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-27T23:13:03.935 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-09-27T23:13:03.952 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-27T23:13:03.952 INFO:tasks.ceph:role: osd.4 2023-09-27T23:13:03.952 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:13:03.952 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-27T23:13:04.471 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:13:04.471 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:13:04.471 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:13:04.471 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2023-09-27T23:13:04.471 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:13:04.472 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:13:04.472 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:13:04.472 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:13:04.472 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:13:04.472 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:13:04.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-09-27T23:13:04.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-09-27T23:13:04.575 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-09-27T23:13:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:13:04.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:13:04.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-09-27T23:13:04.647 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-27T23:13:04.647 INFO:tasks.ceph:role: osd.5 2023-09-27T23:13:04.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:13:04.648 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:13:05.145 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:13:05.146 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:13:05.146 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:13:05.147 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-09-27T23:13:05.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-09-27T23:13:05.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-09-27T23:13:05.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:13:05.242 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:13:05.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-09-27T23:13:05.253 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-27T23:13:05.253 INFO:tasks.ceph:role: osd.6 2023-09-27T23:13:05.253 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:13:05.254 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-27T23:13:05.755 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:13:05.756 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:13:05.757 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:13:05.757 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:13:05.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-09-27T23:13:05.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-09-27T23:13:05.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-09-27T23:13:05.828 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:13:05.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:13:05.830 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-09-27T23:13:05.840 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-27T23:13:05.840 INFO:tasks.ceph:role: osd.7 2023-09-27T23:13:05.841 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:13:05.841 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-27T23:13:06.339 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-27T23:13:06.340 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-27T23:13:06.342 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-09-27T23:13:06.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-09-27T23:13:06.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-09-27T23:13:06.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:13:06.412 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: /sbin/restorecon: command not found 2023-09-27T23:13:06.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:06.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:06.466+0000 7f52c251c3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-09-27T23:13:06.476 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:06.470+0000 7f52c251c3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-09-27T23:13:06.476 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:06.470+0000 7f52c251c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-27T23:13:06.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:06.470+0000 7f52c251c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-27T23:13:06.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:06.470+0000 7f52c251c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-27T23:13:06.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:06.478+0000 7f52c251c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-09-27T23:13:07.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:07.278+0000 7f52c251c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:08.062+0000 7f52c251c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:08.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:08.846+0000 7f52c251c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:09.720 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-09-27T23:13:09.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:09.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:09.762+0000 7f3bf38f13c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-09-27T23:13:09.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:09.766+0000 7f3bf38f13c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-09-27T23:13:09.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:09.766+0000 7f3bf38f13c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-27T23:13:09.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:09.766+0000 7f3bf38f13c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-27T23:13:09.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:09.766+0000 7f3bf38f13c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-27T23:13:09.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:09.766+0000 7f3bf38f13c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-09-27T23:13:10.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:10.326+0000 7f3bf38f13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:11.120 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:11.114+0000 7f3bf38f13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:11.900 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:11.894+0000 7f3bf38f13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:12.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-09-27T23:13:12.782 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:12.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:12.858+0000 7fc3d7e153c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-09-27T23:13:12.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:12.858+0000 7fc3d7e153c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-09-27T23:13:12.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:12.858+0000 7fc3d7e153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-27T23:13:12.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:12.858+0000 7fc3d7e153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-27T23:13:12.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:12.858+0000 7fc3d7e153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-27T23:13:12.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:12.858+0000 7fc3d7e153c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-09-27T23:13:13.418 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:13.410+0000 7fc3d7e153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:14.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:14.202+0000 7fc3d7e153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:14.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:14.982+0000 7fc3d7e153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:15.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-09-27T23:13:15.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:15.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:15.954+0000 7fd3de4153c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-09-27T23:13:15.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:15.954+0000 7fd3de4153c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-09-27T23:13:15.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:15.954+0000 7fd3de4153c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-27T23:13:15.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:15.954+0000 7fd3de4153c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-27T23:13:15.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:15.954+0000 7fd3de4153c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-27T23:13:15.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:15.954+0000 7fd3de4153c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-09-27T23:13:16.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:16.514+0000 7fd3de4153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:17.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:13:17.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:13:17.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:17.306+0000 7fd3de4153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:18.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:18.102+0000 7fd3de4153c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-27T23:13:18.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-09-27T23:13:18.973 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-27T23:13:18.973 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:18.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-27T23:13:18.990 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:18.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-27T23:13:19.005 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-09-27T23:13:19.064 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-27T23:13:19.073 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-27T23:13:19.089 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-27T23:13:19.100 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-27T23:13:19.110 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-27T23:13:19.125 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-09-27T23:13:19.140 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-09-27T23:13:19.154 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-09-27T23:13:19.166 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-09-27T23:13:19.221 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-09-27T23:13:19.235 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-09-27T23:13:19.247 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.247 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-27T23:13:19.254 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:19.254 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-09-27T23:13:19.261 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.261 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-27T23:13:19.297 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:19.297 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-09-27T23:13:19.304 INFO:tasks.ceph:Adding keys to all mons... 2023-09-27T23:13:19.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-27T23:13:19.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ== 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ== 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA== 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2023-09-27T23:13:19.319 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg== 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ== 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA== 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA== 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw== 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw== 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-09-27T23:13:19.320 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g== 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ== 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w== 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w== 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ== 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA== 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2023-09-27T23:13:19.321 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw== 2023-09-27T23:13:19.322 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2023-09-27T23:13:19.322 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg== 2023-09-27T23:13:19.322 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2023-09-27T23:13:19.322 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw== 2023-09-27T23:13:19.323 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2023-09-27T23:13:19.323 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ== 2023-09-27T23:13:19.323 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-09-27T23:13:19.323 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ== 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA== 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg== 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ== 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA== 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA== 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2023-09-27T23:13:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw== 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw== 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g== 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ== 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w== 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w== 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2023-09-27T23:13:19.325 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ== 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA== 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw== 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg== 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2023-09-27T23:13:19.326 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw== 2023-09-27T23:13:19.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-27T23:13:19.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-27T23:13:19.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-27T23:13:19.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-27T23:13:19.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.877 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-27T23:13:19.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:19.971 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:20.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:20.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:20.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:20.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-27T23:13:20.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.284 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-27T23:13:20.372 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-27T23:13:20.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-27T23:13:20.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-27T23:13:20.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-27T23:13:20.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-27T23:13:20.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-27T23:13:20.543 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-27T23:13:20.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-27T23:13:20.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-27T23:13:20.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-27T23:13:20.637 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:20.640 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-27T23:13:20.646 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-27T23:13:20.646 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-27T23:13:20.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-27T23:13:20.649 INFO:tasks.ceph.mon.a:Started 2023-09-27T23:13:20.649 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-27T23:13:20.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-27T23:13:20.689 INFO:tasks.ceph.mon.b:Started 2023-09-27T23:13:20.689 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-27T23:13:20.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-27T23:13:20.691 INFO:tasks.ceph.mon.c:Started 2023-09-27T23:13:20.691 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-27T23:13:20.691 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-27T23:13:20.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-27T23:13:20.694 INFO:tasks.ceph.mgr.y:Started 2023-09-27T23:13:20.694 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-27T23:13:20.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-27T23:13:20.696 INFO:tasks.ceph.mgr.x:Started 2023-09-27T23:13:20.696 INFO:tasks.ceph:Setting crush tunables to default 2023-09-27T23:13:20.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2023-09-27T23:13:20.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.799+0000 7fcd18a27700 1 Processor -- start 2023-09-27T23:13:20.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.799+0000 7fcd18a27700 1 -- start start 2023-09-27T23:13:20.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd18a27700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd18a27700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd18a27700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd140cc7e0 con 0x7fcd140c8f40 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd18a27700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd140cbbf0 con 0x7fcd14142680 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd18a27700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd140cbeb0 con 0x7fcd1406d760 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1259c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcd140c8f40 0x7fcd1406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:50120/0 (socket says 172.21.15.64:50120) 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1259c700 1 -- 172.21.15.64:0/2501642413 learned_addr learned my addr 172.21.15.64:0/2501642413 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1359e700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 msgr2=0x7fcd140ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1359e700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 msgr2=0x7fcd140ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-27T23:13:20.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-27T23:13:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:13:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 msgr2=0x7fcd140c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-27T23:13:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 msgr2=0x7fcd140c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-27T23:13:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-27T23:13:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:20.803+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd1359e700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 msgr2=0x7fcd140ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd1359e700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 msgr2=0x7fcd140ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 msgr2=0x7fcd140c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 msgr2=0x7fcd140c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:13:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-27T23:13:21.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.007+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:13:21.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829547943 0 0) 0x7fcd140cc7e0 con 0x7fcd140c8f40 2023-09-27T23:13:21.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd00003570 con 0x7fcd140c8f40 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505441786 0 0) 0x7fcd00003570 con 0x7fcd140c8f40 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 msgr2=0x7fcd140ca7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.123:3300/0 conn(0x7fcd1406d760 0x7fcd140ca7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 msgr2=0x7fcd140c8a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 --2- 172.21.15.64:0/2501642413 >> v2:172.21.15.64:3300/0 conn(0x7fcd14142680 0x7fcd140c8a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140cc190 con 0x7fcd140c8f40 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fcd04001470 con 0x7fcd140c8f40 2023-09-27T23:13:21.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 >> v1:172.21.15.123:6789/0 conn(0x7fcd140c8f40 legacy=0x7fcd1406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00003b30 0x7fcd00003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00004810 0x7fcd00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd000052e0 0x7fcd0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140cbbf0 con 0x7fcd00003b30 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140cbeb0 con 0x7fcd00004810 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0000c320 con 0x7fcd000052e0 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00003b30 0x7fcd00003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd1259c700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00004810 0x7fcd00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd1359e700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd000052e0 0x7fcd0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd000052e0 msgr2=0x7fcd0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd000052e0 0x7fcd0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00004810 msgr2=0x7fcd00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00004810 0x7fcd00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd1359e700 1 -- 172.21.15.64:0/2501642413 reap_dead start 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 -- 172.21.15.64:0/2501642413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00003570 con 0x7fcd00003b30 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00003b30 0x7fcd00003f40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00dbf0 tx=0x7fccfc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e967732d67ef55d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00d560 con 0x7fcd00003b30 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc00f720 con 0x7fcd00003b30 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.019+0000 7fcd11d9b700 1 -- 172.21.15.64:0/2501642413 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00f8d0 con 0x7fcd00003b30 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 -- 172.21.15.64:0/2501642413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00003b30 msgr2=0x7fcd00003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00003b30 0x7fcd00003f40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00dbf0 tx=0x7fccfc008040 comp rx=0 tx=0).stop 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 -- 172.21.15.64:0/2501642413 shutdown_connections 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 --2- 172.21.15.64:0/2501642413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00003b30 0x7fcd00003f40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 -- 172.21.15.64:0/2501642413 >> 172.21.15.64:0/2501642413 conn(0x7fcd140ccb30 msgr2=0x7fcd140ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 -- 172.21.15.64:0/2501642413 shutdown_connections 2023-09-27T23:13:21.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 -- 172.21.15.64:0/2501642413 wait complete. 2023-09-27T23:13:21.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 Processor -- start 2023-09-27T23:13:21.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.023+0000 7fcd18a27700 1 -- start start 2023-09-27T23:13:21.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd18a27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00003b30 0x7fcd140b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd18a27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 0x7fcd1416d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd18a27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 0x7fcd140c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd18a27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14069ac0 con 0x7fcd1416ce90 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd18a27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c4060 con 0x7fcd00003b30 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd18a27700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c4670 con 0x7fcd140c61f0 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 0x7fcd1416d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 0x7fcd140c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 0x7fcd140c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:47798/0 (socket says 172.21.15.64:47798) 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 0x7fcd1416d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41732/0 (socket says 172.21.15.64:41732) 2023-09-27T23:13:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 -- 172.21.15.64:0/4172986585 learned_addr learned my addr 172.21.15.64:0/4172986585 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:21.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd12d9d700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00003b30 0x7fcd140b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 -- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00003b30 msgr2=0x7fcd140b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00003b30 0x7fcd140b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 -- 172.21.15.64:0/4172986585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 msgr2=0x7fcd1416d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 0x7fcd1416d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 -- 172.21.15.64:0/4172986585 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140c4950 con 0x7fcd140c61f0 2023-09-27T23:13:21.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1259c700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 0x7fcd1416d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:21.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.027+0000 7fcd1359e700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 0x7fcd140c6660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009590 tx=0x7fcd0c0097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.031+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c002d30 con 0x7fcd140c61f0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.031+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c00f040 con 0x7fcd140c61f0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.031+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0166b0 con 0x7fcd140c61f0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.031+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd140c4c30 con 0x7fcd140c61f0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.031+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd140b0a30 con 0x7fcd140c61f0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.035+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fcd0c003040 con 0x7fcd140c61f0 2023-09-27T23:13:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.035+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcd0c020410 con 0x7fcd140c61f0 2023-09-27T23:13:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.035+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd140c41a0 con 0x7fcd140c61f0 2023-09-27T23:13:21.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.039+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7fcd140c41a0 con 0x7fcd140c61f0 2023-09-27T23:13:21.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.115+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcd1404be40 con 0x7fcd140c61f0 2023-09-27T23:13:21.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.123+0000 7fcd0affd700 1 -- 172.21.15.64:0/4172986585 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcd0c002e90 con 0x7fcd140c61f0 2023-09-27T23:13:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 msgr2=0x7fcd140c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 0x7fcd140c6660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009590 tx=0x7fcd0c0097d0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 shutdown_connections 2023-09-27T23:13:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd140c61f0 0x7fcd140c6660 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd1416ce90 0x7fcd1416d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 --2- 172.21.15.64:0/4172986585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd00003b30 0x7fcd140b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 >> 172.21.15.64:0/4172986585 conn(0x7fcd140ccb30 msgr2=0x7fcd140adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 shutdown_connections 2023-09-27T23:13:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:21.127+0000 7fcd18a27700 1 -- 172.21.15.64:0/4172986585 wait complete. 2023-09-27T23:13:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2023-09-27T23:13:21.147 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-27T23:13:21.147 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:21.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-27T23:13:21.163 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:21.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-27T23:13:21.175 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:21.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-27T23:13:21.188 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-27T23:13:21.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-27T23:13:21.200 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:21.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-09-27T23:13:21.215 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:21.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-09-27T23:13:21.246 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:21.238+0000 7fbbaa77e000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:13:21.260 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:21.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-09-27T23:13:21.315 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-27T23:13:21.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-09-27T23:13:21.325 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 915f831e-9c10-4ed9-8012-6e415f3f0fe3 0 2023-09-27T23:13:21.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 Processor -- start 2023-09-27T23:13:21.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 -- start start 2023-09-27T23:13:21.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6438150200 0x7f64380c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64380ace30 0x7f64380aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64380af250 con 0x7f64380ae770 2023-09-27T23:13:21.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64380af530 con 0x7f64380ace30 2023-09-27T23:13:21.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643dd83700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64380ca8a0 con 0x7f6438150200 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6438150200 0x7f64380c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6438150200 0x7f64380c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45474/0 (socket says 172.21.15.123:45474) 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 -- 172.21.15.123:0/1781394147 learned_addr learned my addr 172.21.15.123:0/1781394147 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f6437fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f64380ae770 0x7f64380ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36236/0 (socket says 172.21.15.123:36236) 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643d582700 1 --2- 172.21.15.123:0/1781394147 >> v2:172.21.15.64:3300/0 conn(0x7f64380ace30 0x7f64380aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 -- 172.21.15.123:0/1781394147 >> v2:172.21.15.64:3300/0 conn(0x7f64380ace30 msgr2=0x7f64380aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 --2- 172.21.15.123:0/1781394147 >> v2:172.21.15.64:3300/0 conn(0x7f64380ace30 0x7f64380aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 -- 172.21.15.123:0/1781394147 >> v1:172.21.15.123:6789/0 conn(0x7f64380ae770 legacy=0x7f64380ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f64377fe700 1 -- 172.21.15.123:0/1781394147 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411181790 0 0) 0x7f64380af250 con 0x7f64380ae770 2023-09-27T23:13:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 -- 172.21.15.123:0/1781394147 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380cab80 con 0x7f6438150200 2023-09-27T23:13:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f643cd81700 1 --2- 172.21.15.123:0/1781394147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438150200 0x7f64380c7170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f643806b4e0 tx=0x7f642c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44ce0c1753fc929 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f64377fe700 1 -- 172.21.15.123:0/1781394147 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c009e70 con 0x7f6438150200 2023-09-27T23:13:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f64377fe700 1 -- 172.21.15.123:0/1781394147 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c00b860 con 0x7f6438150200 2023-09-27T23:13:21.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.422+0000 7f64377fe700 1 -- 172.21.15.123:0/1781394147 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00bc00 con 0x7f6438150200 2023-09-27T23:13:21.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- 172.21.15.123:0/1781394147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438150200 msgr2=0x7f64380c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 --2- 172.21.15.123:0/1781394147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438150200 0x7f64380c7170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f643806b4e0 tx=0x7f642c009250 comp rx=0 tx=0).stop 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- 172.21.15.123:0/1781394147 shutdown_connections 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 --2- 172.21.15.123:0/1781394147 >> v2:172.21.15.64:3300/0 conn(0x7f64380ace30 0x7f64380aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 --2- 172.21.15.123:0/1781394147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438150200 0x7f64380c7170 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- 172.21.15.123:0/1781394147 >> 172.21.15.123:0/1781394147 conn(0x7f64380ccb30 msgr2=0x7f64380ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- 172.21.15.123:0/1781394147 shutdown_connections 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- 172.21.15.123:0/1781394147 wait complete. 2023-09-27T23:13:21.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 Processor -- start 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- start start 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64380ace30 0x7f64380a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 0x7f64380aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6438150200 0x7f64380c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380c5d00 con 0x7f64380ace30 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380ab2e0 con 0x7f64380ae770 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643dd83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380c5410 con 0x7f6438150200 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 0x7f64380aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643d582700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6438150200 0x7f64380c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 0x7f64380aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45482/0 (socket says 172.21.15.123:45482) 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643d582700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6438150200 0x7f64380c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:45570/0 (socket says 172.21.15.123:45570) 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 -- 172.21.15.123:0/1764816834 learned_addr learned my addr 172.21.15.123:0/1764816834 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f643cd81700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64380ace30 0x7f64380a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 -- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6438150200 msgr2=0x7f64380c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6438150200 0x7f64380c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 -- 172.21.15.123:0/1764816834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64380ace30 msgr2=0x7f64380a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64380ace30 0x7f64380a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.426+0000 7f6437fff700 1 -- 172.21.15.123:0/1764816834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0094f0 con 0x7f64380ae770 2023-09-27T23:13:21.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f6437fff700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 0x7f64380aada0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64240011e0 tx=0x7f6424007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424005a70 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424005bd0 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64380c56b0 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64240070d0 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64380a4a20 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f642401c440 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f64240076c0 con 0x7f64380ae770 2023-09-27T23:13:21.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.430+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64380c5e70 con 0x7f64380ae770 2023-09-27T23:13:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.434+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f6424017070 con 0x7f64380ae770 2023-09-27T23:13:21.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.510+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "915f831e-9c10-4ed9-8012-6e415f3f0fe3", "id": 0} v 0) v1 -- 0x7f643804be40 con 0x7f64380ae770 2023-09-27T23:13:21.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.518+0000 7f6434ff9700 1 -- 172.21.15.123:0/1764816834 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "915f831e-9c10-4ed9-8012-6e415f3f0fe3", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f642401c880 con 0x7f64380ae770 2023-09-27T23:13:21.527 INFO:teuthology.orchestra.run.smithi123.stdout:0 2023-09-27T23:13:21.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 msgr2=0x7f64380aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 0x7f64380aada0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64240011e0 tx=0x7f6424007e10 comp rx=0 tx=0).stop 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 shutdown_connections 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6438150200 0x7f64380c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64380ae770 0x7f64380aada0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 --2- 172.21.15.123:0/1764816834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64380ace30 0x7f64380a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 >> 172.21.15.123:0/1764816834 conn(0x7f64380ccb30 msgr2=0x7f64380c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 shutdown_connections 2023-09-27T23:13:21.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.522+0000 7f643dd83700 1 -- 172.21.15.123:0/1764816834 wait complete. 2023-09-27T23:13:21.543 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 019649f0-4ab6-44a5-afcd-ec3112182bd9 1 2023-09-27T23:13:21.607 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:21.603+0000 7f66540e5000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:13:21.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.642+0000 7f44fe1bd700 1 Processor -- start 2023-09-27T23:13:21.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.642+0000 7f44fe1bd700 1 -- start start 2023-09-27T23:13:21.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f44f8155a90 0x7f44f80ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f80c68a0 con 0x7f44f8155e60 2023-09-27T23:13:21.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f80c6e60 con 0x7f44f806db70 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f80ae4d0 con 0x7f44f8155a90 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f44f806db70 0x7f44f80ad760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36250/0 (socket says 172.21.15.123:36250) 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 -- 172.21.15.123:0/2681841816 learned_addr learned my addr 172.21.15.123:0/2681841816 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fd1bb700 1 --2- 172.21.15.123:0/2681841816 >> v2:172.21.15.64:3300/0 conn(0x7f44f8155a90 0x7f44f80ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018824439 0 0) 0x7f44f80c6e60 con 0x7f44f806db70 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e4003570 con 0x7f44f806db70 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351046279 0 0) 0x7f44e4003570 con 0x7f44f806db70 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 >> v2:172.21.15.64:3300/0 conn(0x7f44f8155a90 msgr2=0x7f44f80ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 --2- 172.21.15.123:0/2681841816 >> v2:172.21.15.64:3300/0 conn(0x7f44f8155a90 0x7f44f80ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 >> v1:172.21.15.64:6789/0 conn(0x7f44f8155e60 legacy=0x7f44f80c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80ae7b0 con 0x7f44f806db70 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f44e8001470 con 0x7f44f806db70 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 >> v1:172.21.15.123:6789/0 conn(0x7f44f806db70 legacy=0x7f44f80ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44e4003a60 0x7f44e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e40051c0 0x7f44e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80ae4d0 con 0x7f44e4003a60 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e400c050 con 0x7f44e4004810 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e400c210 con 0x7f44e40051c0 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fd1bb700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44e4003a60 0x7f44e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fd9bc700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e40051c0 0x7f44e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 -- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e40051c0 msgr2=0x7f44e400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e40051c0 0x7f44e400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 -- 172.21.15.123:0/2681841816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44e4003a60 msgr2=0x7f44e4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44e4003a60 0x7f44e4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 -- 172.21.15.123:0/2681841816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4003570 con 0x7f44e4004810 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fd9bc700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e40051c0 0x7f44e400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fd9bc700 1 -- 172.21.15.123:0/2681841816 reap_dead start 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fc9ba700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44e4004c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c300 tx=0x7f44f80abf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48be440d63d3e59e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800cb90 con 0x7f44e4004810 2023-09-27T23:13:21.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e800ccf0 con 0x7f44e4004810 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44effff700 1 -- 172.21.15.123:0/2681841816 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8005b50 con 0x7f44e4004810 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 msgr2=0x7f44e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44e4004c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c300 tx=0x7f44f80abf30 comp rx=0 tx=0).stop 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- 172.21.15.123:0/2681841816 shutdown_connections 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 --2- 172.21.15.123:0/2681841816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44e4004c80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- 172.21.15.123:0/2681841816 >> 172.21.15.123:0/2681841816 conn(0x7f44f80c8cd0 msgr2=0x7f44f80c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.646+0000 7f44fe1bd700 1 -- 172.21.15.123:0/2681841816 shutdown_connections 2023-09-27T23:13:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 -- 172.21.15.123:0/2681841816 wait complete. 2023-09-27T23:13:21.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 Processor -- start 2023-09-27T23:13:21.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 -- start start 2023-09-27T23:13:21.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44f80c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44f80af320 0x7f44f80af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44f80afcd0 0x7f44f80b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80c61b0 con 0x7f44f80afcd0 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80c3a10 con 0x7f44e4004810 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fe1bd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80c3cf0 con 0x7f44f80af320 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fd1bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44f80c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fc9ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44f80af320 0x7f44f80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fd1bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44f80c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45506/0 (socket says 172.21.15.123:45506) 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fc9ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44f80af320 0x7f44f80af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:45588/0 (socket says 172.21.15.123:45588) 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.650+0000 7f44fd1bb700 1 -- 172.21.15.123:0/4173811368 learned_addr learned my addr 172.21.15.123:0/4173811368 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd9bc700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44f80afcd0 0x7f44f80b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd1bb700 1 -- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44f80af320 msgr2=0x7f44f80af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd1bb700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44f80af320 0x7f44f80af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd1bb700 1 -- 172.21.15.123:0/4173811368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44f80afcd0 msgr2=0x7f44f80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd1bb700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44f80afcd0 0x7f44f80b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd1bb700 1 -- 172.21.15.123:0/4173811368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80c3fd0 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fd1bb700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44f80c3310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f44f4011d40 tx=0x7f44f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400fa10 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f400fb70 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f80c42b0 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4014440 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f80c2c70 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f44f4014af0 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f44f4018030 con 0x7f44e4004810 2023-09-27T23:13:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.654+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f804be40 con 0x7f44e4004810 2023-09-27T23:13:21.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.658+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f44f403c6b0 con 0x7f44e4004810 2023-09-27T23:13:21.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.734+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "019649f0-4ab6-44a5-afcd-ec3112182bd9", "id": 1} v 0) v1 -- 0x7f44f80c6af0 con 0x7f44e4004810 2023-09-27T23:13:21.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.746+0000 7f44ed7fa700 1 -- 172.21.15.123:0/4173811368 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "019649f0-4ab6-44a5-afcd-ec3112182bd9", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44f402b030 con 0x7f44e4004810 2023-09-27T23:13:21.754 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-09-27T23:13:21.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 msgr2=0x7f44f80c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44f80c3310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f44f4011d40 tx=0x7f44f40091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 shutdown_connections 2023-09-27T23:13:21.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44f80afcd0 0x7f44f80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44f80af320 0x7f44f80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 --2- 172.21.15.123:0/4173811368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e4004810 0x7f44f80c3310 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 >> 172.21.15.123:0/4173811368 conn(0x7f44f80c8cd0 msgr2=0x7f44f80c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 shutdown_connections 2023-09-27T23:13:21.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.750+0000 7f44fe1bd700 1 -- 172.21.15.123:0/4173811368 wait complete. 2023-09-27T23:13:21.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 232c6eb7-ce53-44be-9ec1-f4bfba21c327 2 2023-09-27T23:13:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.866+0000 7fb3f9a98700 1 Processor -- start 2023-09-27T23:13:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.866+0000 7fb3f9a98700 1 -- start start 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f9a98700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3f40cc110 0x7fb3f41564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f9a98700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f4156af0 con 0x7fb3f413fbb0 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f9a98700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f4156dd0 con 0x7fb3f413f7e0 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f9a98700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f4157950 con 0x7fb3f40cc110 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb3f413f7e0 0x7fb3f40af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36256/0 (socket says 172.21.15.123:36256) 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f37fe700 1 -- 172.21.15.123:0/1626326873 learned_addr learned my addr 172.21.15.123:0/1626326873 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-09-27T23:13:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/1626326873 >> v2:172.21.15.64:3300/0 conn(0x7fb3f40cc110 0x7fb3f41564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072508689 0 0) 0x7fb3f4156dd0 con 0x7fb3f413f7e0 2023-09-27T23:13:21.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3dc003570 con 0x7fb3f413f7e0 2023-09-27T23:13:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049784269 0 0) 0x7fb3f4156af0 con 0x7fb3f413fbb0 2023-09-27T23:13:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f4156dd0 con 0x7fb3f413fbb0 2023-09-27T23:13:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4199784521 0 0) 0x7fb3dc003570 con 0x7fb3f413f7e0 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 >> v2:172.21.15.64:3300/0 conn(0x7fb3f40cc110 msgr2=0x7fb3f41564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 --2- 172.21.15.123:0/1626326873 >> v2:172.21.15.64:3300/0 conn(0x7fb3f40cc110 0x7fb3f41564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 >> v1:172.21.15.64:6789/0 conn(0x7fb3f413fbb0 legacy=0x7fb3f40cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f41570b0 con 0x7fb3f413f7e0 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fb3e4001470 con 0x7fb3f413f7e0 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 >> v1:172.21.15.123:6789/0 conn(0x7fb3f413f7e0 legacy=0x7fb3f40af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3dc003c40 0x7fb3dc004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc004810 0x7fb3dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3dc005210 0x7fb3dc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4157950 con 0x7fb3dc003c40 2023-09-27T23:13:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc00c130 con 0x7fb3dc004810 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc00c2c0 con 0x7fb3dc005210 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3dc005210 0x7fb3dc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f2ffd700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc004810 0x7fb3dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f37fe700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3dc003c40 0x7fb3dc004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 -- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc004810 msgr2=0x7fb3dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc004810 0x7fb3dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 -- 172.21.15.123:0/1626326873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3dc003c40 msgr2=0x7fb3dc004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3dc003c40 0x7fb3dc004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 -- 172.21.15.123:0/1626326873 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc003570 con 0x7fb3dc005210 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 -- 172.21.15.123:0/1626326873 reap_dead start 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3dc005210 0x7fb3dc00ba60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40cc510 tx=0x7fb3ec00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe9988de2987e16 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec009ac0 con 0x7fb3dc005210 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec00f040 con 0x7fb3dc005210 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.870+0000 7fb3f27fc700 1 -- 172.21.15.123:0/1626326873 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00ece0 con 0x7fb3dc005210 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3dc005210 msgr2=0x7fb3dc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3dc005210 0x7fb3dc00ba60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40cc510 tx=0x7fb3ec00b040 comp rx=0 tx=0).stop 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- 172.21.15.123:0/1626326873 shutdown_connections 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 --2- 172.21.15.123:0/1626326873 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3dc005210 0x7fb3dc00ba60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- 172.21.15.123:0/1626326873 >> 172.21.15.123:0/1626326873 conn(0x7fb3f40cd8f0 msgr2=0x7fb3f40c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- 172.21.15.123:0/1626326873 shutdown_connections 2023-09-27T23:13:21.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- 172.21.15.123:0/1626326873 wait complete. 2023-09-27T23:13:21.884 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fbbaa77e000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:13:21.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 Processor -- start 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- start start 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc005210 0x7fb3f4134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 0x7fb3f406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3f413c540 0x7fb3f413c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f406dfa0 con 0x7fb3f413c540 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4167260 con 0x7fb3dc005210 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f9a98700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f413d270 con 0x7fb3f406d5f0 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f2ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 0x7fb3f406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f2ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 0x7fb3f406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:45594/0 (socket says 172.21.15.123:45594) 2023-09-27T23:13:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f2ffd700 1 -- 172.21.15.123:0/620904908 learned_addr learned my addr 172.21.15.123:0/620904908 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f37fe700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc005210 0x7fb3f4134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.874+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3f413c540 0x7fb3f413c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f2ffd700 1 -- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc005210 msgr2=0x7fb3f4134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f2ffd700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc005210 0x7fb3f4134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f2ffd700 1 -- 172.21.15.123:0/620904908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3f413c540 msgr2=0x7fb3f413c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f2ffd700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3f413c540 0x7fb3f413c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f2ffd700 1 -- 172.21.15.123:0/620904908 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f413d550 con 0x7fb3f406d5f0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f3fff700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3f413c540 0x7fb3f413c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f2ffd700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 0x7fb3f406da60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8002b20 tx=0x7fb3e8007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800f9a0 con 0x7fb3f406d5f0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e800fb20 con 0x7fb3f406d5f0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f413d830 con 0x7fb3f406d5f0 2023-09-27T23:13:21.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800fee0 con 0x7fb3f406d5f0 2023-09-27T23:13:21.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f413cef0 con 0x7fb3f406d5f0 2023-09-27T23:13:21.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb3e8013880 con 0x7fb3f406d5f0 2023-09-27T23:13:21.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fb3e801c920 con 0x7fb3f406d5f0 2023-09-27T23:13:21.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.878+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f4167480 con 0x7fb3f406d5f0 2023-09-27T23:13:21.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.882+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7fb3f4167480 con 0x7fb3f406d5f0 2023-09-27T23:13:21.938 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:21.930+0000 7fbbaa77e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:13:21.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.958+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "232c6eb7-ce53-44be-9ec1-f4bfba21c327", "id": 2} v 0) v1 -- 0x7fb3f4071120 con 0x7fb3f406d5f0 2023-09-27T23:13:21.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.962+0000 7fb3db7fe700 1 -- 172.21.15.123:0/620904908 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "232c6eb7-ce53-44be-9ec1-f4bfba21c327", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb3e8017020 con 0x7fb3f406d5f0 2023-09-27T23:13:21.971 INFO:teuthology.orchestra.run.smithi123.stdout:2 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 msgr2=0x7fb3f406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 0x7fb3f406da60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8002b20 tx=0x7fb3e8007d30 comp rx=0 tx=0).stop 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 shutdown_connections 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3f413c540 0x7fb3f413c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3f406d5f0 0x7fb3f406da60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 --2- 172.21.15.123:0/620904908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3dc005210 0x7fb3f4134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 >> 172.21.15.123:0/620904908 conn(0x7fb3f40cd8f0 msgr2=0x7fb3f4158830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 shutdown_connections 2023-09-27T23:13:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:21.966+0000 7fb3f9a98700 1 -- 172.21.15.123:0/620904908 wait complete. 2023-09-27T23:13:21.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 3f119991-cb63-4f2e-95b5-db5712f79f3f 3 2023-09-27T23:13:22.056 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:22.050+0000 7fbbaa77e000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:13:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 Processor -- start 2023-09-27T23:13:22.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 -- start start 2023-09-27T23:13:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbb24150200 0x7fbb240c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb240ae770 0x7fbb240aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb240af250 con 0x7fbb240ac720 2023-09-27T23:13:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240af530 con 0x7fbb240ae770 2023-09-27T23:13:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb29701700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240ca8a0 con 0x7fbb24150200 2023-09-27T23:13:22.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb23fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbb24150200 0x7fbb240c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb28f00700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbb240ac720 0x7fbb240acb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36270/0 (socket says 172.21.15.123:36270) 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb28f00700 1 -- 172.21.15.123:0/2334935451 learned_addr learned my addr 172.21.15.123:0/2334935451 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.086+0000 7fbb237fe700 1 --2- 172.21.15.123:0/2334935451 >> v2:172.21.15.64:3300/0 conn(0x7fbb240ae770 0x7fbb240aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb23fff700 1 -- 172.21.15.123:0/2334935451 >> v2:172.21.15.64:3300/0 conn(0x7fbb240ae770 msgr2=0x7fbb240aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb23fff700 1 --2- 172.21.15.123:0/2334935451 >> v2:172.21.15.64:3300/0 conn(0x7fbb240ae770 0x7fbb240aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb23fff700 1 -- 172.21.15.123:0/2334935451 >> v1:172.21.15.123:6789/0 conn(0x7fbb240ac720 legacy=0x7fbb240acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb22ffd700 1 -- 172.21.15.123:0/2334935451 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944195092 0 0) 0x7fbb240af250 con 0x7fbb240ac720 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb23fff700 1 -- 172.21.15.123:0/2334935451 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240cab80 con 0x7fbb24150200 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb237fe700 1 --2- 172.21.15.123:0/2334935451 >> v2:172.21.15.64:3300/0 conn(0x7fbb240ae770 0x7fbb240aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb23fff700 1 --2- 172.21.15.123:0/2334935451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24150200 0x7fbb240c7170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbb2406b4e0 tx=0x7fbb140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e2cbfdc3f5bcb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb22ffd700 1 -- 172.21.15.123:0/2334935451 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400b6b0 con 0x7fbb24150200 2023-09-27T23:13:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb22ffd700 1 -- 172.21.15.123:0/2334935451 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400b810 con 0x7fbb24150200 2023-09-27T23:13:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb22ffd700 1 -- 172.21.15.123:0/2334935451 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400bbb0 con 0x7fbb24150200 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 -- 172.21.15.123:0/2334935451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24150200 msgr2=0x7fbb240c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 --2- 172.21.15.123:0/2334935451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24150200 0x7fbb240c7170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbb2406b4e0 tx=0x7fbb140091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 -- 172.21.15.123:0/2334935451 shutdown_connections 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 --2- 172.21.15.123:0/2334935451 >> v2:172.21.15.64:3300/0 conn(0x7fbb240ae770 0x7fbb240aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 --2- 172.21.15.123:0/2334935451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24150200 0x7fbb240c7170 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 -- 172.21.15.123:0/2334935451 >> 172.21.15.123:0/2334935451 conn(0x7fbb240ccb30 msgr2=0x7fbb240ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 -- 172.21.15.123:0/2334935451 shutdown_connections 2023-09-27T23:13:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.090+0000 7fbb29701700 1 -- 172.21.15.123:0/2334935451 wait complete. 2023-09-27T23:13:22.100 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 Processor -- start 2023-09-27T23:13:22.100 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 -- start start 2023-09-27T23:13:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240ac720 0x7fbb2414b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 0x7fbb240c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24150200 0x7fbb240c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2414c140 con 0x7fbb24150200 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2414f7a0 con 0x7fbb240ac720 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240c5160 con 0x7fbb240ae770 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 0x7fbb240c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 0x7fbb240c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:45608/0 (socket says 172.21.15.123:45608) 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 -- 172.21.15.123:0/1053650522 learned_addr learned my addr 172.21.15.123:0/1053650522 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb23fff700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240ac720 0x7fbb2414b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb28f00700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24150200 0x7fbb240c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 -- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240ac720 msgr2=0x7fbb2414b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240ac720 0x7fbb2414b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 -- 172.21.15.123:0/1053650522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24150200 msgr2=0x7fbb240c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24150200 0x7fbb240c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 -- 172.21.15.123:0/1053650522 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14009420 con 0x7fbb240ae770 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb237fe700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 0x7fbb240c45f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e900 tx=0x7fbb18008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18013410 con 0x7fbb240ae770 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18013570 con 0x7fbb240ae770 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb240c5440 con 0x7fbb240ae770 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18013910 con 0x7fbb240ae770 2023-09-27T23:13:22.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.094+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2414c670 con 0x7fbb240ae770 2023-09-27T23:13:22.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.098+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fbb18013d00 con 0x7fbb240ae770 2023-09-27T23:13:22.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.098+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fbb1801b3f0 con 0x7fbb240ae770 2023-09-27T23:13:22.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.098+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2404be40 con 0x7fbb240ae770 2023-09-27T23:13:22.107 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.098+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7fbb1803a6b0 con 0x7fbb240ae770 2023-09-27T23:13:22.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.174+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3f119991-cb63-4f2e-95b5-db5712f79f3f", "id": 3} v 0) v1 -- 0x7fbb2406cbf0 con 0x7fbb240ae770 2023-09-27T23:13:22.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb0bfff700 1 -- 172.21.15.123:0/1053650522 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f119991-cb63-4f2e-95b5-db5712f79f3f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb18029030 con 0x7fbb240ae770 2023-09-27T23:13:22.192 INFO:teuthology.orchestra.run.smithi123.stdout:3 2023-09-27T23:13:22.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 msgr2=0x7fbb240c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 0x7fbb240c45f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e900 tx=0x7fbb18008040 comp rx=0 tx=0).stop 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 shutdown_connections 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24150200 0x7fbb240c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb240ae770 0x7fbb240c45f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 --2- 172.21.15.123:0/1053650522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240ac720 0x7fbb2414b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 >> 172.21.15.123:0/1053650522 conn(0x7fbb240ccb30 msgr2=0x7fbb240b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 shutdown_connections 2023-09-27T23:13:22.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.186+0000 7fbb29701700 1 -- 172.21.15.123:0/1053650522 wait complete. 2023-09-27T23:13:22.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 7e01ef22-2cfe-4187-ac67-822e22a91142 4 2023-09-27T23:13:22.215 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.211+0000 7f66540e5000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:13:22.275 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.271+0000 7f66540e5000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:13:22.281 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:22.274+0000 7fbbaa77e000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:13:22.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.298+0000 7ff0f759e700 1 Processor -- start 2023-09-27T23:13:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.298+0000 7ff0f759e700 1 -- start start 2023-09-27T23:13:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0f815d140 0x7ff0f815d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0f806ae20 0x7ff0f806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f80c7820 con 0x7ff0f80ac170 2023-09-27T23:13:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f806b790 con 0x7ff0f806ae20 2023-09-27T23:13:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f80ac5c0 con 0x7ff0f815d140 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0f815d140 0x7ff0f815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0effff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff0f80ac170 0x7ff0f806d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44242/0 (socket says 172.21.15.123:44242) 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0effff700 1 -- 172.21.15.123:0/3359772691 learned_addr learned my addr 172.21.15.123:0/3359772691 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f6d9d700 1 --2- 172.21.15.123:0/3359772691 >> v2:172.21.15.64:3300/0 conn(0x7ff0f806ae20 0x7ff0f806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f659c700 1 -- 172.21.15.123:0/3359772691 >> v2:172.21.15.64:3300/0 conn(0x7ff0f806ae20 msgr2=0x7ff0f806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f659c700 1 --2- 172.21.15.123:0/3359772691 >> v2:172.21.15.64:3300/0 conn(0x7ff0f806ae20 0x7ff0f806b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f5d9b700 1 -- 172.21.15.123:0/3359772691 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281676490 0 0) 0x7ff0f80c7820 con 0x7ff0f80ac170 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f659c700 1 -- 172.21.15.123:0/3359772691 >> v1:172.21.15.123:6789/0 conn(0x7ff0f80ac170 legacy=0x7ff0f806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f659c700 1 -- 172.21.15.123:0/3359772691 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80c6c10 con 0x7ff0f815d140 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f659c700 1 --2- 172.21.15.123:0/3359772691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f815d140 0x7ff0f815d510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8068010 tx=0x7ff0e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84073f495f7cf912 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f5d9b700 1 -- 172.21.15.123:0/3359772691 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e800b6b0 con 0x7ff0f815d140 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f5d9b700 1 -- 172.21.15.123:0/3359772691 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e800b810 con 0x7ff0f815d140 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f5d9b700 1 -- 172.21.15.123:0/3359772691 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e800bbb0 con 0x7ff0f815d140 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- 172.21.15.123:0/3359772691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f815d140 msgr2=0x7ff0f815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 --2- 172.21.15.123:0/3359772691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f815d140 0x7ff0f815d510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8068010 tx=0x7ff0e80091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- 172.21.15.123:0/3359772691 shutdown_connections 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 --2- 172.21.15.123:0/3359772691 >> v2:172.21.15.64:3300/0 conn(0x7ff0f806ae20 0x7ff0f806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 --2- 172.21.15.123:0/3359772691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f815d140 0x7ff0f815d510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- 172.21.15.123:0/3359772691 >> 172.21.15.123:0/3359772691 conn(0x7ff0f80c8cd0 msgr2=0x7ff0f80c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- 172.21.15.123:0/3359772691 shutdown_connections 2023-09-27T23:13:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.302+0000 7ff0f759e700 1 -- 172.21.15.123:0/3359772691 wait complete. 2023-09-27T23:13:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 Processor -- start 2023-09-27T23:13:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 -- start start 2023-09-27T23:13:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f806ae20 0x7ff0f80b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 0x7ff0f80c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0f815d140 0x7ff0f80bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8147180 con 0x7ff0f806ae20 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80b1200 con 0x7ff0f80ac170 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80c6630 con 0x7ff0f815d140 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 0x7ff0f80c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 0x7ff0f80c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57262/0 (socket says 172.21.15.123:57262) 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 -- 172.21.15.123:0/2901355605 learned_addr learned my addr 172.21.15.123:0/2901355605 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f6d9d700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0f815d140 0x7ff0f80bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f659c700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f806ae20 0x7ff0f80b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 -- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0f815d140 msgr2=0x7ff0f80bfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0f815d140 0x7ff0f80bfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 -- 172.21.15.123:0/2901355605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f806ae20 msgr2=0x7ff0f80b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f806ae20 0x7ff0f80b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 -- 172.21.15.123:0/2901355605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e8009420 con 0x7ff0f80ac170 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0effff700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 0x7ff0f80c60f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c810 tx=0x7ff0e0007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0005b30 con 0x7ff0f80ac170 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f80b05f0 con 0x7ff0f80ac170 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0005c90 con 0x7ff0f80ac170 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0007300 con 0x7ff0f80ac170 2023-09-27T23:13:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f80b0bb0 con 0x7ff0f80ac170 2023-09-27T23:13:22.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff0e001c440 con 0x7ff0f80ac170 2023-09-27T23:13:22.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.306+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7ff0e0005e00 con 0x7ff0f80ac170 2023-09-27T23:13:22.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.310+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f804be40 con 0x7ff0f80ac170 2023-09-27T23:13:22.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.310+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7ff0e0017070 con 0x7ff0f80ac170 2023-09-27T23:13:22.329 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:22.322+0000 7fbbaa77e000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:13:22.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.386+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7e01ef22-2cfe-4187-ac67-822e22a91142", "id": 4} v 0) v1 -- 0x7ff0f80b0f50 con 0x7ff0f80ac170 2023-09-27T23:13:22.398 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.395+0000 7f66540e5000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:13:22.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.394+0000 7ff0ee7fc700 1 -- 172.21.15.123:0/2901355605 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7e01ef22-2cfe-4187-ac67-822e22a91142", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff0e001c930 con 0x7ff0f80ac170 2023-09-27T23:13:22.403 INFO:teuthology.orchestra.run.smithi123.stdout:4 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 msgr2=0x7ff0f80c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 0x7ff0f80c60f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c810 tx=0x7ff0e0007df0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 shutdown_connections 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0f815d140 0x7ff0f80bfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f80ac170 0x7ff0f80c60f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 --2- 172.21.15.123:0/2901355605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f806ae20 0x7ff0f80b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 >> 172.21.15.123:0/2901355605 conn(0x7ff0f80c8cd0 msgr2=0x7ff0f80adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 shutdown_connections 2023-09-27T23:13:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.398+0000 7ff0f759e700 1 -- 172.21.15.123:0/2901355605 wait complete. 2023-09-27T23:13:22.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 07a4e7d9-c362-4b12-a607-6faf07adf98a 5 2023-09-27T23:13:22.503 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:22.498+0000 7fbbaa77e000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:13:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.514+0000 7fc8e67b7700 1 Processor -- start 2023-09-27T23:13:22.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.514+0000 7fc8e67b7700 1 -- start start 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 0x7fc8e015a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8e00c8f90 0x7fc8e00ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e0159bc0 con 0x7fc8e00cac40 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00cb130 con 0x7fc8e00c8f90 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00cb410 con 0x7fc8e00c8bc0 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 0x7fc8e015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 0x7fc8e015a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40128/0 (socket says 172.21.15.123:40128) 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 -- 172.21.15.123:0/770992019 learned_addr learned my addr 172.21.15.123:0/770992019 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:22.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e4fb4700 1 --2- 172.21.15.123:0/770992019 >> v2:172.21.15.123:3300/0 conn(0x7fc8e00c8f90 0x7fc8e00ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 -- 172.21.15.123:0/770992019 >> v2:172.21.15.123:3300/0 conn(0x7fc8e00c8f90 msgr2=0x7fc8e00ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 --2- 172.21.15.123:0/770992019 >> v2:172.21.15.123:3300/0 conn(0x7fc8e00c8f90 0x7fc8e00ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 -- 172.21.15.123:0/770992019 >> v1:172.21.15.64:6789/0 conn(0x7fc8e00cac40 legacy=0x7fc8e006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 -- 172.21.15.123:0/770992019 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0159850 con 0x7fc8e00c8bc0 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e57b5700 1 --2- 172.21.15.123:0/770992019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 0x7fc8e015a050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0099b0 tx=0x7fc8dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2883162eca3f238 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8d7fff700 1 -- 172.21.15.123:0/770992019 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc009ea0 con 0x7fc8e00c8bc0 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8d7fff700 1 -- 172.21.15.123:0/770992019 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc00b820 con 0x7fc8e00c8bc0 2023-09-27T23:13:22.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8d7fff700 1 -- 172.21.15.123:0/770992019 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00bbc0 con 0x7fc8e00c8bc0 2023-09-27T23:13:22.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- 172.21.15.123:0/770992019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 msgr2=0x7fc8e015a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/770992019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 0x7fc8e015a050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0099b0 tx=0x7fc8dc009250 comp rx=0 tx=0).stop 2023-09-27T23:13:22.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- 172.21.15.123:0/770992019 shutdown_connections 2023-09-27T23:13:22.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/770992019 >> v2:172.21.15.123:3300/0 conn(0x7fc8e00c8f90 0x7fc8e00ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/770992019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00c8bc0 0x7fc8e015a050 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- 172.21.15.123:0/770992019 >> 172.21.15.123:0/770992019 conn(0x7fc8e00ccb30 msgr2=0x7fc8e00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- 172.21.15.123:0/770992019 shutdown_connections 2023-09-27T23:13:22.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.518+0000 7fc8e67b7700 1 -- 172.21.15.123:0/770992019 wait complete. 2023-09-27T23:13:22.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 Processor -- start 2023-09-27T23:13:22.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 -- start start 2023-09-27T23:13:22.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e00c8bc0 0x7fc8e00b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8e00c8f90 0x7fc8e0169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00cac40 0x7fc8e0169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0155f10 con 0x7fc8e00c8bc0 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00b2770 con 0x7fc8e00c8f90 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00674b0 con 0x7fc8e00cac40 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e4fb4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8e00c8f90 0x7fc8e0169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00cac40 0x7fc8e0169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e4fb4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8e00c8f90 0x7fc8e0169140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57276/0 (socket says 172.21.15.123:57276) 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e4fb4700 1 -- 172.21.15.123:0/2942570060 learned_addr learned my addr 172.21.15.123:0/2942570060 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e57b5700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e00c8bc0 0x7fc8e00b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 -- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8e00c8f90 msgr2=0x7fc8e0169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8e00c8f90 0x7fc8e0169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 -- 172.21.15.123:0/2942570060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e00c8bc0 msgr2=0x7fc8e00b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e00c8bc0 0x7fc8e00b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 -- 172.21.15.123:0/2942570060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0094f0 con 0x7fc8e00cac40 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e5fb6700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00cac40 0x7fc8e0169680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00cb060 tx=0x7fc8d8009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800c7c0 con 0x7fc8e00cac40 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d800c920 con 0x7fc8e00cac40 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e00b1ed0 con 0x7fc8e00cac40 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800cce0 con 0x7fc8e00cac40 2023-09-27T23:13:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.522+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e00b2490 con 0x7fc8e00cac40 2023-09-27T23:13:22.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.526+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc8d8014620 con 0x7fc8e00cac40 2023-09-27T23:13:22.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.526+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fc8d801d970 con 0x7fc8e00cac40 2023-09-27T23:13:22.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.526+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0068aa0 con 0x7fc8e00cac40 2023-09-27T23:13:22.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.526+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7fc8d8022060 con 0x7fc8e00cac40 2023-09-27T23:13:22.564 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:22.558+0000 7fbbaa77e000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:13:22.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.602+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "07a4e7d9-c362-4b12-a607-6faf07adf98a", "id": 5} v 0) v1 -- 0x7fc8e004be40 con 0x7fc8e00cac40 2023-09-27T23:13:22.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8d57fa700 1 -- 172.21.15.123:0/2942570060 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07a4e7d9-c362-4b12-a607-6faf07adf98a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc8d8014ab0 con 0x7fc8e00cac40 2023-09-27T23:13:22.622 INFO:teuthology.orchestra.run.smithi123.stdout:5 2023-09-27T23:13:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00cac40 msgr2=0x7fc8e0169680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00cac40 0x7fc8e0169680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00cb060 tx=0x7fc8d8009760 comp rx=0 tx=0).stop 2023-09-27T23:13:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 shutdown_connections 2023-09-27T23:13:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8e00cac40 0x7fc8e0169680 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8e00c8f90 0x7fc8e0169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 --2- 172.21.15.123:0/2942570060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e00c8bc0 0x7fc8e00b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 >> 172.21.15.123:0/2942570060 conn(0x7fc8e00ccb30 msgr2=0x7fc8e0158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 shutdown_connections 2023-09-27T23:13:22.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.614+0000 7fc8e67b7700 1 -- 172.21.15.123:0/2942570060 wait complete. 2023-09-27T23:13:22.625 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:22.618+0000 7fbbaa77e000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:13:22.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 98c04d1b-babe-4851-82e7-a418c0c3e53b 6 2023-09-27T23:13:22.642 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.639+0000 7f66540e5000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:13:22.696 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.691+0000 7f66540e5000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:13:22.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.730+0000 7f7018b66700 1 Processor -- start 2023-09-27T23:13:22.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.730+0000 7f7018b66700 1 -- start start 2023-09-27T23:13:22.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70140cc460 0x7f701415af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701415dc40 con 0x7f701406d650 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701415df20 con 0x7f701406d280 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701415e200 con 0x7f70140cc460 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f70137fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f701406d280 0x7f70140cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44250/0 (socket says 172.21.15.123:44250) 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f70137fe700 1 -- 172.21.15.123:0/1516859359 learned_addr learned my addr 172.21.15.123:0/1516859359 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7013fff700 1 --2- 172.21.15.123:0/1516859359 >> v2:172.21.15.123:3300/0 conn(0x7f70140cc460 0x7f701415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7013fff700 1 -- 172.21.15.123:0/1516859359 >> v1:172.21.15.123:6789/0 conn(0x7f701406d280 legacy=0x7f70140cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7013fff700 1 -- 172.21.15.123:0/1516859359 >> v1:172.21.15.64:6789/0 conn(0x7f701406d650 legacy=0x7f701415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7013fff700 1 -- 172.21.15.123:0/1516859359 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701415e4e0 con 0x7f70140cc460 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7013fff700 1 --2- 172.21.15.123:0/1516859359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70140cc460 0x7f701415af00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e60 tx=0x7f700c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5fd15f57d68b783 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f70127fc700 1 -- 172.21.15.123:0/1516859359 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00e070 con 0x7f70140cc460 2023-09-27T23:13:22.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f70127fc700 1 -- 172.21.15.123:0/1516859359 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00bbf0 con 0x7f70140cc460 2023-09-27T23:13:22.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f70127fc700 1 -- 172.21.15.123:0/1516859359 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c012430 con 0x7f70140cc460 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- 172.21.15.123:0/1516859359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70140cc460 msgr2=0x7f701415af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 --2- 172.21.15.123:0/1516859359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70140cc460 0x7f701415af00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e60 tx=0x7f700c009780 comp rx=0 tx=0).stop 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- 172.21.15.123:0/1516859359 shutdown_connections 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 --2- 172.21.15.123:0/1516859359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70140cc460 0x7f701415af00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- 172.21.15.123:0/1516859359 >> 172.21.15.123:0/1516859359 conn(0x7f70140ccb30 msgr2=0x7f70140ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- 172.21.15.123:0/1516859359 shutdown_connections 2023-09-27T23:13:22.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- 172.21.15.123:0/1516859359 wait complete. 2023-09-27T23:13:22.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 Processor -- start 2023-09-27T23:13:22.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.734+0000 7f7018b66700 1 -- start start 2023-09-27T23:13:22.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701406d280 0x7f7014157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701406d650 0x7f701415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70140cc460 0x7f701415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70140c8d60 con 0x7f701406d650 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7014069aa0 con 0x7f701406d280 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701415d010 con 0x7f70140cc460 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f70137fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701406d280 0x7f7014157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70140cc460 0x7f701415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f70137fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701406d280 0x7f7014157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57290/0 (socket says 172.21.15.123:57290) 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f70137fe700 1 -- 172.21.15.123:0/2090043975 learned_addr learned my addr 172.21.15.123:0/2090043975 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7012ffd700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701406d650 0x7f701415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 -- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701406d280 msgr2=0x7f7014157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701406d280 0x7f7014157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 -- 172.21.15.123:0/2090043975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701406d650 msgr2=0x7f701415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701406d650 0x7f701415c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 -- 172.21.15.123:0/2090043975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c009a20 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7012ffd700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701406d650 0x7f701415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7013fff700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70140cc460 0x7f701415c9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f700c015000 tx=0x7f700c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c01e640 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c01e7a0 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70140c8430 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c01eb60 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70140c89f0 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f700c026400 con 0x7f70140cc460 2023-09-27T23:13:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f700c01d9c0 con 0x7f70140cc460 2023-09-27T23:13:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.738+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70140c8ed0 con 0x7f70140cc460 2023-09-27T23:13:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.742+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f700c01dca0 con 0x7f70140cc460 2023-09-27T23:13:22.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.814+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "98c04d1b-babe-4851-82e7-a418c0c3e53b", "id": 6} v 0) v1 -- 0x7f701404be40 con 0x7f70140cc460 2023-09-27T23:13:22.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f6ffb7fe700 1 -- 172.21.15.123:0/2090043975 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98c04d1b-babe-4851-82e7-a418c0c3e53b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f700c01de40 con 0x7f70140cc460 2023-09-27T23:13:22.833 INFO:teuthology.orchestra.run.smithi123.stdout:6 2023-09-27T23:13:22.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70140cc460 msgr2=0x7f701415c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70140cc460 0x7f701415c9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f700c015000 tx=0x7f700c018000 comp rx=0 tx=0).stop 2023-09-27T23:13:22.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 shutdown_connections 2023-09-27T23:13:22.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70140cc460 0x7f701415c9e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701406d650 0x7f701415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 --2- 172.21.15.123:0/2090043975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701406d280 0x7f7014157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 >> 172.21.15.123:0/2090043975 conn(0x7f70140ccb30 msgr2=0x7f701415eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 shutdown_connections 2023-09-27T23:13:22.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.826+0000 7f7018b66700 1 -- 172.21.15.123:0/2090043975 wait complete. 2023-09-27T23:13:22.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new d48f521b-c431-4a35-8069-a24a0648c612 7 2023-09-27T23:13:22.890 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.887+0000 7f66540e5000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:13:22.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.938+0000 7f819149e700 1 Processor -- start 2023-09-27T23:13:22.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.938+0000 7f819149e700 1 -- start start 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f818c142a50 0x7f818c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0cc7e0 con 0x7f818c0ad650 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0cbbf0 con 0x7f818c142680 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c0cbeb0 con 0x7f818c142a50 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f818c142a50 0x7f818c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f818c142680 0x7f818c0c8a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44256/0 (socket says 172.21.15.123:44256) 2023-09-27T23:13:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818bfff700 1 -- 172.21.15.123:0/2384618764 learned_addr learned my addr 172.21.15.123:0/2384618764 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818b7fe700 1 -- 172.21.15.123:0/2384618764 >> v1:172.21.15.123:6789/0 conn(0x7f818c142680 legacy=0x7f818c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818b7fe700 1 -- 172.21.15.123:0/2384618764 >> v1:172.21.15.64:6789/0 conn(0x7f818c0ad650 legacy=0x7f818c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818affd700 1 -- 172.21.15.123:0/2384618764 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246115452 0 0) 0x7f818c0cbbf0 con 0x7f818c142680 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818b7fe700 1 -- 172.21.15.123:0/2384618764 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0cc190 con 0x7f818c142a50 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818b7fe700 1 --2- 172.21.15.123:0/2384618764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142a50 0x7f818c06d050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81800098e0 tx=0x7f81800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97bd73dc4f0fba6e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818affd700 1 -- 172.21.15.123:0/2384618764 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180009e80 con 0x7f818c142a50 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818affd700 1 -- 172.21.15.123:0/2384618764 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000b860 con 0x7f818c142a50 2023-09-27T23:13:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f818affd700 1 -- 172.21.15.123:0/2384618764 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000bbe0 con 0x7f818c142a50 2023-09-27T23:13:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- 172.21.15.123:0/2384618764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142a50 msgr2=0x7f818c06d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 --2- 172.21.15.123:0/2384618764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142a50 0x7f818c06d050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81800098e0 tx=0x7f81800091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- 172.21.15.123:0/2384618764 shutdown_connections 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 --2- 172.21.15.123:0/2384618764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142a50 0x7f818c06d050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- 172.21.15.123:0/2384618764 >> 172.21.15.123:0/2384618764 conn(0x7f818c0ccb30 msgr2=0x7f818c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- 172.21.15.123:0/2384618764 shutdown_connections 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- 172.21.15.123:0/2384618764 wait complete. 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 Processor -- start 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.942+0000 7f819149e700 1 -- start start 2023-09-27T23:13:22.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f818c0ad650 0x7f818c0670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 0x7f818c0c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f818c142a50 0x7f818c0c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0676c0 con 0x7f818c142a50 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0679a0 con 0x7f818c142680 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c067c80 con 0x7f818c0ad650 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 0x7f818c0c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f818c0ad650 0x7f818c0670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 0x7f818c0c2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57316/0 (socket says 172.21.15.123:57316) 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 -- 172.21.15.123:0/467562889 learned_addr learned my addr 172.21.15.123:0/467562889 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f8190c9d700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f818c142a50 0x7f818c0c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:22.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 -- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f818c0ad650 msgr2=0x7f818c0670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f818c0ad650 0x7f818c0670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 -- 172.21.15.123:0/467562889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f818c142a50 msgr2=0x7f818c0c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f818c142a50 0x7f818c0c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 -- 172.21.15.123:0/467562889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009420 con 0x7f818c142680 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f818b7fe700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 0x7f818c0c2bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f818c06b3a0 tx=0x7f8180017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818001f070 con 0x7f818c142680 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000b9d0 con 0x7f818c142680 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c067f60 con 0x7f818c142680 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011670 con 0x7f818c142680 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c0a8d60 con 0x7f818c142680 2023-09-27T23:13:22.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8180011ca0 con 0x7f818c142680 2023-09-27T23:13:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f8180023a80 con 0x7f818c142680 2023-09-27T23:13:22.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.946+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c04be40 con 0x7f818c142680 2023-09-27T23:13:22.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:22.950+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f8180028070 con 0x7f818c142680 2023-09-27T23:13:22.959 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:22.955+0000 7f66540e5000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:13:23.010 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.002+0000 7fbbaa77e000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:13:23.027 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:23.023+0000 7f66540e5000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:13:23.035 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.030+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d48f521b-c431-4a35-8069-a24a0648c612", "id": 7} v 0) v1 -- 0x7f818c0cc470 con 0x7f818c142680 2023-09-27T23:13:23.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f8173fff700 1 -- 172.21.15.123:0/467562889 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d48f521b-c431-4a35-8069-a24a0648c612", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8180015020 con 0x7f818c142680 2023-09-27T23:13:23.046 INFO:teuthology.orchestra.run.smithi123.stdout:7 2023-09-27T23:13:23.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 msgr2=0x7f818c0c2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:23.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 0x7f818c0c2bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f818c06b3a0 tx=0x7f8180017000 comp rx=0 tx=0).stop 2023-09-27T23:13:23.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 shutdown_connections 2023-09-27T23:13:23.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f818c142a50 0x7f818c0c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818c142680 0x7f818c0c2bc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 --2- 172.21.15.123:0/467562889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f818c0ad650 0x7f818c0670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 >> 172.21.15.123:0/467562889 conn(0x7f818c0ccb30 msgr2=0x7f818c0c1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 shutdown_connections 2023-09-27T23:13:23.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-27T23:13:23.038+0000 7f819149e700 1 -- 172.21.15.123:0/467562889 wait complete. 2023-09-27T23:13:23.058 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-27T23:13:23.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-27T23:13:23.061 INFO:tasks.ceph.osd.0:Started 2023-09-27T23:13:23.061 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-27T23:13:23.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-27T23:13:23.063 INFO:tasks.ceph.osd.1:Started 2023-09-27T23:13:23.063 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-27T23:13:23.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-27T23:13:23.066 INFO:tasks.ceph.osd.2:Started 2023-09-27T23:13:23.066 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-27T23:13:23.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-27T23:13:23.068 INFO:tasks.ceph.osd.3:Started 2023-09-27T23:13:23.068 INFO:tasks.ceph.osd.4:Restarting daemon 2023-09-27T23:13:23.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-09-27T23:13:23.071 INFO:tasks.ceph.osd.4:Started 2023-09-27T23:13:23.071 INFO:tasks.ceph.osd.5:Restarting daemon 2023-09-27T23:13:23.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-09-27T23:13:23.073 INFO:tasks.ceph.osd.5:Started 2023-09-27T23:13:23.073 INFO:tasks.ceph.osd.6:Restarting daemon 2023-09-27T23:13:23.073 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-09-27T23:13:23.075 INFO:tasks.ceph.osd.6:Started 2023-09-27T23:13:23.075 INFO:tasks.ceph.osd.7:Restarting daemon 2023-09-27T23:13:23.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-09-27T23:13:23.078 INFO:tasks.ceph.osd.7:Started 2023-09-27T23:13:23.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:23.138 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.130+0000 7fbbaa77e000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:13:23.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.187+0000 7ff11891f700 1 Processor -- start 2023-09-27T23:13:23.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.187+0000 7ff11891f700 1 -- start start 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff114155490 0x7ff11406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff1140684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140c0b80 con 0x7ff114068a00 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11415a220 con 0x7ff114155490 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff114155490 0x7ff11406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11406de30 con 0x7ff11406d9e0 2023-09-27T23:13:23.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff114155490 0x7ff11406d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41780/0 (socket says 172.21.15.64:41780) 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 -- 172.21.15.64:0/824304635 learned_addr learned my addr 172.21.15.64:0/824304635 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff111d9b700 1 --2- 172.21.15.64:0/824304635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff1140684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 -- 172.21.15.64:0/824304635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 msgr2=0x7ff1140684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 --2- 172.21.15.64:0/824304635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff1140684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 -- 172.21.15.64:0/824304635 >> v1:172.21.15.123:6789/0 conn(0x7ff114068a00 legacy=0x7ff1140670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 -- 172.21.15.64:0/824304635 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1140c0dd0 con 0x7ff114155490 2023-09-27T23:13:23.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11259c700 1 --2- 172.21.15.64:0/824304635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11406d4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff1040099b0 tx=0x7ff104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a194416e1258e6ba server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11159a700 1 -- 172.21.15.64:0/824304635 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10400b860 con 0x7ff114155490 2023-09-27T23:13:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11159a700 1 -- 172.21.15.64:0/824304635 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10400b9c0 con 0x7ff114155490 2023-09-27T23:13:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11159a700 1 -- 172.21.15.64:0/824304635 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10400bd60 con 0x7ff114155490 2023-09-27T23:13:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- 172.21.15.64:0/824304635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 msgr2=0x7ff11406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:23.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 --2- 172.21.15.64:0/824304635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11406d4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff1040099b0 tx=0x7ff104009250 comp rx=0 tx=0).stop 2023-09-27T23:13:23.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- 172.21.15.64:0/824304635 shutdown_connections 2023-09-27T23:13:23.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 --2- 172.21.15.64:0/824304635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff1140684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 --2- 172.21.15.64:0/824304635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11406d4a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- 172.21.15.64:0/824304635 >> 172.21.15.64:0/824304635 conn(0x7ff1140c92c0 msgr2=0x7ff11406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:23.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- 172.21.15.64:0/824304635 shutdown_connections 2023-09-27T23:13:23.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.191+0000 7ff11891f700 1 -- 172.21.15.64:0/824304635 wait complete. 2023-09-27T23:13:23.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 Processor -- start 2023-09-27T23:13:23.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 -- start start 2023-09-27T23:13:23.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff114068a00 0x7ff114156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:23.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff11414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140cb0e0 con 0x7ff114155490 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11414e760 con 0x7ff114068a00 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff114156810 con 0x7ff11406d9e0 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff111d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff11414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11414bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41794/0 (socket says 172.21.15.64:41794) 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 -- 172.21.15.64:0/2108731092 learned_addr learned my addr 172.21.15.64:0/2108731092 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11259c700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff114068a00 0x7ff114156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 -- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 msgr2=0x7ff11414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff11414e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 -- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff114068a00 msgr2=0x7ff114156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff114068a00 0x7ff114156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 -- 172.21.15.64:0/2108731092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040094f0 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff112d9d700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11414bdd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00c870 tx=0x7ff10c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c012630 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c0127b0 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11414c310 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c012b70 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1140caa90 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff10c003030 con 0x7ff114155490 2023-09-27T23:13:23.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.195+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7ff10c01b400 con 0x7ff114155490 2023-09-27T23:13:23.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.199+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff114071120 con 0x7ff114155490 2023-09-27T23:13:23.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.199+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7ff10c01b6c0 con 0x7ff114155490 2023-09-27T23:13:23.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.267+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1140642a0 con 0x7ff114155490 2023-09-27T23:13:23.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff1027fc700 1 -- 172.21.15.64:0/2108731092 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7ff10c024420 con 0x7ff114155490 2023-09-27T23:13:23.273 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:23.274 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:23.035794+0000","last_up_change":"0.000000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 msgr2=0x7ff11414bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11414bdd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00c870 tx=0x7ff10c001670 comp rx=0 tx=0).stop 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 shutdown_connections 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff114155490 0x7ff11414bdd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff11406d9e0 0x7ff11414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 --2- 172.21.15.64:0/2108731092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff114068a00 0x7ff114156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 >> 172.21.15.64:0/2108731092 conn(0x7ff1140c92c0 msgr2=0x7ff1140c7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:23.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 shutdown_connections 2023-09-27T23:13:23.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:23.271+0000 7ff11891f700 1 -- 172.21.15.64:0/2108731092 wait complete. 2023-09-27T23:13:23.288 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-27T23:13:23.289 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-27T23:13:23.426 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-09-27T23:13:23.423+0000 7f0d75ae13c0 -1 Falling back to public interface 2023-09-27T23:13:23.427 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.422+0000 7fbbaa77e000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:13:23.454 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-09-27T23:13:23.451+0000 7f288c0f43c0 -1 Falling back to public interface 2023-09-27T23:13:23.473 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-09-27T23:13:23.466+0000 7f057800f3c0 -1 Falling back to public interface 2023-09-27T23:13:23.480 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:23.475+0000 7f66540e5000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:13:23.590 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.582+0000 7fbbaa77e000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:13:23.604 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:23.599+0000 7f66540e5000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:13:23.656 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.650+0000 7fbbaa77e000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:13:23.699 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-09-27T23:13:23.695+0000 7f1e496363c0 -1 Falling back to public interface 2023-09-27T23:13:23.701 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-09-27T23:13:23.694+0000 7f43096cb3c0 -1 Falling back to public interface 2023-09-27T23:13:23.707 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.698+0000 7fbbaa77e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:13:23.710 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-09-27T23:13:23.707+0000 7fdd5be0a3c0 -1 Falling back to public interface 2023-09-27T23:13:23.713 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-09-27T23:13:23.706+0000 7fc6235363c0 -1 Falling back to public interface 2023-09-27T23:13:23.713 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-09-27T23:13:23.706+0000 7f234c6733c0 -1 Falling back to public interface 2023-09-27T23:13:23.762 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:23.754+0000 7fbbaa77e000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:13:23.935 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:23.931+0000 7f66540e5000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:13:24.036 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:24.030+0000 7fbbaa77e000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:13:24.093 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:24.086+0000 7fbbaa77e000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:13:24.095 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:24.106 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.103+0000 7f66540e5000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:13:24.158 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:24.150+0000 7fbbaa77e000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:13:24.175 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.171+0000 7f66540e5000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:13:24.182 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.179+0000 7fe212bdc700 1 Processor -- start 2023-09-27T23:13:24.183 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.179+0000 7fe212bdc700 1 -- start start 2023-09-27T23:13:24.185 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c158f10 0x7fe20c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:24.185 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe20c0af920 0x7fe20c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:24.185 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe20c06d7e0 0x7fe20c06a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c06ce10 con 0x7fe20c06d7e0 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c06c200 con 0x7fe20c0af920 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c06c4e0 con 0x7fe20c158f10 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe210978700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c158f10 0x7fe20c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe20c06d7e0 0x7fe20c06a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe210978700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c158f10 0x7fe20c06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:47846/0 (socket says 172.21.15.64:47846) 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe210978700 1 -- 172.21.15.64:0/3394359644 learned_addr learned my addr 172.21.15.64:0/3394359644 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:24.186 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe20bfff700 1 --2- 172.21.15.64:0/3394359644 >> v2:172.21.15.123:3300/0 conn(0x7fe20c0af920 0x7fe20c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 -- 172.21.15.64:0/3394359644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c158f10 msgr2=0x7fe20c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 --2- 172.21.15.64:0/3394359644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c158f10 0x7fe20c06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 -- 172.21.15.64:0/3394359644 >> v2:172.21.15.123:3300/0 conn(0x7fe20c0af920 msgr2=0x7fe20c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 --2- 172.21.15.64:0/3394359644 >> v2:172.21.15.123:3300/0 conn(0x7fe20c0af920 0x7fe20c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 -- 172.21.15.64:0/3394359644 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c06c7c0 con 0x7fe20c06d7e0 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe211179700 1 --2- 172.21.15.64:0/3394359644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c06d7e0 0x7fe20c06a260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe20c069c70 tx=0x7fe204009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2988d7d70802170d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:24.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe20b7fe700 1 -- 172.21.15.64:0/3394359644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400e070 con 0x7fe20c06d7e0 2023-09-27T23:13:24.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe20b7fe700 1 -- 172.21.15.64:0/3394359644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20400bab0 con 0x7fe20c06d7e0 2023-09-27T23:13:24.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe20b7fe700 1 -- 172.21.15.64:0/3394359644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400be70 con 0x7fe20c06d7e0 2023-09-27T23:13:24.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- 172.21.15.64:0/3394359644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c06d7e0 msgr2=0x7fe20c06a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:24.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- 172.21.15.64:0/3394359644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c06d7e0 0x7fe20c06a260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe20c069c70 tx=0x7fe204009780 comp rx=0 tx=0).stop 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- 172.21.15.64:0/3394359644 shutdown_connections 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- 172.21.15.64:0/3394359644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c06d7e0 0x7fe20c06a260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- 172.21.15.64:0/3394359644 >> v2:172.21.15.123:3300/0 conn(0x7fe20c0af920 0x7fe20c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 --2- 172.21.15.64:0/3394359644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c158f10 0x7fe20c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- 172.21.15.64:0/3394359644 >> 172.21.15.64:0/3394359644 conn(0x7fe20c0cc480 msgr2=0x7fe20c0cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- 172.21.15.64:0/3394359644 shutdown_connections 2023-09-27T23:13:24.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.183+0000 7fe212bdc700 1 -- 172.21.15.64:0/3394359644 wait complete. 2023-09-27T23:13:24.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 Processor -- start 2023-09-27T23:13:24.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 -- start start 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c06d7e0 0x7fe20c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20c0af920 0x7fe20c168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 0x7fe20c0ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0c1290 con 0x7fe20c158f10 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0aa7b0 con 0x7fe20c0af920 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe212bdc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0aaa90 con 0x7fe20c06d7e0 2023-09-27T23:13:24.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 0x7fe20c0ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 0x7fe20c0ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41806/0 (socket says 172.21.15.64:41806) 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 -- 172.21.15.64:0/394325925 learned_addr learned my addr 172.21.15.64:0/394325925 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe20bfff700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20c0af920 0x7fe20c168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe210978700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c06d7e0 0x7fe20c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 -- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c06d7e0 msgr2=0x7fe20c156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c06d7e0 0x7fe20c156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 -- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20c0af920 msgr2=0x7fe20c168f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20c0af920 0x7fe20c168f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe211179700 1 -- 172.21.15.64:0/394325925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009a20 con 0x7fe20c158f10 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.187+0000 7fe210978700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c06d7e0 0x7fe20c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe211179700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 0x7fe20c0ace00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe204009fd0 tx=0x7fe204019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:24.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204012e80 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204027470 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c0aad70 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204027830 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c0ab3b0 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe204024070 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fe20401dc90 con 0x7fe20c158f10 2023-09-27T23:13:24.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.191+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c1579f0 con 0x7fe20c158f10 2023-09-27T23:13:24.198 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.195+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7fe2040466b0 con 0x7fe20c158f10 2023-09-27T23:13:24.228 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.223+0000 7f66540e5000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:13:24.260 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.255+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe20c0ab540 con 0x7fe20c158f10 2023-09-27T23:13:24.261 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe208ff9700 1 -- 172.21.15.64:0/394325925 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fe204035030 con 0x7fe20c158f10 2023-09-27T23:13:24.261 INFO:teuthology.misc.health.smithi064.stdout: 2023-09-27T23:13:24.261 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:23.035794+0000","last_up_change":"0.000000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:24.262 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 msgr2=0x7fe20c0ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:24.262 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 0x7fe20c0ace00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe204009fd0 tx=0x7fe204019000 comp rx=0 tx=0).stop 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 shutdown_connections 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe20c158f10 0x7fe20c0ace00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe20c0af920 0x7fe20c168f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 --2- 172.21.15.64:0/394325925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20c06d7e0 0x7fe20c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 >> 172.21.15.64:0/394325925 conn(0x7fe20c0cc480 msgr2=0x7fe20c1580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 shutdown_connections 2023-09-27T23:13:24.263 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:24.259+0000 7fe212bdc700 1 -- 172.21.15.64:0/394325925 wait complete. 2023-09-27T23:13:24.279 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-27T23:13:24.288 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.283+0000 7f66540e5000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:13:24.332 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:24.326+0000 7fbbaa77e000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:13:24.399 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:24.390+0000 7fbbaa77e000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:13:24.559 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.555+0000 7f66540e5000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:13:24.618 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.615+0000 7f66540e5000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:13:24.680 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.675+0000 7f66540e5000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:13:24.842 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.839+0000 7f66540e5000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:13:24.909 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:24.907+0000 7f66540e5000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:13:25.107 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:25.098+0000 7fbbaa77e000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:13:25.651 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:13:25.647+0000 7f66540e5000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:13:26.153 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:26.146+0000 7fbb93876700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-27T23:13:26.153 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:13:26.146+0000 7fbb93876700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-27T23:13:27.118 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-09-27T23:13:27.115+0000 7f288c0f43c0 -1 osd.1 0 log_to_monitors true 2023-09-27T23:13:27.275 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-09-27T23:13:27.266+0000 7f43096cb3c0 -1 osd.4 0 log_to_monitors true 2023-09-27T23:13:27.340 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-09-27T23:13:27.335+0000 7fdd5be0a3c0 -1 osd.2 0 log_to_monitors true 2023-09-27T23:13:27.351 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-09-27T23:13:27.347+0000 7f0d75ae13c0 -1 osd.0 0 log_to_monitors true 2023-09-27T23:13:27.362 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-09-27T23:13:27.354+0000 7f234c6733c0 -1 osd.7 0 log_to_monitors true 2023-09-27T23:13:27.389 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-09-27T23:13:27.382+0000 7f057800f3c0 -1 osd.6 0 log_to_monitors true 2023-09-27T23:13:27.520 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-09-27T23:13:27.515+0000 7f1e496363c0 -1 osd.3 0 log_to_monitors true 2023-09-27T23:13:27.539 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-09-27T23:13:27.530+0000 7fc6235363c0 -1 osd.5 0 log_to_monitors true 2023-09-27T23:13:29.331 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-09-27T23:13:29.327+0000 7f2887894700 -1 osd.1 0 waiting for initial osdmap 2023-09-27T23:13:29.560 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-09-27T23:13:29.555+0000 7f2880638700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:30.943 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-09-27T23:13:30.939+0000 7fdd575aa700 -1 osd.2 0 waiting for initial osdmap 2023-09-27T23:13:30.950 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-09-27T23:13:30.947+0000 7fdd5034e700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:30.976 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-09-27T23:13:30.971+0000 7f1e44dd6700 -1 osd.3 0 waiting for initial osdmap 2023-09-27T23:13:30.982 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-09-27T23:13:30.979+0000 7f1e3db7a700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:30.996 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-09-27T23:13:30.991+0000 7f0d71281700 -1 osd.0 0 waiting for initial osdmap 2023-09-27T23:13:31.002 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-09-27T23:13:30.999+0000 7f0d6a025700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:31.085 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:31.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 Processor -- start 2023-09-27T23:13:31.187 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 -- start start 2023-09-27T23:13:31.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5d8c15ada0 0x7f5d8c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:31.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c15e350 0x7f5d8c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:31.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c0c72e0 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c0697c0 con 0x7f5d8c15ada0 2023-09-27T23:13:31.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d9120e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c161da0 con 0x7f5d8c15e350 2023-09-27T23:13:31.188 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5d8c15ada0 0x7f5d8c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:31.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d90a0d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5d8c0c6e60 0x7f5d8c06b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49078/0 (socket says 172.21.15.64:49078) 2023-09-27T23:13:31.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d90a0d700 1 -- 172.21.15.64:0/1533051272 learned_addr learned my addr 172.21.15.64:0/1533051272 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:31.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8b7fe700 1 --2- 172.21.15.64:0/1533051272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c15e350 0x7f5d8c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:31.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8affd700 1 -- 172.21.15.64:0/1533051272 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923390274 0 0) 0x7f5d84001490 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.189 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8affd700 1 -- 172.21.15.64:0/1533051272 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d78003570 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.190 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8affd700 1 -- 172.21.15.64:0/1533051272 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3788652898 0 0) 0x7f5d78003570 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.190 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8bfff700 1 -- 172.21.15.64:0/1533051272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c15e350 msgr2=0x7f5d8c0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:31.190 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8bfff700 1 --2- 172.21.15.64:0/1533051272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c15e350 0x7f5d8c0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.190 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8bfff700 1 -- 172.21.15.64:0/1533051272 >> v1:172.21.15.64:6789/0 conn(0x7f5d8c0c6e60 legacy=0x7f5d8c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:31.190 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8bfff700 1 -- 172.21.15.64:0/1533051272 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c161500 con 0x7f5d8c15ada0 2023-09-27T23:13:31.190 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.183+0000 7f5d8bfff700 1 --2- 172.21.15.64:0/1533051272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c160e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0098e0 tx=0x7f5d7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a551d40229270558 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d8affd700 1 -- 172.21.15.64:0/1533051272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84001980 con 0x7f5d8c15ada0 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d8affd700 1 -- 172.21.15.64:0/1533051272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c011420 con 0x7f5d8c15ada0 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d8affd700 1 -- 172.21.15.64:0/1533051272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0117c0 con 0x7f5d8c15ada0 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- 172.21.15.64:0/1533051272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 msgr2=0x7f5d8c160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 --2- 172.21.15.64:0/1533051272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c160e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0098e0 tx=0x7f5d7c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- 172.21.15.64:0/1533051272 shutdown_connections 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 --2- 172.21.15.64:0/1533051272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c15e350 0x7f5d8c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 --2- 172.21.15.64:0/1533051272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c160e90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- 172.21.15.64:0/1533051272 >> 172.21.15.64:0/1533051272 conn(0x7f5d8c0c8420 msgr2=0x7f5d8c0c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- 172.21.15.64:0/1533051272 shutdown_connections 2023-09-27T23:13:31.191 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- 172.21.15.64:0/1533051272 wait complete. 2023-09-27T23:13:31.192 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 Processor -- start 2023-09-27T23:13:31.192 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- start start 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 0x7f5d8c14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c0c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c15e350 0x7f5d8c0c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1592c0 con 0x7f5d8c15ada0 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b08e0 con 0x7f5d8c15e350 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.187+0000 7f5d9120e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0afcd0 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.193 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 0x7f5d8c14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:31.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 0x7f5d8c14dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:59220/0 (socket says 172.21.15.64:59220) 2023-09-27T23:13:31.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 -- 172.21.15.64:0/2167684990 learned_addr learned my addr 172.21.15.64:0/2167684990 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:31.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8b7fe700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c0c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:31.194 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d90a0d700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c15e350 0x7f5d8c0c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 -- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c15e350 msgr2=0x7f5d8c0c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c15e350 0x7f5d8c0c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 -- 172.21.15.64:0/2167684990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 msgr2=0x7f5d8c0c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c0c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 -- 172.21.15.64:0/2167684990 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009420 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d90a0d700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c15e350 0x7f5d8c0c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d8bfff700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 0x7f5d8c14dbf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0093f0 tx=0x7f5d7c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:31.195 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c01cbe0 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c01cd40 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c0affb0 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c01a660 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c0b0570 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c04be40 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.196 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.191+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 4) v1 ==== 80203+0+0 (secure 0 0 0) 0x7f5d7c01aa90 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.197 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.195+0000 7f5d83fff700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d6c0608f0 0x7f5d6c062da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:31.197 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.195+0000 7f5d8b7fe700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d6c0608f0 0x7f5d6c062da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:31.198 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.195+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2507+0+0 (secure 0 0 0) 0x7f5d7c07d720 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.198 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.195+0000 7f5d8b7fe700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d6c0608f0 0x7f5d6c062da0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b0cf0 tx=0x7f5d740091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:31.204 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.199+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5d7c04e870 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.368 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.363+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d8c06f520 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.369 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d83fff700 1 -- 172.21.15.64:0/2167684990 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7450 (secure 0 0 0) 0x7f5d7c029070 con 0x7f5d8c0c6e60 2023-09-27T23:13:31.369 INFO:teuthology.misc.health.smithi064.stdout: 2023-09-27T23:13:31.370 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":14,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:30.416716+0000","last_up_change":"2023-09-27T23:13:30.416716+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d6c0608f0 msgr2=0x7f5d6c062da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d6c0608f0 0x7f5d6c062da0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b0cf0 tx=0x7f5d740091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 msgr2=0x7f5d8c14dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 0x7f5d8c14dbf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0093f0 tx=0x7f5d7c016000 comp rx=0 tx=0).stop 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 shutdown_connections 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d6c0608f0 0x7f5d6c062da0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c15e350 0x7f5d8c0c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.372 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c15ada0 0x7f5d8c0c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.373 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 --2- 172.21.15.64:0/2167684990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d8c0c6e60 0x7f5d8c14dbf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:31.373 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 >> 172.21.15.64:0/2167684990 conn(0x7f5d8c0c8420 msgr2=0x7f5d8c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:31.373 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 shutdown_connections 2023-09-27T23:13:31.373 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:31.367+0000 7f5d9120e700 1 -- 172.21.15.64:0/2167684990 wait complete. 2023-09-27T23:13:31.383 DEBUG:teuthology.misc:1 of 8 OSDs are up 2023-09-27T23:13:31.419 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-09-27T23:13:31.413+0000 7f0574fd4700 -1 osd.6 0 waiting for initial osdmap 2023-09-27T23:13:31.424 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-09-27T23:13:31.421+0000 7f056c553700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:31.430 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-09-27T23:13:31.425+0000 7fc6204fb700 -1 osd.5 0 waiting for initial osdmap 2023-09-27T23:13:31.432 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-09-27T23:13:31.425+0000 7f4304e6b700 -1 osd.4 0 waiting for initial osdmap 2023-09-27T23:13:31.438 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-09-27T23:13:31.433+0000 7f42fdc0f700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:31.445 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-09-27T23:13:31.437+0000 7fc617a7a700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:31.465 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-09-27T23:13:31.461+0000 7f2347e13700 -1 osd.7 0 waiting for initial osdmap 2023-09-27T23:13:31.470 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-09-27T23:13:31.465+0000 7f2340bb7700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-27T23:13:38.189 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:38.293 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.286+0000 7f2c56021700 1 Processor -- start 2023-09-27T23:13:38.293 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.286+0000 7f2c56021700 1 -- start start 2023-09-27T23:13:38.294 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c50156bc0 0x7f2c500b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5006efa0 con 0x7f2c50069da0 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500af450 con 0x7f2c5006ae80 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c500afc60 con 0x7f2c50156bc0 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c5501f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c50156bc0 0x7f2c500b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c55820700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2c50069da0 0x7f2c5006c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38224/0 (socket says 172.21.15.64:38224) 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c5501f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c50156bc0 0x7f2c500b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49378/0 (socket says 172.21.15.64:49378) 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c55820700 1 -- 172.21.15.64:0/3881802523 learned_addr learned my addr 172.21.15.64:0/3881802523 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c5501f700 1 -- 172.21.15.64:0/3881802523 >> v1:172.21.15.123:6789/0 conn(0x7f2c5006ae80 legacy=0x7f2c50069690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c47fff700 1 -- 172.21.15.64:0/3881802523 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597736746 0 0) 0x7f2c5006efa0 con 0x7f2c50069da0 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c5501f700 1 -- 172.21.15.64:0/3881802523 >> v1:172.21.15.64:6789/0 conn(0x7f2c50069da0 legacy=0x7f2c5006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.295 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c5501f700 1 -- 172.21.15.64:0/3881802523 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5006bc00 con 0x7f2c50156bc0 2023-09-27T23:13:38.296 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c5501f700 1 --2- 172.21.15.64:0/3881802523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50156bc0 0x7f2c500b1110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2c400098e0 tx=0x7f2c400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aff8e0f13922146 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:38.296 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c47fff700 1 -- 172.21.15.64:0/3881802523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40009e80 con 0x7f2c50156bc0 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c47fff700 1 -- 172.21.15.64:0/3881802523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4000b860 con 0x7f2c50156bc0 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c47fff700 1 -- 172.21.15.64:0/3881802523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000bbe0 con 0x7f2c50156bc0 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- 172.21.15.64:0/3881802523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50156bc0 msgr2=0x7f2c500b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 --2- 172.21.15.64:0/3881802523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50156bc0 0x7f2c500b1110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2c400098e0 tx=0x7f2c400091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- 172.21.15.64:0/3881802523 shutdown_connections 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 --2- 172.21.15.64:0/3881802523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50156bc0 0x7f2c500b1110 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- 172.21.15.64:0/3881802523 >> 172.21.15.64:0/3881802523 conn(0x7f2c500c8420 msgr2=0x7f2c500c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- 172.21.15.64:0/3881802523 shutdown_connections 2023-09-27T23:13:38.297 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.290+0000 7f2c56021700 1 -- 172.21.15.64:0/3881802523 wait complete. 2023-09-27T23:13:38.298 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 Processor -- start 2023-09-27T23:13:38.298 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 -- start start 2023-09-27T23:13:38.300 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 0x7f2c5014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.300 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c5006ae80 0x7f2c500c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.300 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c50156bc0 0x7f2c5006a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.300 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50155c60 con 0x7f2c50069da0 2023-09-27T23:13:38.300 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50150090 con 0x7f2c5006ae80 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50155050 con 0x7f2c50156bc0 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 0x7f2c5014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 0x7f2c5014fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49384/0 (socket says 172.21.15.64:49384) 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 -- 172.21.15.64:0/2145814474 learned_addr learned my addr 172.21.15.64:0/2145814474 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c55820700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c50156bc0 0x7f2c5006a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5481e700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c5006ae80 0x7f2c500c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 -- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c50156bc0 msgr2=0x7f2c5006a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c50156bc0 0x7f2c5006a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 -- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c5006ae80 msgr2=0x7f2c500c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c5006ae80 0x7f2c500c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 -- 172.21.15.64:0/2145814474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009420 con 0x7f2c50069da0 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5481e700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c5006ae80 0x7f2c500c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:13:38.301 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c5501f700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 0x7f2c5014fb50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2c50068fe0 tx=0x7f2c40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:38.302 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40011420 con 0x7f2c50069da0 2023-09-27T23:13:38.302 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c40011580 con 0x7f2c50069da0 2023-09-27T23:13:38.302 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50155330 con 0x7f2c50069da0 2023-09-27T23:13:38.302 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40011940 con 0x7f2c50069da0 2023-09-27T23:13:38.302 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.294+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c501558f0 con 0x7f2c50069da0 2023-09-27T23:13:38.302 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.298+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5004be40 con 0x7f2c50069da0 2023-09-27T23:13:38.303 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.298+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f2c40011cf0 con 0x7f2c50069da0 2023-09-27T23:13:38.303 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.298+0000 7f2c457fa700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c34060980 0x7f2c34062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.303 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.298+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2c4007e1d0 con 0x7f2c50069da0 2023-09-27T23:13:38.303 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.298+0000 7f2c5481e700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c34060980 0x7f2c34062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.304 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.298+0000 7f2c5481e700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c34060980 0x7f2c34062e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c500c5b90 tx=0x7f2c4c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:38.313 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.306+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2c4004e900 con 0x7f2c50069da0 2023-09-27T23:13:38.470 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.466+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c50153e20 con 0x7f2c50069da0 2023-09-27T23:13:38.472 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.466+0000 7f2c457fa700 1 -- 172.21.15.64:0/2145814474 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f2c4007ce60 con 0x7f2c50069da0 2023-09-27T23:13:38.472 INFO:teuthology.misc.health.smithi064.stdout: 2023-09-27T23:13:38.472 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":18,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:35.317015+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:38.474 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c34060980 msgr2=0x7f2c34062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.474 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c34060980 0x7f2c34062e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c500c5b90 tx=0x7f2c4c009040 comp rx=0 tx=0).stop 2023-09-27T23:13:38.474 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 msgr2=0x7f2c5014fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 0x7f2c5014fb50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2c50068fe0 tx=0x7f2c40017000 comp rx=0 tx=0).stop 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 shutdown_connections 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c34060980 0x7f2c34062e30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c50156bc0 0x7f2c5006a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c5006ae80 0x7f2c500c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 --2- 172.21.15.64:0/2145814474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50069da0 0x7f2c5014fb50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 >> 172.21.15.64:0/2145814474 conn(0x7f2c500c8420 msgr2=0x7f2c501524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 shutdown_connections 2023-09-27T23:13:38.475 INFO:teuthology.misc.health.smithi064.stderr:2023-09-27T23:13:38.470+0000 7f2c56021700 1 -- 172.21.15.64:0/2145814474 wait complete. 2023-09-27T23:13:38.485 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-09-27T23:13:38.485 INFO:tasks.ceph:Creating RBD pool 2023-09-27T23:13:38.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-27T23:13:38.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 Processor -- start 2023-09-27T23:13:38.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 -- start start 2023-09-27T23:13:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85dc0b08d0 0x7f85dc159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc15a3f0 con 0x7f85dc159fb0 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85dc0b0e40 con 0x7f85dc0b08d0 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.594+0000 7f85e0914700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b1120 con 0x7f85dc0ab870 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85dc0b08d0 0x7f85dc159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85dad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f85dc159fb0 0x7f85dc156470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38238/0 (socket says 172.21.15.64:38238) 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85dc0b08d0 0x7f85dc159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49400/0 (socket says 172.21.15.64:49400) 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85dad9d700 1 -- 172.21.15.64:0/3763214952 learned_addr learned my addr 172.21.15.64:0/3763214952 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:38.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85da59c700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 -- 172.21.15.64:0/3763214952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 msgr2=0x7f85dc0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc0afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 -- 172.21.15.64:0/3763214952 >> v1:172.21.15.64:6789/0 conn(0x7f85dc159fb0 legacy=0x7f85dc156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 -- 172.21.15.64:0/3763214952 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0b1400 con 0x7f85dc0b08d0 2023-09-27T23:13:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85da59c700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d9d9b700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc0b08d0 0x7f85dc159a70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f85c80098e0 tx=0x7f85c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f11a03ec044a9d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:38.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d959a700 1 -- 172.21.15.64:0/3763214952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800b640 con 0x7f85dc0b08d0 2023-09-27T23:13:38.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d959a700 1 -- 172.21.15.64:0/3763214952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c800b7a0 con 0x7f85dc0b08d0 2023-09-27T23:13:38.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85d959a700 1 -- 172.21.15.64:0/3763214952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800bb40 con 0x7f85dc0b08d0 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 -- 172.21.15.64:0/3763214952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc0b08d0 msgr2=0x7f85dc159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc0b08d0 0x7f85dc159a70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f85c80098e0 tx=0x7f85c80091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 -- 172.21.15.64:0/3763214952 shutdown_connections 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc0b08d0 0x7f85dc159a70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 --2- 172.21.15.64:0/3763214952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 -- 172.21.15.64:0/3763214952 >> 172.21.15.64:0/3763214952 conn(0x7f85dc0cc580 msgr2=0x7f85dc0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 -- 172.21.15.64:0/3763214952 shutdown_connections 2023-09-27T23:13:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.598+0000 7f85e0914700 1 -- 172.21.15.64:0/3763214952 wait complete. 2023-09-27T23:13:38.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 Processor -- start 2023-09-27T23:13:38.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 -- start start 2023-09-27T23:13:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85dc0b08d0 0x7f85dc0a8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 0x7f85dc158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc1562e0 con 0x7f85dc159fb0 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0a5470 con 0x7f85dc0b08d0 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.602+0000 7f85e0914700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc156df0 con 0x7f85dc0ab870 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 0x7f85dc158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 0x7f85dc158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49412/0 (socket says 172.21.15.64:49412) 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 -- 172.21.15.64:0/658134328 learned_addr learned my addr 172.21.15.64:0/658134328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85da59c700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85d9d9b700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85dc0b08d0 0x7f85dc0a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 -- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 msgr2=0x7f85dc152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 -- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85dc0b08d0 msgr2=0x7f85dc0a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85dc0b08d0 0x7f85dc0a8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:38.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 -- 172.21.15.64:0/658134328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc009420 con 0x7f85dc159fb0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85d9d9b700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85dc0b08d0 0x7f85dc0a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85dad9d700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 0x7f85dc158c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85d400cca0 tx=0x7f85d4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400a920 con 0x7f85dc159fb0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d400aaa0 con 0x7f85dc159fb0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c8009420 con 0x7f85dc159fb0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400aea0 con 0x7f85dc159fb0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc1564c0 con 0x7f85dc159fb0 2023-09-27T23:13:38.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc071120 con 0x7f85dc159fb0 2023-09-27T23:13:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.606+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f85d4007300 con 0x7f85dc159fb0 2023-09-27T23:13:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.610+0000 7f85d27fc700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85c4060950 0x7f85c4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:38.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.610+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f85d407ccd0 con 0x7f85dc159fb0 2023-09-27T23:13:38.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.610+0000 7f85da59c700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85c4060950 0x7f85c4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:38.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.610+0000 7f85da59c700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85c4060950 0x7f85c4062e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009ea0 tx=0x7f85cc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:38.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.614+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f85d404d480 con 0x7f85dc159fb0 2023-09-27T23:13:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:38.778+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f85dc156f30 con 0x7f85dc159fb0 2023-09-27T23:13:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.350+0000 7f85d27fc700 1 -- 172.21.15.64:0/658134328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v19) v1 ==== 109+0+0 (secure 0 0 0) 0x7f85d4050aa0 con 0x7f85dc159fb0 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85c4060950 msgr2=0x7f85c4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85c4060950 0x7f85c4062e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009ea0 tx=0x7f85cc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 msgr2=0x7f85dc158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 0x7f85dc158c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85d400cca0 tx=0x7f85d4001a40 comp rx=0 tx=0).stop 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 shutdown_connections 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85c4060950 0x7f85c4062e00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85dc159fb0 0x7f85dc158c40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85dc0b08d0 0x7f85dc0a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 --2- 172.21.15.64:0/658134328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85dc0ab870 0x7f85dc152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.354+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 >> 172.21.15.64:0/658134328 conn(0x7f85dc0cc580 msgr2=0x7f85dc0ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:39.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.358+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 shutdown_connections 2023-09-27T23:13:39.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.358+0000 7f85e0914700 1 -- 172.21.15.64:0/658134328 wait complete. 2023-09-27T23:13:39.363 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'rbd' created 2023-09-27T23:13:39.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-27T23:13:39.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.482+0000 7f7c70f92700 1 Processor -- start 2023-09-27T23:13:39.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.482+0000 7f7c70f92700 1 -- start start 2023-09-27T23:13:39.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c6c1569e0 0x7f7c6c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:39.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7c6c0aef60 0x7f7c6c0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c0b2060 con 0x7f7c6c0adf40 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c06beb0 con 0x7f7c6c1569e0 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c06cb80 con 0x7f7c6c0aef60 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c6c1569e0 0x7f7c6c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7c6c0adf40 0x7f7c6c0ae390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38242/0 (socket says 172.21.15.64:38242) 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6bfff700 1 -- 172.21.15.64:0/3661336175 learned_addr learned my addr 172.21.15.64:0/3661336175 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 --2- 172.21.15.64:0/3661336175 >> v2:172.21.15.64:3300/0 conn(0x7f7c6c1569e0 0x7f7c6c156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49424/0 (socket says 172.21.15.64:49424) 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c62ffd700 1 --2- 172.21.15.64:0/3661336175 >> v2:172.21.15.123:3300/0 conn(0x7f7c6c0aef60 0x7f7c6c0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 -- 172.21.15.64:0/3661336175 >> v2:172.21.15.123:3300/0 conn(0x7f7c6c0aef60 msgr2=0x7f7c6c0b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6affd700 1 -- 172.21.15.64:0/3661336175 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291579560 0 0) 0x7f7c6c0b2060 con 0x7f7c6c0adf40 2023-09-27T23:13:39.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 --2- 172.21.15.64:0/3661336175 >> v2:172.21.15.123:3300/0 conn(0x7f7c6c0aef60 0x7f7c6c0b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 -- 172.21.15.64:0/3661336175 >> v1:172.21.15.64:6789/0 conn(0x7f7c6c0adf40 legacy=0x7f7c6c0ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:39.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 -- 172.21.15.64:0/3661336175 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c06ce60 con 0x7f7c6c1569e0 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6b7fe700 1 --2- 172.21.15.64:0/3661336175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c156db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c06a5d0 tx=0x7f7c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84c5898976b13f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6affd700 1 -- 172.21.15.64:0/3661336175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c011630 con 0x7f7c6c1569e0 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6affd700 1 -- 172.21.15.64:0/3661336175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c011790 con 0x7f7c6c1569e0 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c6affd700 1 -- 172.21.15.64:0/3661336175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c011b30 con 0x7f7c6c1569e0 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- 172.21.15.64:0/3661336175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 msgr2=0x7f7c6c156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 --2- 172.21.15.64:0/3661336175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c156db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c06a5d0 tx=0x7f7c5c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- 172.21.15.64:0/3661336175 shutdown_connections 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 --2- 172.21.15.64:0/3661336175 >> v2:172.21.15.123:3300/0 conn(0x7f7c6c0aef60 0x7f7c6c0b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 --2- 172.21.15.64:0/3661336175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c156db0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- 172.21.15.64:0/3661336175 >> 172.21.15.64:0/3661336175 conn(0x7f7c6c0c8c00 msgr2=0x7f7c6c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:39.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- 172.21.15.64:0/3661336175 shutdown_connections 2023-09-27T23:13:39.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.486+0000 7f7c70f92700 1 -- 172.21.15.64:0/3661336175 wait complete. 2023-09-27T23:13:39.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 Processor -- start 2023-09-27T23:13:39.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 -- start start 2023-09-27T23:13:39.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6c0adf40 0x7f7c6c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:39.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c0aef60 0x7f7c6c169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:39.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c157350 con 0x7f7c6c1569e0 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1575f0 con 0x7f7c6c0aef60 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1578d0 con 0x7f7c6c0adf40 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49440/0 (socket says 172.21.15.64:49440) 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 -- 172.21.15.64:0/2331137325 learned_addr learned my addr 172.21.15.64:0/2331137325 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6b7fe700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6c0adf40 0x7f7c6c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:39.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c62ffd700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c0aef60 0x7f7c6c169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:39.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 -- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6c0adf40 msgr2=0x7f7c6c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:39.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6c0adf40 0x7f7c6c0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 -- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c0aef60 msgr2=0x7f7c6c169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:39.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c0aef60 0x7f7c6c169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 -- 172.21.15.64:0/2331137325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c009420 con 0x7f7c6c1569e0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c6bfff700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c0b0b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400cc90 tx=0x7f7c64001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400a8b0 con 0x7f7c6c1569e0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c157bb0 con 0x7f7c6c1569e0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6400aa10 con 0x7f7c6c1569e0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400add0 con 0x7f7c6c1569e0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.490+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c16a240 con 0x7f7c6c1569e0 2023-09-27T23:13:39.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.494+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c04be40 con 0x7f7c6c1569e0 2023-09-27T23:13:39.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.494+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7c64007300 con 0x7f7c6c1569e0 2023-09-27T23:13:39.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.494+0000 7f7c63fff700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c4c060980 0x7f7c4c062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:39.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.494+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f7c6407ce60 con 0x7f7c6c1569e0 2023-09-27T23:13:39.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.494+0000 7f7c6b7fe700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c4c060980 0x7f7c4c062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:39.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.498+0000 7f7c6b7fe700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c4c060980 0x7f7c4c062e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0093f0 tx=0x7f7c5c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:39.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.502+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7c6404d440 con 0x7f7c6c1569e0 2023-09-27T23:13:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:39.666+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7c6c06d140 con 0x7f7c6c1569e0 2023-09-27T23:13:40.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.354+0000 7f7c63fff700 1 -- 172.21.15.64:0/2331137325 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v20) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7c64050a60 con 0x7f7c6c1569e0 2023-09-27T23:13:40.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c4c060980 msgr2=0x7f7c4c062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c4c060980 0x7f7c4c062e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0093f0 tx=0x7f7c5c017000 comp rx=0 tx=0).stop 2023-09-27T23:13:40.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 msgr2=0x7f7c6c0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c0b0b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400cc90 tx=0x7f7c64001a80 comp rx=0 tx=0).stop 2023-09-27T23:13:40.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 shutdown_connections 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c4c060980 0x7f7c4c062e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c6c1569e0 0x7f7c6c0b0b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c0aef60 0x7f7c6c169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 --2- 172.21.15.64:0/2331137325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c6c0adf40 0x7f7c6c0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 >> 172.21.15.64:0/2331137325 conn(0x7f7c6c0c8c00 msgr2=0x7f7c6c067970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 shutdown_connections 2023-09-27T23:13:40.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.358+0000 7f7c70f92700 1 -- 172.21.15.64:0/2331137325 wait complete. 2023-09-27T23:13:40.366 INFO:teuthology.orchestra.run.smithi064.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-27T23:13:40.380 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-27T23:13:40.380 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:13:40.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:13:40.383 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:13:40.383 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:13:40.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:13:40.386 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:13:40.386 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:13:40.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:13:40.389 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:13:40.389 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:13:40.389 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:13:40.392 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:13:40.392 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-27T23:13:40.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:40.455 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:13:40.456 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:13:40.460 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:13:40.465 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:13:40.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.498+0000 7ffa45ee6700 1 Processor -- start 2023-09-27T23:13:40.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.498+0000 7ffa45ee6700 1 -- start start 2023-09-27T23:13:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffa400b0810 0x7ffa400b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa4006be10 0x7ffa40158d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400af530 con 0x7ffa400af9c0 2023-09-27T23:13:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa40159660 con 0x7ffa400b0810 2023-09-27T23:13:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40159940 con 0x7ffa4006be10 2023-09-27T23:13:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffa400b0810 0x7ffa400b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffa400b0810 0x7ffa400b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49468/0 (socket says 172.21.15.64:49468) 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 -- 172.21.15.64:0/3400571943 learned_addr learned my addr 172.21.15.64:0/3400571943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/3400571943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa4006be10 0x7ffa40158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 -- 172.21.15.64:0/3400571943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa4006be10 msgr2=0x7ffa40158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 --2- 172.21.15.64:0/3400571943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa4006be10 0x7ffa40158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 -- 172.21.15.64:0/3400571943 >> v1:172.21.15.64:6789/0 conn(0x7ffa400af9c0 legacy=0x7ffa40154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/3400571943 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429430795 0 0) 0x7ffa400af530 con 0x7ffa400af9c0 2023-09-27T23:13:40.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 -- 172.21.15.64:0/3400571943 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40159c20 con 0x7ffa400b0810 2023-09-27T23:13:40.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa44ee4700 1 --2- 172.21.15.64:0/3400571943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b0810 0x7ffa400b0be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa300099e0 tx=0x7ffa30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d8ae5a27dd3ab5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:40.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/3400571943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000b8b0 con 0x7ffa400b0810 2023-09-27T23:13:40.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/3400571943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3000ba10 con 0x7ffa400b0810 2023-09-27T23:13:40.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/3400571943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000bdb0 con 0x7ffa400b0810 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- 172.21.15.64:0/3400571943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b0810 msgr2=0x7ffa400b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/3400571943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b0810 0x7ffa400b0be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa300099e0 tx=0x7ffa30009250 comp rx=0 tx=0).stop 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- 172.21.15.64:0/3400571943 shutdown_connections 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/3400571943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa4006be10 0x7ffa40158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/3400571943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b0810 0x7ffa400b0be0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- 172.21.15.64:0/3400571943 >> 172.21.15.64:0/3400571943 conn(0x7ffa40067bb0 msgr2=0x7ffa400add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:40.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- 172.21.15.64:0/3400571943 shutdown_connections 2023-09-27T23:13:40.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.502+0000 7ffa45ee6700 1 -- 172.21.15.64:0/3400571943 wait complete. 2023-09-27T23:13:40.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.506+0000 7ffa45ee6700 1 Processor -- start 2023-09-27T23:13:40.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.506+0000 7ffa45ee6700 1 -- start start 2023-09-27T23:13:40.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 0x7ffa4014bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400af9c0 0x7ffa401569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400b0810 0x7ffa40153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401679f0 con 0x7ffa4006be10 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400c1e50 con 0x7ffa400af9c0 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400c1240 con 0x7ffa400b0810 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 0x7ffa4014bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 0x7ffa4014bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49470/0 (socket says 172.21.15.64:49470) 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 -- 172.21.15.64:0/700609288 learned_addr learned my addr 172.21.15.64:0/700609288 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa456e5700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400b0810 0x7ffa40153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400af9c0 0x7ffa401569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 -- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400b0810 msgr2=0x7ffa40153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400b0810 0x7ffa40153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 -- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400af9c0 msgr2=0x7ffa401569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400af9c0 0x7ffa401569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 -- 172.21.15.64:0/700609288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300094f0 con 0x7ffa4006be10 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400af9c0 0x7ffa401569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:40.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa44ee4700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 0x7ffa4014bcc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa30006d20 tx=0x7ffa30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:40.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30026450 con 0x7ffa4006be10 2023-09-27T23:13:40.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa300265b0 con 0x7ffa4006be10 2023-09-27T23:13:40.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa400c1520 con 0x7ffa4006be10 2023-09-27T23:13:40.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30026950 con 0x7ffa4006be10 2023-09-27T23:13:40.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa400c1ae0 con 0x7ffa4006be10 2023-09-27T23:13:40.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.510+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4004be40 con 0x7ffa4006be10 2023-09-27T23:13:40.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.514+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ffa3001a420 con 0x7ffa4006be10 2023-09-27T23:13:40.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.514+0000 7ffa3cff9700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa2c060950 0x7ffa2c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.514+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ffa3007e9e0 con 0x7ffa4006be10 2023-09-27T23:13:40.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.514+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa2c060950 0x7ffa2c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.514+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa2c060950 0x7ffa2c062e00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c8460 tx=0x7ffa34008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.522+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ffa3004f030 con 0x7ffa4006be10 2023-09-27T23:13:40.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.686+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa400af1c0 con 0x7ffa4006be10 2023-09-27T23:13:40.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa3cff9700 1 -- 172.21.15.64:0/700609288 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13660 (secure 0 0 0) 0x7ffa30052650 con 0x7ffa4006be10 2023-09-27T23:13:40.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:40.695 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:40.347215+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:40.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa2c060950 msgr2=0x7ffa2c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa2c060950 0x7ffa2c062e00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffa400c8460 tx=0x7ffa34008040 comp rx=0 tx=0).stop 2023-09-27T23:13:40.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 msgr2=0x7ffa4014bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 0x7ffa4014bcc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa30006d20 tx=0x7ffa30017000 comp rx=0 tx=0).stop 2023-09-27T23:13:40.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 shutdown_connections 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa2c060950 0x7ffa2c062e00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400b0810 0x7ffa40153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400af9c0 0x7ffa401569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 --2- 172.21.15.64:0/700609288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa4006be10 0x7ffa4014bcc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 >> 172.21.15.64:0/700609288 conn(0x7ffa40067bb0 msgr2=0x7ffa401579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 shutdown_connections 2023-09-27T23:13:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.690+0000 7ffa45ee6700 1 -- 172.21.15.64:0/700609288 wait complete. 2023-09-27T23:13:40.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:13:40.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:13:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.822+0000 7f65891f2700 1 Processor -- start 2023-09-27T23:13:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.822+0000 7f65891f2700 1 -- start start 2023-09-27T23:13:40.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.822+0000 7f65891f2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840ace30 0x7f65840cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.822+0000 7f65891f2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65840c7610 con 0x7f6584150200 2023-09-27T23:13:40.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.822+0000 7f65891f2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65840af250 con 0x7f65840c7170 2023-09-27T23:13:40.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.822+0000 7f65891f2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65840af530 con 0x7f65840ace30 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f6583fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6584150200 0x7f65840ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38260/0 (socket says 172.21.15.64:38260) 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f6583fff700 1 -- 172.21.15.64:0/664510562 learned_addr learned my addr 172.21.15.64:0/664510562 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65889f1700 1 --2- 172.21.15.64:0/664510562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840ace30 0x7f65840cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418553745 0 0) 0x7f65840c7610 con 0x7f6584150200 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6564003570 con 0x7f6584150200 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804117228 0 0) 0x7f65840af250 con 0x7f65840c7170 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65840c7610 con 0x7f65840c7170 2023-09-27T23:13:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65889f1700 1 -- 172.21.15.64:0/664510562 >> v1:172.21.15.123:6789/0 conn(0x7f65840c7170 legacy=0x7f65840ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8275614 0 0) 0x7f6564003570 con 0x7f6584150200 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65889f1700 1 -- 172.21.15.64:0/664510562 >> v1:172.21.15.64:6789/0 conn(0x7f6584150200 legacy=0x7f65840ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65889f1700 1 -- 172.21.15.64:0/664510562 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65840ca8a0 con 0x7f65840ace30 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65889f1700 1 --2- 172.21.15.64:0/664510562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840ace30 0x7f65840cb380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f657c009e90 tx=0x7f657c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=366c8cb4c7620672 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c00e070 con 0x7f65840ace30 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c00b870 con 0x7f65840ace30 2023-09-27T23:13:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65837fe700 1 -- 172.21.15.64:0/664510562 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574001470 con 0x7f65840ace30 2023-09-27T23:13:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 -- 172.21.15.64:0/664510562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840ace30 msgr2=0x7f65840cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 --2- 172.21.15.64:0/664510562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840ace30 0x7f65840cb380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f657c009e90 tx=0x7f657c009780 comp rx=0 tx=0).stop 2023-09-27T23:13:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 -- 172.21.15.64:0/664510562 shutdown_connections 2023-09-27T23:13:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 --2- 172.21.15.64:0/664510562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840ace30 0x7f65840cb380 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 -- 172.21.15.64:0/664510562 >> 172.21.15.64:0/664510562 conn(0x7f65840ccb30 msgr2=0x7f65840ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 -- 172.21.15.64:0/664510562 shutdown_connections 2023-09-27T23:13:40.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.826+0000 7f65891f2700 1 -- 172.21.15.64:0/664510562 wait complete. 2023-09-27T23:13:40.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 Processor -- start 2023-09-27T23:13:40.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 -- start start 2023-09-27T23:13:40.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 0x7f65840af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840c7170 0x7f65840b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584150200 0x7f6584149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584167260 con 0x7f65840ace30 2023-09-27T23:13:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584167fe0 con 0x7f6584150200 2023-09-27T23:13:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f65891f2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6584167500 con 0x7f65840c7170 2023-09-27T23:13:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f6583fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 0x7f65840af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f6583fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 0x7f65840af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49484/0 (socket says 172.21.15.64:49484) 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.830+0000 7f6583fff700 1 -- 172.21.15.64:0/4274252797 learned_addr learned my addr 172.21.15.64:0/4274252797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f65889f1700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584150200 0x7f6584149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f657b7fe700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840c7170 0x7f65840b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6583fff700 1 -- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840c7170 msgr2=0x7f65840b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6583fff700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840c7170 0x7f65840b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6583fff700 1 -- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584150200 msgr2=0x7f6584149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6583fff700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584150200 0x7f6584149550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:40.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6583fff700 1 -- 172.21.15.64:0/4274252797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c009a20 con 0x7f65840ace30 2023-09-27T23:13:40.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f65889f1700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584150200 0x7f6584149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6583fff700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 0x7f65840af8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f657400cb70 tx=0x7f6574001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574001d00 con 0x7f65840ace30 2023-09-27T23:13:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6574007360 con 0x7f65840ace30 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574004830 con 0x7f65840ace30 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65841677e0 con 0x7f65840ace30 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65840aff60 con 0x7f65840ace30 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6574004be0 con 0x7f65840ace30 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6580ff9700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6568060950 0x7f6568062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f657407c2f0 con 0x7f65840ace30 2023-09-27T23:13:40.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.834+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658404be40 con 0x7f65840ace30 2023-09-27T23:13:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.838+0000 7f657b7fe700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6568060950 0x7f6568062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:40.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.838+0000 7f657b7fe700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6568060950 0x7f6568062e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f658415b930 tx=0x7f656c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:40.846+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6574018080 con 0x7f65840ace30 2023-09-27T23:13:41.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6584157d70 con 0x7f65840ace30 2023-09-27T23:13:41.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f6580ff9700 1 -- 172.21.15.64:0/4274252797 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f657404c8c0 con 0x7f65840ace30 2023-09-27T23:13:41.015 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:13:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6568060950 msgr2=0x7f6568062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6568060950 0x7f6568062e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f658415b930 tx=0x7f656c007d50 comp rx=0 tx=0).stop 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 msgr2=0x7f65840af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 0x7f65840af8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f657400cb70 tx=0x7f6574001ab0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 shutdown_connections 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6568060950 0x7f6568062e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584150200 0x7f6584149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840c7170 0x7f65840b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 --2- 172.21.15.64:0/4274252797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65840ace30 0x7f65840af8d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 >> 172.21.15.64:0/4274252797 conn(0x7f65840ccb30 msgr2=0x7f658415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 shutdown_connections 2023-09-27T23:13:41.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.010+0000 7f65891f2700 1 -- 172.21.15.64:0/4274252797 wait complete. 2023-09-27T23:13:41.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:13:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 Processor -- start 2023-09-27T23:13:41.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 -- start start 2023-09-27T23:13:41.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4610158c90 0x7f46100caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f461006aeb0 0x7f4610159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461006c850 con 0x7f46101595a0 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461006b470 con 0x7f461006aeb0 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f461754b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461006b750 con 0x7f4610158c90 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f4615d48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f461006aeb0 0x7f4610159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f4616d4a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f46101595a0 0x7f461015c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38266/0 (socket says 172.21.15.64:38266) 2023-09-27T23:13:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.134+0000 7f4616d4a700 1 -- 172.21.15.64:0/561644570 learned_addr learned my addr 172.21.15.64:0/561644570 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:41.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4616549700 1 --2- 172.21.15.64:0/561644570 >> v2:172.21.15.123:3300/0 conn(0x7f4610158c90 0x7f46100caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615547700 1 -- 172.21.15.64:0/561644570 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689796424 0 0) 0x7f461006c850 con 0x7f46101595a0 2023-09-27T23:13:41.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615547700 1 -- 172.21.15.64:0/561644570 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45fc003570 con 0x7f46101595a0 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615d48700 1 -- 172.21.15.64:0/561644570 >> v2:172.21.15.123:3300/0 conn(0x7f4610158c90 msgr2=0x7f46100caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615d48700 1 --2- 172.21.15.64:0/561644570 >> v2:172.21.15.123:3300/0 conn(0x7f4610158c90 0x7f46100caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615d48700 1 -- 172.21.15.64:0/561644570 >> v1:172.21.15.64:6789/0 conn(0x7f46101595a0 legacy=0x7f461015c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615d48700 1 -- 172.21.15.64:0/561644570 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461006ba30 con 0x7f461006aeb0 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615d48700 1 --2- 172.21.15.64:0/561644570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610159060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f46100698f0 tx=0x7f46040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703330e6a0d3d176 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615547700 1 -- 172.21.15.64:0/561644570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604011630 con 0x7f461006aeb0 2023-09-27T23:13:41.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615547700 1 -- 172.21.15.64:0/561644570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040117b0 con 0x7f461006aeb0 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f4615547700 1 -- 172.21.15.64:0/561644570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604011b70 con 0x7f461006aeb0 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 -- 172.21.15.64:0/561644570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 msgr2=0x7f4610159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 --2- 172.21.15.64:0/561644570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610159060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f46100698f0 tx=0x7f46040091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 -- 172.21.15.64:0/561644570 shutdown_connections 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 --2- 172.21.15.64:0/561644570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610159060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 --2- 172.21.15.64:0/561644570 >> v2:172.21.15.123:3300/0 conn(0x7f4610158c90 0x7f46100caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 -- 172.21.15.64:0/561644570 >> 172.21.15.64:0/561644570 conn(0x7f4610067bb0 msgr2=0x7f46100cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 -- 172.21.15.64:0/561644570 shutdown_connections 2023-09-27T23:13:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.138+0000 7f461754b700 1 -- 172.21.15.64:0/561644570 wait complete. 2023-09-27T23:13:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 Processor -- start 2023-09-27T23:13:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 -- start start 2023-09-27T23:13:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610169430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158c90 0x7f4610157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46101595a0 0x7f4610153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100c4e10 con 0x7f461006aeb0 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610158460 con 0x7f46101595a0 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4610169a80 con 0x7f4610158c90 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610169430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610169430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49504/0 (socket says 172.21.15.64:49504) 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 -- 172.21.15.64:0/1589801391 learned_addr learned my addr 172.21.15.64:0/1589801391 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4615d48700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158c90 0x7f4610157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616d4a700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46101595a0 0x7f4610153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 -- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158c90 msgr2=0x7f4610157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158c90 0x7f4610157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 -- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46101595a0 msgr2=0x7f4610153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46101595a0 0x7f4610153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 -- 172.21.15.64:0/1589801391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604009420 con 0x7f461006aeb0 2023-09-27T23:13:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f4616549700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610169430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f460c00ec80 tx=0x7f460c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00c850 con 0x7f461006aeb0 2023-09-27T23:13:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00c9b0 con 0x7f461006aeb0 2023-09-27T23:13:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00cd50 con 0x7f461006aeb0 2023-09-27T23:13:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4610169d60 con 0x7f461006aeb0 2023-09-27T23:13:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.142+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461016a320 con 0x7f461006aeb0 2023-09-27T23:13:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.146+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f460c013680 con 0x7f461006aeb0 2023-09-27T23:13:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.146+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461004be40 con 0x7f461006aeb0 2023-09-27T23:13:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.146+0000 7f46027fc700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45f4060950 0x7f45f4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.146+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f460c07ce80 con 0x7f461006aeb0 2023-09-27T23:13:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.146+0000 7f4615d48700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45f4060950 0x7f45f4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.150+0000 7f4615d48700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45f4060950 0x7f45f4062e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46100698f0 tx=0x7f4604017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.154+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f460c04d3b0 con 0x7f461006aeb0 2023-09-27T23:13:41.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.314+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f461006bd10 con 0x7f461006aeb0 2023-09-27T23:13:41.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f46027fc700 1 -- 172.21.15.64:0/1589801391 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f460c0210a0 con 0x7f461006aeb0 2023-09-27T23:13:41.322 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:13:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45f4060950 msgr2=0x7f45f4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45f4060950 0x7f45f4062e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46100698f0 tx=0x7f4604017000 comp rx=0 tx=0).stop 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 msgr2=0x7f4610169430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610169430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f460c00ec80 tx=0x7f460c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 shutdown_connections 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45f4060950 0x7f45f4062e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46101595a0 0x7f4610153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158c90 0x7f4610157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 --2- 172.21.15.64:0/1589801391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f461006aeb0 0x7f4610169430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 >> 172.21.15.64:0/1589801391 conn(0x7f4610067bb0 msgr2=0x7f46100b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 shutdown_connections 2023-09-27T23:13:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.318+0000 7f461754b700 1 -- 172.21.15.64:0/1589801391 wait complete. 2023-09-27T23:13:41.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:13:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 Processor -- start 2023-09-27T23:13:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- start start 2023-09-27T23:13:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa46815d0b0 0x7fa468163d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa468068dc0 con 0x7fa468158d40 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa468068490 con 0x7fa4680cb920 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4680cc910 con 0x7fa46815d0b0 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa468158d40 0x7fa4680cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38278/0 (socket says 172.21.15.64:38278) 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46659c700 1 -- 172.21.15.64:0/4039981838 learned_addr learned my addr 172.21.15.64:0/4039981838 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa466d9d700 1 --2- 172.21.15.64:0/4039981838 >> v2:172.21.15.123:3300/0 conn(0x7fa46815d0b0 0x7fa468163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46559a700 1 -- 172.21.15.64:0/4039981838 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899023873 0 0) 0x7fa468068dc0 con 0x7fa468158d40 2023-09-27T23:13:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46559a700 1 -- 172.21.15.64:0/4039981838 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa450003570 con 0x7fa468158d40 2023-09-27T23:13:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa466d9d700 1 -- 172.21.15.64:0/4039981838 >> v1:172.21.15.123:6789/0 conn(0x7fa4680cb920 legacy=0x7fa46815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa466d9d700 1 -- 172.21.15.64:0/4039981838 >> v1:172.21.15.64:6789/0 conn(0x7fa468158d40 legacy=0x7fa4680cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa466d9d700 1 -- 172.21.15.64:0/4039981838 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa468068770 con 0x7fa46815d0b0 2023-09-27T23:13:41.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa466d9d700 1 --2- 172.21.15.64:0/4039981838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa46815d0b0 0x7fa468163d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa460009e60 tx=0x7fa460009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f17d7fc1083524 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46559a700 1 -- 172.21.15.64:0/4039981838 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46000e070 con 0x7fa46815d0b0 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46559a700 1 -- 172.21.15.64:0/4039981838 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46000bb60 con 0x7fa46815d0b0 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46559a700 1 -- 172.21.15.64:0/4039981838 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460012430 con 0x7fa46815d0b0 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- 172.21.15.64:0/4039981838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa46815d0b0 msgr2=0x7fa468163d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 --2- 172.21.15.64:0/4039981838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa46815d0b0 0x7fa468163d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa460009e60 tx=0x7fa460009780 comp rx=0 tx=0).stop 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- 172.21.15.64:0/4039981838 shutdown_connections 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 --2- 172.21.15.64:0/4039981838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa46815d0b0 0x7fa468163d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.442+0000 7fa46759e700 1 -- 172.21.15.64:0/4039981838 >> 172.21.15.64:0/4039981838 conn(0x7fa4680cdbc0 msgr2=0x7fa4680cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 -- 172.21.15.64:0/4039981838 shutdown_connections 2023-09-27T23:13:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 -- 172.21.15.64:0/4039981838 wait complete. 2023-09-27T23:13:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 Processor -- start 2023-09-27T23:13:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 -- start start 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 0x7fa468150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468158d40 0x7fa4680c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa46815d0b0 0x7fa4680c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680c4c70 con 0x7fa4680cb920 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680c4e80 con 0x7fa468158d40 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680c5160 con 0x7fa46815d0b0 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 0x7fa468150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 0x7fa468150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49516/0 (socket says 172.21.15.64:49516) 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46659c700 1 -- 172.21.15.64:0/2356340536 learned_addr learned my addr 172.21.15.64:0/2356340536 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa465d9b700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468158d40 0x7fa4680c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa466d9d700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa46815d0b0 0x7fa4680c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46659c700 1 -- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa46815d0b0 msgr2=0x7fa4680c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46659c700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa46815d0b0 0x7fa4680c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.446+0000 7fa46659c700 1 -- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468158d40 msgr2=0x7fa4680c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa46659c700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468158d40 0x7fa4680c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa46659c700 1 -- 172.21.15.64:0/2356340536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460009a20 con 0x7fa4680cb920 2023-09-27T23:13:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa46659c700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 0x7fa468150e20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa45800c7c0 tx=0x7fa458007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458005b30 con 0x7fa4680cb920 2023-09-27T23:13:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458005c90 con 0x7fa4680cb920 2023-09-27T23:13:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4680c7520 con 0x7fa4680cb920 2023-09-27T23:13:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458007300 con 0x7fa4680cb920 2023-09-27T23:13:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4680cb290 con 0x7fa4680cb920 2023-09-27T23:13:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.450+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4680c7380 con 0x7fa4680cb920 2023-09-27T23:13:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.454+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fa45801b3b0 con 0x7fa4680cb920 2023-09-27T23:13:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.454+0000 7fa4567fc700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa448060950 0x7fa448062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.454+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa45807c060 con 0x7fa4680cb920 2023-09-27T23:13:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.454+0000 7fa465d9b700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa448060950 0x7fa448062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.454+0000 7fa465d9b700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa448060950 0x7fa448062e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa46815bb00 tx=0x7fa45c007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.462+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa4680c7380 con 0x7fa4680cb920 2023-09-27T23:13:41.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.654+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4680c7380 con 0x7fa4680cb920 2023-09-27T23:13:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.654+0000 7fa4567fc700 1 -- 172.21.15.64:0/2356340536 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fa4680c7380 con 0x7fa4680cb920 2023-09-27T23:13:41.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:41.661 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa448060950 msgr2=0x7fa448062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa448060950 0x7fa448062e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa46815bb00 tx=0x7fa45c007ea0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 msgr2=0x7fa468150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 0x7fa468150e20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa45800c7c0 tx=0x7fa458007e00 comp rx=0 tx=0).stop 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 shutdown_connections 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa448060950 0x7fa448062e00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa46815d0b0 0x7fa4680c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468158d40 0x7fa4680c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 --2- 172.21.15.64:0/2356340536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4680cb920 0x7fa468150e20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 >> 172.21.15.64:0/2356340536 conn(0x7fa4680cdbc0 msgr2=0x7fa4680b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 shutdown_connections 2023-09-27T23:13:41.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.658+0000 7fa46759e700 1 -- 172.21.15.64:0/2356340536 wait complete. 2023-09-27T23:13:41.666 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2023-09-27T23:13:41.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:41.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572d91700 1 Processor -- start 2023-09-27T23:13:41.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572d91700 1 -- start start 2023-09-27T23:13:41.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572d91700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe56c15bd00 0x7fe56c15c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572d91700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56c06b2c0 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572d91700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56c06e0f0 con 0x7fe56c0cad10 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572d91700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56c06a6b0 con 0x7fe56c15bd00 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe57158e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe56c0cb0e0 0x7fe56c06d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38286/0 (socket says 172.21.15.64:38286) 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe57158e700 1 -- 172.21.15.64:0/480687828 learned_addr learned my addr 172.21.15.64:0/480687828 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572590700 1 --2- 172.21.15.64:0/480687828 >> v2:172.21.15.123:3300/0 conn(0x7fe56c15bd00 0x7fe56c15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660468552 0 0) 0x7fe56c06b2c0 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe554003570 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888615381 0 0) 0x7fe56c06e0f0 con 0x7fe56c0cad10 2023-09-27T23:13:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe56c06b2c0 con 0x7fe56c0cad10 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572590700 1 -- 172.21.15.64:0/480687828 >> v1:172.21.15.123:6789/0 conn(0x7fe56c0cad10 legacy=0x7fe56c06b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 131459671 0 0) 0x7fe554003570 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.782+0000 7fe572590700 1 -- 172.21.15.64:0/480687828 >> v1:172.21.15.64:6789/0 conn(0x7fe56c0cb0e0 legacy=0x7fe56c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572590700 1 -- 172.21.15.64:0/480687828 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c06a990 con 0x7fe56c15bd00 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572590700 1 --2- 172.21.15.64:0/480687828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c15c150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098e0 tx=0x7fe55c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ed92f8cce1da986 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560001470 con 0x7fe56c15bd00 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c011420 con 0x7fe56c15bd00 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe570d8d700 1 -- 172.21.15.64:0/480687828 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0117a0 con 0x7fe56c15bd00 2023-09-27T23:13:41.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 -- 172.21.15.64:0/480687828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 msgr2=0x7fe56c15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 --2- 172.21.15.64:0/480687828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c15c150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098e0 tx=0x7fe55c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 -- 172.21.15.64:0/480687828 shutdown_connections 2023-09-27T23:13:41.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 --2- 172.21.15.64:0/480687828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c15c150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 -- 172.21.15.64:0/480687828 >> 172.21.15.64:0/480687828 conn(0x7fe56c067bb0 msgr2=0x7fe56c0cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 -- 172.21.15.64:0/480687828 shutdown_connections 2023-09-27T23:13:41.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 -- 172.21.15.64:0/480687828 wait complete. 2023-09-27T23:13:41.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 Processor -- start 2023-09-27T23:13:41.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.786+0000 7fe572d91700 1 -- start start 2023-09-27T23:13:41.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c0cad10 0x7fe56c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 0x7fe56c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c0b5150 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c0b5430 con 0x7fe56c15bd00 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c0c6200 con 0x7fe56c0cad10 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 0x7fe56c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 0x7fe56c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49524/0 (socket says 172.21.15.64:49524) 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe571d8f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c0cad10 0x7fe56c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572590700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 -- 172.21.15.64:0/1278005098 learned_addr learned my addr 172.21.15.64:0/1278005098 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 -- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c0cad10 msgr2=0x7fe56c0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c0cad10 0x7fe56c0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 -- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 msgr2=0x7fe56c0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c0c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 -- 172.21.15.64:0/1278005098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009420 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe57158e700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 0x7fe56c0b4680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe56000cb60 tx=0x7fe560001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560007150 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5600072b0 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c0c64e0 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560007670 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.790+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c0c5af0 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.794+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c0c6720 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.794+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe560004a40 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.794+0000 7fe569ffb700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe558060950 0x7fe558062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:41.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.794+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe56007be20 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.794+0000 7fe571d8f700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe558060950 0x7fe558062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:41.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.798+0000 7fe571d8f700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe558060950 0x7fe558062e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5640040a0 tx=0x7fe564007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:41.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.802+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe56c0c6720 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.966+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe56c0c6720 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe569ffb700 1 -- 172.21.15.64:0/1278005098 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+13647 (secure 0 0 0) 0x7fe56c0c6720 con 0x7fe56c0cb0e0 2023-09-27T23:13:41.974 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:41.974 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:41.363261+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe558060950 msgr2=0x7fe558062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe558060950 0x7fe558062e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5640040a0 tx=0x7fe564007f90 comp rx=0 tx=0).stop 2023-09-27T23:13:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 msgr2=0x7fe56c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 0x7fe56c0b4680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe56000cb60 tx=0x7fe560001ad0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 shutdown_connections 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe558060950 0x7fe558062e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c15bd00 0x7fe56c0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56c0cb0e0 0x7fe56c0b4680 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 --2- 172.21.15.64:0/1278005098 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c0cad10 0x7fe56c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 >> 172.21.15.64:0/1278005098 conn(0x7fe56c067bb0 msgr2=0x7fe56c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 shutdown_connections 2023-09-27T23:13:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:41.970+0000 7fe572d91700 1 -- 172.21.15.64:0/1278005098 wait complete. 2023-09-27T23:13:41.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:13:41.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:13:42.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 Processor -- start 2023-09-27T23:13:42.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 -- start start 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6278068c50 0x7f62780b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62780b03f0 0x7f62780b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62780b15e0 con 0x7f62780b1d30 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627815a5d0 con 0x7f62780b03f0 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627f925700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62781595f0 con 0x7f6278068c50 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627d6c1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6278068c50 0x7f62780b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627cec0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62780b03f0 0x7f62780b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627dec2700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f62780b1d30 0x7f627815cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:53452/0 (socket says 172.21.15.64:53452) 2023-09-27T23:13:42.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.098+0000 7f627dec2700 1 -- 172.21.15.64:0/1362882013 learned_addr learned my addr 172.21.15.64:0/1362882013 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:42.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627cec0700 1 -- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6278068c50 msgr2=0x7f62780b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:42.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627cec0700 1 --2- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6278068c50 0x7f62780b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627cec0700 1 -- 172.21.15.64:0/1362882013 >> v1:172.21.15.123:6789/0 conn(0x7f62780b1d30 legacy=0x7f627815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627cec0700 1 -- 172.21.15.64:0/1362882013 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780b3c40 con 0x7f62780b03f0 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f626ffff700 1 -- 172.21.15.64:0/1362882013 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660341364 0 0) 0x7f62780b15e0 con 0x7f62780b1d30 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627d6c1700 1 --2- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6278068c50 0x7f62780b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627cec0700 1 --2- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f62780b17f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6274009910 tx=0x7f62740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6656be20785f481d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f626ffff700 1 -- 172.21.15.64:0/1362882013 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400b640 con 0x7f62780b03f0 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f626ffff700 1 -- 172.21.15.64:0/1362882013 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400b7a0 con 0x7f62780b03f0 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f626ffff700 1 -- 172.21.15.64:0/1362882013 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400bb60 con 0x7f62780b03f0 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 -- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 msgr2=0x7f62780b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 --2- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f62780b17f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6274009910 tx=0x7f62740091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 -- 172.21.15.64:0/1362882013 shutdown_connections 2023-09-27T23:13:42.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 --2- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f62780b17f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 --2- 172.21.15.64:0/1362882013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6278068c50 0x7f62780b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 -- 172.21.15.64:0/1362882013 >> 172.21.15.64:0/1362882013 conn(0x7f627806dce0 msgr2=0x7f62780ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:42.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 -- 172.21.15.64:0/1362882013 shutdown_connections 2023-09-27T23:13:42.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 -- 172.21.15.64:0/1362882013 wait complete. 2023-09-27T23:13:42.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 Processor -- start 2023-09-27T23:13:42.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.102+0000 7f627f925700 1 -- start start 2023-09-27T23:13:42.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 0x7f627815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f6278150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62780b1d30 0x7f6278169420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627815e4b0 con 0x7f6278068c50 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627815d8a0 con 0x7f62780b03f0 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627815db80 con 0x7f62780b1d30 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 0x7f627815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 0x7f627815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49536/0 (socket says 172.21.15.64:49536) 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 -- 172.21.15.64:0/552855094 learned_addr learned my addr 172.21.15.64:0/552855094 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:42.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627dec2700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62780b1d30 0x7f6278169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627cec0700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f6278150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 -- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62780b1d30 msgr2=0x7f6278169420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62780b1d30 0x7f6278169420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 -- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 msgr2=0x7f6278150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f6278150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 -- 172.21.15.64:0/552855094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268009420 con 0x7f6278068c50 2023-09-27T23:13:42.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627d6c1700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 0x7f627815b850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f626800ec80 tx=0x7f62680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800c850 con 0x7f6278068c50 2023-09-27T23:13:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626800c9b0 con 0x7f6278068c50 2023-09-27T23:13:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6274009420 con 0x7f6278068c50 2023-09-27T23:13:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800cd30 con 0x7f6278068c50 2023-09-27T23:13:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.106+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627815e140 con 0x7f6278068c50 2023-09-27T23:13:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.110+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627815be50 con 0x7f6278068c50 2023-09-27T23:13:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.110+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6268013680 con 0x7f6278068c50 2023-09-27T23:13:42.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.110+0000 7f626d7fa700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f625c060950 0x7f625c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.110+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f626807d080 con 0x7f6278068c50 2023-09-27T23:13:42.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.110+0000 7f627cec0700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f625c060950 0x7f625c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.110+0000 7f627cec0700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f625c060950 0x7f625c062e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6274009860 tx=0x7f6274017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.118+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f62680510e0 con 0x7f6278068c50 2023-09-27T23:13:42.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.282+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6278071120 con 0x7f6278068c50 2023-09-27T23:13:42.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.282+0000 7f626d7fa700 1 -- 172.21.15.64:0/552855094 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6268050cd0 con 0x7f6278068c50 2023-09-27T23:13:42.288 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:13:42.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f625c060950 msgr2=0x7f625c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f625c060950 0x7f625c062e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6274009860 tx=0x7f6274017000 comp rx=0 tx=0).stop 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 msgr2=0x7f627815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 0x7f627815b850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f626800ec80 tx=0x7f62680091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 shutdown_connections 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f625c060950 0x7f625c062e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62780b1d30 0x7f6278169420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62780b03f0 0x7f6278150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 --2- 172.21.15.64:0/552855094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278068c50 0x7f627815b850 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 >> 172.21.15.64:0/552855094 conn(0x7f627806dce0 msgr2=0x7f6278158780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 shutdown_connections 2023-09-27T23:13:42.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.286+0000 7f627f925700 1 -- 172.21.15.64:0/552855094 wait complete. 2023-09-27T23:13:42.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:13:42.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.410+0000 7fcb07209700 1 Processor -- start 2023-09-27T23:13:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.410+0000 7fcb07209700 1 -- start start 2023-09-27T23:13:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.410+0000 7fcb07209700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb0006d9f0 0x7fcb0006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.410+0000 7fcb07209700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0006b870 con 0x7fcb00158fe0 2023-09-27T23:13:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.410+0000 7fcb07209700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0006ac60 con 0x7fcb0006c9f0 2023-09-27T23:13:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.410+0000 7fcb07209700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0006af40 con 0x7fcb0006d9f0 2023-09-27T23:13:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb06a08700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcb00158fe0 0x7fcb000afd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38296/0 (socket says 172.21.15.64:38296) 2023-09-27T23:13:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb06a08700 1 -- 172.21.15.64:0/1576461258 learned_addr learned my addr 172.21.15.64:0/1576461258 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:42.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05a06700 1 --2- 172.21.15.64:0/1576461258 >> v2:172.21.15.123:3300/0 conn(0x7fcb0006d9f0 0x7fcb0006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181055284 0 0) 0x7fcb0006b870 con 0x7fcb00158fe0 2023-09-27T23:13:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaec003570 con 0x7fcb00158fe0 2023-09-27T23:13:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333698268 0 0) 0x7fcb0006ac60 con 0x7fcb0006c9f0 2023-09-27T23:13:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb0006b870 con 0x7fcb0006c9f0 2023-09-27T23:13:42.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05a06700 1 -- 172.21.15.64:0/1576461258 >> v1:172.21.15.123:6789/0 conn(0x7fcb0006c9f0 legacy=0x7fcb001588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05a06700 1 -- 172.21.15.64:0/1576461258 >> v1:172.21.15.64:6789/0 conn(0x7fcb00158fe0 legacy=0x7fcb000afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05a06700 1 -- 172.21.15.64:0/1576461258 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0006b220 con 0x7fcb0006d9f0 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05a06700 1 --2- 172.21.15.64:0/1576461258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb0006d9f0 0x7fcb0006cdc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40098e0 tx=0x7fcaf40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e930522b223257 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4011630 con 0x7fcb0006d9f0 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4011790 con 0x7fcb0006d9f0 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb05205700 1 -- 172.21.15.64:0/1576461258 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4011b30 con 0x7fcb0006d9f0 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 -- 172.21.15.64:0/1576461258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb0006d9f0 msgr2=0x7fcb0006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 --2- 172.21.15.64:0/1576461258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb0006d9f0 0x7fcb0006cdc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40098e0 tx=0x7fcaf40091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 -- 172.21.15.64:0/1576461258 shutdown_connections 2023-09-27T23:13:42.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 --2- 172.21.15.64:0/1576461258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb0006d9f0 0x7fcb0006cdc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 -- 172.21.15.64:0/1576461258 >> 172.21.15.64:0/1576461258 conn(0x7fcb00067bb0 msgr2=0x7fcb000cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:42.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 -- 172.21.15.64:0/1576461258 shutdown_connections 2023-09-27T23:13:42.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.414+0000 7fcb07209700 1 -- 172.21.15.64:0/1576461258 wait complete. 2023-09-27T23:13:42.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 Processor -- start 2023-09-27T23:13:42.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 -- start start 2023-09-27T23:13:42.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb0006c9f0 0x7fcb00149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 0x7fcb001575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb00158fe0 0x7fcb00157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b3040 con 0x7fcb0006d9f0 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b4ca0 con 0x7fcb00158fe0 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb07209700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b4f80 con 0x7fcb0006c9f0 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 0x7fcb001575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 0x7fcb001575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49548/0 (socket says 172.21.15.64:49548) 2023-09-27T23:13:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 -- 172.21.15.64:0/2676706261 learned_addr learned my addr 172.21.15.64:0/2676706261 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb06207700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb0006c9f0 0x7fcb00149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb06a08700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb00158fe0 0x7fcb00157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 -- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb0006c9f0 msgr2=0x7fcb00149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb0006c9f0 0x7fcb00149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 -- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb00158fe0 msgr2=0x7fcb00157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb00158fe0 0x7fcb00157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 -- 172.21.15.64:0/2676706261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4009420 con 0x7fcb0006d9f0 2023-09-27T23:13:42.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.418+0000 7fcb05a06700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 0x7fcb001575d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb001594d0 tx=0x7fcaf4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4000d10 con 0x7fcb0006d9f0 2023-09-27T23:13:42.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb000b5260 con 0x7fcb0006d9f0 2023-09-27T23:13:42.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4011900 con 0x7fcb0006d9f0 2023-09-27T23:13:42.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4025420 con 0x7fcb0006d9f0 2023-09-27T23:13:42.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb000b5820 con 0x7fcb0006d9f0 2023-09-27T23:13:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb000b5a10 con 0x7fcb0006d9f0 2023-09-27T23:13:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fcaf4023070 con 0x7fcb0006d9f0 2023-09-27T23:13:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcaf27fc700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcae4060950 0x7fcae4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcaf407e0a0 con 0x7fcb0006d9f0 2023-09-27T23:13:42.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.422+0000 7fcb06207700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcae4060950 0x7fcae4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.426+0000 7fcb06207700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcae4060950 0x7fcae4062e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007fd0 tx=0x7fcafc007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.430+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcaf404e5d0 con 0x7fcb0006d9f0 2023-09-27T23:13:42.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.590+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcb0004be40 con 0x7fcb0006d9f0 2023-09-27T23:13:42.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcaf27fc700 1 -- 172.21.15.64:0/2676706261 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcaf402a0a0 con 0x7fcb0006d9f0 2023-09-27T23:13:42.598 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:13:42.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcae4060950 msgr2=0x7fcae4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcae4060950 0x7fcae4062e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007fd0 tx=0x7fcafc007d30 comp rx=0 tx=0).stop 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 msgr2=0x7fcb001575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 0x7fcb001575d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb001594d0 tx=0x7fcaf4017000 comp rx=0 tx=0).stop 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 shutdown_connections 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcae4060950 0x7fcae4062e00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb00158fe0 0x7fcb00157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0006d9f0 0x7fcb001575d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 --2- 172.21.15.64:0/2676706261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb0006c9f0 0x7fcb00149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 >> 172.21.15.64:0/2676706261 conn(0x7fcb00067bb0 msgr2=0x7fcb0014a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:42.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 shutdown_connections 2023-09-27T23:13:42.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.594+0000 7fcb07209700 1 -- 172.21.15.64:0/2676706261 wait complete. 2023-09-27T23:13:42.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:13:42.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fedbf700 1 Processor -- start 2023-09-27T23:13:42.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fedbf700 1 -- start start 2023-09-27T23:13:42.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fedbf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f8155a90 0x7f29f80cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fedbf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f8155ea0 con 0x7f29f80cc360 2023-09-27T23:13:42.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fedbf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f80cc7c0 con 0x7f29f81556c0 2023-09-27T23:13:42.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fedbf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f80cb640 con 0x7f29f8155a90 2023-09-27T23:13:42.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fe5be700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f29f80cc360 0x7f29f80c84e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38306/0 (socket says 172.21.15.64:38306) 2023-09-27T23:13:42.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fe5be700 1 -- 172.21.15.64:0/3766935960 learned_addr learned my addr 172.21.15.64:0/3766935960 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:42.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fd5bc700 1 --2- 172.21.15.64:0/3766935960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f8155a90 0x7f29f80cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578432921 0 0) 0x7f29f8155ea0 con 0x7f29f80cc360 2023-09-27T23:13:42.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29e4003570 con 0x7f29f80cc360 2023-09-27T23:13:42.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485309824 0 0) 0x7f29f0001490 con 0x7f29f81556c0 2023-09-27T23:13:42.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f8155ea0 con 0x7f29f81556c0 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.714+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813929586 0 0) 0x7f29e4003570 con 0x7f29f80cc360 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fd5bc700 1 -- 172.21.15.64:0/3766935960 >> v1:172.21.15.123:6789/0 conn(0x7f29f81556c0 legacy=0x7f29f80c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fd5bc700 1 -- 172.21.15.64:0/3766935960 >> v1:172.21.15.64:6789/0 conn(0x7f29f80cc360 legacy=0x7f29f80c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fd5bc700 1 -- 172.21.15.64:0/3766935960 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f80c2930 con 0x7f29f8155a90 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fd5bc700 1 --2- 172.21.15.64:0/3766935960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f8155a90 0x7f29f80cbe20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009910 tx=0x7f29e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3faed087dd3ea3ef server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4001470 con 0x7f29f8155a90 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8011420 con 0x7f29f8155a90 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fcdbb700 1 -- 172.21.15.64:0/3766935960 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e80117c0 con 0x7f29f8155a90 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 -- 172.21.15.64:0/3766935960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f8155a90 msgr2=0x7f29f80cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3766935960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f8155a90 0x7f29f80cbe20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009910 tx=0x7f29e80091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 -- 172.21.15.64:0/3766935960 shutdown_connections 2023-09-27T23:13:42.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3766935960 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f8155a90 0x7f29f80cbe20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 -- 172.21.15.64:0/3766935960 >> 172.21.15.64:0/3766935960 conn(0x7f29f806b670 msgr2=0x7f29f806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:42.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 -- 172.21.15.64:0/3766935960 shutdown_connections 2023-09-27T23:13:42.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 -- 172.21.15.64:0/3766935960 wait complete. 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 Processor -- start 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.718+0000 7f29fedbf700 1 -- start start 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f80cc360 0x7f29f8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f81556c0 0x7f29f815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 0x7f29f8142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f8167430 con 0x7f29f8155a90 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f8168290 con 0x7f29f80cc360 2023-09-27T23:13:42.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f8167680 con 0x7f29f81556c0 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 0x7f29f8142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 0x7f29f8142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49560/0 (socket says 172.21.15.64:49560) 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 -- 172.21.15.64:0/3075622342 learned_addr learned my addr 172.21.15.64:0/3075622342 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fd5bc700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f81556c0 0x7f29f815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fddbd700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f80cc360 0x7f29f8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 -- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f81556c0 msgr2=0x7f29f815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f81556c0 0x7f29f815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 -- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f80cc360 msgr2=0x7f29f8158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f80cc360 0x7f29f8158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 -- 172.21.15.64:0/3075622342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8009420 con 0x7f29f8155a90 2023-09-27T23:13:42.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fe5be700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 0x7f29f8142050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29f400cba0 tx=0x7f29f4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4007150 con 0x7f29f8155a90 2023-09-27T23:13:42.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40072d0 con 0x7f29f8155a90 2023-09-27T23:13:42.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f8167960 con 0x7f29f8155a90 2023-09-27T23:13:42.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4007690 con 0x7f29f8155a90 2023-09-27T23:13:42.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f8167f20 con 0x7f29f8155a90 2023-09-27T23:13:42.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.722+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f8168420 con 0x7f29f8155a90 2023-09-27T23:13:42.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.726+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f29f4004a40 con 0x7f29f8155a90 2023-09-27T23:13:42.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.726+0000 7f29edffb700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29dc060950 0x7f29dc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.726+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f29f407c1e0 con 0x7f29f8155a90 2023-09-27T23:13:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.726+0000 7f29fddbd700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29dc060950 0x7f29dc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:42.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.726+0000 7f29fddbd700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29dc060950 0x7f29dc062e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29f0007fd0 tx=0x7f29f0007f40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:42.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.730+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f29f404c810 con 0x7f29f8155a90 2023-09-27T23:13:42.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.894+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f29f804be40 con 0x7f29f8155a90 2023-09-27T23:13:42.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.894+0000 7f29edffb700 1 -- 172.21.15.64:0/3075622342 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v21) v1 ==== 84+0+126 (secure 0 0 0) 0x7f29f404fe30 con 0x7f29f8155a90 2023-09-27T23:13:42.899 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:13:42.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:13:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:13:42.901 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:13:42.901 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:13:42.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29dc060950 msgr2=0x7f29dc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29dc060950 0x7f29dc062e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29f0007fd0 tx=0x7f29f0007f40 comp rx=0 tx=0).stop 2023-09-27T23:13:42.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 msgr2=0x7f29f8142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:42.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 0x7f29f8142050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29f400cba0 tx=0x7f29f4001ad0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 shutdown_connections 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29dc060950 0x7f29dc062e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29f8155a90 0x7f29f8142050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29f81556c0 0x7f29f815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 --2- 172.21.15.64:0/3075622342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f80cc360 0x7f29f8158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 >> 172.21.15.64:0/3075622342 conn(0x7f29f806b670 msgr2=0x7f29f8062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 shutdown_connections 2023-09-27T23:13:42.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:42.898+0000 7f29fedbf700 1 -- 172.21.15.64:0/3075622342 wait complete. 2023-09-27T23:13:42.914 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:13:42.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:13:43.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.022+0000 7fdf20e81700 1 Processor -- start 2023-09-27T23:13:43.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf20e81700 1 -- start start 2023-09-27T23:13:43.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf20e81700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0af190 0x7fdf1c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf20e81700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c0cb3f0 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf20e81700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c06d220 con 0x7fdf1c0cac50 2023-09-27T23:13:43.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf20e81700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0cc140 con 0x7fdf1c0af190 2023-09-27T23:13:43.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdf1c0aedc0 0x7fdf1c06d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38312/0 (socket says 172.21.15.64:38312) 2023-09-27T23:13:43.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1b7fe700 1 -- 172.21.15.64:0/3408350425 learned_addr learned my addr 172.21.15.64:0/3408350425 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:43.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1affd700 1 --2- 172.21.15.64:0/3408350425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0af190 0x7fdf1c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357084342 0 0) 0x7fdf1c0cb3f0 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf04003570 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080703273 0 0) 0x7fdf1c06d220 con 0x7fdf1c0cac50 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf1c0cb3f0 con 0x7fdf1c0cac50 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1affd700 1 -- 172.21.15.64:0/3408350425 >> v1:172.21.15.123:6789/0 conn(0x7fdf1c0cac50 legacy=0x7fdf1c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1affd700 1 -- 172.21.15.64:0/3408350425 >> v1:172.21.15.64:6789/0 conn(0x7fdf1c0aedc0 legacy=0x7fdf1c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1affd700 1 -- 172.21.15.64:0/3408350425 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c06c8f0 con 0x7fdf1c0af190 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1affd700 1 --2- 172.21.15.64:0/3408350425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0af190 0x7fdf1c06af50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009910 tx=0x7fdf100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3beb3777af613173 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10011630 con 0x7fdf1c0af190 2023-09-27T23:13:43.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf10011790 con 0x7fdf1c0af190 2023-09-27T23:13:43.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.026+0000 7fdf1a7fc700 1 -- 172.21.15.64:0/3408350425 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10011b30 con 0x7fdf1c0af190 2023-09-27T23:13:43.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 -- 172.21.15.64:0/3408350425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0af190 msgr2=0x7fdf1c06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 --2- 172.21.15.64:0/3408350425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0af190 0x7fdf1c06af50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009910 tx=0x7fdf100091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 -- 172.21.15.64:0/3408350425 shutdown_connections 2023-09-27T23:13:43.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 --2- 172.21.15.64:0/3408350425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0af190 0x7fdf1c06af50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 -- 172.21.15.64:0/3408350425 >> 172.21.15.64:0/3408350425 conn(0x7fdf1c0cc760 msgr2=0x7fdf1c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 -- 172.21.15.64:0/3408350425 shutdown_connections 2023-09-27T23:13:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 -- 172.21.15.64:0/3408350425 wait complete. 2023-09-27T23:13:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 Processor -- start 2023-09-27T23:13:43.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.030+0000 7fdf20e81700 1 -- start start 2023-09-27T23:13:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 0x7fdf1c15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1c0af190 0x7fdf1c0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0cac50 0x7fdf1c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0b4410 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c169d60 con 0x7fdf1c0af190 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0b4660 con 0x7fdf1c0cac50 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 0x7fdf1c15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 0x7fdf1c15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49574/0 (socket says 172.21.15.64:49574) 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 -- 172.21.15.64:0/1554874569 learned_addr learned my addr 172.21.15.64:0/1554874569 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1bfff700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0cac50 0x7fdf1c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1affd700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1c0af190 0x7fdf1c0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 -- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0cac50 msgr2=0x7fdf1c0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0cac50 0x7fdf1c0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 -- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1c0af190 msgr2=0x7fdf1c0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1c0af190 0x7fdf1c0c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 -- 172.21.15.64:0/1554874569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10009420 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf1b7fe700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 0x7fdf1c15b120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c790 tx=0x7fdf0c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c005b30 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c005c90 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c0b4940 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c007300 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.034+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c15b660 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.038+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c0b4e40 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.038+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fdf0c01b3b0 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.038+0000 7fdf037fe700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf08060950 0x7fdf08062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.038+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdf0c07bfb0 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.038+0000 7fdf1affd700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf08060950 0x7fdf08062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.038+0000 7fdf1affd700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf08060950 0x7fdf08062e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009860 tx=0x7fdf10017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:43.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.046+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdf0c050010 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.214+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdf1c04be40 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.390+0000 7fdf037fe700 1 -- 172.21.15.64:0/1554874569 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v22) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdf0c017020 con 0x7fdf1c0aedc0 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf08060950 msgr2=0x7fdf08062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf08060950 0x7fdf08062e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009860 tx=0x7fdf10017000 comp rx=0 tx=0).stop 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 msgr2=0x7fdf1c15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 0x7fdf1c15b120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c790 tx=0x7fdf0c007ea0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 shutdown_connections 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf08060950 0x7fdf08062e00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf1c0cac50 0x7fdf1c0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1c0af190 0x7fdf1c0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 --2- 172.21.15.64:0/1554874569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf1c0aedc0 0x7fdf1c15b120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 >> 172.21.15.64:0/1554874569 conn(0x7fdf1c0cc760 msgr2=0x7fdf1c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 shutdown_connections 2023-09-27T23:13:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.398+0000 7fdf20e81700 1 -- 172.21.15.64:0/1554874569 wait complete. 2023-09-27T23:13:43.405 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:13:43.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-27T23:13:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 Processor -- start 2023-09-27T23:13:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 -- start start 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb041514a0 0x7fdb04157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb0406b7a0 0x7fdb04158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb040cbee0 con 0x7fdb040cae60 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb04158530 con 0x7fdb0406b7a0 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb08fd6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb040cc370 con 0x7fdb041514a0 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb02d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb0406b7a0 0x7fdb04158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb02d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb0406b7a0 0x7fdb04158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49576/0 (socket says 172.21.15.64:49576) 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.526+0000 7fdb02d9d700 1 -- 172.21.15.64:0/100217390 learned_addr learned my addr 172.21.15.64:0/100217390 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:43.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb0259c700 1 --2- 172.21.15.64:0/100217390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb041514a0 0x7fdb04157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb02d9d700 1 -- 172.21.15.64:0/100217390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb041514a0 msgr2=0x7fdb04157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:43.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb02d9d700 1 --2- 172.21.15.64:0/100217390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb041514a0 0x7fdb04157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb02d9d700 1 -- 172.21.15.64:0/100217390 >> v1:172.21.15.123:6789/0 conn(0x7fdb040cae60 legacy=0x7fdb0406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb02d9d700 1 -- 172.21.15.64:0/100217390 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb040cb2d0 con 0x7fdb0406b7a0 2023-09-27T23:13:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb02d9d700 1 --2- 172.21.15.64:0/100217390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb04158740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009f20 tx=0x7fdafc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548f74835a1855d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb0159a700 1 -- 172.21.15.64:0/100217390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00e070 con 0x7fdb0406b7a0 2023-09-27T23:13:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb0159a700 1 -- 172.21.15.64:0/100217390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc00b9d0 con 0x7fdb0406b7a0 2023-09-27T23:13:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb0159a700 1 -- 172.21.15.64:0/100217390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0092e0 con 0x7fdb0406b7a0 2023-09-27T23:13:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 -- 172.21.15.64:0/100217390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 msgr2=0x7fdb04158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/100217390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb04158740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009f20 tx=0x7fdafc009840 comp rx=0 tx=0).stop 2023-09-27T23:13:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 -- 172.21.15.64:0/100217390 shutdown_connections 2023-09-27T23:13:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/100217390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb04158740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/100217390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb041514a0 0x7fdb04157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 -- 172.21.15.64:0/100217390 >> 172.21.15.64:0/100217390 conn(0x7fdb040cc580 msgr2=0x7fdb040cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:43.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 -- 172.21.15.64:0/100217390 shutdown_connections 2023-09-27T23:13:43.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.530+0000 7fdb08fd6700 1 -- 172.21.15.64:0/100217390 wait complete. 2023-09-27T23:13:43.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 Processor -- start 2023-09-27T23:13:43.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 -- start start 2023-09-27T23:13:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb0414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb040cae60 0x7fdb0414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb041514a0 0x7fdb040b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0414ca00 con 0x7fdb0406b7a0 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0415a810 con 0x7fdb041514a0 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0416a880 con 0x7fdb040cae60 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb0414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb0414cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49586/0 (socket says 172.21.15.64:49586) 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 -- 172.21.15.64:0/1609076553 learned_addr learned my addr 172.21.15.64:0/1609076553 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb01d9b700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb040cae60 0x7fdb0414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb02d9d700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb041514a0 0x7fdb040b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 -- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb040cae60 msgr2=0x7fdb0414c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb040cae60 0x7fdb0414c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 -- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb041514a0 msgr2=0x7fdb040b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb041514a0 0x7fdb040b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 -- 172.21.15.64:0/1609076553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009ae0 con 0x7fdb0406b7a0 2023-09-27T23:13:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb02d9d700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb041514a0 0x7fdb040b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb0259c700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb0414cc20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400e930 tx=0x7fdaf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4013680 con 0x7fdb0406b7a0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf40137e0 con 0x7fdb0406b7a0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0416ab60 con 0x7fdb0406b7a0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4013b80 con 0x7fdb0406b7a0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.534+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0416b120 con 0x7fdb0406b7a0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.538+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb04071120 con 0x7fdb0406b7a0 2023-09-27T23:13:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.538+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fdaf400c460 con 0x7fdb0406b7a0 2023-09-27T23:13:43.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.538+0000 7fdafa7fc700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdae4060950 0x7fdae4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:43.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.538+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fdaf407cdd0 con 0x7fdb0406b7a0 2023-09-27T23:13:43.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.538+0000 7fdb01d9b700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdae4060950 0x7fdae4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:43.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.538+0000 7fdb01d9b700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdae4060950 0x7fdae4062e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb04151a90 tx=0x7fdaec007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:43.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.546+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdaf404d230 con 0x7fdb0406b7a0 2023-09-27T23:13:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:43.710+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fdb040cb5b0 con 0x7fdb0406b7a0 2023-09-27T23:13:44.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.398+0000 7fdafa7fc700 1 -- 172.21.15.64:0/1609076553 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v23) v1 ==== 171+0+0 (secure 0 0 0) 0x7fdaf4050850 con 0x7fdb0406b7a0 2023-09-27T23:13:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.414+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdae4060950 msgr2=0x7fdae4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.414+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdae4060950 0x7fdae4062e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb04151a90 tx=0x7fdaec007ec0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.414+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 msgr2=0x7fdb0414cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.414+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb0414cc20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400e930 tx=0x7fdaf40091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 shutdown_connections 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdae4060950 0x7fdae4062e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb041514a0 0x7fdb040b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb040cae60 0x7fdb0414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 --2- 172.21.15.64:0/1609076553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0406b7a0 0x7fdb0414cc20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 >> 172.21.15.64:0/1609076553 conn(0x7fdb040cc580 msgr2=0x7fdb040cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 shutdown_connections 2023-09-27T23:13:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.418+0000 7fdb08fd6700 1 -- 172.21.15.64:0/1609076553 wait complete. 2023-09-27T23:13:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:13:44.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:13:44.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.538+0000 7fb87b80f700 1 Processor -- start 2023-09-27T23:13:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.538+0000 7fb87b80f700 1 -- start start 2023-09-27T23:13:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8740cecd0 0x7fb87406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740cbdf0 con 0x7fb874158e20 2023-09-27T23:13:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740cc0d0 con 0x7fb8740cb970 2023-09-27T23:13:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8740cc3b0 con 0x7fb8740cecd0 2023-09-27T23:13:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb8795ab700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb874158e20 0x7fb8740ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38314/0 (socket says 172.21.15.64:38314) 2023-09-27T23:13:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb8795ab700 1 -- 172.21.15.64:0/3092954876 learned_addr learned my addr 172.21.15.64:0/3092954876 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb879dac700 1 --2- 172.21.15.64:0/3092954876 >> v2:172.21.15.64:3300/0 conn(0x7fb8740cecd0 0x7fb87406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb873fff700 1 -- 172.21.15.64:0/3092954876 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256034365 0 0) 0x7fb8740cbdf0 con 0x7fb874158e20 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb873fff700 1 -- 172.21.15.64:0/3092954876 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb85c003570 con 0x7fb874158e20 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb879dac700 1 -- 172.21.15.64:0/3092954876 >> v1:172.21.15.123:6789/0 conn(0x7fb8740cb970 legacy=0x7fb87406acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb879dac700 1 -- 172.21.15.64:0/3092954876 >> v1:172.21.15.64:6789/0 conn(0x7fb874158e20 legacy=0x7fb8740ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb879dac700 1 -- 172.21.15.64:0/3092954876 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740cc690 con 0x7fb8740cecd0 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb879dac700 1 --2- 172.21.15.64:0/3092954876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cecd0 0x7fb87406b3d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e90 tx=0x7fb86c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c07ac32a791081 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb873fff700 1 -- 172.21.15.64:0/3092954876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00e070 con 0x7fb8740cecd0 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb873fff700 1 -- 172.21.15.64:0/3092954876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00bc40 con 0x7fb8740cecd0 2023-09-27T23:13:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb873fff700 1 -- 172.21.15.64:0/3092954876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c012430 con 0x7fb8740cecd0 2023-09-27T23:13:44.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- 172.21.15.64:0/3092954876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cecd0 msgr2=0x7fb87406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 --2- 172.21.15.64:0/3092954876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cecd0 0x7fb87406b3d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e90 tx=0x7fb86c009780 comp rx=0 tx=0).stop 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- 172.21.15.64:0/3092954876 shutdown_connections 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 --2- 172.21.15.64:0/3092954876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cecd0 0x7fb87406b3d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- 172.21.15.64:0/3092954876 >> 172.21.15.64:0/3092954876 conn(0x7fb874067bb0 msgr2=0x7fb8740d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- 172.21.15.64:0/3092954876 shutdown_connections 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.542+0000 7fb87b80f700 1 -- 172.21.15.64:0/3092954876 wait complete. 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 Processor -- start 2023-09-27T23:13:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 -- start start 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8740cecd0 0x7fb8740b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb874158e20 0x7fb8740ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 0x7fb87414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87414f800 con 0x7fb8740cd0f0 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740c7630 con 0x7fb8740cecd0 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb87b80f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87414f240 con 0x7fb874158e20 2023-09-27T23:13:44.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 0x7fb87414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 0x7fb87414c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49598/0 (socket says 172.21.15.64:49598) 2023-09-27T23:13:44.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 -- 172.21.15.64:0/2321531001 learned_addr learned my addr 172.21.15.64:0/2321531001 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:44.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb8795ab700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8740cecd0 0x7fb8740b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb878daa700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb874158e20 0x7fb8740ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 -- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb874158e20 msgr2=0x7fb8740ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb874158e20 0x7fb8740ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 -- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8740cecd0 msgr2=0x7fb8740b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8740cecd0 0x7fb8740b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 -- 172.21.15.64:0/2321531001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c009a20 con 0x7fb8740cd0f0 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.546+0000 7fb879dac700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 0x7fb87414c7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb874069a80 tx=0x7fb86c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c026480 con 0x7fb8740cd0f0 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c026600 con 0x7fb8740cd0f0 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87414f520 con 0x7fb8740cd0f0 2023-09-27T23:13:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0269c0 con 0x7fb8740cd0f0 2023-09-27T23:13:44.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8740b5c90 con 0x7fb8740cd0f0 2023-09-27T23:13:44.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb874071120 con 0x7fb8740cd0f0 2023-09-27T23:13:44.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb86c01c400 con 0x7fb8740cd0f0 2023-09-27T23:13:44.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8717fa700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb860060950 0x7fb860062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fb86c07edd0 con 0x7fb8740cd0f0 2023-09-27T23:13:44.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.550+0000 7fb8795ab700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb860060950 0x7fb860062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.554+0000 7fb8795ab700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb860060950 0x7fb860062e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb864007fd0 tx=0x7fb864009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:44.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.558+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb86c04f0b0 con 0x7fb8740cd0f0 2023-09-27T23:13:44.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.754+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb8740c7770 con 0x7fb8740cd0f0 2023-09-27T23:13:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.778+0000 7fb8717fa700 1 -- 172.21.15.64:0/2321531001 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb86c02b0c0 con 0x7fb8740cd0f0 2023-09-27T23:13:44.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb860060950 msgr2=0x7fb860062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb860060950 0x7fb860062e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb864007fd0 tx=0x7fb864009040 comp rx=0 tx=0).stop 2023-09-27T23:13:44.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 msgr2=0x7fb87414c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 0x7fb87414c7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb874069a80 tx=0x7fb86c019000 comp rx=0 tx=0).stop 2023-09-27T23:13:44.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 shutdown_connections 2023-09-27T23:13:44.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb860060950 0x7fb860062e00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8740cd0f0 0x7fb87414c7e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb874158e20 0x7fb8740ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 --2- 172.21.15.64:0/2321531001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8740cecd0 0x7fb8740b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.786+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 >> 172.21.15.64:0/2321531001 conn(0x7fb874067bb0 msgr2=0x7fb8740c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:44.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.790+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 shutdown_connections 2023-09-27T23:13:44.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.790+0000 7fb87b80f700 1 -- 172.21.15.64:0/2321531001 wait complete. 2023-09-27T23:13:44.798 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 3 and data pool 4 2023-09-27T23:13:44.811 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-27T23:13:44.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-27T23:13:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.910+0000 7f0537552700 1 Processor -- start 2023-09-27T23:13:44.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.910+0000 7f0537552700 1 -- start start 2023-09-27T23:13:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0530156b90 0x7f05300ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300c7e70 con 0x7f0530156f60 2023-09-27T23:13:44.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0534aed700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0530156f60 0x7f05300bfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48706/0 (socket says 172.21.15.64:48706) 2023-09-27T23:13:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0534aed700 1 -- 172.21.15.64:0/1532178196 learned_addr learned my addr 172.21.15.64:0/1532178196 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300c72a0 con 0x7f05300ae7c0 2023-09-27T23:13:44.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300c7540 con 0x7f0530156b90 2023-09-27T23:13:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611456216 0 0) 0x7f05300c7e70 con 0x7f0530156f60 2023-09-27T23:13:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f05352ee700 1 --2- 172.21.15.64:0/1532178196 >> v2:172.21.15.123:3300/0 conn(0x7f0530156b90 0x7f05300ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c003570 con 0x7f0530156f60 2023-09-27T23:13:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1286761856 0 0) 0x7f051c003570 con 0x7f0530156f60 2023-09-27T23:13:44.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 >> v2:172.21.15.123:3300/0 conn(0x7f0530156b90 msgr2=0x7f05300ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 --2- 172.21.15.64:0/1532178196 >> v2:172.21.15.123:3300/0 conn(0x7f0530156b90 0x7f05300ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 >> v1:172.21.15.123:6789/0 conn(0x7f05300ae7c0 legacy=0x7f05300c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300c7820 con 0x7f0530156f60 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f0520001470 con 0x7f0530156f60 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 >> v1:172.21.15.64:6789/0 conn(0x7f0530156f60 legacy=0x7f05300bfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f051c003a60 0x7f051c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f051c004810 0x7f051c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f051c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300c7540 con 0x7f051c0051c0 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00c0d0 con 0x7f051c003a60 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00c290 con 0x7f051c004810 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f05352ee700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f051c003a60 0x7f051c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f051c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0534aed700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f051c004810 0x7f051c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 -- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f051c004810 msgr2=0x7f051c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f051c004810 0x7f051c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 -- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f051c003a60 msgr2=0x7f051c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f051c003a60 0x7f051c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 -- 172.21.15.64:0/1532178196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c003570 con 0x7f051c0051c0 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0534aed700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f051c004810 0x7f051c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:44.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f05352ee700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f051c003a60 0x7f051c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 -- 172.21.15.64:0/1532178196 reap_dead start 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0535aef700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f051c00ba60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0528001fc0 tx=0x7f0528001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5261e6abeb4b98 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528001950 con 0x7f051c0051c0 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052800c8b0 con 0x7f051c0051c0 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0527fff700 1 -- 172.21.15.64:0/1532178196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800cc90 con 0x7f051c0051c0 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 msgr2=0x7f051c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f051c00ba60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0528001fc0 tx=0x7f0528001640 comp rx=0 tx=0).stop 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 shutdown_connections 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 --2- 172.21.15.64:0/1532178196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f051c00ba60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.914+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 >> 172.21.15.64:0/1532178196 conn(0x7f05300c8db0 msgr2=0x7f05300c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 shutdown_connections 2023-09-27T23:13:44.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 -- 172.21.15.64:0/1532178196 wait complete. 2023-09-27T23:13:44.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 Processor -- start 2023-09-27T23:13:44.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 -- start start 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f05301538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05300c56f0 0x7f05300b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05300b06d0 0x7f05300c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b0b40 con 0x7f051c0051c0 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530153de0 con 0x7f05300c56f0 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0537552700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0530166c30 con 0x7f05300b06d0 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f05352ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f05301538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f05352ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f05301538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56820/0 (socket says 172.21.15.64:56820) 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f05352ee700 1 -- 172.21.15.64:0/4244953441 learned_addr learned my addr 172.21.15.64:0/4244953441 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:44.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0535aef700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05300b06d0 0x7f05300c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.918+0000 7f0534aed700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05300c56f0 0x7f05300b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05352ee700 1 -- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05300b06d0 msgr2=0x7f05300c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05352ee700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05300b06d0 0x7f05300c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05352ee700 1 -- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05300c56f0 msgr2=0x7f05300b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05352ee700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05300c56f0 0x7f05300b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05352ee700 1 -- 172.21.15.64:0/4244953441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530166f10 con 0x7f051c0051c0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05352ee700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f05301538a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f052c011d70 tx=0x7f052c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c011e70 con 0x7f051c0051c0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c00fcd0 con 0x7f051c0051c0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05301671f0 con 0x7f051c0051c0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c014670 con 0x7f051c0051c0 2023-09-27T23:13:44.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05301677b0 con 0x7f051c0051c0 2023-09-27T23:13:44.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0530071120 con 0x7f051c0051c0 2023-09-27T23:13:44.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.922+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f052c014a20 con 0x7f051c0051c0 2023-09-27T23:13:44.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.926+0000 7f05257fa700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0514060980 0x7f0514062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:44.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.926+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f052c018070 con 0x7f051c0051c0 2023-09-27T23:13:44.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.926+0000 7f0534aed700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0514060980 0x7f0514062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:44.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.926+0000 7f0534aed700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0514060980 0x7f0514062e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05300c5220 tx=0x7f05300c7de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:44.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:44.934+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f052c04dc40 con 0x7f051c0051c0 2023-09-27T23:13:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.102+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f05300c7b00 con 0x7f051c0051c0 2023-09-27T23:13:45.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.770+0000 7f05257fa700 1 -- 172.21.15.64:0/4244953441 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v25) v1 ==== 157+0+0 (secure 0 0 0) 0x7f052c051260 con 0x7f051c0051c0 2023-09-27T23:13:45.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0514060980 msgr2=0x7f0514062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:45.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0514060980 0x7f0514062e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05300c5220 tx=0x7f05300c7de0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 msgr2=0x7f05301538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f05301538a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f052c011d70 tx=0x7f052c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 shutdown_connections 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0514060980 0x7f0514062e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05300b06d0 0x7f05300c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05300c56f0 0x7f05300b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 --2- 172.21.15.64:0/4244953441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f051c0051c0 0x7f05301538a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 >> 172.21.15.64:0/4244953441 conn(0x7f05300c8db0 msgr2=0x7f05300b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 shutdown_connections 2023-09-27T23:13:45.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.778+0000 7f0537552700 1 -- 172.21.15.64:0/4244953441 wait complete. 2023-09-27T23:13:45.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:13:45.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 Processor -- start 2023-09-27T23:13:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 -- start start 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4124150200 0x7f41240c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41240ace30 0x7f41240aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240af250 con 0x7f41240ae770 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41240af530 con 0x7f41240ace30 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f412359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240ca8a0 con 0x7f4124150200 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f4122d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41240ace30 0x7f41240aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f4122d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41240ace30 0x7f41240aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56822/0 (socket says 172.21.15.64:56822) 2023-09-27T23:13:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.918+0000 7f4122d9d700 1 -- 172.21.15.64:0/1809396165 learned_addr learned my addr 172.21.15.64:0/1809396165 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:45.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412259c700 1 --2- 172.21.15.64:0/1809396165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4124150200 0x7f41240c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:45.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4122d9d700 1 -- 172.21.15.64:0/1809396165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4124150200 msgr2=0x7f41240c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:45.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4122d9d700 1 --2- 172.21.15.64:0/1809396165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4124150200 0x7f41240c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4122d9d700 1 -- 172.21.15.64:0/1809396165 >> v1:172.21.15.123:6789/0 conn(0x7f41240ae770 legacy=0x7f41240ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4122d9d700 1 -- 172.21.15.64:0/1809396165 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41240cab80 con 0x7f41240ace30 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4121d9b700 1 -- 172.21.15.64:0/1809396165 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423481387 0 0) 0x7f41240af250 con 0x7f41240ae770 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4122d9d700 1 --2- 172.21.15.64:0/1809396165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41240ace30 0x7f41240aebe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f411c009f20 tx=0x7f411c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb03c2d99bf6bfa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4121d9b700 1 -- 172.21.15.64:0/1809396165 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00e070 con 0x7f41240ace30 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4121d9b700 1 -- 172.21.15.64:0/1809396165 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00b9d0 con 0x7f41240ace30 2023-09-27T23:13:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f4121d9b700 1 -- 172.21.15.64:0/1809396165 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0092e0 con 0x7f41240ace30 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 -- 172.21.15.64:0/1809396165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41240ace30 msgr2=0x7f41240aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 --2- 172.21.15.64:0/1809396165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41240ace30 0x7f41240aebe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f411c009f20 tx=0x7f411c009840 comp rx=0 tx=0).stop 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 -- 172.21.15.64:0/1809396165 shutdown_connections 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 --2- 172.21.15.64:0/1809396165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41240ace30 0x7f41240aebe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 --2- 172.21.15.64:0/1809396165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4124150200 0x7f41240c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 -- 172.21.15.64:0/1809396165 >> 172.21.15.64:0/1809396165 conn(0x7f41240ccb30 msgr2=0x7f41240ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 -- 172.21.15.64:0/1809396165 shutdown_connections 2023-09-27T23:13:45.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.922+0000 7f412359e700 1 -- 172.21.15.64:0/1809396165 wait complete. 2023-09-27T23:13:45.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 Processor -- start 2023-09-27T23:13:45.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 -- start start 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41240ace30 0x7f41240c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41240ae770 0x7f4124069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 0x7f4124149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412406a090 con 0x7f4124150200 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4124149ff0 con 0x7f41240ace30 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240c4eb0 con 0x7f41240ae770 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 0x7f4124149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 0x7f4124149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56830/0 (socket says 172.21.15.64:56830) 2023-09-27T23:13:45.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 -- 172.21.15.64:0/4078506693 learned_addr learned my addr 172.21.15.64:0/4078506693 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:45.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412259c700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41240ace30 0x7f41240c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:45.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f411bfff700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41240ae770 0x7f4124069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 -- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41240ae770 msgr2=0x7f4124069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41240ae770 0x7f4124069b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 -- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41240ace30 msgr2=0x7f41240c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41240ace30 0x7f41240c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 -- 172.21.15.64:0/4078506693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009ae0 con 0x7f4124150200 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f4122d9d700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 0x7f4124149ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f412406b3a0 tx=0x7f411c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c008ad0 con 0x7f4124150200 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c008c30 con 0x7f4124150200 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41240c42a0 con 0x7f4124150200 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c026460 con 0x7f4124150200 2023-09-27T23:13:45.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.926+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41240c4860 con 0x7f4124150200 2023-09-27T23:13:45.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.930+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412404be40 con 0x7f4124150200 2023-09-27T23:13:45.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.930+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f411c003040 con 0x7f4124150200 2023-09-27T23:13:45.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.930+0000 7f411a7fc700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4104060980 0x7f4104062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:45.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.930+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f411c07ed20 con 0x7f4124150200 2023-09-27T23:13:45.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.930+0000 7f412259c700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4104060980 0x7f4104062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:45.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.930+0000 7f412259c700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4104060980 0x7f4104062e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4114009d10 tx=0x7f41140091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:45.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:45.934+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f411c04eed0 con 0x7f4124150200 2023-09-27T23:13:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:46.098+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f41240c4ff0 con 0x7f4124150200 2023-09-27T23:13:46.490 INFO:tasks.ceph.mds.d.smithi123.stderr:2023-09-27T23:13:46.485+0000 7feda7fdc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:13:47.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:13:47.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:13:47.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.794+0000 7f411a7fc700 1 -- 172.21.15.64:0/4078506693 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v27) v1 ==== 228+0+0 (secure 0 0 0) 0x7f411c0524f0 con 0x7f4124150200 2023-09-27T23:13:47.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4104060980 msgr2=0x7f4104062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:47.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4104060980 0x7f4104062e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4114009d10 tx=0x7f41140091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 msgr2=0x7f4124149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 0x7f4124149ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f412406b3a0 tx=0x7f411c018000 comp rx=0 tx=0).stop 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 shutdown_connections 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4104060980 0x7f4104062e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124150200 0x7f4124149ab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41240ae770 0x7f4124069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 --2- 172.21.15.64:0/4078506693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41240ace30 0x7f41240c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 >> 172.21.15.64:0/4078506693 conn(0x7f41240ccb30 msgr2=0x7f412406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 shutdown_connections 2023-09-27T23:13:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.810+0000 7f412359e700 1 -- 172.21.15.64:0/4078506693 wait complete. 2023-09-27T23:13:47.815 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data_ec' created 2023-09-27T23:13:47.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-27T23:13:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.934+0000 7f410b59e700 1 Processor -- start 2023-09-27T23:13:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.934+0000 7f410b59e700 1 -- start start 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f410c06e7b0 0x7f410c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f410c158b10 0x7f410c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c0c82f0 con 0x7f410c158b10 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c06ded0 con 0x7f410c06e7b0 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c7fa0 con 0x7f410c158740 2023-09-27T23:13:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f410c158b10 0x7f410c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f410c158b10 0x7f410c06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56842/0 (socket says 172.21.15.64:56842) 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 -- 172.21.15.64:0/3738106203 learned_addr learned my addr 172.21.15.64:0/3738106203 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410a59c700 1 --2- 172.21.15.64:0/3738106203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f4101d9b700 1 --2- 172.21.15.64:0/3738106203 >> v2:172.21.15.123:3300/0 conn(0x7f410c06e7b0 0x7f410c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 -- 172.21.15.64:0/3738106203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 msgr2=0x7f410c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 --2- 172.21.15.64:0/3738106203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 -- 172.21.15.64:0/3738106203 >> v2:172.21.15.123:3300/0 conn(0x7f410c06e7b0 msgr2=0x7f410c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 --2- 172.21.15.64:0/3738106203 >> v2:172.21.15.123:3300/0 conn(0x7f410c06e7b0 0x7f410c06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 -- 172.21.15.64:0/3738106203 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0c70f0 con 0x7f410c158b10 2023-09-27T23:13:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f410ad9d700 1 --2- 172.21.15.64:0/3738106203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c06d590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f410c0cc0f0 tx=0x7f410400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62dc48fb4d39bc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.938+0000 7f4109d9b700 1 -- 172.21.15.64:0/3738106203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104010070 con 0x7f410c158b10 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f4109d9b700 1 -- 172.21.15.64:0/3738106203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410400db20 con 0x7f410c158b10 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f4109d9b700 1 -- 172.21.15.64:0/3738106203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104016400 con 0x7f410c158b10 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 -- 172.21.15.64:0/3738106203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 msgr2=0x7f410c06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 --2- 172.21.15.64:0/3738106203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c06d590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f410c0cc0f0 tx=0x7f410400ba50 comp rx=0 tx=0).stop 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 -- 172.21.15.64:0/3738106203 shutdown_connections 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 --2- 172.21.15.64:0/3738106203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c06d590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 --2- 172.21.15.64:0/3738106203 >> v2:172.21.15.123:3300/0 conn(0x7f410c06e7b0 0x7f410c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 --2- 172.21.15.64:0/3738106203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 -- 172.21.15.64:0/3738106203 >> 172.21.15.64:0/3738106203 conn(0x7f410c0c8c00 msgr2=0x7f410c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 -- 172.21.15.64:0/3738106203 shutdown_connections 2023-09-27T23:13:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 -- 172.21.15.64:0/3738106203 wait complete. 2023-09-27T23:13:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 Processor -- start 2023-09-27T23:13:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.942+0000 7f410b59e700 1 -- start start 2023-09-27T23:13:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f410c06e7b0 0x7f410c1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c5140 con 0x7f410c158b10 2023-09-27T23:13:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c4530 con 0x7f410c06e7b0 2023-09-27T23:13:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c4810 con 0x7f410c158740 2023-09-27T23:13:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c0ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56856/0 (socket says 172.21.15.64:56856) 2023-09-27T23:13:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 -- 172.21.15.64:0/1300105367 learned_addr learned my addr 172.21.15.64:0/1300105367 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f4101d9b700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410a59c700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f410c06e7b0 0x7f410c1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 -- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 msgr2=0x7f410c0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:47.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c0c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 -- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f410c06e7b0 msgr2=0x7f410c1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:47.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f410c06e7b0 0x7f410c1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:47.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 -- 172.21.15.64:0/1300105367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410400bcf0 con 0x7f410c158b10 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410ad9d700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c0ca560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f410c06a8c0 tx=0x7f410401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040032f0 con 0x7f410c158b10 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104020650 con 0x7f410c158b10 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c0c4af0 con 0x7f410c158b10 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040209d0 con 0x7f410c158b10 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.946+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c0c55d0 con 0x7f410c158b10 2023-09-27T23:13:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.950+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c0c52b0 con 0x7f410c158b10 2023-09-27T23:13:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.950+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4104003040 con 0x7f410c158b10 2023-09-27T23:13:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.950+0000 7f4102ffd700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f40ec060950 0x7f40ec062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.950+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f4104080f90 con 0x7f410c158b10 2023-09-27T23:13:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.950+0000 7f410a59c700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f40ec060950 0x7f40ec062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.950+0000 7f410a59c700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f40ec060950 0x7f40ec062e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009db0 tx=0x7f40fc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:47.958+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4104050ea0 con 0x7f410c158b10 2023-09-27T23:13:48.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.118+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f410c04be40 con 0x7f410c158b10 2023-09-27T23:13:48.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.802+0000 7f4102ffd700 1 -- 172.21.15.64:0/1300105367 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f41040240c0 con 0x7f410c158b10 2023-09-27T23:13:48.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.806+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f40ec060950 msgr2=0x7f40ec062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.806+0000 7f410b59e700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f40ec060950 0x7f40ec062e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009db0 tx=0x7f40fc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.806+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 msgr2=0x7f410c0ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.806+0000 7f410b59e700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c0ca560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f410c06a8c0 tx=0x7f410401c000 comp rx=0 tx=0).stop 2023-09-27T23:13:48.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 shutdown_connections 2023-09-27T23:13:48.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f40ec060950 0x7f40ec062e00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f410c158b10 0x7f410c0ca560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f410c158740 0x7f410c0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 --2- 172.21.15.64:0/1300105367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f410c06e7b0 0x7f410c1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 >> 172.21.15.64:0/1300105367 conn(0x7f410c0c8c00 msgr2=0x7f410c155670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:48.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 shutdown_connections 2023-09-27T23:13:48.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.810+0000 7f410b59e700 1 -- 172.21.15.64:0/1300105367 wait complete. 2023-09-27T23:13:48.817 INFO:teuthology.orchestra.run.smithi064.stderr:set pool 5 allow_ec_overwrites to true 2023-09-27T23:13:48.836 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-27T23:13:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.934+0000 7f6e13fff700 1 Processor -- start 2023-09-27T23:13:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.934+0000 7f6e13fff700 1 -- start start 2023-09-27T23:13:48.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e13fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6e1406ae20 0x7f6e1406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e13fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1406b730 con 0x7f6e140ab6d0 2023-09-27T23:13:48.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e13fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e140ac390 con 0x7f6e140b1440 2023-09-27T23:13:48.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e13fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e140ac670 con 0x7f6e1406ae20 2023-09-27T23:13:48.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6e140ab6d0 0x7f6e140abb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48712/0 (socket says 172.21.15.64:48712) 2023-09-27T23:13:48.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 -- 172.21.15.64:0/4010391291 learned_addr learned my addr 172.21.15.64:0/4010391291 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e12ffd700 1 --2- 172.21.15.64:0/4010391291 >> v2:172.21.15.123:3300/0 conn(0x7f6e1406ae20 0x7f6e1406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025081388 0 0) 0x7f6e1406b730 con 0x7f6e140ab6d0 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dfc003570 con 0x7f6e140ab6d0 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568535682 0 0) 0x7f6e140ac390 con 0x7f6e140b1440 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e1406b730 con 0x7f6e140b1440 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731344447 0 0) 0x7f6dfc003570 con 0x7f6e140ab6d0 2023-09-27T23:13:48.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 >> v2:172.21.15.123:3300/0 conn(0x7f6e1406ae20 msgr2=0x7f6e1406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 --2- 172.21.15.64:0/4010391291 >> v2:172.21.15.123:3300/0 conn(0x7f6e1406ae20 0x7f6e1406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 >> v1:172.21.15.123:6789/0 conn(0x7f6e140b1440 legacy=0x7f6e140b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140ac950 con 0x7f6e140ab6d0 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f6e08001470 con 0x7f6e140ab6d0 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e12ffd700 1 --2- 172.21.15.64:0/4010391291 >> v2:172.21.15.123:3300/0 conn(0x7f6e1406ae20 0x7f6e1406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 >> v1:172.21.15.64:6789/0 conn(0x7f6e140ab6d0 legacy=0x7f6e140abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc003bf0 0x7f6dfc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc0051c0 0x7f6dfc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140ac670 con 0x7f6dfc004810 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc00c160 con 0x7f6dfc003bf0 2023-09-27T23:13:48.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc00c320 con 0x7f6dfc0051c0 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e12ffd700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc003bf0 0x7f6dfc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e137fe700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc0051c0 0x7f6dfc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 -- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc0051c0 msgr2=0x7f6dfc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc0051c0 0x7f6dfc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 -- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc003bf0 msgr2=0x7f6dfc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dfc003bf0 0x7f6dfc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e137fe700 1 -- 172.21.15.64:0/4010391291 reap_dead start 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 -- 172.21.15.64:0/4010391291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc003570 con 0x7f6dfc004810 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.938+0000 7f6e127fc700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c2a0 tx=0x7f6e08001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34010d971242eb4a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800cb70 con 0x7f6dfc004810 2023-09-27T23:13:48.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0800ccf0 con 0x7f6dfc004810 2023-09-27T23:13:48.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e11ffb700 1 -- 172.21.15.64:0/4010391291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08005b70 con 0x7f6dfc004810 2023-09-27T23:13:48.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 -- 172.21.15.64:0/4010391291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc004810 msgr2=0x7f6dfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c2a0 tx=0x7f6e08001e10 comp rx=0 tx=0).stop 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 -- 172.21.15.64:0/4010391291 shutdown_connections 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 --2- 172.21.15.64:0/4010391291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 -- 172.21.15.64:0/4010391291 >> 172.21.15.64:0/4010391291 conn(0x7f6e140c8cd0 msgr2=0x7f6e140c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 -- 172.21.15.64:0/4010391291 shutdown_connections 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 -- 172.21.15.64:0/4010391291 wait complete. 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 Processor -- start 2023-09-27T23:13:48.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.942+0000 7f6e13fff700 1 -- start start 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc004810 0x7f6e14157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 0x7f6e141684a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14169d80 0x7f6e1416a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140c56b0 con 0x7f6e14168030 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141698f0 con 0x7f6e14169d80 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141580a0 con 0x7f6dfc004810 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 0x7f6e141684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 0x7f6e141684a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56880/0 (socket says 172.21.15.64:56880) 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 -- 172.21.15.64:0/590500033 learned_addr learned my addr 172.21.15.64:0/590500033 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e12ffd700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc004810 0x7f6e14157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e137fe700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14169d80 0x7f6e1416a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 -- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc004810 msgr2=0x7f6e14157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc004810 0x7f6e14157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 -- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14169d80 msgr2=0x7f6e1416a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14169d80 0x7f6e1416a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 -- 172.21.15.64:0/590500033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14168ce0 con 0x7f6e14168030 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e127fc700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 0x7f6e141684a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e08007c90 tx=0x7f6e08015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800ce60 con 0x7f6e14168030 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08001650 con 0x7f6e14168030 2023-09-27T23:13:48.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e14168fc0 con 0x7f6e14168030 2023-09-27T23:13:48.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08007300 con 0x7f6e14168030 2023-09-27T23:13:48.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e14169580 con 0x7f6e14168030 2023-09-27T23:13:48.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.946+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e141570f0 con 0x7f6e14168030 2023-09-27T23:13:48.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.950+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6e080076b0 con 0x7f6e14168030 2023-09-27T23:13:48.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.950+0000 7f6dfaffd700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e00060950 0x7f6e00062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:48.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.950+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f6e0807e0e0 con 0x7f6e14168030 2023-09-27T23:13:48.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.950+0000 7f6e12ffd700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e00060950 0x7f6e00062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:48.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.950+0000 7f6e12ffd700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e00060950 0x7f6e00062e00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0094d0 tx=0x7f6e0c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:48.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:48.954+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6e141570f0 con 0x7f6e14168030 2023-09-27T23:13:49.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.150+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f6e141570f0 con 0x7f6e14168030 2023-09-27T23:13:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.162+0000 7f6dfaffd700 1 -- 172.21.15.64:0/590500033 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6e141570f0 con 0x7f6e14168030 2023-09-27T23:13:49.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e00060950 msgr2=0x7f6e00062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e00060950 0x7f6e00062e00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0094d0 tx=0x7f6e0c00b040 comp rx=0 tx=0).stop 2023-09-27T23:13:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 msgr2=0x7f6e141684a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 0x7f6e141684a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e08007c90 tx=0x7f6e08015000 comp rx=0 tx=0).stop 2023-09-27T23:13:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 shutdown_connections 2023-09-27T23:13:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e00060950 0x7f6e00062e00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14169d80 0x7f6e1416a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e14168030 0x7f6e141684a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 --2- 172.21.15.64:0/590500033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dfc004810 0x7f6e14157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 >> 172.21.15.64:0/590500033 conn(0x7f6e140c8cd0 msgr2=0x7f6e140b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 shutdown_connections 2023-09-27T23:13:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.166+0000 7f6e13fff700 1 -- 172.21.15.64:0/590500033 wait complete. 2023-09-27T23:13:49.175 INFO:teuthology.orchestra.run.smithi064.stderr:added data pool 5 to fsmap 2023-09-27T23:13:49.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:13:49.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 Processor -- start 2023-09-27T23:13:49.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- start start 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe76815d320 0x7fe76815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7680c15b0 0x7fe7680aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7680c7190 con 0x7fe76806d690 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680c1b50 con 0x7fe76815d320 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680c1e30 con 0x7fe7680c15b0 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe76815d320 0x7fe76815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76759e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe76806d690 0x7fe7680680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48724/0 (socket says 172.21.15.64:48724) 2023-09-27T23:13:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7680c15b0 0x7fe7680aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe76815d320 0x7fe76815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56890/0 (socket says 172.21.15.64:56890) 2023-09-27T23:13:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76759e700 1 -- 172.21.15.64:0/3044240263 learned_addr learned my addr 172.21.15.64:0/3044240263 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 -- 172.21.15.64:0/3044240263 >> v2:172.21.15.123:3300/0 conn(0x7fe7680c15b0 msgr2=0x7fe7680aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe765d9b700 1 -- 172.21.15.64:0/3044240263 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984214030 0 0) 0x7fe7680c7190 con 0x7fe76806d690 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 --2- 172.21.15.64:0/3044240263 >> v2:172.21.15.123:3300/0 conn(0x7fe7680c15b0 0x7fe7680aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 -- 172.21.15.64:0/3044240263 >> v1:172.21.15.64:6789/0 conn(0x7fe76806d690 legacy=0x7fe7680680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 -- 172.21.15.64:0/3044240263 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680c6e20 con 0x7fe76815d320 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe766d9d700 1 --2- 172.21.15.64:0/3044240263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76815d320 0x7fe76815d6f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe76806f530 tx=0x7fe75c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65d76c7aacfc821 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe765d9b700 1 -- 172.21.15.64:0/3044240263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011630 con 0x7fe76815d320 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe765d9b700 1 -- 172.21.15.64:0/3044240263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c011790 con 0x7fe76815d320 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe765d9b700 1 -- 172.21.15.64:0/3044240263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c011b30 con 0x7fe76815d320 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- 172.21.15.64:0/3044240263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76815d320 msgr2=0x7fe76815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/3044240263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76815d320 0x7fe76815d6f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe76806f530 tx=0x7fe75c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- 172.21.15.64:0/3044240263 shutdown_connections 2023-09-27T23:13:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/3044240263 >> v2:172.21.15.123:3300/0 conn(0x7fe7680c15b0 0x7fe7680aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/3044240263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76815d320 0x7fe76815d6f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- 172.21.15.64:0/3044240263 >> 172.21.15.64:0/3044240263 conn(0x7fe7680c8850 msgr2=0x7fe7680c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- 172.21.15.64:0/3044240263 shutdown_connections 2023-09-27T23:13:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.286+0000 7fe76d1b4700 1 -- 172.21.15.64:0/3044240263 wait complete. 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 Processor -- start 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 -- start start 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 0x7fe7681541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7680c15b0 0x7fe76816d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe76815d320 0x7fe7680c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe768154710 con 0x7fe76806d690 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680aeec0 con 0x7fe7680c15b0 2023-09-27T23:13:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76d1b4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680af1a0 con 0x7fe76815d320 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 0x7fe7681541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 0x7fe7681541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56904/0 (socket says 172.21.15.64:56904) 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 -- 172.21.15.64:0/541159545 learned_addr learned my addr 172.21.15.64:0/541159545 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76659c700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7680c15b0 0x7fe76816d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe76759e700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe76815d320 0x7fe7680c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 -- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe76815d320 msgr2=0x7fe7680c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe76815d320 0x7fe7680c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 -- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7680c15b0 msgr2=0x7fe76816d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7680c15b0 0x7fe76816d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.290+0000 7fe766d9d700 1 -- 172.21.15.64:0/541159545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009420 con 0x7fe76806d690 2023-09-27T23:13:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe766d9d700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 0x7fe7681541d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe76806a2b0 tx=0x7fe75c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c000d10 con 0x7fe76806d690 2023-09-27T23:13:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c011b10 con 0x7fe76806d690 2023-09-27T23:13:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7680af480 con 0x7fe76806d690 2023-09-27T23:13:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c025640 con 0x7fe76806d690 2023-09-27T23:13:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7680afa40 con 0x7fe76806d690 2023-09-27T23:13:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7680afc30 con 0x7fe76806d690 2023-09-27T23:13:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.294+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe75c025a30 con 0x7fe76806d690 2023-09-27T23:13:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.298+0000 7fe74effd700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe754060980 0x7fe754062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.298+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe75c07e7f0 con 0x7fe76806d690 2023-09-27T23:13:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.298+0000 7fe76659c700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe754060980 0x7fe754062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.298+0000 7fe76659c700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe754060980 0x7fe754062e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe768068800 tx=0x7fe7580091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.302+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe75c04e780 con 0x7fe76806d690 2023-09-27T23:13:49.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.502+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe768071120 con 0x7fe76806d690 2023-09-27T23:13:49.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe74effd700 1 -- 172.21.15.64:0/541159545 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4596 (secure 0 0 0) 0x7fe75c051da0 con 0x7fe76806d690 2023-09-27T23:13:49.509 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:49.510 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:49.156431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-09-27T23:13:49.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe754060980 msgr2=0x7fe754062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe754060980 0x7fe754062e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe768068800 tx=0x7fe7580091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 msgr2=0x7fe7681541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 0x7fe7681541d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe76806a2b0 tx=0x7fe75c017000 comp rx=0 tx=0).stop 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 shutdown_connections 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe754060980 0x7fe754062e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe76815d320 0x7fe7680c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7680c15b0 0x7fe76816d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 --2- 172.21.15.64:0/541159545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76806d690 0x7fe7681541d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 >> 172.21.15.64:0/541159545 conn(0x7fe7680c8850 msgr2=0x7fe7680c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 shutdown_connections 2023-09-27T23:13:49.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.506+0000 7fe76d1b4700 1 -- 172.21.15.64:0/541159545 wait complete. 2023-09-27T23:13:49.513 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-09-27T23:13:49.523 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:49.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.622+0000 7f7879163700 1 Processor -- start 2023-09-27T23:13:49.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.622+0000 7f7879163700 1 -- start start 2023-09-27T23:13:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7879163700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f7874159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7879163700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f78740abce0 0x7f78740c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7879163700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740c1910 con 0x7f78740ab910 2023-09-27T23:13:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7879163700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7874159cf0 con 0x7f78740abce0 2023-09-27T23:13:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7879163700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7874159fd0 con 0x7f78740670e0 2023-09-27T23:13:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7873fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f78740ab910 0x7f78740cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48730/0 (socket says 172.21.15.64:48730) 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7873fff700 1 -- 172.21.15.64:0/2418331940 learned_addr learned my addr 172.21.15.64:0/2418331940 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7878962700 1 --2- 172.21.15.64:0/2418331940 >> v2:172.21.15.64:3300/0 conn(0x7f78740abce0 0x7f78740c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f78737fe700 1 --2- 172.21.15.64:0/2418331940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f7874159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7872ffd700 1 -- 172.21.15.64:0/2418331940 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400251696 0 0) 0x7f78740c1910 con 0x7f78740ab910 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7872ffd700 1 -- 172.21.15.64:0/2418331940 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c003570 con 0x7f78740ab910 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7878962700 1 -- 172.21.15.64:0/2418331940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 msgr2=0x7f7874159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7872ffd700 1 -- 172.21.15.64:0/2418331940 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081341586 0 0) 0x7f785c003570 con 0x7f78740ab910 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7878962700 1 --2- 172.21.15.64:0/2418331940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f7874159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7878962700 1 -- 172.21.15.64:0/2418331940 >> v1:172.21.15.64:6789/0 conn(0x7f78740ab910 legacy=0x7f78740cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7878962700 1 -- 172.21.15.64:0/2418331940 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787415a2b0 con 0x7f78740abce0 2023-09-27T23:13:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7878962700 1 --2- 172.21.15.64:0/2418331940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f78740c2f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f786c009e60 tx=0x7f786c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df1c608e03b4090 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7872ffd700 1 -- 172.21.15.64:0/2418331940 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00e070 con 0x7f78740abce0 2023-09-27T23:13:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7872ffd700 1 -- 172.21.15.64:0/2418331940 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00b7a0 con 0x7f78740abce0 2023-09-27T23:13:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.626+0000 7f7872ffd700 1 -- 172.21.15.64:0/2418331940 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864001470 con 0x7f78740abce0 2023-09-27T23:13:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- 172.21.15.64:0/2418331940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 msgr2=0x7f78740c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 --2- 172.21.15.64:0/2418331940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f78740c2f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f786c009e60 tx=0x7f786c009780 comp rx=0 tx=0).stop 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- 172.21.15.64:0/2418331940 shutdown_connections 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 --2- 172.21.15.64:0/2418331940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f78740c2f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 --2- 172.21.15.64:0/2418331940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f7874159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- 172.21.15.64:0/2418331940 >> 172.21.15.64:0/2418331940 conn(0x7f787406c080 msgr2=0x7f78740ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- 172.21.15.64:0/2418331940 shutdown_connections 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- 172.21.15.64:0/2418331940 wait complete. 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 Processor -- start 2023-09-27T23:13:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- start start 2023-09-27T23:13:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f78740cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ab910 0x7f78741536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f7874153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740c6a80 con 0x7f78740abce0 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740c4a40 con 0x7f78740ab910 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7879163700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740c4110 con 0x7f78740670e0 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7878962700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f7874153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7878962700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f7874153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56920/0 (socket says 172.21.15.64:56920) 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7878962700 1 -- 172.21.15.64:0/353492632 learned_addr learned my addr 172.21.15.64:0/353492632 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f7873fff700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f78740cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.630+0000 7f78737fe700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ab910 0x7f78741536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7878962700 1 -- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 msgr2=0x7f78740cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7878962700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f78740cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7878962700 1 -- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ab910 msgr2=0x7f78741536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7878962700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ab910 0x7f78741536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7878962700 1 -- 172.21.15.64:0/353492632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009a20 con 0x7f78740abce0 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f78737fe700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ab910 0x7f78741536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7878962700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f7874153c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f786c0097c0 tx=0x7f786c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c012990 con 0x7f78740abce0 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c021070 con 0x7f78740abce0 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78740c43f0 con 0x7f78740abce0 2023-09-27T23:13:49.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c012da0 con 0x7f78740abce0 2023-09-27T23:13:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78740cc120 con 0x7f78740abce0 2023-09-27T23:13:49.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f786c023030 con 0x7f78740abce0 2023-09-27T23:13:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.634+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78740c4bf0 con 0x7f78740abce0 2023-09-27T23:13:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.638+0000 7f785bfff700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7860060950 0x7f7860062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.638+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f786c07f000 con 0x7f78740abce0 2023-09-27T23:13:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.638+0000 7f7873fff700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7860060950 0x7f7860062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.638+0000 7f7873fff700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7860060950 0x7f7860062e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7864004dd0 tx=0x7f7864001970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.642+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f78740c4bf0 con 0x7f78740abce0 2023-09-27T23:13:49.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.806+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78740c4bb0 con 0x7f78740abce0 2023-09-27T23:13:49.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.806+0000 7f785bfff700 1 -- 172.21.15.64:0/353492632 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f786c04ef90 con 0x7f78740abce0 2023-09-27T23:13:49.812 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:49.813 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7860060950 msgr2=0x7f7860062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7860060950 0x7f7860062e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7864004dd0 tx=0x7f7864001970 comp rx=0 tx=0).stop 2023-09-27T23:13:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 msgr2=0x7f7874153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f7874153c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f786c0097c0 tx=0x7f786c018000 comp rx=0 tx=0).stop 2023-09-27T23:13:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 shutdown_connections 2023-09-27T23:13:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7860060950 0x7f7860062e00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78740abce0 0x7f7874153c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ab910 0x7f78741536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 --2- 172.21.15.64:0/353492632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740670e0 0x7f78740cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 >> 172.21.15.64:0/353492632 conn(0x7f787406c080 msgr2=0x7f7874068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:49.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 shutdown_connections 2023-09-27T23:13:49.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.810+0000 7f7879163700 1 -- 172.21.15.64:0/353492632 wait complete. 2023-09-27T23:13:49.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:49.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 Processor -- start 2023-09-27T23:13:49.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 -- start start 2023-09-27T23:13:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d80cc6c0 0x7ff5d815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5d80ccb10 0x7ff5d815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d806cd40 con 0x7ff5d8161170 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d806c7c0 con 0x7ff5d80ccb10 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5ddac9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80c9930 con 0x7ff5d80cc6c0 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff5d8161170 0x7ff5d8161540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48736/0 (socket says 172.21.15.64:48736) 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d77fe700 1 -- 172.21.15.64:0/2436200774 learned_addr learned my addr 172.21.15.64:0/2436200774 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d6ffd700 1 --2- 172.21.15.64:0/2436200774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d80cc6c0 0x7ff5d815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d7fff700 1 --2- 172.21.15.64:0/2436200774 >> v2:172.21.15.123:3300/0 conn(0x7ff5d80ccb10 0x7ff5d815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d67fc700 1 -- 172.21.15.64:0/2436200774 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683345021 0 0) 0x7ff5d806cd40 con 0x7ff5d8161170 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d67fc700 1 -- 172.21.15.64:0/2436200774 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c4003570 con 0x7ff5d8161170 2023-09-27T23:13:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/2436200774 >> v2:172.21.15.123:3300/0 conn(0x7ff5d80ccb10 msgr2=0x7ff5d815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:49.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d6ffd700 1 --2- 172.21.15.64:0/2436200774 >> v2:172.21.15.123:3300/0 conn(0x7ff5d80ccb10 0x7ff5d815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/2436200774 >> v1:172.21.15.64:6789/0 conn(0x7ff5d8161170 legacy=0x7ff5d8161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/2436200774 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d80c9c10 con 0x7ff5d80cc6c0 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d6ffd700 1 --2- 172.21.15.64:0/2436200774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d80cc6c0 0x7ff5d815aaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009e60 tx=0x7ff5d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88bf36125ff370a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d67fc700 1 -- 172.21.15.64:0/2436200774 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000e070 con 0x7ff5d80cc6c0 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d67fc700 1 -- 172.21.15.64:0/2436200774 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d000bab0 con 0x7ff5d80cc6c0 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.934+0000 7ff5d67fc700 1 -- 172.21.15.64:0/2436200774 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000be30 con 0x7ff5d80cc6c0 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/2436200774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d80cc6c0 msgr2=0x7ff5d815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/2436200774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d80cc6c0 0x7ff5d815aaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009e60 tx=0x7ff5d0009780 comp rx=0 tx=0).stop 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/2436200774 shutdown_connections 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/2436200774 >> v2:172.21.15.123:3300/0 conn(0x7ff5d80ccb10 0x7ff5d815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/2436200774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d80cc6c0 0x7ff5d815aaa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/2436200774 >> 172.21.15.64:0/2436200774 conn(0x7ff5d80ce000 msgr2=0x7ff5d80ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:49.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/2436200774 shutdown_connections 2023-09-27T23:13:49.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/2436200774 wait complete. 2023-09-27T23:13:49.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 Processor -- start 2023-09-27T23:13:49.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- start start 2023-09-27T23:13:49.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80cc6c0 0x7ff5d816ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 0x7ff5d816cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d8161170 0x7ff5d816d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d816ea80 con 0x7ff5d80ccb10 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d816ecd0 con 0x7ff5d80cc6c0 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d816da20 con 0x7ff5d8161170 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 0x7ff5d816cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 0x7ff5d816cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56930/0 (socket says 172.21.15.64:56930) 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/984408196 learned_addr learned my addr 172.21.15.64:0/984408196 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d7fff700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d8161170 0x7ff5d816d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d77fe700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80cc6c0 0x7ff5d816ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d8161170 msgr2=0x7ff5d816d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d8161170 0x7ff5d816d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80cc6c0 msgr2=0x7ff5d816ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80cc6c0 0x7ff5d816ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:49.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 -- 172.21.15.64:0/984408196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009a20 con 0x7ff5d80ccb10 2023-09-27T23:13:49.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5d6ffd700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 0x7ff5d816cf70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00097c0 tx=0x7ff5d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0012a20 con 0x7ff5d80ccb10 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d0021070 con 0x7ff5d80ccb10 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d8159c70 con 0x7ff5d80ccb10 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0012dd0 con 0x7ff5d80ccb10 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.938+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d815a2b0 con 0x7ff5d80ccb10 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.942+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d815a470 con 0x7ff5d80ccb10 2023-09-27T23:13:49.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.942+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ff5d0023030 con 0x7ff5d80ccb10 2023-09-27T23:13:49.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.942+0000 7ff5cf7fe700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff5b8060950 0x7ff5b8062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:49.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.942+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff5d007ef50 con 0x7ff5d80ccb10 2023-09-27T23:13:49.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.942+0000 7ff5d77fe700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff5b8060950 0x7ff5b8062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:49.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.946+0000 7ff5d77fe700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff5b8060950 0x7ff5b8062e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00058d0 tx=0x7ff5c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:49.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:49.946+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff5d815a470 con 0x7ff5d80ccb10 2023-09-27T23:13:50.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.114+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5d815a470 con 0x7ff5d80ccb10 2023-09-27T23:13:50.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.114+0000 7ff5cf7fe700 1 -- 172.21.15.64:0/984408196 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7ff5d815a470 con 0x7ff5d80ccb10 2023-09-27T23:13:50.120 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:50.120 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff5b8060950 msgr2=0x7ff5b8062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff5b8060950 0x7ff5b8062e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00058d0 tx=0x7ff5c0009040 comp rx=0 tx=0).stop 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 msgr2=0x7ff5d816cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 0x7ff5d816cf70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00097c0 tx=0x7ff5d0019000 comp rx=0 tx=0).stop 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 shutdown_connections 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff5b8060950 0x7ff5b8062e00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5d8161170 0x7ff5d816d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d80ccb10 0x7ff5d816cf70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 --2- 172.21.15.64:0/984408196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80cc6c0 0x7ff5d816ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 >> 172.21.15.64:0/984408196 conn(0x7ff5d80ce000 msgr2=0x7ff5d815c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 shutdown_connections 2023-09-27T23:13:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.118+0000 7ff5ddac9700 1 -- 172.21.15.64:0/984408196 wait complete. 2023-09-27T23:13:50.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:13:50.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.246+0000 7fe0ae5c0700 1 Processor -- start 2023-09-27T23:13:50.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.246+0000 7fe0ae5c0700 1 -- start start 2023-09-27T23:13:50.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0a80670e0 0x7fe0a8159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80abce0 0x7fe0a80c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a80c1910 con 0x7fe0a80ab910 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a8159cf0 con 0x7fe0a80670e0 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8159fd0 con 0x7fe0a80abce0 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0a80670e0 0x7fe0a8159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0a80670e0 0x7fe0a8159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56938/0 (socket says 172.21.15.64:56938) 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 -- 172.21.15.64:0/3233418168 learned_addr learned my addr 172.21.15.64:0/3233418168 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:50.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0addbf700 1 --2- 172.21.15.64:0/3233418168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80abce0 0x7fe0a80c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 -- 172.21.15.64:0/3233418168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80abce0 msgr2=0x7fe0a80c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:50.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 --2- 172.21.15.64:0/3233418168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80abce0 0x7fe0a80c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 -- 172.21.15.64:0/3233418168 >> v1:172.21.15.123:6789/0 conn(0x7fe0a80ab910 legacy=0x7fe0a80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 -- 172.21.15.64:0/3233418168 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a815a2b0 con 0x7fe0a80670e0 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0acdbd700 1 --2- 172.21.15.64:0/3233418168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80670e0 0x7fe0a8159710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0980099b0 tx=0x7fe098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf7aca932f5c58f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe09ffff700 1 -- 172.21.15.64:0/3233418168 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098009ea0 con 0x7fe0a80670e0 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe09ffff700 1 -- 172.21.15.64:0/3233418168 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09800b860 con 0x7fe0a80670e0 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe09ffff700 1 -- 172.21.15.64:0/3233418168 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800bc00 con 0x7fe0a80670e0 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/3233418168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80670e0 msgr2=0x7fe0a8159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/3233418168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80670e0 0x7fe0a8159710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0980099b0 tx=0x7fe098009250 comp rx=0 tx=0).stop 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/3233418168 shutdown_connections 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/3233418168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80abce0 0x7fe0a80c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/3233418168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80670e0 0x7fe0a8159710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/3233418168 >> 172.21.15.64:0/3233418168 conn(0x7fe0a806c080 msgr2=0x7fe0a80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:50.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/3233418168 shutdown_connections 2023-09-27T23:13:50.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.250+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/3233418168 wait complete. 2023-09-27T23:13:50.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 Processor -- start 2023-09-27T23:13:50.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 -- start start 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80670e0 0x7fe0a814e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 0x7fe0a8153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a80abce0 0x7fe0a80cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80b1870 con 0x7fe0a80ab910 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80b0f40 con 0x7fe0a80abce0 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80b1220 con 0x7fe0a80670e0 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 0x7fe0a8153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 0x7fe0a8153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56948/0 (socket says 172.21.15.64:56948) 2023-09-27T23:13:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 -- 172.21.15.64:0/2912198525 learned_addr learned my addr 172.21.15.64:0/2912198525 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:50.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ad5be700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80670e0 0x7fe0a814e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0addbf700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a80abce0 0x7fe0a80cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 -- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80670e0 msgr2=0x7fe0a814e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80670e0 0x7fe0a814e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 -- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a80abce0 msgr2=0x7fe0a80cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a80abce0 0x7fe0a80cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 -- 172.21.15.64:0/2912198525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980094f0 con 0x7fe0a80ab910 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0acdbd700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 0x7fe0a8153170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80ca600 tx=0x7fe098017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098020070 con 0x7fe0a80ab910 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09800bbe0 con 0x7fe0a80ab910 2023-09-27T23:13:50.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a80b1500 con 0x7fe0a80ab910 2023-09-27T23:13:50.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098009ea0 con 0x7fe0a80ab910 2023-09-27T23:13:50.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.254+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a80b0b60 con 0x7fe0a80ab910 2023-09-27T23:13:50.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.258+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a80b1a20 con 0x7fe0a80ab910 2023-09-27T23:13:50.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.258+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe0980118a0 con 0x7fe0a80ab910 2023-09-27T23:13:50.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.258+0000 7fe09d7fa700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe08c060950 0x7fe08c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.258+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe098080210 con 0x7fe0a80ab910 2023-09-27T23:13:50.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.258+0000 7fe0ad5be700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe08c060950 0x7fe08c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.258+0000 7fe0ad5be700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe08c060950 0x7fe08c062e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0a81691d0 tx=0x7fe0a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:50.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.266+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe0a80b1a20 con 0x7fe0a80ab910 2023-09-27T23:13:50.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.462+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0a804be40 con 0x7fe0a80ab910 2023-09-27T23:13:50.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.462+0000 7fe09d7fa700 1 -- 172.21.15.64:0/2912198525 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4596 (secure 0 0 0) 0x7fe0980501a0 con 0x7fe0a80ab910 2023-09-27T23:13:50.467 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:50.467 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:49.156431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-09-27T23:13:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe08c060950 msgr2=0x7fe08c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe08c060950 0x7fe08c062e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0a81691d0 tx=0x7fe0a4009040 comp rx=0 tx=0).stop 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 msgr2=0x7fe0a8153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 0x7fe0a8153170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80ca600 tx=0x7fe098017000 comp rx=0 tx=0).stop 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 shutdown_connections 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe08c060950 0x7fe08c062e00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a80abce0 0x7fe0a80cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a80ab910 0x7fe0a8153170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 --2- 172.21.15.64:0/2912198525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0a80670e0 0x7fe0a814e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 >> 172.21.15.64:0/2912198525 conn(0x7fe0a806c080 msgr2=0x7fe0a8158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:50.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 shutdown_connections 2023-09-27T23:13:50.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.466+0000 7fe0ae5c0700 1 -- 172.21.15.64:0/2912198525 wait complete. 2023-09-27T23:13:50.471 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-09-27T23:13:50.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:50.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.578+0000 7f5958855700 1 Processor -- start 2023-09-27T23:13:50.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.578+0000 7f5958855700 1 -- start start 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f595415ae00 0x7f59540c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540add10 con 0x7f59540acc30 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540cc920 con 0x7f59540ad000 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540ad450 con 0x7f595415ae00 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f595415ae00 0x7f59540c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f595415ae00 0x7f59540c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56958/0 (socket says 172.21.15.64:56958) 2023-09-27T23:13:50.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 -- 172.21.15.64:0/1869812052 learned_addr learned my addr 172.21.15.64:0/1869812052 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5952d9d700 1 --1- 172.21.15.64:0/1869812052 >> v1:172.21.15.64:6789/0 conn(0x7f59540acc30 0x7f59540ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48752/0 (socket says 172.21.15.64:48752) 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 -- 172.21.15.64:0/1869812052 >> v1:172.21.15.123:6789/0 conn(0x7f59540ad000 legacy=0x7f595415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 -- 172.21.15.64:0/1869812052 >> v1:172.21.15.64:6789/0 conn(0x7f59540acc30 legacy=0x7f59540ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 -- 172.21.15.64:0/1869812052 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540c9290 con 0x7f595415ae00 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f595359e700 1 --2- 172.21.15.64:0/1869812052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f59540c8280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f594c009f20 tx=0x7f594c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f29b36298f3ec9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5951d9b700 1 -- 172.21.15.64:0/1869812052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00e070 con 0x7f595415ae00 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5951d9b700 1 -- 172.21.15.64:0/1869812052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00bad0 con 0x7f595415ae00 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5951d9b700 1 -- 172.21.15.64:0/1869812052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0092e0 con 0x7f595415ae00 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- 172.21.15.64:0/1869812052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 msgr2=0x7f59540c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 --2- 172.21.15.64:0/1869812052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f59540c8280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f594c009f20 tx=0x7f594c009840 comp rx=0 tx=0).stop 2023-09-27T23:13:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- 172.21.15.64:0/1869812052 shutdown_connections 2023-09-27T23:13:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 --2- 172.21.15.64:0/1869812052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f59540c8280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- 172.21.15.64:0/1869812052 >> 172.21.15.64:0/1869812052 conn(0x7f59540cdbd0 msgr2=0x7f59540cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- 172.21.15.64:0/1869812052 shutdown_connections 2023-09-27T23:13:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- 172.21.15.64:0/1869812052 wait complete. 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 Processor -- start 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.582+0000 7f5958855700 1 -- start start 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59540acc30 0x7f59540b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59540ad000 0x7f59540c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f5954149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b2df0 con 0x7f595415ae00 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b27e0 con 0x7f59540acc30 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540c66f0 con 0x7f59540ad000 2023-09-27T23:13:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f5954149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f5954149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56966/0 (socket says 172.21.15.64:56966) 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 -- 172.21.15.64:0/1952308772 learned_addr learned my addr 172.21.15.64:0/1952308772 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595259c700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59540ad000 0x7f59540c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5952d9d700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59540acc30 0x7f59540b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 -- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59540ad000 msgr2=0x7f59540c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59540ad000 0x7f59540c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 -- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59540acc30 msgr2=0x7f59540b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59540acc30 0x7f59540b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 -- 172.21.15.64:0/1952308772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009ae0 con 0x7f595415ae00 2023-09-27T23:13:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595259c700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59540ad000 0x7f59540c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5952d9d700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59540acc30 0x7f59540b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f595359e700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f5954149570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f594c003000 tx=0x7f594c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c012e60 con 0x7f595415ae00 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c0092e0 con 0x7f595415ae00 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540c6410 con 0x7f595415ae00 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c01d670 con 0x7f595415ae00 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540c6050 con 0x7f595415ae00 2023-09-27T23:13:50.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59540b2640 con 0x7f595415ae00 2023-09-27T23:13:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.586+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f594c01d9e0 con 0x7f595415ae00 2023-09-27T23:13:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.590+0000 7f593affd700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5940060950 0x7f5940062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.590+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f594c07ed00 con 0x7f595415ae00 2023-09-27T23:13:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.590+0000 7f5952d9d700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5940060950 0x7f5940062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:50.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.590+0000 7f5952d9d700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5940060950 0x7f5940062e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5944007fd0 tx=0x7f5944007db0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:50.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.598+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f59540b2640 con 0x7f595415ae00 2023-09-27T23:13:50.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.758+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59540b2640 con 0x7f595415ae00 2023-09-27T23:13:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.762+0000 7f593affd700 1 -- 172.21.15.64:0/1952308772 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f59540b2640 con 0x7f595415ae00 2023-09-27T23:13:50.767 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:50.767 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5940060950 msgr2=0x7f5940062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5940060950 0x7f5940062e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5944007fd0 tx=0x7f5944007db0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 msgr2=0x7f5954149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f5954149570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f594c003000 tx=0x7f594c017000 comp rx=0 tx=0).stop 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 shutdown_connections 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5940060950 0x7f5940062e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595415ae00 0x7f5954149570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59540ad000 0x7f59540c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 --2- 172.21.15.64:0/1952308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59540acc30 0x7f59540b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 >> 172.21.15.64:0/1952308772 conn(0x7f59540cdbd0 msgr2=0x7f59540ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 shutdown_connections 2023-09-27T23:13:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:50.766+0000 7f5958855700 1 -- 172.21.15.64:0/1952308772 wait complete. 2023-09-27T23:13:51.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:13:44.760383+0000', 'modified': '2023-09-27T23:13:49.156431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6835/2280995923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2280995923}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2280995923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-09-27T23:13:51.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:13:51.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-27T23:13:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 Processor -- start 2023-09-27T23:13:51.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- start start 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f616c1550e0 0x7f616c0ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06be30 0x7f616c06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f616c06b320 0x7f616c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c0b1220 con 0x7f616c1550e0 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c06b7c0 con 0x7f616c06b320 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0afdf0 con 0x7f616c06be30 2023-09-27T23:13:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f616c1550e0 0x7f616c0ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f616c1550e0 0x7f616c0ab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56970/0 (socket says 172.21.15.64:56970) 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 -- 172.21.15.64:0/3696355512 learned_addr learned my addr 172.21.15.64:0/3696355512 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616affd700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06be30 0x7f616c06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616bfff700 1 --2- 172.21.15.64:0/3696355512 >> v2:172.21.15.123:3300/0 conn(0x7f616c06b320 0x7f616c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 -- 172.21.15.64:0/3696355512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06be30 msgr2=0x7f616c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06be30 0x7f616c06ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 -- 172.21.15.64:0/3696355512 >> v2:172.21.15.123:3300/0 conn(0x7f616c06b320 msgr2=0x7f616c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 --2- 172.21.15.64:0/3696355512 >> v2:172.21.15.123:3300/0 conn(0x7f616c06b320 0x7f616c06c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 -- 172.21.15.64:0/3696355512 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0af220 con 0x7f616c1550e0 2023-09-27T23:13:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616affd700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06be30 0x7f616c06ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616b7fe700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c1550e0 0x7f616c0ab540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f616c063f50 tx=0x7f6154009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e98b012da667a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616a7fc700 1 -- 172.21.15.64:0/3696355512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400b860 con 0x7f616c1550e0 2023-09-27T23:13:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616a7fc700 1 -- 172.21.15.64:0/3696355512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615400b9c0 con 0x7f616c1550e0 2023-09-27T23:13:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f616a7fc700 1 -- 172.21.15.64:0/3696355512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400bd60 con 0x7f616c1550e0 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- 172.21.15.64:0/3696355512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c1550e0 msgr2=0x7f616c0ab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c1550e0 0x7f616c0ab540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f616c063f50 tx=0x7f6154009250 comp rx=0 tx=0).stop 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- 172.21.15.64:0/3696355512 shutdown_connections 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- 172.21.15.64:0/3696355512 >> v2:172.21.15.123:3300/0 conn(0x7f616c06b320 0x7f616c06c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06be30 0x7f616c06ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 --2- 172.21.15.64:0/3696355512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c1550e0 0x7f616c0ab540 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- 172.21.15.64:0/3696355512 >> 172.21.15.64:0/3696355512 conn(0x7f616c0684d0 msgr2=0x7f616c0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- 172.21.15.64:0/3696355512 shutdown_connections 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.302+0000 7f61709b2700 1 -- 172.21.15.64:0/3696355512 wait complete. 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 Processor -- start 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 -- start start 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 0x7f616c14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 0x7f616c0c1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1550e0 0x7f616c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c14e900 con 0x7f616c06be30 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c152530 con 0x7f616c1550e0 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f61709b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c14ede0 con 0x7f616c06b320 2023-09-27T23:13:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 0x7f616c0c1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 0x7f616c14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 0x7f616c14e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49514/0 (socket says 172.21.15.64:49514) 2023-09-27T23:13:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 0x7f616c0c1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56982/0 (socket says 172.21.15.64:56982) 2023-09-27T23:13:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616b7fe700 1 -- 172.21.15.64:0/1223476679 learned_addr learned my addr 172.21.15.64:0/1223476679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616bfff700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1550e0 0x7f616c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 -- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 msgr2=0x7f616c14e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 0x7f616c14e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 -- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1550e0 msgr2=0x7f616c0c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1550e0 0x7f616c0c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 -- 172.21.15.64:0/1223476679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540094f0 con 0x7f616c06be30 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 0x7f616c14e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616bfff700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1550e0 0x7f616c0c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.306+0000 7f616affd700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 0x7f616c0c1920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f616400e160 tx=0x7f6164009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61637fe700 1 -- 172.21.15.64:0/1223476679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616400ec40 con 0x7f616c06be30 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61637fe700 1 -- 172.21.15.64:0/1223476679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616400eda0 con 0x7f616c06be30 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c152ab0 con 0x7f616c06be30 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61637fe700 1 -- 172.21.15.64:0/1223476679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164014670 con 0x7f616c06be30 2023-09-27T23:13:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c0c3150 con 0x7f616c06be30 2023-09-27T23:13:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61637fe700 1 -- 172.21.15.64:0/1223476679 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f61640149e0 con 0x7f616c06be30 2023-09-27T23:13:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.310+0000 7f61637fe700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f614c060980 0x7f614c062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f61637fe700 1 -- 172.21.15.64:0/1223476679 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f616407d760 con 0x7f616c06be30 2023-09-27T23:13:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f614c060980 0x7f614c062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f614c060980 0x7f614c062e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f615400c010 tx=0x7f6154017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f6173ebd740 1 Processor -- start 2023-09-27T23:13:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f6173ebd740 1 -- start start 2023-09-27T23:13:51.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 0x1db8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1db9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 0x1db98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.314+0000 7f6173ebd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce6f30 con 0x1b86790 2023-09-27T23:13:51.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce2ed0 con 0x1b3bae0 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ca90b0 con 0x1b3b710 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 0x1db98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 0x1db8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1db9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 0x1db8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:42058/0 (socket says 172.21.15.64:42058) 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1db9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49518/0 (socket says 172.21.15.64:49518) 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616b7fe700 1 -- 172.21.15.64:0/3516231044 learned_addr learned my addr 172.21.15.64:0/3516231044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 0x1db98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56984/0 (socket says 172.21.15.64:56984) 2023-09-27T23:13:51.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 -- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 msgr2=0x1db9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1db9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 -- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 msgr2=0x1db8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 0x1db8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 -- 172.21.15.64:0/3516231044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1b879e0 con 0x1b86790 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616affd700 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1db9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616b7fe700 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 0x1db8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f616bfff700 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 0x1db98d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f615c006b00 tx=0x7f615c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625ef35585f8c45d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6160ff9700 1 -- 172.21.15.64:0/3516231044 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c007e00 con 0x1b86790 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6160ff9700 1 -- 172.21.15.64:0/3516231044 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c010bd0 con 0x1b86790 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6160ff9700 1 -- 172.21.15.64:0/3516231044 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c014430 con 0x1b86790 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- 172.21.15.64:0/3516231044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 msgr2=0x1db98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 0x1db98d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f615c006b00 tx=0x7f615c008040 comp rx=0 tx=0).stop 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- 172.21.15.64:0/3516231044 shutdown_connections 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b86790 0x1db98d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1db9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 --2- 172.21.15.64:0/3516231044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3bae0 0x1db8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- 172.21.15.64:0/3516231044 >> 172.21.15.64:0/3516231044 conn(0x1b3beb0 msgr2=0x1d75880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- 172.21.15.64:0/3516231044 shutdown_connections 2023-09-27T23:13:51.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- 172.21.15.64:0/3516231044 wait complete. 2023-09-27T23:13:51.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.318+0000 7f6173ebd740 1 -- start start 2023-09-27T23:13:51.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 Processor -- start 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1d75740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 0x1d9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1d9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ca8d50 con 0x1b3bae0 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c81970 con 0x1b86790 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ce6100 con 0x1b3b710 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 10 client.0 _ll_register_callbacks cb 0x1d2cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:13:51.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 0x1d9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 0x1d9ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56998/0 (socket says 172.21.15.64:56998) 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 -- 172.21.15.64:0/1866067687 learned_addr learned my addr 172.21.15.64:0/1866067687 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1d75740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616bfff700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1d9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 -- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 msgr2=0x1d75740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1d75740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 -- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 msgr2=0x1d9f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1d9f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 -- 172.21.15.64:0/1866067687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c0c3150 con 0x1b3bae0 2023-09-27T23:13:51.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616affd700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 0x1d9ec00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f616407b8c0 tx=0x7f616407b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a34264aa2b3b1f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6149ffb700 1 -- 172.21.15.64:0/1866067687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616404f750 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6149ffb700 1 -- 172.21.15.64:0/1866067687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f61640508a0 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c81cc0 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6149ffb700 1 -- 172.21.15.64:0/1866067687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6164057f00 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 10 client.4449 fetch_fsmap learned FSMap version 6 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7f616c0af220 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6149ffb700 1 -- 172.21.15.64:0/1866067687 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616406f080 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6149ffb700 1 -- 172.21.15.64:0/1866067687 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f61640563f0 con 0x1b3bae0 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 10 client.4449 fetch_fsmap finished waiting for FSMap version 6 2023-09-27T23:13:51.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 10 client.4449 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 20 client.4449 populate_metadata read hostname 'smithi064' 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1dc56b0 0x1dc7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 4 client.4449 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 --> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c853e0 con 0x1dc56b0 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.322+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1dc56b0 0x1dc7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6149ffb700 10 client.4449 ms_handle_connect on v2:172.21.15.123:6834/2280995923 2023-09-27T23:13:51.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1dc56b0 0x1dc7a90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ff7fee42546ff3 server_cookie=e6664c88f2c0be1 in_seq=0 out_seq=0 2023-09-27T23:13:51.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6149ffb700 1 -- 172.21.15.64:0/1866067687 <== mds.0 v2:172.21.15.123:6834/2280995923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1c853e0 con 0x1dc56b0 2023-09-27T23:13:51.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6149ffb700 10 client.4449 handle_command_reply: tid=0 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 client.4449 shutdown 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 msgr2=0x1d9ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 0x1d9ec00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f616407b8c0 tx=0x7f616407b790 comp rx=0 tx=0).stop 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 shutdown_connections 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1dc56b0 0x1dc7a90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1d9f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3bae0 0x1d9ec00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6149ffb700 0 client.4449 ms_handle_reset on v2:172.21.15.123:6834/2280995923 2023-09-27T23:13:51.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1866067687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3b710 0x1d75740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 >> 172.21.15.64:0/1866067687 conn(0x1b3beb0 msgr2=0x1d6ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 shutdown_connections 2023-09-27T23:13:51.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 1 -- 172.21.15.64:0/1866067687 wait complete. 2023-09-27T23:13:51.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.326+0000 7f6173ebd740 20 client.4449 trim_cache size 0 max 16384 2023-09-27T23:13:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 Processor -- start 2023-09-27T23:13:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 -- start start 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 0x1e0e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3bae0 0x1e0ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1e0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cb6f40 con 0x1b3b710 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cad770 con 0x1b86790 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cb8010 con 0x1b3bae0 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 0x1e0e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3bae0 0x1e0ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 0x1e0e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57000/0 (socket says 172.21.15.64:57000) 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616b7fe700 1 -- 172.21.15.64:0/1254427160 learned_addr learned my addr 172.21.15.64:0/1254427160 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616bfff700 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1e0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616affd700 1 -- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 msgr2=0x1e0f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616affd700 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1e0f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616affd700 1 -- 172.21.15.64:0/1254427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 msgr2=0x1e0e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616affd700 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 0x1e0e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616affd700 1 -- 172.21.15.64:0/1254427160 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0c4b50 con 0x1b3bae0 2023-09-27T23:13:51.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.346+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 0x1e0e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f616affd700 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3bae0 0x1e0ec10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f616404f700 tx=0x7f6164063000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3dc568e3dffbcf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f61617fa700 1 -- 172.21.15.64:0/1254427160 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164056770 con 0x1b3bae0 2023-09-27T23:13:51.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f61617fa700 1 -- 172.21.15.64:0/1254427160 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616406f050 con 0x1b3bae0 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f61617fa700 1 -- 172.21.15.64:0/1254427160 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164050d10 con 0x1b3bae0 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3bae0 msgr2=0x1e0ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3bae0 0x1e0ec10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f616404f700 tx=0x7f6164063000 comp rx=0 tx=0).stop 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- 172.21.15.64:0/1254427160 shutdown_connections 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b86790 0x1e0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b3bae0 0x1e0ec10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1254427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b3b710 0x1e0e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- 172.21.15.64:0/1254427160 >> 172.21.15.64:0/1254427160 conn(0x1b3beb0 msgr2=0x1d9dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- 172.21.15.64:0/1254427160 shutdown_connections 2023-09-27T23:13:51.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- 172.21.15.64:0/1254427160 wait complete. 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- start start 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 Processor -- start 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3b710 0x1d9a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b86790 0x1d9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 0x1d9b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce5590 con 0x1d9b380 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ce7ef0 con 0x1b3b710 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cb8550 con 0x1b86790 2023-09-27T23:13:51.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f6173ebd740 10 client.0 _ll_register_callbacks cb 0x1bff0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:13:51.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f616bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 0x1d9b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f616bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 0x1d9b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57012/0 (socket says 172.21.15.64:57012) 2023-09-27T23:13:51.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f616bfff700 1 -- 172.21.15.64:0/1805564785 learned_addr learned my addr 172.21.15.64:0/1805564785 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3b710 0x1d9a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.350+0000 7f616affd700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b86790 0x1d9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616b7fe700 1 -- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b86790 msgr2=0x1d9ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b86790 0x1d9ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616b7fe700 1 -- 172.21.15.64:0/1805564785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 msgr2=0x1d9b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 0x1d9b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616b7fe700 1 -- 172.21.15.64:0/1805564785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6154011990 con 0x1b3b710 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616affd700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b86790 0x1d9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616bfff700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 0x1d9b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616b7fe700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3b710 0x1d9a900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6154003030 tx=0x7f6154029040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddae282fb2eba1c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 1 -- 172.21.15.64:0/1805564785 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615402c040 con 0x1b3b710 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f615c010bd0 con 0x1b3b710 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 1 -- 172.21.15.64:0/1805564785 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f615403a070 con 0x1b3b710 2023-09-27T23:13:51.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 1 -- 172.21.15.64:0/1805564785 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6154020070 con 0x1b3b710 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 10 client.4456 fetch_fsmap learned FSMap version 6 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7f616c0af220 con 0x1b3b710 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 1 -- 172.21.15.64:0/1805564785 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c010bd0 con 0x1b3b710 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 1 -- 172.21.15.64:0/1805564785 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f615401ab60 con 0x1b3b710 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 10 client.4456 fetch_fsmap finished waiting for FSMap version 6 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 10 client.4456 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 20 client.4456 populate_metadata read hostname 'smithi064' 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1e18950 0x1e1ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 4 client.4456 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-09-27T23:13:51.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 --> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1cb7410 con 0x1e18950 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616affd700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1e18950 0x1e1ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 10 client.4456 ms_handle_connect on v2:172.21.15.123:6834/2280995923 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f616affd700 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1e18950 0x1e1ad30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c399803a97005989 server_cookie=7712efca9c2fb26b in_seq=0 out_seq=0 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 1 -- 172.21.15.64:0/1805564785 <== mds.0 v2:172.21.15.123:6834/2280995923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x1cb7410 con 0x1e18950 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6149ffb700 10 client.4456 handle_command_reply: tid=0 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.354+0000 7f6173ebd740 1 client.4456 shutdown 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3b710 msgr2=0x1d9a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3b710 0x1d9a900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6154003030 tx=0x7f6154029040 comp rx=0 tx=0).stop 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 shutdown_connections 2023-09-27T23:13:51.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1e18950 0x1e1ad30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6149ffb700 0 client.4456 ms_handle_reset on v2:172.21.15.123:6834/2280995923 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d9b380 0x1d9b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1b86790 0x1d9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 --2- 172.21.15.64:0/1805564785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1b3b710 0x1d9a900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 >> 172.21.15.64:0/1805564785 conn(0x1b3beb0 msgr2=0x1d9e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 shutdown_connections 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 1 -- 172.21.15.64:0/1805564785 wait complete. 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f6173ebd740 20 client.4456 trim_cache size 0 max 16384 2023-09-27T23:13:51.366 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4299, 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 6.5947963610000002, 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 6, 2023-09-27T23:13:51.367 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 29, 2023-09-27T23:13:51.368 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 28, 2023-09-27T23:13:51.368 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 10.898058520999999 2023-09-27T23:13:51.368 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f614c060980 msgr2=0x7f614c062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f614c060980 0x7f614c062e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f615400c010 tx=0x7f6154017000 comp rx=0 tx=0).stop 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 msgr2=0x7f616c0c1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 0x7f616c0c1920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f616400e160 tx=0x7f6164009940 comp rx=0 tx=0).stop 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 shutdown_connections 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f614c060980 0x7f614c062e30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1550e0 0x7f616c0c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f616c06be30 0x7f616c0c1920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 --2- 172.21.15.64:0/1223476679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f616c06b320 0x7f616c14e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 >> 172.21.15.64:0/1223476679 conn(0x7f616c0684d0 msgr2=0x7f616c0b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 shutdown_connections 2023-09-27T23:13:51.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.358+0000 7f61709b2700 1 -- 172.21.15.64:0/1223476679 wait complete. 2023-09-27T23:13:51.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:13:51.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 Processor -- start 2023-09-27T23:13:51.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 -- start start 2023-09-27T23:13:51.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64f0158f20 0x7f64f01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f006dab0 0x7f64f015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f006c7c0 con 0x7f64f006df00 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f006d620 con 0x7f64f0158f20 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f6657700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f006ca10 con 0x7f64f006dab0 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64effff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64f0158f20 0x7f64f01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f4bf4700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f64f006df00 0x7f64f00aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:38508/0 (socket says 172.21.15.64:38508) 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64f4bf4700 1 -- 172.21.15.64:0/941149498 learned_addr learned my addr 172.21.15.64:0/941149498 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64e77fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f006dab0 0x7f64f015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64ef7fe700 1 -- 172.21.15.64:0/941149498 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605049332 0 0) 0x7f64f006c7c0 con 0x7f64f006df00 2023-09-27T23:13:51.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64ef7fe700 1 -- 172.21.15.64:0/941149498 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64dc003570 con 0x7f64f006df00 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64effff700 1 -- 172.21.15.64:0/941149498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f006dab0 msgr2=0x7f64f015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64effff700 1 --2- 172.21.15.64:0/941149498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f006dab0 0x7f64f015cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64effff700 1 -- 172.21.15.64:0/941149498 >> v1:172.21.15.123:6789/0 conn(0x7f64f006df00 legacy=0x7f64f00aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64ef7fe700 1 -- 172.21.15.64:0/941149498 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363719051 0 0) 0x7f64dc003570 con 0x7f64f006df00 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64effff700 1 -- 172.21.15.64:0/941149498 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f006ccf0 con 0x7f64f0158f20 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64e77fe700 1 --2- 172.21.15.64:0/941149498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f006dab0 0x7f64f015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64effff700 1 --2- 172.21.15.64:0/941149498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f0158f20 0x7f64f01592f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f64e00098e0 tx=0x7f64e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ee60bf926bb711 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64ef7fe700 1 -- 172.21.15.64:0/941149498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80017f0 con 0x7f64f0158f20 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64ef7fe700 1 -- 172.21.15.64:0/941149498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0011420 con 0x7f64f0158f20 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.490+0000 7f64ef7fe700 1 -- 172.21.15.64:0/941149498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00117a0 con 0x7f64f0158f20 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 -- 172.21.15.64:0/941149498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f0158f20 msgr2=0x7f64f01592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 --2- 172.21.15.64:0/941149498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f0158f20 0x7f64f01592f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f64e00098e0 tx=0x7f64e00091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 -- 172.21.15.64:0/941149498 shutdown_connections 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 --2- 172.21.15.64:0/941149498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f006dab0 0x7f64f015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 --2- 172.21.15.64:0/941149498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f0158f20 0x7f64f01592f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 -- 172.21.15.64:0/941149498 >> 172.21.15.64:0/941149498 conn(0x7f64f00ccb30 msgr2=0x7f64f00c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 -- 172.21.15.64:0/941149498 shutdown_connections 2023-09-27T23:13:51.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 -- 172.21.15.64:0/941149498 wait complete. 2023-09-27T23:13:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 Processor -- start 2023-09-27T23:13:51.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.494+0000 7f64f6657700 1 -- start start 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 0x7f64f016d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f006df00 0x7f64f00c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f0158f20 0x7f64f00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00b0500 con 0x7f64f006dab0 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00c4390 con 0x7f64f006df00 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00c4670 con 0x7f64f0158f20 2023-09-27T23:13:51.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 0x7f64f016d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 0x7f64f016d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57028/0 (socket says 172.21.15.64:57028) 2023-09-27T23:13:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 -- 172.21.15.64:0/1500324847 learned_addr learned my addr 172.21.15.64:0/1500324847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64e77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f006df00 0x7f64f00c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f4bf4700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f0158f20 0x7f64f00cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 -- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f0158f20 msgr2=0x7f64f00cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f0158f20 0x7f64f00cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 -- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f006df00 msgr2=0x7f64f00c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f006df00 0x7f64f00c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 -- 172.21.15.64:0/1500324847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d8009420 con 0x7f64f006dab0 2023-09-27T23:13:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64effff700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 0x7f64f016d090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f64f006a6a0 tx=0x7f64e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e001cab0 con 0x7f64f006dab0 2023-09-27T23:13:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e001cc30 con 0x7f64f006dab0 2023-09-27T23:13:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e001a450 con 0x7f64f006dab0 2023-09-27T23:13:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e0009420 con 0x7f64f006dab0 2023-09-27T23:13:51.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.498+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f00c40a0 con 0x7f64f006dab0 2023-09-27T23:13:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.502+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f00c4b70 con 0x7f64f006dab0 2023-09-27T23:13:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.502+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f64e001a860 con 0x7f64f006dab0 2023-09-27T23:13:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.502+0000 7f64ecff9700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64d0060950 0x7f64d0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.502+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f64e007e8e0 con 0x7f64f006dab0 2023-09-27T23:13:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.502+0000 7f64e77fe700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64d0060950 0x7f64d0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:51.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.502+0000 7f64e77fe700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64d0060950 0x7f64d0062e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f64f016c400 tx=0x7f64d80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.510+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f64e004e920 con 0x7f64f006dab0 2023-09-27T23:13:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64f00b02d0 con 0x7f64f006dab0 2023-09-27T23:13:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64ecff9700 1 -- 172.21.15.64:0/1500324847 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f64e0051f40 con 0x7f64f006dab0 2023-09-27T23:13:51.714 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-27T23:13:51.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64d0060950 msgr2=0x7f64d0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64d0060950 0x7f64d0062e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f64f016c400 tx=0x7f64d80091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 msgr2=0x7f64f016d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:51.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 0x7f64f016d090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f64f006a6a0 tx=0x7f64e0016000 comp rx=0 tx=0).stop 2023-09-27T23:13:51.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 shutdown_connections 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64d0060950 0x7f64d0062e00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64f0158f20 0x7f64f00cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f006df00 0x7f64f00c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 --2- 172.21.15.64:0/1500324847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64f006dab0 0x7f64f016d090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 >> 172.21.15.64:0/1500324847 conn(0x7f64f00ccb30 msgr2=0x7f64f015cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 shutdown_connections 2023-09-27T23:13:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:51.710+0000 7f64f6657700 1 -- 172.21.15.64:0/1500324847 wait complete. 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:13:51.728 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-27T23:13:51.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.729 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:13:51.733 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:13:51.733 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:13:51.733 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:13:51.734 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42552sec preferred_lft 42552sec 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:13:51.735 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:13:51.736 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-27T23:13:51.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-27T23:13:51.736 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:13:51.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.822 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:13:51.832 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:13:51.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.833 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:13:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:13:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:13:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:13:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:13:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:13:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:13:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:13:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:13:51.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.864 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:13:51.865 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:13:51.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:13:51.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:13:51.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:13:51.865 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:13:51.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.905 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:13:51.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.956 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-09-27T23:13:51.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.963 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:13:51.963 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:13:51.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-27T23:13:51.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-27T23:13:51.963 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:13:51.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-27T23:13:51.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-27T23:13:51.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-27T23:13:51.998 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:13:52.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:52.124 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:13:52.124 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:13:52.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:13:52.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2023-09-27T23:13:52.124 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:13:52.149 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-27T23:13:52.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:13:52.149 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:52.156 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-27T23:13:52.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:13:52.156 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:52.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:13:52.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:13:52.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:13:52.230 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:13:52.231 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:13:52.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:52.231 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:13:52.237 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:13:52.237 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-27T23:13:52.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:13:52.255 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:13:52.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:13:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:13:52.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:13:52.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:52.313 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:13:52.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:2023-09-27T23:13:52.322+0000 7f46a0713380 -1 init, newargv = 0x55a8a3adb310 newargc=15 2023-09-27T23:13:52.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[39752]: starting ceph client 2023-09-27T23:13:52.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[39752]: starting fuse 2023-09-27T23:13:53.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:13:53.438 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:13:53.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:13:53.496 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:13:53.497 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:13:53.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:53.498 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:13:53.505 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:13:53.505 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:13:53.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:13:53.505 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:13:53.505 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:13:53.505 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:13:53.505 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:13:53.505 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:13:53.506 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.admin")) 2023-09-27T23:13:53.507 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:13:53.588 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.admin.39752.asok 2023-09-27T23:13:53.771 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.39752.asok 2023-09-27T23:13:53.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:53.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.39752.asok status 2023-09-27T23:13:53.867 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "admin", 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "39752", 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:13:53.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4470, 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4470 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 4083345510 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:13:53.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 4083345510 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4470 192.168.0.1:0/4083345510", 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/4083345510", 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 6, 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 29, 2023-09-27T23:13:53.870 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:13:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:13:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:13:53.871 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:13:53.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:53.877 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:13:53.877 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:53.885 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:13:53.885 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:53.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:53.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:53.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:13:53.900 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-27T23:13:53.934 INFO:teuthology.orchestra.run.smithi064.stdout:# file: . 2023-09-27T23:13:53.934 INFO:teuthology.orchestra.run.smithi064.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-27T23:13:53.934 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:53.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:13:53.935 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:13:53.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-27T23:13:54.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[39752]: fuse finished with error 0 and tester_r 0 2023-09-27T23:13:54.050 INFO:teuthology.orchestra.run:waiting for 900 2023-09-27T23:13:54.116 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:13:54.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:13:54.117 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-27T23:13:54.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.242+0000 7fa69a5a9700 1 Processor -- start 2023-09-27T23:13:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.242+0000 7fa69a5a9700 1 -- start start 2023-09-27T23:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa69406b8b0 0x7fa6940c8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6941595d0 0x7fa6940afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69415a3e0 con 0x7fa694159200 2023-09-27T23:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940c8a90 con 0x7fa69406b8b0 2023-09-27T23:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa694159ab0 con 0x7fa6941595d0 2023-09-27T23:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa69406b8b0 0x7fa6940c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa69406b8b0 0x7fa6940c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57040/0 (socket says 172.21.15.64:57040) 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 -- 172.21.15.64:0/3144154683 learned_addr learned my addr 172.21.15.64:0/3144154683 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa698b46700 1 --2- 172.21.15.64:0/3144154683 >> v2:172.21.15.123:3300/0 conn(0x7fa6941595d0 0x7fa6940afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 -- 172.21.15.64:0/3144154683 >> v2:172.21.15.123:3300/0 conn(0x7fa6941595d0 msgr2=0x7fa6940afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 --2- 172.21.15.64:0/3144154683 >> v2:172.21.15.123:3300/0 conn(0x7fa6941595d0 0x7fa6940afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 -- 172.21.15.64:0/3144154683 >> v1:172.21.15.64:6789/0 conn(0x7fa694159200 legacy=0x7fa69406b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 -- 172.21.15.64:0/3144154683 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694159d90 con 0x7fa69406b8b0 2023-09-27T23:13:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa6937fe700 1 --2- 172.21.15.64:0/3144154683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69406b8b0 0x7fa6940c8550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6940b2190 tx=0x7fa688009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f598f15241c20ff server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa692ffd700 1 -- 172.21.15.64:0/3144154683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800b860 con 0x7fa69406b8b0 2023-09-27T23:13:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa692ffd700 1 -- 172.21.15.64:0/3144154683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800b9c0 con 0x7fa69406b8b0 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa692ffd700 1 -- 172.21.15.64:0/3144154683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800bd60 con 0x7fa69406b8b0 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3144154683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69406b8b0 msgr2=0x7fa6940c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3144154683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69406b8b0 0x7fa6940c8550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6940b2190 tx=0x7fa688009250 comp rx=0 tx=0).stop 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3144154683 shutdown_connections 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3144154683 >> v2:172.21.15.123:3300/0 conn(0x7fa6941595d0 0x7fa6940afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3144154683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69406b8b0 0x7fa6940c8550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3144154683 >> 172.21.15.64:0/3144154683 conn(0x7fa694067410 msgr2=0x7fa6940677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3144154683 shutdown_connections 2023-09-27T23:13:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.246+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3144154683 wait complete. 2023-09-27T23:13:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 Processor -- start 2023-09-27T23:13:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 -- start start 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69406b8b0 0x7fa69415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa694159200 0x7fa6941687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 0x7fa6940afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69414eae0 con 0x7fa6941595d0 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415b1b0 con 0x7fa69406b8b0 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa69a5a9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415b450 con 0x7fa694159200 2023-09-27T23:13:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 0x7fa6940afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 0x7fa6940afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57052/0 (socket says 172.21.15.64:57052) 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 -- 172.21.15.64:0/3431689704 learned_addr learned my addr 172.21.15.64:0/3431689704 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa693fff700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69406b8b0 0x7fa69415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa6937fe700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa694159200 0x7fa6941687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 -- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa694159200 msgr2=0x7fa6941687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa694159200 0x7fa6941687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 -- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69406b8b0 msgr2=0x7fa69415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69406b8b0 0x7fa69415ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 -- 172.21.15.64:0/3431689704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6880094f0 con 0x7fa6941595d0 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa6937fe700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa694159200 0x7fa6941687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa693fff700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69406b8b0 0x7fa69415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa698b46700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 0x7fa6940afe40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa68c007f20 tx=0x7fa68c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.250+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00eca0 con 0x7fa6941595d0 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c00ee20 con 0x7fa6941595d0 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c014670 con 0x7fa6941595d0 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69415b730 con 0x7fa6941595d0 2023-09-27T23:13:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69415bcf0 con 0x7fa6941595d0 2023-09-27T23:13:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fa68c0149e0 con 0x7fa6941595d0 2023-09-27T23:13:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa694071120 con 0x7fa6941595d0 2023-09-27T23:13:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa67bfff700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060950 0x7fa680062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa68c07d8e0 con 0x7fa6941595d0 2023-09-27T23:13:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.254+0000 7fa693fff700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060950 0x7fa680062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.258+0000 7fa693fff700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060950 0x7fa680062e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa684007fd0 tx=0x7fa684009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.266+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa68c04d870 con 0x7fa6941595d0 2023-09-27T23:13:54.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.422+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa69406bf80 con 0x7fa6941595d0 2023-09-27T23:13:54.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.422+0000 7fa67bfff700 1 -- 172.21.15.64:0/3431689704 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fa68c050e90 con 0x7fa6941595d0 2023-09-27T23:13:54.429 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:54.429 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060950 msgr2=0x7fa680062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060950 0x7fa680062e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa684007fd0 tx=0x7fa684009040 comp rx=0 tx=0).stop 2023-09-27T23:13:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 msgr2=0x7fa6940afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 0x7fa6940afe40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa68c007f20 tx=0x7fa68c009860 comp rx=0 tx=0).stop 2023-09-27T23:13:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 shutdown_connections 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060950 0x7fa680062e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6941595d0 0x7fa6940afe40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa694159200 0x7fa6941687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 --2- 172.21.15.64:0/3431689704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69406b8b0 0x7fa69415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 >> 172.21.15.64:0/3431689704 conn(0x7fa694067410 msgr2=0x7fa6940cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 shutdown_connections 2023-09-27T23:13:54.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.426+0000 7fa69a5a9700 1 -- 172.21.15.64:0/3431689704 wait complete. 2023-09-27T23:13:54.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 Processor -- start 2023-09-27T23:13:54.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 -- start start 2023-09-27T23:13:54.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc158f70 0x7effbc067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effbc159340 0x7effbc0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc068c70 con 0x7effbc069100 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc068060 con 0x7effbc159340 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc4682700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc068340 con 0x7effbc158f70 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc2c1f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effbc159340 0x7effbc0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc1c1d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7effbc069100 0x7effbc0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48768/0 (socket says 172.21.15.64:48768) 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc1c1d700 1 -- 172.21.15.64:0/2200326242 learned_addr learned my addr 172.21.15.64:0/2200326242 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc241e700 1 --2- 172.21.15.64:0/2200326242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc158f70 0x7effbc067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc141c700 1 -- 172.21.15.64:0/2200326242 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171664663 0 0) 0x7effbc068c70 con 0x7effbc069100 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc141c700 1 -- 172.21.15.64:0/2200326242 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003570 con 0x7effbc069100 2023-09-27T23:13:54.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc2c1f700 1 -- 172.21.15.64:0/2200326242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc158f70 msgr2=0x7effbc067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:54.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc2c1f700 1 --2- 172.21.15.64:0/2200326242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc158f70 0x7effbc067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc2c1f700 1 -- 172.21.15.64:0/2200326242 >> v1:172.21.15.64:6789/0 conn(0x7effbc069100 legacy=0x7effbc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.546+0000 7effc2c1f700 1 -- 172.21.15.64:0/2200326242 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc068620 con 0x7effbc159340 2023-09-27T23:13:54.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc2c1f700 1 --2- 172.21.15.64:0/2200326242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc159340 0x7effbc0cb3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effb40098e0 tx=0x7effb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b7e2cc4e38428b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc141c700 1 -- 172.21.15.64:0/2200326242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4011630 con 0x7effbc159340 2023-09-27T23:13:54.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc141c700 1 -- 172.21.15.64:0/2200326242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb4011790 con 0x7effbc159340 2023-09-27T23:13:54.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc141c700 1 -- 172.21.15.64:0/2200326242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4011b10 con 0x7effbc159340 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 -- 172.21.15.64:0/2200326242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc159340 msgr2=0x7effbc0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 --2- 172.21.15.64:0/2200326242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc159340 0x7effbc0cb3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effb40098e0 tx=0x7effb40091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 -- 172.21.15.64:0/2200326242 shutdown_connections 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 --2- 172.21.15.64:0/2200326242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc159340 0x7effbc0cb3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 --2- 172.21.15.64:0/2200326242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc158f70 0x7effbc067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 -- 172.21.15.64:0/2200326242 >> 172.21.15.64:0/2200326242 conn(0x7effbc0ccc40 msgr2=0x7effbc0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 -- 172.21.15.64:0/2200326242 shutdown_connections 2023-09-27T23:13:54.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 -- 172.21.15.64:0/2200326242 wait complete. 2023-09-27T23:13:54.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 Processor -- start 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.550+0000 7effc4682700 1 -- start start 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc069100 0x7effbc151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 0x7effbc144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc159340 0x7effbc1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc1567b0 con 0x7effbc158f70 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc151d60 con 0x7effbc069100 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc152ba0 con 0x7effbc159340 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 0x7effbc144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 0x7effbc144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57074/0 (socket says 172.21.15.64:57074) 2023-09-27T23:13:54.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 -- 172.21.15.64:0/1045748785 learned_addr learned my addr 172.21.15.64:0/1045748785 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:54.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc2c1f700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc159340 0x7effbc1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc241e700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc069100 0x7effbc151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 -- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc159340 msgr2=0x7effbc1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc159340 0x7effbc1540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 -- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc069100 msgr2=0x7effbc151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc069100 0x7effbc151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 -- 172.21.15.64:0/1045748785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4009420 con 0x7effbc158f70 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc1c1d700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 0x7effbc144d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7effb000c7c0 tx=0x7effb0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0005cd0 con 0x7effbc158f70 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb0005e30 con 0x7effbc158f70 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc151f90 con 0x7effbc158f70 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0007320 con 0x7effbc158f70 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc152550 con 0x7effbc158f70 2023-09-27T23:13:54.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc152d50 con 0x7effbc158f70 2023-09-27T23:13:54.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.554+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7effb0007690 con 0x7effbc158f70 2023-09-27T23:13:54.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.558+0000 7effba7fc700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7effa8060980 0x7effa8062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.558+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7effb007c770 con 0x7effbc158f70 2023-09-27T23:13:54.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.558+0000 7effc241e700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7effa8060980 0x7effa8062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.558+0000 7effc241e700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7effa8060980 0x7effa8062e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effac00f570 tx=0x7effac00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.562+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7effbc152d50 con 0x7effbc158f70 2023-09-27T23:13:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.726+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effbc152d10 con 0x7effbc158f70 2023-09-27T23:13:54.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effba7fc700 1 -- 172.21.15.64:0/1045748785 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7effb004c700 con 0x7effbc158f70 2023-09-27T23:13:54.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:54.733 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:54.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7effa8060980 msgr2=0x7effa8062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7effa8060980 0x7effa8062e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effac00f570 tx=0x7effac00b040 comp rx=0 tx=0).stop 2023-09-27T23:13:54.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 msgr2=0x7effbc144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 0x7effbc144d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7effb000c7c0 tx=0x7effb0007e00 comp rx=0 tx=0).stop 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 shutdown_connections 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7effa8060980 0x7effa8062e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effbc159340 0x7effbc1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc158f70 0x7effbc144d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 --2- 172.21.15.64:0/1045748785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effbc069100 0x7effbc151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 >> 172.21.15.64:0/1045748785 conn(0x7effbc0ccc40 msgr2=0x7effbc0b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 shutdown_connections 2023-09-27T23:13:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.730+0000 7effc4682700 1 -- 172.21.15.64:0/1045748785 wait complete. 2023-09-27T23:13:54.747 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:13:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 Processor -- start 2023-09-27T23:13:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 -- start start 2023-09-27T23:13:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b540caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b5406aeb0 0x7f4b54159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b541595a0 0x7f4b54159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5406c850 con 0x7f4b541595a0 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5406b470 con 0x7f4b5406aeb0 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5af93700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5406b750 con 0x7f4b54158c90 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5a792700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b541595a0 0x7f4b54159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5a792700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b541595a0 0x7f4b54159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44184/0 (socket says 172.21.15.64:44184) 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b5a792700 1 -- 172.21.15.64:0/2902922128 learned_addr learned my addr 172.21.15.64:0/2902922128 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b59f91700 1 --2- 172.21.15.64:0/2902922128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b540caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.854+0000 7f4b59790700 1 --2- 172.21.15.64:0/2902922128 >> v2:172.21.15.123:3300/0 conn(0x7f4b5406aeb0 0x7f4b54159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5a792700 1 -- 172.21.15.64:0/2902922128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 msgr2=0x7f4b540caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5a792700 1 --2- 172.21.15.64:0/2902922128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b540caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5a792700 1 -- 172.21.15.64:0/2902922128 >> v2:172.21.15.123:3300/0 conn(0x7f4b5406aeb0 msgr2=0x7f4b54159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5a792700 1 --2- 172.21.15.64:0/2902922128 >> v2:172.21.15.123:3300/0 conn(0x7f4b5406aeb0 0x7f4b54159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5a792700 1 -- 172.21.15.64:0/2902922128 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5406ba30 con 0x7f4b541595a0 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5a792700 1 --2- 172.21.15.64:0/2902922128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b541595a0 0x7f4b54159a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b5406a000 tx=0x7f4b4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4c0297fd4e404b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b58f8f700 1 -- 172.21.15.64:0/2902922128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00e070 con 0x7f4b541595a0 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b58f8f700 1 -- 172.21.15.64:0/2902922128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00b970 con 0x7f4b541595a0 2023-09-27T23:13:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b58f8f700 1 -- 172.21.15.64:0/2902922128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0092e0 con 0x7f4b541595a0 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 -- 172.21.15.64:0/2902922128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b541595a0 msgr2=0x7f4b54159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 --2- 172.21.15.64:0/2902922128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b541595a0 0x7f4b54159a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b5406a000 tx=0x7f4b4c009840 comp rx=0 tx=0).stop 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 -- 172.21.15.64:0/2902922128 shutdown_connections 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 --2- 172.21.15.64:0/2902922128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b541595a0 0x7f4b54159a20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 --2- 172.21.15.64:0/2902922128 >> v2:172.21.15.123:3300/0 conn(0x7f4b5406aeb0 0x7f4b54159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 --2- 172.21.15.64:0/2902922128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b540caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 -- 172.21.15.64:0/2902922128 >> 172.21.15.64:0/2902922128 conn(0x7f4b54067bb0 msgr2=0x7f4b540cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 -- 172.21.15.64:0/2902922128 shutdown_connections 2023-09-27T23:13:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 -- 172.21.15.64:0/2902922128 wait complete. 2023-09-27T23:13:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 Processor -- start 2023-09-27T23:13:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.858+0000 7f4b5af93700 1 -- start start 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 0x7f4b541446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b5415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b541595a0 0x7f4b5415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54169f20 con 0x7f4b5406aeb0 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415a4f0 con 0x7f4b541595a0 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415a7d0 con 0x7f4b54158c90 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 0x7f4b541446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 0x7f4b541446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44198/0 (socket says 172.21.15.64:44198) 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 -- 172.21.15.64:0/3891477033 learned_addr learned my addr 172.21.15.64:0/3891477033 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59790700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b5415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5a792700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b541595a0 0x7f4b5415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 -- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 msgr2=0x7f4b5415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b5415b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 -- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b541595a0 msgr2=0x7f4b5415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b541595a0 0x7f4b5415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 -- 172.21.15.64:0/3891477033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009ae0 con 0x7f4b5406aeb0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59790700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b5415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5a792700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b541595a0 0x7f4b5415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b59f91700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 0x7f4b541446f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ec80 tx=0x7f4b500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000c9d0 con 0x7f4b5406aeb0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5000cb30 con 0x7f4b5406aeb0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50013470 con 0x7f4b5406aeb0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5415aab0 con 0x7f4b5406aeb0 2023-09-27T23:13:54.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.862+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5415ccb0 con 0x7f4b5406aeb0 2023-09-27T23:13:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.866+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5415af30 con 0x7f4b5406aeb0 2023-09-27T23:13:54.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.866+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4b50013820 con 0x7f4b5406aeb0 2023-09-27T23:13:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.866+0000 7f4b49ffb700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b38060950 0x7f4b38062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.866+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4b5007d550 con 0x7f4b5406aeb0 2023-09-27T23:13:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.866+0000 7f4b59790700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b38060950 0x7f4b38062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.866+0000 7f4b59790700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b38060950 0x7f4b38062e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b44006f00 tx=0x7f4b44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:54.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:54.874+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4b5004d4e0 con 0x7f4b5406aeb0 2023-09-27T23:13:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.078+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b5404be40 con 0x7f4b5406aeb0 2023-09-27T23:13:55.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.850+0000 7f4b49ffb700 1 -- 172.21.15.64:0/3891477033 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b50056070 con 0x7f4b5406aeb0 2023-09-27T23:13:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b38060950 msgr2=0x7f4b38062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b38060950 0x7f4b38062e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b44006f00 tx=0x7f4b44008040 comp rx=0 tx=0).stop 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 msgr2=0x7f4b541446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 0x7f4b541446f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ec80 tx=0x7f4b500091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 shutdown_connections 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b38060950 0x7f4b38062e00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b541595a0 0x7f4b5415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b54158c90 0x7f4b5415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 --2- 172.21.15.64:0/3891477033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5406aeb0 0x7f4b541446f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 >> 172.21.15.64:0/3891477033 conn(0x7f4b54067bb0 msgr2=0x7f4b541599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 shutdown_connections 2023-09-27T23:13:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.854+0000 7f4b5af93700 1 -- 172.21.15.64:0/3891477033 wait complete. 2023-09-27T23:13:55.874 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-09-27T23:13:55.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:13:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 Processor -- start 2023-09-27T23:13:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 -- start start 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f723806aeb0 0x7f7238159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72381595a0 0x7f7238159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723806c850 con 0x7f7238158c90 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380cb650 con 0x7f72381595a0 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723806b470 con 0x7f723806aeb0 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72381595a0 0x7f7238159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72381595a0 0x7f7238159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44202/0 (socket says 172.21.15.64:44202) 2023-09-27T23:13:55.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 -- 172.21.15.64:0/3289821161 learned_addr learned my addr 172.21.15.64:0/3289821161 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:55.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7235d9b700 1 --2- 172.21.15.64:0/3289821161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f723806aeb0 0x7f7238159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 -- 172.21.15.64:0/3289821161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f723806aeb0 msgr2=0x7f7238159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 --2- 172.21.15.64:0/3289821161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f723806aeb0 0x7f7238159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 -- 172.21.15.64:0/3289821161 >> v1:172.21.15.64:6789/0 conn(0x7f7238158c90 legacy=0x7f72380caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 -- 172.21.15.64:0/3289821161 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723806b750 con 0x7f72381595a0 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f7236d9d700 1 --2- 172.21.15.64:0/3289821161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72381595a0 0x7f7238159a20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7230009f20 tx=0x7f7230009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2949497a6ff3ea9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723559a700 1 -- 172.21.15.64:0/3289821161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000e070 con 0x7f72381595a0 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723559a700 1 -- 172.21.15.64:0/3289821161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723000b9d0 con 0x7f72381595a0 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.990+0000 7f723559a700 1 -- 172.21.15.64:0/3289821161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72300092e0 con 0x7f72381595a0 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 -- 172.21.15.64:0/3289821161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72381595a0 msgr2=0x7f7238159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 --2- 172.21.15.64:0/3289821161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72381595a0 0x7f7238159a20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7230009f20 tx=0x7f7230009840 comp rx=0 tx=0).stop 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 -- 172.21.15.64:0/3289821161 shutdown_connections 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 --2- 172.21.15.64:0/3289821161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72381595a0 0x7f7238159a20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 --2- 172.21.15.64:0/3289821161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f723806aeb0 0x7f7238159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 -- 172.21.15.64:0/3289821161 >> 172.21.15.64:0/3289821161 conn(0x7f7238067bb0 msgr2=0x7f72380cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 -- 172.21.15.64:0/3289821161 shutdown_connections 2023-09-27T23:13:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 -- 172.21.15.64:0/3289821161 wait complete. 2023-09-27T23:13:56.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 Processor -- start 2023-09-27T23:13:56.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.994+0000 7f723759e700 1 -- start start 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 0x7f723814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7238158c90 0x7f7238157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72381595a0 0x7f723816c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c9d30 con 0x7f723806aeb0 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c91b0 con 0x7f72381595a0 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c9490 con 0x7f7238158c90 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 0x7f723814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 0x7f723814b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44216/0 (socket says 172.21.15.64:44216) 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 -- 172.21.15.64:0/2168496210 learned_addr learned my addr 172.21.15.64:0/2168496210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f7235d9b700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7238158c90 0x7f7238157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f7236d9d700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72381595a0 0x7f723816c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 -- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7238158c90 msgr2=0x7f7238157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7238158c90 0x7f7238157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 -- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72381595a0 msgr2=0x7f723816c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72381595a0 0x7f723816c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 -- 172.21.15.64:0/2168496210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230009ae0 con 0x7f723806aeb0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723659c700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 0x7f723814b430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f722c00c590 tx=0x7f722c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c011590 con 0x7f723806aeb0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c011710 con 0x7f723806aeb0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380c9770 con 0x7f723806aeb0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c011ad0 con 0x7f723806aeb0 2023-09-27T23:13:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:55.998+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380c8e00 con 0x7f723806aeb0 2023-09-27T23:13:56.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.002+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723804be40 con 0x7f723806aeb0 2023-09-27T23:13:56.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.002+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f722c01b3b0 con 0x7f723806aeb0 2023-09-27T23:13:56.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.002+0000 7f72267fc700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7218060950 0x7f7218062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.002+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f722c07c5d0 con 0x7f723806aeb0 2023-09-27T23:13:56.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.002+0000 7f7235d9b700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7218060950 0x7f7218062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.002+0000 7f7235d9b700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7218060950 0x7f7218062e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7238166cf0 tx=0x7f72280091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:56.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.010+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f722c04c560 con 0x7f723806aeb0 2023-09-27T23:13:56.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.206+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f723806ba30 con 0x7f723806aeb0 2023-09-27T23:13:56.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.854+0000 7f72267fc700 1 -- 172.21.15.64:0/2168496210 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f722c0170c0 con 0x7f723806aeb0 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7218060950 msgr2=0x7f7218062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7218060950 0x7f7218062e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7238166cf0 tx=0x7f72280091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 msgr2=0x7f723814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 0x7f723814b430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f722c00c590 tx=0x7f722c00c8a0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 shutdown_connections 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7218060950 0x7f7218062e00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72381595a0 0x7f723816c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7238158c90 0x7f7238157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 --2- 172.21.15.64:0/2168496210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f723806aeb0 0x7f723814b430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 >> 172.21.15.64:0/2168496210 conn(0x7f7238067bb0 msgr2=0x7f72380b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 shutdown_connections 2023-09-27T23:13:56.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.858+0000 7f723759e700 1 -- 172.21.15.64:0/2168496210 wait complete. 2023-09-27T23:13:56.879 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:13:56.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:13:56.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 Processor -- start 2023-09-27T23:13:56.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 -- start start 2023-09-27T23:13:56.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6c7413f890 0x7f6c740aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c740c2980 0x7f6c740c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c740c1a80 0x7f6c740680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:56.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7406c9f0 con 0x7f6c740c1a80 2023-09-27T23:13:56.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c740c7f20 con 0x7f6c7413f890 2023-09-27T23:13:56.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c78ac7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c740c8190 con 0x7f6c740c2980 2023-09-27T23:13:56.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c72d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c740c1a80 0x7f6c740680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c72d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c740c1a80 0x7f6c740680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44222/0 (socket says 172.21.15.64:44222) 2023-09-27T23:13:56.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c7259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6c7413f890 0x7f6c740aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c72d9d700 1 -- 172.21.15.64:0/3212853219 learned_addr learned my addr 172.21.15.64:0/3212853219 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:56.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.990+0000 7f6c6bfff700 1 --2- 172.21.15.64:0/3212853219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c740c2980 0x7f6c740c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:56.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c72d9d700 1 -- 172.21.15.64:0/3212853219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c740c2980 msgr2=0x7f6c740c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c72d9d700 1 --2- 172.21.15.64:0/3212853219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c740c2980 0x7f6c740c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c72d9d700 1 -- 172.21.15.64:0/3212853219 >> v2:172.21.15.123:3300/0 conn(0x7f6c7413f890 msgr2=0x7f6c740aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c72d9d700 1 --2- 172.21.15.64:0/3212853219 >> v2:172.21.15.123:3300/0 conn(0x7f6c7413f890 0x7f6c740aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c72d9d700 1 -- 172.21.15.64:0/3212853219 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c740c8470 con 0x7f6c740c1a80 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c7259c700 1 --2- 172.21.15.64:0/3212853219 >> v2:172.21.15.123:3300/0 conn(0x7f6c7413f890 0x7f6c740aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c72d9d700 1 --2- 172.21.15.64:0/3212853219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c1a80 0x7f6c740680f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c74068c90 tx=0x7f6c6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856a5374f9735337 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c71d9b700 1 -- 172.21.15.64:0/3212853219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00e070 con 0x7f6c740c1a80 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c71d9b700 1 -- 172.21.15.64:0/3212853219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c00b970 con 0x7f6c740c1a80 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c71d9b700 1 -- 172.21.15.64:0/3212853219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0092e0 con 0x7f6c740c1a80 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- 172.21.15.64:0/3212853219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c1a80 msgr2=0x7f6c740680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:56.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/3212853219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c1a80 0x7f6c740680f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c74068c90 tx=0x7f6c6c009840 comp rx=0 tx=0).stop 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- 172.21.15.64:0/3212853219 shutdown_connections 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/3212853219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c1a80 0x7f6c740680f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/3212853219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c740c2980 0x7f6c740c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/3212853219 >> v2:172.21.15.123:3300/0 conn(0x7f6c7413f890 0x7f6c740aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- 172.21.15.64:0/3212853219 >> 172.21.15.64:0/3212853219 conn(0x7f6c740c8850 msgr2=0x7f6c740c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- 172.21.15.64:0/3212853219 shutdown_connections 2023-09-27T23:13:57.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- 172.21.15.64:0/3212853219 wait complete. 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 Processor -- start 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- start start 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 0x7f6c7413e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c7413f890 0x7f6c7415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c7415c480 0x7f6c740670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c740c4880 con 0x7f6c740c2980 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7413ed60 con 0x7f6c7413f890 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c78ac7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7415c990 con 0x7f6c7415c480 2023-09-27T23:13:57.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c7259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 0x7f6c7413e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c7259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 0x7f6c7413e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44230/0 (socket says 172.21.15.64:44230) 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.994+0000 7f6c7259c700 1 -- 172.21.15.64:0/2569751972 learned_addr learned my addr 172.21.15.64:0/2569751972 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c72d9d700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c7415c480 0x7f6c740670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6bfff700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c7413f890 0x7f6c7415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c7259c700 1 -- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c7415c480 msgr2=0x7f6c740670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c7259c700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c7415c480 0x7f6c740670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c7259c700 1 -- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c7413f890 msgr2=0x7f6c7415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c7259c700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c7413f890 0x7f6c7415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c7259c700 1 -- 172.21.15.64:0/2569751972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009420 con 0x7f6c740c2980 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c7259c700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 0x7f6c7413e820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00ec80 tx=0x7f6c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00c850 con 0x7f6c740c2980 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c00c9b0 con 0x7f6c740c2980 2023-09-27T23:13:57.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c009ae0 con 0x7f6c740c2980 2023-09-27T23:13:57.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00cd30 con 0x7f6c740c2980 2023-09-27T23:13:57.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7415cf50 con 0x7f6c740c2980 2023-09-27T23:13:57.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7415d170 con 0x7f6c740c2980 2023-09-27T23:13:57.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6c5c013680 con 0x7f6c740c2980 2023-09-27T23:13:57.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6a7fc700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6c54060980 0x7f6c54062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6c5c07d370 con 0x7f6c740c2980 2023-09-27T23:13:57.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:56.998+0000 7f6c6bfff700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6c54060980 0x7f6c54062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.002+0000 7f6c6bfff700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6c54060980 0x7f6c54062e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c740680c0 tx=0x7f6c640091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.010+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6c5c050d90 con 0x7f6c740c2980 2023-09-27T23:13:57.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.210+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c74071120 con 0x7f6c740c2980 2023-09-27T23:13:57.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.210+0000 7f6c6a7fc700 1 -- 172.21.15.64:0/2569751972 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f6c5c050920 con 0x7f6c740c2980 2023-09-27T23:13:57.215 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:57.215 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:13:57.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6c54060980 msgr2=0x7f6c54062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6c54060980 0x7f6c54062e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c740680c0 tx=0x7f6c640091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 msgr2=0x7f6c7413e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 0x7f6c7413e820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00ec80 tx=0x7f6c5c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 shutdown_connections 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6c54060980 0x7f6c54062e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c7415c480 0x7f6c740670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c7413f890 0x7f6c7415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 --2- 172.21.15.64:0/2569751972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c740c2980 0x7f6c7413e820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 >> 172.21.15.64:0/2569751972 conn(0x7f6c740c8850 msgr2=0x7f6c740adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 shutdown_connections 2023-09-27T23:13:57.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.214+0000 7f6c78ac7700 1 -- 172.21.15.64:0/2569751972 wait complete. 2023-09-27T23:13:57.219 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:13:57.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:57.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.326+0000 7f47e359e700 1 Processor -- start 2023-09-27T23:13:57.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.326+0000 7f47e359e700 1 -- start start 2023-09-27T23:13:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47e4158e70 0x7f47e40afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e40ca5a0 con 0x7f47e40cc470 2023-09-27T23:13:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e406dac0 con 0x7f47e406a540 2023-09-27T23:13:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e40b0260 con 0x7f47e4158e70 2023-09-27T23:13:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e2d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f47e40cc470 0x7f47e40cc890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33980/0 (socket says 172.21.15.64:33980) 2023-09-27T23:13:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e2d9d700 1 -- 172.21.15.64:0/2555478596 learned_addr learned my addr 172.21.15.64:0/2555478596 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:57.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e259c700 1 --2- 172.21.15.64:0/2555478596 >> v2:172.21.15.123:3300/0 conn(0x7f47e4158e70 0x7f47e40afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e159a700 1 -- 172.21.15.64:0/2555478596 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835496299 0 0) 0x7f47e40ca5a0 con 0x7f47e40cc470 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e159a700 1 -- 172.21.15.64:0/2555478596 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d0003570 con 0x7f47e40cc470 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e259c700 1 -- 172.21.15.64:0/2555478596 >> v1:172.21.15.123:6789/0 conn(0x7f47e406a540 legacy=0x7f47e406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e259c700 1 -- 172.21.15.64:0/2555478596 >> v1:172.21.15.64:6789/0 conn(0x7f47e40cc470 legacy=0x7f47e40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e259c700 1 -- 172.21.15.64:0/2555478596 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e406d220 con 0x7f47e4158e70 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e259c700 1 --2- 172.21.15.64:0/2555478596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e4158e70 0x7f47e40afd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0098e0 tx=0x7f47cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2da0a6d46694dd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e159a700 1 -- 172.21.15.64:0/2555478596 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00b640 con 0x7f47e4158e70 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e159a700 1 -- 172.21.15.64:0/2555478596 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc00b7a0 con 0x7f47e4158e70 2023-09-27T23:13:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.330+0000 7f47e159a700 1 -- 172.21.15.64:0/2555478596 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00bb40 con 0x7f47e4158e70 2023-09-27T23:13:57.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 -- 172.21.15.64:0/2555478596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e4158e70 msgr2=0x7f47e40afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 --2- 172.21.15.64:0/2555478596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e4158e70 0x7f47e40afd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0098e0 tx=0x7f47cc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 -- 172.21.15.64:0/2555478596 shutdown_connections 2023-09-27T23:13:57.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 --2- 172.21.15.64:0/2555478596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e4158e70 0x7f47e40afd20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 -- 172.21.15.64:0/2555478596 >> 172.21.15.64:0/2555478596 conn(0x7f47e40670e0 msgr2=0x7f47e40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 -- 172.21.15.64:0/2555478596 shutdown_connections 2023-09-27T23:13:57.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 -- 172.21.15.64:0/2555478596 wait complete. 2023-09-27T23:13:57.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 Processor -- start 2023-09-27T23:13:57.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.334+0000 7f47e359e700 1 -- start start 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e406a540 0x7f47e4156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 0x7f47e40b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e4158e70 0x7f47e40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40c9130 con 0x7f47e40cc470 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40c9410 con 0x7f47e406a540 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40c96f0 con 0x7f47e4158e70 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 0x7f47e40b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 0x7f47e40b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44244/0 (socket says 172.21.15.64:44244) 2023-09-27T23:13:57.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 -- 172.21.15.64:0/3347951311 learned_addr learned my addr 172.21.15.64:0/3347951311 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:57.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e2d9d700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e4158e70 0x7f47e40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e259c700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e406a540 0x7f47e4156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 -- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e4158e70 msgr2=0x7f47e40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e4158e70 0x7f47e40b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 -- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e406a540 msgr2=0x7f47e4156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e406a540 0x7f47e4156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 -- 172.21.15.64:0/3347951311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009420 con 0x7f47e40cc470 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e1d9b700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 0x7f47e40b3eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47d400cb10 tx=0x7f47d40017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400ce20 con 0x7f47e40cc470 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400a7e0 con 0x7f47e40cc470 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400aba0 con 0x7f47e40cc470 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e40c99d0 con 0x7f47e40cc470 2023-09-27T23:13:57.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.338+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e4156d20 con 0x7f47e40cc470 2023-09-27T23:13:57.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.342+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e404be40 con 0x7f47e40cc470 2023-09-27T23:13:57.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.342+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f47d4003030 con 0x7f47e40cc470 2023-09-27T23:13:57.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.342+0000 7f47da7fc700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c4060950 0x7f47c4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.342+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f47d407cfa0 con 0x7f47e40cc470 2023-09-27T23:13:57.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.342+0000 7f47e259c700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c4060950 0x7f47c4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.346+0000 7f47e259c700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c4060950 0x7f47c4062e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0093f0 tx=0x7f47cc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.350+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f47d404cf30 con 0x7f47e40cc470 2023-09-27T23:13:57.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.510+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e4156d20 con 0x7f47e40cc470 2023-09-27T23:13:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47da7fc700 1 -- 172.21.15.64:0/3347951311 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f47e4156d20 con 0x7f47e40cc470 2023-09-27T23:13:57.518 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:57.518 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c4060950 msgr2=0x7f47c4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c4060950 0x7f47c4062e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0093f0 tx=0x7f47cc017000 comp rx=0 tx=0).stop 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 msgr2=0x7f47e40b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 0x7f47e40b3eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47d400cb10 tx=0x7f47d40017b0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 shutdown_connections 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c4060950 0x7f47c4062e00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e4158e70 0x7f47e40b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e40cc470 0x7f47e40b3eb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 --2- 172.21.15.64:0/3347951311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e406a540 0x7f47e4156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 >> 172.21.15.64:0/3347951311 conn(0x7f47e40670e0 msgr2=0x7f47e40cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 shutdown_connections 2023-09-27T23:13:57.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.514+0000 7f47e359e700 1 -- 172.21.15.64:0/3347951311 wait complete. 2023-09-27T23:13:57.532 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:13:57.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:13:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.630+0000 7f6365803700 1 Processor -- start 2023-09-27T23:13:57.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.630+0000 7f6365803700 1 -- start start 2023-09-27T23:13:57.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 0x7f63600ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636006c7c0 con 0x7f636006a9d0 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636006cd80 con 0x7f636006a580 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636015b680 con 0x7f63600b1410 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365002700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f636006a9d0 0x7f636015af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33996/0 (socket says 172.21.15.64:33996) 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365002700 1 -- 172.21.15.64:0/3379322965 learned_addr learned my addr 172.21.15.64:0/3379322965 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6364801700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 0x7f63600ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993201975 0 0) 0x7f636006c7c0 con 0x7f636006a9d0 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6348003570 con 0x7f636006a9d0 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722014750 0 0) 0x7f6348003570 con 0x7f636006a9d0 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 msgr2=0x7f63600ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 0x7f63600ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 >> v1:172.21.15.123:6789/0 conn(0x7f636006a580 legacy=0x7f636015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636006b8f0 con 0x7f636006a9d0 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f6358001470 con 0x7f636006a9d0 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 >> v1:172.21.15.64:6789/0 conn(0x7f636006a9d0 legacy=0x7f636015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 0x7f6348003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6348004010 0x7f6348004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004d50 0x7f63480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636015b680 con 0x7f6348004010 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634800bbf0 con 0x7f6348004d50 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634800bdb0 con 0x7f63600b1410 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6348004010 0x7f6348004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365002700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004d50 0x7f63480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6364801700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 0x7f6348003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 -- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 msgr2=0x7f6348003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600b1410 0x7f6348003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 -- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004d50 msgr2=0x7f63480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004d50 0x7f63480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 -- 172.21.15.64:0/3379322965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348003570 con 0x7f6348004010 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365002700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004d50 0x7f63480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635ffff700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6348004010 0x7f6348004810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6354001f90 tx=0x7f6354001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0b750984366cdc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354001950 con 0x7f6348004010 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635400c8b0 con 0x7f6348004010 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f635f7fe700 1 -- 172.21.15.64:0/3379322965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400cc50 con 0x7f6348004010 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- 172.21.15.64:0/3379322965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6348004010 msgr2=0x7f6348004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 --2- 172.21.15.64:0/3379322965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6348004010 0x7f6348004810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6354001f90 tx=0x7f6354001640 comp rx=0 tx=0).stop 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365002700 1 -- 172.21.15.64:0/3379322965 reap_dead start 2023-09-27T23:13:57.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- 172.21.15.64:0/3379322965 shutdown_connections 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- 172.21.15.64:0/3379322965 >> 172.21.15.64:0/3379322965 conn(0x7f6360067bb0 msgr2=0x7f63600cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- 172.21.15.64:0/3379322965 shutdown_connections 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.634+0000 7f6365803700 1 -- 172.21.15.64:0/3379322965 wait complete. 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 Processor -- start 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 -- start start 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004010 0x7f63600c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600c3ad0 0x7f63600c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 0x7f63600c1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c0860 con 0x7f63600c4ff0 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c5490 con 0x7f6348004010 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c0a40 con 0x7f63600c3ad0 2023-09-27T23:13:57.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 0x7f63600c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 0x7f63600c1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44262/0 (socket says 172.21.15.64:44262) 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 -- 172.21.15.64:0/2133189514 learned_addr learned my addr 172.21.15.64:0/2133189514 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f635ffff700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600c3ad0 0x7f63600c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6364801700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004010 0x7f63600c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 -- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600c3ad0 msgr2=0x7f63600c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600c3ad0 0x7f63600c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 -- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004010 msgr2=0x7f63600c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004010 0x7f63600c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 -- 172.21.15.64:0/2133189514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600c0d20 con 0x7f63600c4ff0 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6364801700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004010 0x7f63600c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365002700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 0x7f63600c1800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c00 tx=0x7f6358001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800c600 con 0x7f63600c4ff0 2023-09-27T23:13:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800c760 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63600c1000 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800cb20 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.638+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63600c27e0 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.642+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63600c1500 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.642+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f63580133e0 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.642+0000 7f635cff9700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f634c060950 0x7f634c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.642+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f635807caf0 con 0x7f63600c4ff0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.642+0000 7f6364801700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f634c060950 0x7f634c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.642+0000 7f6364801700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f634c060950 0x7f634c062e00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6350009de0 tx=0x7f6350008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.650+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f635804cb00 con 0x7f63600c4ff0 2023-09-27T23:13:57.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.814+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63600c0bb0 con 0x7f634c060950 2023-09-27T23:13:57.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.830+0000 7f635cff9700 1 -- 172.21.15.64:0/2133189514 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+364820 (secure 0 0 0) 0x7f63600c0bb0 con 0x7f634c060950 2023-09-27T23:13:57.840 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f634c060950 msgr2=0x7f634c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f634c060950 0x7f634c062e00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6350009de0 tx=0x7f6350008040 comp rx=0 tx=0).stop 2023-09-27T23:13:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 msgr2=0x7f63600c1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 0x7f63600c1800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c00 tx=0x7f6358001470 comp rx=0 tx=0).stop 2023-09-27T23:13:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 shutdown_connections 2023-09-27T23:13:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f634c060950 0x7f634c062e00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63600c4ff0 0x7f63600c1800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f63600c3ad0 0x7f63600c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 --2- 172.21.15.64:0/2133189514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6348004010 0x7f63600c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 >> 172.21.15.64:0/2133189514 conn(0x7f6360067bb0 msgr2=0x7f63600b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 shutdown_connections 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.838+0000 7f6365803700 1 -- 172.21.15.64:0/2133189514 wait complete. 2023-09-27T23:13:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:13:57.857 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-09-27T23:13:57.147323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":303,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67688,"kb_used_data":3624,"kb_used_omap":0,"kb_used_meta":64000,"kb_avail":754907032,"statfs":{"total":773094113280,"available":773024800768,"internally_reserved":0,"allocated":3710976,"data_stored":2184476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65536000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.350658"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838997+0000","last_change":"2023-09-27T23:13:48.825219+0000","last_active":"2023-09-27T23:13:48.838997+0000","last_peered":"2023-09-27T23:13:48.838997+0000","last_clean":"2023-09-27T23:13:48.838997+0000","last_became_active":"2023-09-27T23:13:48.824822+0000","last_became_peered":"2023-09-27T23:13:48.824822+0000","last_unstale":"2023-09-27T23:13:48.838997+0000","last_undegraded":"2023-09-27T23:13:48.838997+0000","last_fullsized":"2023-09-27T23:13:48.838997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:34.128831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803408+0000","last_change":"2023-09-27T23:13:44.424249+0000","last_active":"2023-09-27T23:13:48.803408+0000","last_peered":"2023-09-27T23:13:48.803408+0000","last_clean":"2023-09-27T23:13:48.803408+0000","last_became_active":"2023-09-27T23:13:44.423915+0000","last_became_peered":"2023-09-27T23:13:44.423915+0000","last_unstale":"2023-09-27T23:13:48.803408+0000","last_undegraded":"2023-09-27T23:13:48.803408+0000","last_fullsized":"2023-09-27T23:13:48.803408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821460+0000","last_change":"2023-09-27T23:13:44.796968+0000","last_active":"2023-09-27T23:13:48.821460+0000","last_peered":"2023-09-27T23:13:48.821460+0000","last_clean":"2023-09-27T23:13:48.821460+0000","last_became_active":"2023-09-27T23:13:44.795519+0000","last_became_peered":"2023-09-27T23:13:44.795519+0000","last_unstale":"2023-09-27T23:13:48.821460+0000","last_undegraded":"2023-09-27T23:13:48.821460+0000","last_fullsized":"2023-09-27T23:13:48.821460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843929+0000","last_change":"2023-09-27T23:13:48.839540+0000","last_active":"2023-09-27T23:13:48.843929+0000","last_peered":"2023-09-27T23:13:48.843929+0000","last_clean":"2023-09-27T23:13:48.843929+0000","last_became_active":"2023-09-27T23:13:48.839079+0000","last_became_peered":"2023-09-27T23:13:48.839079+0000","last_unstale":"2023-09-27T23:13:48.843929+0000","last_undegraded":"2023-09-27T23:13:48.843929+0000","last_fullsized":"2023-09-27T23:13:48.843929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825433+0000","last_change":"2023-09-27T23:13:44.416457+0000","last_active":"2023-09-27T23:13:48.825433+0000","last_peered":"2023-09-27T23:13:48.825433+0000","last_clean":"2023-09-27T23:13:48.825433+0000","last_became_active":"2023-09-27T23:13:44.416135+0000","last_became_peered":"2023-09-27T23:13:44.416135+0000","last_unstale":"2023-09-27T23:13:48.825433+0000","last_undegraded":"2023-09-27T23:13:48.825433+0000","last_fullsized":"2023-09-27T23:13:48.825433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822547+0000","last_change":"2023-09-27T23:13:44.783780+0000","last_active":"2023-09-27T23:13:48.822547+0000","last_peered":"2023-09-27T23:13:48.822547+0000","last_clean":"2023-09-27T23:13:48.822547+0000","last_became_active":"2023-09-27T23:13:44.783441+0000","last_became_peered":"2023-09-27T23:13:44.783441+0000","last_unstale":"2023-09-27T23:13:48.822547+0000","last_undegraded":"2023-09-27T23:13:48.822547+0000","last_fullsized":"2023-09-27T23:13:48.822547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840431+0000","last_change":"2023-09-27T23:13:48.835419+0000","last_active":"2023-09-27T23:13:48.840431+0000","last_peered":"2023-09-27T23:13:48.840431+0000","last_clean":"2023-09-27T23:13:48.840431+0000","last_became_active":"2023-09-27T23:13:48.834981+0000","last_became_peered":"2023-09-27T23:13:48.834981+0000","last_unstale":"2023-09-27T23:13:48.840431+0000","last_undegraded":"2023-09-27T23:13:48.840431+0000","last_fullsized":"2023-09-27T23:13:48.840431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814741+0000","last_change":"2023-09-27T23:13:44.427390+0000","last_active":"2023-09-27T23:13:48.814741+0000","last_peered":"2023-09-27T23:13:48.814741+0000","last_clean":"2023-09-27T23:13:48.814741+0000","last_became_active":"2023-09-27T23:13:44.426407+0000","last_became_peered":"2023-09-27T23:13:44.426407+0000","last_unstale":"2023-09-27T23:13:48.814741+0000","last_undegraded":"2023-09-27T23:13:48.814741+0000","last_fullsized":"2023-09-27T23:13:48.814741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820888+0000","last_change":"2023-09-27T23:13:44.803765+0000","last_active":"2023-09-27T23:13:48.820888+0000","last_peered":"2023-09-27T23:13:48.820888+0000","last_clean":"2023-09-27T23:13:48.820888+0000","last_became_active":"2023-09-27T23:13:44.803448+0000","last_became_peered":"2023-09-27T23:13:44.803448+0000","last_unstale":"2023-09-27T23:13:48.820888+0000","last_undegraded":"2023-09-27T23:13:48.820888+0000","last_fullsized":"2023-09-27T23:13:48.820888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:14.091887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834465+0000","last_change":"2023-09-27T23:13:48.831362+0000","last_active":"2023-09-27T23:13:48.834465+0000","last_peered":"2023-09-27T23:13:48.834465+0000","last_clean":"2023-09-27T23:13:48.834465+0000","last_became_active":"2023-09-27T23:13:48.830786+0000","last_became_peered":"2023-09-27T23:13:48.830786+0000","last_unstale":"2023-09-27T23:13:48.834465+0000","last_undegraded":"2023-09-27T23:13:48.834465+0000","last_fullsized":"2023-09-27T23:13:48.834465+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:24:25.298387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811157+0000","last_change":"2023-09-27T23:13:44.425581+0000","last_active":"2023-09-27T23:13:48.811157+0000","last_peered":"2023-09-27T23:13:48.811157+0000","last_clean":"2023-09-27T23:13:48.811157+0000","last_became_active":"2023-09-27T23:13:44.425084+0000","last_became_peered":"2023-09-27T23:13:44.425084+0000","last_unstale":"2023-09-27T23:13:48.811157+0000","last_undegraded":"2023-09-27T23:13:48.811157+0000","last_fullsized":"2023-09-27T23:13:48.811157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821059+0000","last_change":"2023-09-27T23:13:44.803698+0000","last_active":"2023-09-27T23:13:48.821059+0000","last_peered":"2023-09-27T23:13:48.821059+0000","last_clean":"2023-09-27T23:13:48.821059+0000","last_became_active":"2023-09-27T23:13:44.803316+0000","last_became_peered":"2023-09-27T23:13:44.803316+0000","last_unstale":"2023-09-27T23:13:48.821059+0000","last_undegraded":"2023-09-27T23:13:48.821059+0000","last_fullsized":"2023-09-27T23:13:48.821059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:10:53.031343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843374+0000","last_change":"2023-09-27T23:13:48.835746+0000","last_active":"2023-09-27T23:13:48.843374+0000","last_peered":"2023-09-27T23:13:48.843374+0000","last_clean":"2023-09-27T23:13:48.843374+0000","last_became_active":"2023-09-27T23:13:48.835306+0000","last_became_peered":"2023-09-27T23:13:48.835306+0000","last_unstale":"2023-09-27T23:13:48.843374+0000","last_undegraded":"2023-09-27T23:13:48.843374+0000","last_fullsized":"2023-09-27T23:13:48.843374+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:00:48.785735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804887+0000","last_change":"2023-09-27T23:13:44.428693+0000","last_active":"2023-09-27T23:13:48.804887+0000","last_peered":"2023-09-27T23:13:48.804887+0000","last_clean":"2023-09-27T23:13:48.804887+0000","last_became_active":"2023-09-27T23:13:44.428213+0000","last_became_peered":"2023-09-27T23:13:44.428213+0000","last_unstale":"2023-09-27T23:13:48.804887+0000","last_undegraded":"2023-09-27T23:13:48.804887+0000","last_fullsized":"2023-09-27T23:13:48.804887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810898+0000","last_change":"2023-09-27T23:13:44.791611+0000","last_active":"2023-09-27T23:13:48.810898+0000","last_peered":"2023-09-27T23:13:48.810898+0000","last_clean":"2023-09-27T23:13:48.810898+0000","last_became_active":"2023-09-27T23:13:44.791292+0000","last_became_peered":"2023-09-27T23:13:44.791292+0000","last_unstale":"2023-09-27T23:13:48.810898+0000","last_undegraded":"2023-09-27T23:13:48.810898+0000","last_fullsized":"2023-09-27T23:13:48.810898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:08.690781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840009+0000","last_change":"2023-09-27T23:13:48.827264+0000","last_active":"2023-09-27T23:13:48.840009+0000","last_peered":"2023-09-27T23:13:48.840009+0000","last_clean":"2023-09-27T23:13:48.840009+0000","last_became_active":"2023-09-27T23:13:48.826832+0000","last_became_peered":"2023-09-27T23:13:48.826832+0000","last_unstale":"2023-09-27T23:13:48.840009+0000","last_undegraded":"2023-09-27T23:13:48.840009+0000","last_fullsized":"2023-09-27T23:13:48.840009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:48:20.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821181+0000","last_change":"2023-09-27T23:13:44.423920+0000","last_active":"2023-09-27T23:13:48.821181+0000","last_peered":"2023-09-27T23:13:48.821181+0000","last_clean":"2023-09-27T23:13:48.821181+0000","last_became_active":"2023-09-27T23:13:44.422754+0000","last_became_peered":"2023-09-27T23:13:44.422754+0000","last_unstale":"2023-09-27T23:13:48.821181+0000","last_undegraded":"2023-09-27T23:13:48.821181+0000","last_fullsized":"2023-09-27T23:13:48.821181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:16.888839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825025+0000","last_change":"2023-09-27T23:13:44.790022+0000","last_active":"2023-09-27T23:13:48.825025+0000","last_peered":"2023-09-27T23:13:48.825025+0000","last_clean":"2023-09-27T23:13:48.825025+0000","last_became_active":"2023-09-27T23:13:44.789052+0000","last_became_peered":"2023-09-27T23:13:44.789052+0000","last_unstale":"2023-09-27T23:13:48.825025+0000","last_undegraded":"2023-09-27T23:13:48.825025+0000","last_fullsized":"2023-09-27T23:13:48.825025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.841311+0000","last_change":"2023-09-27T23:13:48.835634+0000","last_active":"2023-09-27T23:13:48.841311+0000","last_peered":"2023-09-27T23:13:48.841311+0000","last_clean":"2023-09-27T23:13:48.841311+0000","last_became_active":"2023-09-27T23:13:48.834989+0000","last_became_peered":"2023-09-27T23:13:48.834989+0000","last_unstale":"2023-09-27T23:13:48.841311+0000","last_undegraded":"2023-09-27T23:13:48.841311+0000","last_fullsized":"2023-09-27T23:13:48.841311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:09.944701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802920+0000","last_change":"2023-09-27T23:13:44.424671+0000","last_active":"2023-09-27T23:13:48.802920+0000","last_peered":"2023-09-27T23:13:48.802920+0000","last_clean":"2023-09-27T23:13:48.802920+0000","last_became_active":"2023-09-27T23:13:44.424321+0000","last_became_peered":"2023-09-27T23:13:44.424321+0000","last_unstale":"2023-09-27T23:13:48.802920+0000","last_undegraded":"2023-09-27T23:13:48.802920+0000","last_fullsized":"2023-09-27T23:13:48.802920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:44:43.453425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803496+0000","last_change":"2023-09-27T23:13:44.792445+0000","last_active":"2023-09-27T23:13:48.803496+0000","last_peered":"2023-09-27T23:13:48.803496+0000","last_clean":"2023-09-27T23:13:48.803496+0000","last_became_active":"2023-09-27T23:13:44.790669+0000","last_became_peered":"2023-09-27T23:13:44.790669+0000","last_unstale":"2023-09-27T23:13:48.803496+0000","last_undegraded":"2023-09-27T23:13:48.803496+0000","last_fullsized":"2023-09-27T23:13:48.803496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833795+0000","last_change":"2023-09-27T23:13:48.824674+0000","last_active":"2023-09-27T23:13:48.833795+0000","last_peered":"2023-09-27T23:13:48.833795+0000","last_clean":"2023-09-27T23:13:48.833795+0000","last_became_active":"2023-09-27T23:13:48.824220+0000","last_became_peered":"2023-09-27T23:13:48.824220+0000","last_unstale":"2023-09-27T23:13:48.833795+0000","last_undegraded":"2023-09-27T23:13:48.833795+0000","last_fullsized":"2023-09-27T23:13:48.833795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822020+0000","last_change":"2023-09-27T23:13:44.415940+0000","last_active":"2023-09-27T23:13:48.822020+0000","last_peered":"2023-09-27T23:13:48.822020+0000","last_clean":"2023-09-27T23:13:48.822020+0000","last_became_active":"2023-09-27T23:13:44.415355+0000","last_became_peered":"2023-09-27T23:13:44.415355+0000","last_unstale":"2023-09-27T23:13:48.822020+0000","last_undegraded":"2023-09-27T23:13:48.822020+0000","last_fullsized":"2023-09-27T23:13:48.822020+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813805+0000","last_change":"2023-09-27T23:13:44.807290+0000","last_active":"2023-09-27T23:13:48.813805+0000","last_peered":"2023-09-27T23:13:48.813805+0000","last_clean":"2023-09-27T23:13:48.813805+0000","last_became_active":"2023-09-27T23:13:44.806991+0000","last_became_peered":"2023-09-27T23:13:44.806991+0000","last_unstale":"2023-09-27T23:13:48.813805+0000","last_undegraded":"2023-09-27T23:13:48.813805+0000","last_fullsized":"2023-09-27T23:13:48.813805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:41.102884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821934+0000","last_change":"2023-09-27T23:13:44.797997+0000","last_active":"2023-09-27T23:13:48.821934+0000","last_peered":"2023-09-27T23:13:48.821934+0000","last_clean":"2023-09-27T23:13:48.821934+0000","last_became_active":"2023-09-27T23:13:44.797689+0000","last_became_peered":"2023-09-27T23:13:44.797689+0000","last_unstale":"2023-09-27T23:13:48.821934+0000","last_undegraded":"2023-09-27T23:13:48.821934+0000","last_fullsized":"2023-09-27T23:13:48.821934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803641+0000","last_change":"2023-09-27T23:13:44.420293+0000","last_active":"2023-09-27T23:13:48.803641+0000","last_peered":"2023-09-27T23:13:48.803641+0000","last_clean":"2023-09-27T23:13:48.803641+0000","last_became_active":"2023-09-27T23:13:44.419584+0000","last_became_peered":"2023-09-27T23:13:44.419584+0000","last_unstale":"2023-09-27T23:13:48.803641+0000","last_undegraded":"2023-09-27T23:13:48.803641+0000","last_fullsized":"2023-09-27T23:13:48.803641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843828+0000","last_change":"2023-09-27T23:13:48.842799+0000","last_active":"2023-09-27T23:13:48.843828+0000","last_peered":"2023-09-27T23:13:48.843828+0000","last_clean":"2023-09-27T23:13:48.843828+0000","last_became_active":"2023-09-27T23:13:48.842271+0000","last_became_peered":"2023-09-27T23:13:48.842271+0000","last_unstale":"2023-09-27T23:13:48.843828+0000","last_undegraded":"2023-09-27T23:13:48.843828+0000","last_fullsized":"2023-09-27T23:13:48.843828+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:26.172918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822463+0000","last_change":"2023-09-27T23:13:44.791363+0000","last_active":"2023-09-27T23:13:48.822463+0000","last_peered":"2023-09-27T23:13:48.822463+0000","last_clean":"2023-09-27T23:13:48.822463+0000","last_became_active":"2023-09-27T23:13:44.786248+0000","last_became_peered":"2023-09-27T23:13:44.786248+0000","last_unstale":"2023-09-27T23:13:48.822463+0000","last_undegraded":"2023-09-27T23:13:48.822463+0000","last_fullsized":"2023-09-27T23:13:48.822463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:22.779025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815748+0000","last_change":"2023-09-27T23:13:44.416087+0000","last_active":"2023-09-27T23:13:48.815748+0000","last_peered":"2023-09-27T23:13:48.815748+0000","last_clean":"2023-09-27T23:13:48.815748+0000","last_became_active":"2023-09-27T23:13:44.415755+0000","last_became_peered":"2023-09-27T23:13:44.415755+0000","last_unstale":"2023-09-27T23:13:48.815748+0000","last_undegraded":"2023-09-27T23:13:48.815748+0000","last_fullsized":"2023-09-27T23:13:48.815748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826650+0000","last_change":"2023-09-27T23:13:48.825181+0000","last_active":"2023-09-27T23:13:48.826650+0000","last_peered":"2023-09-27T23:13:48.826650+0000","last_clean":"2023-09-27T23:13:48.826650+0000","last_became_active":"2023-09-27T23:13:48.824750+0000","last_became_peered":"2023-09-27T23:13:48.824750+0000","last_unstale":"2023-09-27T23:13:48.826650+0000","last_undegraded":"2023-09-27T23:13:48.826650+0000","last_fullsized":"2023-09-27T23:13:48.826650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803126+0000","last_change":"2023-09-27T23:13:44.423841+0000","last_active":"2023-09-27T23:13:48.803126+0000","last_peered":"2023-09-27T23:13:48.803126+0000","last_clean":"2023-09-27T23:13:48.803126+0000","last_became_active":"2023-09-27T23:13:44.423450+0000","last_became_peered":"2023-09-27T23:13:44.423450+0000","last_unstale":"2023-09-27T23:13:48.803126+0000","last_undegraded":"2023-09-27T23:13:48.803126+0000","last_fullsized":"2023-09-27T23:13:48.803126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804382+0000","last_change":"2023-09-27T23:13:44.789353+0000","last_active":"2023-09-27T23:13:48.804382+0000","last_peered":"2023-09-27T23:13:48.804382+0000","last_clean":"2023-09-27T23:13:48.804382+0000","last_became_active":"2023-09-27T23:13:44.788925+0000","last_became_peered":"2023-09-27T23:13:44.788925+0000","last_unstale":"2023-09-27T23:13:48.804382+0000","last_undegraded":"2023-09-27T23:13:48.804382+0000","last_fullsized":"2023-09-27T23:13:48.804382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:24.773257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842925+0000","last_change":"2023-09-27T23:13:48.840258+0000","last_active":"2023-09-27T23:13:48.842925+0000","last_peered":"2023-09-27T23:13:48.842925+0000","last_clean":"2023-09-27T23:13:48.842925+0000","last_became_active":"2023-09-27T23:13:48.839796+0000","last_became_peered":"2023-09-27T23:13:48.839796+0000","last_unstale":"2023-09-27T23:13:48.842925+0000","last_undegraded":"2023-09-27T23:13:48.842925+0000","last_fullsized":"2023-09-27T23:13:48.842925+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821682+0000","last_change":"2023-09-27T23:13:44.422959+0000","last_active":"2023-09-27T23:13:48.821682+0000","last_peered":"2023-09-27T23:13:48.821682+0000","last_clean":"2023-09-27T23:13:48.821682+0000","last_became_active":"2023-09-27T23:13:44.420682+0000","last_became_peered":"2023-09-27T23:13:44.420682+0000","last_unstale":"2023-09-27T23:13:48.821682+0000","last_undegraded":"2023-09-27T23:13:48.821682+0000","last_fullsized":"2023-09-27T23:13:48.821682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821157+0000","last_change":"2023-09-27T23:13:44.792715+0000","last_active":"2023-09-27T23:13:48.821157+0000","last_peered":"2023-09-27T23:13:48.821157+0000","last_clean":"2023-09-27T23:13:48.821157+0000","last_became_active":"2023-09-27T23:13:44.792356+0000","last_became_peered":"2023-09-27T23:13:44.792356+0000","last_unstale":"2023-09-27T23:13:48.821157+0000","last_undegraded":"2023-09-27T23:13:48.821157+0000","last_fullsized":"2023-09-27T23:13:48.821157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843134+0000","last_change":"2023-09-27T23:13:48.830683+0000","last_active":"2023-09-27T23:13:48.843134+0000","last_peered":"2023-09-27T23:13:48.843134+0000","last_clean":"2023-09-27T23:13:48.843134+0000","last_became_active":"2023-09-27T23:13:48.830149+0000","last_became_peered":"2023-09-27T23:13:48.830149+0000","last_unstale":"2023-09-27T23:13:48.843134+0000","last_undegraded":"2023-09-27T23:13:48.843134+0000","last_fullsized":"2023-09-27T23:13:48.843134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:31:07.801049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821012+0000","last_change":"2023-09-27T23:13:44.807208+0000","last_active":"2023-09-27T23:13:48.821012+0000","last_peered":"2023-09-27T23:13:48.821012+0000","last_clean":"2023-09-27T23:13:48.821012+0000","last_became_active":"2023-09-27T23:13:44.806937+0000","last_became_peered":"2023-09-27T23:13:44.806937+0000","last_unstale":"2023-09-27T23:13:48.821012+0000","last_undegraded":"2023-09-27T23:13:48.821012+0000","last_fullsized":"2023-09-27T23:13:48.821012+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:03:17.944613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813854+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:13:48.813854+0000","last_peered":"2023-09-27T23:13:48.813854+0000","last_clean":"2023-09-27T23:13:48.813854+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:13:48.813854+0000","last_undegraded":"2023-09-27T23:13:48.813854+0000","last_fullsized":"2023-09-27T23:13:48.813854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-27T23:13:49.971771+0000","last_change":"2023-09-27T23:13:44.421149+0000","last_active":"2023-09-27T23:13:49.971771+0000","last_peered":"2023-09-27T23:13:49.971771+0000","last_clean":"2023-09-27T23:13:49.971771+0000","last_became_active":"2023-09-27T23:13:44.419644+0000","last_became_peered":"2023-09-27T23:13:44.419644+0000","last_unstale":"2023-09-27T23:13:49.971771+0000","last_undegraded":"2023-09-27T23:13:49.971771+0000","last_fullsized":"2023-09-27T23:13:49.971771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842921+0000","last_change":"2023-09-27T23:13:48.840751+0000","last_active":"2023-09-27T23:13:48.842921+0000","last_peered":"2023-09-27T23:13:48.842921+0000","last_clean":"2023-09-27T23:13:48.842921+0000","last_became_active":"2023-09-27T23:13:48.840353+0000","last_became_peered":"2023-09-27T23:13:48.840353+0000","last_unstale":"2023-09-27T23:13:48.842921+0000","last_undegraded":"2023-09-27T23:13:48.842921+0000","last_fullsized":"2023-09-27T23:13:48.842921+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:59:09.353088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821991+0000","last_change":"2023-09-27T23:13:44.791105+0000","last_active":"2023-09-27T23:13:48.821991+0000","last_peered":"2023-09-27T23:13:48.821991+0000","last_clean":"2023-09-27T23:13:48.821991+0000","last_became_active":"2023-09-27T23:13:44.790824+0000","last_became_peered":"2023-09-27T23:13:44.790824+0000","last_unstale":"2023-09-27T23:13:48.821991+0000","last_undegraded":"2023-09-27T23:13:48.821991+0000","last_fullsized":"2023-09-27T23:13:48.821991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807265+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:13:48.807265+0000","last_peered":"2023-09-27T23:13:48.807265+0000","last_clean":"2023-09-27T23:13:48.807265+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:13:48.807265+0000","last_undegraded":"2023-09-27T23:13:48.807265+0000","last_fullsized":"2023-09-27T23:13:48.807265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825400+0000","last_change":"2023-09-27T23:13:44.421998+0000","last_active":"2023-09-27T23:13:48.825400+0000","last_peered":"2023-09-27T23:13:48.825400+0000","last_clean":"2023-09-27T23:13:48.825400+0000","last_became_active":"2023-09-27T23:13:44.416358+0000","last_became_peered":"2023-09-27T23:13:44.416358+0000","last_unstale":"2023-09-27T23:13:48.825400+0000","last_undegraded":"2023-09-27T23:13:48.825400+0000","last_fullsized":"2023-09-27T23:13:48.825400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840017+0000","last_change":"2023-09-27T23:13:48.827758+0000","last_active":"2023-09-27T23:13:48.840017+0000","last_peered":"2023-09-27T23:13:48.840017+0000","last_clean":"2023-09-27T23:13:48.840017+0000","last_became_active":"2023-09-27T23:13:48.827144+0000","last_became_peered":"2023-09-27T23:13:48.827144+0000","last_unstale":"2023-09-27T23:13:48.840017+0000","last_undegraded":"2023-09-27T23:13:48.840017+0000","last_fullsized":"2023-09-27T23:13:48.840017+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:02:44.973308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804641+0000","last_change":"2023-09-27T23:13:44.807107+0000","last_active":"2023-09-27T23:13:48.804641+0000","last_peered":"2023-09-27T23:13:48.804641+0000","last_clean":"2023-09-27T23:13:48.804641+0000","last_became_active":"2023-09-27T23:13:44.806739+0000","last_became_peered":"2023-09-27T23:13:44.806739+0000","last_unstale":"2023-09-27T23:13:48.804641+0000","last_undegraded":"2023-09-27T23:13:48.804641+0000","last_fullsized":"2023-09-27T23:13:48.804641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803837+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:13:48.803837+0000","last_peered":"2023-09-27T23:13:48.803837+0000","last_clean":"2023-09-27T23:13:48.803837+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:13:48.803837+0000","last_undegraded":"2023-09-27T23:13:48.803837+0000","last_fullsized":"2023-09-27T23:13:48.803837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802757+0000","last_change":"2023-09-27T23:13:44.419685+0000","last_active":"2023-09-27T23:13:48.802757+0000","last_peered":"2023-09-27T23:13:48.802757+0000","last_clean":"2023-09-27T23:13:48.802757+0000","last_became_active":"2023-09-27T23:13:44.419369+0000","last_became_peered":"2023-09-27T23:13:44.419369+0000","last_unstale":"2023-09-27T23:13:48.802757+0000","last_undegraded":"2023-09-27T23:13:48.802757+0000","last_fullsized":"2023-09-27T23:13:48.802757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:04.718338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826218+0000","last_change":"2023-09-27T23:13:48.822456+0000","last_active":"2023-09-27T23:13:48.826218+0000","last_peered":"2023-09-27T23:13:48.826218+0000","last_clean":"2023-09-27T23:13:48.826218+0000","last_became_active":"2023-09-27T23:13:48.821878+0000","last_became_peered":"2023-09-27T23:13:48.821878+0000","last_unstale":"2023-09-27T23:13:48.826218+0000","last_undegraded":"2023-09-27T23:13:48.826218+0000","last_fullsized":"2023-09-27T23:13:48.826218+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807086+0000","last_change":"2023-09-27T23:13:44.777613+0000","last_active":"2023-09-27T23:13:48.807086+0000","last_peered":"2023-09-27T23:13:48.807086+0000","last_clean":"2023-09-27T23:13:48.807086+0000","last_became_active":"2023-09-27T23:13:44.777267+0000","last_became_peered":"2023-09-27T23:13:44.777267+0000","last_unstale":"2023-09-27T23:13:48.807086+0000","last_undegraded":"2023-09-27T23:13:48.807086+0000","last_fullsized":"2023-09-27T23:13:48.807086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:42:36.525422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802984+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:13:48.802984+0000","last_peered":"2023-09-27T23:13:48.802984+0000","last_clean":"2023-09-27T23:13:48.802984+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:13:48.802984+0000","last_undegraded":"2023-09-27T23:13:48.802984+0000","last_fullsized":"2023-09-27T23:13:48.802984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814830+0000","last_change":"2023-09-27T23:13:44.422663+0000","last_active":"2023-09-27T23:13:48.814830+0000","last_peered":"2023-09-27T23:13:48.814830+0000","last_clean":"2023-09-27T23:13:48.814830+0000","last_became_active":"2023-09-27T23:13:44.422361+0000","last_became_peered":"2023-09-27T23:13:44.422361+0000","last_unstale":"2023-09-27T23:13:48.814830+0000","last_undegraded":"2023-09-27T23:13:48.814830+0000","last_fullsized":"2023-09-27T23:13:48.814830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833765+0000","last_change":"2023-09-27T23:13:48.824670+0000","last_active":"2023-09-27T23:13:48.833765+0000","last_peered":"2023-09-27T23:13:48.833765+0000","last_clean":"2023-09-27T23:13:48.833765+0000","last_became_active":"2023-09-27T23:13:48.824249+0000","last_became_peered":"2023-09-27T23:13:48.824249+0000","last_unstale":"2023-09-27T23:13:48.833765+0000","last_undegraded":"2023-09-27T23:13:48.833765+0000","last_fullsized":"2023-09-27T23:13:48.833765+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802514+0000","last_change":"2023-09-27T23:13:44.778136+0000","last_active":"2023-09-27T23:13:48.802514+0000","last_peered":"2023-09-27T23:13:48.802514+0000","last_clean":"2023-09-27T23:13:48.802514+0000","last_became_active":"2023-09-27T23:13:44.777865+0000","last_became_peered":"2023-09-27T23:13:44.777865+0000","last_unstale":"2023-09-27T23:13:48.802514+0000","last_undegraded":"2023-09-27T23:13:48.802514+0000","last_fullsized":"2023-09-27T23:13:48.802514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:46:04.289644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821294+0000","last_change":"2023-09-27T23:13:44.424183+0000","last_active":"2023-09-27T23:13:48.821294+0000","last_peered":"2023-09-27T23:13:48.821294+0000","last_clean":"2023-09-27T23:13:48.821294+0000","last_became_active":"2023-09-27T23:13:44.422919+0000","last_became_peered":"2023-09-27T23:13:44.422919+0000","last_unstale":"2023-09-27T23:13:48.821294+0000","last_undegraded":"2023-09-27T23:13:48.821294+0000","last_fullsized":"2023-09-27T23:13:48.821294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813932+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:13:48.813932+0000","last_peered":"2023-09-27T23:13:48.813932+0000","last_clean":"2023-09-27T23:13:48.813932+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:13:48.813932+0000","last_undegraded":"2023-09-27T23:13:48.813932+0000","last_fullsized":"2023-09-27T23:13:48.813932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839053+0000","last_change":"2023-09-27T23:13:48.836163+0000","last_active":"2023-09-27T23:13:48.839053+0000","last_peered":"2023-09-27T23:13:48.839053+0000","last_clean":"2023-09-27T23:13:48.839053+0000","last_became_active":"2023-09-27T23:13:48.835655+0000","last_became_peered":"2023-09-27T23:13:48.835655+0000","last_unstale":"2023-09-27T23:13:48.839053+0000","last_undegraded":"2023-09-27T23:13:48.839053+0000","last_fullsized":"2023-09-27T23:13:48.839053+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802588+0000","last_change":"2023-09-27T23:13:44.777627+0000","last_active":"2023-09-27T23:13:48.802588+0000","last_peered":"2023-09-27T23:13:48.802588+0000","last_clean":"2023-09-27T23:13:48.802588+0000","last_became_active":"2023-09-27T23:13:44.777239+0000","last_became_peered":"2023-09-27T23:13:44.777239+0000","last_unstale":"2023-09-27T23:13:48.802588+0000","last_undegraded":"2023-09-27T23:13:48.802588+0000","last_fullsized":"2023-09-27T23:13:48.802588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:34.572519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.800241+0000","last_change":"2023-09-27T23:13:44.419827+0000","last_active":"2023-09-27T23:13:48.800241+0000","last_peered":"2023-09-27T23:13:48.800241+0000","last_clean":"2023-09-27T23:13:48.800241+0000","last_became_active":"2023-09-27T23:13:44.419535+0000","last_became_peered":"2023-09-27T23:13:44.419535+0000","last_unstale":"2023-09-27T23:13:48.800241+0000","last_undegraded":"2023-09-27T23:13:48.800241+0000","last_fullsized":"2023-09-27T23:13:48.800241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:08:26.061286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813256+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:13:48.813256+0000","last_peered":"2023-09-27T23:13:48.813256+0000","last_clean":"2023-09-27T23:13:48.813256+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:13:48.813256+0000","last_undegraded":"2023-09-27T23:13:48.813256+0000","last_fullsized":"2023-09-27T23:13:48.813256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842381+0000","last_change":"2023-09-27T23:13:48.839688+0000","last_active":"2023-09-27T23:13:48.842381+0000","last_peered":"2023-09-27T23:13:48.842381+0000","last_clean":"2023-09-27T23:13:48.842381+0000","last_became_active":"2023-09-27T23:13:48.839257+0000","last_became_peered":"2023-09-27T23:13:48.839257+0000","last_unstale":"2023-09-27T23:13:48.842381+0000","last_undegraded":"2023-09-27T23:13:48.842381+0000","last_fullsized":"2023-09-27T23:13:48.842381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:23.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803724+0000","last_change":"2023-09-27T23:13:44.796668+0000","last_active":"2023-09-27T23:13:48.803724+0000","last_peered":"2023-09-27T23:13:48.803724+0000","last_clean":"2023-09-27T23:13:48.803724+0000","last_became_active":"2023-09-27T23:13:44.795170+0000","last_became_peered":"2023-09-27T23:13:44.795170+0000","last_unstale":"2023-09-27T23:13:48.803724+0000","last_undegraded":"2023-09-27T23:13:48.803724+0000","last_fullsized":"2023-09-27T23:13:48.803724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802500+0000","last_change":"2023-09-27T23:13:44.428369+0000","last_active":"2023-09-27T23:13:48.802500+0000","last_peered":"2023-09-27T23:13:48.802500+0000","last_clean":"2023-09-27T23:13:48.802500+0000","last_became_active":"2023-09-27T23:13:44.427956+0000","last_became_peered":"2023-09-27T23:13:44.427956+0000","last_unstale":"2023-09-27T23:13:48.802500+0000","last_undegraded":"2023-09-27T23:13:48.802500+0000","last_fullsized":"2023-09-27T23:13:48.802500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803011+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:13:48.803011+0000","last_peered":"2023-09-27T23:13:48.803011+0000","last_clean":"2023-09-27T23:13:48.803011+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:13:48.803011+0000","last_undegraded":"2023-09-27T23:13:48.803011+0000","last_fullsized":"2023-09-27T23:13:48.803011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834595+0000","last_change":"2023-09-27T23:13:48.830308+0000","last_active":"2023-09-27T23:13:48.834595+0000","last_peered":"2023-09-27T23:13:48.834595+0000","last_clean":"2023-09-27T23:13:48.834595+0000","last_became_active":"2023-09-27T23:13:48.829698+0000","last_became_peered":"2023-09-27T23:13:48.829698+0000","last_unstale":"2023-09-27T23:13:48.834595+0000","last_undegraded":"2023-09-27T23:13:48.834595+0000","last_fullsized":"2023-09-27T23:13:48.834595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:27:58.121695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812509+0000","last_change":"2023-09-27T23:13:44.804857+0000","last_active":"2023-09-27T23:13:48.812509+0000","last_peered":"2023-09-27T23:13:48.812509+0000","last_clean":"2023-09-27T23:13:48.812509+0000","last_became_active":"2023-09-27T23:13:44.804486+0000","last_became_peered":"2023-09-27T23:13:44.804486+0000","last_unstale":"2023-09-27T23:13:48.812509+0000","last_undegraded":"2023-09-27T23:13:48.812509+0000","last_fullsized":"2023-09-27T23:13:48.812509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822742+0000","last_change":"2023-09-27T23:13:44.416101+0000","last_active":"2023-09-27T23:13:48.822742+0000","last_peered":"2023-09-27T23:13:48.822742+0000","last_clean":"2023-09-27T23:13:48.822742+0000","last_became_active":"2023-09-27T23:13:44.415693+0000","last_became_peered":"2023-09-27T23:13:44.415693+0000","last_unstale":"2023-09-27T23:13:48.822742+0000","last_undegraded":"2023-09-27T23:13:48.822742+0000","last_fullsized":"2023-09-27T23:13:48.822742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813962+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:13:48.813962+0000","last_peered":"2023-09-27T23:13:48.813962+0000","last_clean":"2023-09-27T23:13:48.813962+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:13:48.813962+0000","last_undegraded":"2023-09-27T23:13:48.813962+0000","last_fullsized":"2023-09-27T23:13:48.813962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803349+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:13:48.803349+0000","last_peered":"2023-09-27T23:13:48.803349+0000","last_clean":"2023-09-27T23:13:48.803349+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:13:48.803349+0000","last_undegraded":"2023-09-27T23:13:48.803349+0000","last_fullsized":"2023-09-27T23:13:48.803349+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829119+0000","last_change":"2023-09-27T23:13:48.827977+0000","last_active":"2023-09-27T23:13:48.829119+0000","last_peered":"2023-09-27T23:13:48.829119+0000","last_clean":"2023-09-27T23:13:48.829119+0000","last_became_active":"2023-09-27T23:13:48.827465+0000","last_became_peered":"2023-09-27T23:13:48.827465+0000","last_unstale":"2023-09-27T23:13:48.829119+0000","last_undegraded":"2023-09-27T23:13:48.829119+0000","last_fullsized":"2023-09-27T23:13:48.829119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:18:57.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803299+0000","last_change":"2023-09-27T23:13:44.786471+0000","last_active":"2023-09-27T23:13:48.803299+0000","last_peered":"2023-09-27T23:13:48.803299+0000","last_clean":"2023-09-27T23:13:48.803299+0000","last_became_active":"2023-09-27T23:13:44.786182+0000","last_became_peered":"2023-09-27T23:13:44.786182+0000","last_unstale":"2023-09-27T23:13:48.803299+0000","last_undegraded":"2023-09-27T23:13:48.803299+0000","last_fullsized":"2023-09-27T23:13:48.803299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:04:06.713350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822086+0000","last_change":"2023-09-27T23:13:44.422925+0000","last_active":"2023-09-27T23:13:48.822086+0000","last_peered":"2023-09-27T23:13:48.822086+0000","last_clean":"2023-09-27T23:13:48.822086+0000","last_became_active":"2023-09-27T23:13:44.422547+0000","last_became_peered":"2023-09-27T23:13:44.422547+0000","last_unstale":"2023-09-27T23:13:48.822086+0000","last_undegraded":"2023-09-27T23:13:48.822086+0000","last_fullsized":"2023-09-27T23:13:48.822086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839810+0000","last_change":"2023-09-27T23:13:48.824583+0000","last_active":"2023-09-27T23:13:48.839810+0000","last_peered":"2023-09-27T23:13:48.839810+0000","last_clean":"2023-09-27T23:13:48.839810+0000","last_became_active":"2023-09-27T23:13:48.824080+0000","last_became_peered":"2023-09-27T23:13:48.824080+0000","last_unstale":"2023-09-27T23:13:48.839810+0000","last_undegraded":"2023-09-27T23:13:48.839810+0000","last_fullsized":"2023-09-27T23:13:48.839810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:29:19.951565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821416+0000","last_change":"2023-09-27T23:13:44.792934+0000","last_active":"2023-09-27T23:13:48.821416+0000","last_peered":"2023-09-27T23:13:48.821416+0000","last_clean":"2023-09-27T23:13:48.821416+0000","last_became_active":"2023-09-27T23:13:44.791774+0000","last_became_peered":"2023-09-27T23:13:44.791774+0000","last_unstale":"2023-09-27T23:13:48.821416+0000","last_undegraded":"2023-09-27T23:13:48.821416+0000","last_fullsized":"2023-09-27T23:13:48.821416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821113+0000","last_change":"2023-09-27T23:13:44.418284+0000","last_active":"2023-09-27T23:13:48.821113+0000","last_peered":"2023-09-27T23:13:48.821113+0000","last_clean":"2023-09-27T23:13:48.821113+0000","last_became_active":"2023-09-27T23:13:44.417952+0000","last_became_peered":"2023-09-27T23:13:44.417952+0000","last_unstale":"2023-09-27T23:13:48.821113+0000","last_undegraded":"2023-09-27T23:13:48.821113+0000","last_fullsized":"2023-09-27T23:13:48.821113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840441+0000","last_change":"2023-09-27T23:13:48.835487+0000","last_active":"2023-09-27T23:13:48.840441+0000","last_peered":"2023-09-27T23:13:48.840441+0000","last_clean":"2023-09-27T23:13:48.840441+0000","last_became_active":"2023-09-27T23:13:48.835043+0000","last_became_peered":"2023-09-27T23:13:48.835043+0000","last_unstale":"2023-09-27T23:13:48.840441+0000","last_undegraded":"2023-09-27T23:13:48.840441+0000","last_fullsized":"2023-09-27T23:13:48.840441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:43:53.193315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821389+0000","last_change":"2023-09-27T23:13:44.797452+0000","last_active":"2023-09-27T23:13:48.821389+0000","last_peered":"2023-09-27T23:13:48.821389+0000","last_clean":"2023-09-27T23:13:48.821389+0000","last_became_active":"2023-09-27T23:13:44.797056+0000","last_became_peered":"2023-09-27T23:13:44.797056+0000","last_unstale":"2023-09-27T23:13:48.821389+0000","last_undegraded":"2023-09-27T23:13:48.821389+0000","last_fullsized":"2023-09-27T23:13:48.821389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812548+0000","last_change":"2023-09-27T23:13:44.426307+0000","last_active":"2023-09-27T23:13:48.812548+0000","last_peered":"2023-09-27T23:13:48.812548+0000","last_clean":"2023-09-27T23:13:48.812548+0000","last_became_active":"2023-09-27T23:13:44.425925+0000","last_became_peered":"2023-09-27T23:13:44.425925+0000","last_unstale":"2023-09-27T23:13:48.812548+0000","last_undegraded":"2023-09-27T23:13:48.812548+0000","last_fullsized":"2023-09-27T23:13:48.812548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843420+0000","last_change":"2023-09-27T23:13:48.835916+0000","last_active":"2023-09-27T23:13:48.843420+0000","last_peered":"2023-09-27T23:13:48.843420+0000","last_clean":"2023-09-27T23:13:48.843420+0000","last_became_active":"2023-09-27T23:13:48.835201+0000","last_became_peered":"2023-09-27T23:13:48.835201+0000","last_unstale":"2023-09-27T23:13:48.843420+0000","last_undegraded":"2023-09-27T23:13:48.843420+0000","last_fullsized":"2023-09-27T23:13:48.843420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:15:13.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821512+0000","last_change":"2023-09-27T23:13:44.804376+0000","last_active":"2023-09-27T23:13:48.821512+0000","last_peered":"2023-09-27T23:13:48.821512+0000","last_clean":"2023-09-27T23:13:48.821512+0000","last_became_active":"2023-09-27T23:13:44.803820+0000","last_became_peered":"2023-09-27T23:13:44.803820+0000","last_unstale":"2023-09-27T23:13:48.821512+0000","last_undegraded":"2023-09-27T23:13:48.821512+0000","last_fullsized":"2023-09-27T23:13:48.821512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:45:33.486648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825618+0000","last_change":"2023-09-27T23:13:44.422613+0000","last_active":"2023-09-27T23:13:48.825618+0000","last_peered":"2023-09-27T23:13:48.825618+0000","last_clean":"2023-09-27T23:13:48.825618+0000","last_became_active":"2023-09-27T23:13:44.422341+0000","last_became_peered":"2023-09-27T23:13:44.422341+0000","last_unstale":"2023-09-27T23:13:48.825618+0000","last_undegraded":"2023-09-27T23:13:48.825618+0000","last_fullsized":"2023-09-27T23:13:48.825618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825159+0000","last_change":"2023-09-27T23:13:48.821288+0000","last_active":"2023-09-27T23:13:48.825159+0000","last_peered":"2023-09-27T23:13:48.825159+0000","last_clean":"2023-09-27T23:13:48.825159+0000","last_became_active":"2023-09-27T23:13:48.820648+0000","last_became_peered":"2023-09-27T23:13:48.820648+0000","last_unstale":"2023-09-27T23:13:48.825159+0000","last_undegraded":"2023-09-27T23:13:48.825159+0000","last_fullsized":"2023-09-27T23:13:48.825159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:17.485779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804537+0000","last_change":"2023-09-27T23:13:44.795149+0000","last_active":"2023-09-27T23:13:48.804537+0000","last_peered":"2023-09-27T23:13:48.804537+0000","last_clean":"2023-09-27T23:13:48.804537+0000","last_became_active":"2023-09-27T23:13:44.794005+0000","last_became_peered":"2023-09-27T23:13:44.794005+0000","last_unstale":"2023-09-27T23:13:48.804537+0000","last_undegraded":"2023-09-27T23:13:48.804537+0000","last_fullsized":"2023-09-27T23:13:48.804537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811248+0000","last_change":"2023-09-27T23:13:44.422676+0000","last_active":"2023-09-27T23:13:48.811248+0000","last_peered":"2023-09-27T23:13:48.811248+0000","last_clean":"2023-09-27T23:13:48.811248+0000","last_became_active":"2023-09-27T23:13:44.422396+0000","last_became_peered":"2023-09-27T23:13:44.422396+0000","last_unstale":"2023-09-27T23:13:48.811248+0000","last_undegraded":"2023-09-27T23:13:48.811248+0000","last_fullsized":"2023-09-27T23:13:48.811248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838657+0000","last_change":"2023-09-27T23:13:48.835879+0000","last_active":"2023-09-27T23:13:48.838657+0000","last_peered":"2023-09-27T23:13:48.838657+0000","last_clean":"2023-09-27T23:13:48.838657+0000","last_became_active":"2023-09-27T23:13:48.835422+0000","last_became_peered":"2023-09-27T23:13:48.835422+0000","last_unstale":"2023-09-27T23:13:48.838657+0000","last_undegraded":"2023-09-27T23:13:48.838657+0000","last_fullsized":"2023-09-27T23:13:48.838657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:36.782810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810125+0000","last_change":"2023-09-27T23:13:44.795768+0000","last_active":"2023-09-27T23:13:48.810125+0000","last_peered":"2023-09-27T23:13:48.810125+0000","last_clean":"2023-09-27T23:13:48.810125+0000","last_became_active":"2023-09-27T23:13:44.795376+0000","last_became_peered":"2023-09-27T23:13:44.795376+0000","last_unstale":"2023-09-27T23:13:48.810125+0000","last_undegraded":"2023-09-27T23:13:48.810125+0000","last_fullsized":"2023-09-27T23:13:48.810125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:02.136037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803338+0000","last_change":"2023-09-27T23:13:44.415778+0000","last_active":"2023-09-27T23:13:48.803338+0000","last_peered":"2023-09-27T23:13:48.803338+0000","last_clean":"2023-09-27T23:13:48.803338+0000","last_became_active":"2023-09-27T23:13:44.415490+0000","last_became_peered":"2023-09-27T23:13:44.415490+0000","last_unstale":"2023-09-27T23:13:48.803338+0000","last_undegraded":"2023-09-27T23:13:48.803338+0000","last_fullsized":"2023-09-27T23:13:48.803338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.828626+0000","last_change":"2023-09-27T23:13:48.823729+0000","last_active":"2023-09-27T23:13:48.828626+0000","last_peered":"2023-09-27T23:13:48.828626+0000","last_clean":"2023-09-27T23:13:48.828626+0000","last_became_active":"2023-09-27T23:13:48.823284+0000","last_became_peered":"2023-09-27T23:13:48.823284+0000","last_unstale":"2023-09-27T23:13:48.828626+0000","last_undegraded":"2023-09-27T23:13:48.828626+0000","last_fullsized":"2023-09-27T23:13:48.828626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:19.656238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820833+0000","last_change":"2023-09-27T23:13:44.794072+0000","last_active":"2023-09-27T23:13:48.820833+0000","last_peered":"2023-09-27T23:13:48.820833+0000","last_clean":"2023-09-27T23:13:48.820833+0000","last_became_active":"2023-09-27T23:13:44.793773+0000","last_became_peered":"2023-09-27T23:13:44.793773+0000","last_unstale":"2023-09-27T23:13:48.820833+0000","last_undegraded":"2023-09-27T23:13:48.820833+0000","last_fullsized":"2023-09-27T23:13:48.820833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804693+0000","last_change":"2023-09-27T23:13:44.426779+0000","last_active":"2023-09-27T23:13:48.804693+0000","last_peered":"2023-09-27T23:13:48.804693+0000","last_clean":"2023-09-27T23:13:48.804693+0000","last_became_active":"2023-09-27T23:13:44.426515+0000","last_became_peered":"2023-09-27T23:13:44.426515+0000","last_unstale":"2023-09-27T23:13:48.804693+0000","last_undegraded":"2023-09-27T23:13:48.804693+0000","last_fullsized":"2023-09-27T23:13:48.804693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:40.093740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842388+0000","last_change":"2023-09-27T23:13:48.839843+0000","last_active":"2023-09-27T23:13:48.842388+0000","last_peered":"2023-09-27T23:13:48.842388+0000","last_clean":"2023-09-27T23:13:48.842388+0000","last_became_active":"2023-09-27T23:13:48.839383+0000","last_became_peered":"2023-09-27T23:13:48.839383+0000","last_unstale":"2023-09-27T23:13:48.842388+0000","last_undegraded":"2023-09-27T23:13:48.842388+0000","last_fullsized":"2023-09-27T23:13:48.842388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810673+0000","last_change":"2023-09-27T23:13:44.802558+0000","last_active":"2023-09-27T23:13:48.810673+0000","last_peered":"2023-09-27T23:13:48.810673+0000","last_clean":"2023-09-27T23:13:48.810673+0000","last_became_active":"2023-09-27T23:13:44.800406+0000","last_became_peered":"2023-09-27T23:13:44.800406+0000","last_unstale":"2023-09-27T23:13:48.810673+0000","last_undegraded":"2023-09-27T23:13:48.810673+0000","last_fullsized":"2023-09-27T23:13:48.810673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804716+0000","last_change":"2023-09-27T23:13:44.426684+0000","last_active":"2023-09-27T23:13:48.804716+0000","last_peered":"2023-09-27T23:13:48.804716+0000","last_clean":"2023-09-27T23:13:48.804716+0000","last_became_active":"2023-09-27T23:13:44.426422+0000","last_became_peered":"2023-09-27T23:13:44.426422+0000","last_unstale":"2023-09-27T23:13:48.804716+0000","last_undegraded":"2023-09-27T23:13:48.804716+0000","last_fullsized":"2023-09-27T23:13:48.804716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:12.799421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829098+0000","last_change":"2023-09-27T23:13:48.827612+0000","last_active":"2023-09-27T23:13:48.829098+0000","last_peered":"2023-09-27T23:13:48.829098+0000","last_clean":"2023-09-27T23:13:48.829098+0000","last_became_active":"2023-09-27T23:13:48.827019+0000","last_became_peered":"2023-09-27T23:13:48.827019+0000","last_unstale":"2023-09-27T23:13:48.829098+0000","last_undegraded":"2023-09-27T23:13:48.829098+0000","last_fullsized":"2023-09-27T23:13:48.829098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:40:33.151042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.835789+0000","last_change":"2023-09-27T23:13:48.830820+0000","last_active":"2023-09-27T23:13:48.835789+0000","last_peered":"2023-09-27T23:13:48.835789+0000","last_clean":"2023-09-27T23:13:48.835789+0000","last_became_active":"2023-09-27T23:13:48.829846+0000","last_became_peered":"2023-09-27T23:13:48.829846+0000","last_unstale":"2023-09-27T23:13:48.835789+0000","last_undegraded":"2023-09-27T23:13:48.835789+0000","last_fullsized":"2023-09-27T23:13:48.835789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809441+0000","last_change":"2023-09-27T23:13:44.413137+0000","last_active":"2023-09-27T23:13:48.809441+0000","last_peered":"2023-09-27T23:13:48.809441+0000","last_clean":"2023-09-27T23:13:48.809441+0000","last_became_active":"2023-09-27T23:13:44.412799+0000","last_became_peered":"2023-09-27T23:13:44.412799+0000","last_unstale":"2023-09-27T23:13:48.809441+0000","last_undegraded":"2023-09-27T23:13:48.809441+0000","last_fullsized":"2023-09-27T23:13:48.809441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825273+0000","last_change":"2023-09-27T23:13:44.793322+0000","last_active":"2023-09-27T23:13:48.825273+0000","last_peered":"2023-09-27T23:13:48.825273+0000","last_clean":"2023-09-27T23:13:48.825273+0000","last_became_active":"2023-09-27T23:13:44.793057+0000","last_became_peered":"2023-09-27T23:13:44.793057+0000","last_unstale":"2023-09-27T23:13:48.825273+0000","last_undegraded":"2023-09-27T23:13:48.825273+0000","last_fullsized":"2023-09-27T23:13:48.825273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842877+0000","last_change":"2023-09-27T23:13:48.839976+0000","last_active":"2023-09-27T23:13:48.842877+0000","last_peered":"2023-09-27T23:13:48.842877+0000","last_clean":"2023-09-27T23:13:48.842877+0000","last_became_active":"2023-09-27T23:13:48.839568+0000","last_became_peered":"2023-09-27T23:13:48.839568+0000","last_unstale":"2023-09-27T23:13:48.842877+0000","last_undegraded":"2023-09-27T23:13:48.842877+0000","last_fullsized":"2023-09-27T23:13:48.842877+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:01.437279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804236+0000","last_change":"2023-09-27T23:13:44.427379+0000","last_active":"2023-09-27T23:13:48.804236+0000","last_peered":"2023-09-27T23:13:48.804236+0000","last_clean":"2023-09-27T23:13:48.804236+0000","last_became_active":"2023-09-27T23:13:44.427094+0000","last_became_peered":"2023-09-27T23:13:44.427094+0000","last_unstale":"2023-09-27T23:13:48.804236+0000","last_undegraded":"2023-09-27T23:13:48.804236+0000","last_fullsized":"2023-09-27T23:13:48.804236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:33.544941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803930+0000","last_change":"2023-09-27T23:13:44.793100+0000","last_active":"2023-09-27T23:13:48.803930+0000","last_peered":"2023-09-27T23:13:48.803930+0000","last_clean":"2023-09-27T23:13:48.803930+0000","last_became_active":"2023-09-27T23:13:44.792493+0000","last_became_peered":"2023-09-27T23:13:44.792493+0000","last_unstale":"2023-09-27T23:13:48.803930+0000","last_undegraded":"2023-09-27T23:13:48.803930+0000","last_fullsized":"2023-09-27T23:13:48.803930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804152+0000","last_change":"2023-09-27T23:13:44.420128+0000","last_active":"2023-09-27T23:13:48.804152+0000","last_peered":"2023-09-27T23:13:48.804152+0000","last_clean":"2023-09-27T23:13:48.804152+0000","last_became_active":"2023-09-27T23:13:44.419240+0000","last_became_peered":"2023-09-27T23:13:44.419240+0000","last_unstale":"2023-09-27T23:13:48.804152+0000","last_undegraded":"2023-09-27T23:13:48.804152+0000","last_fullsized":"2023-09-27T23:13:48.804152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814644+0000","last_change":"2023-09-27T23:13:44.802448+0000","last_active":"2023-09-27T23:13:48.814644+0000","last_peered":"2023-09-27T23:13:48.814644+0000","last_clean":"2023-09-27T23:13:48.814644+0000","last_became_active":"2023-09-27T23:13:44.801316+0000","last_became_peered":"2023-09-27T23:13:44.801316+0000","last_unstale":"2023-09-27T23:13:48.814644+0000","last_undegraded":"2023-09-27T23:13:48.814644+0000","last_fullsized":"2023-09-27T23:13:48.814644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842951+0000","last_change":"2023-09-27T23:13:48.841624+0000","last_active":"2023-09-27T23:13:48.842951+0000","last_peered":"2023-09-27T23:13:48.842951+0000","last_clean":"2023-09-27T23:13:48.842951+0000","last_became_active":"2023-09-27T23:13:48.841104+0000","last_became_peered":"2023-09-27T23:13:48.841104+0000","last_unstale":"2023-09-27T23:13:48.842951+0000","last_undegraded":"2023-09-27T23:13:48.842951+0000","last_fullsized":"2023-09-27T23:13:48.842951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804318+0000","last_change":"2023-09-27T23:13:44.426217+0000","last_active":"2023-09-27T23:13:48.804318+0000","last_peered":"2023-09-27T23:13:48.804318+0000","last_clean":"2023-09-27T23:13:48.804318+0000","last_became_active":"2023-09-27T23:13:44.425907+0000","last_became_peered":"2023-09-27T23:13:44.425907+0000","last_unstale":"2023-09-27T23:13:48.804318+0000","last_undegraded":"2023-09-27T23:13:48.804318+0000","last_fullsized":"2023-09-27T23:13:48.804318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:01.126893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825997+0000","last_change":"2023-09-27T23:13:44.789234+0000","last_active":"2023-09-27T23:13:48.825997+0000","last_peered":"2023-09-27T23:13:48.825997+0000","last_clean":"2023-09-27T23:13:48.825997+0000","last_became_active":"2023-09-27T23:13:44.788948+0000","last_became_peered":"2023-09-27T23:13:44.788948+0000","last_unstale":"2023-09-27T23:13:48.825997+0000","last_undegraded":"2023-09-27T23:13:48.825997+0000","last_fullsized":"2023-09-27T23:13:48.825997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834520+0000","last_change":"2023-09-27T23:13:48.830283+0000","last_active":"2023-09-27T23:13:48.834520+0000","last_peered":"2023-09-27T23:13:48.834520+0000","last_clean":"2023-09-27T23:13:48.834520+0000","last_became_active":"2023-09-27T23:13:48.829788+0000","last_became_peered":"2023-09-27T23:13:48.829788+0000","last_unstale":"2023-09-27T23:13:48.834520+0000","last_undegraded":"2023-09-27T23:13:48.834520+0000","last_fullsized":"2023-09-27T23:13:48.834520+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809476+0000","last_change":"2023-09-27T23:13:44.794796+0000","last_active":"2023-09-27T23:13:48.809476+0000","last_peered":"2023-09-27T23:13:48.809476+0000","last_clean":"2023-09-27T23:13:48.809476+0000","last_became_active":"2023-09-27T23:13:44.793105+0000","last_became_peered":"2023-09-27T23:13:44.793105+0000","last_unstale":"2023-09-27T23:13:48.809476+0000","last_undegraded":"2023-09-27T23:13:48.809476+0000","last_fullsized":"2023-09-27T23:13:48.809476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:13.808565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822377+0000","last_change":"2023-09-27T23:13:44.789337+0000","last_active":"2023-09-27T23:13:48.822377+0000","last_peered":"2023-09-27T23:13:48.822377+0000","last_clean":"2023-09-27T23:13:48.822377+0000","last_became_active":"2023-09-27T23:13:44.789055+0000","last_became_peered":"2023-09-27T23:13:44.789055+0000","last_unstale":"2023-09-27T23:13:48.822377+0000","last_undegraded":"2023-09-27T23:13:48.822377+0000","last_fullsized":"2023-09-27T23:13:48.822377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821093+0000","last_change":"2023-09-27T23:13:44.792891+0000","last_active":"2023-09-27T23:13:48.821093+0000","last_peered":"2023-09-27T23:13:48.821093+0000","last_clean":"2023-09-27T23:13:48.821093+0000","last_became_active":"2023-09-27T23:13:44.792439+0000","last_became_peered":"2023-09-27T23:13:44.792439+0000","last_unstale":"2023-09-27T23:13:48.821093+0000","last_undegraded":"2023-09-27T23:13:48.821093+0000","last_fullsized":"2023-09-27T23:13:48.821093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:33.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825381+0000","last_change":"2023-09-27T23:13:44.787514+0000","last_active":"2023-09-27T23:13:48.825381+0000","last_peered":"2023-09-27T23:13:48.825381+0000","last_clean":"2023-09-27T23:13:48.825381+0000","last_became_active":"2023-09-27T23:13:44.787130+0000","last_became_peered":"2023-09-27T23:13:44.787130+0000","last_unstale":"2023-09-27T23:13:48.825381+0000","last_undegraded":"2023-09-27T23:13:48.825381+0000","last_fullsized":"2023-09-27T23:13:48.825381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.806701+0000","last_change":"2023-09-27T23:13:44.786768+0000","last_active":"2023-09-27T23:13:48.806701+0000","last_peered":"2023-09-27T23:13:48.806701+0000","last_clean":"2023-09-27T23:13:48.806701+0000","last_became_active":"2023-09-27T23:13:44.785642+0000","last_became_peered":"2023-09-27T23:13:44.785642+0000","last_unstale":"2023-09-27T23:13:48.806701+0000","last_undegraded":"2023-09-27T23:13:48.806701+0000","last_fullsized":"2023-09-27T23:13:48.806701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:11:40.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814552+0000","last_change":"2023-09-27T23:13:44.802393+0000","last_active":"2023-09-27T23:13:48.814552+0000","last_peered":"2023-09-27T23:13:48.814552+0000","last_clean":"2023-09-27T23:13:48.814552+0000","last_became_active":"2023-09-27T23:13:44.801038+0000","last_became_peered":"2023-09-27T23:13:44.801038+0000","last_unstale":"2023-09-27T23:13:48.814552+0000","last_undegraded":"2023-09-27T23:13:48.814552+0000","last_fullsized":"2023-09-27T23:13:48.814552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807068+0000","last_change":"2023-09-27T23:13:44.778165+0000","last_active":"2023-09-27T23:13:48.807068+0000","last_peered":"2023-09-27T23:13:48.807068+0000","last_clean":"2023-09-27T23:13:48.807068+0000","last_became_active":"2023-09-27T23:13:44.777922+0000","last_became_peered":"2023-09-27T23:13:44.777922+0000","last_unstale":"2023-09-27T23:13:48.807068+0000","last_undegraded":"2023-09-27T23:13:48.807068+0000","last_fullsized":"2023-09-27T23:13:48.807068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:31.322306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.801910+0000","last_change":"2023-09-27T23:13:44.792009+0000","last_active":"2023-09-27T23:13:48.801910+0000","last_peered":"2023-09-27T23:13:48.801910+0000","last_clean":"2023-09-27T23:13:48.801910+0000","last_became_active":"2023-09-27T23:13:44.791714+0000","last_became_peered":"2023-09-27T23:13:44.791714+0000","last_unstale":"2023-09-27T23:13:48.801910+0000","last_undegraded":"2023-09-27T23:13:48.801910+0000","last_fullsized":"2023-09-27T23:13:48.801910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:10.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802783+0000","last_change":"2023-09-27T23:13:44.800431+0000","last_active":"2023-09-27T23:13:48.802783+0000","last_peered":"2023-09-27T23:13:48.802783+0000","last_clean":"2023-09-27T23:13:48.802783+0000","last_became_active":"2023-09-27T23:13:44.800056+0000","last_became_peered":"2023-09-27T23:13:44.800056+0000","last_unstale":"2023-09-27T23:13:48.802783+0000","last_undegraded":"2023-09-27T23:13:48.802783+0000","last_fullsized":"2023-09-27T23:13:48.802783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822290+0000","last_change":"2023-09-27T23:13:44.788799+0000","last_active":"2023-09-27T23:13:48.822290+0000","last_peered":"2023-09-27T23:13:48.822290+0000","last_clean":"2023-09-27T23:13:48.822290+0000","last_became_active":"2023-09-27T23:13:44.788526+0000","last_became_peered":"2023-09-27T23:13:44.788526+0000","last_unstale":"2023-09-27T23:13:48.822290+0000","last_undegraded":"2023-09-27T23:13:48.822290+0000","last_fullsized":"2023-09-27T23:13:48.822290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803904+0000","last_change":"2023-09-27T23:13:44.795488+0000","last_active":"2023-09-27T23:13:48.803904+0000","last_peered":"2023-09-27T23:13:48.803904+0000","last_clean":"2023-09-27T23:13:48.803904+0000","last_became_active":"2023-09-27T23:13:44.794082+0000","last_became_peered":"2023-09-27T23:13:44.794082+0000","last_unstale":"2023-09-27T23:13:48.803904+0000","last_undegraded":"2023-09-27T23:13:48.803904+0000","last_fullsized":"2023-09-27T23:13:48.803904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803019+0000","last_change":"2023-09-27T23:13:44.802208+0000","last_active":"2023-09-27T23:13:48.803019+0000","last_peered":"2023-09-27T23:13:48.803019+0000","last_clean":"2023-09-27T23:13:48.803019+0000","last_became_active":"2023-09-27T23:13:44.801858+0000","last_became_peered":"2023-09-27T23:13:44.801858+0000","last_unstale":"2023-09-27T23:13:48.803019+0000","last_undegraded":"2023-09-27T23:13:48.803019+0000","last_fullsized":"2023-09-27T23:13:48.803019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821304+0000","last_change":"2023-09-27T23:13:44.795884+0000","last_active":"2023-09-27T23:13:48.821304+0000","last_peered":"2023-09-27T23:13:48.821304+0000","last_clean":"2023-09-27T23:13:48.821304+0000","last_became_active":"2023-09-27T23:13:44.795490+0000","last_became_peered":"2023-09-27T23:13:44.795490+0000","last_unstale":"2023-09-27T23:13:48.821304+0000","last_undegraded":"2023-09-27T23:13:48.821304+0000","last_fullsized":"2023-09-27T23:13:48.821304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:53:40.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810126+0000","last_change":"2023-09-27T23:13:44.791556+0000","last_active":"2023-09-27T23:13:48.810126+0000","last_peered":"2023-09-27T23:13:48.810126+0000","last_clean":"2023-09-27T23:13:48.810126+0000","last_became_active":"2023-09-27T23:13:44.789598+0000","last_became_peered":"2023-09-27T23:13:44.789598+0000","last_unstale":"2023-09-27T23:13:48.810126+0000","last_undegraded":"2023-09-27T23:13:48.810126+0000","last_fullsized":"2023-09-27T23:13:48.810126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:03.782211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821279+0000","last_change":"2023-09-27T23:13:44.794170+0000","last_active":"2023-09-27T23:13:48.821279+0000","last_peered":"2023-09-27T23:13:48.821279+0000","last_clean":"2023-09-27T23:13:48.821279+0000","last_became_active":"2023-09-27T23:13:44.793843+0000","last_became_peered":"2023-09-27T23:13:44.793843+0000","last_unstale":"2023-09-27T23:13:48.821279+0000","last_undegraded":"2023-09-27T23:13:48.821279+0000","last_fullsized":"2023-09-27T23:13:48.821279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804536+0000","last_change":"2023-09-27T23:13:44.795499+0000","last_active":"2023-09-27T23:13:48.804536+0000","last_peered":"2023-09-27T23:13:48.804536+0000","last_clean":"2023-09-27T23:13:48.804536+0000","last_became_active":"2023-09-27T23:13:44.794256+0000","last_became_peered":"2023-09-27T23:13:44.794256+0000","last_unstale":"2023-09-27T23:13:48.804536+0000","last_undegraded":"2023-09-27T23:13:48.804536+0000","last_fullsized":"2023-09-27T23:13:48.804536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821399+0000","last_change":"2023-09-27T23:13:44.797131+0000","last_active":"2023-09-27T23:13:48.821399+0000","last_peered":"2023-09-27T23:13:48.821399+0000","last_clean":"2023-09-27T23:13:48.821399+0000","last_became_active":"2023-09-27T23:13:44.796814+0000","last_became_peered":"2023-09-27T23:13:44.796814+0000","last_unstale":"2023-09-27T23:13:48.821399+0000","last_undegraded":"2023-09-27T23:13:48.821399+0000","last_fullsized":"2023-09-27T23:13:48.821399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804126+0000","last_change":"2023-09-27T23:13:44.794205+0000","last_active":"2023-09-27T23:13:48.804126+0000","last_peered":"2023-09-27T23:13:48.804126+0000","last_clean":"2023-09-27T23:13:48.804126+0000","last_became_active":"2023-09-27T23:13:44.793765+0000","last_became_peered":"2023-09-27T23:13:44.793765+0000","last_unstale":"2023-09-27T23:13:48.804126+0000","last_undegraded":"2023-09-27T23:13:48.804126+0000","last_fullsized":"2023-09-27T23:13:48.804126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813812+0000","last_change":"2023-09-27T23:13:44.777827+0000","last_active":"2023-09-27T23:13:48.813812+0000","last_peered":"2023-09-27T23:13:48.813812+0000","last_clean":"2023-09-27T23:13:48.813812+0000","last_became_active":"2023-09-27T23:13:44.777461+0000","last_became_peered":"2023-09-27T23:13:44.777461+0000","last_unstale":"2023-09-27T23:13:48.813812+0000","last_undegraded":"2023-09-27T23:13:48.813812+0000","last_fullsized":"2023-09-27T23:13:48.813812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810644+0000","last_change":"2023-09-27T23:13:44.787157+0000","last_active":"2023-09-27T23:13:48.810644+0000","last_peered":"2023-09-27T23:13:48.810644+0000","last_clean":"2023-09-27T23:13:48.810644+0000","last_became_active":"2023-09-27T23:13:44.786486+0000","last_became_peered":"2023-09-27T23:13:44.786486+0000","last_unstale":"2023-09-27T23:13:48.810644+0000","last_undegraded":"2023-09-27T23:13:48.810644+0000","last_fullsized":"2023-09-27T23:13:48.810644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:01.890618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810866+0000","last_change":"2023-09-27T23:13:44.796954+0000","last_active":"2023-09-27T23:13:48.810866+0000","last_peered":"2023-09-27T23:13:48.810866+0000","last_clean":"2023-09-27T23:13:48.810866+0000","last_became_active":"2023-09-27T23:13:44.796634+0000","last_became_peered":"2023-09-27T23:13:44.796634+0000","last_unstale":"2023-09-27T23:13:48.810866+0000","last_undegraded":"2023-09-27T23:13:48.810866+0000","last_fullsized":"2023-09-27T23:13:48.810866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:04.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821286+0000","last_change":"2023-09-27T23:13:44.797615+0000","last_active":"2023-09-27T23:13:48.821286+0000","last_peered":"2023-09-27T23:13:48.821286+0000","last_clean":"2023-09-27T23:13:48.821286+0000","last_became_active":"2023-09-27T23:13:44.797325+0000","last_became_peered":"2023-09-27T23:13:44.797325+0000","last_unstale":"2023-09-27T23:13:48.821286+0000","last_undegraded":"2023-09-27T23:13:48.821286+0000","last_fullsized":"2023-09-27T23:13:48.821286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.819151+0000","last_change":"2023-09-27T23:13:44.801326+0000","last_active":"2023-09-27T23:13:48.819151+0000","last_peered":"2023-09-27T23:13:48.819151+0000","last_clean":"2023-09-27T23:13:48.819151+0000","last_became_active":"2023-09-27T23:13:44.800936+0000","last_became_peered":"2023-09-27T23:13:44.800936+0000","last_unstale":"2023-09-27T23:13:48.819151+0000","last_undegraded":"2023-09-27T23:13:48.819151+0000","last_fullsized":"2023-09-27T23:13:48.819151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803144+0000","last_change":"2023-09-27T23:13:44.800702+0000","last_active":"2023-09-27T23:13:48.803144+0000","last_peered":"2023-09-27T23:13:48.803144+0000","last_clean":"2023-09-27T23:13:48.803144+0000","last_became_active":"2023-09-27T23:13:44.800422+0000","last_became_peered":"2023-09-27T23:13:44.800422+0000","last_unstale":"2023-09-27T23:13:48.803144+0000","last_undegraded":"2023-09-27T23:13:48.803144+0000","last_fullsized":"2023-09-27T23:13:48.803144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810589+0000","last_change":"2023-09-27T23:13:44.802316+0000","last_active":"2023-09-27T23:13:48.810589+0000","last_peered":"2023-09-27T23:13:48.810589+0000","last_clean":"2023-09-27T23:13:48.810589+0000","last_became_active":"2023-09-27T23:13:44.800990+0000","last_became_peered":"2023-09-27T23:13:44.800990+0000","last_unstale":"2023-09-27T23:13:48.810589+0000","last_undegraded":"2023-09-27T23:13:48.810589+0000","last_fullsized":"2023-09-27T23:13:48.810589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804015+0000","last_change":"2023-09-27T23:13:44.794033+0000","last_active":"2023-09-27T23:13:48.804015+0000","last_peered":"2023-09-27T23:13:48.804015+0000","last_clean":"2023-09-27T23:13:48.804015+0000","last_became_active":"2023-09-27T23:13:44.793696+0000","last_became_peered":"2023-09-27T23:13:44.793696+0000","last_unstale":"2023-09-27T23:13:48.804015+0000","last_undegraded":"2023-09-27T23:13:48.804015+0000","last_fullsized":"2023-09-27T23:13:48.804015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820743+0000","last_change":"2023-09-27T23:13:44.795881+0000","last_active":"2023-09-27T23:13:48.820743+0000","last_peered":"2023-09-27T23:13:48.820743+0000","last_clean":"2023-09-27T23:13:48.820743+0000","last_became_active":"2023-09-27T23:13:44.795554+0000","last_became_peered":"2023-09-27T23:13:44.795554+0000","last_unstale":"2023-09-27T23:13:48.820743+0000","last_undegraded":"2023-09-27T23:13:48.820743+0000","last_fullsized":"2023-09-27T23:13:48.820743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:16.262346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804794+0000","last_change":"2023-09-27T23:13:44.801496+0000","last_active":"2023-09-27T23:13:48.804794+0000","last_peered":"2023-09-27T23:13:48.804794+0000","last_clean":"2023-09-27T23:13:48.804794+0000","last_became_active":"2023-09-27T23:13:44.801264+0000","last_became_peered":"2023-09-27T23:13:44.801264+0000","last_unstale":"2023-09-27T23:13:48.804794+0000","last_undegraded":"2023-09-27T23:13:48.804794+0000","last_fullsized":"2023-09-27T23:13:48.804794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812717+0000","last_change":"2023-09-27T23:13:44.790757+0000","last_active":"2023-09-27T23:13:48.812717+0000","last_peered":"2023-09-27T23:13:48.812717+0000","last_clean":"2023-09-27T23:13:48.812717+0000","last_became_active":"2023-09-27T23:13:44.790388+0000","last_became_peered":"2023-09-27T23:13:44.790388+0000","last_unstale":"2023-09-27T23:13:48.812717+0000","last_undegraded":"2023-09-27T23:13:48.812717+0000","last_fullsized":"2023-09-27T23:13:48.812717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821180+0000","last_change":"2023-09-27T23:13:44.799201+0000","last_active":"2023-09-27T23:13:48.821180+0000","last_peered":"2023-09-27T23:13:48.821180+0000","last_clean":"2023-09-27T23:13:48.821180+0000","last_became_active":"2023-09-27T23:13:44.798712+0000","last_became_peered":"2023-09-27T23:13:44.798712+0000","last_unstale":"2023-09-27T23:13:48.821180+0000","last_undegraded":"2023-09-27T23:13:48.821180+0000","last_fullsized":"2023-09-27T23:13:48.821180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815177+0000","last_change":"2023-09-27T23:13:44.780452+0000","last_active":"2023-09-27T23:13:48.815177+0000","last_peered":"2023-09-27T23:13:48.815177+0000","last_clean":"2023-09-27T23:13:48.815177+0000","last_became_active":"2023-09-27T23:13:44.780111+0000","last_became_peered":"2023-09-27T23:13:44.780111+0000","last_unstale":"2023-09-27T23:13:48.815177+0000","last_undegraded":"2023-09-27T23:13:48.815177+0000","last_fullsized":"2023-09-27T23:13:48.815177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810755+0000","last_change":"2023-09-27T23:13:44.794473+0000","last_active":"2023-09-27T23:13:48.810755+0000","last_peered":"2023-09-27T23:13:48.810755+0000","last_clean":"2023-09-27T23:13:48.810755+0000","last_became_active":"2023-09-27T23:13:44.794187+0000","last_became_peered":"2023-09-27T23:13:44.794187+0000","last_unstale":"2023-09-27T23:13:48.810755+0000","last_undegraded":"2023-09-27T23:13:48.810755+0000","last_fullsized":"2023-09-27T23:13:48.810755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9300,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362540,"statfs":{"total":96636764160,"available":96627240960,"internally_reserved":0,"allocated":929792,"data_stored":717011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8188,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363652,"statfs":{"total":96636764160,"available":96628379648,"internally_reserved":0,"allocated":315392,"data_stored":125515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8264,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363576,"statfs":{"total":96636764160,"available":96628301824,"internally_reserved":0,"allocated":327680,"data_stored":126483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":124981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":905216,"data_stored":715813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:13:57.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 Processor -- start 2023-09-27T23:13:57.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 -- start start 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3078158e70 0x7f30780afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30780cc470 0x7f30780cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30780ca5a0 con 0x7f307806a540 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307806dac0 con 0x7f3078158e70 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.962+0000 7f307e9e5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30780b0260 con 0x7f30780cc470 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e1e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30780cc470 0x7f30780cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307d9e3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3078158e70 0x7f30780afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307d1e2700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f307806a540 0x7f307806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59574/0 (socket says 172.21.15.64:59574) 2023-09-27T23:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307d1e2700 1 -- 172.21.15.64:0/2319131236 learned_addr learned my addr 172.21.15.64:0/2319131236 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e1e4700 1 -- 172.21.15.64:0/2319131236 >> v2:172.21.15.123:3300/0 conn(0x7f3078158e70 msgr2=0x7f30780afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e1e4700 1 --2- 172.21.15.64:0/2319131236 >> v2:172.21.15.123:3300/0 conn(0x7f3078158e70 0x7f30780afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e1e4700 1 -- 172.21.15.64:0/2319131236 >> v1:172.21.15.123:6789/0 conn(0x7f307806a540 legacy=0x7f307806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e1e4700 1 -- 172.21.15.64:0/2319131236 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307806d220 con 0x7f30780cc470 2023-09-27T23:13:57.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307d9e3700 1 --2- 172.21.15.64:0/2319131236 >> v2:172.21.15.123:3300/0 conn(0x7f3078158e70 0x7f30780afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:57.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e1e4700 1 --2- 172.21.15.64:0/2319131236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30780cc470 0x7f30780cc8d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30700098e0 tx=0x7f30700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5896f9d0e105cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307c9e1700 1 -- 172.21.15.64:0/2319131236 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070011630 con 0x7f30780cc470 2023-09-27T23:13:57.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307c9e1700 1 -- 172.21.15.64:0/2319131236 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070011790 con 0x7f30780cc470 2023-09-27T23:13:57.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307c9e1700 1 -- 172.21.15.64:0/2319131236 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070011b30 con 0x7f30780cc470 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 -- 172.21.15.64:0/2319131236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30780cc470 msgr2=0x7f30780cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 --2- 172.21.15.64:0/2319131236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30780cc470 0x7f30780cc8d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30700098e0 tx=0x7f30700091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 -- 172.21.15.64:0/2319131236 shutdown_connections 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 --2- 172.21.15.64:0/2319131236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30780cc470 0x7f30780cc8d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 --2- 172.21.15.64:0/2319131236 >> v2:172.21.15.123:3300/0 conn(0x7f3078158e70 0x7f30780afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 -- 172.21.15.64:0/2319131236 >> 172.21.15.64:0/2319131236 conn(0x7f30780670e0 msgr2=0x7f30780674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 -- 172.21.15.64:0/2319131236 shutdown_connections 2023-09-27T23:13:57.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.966+0000 7f307e9e5700 1 -- 172.21.15.64:0/2319131236 wait complete. 2023-09-27T23:13:57.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 Processor -- start 2023-09-27T23:13:57.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 -- start start 2023-09-27T23:13:57.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f307806a540 0x7f30780b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 0x7f30780c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3078158e70 0x7f307815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30780c4f20 con 0x7f30780cc470 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307815b1b0 con 0x7f307806a540 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e9e5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307815a900 con 0x7f3078158e70 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 0x7f30780c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 0x7f30780c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44266/0 (socket says 172.21.15.64:44266) 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d9e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f307806a540 0x7f30780b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 -- 172.21.15.64:0/3175456371 learned_addr learned my addr 172.21.15.64:0/3175456371 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307e1e4700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3078158e70 0x7f307815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 -- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3078158e70 msgr2=0x7f307815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3078158e70 0x7f307815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 -- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f307806a540 msgr2=0x7f30780b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:57.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f307806a540 0x7f30780b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:57.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 -- 172.21.15.64:0/3175456371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068009a20 con 0x7f30780cc470 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d9e3700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f307806a540 0x7f30780b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f307d1e2700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 0x7f30780c42d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f307400c810 tx=0x7f3074007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074005b30 con 0x7f30780cc470 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.970+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074005cb0 con 0x7f30780cc470 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.974+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3070009420 con 0x7f30780cc470 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.974+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074007340 con 0x7f30780cc470 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.974+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30780c4c40 con 0x7f30780cc470 2023-09-27T23:13:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.974+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307804be40 con 0x7f30780cc470 2023-09-27T23:13:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.974+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f307401b400 con 0x7f30780cc470 2023-09-27T23:13:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.974+0000 7f306dffb700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3064060950 0x7f3064062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.978+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f307407cc00 con 0x7f30780cc470 2023-09-27T23:13:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.978+0000 7f307d9e3700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3064060950 0x7f3064062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:57.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.978+0000 7f307d9e3700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3064060950 0x7f3064062e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30680099f0 tx=0x7f306800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:57.986+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3074083080 con 0x7f30780cc470 2023-09-27T23:13:58.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.142+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30780cd300 con 0x7f3064060950 2023-09-27T23:13:58.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.162+0000 7f306dffb700 1 -- 172.21.15.64:0/3175456371 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+364820 (secure 0 0 0) 0x7f30780cd300 con 0x7f3064060950 2023-09-27T23:13:58.170 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3064060950 msgr2=0x7f3064062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3064060950 0x7f3064062e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30680099f0 tx=0x7f306800b040 comp rx=0 tx=0).stop 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 msgr2=0x7f30780c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 0x7f30780c42d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f307400c810 tx=0x7f3074007f30 comp rx=0 tx=0).stop 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 shutdown_connections 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3064060950 0x7f3064062e00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3078158e70 0x7f307815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30780cc470 0x7f30780c42d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 --2- 172.21.15.64:0/3175456371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f307806a540 0x7f30780b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.166+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 >> 172.21.15.64:0/3175456371 conn(0x7f30780670e0 msgr2=0x7f3078158190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:58.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.170+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 shutdown_connections 2023-09-27T23:13:58.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.170+0000 7f307e9e5700 1 -- 172.21.15.64:0/3175456371 wait complete. 2023-09-27T23:13:58.174 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:13:58.188 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-09-27T23:13:57.147323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":303,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67688,"kb_used_data":3624,"kb_used_omap":0,"kb_used_meta":64000,"kb_avail":754907032,"statfs":{"total":773094113280,"available":773024800768,"internally_reserved":0,"allocated":3710976,"data_stored":2184476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65536000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.350658"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838997+0000","last_change":"2023-09-27T23:13:48.825219+0000","last_active":"2023-09-27T23:13:48.838997+0000","last_peered":"2023-09-27T23:13:48.838997+0000","last_clean":"2023-09-27T23:13:48.838997+0000","last_became_active":"2023-09-27T23:13:48.824822+0000","last_became_peered":"2023-09-27T23:13:48.824822+0000","last_unstale":"2023-09-27T23:13:48.838997+0000","last_undegraded":"2023-09-27T23:13:48.838997+0000","last_fullsized":"2023-09-27T23:13:48.838997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:34.128831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803408+0000","last_change":"2023-09-27T23:13:44.424249+0000","last_active":"2023-09-27T23:13:48.803408+0000","last_peered":"2023-09-27T23:13:48.803408+0000","last_clean":"2023-09-27T23:13:48.803408+0000","last_became_active":"2023-09-27T23:13:44.423915+0000","last_became_peered":"2023-09-27T23:13:44.423915+0000","last_unstale":"2023-09-27T23:13:48.803408+0000","last_undegraded":"2023-09-27T23:13:48.803408+0000","last_fullsized":"2023-09-27T23:13:48.803408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821460+0000","last_change":"2023-09-27T23:13:44.796968+0000","last_active":"2023-09-27T23:13:48.821460+0000","last_peered":"2023-09-27T23:13:48.821460+0000","last_clean":"2023-09-27T23:13:48.821460+0000","last_became_active":"2023-09-27T23:13:44.795519+0000","last_became_peered":"2023-09-27T23:13:44.795519+0000","last_unstale":"2023-09-27T23:13:48.821460+0000","last_undegraded":"2023-09-27T23:13:48.821460+0000","last_fullsized":"2023-09-27T23:13:48.821460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843929+0000","last_change":"2023-09-27T23:13:48.839540+0000","last_active":"2023-09-27T23:13:48.843929+0000","last_peered":"2023-09-27T23:13:48.843929+0000","last_clean":"2023-09-27T23:13:48.843929+0000","last_became_active":"2023-09-27T23:13:48.839079+0000","last_became_peered":"2023-09-27T23:13:48.839079+0000","last_unstale":"2023-09-27T23:13:48.843929+0000","last_undegraded":"2023-09-27T23:13:48.843929+0000","last_fullsized":"2023-09-27T23:13:48.843929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825433+0000","last_change":"2023-09-27T23:13:44.416457+0000","last_active":"2023-09-27T23:13:48.825433+0000","last_peered":"2023-09-27T23:13:48.825433+0000","last_clean":"2023-09-27T23:13:48.825433+0000","last_became_active":"2023-09-27T23:13:44.416135+0000","last_became_peered":"2023-09-27T23:13:44.416135+0000","last_unstale":"2023-09-27T23:13:48.825433+0000","last_undegraded":"2023-09-27T23:13:48.825433+0000","last_fullsized":"2023-09-27T23:13:48.825433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822547+0000","last_change":"2023-09-27T23:13:44.783780+0000","last_active":"2023-09-27T23:13:48.822547+0000","last_peered":"2023-09-27T23:13:48.822547+0000","last_clean":"2023-09-27T23:13:48.822547+0000","last_became_active":"2023-09-27T23:13:44.783441+0000","last_became_peered":"2023-09-27T23:13:44.783441+0000","last_unstale":"2023-09-27T23:13:48.822547+0000","last_undegraded":"2023-09-27T23:13:48.822547+0000","last_fullsized":"2023-09-27T23:13:48.822547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840431+0000","last_change":"2023-09-27T23:13:48.835419+0000","last_active":"2023-09-27T23:13:48.840431+0000","last_peered":"2023-09-27T23:13:48.840431+0000","last_clean":"2023-09-27T23:13:48.840431+0000","last_became_active":"2023-09-27T23:13:48.834981+0000","last_became_peered":"2023-09-27T23:13:48.834981+0000","last_unstale":"2023-09-27T23:13:48.840431+0000","last_undegraded":"2023-09-27T23:13:48.840431+0000","last_fullsized":"2023-09-27T23:13:48.840431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814741+0000","last_change":"2023-09-27T23:13:44.427390+0000","last_active":"2023-09-27T23:13:48.814741+0000","last_peered":"2023-09-27T23:13:48.814741+0000","last_clean":"2023-09-27T23:13:48.814741+0000","last_became_active":"2023-09-27T23:13:44.426407+0000","last_became_peered":"2023-09-27T23:13:44.426407+0000","last_unstale":"2023-09-27T23:13:48.814741+0000","last_undegraded":"2023-09-27T23:13:48.814741+0000","last_fullsized":"2023-09-27T23:13:48.814741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820888+0000","last_change":"2023-09-27T23:13:44.803765+0000","last_active":"2023-09-27T23:13:48.820888+0000","last_peered":"2023-09-27T23:13:48.820888+0000","last_clean":"2023-09-27T23:13:48.820888+0000","last_became_active":"2023-09-27T23:13:44.803448+0000","last_became_peered":"2023-09-27T23:13:44.803448+0000","last_unstale":"2023-09-27T23:13:48.820888+0000","last_undegraded":"2023-09-27T23:13:48.820888+0000","last_fullsized":"2023-09-27T23:13:48.820888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:14.091887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834465+0000","last_change":"2023-09-27T23:13:48.831362+0000","last_active":"2023-09-27T23:13:48.834465+0000","last_peered":"2023-09-27T23:13:48.834465+0000","last_clean":"2023-09-27T23:13:48.834465+0000","last_became_active":"2023-09-27T23:13:48.830786+0000","last_became_peered":"2023-09-27T23:13:48.830786+0000","last_unstale":"2023-09-27T23:13:48.834465+0000","last_undegraded":"2023-09-27T23:13:48.834465+0000","last_fullsized":"2023-09-27T23:13:48.834465+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:24:25.298387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811157+0000","last_change":"2023-09-27T23:13:44.425581+0000","last_active":"2023-09-27T23:13:48.811157+0000","last_peered":"2023-09-27T23:13:48.811157+0000","last_clean":"2023-09-27T23:13:48.811157+0000","last_became_active":"2023-09-27T23:13:44.425084+0000","last_became_peered":"2023-09-27T23:13:44.425084+0000","last_unstale":"2023-09-27T23:13:48.811157+0000","last_undegraded":"2023-09-27T23:13:48.811157+0000","last_fullsized":"2023-09-27T23:13:48.811157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821059+0000","last_change":"2023-09-27T23:13:44.803698+0000","last_active":"2023-09-27T23:13:48.821059+0000","last_peered":"2023-09-27T23:13:48.821059+0000","last_clean":"2023-09-27T23:13:48.821059+0000","last_became_active":"2023-09-27T23:13:44.803316+0000","last_became_peered":"2023-09-27T23:13:44.803316+0000","last_unstale":"2023-09-27T23:13:48.821059+0000","last_undegraded":"2023-09-27T23:13:48.821059+0000","last_fullsized":"2023-09-27T23:13:48.821059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:10:53.031343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843374+0000","last_change":"2023-09-27T23:13:48.835746+0000","last_active":"2023-09-27T23:13:48.843374+0000","last_peered":"2023-09-27T23:13:48.843374+0000","last_clean":"2023-09-27T23:13:48.843374+0000","last_became_active":"2023-09-27T23:13:48.835306+0000","last_became_peered":"2023-09-27T23:13:48.835306+0000","last_unstale":"2023-09-27T23:13:48.843374+0000","last_undegraded":"2023-09-27T23:13:48.843374+0000","last_fullsized":"2023-09-27T23:13:48.843374+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:00:48.785735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804887+0000","last_change":"2023-09-27T23:13:44.428693+0000","last_active":"2023-09-27T23:13:48.804887+0000","last_peered":"2023-09-27T23:13:48.804887+0000","last_clean":"2023-09-27T23:13:48.804887+0000","last_became_active":"2023-09-27T23:13:44.428213+0000","last_became_peered":"2023-09-27T23:13:44.428213+0000","last_unstale":"2023-09-27T23:13:48.804887+0000","last_undegraded":"2023-09-27T23:13:48.804887+0000","last_fullsized":"2023-09-27T23:13:48.804887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810898+0000","last_change":"2023-09-27T23:13:44.791611+0000","last_active":"2023-09-27T23:13:48.810898+0000","last_peered":"2023-09-27T23:13:48.810898+0000","last_clean":"2023-09-27T23:13:48.810898+0000","last_became_active":"2023-09-27T23:13:44.791292+0000","last_became_peered":"2023-09-27T23:13:44.791292+0000","last_unstale":"2023-09-27T23:13:48.810898+0000","last_undegraded":"2023-09-27T23:13:48.810898+0000","last_fullsized":"2023-09-27T23:13:48.810898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:08.690781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840009+0000","last_change":"2023-09-27T23:13:48.827264+0000","last_active":"2023-09-27T23:13:48.840009+0000","last_peered":"2023-09-27T23:13:48.840009+0000","last_clean":"2023-09-27T23:13:48.840009+0000","last_became_active":"2023-09-27T23:13:48.826832+0000","last_became_peered":"2023-09-27T23:13:48.826832+0000","last_unstale":"2023-09-27T23:13:48.840009+0000","last_undegraded":"2023-09-27T23:13:48.840009+0000","last_fullsized":"2023-09-27T23:13:48.840009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:48:20.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821181+0000","last_change":"2023-09-27T23:13:44.423920+0000","last_active":"2023-09-27T23:13:48.821181+0000","last_peered":"2023-09-27T23:13:48.821181+0000","last_clean":"2023-09-27T23:13:48.821181+0000","last_became_active":"2023-09-27T23:13:44.422754+0000","last_became_peered":"2023-09-27T23:13:44.422754+0000","last_unstale":"2023-09-27T23:13:48.821181+0000","last_undegraded":"2023-09-27T23:13:48.821181+0000","last_fullsized":"2023-09-27T23:13:48.821181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:16.888839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825025+0000","last_change":"2023-09-27T23:13:44.790022+0000","last_active":"2023-09-27T23:13:48.825025+0000","last_peered":"2023-09-27T23:13:48.825025+0000","last_clean":"2023-09-27T23:13:48.825025+0000","last_became_active":"2023-09-27T23:13:44.789052+0000","last_became_peered":"2023-09-27T23:13:44.789052+0000","last_unstale":"2023-09-27T23:13:48.825025+0000","last_undegraded":"2023-09-27T23:13:48.825025+0000","last_fullsized":"2023-09-27T23:13:48.825025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.841311+0000","last_change":"2023-09-27T23:13:48.835634+0000","last_active":"2023-09-27T23:13:48.841311+0000","last_peered":"2023-09-27T23:13:48.841311+0000","last_clean":"2023-09-27T23:13:48.841311+0000","last_became_active":"2023-09-27T23:13:48.834989+0000","last_became_peered":"2023-09-27T23:13:48.834989+0000","last_unstale":"2023-09-27T23:13:48.841311+0000","last_undegraded":"2023-09-27T23:13:48.841311+0000","last_fullsized":"2023-09-27T23:13:48.841311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:09.944701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802920+0000","last_change":"2023-09-27T23:13:44.424671+0000","last_active":"2023-09-27T23:13:48.802920+0000","last_peered":"2023-09-27T23:13:48.802920+0000","last_clean":"2023-09-27T23:13:48.802920+0000","last_became_active":"2023-09-27T23:13:44.424321+0000","last_became_peered":"2023-09-27T23:13:44.424321+0000","last_unstale":"2023-09-27T23:13:48.802920+0000","last_undegraded":"2023-09-27T23:13:48.802920+0000","last_fullsized":"2023-09-27T23:13:48.802920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:44:43.453425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803496+0000","last_change":"2023-09-27T23:13:44.792445+0000","last_active":"2023-09-27T23:13:48.803496+0000","last_peered":"2023-09-27T23:13:48.803496+0000","last_clean":"2023-09-27T23:13:48.803496+0000","last_became_active":"2023-09-27T23:13:44.790669+0000","last_became_peered":"2023-09-27T23:13:44.790669+0000","last_unstale":"2023-09-27T23:13:48.803496+0000","last_undegraded":"2023-09-27T23:13:48.803496+0000","last_fullsized":"2023-09-27T23:13:48.803496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833795+0000","last_change":"2023-09-27T23:13:48.824674+0000","last_active":"2023-09-27T23:13:48.833795+0000","last_peered":"2023-09-27T23:13:48.833795+0000","last_clean":"2023-09-27T23:13:48.833795+0000","last_became_active":"2023-09-27T23:13:48.824220+0000","last_became_peered":"2023-09-27T23:13:48.824220+0000","last_unstale":"2023-09-27T23:13:48.833795+0000","last_undegraded":"2023-09-27T23:13:48.833795+0000","last_fullsized":"2023-09-27T23:13:48.833795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822020+0000","last_change":"2023-09-27T23:13:44.415940+0000","last_active":"2023-09-27T23:13:48.822020+0000","last_peered":"2023-09-27T23:13:48.822020+0000","last_clean":"2023-09-27T23:13:48.822020+0000","last_became_active":"2023-09-27T23:13:44.415355+0000","last_became_peered":"2023-09-27T23:13:44.415355+0000","last_unstale":"2023-09-27T23:13:48.822020+0000","last_undegraded":"2023-09-27T23:13:48.822020+0000","last_fullsized":"2023-09-27T23:13:48.822020+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813805+0000","last_change":"2023-09-27T23:13:44.807290+0000","last_active":"2023-09-27T23:13:48.813805+0000","last_peered":"2023-09-27T23:13:48.813805+0000","last_clean":"2023-09-27T23:13:48.813805+0000","last_became_active":"2023-09-27T23:13:44.806991+0000","last_became_peered":"2023-09-27T23:13:44.806991+0000","last_unstale":"2023-09-27T23:13:48.813805+0000","last_undegraded":"2023-09-27T23:13:48.813805+0000","last_fullsized":"2023-09-27T23:13:48.813805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:41.102884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821934+0000","last_change":"2023-09-27T23:13:44.797997+0000","last_active":"2023-09-27T23:13:48.821934+0000","last_peered":"2023-09-27T23:13:48.821934+0000","last_clean":"2023-09-27T23:13:48.821934+0000","last_became_active":"2023-09-27T23:13:44.797689+0000","last_became_peered":"2023-09-27T23:13:44.797689+0000","last_unstale":"2023-09-27T23:13:48.821934+0000","last_undegraded":"2023-09-27T23:13:48.821934+0000","last_fullsized":"2023-09-27T23:13:48.821934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803641+0000","last_change":"2023-09-27T23:13:44.420293+0000","last_active":"2023-09-27T23:13:48.803641+0000","last_peered":"2023-09-27T23:13:48.803641+0000","last_clean":"2023-09-27T23:13:48.803641+0000","last_became_active":"2023-09-27T23:13:44.419584+0000","last_became_peered":"2023-09-27T23:13:44.419584+0000","last_unstale":"2023-09-27T23:13:48.803641+0000","last_undegraded":"2023-09-27T23:13:48.803641+0000","last_fullsized":"2023-09-27T23:13:48.803641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843828+0000","last_change":"2023-09-27T23:13:48.842799+0000","last_active":"2023-09-27T23:13:48.843828+0000","last_peered":"2023-09-27T23:13:48.843828+0000","last_clean":"2023-09-27T23:13:48.843828+0000","last_became_active":"2023-09-27T23:13:48.842271+0000","last_became_peered":"2023-09-27T23:13:48.842271+0000","last_unstale":"2023-09-27T23:13:48.843828+0000","last_undegraded":"2023-09-27T23:13:48.843828+0000","last_fullsized":"2023-09-27T23:13:48.843828+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:26.172918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822463+0000","last_change":"2023-09-27T23:13:44.791363+0000","last_active":"2023-09-27T23:13:48.822463+0000","last_peered":"2023-09-27T23:13:48.822463+0000","last_clean":"2023-09-27T23:13:48.822463+0000","last_became_active":"2023-09-27T23:13:44.786248+0000","last_became_peered":"2023-09-27T23:13:44.786248+0000","last_unstale":"2023-09-27T23:13:48.822463+0000","last_undegraded":"2023-09-27T23:13:48.822463+0000","last_fullsized":"2023-09-27T23:13:48.822463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:22.779025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815748+0000","last_change":"2023-09-27T23:13:44.416087+0000","last_active":"2023-09-27T23:13:48.815748+0000","last_peered":"2023-09-27T23:13:48.815748+0000","last_clean":"2023-09-27T23:13:48.815748+0000","last_became_active":"2023-09-27T23:13:44.415755+0000","last_became_peered":"2023-09-27T23:13:44.415755+0000","last_unstale":"2023-09-27T23:13:48.815748+0000","last_undegraded":"2023-09-27T23:13:48.815748+0000","last_fullsized":"2023-09-27T23:13:48.815748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826650+0000","last_change":"2023-09-27T23:13:48.825181+0000","last_active":"2023-09-27T23:13:48.826650+0000","last_peered":"2023-09-27T23:13:48.826650+0000","last_clean":"2023-09-27T23:13:48.826650+0000","last_became_active":"2023-09-27T23:13:48.824750+0000","last_became_peered":"2023-09-27T23:13:48.824750+0000","last_unstale":"2023-09-27T23:13:48.826650+0000","last_undegraded":"2023-09-27T23:13:48.826650+0000","last_fullsized":"2023-09-27T23:13:48.826650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803126+0000","last_change":"2023-09-27T23:13:44.423841+0000","last_active":"2023-09-27T23:13:48.803126+0000","last_peered":"2023-09-27T23:13:48.803126+0000","last_clean":"2023-09-27T23:13:48.803126+0000","last_became_active":"2023-09-27T23:13:44.423450+0000","last_became_peered":"2023-09-27T23:13:44.423450+0000","last_unstale":"2023-09-27T23:13:48.803126+0000","last_undegraded":"2023-09-27T23:13:48.803126+0000","last_fullsized":"2023-09-27T23:13:48.803126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804382+0000","last_change":"2023-09-27T23:13:44.789353+0000","last_active":"2023-09-27T23:13:48.804382+0000","last_peered":"2023-09-27T23:13:48.804382+0000","last_clean":"2023-09-27T23:13:48.804382+0000","last_became_active":"2023-09-27T23:13:44.788925+0000","last_became_peered":"2023-09-27T23:13:44.788925+0000","last_unstale":"2023-09-27T23:13:48.804382+0000","last_undegraded":"2023-09-27T23:13:48.804382+0000","last_fullsized":"2023-09-27T23:13:48.804382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:24.773257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842925+0000","last_change":"2023-09-27T23:13:48.840258+0000","last_active":"2023-09-27T23:13:48.842925+0000","last_peered":"2023-09-27T23:13:48.842925+0000","last_clean":"2023-09-27T23:13:48.842925+0000","last_became_active":"2023-09-27T23:13:48.839796+0000","last_became_peered":"2023-09-27T23:13:48.839796+0000","last_unstale":"2023-09-27T23:13:48.842925+0000","last_undegraded":"2023-09-27T23:13:48.842925+0000","last_fullsized":"2023-09-27T23:13:48.842925+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821682+0000","last_change":"2023-09-27T23:13:44.422959+0000","last_active":"2023-09-27T23:13:48.821682+0000","last_peered":"2023-09-27T23:13:48.821682+0000","last_clean":"2023-09-27T23:13:48.821682+0000","last_became_active":"2023-09-27T23:13:44.420682+0000","last_became_peered":"2023-09-27T23:13:44.420682+0000","last_unstale":"2023-09-27T23:13:48.821682+0000","last_undegraded":"2023-09-27T23:13:48.821682+0000","last_fullsized":"2023-09-27T23:13:48.821682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821157+0000","last_change":"2023-09-27T23:13:44.792715+0000","last_active":"2023-09-27T23:13:48.821157+0000","last_peered":"2023-09-27T23:13:48.821157+0000","last_clean":"2023-09-27T23:13:48.821157+0000","last_became_active":"2023-09-27T23:13:44.792356+0000","last_became_peered":"2023-09-27T23:13:44.792356+0000","last_unstale":"2023-09-27T23:13:48.821157+0000","last_undegraded":"2023-09-27T23:13:48.821157+0000","last_fullsized":"2023-09-27T23:13:48.821157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843134+0000","last_change":"2023-09-27T23:13:48.830683+0000","last_active":"2023-09-27T23:13:48.843134+0000","last_peered":"2023-09-27T23:13:48.843134+0000","last_clean":"2023-09-27T23:13:48.843134+0000","last_became_active":"2023-09-27T23:13:48.830149+0000","last_became_peered":"2023-09-27T23:13:48.830149+0000","last_unstale":"2023-09-27T23:13:48.843134+0000","last_undegraded":"2023-09-27T23:13:48.843134+0000","last_fullsized":"2023-09-27T23:13:48.843134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:31:07.801049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821012+0000","last_change":"2023-09-27T23:13:44.807208+0000","last_active":"2023-09-27T23:13:48.821012+0000","last_peered":"2023-09-27T23:13:48.821012+0000","last_clean":"2023-09-27T23:13:48.821012+0000","last_became_active":"2023-09-27T23:13:44.806937+0000","last_became_peered":"2023-09-27T23:13:44.806937+0000","last_unstale":"2023-09-27T23:13:48.821012+0000","last_undegraded":"2023-09-27T23:13:48.821012+0000","last_fullsized":"2023-09-27T23:13:48.821012+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:03:17.944613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813854+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:13:48.813854+0000","last_peered":"2023-09-27T23:13:48.813854+0000","last_clean":"2023-09-27T23:13:48.813854+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:13:48.813854+0000","last_undegraded":"2023-09-27T23:13:48.813854+0000","last_fullsized":"2023-09-27T23:13:48.813854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-27T23:13:49.971771+0000","last_change":"2023-09-27T23:13:44.421149+0000","last_active":"2023-09-27T23:13:49.971771+0000","last_peered":"2023-09-27T23:13:49.971771+0000","last_clean":"2023-09-27T23:13:49.971771+0000","last_became_active":"2023-09-27T23:13:44.419644+0000","last_became_peered":"2023-09-27T23:13:44.419644+0000","last_unstale":"2023-09-27T23:13:49.971771+0000","last_undegraded":"2023-09-27T23:13:49.971771+0000","last_fullsized":"2023-09-27T23:13:49.971771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842921+0000","last_change":"2023-09-27T23:13:48.840751+0000","last_active":"2023-09-27T23:13:48.842921+0000","last_peered":"2023-09-27T23:13:48.842921+0000","last_clean":"2023-09-27T23:13:48.842921+0000","last_became_active":"2023-09-27T23:13:48.840353+0000","last_became_peered":"2023-09-27T23:13:48.840353+0000","last_unstale":"2023-09-27T23:13:48.842921+0000","last_undegraded":"2023-09-27T23:13:48.842921+0000","last_fullsized":"2023-09-27T23:13:48.842921+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:59:09.353088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821991+0000","last_change":"2023-09-27T23:13:44.791105+0000","last_active":"2023-09-27T23:13:48.821991+0000","last_peered":"2023-09-27T23:13:48.821991+0000","last_clean":"2023-09-27T23:13:48.821991+0000","last_became_active":"2023-09-27T23:13:44.790824+0000","last_became_peered":"2023-09-27T23:13:44.790824+0000","last_unstale":"2023-09-27T23:13:48.821991+0000","last_undegraded":"2023-09-27T23:13:48.821991+0000","last_fullsized":"2023-09-27T23:13:48.821991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807265+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:13:48.807265+0000","last_peered":"2023-09-27T23:13:48.807265+0000","last_clean":"2023-09-27T23:13:48.807265+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:13:48.807265+0000","last_undegraded":"2023-09-27T23:13:48.807265+0000","last_fullsized":"2023-09-27T23:13:48.807265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825400+0000","last_change":"2023-09-27T23:13:44.421998+0000","last_active":"2023-09-27T23:13:48.825400+0000","last_peered":"2023-09-27T23:13:48.825400+0000","last_clean":"2023-09-27T23:13:48.825400+0000","last_became_active":"2023-09-27T23:13:44.416358+0000","last_became_peered":"2023-09-27T23:13:44.416358+0000","last_unstale":"2023-09-27T23:13:48.825400+0000","last_undegraded":"2023-09-27T23:13:48.825400+0000","last_fullsized":"2023-09-27T23:13:48.825400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840017+0000","last_change":"2023-09-27T23:13:48.827758+0000","last_active":"2023-09-27T23:13:48.840017+0000","last_peered":"2023-09-27T23:13:48.840017+0000","last_clean":"2023-09-27T23:13:48.840017+0000","last_became_active":"2023-09-27T23:13:48.827144+0000","last_became_peered":"2023-09-27T23:13:48.827144+0000","last_unstale":"2023-09-27T23:13:48.840017+0000","last_undegraded":"2023-09-27T23:13:48.840017+0000","last_fullsized":"2023-09-27T23:13:48.840017+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:02:44.973308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804641+0000","last_change":"2023-09-27T23:13:44.807107+0000","last_active":"2023-09-27T23:13:48.804641+0000","last_peered":"2023-09-27T23:13:48.804641+0000","last_clean":"2023-09-27T23:13:48.804641+0000","last_became_active":"2023-09-27T23:13:44.806739+0000","last_became_peered":"2023-09-27T23:13:44.806739+0000","last_unstale":"2023-09-27T23:13:48.804641+0000","last_undegraded":"2023-09-27T23:13:48.804641+0000","last_fullsized":"2023-09-27T23:13:48.804641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803837+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:13:48.803837+0000","last_peered":"2023-09-27T23:13:48.803837+0000","last_clean":"2023-09-27T23:13:48.803837+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:13:48.803837+0000","last_undegraded":"2023-09-27T23:13:48.803837+0000","last_fullsized":"2023-09-27T23:13:48.803837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802757+0000","last_change":"2023-09-27T23:13:44.419685+0000","last_active":"2023-09-27T23:13:48.802757+0000","last_peered":"2023-09-27T23:13:48.802757+0000","last_clean":"2023-09-27T23:13:48.802757+0000","last_became_active":"2023-09-27T23:13:44.419369+0000","last_became_peered":"2023-09-27T23:13:44.419369+0000","last_unstale":"2023-09-27T23:13:48.802757+0000","last_undegraded":"2023-09-27T23:13:48.802757+0000","last_fullsized":"2023-09-27T23:13:48.802757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:04.718338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826218+0000","last_change":"2023-09-27T23:13:48.822456+0000","last_active":"2023-09-27T23:13:48.826218+0000","last_peered":"2023-09-27T23:13:48.826218+0000","last_clean":"2023-09-27T23:13:48.826218+0000","last_became_active":"2023-09-27T23:13:48.821878+0000","last_became_peered":"2023-09-27T23:13:48.821878+0000","last_unstale":"2023-09-27T23:13:48.826218+0000","last_undegraded":"2023-09-27T23:13:48.826218+0000","last_fullsized":"2023-09-27T23:13:48.826218+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807086+0000","last_change":"2023-09-27T23:13:44.777613+0000","last_active":"2023-09-27T23:13:48.807086+0000","last_peered":"2023-09-27T23:13:48.807086+0000","last_clean":"2023-09-27T23:13:48.807086+0000","last_became_active":"2023-09-27T23:13:44.777267+0000","last_became_peered":"2023-09-27T23:13:44.777267+0000","last_unstale":"2023-09-27T23:13:48.807086+0000","last_undegraded":"2023-09-27T23:13:48.807086+0000","last_fullsized":"2023-09-27T23:13:48.807086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:42:36.525422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802984+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:13:48.802984+0000","last_peered":"2023-09-27T23:13:48.802984+0000","last_clean":"2023-09-27T23:13:48.802984+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:13:48.802984+0000","last_undegraded":"2023-09-27T23:13:48.802984+0000","last_fullsized":"2023-09-27T23:13:48.802984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814830+0000","last_change":"2023-09-27T23:13:44.422663+0000","last_active":"2023-09-27T23:13:48.814830+0000","last_peered":"2023-09-27T23:13:48.814830+0000","last_clean":"2023-09-27T23:13:48.814830+0000","last_became_active":"2023-09-27T23:13:44.422361+0000","last_became_peered":"2023-09-27T23:13:44.422361+0000","last_unstale":"2023-09-27T23:13:48.814830+0000","last_undegraded":"2023-09-27T23:13:48.814830+0000","last_fullsized":"2023-09-27T23:13:48.814830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833765+0000","last_change":"2023-09-27T23:13:48.824670+0000","last_active":"2023-09-27T23:13:48.833765+0000","last_peered":"2023-09-27T23:13:48.833765+0000","last_clean":"2023-09-27T23:13:48.833765+0000","last_became_active":"2023-09-27T23:13:48.824249+0000","last_became_peered":"2023-09-27T23:13:48.824249+0000","last_unstale":"2023-09-27T23:13:48.833765+0000","last_undegraded":"2023-09-27T23:13:48.833765+0000","last_fullsized":"2023-09-27T23:13:48.833765+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802514+0000","last_change":"2023-09-27T23:13:44.778136+0000","last_active":"2023-09-27T23:13:48.802514+0000","last_peered":"2023-09-27T23:13:48.802514+0000","last_clean":"2023-09-27T23:13:48.802514+0000","last_became_active":"2023-09-27T23:13:44.777865+0000","last_became_peered":"2023-09-27T23:13:44.777865+0000","last_unstale":"2023-09-27T23:13:48.802514+0000","last_undegraded":"2023-09-27T23:13:48.802514+0000","last_fullsized":"2023-09-27T23:13:48.802514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:46:04.289644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821294+0000","last_change":"2023-09-27T23:13:44.424183+0000","last_active":"2023-09-27T23:13:48.821294+0000","last_peered":"2023-09-27T23:13:48.821294+0000","last_clean":"2023-09-27T23:13:48.821294+0000","last_became_active":"2023-09-27T23:13:44.422919+0000","last_became_peered":"2023-09-27T23:13:44.422919+0000","last_unstale":"2023-09-27T23:13:48.821294+0000","last_undegraded":"2023-09-27T23:13:48.821294+0000","last_fullsized":"2023-09-27T23:13:48.821294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813932+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:13:48.813932+0000","last_peered":"2023-09-27T23:13:48.813932+0000","last_clean":"2023-09-27T23:13:48.813932+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:13:48.813932+0000","last_undegraded":"2023-09-27T23:13:48.813932+0000","last_fullsized":"2023-09-27T23:13:48.813932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839053+0000","last_change":"2023-09-27T23:13:48.836163+0000","last_active":"2023-09-27T23:13:48.839053+0000","last_peered":"2023-09-27T23:13:48.839053+0000","last_clean":"2023-09-27T23:13:48.839053+0000","last_became_active":"2023-09-27T23:13:48.835655+0000","last_became_peered":"2023-09-27T23:13:48.835655+0000","last_unstale":"2023-09-27T23:13:48.839053+0000","last_undegraded":"2023-09-27T23:13:48.839053+0000","last_fullsized":"2023-09-27T23:13:48.839053+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802588+0000","last_change":"2023-09-27T23:13:44.777627+0000","last_active":"2023-09-27T23:13:48.802588+0000","last_peered":"2023-09-27T23:13:48.802588+0000","last_clean":"2023-09-27T23:13:48.802588+0000","last_became_active":"2023-09-27T23:13:44.777239+0000","last_became_peered":"2023-09-27T23:13:44.777239+0000","last_unstale":"2023-09-27T23:13:48.802588+0000","last_undegraded":"2023-09-27T23:13:48.802588+0000","last_fullsized":"2023-09-27T23:13:48.802588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:34.572519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.800241+0000","last_change":"2023-09-27T23:13:44.419827+0000","last_active":"2023-09-27T23:13:48.800241+0000","last_peered":"2023-09-27T23:13:48.800241+0000","last_clean":"2023-09-27T23:13:48.800241+0000","last_became_active":"2023-09-27T23:13:44.419535+0000","last_became_peered":"2023-09-27T23:13:44.419535+0000","last_unstale":"2023-09-27T23:13:48.800241+0000","last_undegraded":"2023-09-27T23:13:48.800241+0000","last_fullsized":"2023-09-27T23:13:48.800241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:08:26.061286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813256+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:13:48.813256+0000","last_peered":"2023-09-27T23:13:48.813256+0000","last_clean":"2023-09-27T23:13:48.813256+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:13:48.813256+0000","last_undegraded":"2023-09-27T23:13:48.813256+0000","last_fullsized":"2023-09-27T23:13:48.813256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842381+0000","last_change":"2023-09-27T23:13:48.839688+0000","last_active":"2023-09-27T23:13:48.842381+0000","last_peered":"2023-09-27T23:13:48.842381+0000","last_clean":"2023-09-27T23:13:48.842381+0000","last_became_active":"2023-09-27T23:13:48.839257+0000","last_became_peered":"2023-09-27T23:13:48.839257+0000","last_unstale":"2023-09-27T23:13:48.842381+0000","last_undegraded":"2023-09-27T23:13:48.842381+0000","last_fullsized":"2023-09-27T23:13:48.842381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:23.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803724+0000","last_change":"2023-09-27T23:13:44.796668+0000","last_active":"2023-09-27T23:13:48.803724+0000","last_peered":"2023-09-27T23:13:48.803724+0000","last_clean":"2023-09-27T23:13:48.803724+0000","last_became_active":"2023-09-27T23:13:44.795170+0000","last_became_peered":"2023-09-27T23:13:44.795170+0000","last_unstale":"2023-09-27T23:13:48.803724+0000","last_undegraded":"2023-09-27T23:13:48.803724+0000","last_fullsized":"2023-09-27T23:13:48.803724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802500+0000","last_change":"2023-09-27T23:13:44.428369+0000","last_active":"2023-09-27T23:13:48.802500+0000","last_peered":"2023-09-27T23:13:48.802500+0000","last_clean":"2023-09-27T23:13:48.802500+0000","last_became_active":"2023-09-27T23:13:44.427956+0000","last_became_peered":"2023-09-27T23:13:44.427956+0000","last_unstale":"2023-09-27T23:13:48.802500+0000","last_undegraded":"2023-09-27T23:13:48.802500+0000","last_fullsized":"2023-09-27T23:13:48.802500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803011+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:13:48.803011+0000","last_peered":"2023-09-27T23:13:48.803011+0000","last_clean":"2023-09-27T23:13:48.803011+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:13:48.803011+0000","last_undegraded":"2023-09-27T23:13:48.803011+0000","last_fullsized":"2023-09-27T23:13:48.803011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834595+0000","last_change":"2023-09-27T23:13:48.830308+0000","last_active":"2023-09-27T23:13:48.834595+0000","last_peered":"2023-09-27T23:13:48.834595+0000","last_clean":"2023-09-27T23:13:48.834595+0000","last_became_active":"2023-09-27T23:13:48.829698+0000","last_became_peered":"2023-09-27T23:13:48.829698+0000","last_unstale":"2023-09-27T23:13:48.834595+0000","last_undegraded":"2023-09-27T23:13:48.834595+0000","last_fullsized":"2023-09-27T23:13:48.834595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:27:58.121695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812509+0000","last_change":"2023-09-27T23:13:44.804857+0000","last_active":"2023-09-27T23:13:48.812509+0000","last_peered":"2023-09-27T23:13:48.812509+0000","last_clean":"2023-09-27T23:13:48.812509+0000","last_became_active":"2023-09-27T23:13:44.804486+0000","last_became_peered":"2023-09-27T23:13:44.804486+0000","last_unstale":"2023-09-27T23:13:48.812509+0000","last_undegraded":"2023-09-27T23:13:48.812509+0000","last_fullsized":"2023-09-27T23:13:48.812509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822742+0000","last_change":"2023-09-27T23:13:44.416101+0000","last_active":"2023-09-27T23:13:48.822742+0000","last_peered":"2023-09-27T23:13:48.822742+0000","last_clean":"2023-09-27T23:13:48.822742+0000","last_became_active":"2023-09-27T23:13:44.415693+0000","last_became_peered":"2023-09-27T23:13:44.415693+0000","last_unstale":"2023-09-27T23:13:48.822742+0000","last_undegraded":"2023-09-27T23:13:48.822742+0000","last_fullsized":"2023-09-27T23:13:48.822742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813962+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:13:48.813962+0000","last_peered":"2023-09-27T23:13:48.813962+0000","last_clean":"2023-09-27T23:13:48.813962+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:13:48.813962+0000","last_undegraded":"2023-09-27T23:13:48.813962+0000","last_fullsized":"2023-09-27T23:13:48.813962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803349+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:13:48.803349+0000","last_peered":"2023-09-27T23:13:48.803349+0000","last_clean":"2023-09-27T23:13:48.803349+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:13:48.803349+0000","last_undegraded":"2023-09-27T23:13:48.803349+0000","last_fullsized":"2023-09-27T23:13:48.803349+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829119+0000","last_change":"2023-09-27T23:13:48.827977+0000","last_active":"2023-09-27T23:13:48.829119+0000","last_peered":"2023-09-27T23:13:48.829119+0000","last_clean":"2023-09-27T23:13:48.829119+0000","last_became_active":"2023-09-27T23:13:48.827465+0000","last_became_peered":"2023-09-27T23:13:48.827465+0000","last_unstale":"2023-09-27T23:13:48.829119+0000","last_undegraded":"2023-09-27T23:13:48.829119+0000","last_fullsized":"2023-09-27T23:13:48.829119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:18:57.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803299+0000","last_change":"2023-09-27T23:13:44.786471+0000","last_active":"2023-09-27T23:13:48.803299+0000","last_peered":"2023-09-27T23:13:48.803299+0000","last_clean":"2023-09-27T23:13:48.803299+0000","last_became_active":"2023-09-27T23:13:44.786182+0000","last_became_peered":"2023-09-27T23:13:44.786182+0000","last_unstale":"2023-09-27T23:13:48.803299+0000","last_undegraded":"2023-09-27T23:13:48.803299+0000","last_fullsized":"2023-09-27T23:13:48.803299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:04:06.713350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822086+0000","last_change":"2023-09-27T23:13:44.422925+0000","last_active":"2023-09-27T23:13:48.822086+0000","last_peered":"2023-09-27T23:13:48.822086+0000","last_clean":"2023-09-27T23:13:48.822086+0000","last_became_active":"2023-09-27T23:13:44.422547+0000","last_became_peered":"2023-09-27T23:13:44.422547+0000","last_unstale":"2023-09-27T23:13:48.822086+0000","last_undegraded":"2023-09-27T23:13:48.822086+0000","last_fullsized":"2023-09-27T23:13:48.822086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839810+0000","last_change":"2023-09-27T23:13:48.824583+0000","last_active":"2023-09-27T23:13:48.839810+0000","last_peered":"2023-09-27T23:13:48.839810+0000","last_clean":"2023-09-27T23:13:48.839810+0000","last_became_active":"2023-09-27T23:13:48.824080+0000","last_became_peered":"2023-09-27T23:13:48.824080+0000","last_unstale":"2023-09-27T23:13:48.839810+0000","last_undegraded":"2023-09-27T23:13:48.839810+0000","last_fullsized":"2023-09-27T23:13:48.839810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:29:19.951565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821416+0000","last_change":"2023-09-27T23:13:44.792934+0000","last_active":"2023-09-27T23:13:48.821416+0000","last_peered":"2023-09-27T23:13:48.821416+0000","last_clean":"2023-09-27T23:13:48.821416+0000","last_became_active":"2023-09-27T23:13:44.791774+0000","last_became_peered":"2023-09-27T23:13:44.791774+0000","last_unstale":"2023-09-27T23:13:48.821416+0000","last_undegraded":"2023-09-27T23:13:48.821416+0000","last_fullsized":"2023-09-27T23:13:48.821416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821113+0000","last_change":"2023-09-27T23:13:44.418284+0000","last_active":"2023-09-27T23:13:48.821113+0000","last_peered":"2023-09-27T23:13:48.821113+0000","last_clean":"2023-09-27T23:13:48.821113+0000","last_became_active":"2023-09-27T23:13:44.417952+0000","last_became_peered":"2023-09-27T23:13:44.417952+0000","last_unstale":"2023-09-27T23:13:48.821113+0000","last_undegraded":"2023-09-27T23:13:48.821113+0000","last_fullsized":"2023-09-27T23:13:48.821113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840441+0000","last_change":"2023-09-27T23:13:48.835487+0000","last_active":"2023-09-27T23:13:48.840441+0000","last_peered":"2023-09-27T23:13:48.840441+0000","last_clean":"2023-09-27T23:13:48.840441+0000","last_became_active":"2023-09-27T23:13:48.835043+0000","last_became_peered":"2023-09-27T23:13:48.835043+0000","last_unstale":"2023-09-27T23:13:48.840441+0000","last_undegraded":"2023-09-27T23:13:48.840441+0000","last_fullsized":"2023-09-27T23:13:48.840441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:43:53.193315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821389+0000","last_change":"2023-09-27T23:13:44.797452+0000","last_active":"2023-09-27T23:13:48.821389+0000","last_peered":"2023-09-27T23:13:48.821389+0000","last_clean":"2023-09-27T23:13:48.821389+0000","last_became_active":"2023-09-27T23:13:44.797056+0000","last_became_peered":"2023-09-27T23:13:44.797056+0000","last_unstale":"2023-09-27T23:13:48.821389+0000","last_undegraded":"2023-09-27T23:13:48.821389+0000","last_fullsized":"2023-09-27T23:13:48.821389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812548+0000","last_change":"2023-09-27T23:13:44.426307+0000","last_active":"2023-09-27T23:13:48.812548+0000","last_peered":"2023-09-27T23:13:48.812548+0000","last_clean":"2023-09-27T23:13:48.812548+0000","last_became_active":"2023-09-27T23:13:44.425925+0000","last_became_peered":"2023-09-27T23:13:44.425925+0000","last_unstale":"2023-09-27T23:13:48.812548+0000","last_undegraded":"2023-09-27T23:13:48.812548+0000","last_fullsized":"2023-09-27T23:13:48.812548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843420+0000","last_change":"2023-09-27T23:13:48.835916+0000","last_active":"2023-09-27T23:13:48.843420+0000","last_peered":"2023-09-27T23:13:48.843420+0000","last_clean":"2023-09-27T23:13:48.843420+0000","last_became_active":"2023-09-27T23:13:48.835201+0000","last_became_peered":"2023-09-27T23:13:48.835201+0000","last_unstale":"2023-09-27T23:13:48.843420+0000","last_undegraded":"2023-09-27T23:13:48.843420+0000","last_fullsized":"2023-09-27T23:13:48.843420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:15:13.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821512+0000","last_change":"2023-09-27T23:13:44.804376+0000","last_active":"2023-09-27T23:13:48.821512+0000","last_peered":"2023-09-27T23:13:48.821512+0000","last_clean":"2023-09-27T23:13:48.821512+0000","last_became_active":"2023-09-27T23:13:44.803820+0000","last_became_peered":"2023-09-27T23:13:44.803820+0000","last_unstale":"2023-09-27T23:13:48.821512+0000","last_undegraded":"2023-09-27T23:13:48.821512+0000","last_fullsized":"2023-09-27T23:13:48.821512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:45:33.486648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825618+0000","last_change":"2023-09-27T23:13:44.422613+0000","last_active":"2023-09-27T23:13:48.825618+0000","last_peered":"2023-09-27T23:13:48.825618+0000","last_clean":"2023-09-27T23:13:48.825618+0000","last_became_active":"2023-09-27T23:13:44.422341+0000","last_became_peered":"2023-09-27T23:13:44.422341+0000","last_unstale":"2023-09-27T23:13:48.825618+0000","last_undegraded":"2023-09-27T23:13:48.825618+0000","last_fullsized":"2023-09-27T23:13:48.825618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825159+0000","last_change":"2023-09-27T23:13:48.821288+0000","last_active":"2023-09-27T23:13:48.825159+0000","last_peered":"2023-09-27T23:13:48.825159+0000","last_clean":"2023-09-27T23:13:48.825159+0000","last_became_active":"2023-09-27T23:13:48.820648+0000","last_became_peered":"2023-09-27T23:13:48.820648+0000","last_unstale":"2023-09-27T23:13:48.825159+0000","last_undegraded":"2023-09-27T23:13:48.825159+0000","last_fullsized":"2023-09-27T23:13:48.825159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:17.485779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804537+0000","last_change":"2023-09-27T23:13:44.795149+0000","last_active":"2023-09-27T23:13:48.804537+0000","last_peered":"2023-09-27T23:13:48.804537+0000","last_clean":"2023-09-27T23:13:48.804537+0000","last_became_active":"2023-09-27T23:13:44.794005+0000","last_became_peered":"2023-09-27T23:13:44.794005+0000","last_unstale":"2023-09-27T23:13:48.804537+0000","last_undegraded":"2023-09-27T23:13:48.804537+0000","last_fullsized":"2023-09-27T23:13:48.804537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811248+0000","last_change":"2023-09-27T23:13:44.422676+0000","last_active":"2023-09-27T23:13:48.811248+0000","last_peered":"2023-09-27T23:13:48.811248+0000","last_clean":"2023-09-27T23:13:48.811248+0000","last_became_active":"2023-09-27T23:13:44.422396+0000","last_became_peered":"2023-09-27T23:13:44.422396+0000","last_unstale":"2023-09-27T23:13:48.811248+0000","last_undegraded":"2023-09-27T23:13:48.811248+0000","last_fullsized":"2023-09-27T23:13:48.811248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838657+0000","last_change":"2023-09-27T23:13:48.835879+0000","last_active":"2023-09-27T23:13:48.838657+0000","last_peered":"2023-09-27T23:13:48.838657+0000","last_clean":"2023-09-27T23:13:48.838657+0000","last_became_active":"2023-09-27T23:13:48.835422+0000","last_became_peered":"2023-09-27T23:13:48.835422+0000","last_unstale":"2023-09-27T23:13:48.838657+0000","last_undegraded":"2023-09-27T23:13:48.838657+0000","last_fullsized":"2023-09-27T23:13:48.838657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:36.782810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810125+0000","last_change":"2023-09-27T23:13:44.795768+0000","last_active":"2023-09-27T23:13:48.810125+0000","last_peered":"2023-09-27T23:13:48.810125+0000","last_clean":"2023-09-27T23:13:48.810125+0000","last_became_active":"2023-09-27T23:13:44.795376+0000","last_became_peered":"2023-09-27T23:13:44.795376+0000","last_unstale":"2023-09-27T23:13:48.810125+0000","last_undegraded":"2023-09-27T23:13:48.810125+0000","last_fullsized":"2023-09-27T23:13:48.810125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:02.136037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803338+0000","last_change":"2023-09-27T23:13:44.415778+0000","last_active":"2023-09-27T23:13:48.803338+0000","last_peered":"2023-09-27T23:13:48.803338+0000","last_clean":"2023-09-27T23:13:48.803338+0000","last_became_active":"2023-09-27T23:13:44.415490+0000","last_became_peered":"2023-09-27T23:13:44.415490+0000","last_unstale":"2023-09-27T23:13:48.803338+0000","last_undegraded":"2023-09-27T23:13:48.803338+0000","last_fullsized":"2023-09-27T23:13:48.803338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.828626+0000","last_change":"2023-09-27T23:13:48.823729+0000","last_active":"2023-09-27T23:13:48.828626+0000","last_peered":"2023-09-27T23:13:48.828626+0000","last_clean":"2023-09-27T23:13:48.828626+0000","last_became_active":"2023-09-27T23:13:48.823284+0000","last_became_peered":"2023-09-27T23:13:48.823284+0000","last_unstale":"2023-09-27T23:13:48.828626+0000","last_undegraded":"2023-09-27T23:13:48.828626+0000","last_fullsized":"2023-09-27T23:13:48.828626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:19.656238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820833+0000","last_change":"2023-09-27T23:13:44.794072+0000","last_active":"2023-09-27T23:13:48.820833+0000","last_peered":"2023-09-27T23:13:48.820833+0000","last_clean":"2023-09-27T23:13:48.820833+0000","last_became_active":"2023-09-27T23:13:44.793773+0000","last_became_peered":"2023-09-27T23:13:44.793773+0000","last_unstale":"2023-09-27T23:13:48.820833+0000","last_undegraded":"2023-09-27T23:13:48.820833+0000","last_fullsized":"2023-09-27T23:13:48.820833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804693+0000","last_change":"2023-09-27T23:13:44.426779+0000","last_active":"2023-09-27T23:13:48.804693+0000","last_peered":"2023-09-27T23:13:48.804693+0000","last_clean":"2023-09-27T23:13:48.804693+0000","last_became_active":"2023-09-27T23:13:44.426515+0000","last_became_peered":"2023-09-27T23:13:44.426515+0000","last_unstale":"2023-09-27T23:13:48.804693+0000","last_undegraded":"2023-09-27T23:13:48.804693+0000","last_fullsized":"2023-09-27T23:13:48.804693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:40.093740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842388+0000","last_change":"2023-09-27T23:13:48.839843+0000","last_active":"2023-09-27T23:13:48.842388+0000","last_peered":"2023-09-27T23:13:48.842388+0000","last_clean":"2023-09-27T23:13:48.842388+0000","last_became_active":"2023-09-27T23:13:48.839383+0000","last_became_peered":"2023-09-27T23:13:48.839383+0000","last_unstale":"2023-09-27T23:13:48.842388+0000","last_undegraded":"2023-09-27T23:13:48.842388+0000","last_fullsized":"2023-09-27T23:13:48.842388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810673+0000","last_change":"2023-09-27T23:13:44.802558+0000","last_active":"2023-09-27T23:13:48.810673+0000","last_peered":"2023-09-27T23:13:48.810673+0000","last_clean":"2023-09-27T23:13:48.810673+0000","last_became_active":"2023-09-27T23:13:44.800406+0000","last_became_peered":"2023-09-27T23:13:44.800406+0000","last_unstale":"2023-09-27T23:13:48.810673+0000","last_undegraded":"2023-09-27T23:13:48.810673+0000","last_fullsized":"2023-09-27T23:13:48.810673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804716+0000","last_change":"2023-09-27T23:13:44.426684+0000","last_active":"2023-09-27T23:13:48.804716+0000","last_peered":"2023-09-27T23:13:48.804716+0000","last_clean":"2023-09-27T23:13:48.804716+0000","last_became_active":"2023-09-27T23:13:44.426422+0000","last_became_peered":"2023-09-27T23:13:44.426422+0000","last_unstale":"2023-09-27T23:13:48.804716+0000","last_undegraded":"2023-09-27T23:13:48.804716+0000","last_fullsized":"2023-09-27T23:13:48.804716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:12.799421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829098+0000","last_change":"2023-09-27T23:13:48.827612+0000","last_active":"2023-09-27T23:13:48.829098+0000","last_peered":"2023-09-27T23:13:48.829098+0000","last_clean":"2023-09-27T23:13:48.829098+0000","last_became_active":"2023-09-27T23:13:48.827019+0000","last_became_peered":"2023-09-27T23:13:48.827019+0000","last_unstale":"2023-09-27T23:13:48.829098+0000","last_undegraded":"2023-09-27T23:13:48.829098+0000","last_fullsized":"2023-09-27T23:13:48.829098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:40:33.151042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.835789+0000","last_change":"2023-09-27T23:13:48.830820+0000","last_active":"2023-09-27T23:13:48.835789+0000","last_peered":"2023-09-27T23:13:48.835789+0000","last_clean":"2023-09-27T23:13:48.835789+0000","last_became_active":"2023-09-27T23:13:48.829846+0000","last_became_peered":"2023-09-27T23:13:48.829846+0000","last_unstale":"2023-09-27T23:13:48.835789+0000","last_undegraded":"2023-09-27T23:13:48.835789+0000","last_fullsized":"2023-09-27T23:13:48.835789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809441+0000","last_change":"2023-09-27T23:13:44.413137+0000","last_active":"2023-09-27T23:13:48.809441+0000","last_peered":"2023-09-27T23:13:48.809441+0000","last_clean":"2023-09-27T23:13:48.809441+0000","last_became_active":"2023-09-27T23:13:44.412799+0000","last_became_peered":"2023-09-27T23:13:44.412799+0000","last_unstale":"2023-09-27T23:13:48.809441+0000","last_undegraded":"2023-09-27T23:13:48.809441+0000","last_fullsized":"2023-09-27T23:13:48.809441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825273+0000","last_change":"2023-09-27T23:13:44.793322+0000","last_active":"2023-09-27T23:13:48.825273+0000","last_peered":"2023-09-27T23:13:48.825273+0000","last_clean":"2023-09-27T23:13:48.825273+0000","last_became_active":"2023-09-27T23:13:44.793057+0000","last_became_peered":"2023-09-27T23:13:44.793057+0000","last_unstale":"2023-09-27T23:13:48.825273+0000","last_undegraded":"2023-09-27T23:13:48.825273+0000","last_fullsized":"2023-09-27T23:13:48.825273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842877+0000","last_change":"2023-09-27T23:13:48.839976+0000","last_active":"2023-09-27T23:13:48.842877+0000","last_peered":"2023-09-27T23:13:48.842877+0000","last_clean":"2023-09-27T23:13:48.842877+0000","last_became_active":"2023-09-27T23:13:48.839568+0000","last_became_peered":"2023-09-27T23:13:48.839568+0000","last_unstale":"2023-09-27T23:13:48.842877+0000","last_undegraded":"2023-09-27T23:13:48.842877+0000","last_fullsized":"2023-09-27T23:13:48.842877+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:01.437279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804236+0000","last_change":"2023-09-27T23:13:44.427379+0000","last_active":"2023-09-27T23:13:48.804236+0000","last_peered":"2023-09-27T23:13:48.804236+0000","last_clean":"2023-09-27T23:13:48.804236+0000","last_became_active":"2023-09-27T23:13:44.427094+0000","last_became_peered":"2023-09-27T23:13:44.427094+0000","last_unstale":"2023-09-27T23:13:48.804236+0000","last_undegraded":"2023-09-27T23:13:48.804236+0000","last_fullsized":"2023-09-27T23:13:48.804236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:33.544941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803930+0000","last_change":"2023-09-27T23:13:44.793100+0000","last_active":"2023-09-27T23:13:48.803930+0000","last_peered":"2023-09-27T23:13:48.803930+0000","last_clean":"2023-09-27T23:13:48.803930+0000","last_became_active":"2023-09-27T23:13:44.792493+0000","last_became_peered":"2023-09-27T23:13:44.792493+0000","last_unstale":"2023-09-27T23:13:48.803930+0000","last_undegraded":"2023-09-27T23:13:48.803930+0000","last_fullsized":"2023-09-27T23:13:48.803930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804152+0000","last_change":"2023-09-27T23:13:44.420128+0000","last_active":"2023-09-27T23:13:48.804152+0000","last_peered":"2023-09-27T23:13:48.804152+0000","last_clean":"2023-09-27T23:13:48.804152+0000","last_became_active":"2023-09-27T23:13:44.419240+0000","last_became_peered":"2023-09-27T23:13:44.419240+0000","last_unstale":"2023-09-27T23:13:48.804152+0000","last_undegraded":"2023-09-27T23:13:48.804152+0000","last_fullsized":"2023-09-27T23:13:48.804152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814644+0000","last_change":"2023-09-27T23:13:44.802448+0000","last_active":"2023-09-27T23:13:48.814644+0000","last_peered":"2023-09-27T23:13:48.814644+0000","last_clean":"2023-09-27T23:13:48.814644+0000","last_became_active":"2023-09-27T23:13:44.801316+0000","last_became_peered":"2023-09-27T23:13:44.801316+0000","last_unstale":"2023-09-27T23:13:48.814644+0000","last_undegraded":"2023-09-27T23:13:48.814644+0000","last_fullsized":"2023-09-27T23:13:48.814644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842951+0000","last_change":"2023-09-27T23:13:48.841624+0000","last_active":"2023-09-27T23:13:48.842951+0000","last_peered":"2023-09-27T23:13:48.842951+0000","last_clean":"2023-09-27T23:13:48.842951+0000","last_became_active":"2023-09-27T23:13:48.841104+0000","last_became_peered":"2023-09-27T23:13:48.841104+0000","last_unstale":"2023-09-27T23:13:48.842951+0000","last_undegraded":"2023-09-27T23:13:48.842951+0000","last_fullsized":"2023-09-27T23:13:48.842951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804318+0000","last_change":"2023-09-27T23:13:44.426217+0000","last_active":"2023-09-27T23:13:48.804318+0000","last_peered":"2023-09-27T23:13:48.804318+0000","last_clean":"2023-09-27T23:13:48.804318+0000","last_became_active":"2023-09-27T23:13:44.425907+0000","last_became_peered":"2023-09-27T23:13:44.425907+0000","last_unstale":"2023-09-27T23:13:48.804318+0000","last_undegraded":"2023-09-27T23:13:48.804318+0000","last_fullsized":"2023-09-27T23:13:48.804318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:01.126893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825997+0000","last_change":"2023-09-27T23:13:44.789234+0000","last_active":"2023-09-27T23:13:48.825997+0000","last_peered":"2023-09-27T23:13:48.825997+0000","last_clean":"2023-09-27T23:13:48.825997+0000","last_became_active":"2023-09-27T23:13:44.788948+0000","last_became_peered":"2023-09-27T23:13:44.788948+0000","last_unstale":"2023-09-27T23:13:48.825997+0000","last_undegraded":"2023-09-27T23:13:48.825997+0000","last_fullsized":"2023-09-27T23:13:48.825997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834520+0000","last_change":"2023-09-27T23:13:48.830283+0000","last_active":"2023-09-27T23:13:48.834520+0000","last_peered":"2023-09-27T23:13:48.834520+0000","last_clean":"2023-09-27T23:13:48.834520+0000","last_became_active":"2023-09-27T23:13:48.829788+0000","last_became_peered":"2023-09-27T23:13:48.829788+0000","last_unstale":"2023-09-27T23:13:48.834520+0000","last_undegraded":"2023-09-27T23:13:48.834520+0000","last_fullsized":"2023-09-27T23:13:48.834520+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809476+0000","last_change":"2023-09-27T23:13:44.794796+0000","last_active":"2023-09-27T23:13:48.809476+0000","last_peered":"2023-09-27T23:13:48.809476+0000","last_clean":"2023-09-27T23:13:48.809476+0000","last_became_active":"2023-09-27T23:13:44.793105+0000","last_became_peered":"2023-09-27T23:13:44.793105+0000","last_unstale":"2023-09-27T23:13:48.809476+0000","last_undegraded":"2023-09-27T23:13:48.809476+0000","last_fullsized":"2023-09-27T23:13:48.809476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:13.808565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822377+0000","last_change":"2023-09-27T23:13:44.789337+0000","last_active":"2023-09-27T23:13:48.822377+0000","last_peered":"2023-09-27T23:13:48.822377+0000","last_clean":"2023-09-27T23:13:48.822377+0000","last_became_active":"2023-09-27T23:13:44.789055+0000","last_became_peered":"2023-09-27T23:13:44.789055+0000","last_unstale":"2023-09-27T23:13:48.822377+0000","last_undegraded":"2023-09-27T23:13:48.822377+0000","last_fullsized":"2023-09-27T23:13:48.822377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821093+0000","last_change":"2023-09-27T23:13:44.792891+0000","last_active":"2023-09-27T23:13:48.821093+0000","last_peered":"2023-09-27T23:13:48.821093+0000","last_clean":"2023-09-27T23:13:48.821093+0000","last_became_active":"2023-09-27T23:13:44.792439+0000","last_became_peered":"2023-09-27T23:13:44.792439+0000","last_unstale":"2023-09-27T23:13:48.821093+0000","last_undegraded":"2023-09-27T23:13:48.821093+0000","last_fullsized":"2023-09-27T23:13:48.821093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:33.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825381+0000","last_change":"2023-09-27T23:13:44.787514+0000","last_active":"2023-09-27T23:13:48.825381+0000","last_peered":"2023-09-27T23:13:48.825381+0000","last_clean":"2023-09-27T23:13:48.825381+0000","last_became_active":"2023-09-27T23:13:44.787130+0000","last_became_peered":"2023-09-27T23:13:44.787130+0000","last_unstale":"2023-09-27T23:13:48.825381+0000","last_undegraded":"2023-09-27T23:13:48.825381+0000","last_fullsized":"2023-09-27T23:13:48.825381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.806701+0000","last_change":"2023-09-27T23:13:44.786768+0000","last_active":"2023-09-27T23:13:48.806701+0000","last_peered":"2023-09-27T23:13:48.806701+0000","last_clean":"2023-09-27T23:13:48.806701+0000","last_became_active":"2023-09-27T23:13:44.785642+0000","last_became_peered":"2023-09-27T23:13:44.785642+0000","last_unstale":"2023-09-27T23:13:48.806701+0000","last_undegraded":"2023-09-27T23:13:48.806701+0000","last_fullsized":"2023-09-27T23:13:48.806701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:11:40.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814552+0000","last_change":"2023-09-27T23:13:44.802393+0000","last_active":"2023-09-27T23:13:48.814552+0000","last_peered":"2023-09-27T23:13:48.814552+0000","last_clean":"2023-09-27T23:13:48.814552+0000","last_became_active":"2023-09-27T23:13:44.801038+0000","last_became_peered":"2023-09-27T23:13:44.801038+0000","last_unstale":"2023-09-27T23:13:48.814552+0000","last_undegraded":"2023-09-27T23:13:48.814552+0000","last_fullsized":"2023-09-27T23:13:48.814552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807068+0000","last_change":"2023-09-27T23:13:44.778165+0000","last_active":"2023-09-27T23:13:48.807068+0000","last_peered":"2023-09-27T23:13:48.807068+0000","last_clean":"2023-09-27T23:13:48.807068+0000","last_became_active":"2023-09-27T23:13:44.777922+0000","last_became_peered":"2023-09-27T23:13:44.777922+0000","last_unstale":"2023-09-27T23:13:48.807068+0000","last_undegraded":"2023-09-27T23:13:48.807068+0000","last_fullsized":"2023-09-27T23:13:48.807068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:31.322306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.801910+0000","last_change":"2023-09-27T23:13:44.792009+0000","last_active":"2023-09-27T23:13:48.801910+0000","last_peered":"2023-09-27T23:13:48.801910+0000","last_clean":"2023-09-27T23:13:48.801910+0000","last_became_active":"2023-09-27T23:13:44.791714+0000","last_became_peered":"2023-09-27T23:13:44.791714+0000","last_unstale":"2023-09-27T23:13:48.801910+0000","last_undegraded":"2023-09-27T23:13:48.801910+0000","last_fullsized":"2023-09-27T23:13:48.801910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:10.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802783+0000","last_change":"2023-09-27T23:13:44.800431+0000","last_active":"2023-09-27T23:13:48.802783+0000","last_peered":"2023-09-27T23:13:48.802783+0000","last_clean":"2023-09-27T23:13:48.802783+0000","last_became_active":"2023-09-27T23:13:44.800056+0000","last_became_peered":"2023-09-27T23:13:44.800056+0000","last_unstale":"2023-09-27T23:13:48.802783+0000","last_undegraded":"2023-09-27T23:13:48.802783+0000","last_fullsized":"2023-09-27T23:13:48.802783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822290+0000","last_change":"2023-09-27T23:13:44.788799+0000","last_active":"2023-09-27T23:13:48.822290+0000","last_peered":"2023-09-27T23:13:48.822290+0000","last_clean":"2023-09-27T23:13:48.822290+0000","last_became_active":"2023-09-27T23:13:44.788526+0000","last_became_peered":"2023-09-27T23:13:44.788526+0000","last_unstale":"2023-09-27T23:13:48.822290+0000","last_undegraded":"2023-09-27T23:13:48.822290+0000","last_fullsized":"2023-09-27T23:13:48.822290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803904+0000","last_change":"2023-09-27T23:13:44.795488+0000","last_active":"2023-09-27T23:13:48.803904+0000","last_peered":"2023-09-27T23:13:48.803904+0000","last_clean":"2023-09-27T23:13:48.803904+0000","last_became_active":"2023-09-27T23:13:44.794082+0000","last_became_peered":"2023-09-27T23:13:44.794082+0000","last_unstale":"2023-09-27T23:13:48.803904+0000","last_undegraded":"2023-09-27T23:13:48.803904+0000","last_fullsized":"2023-09-27T23:13:48.803904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803019+0000","last_change":"2023-09-27T23:13:44.802208+0000","last_active":"2023-09-27T23:13:48.803019+0000","last_peered":"2023-09-27T23:13:48.803019+0000","last_clean":"2023-09-27T23:13:48.803019+0000","last_became_active":"2023-09-27T23:13:44.801858+0000","last_became_peered":"2023-09-27T23:13:44.801858+0000","last_unstale":"2023-09-27T23:13:48.803019+0000","last_undegraded":"2023-09-27T23:13:48.803019+0000","last_fullsized":"2023-09-27T23:13:48.803019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821304+0000","last_change":"2023-09-27T23:13:44.795884+0000","last_active":"2023-09-27T23:13:48.821304+0000","last_peered":"2023-09-27T23:13:48.821304+0000","last_clean":"2023-09-27T23:13:48.821304+0000","last_became_active":"2023-09-27T23:13:44.795490+0000","last_became_peered":"2023-09-27T23:13:44.795490+0000","last_unstale":"2023-09-27T23:13:48.821304+0000","last_undegraded":"2023-09-27T23:13:48.821304+0000","last_fullsized":"2023-09-27T23:13:48.821304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:53:40.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810126+0000","last_change":"2023-09-27T23:13:44.791556+0000","last_active":"2023-09-27T23:13:48.810126+0000","last_peered":"2023-09-27T23:13:48.810126+0000","last_clean":"2023-09-27T23:13:48.810126+0000","last_became_active":"2023-09-27T23:13:44.789598+0000","last_became_peered":"2023-09-27T23:13:44.789598+0000","last_unstale":"2023-09-27T23:13:48.810126+0000","last_undegraded":"2023-09-27T23:13:48.810126+0000","last_fullsized":"2023-09-27T23:13:48.810126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:03.782211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821279+0000","last_change":"2023-09-27T23:13:44.794170+0000","last_active":"2023-09-27T23:13:48.821279+0000","last_peered":"2023-09-27T23:13:48.821279+0000","last_clean":"2023-09-27T23:13:48.821279+0000","last_became_active":"2023-09-27T23:13:44.793843+0000","last_became_peered":"2023-09-27T23:13:44.793843+0000","last_unstale":"2023-09-27T23:13:48.821279+0000","last_undegraded":"2023-09-27T23:13:48.821279+0000","last_fullsized":"2023-09-27T23:13:48.821279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804536+0000","last_change":"2023-09-27T23:13:44.795499+0000","last_active":"2023-09-27T23:13:48.804536+0000","last_peered":"2023-09-27T23:13:48.804536+0000","last_clean":"2023-09-27T23:13:48.804536+0000","last_became_active":"2023-09-27T23:13:44.794256+0000","last_became_peered":"2023-09-27T23:13:44.794256+0000","last_unstale":"2023-09-27T23:13:48.804536+0000","last_undegraded":"2023-09-27T23:13:48.804536+0000","last_fullsized":"2023-09-27T23:13:48.804536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821399+0000","last_change":"2023-09-27T23:13:44.797131+0000","last_active":"2023-09-27T23:13:48.821399+0000","last_peered":"2023-09-27T23:13:48.821399+0000","last_clean":"2023-09-27T23:13:48.821399+0000","last_became_active":"2023-09-27T23:13:44.796814+0000","last_became_peered":"2023-09-27T23:13:44.796814+0000","last_unstale":"2023-09-27T23:13:48.821399+0000","last_undegraded":"2023-09-27T23:13:48.821399+0000","last_fullsized":"2023-09-27T23:13:48.821399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804126+0000","last_change":"2023-09-27T23:13:44.794205+0000","last_active":"2023-09-27T23:13:48.804126+0000","last_peered":"2023-09-27T23:13:48.804126+0000","last_clean":"2023-09-27T23:13:48.804126+0000","last_became_active":"2023-09-27T23:13:44.793765+0000","last_became_peered":"2023-09-27T23:13:44.793765+0000","last_unstale":"2023-09-27T23:13:48.804126+0000","last_undegraded":"2023-09-27T23:13:48.804126+0000","last_fullsized":"2023-09-27T23:13:48.804126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813812+0000","last_change":"2023-09-27T23:13:44.777827+0000","last_active":"2023-09-27T23:13:48.813812+0000","last_peered":"2023-09-27T23:13:48.813812+0000","last_clean":"2023-09-27T23:13:48.813812+0000","last_became_active":"2023-09-27T23:13:44.777461+0000","last_became_peered":"2023-09-27T23:13:44.777461+0000","last_unstale":"2023-09-27T23:13:48.813812+0000","last_undegraded":"2023-09-27T23:13:48.813812+0000","last_fullsized":"2023-09-27T23:13:48.813812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810644+0000","last_change":"2023-09-27T23:13:44.787157+0000","last_active":"2023-09-27T23:13:48.810644+0000","last_peered":"2023-09-27T23:13:48.810644+0000","last_clean":"2023-09-27T23:13:48.810644+0000","last_became_active":"2023-09-27T23:13:44.786486+0000","last_became_peered":"2023-09-27T23:13:44.786486+0000","last_unstale":"2023-09-27T23:13:48.810644+0000","last_undegraded":"2023-09-27T23:13:48.810644+0000","last_fullsized":"2023-09-27T23:13:48.810644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:01.890618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810866+0000","last_change":"2023-09-27T23:13:44.796954+0000","last_active":"2023-09-27T23:13:48.810866+0000","last_peered":"2023-09-27T23:13:48.810866+0000","last_clean":"2023-09-27T23:13:48.810866+0000","last_became_active":"2023-09-27T23:13:44.796634+0000","last_became_peered":"2023-09-27T23:13:44.796634+0000","last_unstale":"2023-09-27T23:13:48.810866+0000","last_undegraded":"2023-09-27T23:13:48.810866+0000","last_fullsized":"2023-09-27T23:13:48.810866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:04.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821286+0000","last_change":"2023-09-27T23:13:44.797615+0000","last_active":"2023-09-27T23:13:48.821286+0000","last_peered":"2023-09-27T23:13:48.821286+0000","last_clean":"2023-09-27T23:13:48.821286+0000","last_became_active":"2023-09-27T23:13:44.797325+0000","last_became_peered":"2023-09-27T23:13:44.797325+0000","last_unstale":"2023-09-27T23:13:48.821286+0000","last_undegraded":"2023-09-27T23:13:48.821286+0000","last_fullsized":"2023-09-27T23:13:48.821286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.819151+0000","last_change":"2023-09-27T23:13:44.801326+0000","last_active":"2023-09-27T23:13:48.819151+0000","last_peered":"2023-09-27T23:13:48.819151+0000","last_clean":"2023-09-27T23:13:48.819151+0000","last_became_active":"2023-09-27T23:13:44.800936+0000","last_became_peered":"2023-09-27T23:13:44.800936+0000","last_unstale":"2023-09-27T23:13:48.819151+0000","last_undegraded":"2023-09-27T23:13:48.819151+0000","last_fullsized":"2023-09-27T23:13:48.819151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803144+0000","last_change":"2023-09-27T23:13:44.800702+0000","last_active":"2023-09-27T23:13:48.803144+0000","last_peered":"2023-09-27T23:13:48.803144+0000","last_clean":"2023-09-27T23:13:48.803144+0000","last_became_active":"2023-09-27T23:13:44.800422+0000","last_became_peered":"2023-09-27T23:13:44.800422+0000","last_unstale":"2023-09-27T23:13:48.803144+0000","last_undegraded":"2023-09-27T23:13:48.803144+0000","last_fullsized":"2023-09-27T23:13:48.803144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810589+0000","last_change":"2023-09-27T23:13:44.802316+0000","last_active":"2023-09-27T23:13:48.810589+0000","last_peered":"2023-09-27T23:13:48.810589+0000","last_clean":"2023-09-27T23:13:48.810589+0000","last_became_active":"2023-09-27T23:13:44.800990+0000","last_became_peered":"2023-09-27T23:13:44.800990+0000","last_unstale":"2023-09-27T23:13:48.810589+0000","last_undegraded":"2023-09-27T23:13:48.810589+0000","last_fullsized":"2023-09-27T23:13:48.810589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804015+0000","last_change":"2023-09-27T23:13:44.794033+0000","last_active":"2023-09-27T23:13:48.804015+0000","last_peered":"2023-09-27T23:13:48.804015+0000","last_clean":"2023-09-27T23:13:48.804015+0000","last_became_active":"2023-09-27T23:13:44.793696+0000","last_became_peered":"2023-09-27T23:13:44.793696+0000","last_unstale":"2023-09-27T23:13:48.804015+0000","last_undegraded":"2023-09-27T23:13:48.804015+0000","last_fullsized":"2023-09-27T23:13:48.804015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820743+0000","last_change":"2023-09-27T23:13:44.795881+0000","last_active":"2023-09-27T23:13:48.820743+0000","last_peered":"2023-09-27T23:13:48.820743+0000","last_clean":"2023-09-27T23:13:48.820743+0000","last_became_active":"2023-09-27T23:13:44.795554+0000","last_became_peered":"2023-09-27T23:13:44.795554+0000","last_unstale":"2023-09-27T23:13:48.820743+0000","last_undegraded":"2023-09-27T23:13:48.820743+0000","last_fullsized":"2023-09-27T23:13:48.820743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:16.262346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804794+0000","last_change":"2023-09-27T23:13:44.801496+0000","last_active":"2023-09-27T23:13:48.804794+0000","last_peered":"2023-09-27T23:13:48.804794+0000","last_clean":"2023-09-27T23:13:48.804794+0000","last_became_active":"2023-09-27T23:13:44.801264+0000","last_became_peered":"2023-09-27T23:13:44.801264+0000","last_unstale":"2023-09-27T23:13:48.804794+0000","last_undegraded":"2023-09-27T23:13:48.804794+0000","last_fullsized":"2023-09-27T23:13:48.804794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812717+0000","last_change":"2023-09-27T23:13:44.790757+0000","last_active":"2023-09-27T23:13:48.812717+0000","last_peered":"2023-09-27T23:13:48.812717+0000","last_clean":"2023-09-27T23:13:48.812717+0000","last_became_active":"2023-09-27T23:13:44.790388+0000","last_became_peered":"2023-09-27T23:13:44.790388+0000","last_unstale":"2023-09-27T23:13:48.812717+0000","last_undegraded":"2023-09-27T23:13:48.812717+0000","last_fullsized":"2023-09-27T23:13:48.812717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821180+0000","last_change":"2023-09-27T23:13:44.799201+0000","last_active":"2023-09-27T23:13:48.821180+0000","last_peered":"2023-09-27T23:13:48.821180+0000","last_clean":"2023-09-27T23:13:48.821180+0000","last_became_active":"2023-09-27T23:13:44.798712+0000","last_became_peered":"2023-09-27T23:13:44.798712+0000","last_unstale":"2023-09-27T23:13:48.821180+0000","last_undegraded":"2023-09-27T23:13:48.821180+0000","last_fullsized":"2023-09-27T23:13:48.821180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815177+0000","last_change":"2023-09-27T23:13:44.780452+0000","last_active":"2023-09-27T23:13:48.815177+0000","last_peered":"2023-09-27T23:13:48.815177+0000","last_clean":"2023-09-27T23:13:48.815177+0000","last_became_active":"2023-09-27T23:13:44.780111+0000","last_became_peered":"2023-09-27T23:13:44.780111+0000","last_unstale":"2023-09-27T23:13:48.815177+0000","last_undegraded":"2023-09-27T23:13:48.815177+0000","last_fullsized":"2023-09-27T23:13:48.815177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810755+0000","last_change":"2023-09-27T23:13:44.794473+0000","last_active":"2023-09-27T23:13:48.810755+0000","last_peered":"2023-09-27T23:13:48.810755+0000","last_clean":"2023-09-27T23:13:48.810755+0000","last_became_active":"2023-09-27T23:13:44.794187+0000","last_became_peered":"2023-09-27T23:13:44.794187+0000","last_unstale":"2023-09-27T23:13:48.810755+0000","last_undegraded":"2023-09-27T23:13:48.810755+0000","last_fullsized":"2023-09-27T23:13:48.810755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9300,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362540,"statfs":{"total":96636764160,"available":96627240960,"internally_reserved":0,"allocated":929792,"data_stored":717011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8188,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363652,"statfs":{"total":96636764160,"available":96628379648,"internally_reserved":0,"allocated":315392,"data_stored":125515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8264,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363576,"statfs":{"total":96636764160,"available":96628301824,"internally_reserved":0,"allocated":327680,"data_stored":126483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":124981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":905216,"data_stored":715813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:13:58.195 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:13:58.195 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-27T23:13:58.196 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-27T23:13:58.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-27T23:13:58.196 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-27T23:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 Processor -- start 2023-09-27T23:13:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- start start 2023-09-27T23:13:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9380c15b0 0x7fb9380aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380c7190 con 0x7fb93806d690 2023-09-27T23:13:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9380c1b50 con 0x7fb9380c15b0 2023-09-27T23:13:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380c1e30 con 0x7fb93815d320 2023-09-27T23:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9380c15b0 0x7fb9380aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9380c15b0 0x7fb9380aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44282/0 (socket says 172.21.15.64:44282) 2023-09-27T23:13:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 -- 172.21.15.64:0/3667310551 learned_addr learned my addr 172.21.15.64:0/3667310551 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb936d9d700 1 --2- 172.21.15.64:0/3667310551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 -- 172.21.15.64:0/3667310551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 msgr2=0x7fb93815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 --2- 172.21.15.64:0/3667310551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 -- 172.21.15.64:0/3667310551 >> v1:172.21.15.123:6789/0 conn(0x7fb93806d690 legacy=0x7fb9380680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 -- 172.21.15.64:0/3667310551 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380c6e20 con 0x7fb9380c15b0 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93659c700 1 --2- 172.21.15.64:0/3667310551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9380c15b0 0x7fb9380aeba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb930009f20 tx=0x7fb930009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fbeae9da08b15a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb935d9b700 1 -- 172.21.15.64:0/3667310551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000e070 con 0x7fb9380c15b0 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb935d9b700 1 -- 172.21.15.64:0/3667310551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93000ba80 con 0x7fb9380c15b0 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb935d9b700 1 -- 172.21.15.64:0/3667310551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9300092e0 con 0x7fb9380c15b0 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- 172.21.15.64:0/3667310551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9380c15b0 msgr2=0x7fb9380aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 --2- 172.21.15.64:0/3667310551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9380c15b0 0x7fb9380aeba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb930009f20 tx=0x7fb930009840 comp rx=0 tx=0).stop 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- 172.21.15.64:0/3667310551 shutdown_connections 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 --2- 172.21.15.64:0/3667310551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9380c15b0 0x7fb9380aeba0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 --2- 172.21.15.64:0/3667310551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- 172.21.15.64:0/3667310551 >> 172.21.15.64:0/3667310551 conn(0x7fb9380c8850 msgr2=0x7fb9380c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- 172.21.15.64:0/3667310551 shutdown_connections 2023-09-27T23:13:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.302+0000 7fb93d12d700 1 -- 172.21.15.64:0/3667310551 wait complete. 2023-09-27T23:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 Processor -- start 2023-09-27T23:13:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 -- start start 2023-09-27T23:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 0x7fb9380c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9380c15b0 0x7fb938159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815dc30 con 0x7fb93806d690 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815ea70 con 0x7fb9380c15b0 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815de60 con 0x7fb93815d320 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 0x7fb9380c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9380c15b0 0x7fb938159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 0x7fb9380c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44288/0 (socket says 172.21.15.64:44288) 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 -- 172.21.15.64:0/2455690517 learned_addr learned my addr 172.21.15.64:0/2455690517 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93759e700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 -- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 msgr2=0x7fb93815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 -- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9380c15b0 msgr2=0x7fb938159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9380c15b0 0x7fb938159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 -- 172.21.15.64:0/2455690517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009ae0 con 0x7fb93806d690 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb936d9d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 0x7fb9380c5810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d860 tx=0x7fb928008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9280125f0 con 0x7fb93806d690 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb928012750 con 0x7fb93806d690 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93815e140 con 0x7fb93806d690 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928012b10 con 0x7fb93806d690 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.306+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93815e700 con 0x7fb93806d690 2023-09-27T23:13:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.310+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938071120 con 0x7fb93806d690 2023-09-27T23:13:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.310+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb92801e460 con 0x7fb93806d690 2023-09-27T23:13:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.310+0000 7fb92effd700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb918060950 0x7fb918062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.310+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb92807d760 con 0x7fb93806d690 2023-09-27T23:13:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.310+0000 7fb93659c700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb918060950 0x7fb918062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.310+0000 7fb93659c700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb918060950 0x7fb918062e00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb930009880 tx=0x7fb930018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.318+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb9280510e0 con 0x7fb93806d690 2023-09-27T23:13:58.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.514+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb9380642a0 con 0x7fb93806d690 2023-09-27T23:13:58.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.522+0000 7fb92effd700 1 -- 172.21.15.64:0/2455690517 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154656 (secure 0 0 0) 0x7fb928050c70 con 0x7fb93806d690 2023-09-27T23:13:58.529 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:58.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb918060950 msgr2=0x7fb918062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb918060950 0x7fb918062e00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb930009880 tx=0x7fb930018000 comp rx=0 tx=0).stop 2023-09-27T23:13:58.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 msgr2=0x7fb9380c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 0x7fb9380c5810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d860 tx=0x7fb928008040 comp rx=0 tx=0).stop 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 shutdown_connections 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb918060950 0x7fb918062e00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93815d320 0x7fb93815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9380c15b0 0x7fb938159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 --2- 172.21.15.64:0/2455690517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93806d690 0x7fb9380c5810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 >> 172.21.15.64:0/2455690517 conn(0x7fb9380c8850 msgr2=0x7fb93806b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 shutdown_connections 2023-09-27T23:13:58.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.526+0000 7fb93d12d700 1 -- 172.21.15.64:0/2455690517 wait complete. 2023-09-27T23:13:58.543 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":33228},{"type":"v1","addr":"172.21.15.123:6833","nonce":33228}]},"active_addr":"172.21.15.123:6833/33228","active_change":"2023-09-27T23:13:25.125821+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4114,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3145850360}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":118144335}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1007482582}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":439915678}]}]} 2023-09-27T23:13:58.546 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-27T23:13:58.547 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-27T23:13:58.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:58.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c8982700 1 Processor -- start 2023-09-27T23:13:58.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c8982700 1 -- start start 2023-09-27T23:13:58.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c8982700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9c406cac0 0x7fa9c406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c8982700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c406e1c0 con 0x7fa9c415cfd0 2023-09-27T23:13:58.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c8982700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c406d5b0 con 0x7fa9c40cc130 2023-09-27T23:13:58.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c8982700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c406d3d0 con 0x7fa9c406cac0 2023-09-27T23:13:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c2d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9c406cac0 0x7fa9c406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c359e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa9c415cfd0 0x7fa9c40678e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34004/0 (socket says 172.21.15.64:34004) 2023-09-27T23:13:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c2d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9c406cac0 0x7fa9c406ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44300/0 (socket says 172.21.15.64:44300) 2023-09-27T23:13:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.650+0000 7fa9c359e700 1 -- 172.21.15.64:0/1121487363 learned_addr learned my addr 172.21.15.64:0/1121487363 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:58.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/1121487363 >> v1:172.21.15.123:6789/0 conn(0x7fa9c40cc130 legacy=0x7fa9c415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/1121487363 >> v1:172.21.15.64:6789/0 conn(0x7fa9c415cfd0 legacy=0x7fa9c40678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/1121487363 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c406d890 con 0x7fa9c406cac0 2023-09-27T23:13:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c2d9d700 1 --2- 172.21.15.64:0/1121487363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c406ce90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40098e0 tx=0x7fa9b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce22d5c06976af14 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c1d9b700 1 -- 172.21.15.64:0/1121487363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4009e80 con 0x7fa9c406cac0 2023-09-27T23:13:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c1d9b700 1 -- 172.21.15.64:0/1121487363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400b860 con 0x7fa9c406cac0 2023-09-27T23:13:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c1d9b700 1 -- 172.21.15.64:0/1121487363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400bbe0 con 0x7fa9c406cac0 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 -- 172.21.15.64:0/1121487363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 msgr2=0x7fa9c406ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 --2- 172.21.15.64:0/1121487363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c406ce90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40098e0 tx=0x7fa9b40091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 -- 172.21.15.64:0/1121487363 shutdown_connections 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 --2- 172.21.15.64:0/1121487363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c406ce90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 -- 172.21.15.64:0/1121487363 >> 172.21.15.64:0/1121487363 conn(0x7fa9c40cdbd0 msgr2=0x7fa9c40cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 -- 172.21.15.64:0/1121487363 shutdown_connections 2023-09-27T23:13:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 -- 172.21.15.64:0/1121487363 wait complete. 2023-09-27T23:13:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 Processor -- start 2023-09-27T23:13:58.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.654+0000 7fa9c8982700 1 -- start start 2023-09-27T23:13:58.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c40b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9c40cc130 0x7fa9c414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9c415cfd0 0x7fa9c40b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40b3430 con 0x7fa9c406cac0 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40b1d70 con 0x7fa9c415cfd0 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40b2820 con 0x7fa9c40cc130 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c40b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c40b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44312/0 (socket says 172.21.15.64:44312) 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/3749640769 learned_addr learned my addr 172.21.15.64:0/3749640769 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c259c700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9c40cc130 0x7fa9c414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c359e700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9c415cfd0 0x7fa9c40b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9c40cc130 msgr2=0x7fa9c414ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9c40cc130 0x7fa9c414ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9c415cfd0 msgr2=0x7fa9c40b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9c415cfd0 0x7fa9c40b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 -- 172.21.15.64:0/3749640769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4009420 con 0x7fa9c406cac0 2023-09-27T23:13:58.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c359e700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9c415cfd0 0x7fa9c40b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c2d9d700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c40b2240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa9c406b560 tx=0x7fa9b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400bb20 con 0x7fa9c406cac0 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b4025420 con 0x7fa9c406cac0 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40257c0 con 0x7fa9c406cac0 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c40b2b00 con 0x7fa9c406cac0 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.658+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c40b30c0 con 0x7fa9c406cac0 2023-09-27T23:13:58.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.662+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c404be40 con 0x7fa9c406cac0 2023-09-27T23:13:58.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.662+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fa9b4023070 con 0x7fa9c406cac0 2023-09-27T23:13:58.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.662+0000 7fa9baffd700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9a4060980 0x7fa9a4062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.662+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa9b407e760 con 0x7fa9c406cac0 2023-09-27T23:13:58.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.662+0000 7fa9c259c700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9a4060980 0x7fa9a4062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.662+0000 7fa9c259c700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9a4060980 0x7fa9a4062e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40c41f0 tx=0x7fa9ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.670+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa9b404e6f0 con 0x7fa9c406cac0 2023-09-27T23:13:58.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.830+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9c40671b0 con 0x7fa9c406cac0 2023-09-27T23:13:58.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9baffd700 1 -- 172.21.15.64:0/3749640769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fa9b4051d10 con 0x7fa9c406cac0 2023-09-27T23:13:58.838 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:58.838 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9a4060980 msgr2=0x7fa9a4062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9a4060980 0x7fa9a4062e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40c41f0 tx=0x7fa9ac009040 comp rx=0 tx=0).stop 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 msgr2=0x7fa9c40b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c40b2240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa9c406b560 tx=0x7fa9b4017000 comp rx=0 tx=0).stop 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 shutdown_connections 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9a4060980 0x7fa9a4062e30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9c415cfd0 0x7fa9c40b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9c40cc130 0x7fa9c414ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 --2- 172.21.15.64:0/3749640769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c406cac0 0x7fa9c40b2240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 >> 172.21.15.64:0/3749640769 conn(0x7fa9c40cdbd0 msgr2=0x7fa9c40c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:58.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 shutdown_connections 2023-09-27T23:13:58.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.834+0000 7fa9c8982700 1 -- 172.21.15.64:0/3749640769 wait complete. 2023-09-27T23:13:58.852 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-27T23:13:58.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:13:58.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 Processor -- start 2023-09-27T23:13:58.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 -- start start 2023-09-27T23:13:58.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f828806ce90 0x7f828815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828815ce00 0x7f82880678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828806e1c0 con 0x7f828806cac0 2023-09-27T23:13:58.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828815d280 con 0x7f828806ce90 2023-09-27T23:13:58.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828ecc7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82880cc840 con 0x7f828815ce00 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828d4c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f828806ce90 0x7f828815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828d4c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f828806ce90 0x7f828815c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60358/0 (socket says 172.21.15.64:60358) 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828d4c4700 1 -- 172.21.15.64:0/929941446 learned_addr learned my addr 172.21.15.64:0/929941446 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.954+0000 7f828e4c6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828815ce00 0x7f82880678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828d4c4700 1 -- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828815ce00 msgr2=0x7f82880678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828d4c4700 1 --2- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828815ce00 0x7f82880678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828d4c4700 1 -- 172.21.15.64:0/929941446 >> v1:172.21.15.123:6789/0 conn(0x7f828806cac0 legacy=0x7f82880cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828d4c4700 1 -- 172.21.15.64:0/929941446 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828806d2e0 con 0x7f828806ce90 2023-09-27T23:13:58.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828d4c4700 1 --2- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828806ce90 0x7f828815c8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f827c009f20 tx=0x7f827c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3797b83ab6ba6f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ccc3700 1 -- 172.21.15.64:0/929941446 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00e070 con 0x7f828806ce90 2023-09-27T23:13:58.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ccc3700 1 -- 172.21.15.64:0/929941446 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00b9d0 con 0x7f828806ce90 2023-09-27T23:13:58.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ccc3700 1 -- 172.21.15.64:0/929941446 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0092e0 con 0x7f828806ce90 2023-09-27T23:13:58.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 -- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828806ce90 msgr2=0x7f828815c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 --2- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828806ce90 0x7f828815c8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f827c009f20 tx=0x7f827c009840 comp rx=0 tx=0).stop 2023-09-27T23:13:58.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 -- 172.21.15.64:0/929941446 shutdown_connections 2023-09-27T23:13:58.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 --2- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828815ce00 0x7f82880678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 --2- 172.21.15.64:0/929941446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828806ce90 0x7f828815c8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 -- 172.21.15.64:0/929941446 >> 172.21.15.64:0/929941446 conn(0x7f82880cdbd0 msgr2=0x7f82880cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:58.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 -- 172.21.15.64:0/929941446 shutdown_connections 2023-09-27T23:13:58.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.958+0000 7f828ecc7700 1 -- 172.21.15.64:0/929941446 wait complete. 2023-09-27T23:13:58.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 Processor -- start 2023-09-27T23:13:58.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 -- start start 2023-09-27T23:13:58.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 0x7f8288140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828806ce90 0x7f82880b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828815ce00 0x7f82880aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82880b2fd0 con 0x7f828806cac0 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82880b23c0 con 0x7f828815ce00 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828ecc7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82880b26a0 con 0x7f828806ce90 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 0x7f8288140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828e4c6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828815ce00 0x7f82880aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828d4c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828806ce90 0x7f82880b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 0x7f8288140460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44320/0 (socket says 172.21.15.64:44320) 2023-09-27T23:13:58.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 -- 172.21.15.64:0/3421511094 learned_addr learned my addr 172.21.15.64:0/3421511094 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 -- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828806ce90 msgr2=0x7f82880b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828806ce90 0x7f82880b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 -- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828815ce00 msgr2=0x7f82880aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828815ce00 0x7f82880aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828dcc5700 1 -- 172.21.15.64:0/3421511094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009ae0 con 0x7f828806cac0 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.962+0000 7f828e4c6700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828815ce00 0x7f82880aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f828dcc5700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 0x7f8288140460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f827800c4e0 tx=0x7f827800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278011590 con 0x7f828806cac0 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278011710 con 0x7f828806cac0 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82880b2980 con 0x7f828806cac0 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278011ad0 con 0x7f828806cac0 2023-09-27T23:13:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828815c620 con 0x7f828806cac0 2023-09-27T23:13:58.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828815a230 con 0x7f828806cac0 2023-09-27T23:13:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8278001020 con 0x7f828806cac0 2023-09-27T23:13:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.966+0000 7f8285ffb700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8274060980 0x7f8274062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.970+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f827807c4b0 con 0x7f828806cac0 2023-09-27T23:13:58.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.970+0000 7f828d4c4700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8274060980 0x7f8274062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:58.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.970+0000 7f828d4c4700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8274060980 0x7f8274062e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f827c009880 tx=0x7f827c018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:58.978+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f827804c440 con 0x7f828806cac0 2023-09-27T23:13:59.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.142+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f828804be40 con 0x7f828806cac0 2023-09-27T23:13:59.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.142+0000 7f8285ffb700 1 -- 172.21.15.64:0/3421511094 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f8278017090 con 0x7f828806cac0 2023-09-27T23:13:59.149 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:13:59.149 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8274060980 msgr2=0x7f8274062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8274060980 0x7f8274062e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f827c009880 tx=0x7f827c018000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 msgr2=0x7f8288140460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 0x7f8288140460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f827800c4e0 tx=0x7f827800c7f0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 shutdown_connections 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8274060980 0x7f8274062e30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f828815ce00 0x7f82880aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f828806ce90 0x7f82880b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 --2- 172.21.15.64:0/3421511094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f828806cac0 0x7f8288140460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 >> 172.21.15.64:0/3421511094 conn(0x7f82880cdbd0 msgr2=0x7f82880cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 shutdown_connections 2023-09-27T23:13:59.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.146+0000 7f828ecc7700 1 -- 172.21.15.64:0/3421511094 wait complete. 2023-09-27T23:13:59.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-27T23:13:59.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-27T23:13:59.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-27T23:13:59.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-27T23:13:59.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-09-27T23:13:59.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-09-27T23:13:59.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-09-27T23:13:59.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-09-27T23:13:59.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.302+0000 7fdae7fff700 1 Processor -- start 2023-09-27T23:13:59.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.302+0000 7fdae7fff700 1 -- start start 2023-09-27T23:13:59.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdae8159160 0x7fdae8159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdae80c5fd0 0x7fdae80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae80c6400 con 0x7fdae80ae210 2023-09-27T23:13:59.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae806a7b0 con 0x7fdae80c5fd0 2023-09-27T23:13:59.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae80ad550 con 0x7fdae8159160 2023-09-27T23:13:59.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae67fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdae80c5fd0 0x7fdae80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae67fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdae80c5fd0 0x7fdae80adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44322/0 (socket says 172.21.15.64:44322) 2023-09-27T23:13:59.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae77fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdae80ae210 0x7fdae8158200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59612/0 (socket says 172.21.15.64:59612) 2023-09-27T23:13:59.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae77fe700 1 -- 172.21.15.64:0/973931249 learned_addr learned my addr 172.21.15.64:0/973931249 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae6ffd700 1 --2- 172.21.15.64:0/973931249 >> v2:172.21.15.123:3300/0 conn(0x7fdae8159160 0x7fdae8159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae6ffd700 1 -- 172.21.15.64:0/973931249 >> v2:172.21.15.64:3300/0 conn(0x7fdae80c5fd0 msgr2=0x7fdae80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae6ffd700 1 --2- 172.21.15.64:0/973931249 >> v2:172.21.15.64:3300/0 conn(0x7fdae80c5fd0 0x7fdae80adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae6ffd700 1 -- 172.21.15.64:0/973931249 >> v1:172.21.15.123:6789/0 conn(0x7fdae80ae210 legacy=0x7fdae8158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae6ffd700 1 -- 172.21.15.64:0/973931249 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80accb0 con 0x7fdae8159160 2023-09-27T23:13:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae6ffd700 1 --2- 172.21.15.64:0/973931249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae8159160 0x7fdae8159530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdae80c94d0 tx=0x7fdadc00a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b20848f3b6273f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae5ffb700 1 -- 172.21.15.64:0/973931249 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00c870 con 0x7fdae8159160 2023-09-27T23:13:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae5ffb700 1 -- 172.21.15.64:0/973931249 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc00c9d0 con 0x7fdae8159160 2023-09-27T23:13:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae5ffb700 1 -- 172.21.15.64:0/973931249 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00cd70 con 0x7fdae8159160 2023-09-27T23:13:59.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fa42e667700 1 Processor -- start 2023-09-27T23:13:59.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7f3dc20cb700 1 Processor -- start 2023-09-27T23:13:59.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7f3dc20cb700 1 -- start start 2023-09-27T23:13:59.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7ff7adc89700 1 Processor -- start 2023-09-27T23:13:59.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 -- 172.21.15.64:0/973931249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae8159160 msgr2=0x7fdae8159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fdae7fff700 1 --2- 172.21.15.64:0/973931249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae8159160 0x7fdae8159530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdae80c94d0 tx=0x7fdadc00a330 comp rx=0 tx=0).stop 2023-09-27T23:13:59.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7f69d319d700 1 Processor -- start 2023-09-27T23:13:59.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7f69d319d700 1 -- start start 2023-09-27T23:13:59.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7f0147fff700 1 Processor -- start 2023-09-27T23:13:59.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7f0147fff700 1 -- start start 2023-09-27T23:13:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fa42e667700 1 -- start start 2023-09-27T23:13:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fa42e667700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa428154c20 0x7fa4280abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7fa42e667700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa428154ff0 0x7fa42806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280c5fd0 con 0x7fa42806cd00 2023-09-27T23:13:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280c5c70 con 0x7fa428154ff0 2023-09-27T23:13:59.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42806d130 con 0x7fa428154c20 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc20cb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3dbc06de20 0x7f3dbc0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc20cb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dbc06d990 con 0x7f3dbc06c7c0 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc20cb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dbc06cd80 con 0x7f3dbc155960 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc20cb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dbc06d060 con 0x7f3dbc06de20 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa428154ff0 0x7fa42806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42de66700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa42806cd00 0x7fa42806a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34022/0 (socket says 172.21.15.64:34022) 2023-09-27T23:13:59.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa428154ff0 0x7fa42806c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60366/0 (socket says 172.21.15.64:60366) 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42de66700 1 -- 172.21.15.64:0/607522948 learned_addr learned my addr 172.21.15.64:0/607522948 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42d665700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa428154c20 0x7fa4280abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc18ca700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3dbc06c7c0 0x7f3dbc0aca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34020/0 (socket says 172.21.15.64:34020) 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc18ca700 1 -- 172.21.15.64:0/1547230321 learned_addr learned my addr 172.21.15.64:0/1547230321 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69d319d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f69cc15a9f0 0x7f69cc0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69d319d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69cc15adc0 0x7f69cc0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69d319d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc15e3a0 con 0x7f69cc06b200 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69d319d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc15e680 con 0x7f69cc15adc0 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69d319d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc06a720 con 0x7f69cc15a9f0 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69cc15adc0 0x7f69cc0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69cc15adc0 0x7f69cc0c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44324/0 (socket says 172.21.15.64:44324) 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 -- 172.21.15.64:0/45524887 learned_addr learned my addr 172.21.15.64:0/45524887 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 -- 172.21.15.64:0/973931249 shutdown_connections 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 --2- 172.21.15.64:0/973931249 >> v2:172.21.15.64:3300/0 conn(0x7fdae80c5fd0 0x7fdae80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 --2- 172.21.15.64:0/973931249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae8159160 0x7fdae8159530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 -- 172.21.15.64:0/973931249 >> 172.21.15.64:0/973931249 conn(0x7fdae8067bb0 msgr2=0x7fdae80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.306+0000 7ff7adc89700 1 -- start start 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7a8154ff0 0x7ff7a806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7a806cd00 0x7ff7a806a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a80c5c70 con 0x7ff7a806cd00 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a80c5fd0 con 0x7ff7a8154ff0 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a806d130 con 0x7ff7a8154c20 2023-09-27T23:13:59.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7a806cd00 0x7ff7a806a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7a806cd00 0x7ff7a806a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44332/0 (socket says 172.21.15.64:44332) 2023-09-27T23:13:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 -- 172.21.15.64:0/129937452 learned_addr learned my addr 172.21.15.64:0/129937452 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7a7fff700 1 --2- 172.21.15.64:0/129937452 >> v2:172.21.15.123:3300/0 conn(0x7ff7a8154ff0 0x7ff7a806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc08c8700 1 --2- 172.21.15.64:0/1547230321 >> v2:172.21.15.123:3300/0 conn(0x7f3dbc06de20 0x7f3dbc0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69d0f39700 1 --2- 172.21.15.64:0/45524887 >> v2:172.21.15.123:3300/0 conn(0x7f69cc15a9f0 0x7f69cc0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 -- 172.21.15.64:0/973931249 shutdown_connections 2023-09-27T23:13:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640233481 0 0) 0x7f3dbc06d990 con 0x7f3dbc06c7c0 2023-09-27T23:13:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 -- 172.21.15.64:0/45524887 >> v2:172.21.15.123:3300/0 conn(0x7f69cc15a9f0 msgr2=0x7f69cc0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 -- 172.21.15.64:0/973931249 wait complete. 2023-09-27T23:13:59.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da8003570 con 0x7f3dbc06c7c0 2023-09-27T23:13:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7acc87700 1 --2- 172.21.15.64:0/129937452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 --2- 172.21.15.64:0/45524887 >> v2:172.21.15.123:3300/0 conn(0x7f69cc15a9f0 0x7f69cc0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 -- 172.21.15.64:0/45524887 >> v1:172.21.15.123:6789/0 conn(0x7f69cc06b200 legacy=0x7f69cc15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 -- 172.21.15.64:0/45524887 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc06aa00 con 0x7f69cc15adc0 2023-09-27T23:13:59.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 -- 172.21.15.64:0/129937452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 msgr2=0x7ff7a80abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 --2- 172.21.15.64:0/129937452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 -- 172.21.15.64:0/129937452 >> v2:172.21.15.123:3300/0 conn(0x7ff7a8154ff0 msgr2=0x7ff7a806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 --2- 172.21.15.64:0/129937452 >> v2:172.21.15.123:3300/0 conn(0x7ff7a8154ff0 0x7ff7a806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 -- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa428154c20 msgr2=0x7fa4280abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa428154c20 0x7fa4280abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 -- 172.21.15.64:0/607522948 >> v1:172.21.15.64:6789/0 conn(0x7fa42806cd00 legacy=0x7fa42806a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 -- 172.21.15.64:0/607522948 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42806d3a0 con 0x7fa428154ff0 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42d665700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa428154c20 0x7fa4280abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331785502 0 0) 0x7f3da8003570 con 0x7f3dbc06c7c0 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 >> v2:172.21.15.123:3300/0 conn(0x7f3dbc06de20 msgr2=0x7f3dbc0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 --2- 172.21.15.64:0/1547230321 >> v2:172.21.15.123:3300/0 conn(0x7f3dbc06de20 0x7f3dbc0c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 >> v1:172.21.15.123:6789/0 conn(0x7f3dbc155960 legacy=0x7f3dbc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc06d340 con 0x7f3dbc06c7c0 2023-09-27T23:13:59.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f3db8001470 con 0x7f3dbc06c7c0 2023-09-27T23:13:59.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0147fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0148158bc0 0x7f0148158f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 >> v1:172.21.15.64:6789/0 conn(0x7f3dbc06c7c0 legacy=0x7f3dbc0aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0147fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f01480c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 -- 172.21.15.64:0/129937452 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a806d3a0 con 0x7ff7a806cd00 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da8003b30 0x7f3da8003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da8004810 0x7f3da8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3da800fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc06d060 con 0x7f3da8003b30 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80105a0 con 0x7f3da8004810 2023-09-27T23:13:59.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8010760 con 0x7f3da80052e0 2023-09-27T23:13:59.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7ad488700 1 --2- 172.21.15.64:0/129937452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a806a580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8069cb0 tx=0x7ff7a0007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92905aec3c7a0d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0148158bc0 0x7f0148158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0148158bc0 0x7f0148158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44334/0 (socket says 172.21.15.64:44334) 2023-09-27T23:13:59.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 -- 172.21.15.64:0/4171765832 learned_addr learned my addr 172.21.15.64:0/4171765832 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014806b570 con 0x7f01480cc5a0 2023-09-27T23:13:59.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7a77fe700 1 -- 172.21.15.64:0/129937452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0010070 con 0x7ff7a806cd00 2023-09-27T23:13:59.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f01467fc700 1 --2- 172.21.15.64:0/4171765832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f01480c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014806a960 con 0x7f0148158bc0 2023-09-27T23:13:59.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42ce64700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154ff0 0x7fa42806c7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa42400aa90 tx=0x7fa42400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3e389cecf71e24 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014806ac40 con 0x7f01480cc150 2023-09-27T23:13:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7a77fe700 1 -- 172.21.15.64:0/129937452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a000dbd0 con 0x7ff7a806cd00 2023-09-27T23:13:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7a77fe700 1 -- 172.21.15.64:0/129937452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0007b40 con 0x7ff7a806cd00 2023-09-27T23:13:59.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cbfff700 1 --2- 172.21.15.64:0/45524887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15adc0 0x7f69cc0c8690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69cc06b6b0 tx=0x7f69c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6100c6196f99b9f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc10c9700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da8003b30 0x7f3da8003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3da800fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa41ffff700 1 -- 172.21.15.64:0/607522948 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400c930 con 0x7fa428154ff0 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa41ffff700 1 -- 172.21.15.64:0/607522948 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42400ca90 con 0x7fa428154ff0 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa41ffff700 1 -- 172.21.15.64:0/607522948 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400ce30 con 0x7fa428154ff0 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 -- 172.21.15.64:0/4171765832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 msgr2=0x7f01480c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 --2- 172.21.15.64:0/4171765832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f01480c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 -- 172.21.15.64:0/4171765832 >> v1:172.21.15.123:6789/0 conn(0x7f01480cc5a0 legacy=0x7f014815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 -- 172.21.15.64:0/4171765832 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014806af20 con 0x7f0148158bc0 2023-09-27T23:13:59.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cb7fe700 1 -- 172.21.15.64:0/45524887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000b820 con 0x7f69cc15adc0 2023-09-27T23:13:59.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0146ffd700 1 --2- 172.21.15.64:0/4171765832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f0148158f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01380099b0 tx=0x7f0138009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99ccc84c3fead90 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 Processor -- start 2023-09-27T23:13:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f0145ffb700 1 -- 172.21.15.64:0/4171765832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138009ea0 con 0x7f0148158bc0 2023-09-27T23:13:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc18ca700 1 -- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da8004810 msgr2=0x7f3da8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da8004810 0x7f3da8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f3dc18ca700 1 -- 172.21.15.64:0/1547230321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da8003b30 msgr2=0x7f3da8003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 -- 172.21.15.64:0/129937452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 msgr2=0x7ff7a806a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- 172.21.15.64:0/129937452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a806a580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8069cb0 tx=0x7ff7a0007690 comp rx=0 tx=0).stop 2023-09-27T23:13:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fdae7fff700 1 -- start start 2023-09-27T23:13:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cb7fe700 1 -- 172.21.15.64:0/45524887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c000b980 con 0x7f69cc15adc0 2023-09-27T23:13:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7f69cb7fe700 1 -- 172.21.15.64:0/45524887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000bd20 con 0x7f69cc15adc0 2023-09-27T23:13:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 -- 172.21.15.64:0/129937452 shutdown_connections 2023-09-27T23:13:59.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- 172.21.15.64:0/129937452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a806a580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- 172.21.15.64:0/129937452 >> v2:172.21.15.123:3300/0 conn(0x7ff7a8154ff0 0x7ff7a806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 --2- 172.21.15.64:0/129937452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7ff7adc89700 1 -- 172.21.15.64:0/129937452 >> 172.21.15.64:0/129937452 conn(0x7ff7a8067bb0 msgr2=0x7ff7a80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154ff0 msgr2=0x7fa42806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154ff0 0x7fa42806c7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa42400aa90 tx=0x7fa42400a330 comp rx=0 tx=0).stop 2023-09-27T23:13:59.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- 172.21.15.64:0/607522948 shutdown_connections 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154ff0 0x7fa42806c7c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 --2- 172.21.15.64:0/607522948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa428154c20 0x7fa4280abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- 172.21.15.64:0/607522948 >> 172.21.15.64:0/607522948 conn(0x7fa428067bb0 msgr2=0x7fa4280cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- 172.21.15.64:0/607522948 shutdown_connections 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.310+0000 7fa42e667700 1 -- 172.21.15.64:0/607522948 wait complete. 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da8003b30 0x7f3da8003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc18ca700 1 -- 172.21.15.64:0/1547230321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8003570 con 0x7f3da80052e0 2023-09-27T23:13:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0145ffb700 1 -- 172.21.15.64:0/4171765832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013800b820 con 0x7f0148158bc0 2023-09-27T23:13:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0145ffb700 1 -- 172.21.15.64:0/4171765832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800bbc0 con 0x7f0148158bc0 2023-09-27T23:13:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc18ca700 1 -- 172.21.15.64:0/1547230321 reap_dead start 2023-09-27T23:13:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- 172.21.15.64:0/45524887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15adc0 msgr2=0x7f69cc0c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 --2- 172.21.15.64:0/45524887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15adc0 0x7f69cc0c8690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69cc06b6b0 tx=0x7f69c0009250 comp rx=0 tx=0).stop 2023-09-27T23:13:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdae80ae210 0x7fdae8157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae80c5fd0 0x7fdae8154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae8159160 0x7fdae8150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc10c9700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da8003b30 0x7f3da8003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:59.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- 172.21.15.64:0/45524887 shutdown_connections 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 --2- 172.21.15.64:0/45524887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15adc0 0x7f69cc0c8690 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 --2- 172.21.15.64:0/45524887 >> v2:172.21.15.123:3300/0 conn(0x7f69cc15a9f0 0x7f69cc0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- 172.21.15.64:0/45524887 >> 172.21.15.64:0/45524887 conn(0x7f69cc0670e0 msgr2=0x7f69cc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8156c90 con 0x7fdae8159160 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8156f70 con 0x7fdae80c5fd0 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80ae980 con 0x7fdae80ae210 2023-09-27T23:13:59.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3da800fe70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3db800c2d0 tx=0x7f3db8001da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42dd39c10881bff9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae8159160 0x7fdae8150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae80c5fd0 0x7fdae8154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae8159160 0x7fdae8150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44354/0 (socket says 172.21.15.64:44354) 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 -- 172.21.15.64:0/4047572009 learned_addr learned my addr 172.21.15.64:0/4047572009 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae80c5fd0 0x7fdae8154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60424/0 (socket says 172.21.15.64:60424) 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae6ffd700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdae80ae210 0x7fdae8157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- 172.21.15.64:0/45524887 shutdown_connections 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- 172.21.15.64:0/45524887 wait complete. 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 Processor -- start 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 -- start start 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db800cba0 con 0x7f3da80052e0 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db800cd40 con 0x7f3da80052e0 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3db3fff700 1 -- 172.21.15.64:0/1547230321 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8005b70 con 0x7f3da80052e0 2023-09-27T23:13:59.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 -- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 msgr2=0x7f3da800fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3da800fe70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3db800c2d0 tx=0x7f3db8001da0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 -- 172.21.15.64:0/1547230321 shutdown_connections 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 --2- 172.21.15.64:0/1547230321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3da800fe70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 -- 172.21.15.64:0/1547230321 >> 172.21.15.64:0/1547230321 conn(0x7f3dbc067bb0 msgr2=0x7f3dbc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 -- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdae80ae210 msgr2=0x7fdae8157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdae80ae210 0x7fdae8157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 -- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae80c5fd0 msgr2=0x7fdae8154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdae80c5fd0 0x7fdae8154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 -- 172.21.15.64:0/4047572009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc00a5d0 con 0x7fdae8159160 2023-09-27T23:13:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae77fe700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae8159160 0x7fdae8150130 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdae000cca0 tx=0x7fdae0001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000a9c0 con 0x7fdae8159160 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae000ab40 con 0x7fdae8159160 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00070f0 con 0x7fdae8159160 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae80aec60 con 0x7fdae8159160 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae8157bc0 con 0x7fdae8159160 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7ff7adc89700 1 -- 172.21.15.64:0/129937452 shutdown_connections 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7ff7adc89700 1 -- 172.21.15.64:0/129937452 wait complete. 2023-09-27T23:13:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 -- 172.21.15.64:0/1547230321 shutdown_connections 2023-09-27T23:13:59.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdae804be40 con 0x7fdae8159160 2023-09-27T23:13:59.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa42806cd00 0x7fa4280afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154c20 0x7fa4281656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa42814c0a0 0x7fa42814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa428165c20 con 0x7fa42806cd00 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42814f9f0 con 0x7fa428154c20 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42ce64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154c20 0x7fa4281656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42ce64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154c20 0x7fa4281656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60434/0 (socket says 172.21.15.64:60434) 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42ce64700 1 -- 172.21.15.64:0/137371346 learned_addr learned my addr 172.21.15.64:0/137371346 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7fa42e667700 1 -- 172.21.15.64:0/137371346 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42814ede0 con 0x7fa42814c0a0 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 Processor -- start 2023-09-27T23:13:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- start start 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc06b200 0x7f69cc151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15a9f0 0x7f69cc15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 Processor -- start 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15adc0 0x7f69cc15fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0af730 con 0x7f69cc15a9f0 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0afa10 con 0x7f69cc15adc0 2023-09-27T23:13:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f69d319d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0afcf0 con 0x7f69cc06b200 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 msgr2=0x7f0148158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0147fff700 1 --2- 172.21.15.64:0/4171765832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f0148158f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01380099b0 tx=0x7f0138009250 comp rx=0 tx=0).stop 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 shutdown_connections 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0147fff700 1 --2- 172.21.15.64:0/4171765832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f01480c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0147fff700 1 --2- 172.21.15.64:0/4171765832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f0148158f90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 >> 172.21.15.64:0/4171765832 conn(0x7f0148067bb0 msgr2=0x7f01480cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15a9f0 0x7f69cc15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.314+0000 7f3dc20cb700 1 -- 172.21.15.64:0/1547230321 wait complete. 2023-09-27T23:13:59.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 -- start start 2023-09-27T23:13:59.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42de66700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa42814c0a0 0x7fa42814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42d665700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa42806cd00 0x7fa4280afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fdae0007520 con 0x7fdae8159160 2023-09-27T23:13:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42ce64700 1 -- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa42814c0a0 msgr2=0x7fa42814fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42ce64700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa42814c0a0 0x7fa42814fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42ce64700 1 -- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa42806cd00 msgr2=0x7fa4280afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42ce64700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa42806cd00 0x7fa4280afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42ce64700 1 -- 172.21.15.64:0/137371346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009a20 con 0x7fa428154c20 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15a9f0 0x7f69cc15f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44358/0 (socket says 172.21.15.64:44358) 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69cbfff700 1 -- 172.21.15.64:0/710930846 learned_addr learned my addr 172.21.15.64:0/710930846 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc06b200 0x7f69cc151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d173a700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15adc0 0x7f69cc15fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd64de700 1 Processor -- start 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 -- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15adc0 msgr2=0x7f69cc15fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15adc0 0x7f69cc15fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 -- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15a9f0 msgr2=0x7f69cc15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15a9f0 0x7f69cc15f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 -- 172.21.15.64:0/710930846 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c00094f0 con 0x7f69cc06b200 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69cbfff700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc15a9f0 0x7f69cc15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 shutdown_connections 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f0147fff700 1 -- 172.21.15.64:0/4171765832 wait complete. 2023-09-27T23:13:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd64de700 1 -- start start 2023-09-27T23:13:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d0f39700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc06b200 0x7f69cc151040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00d890 tx=0x7f69bc008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdaceffd700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad4060a20 0x7fdad4062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a8155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a8151bf0 0x7ff7a816d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80ca3b0 con 0x7ff7a806cd00 2023-09-27T23:13:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81571f0 con 0x7ff7a8151bf0 2023-09-27T23:13:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81565e0 con 0x7ff7a8154c20 2023-09-27T23:13:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42ce64700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154c20 0x7fa4281656e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4240030c0 tx=0x7fa424019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a8155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a8155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44384/0 (socket says 172.21.15.64:44384) 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 -- 172.21.15.64:0/1514377888 learned_addr learned my addr 172.21.15.64:0/1514377888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7ad488700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a8151bf0 0x7ff7a816d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424018dc0 con 0x7fa428154c20 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424022410 con 0x7fa428154c20 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240227b0 con 0x7fa428154c20 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 -- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 msgr2=0x7ff7a80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7a7fff700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd64de700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdfd0161410 0x7fdfd00c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd64de700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd00c74d0 con 0x7fdfd006d650 2023-09-27T23:13:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd64de700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd0161c80 con 0x7fdfd006d280 2023-09-27T23:13:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd64de700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd00c88f0 con 0x7fdfd0161410 2023-09-27T23:13:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 Processor -- start 2023-09-27T23:13:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 -- start start 2023-09-27T23:13:59.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0125f0 con 0x7f69cc06b200 2023-09-27T23:13:59.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc012770 con 0x7f69cc06b200 2023-09-27T23:13:59.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc012b30 con 0x7f69cc06b200 2023-09-27T23:13:59.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 -- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a8151bf0 msgr2=0x7ff7a816d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a8151bf0 0x7ff7a816d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 -- 172.21.15.64:0/1514377888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a000be70 con 0x7ff7a806cd00 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7a7fff700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a8154c20 0x7ff7a80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdae007e070 con 0x7fdae8159160 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdae6ffd700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad4060a20 0x7fdad4062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdaceffd700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7fdad406a6f0 0x7fdad406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 --> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdad406d1f0 con 0x7fdad406a6f0 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdae007e360 con 0x7fdae8159160 2023-09-27T23:13:59.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdae67fc700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7fdad406a6f0 0x7fdad406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdae67fc700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7fdad406a6f0 0x7fdad406cb20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdae6ffd700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad4060a20 0x7fdad4062ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0030c0 tx=0x7fdadc019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f0147fff700 1 Processor -- start 2023-09-27T23:13:59.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d319d700 1 -- 172.21.15.64:0/710930846 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc0affd0 con 0x7f69cc06b200 2023-09-27T23:13:59.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfcf7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdfd006d650 0x7fdfd0160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34030/0 (socket says 172.21.15.64:34030) 2023-09-27T23:13:59.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfcf7fe700 1 -- 172.21.15.64:0/382796369 learned_addr learned my addr 172.21.15.64:0/382796369 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfd4a7b700 1 --2- 172.21.15.64:0/382796369 >> v2:172.21.15.123:3300/0 conn(0x7fdfd0161410 0x7fdfd00c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d319d700 1 -- 172.21.15.64:0/710930846 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc0ae9d0 con 0x7f69cc06b200 2023-09-27T23:13:59.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898689209 0 0) 0x7fdfd00c74d0 con 0x7fdfd006d650 2023-09-27T23:13:59.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb8003570 con 0x7fdfd006d650 2023-09-27T23:13:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42e667700 1 -- 172.21.15.64:0/137371346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42400a5d0 con 0x7fa428154c20 2023-09-27T23:13:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7fa42e667700 1 -- 172.21.15.64:0/137371346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42814f3a0 con 0x7fa428154c20 2023-09-27T23:13:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f0147fff700 1 -- start start 2023-09-27T23:13:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3dbc1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dbc158540 0x7f3dbc1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dbc1507f0 0x7f3dbc155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc150c90 con 0x7f3dbc158540 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc149d00 con 0x7f3dbc1507f0 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc20cb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc156980 con 0x7f3da80052e0 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc18ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dbc1507f0 0x7f3dbc155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc18ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dbc1507f0 0x7f3dbc155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60472/0 (socket says 172.21.15.64:60472) 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc18ca700 1 -- 172.21.15.64:0/3074856039 learned_addr learned my addr 172.21.15.64:0/3074856039 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f3dc08c8700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dbc158540 0x7f3dbc1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f014816d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01480cc5a0 0x7f014815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f01481484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01480b0590 con 0x7f0148158bc0 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014815ae60 con 0x7f01480cc5a0 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0148159cc0 con 0x7f01480cc150 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc18ca700 1 -- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 msgr2=0x7f3dbc1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80052e0 0x7f3dbc1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7f69d319d700 1 -- 172.21.15.64:0/710930846 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69cc04be40 con 0x7f69cc06b200 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc18ca700 1 -- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dbc158540 msgr2=0x7f3dbc1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dbc158540 0x7f3dbc1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc18ca700 1 -- 172.21.15.64:0/3074856039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc156c60 con 0x7f3dbc1507f0 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01480cc5a0 0x7f014815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01480cc5a0 0x7f014815a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60486/0 (socket says 172.21.15.64:60486) 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 -- 172.21.15.64:0/1537059261 learned_addr learned my addr 172.21.15.64:0/1537059261 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0146ffd700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f014816d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7acc87700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a8155f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff79800d890 tx=0x7ff798008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7980125f0 con 0x7ff7a806cd00 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798012750 con 0x7ff7a806cd00 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798012ad0 con 0x7ff7a806cd00 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a81568c0 con 0x7ff7a806cd00 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.318+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a8156e80 con 0x7ff7a806cd00 2023-09-27T23:13:59.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917642583 0 0) 0x7fdfd0161c80 con 0x7fdfd006d280 2023-09-27T23:13:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd00c74d0 con 0x7fdfd006d280 2023-09-27T23:13:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041951368 0 0) 0x7fdfb8003570 con 0x7fdfd006d650 2023-09-27T23:13:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfd4a7b700 1 -- 172.21.15.64:0/382796369 >> v1:172.21.15.123:6789/0 conn(0x7fdfd006d280 legacy=0x7fdfd00c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfd4a7b700 1 -- 172.21.15.64:0/382796369 >> v1:172.21.15.64:6789/0 conn(0x7fdfd006d650 legacy=0x7fdfd0160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfd4a7b700 1 -- 172.21.15.64:0/382796369 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd01619a0 con 0x7fdfd0161410 2023-09-27T23:13:59.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== osd.3 v2:172.21.15.64:6816/33401 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdad406d1f0 con 0x7fdad406a6f0 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 -- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 msgr2=0x7f014816d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01480cc150 0x7f014816d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 -- 172.21.15.64:0/1537059261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 msgr2=0x7f01481484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01477fe700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f01481484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f01481484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 -- 172.21.15.64:0/1537059261 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380094f0 con 0x7f01480cc5a0 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01477fe700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0148158bc0 0x7f01481484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:13:59.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f01467fc700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01480cc5a0 0x7f014815a920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f013c00ec80 tx=0x7f013c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7a804be40 con 0x7ff7a806cd00 2023-09-27T23:13:59.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc08c8700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dbc158540 0x7f3dbc1502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:59.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc18ca700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dbc1507f0 0x7f3dbc155d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0b1bb0 tx=0x7f3db8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f69bc013480 con 0x7f69cc06b200 2023-09-27T23:13:59.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00c850 con 0x7f01480cc5a0 2023-09-27T23:13:59.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01480af980 con 0x7f01480cc5a0 2023-09-27T23:13:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfd4a7b700 1 --2- 172.21.15.64:0/382796369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 0x7fdfd00c1d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e90 tx=0x7fdfc8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5f2d5587b7929f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc800e070 con 0x7fdfd0161410 2023-09-27T23:13:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc800b8c0 con 0x7fdfd0161410 2023-09-27T23:13:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfceffd700 1 -- 172.21.15.64:0/382796369 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40013f0 con 0x7fdfd0161410 2023-09-27T23:13:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01480aff40 con 0x7f01480cc5a0 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69c8ff9700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69b8060a20 0x7f69b8062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f69bc07dec0 con 0x7f69cc06b200 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69c8ff9700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7f69b806a6f0 0x7f69b806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 --> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69b806d1f0 con 0x7f69b806a6f0 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69bc07e290 con 0x7f69cc06b200 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c00c9b0 con 0x7f01480cc5a0 2023-09-27T23:13:59.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ff79801e460 con 0x7ff7a806cd00 2023-09-27T23:13:59.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a4ff9700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff794060a20 0x7ff794062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff79807d730 con 0x7ff7a806cd00 2023-09-27T23:13:59.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a4ff9700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7ff79406a670 0x7ff79406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00cd50 con 0x7f01480cc5a0 2023-09-27T23:13:59.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 --> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff79406d170 con 0x7ff79406a670 2023-09-27T23:13:59.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfd64de700 1 -- 172.21.15.64:0/382796369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 msgr2=0x7fdfd00c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fdfd64de700 1 --2- 172.21.15.64:0/382796369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 0x7fdfd00c1d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e90 tx=0x7fdfc8009780 comp rx=0 tx=0).stop 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8020070 con 0x7f3dbc1507f0 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8000ed0 con 0x7f3dbc1507f0 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8007300 con 0x7f3dbc1507f0 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc20cb700 1 -- 172.21.15.64:0/3074856039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc156f40 con 0x7f3dbc1507f0 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3dc20cb700 1 -- 172.21.15.64:0/3074856039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dbc1536d0 con 0x7f3dbc1507f0 2023-09-27T23:13:59.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3dbc04be40 con 0x7f3dbc1507f0 2023-09-27T23:13:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fa424020070 con 0x7fa428154c20 2023-09-27T23:13:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0128000fd0 con 0x7f01480cc5a0 2023-09-27T23:13:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7ad488700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7ff79406a670 0x7ff79406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff79804d740 con 0x7ff7a806cd00 2023-09-27T23:13:59.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7ff7a7fff700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff794060a20 0x7ff794062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7ff7ad488700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7ff79406a670 0x7ff79406caa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa41d7fa700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa40c060aa0 0x7fa40c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa42407e400 con 0x7fa428154c20 2023-09-27T23:13:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa42d665700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa40c060aa0 0x7fa40c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69d173a700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7f69b806a6f0 0x7f69b806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69d173a700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7f69b806a6f0 0x7f69b806cb20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.322+0000 7f69cbfff700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69b8060a20 0x7f69b8062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f69cbfff700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69b8060a20 0x7f69b8062ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69cc155550 tx=0x7f69c0016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== osd.2 v2:172.21.15.64:6824/33402 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f69b806d1f0 con 0x7f69b806a6f0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f013c013680 con 0x7f01480cc5a0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0134060980 0x7f0134062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f013c07d6f0 con 0x7f01480cc5a0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f0146ffd700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0134060980 0x7f0134062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f013406a460 0x7f013406c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 --> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f013406cf60 con 0x7f013406a460 2023-09-27T23:13:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f013c07fd30 con 0x7f01480cc5a0 2023-09-27T23:13:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fdfd64de700 1 -- 172.21.15.64:0/382796369 shutdown_connections 2023-09-27T23:13:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fdfd64de700 1 --2- 172.21.15.64:0/382796369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 0x7fdfd00c1d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fdfd64de700 1 -- 172.21.15.64:0/382796369 >> 172.21.15.64:0/382796369 conn(0x7fdfd006b670 msgr2=0x7fdfd006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f01477fe700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f013406a460 0x7f013406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f3db8007670 con 0x7f3dbc1507f0 2023-09-27T23:13:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f0146ffd700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0134060980 0x7f0134062e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f01380099b0 tx=0x7f0138016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa42d665700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa40c060aa0 0x7fa40c062f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4200099f0 tx=0x7fa42000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f01477fe700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f013406a460 0x7f013406c890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3db17fa700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3da0060aa0 0x7f3da0062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== osd.5 v2:172.21.15.123:6817/33567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff79406d170 con 0x7ff79406a670 2023-09-27T23:13:59.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3db807d950 con 0x7f3dbc1507f0 2023-09-27T23:13:59.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3dc10c9700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3da0060aa0 0x7f3da0062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3db17fa700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7f3da006a770 0x7f3da006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa412ffd700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7fa428154ff0 0x7fa428151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 --> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa42806d680 con 0x7fa428154ff0 2023-09-27T23:13:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 --> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3da006d270 con 0x7f3da006a770 2023-09-27T23:13:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db807dc80 con 0x7f3dbc1507f0 2023-09-27T23:13:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3dc08c8700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7f3da006a770 0x7f3da006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fdfd64de700 1 -- 172.21.15.64:0/382796369 shutdown_connections 2023-09-27T23:13:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fdfd64de700 1 -- 172.21.15.64:0/382796369 wait complete. 2023-09-27T23:13:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7fa42de66700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7fa428154ff0 0x7fa428151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7ff7a7fff700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff794060a20 0x7ff794062ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff79c006f00 tx=0x7ff79c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3dc10c9700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3da0060aa0 0x7f3da0062f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc04ccf0 tx=0x7f3dac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f3dc08c8700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7f3da006a770 0x7f3da006cba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.326+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== osd.4 v2:172.21.15.123:6808/33568 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f013406cf60 con 0x7f013406a460 2023-09-27T23:13:59.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== osd.1 v2:172.21.15.64:6808/33405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3da006d270 con 0x7f3da006a770 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 Processor -- start 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 -- start start 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd006d650 0x7fdfd013fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 0x7fdfd0140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfd0157d80 0x7fdfd0158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01408b0 con 0x7fdfd006d650 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00c50f0 con 0x7fdfd0161410 2023-09-27T23:13:59.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd64de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00c4200 con 0x7fdfd0157d80 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd006d650 0x7fdfd013fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd006d650 0x7fdfd013fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44420/0 (socket says 172.21.15.64:44420) 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 -- 172.21.15.64:0/4247139639 learned_addr learned my addr 172.21.15.64:0/4247139639 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfd4a7b700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfd0157d80 0x7fdfd0158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 -- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfd0157d80 msgr2=0x7fdfd0158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfd0157d80 0x7fdfd0158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 -- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 msgr2=0x7fdfd0140370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfd0161410 0x7fdfd0140370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 -- 172.21.15.64:0/4247139639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8009a20 con 0x7fdfd006d650 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.330+0000 7fdfcffff700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd006d650 0x7fdfd013fe30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00010e0 tx=0x7fdfc0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0010510 con 0x7fdfd006d650 2023-09-27T23:13:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fa42de66700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7fa428154ff0 0x7fa428151640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd00c44e0 con 0x7fdfd006d650 2023-09-27T23:13:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd00c4aa0 con 0x7fdfd006d650 2023-09-27T23:13:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0010670 con 0x7fdfd006d650 2023-09-27T23:13:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00109f0 con 0x7fdfd006d650 2023-09-27T23:13:59.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== osd.0 v2:172.21.15.64:6800/33400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa42806d680 con 0x7fa428154ff0 2023-09-27T23:13:59.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fdfc001b400 con 0x7fdfd006d650 2023-09-27T23:13:59.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfb7fff700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdfb8060e90 0x7fdfb8063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdfc007cea0 con 0x7fdfd006d650 2023-09-27T23:13:59.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fbeaad7f700 1 Processor -- start 2023-09-27T23:13:59.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.334+0000 7fbeaad7f700 1 -- start start 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbeaad7f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbea4159530 0x7fbea40c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbeaad7f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea40accb0 con 0x7fbea4158200 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbeaad7f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea40acf90 con 0x7fbea4159160 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbeaad7f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea4158d70 con 0x7fbea4159530 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbea4159530 0x7fbea40c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbea4159530 0x7fbea40c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44426/0 (socket says 172.21.15.64:44426) 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 -- 172.21.15.64:0/742122212 learned_addr learned my addr 172.21.15.64:0/742122212 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 -- 172.21.15.64:0/742122212 >> v1:172.21.15.123:6789/0 conn(0x7fbea4159160 legacy=0x7fbea40adcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 -- 172.21.15.64:0/742122212 >> v1:172.21.15.64:6789/0 conn(0x7fbea4158200 legacy=0x7fbea4158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 -- 172.21.15.64:0/742122212 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea40ad270 con 0x7fbea4159530 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.338+0000 7fbea957c700 1 --2- 172.21.15.64:0/742122212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4159530 0x7fbea40c5fd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009e10 tx=0x7fbe9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6a7451e65f70d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fbea8d7b700 1 -- 172.21.15.64:0/742122212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0092e0 con 0x7fbea4159530 2023-09-27T23:13:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fdfd64de700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7fdfbc001620 0x7fdfbc003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fbea8d7b700 1 -- 172.21.15.64:0/742122212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c00caf0 con 0x7fbea4159530 2023-09-27T23:13:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 --> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfbc006bc0 con 0x7fdfbc001620 2023-09-27T23:13:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fdfd4a7b700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7fdfbc001620 0x7fdfbc003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fbea8d7b700 1 -- 172.21.15.64:0/742122212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c011400 con 0x7fbea4159530 2023-09-27T23:13:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fbeaad7f700 1 -- 172.21.15.64:0/742122212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4159530 msgr2=0x7fbea40c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.342+0000 7fbeaad7f700 1 --2- 172.21.15.64:0/742122212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4159530 0x7fbea40c5fd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009e10 tx=0x7fbe9c009840 comp rx=0 tx=0).stop 2023-09-27T23:13:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fdfcf7fe700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdfb8060e90 0x7fdfb8063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fdfd4a7b700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7fdfbc001620 0x7fdfbc003ad0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fdfcf7fe700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdfb8060e90 0x7fdfb8063340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfd01708b0 tx=0x7fdfc40012c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fbeaad7f700 1 -- 172.21.15.64:0/742122212 shutdown_connections 2023-09-27T23:13:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== osd.7 v2:172.21.15.123:6816/33569 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdfbc006bc0 con 0x7fdfbc001620 2023-09-27T23:13:59.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69d319d700 1 -- 172.21.15.64:0/710930846 --> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69cc0b0160 con 0x7f69b806a6f0 2023-09-27T23:13:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fbeaad7f700 1 --2- 172.21.15.64:0/742122212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4159530 0x7fbea40c5fd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fbeaad7f700 1 -- 172.21.15.64:0/742122212 >> 172.21.15.64:0/742122212 conn(0x7fbea4067bb0 msgr2=0x7fbea40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fbeaad7f700 1 -- 172.21.15.64:0/742122212 shutdown_connections 2023-09-27T23:13:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fbeaad7f700 1 -- 172.21.15.64:0/742122212 wait complete. 2023-09-27T23:13:59.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69c8ff9700 1 -- 172.21.15.64:0/710930846 <== osd.2 v2:172.21.15.64:6824/33402 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f69cc0b0160 con 0x7f69b806a6f0 2023-09-27T23:13:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7f69b806a6f0 msgr2=0x7f69b806cb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7f69b806a6f0 0x7f69b806cb20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 --> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7a806d680 con 0x7ff79406a670 2023-09-27T23:13:59.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69b8060a20 msgr2=0x7f69b8062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 --> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdae804be40 con 0x7fdad406a6f0 2023-09-27T23:13:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69b8060a20 0x7f69b8062ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69cc155550 tx=0x7f69c0016000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc06b200 msgr2=0x7f69cc151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 --2- 172.21.15.64:0/710930846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc06b200 0x7f69cc151040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00d890 tx=0x7f69bc008040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69d173a700 1 -- 172.21.15.64:0/710930846 reap_dead start 2023-09-27T23:13:59.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 shutdown_connections 2023-09-27T23:13:59.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.346+0000 7ff7a4ff9700 1 -- 172.21.15.64:0/1514377888 <== osd.5 v2:172.21.15.123:6817/33567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff7a806d680 con 0x7ff79406a670 2023-09-27T23:13:59.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 >> 172.21.15.64:0/710930846 conn(0x7f69cc0670e0 msgr2=0x7f69cc0674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdaceffd700 1 -- 172.21.15.64:0/4047572009 <== osd.3 v2:172.21.15.64:6816/33401 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdae804be40 con 0x7fdad406a6f0 2023-09-27T23:13:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 Processor -- start 2023-09-27T23:13:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 -- start start 2023-09-27T23:13:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 shutdown_connections 2023-09-27T23:13:59.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7f69b27fc700 1 -- 172.21.15.64:0/710930846 wait complete. 2023-09-27T23:13:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4158200 0x7fbea40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7fdad406a6f0 msgr2=0x7fdad406cb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7fdad406a6f0 0x7fdad406cb20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad4060a20 msgr2=0x7fdad4062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad4060a20 0x7fdad4062ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0030c0 tx=0x7fdadc019000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbea4159160 0x7fbea4151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea4159530 0x7fbea41412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4152f90 con 0x7fbea4158200 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea41520e0 con 0x7fbea4159530 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaad7f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4152380 con 0x7fbea4159160 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4158200 0x7fbea40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4158200 0x7fbea40b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44434/0 (socket says 172.21.15.64:44434) 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 -- 172.21.15.64:0/1382794153 learned_addr learned my addr 172.21.15.64:0/1382794153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbeaa57e700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea4159530 0x7fbea41412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea957c700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbea4159160 0x7fbea4151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 -- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbea4159160 msgr2=0x7fbea4151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbea4159160 0x7fbea4151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 -- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea4159530 msgr2=0x7fbea41412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae8159160 msgr2=0x7fdae8150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea4159530 0x7fbea41412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 -- 172.21.15.64:0/1382794153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c00d040 con 0x7fbea4158200 2023-09-27T23:13:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea957c700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbea4159160 0x7fbea4151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 --2- 172.21.15.64:0/4047572009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae8159160 0x7fdae8150130 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdae000cca0 tx=0x7fdae0001a90 comp rx=0 tx=0).stop 2023-09-27T23:13:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae77fe700 1 -- 172.21.15.64:0/4047572009 reap_dead start 2023-09-27T23:13:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 shutdown_connections 2023-09-27T23:13:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 >> 172.21.15.64:0/4047572009 conn(0x7fdae8067bb0 msgr2=0x7fdae806cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 shutdown_connections 2023-09-27T23:13:59.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fbea9d7d700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4158200 0x7fbea40b1510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b600 tx=0x7fbe940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.350+0000 7fdae7fff700 1 -- 172.21.15.64:0/4047572009 wait complete. 2023-09-27T23:13:59.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 --> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0128002d30 con 0x7f013406a460 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7ff79406a670 msgr2=0x7ff79406caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7ff79406a670 0x7ff79406caa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff794060a20 msgr2=0x7ff794062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff794060a20 0x7ff794062ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff79c006f00 tx=0x7ff79c008040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 msgr2=0x7ff7a8155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 --2- 172.21.15.64:0/1514377888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a806cd00 0x7ff7a8155f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff79800d890 tx=0x7ff798008040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7ad488700 1 -- 172.21.15.64:0/1514377888 reap_dead start 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 shutdown_connections 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 >> 172.21.15.64:0/1514377888 conn(0x7ff7a8067bb0 msgr2=0x7ff7a8153680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 shutdown_connections 2023-09-27T23:13:59.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7ff7adc89700 1 -- 172.21.15.64:0/1514377888 wait complete. 2023-09-27T23:13:59.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7f012effd700 1 -- 172.21.15.64:0/1537059261 <== osd.4 v2:172.21.15.123:6808/33568 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0128002d30 con 0x7f013406a460 2023-09-27T23:13:59.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 --> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3dbc1570d0 con 0x7f3da006a770 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400f510 con 0x7fbea4158200 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9400f670 con 0x7fbea4158200 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea4152660 con 0x7fbea4158200 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea4152c20 con 0x7fbea4158200 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400fa30 con 0x7fbea4158200 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.354+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe84000fd0 con 0x7fbea4158200 2023-09-27T23:13:59.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3db17fa700 1 -- 172.21.15.64:0/3074856039 <== osd.1 v2:172.21.15.64:6808/33405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3dbc1570d0 con 0x7f3da006a770 2023-09-27T23:13:59.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f013406a460 msgr2=0x7f013406c890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f013406a460 0x7f013406c890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0134060980 msgr2=0x7f0134062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0134060980 0x7f0134062e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f01380099b0 tx=0x7f0138016000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbe9401a400 con 0x7fbea4158200 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01480cc5a0 msgr2=0x7f014815a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 --2- 172.21.15.64:0/1537059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01480cc5a0 0x7f014815a920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f013c00ec80 tx=0x7f013c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f01477fe700 1 -- 172.21.15.64:0/1537059261 reap_dead start 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 shutdown_connections 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 >> 172.21.15.64:0/1537059261 conn(0x7f0148067bb0 msgr2=0x7f01480cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 shutdown_connections 2023-09-27T23:13:59.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f0147fff700 1 -- 172.21.15.64:0/1537059261 wait complete. 2023-09-27T23:13:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbe99ffb700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe880609f0 0x7fbe88062ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbe9407cbe0 con 0x7fbea4158200 2023-09-27T23:13:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbea957c700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe880609f0 0x7fbe88062ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbe99ffb700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7fbe8806a6c0 0x7fbe8806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 --> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe8806d1c0 con 0x7fbe8806a6c0 2023-09-27T23:13:59.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7f3da006a770 msgr2=0x7f3da006cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7f3da006a770 0x7f3da006cba0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe9407f270 con 0x7fbea4158200 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3da0060aa0 msgr2=0x7f3da0062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3da0060aa0 0x7f3da0062f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc04ccf0 tx=0x7f3dac009040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dbc1507f0 msgr2=0x7f3dbc155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 --2- 172.21.15.64:0/3074856039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dbc1507f0 0x7f3dbc155d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0b1bb0 tx=0x7f3db8013000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3dc18ca700 1 -- 172.21.15.64:0/3074856039 reap_dead start 2023-09-27T23:13:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 shutdown_connections 2023-09-27T23:13:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 >> 172.21.15.64:0/3074856039 conn(0x7f3dbc067bb0 msgr2=0x7f3dbc0c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 shutdown_connections 2023-09-27T23:13:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbeaa57e700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7fbe8806a6c0 0x7fbe8806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7f3da6ffd700 1 -- 172.21.15.64:0/3074856039 wait complete. 2023-09-27T23:13:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbea957c700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe880609f0 0x7fbe88062ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009880 tx=0x7fbe9c018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.358+0000 7fbeaa57e700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7fbe8806a6c0 0x7fbe8806caf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.362+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== osd.6 v2:172.21.15.123:6800/33584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fbe8806d1c0 con 0x7fbe8806a6c0 2023-09-27T23:13:59.406 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:13:59.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-27T23:13:59.407 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:13:59.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-27T23:13:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 --> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfb0000f90 con 0x7fdfbc001620 2023-09-27T23:13:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 --> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4280636a0 con 0x7fa428154ff0 2023-09-27T23:13:59.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa41d7fa700 1 -- 172.21.15.64:0/137371346 <== osd.0 v2:172.21.15.64:6800/33400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa4280636a0 con 0x7fa428154ff0 2023-09-27T23:13:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7fa428154ff0 msgr2=0x7fa428151640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7fa428154ff0 0x7fa428151640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa40c060aa0 msgr2=0x7fa40c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa40c060aa0 0x7fa40c062f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4200099f0 tx=0x7fa42000b040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154c20 msgr2=0x7fa4281656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 --2- 172.21.15.64:0/137371346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa428154c20 0x7fa4281656e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4240030c0 tx=0x7fa424019000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa42de66700 1 -- 172.21.15.64:0/137371346 reap_dead start 2023-09-27T23:13:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfb7fff700 1 -- 172.21.15.64:0/4247139639 <== osd.7 v2:172.21.15.123:6816/33569 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdfb0000f90 con 0x7fdfbc001620 2023-09-27T23:13:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 shutdown_connections 2023-09-27T23:13:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 >> 172.21.15.64:0/137371346 conn(0x7fa428067bb0 msgr2=0x7fa42814deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 shutdown_connections 2023-09-27T23:13:59.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fa412ffd700 1 -- 172.21.15.64:0/137371346 wait complete. 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7fdfbc001620 msgr2=0x7fdfbc003ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7fdfbc001620 0x7fdfbc003ad0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdfb8060e90 msgr2=0x7fdfb8063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdfb8060e90 0x7fdfb8063340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfd01708b0 tx=0x7fdfc40012c0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd006d650 msgr2=0x7fdfd013fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 --2- 172.21.15.64:0/4247139639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfd006d650 0x7fdfd013fe30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00010e0 tx=0x7fdfc0007e10 comp rx=0 tx=0).stop 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd4a7b700 1 -- 172.21.15.64:0/4247139639 reap_dead start 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 shutdown_connections 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 >> 172.21.15.64:0/4247139639 conn(0x7fdfd006b670 msgr2=0x7fdfd00c3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 shutdown_connections 2023-09-27T23:13:59.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.370+0000 7fdfd64de700 1 -- 172.21.15.64:0/4247139639 wait complete. 2023-09-27T23:13:59.413 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:13:59.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-27T23:13:59.414 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:13:59.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-27T23:13:59.415 INFO:teuthology.orchestra.run.smithi064.stdout:60129542153 2023-09-27T23:13:59.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-27T23:13:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.382+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 --> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe84002d30 con 0x7fbe8806a6c0 2023-09-27T23:13:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbe99ffb700 1 -- 172.21.15.64:0/1382794153 <== osd.6 v2:172.21.15.123:6800/33584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe84002d30 con 0x7fbe8806a6c0 2023-09-27T23:13:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7fbe8806a6c0 msgr2=0x7fbe8806caf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7fbe8806a6c0 0x7fbe8806caf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe880609f0 msgr2=0x7fbe88062ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe880609f0 0x7fbe88062ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009880 tx=0x7fbe9c018000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4158200 msgr2=0x7fbea40b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 --2- 172.21.15.64:0/1382794153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbea4158200 0x7fbea40b1510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b600 tx=0x7fbe940091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaa57e700 1 -- 172.21.15.64:0/1382794153 reap_dead start 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 shutdown_connections 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 >> 172.21.15.64:0/1382794153 conn(0x7fbea4067bb0 msgr2=0x7fbea4153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 shutdown_connections 2023-09-27T23:13:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.386+0000 7fbeaad7f700 1 -- 172.21.15.64:0/1382794153 wait complete. 2023-09-27T23:13:59.418 INFO:teuthology.orchestra.run.smithi064.stdout:64424509449 2023-09-27T23:13:59.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-27T23:13:59.419 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:13:59.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-27T23:13:59.420 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:13:59.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-27T23:13:59.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 Processor -- start 2023-09-27T23:13:59.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 Processor -- start 2023-09-27T23:13:59.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 -- start start 2023-09-27T23:13:59.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23180c7d90 0x7f23180c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231813f880 0x7f23180ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231806dc00 con 0x7f231806c7c0 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231806e090 con 0x7f23180c7d90 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f231d42e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231806cff0 con 0x7f231813f880 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271f59e700 1 Processor -- start 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271f59e700 1 -- start start 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2720158bc0 0x7f2720158f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272006b570 con 0x7f27200cc150 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272006a960 con 0x7f27200c5380 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272006ac40 con 0x7f2720158bc0 2023-09-27T23:13:59.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271dd9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f27200cc150 0x7f27200c4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34056/0 (socket says 172.21.15.64:34056) 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271dd9b700 1 -- 172.21.15.64:0/3152895269 learned_addr learned my addr 172.21.15.64:0/3152895269 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f271e59c700 1 --2- 172.21.15.64:0/3152895269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2720158bc0 0x7f2720158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 -- start start 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff60155960 0x7eff6015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff600c7f00 0x7eff6006a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6006d990 con 0x7eff6006de20 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6015a570 con 0x7eff60155960 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff668fd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6006cd80 con 0x7eff600c7f00 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff650fa700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7eff6006de20 0x7eff600c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59670/0 (socket says 172.21.15.64:59670) 2023-09-27T23:13:59.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff650fa700 1 -- 172.21.15.64:0/2067963767 learned_addr learned my addr 172.21.15.64:0/2067963767 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff660fc700 1 --2- 172.21.15.64:0/2067963767 >> v2:172.21.15.123:3300/0 conn(0x7eff600c7f00 0x7eff6006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f23167fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f231806c7c0 0x7f231813f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34062/0 (socket says 172.21.15.64:34062) 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f23167fc700 1 -- 172.21.15.64:0/458089404 learned_addr learned my addr 172.21.15.64:0/458089404 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff660fc700 1 -- 172.21.15.64:0/2067963767 >> v2:172.21.15.64:3300/0 conn(0x7eff60155960 msgr2=0x7eff6015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff660fc700 1 --2- 172.21.15.64:0/2067963767 >> v2:172.21.15.64:3300/0 conn(0x7eff60155960 0x7eff6015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff660fc700 1 -- 172.21.15.64:0/2067963767 >> v1:172.21.15.123:6789/0 conn(0x7eff6006de20 legacy=0x7eff600c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7eff660fc700 1 -- 172.21.15.64:0/2067963767 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6006d060 con 0x7eff600c7f00 2023-09-27T23:13:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.598+0000 7f2316ffd700 1 --2- 172.21.15.64:0/458089404 >> v2:172.21.15.123:3300/0 conn(0x7f23180c7d90 0x7f23180c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f23177fe700 1 --2- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231813f880 0x7f23180ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271e59c700 1 -- 172.21.15.64:0/3152895269 >> v1:172.21.15.123:6789/0 conn(0x7f27200c5380 legacy=0x7f272006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271e59c700 1 -- 172.21.15.64:0/3152895269 >> v1:172.21.15.64:6789/0 conn(0x7f27200cc150 legacy=0x7f27200c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271e59c700 1 -- 172.21.15.64:0/3152895269 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272006af20 con 0x7f2720158bc0 2023-09-27T23:13:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff660fc700 1 --2- 172.21.15.64:0/2067963767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600c7f00 0x7eff6006a580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff600c5f50 tx=0x7eff5c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450b0c399c74428a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2316ffd700 1 -- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231813f880 msgr2=0x7f23180ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2316ffd700 1 --2- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231813f880 0x7f23180ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2316ffd700 1 -- 172.21.15.64:0/458089404 >> v1:172.21.15.64:6789/0 conn(0x7f231806c7c0 legacy=0x7f231813f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2316ffd700 1 -- 172.21.15.64:0/458089404 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231806d2d0 con 0x7f23180c7d90 2023-09-27T23:13:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271e59c700 1 --2- 172.21.15.64:0/3152895269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2720158bc0 0x7f2720158f90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2710009910 tx=0x7f27100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a26bd60ca1b6f8e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff648f9700 1 -- 172.21.15.64:0/2067963767 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00b6b0 con 0x7eff600c7f00 2023-09-27T23:13:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff648f9700 1 -- 172.21.15.64:0/2067963767 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c00b810 con 0x7eff600c7f00 2023-09-27T23:13:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff648f9700 1 -- 172.21.15.64:0/2067963767 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00bbd0 con 0x7eff600c7f00 2023-09-27T23:13:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2316ffd700 1 --2- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23180c7d90 0x7f23180c8160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23080098e0 tx=0x7f23080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa86ae62562ea316 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 -- 172.21.15.64:0/2067963767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600c7f00 msgr2=0x7eff6006a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 --2- 172.21.15.64:0/2067963767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600c7f00 0x7eff6006a580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff600c5f50 tx=0x7eff5c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271d59a700 1 -- 172.21.15.64:0/3152895269 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710009eb0 con 0x7f2720158bc0 2023-09-27T23:13:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271d59a700 1 -- 172.21.15.64:0/3152895269 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271000b760 con 0x7f2720158bc0 2023-09-27T23:13:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f271d59a700 1 -- 172.21.15.64:0/3152895269 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271000bb00 con 0x7f2720158bc0 2023-09-27T23:13:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2315ffb700 1 -- 172.21.15.64:0/458089404 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308011630 con 0x7f23180c7d90 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2315ffb700 1 -- 172.21.15.64:0/458089404 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308011790 con 0x7f23180c7d90 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f2315ffb700 1 -- 172.21.15.64:0/458089404 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308011b30 con 0x7f23180c7d90 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 -- 172.21.15.64:0/2067963767 shutdown_connections 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 --2- 172.21.15.64:0/2067963767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600c7f00 0x7eff6006a580 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 --2- 172.21.15.64:0/2067963767 >> v2:172.21.15.64:3300/0 conn(0x7eff60155960 0x7eff6015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 -- 172.21.15.64:0/2067963767 >> 172.21.15.64:0/2067963767 conn(0x7eff60067bb0 msgr2=0x7eff600cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 -- 172.21.15.64:0/2067963767 shutdown_connections 2023-09-27T23:13:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 Processor -- start 2023-09-27T23:13:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7eff668fd700 1 -- 172.21.15.64:0/2067963767 wait complete. 2023-09-27T23:13:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 -- start start 2023-09-27T23:13:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f231d42e700 1 -- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23180c7d90 msgr2=0x7f23180c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f231d42e700 1 --2- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23180c7d90 0x7f23180c8160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23080098e0 tx=0x7f23080091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f6207609700 1 Processor -- start 2023-09-27T23:13:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f6207609700 1 -- start start 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c78158bc0 0x7f0c78158f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0c780cc150 0x7f0c780c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c780cc5a0 0x7f0c780c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7806b570 con 0x7f0c780cc5a0 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7806a960 con 0x7f0c780cc150 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c7759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7806ac40 con 0x7f0c78158bc0 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7f0c76d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c780cc5a0 0x7f0c780c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c780cc5a0 0x7f0c780c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44452/0 (socket says 172.21.15.64:44452) 2023-09-27T23:13:59.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 -- 172.21.15.64:0/2582806292 learned_addr learned my addr 172.21.15.64:0/2582806292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 Processor -- start 2023-09-27T23:13:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c75d9b700 1 --2- 172.21.15.64:0/2582806292 >> v2:172.21.15.123:3300/0 conn(0x7f0c780cc150 0x7f0c780c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 -- 172.21.15.64:0/458089404 shutdown_connections 2023-09-27T23:13:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 --2- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231813f880 0x7f23180ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 --2- 172.21.15.64:0/458089404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23180c7d90 0x7f23180c8160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 -- 172.21.15.64:0/458089404 >> 172.21.15.64:0/458089404 conn(0x7f2318067b60 msgr2=0x7f23180ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7fd964e15700 1 Processor -- start 2023-09-27T23:13:59.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.602+0000 7fd964e15700 1 -- start start 2023-09-27T23:13:59.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 -- start start 2023-09-27T23:13:59.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 -- 172.21.15.64:0/2582806292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c78158bc0 msgr2=0x7f0c78158f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:13:59.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 --2- 172.21.15.64:0/2582806292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c78158bc0 0x7f0c78158f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 -- 172.21.15.64:0/2582806292 >> v2:172.21.15.123:3300/0 conn(0x7f0c780cc150 msgr2=0x7f0c780c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 --2- 172.21.15.64:0/2582806292 >> v2:172.21.15.123:3300/0 conn(0x7f0c780cc150 0x7f0c780c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 -- 172.21.15.64:0/2582806292 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7806af20 con 0x7f0c780cc5a0 2023-09-27T23:13:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 Processor -- start 2023-09-27T23:13:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 -- start start 2023-09-27T23:13:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c76d9d700 1 --2- 172.21.15.64:0/2582806292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c780c51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c78063a20 tx=0x7f0c70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7049dbd2c0d83f6b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 -- 172.21.15.64:0/458089404 shutdown_connections 2023-09-27T23:13:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f271f59e700 1 -- 172.21.15.64:0/3152895269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2720158bc0 msgr2=0x7f2720158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f271f59e700 1 --2- 172.21.15.64:0/3152895269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2720158bc0 0x7f2720158f90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2710009910 tx=0x7f27100091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7559a700 1 -- 172.21.15.64:0/2582806292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000e070 con 0x7f0c780cc5a0 2023-09-27T23:13:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f012ee3f700 1 Processor -- start 2023-09-27T23:13:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f6207609700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62000c7d90 0x7f62000c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f6207609700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f620006c7c0 0x7f620013f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f6207609700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620006dc00 con 0x7f620013f6b0 2023-09-27T23:13:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f6207609700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620013faf0 con 0x7f62000c7d90 2023-09-27T23:13:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f6207609700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620006e090 con 0x7f620006c7c0 2023-09-27T23:13:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f012ee3f700 1 -- start start 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd964e15700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd96006ba80 0x7fd96016a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd964e15700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd96016acb0 0x7fd9600c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd964e15700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd9600c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd964e15700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96006e210 con 0x7fd96006ba80 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd964e15700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96006b5f0 con 0x7fd96016acb0 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd964e15700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96006be50 con 0x7fd9600c8950 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd96006ba80 0x7fd96016a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd96006ba80 0x7fd96016a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44456/0 (socket says 172.21.15.64:44456) 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 -- 172.21.15.64:0/1172410938 learned_addr learned my addr 172.21.15.64:0/1172410938 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95dd9b700 1 --2- 172.21.15.64:0/1172410938 >> v2:172.21.15.123:3300/0 conn(0x7fd96016acb0 0x7fd9600c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 -- 172.21.15.64:0/458089404 wait complete. 2023-09-27T23:13:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7559a700 1 -- 172.21.15.64:0/2582806292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7000ba20 con 0x7f0c780cc5a0 2023-09-27T23:13:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7559a700 1 -- 172.21.15.64:0/2582806292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700092e0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62000c7d90 0x7f62000c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62000c7d90 0x7f62000c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44468/0 (socket says 172.21.15.64:44468) 2023-09-27T23:13:59.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 -- 172.21.15.64:0/896660205 learned_addr learned my addr 172.21.15.64:0/896660205 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff6006de20 0x7eff600b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c7f00 0x7eff60146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff60155960 0x7eff600c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95ed9d700 1 --2- 172.21.15.64:0/1172410938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd9600c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 -- 172.21.15.64:0/1172410938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 msgr2=0x7fd9600c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 --2- 172.21.15.64:0/1172410938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd9600c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 -- 172.21.15.64:0/1172410938 >> v2:172.21.15.123:3300/0 conn(0x7fd96016acb0 msgr2=0x7fd9600c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 --2- 172.21.15.64:0/1172410938 >> v2:172.21.15.123:3300/0 conn(0x7fd96016acb0 0x7fd9600c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 -- 172.21.15.64:0/1172410938 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96006dc50 con 0x7fd96006ba80 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3634068670 0x7f363406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f363406dd10 0x7f36340ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363406e170 con 0x7f36340ab8e0 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36340ad6d0 con 0x7f3634068670 2023-09-27T23:13:59.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36340abd20 con 0x7f363406dd10 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600afed0 con 0x7eff600c7f00 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600aee30 con 0x7eff6006de20 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff658fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff6006de20 0x7eff600b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff658fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff6006de20 0x7eff600b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60546/0 (socket says 172.21.15.64:60546) 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c7f00 0x7eff60146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff658fb700 1 -- 172.21.15.64:0/300714457 learned_addr learned my addr 172.21.15.64:0/300714457 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 -- 172.21.15.64:0/300714457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600af110 con 0x7eff60155960 2023-09-27T23:13:59.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f6204ba4700 1 --2- 172.21.15.64:0/896660205 >> v2:172.21.15.123:3300/0 conn(0x7f620006c7c0 0x7f620013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 -- 172.21.15.64:0/896660205 >> v2:172.21.15.123:3300/0 conn(0x7f620006c7c0 msgr2=0x7f620013f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 --2- 172.21.15.64:0/896660205 >> v2:172.21.15.123:3300/0 conn(0x7f620006c7c0 0x7f620013f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 -- 172.21.15.64:0/896660205 >> v1:172.21.15.64:6789/0 conn(0x7f620013f6b0 legacy=0x7f620006a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 -- 172.21.15.64:0/896660205 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620006cff0 con 0x7f62000c7d90 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f271f59e700 1 -- 172.21.15.64:0/3152895269 shutdown_connections 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f271f59e700 1 --2- 172.21.15.64:0/3152895269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2720158bc0 0x7f2720158f90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f271f59e700 1 -- 172.21.15.64:0/3152895269 >> 172.21.15.64:0/3152895269 conn(0x7f2720067bb0 msgr2=0x7f27200cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff660fc700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff60155960 0x7eff600c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 -- 172.21.15.64:0/2582806292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 msgr2=0x7f0c780c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 --2- 172.21.15.64:0/2582806292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c780c51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c78063a20 tx=0x7f0c70009840 comp rx=0 tx=0).stop 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 -- 172.21.15.64:0/2582806292 shutdown_connections 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 --2- 172.21.15.64:0/2582806292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c780c51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 --2- 172.21.15.64:0/2582806292 >> v2:172.21.15.123:3300/0 conn(0x7f0c780cc150 0x7f0c780c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 --2- 172.21.15.64:0/2582806292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c78158bc0 0x7f0c78158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 -- 172.21.15.64:0/2582806292 >> 172.21.15.64:0/2582806292 conn(0x7f0c78067bb0 msgr2=0x7f0c780cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 Processor -- start 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 -- start start 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f3631d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f363406dd10 0x7f36340ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f3631d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f363406dd10 0x7f36340ab3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60552/0 (socket says 172.21.15.64:60552) 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f3631d9b700 1 -- 172.21.15.64:0/361402387 learned_addr learned my addr 172.21.15.64:0/361402387 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363259c700 1 --2- 172.21.15.64:0/361402387 >> v2:172.21.15.64:3300/0 conn(0x7f3634068670 0x7f363406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f62053a5700 1 --2- 172.21.15.64:0/896660205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62000c7d90 0x7f62000c8160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62000ae4a0 tx=0x7f61fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93e5a88b45db5ee server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 -- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff60155960 msgr2=0x7eff600c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff60155960 0x7eff600c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 -- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff6006de20 msgr2=0x7eff600b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95e59c700 1 --2- 172.21.15.64:0/1172410938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd96006ba80 0x7fd96016a770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd9600ae760 tx=0x7fd958009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d280f8badb2aaa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff6006de20 0x7eff600b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 -- 172.21.15.64:0/300714457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c009420 con 0x7eff600c7f00 2023-09-27T23:13:59.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f0c7759e700 1 -- 172.21.15.64:0/2582806292 shutdown_connections 2023-09-27T23:13:59.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231806c7c0 0x7f23180ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f231d42e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 0x7f2318168b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363259c700 1 -- 172.21.15.64:0/361402387 >> v2:172.21.15.123:3300/0 conn(0x7f363406dd10 msgr2=0x7f36340ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363259c700 1 --2- 172.21.15.64:0/361402387 >> v2:172.21.15.123:3300/0 conn(0x7f363406dd10 0x7f36340ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363259c700 1 -- 172.21.15.64:0/361402387 >> v1:172.21.15.123:6789/0 conn(0x7f36340ab8e0 legacy=0x7f3634154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f363259c700 1 -- 172.21.15.64:0/361402387 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340abfd0 con 0x7f3634068670 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f61f7fff700 1 -- 172.21.15.64:0/896660205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00b860 con 0x7f62000c7d90 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7f61f7fff700 1 -- 172.21.15.64:0/896660205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc00b9c0 con 0x7f62000c7d90 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012ee3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280c7df0 0x7f01280c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012ee3f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01280afbf0 0x7f01280b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff660fc700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff60155960 0x7eff600c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff650fa700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c7f00 0x7eff60146310 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff50015ca0 tx=0x7eff50001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000f050 con 0x7eff600c7f00 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff50013a30 con 0x7eff600c7f00 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 -- 172.21.15.64:0/300714457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff600af3f0 con 0x7eff600c7f00 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff668fd700 1 -- 172.21.15.64:0/300714457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff600af9b0 con 0x7eff600c7f00 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50013e10 con 0x7eff600c7f00 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95d59a700 1 -- 172.21.15.64:0/1172410938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95800e070 con 0x7fd96006ba80 2023-09-27T23:13:59.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95d59a700 1 -- 172.21.15.64:0/1172410938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95800ba50 con 0x7fd96006ba80 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.606+0000 7fd95d59a700 1 -- 172.21.15.64:0/1172410938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9580092e0 con 0x7fd96006ba80 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f231d42e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f231813f880 0x7f23180c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f231d42e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180afcd0 con 0x7f23180c7d90 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f231d42e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318169080 con 0x7f231813f880 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f231d42e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23180af050 con 0x7f231806c7c0 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 -- 172.21.15.64:0/2582806292 wait complete. 2023-09-27T23:13:59.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 0x7f2318168b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f2316ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231806c7c0 0x7f23180ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 0x7f2318168b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44502/0 (socket says 172.21.15.64:44502) 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 -- 172.21.15.64:0/3421007839 learned_addr learned my addr 172.21.15.64:0/3421007839 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01280afbf0 0x7f01280b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01280afbf0 0x7f01280b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44514/0 (socket says 172.21.15.64:44514) 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 -- 172.21.15.64:0/137795838 learned_addr learned my addr 172.21.15.64:0/137795838 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0128159e20 con 0x7f01281599b0 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012806ae50 con 0x7f01280afbf0 2023-09-27T23:13:59.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012806dbb0 con 0x7f01280c7df0 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 -- 172.21.15.64:0/1172410938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd96006ba80 msgr2=0x7fd96016a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 --2- 172.21.15.64:0/1172410938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd96006ba80 0x7fd96016a770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd9600ae760 tx=0x7fd958009840 comp rx=0 tx=0).stop 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 -- 172.21.15.64:0/1172410938 shutdown_connections 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 --2- 172.21.15.64:0/1172410938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd9600c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 --2- 172.21.15.64:0/1172410938 >> v2:172.21.15.123:3300/0 conn(0x7fd96016acb0 0x7fd9600c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 --2- 172.21.15.64:0/1172410938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd96006ba80 0x7fd96016a770 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd9600ae760 tx=0x7fd958009840 comp rx=0 tx=0).stop 2023-09-27T23:13:59.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 -- 172.21.15.64:0/1172410938 >> 172.21.15.64:0/1172410938 conn(0x7fd960067bb0 msgr2=0x7fd9600ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 -- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231806c7c0 msgr2=0x7f23180ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231806c7c0 0x7f23180ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 -- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f231813f880 msgr2=0x7f23180c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f231813f880 0x7f23180c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 -- 172.21.15.64:0/3421007839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308009420 con 0x7f23180c7d90 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f61f7fff700 1 -- 172.21.15.64:0/896660205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00bd60 con 0x7f62000c7d90 2023-09-27T23:13:59.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f363259c700 1 --2- 172.21.15.64:0/361402387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634068670 0x7f363406d7d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f363406f7e0 tx=0x7f36240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8382d569f73bc5e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 -- 172.21.15.64:0/137795838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280c7df0 msgr2=0x7f01280c81c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:13:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 --2- 172.21.15.64:0/137795838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280c7df0 0x7f01280c81c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 -- 172.21.15.64:0/137795838 >> v1:172.21.15.123:6789/0 conn(0x7f01281599b0 legacy=0x7f012806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:13:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 -- 172.21.15.64:0/137795838 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280b0760 con 0x7f01280afbf0 2023-09-27T23:13:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f271f59e700 1 -- 172.21.15.64:0/3152895269 shutdown_connections 2023-09-27T23:13:59.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 Processor -- start 2023-09-27T23:13:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7eff668fd700 1 -- 172.21.15.64:0/300714457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff600afbd0 con 0x7eff600c7f00 2023-09-27T23:13:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7eff50019690 con 0x7eff600c7f00 2023-09-27T23:13:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 -- start start 2023-09-27T23:13:59.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f23167fc700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 0x7f2318168b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f231000ce40 tx=0x7f2310001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7eff55ffb700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff44060970 0x7eff44062e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7eff50085970 con 0x7eff600c7f00 2023-09-27T23:13:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f6207609700 1 -- 172.21.15.64:0/896660205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62000c7d90 msgr2=0x7f62000c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f6207609700 1 --2- 172.21.15.64:0/896660205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62000c7d90 0x7f62000c8160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62000ae4a0 tx=0x7f61fc009250 comp rx=0 tx=0).stop 2023-09-27T23:13:59.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7eff658fb700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff44060970 0x7eff44062e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f271f59e700 1 -- 172.21.15.64:0/3152895269 wait complete. 2023-09-27T23:13:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f271f59e700 1 Processor -- start 2023-09-27T23:13:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f271f59e700 1 -- start start 2023-09-27T23:13:59.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f012d3dc700 1 --2- 172.21.15.64:0/137795838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280afbf0 0x7f01280b0070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0120009f20 tx=0x7f0120009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3d48018590fa25 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c780cc150 0x7f0c78140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c781554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78158bc0 0x7f0c781594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c781567e0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78155bd0 con 0x7f0c78158bc0 2023-09-27T23:13:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78155eb0 con 0x7f0c780cc150 2023-09-27T23:13:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7eff658fb700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff44060970 0x7eff44062e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff58009ee0 tx=0x7eff58009840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 -- 172.21.15.64:0/1172410938 shutdown_connections 2023-09-27T23:13:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7fd964e15700 1 -- 172.21.15.64:0/1172410938 wait complete. 2023-09-27T23:13:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f6207609700 1 -- 172.21.15.64:0/896660205 shutdown_connections 2023-09-27T23:13:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f6207609700 1 --2- 172.21.15.64:0/896660205 >> v2:172.21.15.123:3300/0 conn(0x7f620006c7c0 0x7f620013f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f6207609700 1 --2- 172.21.15.64:0/896660205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62000c7d90 0x7f62000c8160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f6207609700 1 -- 172.21.15.64:0/896660205 >> 172.21.15.64:0/896660205 conn(0x7f6200067b60 msgr2=0x7f62000ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c781554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c781554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44528/0 (socket says 172.21.15.64:44528) 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 -- 172.21.15.64:0/1933566668 learned_addr learned my addr 172.21.15.64:0/1933566668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c76d9d700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78158bc0 0x7f0c781594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c7659c700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c780cc150 0x7f0c78140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 -- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c780cc150 msgr2=0x7f0c78140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c780cc150 0x7f0c78140f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 -- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78158bc0 msgr2=0x7f0c781594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78158bc0 0x7f0c781594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 -- 172.21.15.64:0/1933566668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70009ae0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.610+0000 7f0c75d9b700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c781554d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00ec80 tx=0x7f0c6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27200c5380 0x7f27201574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27200cc150 0x7f27201762a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2720158bc0 0x7f27201767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272015aed0 con 0x7f2720158bc0 2023-09-27T23:13:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272015bcf0 con 0x7f27200cc150 2023-09-27T23:13:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272015b0e0 con 0x7f27200c5380 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27200c5380 0x7f27201574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2720158bc0 0x7f27201767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27200c5380 0x7f27201574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:52824/0 (socket says 172.21.15.64:52824) 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27200cc150 0x7f27201762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271e59c700 1 -- 172.21.15.64:0/2723615757 learned_addr learned my addr 172.21.15.64:0/2723615757 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 -- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27200c5380 msgr2=0x7f27201574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27200c5380 0x7f27201574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 -- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27200cc150 msgr2=0x7f27201762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27200cc150 0x7f27201762a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 -- 172.21.15.64:0/2723615757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2710009420 con 0x7f2720158bc0 2023-09-27T23:13:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f271ed9d700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2720158bc0 0x7f27201767e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2720068060 tx=0x7f27180016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 Processor -- start 2023-09-27T23:13:59.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 -- 172.21.15.64:0/896660205 shutdown_connections 2023-09-27T23:13:59.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 -- 172.21.15.64:0/896660205 wait complete. 2023-09-27T23:13:59.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 -- start start 2023-09-27T23:13:59.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd960159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd96016acb0 0x7fd960159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 0x7fd960182240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd960167190 con 0x7fd9600c8350 2023-09-27T23:13:59.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600aa970 con 0x7fd96016acb0 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd964e15700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600aac50 con 0x7fd9600c8950 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 0x7fd960182240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 0x7fd960182240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44554/0 (socket says 172.21.15.64:44554) 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd960159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 -- 172.21.15.64:0/2664624733 learned_addr learned my addr 172.21.15.64:0/2664624733 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95dd9b700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd96016acb0 0x7fd960159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 Processor -- start 2023-09-27T23:13:59.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 -- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 msgr2=0x7fd960159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd960159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 -- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd96016acb0 msgr2=0x7fd960159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd96016acb0 0x7fd960159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 -- 172.21.15.64:0/2664624733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd958009ae0 con 0x7fd9600c8350 2023-09-27T23:13:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 -- start start 2023-09-27T23:13:59.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7fd95ed9d700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 0x7fd960182240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d890 tx=0x7fd950008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f620006c7c0 0x7f620013b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62000c7d90 0x7f6200159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 0x7f6200159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620015a410 con 0x7f620013f6b0 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200159e50 con 0x7f620006c7c0 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6207609700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620015a130 con 0x7f62000c7d90 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 0x7f6200159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 0x7f6200159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44560/0 (socket says 172.21.15.64:44560) 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 -- 172.21.15.64:0/1117495069 learned_addr learned my addr 172.21.15.64:0/1117495069 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f62053a5700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f620006c7c0 0x7f620013b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6204ba4700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62000c7d90 0x7f6200159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 -- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62000c7d90 msgr2=0x7f6200159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00c850 con 0x7f0c780cc5a0 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000ac00 con 0x7f23180c7d90 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23180af330 con 0x7f23180c7d90 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23180af8f0 con 0x7f23180c7d90 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231000ad80 con 0x7f23180c7d90 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310007340 con 0x7f23180c7d90 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363159a700 1 -- 172.21.15.64:0/361402387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400b6b0 con 0x7f3634068670 2023-09-27T23:13:59.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363159a700 1 -- 172.21.15.64:0/361402387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362400b810 con 0x7f3634068670 2023-09-27T23:13:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62000c7d90 0x7f6200159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f01277fe700 1 -- 172.21.15.64:0/137795838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012000e070 con 0x7f01280afbf0 2023-09-27T23:13:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f01277fe700 1 -- 172.21.15.64:0/137795838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012000ba40 con 0x7f01280afbf0 2023-09-27T23:13:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff50055900 con 0x7eff600c7f00 2023-09-27T23:13:59.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500125f0 con 0x7fd9600c8350 2023-09-27T23:13:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f01277fe700 1 -- 172.21.15.64:0/137795838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200092e0 con 0x7f01280afbf0 2023-09-27T23:13:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c78156190 con 0x7f0c780cc5a0 2023-09-27T23:13:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c78159a30 con 0x7f0c780cc5a0 2023-09-27T23:13:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c00c9b0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00cd50 con 0x7f0c780cc5a0 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718012630 con 0x7f2720158bc0 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272015b3c0 con 0x7f2720158bc0 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272015b980 con 0x7f2720158bc0 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 -- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f620006c7c0 msgr2=0x7f620013b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f620006c7c0 0x7f620013b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.614+0000 7f6205ba6700 1 -- 172.21.15.64:0/1117495069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0094f0 con 0x7f620013f6b0 2023-09-27T23:13:59.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f6205ba6700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 0x7f6200159860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c4c0 tx=0x7f61f0007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9600aaf30 con 0x7fd9600c8350 2023-09-27T23:13:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9600aa5d0 con 0x7fd9600c8350 2023-09-27T23:13:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f80052b0 con 0x7f23180c7d90 2023-09-27T23:13:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27180127b0 con 0x7f2720158bc0 2023-09-27T23:13:59.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950012960 con 0x7fd9600c8350 2023-09-27T23:13:59.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950012ad0 con 0x7fd9600c8350 2023-09-27T23:13:59.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000ce30 con 0x7f620013f6b0 2023-09-27T23:13:59.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363159a700 1 -- 172.21.15.64:0/361402387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400bbb0 con 0x7f3634068670 2023-09-27T23:13:59.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62001670b0 con 0x7f620013f6b0 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200150c90 con 0x7f620013f6b0 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 -- 172.21.15.64:0/361402387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634068670 msgr2=0x7f363406d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 --2- 172.21.15.64:0/361402387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634068670 0x7f363406d7d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f363406f7e0 tx=0x7f36240091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 -- 172.21.15.64:0/361402387 shutdown_connections 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 --2- 172.21.15.64:0/361402387 >> v2:172.21.15.123:3300/0 conn(0x7f363406dd10 0x7f36340ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 --2- 172.21.15.64:0/361402387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634068670 0x7f363406d7d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 -- 172.21.15.64:0/361402387 >> 172.21.15.64:0/361402387 conn(0x7f36340cb990 msgr2=0x7f36340cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c0052b0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f2310007730 con 0x7f23180c7d90 2023-09-27T23:13:59.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f22feffd700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2304060950 0x7f2304062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f231007d7a0 con 0x7f23180c7d90 2023-09-27T23:13:59.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f363359e700 1 -- 172.21.15.64:0/361402387 shutdown_connections 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718012b70 con 0x7f2720158bc0 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27040052b0 con 0x7f2720158bc0 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00117a0 con 0x7f620013f6b0 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6200166f10 con 0x7f620013f6b0 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0011910 con 0x7f620013f6b0 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280afbf0 msgr2=0x7f01280b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 --2- 172.21.15.64:0/137795838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280afbf0 0x7f01280b0070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0120009f20 tx=0x7f0120009840 comp rx=0 tx=0).stop 2023-09-27T23:13:59.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 shutdown_connections 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 --2- 172.21.15.64:0/137795838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280afbf0 0x7f01280b0070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 --2- 172.21.15.64:0/137795838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280c7df0 0x7f01280c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 >> 172.21.15.64:0/137795838 conn(0x7f0128067bb0 msgr2=0x7f01280ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 shutdown_connections 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f012ee3f700 1 -- 172.21.15.64:0/137795838 wait complete. 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9440052b0 con 0x7fd9600c8350 2023-09-27T23:13:59.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.618+0000 7f2316ffd700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2304060950 0x7f2304062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fd950012c30 con 0x7fd9600c8350 2023-09-27T23:13:59.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7fd94e7fc700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd940060950 0x7fd940062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd95007d4c0 con 0x7fd9600c8350 2023-09-27T23:13:59.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 Processor -- start 2023-09-27T23:13:59.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f0c6c013680 con 0x7f0c780cc5a0 2023-09-27T23:13:59.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 -- start start 2023-09-27T23:13:59.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7fd95e59c700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd940060950 0x7fd940062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f61f0011cc0 con 0x7f620013f6b0 2023-09-27T23:13:59.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 -- 172.21.15.64:0/361402387 wait complete. 2023-09-27T23:13:59.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 Processor -- start 2023-09-27T23:13:59.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0c667fc700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c58060950 0x7f0c58062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 -- start start 2023-09-27T23:13:59.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7fd95e59c700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd940060950 0x7fd940062e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd96006a1f0 tx=0x7fd958018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f61f57fa700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f61e4060950 0x7f61e4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0c7659c700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c58060950 0x7f0c58062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f2316ffd700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2304060950 0x7f2304062e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23080093f0 tx=0x7f2308016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280afbf0 0x7f01281561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 0x7f01280cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01281599b0 0x7f0128175b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280c9f00 con 0x7f01280c7df0 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280c8b10 con 0x7f01281599b0 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012ee3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0128156720 con 0x7f01280afbf0 2023-09-27T23:13:59.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f62053a5700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f61e4060950 0x7f61e4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f271801a3a0 con 0x7f2720158bc0 2023-09-27T23:13:59.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f270e7fc700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27000609a0 0x7f2700062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f271807d260 con 0x7f2720158bc0 2023-09-27T23:13:59.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 0x7f01280cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 0x7f01280cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44576/0 (socket says 172.21.15.64:44576) 2023-09-27T23:13:59.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 -- 172.21.15.64:0/28070582 learned_addr learned my addr 172.21.15.64:0/28070582 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012d3dc700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01281599b0 0x7f0128175b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f271e59c700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27000609a0 0x7f2700062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012cbdb700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280afbf0 0x7f01281561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3634068670 0x7f3634175480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 0x7f363413b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36340ab8e0 0x7f363413b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363413da20 con 0x7f363406dd10 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634141450 con 0x7f36340ab8e0 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3634140880 con 0x7f3634068670 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 -- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280afbf0 msgr2=0x7f01281561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280afbf0 0x7f01281561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 -- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01281599b0 msgr2=0x7f0128175b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01281599b0 0x7f0128175b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 -- 172.21.15.64:0/28070582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120009ae0 con 0x7f01280c7df0 2023-09-27T23:13:59.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0c7659c700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c58060950 0x7f0c58062e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006fd0 tx=0x7f0c68009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f012d3dc700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01281599b0 0x7f0128175b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:13:59.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 0x7f363413b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f363259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3634068670 0x7f3634175480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 0x7f363413b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44584/0 (socket says 172.21.15.64:44584) 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 -- 172.21.15.64:0/1106245850 learned_addr learned my addr 172.21.15.64:0/1106245850 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3632d9d700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36340ab8e0 0x7f363413b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f0127fff700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 0x7f01280cab40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f011800b610 tx=0x7f01180090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 -- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3634068670 msgr2=0x7f3634175480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3634068670 0x7f3634175480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f62053a5700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f61e4060950 0x7f61e4062e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f62000ae4a0 tx=0x7f61fc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 -- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36340ab8e0 msgr2=0x7f363413b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36340ab8e0 0x7f363413b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f3631d9b700 1 -- 172.21.15.64:0/1106245850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624009420 con 0x7f363406dd10 2023-09-27T23:13:59.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.622+0000 7f271e59c700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27000609a0 0x7f2700062e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2710009860 tx=0x7f2710017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.626+0000 7f3631d9b700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 0x7f363413b460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f362c007eb0 tx=0x7f362c009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.626+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0c6c051ed0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.626+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f231004d720 con 0x7f23180c7d90 2023-09-27T23:13:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.630+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f271804d1d0 con 0x7f2720158bc0 2023-09-27T23:13:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f61f0053bc0 con 0x7f620013f6b0 2023-09-27T23:13:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011800f510 con 0x7f01280c7df0 2023-09-27T23:13:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00dab0 con 0x7f363406dd10 2023-09-27T23:13:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0128157f50 con 0x7f01280c7df0 2023-09-27T23:13:59.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01281587f0 con 0x7f01280c7df0 2023-09-27T23:13:59.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd95004d500 con 0x7fd9600c8350 2023-09-27T23:13:59.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3634140b20 con 0x7f363406dd10 2023-09-27T23:13:59.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.638+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36341410e0 con 0x7f363406dd10 2023-09-27T23:13:59.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.642+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128071120 con 0x7f01280c7df0 2023-09-27T23:13:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.642+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363404be40 con 0x7f363406dd10 2023-09-27T23:13:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.646+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0c6c04dad0 con 0x7f0c780cc5a0 2023-09-27T23:13:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.654+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011800f880 con 0x7f01280c7df0 2023-09-27T23:13:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.654+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011800f9f0 con 0x7f01280c7df0 2023-09-27T23:13:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.654+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f61f004cd20 con 0x7f620013f6b0 2023-09-27T23:13:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.654+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00f040 con 0x7f363406dd10 2023-09-27T23:13:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.654+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00de20 con 0x7f363406dd10 2023-09-27T23:13:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f011800fb50 con 0x7f01280c7df0 2023-09-27T23:13:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f0124ff9700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f01140609d0 0x7f0114062e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f011807d500 con 0x7f01280c7df0 2023-09-27T23:13:59.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f012cbdb700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f01140609d0 0x7f0114062e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f362c01e400 con 0x7f363406dd10 2023-09-27T23:13:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f36227fc700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3614060950 0x7f3614062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:13:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f362c07e0d0 con 0x7f363406dd10 2023-09-27T23:13:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f363259c700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3614060950 0x7f3614062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:13:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f012cbdb700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f01140609d0 0x7f0114062e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f011c006fd0 tx=0x7f011c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.658+0000 7f363259c700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3614060950 0x7f3614062e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36240093f0 tx=0x7f3624017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:13:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.666+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f011804d4d0 con 0x7f01280c7df0 2023-09-27T23:13:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.674+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f362c04e060 con 0x7f363406dd10 2023-09-27T23:13:59.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.918+0000 7eff668fd700 1 -- 172.21.15.64:0/300714457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7eff6004be40 con 0x7eff600c7f00 2023-09-27T23:13:59.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.918+0000 7eff55ffb700 1 -- 172.21.15.64:0/300714457 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff50083e60 con 0x7eff600c7f00 2023-09-27T23:13:59.924 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:13:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 -- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff44060970 msgr2=0x7eff44062e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff44060970 0x7eff44062e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff58009ee0 tx=0x7eff58009840 comp rx=0 tx=0).stop 2023-09-27T23:13:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 -- 172.21.15.64:0/300714457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c7f00 msgr2=0x7eff60146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c7f00 0x7eff60146310 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff50015ca0 tx=0x7eff50001720 comp rx=0 tx=0).stop 2023-09-27T23:13:59.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 -- 172.21.15.64:0/300714457 shutdown_connections 2023-09-27T23:13:59.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff44060970 0x7eff44062e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff60155960 0x7eff600c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c7f00 0x7eff60146310 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 --2- 172.21.15.64:0/300714457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff6006de20 0x7eff600b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 -- 172.21.15.64:0/300714457 >> 172.21.15.64:0/300714457 conn(0x7eff60067bb0 msgr2=0x7eff6014dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 -- 172.21.15.64:0/300714457 shutdown_connections 2023-09-27T23:13:59.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.926+0000 7eff437fe700 1 -- 172.21.15.64:0/300714457 wait complete. 2023-09-27T23:13:59.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2023-09-27T23:13:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.942+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22f8005560 con 0x7f23180c7d90 2023-09-27T23:13:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.942+0000 7f22feffd700 1 -- 172.21.15.64:0/3421007839 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2310050d40 con 0x7f23180c7d90 2023-09-27T23:13:59.946 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2023-09-27T23:13:59.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2704005560 con 0x7f2720158bc0 2023-09-27T23:13:59.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f270e7fc700 1 -- 172.21.15.64:0/2723615757 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2718054e60 con 0x7f2720158bc0 2023-09-27T23:13:59.950 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2023-09-27T23:13:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27000609a0 msgr2=0x7f2700062e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27000609a0 0x7f2700062e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2710009860 tx=0x7f2710017000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2720158bc0 msgr2=0x7f27201767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2720158bc0 0x7f27201767e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2720068060 tx=0x7f27180016a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 shutdown_connections 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27000609a0 0x7f2700062e50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2720158bc0 0x7f27201767e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27200cc150 0x7f27201762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 --2- 172.21.15.64:0/2723615757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27200c5380 0x7f27201574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 >> 172.21.15.64:0/2723615757 conn(0x7f2720067bb0 msgr2=0x7f2720153740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 shutdown_connections 2023-09-27T23:13:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.946+0000 7f271f59e700 1 -- 172.21.15.64:0/2723615757 wait complete. 2023-09-27T23:13:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2304060950 msgr2=0x7f2304062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2304060950 0x7f2304062e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23080093f0 tx=0x7f2308016000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 msgr2=0x7f2318168b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 0x7f2318168b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f231000ce40 tx=0x7f2310001be0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 shutdown_connections 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2304060950 0x7f2304062e00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f231813f880 0x7f23180c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23180c7d90 0x7f2318168b40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 --2- 172.21.15.64:0/3421007839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f231806c7c0 0x7f23180ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 >> 172.21.15.64:0/3421007839 conn(0x7f2318067b60 msgr2=0x7f2318156700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 shutdown_connections 2023-09-27T23:13:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.950+0000 7f231d42e700 1 -- 172.21.15.64:0/3421007839 wait complete. 2023-09-27T23:13:59.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.954+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd944005830 con 0x7fd9600c8350 2023-09-27T23:13:59.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.954+0000 7fd94e7fc700 1 -- 172.21.15.64:0/2664624733 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd950050b20 con 0x7fd9600c8350 2023-09-27T23:13:59.959 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2023-09-27T23:13:59.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd940060950 msgr2=0x7fd940062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd940060950 0x7fd940062e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd96006a1f0 tx=0x7fd958018000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 msgr2=0x7fd960182240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 0x7fd960182240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d890 tx=0x7fd950008040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 shutdown_connections 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd940060950 0x7fd940062e00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9600c8350 0x7fd960182240 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd96016acb0 0x7fd960159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 --2- 172.21.15.64:0/2664624733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9600c8950 0x7fd960159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 >> 172.21.15.64:0/2664624733 conn(0x7fd960067bb0 msgr2=0x7fd9600cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 shutdown_connections 2023-09-27T23:13:59.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.958+0000 7fd964e15700 1 -- 172.21.15.64:0/2664624733 wait complete. 2023-09-27T23:13:59.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2023-09-27T23:13:59.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2023-09-27T23:13:59.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6200166f10 con 0x7f620013f6b0 2023-09-27T23:13:59.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f61f57fa700 1 -- 172.21.15.64:0/1117495069 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6200166f10 con 0x7f620013f6b0 2023-09-27T23:13:59.970 INFO:teuthology.orchestra.run.smithi064.stdout:60129542152 2023-09-27T23:13:59.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f61e4060950 msgr2=0x7f61e4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f61e4060950 0x7f61e4062e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f62000ae4a0 tx=0x7f61fc017000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 msgr2=0x7f6200159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 0x7f6200159860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c4c0 tx=0x7f61f0007f00 comp rx=0 tx=0).stop 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 shutdown_connections 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f61e4060950 0x7f61e4062e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f620013f6b0 0x7f6200159860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62000c7d90 0x7f6200159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 --2- 172.21.15.64:0/1117495069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f620006c7c0 0x7f620013b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.966+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 >> 172.21.15.64:0/1117495069 conn(0x7f6200067b60 msgr2=0x7f62000af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.970+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 shutdown_connections 2023-09-27T23:13:59.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.970+0000 7f6207609700 1 -- 172.21.15.64:0/1117495069 wait complete. 2023-09-27T23:13:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01280ae080 con 0x7f01280c7df0 2023-09-27T23:13:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f0124ff9700 1 -- 172.21.15.64:0/28070582 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0118050af0 con 0x7f01280c7df0 2023-09-27T23:13:59.977 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f01140609d0 msgr2=0x7f0114062e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f01140609d0 0x7f0114062e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f011c006fd0 tx=0x7f011c008040 comp rx=0 tx=0).stop 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 msgr2=0x7f01280cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 0x7f01280cab40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f011800b610 tx=0x7f01180090f0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 shutdown_connections 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f01140609d0 0x7f0114062e80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01281599b0 0x7f0128175b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01280c7df0 0x7f01280cab40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 --2- 172.21.15.64:0/28070582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01280afbf0 0x7f01281561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 >> 172.21.15.64:0/28070582 conn(0x7f0128067bb0 msgr2=0x7f012806a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 shutdown_connections 2023-09-27T23:13:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.974+0000 7f012ee3f700 1 -- 172.21.15.64:0/28070582 wait complete. 2023-09-27T23:13:59.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.1 2023-09-27T23:13:59.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0c5c005830 con 0x7f0c780cc5a0 2023-09-27T23:13:59.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c667fc700 1 -- 172.21.15.64:0/1933566668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c6c017090 con 0x7f0c780cc5a0 2023-09-27T23:13:59.986 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:13:59.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c58060950 msgr2=0x7f0c58062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c58060950 0x7f0c58062e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006fd0 tx=0x7f0c68009060 comp rx=0 tx=0).stop 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 msgr2=0x7f0c781554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c781554d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00ec80 tx=0x7f0c6c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 shutdown_connections 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c58060950 0x7f0c58062e00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c78158bc0 0x7f0c781594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c780cc5a0 0x7f0c781554d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 --2- 172.21.15.64:0/1933566668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c780cc150 0x7f0c78140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 >> 172.21.15.64:0/1933566668 conn(0x7f0c78067bb0 msgr2=0x7f0c780c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 shutdown_connections 2023-09-27T23:13:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.982+0000 7f0c7759e700 1 -- 172.21.15.64:0/1933566668 wait complete. 2023-09-27T23:13:59.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.0 2023-09-27T23:13:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f36340acd90 con 0x7f363406dd10 2023-09-27T23:13:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f36227fc700 1 -- 172.21.15.64:0/1106245850 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f362c051680 con 0x7f363406dd10 2023-09-27T23:13:59.994 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3614060950 msgr2=0x7f3614062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3614060950 0x7f3614062e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36240093f0 tx=0x7f3624017000 comp rx=0 tx=0).stop 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 msgr2=0x7f363413b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 0x7f363413b460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f362c007eb0 tx=0x7f362c009330 comp rx=0 tx=0).stop 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 shutdown_connections 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3614060950 0x7f3614062e00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36340ab8e0 0x7f363413b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363406dd10 0x7f363413b460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 --2- 172.21.15.64:0/1106245850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3634068670 0x7f3634175480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:13:59.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.990+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 >> 172.21.15.64:0/1106245850 conn(0x7f36340cb990 msgr2=0x7f3634143560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:13:59.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.994+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 shutdown_connections 2023-09-27T23:13:59.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:13:59.994+0000 7f363359e700 1 -- 172.21.15.64:0/1106245850 wait complete. 2023-09-27T23:13:59.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.2 2023-09-27T23:14:00.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2023-09-27T23:14:00.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-27T23:14:00.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-27T23:14:00.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-27T23:14:00.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-27T23:14:00.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-27T23:14:00.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-27T23:14:00.999 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-27T23:14:01.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-27T23:14:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e6c3f700 1 Processor -- start 2023-09-27T23:14:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e6c3f700 1 -- start start 2023-09-27T23:14:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e6c3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006de60 0x7f56e016a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e6c3f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e006b030 con 0x7f56e00c66d0 2023-09-27T23:14:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e6c3f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e006b310 con 0x7f56e006d9e0 2023-09-27T23:14:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e6c3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e006b5f0 con 0x7f56e006de60 2023-09-27T23:14:01.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e5c3d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f56e00c66d0 0x7f56e00c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34078/0 (socket says 172.21.15.64:34078) 2023-09-27T23:14:01.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e5c3d700 1 -- 172.21.15.64:0/2890613745 learned_addr learned my addr 172.21.15.64:0/2890613745 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:01.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.078+0000 7f56e643e700 1 --2- 172.21.15.64:0/2890613745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006de60 0x7f56e016a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f420eed1700 1 Processor -- start 2023-09-27T23:14:01.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f420eed1700 1 -- start start 2023-09-27T23:14:01.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420eed1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42080c66d0 0x7f42080c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539750383 0 0) 0x7f56e006b030 con 0x7f56e00c66d0 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56cc003550 con 0x7f56e00c66d0 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819772796 0 0) 0x7f56e006b310 con 0x7f56e006d9e0 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e006b030 con 0x7f56e006d9e0 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e643e700 1 -- 172.21.15.64:0/2890613745 >> v1:172.21.15.123:6789/0 conn(0x7f56e006d9e0 legacy=0x7f56e0169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e643e700 1 -- 172.21.15.64:0/2890613745 >> v1:172.21.15.64:6789/0 conn(0x7f56e00c66d0 legacy=0x7f56e00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e643e700 1 -- 172.21.15.64:0/2890613745 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e006b8d0 con 0x7f56e006de60 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e643e700 1 --2- 172.21.15.64:0/2890613745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006de60 0x7f56e016a2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009930 tx=0x7f56dc0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1114874f712d1ea7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0001470 con 0x7f56e006de60 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc014b60 con 0x7f56e006de60 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e4c3b700 1 -- 172.21.15.64:0/2890613745 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc01c410 con 0x7f56e006de60 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 -- 172.21.15.64:0/2890613745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006de60 msgr2=0x7f56e016a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 --2- 172.21.15.64:0/2890613745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006de60 0x7f56e016a2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009930 tx=0x7f56dc0095f0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 -- 172.21.15.64:0/2890613745 shutdown_connections 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 --2- 172.21.15.64:0/2890613745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006de60 0x7f56e016a2c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 -- 172.21.15.64:0/2890613745 >> 172.21.15.64:0/2890613745 conn(0x7f56e0067bb0 msgr2=0x7f56e00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 -- 172.21.15.64:0/2890613745 shutdown_connections 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.086+0000 7f56e6c3f700 1 -- 172.21.15.64:0/2890613745 wait complete. 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 Processor -- start 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 -- start start 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006d9e0 0x7f56e018aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 0x7f56e00c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56e00c66d0 0x7f56e00ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e018b400 con 0x7f56e006de60 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e018b760 con 0x7f56e00c66d0 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e6c3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e019b520 con 0x7f56e006d9e0 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 0x7f56e00c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e5c3d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006d9e0 0x7f56e018aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 0x7f56e00c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44602/0 (socket says 172.21.15.64:44602) 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 -- 172.21.15.64:0/187419578 learned_addr learned my addr 172.21.15.64:0/187419578 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e643e700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56e00c66d0 0x7f56e00ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420eed1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 0x7f420806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 -- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006d9e0 msgr2=0x7f56e018aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006d9e0 0x7f56e018aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 -- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56e00c66d0 msgr2=0x7f56e00ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56e00c66d0 0x7f56e00ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 -- 172.21.15.64:0/187419578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc009160 con 0x7f56e006de60 2023-09-27T23:14:01.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f56e543c700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 0x7f56e00c50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c910 tx=0x7f56d8002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420eed1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f420806de30 0x7f420806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420eed1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42080c6fe0 con 0x7f42080c66d0 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420eed1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420806b030 con 0x7f420806de30 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420eed1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420806b310 con 0x7f420806d9e0 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 0x7f420806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420e6d0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f420806de30 0x7f420806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 0x7f420806c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:52856/0 (socket says 172.21.15.64:52856) 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 -- 172.21.15.64:0/2934687469 learned_addr learned my addr 172.21.15.64:0/2934687469 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420decf700 1 --2- 172.21.15.64:0/2934687469 >> v2:172.21.15.64:3300/0 conn(0x7f42080c66d0 0x7f42080c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 -- 172.21.15.64:0/2934687469 >> v2:172.21.15.123:3300/0 conn(0x7f420806de30 msgr2=0x7f420806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8016070 con 0x7f56e006de60 2023-09-27T23:14:01.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8002260 con 0x7f56e006de60 2023-09-27T23:14:01.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8006df0 con 0x7f56e006de60 2023-09-27T23:14:01.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56e6c3f700 1 -- 172.21.15.64:0/187419578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e019b800 con 0x7f56e006de60 2023-09-27T23:14:01.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56e6c3f700 1 -- 172.21.15.64:0/187419578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e019bdc0 con 0x7f56e006de60 2023-09-27T23:14:01.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 --2- 172.21.15.64:0/2934687469 >> v2:172.21.15.123:3300/0 conn(0x7f420806de30 0x7f420806cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 -- 172.21.15.64:0/2934687469 >> v2:172.21.15.64:3300/0 conn(0x7f42080c66d0 msgr2=0x7f42080c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 --2- 172.21.15.64:0/2934687469 >> v2:172.21.15.64:3300/0 conn(0x7f42080c66d0 0x7f42080c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 -- 172.21.15.64:0/2934687469 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420806b5f0 con 0x7f420806d9e0 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420d6ce700 1 --2- 172.21.15.64:0/2934687469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 0x7f420806c7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f420806af40 tx=0x7f4200008840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c4259357edba66 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420cecd700 1 -- 172.21.15.64:0/2934687469 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000b070 con 0x7f420806d9e0 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420cecd700 1 -- 172.21.15.64:0/2934687469 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420000cc00 con 0x7f420806d9e0 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.090+0000 7f420cecd700 1 -- 172.21.15.64:0/2934687469 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000c040 con 0x7f420806d9e0 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 -- 172.21.15.64:0/2934687469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 msgr2=0x7f420806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 --2- 172.21.15.64:0/2934687469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 0x7f420806c7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f420806af40 tx=0x7f4200008840 comp rx=0 tx=0).stop 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 -- 172.21.15.64:0/2934687469 shutdown_connections 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 --2- 172.21.15.64:0/2934687469 >> v2:172.21.15.123:3300/0 conn(0x7f420806de30 0x7f420806cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 --2- 172.21.15.64:0/2934687469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f420806d9e0 0x7f420806c7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 --2- 172.21.15.64:0/2934687469 >> v2:172.21.15.64:3300/0 conn(0x7f42080c66d0 0x7f42080c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 -- 172.21.15.64:0/2934687469 >> 172.21.15.64:0/2934687469 conn(0x7f4208067bb0 msgr2=0x7f42080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 -- 172.21.15.64:0/2934687469 shutdown_connections 2023-09-27T23:14:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56e6c3f700 1 -- 172.21.15.64:0/187419578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e004be40 con 0x7f56e006de60 2023-09-27T23:14:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f56d8012650 con 0x7f56e006de60 2023-09-27T23:14:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f420eed1700 1 -- 172.21.15.64:0/2934687469 wait complete. 2023-09-27T23:14:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56d5ffb700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56c4060a20 0x7f56c4062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.094+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f56d807cbb0 con 0x7f56e006de60 2023-09-27T23:14:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f56e5c3d700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56c4060a20 0x7f56c4062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f56e5c3d700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56c4060a20 0x7f56c4062ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56d00015a0 tx=0x7f56d004d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 Processor -- start 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 -- start start 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420806d9e0 0x7f4208169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 0x7f420816ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42080c66d0 0x7f420815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420816b1c0 con 0x7f420806de30 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080b0cd0 con 0x7f420806d9e0 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420eed1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420816b3b0 con 0x7f42080c66d0 2023-09-27T23:14:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 0x7f420816ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 0x7f420816ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44610/0 (socket says 172.21.15.64:44610) 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 -- 172.21.15.64:0/4245568904 learned_addr learned my addr 172.21.15.64:0/4245568904 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420decf700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420806d9e0 0x7f4208169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420e6d0700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42080c66d0 0x7f420815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 -- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42080c66d0 msgr2=0x7f420815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42080c66d0 0x7f420815da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 -- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420806d9e0 msgr2=0x7f4208169cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420806d9e0 0x7f4208169cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.098+0000 7f420d6ce700 1 -- 172.21.15.64:0/4245568904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4200008ae0 con 0x7f420806de30 2023-09-27T23:14:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.102+0000 7f420d6ce700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 0x7f420816ac80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f420806b450 tx=0x7f4200018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.102+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200020070 con 0x7f420806de30 2023-09-27T23:14:01.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.102+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f56d804cbc0 con 0x7f56e006de60 2023-09-27T23:14:01.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.102+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42080b03a0 con 0x7f420806de30 2023-09-27T23:14:01.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42080b0960 con 0x7f420806de30 2023-09-27T23:14:01.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200007a10 con 0x7f420806de30 2023-09-27T23:14:01.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200007d90 con 0x7f420806de30 2023-09-27T23:14:01.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e80052b0 con 0x7f420806de30 2023-09-27T23:14:01.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f420000f070 con 0x7f420806de30 2023-09-27T23:14:01.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f41fdffb700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f41f4071ae0 0x7f41f4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f420007e5a0 con 0x7f420806de30 2023-09-27T23:14:01.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f4e53fff700 1 Processor -- start 2023-09-27T23:14:01.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f420decf700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f41f4071ae0 0x7f41f4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f420decf700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f41f4071ae0 0x7f41f4073f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4208167fd0 tx=0x7f41f800f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.106+0000 7f4e53fff700 1 -- start start 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e53fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4e54154c20 0x7f4e540abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e53fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e540c5fd0 con 0x7f4e54154ff0 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e53fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5406d3a0 con 0x7f4e5406ced0 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e53fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e540c5c70 con 0x7f4e54154c20 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e527fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e54154ff0 0x7f4e5406c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34090/0 (socket says 172.21.15.64:34090) 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e537fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4e5406ced0 0x7f4e5406a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59716/0 (socket says 172.21.15.64:59716) 2023-09-27T23:14:01.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e537fe700 1 -- 172.21.15.64:0/2864060814 learned_addr learned my addr 172.21.15.64:0/2864060814 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:01.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730307764 0 0) 0x7f4e5406d3a0 con 0x7f4e5406ced0 2023-09-27T23:14:01.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e3c003570 con 0x7f4e5406ced0 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/2864060814 >> v2:172.21.15.123:3300/0 conn(0x7f4e54154c20 0x7f4e540abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1582214303 0 0) 0x7f4e3c003570 con 0x7f4e5406ced0 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 >> v2:172.21.15.123:3300/0 conn(0x7f4e54154c20 msgr2=0x7f4e540abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 --2- 172.21.15.64:0/2864060814 >> v2:172.21.15.123:3300/0 conn(0x7f4e54154c20 0x7f4e540abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 >> v1:172.21.15.64:6789/0 conn(0x7f4e54154ff0 legacy=0x7f4e5406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5406d680 con 0x7f4e5406ced0 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/2864060814 >> v2:172.21.15.123:3300/0 conn(0x7f4e54154c20 0x7f4e540abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f4e44001470 con 0x7f4e5406ced0 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 >> v1:172.21.15.123:6789/0 conn(0x7f4e5406ced0 legacy=0x7f4e5406a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e3c003b30 0x7f4e3c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c004810 0x7f4e3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c005230 0x7f4e3c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540c5c70 con 0x7f4e3c003b30 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c00c160 con 0x7f4e3c005230 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c00c320 con 0x7f4e3c004810 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e3c003b30 0x7f4e3c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e527fc700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c004810 0x7f4e3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.110+0000 7f4e537fe700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c005230 0x7f4e3c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e52ffd700 1 -- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c004810 msgr2=0x7f4e3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c004810 0x7f4e3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e52ffd700 1 -- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c005230 msgr2=0x7f4e3c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c005230 0x7f4e3c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e52ffd700 1 -- 172.21.15.64:0/2864060814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c003570 con 0x7f4e3c003b30 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e537fe700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c005230 0x7f4e3c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e537fe700 1 -- 172.21.15.64:0/2864060814 reap_dead start 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e3c003b30 0x7f4e3c003f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c010620 tx=0x7f4e4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7794b8a4ab7b919a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c009ab0 con 0x7f4e3c003b30 2023-09-27T23:14:01.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c010b90 con 0x7f4e3c003b30 2023-09-27T23:14:01.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e51ffb700 1 -- 172.21.15.64:0/2864060814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c015820 con 0x7f4e3c003b30 2023-09-27T23:14:01.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f420004e530 con 0x7f420806de30 2023-09-27T23:14:01.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e53fff700 1 -- 172.21.15.64:0/2864060814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e3c003b30 msgr2=0x7f4e3c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.114+0000 7f4e53fff700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e3c003b30 0x7f4e3c003f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c010620 tx=0x7f4e4c00b040 comp rx=0 tx=0).stop 2023-09-27T23:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.118+0000 7f4e53fff700 1 -- 172.21.15.64:0/2864060814 shutdown_connections 2023-09-27T23:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.118+0000 7f4e53fff700 1 --2- 172.21.15.64:0/2864060814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e3c003b30 0x7f4e3c003f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.118+0000 7f4e53fff700 1 -- 172.21.15.64:0/2864060814 >> 172.21.15.64:0/2864060814 conn(0x7f4e54067bb0 msgr2=0x7f4e540cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.118+0000 7f4e53fff700 1 -- 172.21.15.64:0/2864060814 shutdown_connections 2023-09-27T23:14:01.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.118+0000 7f4e53fff700 1 -- 172.21.15.64:0/2864060814 wait complete. 2023-09-27T23:14:01.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.122+0000 7f4e53fff700 1 Processor -- start 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 -- start start 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c003b30 0x7f4e5416d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 0x7f4e5414ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e5414d290 0x7f4e5414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 0x7f4e5414ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 0x7f4e5414ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44622/0 (socket says 172.21.15.64:44622) 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e52ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c003b30 0x7f4e5416d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e52ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c003b30 0x7f4e5416d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:52878/0 (socket says 172.21.15.64:52878) 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 -- 172.21.15.64:0/1893700833 learned_addr learned my addr 172.21.15.64:0/1893700833 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5414e310 con 0x7f4e54143c30 2023-09-27T23:14:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e540b1690 con 0x7f4e5414d290 2023-09-27T23:14:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5416ddc0 con 0x7f4e3c003b30 2023-09-27T23:14:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e537fe700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e5414d290 0x7f4e5414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 -- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c003b30 msgr2=0x7f4e5416d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c003b30 0x7f4e5416d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 -- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e5414d290 msgr2=0x7f4e5414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e5414d290 0x7f4e5414f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5414d790 con 0x7f4e54143c30 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e527fc700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 0x7f4e5414ee40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e48001560 tx=0x7f4e48007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48012560 con 0x7f4e54143c30 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5414da70 con 0x7f4e54143c30 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5414e030 con 0x7f4e54143c30 2023-09-27T23:14:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.126+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480126c0 con 0x7f4e54143c30 2023-09-27T23:14:01.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.130+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48012a60 con 0x7f4e54143c30 2023-09-27T23:14:01.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.130+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e340052b0 con 0x7f4e54143c30 2023-09-27T23:14:01.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.130+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4e4801d460 con 0x7f4e54143c30 2023-09-27T23:14:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.130+0000 7f4e3affd700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e40060a50 0x7f4e40062f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.130+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4e4807db70 con 0x7f4e54143c30 2023-09-27T23:14:01.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.130+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e40060a50 0x7f4e40062f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.134+0000 7f4e52ffd700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e40060a50 0x7f4e40062f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0094a0 tx=0x7f4e4c021000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.138+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4e4804db60 con 0x7f4e54143c30 2023-09-27T23:14:01.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f88462e0700 1 Processor -- start 2023-09-27T23:14:01.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f88462e0700 1 -- start start 2023-09-27T23:14:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f88462e0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f884006b300 0x7f8840154de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f884487d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f88400c6880 0x7f88400ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34100/0 (socket says 172.21.15.64:34100) 2023-09-27T23:14:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f884487d700 1 -- 172.21.15.64:0/869192627 learned_addr learned my addr 172.21.15.64:0/869192627 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884006ceb0 con 0x7f88400c6880 2023-09-27T23:14:01.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400addf0 con 0x7f8840154a10 2023-09-27T23:14:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400ae330 con 0x7f884006b300 2023-09-27T23:14:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f883f7fe700 1 --2- 172.21.15.64:0/869192627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f884006b300 0x7f8840154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871087588 0 0) 0x7f884006ceb0 con 0x7f88400c6880 2023-09-27T23:14:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.150+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8828003570 con 0x7f88400c6880 2023-09-27T23:14:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681771699 0 0) 0x7f88400addf0 con 0x7f8840154a10 2023-09-27T23:14:01.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f884006ceb0 con 0x7f8840154a10 2023-09-27T23:14:01.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 Processor -- start 2023-09-27T23:14:01.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 -- start start 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901267437 0 0) 0x7f8828003570 con 0x7f88400c6880 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883f7fe700 1 -- 172.21.15.64:0/869192627 >> v1:172.21.15.123:6789/0 conn(0x7f8840154a10 legacy=0x7f88400ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883f7fe700 1 -- 172.21.15.64:0/869192627 >> v1:172.21.15.64:6789/0 conn(0x7f88400c6880 legacy=0x7f88400ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883f7fe700 1 -- 172.21.15.64:0/869192627 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400c6ce0 con 0x7f884006b300 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883f7fe700 1 --2- 172.21.15.64:0/869192627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f884006b300 0x7f8840154de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8830007320 tx=0x7f883000abc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d59ccefeff95a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000f070 con 0x7f884006b300 2023-09-27T23:14:01.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f883effd700 1 -- 172.21.15.64:0/869192627 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830007760 con 0x7f884006b300 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbf2c0c7d90 0x7fbf2c0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c13f880 0x7fbf2c0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2c06dc00 con 0x7fbf2c06c7c0 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2c06e090 con 0x7fbf2c0c7d90 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf30e14700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c06cff0 con 0x7fbf2c13f880 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf29d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbf2c06c7c0 0x7fbf2c13f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34102/0 (socket says 172.21.15.64:34102) 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf29d9b700 1 -- 172.21.15.64:0/2347819758 learned_addr learned my addr 172.21.15.64:0/2347819758 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2a59c700 1 --2- 172.21.15.64:0/2347819758 >> v2:172.21.15.123:3300/0 conn(0x7fbf2c0c7d90 0x7fbf2c0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2ad9d700 1 --2- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c13f880 0x7fbf2c0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f884006b300 msgr2=0x7f8840154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 --2- 172.21.15.64:0/869192627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f884006b300 0x7f8840154de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8830007320 tx=0x7f883000abc0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2a59c700 1 -- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c13f880 msgr2=0x7fbf2c0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2a59c700 1 --2- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c13f880 0x7fbf2c0ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2a59c700 1 -- 172.21.15.64:0/2347819758 >> v1:172.21.15.64:6789/0 conn(0x7fbf2c06c7c0 legacy=0x7fbf2c13f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2a59c700 1 -- 172.21.15.64:0/2347819758 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c06d2d0 con 0x7fbf2c0c7d90 2023-09-27T23:14:01.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 shutdown_connections 2023-09-27T23:14:01.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 --2- 172.21.15.64:0/869192627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f884006b300 0x7f8840154de0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 >> 172.21.15.64:0/869192627 conn(0x7f88400670e0 msgr2=0x7f88400674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 shutdown_connections 2023-09-27T23:14:01.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7f88462e0700 1 -- 172.21.15.64:0/869192627 wait complete. 2023-09-27T23:14:01.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2a59c700 1 --2- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c0c7d90 0x7fbf2c0c8160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098e0 tx=0x7fbf1c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5337cc039bb5834 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fd67a0c1700 1 Processor -- start 2023-09-27T23:14:01.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fd67a0c1700 1 -- start start 2023-09-27T23:14:01.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2959a700 1 -- 172.21.15.64:0/2347819758 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c011630 con 0x7fbf2c0c7d90 2023-09-27T23:14:01.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2959a700 1 -- 172.21.15.64:0/2347819758 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c011790 con 0x7fbf2c0c7d90 2023-09-27T23:14:01.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.154+0000 7fbf2959a700 1 -- 172.21.15.64:0/2347819758 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c011b30 con 0x7fbf2c0c7d90 2023-09-27T23:14:01.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 -- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c0c7d90 msgr2=0x7fbf2c0c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 Processor -- start 2023-09-27T23:14:01.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 -- start start 2023-09-27T23:14:01.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd67406d370 0x7fd67406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd67406cd00 0x7fd6740a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67406bc30 con 0x7fd674158e50 2023-09-27T23:14:01.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67406d1a0 con 0x7fd67406cd00 2023-09-27T23:14:01.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67406b020 con 0x7fd67406d370 2023-09-27T23:14:01.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 --2- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c0c7d90 0x7fbf2c0c8160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098e0 tx=0x7fbf1c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd67406cd00 0x7fd6740a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd672ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd67406d370 0x7fd67406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd67406cd00 0x7fd6740a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44628/0 (socket says 172.21.15.64:44628) 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 -- 172.21.15.64:0/3621295463 learned_addr learned my addr 172.21.15.64:0/3621295463 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 -- 172.21.15.64:0/3621295463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd67406d370 msgr2=0x7fd67406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 --2- 172.21.15.64:0/3621295463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd67406d370 0x7fd67406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 -- 172.21.15.64:0/3621295463 >> v1:172.21.15.64:6789/0 conn(0x7fd674158e50 legacy=0x7fd67406a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 -- 172.21.15.64:0/3621295463 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67406b300 con 0x7fd67406cd00 2023-09-27T23:14:01.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd673fff700 1 --2- 172.21.15.64:0/3621295463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd6740a8af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009f20 tx=0x7fd66c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abc574c7b1eb4df server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 -- 172.21.15.64:0/2347819758 shutdown_connections 2023-09-27T23:14:01.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 --2- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c13f880 0x7fbf2c0ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 --2- 172.21.15.64:0/2347819758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c0c7d90 0x7fbf2c0c8160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 -- 172.21.15.64:0/2347819758 >> 172.21.15.64:0/2347819758 conn(0x7fbf2c067b60 msgr2=0x7fbf2c0ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 -- 172.21.15.64:0/2347819758 shutdown_connections 2023-09-27T23:14:01.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 -- 172.21.15.64:0/2347819758 wait complete. 2023-09-27T23:14:01.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 0x7f884014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88400c6880 0x7f88400c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8840154a10 0x7f88400af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400c42b0 con 0x7f884006b300 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400c4590 con 0x7f88400c6880 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400c4870 con 0x7f8840154a10 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88400c6880 0x7f88400c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 0x7f884014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f884487d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8840154a10 0x7f88400af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 0x7f884014c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44638/0 (socket says 172.21.15.64:44638) 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 -- 172.21.15.64:0/4158484809 learned_addr learned my addr 172.21.15.64:0/4158484809 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 -- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8840154a10 msgr2=0x7f88400af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8840154a10 0x7f88400af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 -- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88400c6880 msgr2=0x7f88400c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88400c6880 0x7f88400c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 -- 172.21.15.64:0/4158484809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400c4b50 con 0x7f884006b300 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f883ffff700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 0x7f884014c380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f883400c790 tx=0x7f8834007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd6727fc700 1 -- 172.21.15.64:0/3621295463 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00e070 con 0x7fd67406cd00 2023-09-27T23:14:01.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd6727fc700 1 -- 172.21.15.64:0/3621295463 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c00b8d0 con 0x7fd67406cd00 2023-09-27T23:14:01.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834010510 con 0x7f884006b300 2023-09-27T23:14:01.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f251359e700 1 Processor -- start 2023-09-27T23:14:01.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f251359e700 1 -- start start 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd6727fc700 1 -- 172.21.15.64:0/3621295463 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0092e0 con 0x7fd67406cd00 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3621295463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 msgr2=0x7fd6740a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3621295463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd6740a8af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009f20 tx=0x7fd66c009840 comp rx=0 tx=0).stop 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834010670 con 0x7f884006b300 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834010a10 con 0x7f884006b300 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 -- 172.21.15.64:0/4158484809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88400c4e30 con 0x7f884006b300 2023-09-27T23:14:01.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7f88462e0700 1 -- 172.21.15.64:0/4158484809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884014cae0 con 0x7f884006b300 2023-09-27T23:14:01.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25140abcb0 0x7f25140ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.158+0000 7fbf30e14700 1 Processor -- start 2023-09-27T23:14:01.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2514159ad0 0x7f2514159f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251406ae90 con 0x7f251406ba10 2023-09-27T23:14:01.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140ad090 con 0x7f25140abcb0 2023-09-27T23:14:01.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140ab3c0 con 0x7f2514159ad0 2023-09-27T23:14:01.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 -- start start 2023-09-27T23:14:01.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25140abcb0 0x7f25140ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25140abcb0 0x7f25140ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60734/0 (socket says 172.21.15.64:60734) 2023-09-27T23:14:01.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 -- 172.21.15.64:0/1903314180 learned_addr learned my addr 172.21.15.64:0/1903314180 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3621295463 shutdown_connections 2023-09-27T23:14:01.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3621295463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd6740a8af0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3621295463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd67406d370 0x7fd67406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3621295463 >> 172.21.15.64:0/3621295463 conn(0x7fd674067bb0 msgr2=0x7fd6740ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 -- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2514159ad0 msgr2=0x7f2514159f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 --2- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2514159ad0 0x7f2514159f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 -- 172.21.15.64:0/1903314180 >> v1:172.21.15.123:6789/0 conn(0x7f251406ba10 legacy=0x7f25140adb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 -- 172.21.15.64:0/1903314180 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251406b320 con 0x7f25140abcb0 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f251259c700 1 --2- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f25140ac080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f25080099e0 tx=0x7f2508009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5a10775e7ad41 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c06c7c0 0x7fbf2c13d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c0c7d90 0x7fbf2c16a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 0x7fbf2c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f883401b3b0 con 0x7f884006b300 2023-09-27T23:14:01.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c1671f0 con 0x7fbf2c13f880 2023-09-27T23:14:01.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c16a9e0 con 0x7fbf2c06c7c0 2023-09-27T23:14:01.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c166ea0 con 0x7fbf2c0c7d90 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c06c7c0 0x7fbf2c13d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 0x7fbf2c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf29d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c0c7d90 0x7fbf2c16a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf29d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c0c7d90 0x7fbf2c16a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:52924/0 (socket says 172.21.15.64:52924) 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 0x7fbf2c1569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44650/0 (socket says 172.21.15.64:44650) 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf29d9b700 1 -- 172.21.15.64:0/307952850 learned_addr learned my addr 172.21.15.64:0/307952850 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f8827fff700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8828060ed0 0x7f8828063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f883407ca30 con 0x7f884006b300 2023-09-27T23:14:01.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7f883f7fe700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8828060ed0 0x7f8828063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 -- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c0c7d90 msgr2=0x7fbf2c16a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c0c7d90 0x7fbf2c16a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 -- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c06c7c0 msgr2=0x7fbf2c13d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c06c7c0 0x7fbf2c13d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 -- 172.21.15.64:0/307952850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c009420 con 0x7fbf2c13f880 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2a59c700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c06c7c0 0x7fbf2c13d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf2ad9d700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 0x7fbf2c1569d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ec90 tx=0x7fbf2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3621295463 shutdown_connections 2023-09-27T23:14:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3621295463 wait complete. 2023-09-27T23:14:01.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884004be40 con 0x7f884006b300 2023-09-27T23:14:01.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 Processor -- start 2023-09-27T23:14:01.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400c890 con 0x7fbf2c13f880 2023-09-27T23:14:01.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2400f040 con 0x7fbf2c13f880 2023-09-27T23:14:01.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400cc60 con 0x7fbf2c13f880 2023-09-27T23:14:01.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2c16ac40 con 0x7fbf2c13f880 2023-09-27T23:14:01.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.162+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c16b200 con 0x7fbf2c13f880 2023-09-27T23:14:01.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 -- start start 2023-09-27T23:14:01.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7f883f7fe700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8828060ed0 0x7f8828063380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f884006b6d0 tx=0x7f8830017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7f251159a700 1 -- 172.21.15.64:0/1903314180 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250800b7e0 con 0x7f25140abcb0 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7f251159a700 1 -- 172.21.15.64:0/1903314180 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250800b940 con 0x7f25140abcb0 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd674140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd67406d370 0x7fd6741595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd674158e50 0x7fd6740c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67414e350 con 0x7fd67406cd00 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67414eb60 con 0x7fd67406d370 2023-09-27T23:14:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd67a0c1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67414e5a0 con 0x7fd674158e50 2023-09-27T23:14:01.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7f251359e700 1 -- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 msgr2=0x7f25140ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7f251359e700 1 --2- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f25140ac080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f25080099e0 tx=0x7f2508009250 comp rx=0 tx=0).stop 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd674140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd674140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44652/0 (socket says 172.21.15.64:44652) 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 -- 172.21.15.64:0/3805058603 learned_addr learned my addr 172.21.15.64:0/3805058603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd673fff700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd674158e50 0x7fd6740c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd672ffd700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd67406d370 0x7fd6741595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 -- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd674158e50 msgr2=0x7fd6740c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd674158e50 0x7fd6740c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 -- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd67406d370 msgr2=0x7fd6741595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd67406d370 0x7fd6741595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 -- 172.21.15.64:0/3805058603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c009ae0 con 0x7fd67406cd00 2023-09-27T23:14:01.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.166+0000 7fd6737fe700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd674140480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c840 tx=0x7fd664007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbf24014690 con 0x7fbf2c13f880 2023-09-27T23:14:01.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 -- 172.21.15.64:0/1903314180 shutdown_connections 2023-09-27T23:14:01.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 --2- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2514159ad0 0x7f2514159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 --2- 172.21.15.64:0/1903314180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f25140ac080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 -- 172.21.15.64:0/1903314180 >> 172.21.15.64:0/1903314180 conn(0x7f2514067bb0 msgr2=0x7f25140cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fbf1a7fc700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf0c0609e0 0x7fbf0c062e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbf2407dc00 con 0x7fbf2c13f880 2023-09-27T23:14:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fbf2a59c700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf0c0609e0 0x7fbf0c062e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 -- 172.21.15.64:0/1903314180 shutdown_connections 2023-09-27T23:14:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 -- 172.21.15.64:0/1903314180 wait complete. 2023-09-27T23:14:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664011590 con 0x7fd67406cd00 2023-09-27T23:14:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664011710 con 0x7fd67406cd00 2023-09-27T23:14:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664011ad0 con 0x7fd67406cd00 2023-09-27T23:14:01.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f251359e700 1 Processor -- start 2023-09-27T23:14:01.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c167360 con 0x7fbf2c13f880 2023-09-27T23:14:01.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67414e880 con 0x7fd67406cd00 2023-09-27T23:14:01.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6741409c0 con 0x7fd67406cd00 2023-09-27T23:14:01.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 -- start start 2023-09-27T23:14:01.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.170+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f883404c9a0 con 0x7f884006b300 2023-09-27T23:14:01.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7fbf2a59c700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf0c0609e0 0x7fbf0c062e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0093f0 tx=0x7fbf1c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6740c9580 con 0x7fd67406cd00 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 Processor -- start 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 -- start start 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c0c68e0 0x7f4d6c0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d6c1618c0 0x7f4d6c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d6c0aba00 0x7f4d6c0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c161d10 con 0x7f4d6c0aba00 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c06df20 con 0x7f4d6c1618c0 2023-09-27T23:14:01.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d733a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0ac370 con 0x7f4d6c0c68e0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d6c1618c0 0x7f4d6c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d723a5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c0c68e0 0x7f4d6c0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d6c1618c0 0x7f4d6c06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60782/0 (socket says 172.21.15.64:60782) 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 -- 172.21.15.64:0/1422412132 learned_addr learned my addr 172.21.15.64:0/1422412132 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 -- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c0c68e0 msgr2=0x7f4d6c0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c0c68e0 0x7f4d6c0c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 -- 172.21.15.64:0/1422412132 >> v2:172.21.15.64:3300/0 conn(0x7f4d6c0aba00 msgr2=0x7f4d6c0abe30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/1422412132 >> v2:172.21.15.64:3300/0 conn(0x7f4d6c0aba00 0x7f4d6c0abe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 -- 172.21.15.64:0/1422412132 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c06e130 con 0x7f4d6c1618c0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c1618c0 0x7f4d6c06d9e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c063a20 tx=0x7f4d68009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75467448f3e0b8a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.178+0000 7f4d713a3700 1 -- 172.21.15.64:0/1422412132 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68009e70 con 0x7f4d6c1618c0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.178+0000 7f4d713a3700 1 -- 172.21.15.64:0/1422412132 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6800b860 con 0x7f4d6c1618c0 2023-09-27T23:14:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.178+0000 7f4d713a3700 1 -- 172.21.15.64:0/1422412132 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800bc00 con 0x7f4d6c1618c0 2023-09-27T23:14:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.178+0000 7f4d733a7700 1 -- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c1618c0 msgr2=0x7f4d6c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.178+0000 7f4d733a7700 1 --2- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c1618c0 0x7f4d6c06d9e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c063a20 tx=0x7f4d68009250 comp rx=0 tx=0).stop 2023-09-27T23:14:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251406ba10 0x7f25141550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f251416adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2514159ad0 0x7f25140b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514152b60 con 0x7f2514159ad0 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251416df10 con 0x7f25140abcb0 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251416e1f0 con 0x7f251406ba10 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251406ba10 0x7f25141550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251406ba10 0x7f25141550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:52940/0 (socket says 172.21.15.64:52940) 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f251259c700 1 -- 172.21.15.64:0/4188579896 learned_addr learned my addr 172.21.15.64:0/4188579896 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2511d9b700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f251416adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2514159ad0 0x7f25140b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 -- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251406ba10 msgr2=0x7f25141550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251406ba10 0x7f25141550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 -- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 msgr2=0x7f251416adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f251416adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fd66401b3b0 con 0x7fd67406cd00 2023-09-27T23:14:01.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 -- 172.21.15.64:0/4188579896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080094f0 con 0x7f2514159ad0 2023-09-27T23:14:01.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.174+0000 7f2512d9d700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2514159ad0 0x7f25140b0ac0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f250400b490 tx=0x7f2504006f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 -- 172.21.15.64:0/1422412132 shutdown_connections 2023-09-27T23:14:01.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 --2- 172.21.15.64:0/1422412132 >> v2:172.21.15.64:3300/0 conn(0x7f4d6c0aba00 0x7f4d6c0abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 --2- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c1618c0 0x7f4d6c06d9e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 --2- 172.21.15.64:0/1422412132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c0c68e0 0x7f4d6c0c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 -- 172.21.15.64:0/1422412132 >> 172.21.15.64:0/1422412132 conn(0x7f4d6c067bb0 msgr2=0x7f4d6c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7fd65b7fe700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd660060aa0 0x7fd660062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd66407c730 con 0x7fd67406cd00 2023-09-27T23:14:01.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7fd672ffd700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd660060aa0 0x7fd660062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 -- 172.21.15.64:0/1422412132 shutdown_connections 2023-09-27T23:14:01.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 -- 172.21.15.64:0/1422412132 wait complete. 2023-09-27T23:14:01.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbf2404daf0 con 0x7fbf2c13f880 2023-09-27T23:14:01.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7fd672ffd700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd660060aa0 0x7fd660062f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd67415c310 tx=0x7fd6680091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 Processor -- start 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.182+0000 7f4d733a7700 1 -- start start 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d733a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 0x7f4d6c16e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d733a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c0c68e0 0x7f4d6c148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d733a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c1618c0 0x7f4d6c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d733a7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c14deb0 con 0x7f4d6c0aba00 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d733a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0b0040 con 0x7f4d6c0c68e0 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d733a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c14d580 con 0x7f4d6c1618c0 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 0x7f4d6c16e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 0x7f4d6c16e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44666/0 (socket says 172.21.15.64:44666) 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 -- 172.21.15.64:0/2205826841 learned_addr learned my addr 172.21.15.64:0/2205826841 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c0c68e0 0x7f4d6c148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d72ba6700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c1618c0 0x7f4d6c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 -- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c1618c0 msgr2=0x7f4d6c0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c1618c0 0x7f4d6c0cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 -- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c0c68e0 msgr2=0x7f4d6c148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c0c68e0 0x7f4d6c148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 -- 172.21.15.64:0/2205826841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d680094f0 con 0x7f4d6c0aba00 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c0c68e0 0x7f4d6c148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.186+0000 7f4d723a5700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 0x7f4d6c16e200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c240 tx=0x7f4d6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011400 con 0x7f2514159ad0 2023-09-27T23:14:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251416e4d0 con 0x7f2514159ad0 2023-09-27T23:14:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251416b330 con 0x7f2514159ad0 2023-09-27T23:14:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd6740c9580 con 0x7fd67406cd00 2023-09-27T23:14:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504011580 con 0x7f2514159ad0 2023-09-27T23:14:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011940 con 0x7f2514159ad0 2023-09-27T23:14:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400cb70 con 0x7f4d6c0aba00 2023-09-27T23:14:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c14d860 con 0x7f4d6c0aba00 2023-09-27T23:14:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c1494f0 con 0x7f4d6c0aba00 2023-09-27T23:14:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6400f040 con 0x7f4d6c0aba00 2023-09-27T23:14:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.190+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64014480 con 0x7f4d6c0aba00 2023-09-27T23:14:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f2504011cf0 con 0x7f2514159ad0 2023-09-27T23:14:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f80052b0 con 0x7f2514159ad0 2023-09-27T23:14:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f25027fc700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24f40609a0 0x7f24f4062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f250407ca60 con 0x7f2514159ad0 2023-09-27T23:14:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f251259c700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24f40609a0 0x7f24f4062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4d640147f0 con 0x7f4d6c0aba00 2023-09-27T23:14:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f251259c700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24f40609a0 0x7f24f4062e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2508009290 tx=0x7f2508016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f4d627fc700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d50060950 0x7f4d50062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4d6407db90 con 0x7f4d6c0aba00 2023-09-27T23:14:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.194+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d50060950 0x7f4d50062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.198+0000 7f4d71ba4700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d50060950 0x7f4d50062e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c063a20 tx=0x7f4d68017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.198+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d540052b0 con 0x7f4d6c0aba00 2023-09-27T23:14:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.210+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f250404c9d0 con 0x7f2514159ad0 2023-09-27T23:14:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.218+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4d6404da80 con 0x7f4d6c0aba00 2023-09-27T23:14:01.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.386+0000 7f56e6c3f700 1 -- 172.21.15.64:0/187419578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f56e018b8a0 con 0x7f56e006de60 2023-09-27T23:14:01.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.386+0000 7f56d5ffb700 1 -- 172.21.15.64:0/187419578 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56d80501e0 con 0x7f56e006de60 2023-09-27T23:14:01.394 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 -- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56c4060a20 msgr2=0x7f56c4062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56c4060a20 0x7f56c4062ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56d00015a0 tx=0x7f56d004d040 comp rx=0 tx=0).stop 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 -- 172.21.15.64:0/187419578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 msgr2=0x7f56e00c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 0x7f56e00c50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c910 tx=0x7f56d8002700 comp rx=0 tx=0).stop 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 -- 172.21.15.64:0/187419578 shutdown_connections 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56c4060a20 0x7f56c4062ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56e00c66d0 0x7f56e00ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e006de60 0x7f56e00c50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 --2- 172.21.15.64:0/187419578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56e006d9e0 0x7f56e018aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 -- 172.21.15.64:0/187419578 >> 172.21.15.64:0/187419578 conn(0x7f56e0067bb0 msgr2=0x7f56e00c6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 -- 172.21.15.64:0/187419578 shutdown_connections 2023-09-27T23:14:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.394+0000 7f56c37fe700 1 -- 172.21.15.64:0/187419578 wait complete. 2023-09-27T23:14:01.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2023-09-27T23:14:01.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.422+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f41e8005830 con 0x7f420806de30 2023-09-27T23:14:01.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.422+0000 7f41fdffb700 1 -- 172.21.15.64:0/4245568904 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4200051b50 con 0x7f420806de30 2023-09-27T23:14:01.427 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2023-09-27T23:14:01.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f41f4071ae0 msgr2=0x7f41f4073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f41f4071ae0 0x7f41f4073f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4208167fd0 tx=0x7f41f800f040 comp rx=0 tx=0).stop 2023-09-27T23:14:01.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 msgr2=0x7f420816ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 0x7f420816ac80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f420806b450 tx=0x7f4200018000 comp rx=0 tx=0).stop 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 shutdown_connections 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f41f4071ae0 0x7f41f4073f90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42080c66d0 0x7f420815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420806de30 0x7f420816ac80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 --2- 172.21.15.64:0/4245568904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420806d9e0 0x7f4208169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 >> 172.21.15.64:0/4245568904 conn(0x7f4208067bb0 msgr2=0x7f42080ac490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 shutdown_connections 2023-09-27T23:14:01.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f420eed1700 1 -- 172.21.15.64:0/4245568904 wait complete. 2023-09-27T23:14:01.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.426+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4e34005830 con 0x7f4e54143c30 2023-09-27T23:14:01.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e3affd700 1 -- 172.21.15.64:0/1893700833 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e48051180 con 0x7f4e54143c30 2023-09-27T23:14:01.433 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2023-09-27T23:14:01.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e40060a50 msgr2=0x7f4e40062f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e40060a50 0x7f4e40062f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0094a0 tx=0x7f4e4c021000 comp rx=0 tx=0).stop 2023-09-27T23:14:01.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 msgr2=0x7f4e5414ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 0x7f4e5414ee40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e48001560 tx=0x7f4e48007d20 comp rx=0 tx=0).stop 2023-09-27T23:14:01.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 shutdown_connections 2023-09-27T23:14:01.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e40060a50 0x7f4e40062f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e5414d290 0x7f4e5414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e54143c30 0x7f4e5414ee40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 --2- 172.21.15.64:0/1893700833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e3c003b30 0x7f4e5416d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 >> 172.21.15.64:0/1893700833 conn(0x7f4e54067bb0 msgr2=0x7f4e540c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 shutdown_connections 2023-09-27T23:14:01.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.430+0000 7f4e53fff700 1 -- 172.21.15.64:0/1893700833 wait complete. 2023-09-27T23:14:01.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2023-09-27T23:14:01.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2023-09-27T23:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24f8005560 con 0x7f2514159ad0 2023-09-27T23:14:01.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f25027fc700 1 -- 172.21.15.64:0/4188579896 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f250404fff0 con 0x7f2514159ad0 2023-09-27T23:14:01.469 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:14:01.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbf2c071120 con 0x7fbf2c13f880 2023-09-27T23:14:01.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7fbf1a7fc700 1 -- 172.21.15.64:0/307952850 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf24018020 con 0x7fbf2c13f880 2023-09-27T23:14:01.472 INFO:teuthology.orchestra.run.smithi064.stdout:64424509446 2023-09-27T23:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24f40609a0 msgr2=0x7f24f4062e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24f40609a0 0x7f24f4062e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2508009290 tx=0x7f2508016000 comp rx=0 tx=0).stop 2023-09-27T23:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2514159ad0 msgr2=0x7f25140b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2514159ad0 0x7f25140b0ac0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f250400b490 tx=0x7f2504006f50 comp rx=0 tx=0).stop 2023-09-27T23:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 shutdown_connections 2023-09-27T23:14:01.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24f40609a0 0x7f24f4062e50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2514159ad0 0x7f25140b0ac0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25140abcb0 0x7f251416adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 --2- 172.21.15.64:0/4188579896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f251406ba10 0x7f25141550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 >> 172.21.15.64:0/4188579896 conn(0x7f2514067bb0 msgr2=0x7f25140c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 shutdown_connections 2023-09-27T23:14:01.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.466+0000 7f251359e700 1 -- 172.21.15.64:0/4188579896 wait complete. 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf0c0609e0 msgr2=0x7fbf0c062e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf0c0609e0 0x7fbf0c062e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0093f0 tx=0x7fbf1c016000 comp rx=0 tx=0).stop 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 msgr2=0x7fbf2c1569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 0x7fbf2c1569d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ec90 tx=0x7fbf2400b040 comp rx=0 tx=0).stop 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 shutdown_connections 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf0c0609e0 0x7fbf0c062e90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf2c13f880 0x7fbf2c1569d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf2c0c7d90 0x7fbf2c16a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 --2- 172.21.15.64:0/307952850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf2c06c7c0 0x7fbf2c13d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 >> 172.21.15.64:0/307952850 conn(0x7fbf2c067b60 msgr2=0x7fbf2c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 shutdown_connections 2023-09-27T23:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.470+0000 7fbf30e14700 1 -- 172.21.15.64:0/307952850 wait complete. 2023-09-27T23:14:01.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.0 2023-09-27T23:14:01.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2023-09-27T23:14:01.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.486+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f88400ae050 con 0x7f884006b300 2023-09-27T23:14:01.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.486+0000 7f8827fff700 1 -- 172.21.15.64:0/4158484809 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8834017030 con 0x7f884006b300 2023-09-27T23:14:01.491 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:14:01.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.486+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4d54005560 con 0x7f4d6c0aba00 2023-09-27T23:14:01.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.486+0000 7f4d627fc700 1 -- 172.21.15.64:0/2205826841 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d64018030 con 0x7f4d6c0aba00 2023-09-27T23:14:01.491 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:14:01.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d50060950 msgr2=0x7f4d50062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d50060950 0x7f4d50062e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c063a20 tx=0x7f4d68017000 comp rx=0 tx=0).stop 2023-09-27T23:14:01.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 msgr2=0x7f4d6c16e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 0x7f4d6c16e200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c240 tx=0x7f4d6400b040 comp rx=0 tx=0).stop 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 shutdown_connections 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d50060950 0x7f4d50062e00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c1618c0 0x7f4d6c0cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c0c68e0 0x7f4d6c148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 --2- 172.21.15.64:0/2205826841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0aba00 0x7f4d6c16e200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 >> 172.21.15.64:0/2205826841 conn(0x7f4d6c067bb0 msgr2=0x7f4d6c0ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8828060ed0 msgr2=0x7f8828063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8828060ed0 0x7f8828063380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f884006b6d0 tx=0x7f8830017000 comp rx=0 tx=0).stop 2023-09-27T23:14:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 msgr2=0x7f884014c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 0x7f884014c380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f883400c790 tx=0x7f8834007e10 comp rx=0 tx=0).stop 2023-09-27T23:14:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 shutdown_connections 2023-09-27T23:14:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f4d733a7700 1 -- 172.21.15.64:0/2205826841 wait complete. 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 shutdown_connections 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8828060ed0 0x7f8828063380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8840154a10 0x7f88400af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88400c6880 0x7f88400c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 --2- 172.21.15.64:0/4158484809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884006b300 0x7f884014c380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 >> 172.21.15.64:0/4158484809 conn(0x7f88400670e0 msgr2=0x7f88400c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 shutdown_connections 2023-09-27T23:14:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.490+0000 7f8825ffb700 1 -- 172.21.15.64:0/4158484809 wait complete. 2023-09-27T23:14:01.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2023-09-27T23:14:01.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.2 2023-09-27T23:14:01.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.502+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd6740c9580 con 0x7fd67406cd00 2023-09-27T23:14:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.502+0000 7fd65b7fe700 1 -- 172.21.15.64:0/3805058603 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd664017020 con 0x7fd67406cd00 2023-09-27T23:14:01.508 INFO:teuthology.orchestra.run.smithi064.stdout:60129542152 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd660060aa0 msgr2=0x7fd660062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd660060aa0 0x7fd660062f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd67415c310 tx=0x7fd6680091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 msgr2=0x7fd674140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd674140480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c840 tx=0x7fd664007ee0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 shutdown_connections 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd660060aa0 0x7fd660062f50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd674158e50 0x7fd6740c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd67406d370 0x7fd6741595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 --2- 172.21.15.64:0/3805058603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd67406cd00 0x7fd674140480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 >> 172.21.15.64:0/3805058603 conn(0x7fd674067bb0 msgr2=0x7fd6740af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 shutdown_connections 2023-09-27T23:14:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:01.506+0000 7fd67a0c1700 1 -- 172.21.15.64:0/3805058603 wait complete. 2023-09-27T23:14:01.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.1 2023-09-27T23:14:02.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-27T23:14:02.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-27T23:14:02.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-27T23:14:02.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-27T23:14:02.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-27T23:14:02.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-27T23:14:02.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-27T23:14:02.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-27T23:14:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 Processor -- start 2023-09-27T23:14:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 -- start start 2023-09-27T23:14:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb5ec155960 0x7fb5ec15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec06d990 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec15a570 con 0x7fb5ec155960 2023-09-27T23:14:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb5ec155960 0x7fb5ec15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb5ec155960 0x7fb5ec15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44670/0 (socket says 172.21.15.64:44670) 2023-09-27T23:14:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 -- 172.21.15.64:0/242189789 learned_addr learned my addr 172.21.15.64:0/242189789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5eb59e700 1 -- 172.21.15.64:0/242189789 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec06cd80 con 0x7fb5ec06de20 2023-09-27T23:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 -- 172.21.15.64:0/242189789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 msgr2=0x7fb5ec0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 --2- 172.21.15.64:0/242189789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 -- 172.21.15.64:0/242189789 >> v1:172.21.15.123:6789/0 conn(0x7fb5ec06c7c0 legacy=0x7fb5ec0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 -- 172.21.15.64:0/242189789 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec06d060 con 0x7fb5ec155960 2023-09-27T23:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.554+0000 7fb5ea59c700 1 --2- 172.21.15.64:0/242189789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec155960 0x7fb5ec15a030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009f20 tx=0x7fb5e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a7d32a0d9339af server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5e959a700 1 -- 172.21.15.64:0/242189789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400e070 con 0x7fb5ec155960 2023-09-27T23:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5e959a700 1 -- 172.21.15.64:0/242189789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e400ba40 con 0x7fb5ec155960 2023-09-27T23:14:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5e959a700 1 -- 172.21.15.64:0/242189789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40092e0 con 0x7fb5ec155960 2023-09-27T23:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 -- 172.21.15.64:0/242189789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec155960 msgr2=0x7fb5ec15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 --2- 172.21.15.64:0/242189789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec155960 0x7fb5ec15a030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009f20 tx=0x7fb5e4009840 comp rx=0 tx=0).stop 2023-09-27T23:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 -- 172.21.15.64:0/242189789 shutdown_connections 2023-09-27T23:14:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 --2- 172.21.15.64:0/242189789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 --2- 172.21.15.64:0/242189789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec155960 0x7fb5ec15a030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 -- 172.21.15.64:0/242189789 >> 172.21.15.64:0/242189789 conn(0x7fb5ec067bb0 msgr2=0x7fb5ec0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 -- 172.21.15.64:0/242189789 shutdown_connections 2023-09-27T23:14:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.558+0000 7fb5eb59e700 1 -- 172.21.15.64:0/242189789 wait complete. 2023-09-27T23:14:02.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 Processor -- start 2023-09-27T23:14:02.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 -- start start 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 0x7fb5ec147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ec155960 0x7fb5ec142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14aae0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14a2d0 con 0x7fb5ec155960 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5eb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec147960 con 0x7fb5ec06de20 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 0x7fb5ec147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 0x7fb5ec147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44680/0 (socket says 172.21.15.64:44680) 2023-09-27T23:14:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 -- 172.21.15.64:0/4233157576 learned_addr learned my addr 172.21.15.64:0/4233157576 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5e9d9b700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ead9d700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ec155960 0x7fb5ec142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 -- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 msgr2=0x7fb5ec0c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 -- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ec155960 msgr2=0x7fb5ec142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ec155960 0x7fb5ec142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 -- 172.21.15.64:0/4233157576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e4009ae0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5e9d9b700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.562+0000 7fb5ea59c700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 0x7fb5ec147180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec064040 tx=0x7fb5e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.570+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4012bd0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.570+0000 7fb5eb59e700 1 -- 172.21.15.64:0/4233157576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec14a520 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.570+0000 7fb5eb59e700 1 -- 172.21.15.64:0/4233157576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec142690 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.570+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4012d70 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.570+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e401e680 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.570+0000 7fb5eb59e700 1 -- 172.21.15.64:0/4233157576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d00052b0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.574+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb5e401e9f0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.574+0000 7fb5da7fc700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb5cc060ad0 0x7fb5cc062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.574+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb5e407ecc0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.578+0000 7fb5e9d9b700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb5cc060ad0 0x7fb5cc062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.582+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb5e404ecb0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.582+0000 7fb5e9d9b700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb5cc060ad0 0x7fb5cc062f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec1498e0 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54310d8700 1 Processor -- start 2023-09-27T23:14:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54310d8700 1 -- start start 2023-09-27T23:14:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54310d8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f542c154ff0 0x7f542c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54310d8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c06d160 con 0x7f542c06cd00 2023-09-27T23:14:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54310d8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0c5fd0 con 0x7f542c154c20 2023-09-27T23:14:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54310d8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0c5c70 con 0x7f542c154ff0 2023-09-27T23:14:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 Processor -- start 2023-09-27T23:14:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54308d7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f542c06cd00 0x7f542c06a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34114/0 (socket says 172.21.15.64:34114) 2023-09-27T23:14:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f54308d7700 1 -- 172.21.15.64:0/2348415782 learned_addr learned my addr 172.21.15.64:0/2348415782 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542b7fe700 1 --2- 172.21.15.64:0/2348415782 >> v2:172.21.15.123:3300/0 conn(0x7f542c154ff0 0x7f542c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 -- start start 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705883287 0 0) 0x7f541c001110 con 0x7f542c06cd00 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5414003570 con 0x7f542c06cd00 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252414059 0 0) 0x7f542c0c5fd0 con 0x7f542c154c20 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f541c001110 con 0x7f542c154c20 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1419372014 0 0) 0x7f5414003570 con 0x7f542c06cd00 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542b7fe700 1 -- 172.21.15.64:0/2348415782 >> v1:172.21.15.123:6789/0 conn(0x7f542c154c20 legacy=0x7f542c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542b7fe700 1 -- 172.21.15.64:0/2348415782 >> v1:172.21.15.64:6789/0 conn(0x7f542c06cd00 legacy=0x7f542c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542b7fe700 1 -- 172.21.15.64:0/2348415782 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c06d3a0 con 0x7f542c154ff0 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8350154c20 0x7f83500abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8350154ff0 0x7f835006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f835006cd00 0x7f835006a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83500c5c70 con 0x7f8350154ff0 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835006d160 con 0x7f8350154c20 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8355eb6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83500c5fd0 con 0x7f835006cd00 2023-09-27T23:14:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8350154ff0 0x7f835006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8350154ff0 0x7f835006c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44686/0 (socket says 172.21.15.64:44686) 2023-09-27T23:14:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 -- 172.21.15.64:0/2234713151 learned_addr learned my addr 172.21.15.64:0/2234713151 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f542b7fe700 1 --2- 172.21.15.64:0/2348415782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154ff0 0x7f542c06c7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f542000b310 tx=0x7f54200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f0556bd5320c05 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f83556b5700 1 --2- 172.21.15.64:0/2234713151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f835006cd00 0x7f835006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f8354eb4700 1 --2- 172.21.15.64:0/2234713151 >> v2:172.21.15.123:3300/0 conn(0x7f8350154c20 0x7f83500abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 -- 172.21.15.64:0/2234713151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f835006cd00 msgr2=0x7f835006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 --2- 172.21.15.64:0/2234713151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f835006cd00 0x7f835006a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 -- 172.21.15.64:0/2234713151 >> v2:172.21.15.123:3300/0 conn(0x7f8350154c20 msgr2=0x7f83500abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 --2- 172.21.15.64:0/2234713151 >> v2:172.21.15.123:3300/0 conn(0x7f8350154c20 0x7f83500abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.594+0000 7f834ffff700 1 -- 172.21.15.64:0/2234713151 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835006d3a0 con 0x7f8350154ff0 2023-09-27T23:14:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c001600 con 0x7f542c154ff0 2023-09-27T23:14:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542000bb00 con 0x7f542c154ff0 2023-09-27T23:14:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f834ffff700 1 --2- 172.21.15.64:0/2234713151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350154ff0 0x7f835006c7c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83500696f0 tx=0x7f83440082d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc232080fc1392 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f834f7fe700 1 -- 172.21.15.64:0/2234713151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344010070 con 0x7f8350154ff0 2023-09-27T23:14:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f834f7fe700 1 -- 172.21.15.64:0/2234713151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834400db00 con 0x7f8350154ff0 2023-09-27T23:14:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f542affd700 1 -- 172.21.15.64:0/2348415782 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000bec0 con 0x7f542c154ff0 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 -- 172.21.15.64:0/2348415782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154ff0 msgr2=0x7f542c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 --2- 172.21.15.64:0/2348415782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154ff0 0x7f542c06c7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f542000b310 tx=0x7f54200091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 -- 172.21.15.64:0/2348415782 shutdown_connections 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 --2- 172.21.15.64:0/2348415782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154ff0 0x7f542c06c7c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 -- 172.21.15.64:0/2348415782 >> 172.21.15.64:0/2348415782 conn(0x7f542c067bb0 msgr2=0x7f542c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 -- 172.21.15.64:0/2234713151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350154ff0 msgr2=0x7f835006c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 --2- 172.21.15.64:0/2234713151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350154ff0 0x7f835006c7c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83500696f0 tx=0x7f83440082d0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 -- 172.21.15.64:0/2348415782 shutdown_connections 2023-09-27T23:14:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 -- 172.21.15.64:0/2234713151 shutdown_connections 2023-09-27T23:14:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 --2- 172.21.15.64:0/2234713151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f835006cd00 0x7f835006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 --2- 172.21.15.64:0/2234713151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8350154ff0 0x7f835006c7c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 --2- 172.21.15.64:0/2234713151 >> v2:172.21.15.123:3300/0 conn(0x7f8350154c20 0x7f83500abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 -- 172.21.15.64:0/2234713151 >> 172.21.15.64:0/2234713151 conn(0x7f8350067bb0 msgr2=0x7f83500cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 -- 172.21.15.64:0/2348415782 wait complete. 2023-09-27T23:14:02.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 -- 172.21.15.64:0/2234713151 shutdown_connections 2023-09-27T23:14:02.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 -- 172.21.15.64:0/2234713151 wait complete. 2023-09-27T23:14:02.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 Processor -- start 2023-09-27T23:14:02.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 Processor -- start 2023-09-27T23:14:02.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 -- start start 2023-09-27T23:14:02.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f54310d8700 1 -- start start 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.598+0000 7f8355eb6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835006cd00 0x7f83500ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8350154c20 0x7f8350178fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 0x7f8350179500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350086d20 con 0x7f83500ca720 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350150220 con 0x7f835006cd00 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8350179e80 con 0x7f8350154c20 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 0x7f8350179500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 0x7f8350179500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44698/0 (socket says 172.21.15.64:44698) 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 -- 172.21.15.64:0/4216846783 learned_addr learned my addr 172.21.15.64:0/4216846783 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f834ffff700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8350154c20 0x7f8350178fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8354eb4700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835006cd00 0x7f83500ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 -- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8350154c20 msgr2=0x7f8350178fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8350154c20 0x7f8350178fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 -- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835006cd00 msgr2=0x7f83500ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835006cd00 0x7f83500ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 -- 172.21.15.64:0/4216846783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344008570 con 0x7f83500ca720 2023-09-27T23:14:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 0x7f542c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154c20 0x7f542c172ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f542c173000 0x7f542c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c14fab0 con 0x7f542c06cd00 2023-09-27T23:14:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c14ee30 con 0x7f542c154c20 2023-09-27T23:14:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c14f110 con 0x7f542c173000 2023-09-27T23:14:02.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f83556b5700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 0x7f8350179500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8348009fd0 tx=0x7f8348009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 0x7f542c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 0x7f542c0b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44708/0 (socket says 172.21.15.64:44708) 2023-09-27T23:14:02.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 -- 172.21.15.64:0/2254766973 learned_addr learned my addr 172.21.15.64:0/2254766973 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542b7fe700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154c20 0x7f542c172ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54308d7700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f542c173000 0x7f542c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800ebf0 con 0x7f83500ca720 2023-09-27T23:14:02.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 -- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f542c173000 msgr2=0x7f542c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f542c173000 0x7f542c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 -- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154c20 msgr2=0x7f542c172ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154c20 0x7f542c172ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 -- 172.21.15.64:0/2254766973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420009420 con 0x7f542c06cd00 2023-09-27T23:14:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800ed50 con 0x7f83500ca720 2023-09-27T23:14:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348014670 con 0x7f83500ca720 2023-09-27T23:14:02.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f542bfff700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 0x7f542c0b1320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f542400cb10 tx=0x7f54240027c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8350150aa0 con 0x7f83500ca720 2023-09-27T23:14:02.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83501504d0 con 0x7f83500ca720 2023-09-27T23:14:02.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400ce20 con 0x7f542c06cd00 2023-09-27T23:14:02.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424002020 con 0x7f542c06cd00 2023-09-27T23:14:02.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54240023e0 con 0x7f542c06cd00 2023-09-27T23:14:02.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f83480149e0 con 0x7f83500ca720 2023-09-27T23:14:02.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c14f3f0 con 0x7f542c06cd00 2023-09-27T23:14:02.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.602+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0b1860 con 0x7f542c06cd00 2023-09-27T23:14:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f834cff9700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f833c060a20 0x7f833c062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f834807d6f0 con 0x7f83500ca720 2023-09-27T23:14:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c04be40 con 0x7f542c06cd00 2023-09-27T23:14:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f542400a9f0 con 0x7f542c06cd00 2023-09-27T23:14:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f5413fff700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5418060950 0x7f5418062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.606+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f542407d0f0 con 0x7f542c06cd00 2023-09-27T23:14:02.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.610+0000 7f542b7fe700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5418060950 0x7f5418062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.610+0000 7f8354eb4700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f833c060a20 0x7f833c062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.610+0000 7f8354eb4700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f833c060a20 0x7f833c062ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8340006fd0 tx=0x7f8340008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.610+0000 7f542b7fe700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5418060950 0x7f5418062e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f54200091e0 tx=0x7f5420016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.614+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83300052b0 con 0x7f83500ca720 2023-09-27T23:14:02.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.614+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f542404d080 con 0x7f542c06cd00 2023-09-27T23:14:02.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 Processor -- start 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 -- start start 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40640ac110 0x7f40640c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 0x7f406406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406406e130 con 0x7f40640c68e0 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4064161d30 con 0x7f40640ac110 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f406a015700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406406df20 con 0x7f40641618c0 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 0x7f406406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 0x7f406406d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:53014/0 (socket says 172.21.15.64:53014) 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 -- 172.21.15.64:0/2061002806 learned_addr learned my addr 172.21.15.64:0/2061002806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4068812700 1 --2- 172.21.15.64:0/2061002806 >> v2:172.21.15.64:3300/0 conn(0x7f40640ac110 0x7f40640c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 -- 172.21.15.64:0/2061002806 >> v2:172.21.15.64:3300/0 conn(0x7f40640ac110 msgr2=0x7f40640c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 --2- 172.21.15.64:0/2061002806 >> v2:172.21.15.64:3300/0 conn(0x7f40640ac110 0x7f40640c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 -- 172.21.15.64:0/2061002806 >> v1:172.21.15.64:6789/0 conn(0x7f40640c68e0 legacy=0x7f40640aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 -- 172.21.15.64:0/2061002806 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406406a6b0 con 0x7f40641618c0 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4068812700 1 --2- 172.21.15.64:0/2061002806 >> v2:172.21.15.64:3300/0 conn(0x7f40640ac110 0x7f40640c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f4069814700 1 --2- 172.21.15.64:0/2061002806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 0x7f406406d9e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f405c009f20 tx=0x7f405c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848b366a8d188dc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f405bfff700 1 -- 172.21.15.64:0/2061002806 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c00e070 con 0x7f40641618c0 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.618+0000 7f405bfff700 1 -- 172.21.15.64:0/2061002806 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c00b9d0 con 0x7f40641618c0 2023-09-27T23:14:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f405bfff700 1 -- 172.21.15.64:0/2061002806 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0092e0 con 0x7f40641618c0 2023-09-27T23:14:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 -- 172.21.15.64:0/2061002806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 msgr2=0x7f406406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 --2- 172.21.15.64:0/2061002806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 0x7f406406d9e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f405c009f20 tx=0x7f405c009840 comp rx=0 tx=0).stop 2023-09-27T23:14:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 -- 172.21.15.64:0/2061002806 shutdown_connections 2023-09-27T23:14:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 --2- 172.21.15.64:0/2061002806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40641618c0 0x7f406406d9e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 --2- 172.21.15.64:0/2061002806 >> v2:172.21.15.64:3300/0 conn(0x7f40640ac110 0x7f40640c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 -- 172.21.15.64:0/2061002806 >> 172.21.15.64:0/2061002806 conn(0x7f4064067bb0 msgr2=0x7f40640cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 -- 172.21.15.64:0/2061002806 shutdown_connections 2023-09-27T23:14:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 -- 172.21.15.64:0/2061002806 wait complete. 2023-09-27T23:14:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 Processor -- start 2023-09-27T23:14:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f834804d700 con 0x7f83500ca720 2023-09-27T23:14:02.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.622+0000 7f406a015700 1 -- start start 2023-09-27T23:14:02.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f406a015700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40640ac110 0x7f40640ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f406a015700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 0x7f4064157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f406a015700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40641618c0 0x7f406413b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f406a015700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406413ada0 con 0x7f40640c68e0 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f406a015700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641510d0 con 0x7f40641618c0 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f406a015700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4064152520 con 0x7f40640ac110 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 0x7f4064157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 0x7f4064157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44732/0 (socket says 172.21.15.64:44732) 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 -- 172.21.15.64:0/1634851205 learned_addr learned my addr 172.21.15.64:0/1634851205 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4069013700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40640ac110 0x7f40640ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4069814700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40641618c0 0x7f406413b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 -- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40640ac110 msgr2=0x7f40640ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40640ac110 0x7f40640ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 -- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40641618c0 msgr2=0x7f406413b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40641618c0 0x7f406413b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 -- 172.21.15.64:0/1634851205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4060009420 con 0x7f40640c68e0 2023-09-27T23:14:02.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.626+0000 7f4068812700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 0x7f4064157e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f406000ecb0 tx=0x7f40600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.630+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000c920 con 0x7f40640c68e0 2023-09-27T23:14:02.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.630+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c009ae0 con 0x7f40640c68e0 2023-09-27T23:14:02.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.630+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4064152a80 con 0x7f40640c68e0 2023-09-27T23:14:02.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000ca80 con 0x7f40640c68e0 2023-09-27T23:14:02.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000ce20 con 0x7f40640c68e0 2023-09-27T23:14:02.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4060013680 con 0x7f40640c68e0 2023-09-27T23:14:02.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f40597fa700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4048060ad0 0x7f4048062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f406007d230 con 0x7f40640c68e0 2023-09-27T23:14:02.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 Processor -- start 2023-09-27T23:14:02.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f4069013700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4048060ad0 0x7f4048062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f4069013700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4048060ad0 0x7f4048062f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4054003a60 tx=0x7f4054007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 -- start start 2023-09-27T23:14:02.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f40154ff0 0x7f0f4006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f4006cd00 0x7f0f4006a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4006d160 con 0x7f0f40154c20 2023-09-27T23:14:02.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f400c5c70 con 0x7f0f4006cd00 2023-09-27T23:14:02.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.634+0000 7f0f46be0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f400ac3b0 con 0x7f0f40154ff0 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f4006cd00 0x7f0f4006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f4006cd00 0x7f0f4006a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44744/0 (socket says 172.21.15.64:44744) 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f45bde700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0f40154c20 0x7f0f400abca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34130/0 (socket says 172.21.15.64:34130) 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 -- 172.21.15.64:0/1795323935 learned_addr learned my addr 172.21.15.64:0/1795323935 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 -- 172.21.15.64:0/1795323935 >> v2:172.21.15.123:3300/0 conn(0x7f0f40154ff0 msgr2=0x7f0f4006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 --2- 172.21.15.64:0/1795323935 >> v2:172.21.15.123:3300/0 conn(0x7f0f40154ff0 0x7f0f4006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 -- 172.21.15.64:0/1795323935 >> v1:172.21.15.64:6789/0 conn(0x7f0f40154c20 legacy=0x7f0f400abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f44bdc700 1 -- 172.21.15.64:0/1795323935 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360637713 0 0) 0x7f0f38001490 con 0x7f0f40154c20 2023-09-27T23:14:02.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 -- 172.21.15.64:0/1795323935 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f400c5fd0 con 0x7f0f4006cd00 2023-09-27T23:14:02.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f463df700 1 --2- 172.21.15.64:0/1795323935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4006cd00 0x7f0f4006a580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f4006a030 tx=0x7f0f3c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e7e65b8bec1e44 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f44bdc700 1 -- 172.21.15.64:0/1795323935 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c018430 con 0x7f0f4006cd00 2023-09-27T23:14:02.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f44bdc700 1 -- 172.21.15.64:0/1795323935 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c018590 con 0x7f0f4006cd00 2023-09-27T23:14:02.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f44bdc700 1 -- 172.21.15.64:0/1795323935 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c018930 con 0x7f0f4006cd00 2023-09-27T23:14:02.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 -- 172.21.15.64:0/1795323935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4006cd00 msgr2=0x7f0f4006a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1795323935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4006cd00 0x7f0f4006a580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f4006a030 tx=0x7f0f3c0090f0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40440052b0 con 0x7f40640c68e0 2023-09-27T23:14:02.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 -- 172.21.15.64:0/1795323935 shutdown_connections 2023-09-27T23:14:02.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1795323935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4006cd00 0x7f0f4006a580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1795323935 >> v2:172.21.15.123:3300/0 conn(0x7f0f40154ff0 0x7f0f4006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 -- 172.21.15.64:0/1795323935 >> 172.21.15.64:0/1795323935 conn(0x7f0f40067bb0 msgr2=0x7f0f400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7fb20a0e2700 1 Processor -- start 2023-09-27T23:14:02.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 -- 172.21.15.64:0/1795323935 shutdown_connections 2023-09-27T23:14:02.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7f0f46be0700 1 -- 172.21.15.64:0/1795323935 wait complete. 2023-09-27T23:14:02.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.638+0000 7fb20a0e2700 1 -- start start 2023-09-27T23:14:02.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 Processor -- start 2023-09-27T23:14:02.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 -- start start 2023-09-27T23:14:02.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb20a0e2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb204152cd0 0x7fb2040c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb20a0e2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb2040cb4f0 0x7fb2040c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb20a0e2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20406b4e0 con 0x7fb204152900 2023-09-27T23:14:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb20a0e2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb204156220 con 0x7fb2040cb4f0 2023-09-27T23:14:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb20a0e2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb204155610 con 0x7fb204152cd0 2023-09-27T23:14:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb202ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb204152cd0 0x7fb2040c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb2037fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb204152900 0x7fb2040cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59752/0 (socket says 172.21.15.64:59752) 2023-09-27T23:14:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb2037fe700 1 -- 172.21.15.64:0/3594702929 learned_addr learned my addr 172.21.15.64:0/3594702929 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:02.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f4006cd00 0x7f0f40140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 0x7f0f4014ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f401579d0 0x7f0f40157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f400ca440 con 0x7f0f40154c20 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f400c9830 con 0x7f0f401579d0 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f46be0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f400c9b10 con 0x7f0f4006cd00 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 0x7f0f4014ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb202ffd700 1 -- 172.21.15.64:0/3594702929 >> v2:172.21.15.64:3300/0 conn(0x7fb2040cb4f0 msgr2=0x7fb2040c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb202ffd700 1 --2- 172.21.15.64:0/3594702929 >> v2:172.21.15.64:3300/0 conn(0x7fb2040cb4f0 0x7fb2040c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb202ffd700 1 -- 172.21.15.64:0/3594702929 >> v1:172.21.15.123:6789/0 conn(0x7fb204152900 legacy=0x7fb2040cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb202ffd700 1 -- 172.21.15.64:0/3594702929 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2041558f0 con 0x7fb204152cd0 2023-09-27T23:14:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 0x7f0f4014ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44774/0 (socket says 172.21.15.64:44774) 2023-09-27T23:14:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 -- 172.21.15.64:0/1753416015 learned_addr learned my addr 172.21.15.64:0/1753416015 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f463df700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f401579d0 0x7f0f40157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f45bde700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f4006cd00 0x7f0f40140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb202ffd700 1 --2- 172.21.15.64:0/3594702929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb204152cd0 0x7fb2040c7b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb204063450 tx=0x7fb1fc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37ddf2c8b965c5b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 -- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f4006cd00 msgr2=0x7f0f40140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f4006cd00 0x7f0f40140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 -- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f401579d0 msgr2=0x7f0f40157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f401579d0 0x7f0f40157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 -- 172.21.15.64:0/1753416015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c0107d0 con 0x7f0f40154c20 2023-09-27T23:14:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb2027fc700 1 -- 172.21.15.64:0/3594702929 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc00e070 con 0x7fb204152cd0 2023-09-27T23:14:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb2027fc700 1 -- 172.21.15.64:0/3594702929 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc00bc30 con 0x7fb204152cd0 2023-09-27T23:14:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7fb2027fc700 1 -- 172.21.15.64:0/3594702929 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc012430 con 0x7fb204152cd0 2023-09-27T23:14:02.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.642+0000 7f0f453dd700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 0x7f0f4014ecb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b810 tx=0x7f0f300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 -- 172.21.15.64:0/3594702929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb204152cd0 msgr2=0x7fb2040c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 --2- 172.21.15.64:0/3594702929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb204152cd0 0x7fb2040c7b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb204063450 tx=0x7fb1fc009780 comp rx=0 tx=0).stop 2023-09-27T23:14:02.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 -- 172.21.15.64:0/3594702929 shutdown_connections 2023-09-27T23:14:02.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 --2- 172.21.15.64:0/3594702929 >> v2:172.21.15.64:3300/0 conn(0x7fb2040cb4f0 0x7fb2040c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 --2- 172.21.15.64:0/3594702929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb204152cd0 0x7fb2040c7b40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 -- 172.21.15.64:0/3594702929 >> 172.21.15.64:0/3594702929 conn(0x7fb204067bb0 msgr2=0x7fb2040ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 -- 172.21.15.64:0/3594702929 shutdown_connections 2023-09-27T23:14:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 -- 172.21.15.64:0/3594702929 wait complete. 2023-09-27T23:14:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 Processor -- start 2023-09-27T23:14:02.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f406004d1a0 con 0x7f40640c68e0 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.646+0000 7fb20a0e2700 1 -- start start 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2040cb4f0 0x7fb204145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb204152900 0x7fb204147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 0x7fb20406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204149780 con 0x7fb2041485d0 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204148b70 con 0x7fb2040cb4f0 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb204148e50 con 0x7fb204152900 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 0x7fb20406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 0x7fb20406a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44776/0 (socket says 172.21.15.64:44776) 2023-09-27T23:14:02.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 -- 172.21.15.64:0/1231017943 learned_addr learned my addr 172.21.15.64:0/1231017943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb2037fe700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2040cb4f0 0x7fb204145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000f510 con 0x7f0f40154c20 2023-09-27T23:14:02.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3000f670 con 0x7f0f40154c20 2023-09-27T23:14:02.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f400c9df0 con 0x7f0f40154c20 2023-09-27T23:14:02.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f401414e0 con 0x7f0f40154c20 2023-09-27T23:14:02.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000fa30 con 0x7f0f40154c20 2023-09-27T23:14:02.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f0f3001a400 con 0x7f0f40154c20 2023-09-27T23:14:02.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f35ffb700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0f240609a0 0x7f0f24062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0f3007c300 con 0x7f0f40154c20 2023-09-27T23:14:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.658+0000 7f0f45bde700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0f240609a0 0x7f0f24062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 -- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb204152900 msgr2=0x7fb204147940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb204152900 0x7fb204147940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 -- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2040cb4f0 msgr2=0x7fb204145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2040cb4f0 0x7fb204145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 -- 172.21.15.64:0/1231017943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc009a20 con 0x7fb2041485d0 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb203fff700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 0x7fb20406a740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800b530 tx=0x7fb1f80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800f510 con 0x7fb2041485d0 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 -- 172.21.15.64:0/1231017943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb204149130 con 0x7fb2041485d0 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb20a0e2700 1 -- 172.21.15.64:0/1231017943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2041462b0 con 0x7fb2041485d0 2023-09-27T23:14:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f800f690 con 0x7fb2041485d0 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800fa50 con 0x7fb2041485d0 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.650+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb1f801a400 con 0x7fb2041485d0 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.654+0000 7fb1eb7fe700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1ec071b40 0x7fb1ec073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.654+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb1f807cb60 con 0x7fb2041485d0 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.654+0000 7fb2037fe700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1ec071b40 0x7fb1ec073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.654+0000 7fb2037fe700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1ec071b40 0x7fb1ec073ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4007130 tx=0x7fb1f4007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.662+0000 7f0f45bde700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0f240609a0 0x7f0f24062e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f38002130 tx=0x7f0f38001ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.662+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4004be40 con 0x7f0f40154c20 2023-09-27T23:14:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.662+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e40052b0 con 0x7fb2041485d0 2023-09-27T23:14:02.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.674+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0f30081060 con 0x7f0f40154c20 2023-09-27T23:14:02.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.686+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb1f804cc50 con 0x7fb2041485d0 2023-09-27T23:14:02.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664b59e700 1 Processor -- start 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664b59e700 1 -- start start 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f664c154c20 0x7f664c0abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0c5fd0 con 0x7f664c06ced0 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c06d3a0 con 0x7f664c154ff0 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0c5c70 con 0x7f664c154c20 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664ad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f664c06ced0 0x7f664c06a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34136/0 (socket says 172.21.15.64:34136) 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f664c154c20 0x7f664c0abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f664c154c20 0x7f664c0abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44792/0 (socket says 172.21.15.64:44792) 2023-09-27T23:14:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664ad9d700 1 -- 172.21.15.64:0/3836738600 learned_addr learned my addr 172.21.15.64:0/3836738600 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664a59c700 1 -- 172.21.15.64:0/3836738600 >> v1:172.21.15.123:6789/0 conn(0x7f664c154ff0 legacy=0x7f664c06c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-27T23:14:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664a59c700 1 -- 172.21.15.64:0/3836738600 >> v1:172.21.15.64:6789/0 conn(0x7f664c06ced0 legacy=0x7f664c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664a59c700 1 -- 172.21.15.64:0/3836738600 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c06d680 con 0x7f664c154c20 2023-09-27T23:14:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.690+0000 7f664959a700 1 -- 172.21.15.64:0/3836738600 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016404577 0 0) 0x7f664c0c5fd0 con 0x7f664c06ced0 2023-09-27T23:14:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664a59c700 1 --2- 172.21.15.64:0/3836738600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c154c20 0x7f664c0abca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f663c0098e0 tx=0x7f663c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bf0f327002fa0b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664959a700 1 -- 172.21.15.64:0/3836738600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c009e80 con 0x7f664c154c20 2023-09-27T23:14:02.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664959a700 1 -- 172.21.15.64:0/3836738600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c00b820 con 0x7f664c154c20 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 -- 172.21.15.64:0/3836738600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c154c20 msgr2=0x7f664c0abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 --2- 172.21.15.64:0/3836738600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c154c20 0x7f664c0abca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f663c0098e0 tx=0x7f663c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664959a700 1 -- 172.21.15.64:0/3836738600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c00bbc0 con 0x7f664c154c20 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 -- 172.21.15.64:0/3836738600 shutdown_connections 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 --2- 172.21.15.64:0/3836738600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c154c20 0x7f664c0abca0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 -- 172.21.15.64:0/3836738600 >> 172.21.15.64:0/3836738600 conn(0x7f664c067bb0 msgr2=0x7f664c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 -- 172.21.15.64:0/3836738600 shutdown_connections 2023-09-27T23:14:02.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 -- 172.21.15.64:0/3836738600 wait complete. 2023-09-27T23:14:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 Processor -- start 2023-09-27T23:14:02.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.694+0000 7f664b59e700 1 -- start start 2023-09-27T23:14:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 0x7f664c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f664c154c20 0x7f664c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f664c0b1730 0x7f664c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b0d60 con 0x7f664c06ced0 2023-09-27T23:14:02.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0af7d0 con 0x7f664c154c20 2023-09-27T23:14:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b0430 con 0x7f664c0b1730 2023-09-27T23:14:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 0x7f664c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 0x7f664c0b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44794/0 (socket says 172.21.15.64:44794) 2023-09-27T23:14:02.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 -- 172.21.15.64:0/3215832677 learned_addr learned my addr 172.21.15.64:0/3215832677 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664ad9d700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f664c0b1730 0x7f664c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f6649d9b700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f664c154c20 0x7f664c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 -- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f664c0b1730 msgr2=0x7f664c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f664c0b1730 0x7f664c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 -- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f664c154c20 msgr2=0x7f664c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f664c154c20 0x7f664c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 -- 172.21.15.64:0/3215832677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c009420 con 0x7f664c06ced0 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664a59c700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 0x7f664c0b11f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f663c0091e0 tx=0x7f663c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c022070 con 0x7f664c06ced0 2023-09-27T23:14:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c0b0710 con 0x7f664c06ced0 2023-09-27T23:14:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c137110 con 0x7f664c06ced0 2023-09-27T23:14:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c028420 con 0x7f664c06ced0 2023-09-27T23:14:02.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.698+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c0287c0 con 0x7f664c06ced0 2023-09-27T23:14:02.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.702+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f663c026070 con 0x7f664c06ced0 2023-09-27T23:14:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.702+0000 7f663a7fc700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6634060b40 0x7f6634062ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.702+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f663c082610 con 0x7f664c06ced0 2023-09-27T23:14:02.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.702+0000 7f6649d9b700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6634060b40 0x7f6634062ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.702+0000 7f6649d9b700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6634060b40 0x7f6634062ff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f664c154190 tx=0x7f6640009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.710+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c0052e0 con 0x7f664c06ced0 2023-09-27T23:14:02.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 Processor -- start 2023-09-27T23:14:02.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 -- start start 2023-09-27T23:14:02.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75240aa8e0 0x7f75240aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75240afe50 con 0x7f75240ae860 2023-09-27T23:14:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75240af240 con 0x7f75240aa8e0 2023-09-27T23:14:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.714+0000 7f7529164700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75240af520 con 0x7f75240aecc0 2023-09-27T23:14:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7523fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75240aa8e0 0x7f75240aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7523fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75240aa8e0 0x7f75240aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60866/0 (socket says 172.21.15.64:60866) 2023-09-27T23:14:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7523fff700 1 -- 172.21.15.64:0/3473304215 learned_addr learned my addr 172.21.15.64:0/3473304215 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7528963700 1 --2- 172.21.15.64:0/3473304215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7528963700 1 -- 172.21.15.64:0/3473304215 >> v2:172.21.15.123:3300/0 conn(0x7f75240aa8e0 msgr2=0x7f75240aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7528963700 1 --2- 172.21.15.64:0/3473304215 >> v2:172.21.15.123:3300/0 conn(0x7f75240aa8e0 0x7f75240aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7528963700 1 -- 172.21.15.64:0/3473304215 >> v1:172.21.15.123:6789/0 conn(0x7f75240ae860 legacy=0x7f752406a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7528963700 1 -- 172.21.15.64:0/3473304215 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75240af800 con 0x7f75240aecc0 2023-09-27T23:14:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f663c052620 con 0x7f664c06ced0 2023-09-27T23:14:02.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.718+0000 7f7528963700 1 --2- 172.21.15.64:0/3473304215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752406ac90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f751c009e60 tx=0x7f751c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1d77adcac3b44c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7522ffd700 1 -- 172.21.15.64:0/3473304215 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c00e070 con 0x7f75240aecc0 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7522ffd700 1 -- 172.21.15.64:0/3473304215 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c00ba60 con 0x7f75240aecc0 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7522ffd700 1 -- 172.21.15.64:0/3473304215 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c00be20 con 0x7f75240aecc0 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 -- 172.21.15.64:0/3473304215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 msgr2=0x7f752406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 --2- 172.21.15.64:0/3473304215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752406ac90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f751c009e60 tx=0x7f751c009780 comp rx=0 tx=0).stop 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 -- 172.21.15.64:0/3473304215 shutdown_connections 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 --2- 172.21.15.64:0/3473304215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752406ac90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 --2- 172.21.15.64:0/3473304215 >> v2:172.21.15.123:3300/0 conn(0x7f75240aa8e0 0x7f75240aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 -- 172.21.15.64:0/3473304215 >> 172.21.15.64:0/3473304215 conn(0x7f7524067bb0 msgr2=0x7f75240ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 -- 172.21.15.64:0/3473304215 shutdown_connections 2023-09-27T23:14:02.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 -- 172.21.15.64:0/3473304215 wait complete. 2023-09-27T23:14:02.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 Processor -- start 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.722+0000 7f7529164700 1 -- start start 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7529164700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75240aa8e0 0x7f7524144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7529164700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 0x7f7524147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7529164700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752414adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7529164700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7524148c10 con 0x7f75240ae860 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7529164700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7524148000 con 0x7f75240aa8e0 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7529164700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75241482e0 con 0x7f75240aecc0 2023-09-27T23:14:02.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 0x7f7524147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 0x7f7524147700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44798/0 (socket says 172.21.15.64:44798) 2023-09-27T23:14:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 -- 172.21.15.64:0/2983237255 learned_addr learned my addr 172.21.15.64:0/2983237255 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7523fff700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75240aa8e0 0x7f7524144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f7528963700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752414adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 -- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 msgr2=0x7f752414adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752414adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 -- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75240aa8e0 msgr2=0x7f7524144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75240aa8e0 0x7f7524144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 -- 172.21.15.64:0/2983237255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009a20 con 0x7f75240ae860 2023-09-27T23:14:02.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.726+0000 7f75237fe700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 0x7f7524147700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f751800c810 tx=0x7f7518007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.730+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518012630 con 0x7f75240ae860 2023-09-27T23:14:02.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.730+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75180127b0 con 0x7f75240ae860 2023-09-27T23:14:02.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.730+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75241485c0 con 0x7f75240ae860 2023-09-27T23:14:02.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.730+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75241490a0 con 0x7f75240ae860 2023-09-27T23:14:02.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.730+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518012b70 con 0x7f75240ae860 2023-09-27T23:14:02.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.734+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f751801b400 con 0x7f75240ae860 2023-09-27T23:14:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.734+0000 7f750bfff700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7510060a20 0x7f7510062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:02.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.734+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f751807cd60 con 0x7f75240ae860 2023-09-27T23:14:02.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.734+0000 7f7523fff700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7510060a20 0x7f7510062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:02.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.734+0000 7f7523fff700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7510060a20 0x7f7510062ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7514006d40 tx=0x7f751400d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.742+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75040052b0 con 0x7f75240ae860 2023-09-27T23:14:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.750+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f751804ccd0 con 0x7f75240ae860 2023-09-27T23:14:02.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.870+0000 7fb5eb59e700 1 -- 172.21.15.64:0/4233157576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb5d0005560 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.870+0000 7fb5da7fc700 1 -- 172.21.15.64:0/4233157576 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5e40522d0 con 0x7fb5ec06c7c0 2023-09-27T23:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 -- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb5cc060ad0 msgr2=0x7fb5cc062f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb5cc060ad0 0x7fb5cc062f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec1498e0 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 -- 172.21.15.64:0/4233157576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 msgr2=0x7fb5ec147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 0x7fb5ec147180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec064040 tx=0x7fb5e4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 -- 172.21.15.64:0/4233157576 shutdown_connections 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb5cc060ad0 0x7fb5cc062f80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5ec155960 0x7fb5ec142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5ec06de20 0x7fb5ec0c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 --2- 172.21.15.64:0/4233157576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5ec06c7c0 0x7fb5ec147180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 -- 172.21.15.64:0/4233157576 >> 172.21.15.64:0/4233157576 conn(0x7fb5ec067bb0 msgr2=0x7fb5ec06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 -- 172.21.15.64:0/4233157576 shutdown_connections 2023-09-27T23:14:02.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.878+0000 7fb5cbfff700 1 -- 172.21.15.64:0/4233157576 wait complete. 2023-09-27T23:14:02.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2023-09-27T23:14:02.896 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.902+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f542c06dc80 con 0x7f542c06cd00 2023-09-27T23:14:02.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.906+0000 7f5413fff700 1 -- 172.21.15.64:0/2254766973 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5424018300 con 0x7f542c06cd00 2023-09-27T23:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:14:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5418060950 msgr2=0x7f5418062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5418060950 0x7f5418062e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f54200091e0 tx=0x7f5420016000 comp rx=0 tx=0).stop 2023-09-27T23:14:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 msgr2=0x7f542c0b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 0x7f542c0b1320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f542400cb10 tx=0x7f54240027c0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 shutdown_connections 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5418060950 0x7f5418062e00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f542c173000 0x7f542c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542c154c20 0x7f542c172ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 --2- 172.21.15.64:0/2254766973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f542c06cd00 0x7f542c0b1320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 >> 172.21.15.64:0/2254766973 conn(0x7f542c067bb0 msgr2=0x7f542c0c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 shutdown_connections 2023-09-27T23:14:02.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.910+0000 7f54310d8700 1 -- 172.21.15.64:0/2254766973 wait complete. 2023-09-27T23:14:02.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8330005830 con 0x7f83500ca720 2023-09-27T23:14:02.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f834cff9700 1 -- 172.21.15.64:0/4216846783 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8348050d20 con 0x7f83500ca720 2023-09-27T23:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f833c060a20 msgr2=0x7f833c062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f833c060a20 0x7f833c062ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8340006fd0 tx=0x7f8340008040 comp rx=0 tx=0).stop 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 msgr2=0x7f8350179500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 0x7f8350179500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8348009fd0 tx=0x7f8348009900 comp rx=0 tx=0).stop 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 shutdown_connections 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f833c060a20 0x7f833c062ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83500ca720 0x7f8350179500 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8350154c20 0x7f8350178fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 --2- 172.21.15.64:0/4216846783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835006cd00 0x7f83500ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.918+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 >> 172.21.15.64:0/4216846783 conn(0x7f8350067bb0 msgr2=0x7f835006b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.922+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 shutdown_connections 2023-09-27T23:14:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.922+0000 7f8355eb6700 1 -- 172.21.15.64:0/4216846783 wait complete. 2023-09-27T23:14:02.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2023-09-27T23:14:02.927 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:02.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2023-09-27T23:14:02.938 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:02.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.934+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4044005560 con 0x7f40640c68e0 2023-09-27T23:14:02.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.934+0000 7f40597fa700 1 -- 172.21.15.64:0/1634851205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4060017090 con 0x7f40640c68e0 2023-09-27T23:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout:64424509447 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4048060ad0 msgr2=0x7f4048062f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4048060ad0 0x7f4048062f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4054003a60 tx=0x7f4054007ec0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 msgr2=0x7f4064157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 0x7f4064157e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f406000ecb0 tx=0x7f40600091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 shutdown_connections 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4048060ad0 0x7f4048062f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40641618c0 0x7f406413b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40640c68e0 0x7f4064157e70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 --2- 172.21.15.64:0/1634851205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40640ac110 0x7f40640ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 >> 172.21.15.64:0/1634851205 conn(0x7f4064067bb0 msgr2=0x7f406413eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 shutdown_connections 2023-09-27T23:14:02.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.938+0000 7f406a015700 1 -- 172.21.15.64:0/1634851205 wait complete. 2023-09-27T23:14:02.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.946+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb1e4005830 con 0x7fb2041485d0 2023-09-27T23:14:02.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.946+0000 7fb1eb7fe700 1 -- 172.21.15.64:0/1231017943 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1f8050270 con 0x7fb2041485d0 2023-09-27T23:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1ec071b40 msgr2=0x7fb1ec073ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1ec071b40 0x7fb1ec073ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4007130 tx=0x7fb1f4007eb0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 msgr2=0x7fb20406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 0x7fb20406a740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800b530 tx=0x7fb1f80090f0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 shutdown_connections 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1ec071b40 0x7fb1ec073ff0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2041485d0 0x7fb20406a740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb204152900 0x7fb204147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 --2- 172.21.15.64:0/1231017943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2040cb4f0 0x7fb204145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 >> 172.21.15.64:0/1231017943 conn(0x7fb204067bb0 msgr2=0x7fb204156850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 shutdown_connections 2023-09-27T23:14:02.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.950+0000 7fb1e97fa700 1 -- 172.21.15.64:0/1231017943 wait complete. 2023-09-27T23:14:02.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2023-09-27T23:14:02.957 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:02.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2023-09-27T23:14:02.964 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:02.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0f400ca580 con 0x7f0f40154c20 2023-09-27T23:14:02.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f35ffb700 1 -- 172.21.15.64:0/1753416015 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f3004c300 con 0x7f0f40154c20 2023-09-27T23:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout:64424509449 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0f240609a0 msgr2=0x7f0f24062e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0f240609a0 0x7f0f24062e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f38002130 tx=0x7f0f38001ad0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 msgr2=0x7f0f4014ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 0x7f0f4014ecb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b810 tx=0x7f0f300091d0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 shutdown_connections 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0f240609a0 0x7f0f24062e50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f401579d0 0x7f0f40157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f40154c20 0x7f0f4014ecb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 --2- 172.21.15.64:0/1753416015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f4006cd00 0x7f0f40140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 >> 172.21.15.64:0/1753416015 conn(0x7f0f40067bb0 msgr2=0x7f0f400c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 shutdown_connections 2023-09-27T23:14:02.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.966+0000 7f0f46be0700 1 -- 172.21.15.64:0/1753416015 wait complete. 2023-09-27T23:14:02.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2023-09-27T23:14:02.981 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:02.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f662c005860 con 0x7f664c06ced0 2023-09-27T23:14:02.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f663a7fc700 1 -- 172.21.15.64:0/3215832677 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f663c02d070 con 0x7f664c06ced0 2023-09-27T23:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout:60129542153 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6634060b40 msgr2=0x7f6634062ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6634060b40 0x7f6634062ff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f664c154190 tx=0x7f6640009060 comp rx=0 tx=0).stop 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 msgr2=0x7f664c0b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 0x7f664c0b11f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f663c0091e0 tx=0x7f663c016000 comp rx=0 tx=0).stop 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 shutdown_connections 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6634060b40 0x7f6634062ff0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f664c0b1730 0x7f664c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f664c154c20 0x7f664c156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 --2- 172.21.15.64:0/3215832677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f664c06ced0 0x7f664c0b11f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 >> 172.21.15.64:0/3215832677 conn(0x7f664c067bb0 msgr2=0x7f664c0ac750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 shutdown_connections 2023-09-27T23:14:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.986+0000 7f664b59e700 1 -- 172.21.15.64:0/3215832677 wait complete. 2023-09-27T23:14:03.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.1 2023-09-27T23:14:03.001 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:03.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7504005560 con 0x7f75240ae860 2023-09-27T23:14:03.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f750bfff700 1 -- 172.21.15.64:0/2983237255 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75180502f0 con 0x7f75240ae860 2023-09-27T23:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout:64424509448 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7510060a20 msgr2=0x7f7510062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7510060a20 0x7f7510062ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7514006d40 tx=0x7f751400d040 comp rx=0 tx=0).stop 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 msgr2=0x7f7524147700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 0x7f7524147700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f751800c810 tx=0x7f7518007ee0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 shutdown_connections 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7510060a20 0x7f7510062ed0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75240aecc0 0x7f752414adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75240ae860 0x7f7524147700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 --2- 172.21.15.64:0/2983237255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75240aa8e0 0x7f7524144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:02.998+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 >> 172.21.15.64:0/2983237255 conn(0x7f7524067bb0 msgr2=0x7f75241577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.002+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 shutdown_connections 2023-09-27T23:14:03.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.002+0000 7f7529164700 1 -- 172.21.15.64:0/2983237255 wait complete. 2023-09-27T23:14:03.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2023-09-27T23:14:03.014 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:03.015 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-27T23:14:03.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.154+0000 7f07ed55e700 1 Processor -- start 2023-09-27T23:14:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- start start 2023-09-27T23:14:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07e8158df0 0x7f07e806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e8159cd0 con 0x7f07e81598d0 2023-09-27T23:14:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e8159f90 con 0x7f07e806d590 2023-09-27T23:14:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e806da10 con 0x7f07e8158df0 2023-09-27T23:14:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ecd5d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f07e81598d0 0x7f07e80c8130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34152/0 (socket says 172.21.15.64:34152) 2023-09-27T23:14:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ecd5d700 1 -- 172.21.15.64:0/986472389 learned_addr learned my addr 172.21.15.64:0/986472389 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e7fff700 1 --2- 172.21.15.64:0/986472389 >> v2:172.21.15.64:3300/0 conn(0x7f07e8158df0 0x7f07e806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234204375 0 0) 0x7f07e8159cd0 con 0x7f07e81598d0 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d0003570 con 0x7f07e81598d0 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238643108 0 0) 0x7f07e8159f90 con 0x7f07e806d590 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e8159cd0 con 0x7f07e806d590 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1785328317 0 0) 0x7f07d0003570 con 0x7f07e81598d0 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e7fff700 1 -- 172.21.15.64:0/986472389 >> v1:172.21.15.123:6789/0 conn(0x7f07e806d590 legacy=0x7f07e81591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e7fff700 1 -- 172.21.15.64:0/986472389 >> v1:172.21.15.64:6789/0 conn(0x7f07e81598d0 legacy=0x7f07e80c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e7fff700 1 -- 172.21.15.64:0/986472389 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e815a270 con 0x7f07e8158df0 2023-09-27T23:14:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e7fff700 1 --2- 172.21.15.64:0/986472389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e806d050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07d80098e0 tx=0x7f07d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898ada6c4feabfd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00017f0 con 0x7f07e8158df0 2023-09-27T23:14:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8011420 con 0x7f07e8158df0 2023-09-27T23:14:03.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07e6ffd700 1 -- 172.21.15.64:0/986472389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80117c0 con 0x7f07e8158df0 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- 172.21.15.64:0/986472389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 msgr2=0x7f07e806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 --2- 172.21.15.64:0/986472389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e806d050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07d80098e0 tx=0x7f07d80091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- 172.21.15.64:0/986472389 shutdown_connections 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 --2- 172.21.15.64:0/986472389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e806d050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- 172.21.15.64:0/986472389 >> 172.21.15.64:0/986472389 conn(0x7f07e806c080 msgr2=0x7f07e80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- 172.21.15.64:0/986472389 shutdown_connections 2023-09-27T23:14:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.158+0000 7f07ed55e700 1 -- 172.21.15.64:0/986472389 wait complete. 2023-09-27T23:14:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 Processor -- start 2023-09-27T23:14:03.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 -- start start 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e806d590 0x7f07e80b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07e81598d0 0x7f07e815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b2200 con 0x7f07e8158df0 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80c4420 con 0x7f07e806d590 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ed55e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b1c40 con 0x7f07e81598d0 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e815c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44812/0 (socket says 172.21.15.64:44812) 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 -- 172.21.15.64:0/1178587437 learned_addr learned my addr 172.21.15.64:0/1178587437 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e7fff700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e806d590 0x7f07e80b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07ecd5d700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07e81598d0 0x7f07e815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 -- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07e81598d0 msgr2=0x7f07e815c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07e81598d0 0x7f07e815c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 -- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e806d590 msgr2=0x7f07e80b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e806d590 0x7f07e80b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 -- 172.21.15.64:0/1178587437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8009420 con 0x7f07e8158df0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07e77fe700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e815c250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00c580 tx=0x7f07dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.162+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc010510 con 0x7f07e8158df0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e80b1f20 con 0x7f07e8158df0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc010670 con 0x7f07e8158df0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc010a30 con 0x7f07e8158df0 2023-09-27T23:14:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e80c6a00 con 0x7f07e8158df0 2023-09-27T23:14:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e80c3f10 con 0x7f07e8158df0 2023-09-27T23:14:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f07dc01b400 con 0x7f07e8158df0 2023-09-27T23:14:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07cffff700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07d4060950 0x7f07d4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f07dc07ce30 con 0x7f07e8158df0 2023-09-27T23:14:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07e7fff700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07d4060950 0x7f07d4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.166+0000 7f07e7fff700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07d4060950 0x7f07d4062e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07d80093f0 tx=0x7f07d8016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.174+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f07e80c3f10 con 0x7f07e8158df0 2023-09-27T23:14:03.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.322+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07e80c6c80 con 0x7f07d4060950 2023-09-27T23:14:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.342+0000 7f07cffff700 1 -- 172.21.15.64:0/1178587437 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+364695 (secure 0 0 0) 0x7f07e80c6c80 con 0x7f07d4060950 2023-09-27T23:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07d4060950 msgr2=0x7f07d4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07d4060950 0x7f07d4062e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07d80093f0 tx=0x7f07d8016000 comp rx=0 tx=0).stop 2023-09-27T23:14:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 msgr2=0x7f07e815c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e815c250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00c580 tx=0x7f07dc007e10 comp rx=0 tx=0).stop 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 shutdown_connections 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07d4060950 0x7f07d4062e00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07e81598d0 0x7f07e815c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e8158df0 0x7f07e815c250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 --2- 172.21.15.64:0/1178587437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e806d590 0x7f07e80b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 >> 172.21.15.64:0/1178587437 conn(0x7f07e806c080 msgr2=0x7f07e80c8030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 shutdown_connections 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.350+0000 7f07ed55e700 1 -- 172.21.15.64:0/1178587437 wait complete. 2023-09-27T23:14:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-09-27T23:14:03.150421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67688,"kb_used_data":3624,"kb_used_omap":0,"kb_used_meta":64000,"kb_avail":754907032,"statfs":{"total":773094113280,"available":773024800768,"internally_reserved":0,"allocated":3710976,"data_stored":2188832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65536000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005543"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838997+0000","last_change":"2023-09-27T23:13:48.825219+0000","last_active":"2023-09-27T23:13:48.838997+0000","last_peered":"2023-09-27T23:13:48.838997+0000","last_clean":"2023-09-27T23:13:48.838997+0000","last_became_active":"2023-09-27T23:13:48.824822+0000","last_became_peered":"2023-09-27T23:13:48.824822+0000","last_unstale":"2023-09-27T23:13:48.838997+0000","last_undegraded":"2023-09-27T23:13:48.838997+0000","last_fullsized":"2023-09-27T23:13:48.838997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:34.128831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803408+0000","last_change":"2023-09-27T23:13:44.424249+0000","last_active":"2023-09-27T23:13:48.803408+0000","last_peered":"2023-09-27T23:13:48.803408+0000","last_clean":"2023-09-27T23:13:48.803408+0000","last_became_active":"2023-09-27T23:13:44.423915+0000","last_became_peered":"2023-09-27T23:13:44.423915+0000","last_unstale":"2023-09-27T23:13:48.803408+0000","last_undegraded":"2023-09-27T23:13:48.803408+0000","last_fullsized":"2023-09-27T23:13:48.803408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821460+0000","last_change":"2023-09-27T23:13:44.796968+0000","last_active":"2023-09-27T23:13:48.821460+0000","last_peered":"2023-09-27T23:13:48.821460+0000","last_clean":"2023-09-27T23:13:48.821460+0000","last_became_active":"2023-09-27T23:13:44.795519+0000","last_became_peered":"2023-09-27T23:13:44.795519+0000","last_unstale":"2023-09-27T23:13:48.821460+0000","last_undegraded":"2023-09-27T23:13:48.821460+0000","last_fullsized":"2023-09-27T23:13:48.821460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843929+0000","last_change":"2023-09-27T23:13:48.839540+0000","last_active":"2023-09-27T23:13:48.843929+0000","last_peered":"2023-09-27T23:13:48.843929+0000","last_clean":"2023-09-27T23:13:48.843929+0000","last_became_active":"2023-09-27T23:13:48.839079+0000","last_became_peered":"2023-09-27T23:13:48.839079+0000","last_unstale":"2023-09-27T23:13:48.843929+0000","last_undegraded":"2023-09-27T23:13:48.843929+0000","last_fullsized":"2023-09-27T23:13:48.843929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825433+0000","last_change":"2023-09-27T23:13:44.416457+0000","last_active":"2023-09-27T23:13:48.825433+0000","last_peered":"2023-09-27T23:13:48.825433+0000","last_clean":"2023-09-27T23:13:48.825433+0000","last_became_active":"2023-09-27T23:13:44.416135+0000","last_became_peered":"2023-09-27T23:13:44.416135+0000","last_unstale":"2023-09-27T23:13:48.825433+0000","last_undegraded":"2023-09-27T23:13:48.825433+0000","last_fullsized":"2023-09-27T23:13:48.825433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822547+0000","last_change":"2023-09-27T23:13:44.783780+0000","last_active":"2023-09-27T23:13:48.822547+0000","last_peered":"2023-09-27T23:13:48.822547+0000","last_clean":"2023-09-27T23:13:48.822547+0000","last_became_active":"2023-09-27T23:13:44.783441+0000","last_became_peered":"2023-09-27T23:13:44.783441+0000","last_unstale":"2023-09-27T23:13:48.822547+0000","last_undegraded":"2023-09-27T23:13:48.822547+0000","last_fullsized":"2023-09-27T23:13:48.822547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840431+0000","last_change":"2023-09-27T23:13:48.835419+0000","last_active":"2023-09-27T23:13:48.840431+0000","last_peered":"2023-09-27T23:13:48.840431+0000","last_clean":"2023-09-27T23:13:48.840431+0000","last_became_active":"2023-09-27T23:13:48.834981+0000","last_became_peered":"2023-09-27T23:13:48.834981+0000","last_unstale":"2023-09-27T23:13:48.840431+0000","last_undegraded":"2023-09-27T23:13:48.840431+0000","last_fullsized":"2023-09-27T23:13:48.840431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814741+0000","last_change":"2023-09-27T23:13:44.427390+0000","last_active":"2023-09-27T23:13:48.814741+0000","last_peered":"2023-09-27T23:13:48.814741+0000","last_clean":"2023-09-27T23:13:48.814741+0000","last_became_active":"2023-09-27T23:13:44.426407+0000","last_became_peered":"2023-09-27T23:13:44.426407+0000","last_unstale":"2023-09-27T23:13:48.814741+0000","last_undegraded":"2023-09-27T23:13:48.814741+0000","last_fullsized":"2023-09-27T23:13:48.814741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820888+0000","last_change":"2023-09-27T23:13:44.803765+0000","last_active":"2023-09-27T23:13:48.820888+0000","last_peered":"2023-09-27T23:13:48.820888+0000","last_clean":"2023-09-27T23:13:48.820888+0000","last_became_active":"2023-09-27T23:13:44.803448+0000","last_became_peered":"2023-09-27T23:13:44.803448+0000","last_unstale":"2023-09-27T23:13:48.820888+0000","last_undegraded":"2023-09-27T23:13:48.820888+0000","last_fullsized":"2023-09-27T23:13:48.820888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:14.091887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834465+0000","last_change":"2023-09-27T23:13:48.831362+0000","last_active":"2023-09-27T23:13:48.834465+0000","last_peered":"2023-09-27T23:13:48.834465+0000","last_clean":"2023-09-27T23:13:48.834465+0000","last_became_active":"2023-09-27T23:13:48.830786+0000","last_became_peered":"2023-09-27T23:13:48.830786+0000","last_unstale":"2023-09-27T23:13:48.834465+0000","last_undegraded":"2023-09-27T23:13:48.834465+0000","last_fullsized":"2023-09-27T23:13:48.834465+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:24:25.298387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811157+0000","last_change":"2023-09-27T23:13:44.425581+0000","last_active":"2023-09-27T23:13:48.811157+0000","last_peered":"2023-09-27T23:13:48.811157+0000","last_clean":"2023-09-27T23:13:48.811157+0000","last_became_active":"2023-09-27T23:13:44.425084+0000","last_became_peered":"2023-09-27T23:13:44.425084+0000","last_unstale":"2023-09-27T23:13:48.811157+0000","last_undegraded":"2023-09-27T23:13:48.811157+0000","last_fullsized":"2023-09-27T23:13:48.811157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821059+0000","last_change":"2023-09-27T23:13:44.803698+0000","last_active":"2023-09-27T23:13:48.821059+0000","last_peered":"2023-09-27T23:13:48.821059+0000","last_clean":"2023-09-27T23:13:48.821059+0000","last_became_active":"2023-09-27T23:13:44.803316+0000","last_became_peered":"2023-09-27T23:13:44.803316+0000","last_unstale":"2023-09-27T23:13:48.821059+0000","last_undegraded":"2023-09-27T23:13:48.821059+0000","last_fullsized":"2023-09-27T23:13:48.821059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:10:53.031343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843374+0000","last_change":"2023-09-27T23:13:48.835746+0000","last_active":"2023-09-27T23:13:48.843374+0000","last_peered":"2023-09-27T23:13:48.843374+0000","last_clean":"2023-09-27T23:13:48.843374+0000","last_became_active":"2023-09-27T23:13:48.835306+0000","last_became_peered":"2023-09-27T23:13:48.835306+0000","last_unstale":"2023-09-27T23:13:48.843374+0000","last_undegraded":"2023-09-27T23:13:48.843374+0000","last_fullsized":"2023-09-27T23:13:48.843374+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:00:48.785735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804887+0000","last_change":"2023-09-27T23:13:44.428693+0000","last_active":"2023-09-27T23:13:48.804887+0000","last_peered":"2023-09-27T23:13:48.804887+0000","last_clean":"2023-09-27T23:13:48.804887+0000","last_became_active":"2023-09-27T23:13:44.428213+0000","last_became_peered":"2023-09-27T23:13:44.428213+0000","last_unstale":"2023-09-27T23:13:48.804887+0000","last_undegraded":"2023-09-27T23:13:48.804887+0000","last_fullsized":"2023-09-27T23:13:48.804887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810898+0000","last_change":"2023-09-27T23:13:44.791611+0000","last_active":"2023-09-27T23:13:48.810898+0000","last_peered":"2023-09-27T23:13:48.810898+0000","last_clean":"2023-09-27T23:13:48.810898+0000","last_became_active":"2023-09-27T23:13:44.791292+0000","last_became_peered":"2023-09-27T23:13:44.791292+0000","last_unstale":"2023-09-27T23:13:48.810898+0000","last_undegraded":"2023-09-27T23:13:48.810898+0000","last_fullsized":"2023-09-27T23:13:48.810898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:08.690781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840009+0000","last_change":"2023-09-27T23:13:48.827264+0000","last_active":"2023-09-27T23:13:48.840009+0000","last_peered":"2023-09-27T23:13:48.840009+0000","last_clean":"2023-09-27T23:13:48.840009+0000","last_became_active":"2023-09-27T23:13:48.826832+0000","last_became_peered":"2023-09-27T23:13:48.826832+0000","last_unstale":"2023-09-27T23:13:48.840009+0000","last_undegraded":"2023-09-27T23:13:48.840009+0000","last_fullsized":"2023-09-27T23:13:48.840009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:48:20.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821181+0000","last_change":"2023-09-27T23:13:44.423920+0000","last_active":"2023-09-27T23:13:48.821181+0000","last_peered":"2023-09-27T23:13:48.821181+0000","last_clean":"2023-09-27T23:13:48.821181+0000","last_became_active":"2023-09-27T23:13:44.422754+0000","last_became_peered":"2023-09-27T23:13:44.422754+0000","last_unstale":"2023-09-27T23:13:48.821181+0000","last_undegraded":"2023-09-27T23:13:48.821181+0000","last_fullsized":"2023-09-27T23:13:48.821181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:16.888839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825025+0000","last_change":"2023-09-27T23:13:44.790022+0000","last_active":"2023-09-27T23:13:48.825025+0000","last_peered":"2023-09-27T23:13:48.825025+0000","last_clean":"2023-09-27T23:13:48.825025+0000","last_became_active":"2023-09-27T23:13:44.789052+0000","last_became_peered":"2023-09-27T23:13:44.789052+0000","last_unstale":"2023-09-27T23:13:48.825025+0000","last_undegraded":"2023-09-27T23:13:48.825025+0000","last_fullsized":"2023-09-27T23:13:48.825025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.841311+0000","last_change":"2023-09-27T23:13:48.835634+0000","last_active":"2023-09-27T23:13:48.841311+0000","last_peered":"2023-09-27T23:13:48.841311+0000","last_clean":"2023-09-27T23:13:48.841311+0000","last_became_active":"2023-09-27T23:13:48.834989+0000","last_became_peered":"2023-09-27T23:13:48.834989+0000","last_unstale":"2023-09-27T23:13:48.841311+0000","last_undegraded":"2023-09-27T23:13:48.841311+0000","last_fullsized":"2023-09-27T23:13:48.841311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:09.944701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802920+0000","last_change":"2023-09-27T23:13:44.424671+0000","last_active":"2023-09-27T23:13:48.802920+0000","last_peered":"2023-09-27T23:13:48.802920+0000","last_clean":"2023-09-27T23:13:48.802920+0000","last_became_active":"2023-09-27T23:13:44.424321+0000","last_became_peered":"2023-09-27T23:13:44.424321+0000","last_unstale":"2023-09-27T23:13:48.802920+0000","last_undegraded":"2023-09-27T23:13:48.802920+0000","last_fullsized":"2023-09-27T23:13:48.802920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:44:43.453425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803496+0000","last_change":"2023-09-27T23:13:44.792445+0000","last_active":"2023-09-27T23:13:48.803496+0000","last_peered":"2023-09-27T23:13:48.803496+0000","last_clean":"2023-09-27T23:13:48.803496+0000","last_became_active":"2023-09-27T23:13:44.790669+0000","last_became_peered":"2023-09-27T23:13:44.790669+0000","last_unstale":"2023-09-27T23:13:48.803496+0000","last_undegraded":"2023-09-27T23:13:48.803496+0000","last_fullsized":"2023-09-27T23:13:48.803496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833795+0000","last_change":"2023-09-27T23:13:48.824674+0000","last_active":"2023-09-27T23:13:48.833795+0000","last_peered":"2023-09-27T23:13:48.833795+0000","last_clean":"2023-09-27T23:13:48.833795+0000","last_became_active":"2023-09-27T23:13:48.824220+0000","last_became_peered":"2023-09-27T23:13:48.824220+0000","last_unstale":"2023-09-27T23:13:48.833795+0000","last_undegraded":"2023-09-27T23:13:48.833795+0000","last_fullsized":"2023-09-27T23:13:48.833795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822020+0000","last_change":"2023-09-27T23:13:44.415940+0000","last_active":"2023-09-27T23:13:48.822020+0000","last_peered":"2023-09-27T23:13:48.822020+0000","last_clean":"2023-09-27T23:13:48.822020+0000","last_became_active":"2023-09-27T23:13:44.415355+0000","last_became_peered":"2023-09-27T23:13:44.415355+0000","last_unstale":"2023-09-27T23:13:48.822020+0000","last_undegraded":"2023-09-27T23:13:48.822020+0000","last_fullsized":"2023-09-27T23:13:48.822020+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813805+0000","last_change":"2023-09-27T23:13:44.807290+0000","last_active":"2023-09-27T23:13:48.813805+0000","last_peered":"2023-09-27T23:13:48.813805+0000","last_clean":"2023-09-27T23:13:48.813805+0000","last_became_active":"2023-09-27T23:13:44.806991+0000","last_became_peered":"2023-09-27T23:13:44.806991+0000","last_unstale":"2023-09-27T23:13:48.813805+0000","last_undegraded":"2023-09-27T23:13:48.813805+0000","last_fullsized":"2023-09-27T23:13:48.813805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:41.102884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821934+0000","last_change":"2023-09-27T23:13:44.797997+0000","last_active":"2023-09-27T23:13:48.821934+0000","last_peered":"2023-09-27T23:13:48.821934+0000","last_clean":"2023-09-27T23:13:48.821934+0000","last_became_active":"2023-09-27T23:13:44.797689+0000","last_became_peered":"2023-09-27T23:13:44.797689+0000","last_unstale":"2023-09-27T23:13:48.821934+0000","last_undegraded":"2023-09-27T23:13:48.821934+0000","last_fullsized":"2023-09-27T23:13:48.821934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803641+0000","last_change":"2023-09-27T23:13:44.420293+0000","last_active":"2023-09-27T23:13:48.803641+0000","last_peered":"2023-09-27T23:13:48.803641+0000","last_clean":"2023-09-27T23:13:48.803641+0000","last_became_active":"2023-09-27T23:13:44.419584+0000","last_became_peered":"2023-09-27T23:13:44.419584+0000","last_unstale":"2023-09-27T23:13:48.803641+0000","last_undegraded":"2023-09-27T23:13:48.803641+0000","last_fullsized":"2023-09-27T23:13:48.803641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843828+0000","last_change":"2023-09-27T23:13:48.842799+0000","last_active":"2023-09-27T23:13:48.843828+0000","last_peered":"2023-09-27T23:13:48.843828+0000","last_clean":"2023-09-27T23:13:48.843828+0000","last_became_active":"2023-09-27T23:13:48.842271+0000","last_became_peered":"2023-09-27T23:13:48.842271+0000","last_unstale":"2023-09-27T23:13:48.843828+0000","last_undegraded":"2023-09-27T23:13:48.843828+0000","last_fullsized":"2023-09-27T23:13:48.843828+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:26.172918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822463+0000","last_change":"2023-09-27T23:13:44.791363+0000","last_active":"2023-09-27T23:13:48.822463+0000","last_peered":"2023-09-27T23:13:48.822463+0000","last_clean":"2023-09-27T23:13:48.822463+0000","last_became_active":"2023-09-27T23:13:44.786248+0000","last_became_peered":"2023-09-27T23:13:44.786248+0000","last_unstale":"2023-09-27T23:13:48.822463+0000","last_undegraded":"2023-09-27T23:13:48.822463+0000","last_fullsized":"2023-09-27T23:13:48.822463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:22.779025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815748+0000","last_change":"2023-09-27T23:13:44.416087+0000","last_active":"2023-09-27T23:13:48.815748+0000","last_peered":"2023-09-27T23:13:48.815748+0000","last_clean":"2023-09-27T23:13:48.815748+0000","last_became_active":"2023-09-27T23:13:44.415755+0000","last_became_peered":"2023-09-27T23:13:44.415755+0000","last_unstale":"2023-09-27T23:13:48.815748+0000","last_undegraded":"2023-09-27T23:13:48.815748+0000","last_fullsized":"2023-09-27T23:13:48.815748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826650+0000","last_change":"2023-09-27T23:13:48.825181+0000","last_active":"2023-09-27T23:13:48.826650+0000","last_peered":"2023-09-27T23:13:48.826650+0000","last_clean":"2023-09-27T23:13:48.826650+0000","last_became_active":"2023-09-27T23:13:48.824750+0000","last_became_peered":"2023-09-27T23:13:48.824750+0000","last_unstale":"2023-09-27T23:13:48.826650+0000","last_undegraded":"2023-09-27T23:13:48.826650+0000","last_fullsized":"2023-09-27T23:13:48.826650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803126+0000","last_change":"2023-09-27T23:13:44.423841+0000","last_active":"2023-09-27T23:13:48.803126+0000","last_peered":"2023-09-27T23:13:48.803126+0000","last_clean":"2023-09-27T23:13:48.803126+0000","last_became_active":"2023-09-27T23:13:44.423450+0000","last_became_peered":"2023-09-27T23:13:44.423450+0000","last_unstale":"2023-09-27T23:13:48.803126+0000","last_undegraded":"2023-09-27T23:13:48.803126+0000","last_fullsized":"2023-09-27T23:13:48.803126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804382+0000","last_change":"2023-09-27T23:13:44.789353+0000","last_active":"2023-09-27T23:13:48.804382+0000","last_peered":"2023-09-27T23:13:48.804382+0000","last_clean":"2023-09-27T23:13:48.804382+0000","last_became_active":"2023-09-27T23:13:44.788925+0000","last_became_peered":"2023-09-27T23:13:44.788925+0000","last_unstale":"2023-09-27T23:13:48.804382+0000","last_undegraded":"2023-09-27T23:13:48.804382+0000","last_fullsized":"2023-09-27T23:13:48.804382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:24.773257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842925+0000","last_change":"2023-09-27T23:13:48.840258+0000","last_active":"2023-09-27T23:13:48.842925+0000","last_peered":"2023-09-27T23:13:48.842925+0000","last_clean":"2023-09-27T23:13:48.842925+0000","last_became_active":"2023-09-27T23:13:48.839796+0000","last_became_peered":"2023-09-27T23:13:48.839796+0000","last_unstale":"2023-09-27T23:13:48.842925+0000","last_undegraded":"2023-09-27T23:13:48.842925+0000","last_fullsized":"2023-09-27T23:13:48.842925+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"29'9","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-27T23:13:53.926546+0000","last_change":"2023-09-27T23:13:44.422959+0000","last_active":"2023-09-27T23:13:53.926546+0000","last_peered":"2023-09-27T23:13:53.926546+0000","last_clean":"2023-09-27T23:13:53.926546+0000","last_became_active":"2023-09-27T23:13:44.420682+0000","last_became_peered":"2023-09-27T23:13:44.420682+0000","last_unstale":"2023-09-27T23:13:53.926546+0000","last_undegraded":"2023-09-27T23:13:53.926546+0000","last_fullsized":"2023-09-27T23:13:53.926546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821157+0000","last_change":"2023-09-27T23:13:44.792715+0000","last_active":"2023-09-27T23:13:48.821157+0000","last_peered":"2023-09-27T23:13:48.821157+0000","last_clean":"2023-09-27T23:13:48.821157+0000","last_became_active":"2023-09-27T23:13:44.792356+0000","last_became_peered":"2023-09-27T23:13:44.792356+0000","last_unstale":"2023-09-27T23:13:48.821157+0000","last_undegraded":"2023-09-27T23:13:48.821157+0000","last_fullsized":"2023-09-27T23:13:48.821157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843134+0000","last_change":"2023-09-27T23:13:48.830683+0000","last_active":"2023-09-27T23:13:48.843134+0000","last_peered":"2023-09-27T23:13:48.843134+0000","last_clean":"2023-09-27T23:13:48.843134+0000","last_became_active":"2023-09-27T23:13:48.830149+0000","last_became_peered":"2023-09-27T23:13:48.830149+0000","last_unstale":"2023-09-27T23:13:48.843134+0000","last_undegraded":"2023-09-27T23:13:48.843134+0000","last_fullsized":"2023-09-27T23:13:48.843134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:31:07.801049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821012+0000","last_change":"2023-09-27T23:13:44.807208+0000","last_active":"2023-09-27T23:13:48.821012+0000","last_peered":"2023-09-27T23:13:48.821012+0000","last_clean":"2023-09-27T23:13:48.821012+0000","last_became_active":"2023-09-27T23:13:44.806937+0000","last_became_peered":"2023-09-27T23:13:44.806937+0000","last_unstale":"2023-09-27T23:13:48.821012+0000","last_undegraded":"2023-09-27T23:13:48.821012+0000","last_fullsized":"2023-09-27T23:13:48.821012+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:03:17.944613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813854+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:13:48.813854+0000","last_peered":"2023-09-27T23:13:48.813854+0000","last_clean":"2023-09-27T23:13:48.813854+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:13:48.813854+0000","last_undegraded":"2023-09-27T23:13:48.813854+0000","last_fullsized":"2023-09-27T23:13:48.813854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-27T23:13:49.971771+0000","last_change":"2023-09-27T23:13:44.421149+0000","last_active":"2023-09-27T23:13:49.971771+0000","last_peered":"2023-09-27T23:13:49.971771+0000","last_clean":"2023-09-27T23:13:49.971771+0000","last_became_active":"2023-09-27T23:13:44.419644+0000","last_became_peered":"2023-09-27T23:13:44.419644+0000","last_unstale":"2023-09-27T23:13:49.971771+0000","last_undegraded":"2023-09-27T23:13:49.971771+0000","last_fullsized":"2023-09-27T23:13:49.971771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842921+0000","last_change":"2023-09-27T23:13:48.840751+0000","last_active":"2023-09-27T23:13:48.842921+0000","last_peered":"2023-09-27T23:13:48.842921+0000","last_clean":"2023-09-27T23:13:48.842921+0000","last_became_active":"2023-09-27T23:13:48.840353+0000","last_became_peered":"2023-09-27T23:13:48.840353+0000","last_unstale":"2023-09-27T23:13:48.842921+0000","last_undegraded":"2023-09-27T23:13:48.842921+0000","last_fullsized":"2023-09-27T23:13:48.842921+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:59:09.353088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821991+0000","last_change":"2023-09-27T23:13:44.791105+0000","last_active":"2023-09-27T23:13:48.821991+0000","last_peered":"2023-09-27T23:13:48.821991+0000","last_clean":"2023-09-27T23:13:48.821991+0000","last_became_active":"2023-09-27T23:13:44.790824+0000","last_became_peered":"2023-09-27T23:13:44.790824+0000","last_unstale":"2023-09-27T23:13:48.821991+0000","last_undegraded":"2023-09-27T23:13:48.821991+0000","last_fullsized":"2023-09-27T23:13:48.821991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807265+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:13:48.807265+0000","last_peered":"2023-09-27T23:13:48.807265+0000","last_clean":"2023-09-27T23:13:48.807265+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:13:48.807265+0000","last_undegraded":"2023-09-27T23:13:48.807265+0000","last_fullsized":"2023-09-27T23:13:48.807265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825400+0000","last_change":"2023-09-27T23:13:44.421998+0000","last_active":"2023-09-27T23:13:48.825400+0000","last_peered":"2023-09-27T23:13:48.825400+0000","last_clean":"2023-09-27T23:13:48.825400+0000","last_became_active":"2023-09-27T23:13:44.416358+0000","last_became_peered":"2023-09-27T23:13:44.416358+0000","last_unstale":"2023-09-27T23:13:48.825400+0000","last_undegraded":"2023-09-27T23:13:48.825400+0000","last_fullsized":"2023-09-27T23:13:48.825400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840017+0000","last_change":"2023-09-27T23:13:48.827758+0000","last_active":"2023-09-27T23:13:48.840017+0000","last_peered":"2023-09-27T23:13:48.840017+0000","last_clean":"2023-09-27T23:13:48.840017+0000","last_became_active":"2023-09-27T23:13:48.827144+0000","last_became_peered":"2023-09-27T23:13:48.827144+0000","last_unstale":"2023-09-27T23:13:48.840017+0000","last_undegraded":"2023-09-27T23:13:48.840017+0000","last_fullsized":"2023-09-27T23:13:48.840017+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:02:44.973308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804641+0000","last_change":"2023-09-27T23:13:44.807107+0000","last_active":"2023-09-27T23:13:48.804641+0000","last_peered":"2023-09-27T23:13:48.804641+0000","last_clean":"2023-09-27T23:13:48.804641+0000","last_became_active":"2023-09-27T23:13:44.806739+0000","last_became_peered":"2023-09-27T23:13:44.806739+0000","last_unstale":"2023-09-27T23:13:48.804641+0000","last_undegraded":"2023-09-27T23:13:48.804641+0000","last_fullsized":"2023-09-27T23:13:48.804641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803837+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:13:48.803837+0000","last_peered":"2023-09-27T23:13:48.803837+0000","last_clean":"2023-09-27T23:13:48.803837+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:13:48.803837+0000","last_undegraded":"2023-09-27T23:13:48.803837+0000","last_fullsized":"2023-09-27T23:13:48.803837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802757+0000","last_change":"2023-09-27T23:13:44.419685+0000","last_active":"2023-09-27T23:13:48.802757+0000","last_peered":"2023-09-27T23:13:48.802757+0000","last_clean":"2023-09-27T23:13:48.802757+0000","last_became_active":"2023-09-27T23:13:44.419369+0000","last_became_peered":"2023-09-27T23:13:44.419369+0000","last_unstale":"2023-09-27T23:13:48.802757+0000","last_undegraded":"2023-09-27T23:13:48.802757+0000","last_fullsized":"2023-09-27T23:13:48.802757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:04.718338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826218+0000","last_change":"2023-09-27T23:13:48.822456+0000","last_active":"2023-09-27T23:13:48.826218+0000","last_peered":"2023-09-27T23:13:48.826218+0000","last_clean":"2023-09-27T23:13:48.826218+0000","last_became_active":"2023-09-27T23:13:48.821878+0000","last_became_peered":"2023-09-27T23:13:48.821878+0000","last_unstale":"2023-09-27T23:13:48.826218+0000","last_undegraded":"2023-09-27T23:13:48.826218+0000","last_fullsized":"2023-09-27T23:13:48.826218+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807086+0000","last_change":"2023-09-27T23:13:44.777613+0000","last_active":"2023-09-27T23:13:48.807086+0000","last_peered":"2023-09-27T23:13:48.807086+0000","last_clean":"2023-09-27T23:13:48.807086+0000","last_became_active":"2023-09-27T23:13:44.777267+0000","last_became_peered":"2023-09-27T23:13:44.777267+0000","last_unstale":"2023-09-27T23:13:48.807086+0000","last_undegraded":"2023-09-27T23:13:48.807086+0000","last_fullsized":"2023-09-27T23:13:48.807086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:42:36.525422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802984+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:13:48.802984+0000","last_peered":"2023-09-27T23:13:48.802984+0000","last_clean":"2023-09-27T23:13:48.802984+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:13:48.802984+0000","last_undegraded":"2023-09-27T23:13:48.802984+0000","last_fullsized":"2023-09-27T23:13:48.802984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814830+0000","last_change":"2023-09-27T23:13:44.422663+0000","last_active":"2023-09-27T23:13:48.814830+0000","last_peered":"2023-09-27T23:13:48.814830+0000","last_clean":"2023-09-27T23:13:48.814830+0000","last_became_active":"2023-09-27T23:13:44.422361+0000","last_became_peered":"2023-09-27T23:13:44.422361+0000","last_unstale":"2023-09-27T23:13:48.814830+0000","last_undegraded":"2023-09-27T23:13:48.814830+0000","last_fullsized":"2023-09-27T23:13:48.814830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833765+0000","last_change":"2023-09-27T23:13:48.824670+0000","last_active":"2023-09-27T23:13:48.833765+0000","last_peered":"2023-09-27T23:13:48.833765+0000","last_clean":"2023-09-27T23:13:48.833765+0000","last_became_active":"2023-09-27T23:13:48.824249+0000","last_became_peered":"2023-09-27T23:13:48.824249+0000","last_unstale":"2023-09-27T23:13:48.833765+0000","last_undegraded":"2023-09-27T23:13:48.833765+0000","last_fullsized":"2023-09-27T23:13:48.833765+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802514+0000","last_change":"2023-09-27T23:13:44.778136+0000","last_active":"2023-09-27T23:13:48.802514+0000","last_peered":"2023-09-27T23:13:48.802514+0000","last_clean":"2023-09-27T23:13:48.802514+0000","last_became_active":"2023-09-27T23:13:44.777865+0000","last_became_peered":"2023-09-27T23:13:44.777865+0000","last_unstale":"2023-09-27T23:13:48.802514+0000","last_undegraded":"2023-09-27T23:13:48.802514+0000","last_fullsized":"2023-09-27T23:13:48.802514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:46:04.289644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821294+0000","last_change":"2023-09-27T23:13:44.424183+0000","last_active":"2023-09-27T23:13:48.821294+0000","last_peered":"2023-09-27T23:13:48.821294+0000","last_clean":"2023-09-27T23:13:48.821294+0000","last_became_active":"2023-09-27T23:13:44.422919+0000","last_became_peered":"2023-09-27T23:13:44.422919+0000","last_unstale":"2023-09-27T23:13:48.821294+0000","last_undegraded":"2023-09-27T23:13:48.821294+0000","last_fullsized":"2023-09-27T23:13:48.821294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813932+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:13:48.813932+0000","last_peered":"2023-09-27T23:13:48.813932+0000","last_clean":"2023-09-27T23:13:48.813932+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:13:48.813932+0000","last_undegraded":"2023-09-27T23:13:48.813932+0000","last_fullsized":"2023-09-27T23:13:48.813932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839053+0000","last_change":"2023-09-27T23:13:48.836163+0000","last_active":"2023-09-27T23:13:48.839053+0000","last_peered":"2023-09-27T23:13:48.839053+0000","last_clean":"2023-09-27T23:13:48.839053+0000","last_became_active":"2023-09-27T23:13:48.835655+0000","last_became_peered":"2023-09-27T23:13:48.835655+0000","last_unstale":"2023-09-27T23:13:48.839053+0000","last_undegraded":"2023-09-27T23:13:48.839053+0000","last_fullsized":"2023-09-27T23:13:48.839053+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802588+0000","last_change":"2023-09-27T23:13:44.777627+0000","last_active":"2023-09-27T23:13:48.802588+0000","last_peered":"2023-09-27T23:13:48.802588+0000","last_clean":"2023-09-27T23:13:48.802588+0000","last_became_active":"2023-09-27T23:13:44.777239+0000","last_became_peered":"2023-09-27T23:13:44.777239+0000","last_unstale":"2023-09-27T23:13:48.802588+0000","last_undegraded":"2023-09-27T23:13:48.802588+0000","last_fullsized":"2023-09-27T23:13:48.802588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:34.572519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.800241+0000","last_change":"2023-09-27T23:13:44.419827+0000","last_active":"2023-09-27T23:13:48.800241+0000","last_peered":"2023-09-27T23:13:48.800241+0000","last_clean":"2023-09-27T23:13:48.800241+0000","last_became_active":"2023-09-27T23:13:44.419535+0000","last_became_peered":"2023-09-27T23:13:44.419535+0000","last_unstale":"2023-09-27T23:13:48.800241+0000","last_undegraded":"2023-09-27T23:13:48.800241+0000","last_fullsized":"2023-09-27T23:13:48.800241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:08:26.061286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813256+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:13:48.813256+0000","last_peered":"2023-09-27T23:13:48.813256+0000","last_clean":"2023-09-27T23:13:48.813256+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:13:48.813256+0000","last_undegraded":"2023-09-27T23:13:48.813256+0000","last_fullsized":"2023-09-27T23:13:48.813256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842381+0000","last_change":"2023-09-27T23:13:48.839688+0000","last_active":"2023-09-27T23:13:48.842381+0000","last_peered":"2023-09-27T23:13:48.842381+0000","last_clean":"2023-09-27T23:13:48.842381+0000","last_became_active":"2023-09-27T23:13:48.839257+0000","last_became_peered":"2023-09-27T23:13:48.839257+0000","last_unstale":"2023-09-27T23:13:48.842381+0000","last_undegraded":"2023-09-27T23:13:48.842381+0000","last_fullsized":"2023-09-27T23:13:48.842381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:23.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803724+0000","last_change":"2023-09-27T23:13:44.796668+0000","last_active":"2023-09-27T23:13:48.803724+0000","last_peered":"2023-09-27T23:13:48.803724+0000","last_clean":"2023-09-27T23:13:48.803724+0000","last_became_active":"2023-09-27T23:13:44.795170+0000","last_became_peered":"2023-09-27T23:13:44.795170+0000","last_unstale":"2023-09-27T23:13:48.803724+0000","last_undegraded":"2023-09-27T23:13:48.803724+0000","last_fullsized":"2023-09-27T23:13:48.803724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802500+0000","last_change":"2023-09-27T23:13:44.428369+0000","last_active":"2023-09-27T23:13:48.802500+0000","last_peered":"2023-09-27T23:13:48.802500+0000","last_clean":"2023-09-27T23:13:48.802500+0000","last_became_active":"2023-09-27T23:13:44.427956+0000","last_became_peered":"2023-09-27T23:13:44.427956+0000","last_unstale":"2023-09-27T23:13:48.802500+0000","last_undegraded":"2023-09-27T23:13:48.802500+0000","last_fullsized":"2023-09-27T23:13:48.802500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803011+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:13:48.803011+0000","last_peered":"2023-09-27T23:13:48.803011+0000","last_clean":"2023-09-27T23:13:48.803011+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:13:48.803011+0000","last_undegraded":"2023-09-27T23:13:48.803011+0000","last_fullsized":"2023-09-27T23:13:48.803011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834595+0000","last_change":"2023-09-27T23:13:48.830308+0000","last_active":"2023-09-27T23:13:48.834595+0000","last_peered":"2023-09-27T23:13:48.834595+0000","last_clean":"2023-09-27T23:13:48.834595+0000","last_became_active":"2023-09-27T23:13:48.829698+0000","last_became_peered":"2023-09-27T23:13:48.829698+0000","last_unstale":"2023-09-27T23:13:48.834595+0000","last_undegraded":"2023-09-27T23:13:48.834595+0000","last_fullsized":"2023-09-27T23:13:48.834595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:27:58.121695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812509+0000","last_change":"2023-09-27T23:13:44.804857+0000","last_active":"2023-09-27T23:13:48.812509+0000","last_peered":"2023-09-27T23:13:48.812509+0000","last_clean":"2023-09-27T23:13:48.812509+0000","last_became_active":"2023-09-27T23:13:44.804486+0000","last_became_peered":"2023-09-27T23:13:44.804486+0000","last_unstale":"2023-09-27T23:13:48.812509+0000","last_undegraded":"2023-09-27T23:13:48.812509+0000","last_fullsized":"2023-09-27T23:13:48.812509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822742+0000","last_change":"2023-09-27T23:13:44.416101+0000","last_active":"2023-09-27T23:13:48.822742+0000","last_peered":"2023-09-27T23:13:48.822742+0000","last_clean":"2023-09-27T23:13:48.822742+0000","last_became_active":"2023-09-27T23:13:44.415693+0000","last_became_peered":"2023-09-27T23:13:44.415693+0000","last_unstale":"2023-09-27T23:13:48.822742+0000","last_undegraded":"2023-09-27T23:13:48.822742+0000","last_fullsized":"2023-09-27T23:13:48.822742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813962+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:13:48.813962+0000","last_peered":"2023-09-27T23:13:48.813962+0000","last_clean":"2023-09-27T23:13:48.813962+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:13:48.813962+0000","last_undegraded":"2023-09-27T23:13:48.813962+0000","last_fullsized":"2023-09-27T23:13:48.813962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803349+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:13:48.803349+0000","last_peered":"2023-09-27T23:13:48.803349+0000","last_clean":"2023-09-27T23:13:48.803349+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:13:48.803349+0000","last_undegraded":"2023-09-27T23:13:48.803349+0000","last_fullsized":"2023-09-27T23:13:48.803349+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829119+0000","last_change":"2023-09-27T23:13:48.827977+0000","last_active":"2023-09-27T23:13:48.829119+0000","last_peered":"2023-09-27T23:13:48.829119+0000","last_clean":"2023-09-27T23:13:48.829119+0000","last_became_active":"2023-09-27T23:13:48.827465+0000","last_became_peered":"2023-09-27T23:13:48.827465+0000","last_unstale":"2023-09-27T23:13:48.829119+0000","last_undegraded":"2023-09-27T23:13:48.829119+0000","last_fullsized":"2023-09-27T23:13:48.829119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:18:57.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803299+0000","last_change":"2023-09-27T23:13:44.786471+0000","last_active":"2023-09-27T23:13:48.803299+0000","last_peered":"2023-09-27T23:13:48.803299+0000","last_clean":"2023-09-27T23:13:48.803299+0000","last_became_active":"2023-09-27T23:13:44.786182+0000","last_became_peered":"2023-09-27T23:13:44.786182+0000","last_unstale":"2023-09-27T23:13:48.803299+0000","last_undegraded":"2023-09-27T23:13:48.803299+0000","last_fullsized":"2023-09-27T23:13:48.803299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:04:06.713350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822086+0000","last_change":"2023-09-27T23:13:44.422925+0000","last_active":"2023-09-27T23:13:48.822086+0000","last_peered":"2023-09-27T23:13:48.822086+0000","last_clean":"2023-09-27T23:13:48.822086+0000","last_became_active":"2023-09-27T23:13:44.422547+0000","last_became_peered":"2023-09-27T23:13:44.422547+0000","last_unstale":"2023-09-27T23:13:48.822086+0000","last_undegraded":"2023-09-27T23:13:48.822086+0000","last_fullsized":"2023-09-27T23:13:48.822086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839810+0000","last_change":"2023-09-27T23:13:48.824583+0000","last_active":"2023-09-27T23:13:48.839810+0000","last_peered":"2023-09-27T23:13:48.839810+0000","last_clean":"2023-09-27T23:13:48.839810+0000","last_became_active":"2023-09-27T23:13:48.824080+0000","last_became_peered":"2023-09-27T23:13:48.824080+0000","last_unstale":"2023-09-27T23:13:48.839810+0000","last_undegraded":"2023-09-27T23:13:48.839810+0000","last_fullsized":"2023-09-27T23:13:48.839810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:29:19.951565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821416+0000","last_change":"2023-09-27T23:13:44.792934+0000","last_active":"2023-09-27T23:13:48.821416+0000","last_peered":"2023-09-27T23:13:48.821416+0000","last_clean":"2023-09-27T23:13:48.821416+0000","last_became_active":"2023-09-27T23:13:44.791774+0000","last_became_peered":"2023-09-27T23:13:44.791774+0000","last_unstale":"2023-09-27T23:13:48.821416+0000","last_undegraded":"2023-09-27T23:13:48.821416+0000","last_fullsized":"2023-09-27T23:13:48.821416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821113+0000","last_change":"2023-09-27T23:13:44.418284+0000","last_active":"2023-09-27T23:13:48.821113+0000","last_peered":"2023-09-27T23:13:48.821113+0000","last_clean":"2023-09-27T23:13:48.821113+0000","last_became_active":"2023-09-27T23:13:44.417952+0000","last_became_peered":"2023-09-27T23:13:44.417952+0000","last_unstale":"2023-09-27T23:13:48.821113+0000","last_undegraded":"2023-09-27T23:13:48.821113+0000","last_fullsized":"2023-09-27T23:13:48.821113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840441+0000","last_change":"2023-09-27T23:13:48.835487+0000","last_active":"2023-09-27T23:13:48.840441+0000","last_peered":"2023-09-27T23:13:48.840441+0000","last_clean":"2023-09-27T23:13:48.840441+0000","last_became_active":"2023-09-27T23:13:48.835043+0000","last_became_peered":"2023-09-27T23:13:48.835043+0000","last_unstale":"2023-09-27T23:13:48.840441+0000","last_undegraded":"2023-09-27T23:13:48.840441+0000","last_fullsized":"2023-09-27T23:13:48.840441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:43:53.193315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821389+0000","last_change":"2023-09-27T23:13:44.797452+0000","last_active":"2023-09-27T23:13:48.821389+0000","last_peered":"2023-09-27T23:13:48.821389+0000","last_clean":"2023-09-27T23:13:48.821389+0000","last_became_active":"2023-09-27T23:13:44.797056+0000","last_became_peered":"2023-09-27T23:13:44.797056+0000","last_unstale":"2023-09-27T23:13:48.821389+0000","last_undegraded":"2023-09-27T23:13:48.821389+0000","last_fullsized":"2023-09-27T23:13:48.821389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812548+0000","last_change":"2023-09-27T23:13:44.426307+0000","last_active":"2023-09-27T23:13:48.812548+0000","last_peered":"2023-09-27T23:13:48.812548+0000","last_clean":"2023-09-27T23:13:48.812548+0000","last_became_active":"2023-09-27T23:13:44.425925+0000","last_became_peered":"2023-09-27T23:13:44.425925+0000","last_unstale":"2023-09-27T23:13:48.812548+0000","last_undegraded":"2023-09-27T23:13:48.812548+0000","last_fullsized":"2023-09-27T23:13:48.812548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843420+0000","last_change":"2023-09-27T23:13:48.835916+0000","last_active":"2023-09-27T23:13:48.843420+0000","last_peered":"2023-09-27T23:13:48.843420+0000","last_clean":"2023-09-27T23:13:48.843420+0000","last_became_active":"2023-09-27T23:13:48.835201+0000","last_became_peered":"2023-09-27T23:13:48.835201+0000","last_unstale":"2023-09-27T23:13:48.843420+0000","last_undegraded":"2023-09-27T23:13:48.843420+0000","last_fullsized":"2023-09-27T23:13:48.843420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:15:13.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821512+0000","last_change":"2023-09-27T23:13:44.804376+0000","last_active":"2023-09-27T23:13:48.821512+0000","last_peered":"2023-09-27T23:13:48.821512+0000","last_clean":"2023-09-27T23:13:48.821512+0000","last_became_active":"2023-09-27T23:13:44.803820+0000","last_became_peered":"2023-09-27T23:13:44.803820+0000","last_unstale":"2023-09-27T23:13:48.821512+0000","last_undegraded":"2023-09-27T23:13:48.821512+0000","last_fullsized":"2023-09-27T23:13:48.821512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:45:33.486648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825618+0000","last_change":"2023-09-27T23:13:44.422613+0000","last_active":"2023-09-27T23:13:48.825618+0000","last_peered":"2023-09-27T23:13:48.825618+0000","last_clean":"2023-09-27T23:13:48.825618+0000","last_became_active":"2023-09-27T23:13:44.422341+0000","last_became_peered":"2023-09-27T23:13:44.422341+0000","last_unstale":"2023-09-27T23:13:48.825618+0000","last_undegraded":"2023-09-27T23:13:48.825618+0000","last_fullsized":"2023-09-27T23:13:48.825618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825159+0000","last_change":"2023-09-27T23:13:48.821288+0000","last_active":"2023-09-27T23:13:48.825159+0000","last_peered":"2023-09-27T23:13:48.825159+0000","last_clean":"2023-09-27T23:13:48.825159+0000","last_became_active":"2023-09-27T23:13:48.820648+0000","last_became_peered":"2023-09-27T23:13:48.820648+0000","last_unstale":"2023-09-27T23:13:48.825159+0000","last_undegraded":"2023-09-27T23:13:48.825159+0000","last_fullsized":"2023-09-27T23:13:48.825159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:17.485779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804537+0000","last_change":"2023-09-27T23:13:44.795149+0000","last_active":"2023-09-27T23:13:48.804537+0000","last_peered":"2023-09-27T23:13:48.804537+0000","last_clean":"2023-09-27T23:13:48.804537+0000","last_became_active":"2023-09-27T23:13:44.794005+0000","last_became_peered":"2023-09-27T23:13:44.794005+0000","last_unstale":"2023-09-27T23:13:48.804537+0000","last_undegraded":"2023-09-27T23:13:48.804537+0000","last_fullsized":"2023-09-27T23:13:48.804537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811248+0000","last_change":"2023-09-27T23:13:44.422676+0000","last_active":"2023-09-27T23:13:48.811248+0000","last_peered":"2023-09-27T23:13:48.811248+0000","last_clean":"2023-09-27T23:13:48.811248+0000","last_became_active":"2023-09-27T23:13:44.422396+0000","last_became_peered":"2023-09-27T23:13:44.422396+0000","last_unstale":"2023-09-27T23:13:48.811248+0000","last_undegraded":"2023-09-27T23:13:48.811248+0000","last_fullsized":"2023-09-27T23:13:48.811248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838657+0000","last_change":"2023-09-27T23:13:48.835879+0000","last_active":"2023-09-27T23:13:48.838657+0000","last_peered":"2023-09-27T23:13:48.838657+0000","last_clean":"2023-09-27T23:13:48.838657+0000","last_became_active":"2023-09-27T23:13:48.835422+0000","last_became_peered":"2023-09-27T23:13:48.835422+0000","last_unstale":"2023-09-27T23:13:48.838657+0000","last_undegraded":"2023-09-27T23:13:48.838657+0000","last_fullsized":"2023-09-27T23:13:48.838657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:36.782810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810125+0000","last_change":"2023-09-27T23:13:44.795768+0000","last_active":"2023-09-27T23:13:48.810125+0000","last_peered":"2023-09-27T23:13:48.810125+0000","last_clean":"2023-09-27T23:13:48.810125+0000","last_became_active":"2023-09-27T23:13:44.795376+0000","last_became_peered":"2023-09-27T23:13:44.795376+0000","last_unstale":"2023-09-27T23:13:48.810125+0000","last_undegraded":"2023-09-27T23:13:48.810125+0000","last_fullsized":"2023-09-27T23:13:48.810125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:02.136037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803338+0000","last_change":"2023-09-27T23:13:44.415778+0000","last_active":"2023-09-27T23:13:48.803338+0000","last_peered":"2023-09-27T23:13:48.803338+0000","last_clean":"2023-09-27T23:13:48.803338+0000","last_became_active":"2023-09-27T23:13:44.415490+0000","last_became_peered":"2023-09-27T23:13:44.415490+0000","last_unstale":"2023-09-27T23:13:48.803338+0000","last_undegraded":"2023-09-27T23:13:48.803338+0000","last_fullsized":"2023-09-27T23:13:48.803338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.828626+0000","last_change":"2023-09-27T23:13:48.823729+0000","last_active":"2023-09-27T23:13:48.828626+0000","last_peered":"2023-09-27T23:13:48.828626+0000","last_clean":"2023-09-27T23:13:48.828626+0000","last_became_active":"2023-09-27T23:13:48.823284+0000","last_became_peered":"2023-09-27T23:13:48.823284+0000","last_unstale":"2023-09-27T23:13:48.828626+0000","last_undegraded":"2023-09-27T23:13:48.828626+0000","last_fullsized":"2023-09-27T23:13:48.828626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:19.656238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820833+0000","last_change":"2023-09-27T23:13:44.794072+0000","last_active":"2023-09-27T23:13:48.820833+0000","last_peered":"2023-09-27T23:13:48.820833+0000","last_clean":"2023-09-27T23:13:48.820833+0000","last_became_active":"2023-09-27T23:13:44.793773+0000","last_became_peered":"2023-09-27T23:13:44.793773+0000","last_unstale":"2023-09-27T23:13:48.820833+0000","last_undegraded":"2023-09-27T23:13:48.820833+0000","last_fullsized":"2023-09-27T23:13:48.820833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804693+0000","last_change":"2023-09-27T23:13:44.426779+0000","last_active":"2023-09-27T23:13:48.804693+0000","last_peered":"2023-09-27T23:13:48.804693+0000","last_clean":"2023-09-27T23:13:48.804693+0000","last_became_active":"2023-09-27T23:13:44.426515+0000","last_became_peered":"2023-09-27T23:13:44.426515+0000","last_unstale":"2023-09-27T23:13:48.804693+0000","last_undegraded":"2023-09-27T23:13:48.804693+0000","last_fullsized":"2023-09-27T23:13:48.804693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:40.093740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842388+0000","last_change":"2023-09-27T23:13:48.839843+0000","last_active":"2023-09-27T23:13:48.842388+0000","last_peered":"2023-09-27T23:13:48.842388+0000","last_clean":"2023-09-27T23:13:48.842388+0000","last_became_active":"2023-09-27T23:13:48.839383+0000","last_became_peered":"2023-09-27T23:13:48.839383+0000","last_unstale":"2023-09-27T23:13:48.842388+0000","last_undegraded":"2023-09-27T23:13:48.842388+0000","last_fullsized":"2023-09-27T23:13:48.842388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810673+0000","last_change":"2023-09-27T23:13:44.802558+0000","last_active":"2023-09-27T23:13:48.810673+0000","last_peered":"2023-09-27T23:13:48.810673+0000","last_clean":"2023-09-27T23:13:48.810673+0000","last_became_active":"2023-09-27T23:13:44.800406+0000","last_became_peered":"2023-09-27T23:13:44.800406+0000","last_unstale":"2023-09-27T23:13:48.810673+0000","last_undegraded":"2023-09-27T23:13:48.810673+0000","last_fullsized":"2023-09-27T23:13:48.810673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804716+0000","last_change":"2023-09-27T23:13:44.426684+0000","last_active":"2023-09-27T23:13:48.804716+0000","last_peered":"2023-09-27T23:13:48.804716+0000","last_clean":"2023-09-27T23:13:48.804716+0000","last_became_active":"2023-09-27T23:13:44.426422+0000","last_became_peered":"2023-09-27T23:13:44.426422+0000","last_unstale":"2023-09-27T23:13:48.804716+0000","last_undegraded":"2023-09-27T23:13:48.804716+0000","last_fullsized":"2023-09-27T23:13:48.804716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:12.799421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829098+0000","last_change":"2023-09-27T23:13:48.827612+0000","last_active":"2023-09-27T23:13:48.829098+0000","last_peered":"2023-09-27T23:13:48.829098+0000","last_clean":"2023-09-27T23:13:48.829098+0000","last_became_active":"2023-09-27T23:13:48.827019+0000","last_became_peered":"2023-09-27T23:13:48.827019+0000","last_unstale":"2023-09-27T23:13:48.829098+0000","last_undegraded":"2023-09-27T23:13:48.829098+0000","last_fullsized":"2023-09-27T23:13:48.829098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:40:33.151042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.835789+0000","last_change":"2023-09-27T23:13:48.830820+0000","last_active":"2023-09-27T23:13:48.835789+0000","last_peered":"2023-09-27T23:13:48.835789+0000","last_clean":"2023-09-27T23:13:48.835789+0000","last_became_active":"2023-09-27T23:13:48.829846+0000","last_became_peered":"2023-09-27T23:13:48.829846+0000","last_unstale":"2023-09-27T23:13:48.835789+0000","last_undegraded":"2023-09-27T23:13:48.835789+0000","last_fullsized":"2023-09-27T23:13:48.835789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809441+0000","last_change":"2023-09-27T23:13:44.413137+0000","last_active":"2023-09-27T23:13:48.809441+0000","last_peered":"2023-09-27T23:13:48.809441+0000","last_clean":"2023-09-27T23:13:48.809441+0000","last_became_active":"2023-09-27T23:13:44.412799+0000","last_became_peered":"2023-09-27T23:13:44.412799+0000","last_unstale":"2023-09-27T23:13:48.809441+0000","last_undegraded":"2023-09-27T23:13:48.809441+0000","last_fullsized":"2023-09-27T23:13:48.809441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825273+0000","last_change":"2023-09-27T23:13:44.793322+0000","last_active":"2023-09-27T23:13:48.825273+0000","last_peered":"2023-09-27T23:13:48.825273+0000","last_clean":"2023-09-27T23:13:48.825273+0000","last_became_active":"2023-09-27T23:13:44.793057+0000","last_became_peered":"2023-09-27T23:13:44.793057+0000","last_unstale":"2023-09-27T23:13:48.825273+0000","last_undegraded":"2023-09-27T23:13:48.825273+0000","last_fullsized":"2023-09-27T23:13:48.825273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842877+0000","last_change":"2023-09-27T23:13:48.839976+0000","last_active":"2023-09-27T23:13:48.842877+0000","last_peered":"2023-09-27T23:13:48.842877+0000","last_clean":"2023-09-27T23:13:48.842877+0000","last_became_active":"2023-09-27T23:13:48.839568+0000","last_became_peered":"2023-09-27T23:13:48.839568+0000","last_unstale":"2023-09-27T23:13:48.842877+0000","last_undegraded":"2023-09-27T23:13:48.842877+0000","last_fullsized":"2023-09-27T23:13:48.842877+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:01.437279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804236+0000","last_change":"2023-09-27T23:13:44.427379+0000","last_active":"2023-09-27T23:13:48.804236+0000","last_peered":"2023-09-27T23:13:48.804236+0000","last_clean":"2023-09-27T23:13:48.804236+0000","last_became_active":"2023-09-27T23:13:44.427094+0000","last_became_peered":"2023-09-27T23:13:44.427094+0000","last_unstale":"2023-09-27T23:13:48.804236+0000","last_undegraded":"2023-09-27T23:13:48.804236+0000","last_fullsized":"2023-09-27T23:13:48.804236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:33.544941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803930+0000","last_change":"2023-09-27T23:13:44.793100+0000","last_active":"2023-09-27T23:13:48.803930+0000","last_peered":"2023-09-27T23:13:48.803930+0000","last_clean":"2023-09-27T23:13:48.803930+0000","last_became_active":"2023-09-27T23:13:44.792493+0000","last_became_peered":"2023-09-27T23:13:44.792493+0000","last_unstale":"2023-09-27T23:13:48.803930+0000","last_undegraded":"2023-09-27T23:13:48.803930+0000","last_fullsized":"2023-09-27T23:13:48.803930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804152+0000","last_change":"2023-09-27T23:13:44.420128+0000","last_active":"2023-09-27T23:13:48.804152+0000","last_peered":"2023-09-27T23:13:48.804152+0000","last_clean":"2023-09-27T23:13:48.804152+0000","last_became_active":"2023-09-27T23:13:44.419240+0000","last_became_peered":"2023-09-27T23:13:44.419240+0000","last_unstale":"2023-09-27T23:13:48.804152+0000","last_undegraded":"2023-09-27T23:13:48.804152+0000","last_fullsized":"2023-09-27T23:13:48.804152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814644+0000","last_change":"2023-09-27T23:13:44.802448+0000","last_active":"2023-09-27T23:13:48.814644+0000","last_peered":"2023-09-27T23:13:48.814644+0000","last_clean":"2023-09-27T23:13:48.814644+0000","last_became_active":"2023-09-27T23:13:44.801316+0000","last_became_peered":"2023-09-27T23:13:44.801316+0000","last_unstale":"2023-09-27T23:13:48.814644+0000","last_undegraded":"2023-09-27T23:13:48.814644+0000","last_fullsized":"2023-09-27T23:13:48.814644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842951+0000","last_change":"2023-09-27T23:13:48.841624+0000","last_active":"2023-09-27T23:13:48.842951+0000","last_peered":"2023-09-27T23:13:48.842951+0000","last_clean":"2023-09-27T23:13:48.842951+0000","last_became_active":"2023-09-27T23:13:48.841104+0000","last_became_peered":"2023-09-27T23:13:48.841104+0000","last_unstale":"2023-09-27T23:13:48.842951+0000","last_undegraded":"2023-09-27T23:13:48.842951+0000","last_fullsized":"2023-09-27T23:13:48.842951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804318+0000","last_change":"2023-09-27T23:13:44.426217+0000","last_active":"2023-09-27T23:13:48.804318+0000","last_peered":"2023-09-27T23:13:48.804318+0000","last_clean":"2023-09-27T23:13:48.804318+0000","last_became_active":"2023-09-27T23:13:44.425907+0000","last_became_peered":"2023-09-27T23:13:44.425907+0000","last_unstale":"2023-09-27T23:13:48.804318+0000","last_undegraded":"2023-09-27T23:13:48.804318+0000","last_fullsized":"2023-09-27T23:13:48.804318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:01.126893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825997+0000","last_change":"2023-09-27T23:13:44.789234+0000","last_active":"2023-09-27T23:13:48.825997+0000","last_peered":"2023-09-27T23:13:48.825997+0000","last_clean":"2023-09-27T23:13:48.825997+0000","last_became_active":"2023-09-27T23:13:44.788948+0000","last_became_peered":"2023-09-27T23:13:44.788948+0000","last_unstale":"2023-09-27T23:13:48.825997+0000","last_undegraded":"2023-09-27T23:13:48.825997+0000","last_fullsized":"2023-09-27T23:13:48.825997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834520+0000","last_change":"2023-09-27T23:13:48.830283+0000","last_active":"2023-09-27T23:13:48.834520+0000","last_peered":"2023-09-27T23:13:48.834520+0000","last_clean":"2023-09-27T23:13:48.834520+0000","last_became_active":"2023-09-27T23:13:48.829788+0000","last_became_peered":"2023-09-27T23:13:48.829788+0000","last_unstale":"2023-09-27T23:13:48.834520+0000","last_undegraded":"2023-09-27T23:13:48.834520+0000","last_fullsized":"2023-09-27T23:13:48.834520+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809476+0000","last_change":"2023-09-27T23:13:44.794796+0000","last_active":"2023-09-27T23:13:48.809476+0000","last_peered":"2023-09-27T23:13:48.809476+0000","last_clean":"2023-09-27T23:13:48.809476+0000","last_became_active":"2023-09-27T23:13:44.793105+0000","last_became_peered":"2023-09-27T23:13:44.793105+0000","last_unstale":"2023-09-27T23:13:48.809476+0000","last_undegraded":"2023-09-27T23:13:48.809476+0000","last_fullsized":"2023-09-27T23:13:48.809476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:13.808565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822377+0000","last_change":"2023-09-27T23:13:44.789337+0000","last_active":"2023-09-27T23:13:48.822377+0000","last_peered":"2023-09-27T23:13:48.822377+0000","last_clean":"2023-09-27T23:13:48.822377+0000","last_became_active":"2023-09-27T23:13:44.789055+0000","last_became_peered":"2023-09-27T23:13:44.789055+0000","last_unstale":"2023-09-27T23:13:48.822377+0000","last_undegraded":"2023-09-27T23:13:48.822377+0000","last_fullsized":"2023-09-27T23:13:48.822377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821093+0000","last_change":"2023-09-27T23:13:44.792891+0000","last_active":"2023-09-27T23:13:48.821093+0000","last_peered":"2023-09-27T23:13:48.821093+0000","last_clean":"2023-09-27T23:13:48.821093+0000","last_became_active":"2023-09-27T23:13:44.792439+0000","last_became_peered":"2023-09-27T23:13:44.792439+0000","last_unstale":"2023-09-27T23:13:48.821093+0000","last_undegraded":"2023-09-27T23:13:48.821093+0000","last_fullsized":"2023-09-27T23:13:48.821093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:33.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825381+0000","last_change":"2023-09-27T23:13:44.787514+0000","last_active":"2023-09-27T23:13:48.825381+0000","last_peered":"2023-09-27T23:13:48.825381+0000","last_clean":"2023-09-27T23:13:48.825381+0000","last_became_active":"2023-09-27T23:13:44.787130+0000","last_became_peered":"2023-09-27T23:13:44.787130+0000","last_unstale":"2023-09-27T23:13:48.825381+0000","last_undegraded":"2023-09-27T23:13:48.825381+0000","last_fullsized":"2023-09-27T23:13:48.825381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.806701+0000","last_change":"2023-09-27T23:13:44.786768+0000","last_active":"2023-09-27T23:13:48.806701+0000","last_peered":"2023-09-27T23:13:48.806701+0000","last_clean":"2023-09-27T23:13:48.806701+0000","last_became_active":"2023-09-27T23:13:44.785642+0000","last_became_peered":"2023-09-27T23:13:44.785642+0000","last_unstale":"2023-09-27T23:13:48.806701+0000","last_undegraded":"2023-09-27T23:13:48.806701+0000","last_fullsized":"2023-09-27T23:13:48.806701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:11:40.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814552+0000","last_change":"2023-09-27T23:13:44.802393+0000","last_active":"2023-09-27T23:13:48.814552+0000","last_peered":"2023-09-27T23:13:48.814552+0000","last_clean":"2023-09-27T23:13:48.814552+0000","last_became_active":"2023-09-27T23:13:44.801038+0000","last_became_peered":"2023-09-27T23:13:44.801038+0000","last_unstale":"2023-09-27T23:13:48.814552+0000","last_undegraded":"2023-09-27T23:13:48.814552+0000","last_fullsized":"2023-09-27T23:13:48.814552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807068+0000","last_change":"2023-09-27T23:13:44.778165+0000","last_active":"2023-09-27T23:13:48.807068+0000","last_peered":"2023-09-27T23:13:48.807068+0000","last_clean":"2023-09-27T23:13:48.807068+0000","last_became_active":"2023-09-27T23:13:44.777922+0000","last_became_peered":"2023-09-27T23:13:44.777922+0000","last_unstale":"2023-09-27T23:13:48.807068+0000","last_undegraded":"2023-09-27T23:13:48.807068+0000","last_fullsized":"2023-09-27T23:13:48.807068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:31.322306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.801910+0000","last_change":"2023-09-27T23:13:44.792009+0000","last_active":"2023-09-27T23:13:48.801910+0000","last_peered":"2023-09-27T23:13:48.801910+0000","last_clean":"2023-09-27T23:13:48.801910+0000","last_became_active":"2023-09-27T23:13:44.791714+0000","last_became_peered":"2023-09-27T23:13:44.791714+0000","last_unstale":"2023-09-27T23:13:48.801910+0000","last_undegraded":"2023-09-27T23:13:48.801910+0000","last_fullsized":"2023-09-27T23:13:48.801910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:10.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802783+0000","last_change":"2023-09-27T23:13:44.800431+0000","last_active":"2023-09-27T23:13:48.802783+0000","last_peered":"2023-09-27T23:13:48.802783+0000","last_clean":"2023-09-27T23:13:48.802783+0000","last_became_active":"2023-09-27T23:13:44.800056+0000","last_became_peered":"2023-09-27T23:13:44.800056+0000","last_unstale":"2023-09-27T23:13:48.802783+0000","last_undegraded":"2023-09-27T23:13:48.802783+0000","last_fullsized":"2023-09-27T23:13:48.802783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822290+0000","last_change":"2023-09-27T23:13:44.788799+0000","last_active":"2023-09-27T23:13:48.822290+0000","last_peered":"2023-09-27T23:13:48.822290+0000","last_clean":"2023-09-27T23:13:48.822290+0000","last_became_active":"2023-09-27T23:13:44.788526+0000","last_became_peered":"2023-09-27T23:13:44.788526+0000","last_unstale":"2023-09-27T23:13:48.822290+0000","last_undegraded":"2023-09-27T23:13:48.822290+0000","last_fullsized":"2023-09-27T23:13:48.822290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803904+0000","last_change":"2023-09-27T23:13:44.795488+0000","last_active":"2023-09-27T23:13:48.803904+0000","last_peered":"2023-09-27T23:13:48.803904+0000","last_clean":"2023-09-27T23:13:48.803904+0000","last_became_active":"2023-09-27T23:13:44.794082+0000","last_became_peered":"2023-09-27T23:13:44.794082+0000","last_unstale":"2023-09-27T23:13:48.803904+0000","last_undegraded":"2023-09-27T23:13:48.803904+0000","last_fullsized":"2023-09-27T23:13:48.803904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803019+0000","last_change":"2023-09-27T23:13:44.802208+0000","last_active":"2023-09-27T23:13:48.803019+0000","last_peered":"2023-09-27T23:13:48.803019+0000","last_clean":"2023-09-27T23:13:48.803019+0000","last_became_active":"2023-09-27T23:13:44.801858+0000","last_became_peered":"2023-09-27T23:13:44.801858+0000","last_unstale":"2023-09-27T23:13:48.803019+0000","last_undegraded":"2023-09-27T23:13:48.803019+0000","last_fullsized":"2023-09-27T23:13:48.803019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821304+0000","last_change":"2023-09-27T23:13:44.795884+0000","last_active":"2023-09-27T23:13:48.821304+0000","last_peered":"2023-09-27T23:13:48.821304+0000","last_clean":"2023-09-27T23:13:48.821304+0000","last_became_active":"2023-09-27T23:13:44.795490+0000","last_became_peered":"2023-09-27T23:13:44.795490+0000","last_unstale":"2023-09-27T23:13:48.821304+0000","last_undegraded":"2023-09-27T23:13:48.821304+0000","last_fullsized":"2023-09-27T23:13:48.821304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:53:40.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810126+0000","last_change":"2023-09-27T23:13:44.791556+0000","last_active":"2023-09-27T23:13:48.810126+0000","last_peered":"2023-09-27T23:13:48.810126+0000","last_clean":"2023-09-27T23:13:48.810126+0000","last_became_active":"2023-09-27T23:13:44.789598+0000","last_became_peered":"2023-09-27T23:13:44.789598+0000","last_unstale":"2023-09-27T23:13:48.810126+0000","last_undegraded":"2023-09-27T23:13:48.810126+0000","last_fullsized":"2023-09-27T23:13:48.810126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:03.782211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821279+0000","last_change":"2023-09-27T23:13:44.794170+0000","last_active":"2023-09-27T23:13:48.821279+0000","last_peered":"2023-09-27T23:13:48.821279+0000","last_clean":"2023-09-27T23:13:48.821279+0000","last_became_active":"2023-09-27T23:13:44.793843+0000","last_became_peered":"2023-09-27T23:13:44.793843+0000","last_unstale":"2023-09-27T23:13:48.821279+0000","last_undegraded":"2023-09-27T23:13:48.821279+0000","last_fullsized":"2023-09-27T23:13:48.821279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804536+0000","last_change":"2023-09-27T23:13:44.795499+0000","last_active":"2023-09-27T23:13:48.804536+0000","last_peered":"2023-09-27T23:13:48.804536+0000","last_clean":"2023-09-27T23:13:48.804536+0000","last_became_active":"2023-09-27T23:13:44.794256+0000","last_became_peered":"2023-09-27T23:13:44.794256+0000","last_unstale":"2023-09-27T23:13:48.804536+0000","last_undegraded":"2023-09-27T23:13:48.804536+0000","last_fullsized":"2023-09-27T23:13:48.804536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821399+0000","last_change":"2023-09-27T23:13:44.797131+0000","last_active":"2023-09-27T23:13:48.821399+0000","last_peered":"2023-09-27T23:13:48.821399+0000","last_clean":"2023-09-27T23:13:48.821399+0000","last_became_active":"2023-09-27T23:13:44.796814+0000","last_became_peered":"2023-09-27T23:13:44.796814+0000","last_unstale":"2023-09-27T23:13:48.821399+0000","last_undegraded":"2023-09-27T23:13:48.821399+0000","last_fullsized":"2023-09-27T23:13:48.821399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804126+0000","last_change":"2023-09-27T23:13:44.794205+0000","last_active":"2023-09-27T23:13:48.804126+0000","last_peered":"2023-09-27T23:13:48.804126+0000","last_clean":"2023-09-27T23:13:48.804126+0000","last_became_active":"2023-09-27T23:13:44.793765+0000","last_became_peered":"2023-09-27T23:13:44.793765+0000","last_unstale":"2023-09-27T23:13:48.804126+0000","last_undegraded":"2023-09-27T23:13:48.804126+0000","last_fullsized":"2023-09-27T23:13:48.804126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813812+0000","last_change":"2023-09-27T23:13:44.777827+0000","last_active":"2023-09-27T23:13:48.813812+0000","last_peered":"2023-09-27T23:13:48.813812+0000","last_clean":"2023-09-27T23:13:48.813812+0000","last_became_active":"2023-09-27T23:13:44.777461+0000","last_became_peered":"2023-09-27T23:13:44.777461+0000","last_unstale":"2023-09-27T23:13:48.813812+0000","last_undegraded":"2023-09-27T23:13:48.813812+0000","last_fullsized":"2023-09-27T23:13:48.813812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810644+0000","last_change":"2023-09-27T23:13:44.787157+0000","last_active":"2023-09-27T23:13:48.810644+0000","last_peered":"2023-09-27T23:13:48.810644+0000","last_clean":"2023-09-27T23:13:48.810644+0000","last_became_active":"2023-09-27T23:13:44.786486+0000","last_became_peered":"2023-09-27T23:13:44.786486+0000","last_unstale":"2023-09-27T23:13:48.810644+0000","last_undegraded":"2023-09-27T23:13:48.810644+0000","last_fullsized":"2023-09-27T23:13:48.810644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:01.890618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810866+0000","last_change":"2023-09-27T23:13:44.796954+0000","last_active":"2023-09-27T23:13:48.810866+0000","last_peered":"2023-09-27T23:13:48.810866+0000","last_clean":"2023-09-27T23:13:48.810866+0000","last_became_active":"2023-09-27T23:13:44.796634+0000","last_became_peered":"2023-09-27T23:13:44.796634+0000","last_unstale":"2023-09-27T23:13:48.810866+0000","last_undegraded":"2023-09-27T23:13:48.810866+0000","last_fullsized":"2023-09-27T23:13:48.810866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:04.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821286+0000","last_change":"2023-09-27T23:13:44.797615+0000","last_active":"2023-09-27T23:13:48.821286+0000","last_peered":"2023-09-27T23:13:48.821286+0000","last_clean":"2023-09-27T23:13:48.821286+0000","last_became_active":"2023-09-27T23:13:44.797325+0000","last_became_peered":"2023-09-27T23:13:44.797325+0000","last_unstale":"2023-09-27T23:13:48.821286+0000","last_undegraded":"2023-09-27T23:13:48.821286+0000","last_fullsized":"2023-09-27T23:13:48.821286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.819151+0000","last_change":"2023-09-27T23:13:44.801326+0000","last_active":"2023-09-27T23:13:48.819151+0000","last_peered":"2023-09-27T23:13:48.819151+0000","last_clean":"2023-09-27T23:13:48.819151+0000","last_became_active":"2023-09-27T23:13:44.800936+0000","last_became_peered":"2023-09-27T23:13:44.800936+0000","last_unstale":"2023-09-27T23:13:48.819151+0000","last_undegraded":"2023-09-27T23:13:48.819151+0000","last_fullsized":"2023-09-27T23:13:48.819151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803144+0000","last_change":"2023-09-27T23:13:44.800702+0000","last_active":"2023-09-27T23:13:48.803144+0000","last_peered":"2023-09-27T23:13:48.803144+0000","last_clean":"2023-09-27T23:13:48.803144+0000","last_became_active":"2023-09-27T23:13:44.800422+0000","last_became_peered":"2023-09-27T23:13:44.800422+0000","last_unstale":"2023-09-27T23:13:48.803144+0000","last_undegraded":"2023-09-27T23:13:48.803144+0000","last_fullsized":"2023-09-27T23:13:48.803144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810589+0000","last_change":"2023-09-27T23:13:44.802316+0000","last_active":"2023-09-27T23:13:48.810589+0000","last_peered":"2023-09-27T23:13:48.810589+0000","last_clean":"2023-09-27T23:13:48.810589+0000","last_became_active":"2023-09-27T23:13:44.800990+0000","last_became_peered":"2023-09-27T23:13:44.800990+0000","last_unstale":"2023-09-27T23:13:48.810589+0000","last_undegraded":"2023-09-27T23:13:48.810589+0000","last_fullsized":"2023-09-27T23:13:48.810589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804015+0000","last_change":"2023-09-27T23:13:44.794033+0000","last_active":"2023-09-27T23:13:48.804015+0000","last_peered":"2023-09-27T23:13:48.804015+0000","last_clean":"2023-09-27T23:13:48.804015+0000","last_became_active":"2023-09-27T23:13:44.793696+0000","last_became_peered":"2023-09-27T23:13:44.793696+0000","last_unstale":"2023-09-27T23:13:48.804015+0000","last_undegraded":"2023-09-27T23:13:48.804015+0000","last_fullsized":"2023-09-27T23:13:48.804015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820743+0000","last_change":"2023-09-27T23:13:44.795881+0000","last_active":"2023-09-27T23:13:48.820743+0000","last_peered":"2023-09-27T23:13:48.820743+0000","last_clean":"2023-09-27T23:13:48.820743+0000","last_became_active":"2023-09-27T23:13:44.795554+0000","last_became_peered":"2023-09-27T23:13:44.795554+0000","last_unstale":"2023-09-27T23:13:48.820743+0000","last_undegraded":"2023-09-27T23:13:48.820743+0000","last_fullsized":"2023-09-27T23:13:48.820743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:16.262346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804794+0000","last_change":"2023-09-27T23:13:44.801496+0000","last_active":"2023-09-27T23:13:48.804794+0000","last_peered":"2023-09-27T23:13:48.804794+0000","last_clean":"2023-09-27T23:13:48.804794+0000","last_became_active":"2023-09-27T23:13:44.801264+0000","last_became_peered":"2023-09-27T23:13:44.801264+0000","last_unstale":"2023-09-27T23:13:48.804794+0000","last_undegraded":"2023-09-27T23:13:48.804794+0000","last_fullsized":"2023-09-27T23:13:48.804794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812717+0000","last_change":"2023-09-27T23:13:44.790757+0000","last_active":"2023-09-27T23:13:48.812717+0000","last_peered":"2023-09-27T23:13:48.812717+0000","last_clean":"2023-09-27T23:13:48.812717+0000","last_became_active":"2023-09-27T23:13:44.790388+0000","last_became_peered":"2023-09-27T23:13:44.790388+0000","last_unstale":"2023-09-27T23:13:48.812717+0000","last_undegraded":"2023-09-27T23:13:48.812717+0000","last_fullsized":"2023-09-27T23:13:48.812717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821180+0000","last_change":"2023-09-27T23:13:44.799201+0000","last_active":"2023-09-27T23:13:48.821180+0000","last_peered":"2023-09-27T23:13:48.821180+0000","last_clean":"2023-09-27T23:13:48.821180+0000","last_became_active":"2023-09-27T23:13:44.798712+0000","last_became_peered":"2023-09-27T23:13:44.798712+0000","last_unstale":"2023-09-27T23:13:48.821180+0000","last_undegraded":"2023-09-27T23:13:48.821180+0000","last_fullsized":"2023-09-27T23:13:48.821180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815177+0000","last_change":"2023-09-27T23:13:44.780452+0000","last_active":"2023-09-27T23:13:48.815177+0000","last_peered":"2023-09-27T23:13:48.815177+0000","last_clean":"2023-09-27T23:13:48.815177+0000","last_became_active":"2023-09-27T23:13:44.780111+0000","last_became_peered":"2023-09-27T23:13:44.780111+0000","last_unstale":"2023-09-27T23:13:48.815177+0000","last_undegraded":"2023-09-27T23:13:48.815177+0000","last_fullsized":"2023-09-27T23:13:48.815177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810755+0000","last_change":"2023-09-27T23:13:44.794473+0000","last_active":"2023-09-27T23:13:48.810755+0000","last_peered":"2023-09-27T23:13:48.810755+0000","last_clean":"2023-09-27T23:13:48.810755+0000","last_became_active":"2023-09-27T23:13:44.794187+0000","last_became_peered":"2023-09-27T23:13:44.794187+0000","last_unstale":"2023-09-27T23:13:48.810755+0000","last_undegraded":"2023-09-27T23:13:48.810755+0000","last_fullsized":"2023-09-27T23:13:48.810755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9300,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362540,"statfs":{"total":96636764160,"available":96627240960,"internally_reserved":0,"allocated":929792,"data_stored":719189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]}]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8188,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363652,"statfs":{"total":96636764160,"available":96628379648,"internally_reserved":0,"allocated":315392,"data_stored":125515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8264,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363576,"statfs":{"total":96636764160,"available":96628301824,"internally_reserved":0,"allocated":327680,"data_stored":128661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":124981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]}]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":905216,"data_stored":715813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:03.374 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.474+0000 7f928d5fe700 1 Processor -- start 2023-09-27T23:14:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.474+0000 7f928d5fe700 1 -- start start 2023-09-27T23:14:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9288155960 0x7f928815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f928806de20 0x7f92880c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928806d990 con 0x7f928806c7c0 2023-09-27T23:14:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928815a570 con 0x7f9288155960 2023-09-27T23:14:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928806cd80 con 0x7f928806de20 2023-09-27T23:14:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928cdfd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f928806c7c0 0x7f92880c5fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34160/0 (socket says 172.21.15.64:34160) 2023-09-27T23:14:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9287fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9288155960 0x7f928815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928cdfd700 1 -- 172.21.15.64:0/3548251521 learned_addr learned my addr 172.21.15.64:0/3548251521 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:03.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f92877fe700 1 --2- 172.21.15.64:0/3548251521 >> v2:172.21.15.123:3300/0 conn(0x7f928806de20 0x7f92880c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9286ffd700 1 -- 172.21.15.64:0/3548251521 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876894397 0 0) 0x7f928806d990 con 0x7f928806c7c0 2023-09-27T23:14:03.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9286ffd700 1 -- 172.21.15.64:0/3548251521 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9270003570 con 0x7f928806c7c0 2023-09-27T23:14:03.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9287fff700 1 -- 172.21.15.64:0/3548251521 >> v2:172.21.15.123:3300/0 conn(0x7f928806de20 msgr2=0x7f92880c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9287fff700 1 --2- 172.21.15.64:0/3548251521 >> v2:172.21.15.123:3300/0 conn(0x7f928806de20 0x7f92880c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9287fff700 1 -- 172.21.15.64:0/3548251521 >> v1:172.21.15.64:6789/0 conn(0x7f928806c7c0 legacy=0x7f92880c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9287fff700 1 -- 172.21.15.64:0/3548251521 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928806d060 con 0x7f9288155960 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9287fff700 1 --2- 172.21.15.64:0/3548251521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288155960 0x7f928815a030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f928806bf60 tx=0x7f92780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a283f0aa7e4d84ad server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9286ffd700 1 -- 172.21.15.64:0/3548251521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278011630 con 0x7f9288155960 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9286ffd700 1 -- 172.21.15.64:0/3548251521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780117b0 con 0x7f9288155960 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f9286ffd700 1 -- 172.21.15.64:0/3548251521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278011b30 con 0x7f9288155960 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- 172.21.15.64:0/3548251521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288155960 msgr2=0x7f928815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 --2- 172.21.15.64:0/3548251521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288155960 0x7f928815a030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f928806bf60 tx=0x7f92780091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- 172.21.15.64:0/3548251521 shutdown_connections 2023-09-27T23:14:03.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 --2- 172.21.15.64:0/3548251521 >> v2:172.21.15.123:3300/0 conn(0x7f928806de20 0x7f92880c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 --2- 172.21.15.64:0/3548251521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288155960 0x7f928815a030 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- 172.21.15.64:0/3548251521 >> 172.21.15.64:0/3548251521 conn(0x7f9288067bb0 msgr2=0x7f92880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- 172.21.15.64:0/3548251521 shutdown_connections 2023-09-27T23:14:03.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.478+0000 7f928d5fe700 1 -- 172.21.15.64:0/3548251521 wait complete. 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 Processor -- start 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 -- start start 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f928806c7c0 0x7f928816bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f928806de20 0x7f9288157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 0x7f92880ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288158ef0 con 0x7f928815a070 2023-09-27T23:14:03.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288158010 con 0x7f928806c7c0 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92881582e0 con 0x7f928806de20 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 0x7f92880ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 0x7f92880ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44822/0 (socket says 172.21.15.64:44822) 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 -- 172.21.15.64:0/60035529 learned_addr learned my addr 172.21.15.64:0/60035529 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f9287fff700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f928806c7c0 0x7f928816bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f92877fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f928806de20 0x7f9288157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 -- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f928806de20 msgr2=0x7f9288157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f928806de20 0x7f9288157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 -- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f928806c7c0 msgr2=0x7f928816bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f928806c7c0 0x7f928816bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 -- 172.21.15.64:0/60035529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278009420 con 0x7f928815a070 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f92877fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f928806de20 0x7f9288157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928cdfd700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 0x7f92880ca4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9280001460 tx=0x7f9280001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000a940 con 0x7f928815a070 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928000aaa0 con 0x7f928815a070 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92881585c0 con 0x7f928815a070 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000ae20 con 0x7f928815a070 2023-09-27T23:14:03.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.482+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9288158b80 con 0x7f928815a070 2023-09-27T23:14:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.486+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9288169140 con 0x7f928815a070 2023-09-27T23:14:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.486+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f9280007300 con 0x7f928815a070 2023-09-27T23:14:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.486+0000 7f926ffff700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9274060980 0x7f9274062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.486+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f928007d630 con 0x7f928815a070 2023-09-27T23:14:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.486+0000 7f9287fff700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9274060980 0x7f9274062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.486+0000 7f9287fff700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9274060980 0x7f9274062e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f928806bf60 tx=0x7f9278017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.490+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f928004d5b0 con 0x7f928815a070 2023-09-27T23:14:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.650+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f928804be40 con 0x7f9274060980 2023-09-27T23:14:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.666+0000 7f926ffff700 1 -- 172.21.15.64:0/60035529 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+364695 (secure 0 0 0) 0x7f928804be40 con 0x7f9274060980 2023-09-27T23:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9274060980 msgr2=0x7f9274062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9274060980 0x7f9274062e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f928806bf60 tx=0x7f9278017000 comp rx=0 tx=0).stop 2023-09-27T23:14:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 msgr2=0x7f92880ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 0x7f92880ca4d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9280001460 tx=0x7f9280001ae0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 shutdown_connections 2023-09-27T23:14:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9274060980 0x7f9274062e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a070 0x7f92880ca4d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f928806de20 0x7f9288157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 --2- 172.21.15.64:0/60035529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f928806c7c0 0x7f928816bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 >> 172.21.15.64:0/60035529 conn(0x7f9288067bb0 msgr2=0x7f9288150190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 shutdown_connections 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.674+0000 7f928d5fe700 1 -- 172.21.15.64:0/60035529 wait complete. 2023-09-27T23:14:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-09-27T23:14:03.150421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67688,"kb_used_data":3624,"kb_used_omap":0,"kb_used_meta":64000,"kb_avail":754907032,"statfs":{"total":773094113280,"available":773024800768,"internally_reserved":0,"allocated":3710976,"data_stored":2188832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65536000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005543"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838997+0000","last_change":"2023-09-27T23:13:48.825219+0000","last_active":"2023-09-27T23:13:48.838997+0000","last_peered":"2023-09-27T23:13:48.838997+0000","last_clean":"2023-09-27T23:13:48.838997+0000","last_became_active":"2023-09-27T23:13:48.824822+0000","last_became_peered":"2023-09-27T23:13:48.824822+0000","last_unstale":"2023-09-27T23:13:48.838997+0000","last_undegraded":"2023-09-27T23:13:48.838997+0000","last_fullsized":"2023-09-27T23:13:48.838997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:34.128831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803408+0000","last_change":"2023-09-27T23:13:44.424249+0000","last_active":"2023-09-27T23:13:48.803408+0000","last_peered":"2023-09-27T23:13:48.803408+0000","last_clean":"2023-09-27T23:13:48.803408+0000","last_became_active":"2023-09-27T23:13:44.423915+0000","last_became_peered":"2023-09-27T23:13:44.423915+0000","last_unstale":"2023-09-27T23:13:48.803408+0000","last_undegraded":"2023-09-27T23:13:48.803408+0000","last_fullsized":"2023-09-27T23:13:48.803408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821460+0000","last_change":"2023-09-27T23:13:44.796968+0000","last_active":"2023-09-27T23:13:48.821460+0000","last_peered":"2023-09-27T23:13:48.821460+0000","last_clean":"2023-09-27T23:13:48.821460+0000","last_became_active":"2023-09-27T23:13:44.795519+0000","last_became_peered":"2023-09-27T23:13:44.795519+0000","last_unstale":"2023-09-27T23:13:48.821460+0000","last_undegraded":"2023-09-27T23:13:48.821460+0000","last_fullsized":"2023-09-27T23:13:48.821460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843929+0000","last_change":"2023-09-27T23:13:48.839540+0000","last_active":"2023-09-27T23:13:48.843929+0000","last_peered":"2023-09-27T23:13:48.843929+0000","last_clean":"2023-09-27T23:13:48.843929+0000","last_became_active":"2023-09-27T23:13:48.839079+0000","last_became_peered":"2023-09-27T23:13:48.839079+0000","last_unstale":"2023-09-27T23:13:48.843929+0000","last_undegraded":"2023-09-27T23:13:48.843929+0000","last_fullsized":"2023-09-27T23:13:48.843929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825433+0000","last_change":"2023-09-27T23:13:44.416457+0000","last_active":"2023-09-27T23:13:48.825433+0000","last_peered":"2023-09-27T23:13:48.825433+0000","last_clean":"2023-09-27T23:13:48.825433+0000","last_became_active":"2023-09-27T23:13:44.416135+0000","last_became_peered":"2023-09-27T23:13:44.416135+0000","last_unstale":"2023-09-27T23:13:48.825433+0000","last_undegraded":"2023-09-27T23:13:48.825433+0000","last_fullsized":"2023-09-27T23:13:48.825433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822547+0000","last_change":"2023-09-27T23:13:44.783780+0000","last_active":"2023-09-27T23:13:48.822547+0000","last_peered":"2023-09-27T23:13:48.822547+0000","last_clean":"2023-09-27T23:13:48.822547+0000","last_became_active":"2023-09-27T23:13:44.783441+0000","last_became_peered":"2023-09-27T23:13:44.783441+0000","last_unstale":"2023-09-27T23:13:48.822547+0000","last_undegraded":"2023-09-27T23:13:48.822547+0000","last_fullsized":"2023-09-27T23:13:48.822547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840431+0000","last_change":"2023-09-27T23:13:48.835419+0000","last_active":"2023-09-27T23:13:48.840431+0000","last_peered":"2023-09-27T23:13:48.840431+0000","last_clean":"2023-09-27T23:13:48.840431+0000","last_became_active":"2023-09-27T23:13:48.834981+0000","last_became_peered":"2023-09-27T23:13:48.834981+0000","last_unstale":"2023-09-27T23:13:48.840431+0000","last_undegraded":"2023-09-27T23:13:48.840431+0000","last_fullsized":"2023-09-27T23:13:48.840431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814741+0000","last_change":"2023-09-27T23:13:44.427390+0000","last_active":"2023-09-27T23:13:48.814741+0000","last_peered":"2023-09-27T23:13:48.814741+0000","last_clean":"2023-09-27T23:13:48.814741+0000","last_became_active":"2023-09-27T23:13:44.426407+0000","last_became_peered":"2023-09-27T23:13:44.426407+0000","last_unstale":"2023-09-27T23:13:48.814741+0000","last_undegraded":"2023-09-27T23:13:48.814741+0000","last_fullsized":"2023-09-27T23:13:48.814741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820888+0000","last_change":"2023-09-27T23:13:44.803765+0000","last_active":"2023-09-27T23:13:48.820888+0000","last_peered":"2023-09-27T23:13:48.820888+0000","last_clean":"2023-09-27T23:13:48.820888+0000","last_became_active":"2023-09-27T23:13:44.803448+0000","last_became_peered":"2023-09-27T23:13:44.803448+0000","last_unstale":"2023-09-27T23:13:48.820888+0000","last_undegraded":"2023-09-27T23:13:48.820888+0000","last_fullsized":"2023-09-27T23:13:48.820888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:14.091887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834465+0000","last_change":"2023-09-27T23:13:48.831362+0000","last_active":"2023-09-27T23:13:48.834465+0000","last_peered":"2023-09-27T23:13:48.834465+0000","last_clean":"2023-09-27T23:13:48.834465+0000","last_became_active":"2023-09-27T23:13:48.830786+0000","last_became_peered":"2023-09-27T23:13:48.830786+0000","last_unstale":"2023-09-27T23:13:48.834465+0000","last_undegraded":"2023-09-27T23:13:48.834465+0000","last_fullsized":"2023-09-27T23:13:48.834465+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:24:25.298387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811157+0000","last_change":"2023-09-27T23:13:44.425581+0000","last_active":"2023-09-27T23:13:48.811157+0000","last_peered":"2023-09-27T23:13:48.811157+0000","last_clean":"2023-09-27T23:13:48.811157+0000","last_became_active":"2023-09-27T23:13:44.425084+0000","last_became_peered":"2023-09-27T23:13:44.425084+0000","last_unstale":"2023-09-27T23:13:48.811157+0000","last_undegraded":"2023-09-27T23:13:48.811157+0000","last_fullsized":"2023-09-27T23:13:48.811157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821059+0000","last_change":"2023-09-27T23:13:44.803698+0000","last_active":"2023-09-27T23:13:48.821059+0000","last_peered":"2023-09-27T23:13:48.821059+0000","last_clean":"2023-09-27T23:13:48.821059+0000","last_became_active":"2023-09-27T23:13:44.803316+0000","last_became_peered":"2023-09-27T23:13:44.803316+0000","last_unstale":"2023-09-27T23:13:48.821059+0000","last_undegraded":"2023-09-27T23:13:48.821059+0000","last_fullsized":"2023-09-27T23:13:48.821059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:10:53.031343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843374+0000","last_change":"2023-09-27T23:13:48.835746+0000","last_active":"2023-09-27T23:13:48.843374+0000","last_peered":"2023-09-27T23:13:48.843374+0000","last_clean":"2023-09-27T23:13:48.843374+0000","last_became_active":"2023-09-27T23:13:48.835306+0000","last_became_peered":"2023-09-27T23:13:48.835306+0000","last_unstale":"2023-09-27T23:13:48.843374+0000","last_undegraded":"2023-09-27T23:13:48.843374+0000","last_fullsized":"2023-09-27T23:13:48.843374+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:00:48.785735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804887+0000","last_change":"2023-09-27T23:13:44.428693+0000","last_active":"2023-09-27T23:13:48.804887+0000","last_peered":"2023-09-27T23:13:48.804887+0000","last_clean":"2023-09-27T23:13:48.804887+0000","last_became_active":"2023-09-27T23:13:44.428213+0000","last_became_peered":"2023-09-27T23:13:44.428213+0000","last_unstale":"2023-09-27T23:13:48.804887+0000","last_undegraded":"2023-09-27T23:13:48.804887+0000","last_fullsized":"2023-09-27T23:13:48.804887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810898+0000","last_change":"2023-09-27T23:13:44.791611+0000","last_active":"2023-09-27T23:13:48.810898+0000","last_peered":"2023-09-27T23:13:48.810898+0000","last_clean":"2023-09-27T23:13:48.810898+0000","last_became_active":"2023-09-27T23:13:44.791292+0000","last_became_peered":"2023-09-27T23:13:44.791292+0000","last_unstale":"2023-09-27T23:13:48.810898+0000","last_undegraded":"2023-09-27T23:13:48.810898+0000","last_fullsized":"2023-09-27T23:13:48.810898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:08.690781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840009+0000","last_change":"2023-09-27T23:13:48.827264+0000","last_active":"2023-09-27T23:13:48.840009+0000","last_peered":"2023-09-27T23:13:48.840009+0000","last_clean":"2023-09-27T23:13:48.840009+0000","last_became_active":"2023-09-27T23:13:48.826832+0000","last_became_peered":"2023-09-27T23:13:48.826832+0000","last_unstale":"2023-09-27T23:13:48.840009+0000","last_undegraded":"2023-09-27T23:13:48.840009+0000","last_fullsized":"2023-09-27T23:13:48.840009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:48:20.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821181+0000","last_change":"2023-09-27T23:13:44.423920+0000","last_active":"2023-09-27T23:13:48.821181+0000","last_peered":"2023-09-27T23:13:48.821181+0000","last_clean":"2023-09-27T23:13:48.821181+0000","last_became_active":"2023-09-27T23:13:44.422754+0000","last_became_peered":"2023-09-27T23:13:44.422754+0000","last_unstale":"2023-09-27T23:13:48.821181+0000","last_undegraded":"2023-09-27T23:13:48.821181+0000","last_fullsized":"2023-09-27T23:13:48.821181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:16.888839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825025+0000","last_change":"2023-09-27T23:13:44.790022+0000","last_active":"2023-09-27T23:13:48.825025+0000","last_peered":"2023-09-27T23:13:48.825025+0000","last_clean":"2023-09-27T23:13:48.825025+0000","last_became_active":"2023-09-27T23:13:44.789052+0000","last_became_peered":"2023-09-27T23:13:44.789052+0000","last_unstale":"2023-09-27T23:13:48.825025+0000","last_undegraded":"2023-09-27T23:13:48.825025+0000","last_fullsized":"2023-09-27T23:13:48.825025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.841311+0000","last_change":"2023-09-27T23:13:48.835634+0000","last_active":"2023-09-27T23:13:48.841311+0000","last_peered":"2023-09-27T23:13:48.841311+0000","last_clean":"2023-09-27T23:13:48.841311+0000","last_became_active":"2023-09-27T23:13:48.834989+0000","last_became_peered":"2023-09-27T23:13:48.834989+0000","last_unstale":"2023-09-27T23:13:48.841311+0000","last_undegraded":"2023-09-27T23:13:48.841311+0000","last_fullsized":"2023-09-27T23:13:48.841311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:09.944701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802920+0000","last_change":"2023-09-27T23:13:44.424671+0000","last_active":"2023-09-27T23:13:48.802920+0000","last_peered":"2023-09-27T23:13:48.802920+0000","last_clean":"2023-09-27T23:13:48.802920+0000","last_became_active":"2023-09-27T23:13:44.424321+0000","last_became_peered":"2023-09-27T23:13:44.424321+0000","last_unstale":"2023-09-27T23:13:48.802920+0000","last_undegraded":"2023-09-27T23:13:48.802920+0000","last_fullsized":"2023-09-27T23:13:48.802920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:44:43.453425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803496+0000","last_change":"2023-09-27T23:13:44.792445+0000","last_active":"2023-09-27T23:13:48.803496+0000","last_peered":"2023-09-27T23:13:48.803496+0000","last_clean":"2023-09-27T23:13:48.803496+0000","last_became_active":"2023-09-27T23:13:44.790669+0000","last_became_peered":"2023-09-27T23:13:44.790669+0000","last_unstale":"2023-09-27T23:13:48.803496+0000","last_undegraded":"2023-09-27T23:13:48.803496+0000","last_fullsized":"2023-09-27T23:13:48.803496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833795+0000","last_change":"2023-09-27T23:13:48.824674+0000","last_active":"2023-09-27T23:13:48.833795+0000","last_peered":"2023-09-27T23:13:48.833795+0000","last_clean":"2023-09-27T23:13:48.833795+0000","last_became_active":"2023-09-27T23:13:48.824220+0000","last_became_peered":"2023-09-27T23:13:48.824220+0000","last_unstale":"2023-09-27T23:13:48.833795+0000","last_undegraded":"2023-09-27T23:13:48.833795+0000","last_fullsized":"2023-09-27T23:13:48.833795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:22.013842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822020+0000","last_change":"2023-09-27T23:13:44.415940+0000","last_active":"2023-09-27T23:13:48.822020+0000","last_peered":"2023-09-27T23:13:48.822020+0000","last_clean":"2023-09-27T23:13:48.822020+0000","last_became_active":"2023-09-27T23:13:44.415355+0000","last_became_peered":"2023-09-27T23:13:44.415355+0000","last_unstale":"2023-09-27T23:13:48.822020+0000","last_undegraded":"2023-09-27T23:13:48.822020+0000","last_fullsized":"2023-09-27T23:13:48.822020+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813805+0000","last_change":"2023-09-27T23:13:44.807290+0000","last_active":"2023-09-27T23:13:48.813805+0000","last_peered":"2023-09-27T23:13:48.813805+0000","last_clean":"2023-09-27T23:13:48.813805+0000","last_became_active":"2023-09-27T23:13:44.806991+0000","last_became_peered":"2023-09-27T23:13:44.806991+0000","last_unstale":"2023-09-27T23:13:48.813805+0000","last_undegraded":"2023-09-27T23:13:48.813805+0000","last_fullsized":"2023-09-27T23:13:48.813805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:41.102884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821934+0000","last_change":"2023-09-27T23:13:44.797997+0000","last_active":"2023-09-27T23:13:48.821934+0000","last_peered":"2023-09-27T23:13:48.821934+0000","last_clean":"2023-09-27T23:13:48.821934+0000","last_became_active":"2023-09-27T23:13:44.797689+0000","last_became_peered":"2023-09-27T23:13:44.797689+0000","last_unstale":"2023-09-27T23:13:48.821934+0000","last_undegraded":"2023-09-27T23:13:48.821934+0000","last_fullsized":"2023-09-27T23:13:48.821934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803641+0000","last_change":"2023-09-27T23:13:44.420293+0000","last_active":"2023-09-27T23:13:48.803641+0000","last_peered":"2023-09-27T23:13:48.803641+0000","last_clean":"2023-09-27T23:13:48.803641+0000","last_became_active":"2023-09-27T23:13:44.419584+0000","last_became_peered":"2023-09-27T23:13:44.419584+0000","last_unstale":"2023-09-27T23:13:48.803641+0000","last_undegraded":"2023-09-27T23:13:48.803641+0000","last_fullsized":"2023-09-27T23:13:48.803641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843828+0000","last_change":"2023-09-27T23:13:48.842799+0000","last_active":"2023-09-27T23:13:48.843828+0000","last_peered":"2023-09-27T23:13:48.843828+0000","last_clean":"2023-09-27T23:13:48.843828+0000","last_became_active":"2023-09-27T23:13:48.842271+0000","last_became_peered":"2023-09-27T23:13:48.842271+0000","last_unstale":"2023-09-27T23:13:48.843828+0000","last_undegraded":"2023-09-27T23:13:48.843828+0000","last_fullsized":"2023-09-27T23:13:48.843828+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:26.172918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822463+0000","last_change":"2023-09-27T23:13:44.791363+0000","last_active":"2023-09-27T23:13:48.822463+0000","last_peered":"2023-09-27T23:13:48.822463+0000","last_clean":"2023-09-27T23:13:48.822463+0000","last_became_active":"2023-09-27T23:13:44.786248+0000","last_became_peered":"2023-09-27T23:13:44.786248+0000","last_unstale":"2023-09-27T23:13:48.822463+0000","last_undegraded":"2023-09-27T23:13:48.822463+0000","last_fullsized":"2023-09-27T23:13:48.822463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:22.779025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815748+0000","last_change":"2023-09-27T23:13:44.416087+0000","last_active":"2023-09-27T23:13:48.815748+0000","last_peered":"2023-09-27T23:13:48.815748+0000","last_clean":"2023-09-27T23:13:48.815748+0000","last_became_active":"2023-09-27T23:13:44.415755+0000","last_became_peered":"2023-09-27T23:13:44.415755+0000","last_unstale":"2023-09-27T23:13:48.815748+0000","last_undegraded":"2023-09-27T23:13:48.815748+0000","last_fullsized":"2023-09-27T23:13:48.815748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:00.881473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826650+0000","last_change":"2023-09-27T23:13:48.825181+0000","last_active":"2023-09-27T23:13:48.826650+0000","last_peered":"2023-09-27T23:13:48.826650+0000","last_clean":"2023-09-27T23:13:48.826650+0000","last_became_active":"2023-09-27T23:13:48.824750+0000","last_became_peered":"2023-09-27T23:13:48.824750+0000","last_unstale":"2023-09-27T23:13:48.826650+0000","last_undegraded":"2023-09-27T23:13:48.826650+0000","last_fullsized":"2023-09-27T23:13:48.826650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803126+0000","last_change":"2023-09-27T23:13:44.423841+0000","last_active":"2023-09-27T23:13:48.803126+0000","last_peered":"2023-09-27T23:13:48.803126+0000","last_clean":"2023-09-27T23:13:48.803126+0000","last_became_active":"2023-09-27T23:13:44.423450+0000","last_became_peered":"2023-09-27T23:13:44.423450+0000","last_unstale":"2023-09-27T23:13:48.803126+0000","last_undegraded":"2023-09-27T23:13:48.803126+0000","last_fullsized":"2023-09-27T23:13:48.803126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804382+0000","last_change":"2023-09-27T23:13:44.789353+0000","last_active":"2023-09-27T23:13:48.804382+0000","last_peered":"2023-09-27T23:13:48.804382+0000","last_clean":"2023-09-27T23:13:48.804382+0000","last_became_active":"2023-09-27T23:13:44.788925+0000","last_became_peered":"2023-09-27T23:13:44.788925+0000","last_unstale":"2023-09-27T23:13:48.804382+0000","last_undegraded":"2023-09-27T23:13:48.804382+0000","last_fullsized":"2023-09-27T23:13:48.804382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:24.773257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842925+0000","last_change":"2023-09-27T23:13:48.840258+0000","last_active":"2023-09-27T23:13:48.842925+0000","last_peered":"2023-09-27T23:13:48.842925+0000","last_clean":"2023-09-27T23:13:48.842925+0000","last_became_active":"2023-09-27T23:13:48.839796+0000","last_became_peered":"2023-09-27T23:13:48.839796+0000","last_unstale":"2023-09-27T23:13:48.842925+0000","last_undegraded":"2023-09-27T23:13:48.842925+0000","last_fullsized":"2023-09-27T23:13:48.842925+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"29'9","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-27T23:13:53.926546+0000","last_change":"2023-09-27T23:13:44.422959+0000","last_active":"2023-09-27T23:13:53.926546+0000","last_peered":"2023-09-27T23:13:53.926546+0000","last_clean":"2023-09-27T23:13:53.926546+0000","last_became_active":"2023-09-27T23:13:44.420682+0000","last_became_peered":"2023-09-27T23:13:44.420682+0000","last_unstale":"2023-09-27T23:13:53.926546+0000","last_undegraded":"2023-09-27T23:13:53.926546+0000","last_fullsized":"2023-09-27T23:13:53.926546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821157+0000","last_change":"2023-09-27T23:13:44.792715+0000","last_active":"2023-09-27T23:13:48.821157+0000","last_peered":"2023-09-27T23:13:48.821157+0000","last_clean":"2023-09-27T23:13:48.821157+0000","last_became_active":"2023-09-27T23:13:44.792356+0000","last_became_peered":"2023-09-27T23:13:44.792356+0000","last_unstale":"2023-09-27T23:13:48.821157+0000","last_undegraded":"2023-09-27T23:13:48.821157+0000","last_fullsized":"2023-09-27T23:13:48.821157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843134+0000","last_change":"2023-09-27T23:13:48.830683+0000","last_active":"2023-09-27T23:13:48.843134+0000","last_peered":"2023-09-27T23:13:48.843134+0000","last_clean":"2023-09-27T23:13:48.843134+0000","last_became_active":"2023-09-27T23:13:48.830149+0000","last_became_peered":"2023-09-27T23:13:48.830149+0000","last_unstale":"2023-09-27T23:13:48.843134+0000","last_undegraded":"2023-09-27T23:13:48.843134+0000","last_fullsized":"2023-09-27T23:13:48.843134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:31:07.801049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821012+0000","last_change":"2023-09-27T23:13:44.807208+0000","last_active":"2023-09-27T23:13:48.821012+0000","last_peered":"2023-09-27T23:13:48.821012+0000","last_clean":"2023-09-27T23:13:48.821012+0000","last_became_active":"2023-09-27T23:13:44.806937+0000","last_became_peered":"2023-09-27T23:13:44.806937+0000","last_unstale":"2023-09-27T23:13:48.821012+0000","last_undegraded":"2023-09-27T23:13:48.821012+0000","last_fullsized":"2023-09-27T23:13:48.821012+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:03:17.944613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813854+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:13:48.813854+0000","last_peered":"2023-09-27T23:13:48.813854+0000","last_clean":"2023-09-27T23:13:48.813854+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:13:48.813854+0000","last_undegraded":"2023-09-27T23:13:48.813854+0000","last_fullsized":"2023-09-27T23:13:48.813854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-09-27T23:13:49.971771+0000","last_change":"2023-09-27T23:13:44.421149+0000","last_active":"2023-09-27T23:13:49.971771+0000","last_peered":"2023-09-27T23:13:49.971771+0000","last_clean":"2023-09-27T23:13:49.971771+0000","last_became_active":"2023-09-27T23:13:44.419644+0000","last_became_peered":"2023-09-27T23:13:44.419644+0000","last_unstale":"2023-09-27T23:13:49.971771+0000","last_undegraded":"2023-09-27T23:13:49.971771+0000","last_fullsized":"2023-09-27T23:13:49.971771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842921+0000","last_change":"2023-09-27T23:13:48.840751+0000","last_active":"2023-09-27T23:13:48.842921+0000","last_peered":"2023-09-27T23:13:48.842921+0000","last_clean":"2023-09-27T23:13:48.842921+0000","last_became_active":"2023-09-27T23:13:48.840353+0000","last_became_peered":"2023-09-27T23:13:48.840353+0000","last_unstale":"2023-09-27T23:13:48.842921+0000","last_undegraded":"2023-09-27T23:13:48.842921+0000","last_fullsized":"2023-09-27T23:13:48.842921+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:59:09.353088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821991+0000","last_change":"2023-09-27T23:13:44.791105+0000","last_active":"2023-09-27T23:13:48.821991+0000","last_peered":"2023-09-27T23:13:48.821991+0000","last_clean":"2023-09-27T23:13:48.821991+0000","last_became_active":"2023-09-27T23:13:44.790824+0000","last_became_peered":"2023-09-27T23:13:44.790824+0000","last_unstale":"2023-09-27T23:13:48.821991+0000","last_undegraded":"2023-09-27T23:13:48.821991+0000","last_fullsized":"2023-09-27T23:13:48.821991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807265+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:13:48.807265+0000","last_peered":"2023-09-27T23:13:48.807265+0000","last_clean":"2023-09-27T23:13:48.807265+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:13:48.807265+0000","last_undegraded":"2023-09-27T23:13:48.807265+0000","last_fullsized":"2023-09-27T23:13:48.807265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825400+0000","last_change":"2023-09-27T23:13:44.421998+0000","last_active":"2023-09-27T23:13:48.825400+0000","last_peered":"2023-09-27T23:13:48.825400+0000","last_clean":"2023-09-27T23:13:48.825400+0000","last_became_active":"2023-09-27T23:13:44.416358+0000","last_became_peered":"2023-09-27T23:13:44.416358+0000","last_unstale":"2023-09-27T23:13:48.825400+0000","last_undegraded":"2023-09-27T23:13:48.825400+0000","last_fullsized":"2023-09-27T23:13:48.825400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840017+0000","last_change":"2023-09-27T23:13:48.827758+0000","last_active":"2023-09-27T23:13:48.840017+0000","last_peered":"2023-09-27T23:13:48.840017+0000","last_clean":"2023-09-27T23:13:48.840017+0000","last_became_active":"2023-09-27T23:13:48.827144+0000","last_became_peered":"2023-09-27T23:13:48.827144+0000","last_unstale":"2023-09-27T23:13:48.840017+0000","last_undegraded":"2023-09-27T23:13:48.840017+0000","last_fullsized":"2023-09-27T23:13:48.840017+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:02:44.973308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804641+0000","last_change":"2023-09-27T23:13:44.807107+0000","last_active":"2023-09-27T23:13:48.804641+0000","last_peered":"2023-09-27T23:13:48.804641+0000","last_clean":"2023-09-27T23:13:48.804641+0000","last_became_active":"2023-09-27T23:13:44.806739+0000","last_became_peered":"2023-09-27T23:13:44.806739+0000","last_unstale":"2023-09-27T23:13:48.804641+0000","last_undegraded":"2023-09-27T23:13:48.804641+0000","last_fullsized":"2023-09-27T23:13:48.804641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803837+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:13:48.803837+0000","last_peered":"2023-09-27T23:13:48.803837+0000","last_clean":"2023-09-27T23:13:48.803837+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:13:48.803837+0000","last_undegraded":"2023-09-27T23:13:48.803837+0000","last_fullsized":"2023-09-27T23:13:48.803837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802757+0000","last_change":"2023-09-27T23:13:44.419685+0000","last_active":"2023-09-27T23:13:48.802757+0000","last_peered":"2023-09-27T23:13:48.802757+0000","last_clean":"2023-09-27T23:13:48.802757+0000","last_became_active":"2023-09-27T23:13:44.419369+0000","last_became_peered":"2023-09-27T23:13:44.419369+0000","last_unstale":"2023-09-27T23:13:48.802757+0000","last_undegraded":"2023-09-27T23:13:48.802757+0000","last_fullsized":"2023-09-27T23:13:48.802757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:04.718338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.826218+0000","last_change":"2023-09-27T23:13:48.822456+0000","last_active":"2023-09-27T23:13:48.826218+0000","last_peered":"2023-09-27T23:13:48.826218+0000","last_clean":"2023-09-27T23:13:48.826218+0000","last_became_active":"2023-09-27T23:13:48.821878+0000","last_became_peered":"2023-09-27T23:13:48.821878+0000","last_unstale":"2023-09-27T23:13:48.826218+0000","last_undegraded":"2023-09-27T23:13:48.826218+0000","last_fullsized":"2023-09-27T23:13:48.826218+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807086+0000","last_change":"2023-09-27T23:13:44.777613+0000","last_active":"2023-09-27T23:13:48.807086+0000","last_peered":"2023-09-27T23:13:48.807086+0000","last_clean":"2023-09-27T23:13:48.807086+0000","last_became_active":"2023-09-27T23:13:44.777267+0000","last_became_peered":"2023-09-27T23:13:44.777267+0000","last_unstale":"2023-09-27T23:13:48.807086+0000","last_undegraded":"2023-09-27T23:13:48.807086+0000","last_fullsized":"2023-09-27T23:13:48.807086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:42:36.525422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802984+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:13:48.802984+0000","last_peered":"2023-09-27T23:13:48.802984+0000","last_clean":"2023-09-27T23:13:48.802984+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:13:48.802984+0000","last_undegraded":"2023-09-27T23:13:48.802984+0000","last_fullsized":"2023-09-27T23:13:48.802984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814830+0000","last_change":"2023-09-27T23:13:44.422663+0000","last_active":"2023-09-27T23:13:48.814830+0000","last_peered":"2023-09-27T23:13:48.814830+0000","last_clean":"2023-09-27T23:13:48.814830+0000","last_became_active":"2023-09-27T23:13:44.422361+0000","last_became_peered":"2023-09-27T23:13:44.422361+0000","last_unstale":"2023-09-27T23:13:48.814830+0000","last_undegraded":"2023-09-27T23:13:48.814830+0000","last_fullsized":"2023-09-27T23:13:48.814830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:02.773067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.833765+0000","last_change":"2023-09-27T23:13:48.824670+0000","last_active":"2023-09-27T23:13:48.833765+0000","last_peered":"2023-09-27T23:13:48.833765+0000","last_clean":"2023-09-27T23:13:48.833765+0000","last_became_active":"2023-09-27T23:13:48.824249+0000","last_became_peered":"2023-09-27T23:13:48.824249+0000","last_unstale":"2023-09-27T23:13:48.833765+0000","last_undegraded":"2023-09-27T23:13:48.833765+0000","last_fullsized":"2023-09-27T23:13:48.833765+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:03.729907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802514+0000","last_change":"2023-09-27T23:13:44.778136+0000","last_active":"2023-09-27T23:13:48.802514+0000","last_peered":"2023-09-27T23:13:48.802514+0000","last_clean":"2023-09-27T23:13:48.802514+0000","last_became_active":"2023-09-27T23:13:44.777865+0000","last_became_peered":"2023-09-27T23:13:44.777865+0000","last_unstale":"2023-09-27T23:13:48.802514+0000","last_undegraded":"2023-09-27T23:13:48.802514+0000","last_fullsized":"2023-09-27T23:13:48.802514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:46:04.289644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821294+0000","last_change":"2023-09-27T23:13:44.424183+0000","last_active":"2023-09-27T23:13:48.821294+0000","last_peered":"2023-09-27T23:13:48.821294+0000","last_clean":"2023-09-27T23:13:48.821294+0000","last_became_active":"2023-09-27T23:13:44.422919+0000","last_became_peered":"2023-09-27T23:13:44.422919+0000","last_unstale":"2023-09-27T23:13:48.821294+0000","last_undegraded":"2023-09-27T23:13:48.821294+0000","last_fullsized":"2023-09-27T23:13:48.821294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813932+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:13:48.813932+0000","last_peered":"2023-09-27T23:13:48.813932+0000","last_clean":"2023-09-27T23:13:48.813932+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:13:48.813932+0000","last_undegraded":"2023-09-27T23:13:48.813932+0000","last_fullsized":"2023-09-27T23:13:48.813932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839053+0000","last_change":"2023-09-27T23:13:48.836163+0000","last_active":"2023-09-27T23:13:48.839053+0000","last_peered":"2023-09-27T23:13:48.839053+0000","last_clean":"2023-09-27T23:13:48.839053+0000","last_became_active":"2023-09-27T23:13:48.835655+0000","last_became_peered":"2023-09-27T23:13:48.835655+0000","last_unstale":"2023-09-27T23:13:48.839053+0000","last_undegraded":"2023-09-27T23:13:48.839053+0000","last_fullsized":"2023-09-27T23:13:48.839053+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802588+0000","last_change":"2023-09-27T23:13:44.777627+0000","last_active":"2023-09-27T23:13:48.802588+0000","last_peered":"2023-09-27T23:13:48.802588+0000","last_clean":"2023-09-27T23:13:48.802588+0000","last_became_active":"2023-09-27T23:13:44.777239+0000","last_became_peered":"2023-09-27T23:13:44.777239+0000","last_unstale":"2023-09-27T23:13:48.802588+0000","last_undegraded":"2023-09-27T23:13:48.802588+0000","last_fullsized":"2023-09-27T23:13:48.802588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:34.572519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.800241+0000","last_change":"2023-09-27T23:13:44.419827+0000","last_active":"2023-09-27T23:13:48.800241+0000","last_peered":"2023-09-27T23:13:48.800241+0000","last_clean":"2023-09-27T23:13:48.800241+0000","last_became_active":"2023-09-27T23:13:44.419535+0000","last_became_peered":"2023-09-27T23:13:44.419535+0000","last_unstale":"2023-09-27T23:13:48.800241+0000","last_undegraded":"2023-09-27T23:13:48.800241+0000","last_fullsized":"2023-09-27T23:13:48.800241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:08:26.061286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813256+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:13:48.813256+0000","last_peered":"2023-09-27T23:13:48.813256+0000","last_clean":"2023-09-27T23:13:48.813256+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:13:48.813256+0000","last_undegraded":"2023-09-27T23:13:48.813256+0000","last_fullsized":"2023-09-27T23:13:48.813256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842381+0000","last_change":"2023-09-27T23:13:48.839688+0000","last_active":"2023-09-27T23:13:48.842381+0000","last_peered":"2023-09-27T23:13:48.842381+0000","last_clean":"2023-09-27T23:13:48.842381+0000","last_became_active":"2023-09-27T23:13:48.839257+0000","last_became_peered":"2023-09-27T23:13:48.839257+0000","last_unstale":"2023-09-27T23:13:48.842381+0000","last_undegraded":"2023-09-27T23:13:48.842381+0000","last_fullsized":"2023-09-27T23:13:48.842381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:23.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803724+0000","last_change":"2023-09-27T23:13:44.796668+0000","last_active":"2023-09-27T23:13:48.803724+0000","last_peered":"2023-09-27T23:13:48.803724+0000","last_clean":"2023-09-27T23:13:48.803724+0000","last_became_active":"2023-09-27T23:13:44.795170+0000","last_became_peered":"2023-09-27T23:13:44.795170+0000","last_unstale":"2023-09-27T23:13:48.803724+0000","last_undegraded":"2023-09-27T23:13:48.803724+0000","last_fullsized":"2023-09-27T23:13:48.803724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802500+0000","last_change":"2023-09-27T23:13:44.428369+0000","last_active":"2023-09-27T23:13:48.802500+0000","last_peered":"2023-09-27T23:13:48.802500+0000","last_clean":"2023-09-27T23:13:48.802500+0000","last_became_active":"2023-09-27T23:13:44.427956+0000","last_became_peered":"2023-09-27T23:13:44.427956+0000","last_unstale":"2023-09-27T23:13:48.802500+0000","last_undegraded":"2023-09-27T23:13:48.802500+0000","last_fullsized":"2023-09-27T23:13:48.802500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:45:03.020732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803011+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:13:48.803011+0000","last_peered":"2023-09-27T23:13:48.803011+0000","last_clean":"2023-09-27T23:13:48.803011+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:13:48.803011+0000","last_undegraded":"2023-09-27T23:13:48.803011+0000","last_fullsized":"2023-09-27T23:13:48.803011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834595+0000","last_change":"2023-09-27T23:13:48.830308+0000","last_active":"2023-09-27T23:13:48.834595+0000","last_peered":"2023-09-27T23:13:48.834595+0000","last_clean":"2023-09-27T23:13:48.834595+0000","last_became_active":"2023-09-27T23:13:48.829698+0000","last_became_peered":"2023-09-27T23:13:48.829698+0000","last_unstale":"2023-09-27T23:13:48.834595+0000","last_undegraded":"2023-09-27T23:13:48.834595+0000","last_fullsized":"2023-09-27T23:13:48.834595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:27:58.121695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812509+0000","last_change":"2023-09-27T23:13:44.804857+0000","last_active":"2023-09-27T23:13:48.812509+0000","last_peered":"2023-09-27T23:13:48.812509+0000","last_clean":"2023-09-27T23:13:48.812509+0000","last_became_active":"2023-09-27T23:13:44.804486+0000","last_became_peered":"2023-09-27T23:13:44.804486+0000","last_unstale":"2023-09-27T23:13:48.812509+0000","last_undegraded":"2023-09-27T23:13:48.812509+0000","last_fullsized":"2023-09-27T23:13:48.812509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822742+0000","last_change":"2023-09-27T23:13:44.416101+0000","last_active":"2023-09-27T23:13:48.822742+0000","last_peered":"2023-09-27T23:13:48.822742+0000","last_clean":"2023-09-27T23:13:48.822742+0000","last_became_active":"2023-09-27T23:13:44.415693+0000","last_became_peered":"2023-09-27T23:13:44.415693+0000","last_unstale":"2023-09-27T23:13:48.822742+0000","last_undegraded":"2023-09-27T23:13:48.822742+0000","last_fullsized":"2023-09-27T23:13:48.822742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:31.796132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813962+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:13:48.813962+0000","last_peered":"2023-09-27T23:13:48.813962+0000","last_clean":"2023-09-27T23:13:48.813962+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:13:48.813962+0000","last_undegraded":"2023-09-27T23:13:48.813962+0000","last_fullsized":"2023-09-27T23:13:48.813962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803349+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:13:48.803349+0000","last_peered":"2023-09-27T23:13:48.803349+0000","last_clean":"2023-09-27T23:13:48.803349+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:13:48.803349+0000","last_undegraded":"2023-09-27T23:13:48.803349+0000","last_fullsized":"2023-09-27T23:13:48.803349+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829119+0000","last_change":"2023-09-27T23:13:48.827977+0000","last_active":"2023-09-27T23:13:48.829119+0000","last_peered":"2023-09-27T23:13:48.829119+0000","last_clean":"2023-09-27T23:13:48.829119+0000","last_became_active":"2023-09-27T23:13:48.827465+0000","last_became_peered":"2023-09-27T23:13:48.827465+0000","last_unstale":"2023-09-27T23:13:48.829119+0000","last_undegraded":"2023-09-27T23:13:48.829119+0000","last_fullsized":"2023-09-27T23:13:48.829119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:18:57.307510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803299+0000","last_change":"2023-09-27T23:13:44.786471+0000","last_active":"2023-09-27T23:13:48.803299+0000","last_peered":"2023-09-27T23:13:48.803299+0000","last_clean":"2023-09-27T23:13:48.803299+0000","last_became_active":"2023-09-27T23:13:44.786182+0000","last_became_peered":"2023-09-27T23:13:44.786182+0000","last_unstale":"2023-09-27T23:13:48.803299+0000","last_undegraded":"2023-09-27T23:13:48.803299+0000","last_fullsized":"2023-09-27T23:13:48.803299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:04:06.713350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822086+0000","last_change":"2023-09-27T23:13:44.422925+0000","last_active":"2023-09-27T23:13:48.822086+0000","last_peered":"2023-09-27T23:13:48.822086+0000","last_clean":"2023-09-27T23:13:48.822086+0000","last_became_active":"2023-09-27T23:13:44.422547+0000","last_became_peered":"2023-09-27T23:13:44.422547+0000","last_unstale":"2023-09-27T23:13:48.822086+0000","last_undegraded":"2023-09-27T23:13:48.822086+0000","last_fullsized":"2023-09-27T23:13:48.822086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.839810+0000","last_change":"2023-09-27T23:13:48.824583+0000","last_active":"2023-09-27T23:13:48.839810+0000","last_peered":"2023-09-27T23:13:48.839810+0000","last_clean":"2023-09-27T23:13:48.839810+0000","last_became_active":"2023-09-27T23:13:48.824080+0000","last_became_peered":"2023-09-27T23:13:48.824080+0000","last_unstale":"2023-09-27T23:13:48.839810+0000","last_undegraded":"2023-09-27T23:13:48.839810+0000","last_fullsized":"2023-09-27T23:13:48.839810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:29:19.951565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821416+0000","last_change":"2023-09-27T23:13:44.792934+0000","last_active":"2023-09-27T23:13:48.821416+0000","last_peered":"2023-09-27T23:13:48.821416+0000","last_clean":"2023-09-27T23:13:48.821416+0000","last_became_active":"2023-09-27T23:13:44.791774+0000","last_became_peered":"2023-09-27T23:13:44.791774+0000","last_unstale":"2023-09-27T23:13:48.821416+0000","last_undegraded":"2023-09-27T23:13:48.821416+0000","last_fullsized":"2023-09-27T23:13:48.821416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821113+0000","last_change":"2023-09-27T23:13:44.418284+0000","last_active":"2023-09-27T23:13:48.821113+0000","last_peered":"2023-09-27T23:13:48.821113+0000","last_clean":"2023-09-27T23:13:48.821113+0000","last_became_active":"2023-09-27T23:13:44.417952+0000","last_became_peered":"2023-09-27T23:13:44.417952+0000","last_unstale":"2023-09-27T23:13:48.821113+0000","last_undegraded":"2023-09-27T23:13:48.821113+0000","last_fullsized":"2023-09-27T23:13:48.821113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.840441+0000","last_change":"2023-09-27T23:13:48.835487+0000","last_active":"2023-09-27T23:13:48.840441+0000","last_peered":"2023-09-27T23:13:48.840441+0000","last_clean":"2023-09-27T23:13:48.840441+0000","last_became_active":"2023-09-27T23:13:48.835043+0000","last_became_peered":"2023-09-27T23:13:48.835043+0000","last_unstale":"2023-09-27T23:13:48.840441+0000","last_undegraded":"2023-09-27T23:13:48.840441+0000","last_fullsized":"2023-09-27T23:13:48.840441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:43:53.193315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821389+0000","last_change":"2023-09-27T23:13:44.797452+0000","last_active":"2023-09-27T23:13:48.821389+0000","last_peered":"2023-09-27T23:13:48.821389+0000","last_clean":"2023-09-27T23:13:48.821389+0000","last_became_active":"2023-09-27T23:13:44.797056+0000","last_became_peered":"2023-09-27T23:13:44.797056+0000","last_unstale":"2023-09-27T23:13:48.821389+0000","last_undegraded":"2023-09-27T23:13:48.821389+0000","last_fullsized":"2023-09-27T23:13:48.821389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812548+0000","last_change":"2023-09-27T23:13:44.426307+0000","last_active":"2023-09-27T23:13:48.812548+0000","last_peered":"2023-09-27T23:13:48.812548+0000","last_clean":"2023-09-27T23:13:48.812548+0000","last_became_active":"2023-09-27T23:13:44.425925+0000","last_became_peered":"2023-09-27T23:13:44.425925+0000","last_unstale":"2023-09-27T23:13:48.812548+0000","last_undegraded":"2023-09-27T23:13:48.812548+0000","last_fullsized":"2023-09-27T23:13:48.812548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:44:53.089822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.843420+0000","last_change":"2023-09-27T23:13:48.835916+0000","last_active":"2023-09-27T23:13:48.843420+0000","last_peered":"2023-09-27T23:13:48.843420+0000","last_clean":"2023-09-27T23:13:48.843420+0000","last_became_active":"2023-09-27T23:13:48.835201+0000","last_became_peered":"2023-09-27T23:13:48.835201+0000","last_unstale":"2023-09-27T23:13:48.843420+0000","last_undegraded":"2023-09-27T23:13:48.843420+0000","last_fullsized":"2023-09-27T23:13:48.843420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:15:13.817042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821512+0000","last_change":"2023-09-27T23:13:44.804376+0000","last_active":"2023-09-27T23:13:48.821512+0000","last_peered":"2023-09-27T23:13:48.821512+0000","last_clean":"2023-09-27T23:13:48.821512+0000","last_became_active":"2023-09-27T23:13:44.803820+0000","last_became_peered":"2023-09-27T23:13:44.803820+0000","last_unstale":"2023-09-27T23:13:48.821512+0000","last_undegraded":"2023-09-27T23:13:48.821512+0000","last_fullsized":"2023-09-27T23:13:48.821512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:45:33.486648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825618+0000","last_change":"2023-09-27T23:13:44.422613+0000","last_active":"2023-09-27T23:13:48.825618+0000","last_peered":"2023-09-27T23:13:48.825618+0000","last_clean":"2023-09-27T23:13:48.825618+0000","last_became_active":"2023-09-27T23:13:44.422341+0000","last_became_peered":"2023-09-27T23:13:44.422341+0000","last_unstale":"2023-09-27T23:13:48.825618+0000","last_undegraded":"2023-09-27T23:13:48.825618+0000","last_fullsized":"2023-09-27T23:13:48.825618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:49:26.757372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825159+0000","last_change":"2023-09-27T23:13:48.821288+0000","last_active":"2023-09-27T23:13:48.825159+0000","last_peered":"2023-09-27T23:13:48.825159+0000","last_clean":"2023-09-27T23:13:48.825159+0000","last_became_active":"2023-09-27T23:13:48.820648+0000","last_became_peered":"2023-09-27T23:13:48.820648+0000","last_unstale":"2023-09-27T23:13:48.825159+0000","last_undegraded":"2023-09-27T23:13:48.825159+0000","last_fullsized":"2023-09-27T23:13:48.825159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:33:17.485779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804537+0000","last_change":"2023-09-27T23:13:44.795149+0000","last_active":"2023-09-27T23:13:48.804537+0000","last_peered":"2023-09-27T23:13:48.804537+0000","last_clean":"2023-09-27T23:13:48.804537+0000","last_became_active":"2023-09-27T23:13:44.794005+0000","last_became_peered":"2023-09-27T23:13:44.794005+0000","last_unstale":"2023-09-27T23:13:48.804537+0000","last_undegraded":"2023-09-27T23:13:48.804537+0000","last_fullsized":"2023-09-27T23:13:48.804537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.811248+0000","last_change":"2023-09-27T23:13:44.422676+0000","last_active":"2023-09-27T23:13:48.811248+0000","last_peered":"2023-09-27T23:13:48.811248+0000","last_clean":"2023-09-27T23:13:48.811248+0000","last_became_active":"2023-09-27T23:13:44.422396+0000","last_became_peered":"2023-09-27T23:13:44.422396+0000","last_unstale":"2023-09-27T23:13:48.811248+0000","last_undegraded":"2023-09-27T23:13:48.811248+0000","last_fullsized":"2023-09-27T23:13:48.811248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:19:07.681636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.838657+0000","last_change":"2023-09-27T23:13:48.835879+0000","last_active":"2023-09-27T23:13:48.838657+0000","last_peered":"2023-09-27T23:13:48.838657+0000","last_clean":"2023-09-27T23:13:48.838657+0000","last_became_active":"2023-09-27T23:13:48.835422+0000","last_became_peered":"2023-09-27T23:13:48.835422+0000","last_unstale":"2023-09-27T23:13:48.838657+0000","last_undegraded":"2023-09-27T23:13:48.838657+0000","last_fullsized":"2023-09-27T23:13:48.838657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:36.782810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810125+0000","last_change":"2023-09-27T23:13:44.795768+0000","last_active":"2023-09-27T23:13:48.810125+0000","last_peered":"2023-09-27T23:13:48.810125+0000","last_clean":"2023-09-27T23:13:48.810125+0000","last_became_active":"2023-09-27T23:13:44.795376+0000","last_became_peered":"2023-09-27T23:13:44.795376+0000","last_unstale":"2023-09-27T23:13:48.810125+0000","last_undegraded":"2023-09-27T23:13:48.810125+0000","last_fullsized":"2023-09-27T23:13:48.810125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:02.136037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803338+0000","last_change":"2023-09-27T23:13:44.415778+0000","last_active":"2023-09-27T23:13:48.803338+0000","last_peered":"2023-09-27T23:13:48.803338+0000","last_clean":"2023-09-27T23:13:48.803338+0000","last_became_active":"2023-09-27T23:13:44.415490+0000","last_became_peered":"2023-09-27T23:13:44.415490+0000","last_unstale":"2023-09-27T23:13:48.803338+0000","last_undegraded":"2023-09-27T23:13:48.803338+0000","last_fullsized":"2023-09-27T23:13:48.803338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:03.148603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.828626+0000","last_change":"2023-09-27T23:13:48.823729+0000","last_active":"2023-09-27T23:13:48.828626+0000","last_peered":"2023-09-27T23:13:48.828626+0000","last_clean":"2023-09-27T23:13:48.828626+0000","last_became_active":"2023-09-27T23:13:48.823284+0000","last_became_peered":"2023-09-27T23:13:48.823284+0000","last_unstale":"2023-09-27T23:13:48.828626+0000","last_undegraded":"2023-09-27T23:13:48.828626+0000","last_fullsized":"2023-09-27T23:13:48.828626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:19.656238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820833+0000","last_change":"2023-09-27T23:13:44.794072+0000","last_active":"2023-09-27T23:13:48.820833+0000","last_peered":"2023-09-27T23:13:48.820833+0000","last_clean":"2023-09-27T23:13:48.820833+0000","last_became_active":"2023-09-27T23:13:44.793773+0000","last_became_peered":"2023-09-27T23:13:44.793773+0000","last_unstale":"2023-09-27T23:13:48.820833+0000","last_undegraded":"2023-09-27T23:13:48.820833+0000","last_fullsized":"2023-09-27T23:13:48.820833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804693+0000","last_change":"2023-09-27T23:13:44.426779+0000","last_active":"2023-09-27T23:13:48.804693+0000","last_peered":"2023-09-27T23:13:48.804693+0000","last_clean":"2023-09-27T23:13:48.804693+0000","last_became_active":"2023-09-27T23:13:44.426515+0000","last_became_peered":"2023-09-27T23:13:44.426515+0000","last_unstale":"2023-09-27T23:13:48.804693+0000","last_undegraded":"2023-09-27T23:13:48.804693+0000","last_fullsized":"2023-09-27T23:13:48.804693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:40.093740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842388+0000","last_change":"2023-09-27T23:13:48.839843+0000","last_active":"2023-09-27T23:13:48.842388+0000","last_peered":"2023-09-27T23:13:48.842388+0000","last_clean":"2023-09-27T23:13:48.842388+0000","last_became_active":"2023-09-27T23:13:48.839383+0000","last_became_peered":"2023-09-27T23:13:48.839383+0000","last_unstale":"2023-09-27T23:13:48.842388+0000","last_undegraded":"2023-09-27T23:13:48.842388+0000","last_fullsized":"2023-09-27T23:13:48.842388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:44:03.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810673+0000","last_change":"2023-09-27T23:13:44.802558+0000","last_active":"2023-09-27T23:13:48.810673+0000","last_peered":"2023-09-27T23:13:48.810673+0000","last_clean":"2023-09-27T23:13:48.810673+0000","last_became_active":"2023-09-27T23:13:44.800406+0000","last_became_peered":"2023-09-27T23:13:44.800406+0000","last_unstale":"2023-09-27T23:13:48.810673+0000","last_undegraded":"2023-09-27T23:13:48.810673+0000","last_fullsized":"2023-09-27T23:13:48.810673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804716+0000","last_change":"2023-09-27T23:13:44.426684+0000","last_active":"2023-09-27T23:13:48.804716+0000","last_peered":"2023-09-27T23:13:48.804716+0000","last_clean":"2023-09-27T23:13:48.804716+0000","last_became_active":"2023-09-27T23:13:44.426422+0000","last_became_peered":"2023-09-27T23:13:44.426422+0000","last_unstale":"2023-09-27T23:13:48.804716+0000","last_undegraded":"2023-09-27T23:13:48.804716+0000","last_fullsized":"2023-09-27T23:13:48.804716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:12.799421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.829098+0000","last_change":"2023-09-27T23:13:48.827612+0000","last_active":"2023-09-27T23:13:48.829098+0000","last_peered":"2023-09-27T23:13:48.829098+0000","last_clean":"2023-09-27T23:13:48.829098+0000","last_became_active":"2023-09-27T23:13:48.827019+0000","last_became_peered":"2023-09-27T23:13:48.827019+0000","last_unstale":"2023-09-27T23:13:48.829098+0000","last_undegraded":"2023-09-27T23:13:48.829098+0000","last_fullsized":"2023-09-27T23:13:48.829098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:40:33.151042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.835789+0000","last_change":"2023-09-27T23:13:48.830820+0000","last_active":"2023-09-27T23:13:48.835789+0000","last_peered":"2023-09-27T23:13:48.835789+0000","last_clean":"2023-09-27T23:13:48.835789+0000","last_became_active":"2023-09-27T23:13:48.829846+0000","last_became_peered":"2023-09-27T23:13:48.829846+0000","last_unstale":"2023-09-27T23:13:48.835789+0000","last_undegraded":"2023-09-27T23:13:48.835789+0000","last_fullsized":"2023-09-27T23:13:48.835789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:40:15.056097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809441+0000","last_change":"2023-09-27T23:13:44.413137+0000","last_active":"2023-09-27T23:13:48.809441+0000","last_peered":"2023-09-27T23:13:48.809441+0000","last_clean":"2023-09-27T23:13:48.809441+0000","last_became_active":"2023-09-27T23:13:44.412799+0000","last_became_peered":"2023-09-27T23:13:44.412799+0000","last_unstale":"2023-09-27T23:13:48.809441+0000","last_undegraded":"2023-09-27T23:13:48.809441+0000","last_fullsized":"2023-09-27T23:13:48.809441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825273+0000","last_change":"2023-09-27T23:13:44.793322+0000","last_active":"2023-09-27T23:13:48.825273+0000","last_peered":"2023-09-27T23:13:48.825273+0000","last_clean":"2023-09-27T23:13:48.825273+0000","last_became_active":"2023-09-27T23:13:44.793057+0000","last_became_peered":"2023-09-27T23:13:44.793057+0000","last_unstale":"2023-09-27T23:13:48.825273+0000","last_undegraded":"2023-09-27T23:13:48.825273+0000","last_fullsized":"2023-09-27T23:13:48.825273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842877+0000","last_change":"2023-09-27T23:13:48.839976+0000","last_active":"2023-09-27T23:13:48.842877+0000","last_peered":"2023-09-27T23:13:48.842877+0000","last_clean":"2023-09-27T23:13:48.842877+0000","last_became_active":"2023-09-27T23:13:48.839568+0000","last_became_peered":"2023-09-27T23:13:48.839568+0000","last_unstale":"2023-09-27T23:13:48.842877+0000","last_undegraded":"2023-09-27T23:13:48.842877+0000","last_fullsized":"2023-09-27T23:13:48.842877+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:01.437279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804236+0000","last_change":"2023-09-27T23:13:44.427379+0000","last_active":"2023-09-27T23:13:48.804236+0000","last_peered":"2023-09-27T23:13:48.804236+0000","last_clean":"2023-09-27T23:13:48.804236+0000","last_became_active":"2023-09-27T23:13:44.427094+0000","last_became_peered":"2023-09-27T23:13:44.427094+0000","last_unstale":"2023-09-27T23:13:48.804236+0000","last_undegraded":"2023-09-27T23:13:48.804236+0000","last_fullsized":"2023-09-27T23:13:48.804236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:33.544941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803930+0000","last_change":"2023-09-27T23:13:44.793100+0000","last_active":"2023-09-27T23:13:48.803930+0000","last_peered":"2023-09-27T23:13:48.803930+0000","last_clean":"2023-09-27T23:13:48.803930+0000","last_became_active":"2023-09-27T23:13:44.792493+0000","last_became_peered":"2023-09-27T23:13:44.792493+0000","last_unstale":"2023-09-27T23:13:48.803930+0000","last_undegraded":"2023-09-27T23:13:48.803930+0000","last_fullsized":"2023-09-27T23:13:48.803930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804152+0000","last_change":"2023-09-27T23:13:44.420128+0000","last_active":"2023-09-27T23:13:48.804152+0000","last_peered":"2023-09-27T23:13:48.804152+0000","last_clean":"2023-09-27T23:13:48.804152+0000","last_became_active":"2023-09-27T23:13:44.419240+0000","last_became_peered":"2023-09-27T23:13:44.419240+0000","last_unstale":"2023-09-27T23:13:48.804152+0000","last_undegraded":"2023-09-27T23:13:48.804152+0000","last_fullsized":"2023-09-27T23:13:48.804152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:34.691147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814644+0000","last_change":"2023-09-27T23:13:44.802448+0000","last_active":"2023-09-27T23:13:48.814644+0000","last_peered":"2023-09-27T23:13:48.814644+0000","last_clean":"2023-09-27T23:13:48.814644+0000","last_became_active":"2023-09-27T23:13:44.801316+0000","last_became_peered":"2023-09-27T23:13:44.801316+0000","last_unstale":"2023-09-27T23:13:48.814644+0000","last_undegraded":"2023-09-27T23:13:48.814644+0000","last_fullsized":"2023-09-27T23:13:48.814644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:02.029091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.842951+0000","last_change":"2023-09-27T23:13:48.841624+0000","last_active":"2023-09-27T23:13:48.842951+0000","last_peered":"2023-09-27T23:13:48.842951+0000","last_clean":"2023-09-27T23:13:48.842951+0000","last_became_active":"2023-09-27T23:13:48.841104+0000","last_became_peered":"2023-09-27T23:13:48.841104+0000","last_unstale":"2023-09-27T23:13:48.842951+0000","last_undegraded":"2023-09-27T23:13:48.842951+0000","last_fullsized":"2023-09-27T23:13:48.842951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:15.763356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804318+0000","last_change":"2023-09-27T23:13:44.426217+0000","last_active":"2023-09-27T23:13:48.804318+0000","last_peered":"2023-09-27T23:13:48.804318+0000","last_clean":"2023-09-27T23:13:48.804318+0000","last_became_active":"2023-09-27T23:13:44.425907+0000","last_became_peered":"2023-09-27T23:13:44.425907+0000","last_unstale":"2023-09-27T23:13:48.804318+0000","last_undegraded":"2023-09-27T23:13:48.804318+0000","last_fullsized":"2023-09-27T23:13:48.804318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:43.378286+0000","last_clean_scrub_stamp":"2023-09-27T23:13:43.378286+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:01.126893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825997+0000","last_change":"2023-09-27T23:13:44.789234+0000","last_active":"2023-09-27T23:13:48.825997+0000","last_peered":"2023-09-27T23:13:48.825997+0000","last_clean":"2023-09-27T23:13:48.825997+0000","last_became_active":"2023-09-27T23:13:44.788948+0000","last_became_peered":"2023-09-27T23:13:44.788948+0000","last_unstale":"2023-09-27T23:13:48.825997+0000","last_undegraded":"2023-09-27T23:13:48.825997+0000","last_fullsized":"2023-09-27T23:13:48.825997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:42:41.803210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.834520+0000","last_change":"2023-09-27T23:13:48.830283+0000","last_active":"2023-09-27T23:13:48.834520+0000","last_peered":"2023-09-27T23:13:48.834520+0000","last_clean":"2023-09-27T23:13:48.834520+0000","last_became_active":"2023-09-27T23:13:48.829788+0000","last_became_peered":"2023-09-27T23:13:48.829788+0000","last_unstale":"2023-09-27T23:13:48.834520+0000","last_undegraded":"2023-09-27T23:13:48.834520+0000","last_fullsized":"2023-09-27T23:13:48.834520+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:47.781322+0000","last_clean_scrub_stamp":"2023-09-27T23:13:47.781322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:12.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.809476+0000","last_change":"2023-09-27T23:13:44.794796+0000","last_active":"2023-09-27T23:13:48.809476+0000","last_peered":"2023-09-27T23:13:48.809476+0000","last_clean":"2023-09-27T23:13:48.809476+0000","last_became_active":"2023-09-27T23:13:44.793105+0000","last_became_peered":"2023-09-27T23:13:44.793105+0000","last_unstale":"2023-09-27T23:13:48.809476+0000","last_undegraded":"2023-09-27T23:13:48.809476+0000","last_fullsized":"2023-09-27T23:13:48.809476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:13.808565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822377+0000","last_change":"2023-09-27T23:13:44.789337+0000","last_active":"2023-09-27T23:13:48.822377+0000","last_peered":"2023-09-27T23:13:48.822377+0000","last_clean":"2023-09-27T23:13:48.822377+0000","last_became_active":"2023-09-27T23:13:44.789055+0000","last_became_peered":"2023-09-27T23:13:44.789055+0000","last_unstale":"2023-09-27T23:13:48.822377+0000","last_undegraded":"2023-09-27T23:13:48.822377+0000","last_fullsized":"2023-09-27T23:13:48.822377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821093+0000","last_change":"2023-09-27T23:13:44.792891+0000","last_active":"2023-09-27T23:13:48.821093+0000","last_peered":"2023-09-27T23:13:48.821093+0000","last_clean":"2023-09-27T23:13:48.821093+0000","last_became_active":"2023-09-27T23:13:44.792439+0000","last_became_peered":"2023-09-27T23:13:44.792439+0000","last_unstale":"2023-09-27T23:13:48.821093+0000","last_undegraded":"2023-09-27T23:13:48.821093+0000","last_fullsized":"2023-09-27T23:13:48.821093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:33.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.825381+0000","last_change":"2023-09-27T23:13:44.787514+0000","last_active":"2023-09-27T23:13:48.825381+0000","last_peered":"2023-09-27T23:13:48.825381+0000","last_clean":"2023-09-27T23:13:48.825381+0000","last_became_active":"2023-09-27T23:13:44.787130+0000","last_became_peered":"2023-09-27T23:13:44.787130+0000","last_unstale":"2023-09-27T23:13:48.825381+0000","last_undegraded":"2023-09-27T23:13:48.825381+0000","last_fullsized":"2023-09-27T23:13:48.825381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.806701+0000","last_change":"2023-09-27T23:13:44.786768+0000","last_active":"2023-09-27T23:13:48.806701+0000","last_peered":"2023-09-27T23:13:48.806701+0000","last_clean":"2023-09-27T23:13:48.806701+0000","last_became_active":"2023-09-27T23:13:44.785642+0000","last_became_peered":"2023-09-27T23:13:44.785642+0000","last_unstale":"2023-09-27T23:13:48.806701+0000","last_undegraded":"2023-09-27T23:13:48.806701+0000","last_fullsized":"2023-09-27T23:13:48.806701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:11:40.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.814552+0000","last_change":"2023-09-27T23:13:44.802393+0000","last_active":"2023-09-27T23:13:48.814552+0000","last_peered":"2023-09-27T23:13:48.814552+0000","last_clean":"2023-09-27T23:13:48.814552+0000","last_became_active":"2023-09-27T23:13:44.801038+0000","last_became_peered":"2023-09-27T23:13:44.801038+0000","last_unstale":"2023-09-27T23:13:48.814552+0000","last_undegraded":"2023-09-27T23:13:48.814552+0000","last_fullsized":"2023-09-27T23:13:48.814552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:43.071719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.807068+0000","last_change":"2023-09-27T23:13:44.778165+0000","last_active":"2023-09-27T23:13:48.807068+0000","last_peered":"2023-09-27T23:13:48.807068+0000","last_clean":"2023-09-27T23:13:48.807068+0000","last_became_active":"2023-09-27T23:13:44.777922+0000","last_became_peered":"2023-09-27T23:13:44.777922+0000","last_unstale":"2023-09-27T23:13:48.807068+0000","last_undegraded":"2023-09-27T23:13:48.807068+0000","last_fullsized":"2023-09-27T23:13:48.807068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:31.322306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.801910+0000","last_change":"2023-09-27T23:13:44.792009+0000","last_active":"2023-09-27T23:13:48.801910+0000","last_peered":"2023-09-27T23:13:48.801910+0000","last_clean":"2023-09-27T23:13:48.801910+0000","last_became_active":"2023-09-27T23:13:44.791714+0000","last_became_peered":"2023-09-27T23:13:44.791714+0000","last_unstale":"2023-09-27T23:13:48.801910+0000","last_undegraded":"2023-09-27T23:13:48.801910+0000","last_fullsized":"2023-09-27T23:13:48.801910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:10.801409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.802783+0000","last_change":"2023-09-27T23:13:44.800431+0000","last_active":"2023-09-27T23:13:48.802783+0000","last_peered":"2023-09-27T23:13:48.802783+0000","last_clean":"2023-09-27T23:13:48.802783+0000","last_became_active":"2023-09-27T23:13:44.800056+0000","last_became_peered":"2023-09-27T23:13:44.800056+0000","last_unstale":"2023-09-27T23:13:48.802783+0000","last_undegraded":"2023-09-27T23:13:48.802783+0000","last_fullsized":"2023-09-27T23:13:48.802783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.822290+0000","last_change":"2023-09-27T23:13:44.788799+0000","last_active":"2023-09-27T23:13:48.822290+0000","last_peered":"2023-09-27T23:13:48.822290+0000","last_clean":"2023-09-27T23:13:48.822290+0000","last_became_active":"2023-09-27T23:13:44.788526+0000","last_became_peered":"2023-09-27T23:13:44.788526+0000","last_unstale":"2023-09-27T23:13:48.822290+0000","last_undegraded":"2023-09-27T23:13:48.822290+0000","last_fullsized":"2023-09-27T23:13:48.822290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803904+0000","last_change":"2023-09-27T23:13:44.795488+0000","last_active":"2023-09-27T23:13:48.803904+0000","last_peered":"2023-09-27T23:13:48.803904+0000","last_clean":"2023-09-27T23:13:48.803904+0000","last_became_active":"2023-09-27T23:13:44.794082+0000","last_became_peered":"2023-09-27T23:13:44.794082+0000","last_unstale":"2023-09-27T23:13:48.803904+0000","last_undegraded":"2023-09-27T23:13:48.803904+0000","last_fullsized":"2023-09-27T23:13:48.803904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:20.319376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803019+0000","last_change":"2023-09-27T23:13:44.802208+0000","last_active":"2023-09-27T23:13:48.803019+0000","last_peered":"2023-09-27T23:13:48.803019+0000","last_clean":"2023-09-27T23:13:48.803019+0000","last_became_active":"2023-09-27T23:13:44.801858+0000","last_became_peered":"2023-09-27T23:13:44.801858+0000","last_unstale":"2023-09-27T23:13:48.803019+0000","last_undegraded":"2023-09-27T23:13:48.803019+0000","last_fullsized":"2023-09-27T23:13:48.803019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821304+0000","last_change":"2023-09-27T23:13:44.795884+0000","last_active":"2023-09-27T23:13:48.821304+0000","last_peered":"2023-09-27T23:13:48.821304+0000","last_clean":"2023-09-27T23:13:48.821304+0000","last_became_active":"2023-09-27T23:13:44.795490+0000","last_became_peered":"2023-09-27T23:13:44.795490+0000","last_unstale":"2023-09-27T23:13:48.821304+0000","last_undegraded":"2023-09-27T23:13:48.821304+0000","last_fullsized":"2023-09-27T23:13:48.821304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:53:40.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810126+0000","last_change":"2023-09-27T23:13:44.791556+0000","last_active":"2023-09-27T23:13:48.810126+0000","last_peered":"2023-09-27T23:13:48.810126+0000","last_clean":"2023-09-27T23:13:48.810126+0000","last_became_active":"2023-09-27T23:13:44.789598+0000","last_became_peered":"2023-09-27T23:13:44.789598+0000","last_unstale":"2023-09-27T23:13:48.810126+0000","last_undegraded":"2023-09-27T23:13:48.810126+0000","last_fullsized":"2023-09-27T23:13:48.810126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:03.782211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821279+0000","last_change":"2023-09-27T23:13:44.794170+0000","last_active":"2023-09-27T23:13:48.821279+0000","last_peered":"2023-09-27T23:13:48.821279+0000","last_clean":"2023-09-27T23:13:48.821279+0000","last_became_active":"2023-09-27T23:13:44.793843+0000","last_became_peered":"2023-09-27T23:13:44.793843+0000","last_unstale":"2023-09-27T23:13:48.821279+0000","last_undegraded":"2023-09-27T23:13:48.821279+0000","last_fullsized":"2023-09-27T23:13:48.821279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:59.878365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804536+0000","last_change":"2023-09-27T23:13:44.795499+0000","last_active":"2023-09-27T23:13:48.804536+0000","last_peered":"2023-09-27T23:13:48.804536+0000","last_clean":"2023-09-27T23:13:48.804536+0000","last_became_active":"2023-09-27T23:13:44.794256+0000","last_became_peered":"2023-09-27T23:13:44.794256+0000","last_unstale":"2023-09-27T23:13:48.804536+0000","last_undegraded":"2023-09-27T23:13:48.804536+0000","last_fullsized":"2023-09-27T23:13:48.804536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821399+0000","last_change":"2023-09-27T23:13:44.797131+0000","last_active":"2023-09-27T23:13:48.821399+0000","last_peered":"2023-09-27T23:13:48.821399+0000","last_clean":"2023-09-27T23:13:48.821399+0000","last_became_active":"2023-09-27T23:13:44.796814+0000","last_became_peered":"2023-09-27T23:13:44.796814+0000","last_unstale":"2023-09-27T23:13:48.821399+0000","last_undegraded":"2023-09-27T23:13:48.821399+0000","last_fullsized":"2023-09-27T23:13:48.821399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:06.550809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804126+0000","last_change":"2023-09-27T23:13:44.794205+0000","last_active":"2023-09-27T23:13:48.804126+0000","last_peered":"2023-09-27T23:13:48.804126+0000","last_clean":"2023-09-27T23:13:48.804126+0000","last_became_active":"2023-09-27T23:13:44.793765+0000","last_became_peered":"2023-09-27T23:13:44.793765+0000","last_unstale":"2023-09-27T23:13:48.804126+0000","last_undegraded":"2023-09-27T23:13:48.804126+0000","last_fullsized":"2023-09-27T23:13:48.804126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:29:00.547223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.813812+0000","last_change":"2023-09-27T23:13:44.777827+0000","last_active":"2023-09-27T23:13:48.813812+0000","last_peered":"2023-09-27T23:13:48.813812+0000","last_clean":"2023-09-27T23:13:48.813812+0000","last_became_active":"2023-09-27T23:13:44.777461+0000","last_became_peered":"2023-09-27T23:13:44.777461+0000","last_unstale":"2023-09-27T23:13:48.813812+0000","last_undegraded":"2023-09-27T23:13:48.813812+0000","last_fullsized":"2023-09-27T23:13:48.813812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:09:34.554085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810644+0000","last_change":"2023-09-27T23:13:44.787157+0000","last_active":"2023-09-27T23:13:48.810644+0000","last_peered":"2023-09-27T23:13:48.810644+0000","last_clean":"2023-09-27T23:13:48.810644+0000","last_became_active":"2023-09-27T23:13:44.786486+0000","last_became_peered":"2023-09-27T23:13:44.786486+0000","last_unstale":"2023-09-27T23:13:48.810644+0000","last_undegraded":"2023-09-27T23:13:48.810644+0000","last_fullsized":"2023-09-27T23:13:48.810644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:46:01.890618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810866+0000","last_change":"2023-09-27T23:13:44.796954+0000","last_active":"2023-09-27T23:13:48.810866+0000","last_peered":"2023-09-27T23:13:48.810866+0000","last_clean":"2023-09-27T23:13:48.810866+0000","last_became_active":"2023-09-27T23:13:44.796634+0000","last_became_peered":"2023-09-27T23:13:44.796634+0000","last_unstale":"2023-09-27T23:13:48.810866+0000","last_undegraded":"2023-09-27T23:13:48.810866+0000","last_fullsized":"2023-09-27T23:13:48.810866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:40:04.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821286+0000","last_change":"2023-09-27T23:13:44.797615+0000","last_active":"2023-09-27T23:13:48.821286+0000","last_peered":"2023-09-27T23:13:48.821286+0000","last_clean":"2023-09-27T23:13:48.821286+0000","last_became_active":"2023-09-27T23:13:44.797325+0000","last_became_peered":"2023-09-27T23:13:44.797325+0000","last_unstale":"2023-09-27T23:13:48.821286+0000","last_undegraded":"2023-09-27T23:13:48.821286+0000","last_fullsized":"2023-09-27T23:13:48.821286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.819151+0000","last_change":"2023-09-27T23:13:44.801326+0000","last_active":"2023-09-27T23:13:48.819151+0000","last_peered":"2023-09-27T23:13:48.819151+0000","last_clean":"2023-09-27T23:13:48.819151+0000","last_became_active":"2023-09-27T23:13:44.800936+0000","last_became_peered":"2023-09-27T23:13:44.800936+0000","last_unstale":"2023-09-27T23:13:48.819151+0000","last_undegraded":"2023-09-27T23:13:48.819151+0000","last_fullsized":"2023-09-27T23:13:48.819151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:39.974025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.803144+0000","last_change":"2023-09-27T23:13:44.800702+0000","last_active":"2023-09-27T23:13:48.803144+0000","last_peered":"2023-09-27T23:13:48.803144+0000","last_clean":"2023-09-27T23:13:48.803144+0000","last_became_active":"2023-09-27T23:13:44.800422+0000","last_became_peered":"2023-09-27T23:13:44.800422+0000","last_unstale":"2023-09-27T23:13:48.803144+0000","last_undegraded":"2023-09-27T23:13:48.803144+0000","last_fullsized":"2023-09-27T23:13:48.803144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810589+0000","last_change":"2023-09-27T23:13:44.802316+0000","last_active":"2023-09-27T23:13:48.810589+0000","last_peered":"2023-09-27T23:13:48.810589+0000","last_clean":"2023-09-27T23:13:48.810589+0000","last_became_active":"2023-09-27T23:13:44.800990+0000","last_became_peered":"2023-09-27T23:13:44.800990+0000","last_unstale":"2023-09-27T23:13:48.810589+0000","last_undegraded":"2023-09-27T23:13:48.810589+0000","last_fullsized":"2023-09-27T23:13:48.810589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:20.007629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804015+0000","last_change":"2023-09-27T23:13:44.794033+0000","last_active":"2023-09-27T23:13:48.804015+0000","last_peered":"2023-09-27T23:13:48.804015+0000","last_clean":"2023-09-27T23:13:48.804015+0000","last_became_active":"2023-09-27T23:13:44.793696+0000","last_became_peered":"2023-09-27T23:13:44.793696+0000","last_unstale":"2023-09-27T23:13:48.804015+0000","last_undegraded":"2023-09-27T23:13:48.804015+0000","last_fullsized":"2023-09-27T23:13:48.804015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.820743+0000","last_change":"2023-09-27T23:13:44.795881+0000","last_active":"2023-09-27T23:13:48.820743+0000","last_peered":"2023-09-27T23:13:48.820743+0000","last_clean":"2023-09-27T23:13:48.820743+0000","last_became_active":"2023-09-27T23:13:44.795554+0000","last_became_peered":"2023-09-27T23:13:44.795554+0000","last_unstale":"2023-09-27T23:13:48.820743+0000","last_undegraded":"2023-09-27T23:13:48.820743+0000","last_fullsized":"2023-09-27T23:13:48.820743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:48:16.262346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.804794+0000","last_change":"2023-09-27T23:13:44.801496+0000","last_active":"2023-09-27T23:13:48.804794+0000","last_peered":"2023-09-27T23:13:48.804794+0000","last_clean":"2023-09-27T23:13:48.804794+0000","last_became_active":"2023-09-27T23:13:44.801264+0000","last_became_peered":"2023-09-27T23:13:44.801264+0000","last_unstale":"2023-09-27T23:13:48.804794+0000","last_undegraded":"2023-09-27T23:13:48.804794+0000","last_fullsized":"2023-09-27T23:13:48.804794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:00.818507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.812717+0000","last_change":"2023-09-27T23:13:44.790757+0000","last_active":"2023-09-27T23:13:48.812717+0000","last_peered":"2023-09-27T23:13:48.812717+0000","last_clean":"2023-09-27T23:13:48.812717+0000","last_became_active":"2023-09-27T23:13:44.790388+0000","last_became_peered":"2023-09-27T23:13:44.790388+0000","last_unstale":"2023-09-27T23:13:48.812717+0000","last_undegraded":"2023-09-27T23:13:48.812717+0000","last_fullsized":"2023-09-27T23:13:48.812717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.821180+0000","last_change":"2023-09-27T23:13:44.799201+0000","last_active":"2023-09-27T23:13:48.821180+0000","last_peered":"2023-09-27T23:13:48.821180+0000","last_clean":"2023-09-27T23:13:48.821180+0000","last_became_active":"2023-09-27T23:13:44.798712+0000","last_became_peered":"2023-09-27T23:13:44.798712+0000","last_unstale":"2023-09-27T23:13:48.821180+0000","last_undegraded":"2023-09-27T23:13:48.821180+0000","last_fullsized":"2023-09-27T23:13:48.821180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:53:58.895871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.815177+0000","last_change":"2023-09-27T23:13:44.780452+0000","last_active":"2023-09-27T23:13:48.815177+0000","last_peered":"2023-09-27T23:13:48.815177+0000","last_clean":"2023-09-27T23:13:48.815177+0000","last_became_active":"2023-09-27T23:13:44.780111+0000","last_became_peered":"2023-09-27T23:13:44.780111+0000","last_unstale":"2023-09-27T23:13:48.815177+0000","last_undegraded":"2023-09-27T23:13:48.815177+0000","last_fullsized":"2023-09-27T23:13:48.815177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:46.470679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-09-27T23:13:48.810755+0000","last_change":"2023-09-27T23:13:44.794473+0000","last_active":"2023-09-27T23:13:48.810755+0000","last_peered":"2023-09-27T23:13:48.810755+0000","last_clean":"2023-09-27T23:13:48.810755+0000","last_became_active":"2023-09-27T23:13:44.794187+0000","last_became_peered":"2023-09-27T23:13:44.794187+0000","last_unstale":"2023-09-27T23:13:48.810755+0000","last_undegraded":"2023-09-27T23:13:48.810755+0000","last_fullsized":"2023-09-27T23:13:48.810755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:44.387430+0000","last_clean_scrub_stamp":"2023-09-27T23:13:44.387430+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:32.805277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9300,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362540,"statfs":{"total":96636764160,"available":96627240960,"internally_reserved":0,"allocated":929792,"data_stored":719189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]}]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8188,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363652,"statfs":{"total":96636764160,"available":96628379648,"internally_reserved":0,"allocated":315392,"data_stored":125515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8264,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363576,"statfs":{"total":96636764160,"available":96628301824,"internally_reserved":0,"allocated":327680,"data_stored":128661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":124981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8116,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363724,"statfs":{"total":96636764160,"available":96628453376,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]}]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":905216,"data_stored":715813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:03.700 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-27T23:14:03.700 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-27T23:14:03.700 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-27T23:14:03.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-27T23:14:03.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398e282700 1 Processor -- start 2023-09-27T23:14:03.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398e282700 1 -- start start 2023-09-27T23:14:03.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398e282700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806a580 0x7f398806a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398e282700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398806c7c0 con 0x7f398806d210 2023-09-27T23:14:03.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398e282700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398806cd80 con 0x7f39880b1410 2023-09-27T23:14:03.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398e282700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398806af10 con 0x7f398806a580 2023-09-27T23:14:03.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.798+0000 7f398da81700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f398806d210 0x7f398806d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34174/0 (socket says 172.21.15.64:34174) 2023-09-27T23:14:03.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398da81700 1 -- 172.21.15.64:0/2103693535 learned_addr learned my addr 172.21.15.64:0/2103693535 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398ca7f700 1 --2- 172.21.15.64:0/2103693535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806a580 0x7f398806a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112672340 0 0) 0x7f398806c7c0 con 0x7f398806d210 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3974003570 con 0x7f398806d210 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185669507 0 0) 0x7f398806cd80 con 0x7f39880b1410 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f398806c7c0 con 0x7f39880b1410 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398ca7f700 1 -- 172.21.15.64:0/2103693535 >> v1:172.21.15.123:6789/0 conn(0x7f39880b1410 legacy=0x7f39880ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398ca7f700 1 -- 172.21.15.64:0/2103693535 >> v1:172.21.15.64:6789/0 conn(0x7f398806d210 legacy=0x7f398806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398ca7f700 1 -- 172.21.15.64:0/2103693535 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398806b8f0 con 0x7f398806a580 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398ca7f700 1 --2- 172.21.15.64:0/2103693535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806a580 0x7f398806a9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3978009910 tx=0x7f39780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d99f511971ea35 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978011630 con 0x7f398806a580 2023-09-27T23:14:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3978011790 con 0x7f398806a580 2023-09-27T23:14:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f397ffff700 1 -- 172.21.15.64:0/2103693535 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978011b10 con 0x7f398806a580 2023-09-27T23:14:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 -- 172.21.15.64:0/2103693535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806a580 msgr2=0x7f398806a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 --2- 172.21.15.64:0/2103693535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806a580 0x7f398806a9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3978009910 tx=0x7f39780091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 -- 172.21.15.64:0/2103693535 shutdown_connections 2023-09-27T23:14:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 --2- 172.21.15.64:0/2103693535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806a580 0x7f398806a9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 -- 172.21.15.64:0/2103693535 >> 172.21.15.64:0/2103693535 conn(0x7f3988067bb0 msgr2=0x7f39880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 -- 172.21.15.64:0/2103693535 shutdown_connections 2023-09-27T23:14:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.802+0000 7f398e282700 1 -- 172.21.15.64:0/2103693535 wait complete. 2023-09-27T23:14:03.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.806+0000 7f398e282700 1 Processor -- start 2023-09-27T23:14:03.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.806+0000 7f398e282700 1 -- start start 2023-09-27T23:14:03.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.806+0000 7f398e282700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 0x7f39880c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.806+0000 7f398e282700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806d210 0x7f398814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.806+0000 7f398e282700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39880b1410 0x7f398806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398e282700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880c55e0 con 0x7f398806a580 2023-09-27T23:14:03.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398e282700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880c58c0 con 0x7f39880b1410 2023-09-27T23:14:03.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398e282700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39880c5ba0 con 0x7f398806d210 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 0x7f39880c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 0x7f39880c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44824/0 (socket says 172.21.15.64:44824) 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 -- 172.21.15.64:0/2247271722 learned_addr learned my addr 172.21.15.64:0/2247271722 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398da81700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39880b1410 0x7f398806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398ca7f700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806d210 0x7f398814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 -- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806d210 msgr2=0x7f398814a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806d210 0x7f398814a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 -- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39880b1410 msgr2=0x7f398806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39880b1410 0x7f398806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 -- 172.21.15.64:0/2247271722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978009420 con 0x7f398806a580 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398d280700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 0x7f39880c4c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f398400c590 tx=0x7f3984007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984010510 con 0x7f398806a580 2023-09-27T23:14:03.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984010670 con 0x7f398806a580 2023-09-27T23:14:03.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39880c5e80 con 0x7f398806a580 2023-09-27T23:14:03.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984010a30 con 0x7f398806a580 2023-09-27T23:14:03.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39880c51b0 con 0x7f398806a580 2023-09-27T23:14:03.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.810+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398804be40 con 0x7f398806a580 2023-09-27T23:14:03.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.814+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f3984025430 con 0x7f398806a580 2023-09-27T23:14:03.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.814+0000 7f397d7fa700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f396c060950 0x7f396c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.814+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f398407d850 con 0x7f398806a580 2023-09-27T23:14:03.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.814+0000 7f398ca7f700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f396c060950 0x7f396c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:03.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.814+0000 7f398ca7f700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f396c060950 0x7f396c062e00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3978009860 tx=0x7f3978017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:03.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:03.822+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f398404d860 con 0x7f398806a580 2023-09-27T23:14:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3988069eb0 con 0x7f398806a580 2023-09-27T23:14:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f397d7fa700 1 -- 172.21.15.64:0/2247271722 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3984050e80 con 0x7f398806a580 2023-09-27T23:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-27T23:14:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f396c060950 msgr2=0x7f396c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f396c060950 0x7f396c062e00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3978009860 tx=0x7f3978017000 comp rx=0 tx=0).stop 2023-09-27T23:14:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 msgr2=0x7f39880c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 0x7f39880c4c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f398400c590 tx=0x7f3984007dc0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 shutdown_connections 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f396c060950 0x7f396c062e00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39880b1410 0x7f398806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398806d210 0x7f398814a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 --2- 172.21.15.64:0/2247271722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f398806a580 0x7f39880c4c70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 >> 172.21.15.64:0/2247271722 conn(0x7f3988067bb0 msgr2=0x7f39881565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 shutdown_connections 2023-09-27T23:14:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.014+0000 7f398e282700 1 -- 172.21.15.64:0/2247271722 wait complete. 2023-09-27T23:14:04.031 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-27T23:14:04.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:04.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 Processor -- start 2023-09-27T23:14:04.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 -- start start 2023-09-27T23:14:04.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70980cad10 0x7f70980cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806b750 0x7f709806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f709806df20 0x7f709815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709806b2c0 con 0x7f709806df20 2023-09-27T23:14:04.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709806a6b0 con 0x7f70980cad10 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7097fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709806a990 con 0x7f709806b750 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f709806df20 0x7f709815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f709806df20 0x7f709815bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44834/0 (socket says 172.21.15.64:44834) 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 -- 172.21.15.64:0/2831532528 learned_addr learned my addr 172.21.15.64:0/2831532528 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7096ffd700 1 --2- 172.21.15.64:0/2831532528 >> v2:172.21.15.123:3300/0 conn(0x7f70980cad10 0x7f70980cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70967fc700 1 --2- 172.21.15.64:0/2831532528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806b750 0x7f709806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 -- 172.21.15.64:0/2831532528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806b750 msgr2=0x7f709806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 --2- 172.21.15.64:0/2831532528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806b750 0x7f709806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 -- 172.21.15.64:0/2831532528 >> v2:172.21.15.123:3300/0 conn(0x7f70980cad10 msgr2=0x7f70980cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 --2- 172.21.15.64:0/2831532528 >> v2:172.21.15.123:3300/0 conn(0x7f70980cad10 0x7f70980cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 -- 172.21.15.64:0/2831532528 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709806ac70 con 0x7f709806df20 2023-09-27T23:14:04.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f70977fe700 1 --2- 172.21.15.64:0/2831532528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806df20 0x7f709815bd00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7098069f90 tx=0x7f7088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d254610ffa12a6dc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7095ffb700 1 -- 172.21.15.64:0/2831532528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088009e70 con 0x7f709806df20 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7095ffb700 1 -- 172.21.15.64:0/2831532528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708800b860 con 0x7f709806df20 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.130+0000 7f7095ffb700 1 -- 172.21.15.64:0/2831532528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800bc00 con 0x7f709806df20 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 -- 172.21.15.64:0/2831532528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806df20 msgr2=0x7f709815bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 --2- 172.21.15.64:0/2831532528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806df20 0x7f709815bd00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7098069f90 tx=0x7f7088009250 comp rx=0 tx=0).stop 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 -- 172.21.15.64:0/2831532528 shutdown_connections 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 --2- 172.21.15.64:0/2831532528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806df20 0x7f709815bd00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 --2- 172.21.15.64:0/2831532528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806b750 0x7f709806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 --2- 172.21.15.64:0/2831532528 >> v2:172.21.15.123:3300/0 conn(0x7f70980cad10 0x7f70980cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 -- 172.21.15.64:0/2831532528 >> 172.21.15.64:0/2831532528 conn(0x7f7098067bb0 msgr2=0x7f70980cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 -- 172.21.15.64:0/2831532528 shutdown_connections 2023-09-27T23:14:04.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 -- 172.21.15.64:0/2831532528 wait complete. 2023-09-27T23:14:04.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 Processor -- start 2023-09-27T23:14:04.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.134+0000 7f7097fff700 1 -- start start 2023-09-27T23:14:04.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 0x7f70980b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806df20 0x7f709814e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70980cad10 0x7f709814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709814ed20 con 0x7f709806b750 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709815f210 con 0x7f70980cad10 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709815e600 con 0x7f709806df20 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 0x7f70980b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 0x7f70980b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44838/0 (socket says 172.21.15.64:44838) 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 -- 172.21.15.64:0/1746974515 learned_addr learned my addr 172.21.15.64:0/1746974515 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f70967fc700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806df20 0x7f709814e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f70977fe700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70980cad10 0x7f709814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 -- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806df20 msgr2=0x7f709814e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806df20 0x7f709814e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 -- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70980cad10 msgr2=0x7f709814e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70980cad10 0x7f709814e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 -- 172.21.15.64:0/1746974515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880094f0 con 0x7f709806b750 2023-09-27T23:14:04.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7096ffd700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 0x7f70980b3730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7090009e50 tx=0x7f709000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000cb30 con 0x7f709806b750 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709000f040 con 0x7f709806b750 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090014480 con 0x7f709806b750 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709815e8e0 con 0x7f709806b750 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709815eea0 con 0x7f709806b750 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.138+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709804be40 con 0x7f709806b750 2023-09-27T23:14:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.142+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f70900147f0 con 0x7f709806b750 2023-09-27T23:14:04.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.142+0000 7f707effd700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7084060950 0x7f7084062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.142+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f709007dac0 con 0x7f709806b750 2023-09-27T23:14:04.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.142+0000 7f70967fc700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7084060950 0x7f7084062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.142+0000 7f70967fc700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7084060950 0x7f7084062e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70980b5cf0 tx=0x7f708c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.150+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f709004da50 con 0x7f709806b750 2023-09-27T23:14:04.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.306+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f709806af50 con 0x7f709806b750 2023-09-27T23:14:04.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f707effd700 1 -- 172.21.15.64:0/1746974515 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f709007de60 con 0x7f709806b750 2023-09-27T23:14:04.313 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:04.313 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7084060950 msgr2=0x7f7084062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7084060950 0x7f7084062e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70980b5cf0 tx=0x7f708c008040 comp rx=0 tx=0).stop 2023-09-27T23:14:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 msgr2=0x7f70980b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 0x7f70980b3730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7090009e50 tx=0x7f709000b040 comp rx=0 tx=0).stop 2023-09-27T23:14:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 shutdown_connections 2023-09-27T23:14:04.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7084060950 0x7f7084062e00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70980cad10 0x7f709814e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f709806df20 0x7f709814e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 --2- 172.21.15.64:0/1746974515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f709806b750 0x7f70980b3730 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 >> 172.21.15.64:0/1746974515 conn(0x7f7098067bb0 msgr2=0x7f70981579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 shutdown_connections 2023-09-27T23:14:04.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.310+0000 7f7097fff700 1 -- 172.21.15.64:0/1746974515 wait complete. 2023-09-27T23:14:04.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:04.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:04.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 Processor -- start 2023-09-27T23:14:04.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 -- start start 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee2c0adab0 0x7fee2c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c15ae30 con 0x7fee2c154780 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0abf70 con 0x7fee2c0adab0 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.434+0000 7fee324ce700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c154b50 con 0x7fee2c06baa0 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee314cc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fee2c154780 0x7fee2c157b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34178/0 (socket says 172.21.15.64:34178) 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee314cc700 1 -- 172.21.15.64:0/3456921432 learned_addr learned my addr 172.21.15.64:0/3456921432 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee30ccb700 1 --2- 172.21.15.64:0/3456921432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee31ccd700 1 --2- 172.21.15.64:0/3456921432 >> v2:172.21.15.123:3300/0 conn(0x7fee2c0adab0 0x7fee2c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee2bfff700 1 -- 172.21.15.64:0/3456921432 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654540635 0 0) 0x7fee2c15ae30 con 0x7fee2c154780 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee2bfff700 1 -- 172.21.15.64:0/3456921432 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee14003570 con 0x7fee2c154780 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee30ccb700 1 -- 172.21.15.64:0/3456921432 >> v2:172.21.15.123:3300/0 conn(0x7fee2c0adab0 msgr2=0x7fee2c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee30ccb700 1 --2- 172.21.15.64:0/3456921432 >> v2:172.21.15.123:3300/0 conn(0x7fee2c0adab0 0x7fee2c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee2bfff700 1 -- 172.21.15.64:0/3456921432 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 128780011 0 0) 0x7fee14003570 con 0x7fee2c154780 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee30ccb700 1 -- 172.21.15.64:0/3456921432 >> v1:172.21.15.64:6789/0 conn(0x7fee2c154780 legacy=0x7fee2c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee30ccb700 1 -- 172.21.15.64:0/3456921432 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c0ac210 con 0x7fee2c06baa0 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee30ccb700 1 --2- 172.21.15.64:0/3456921432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c15b2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee20009e60 tx=0x7fee20009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1501bb34f54c9385 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee2bfff700 1 -- 172.21.15.64:0/3456921432 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000e070 con 0x7fee2c06baa0 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee2bfff700 1 -- 172.21.15.64:0/3456921432 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2000b7a0 con 0x7fee2c06baa0 2023-09-27T23:14:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee2bfff700 1 -- 172.21.15.64:0/3456921432 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c001470 con 0x7fee2c06baa0 2023-09-27T23:14:04.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 -- 172.21.15.64:0/3456921432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 msgr2=0x7fee2c15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 --2- 172.21.15.64:0/3456921432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c15b2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee20009e60 tx=0x7fee20009780 comp rx=0 tx=0).stop 2023-09-27T23:14:04.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 -- 172.21.15.64:0/3456921432 shutdown_connections 2023-09-27T23:14:04.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 --2- 172.21.15.64:0/3456921432 >> v2:172.21.15.123:3300/0 conn(0x7fee2c0adab0 0x7fee2c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 --2- 172.21.15.64:0/3456921432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c15b2c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 -- 172.21.15.64:0/3456921432 >> 172.21.15.64:0/3456921432 conn(0x7fee2c067b50 msgr2=0x7fee2c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 -- 172.21.15.64:0/3456921432 shutdown_connections 2023-09-27T23:14:04.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.438+0000 7fee324ce700 1 -- 172.21.15.64:0/3456921432 wait complete. 2023-09-27T23:14:04.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 Processor -- start 2023-09-27T23:14:04.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 -- start start 2023-09-27T23:14:04.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 0x7fee2c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee2c154780 0x7fee2c146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c15b8a0 con 0x7fee2c0adab0 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c15be60 con 0x7fee2c154780 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c159c20 con 0x7fee2c06baa0 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 0x7fee2c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 0x7fee2c1591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44840/0 (socket says 172.21.15.64:44840) 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 -- 172.21.15.64:0/1777490327 learned_addr learned my addr 172.21.15.64:0/1777490327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee314cc700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee31ccd700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee2c154780 0x7fee2c146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 -- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 msgr2=0x7fee2c145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 -- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee2c154780 msgr2=0x7fee2c146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee2c154780 0x7fee2c146630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 -- 172.21.15.64:0/1777490327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009a20 con 0x7fee2c0adab0 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee31ccd700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee2c154780 0x7fee2c146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee30ccb700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 0x7fee2c1591d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fee2c06bff0 tx=0x7fee20018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20012ba0 con 0x7fee2c0adab0 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20012d00 con 0x7fee2c0adab0 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c159f00 con 0x7fee2c0adab0 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2001d670 con 0x7fee2c0adab0 2023-09-27T23:14:04.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.442+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c1458a0 con 0x7fee2c0adab0 2023-09-27T23:14:04.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.446+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c04be40 con 0x7fee2c0adab0 2023-09-27T23:14:04.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.446+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fee2001da20 con 0x7fee2c0adab0 2023-09-27T23:14:04.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.446+0000 7fee297fa700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fee18060980 0x7fee18062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.446+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fee2007f170 con 0x7fee2c0adab0 2023-09-27T23:14:04.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.446+0000 7fee314cc700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fee18060980 0x7fee18062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.446+0000 7fee314cc700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fee18060980 0x7fee18062e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee1c004e60 tx=0x7fee1c001a10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.454+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fee2004f110 con 0x7fee2c0adab0 2023-09-27T23:14:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.610+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee2c0c5d40 con 0x7fee2c0adab0 2023-09-27T23:14:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.610+0000 7fee297fa700 1 -- 172.21.15.64:0/1777490327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee20052730 con 0x7fee2c0adab0 2023-09-27T23:14:04.616 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fee18060980 msgr2=0x7fee18062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fee18060980 0x7fee18062e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee1c004e60 tx=0x7fee1c001a10 comp rx=0 tx=0).stop 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 msgr2=0x7fee2c1591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 0x7fee2c1591d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fee2c06bff0 tx=0x7fee20018000 comp rx=0 tx=0).stop 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 shutdown_connections 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fee18060980 0x7fee18062e30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee2c154780 0x7fee2c146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c0adab0 0x7fee2c1591d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 --2- 172.21.15.64:0/1777490327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee2c06baa0 0x7fee2c145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 >> 172.21.15.64:0/1777490327 conn(0x7fee2c067b50 msgr2=0x7fee2c1482d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 shutdown_connections 2023-09-27T23:14:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.614+0000 7fee324ce700 1 -- 172.21.15.64:0/1777490327 wait complete. 2023-09-27T23:14:04.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:04.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 Processor -- start 2023-09-27T23:14:04.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 -- start start 2023-09-27T23:14:04.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c20155520 0x7f5c200c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c2006b530 0x7f5c200ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200ad020 con 0x7f5c200ae6c0 2023-09-27T23:14:04.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200c81e0 con 0x7f5c20155520 2023-09-27T23:14:04.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c264a0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200aca10 con 0x7f5c2006b530 2023-09-27T23:14:04.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c2549e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c20155520 0x7f5c200c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c24c9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5c200ae6c0 0x7f5c2006ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44962/0 (socket says 172.21.15.64:44962) 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c24c9d700 1 -- 172.21.15.64:0/3144938694 learned_addr learned my addr 172.21.15.64:0/3144938694 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c25c9f700 1 --2- 172.21.15.64:0/3144938694 >> v2:172.21.15.123:3300/0 conn(0x7f5c2006b530 0x7f5c200ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c13fff700 1 -- 172.21.15.64:0/3144938694 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738311926 0 0) 0x7f5c200ad020 con 0x7f5c200ae6c0 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c13fff700 1 -- 172.21.15.64:0/3144938694 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c003570 con 0x7f5c200ae6c0 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c2549e700 1 -- 172.21.15.64:0/3144938694 >> v2:172.21.15.123:3300/0 conn(0x7f5c2006b530 msgr2=0x7f5c200ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c2549e700 1 --2- 172.21.15.64:0/3144938694 >> v2:172.21.15.123:3300/0 conn(0x7f5c2006b530 0x7f5c200ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c2549e700 1 -- 172.21.15.64:0/3144938694 >> v1:172.21.15.64:6789/0 conn(0x7f5c200ae6c0 legacy=0x7f5c2006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c2549e700 1 -- 172.21.15.64:0/3144938694 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200c7c20 con 0x7f5c20155520 2023-09-27T23:14:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c25c9f700 1 --2- 172.21.15.64:0/3144938694 >> v2:172.21.15.123:3300/0 conn(0x7f5c2006b530 0x7f5c200ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.742+0000 7f5c2549e700 1 --2- 172.21.15.64:0/3144938694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c7130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c200c6830 tx=0x7f5c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d73d05b47196cd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c13fff700 1 -- 172.21.15.64:0/3144938694 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00b690 con 0x7f5c20155520 2023-09-27T23:14:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c13fff700 1 -- 172.21.15.64:0/3144938694 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c00b7f0 con 0x7f5c20155520 2023-09-27T23:14:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c13fff700 1 -- 172.21.15.64:0/3144938694 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00bb90 con 0x7f5c20155520 2023-09-27T23:14:04.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 -- 172.21.15.64:0/3144938694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 msgr2=0x7f5c200c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 --2- 172.21.15.64:0/3144938694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c7130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c200c6830 tx=0x7f5c1c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 -- 172.21.15.64:0/3144938694 shutdown_connections 2023-09-27T23:14:04.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 --2- 172.21.15.64:0/3144938694 >> v2:172.21.15.123:3300/0 conn(0x7f5c2006b530 0x7f5c200ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 --2- 172.21.15.64:0/3144938694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c7130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 -- 172.21.15.64:0/3144938694 >> 172.21.15.64:0/3144938694 conn(0x7f5c200c8cd0 msgr2=0x7f5c200c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 -- 172.21.15.64:0/3144938694 shutdown_connections 2023-09-27T23:14:04.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 -- 172.21.15.64:0/3144938694 wait complete. 2023-09-27T23:14:04.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 Processor -- start 2023-09-27T23:14:04.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.746+0000 7f5c264a0700 1 -- start start 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c2006b530 0x7f5c20154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c200ae6c0 0x7f5c200c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2014dc90 con 0x7f5c20155520 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2014d680 con 0x7f5c200ae6c0 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c20154cf0 con 0x7f5c2006b530 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36498/0 (socket says 172.21.15.64:36498) 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 -- 172.21.15.64:0/158509767 learned_addr learned my addr 172.21.15.64:0/158509767 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c2549e700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c2006b530 0x7f5c20154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c24c9d700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c200ae6c0 0x7f5c200c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 -- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c2006b530 msgr2=0x7f5c20154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c2006b530 0x7f5c20154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 -- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c200ae6c0 msgr2=0x7f5c200c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c200ae6c0 0x7f5c200c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 -- 172.21.15.64:0/158509767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c009420 con 0x7f5c20155520 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c2549e700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c2006b530 0x7f5c20154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c25c9f700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c5a60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800d0d0 tx=0x7f5c180099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800dc10 con 0x7f5c20155520 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1800f040 con 0x7f5c20155520 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c20154fd0 con 0x7f5c20155520 2023-09-27T23:14:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18015440 con 0x7f5c20155520 2023-09-27T23:14:04.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.750+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c20151de0 con 0x7f5c20155520 2023-09-27T23:14:04.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.754+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2014de00 con 0x7f5c20155520 2023-09-27T23:14:04.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.754+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f5c1801e3b0 con 0x7f5c20155520 2023-09-27T23:14:04.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.754+0000 7f5c117fa700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5c04060950 0x7f5c04062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:04.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.754+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5c1807d640 con 0x7f5c20155520 2023-09-27T23:14:04.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.754+0000 7f5c2549e700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5c04060950 0x7f5c04062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:04.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.754+0000 7f5c2549e700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5c04060950 0x7f5c04062e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0091e0 tx=0x7f5c1c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:04.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.762+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5c1804d5d0 con 0x7f5c20155520 2023-09-27T23:14:04.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.922+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c2004be40 con 0x7f5c20155520 2023-09-27T23:14:04.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.922+0000 7f5c117fa700 1 -- 172.21.15.64:0/158509767 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c18050bf0 con 0x7f5c20155520 2023-09-27T23:14:04.929 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5c04060950 msgr2=0x7f5c04062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5c04060950 0x7f5c04062e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0091e0 tx=0x7f5c1c017000 comp rx=0 tx=0).stop 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 msgr2=0x7f5c200c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c5a60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800d0d0 tx=0x7f5c180099f0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 shutdown_connections 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5c04060950 0x7f5c04062e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20155520 0x7f5c200c5a60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c200ae6c0 0x7f5c200c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 --2- 172.21.15.64:0/158509767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c2006b530 0x7f5c20154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 >> 172.21.15.64:0/158509767 conn(0x7f5c200c8cd0 msgr2=0x7f5c200685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:04.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.926+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 shutdown_connections 2023-09-27T23:14:04.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:04.930+0000 7f5c264a0700 1 -- 172.21.15.64:0/158509767 wait complete. 2023-09-27T23:14:04.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e25502700 1 Processor -- start 2023-09-27T23:14:05.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e25502700 1 -- start start 2023-09-27T23:14:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e25502700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e25502700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e20068750 con 0x7f3e2015bd60 2023-09-27T23:14:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e25502700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e20068a30 con 0x7f3e200b1410 2023-09-27T23:14:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e25502700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200ad470 con 0x7f3e200ad020 2023-09-27T23:14:05.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e24d01700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3e2015bd60 0x7f3e2015c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44978/0 (socket says 172.21.15.64:44978) 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e24d01700 1 -- 172.21.15.64:0/3262095245 learned_addr learned my addr 172.21.15.64:0/3262095245 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1f7fe700 1 --2- 172.21.15.64:0/3262095245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790333168 0 0) 0x7f3e20068750 con 0x7f3e2015bd60 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e08003570 con 0x7f3e2015bd60 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458887993 0 0) 0x7f3e20068a30 con 0x7f3e200b1410 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e20068750 con 0x7f3e200b1410 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/3262095245 >> v1:172.21.15.123:6789/0 conn(0x7f3e200b1410 legacy=0x7f3e200b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/3262095245 >> v1:172.21.15.64:6789/0 conn(0x7f3e2015bd60 legacy=0x7f3e2015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.058+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/3262095245 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20068d10 con 0x7f3e200ad020 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e1f7fe700 1 --2- 172.21.15.64:0/3262095245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200b17e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009910 tx=0x7f3e140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b772f7d2dc0f655b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14011630 con 0x7f3e200ad020 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14011790 con 0x7f3e200ad020 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e1effd700 1 -- 172.21.15.64:0/3262095245 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14011b30 con 0x7f3e200ad020 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 -- 172.21.15.64:0/3262095245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 msgr2=0x7f3e200b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 --2- 172.21.15.64:0/3262095245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200b17e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009910 tx=0x7f3e140091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 -- 172.21.15.64:0/3262095245 shutdown_connections 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 --2- 172.21.15.64:0/3262095245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200b17e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 -- 172.21.15.64:0/3262095245 >> 172.21.15.64:0/3262095245 conn(0x7f3e200cdc20 msgr2=0x7f3e200ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 -- 172.21.15.64:0/3262095245 shutdown_connections 2023-09-27T23:14:05.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.062+0000 7f3e25502700 1 -- 172.21.15.64:0/3262095245 wait complete. 2023-09-27T23:14:05.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 Processor -- start 2023-09-27T23:14:05.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 -- start start 2023-09-27T23:14:05.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 0x7f3e200ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e2015bd60 0x7f3e200cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200cc180 con 0x7f3e200b1410 2023-09-27T23:14:05.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c77a0 con 0x7f3e2015bd60 2023-09-27T23:14:05.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2016a130 con 0x7f3e200ad020 2023-09-27T23:14:05.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 0x7f3e200ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 0x7f3e200ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36504/0 (socket says 172.21.15.64:36504) 2023-09-27T23:14:05.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/253188780 learned_addr learned my addr 172.21.15.64:0/253188780 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:05.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1ffff700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e24d01700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e2015bd60 0x7f3e200cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 msgr2=0x7f3e200cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e2015bd60 msgr2=0x7f3e200cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e2015bd60 0x7f3e200cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 -- 172.21.15.64:0/253188780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14009420 con 0x7f3e200b1410 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1ffff700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e1f7fe700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 0x7f3e200ccbd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009f00 tx=0x7f3e14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1401f070 con 0x7f3e200b1410 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14002f70 con 0x7f3e200b1410 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e200c6e70 con 0x7f3e200b1410 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14006af0 con 0x7f3e200b1410 2023-09-27T23:14:05.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.066+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e200c7430 con 0x7f3e200b1410 2023-09-27T23:14:05.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.070+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f3e14023070 con 0x7f3e200b1410 2023-09-27T23:14:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.070+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2006ddb0 con 0x7f3e200b1410 2023-09-27T23:14:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.070+0000 7f3e07fff700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3e0c060950 0x7f3e0c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.070+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3e1407e340 con 0x7f3e200b1410 2023-09-27T23:14:05.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.074+0000 7f3e1ffff700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3e0c060950 0x7f3e0c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.074+0000 7f3e1ffff700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3e0c060950 0x7f3e0c062e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3e10007fd0 tx=0x7f3e10007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.078+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3e2006ddb0 con 0x7f3e200b1410 2023-09-27T23:14:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e2006ddb0 con 0x7f3e200b1410 2023-09-27T23:14:05.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e07fff700 1 -- 172.21.15.64:0/253188780 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e2006ddb0 con 0x7f3e200b1410 2023-09-27T23:14:05.246 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3e0c060950 msgr2=0x7f3e0c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3e0c060950 0x7f3e0c062e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3e10007fd0 tx=0x7f3e10007d50 comp rx=0 tx=0).stop 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 msgr2=0x7f3e200ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 0x7f3e200ccbd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009f00 tx=0x7f3e14017000 comp rx=0 tx=0).stop 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 shutdown_connections 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3e0c060950 0x7f3e0c062e00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e2015bd60 0x7f3e200cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e200b1410 0x7f3e200ccbd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 --2- 172.21.15.64:0/253188780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e200ad020 0x7f3e200cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 >> 172.21.15.64:0/253188780 conn(0x7f3e200cdc20 msgr2=0x7f3e2006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 shutdown_connections 2023-09-27T23:14:05.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.242+0000 7f3e25502700 1 -- 172.21.15.64:0/253188780 wait complete. 2023-09-27T23:14:05.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.366+0000 7f3596bc5700 1 Processor -- start 2023-09-27T23:14:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.366+0000 7f3596bc5700 1 -- start start 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.366+0000 7f3596bc5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35900ce2d0 0x7f35900ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.366+0000 7f3596bc5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900caa10 0x7f359015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35900cc040 0x7f3590158900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359015d960 con 0x7f35900ce2d0 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35900cbbb0 con 0x7f35900cc040 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359006dc90 con 0x7f35900caa10 2023-09-27T23:14:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35900ce2d0 0x7f35900ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35900ce2d0 0x7f35900ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36510/0 (socket says 172.21.15.64:36510) 2023-09-27T23:14:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 -- 172.21.15.64:0/2806843956 learned_addr learned my addr 172.21.15.64:0/2806843956 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f35953c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900caa10 0x7f359015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f35963c4700 1 --2- 172.21.15.64:0/2806843956 >> v2:172.21.15.123:3300/0 conn(0x7f35900cc040 0x7f3590158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 -- 172.21.15.64:0/2806843956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900caa10 msgr2=0x7f359015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 --2- 172.21.15.64:0/2806843956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900caa10 0x7f359015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 -- 172.21.15.64:0/2806843956 >> v2:172.21.15.123:3300/0 conn(0x7f35900cc040 msgr2=0x7f3590158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 --2- 172.21.15.64:0/2806843956 >> v2:172.21.15.123:3300/0 conn(0x7f35900cc040 0x7f3590158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 -- 172.21.15.64:0/2806843956 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900cafa0 con 0x7f35900ce2d0 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f35963c4700 1 --2- 172.21.15.64:0/2806843956 >> v2:172.21.15.123:3300/0 conn(0x7f35900cc040 0x7f3590158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3595bc3700 1 --2- 172.21.15.64:0/2806843956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900ce2d0 0x7f35900ce6a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f35900b2ab0 tx=0x7f3580009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434935b7e9a7b928 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3594bc1700 1 -- 172.21.15.64:0/2806843956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000b640 con 0x7f35900ce2d0 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3594bc1700 1 -- 172.21.15.64:0/2806843956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000b7a0 con 0x7f35900ce2d0 2023-09-27T23:14:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3594bc1700 1 -- 172.21.15.64:0/2806843956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000bb60 con 0x7f35900ce2d0 2023-09-27T23:14:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- 172.21.15.64:0/2806843956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900ce2d0 msgr2=0x7f35900ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 --2- 172.21.15.64:0/2806843956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900ce2d0 0x7f35900ce6a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f35900b2ab0 tx=0x7f3580009250 comp rx=0 tx=0).stop 2023-09-27T23:14:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- 172.21.15.64:0/2806843956 shutdown_connections 2023-09-27T23:14:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 --2- 172.21.15.64:0/2806843956 >> v2:172.21.15.123:3300/0 conn(0x7f35900cc040 0x7f3590158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 --2- 172.21.15.64:0/2806843956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900caa10 0x7f359015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 --2- 172.21.15.64:0/2806843956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900ce2d0 0x7f35900ce6a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- 172.21.15.64:0/2806843956 >> 172.21.15.64:0/2806843956 conn(0x7f3590067bb0 msgr2=0x7f35900cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- 172.21.15.64:0/2806843956 shutdown_connections 2023-09-27T23:14:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.370+0000 7f3596bc5700 1 -- 172.21.15.64:0/2806843956 wait complete. 2023-09-27T23:14:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 Processor -- start 2023-09-27T23:14:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 -- start start 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35900caa10 0x7f3590147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 0x7f35900b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900ce2d0 0x7f35900c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35900b46d0 con 0x7f35900cc040 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35900b4110 con 0x7f35900caa10 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35900b43f0 con 0x7f35900ce2d0 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 0x7f35900b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 0x7f35900b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36514/0 (socket says 172.21.15.64:36514) 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 -- 172.21.15.64:0/886702718 learned_addr learned my addr 172.21.15.64:0/886702718 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3595bc3700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35900caa10 0x7f3590147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35963c4700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900ce2d0 0x7f35900c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 -- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900ce2d0 msgr2=0x7f35900c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900ce2d0 0x7f35900c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 -- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35900caa10 msgr2=0x7f3590147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35900caa10 0x7f3590147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 -- 172.21.15.64:0/886702718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588009a20 con 0x7f35900cc040 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35963c4700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900ce2d0 0x7f35900c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3595bc3700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35900caa10 0x7f3590147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f35953c2700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 0x7f35900b36b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f358c00ec80 tx=0x7f358c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00c9d0 con 0x7f35900cc040 2023-09-27T23:14:05.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00cb30 con 0x7f35900cc040 2023-09-27T23:14:05.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35800094f0 con 0x7f35900cc040 2023-09-27T23:14:05.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c013470 con 0x7f35900cc040 2023-09-27T23:14:05.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.374+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35900c7310 con 0x7f35900cc040 2023-09-27T23:14:05.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.378+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35900c6e10 con 0x7f35900cc040 2023-09-27T23:14:05.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.378+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f358c013820 con 0x7f35900cc040 2023-09-27T23:14:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.378+0000 7f3585ffb700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3574060950 0x7f3574062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.378+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f358c07d520 con 0x7f35900cc040 2023-09-27T23:14:05.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.378+0000 7f3595bc3700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3574060950 0x7f3574062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.382+0000 7f3595bc3700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3574060950 0x7f3574062e00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35900b2ab0 tx=0x7f3580017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.386+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f35900c6e10 con 0x7f35900cc040 2023-09-27T23:14:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.542+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35900c6e10 con 0x7f35900cc040 2023-09-27T23:14:05.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.542+0000 7f3585ffb700 1 -- 172.21.15.64:0/886702718 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35900c6e10 con 0x7f35900cc040 2023-09-27T23:14:05.548 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3574060950 msgr2=0x7f3574062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3574060950 0x7f3574062e00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35900b2ab0 tx=0x7f3580017000 comp rx=0 tx=0).stop 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 msgr2=0x7f35900b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 0x7f35900b36b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f358c00ec80 tx=0x7f358c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 shutdown_connections 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3574060950 0x7f3574062e00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35900ce2d0 0x7f35900c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35900cc040 0x7f35900b36b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 --2- 172.21.15.64:0/886702718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35900caa10 0x7f3590147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 >> 172.21.15.64:0/886702718 conn(0x7f3590067bb0 msgr2=0x7f3590155600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 shutdown_connections 2023-09-27T23:14:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.546+0000 7f3596bc5700 1 -- 172.21.15.64:0/886702718 wait complete. 2023-09-27T23:14:05.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.670+0000 7effa17c4700 1 Processor -- start 2023-09-27T23:14:05.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.670+0000 7effa17c4700 1 -- start start 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff9c06aeb0 0x7eff9c159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff9c1595a0 0x7eff9c1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c06c850 con 0x7eff9c158c90 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c0cb650 con 0x7eff9c1595a0 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c06b470 con 0x7eff9c06aeb0 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff9c1595a0 0x7eff9c1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7eff9c158c90 0x7eff9c0caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44982/0 (socket says 172.21.15.64:44982) 2023-09-27T23:14:05.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff9c1595a0 0x7eff9c1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36522/0 (socket says 172.21.15.64:36522) 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9bfff700 1 -- 172.21.15.64:0/1307339529 learned_addr learned my addr 172.21.15.64:0/1307339529 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9b7fe700 1 --2- 172.21.15.64:0/1307339529 >> v2:172.21.15.123:3300/0 conn(0x7eff9c06aeb0 0x7eff9c159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 -- 172.21.15.64:0/1307339529 >> v2:172.21.15.123:3300/0 conn(0x7eff9c06aeb0 msgr2=0x7eff9c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 --2- 172.21.15.64:0/1307339529 >> v2:172.21.15.123:3300/0 conn(0x7eff9c06aeb0 0x7eff9c159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 -- 172.21.15.64:0/1307339529 >> v1:172.21.15.64:6789/0 conn(0x7eff9c158c90 legacy=0x7eff9c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 -- 172.21.15.64:0/1307339529 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c06b750 con 0x7eff9c1595a0 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9affd700 1 -- 172.21.15.64:0/1307339529 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295009402 0 0) 0x7eff9c06c850 con 0x7eff9c158c90 2023-09-27T23:14:05.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa0fc3700 1 --2- 172.21.15.64:0/1307339529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c1595a0 0x7eff9c1599c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff9c068000 tx=0x7eff94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b85985551932db1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9affd700 1 -- 172.21.15.64:0/1307339529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400e070 con 0x7eff9c1595a0 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9affd700 1 -- 172.21.15.64:0/1307339529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9400bb60 con 0x7eff9c1595a0 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7eff9affd700 1 -- 172.21.15.64:0/1307339529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94012430 con 0x7eff9c1595a0 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- 172.21.15.64:0/1307339529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c1595a0 msgr2=0x7eff9c1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 --2- 172.21.15.64:0/1307339529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c1595a0 0x7eff9c1599c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff9c068000 tx=0x7eff94009780 comp rx=0 tx=0).stop 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- 172.21.15.64:0/1307339529 shutdown_connections 2023-09-27T23:14:05.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 --2- 172.21.15.64:0/1307339529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c1595a0 0x7eff9c1599c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 --2- 172.21.15.64:0/1307339529 >> v2:172.21.15.123:3300/0 conn(0x7eff9c06aeb0 0x7eff9c159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- 172.21.15.64:0/1307339529 >> 172.21.15.64:0/1307339529 conn(0x7eff9c067bb0 msgr2=0x7eff9c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- 172.21.15.64:0/1307339529 shutdown_connections 2023-09-27T23:14:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.674+0000 7effa17c4700 1 -- 172.21.15.64:0/1307339529 wait complete. 2023-09-27T23:14:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 Processor -- start 2023-09-27T23:14:05.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 -- start start 2023-09-27T23:14:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff9c06aeb0 0x7eff9c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 0x7eff9c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff9c1595a0 0x7eff9c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c155c00 con 0x7eff9c158c90 2023-09-27T23:14:05.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0cdb30 con 0x7eff9c1595a0 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c155360 con 0x7eff9c06aeb0 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 0x7eff9c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 0x7eff9c0b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36524/0 (socket says 172.21.15.64:36524) 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 -- 172.21.15.64:0/1062394641 learned_addr learned my addr 172.21.15.64:0/1062394641 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9bfff700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff9c06aeb0 0x7eff9c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa0fc3700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff9c1595a0 0x7eff9c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 -- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff9c06aeb0 msgr2=0x7eff9c0b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff9c06aeb0 0x7eff9c0b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 -- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff9c1595a0 msgr2=0x7eff9c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff9c1595a0 0x7eff9c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 -- 172.21.15.64:0/1062394641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009a20 con 0x7eff9c158c90 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa0fc3700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff9c1595a0 0x7eff9c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff9b7fe700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 0x7eff9c0b32a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff8400ec80 tx=0x7eff840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400c9d0 con 0x7eff9c158c90 2023-09-27T23:14:05.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8400cb50 con 0x7eff9c158c90 2023-09-27T23:14:05.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c155640 con 0x7eff9c158c90 2023-09-27T23:14:05.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84013470 con 0x7eff9c158c90 2023-09-27T23:14:05.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.678+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c168cc0 con 0x7eff9c158c90 2023-09-27T23:14:05.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.682+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c0cd8e0 con 0x7eff9c158c90 2023-09-27T23:14:05.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.682+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7eff84013860 con 0x7eff9c158c90 2023-09-27T23:14:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.682+0000 7eff93fff700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff7c060950 0x7eff7c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.682+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7eff8407d630 con 0x7eff9c158c90 2023-09-27T23:14:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.682+0000 7eff9bfff700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff7c060950 0x7eff7c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.682+0000 7eff9bfff700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff7c060950 0x7eff7c062e00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0c9b20 tx=0x7eff8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.690+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff8404d520 con 0x7eff9c158c90 2023-09-27T23:14:05.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.854+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7eff9c04be40 con 0x7eff9c158c90 2023-09-27T23:14:05.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.854+0000 7eff93fff700 1 -- 172.21.15.64:0/1062394641 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7eff84050b40 con 0x7eff9c158c90 2023-09-27T23:14:05.859 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:05.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff7c060950 msgr2=0x7eff7c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff7c060950 0x7eff7c062e00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0c9b20 tx=0x7eff8c009040 comp rx=0 tx=0).stop 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 msgr2=0x7eff9c0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 0x7eff9c0b32a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff8400ec80 tx=0x7eff840091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 shutdown_connections 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff7c060950 0x7eff7c062e00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff9c1595a0 0x7eff9c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff9c158c90 0x7eff9c0b32a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 --2- 172.21.15.64:0/1062394641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff9c06aeb0 0x7eff9c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 >> 172.21.15.64:0/1062394641 conn(0x7eff9c067bb0 msgr2=0x7eff9c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 shutdown_connections 2023-09-27T23:14:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.858+0000 7effa17c4700 1 -- 172.21.15.64:0/1062394641 wait complete. 2023-09-27T23:14:05.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:05.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 Processor -- start 2023-09-27T23:14:05.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 -- start start 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ac0067c90 0x7f3ac0068060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac014fba0 0x7f3ac014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac00c6630 con 0x7f3ac00c5900 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00c6440 con 0x7f3ac0067c90 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac7682700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00c6910 con 0x7f3ac014fba0 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac4c1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac014fba0 0x7f3ac014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac5c1f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3ac00c5900 0x7f3ac00c5d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44996/0 (socket says 172.21.15.64:44996) 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac5c1f700 1 -- 172.21.15.64:0/540756253 learned_addr learned my addr 172.21.15.64:0/540756253 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:05.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac541e700 1 --2- 172.21.15.64:0/540756253 >> v2:172.21.15.123:3300/0 conn(0x7f3ac0067c90 0x7f3ac0068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac4c1d700 1 -- 172.21.15.64:0/540756253 >> v2:172.21.15.123:3300/0 conn(0x7f3ac0067c90 msgr2=0x7f3ac0068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac4c1d700 1 --2- 172.21.15.64:0/540756253 >> v2:172.21.15.123:3300/0 conn(0x7f3ac0067c90 0x7f3ac0068060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac4c1d700 1 -- 172.21.15.64:0/540756253 >> v1:172.21.15.64:6789/0 conn(0x7f3ac00c5900 legacy=0x7f3ac00c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ab3fff700 1 -- 172.21.15.64:0/540756253 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131629185 0 0) 0x7f3ac00c6630 con 0x7f3ac00c5900 2023-09-27T23:14:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac4c1d700 1 -- 172.21.15.64:0/540756253 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00c6bf0 con 0x7f3ac014fba0 2023-09-27T23:14:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ac4c1d700 1 --2- 172.21.15.64:0/540756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac014fba0 0x7f3ac014ffd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009e60 tx=0x7f3ab8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc532496fbc5ad9b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ab3fff700 1 -- 172.21.15.64:0/540756253 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab800e070 con 0x7f3ac014fba0 2023-09-27T23:14:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ab3fff700 1 -- 172.21.15.64:0/540756253 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab800bb60 con 0x7f3ac014fba0 2023-09-27T23:14:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.974+0000 7f3ab3fff700 1 -- 172.21.15.64:0/540756253 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8012430 con 0x7f3ac014fba0 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 -- 172.21.15.64:0/540756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac014fba0 msgr2=0x7f3ac014ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 --2- 172.21.15.64:0/540756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac014fba0 0x7f3ac014ffd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009e60 tx=0x7f3ab8009780 comp rx=0 tx=0).stop 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 -- 172.21.15.64:0/540756253 shutdown_connections 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 --2- 172.21.15.64:0/540756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac014fba0 0x7f3ac014ffd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 --2- 172.21.15.64:0/540756253 >> v2:172.21.15.123:3300/0 conn(0x7f3ac0067c90 0x7f3ac0068060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 -- 172.21.15.64:0/540756253 >> 172.21.15.64:0/540756253 conn(0x7f3ac00cd6d0 msgr2=0x7f3ac006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 -- 172.21.15.64:0/540756253 shutdown_connections 2023-09-27T23:14:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 -- 172.21.15.64:0/540756253 wait complete. 2023-09-27T23:14:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 Processor -- start 2023-09-27T23:14:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.978+0000 7f3ac7682700 1 -- start start 2023-09-27T23:14:05.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac0067c90 0x7f3ac01510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac00c5900 0x7f3ac0152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 0x7f3ac00c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00c86e0 con 0x7f3ac014fba0 2023-09-27T23:14:05.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00b4490 con 0x7f3ac00c5900 2023-09-27T23:14:05.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00b4d30 con 0x7f3ac0067c90 2023-09-27T23:14:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 0x7f3ac00c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 0x7f3ac00c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36540/0 (socket says 172.21.15.64:36540) 2023-09-27T23:14:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 -- 172.21.15.64:0/2145373312 learned_addr learned my addr 172.21.15.64:0/2145373312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac541e700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac0067c90 0x7f3ac01510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac4c1d700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac00c5900 0x7f3ac0152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 -- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac0067c90 msgr2=0x7f3ac01510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac0067c90 0x7f3ac01510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 -- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac00c5900 msgr2=0x7f3ac0152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac00c5900 0x7f3ac0152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 -- 172.21.15.64:0/2145373312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8009a20 con 0x7f3ac014fba0 2023-09-27T23:14:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac541e700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac0067c90 0x7f3ac01510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac5c1f700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 0x7f3ac00c9ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400c7e0 tx=0x7f3ab4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4005b30 con 0x7f3ac014fba0 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab4005c90 con 0x7f3ac014fba0 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac016a1b0 con 0x7f3ac014fba0 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4007320 con 0x7f3ac014fba0 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.982+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac00b49c0 con 0x7f3ac014fba0 2023-09-27T23:14:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.986+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac00b4ea0 con 0x7f3ac014fba0 2023-09-27T23:14:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.986+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f3ab401b3b0 con 0x7f3ac014fba0 2023-09-27T23:14:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.986+0000 7f3ab17fa700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3aa4060950 0x7f3aa4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.986+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3ab407c580 con 0x7f3ac014fba0 2023-09-27T23:14:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.986+0000 7f3ac541e700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3aa4060950 0x7f3aa4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.990+0000 7f3ac541e700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3aa4060950 0x7f3aa4062e00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0113e0 tx=0x7f3abc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:05.994+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3ab404c510 con 0x7f3ac014fba0 2023-09-27T23:14:06.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ac0071120 con 0x7f3ac014fba0 2023-09-27T23:14:06.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ab17fa700 1 -- 172.21.15.64:0/2145373312 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f3ab404fb30 con 0x7f3ac014fba0 2023-09-27T23:14:06.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:06.198 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3aa4060950 msgr2=0x7f3aa4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3aa4060950 0x7f3aa4062e00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0113e0 tx=0x7f3abc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 msgr2=0x7f3ac00c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 0x7f3ac00c9ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400c7e0 tx=0x7f3ab4007d90 comp rx=0 tx=0).stop 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 shutdown_connections 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3aa4060950 0x7f3aa4062e00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ac014fba0 0x7f3ac00c9ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac00c5900 0x7f3ac0152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 --2- 172.21.15.64:0/2145373312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac0067c90 0x7f3ac01510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 >> 172.21.15.64:0/2145373312 conn(0x7f3ac00cd6d0 msgr2=0x7f3ac00ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 shutdown_connections 2023-09-27T23:14:06.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.194+0000 7f3ac7682700 1 -- 172.21.15.64:0/2145373312 wait complete. 2023-09-27T23:14:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:06.213 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 Processor -- start 2023-09-27T23:14:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 -- start start 2023-09-27T23:14:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd3840b1410 0x7fd3840ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd38406a580 0x7fd38406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38406c7c0 con 0x7fd38415a860 2023-09-27T23:14:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38406af40 con 0x7fd3840b1410 2023-09-27T23:14:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38406cd80 con 0x7fd38406a580 2023-09-27T23:14:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd3840b1410 0x7fd3840ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd3840b1410 0x7fd3840ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36544/0 (socket says 172.21.15.64:36544) 2023-09-27T23:14:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 -- 172.21.15.64:0/4044045551 learned_addr learned my addr 172.21.15.64:0/4044045551 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:06.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd381d9b700 1 --2- 172.21.15.64:0/4044045551 >> v2:172.21.15.123:3300/0 conn(0x7fd38406a580 0x7fd38406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 -- 172.21.15.64:0/4044045551 >> v2:172.21.15.123:3300/0 conn(0x7fd38406a580 msgr2=0x7fd38406aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 --2- 172.21.15.64:0/4044045551 >> v2:172.21.15.123:3300/0 conn(0x7fd38406a580 0x7fd38406aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 -- 172.21.15.64:0/4044045551 >> v1:172.21.15.123:6789/0 conn(0x7fd38415a860 legacy=0x7fd38415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 -- 172.21.15.64:0/4044045551 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38406b8f0 con 0x7fd3840b1410 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38259c700 1 --2- 172.21.15.64:0/4044045551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840ab440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd384068040 tx=0x7fd374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ee428756f0d1d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38159a700 1 -- 172.21.15.64:0/4044045551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374009e70 con 0x7fd3840b1410 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38159a700 1 -- 172.21.15.64:0/4044045551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400b860 con 0x7fd3840b1410 2023-09-27T23:14:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.318+0000 7fd38159a700 1 -- 172.21.15.64:0/4044045551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bc00 con 0x7fd3840b1410 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/4044045551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 msgr2=0x7fd3840ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 --2- 172.21.15.64:0/4044045551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840ab440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd384068040 tx=0x7fd374009250 comp rx=0 tx=0).stop 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/4044045551 shutdown_connections 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 --2- 172.21.15.64:0/4044045551 >> v2:172.21.15.123:3300/0 conn(0x7fd38406a580 0x7fd38406aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 --2- 172.21.15.64:0/4044045551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840ab440 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/4044045551 >> 172.21.15.64:0/4044045551 conn(0x7fd384067bb0 msgr2=0x7fd3840cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/4044045551 shutdown_connections 2023-09-27T23:14:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/4044045551 wait complete. 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 Processor -- start 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- start start 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38406a580 0x7fd384148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd38415a860 0x7fd3840c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840c66c0 con 0x7fd3840b1410 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840a4db0 con 0x7fd38406a580 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840c6070 con 0x7fd38415a860 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36554/0 (socket says 172.21.15.64:36554) 2023-09-27T23:14:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 -- 172.21.15.64:0/955760205 learned_addr learned my addr 172.21.15.64:0/955760205 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38259c700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38406a580 0x7fd384148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 -- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd38415a860 msgr2=0x7fd3840c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd382d9d700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd38415a860 0x7fd3840c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd38415a860 0x7fd3840c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 -- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38406a580 msgr2=0x7fd384148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38406a580 0x7fd384148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 -- 172.21.15.64:0/955760205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740094f0 con 0x7fd3840b1410 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38259c700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38406a580 0x7fd384148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd382d9d700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd38415a860 0x7fd3840c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd381d9b700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840a4440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e930 tx=0x7fd36c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00eed0 con 0x7fd3840b1410 2023-09-27T23:14:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00c850 con 0x7fd3840b1410 2023-09-27T23:14:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3840c6350 con 0x7fd3840b1410 2023-09-27T23:14:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00cbf0 con 0x7fd3840b1410 2023-09-27T23:14:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.322+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd384148870 con 0x7fd3840b1410 2023-09-27T23:14:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.326+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38404be40 con 0x7fd3840b1410 2023-09-27T23:14:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.326+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fd36c013470 con 0x7fd3840b1410 2023-09-27T23:14:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.326+0000 7fd37a7fc700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd364060950 0x7fd364062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.326+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd36c07d410 con 0x7fd3840b1410 2023-09-27T23:14:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.326+0000 7fd38259c700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd364060950 0x7fd364062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.326+0000 7fd38259c700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd364060950 0x7fd364062e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd384068040 tx=0x7fd374017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.334+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd36c04d3a0 con 0x7fd3840b1410 2023-09-27T23:14:06.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.494+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3840c6800 con 0x7fd3840b1410 2023-09-27T23:14:06.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd37a7fc700 1 -- 172.21.15.64:0/955760205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fd36c0509c0 con 0x7fd3840b1410 2023-09-27T23:14:06.501 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:06.501 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd364060950 msgr2=0x7fd364062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd364060950 0x7fd364062e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd384068040 tx=0x7fd374017000 comp rx=0 tx=0).stop 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 msgr2=0x7fd3840a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840a4440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e930 tx=0x7fd36c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 shutdown_connections 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd364060950 0x7fd364062e00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd38415a860 0x7fd3840c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3840b1410 0x7fd3840a4440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 --2- 172.21.15.64:0/955760205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38406a580 0x7fd384148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 >> 172.21.15.64:0/955760205 conn(0x7fd384067bb0 msgr2=0x7fd384155630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:06.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 shutdown_connections 2023-09-27T23:14:06.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.498+0000 7fd38359e700 1 -- 172.21.15.64:0/955760205 wait complete. 2023-09-27T23:14:06.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:06.517 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:06.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 Processor -- start 2023-09-27T23:14:06.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 -- start start 2023-09-27T23:14:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff04813f2d0 0x7ff0480ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff04815a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480adc60 con 0x7ff0480c7780 2023-09-27T23:14:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04806d7d0 con 0x7ff04813f2d0 2023-09-27T23:14:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04dc4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04815a5c0 con 0x7ff04813f6a0 2023-09-27T23:14:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04d44c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff0480c7780 0x7ff0480c5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45010/0 (socket says 172.21.15.64:45010) 2023-09-27T23:14:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04d44c700 1 -- 172.21.15.64:0/2463821034 learned_addr learned my addr 172.21.15.64:0/2463821034 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff047fff700 1 --2- 172.21.15.64:0/2463821034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff04815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff04cc4b700 1 --2- 172.21.15.64:0/2463821034 >> v2:172.21.15.123:3300/0 conn(0x7ff04813f2d0 0x7ff0480ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff0477fe700 1 -- 172.21.15.64:0/2463821034 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119971051 0 0) 0x7ff0480adc60 con 0x7ff0480c7780 2023-09-27T23:14:06.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.626+0000 7ff0477fe700 1 -- 172.21.15.64:0/2463821034 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff034003570 con 0x7ff0480c7780 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff047fff700 1 -- 172.21.15.64:0/2463821034 >> v2:172.21.15.123:3300/0 conn(0x7ff04813f2d0 msgr2=0x7ff0480ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff0477fe700 1 -- 172.21.15.64:0/2463821034 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359811690 0 0) 0x7ff034003570 con 0x7ff0480c7780 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff047fff700 1 --2- 172.21.15.64:0/2463821034 >> v2:172.21.15.123:3300/0 conn(0x7ff04813f2d0 0x7ff0480ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff047fff700 1 -- 172.21.15.64:0/2463821034 >> v1:172.21.15.64:6789/0 conn(0x7ff0480c7780 legacy=0x7ff0480c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff047fff700 1 -- 172.21.15.64:0/2463821034 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480ad050 con 0x7ff04813f6a0 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff047fff700 1 --2- 172.21.15.64:0/2463821034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff04815a080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff038009910 tx=0x7ff0380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e952acc6c06ceb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff0477fe700 1 -- 172.21.15.64:0/2463821034 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400017f0 con 0x7ff04813f6a0 2023-09-27T23:14:06.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff0477fe700 1 -- 172.21.15.64:0/2463821034 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038011420 con 0x7ff04813f6a0 2023-09-27T23:14:06.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff0477fe700 1 -- 172.21.15.64:0/2463821034 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380117a0 con 0x7ff04813f6a0 2023-09-27T23:14:06.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 -- 172.21.15.64:0/2463821034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 msgr2=0x7ff04815a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/2463821034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff04815a080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff038009910 tx=0x7ff0380091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 -- 172.21.15.64:0/2463821034 shutdown_connections 2023-09-27T23:14:06.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/2463821034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff04815a080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/2463821034 >> v2:172.21.15.123:3300/0 conn(0x7ff04813f2d0 0x7ff0480ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 -- 172.21.15.64:0/2463821034 >> 172.21.15.64:0/2463821034 conn(0x7ff048067b50 msgr2=0x7ff0480cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:06.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 -- 172.21.15.64:0/2463821034 shutdown_connections 2023-09-27T23:14:06.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.630+0000 7ff04dc4d700 1 -- 172.21.15.64:0/2463821034 wait complete. 2023-09-27T23:14:06.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 Processor -- start 2023-09-27T23:14:06.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 -- start start 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 0x7ff048136df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff04813f2d0 0x7ff0480ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff0480c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480b0660 con 0x7ff0480c7780 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480af330 con 0x7ff04813f2d0 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04dc4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480af610 con 0x7ff04813f6a0 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04cc4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 0x7ff048136df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04cc4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 0x7ff048136df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36558/0 (socket says 172.21.15.64:36558) 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04cc4b700 1 -- 172.21.15.64:0/4160175474 learned_addr learned my addr 172.21.15.64:0/4160175474 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:06.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04d44c700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff0480c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff047fff700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff04813f2d0 0x7ff0480ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.634+0000 7ff04cc4b700 1 -- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 msgr2=0x7ff0480c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04cc4b700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff0480c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04cc4b700 1 -- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff04813f2d0 msgr2=0x7ff0480ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04cc4b700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff04813f2d0 0x7ff0480ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04cc4b700 1 -- 172.21.15.64:0/4160175474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009420 con 0x7ff0480c7780 2023-09-27T23:14:06.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04cc4b700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 0x7ff048136df0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ec80 tx=0x7ff0300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000c850 con 0x7ff0480c7780 2023-09-27T23:14:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03000c9b0 con 0x7ff0480c7780 2023-09-27T23:14:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0480b02f0 con 0x7ff0480c7780 2023-09-27T23:14:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000cd50 con 0x7ff0480c7780 2023-09-27T23:14:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0481555c0 con 0x7ff0480c7780 2023-09-27T23:14:06.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0480b0150 con 0x7ff0480c7780 2023-09-27T23:14:06.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.638+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ff030013680 con 0x7ff0480c7780 2023-09-27T23:14:06.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.642+0000 7ff044ff9700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff028060950 0x7ff028062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.642+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff03007da20 con 0x7ff0480c7780 2023-09-27T23:14:06.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.642+0000 7ff047fff700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff028060950 0x7ff028062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.642+0000 7ff047fff700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff028060950 0x7ff028062e00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0380091e0 tx=0x7ff038016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.650+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff03004d990 con 0x7ff0480c7780 2023-09-27T23:14:06.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.810+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0480b0150 con 0x7ff0480c7780 2023-09-27T23:14:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.810+0000 7ff044ff9700 1 -- 172.21.15.64:0/4160175474 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff030021020 con 0x7ff0480c7780 2023-09-27T23:14:06.816 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff028060950 msgr2=0x7ff028062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff028060950 0x7ff028062e00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0380091e0 tx=0x7ff038016000 comp rx=0 tx=0).stop 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 msgr2=0x7ff048136df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 0x7ff048136df0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ec80 tx=0x7ff0300091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 shutdown_connections 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff028060950 0x7ff028062e00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff04813f6a0 0x7ff0480c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff04813f2d0 0x7ff0480ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 --2- 172.21.15.64:0/4160175474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0480c7780 0x7ff048136df0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 >> 172.21.15.64:0/4160175474 conn(0x7ff048067b50 msgr2=0x7ff0480c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 shutdown_connections 2023-09-27T23:14:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.814+0000 7ff04dc4d700 1 -- 172.21.15.64:0/4160175474 wait complete. 2023-09-27T23:14:06.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:06.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 Processor -- start 2023-09-27T23:14:06.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 -- start start 2023-09-27T23:14:06.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f721c06b420 0x7f721c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c06da40 con 0x7f721c06b050 2023-09-27T23:14:06.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c06cec0 con 0x7f721c06b420 2023-09-27T23:14:06.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c06d1a0 con 0x7f721c15d4e0 2023-09-27T23:14:06.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f721c06b420 0x7f721c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f721c06b420 0x7f721c06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36562/0 (socket says 172.21.15.64:36562) 2023-09-27T23:14:06.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721a7fc700 1 -- 172.21.15.64:0/287212268 learned_addr learned my addr 172.21.15.64:0/287212268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:06.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.942+0000 7f721b7fe700 1 --2- 172.21.15.64:0/287212268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721a7fc700 1 -- 172.21.15.64:0/287212268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 msgr2=0x7f721c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:06.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721a7fc700 1 --2- 172.21.15.64:0/287212268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721a7fc700 1 -- 172.21.15.64:0/287212268 >> v1:172.21.15.123:6789/0 conn(0x7f721c06b050 legacy=0x7f721c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721a7fc700 1 -- 172.21.15.64:0/287212268 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c06d480 con 0x7f721c06b420 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721a7fc700 1 --2- 172.21.15.64:0/287212268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c06c7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f720c0099b0 tx=0x7f720c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a52c0d4b4997bb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f7219ffb700 1 -- 172.21.15.64:0/287212268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c009ea0 con 0x7f721c06b420 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f7219ffb700 1 -- 172.21.15.64:0/287212268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00b860 con 0x7f721c06b420 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f7219ffb700 1 -- 172.21.15.64:0/287212268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00bc00 con 0x7f721c06b420 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 -- 172.21.15.64:0/287212268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 msgr2=0x7f721c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 --2- 172.21.15.64:0/287212268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c06c7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f720c0099b0 tx=0x7f720c009250 comp rx=0 tx=0).stop 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 -- 172.21.15.64:0/287212268 shutdown_connections 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 --2- 172.21.15.64:0/287212268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 --2- 172.21.15.64:0/287212268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c06c7c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 -- 172.21.15.64:0/287212268 >> 172.21.15.64:0/287212268 conn(0x7f721c0c8cd0 msgr2=0x7f721c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 -- 172.21.15.64:0/287212268 shutdown_connections 2023-09-27T23:14:06.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 -- 172.21.15.64:0/287212268 wait complete. 2023-09-27T23:14:06.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 Processor -- start 2023-09-27T23:14:06.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.946+0000 7f721bfff700 1 -- start start 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f721c06b050 0x7f721c0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c0a8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c53a0 con 0x7f721c06b420 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c3060 con 0x7f721c06b050 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c4790 con 0x7f721c15d4e0 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36568/0 (socket says 172.21.15.64:36568) 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 -- 172.21.15.64:0/1025717745 learned_addr learned my addr 172.21.15.64:0/1025717745 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721affd700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f721c06b050 0x7f721c0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721b7fe700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c0a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 -- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 msgr2=0x7f721c0a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c0a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 -- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f721c06b050 msgr2=0x7f721c0c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f721c06b050 0x7f721c0c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 -- 172.21.15.64:0/1025717745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0094f0 con 0x7f721c06b420 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721b7fe700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c0a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721a7fc700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c155b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f721c0680c0 tx=0x7f720c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00bbe0 con 0x7f721c06b420 2023-09-27T23:14:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c011420 con 0x7f721c06b420 2023-09-27T23:14:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0117c0 con 0x7f721c06b420 2023-09-27T23:14:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c0c4a70 con 0x7f721c06b420 2023-09-27T23:14:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.950+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c0c5030 con 0x7f721c06b420 2023-09-27T23:14:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.954+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c0c5510 con 0x7f721c06b420 2023-09-27T23:14:06.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.954+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f720c011bf0 con 0x7f721c06b420 2023-09-27T23:14:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.954+0000 7f7212ffd700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f71fc060950 0x7f71fc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.954+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f720c07e9c0 con 0x7f721c06b420 2023-09-27T23:14:06.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.954+0000 7f721affd700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f71fc060950 0x7f71fc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:06.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.958+0000 7f721affd700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f71fc060950 0x7f71fc062e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7214007fd0 tx=0x7f72140016f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:06.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:06.962+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f720c04e950 con 0x7f721c06b420 2023-09-27T23:14:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.126+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f721c04be40 con 0x7f721c06b420 2023-09-27T23:14:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.126+0000 7f7212ffd700 1 -- 172.21.15.64:0/1025717745 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f720c051f70 con 0x7f721c06b420 2023-09-27T23:14:07.131 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.126+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f71fc060950 msgr2=0x7f71fc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.126+0000 7f721bfff700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f71fc060950 0x7f71fc062e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7214007fd0 tx=0x7f72140016f0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.126+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 msgr2=0x7f721c155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.126+0000 7f721bfff700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c155b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f721c0680c0 tx=0x7f720c017000 comp rx=0 tx=0).stop 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 shutdown_connections 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f71fc060950 0x7f71fc062e00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f721c15d4e0 0x7f721c0a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c06b420 0x7f721c155b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 --2- 172.21.15.64:0/1025717745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f721c06b050 0x7f721c0c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 >> 172.21.15.64:0/1025717745 conn(0x7f721c0c8cd0 msgr2=0x7f721c1585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:07.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 shutdown_connections 2023-09-27T23:14:07.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.130+0000 7f721bfff700 1 -- 172.21.15.64:0/1025717745 wait complete. 2023-09-27T23:14:07.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.250+0000 7fcd42393700 1 Processor -- start 2023-09-27T23:14:07.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.250+0000 7fcd42393700 1 -- start start 2023-09-27T23:14:07.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcd3c158c90 0x7fcd3c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd3c159770 0x7fcd3c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c06c850 con 0x7fcd3c06aeb0 2023-09-27T23:14:07.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c06b470 con 0x7fcd3c159770 2023-09-27T23:14:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c06b750 con 0x7fcd3c158c90 2023-09-27T23:14:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd3c159770 0x7fcd3c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd3c159770 0x7fcd3c06cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36578/0 (socket says 172.21.15.64:36578) 2023-09-27T23:14:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 -- 172.21.15.64:0/460592349 learned_addr learned my addr 172.21.15.64:0/460592349 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:07.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41391700 1 --2- 172.21.15.64:0/460592349 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c158c90 0x7fcd3c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 -- 172.21.15.64:0/460592349 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c158c90 msgr2=0x7fcd3c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:07.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 --2- 172.21.15.64:0/460592349 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c158c90 0x7fcd3c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 -- 172.21.15.64:0/460592349 >> v1:172.21.15.123:6789/0 conn(0x7fcd3c06aeb0 legacy=0x7fcd3c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 -- 172.21.15.64:0/460592349 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c06ba30 con 0x7fcd3c159770 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41391700 1 --2- 172.21.15.64:0/460592349 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c158c90 0x7fcd3c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd41b92700 1 --2- 172.21.15.64:0/460592349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c06cce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0698f0 tx=0x7fcd34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd61c9130841d69 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd2ffff700 1 -- 172.21.15.64:0/460592349 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400e070 con 0x7fcd3c159770 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd2ffff700 1 -- 172.21.15.64:0/460592349 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400ba50 con 0x7fcd3c159770 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd2ffff700 1 -- 172.21.15.64:0/460592349 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340092e0 con 0x7fcd3c159770 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- 172.21.15.64:0/460592349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 msgr2=0x7fcd3c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 --2- 172.21.15.64:0/460592349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c06cce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0698f0 tx=0x7fcd34009840 comp rx=0 tx=0).stop 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- 172.21.15.64:0/460592349 shutdown_connections 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 --2- 172.21.15.64:0/460592349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c06cce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 --2- 172.21.15.64:0/460592349 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c158c90 0x7fcd3c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- 172.21.15.64:0/460592349 >> 172.21.15.64:0/460592349 conn(0x7fcd3c067bb0 msgr2=0x7fcd3c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- 172.21.15.64:0/460592349 shutdown_connections 2023-09-27T23:14:07.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- 172.21.15.64:0/460592349 wait complete. 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 Processor -- start 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.254+0000 7fcd42393700 1 -- start start 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c06aeb0 0x7fcd3c1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c158c90 0x7fcd3c0c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0c60d0 con 0x7fcd3c159770 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c153650 con 0x7fcd3c158c90 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0c5220 con 0x7fcd3c06aeb0 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36592/0 (socket says 172.21.15.64:36592) 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 -- 172.21.15.64:0/3196927952 learned_addr learned my addr 172.21.15.64:0/3196927952 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd40b90700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c158c90 0x7fcd3c0c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41391700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c06aeb0 0x7fcd3c1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 -- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c06aeb0 msgr2=0x7fcd3c1573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c06aeb0 0x7fcd3c1573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 -- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c158c90 msgr2=0x7fcd3c0c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c158c90 0x7fcd3c0c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 -- 172.21.15.64:0/3196927952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34009ae0 con 0x7fcd3c159770 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd40b90700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c158c90 0x7fcd3c0c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd41b92700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c154880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c069b70 tx=0x7fcd34018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34008e40 con 0x7fcd3c159770 2023-09-27T23:14:07.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400be60 con 0x7fcd3c159770 2023-09-27T23:14:07.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c0c54c0 con 0x7fcd3c159770 2023-09-27T23:14:07.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3401c640 con 0x7fcd3c159770 2023-09-27T23:14:07.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c0c5a80 con 0x7fcd3c159770 2023-09-27T23:14:07.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.258+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c04be40 con 0x7fcd3c159770 2023-09-27T23:14:07.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.262+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fcd34003040 con 0x7fcd3c159770 2023-09-27T23:14:07.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.262+0000 7fcd2d7fa700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd20060950 0x7fcd20062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.262+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcd3407ea60 con 0x7fcd3c159770 2023-09-27T23:14:07.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.262+0000 7fcd41391700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd20060950 0x7fcd20062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.262+0000 7fcd41391700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd20060950 0x7fcd20062e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcd38006f00 tx=0x7fcd38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.266+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcd3404e9f0 con 0x7fcd3c159770 2023-09-27T23:14:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.426+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd3c0c6210 con 0x7fcd3c159770 2023-09-27T23:14:07.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.426+0000 7fcd2d7fa700 1 -- 172.21.15.64:0/3196927952 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd34009940 con 0x7fcd3c159770 2023-09-27T23:14:07.432 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd20060950 msgr2=0x7fcd20062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd20060950 0x7fcd20062e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcd38006f00 tx=0x7fcd38008040 comp rx=0 tx=0).stop 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 msgr2=0x7fcd3c154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c154880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c069b70 tx=0x7fcd34018000 comp rx=0 tx=0).stop 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 shutdown_connections 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd20060950 0x7fcd20062e00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c159770 0x7fcd3c154880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c158c90 0x7fcd3c0c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 --2- 172.21.15.64:0/3196927952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c06aeb0 0x7fcd3c1573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 >> 172.21.15.64:0/3196927952 conn(0x7fcd3c067bb0 msgr2=0x7fcd3c0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 shutdown_connections 2023-09-27T23:14:07.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.430+0000 7fcd42393700 1 -- 172.21.15.64:0/3196927952 wait complete. 2023-09-27T23:14:07.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:07.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.558+0000 7f077b59e700 1 Processor -- start 2023-09-27T23:14:07.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.558+0000 7f077b59e700 1 -- start start 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f077c0ace10 0x7f077c0ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0ae110 con 0x7f077c06daa0 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c06e0c0 con 0x7f077c0c84e0 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c157110 con 0x7f077c0ace10 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077ad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f077c06daa0 0x7f077c0ae5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45022/0 (socket says 172.21.15.64:45022) 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077ad9d700 1 -- 172.21.15.64:0/1089565954 learned_addr learned my addr 172.21.15.64:0/1089565954 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077a59c700 1 --2- 172.21.15.64:0/1089565954 >> v2:172.21.15.123:3300/0 conn(0x7f077c0ace10 0x7f077c0ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077959a700 1 -- 172.21.15.64:0/1089565954 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351433421 0 0) 0x7f077c0ae110 con 0x7f077c06daa0 2023-09-27T23:14:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077959a700 1 -- 172.21.15.64:0/1089565954 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0764003570 con 0x7f077c06daa0 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077a59c700 1 -- 172.21.15.64:0/1089565954 >> v1:172.21.15.123:6789/0 conn(0x7f077c0c84e0 legacy=0x7f077c06d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077959a700 1 -- 172.21.15.64:0/1089565954 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834070455 0 0) 0x7f0764003570 con 0x7f077c06daa0 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077a59c700 1 -- 172.21.15.64:0/1089565954 >> v1:172.21.15.64:6789/0 conn(0x7f077c06daa0 legacy=0x7f077c0ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077a59c700 1 -- 172.21.15.64:0/1089565954 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0ad7e0 con 0x7f077c0ace10 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077a59c700 1 --2- 172.21.15.64:0/1089565954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c0ace10 0x7f077c0ad1e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07700098e0 tx=0x7f07700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6486af63ff6ac server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077959a700 1 -- 172.21.15.64:0/1089565954 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740017f0 con 0x7f077c0ace10 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077959a700 1 -- 172.21.15.64:0/1089565954 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770011420 con 0x7f077c0ace10 2023-09-27T23:14:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077959a700 1 -- 172.21.15.64:0/1089565954 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700117a0 con 0x7f077c0ace10 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- 172.21.15.64:0/1089565954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c0ace10 msgr2=0x7f077c0ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 --2- 172.21.15.64:0/1089565954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c0ace10 0x7f077c0ad1e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07700098e0 tx=0x7f07700091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- 172.21.15.64:0/1089565954 shutdown_connections 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 --2- 172.21.15.64:0/1089565954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c0ace10 0x7f077c0ad1e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- 172.21.15.64:0/1089565954 >> 172.21.15.64:0/1089565954 conn(0x7f077c0cdc00 msgr2=0x7f077c0ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- 172.21.15.64:0/1089565954 shutdown_connections 2023-09-27T23:14:07.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.562+0000 7f077b59e700 1 -- 172.21.15.64:0/1089565954 wait complete. 2023-09-27T23:14:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 Processor -- start 2023-09-27T23:14:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 -- start start 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 0x7f077c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f077c0ace10 0x7f077c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f077c0c84e0 0x7f077c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1553b0 con 0x7f077c0ace10 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1549e0 con 0x7f077c06daa0 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c153dd0 con 0x7f077c0c84e0 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f077c0ace10 0x7f077c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 0x7f077c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077ad9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f077c0c84e0 0x7f077c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 0x7f077c154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:55992/0 (socket says 172.21.15.64:55992) 2023-09-27T23:14:07.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077a59c700 1 -- 172.21.15.64:0/2461948789 learned_addr learned my addr 172.21.15.64:0/2461948789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077ad9d700 1 -- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 msgr2=0x7f077c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077ad9d700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 0x7f077c154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077ad9d700 1 -- 172.21.15.64:0/2461948789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f077c0ace10 msgr2=0x7f077c0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077ad9d700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f077c0ace10 0x7f077c0b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077ad9d700 1 -- 172.21.15.64:0/2461948789 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009420 con 0x7f077c0c84e0 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f0779d9b700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f077c0ace10 0x7f077c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.566+0000 7f077a59c700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 0x7f077c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f077ad9d700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f077c0c84e0 0x7f077c0ccb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f077400cbc0 tx=0x7f0774007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774004dc0 con 0x7f077c0c84e0 2023-09-27T23:14:07.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400a590 con 0x7f077c0c84e0 2023-09-27T23:14:07.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c1540b0 con 0x7f077c0c84e0 2023-09-27T23:14:07.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400a950 con 0x7f077c0c84e0 2023-09-27T23:14:07.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c154670 con 0x7f077c0c84e0 2023-09-27T23:14:07.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c154b70 con 0x7f077c0c84e0 2023-09-27T23:14:07.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.570+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f077400acc0 con 0x7f077c0c84e0 2023-09-27T23:14:07.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.574+0000 7f076a7fc700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f075c060950 0x7f075c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.574+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f077407c730 con 0x7f077c0c84e0 2023-09-27T23:14:07.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.574+0000 7f077a59c700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f075c060950 0x7f075c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.574+0000 7f077a59c700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f075c060950 0x7f075c062e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0770003080 tx=0x7f0770016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.586+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f077404c6c0 con 0x7f077c0c84e0 2023-09-27T23:14:07.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.746+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f077c04be40 con 0x7f077c0c84e0 2023-09-27T23:14:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.746+0000 7f076a7fc700 1 -- 172.21.15.64:0/2461948789 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0774018020 con 0x7f077c0c84e0 2023-09-27T23:14:07.752 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f075c060950 msgr2=0x7f075c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f075c060950 0x7f075c062e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0770003080 tx=0x7f0770016000 comp rx=0 tx=0).stop 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f077c0c84e0 msgr2=0x7f077c0ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f077c0c84e0 0x7f077c0ccb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f077400cbc0 tx=0x7f0774007f30 comp rx=0 tx=0).stop 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 shutdown_connections 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f075c060950 0x7f075c062e00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f077c0c84e0 0x7f077c0ccb90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f077c0ace10 0x7f077c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 --2- 172.21.15.64:0/2461948789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c06daa0 0x7f077c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.750+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 >> 172.21.15.64:0/2461948789 conn(0x7f077c0cdc00 msgr2=0x7f077c156620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.754+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 shutdown_connections 2023-09-27T23:14:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.754+0000 7f077b59e700 1 -- 172.21.15.64:0/2461948789 wait complete. 2023-09-27T23:14:07.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:07.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.882+0000 7fbfd3fff700 1 Processor -- start 2023-09-27T23:14:07.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.882+0000 7fbfd3fff700 1 -- start start 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd406da40 con 0x7fbfd406b050 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd406cec0 con 0x7fbfd415d4e0 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd406d1a0 con 0x7fbfd406b420 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd2ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbfd406b050 0x7fbfd41594b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45034/0 (socket says 172.21.15.64:45034) 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd2ffd700 1 -- 172.21.15.64:0/4033234404 learned_addr learned my addr 172.21.15.64:0/4033234404 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:07.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd27fc700 1 --2- 172.21.15.64:0/4033234404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888950612 0 0) 0x7fbfd406da40 con 0x7fbfd406b050 2023-09-27T23:14:07.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc0003570 con 0x7fbfd406b050 2023-09-27T23:14:07.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690753143 0 0) 0x7fbfd406cec0 con 0x7fbfd415d4e0 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd406da40 con 0x7fbfd415d4e0 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd27fc700 1 -- 172.21.15.64:0/4033234404 >> v1:172.21.15.123:6789/0 conn(0x7fbfd415d4e0 legacy=0x7fbfd4159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808095884 0 0) 0x7fbfc0003570 con 0x7fbfd406b050 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd27fc700 1 -- 172.21.15.64:0/4033234404 >> v1:172.21.15.64:6789/0 conn(0x7fbfd406b050 legacy=0x7fbfd41594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd27fc700 1 -- 172.21.15.64:0/4033234404 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd406d480 con 0x7fbfd406b420 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd27fc700 1 --2- 172.21.15.64:0/4033234404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd406c7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c676d9c8afcfe985 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4001470 con 0x7fbfd406b420 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc011420 con 0x7fbfd406b420 2023-09-27T23:14:07.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd1ffb700 1 -- 172.21.15.64:0/4033234404 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0117c0 con 0x7fbfd406b420 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- 172.21.15.64:0/4033234404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 msgr2=0x7fbfd406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/4033234404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd406c7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- 172.21.15.64:0/4033234404 shutdown_connections 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/4033234404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd406c7c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- 172.21.15.64:0/4033234404 >> 172.21.15.64:0/4033234404 conn(0x7fbfd40c8cd0 msgr2=0x7fbfd40c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.886+0000 7fbfd3fff700 1 -- 172.21.15.64:0/4033234404 shutdown_connections 2023-09-27T23:14:07.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.890+0000 7fbfd3fff700 1 -- 172.21.15.64:0/4033234404 wait complete. 2023-09-27T23:14:07.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.890+0000 7fbfd3fff700 1 Processor -- start 2023-09-27T23:14:07.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.890+0000 7fbfd3fff700 1 -- start start 2023-09-27T23:14:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 0x7fbfd40c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfd415d4e0 0x7fbfd4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40c3080 con 0x7fbfd406b050 2023-09-27T23:14:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40c4500 con 0x7fbfd415d4e0 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40c38f0 con 0x7fbfd406b420 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 0x7fbfd40c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 0x7fbfd40c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36614/0 (socket says 172.21.15.64:36614) 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 -- 172.21.15.64:0/3841592441 learned_addr learned my addr 172.21.15.64:0/3841592441 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd27fc700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd37fe700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfd415d4e0 0x7fbfd4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 -- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 msgr2=0x7fbfd40c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd40c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 -- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfd415d4e0 msgr2=0x7fbfd4155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfd415d4e0 0x7fbfd4155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 -- 172.21.15.64:0/3841592441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009420 con 0x7fbfd406b050 2023-09-27T23:14:07.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd2ffd700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 0x7fbfd40c3310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400cb70 tx=0x7fbfc4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4007150 con 0x7fbfd406b050 2023-09-27T23:14:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc40072b0 con 0x7fbfd406b050 2023-09-27T23:14:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4007670 con 0x7fbfd406b050 2023-09-27T23:14:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40c3bd0 con 0x7fbfd406b050 2023-09-27T23:14:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd40c4190 con 0x7fbfd406b050 2023-09-27T23:14:07.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.894+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd404be40 con 0x7fbfd406b050 2023-09-27T23:14:07.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.898+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbfc4004a40 con 0x7fbfd406b050 2023-09-27T23:14:07.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.898+0000 7fbfcaffd700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbfb4060950 0x7fbfb4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:07.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.898+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbfc407c3d0 con 0x7fbfd406b050 2023-09-27T23:14:07.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.898+0000 7fbfd27fc700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbfb4060950 0x7fbfb4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:07.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.898+0000 7fbfd27fc700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbfb4060950 0x7fbfb4062e00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0030c0 tx=0x7fbfbc016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:07.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:07.906+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbfc404fe70 con 0x7fbfd406b050 2023-09-27T23:14:08.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.070+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbfd40cbe30 con 0x7fbfd406b050 2023-09-27T23:14:08.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.070+0000 7fbfcaffd700 1 -- 172.21.15.64:0/3841592441 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbfc4018020 con 0x7fbfd406b050 2023-09-27T23:14:08.074 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbfb4060950 msgr2=0x7fbfb4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbfb4060950 0x7fbfb4062e00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0030c0 tx=0x7fbfbc016000 comp rx=0 tx=0).stop 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 msgr2=0x7fbfd40c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 0x7fbfd40c3310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400cb70 tx=0x7fbfc4001ab0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 shutdown_connections 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbfb4060950 0x7fbfb4062e00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfd415d4e0 0x7fbfd4155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfd406b420 0x7fbfd40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 --2- 172.21.15.64:0/3841592441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfd406b050 0x7fbfd40c3310 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 >> 172.21.15.64:0/3841592441 conn(0x7fbfd40c8cd0 msgr2=0x7fbfd415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:08.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 shutdown_connections 2023-09-27T23:14:08.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.074+0000 7fbfd3fff700 1 -- 172.21.15.64:0/3841592441 wait complete. 2023-09-27T23:14:08.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:08.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516adc6700 1 Processor -- start 2023-09-27T23:14:08.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516adc6700 1 -- start start 2023-09-27T23:14:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516adc6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51640c6cd0 0x7f516406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516adc6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516406b470 con 0x7f51640c6900 2023-09-27T23:14:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516adc6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516406e110 con 0x7f516415cc20 2023-09-27T23:14:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516adc6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516406be40 con 0x7f51640c6cd0 2023-09-27T23:14:08.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f51640c6900 0x7f516406da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45044/0 (socket says 172.21.15.64:45044) 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 -- 172.21.15.64:0/3189896563 learned_addr learned my addr 172.21.15.64:0/3189896563 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f51695c3700 1 --2- 172.21.15.64:0/3189896563 >> v2:172.21.15.123:3300/0 conn(0x7f51640c6cd0 0x7f516406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73662331 0 0) 0x7f516406b470 con 0x7f51640c6900 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5150003570 con 0x7f51640c6900 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153833830 0 0) 0x7f5150003570 con 0x7f51640c6900 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 >> v2:172.21.15.123:3300/0 conn(0x7f51640c6cd0 msgr2=0x7f516406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 --2- 172.21.15.64:0/3189896563 >> v2:172.21.15.123:3300/0 conn(0x7f51640c6cd0 0x7f516406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 >> v1:172.21.15.123:6789/0 conn(0x7f516415cc20 legacy=0x7f516415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164161d10 con 0x7f51640c6900 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f5160001470 con 0x7f51640c6900 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 >> v1:172.21.15.64:6789/0 conn(0x7f51640c6900 legacy=0x7f516406da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f5150003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5150004810 0x7f5150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51500051c0 0x7f515000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515000c0a0 con 0x7f5150003a60 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515000c210 con 0x7f5150004810 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515000c3d0 con 0x7f51500051c0 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f5150003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f51695c3700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5150004810 0x7f5150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 -- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51500051c0 msgr2=0x7f515000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516a5c5700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51500051c0 0x7f515000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51500051c0 0x7f515000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 -- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5150004810 msgr2=0x7f5150004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5150004810 0x7f5150004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 -- 172.21.15.64:0/3189896563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150003570 con 0x7f5150003a60 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f51695c3700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5150004810 0x7f5150004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516a5c5700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51500051c0 0x7f515000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f516a5c5700 1 -- 172.21.15.64:0/3189896563 reap_dead start 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5169dc4700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f5150003e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f516000c3c0 tx=0x7f5160001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce72d06beddae779 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000cc90 con 0x7f5150003a60 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516000cdf0 con 0x7f5150003a60 2023-09-27T23:14:08.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.206+0000 7f5168dc2700 1 -- 172.21.15.64:0/3189896563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160005b50 con 0x7f5150003a60 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- 172.21.15.64:0/3189896563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 msgr2=0x7f5150003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f5150003e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f516000c3c0 tx=0x7f5160001e50 comp rx=0 tx=0).stop 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- 172.21.15.64:0/3189896563 shutdown_connections 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 --2- 172.21.15.64:0/3189896563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f5150003e70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- 172.21.15.64:0/3189896563 >> 172.21.15.64:0/3189896563 conn(0x7f5164067bb0 msgr2=0x7f51640cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- 172.21.15.64:0/3189896563 shutdown_connections 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- 172.21.15.64:0/3189896563 wait complete. 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 Processor -- start 2023-09-27T23:14:08.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- start start 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f516416e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f516416d800 0x7f51640acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f516416dc70 0x7f51640ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516415dc60 con 0x7f5150003a60 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516415d020 con 0x7f516416dc70 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516adc6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516415d2c0 con 0x7f516416d800 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f5169dc4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f516416e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f5169dc4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f516416e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36628/0 (socket says 172.21.15.64:36628) 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f5169dc4700 1 -- 172.21.15.64:0/1976467243 learned_addr learned my addr 172.21.15.64:0/1976467243 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f516a5c5700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f516416dc70 0x7f51640ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.210+0000 7f51695c3700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f516416d800 0x7f51640acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5169dc4700 1 -- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f516416d800 msgr2=0x7f51640acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5169dc4700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f516416d800 0x7f51640acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5169dc4700 1 -- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f516416dc70 msgr2=0x7f51640ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5169dc4700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f516416dc70 0x7f51640ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5169dc4700 1 -- 172.21.15.64:0/1976467243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516415d5a0 con 0x7f5150003a60 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f51695c3700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f516416d800 0x7f51640acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f516a5c5700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f516416dc70 0x7f51640ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5169dc4700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f516416e1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f515000c1e0 tx=0x7f5160015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000cc90 con 0x7f5150003a60 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5160004e40 con 0x7f5150003a60 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516415d880 con 0x7f5150003a60 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160007510 con 0x7f5150003a60 2023-09-27T23:14:08.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51640b0360 con 0x7f5150003a60 2023-09-27T23:14:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f5160028460 con 0x7f5150003a60 2023-09-27T23:14:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5159ffb700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5148060950 0x7f5148062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516404be40 con 0x7f5150003a60 2023-09-27T23:14:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f516007e770 con 0x7f5150003a60 2023-09-27T23:14:08.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.214+0000 7f51695c3700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5148060950 0x7f5148062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.218+0000 7f51695c3700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5148060950 0x7f5148062e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5154006dc0 tx=0x7f5154008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.222+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5160052190 con 0x7f5150003a60 2023-09-27T23:14:08.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.418+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f516406a580 con 0x7f5150003a60 2023-09-27T23:14:08.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f5159ffb700 1 -- 172.21.15.64:0/1976467243 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f5160017020 con 0x7f5150003a60 2023-09-27T23:14:08.426 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:08.426 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5148060950 msgr2=0x7f5148062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5148060950 0x7f5148062e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5154006dc0 tx=0x7f5154008040 comp rx=0 tx=0).stop 2023-09-27T23:14:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 msgr2=0x7f516416e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f516416e1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f515000c1e0 tx=0x7f5160015000 comp rx=0 tx=0).stop 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 shutdown_connections 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5148060950 0x7f5148062e00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f516416dc70 0x7f51640ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f516416d800 0x7f51640acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 --2- 172.21.15.64:0/1976467243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5150003a60 0x7f516416e1a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 >> 172.21.15.64:0/1976467243 conn(0x7f5164067bb0 msgr2=0x7f516415c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 shutdown_connections 2023-09-27T23:14:08.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.422+0000 7f516adc6700 1 -- 172.21.15.64:0/1976467243 wait complete. 2023-09-27T23:14:08.430 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:08.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:08.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 Processor -- start 2023-09-27T23:14:08.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 -- start start 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe170155990 0x7fe1700c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b0670 0x7fe1700b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17006cf00 con 0x7fe1700b13b0 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1700ae7e0 con 0x7fe170155990 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.542+0000 7fe1756de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700cbc50 con 0x7fe1700b0670 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe170155990 0x7fe1700c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe170155990 0x7fe1700c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36638/0 (socket says 172.21.15.64:36638) 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 -- 172.21.15.64:0/1033253911 learned_addr learned my addr 172.21.15.64:0/1033253911 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:08.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe174edd700 1 --1- 172.21.15.64:0/1033253911 >> v1:172.21.15.64:6789/0 conn(0x7fe1700b13b0 0x7fe17006d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45048/0 (socket says 172.21.15.64:45048) 2023-09-27T23:14:08.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16f7fe700 1 --2- 172.21.15.64:0/1033253911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b0670 0x7fe1700b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 -- 172.21.15.64:0/1033253911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b0670 msgr2=0x7fe1700b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 --2- 172.21.15.64:0/1033253911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b0670 0x7fe1700b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 -- 172.21.15.64:0/1033253911 >> v1:172.21.15.64:6789/0 conn(0x7fe1700b13b0 legacy=0x7fe17006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 -- 172.21.15.64:0/1033253911 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700b01d0 con 0x7fe170155990 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16ffff700 1 --2- 172.21.15.64:0/1033253911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe170155990 0x7fe1700c97d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1680099b0 tx=0x7fe168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9018af7148ec2aa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16effd700 1 -- 172.21.15.64:0/1033253911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800b860 con 0x7fe170155990 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16effd700 1 -- 172.21.15.64:0/1033253911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16800b9c0 con 0x7fe170155990 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe16effd700 1 -- 172.21.15.64:0/1033253911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800bd60 con 0x7fe170155990 2023-09-27T23:14:08.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 -- 172.21.15.64:0/1033253911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe170155990 msgr2=0x7fe1700c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 --2- 172.21.15.64:0/1033253911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe170155990 0x7fe1700c97d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1680099b0 tx=0x7fe168009250 comp rx=0 tx=0).stop 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 -- 172.21.15.64:0/1033253911 shutdown_connections 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 --2- 172.21.15.64:0/1033253911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b0670 0x7fe1700b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 --2- 172.21.15.64:0/1033253911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe170155990 0x7fe1700c97d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 -- 172.21.15.64:0/1033253911 >> 172.21.15.64:0/1033253911 conn(0x7fe170067bb0 msgr2=0x7fe1700add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 -- 172.21.15.64:0/1033253911 shutdown_connections 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.546+0000 7fe1756de700 1 -- 172.21.15.64:0/1033253911 wait complete. 2023-09-27T23:14:08.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 Processor -- start 2023-09-27T23:14:08.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 -- start start 2023-09-27T23:14:08.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 0x7fe17014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b13b0 0x7fe1701696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170155990 0x7fe170152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17014f600 con 0x7fe1700b0670 2023-09-27T23:14:08.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17014ffd0 con 0x7fe170155990 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17014ecd0 con 0x7fe1700b13b0 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 0x7fe17014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 0x7fe17014fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36654/0 (socket says 172.21.15.64:36654) 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 -- 172.21.15.64:0/88467603 learned_addr learned my addr 172.21.15.64:0/88467603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe174edd700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170155990 0x7fe170152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16f7fe700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b13b0 0x7fe1701696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 -- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b13b0 msgr2=0x7fe1701696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b13b0 0x7fe1701696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 -- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170155990 msgr2=0x7fe170152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170155990 0x7fe170152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 -- 172.21.15.64:0/88467603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680094f0 con 0x7fe1700b0670 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe174edd700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170155990 0x7fe170152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16f7fe700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b13b0 0x7fe1701696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe16ffff700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 0x7fe17014fa90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe170069fb0 tx=0x7fe168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168024440 con 0x7fe1700b0670 2023-09-27T23:14:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17014efb0 con 0x7fe1700b0670 2023-09-27T23:14:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1680245a0 con 0x7fe1700b0670 2023-09-27T23:14:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe170169be0 con 0x7fe1700b0670 2023-09-27T23:14:08.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.550+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168006cf0 con 0x7fe1700b0670 2023-09-27T23:14:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.554+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17014ee10 con 0x7fe1700b0670 2023-09-27T23:14:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.554+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe168024920 con 0x7fe1700b0670 2023-09-27T23:14:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.554+0000 7fe167fff700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe150060980 0x7fe150062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.554+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe16807e890 con 0x7fe1700b0670 2023-09-27T23:14:08.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.554+0000 7fe16f7fe700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe150060980 0x7fe150062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.554+0000 7fe16f7fe700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe150060980 0x7fe150062e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe1701668d0 tx=0x7fe1580097e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.558+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe17014ee10 con 0x7fe1700b0670 2023-09-27T23:14:08.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.718+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe17014ee10 con 0x7fe1700b0670 2023-09-27T23:14:08.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe167fff700 1 -- 172.21.15.64:0/88467603 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fe17014ee10 con 0x7fe1700b0670 2023-09-27T23:14:08.727 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:08.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe150060980 msgr2=0x7fe150062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe150060980 0x7fe150062e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe1701668d0 tx=0x7fe1580097e0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 msgr2=0x7fe17014fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 0x7fe17014fa90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe170069fb0 tx=0x7fe168017000 comp rx=0 tx=0).stop 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 shutdown_connections 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe150060980 0x7fe150062e30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170155990 0x7fe170152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1700b13b0 0x7fe1701696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 --2- 172.21.15.64:0/88467603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700b0670 0x7fe17014fa90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 >> 172.21.15.64:0/88467603 conn(0x7fe170067bb0 msgr2=0x7fe1700cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 shutdown_connections 2023-09-27T23:14:08.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.722+0000 7fe1756de700 1 -- 172.21.15.64:0/88467603 wait complete. 2023-09-27T23:14:08.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:08.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 Processor -- start 2023-09-27T23:14:08.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 -- start start 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 0x7f16d40b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16d40b13b0 0x7f16d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d40c9c20 con 0x7f16d4155990 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d40ae7e0 con 0x7f16d40b13b0 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.838+0000 7f16d8a8a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40cbc50 con 0x7f16d40c97d0 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16d40b13b0 0x7f16d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16cbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 0x7f16d40b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16cbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 0x7f16d40b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:39888/0 (socket says 172.21.15.64:39888) 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16d40b13b0 0x7f16d406d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:56066/0 (socket says 172.21.15.64:56066) 2023-09-27T23:14:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16cbfff700 1 -- 172.21.15.64:0/2459836547 learned_addr learned my addr 172.21.15.64:0/2459836547 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f16d4155990 0x7f16d40b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:43840/0 (socket says 172.21.15.64:43840) 2023-09-27T23:14:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 -- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 msgr2=0x7f16d40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 --2- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 0x7f16d40b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 -- 172.21.15.64:0/2459836547 >> v1:172.21.15.123:6789/0 conn(0x7f16d4155990 legacy=0x7f16d40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 -- 172.21.15.64:0/2459836547 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d406cf00 con 0x7f16d40b13b0 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16cbfff700 1 --2- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 0x7f16d40b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d3fff700 1 --2- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d406d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009e60 tx=0x7f16cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44780889374bd40 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d2ffd700 1 -- 172.21.15.64:0/2459836547 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00e070 con 0x7f16d40b13b0 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d2ffd700 1 -- 172.21.15.64:0/2459836547 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc00bb60 con 0x7f16d40b13b0 2023-09-27T23:14:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d2ffd700 1 -- 172.21.15.64:0/2459836547 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc012430 con 0x7f16d40b13b0 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 msgr2=0x7f16d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d406d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009e60 tx=0x7f16cc009780 comp rx=0 tx=0).stop 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2459836547 shutdown_connections 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d406d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2459836547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d40c97d0 0x7f16d40b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2459836547 >> 172.21.15.64:0/2459836547 conn(0x7f16d4067bb0 msgr2=0x7f16d40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2459836547 shutdown_connections 2023-09-27T23:14:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.842+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2459836547 wait complete. 2023-09-27T23:14:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 Processor -- start 2023-09-27T23:14:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 -- start start 2023-09-27T23:14:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d40cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 0x7f16d4147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d4155990 0x7f16d40cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40cb770 con 0x7f16d40c97d0 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d414c730 con 0x7f16d40b13b0 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d8a8a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d414ca10 con 0x7f16d4155990 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 0x7f16d4147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 0x7f16d4147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36666/0 (socket says 172.21.15.64:36666) 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 -- 172.21.15.64:0/2334314250 learned_addr learned my addr 172.21.15.64:0/2334314250 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d37fe700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d40cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16d3fff700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d4155990 0x7f16d40cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 -- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d4155990 msgr2=0x7f16d40cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d4155990 0x7f16d40cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 -- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 msgr2=0x7f16d40cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d40cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 -- 172.21.15.64:0/2334314250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0094f0 con 0x7f16d40c97d0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.846+0000 7f16cbfff700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 0x7f16d4147e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00ec50 tx=0x7f16bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00c850 con 0x7f16d40c97d0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc00c9b0 con 0x7f16d40c97d0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc009a20 con 0x7f16d40c97d0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00cd50 con 0x7f16d40c97d0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d414cda0 con 0x7f16d40c97d0 2023-09-27T23:14:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d404be40 con 0x7f16d40c97d0 2023-09-27T23:14:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f16bc013680 con 0x7f16d40c97d0 2023-09-27T23:14:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16cb7fe700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f16b4060950 0x7f16b4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.850+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f16bc07d220 con 0x7f16d40c97d0 2023-09-27T23:14:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.854+0000 7f16d37fe700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f16b4060950 0x7f16b4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.854+0000 7f16d37fe700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f16b4060950 0x7f16b4062e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007130 tx=0x7f16c4000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:08.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:08.858+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f16bc04d110 con 0x7f16d40c97d0 2023-09-27T23:14:09.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.054+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16d406b0f0 con 0x7f16d40c97d0 2023-09-27T23:14:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.054+0000 7f16cb7fe700 1 -- 172.21.15.64:0/2334314250 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f16bc050730 con 0x7f16d40c97d0 2023-09-27T23:14:09.060 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:09.060 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:09.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f16b4060950 msgr2=0x7f16b4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f16b4060950 0x7f16b4062e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007130 tx=0x7f16c4000bc0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 msgr2=0x7f16d4147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 0x7f16d4147e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00ec50 tx=0x7f16bc009250 comp rx=0 tx=0).stop 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 shutdown_connections 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f16b4060950 0x7f16b4062e00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16d4155990 0x7f16d40cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16d40c97d0 0x7f16d4147e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 --2- 172.21.15.64:0/2334314250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d40b13b0 0x7f16d40cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 >> 172.21.15.64:0/2334314250 conn(0x7f16d4067bb0 msgr2=0x7f16d406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 shutdown_connections 2023-09-27T23:14:09.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.058+0000 7f16d8a8a700 1 -- 172.21.15.64:0/2334314250 wait complete. 2023-09-27T23:14:09.064 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:09.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:13:44.760383+0000', 'modified': '2023-09-27T23:13:56.848999+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6835/2280995923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2280995923}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2280995923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:14:09.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:14:09.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-27T23:14:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 Processor -- start 2023-09-27T23:14:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 -- start start 2023-09-27T23:14:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9d9c06d280 0x7f9d9c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9c0ad070 con 0x7f9d9c068160 2023-09-27T23:14:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9c0ac780 con 0x7f9d9c06d280 2023-09-27T23:14:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.594+0000 7f9da3511700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c068e90 con 0x7f9d9c15aa10 2023-09-27T23:14:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da250f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9d9c06d280 0x7f9d9c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c15ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:39904/0 (socket says 172.21.15.64:39904) 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da1d0e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9d9c068160 0x7f9d9c0685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:43850/0 (socket says 172.21.15.64:43850) 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 -- 172.21.15.64:0/1013905675 learned_addr learned my addr 172.21.15.64:0/1013905675 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 -- 172.21.15.64:0/1013905675 >> v2:172.21.15.123:3300/0 conn(0x7f9d9c06d280 msgr2=0x7f9d9c06d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 --2- 172.21.15.64:0/1013905675 >> v2:172.21.15.123:3300/0 conn(0x7f9d9c06d280 0x7f9d9c06d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 -- 172.21.15.64:0/1013905675 >> v1:172.21.15.123:6789/0 conn(0x7f9d9c068160 legacy=0x7f9d9c0685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da150d700 1 -- 172.21.15.64:0/1013905675 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402577850 0 0) 0x7f9d9c0ad070 con 0x7f9d9c068160 2023-09-27T23:14:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 -- 172.21.15.64:0/1013905675 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c0aca20 con 0x7f9d9c15aa10 2023-09-27T23:14:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da2d10700 1 --2- 172.21.15.64:0/1013905675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c15ae70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009f20 tx=0x7f9d94009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cacec4dc6eaa4cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da150d700 1 -- 172.21.15.64:0/1013905675 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400e070 con 0x7f9d9c15aa10 2023-09-27T23:14:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da150d700 1 -- 172.21.15.64:0/1013905675 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9400ba60 con 0x7f9d9c15aa10 2023-09-27T23:14:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da150d700 1 -- 172.21.15.64:0/1013905675 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940092e0 con 0x7f9d9c15aa10 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 -- 172.21.15.64:0/1013905675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 msgr2=0x7f9d9c15ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 --2- 172.21.15.64:0/1013905675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c15ae70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009f20 tx=0x7f9d94009840 comp rx=0 tx=0).stop 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 -- 172.21.15.64:0/1013905675 shutdown_connections 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 --2- 172.21.15.64:0/1013905675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c15ae70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 --2- 172.21.15.64:0/1013905675 >> v2:172.21.15.123:3300/0 conn(0x7f9d9c06d280 0x7f9d9c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 -- 172.21.15.64:0/1013905675 >> 172.21.15.64:0/1013905675 conn(0x7f9d9c06c080 msgr2=0x7f9d9c0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 -- 172.21.15.64:0/1013905675 shutdown_connections 2023-09-27T23:14:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.598+0000 7f9da3511700 1 -- 172.21.15.64:0/1013905675 wait complete. 2023-09-27T23:14:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 Processor -- start 2023-09-27T23:14:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 -- start start 2023-09-27T23:14:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 0x7f9d9c0abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c06d280 0x7f9d9c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0aa320 con 0x7f9d9c068160 2023-09-27T23:14:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0a9710 con 0x7f9d9c06d280 2023-09-27T23:14:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0a99f0 con 0x7f9d9c15aa10 2023-09-27T23:14:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 0x7f9d9c0abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 0x7f9d9c0abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36682/0 (socket says 172.21.15.64:36682) 2023-09-27T23:14:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 -- 172.21.15.64:0/1503487046 learned_addr learned my addr 172.21.15.64:0/1503487046 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c06d280 0x7f9d9c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da2d10700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 -- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 msgr2=0x7f9d9c150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 -- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c06d280 msgr2=0x7f9d9c150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c06d280 0x7f9d9c150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 -- 172.21.15.64:0/1503487046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009ae0 con 0x7f9d9c068160 2023-09-27T23:14:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da250f700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 0x7f9d9c0abad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800ec80 tx=0x7f9d980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9d927fc700 1 -- 172.21.15.64:0/1503487046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800c850 con 0x7f9d9c068160 2023-09-27T23:14:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c0a9cd0 con 0x7f9d9c068160 2023-09-27T23:14:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9d927fc700 1 -- 172.21.15.64:0/1503487046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9800c9b0 con 0x7f9d9c068160 2023-09-27T23:14:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9d927fc700 1 -- 172.21.15.64:0/1503487046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800cd50 con 0x7f9d9c068160 2023-09-27T23:14:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.602+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c0ac010 con 0x7f9d9c068160 2023-09-27T23:14:09.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.606+0000 7f9d927fc700 1 -- 172.21.15.64:0/1503487046 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f9d98013680 con 0x7f9d9c068160 2023-09-27T23:14:09.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.606+0000 7f9d927fc700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d80060950 0x7f9d80062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.606+0000 7f9d927fc700 1 -- 172.21.15.64:0/1503487046 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9d9807d480 con 0x7f9d9c068160 2023-09-27T23:14:09.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.606+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d80060950 0x7f9d80062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.606+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d80060950 0x7f9d80062e00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c16d660 tx=0x7f9d8c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 Processor -- start 2023-09-27T23:14:09.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 -- start start 2023-09-27T23:14:09.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c54ae0 0x1ed1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1ed2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 0x1ed28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dfff30 con 0x1c9f790 2023-09-27T23:14:09.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dfbed0 con 0x1c54ae0 2023-09-27T23:14:09.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dc20b0 con 0x1c54710 2023-09-27T23:14:09.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 0x1ed28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da1d0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1ed2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 0x1ed28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36696/0 (socket says 172.21.15.64:36696) 2023-09-27T23:14:09.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 -- 172.21.15.64:0/3096486772 learned_addr learned my addr 172.21.15.64:0/3096486772 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:09.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da250f700 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c54ae0 0x1ed1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 -- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 msgr2=0x1ed2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1ed2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 -- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c54ae0 msgr2=0x1ed1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c54ae0 0x1ed1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 -- 172.21.15.64:0/3096486772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ca09e0 con 0x1c9f790 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9da2d10700 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 0x1ed28d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c068fd0 tx=0x7f9d94018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7938f54d3035b30 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9d7f7fe700 1 -- 172.21.15.64:0/3096486772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94007240 con 0x1c9f790 2023-09-27T23:14:09.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9d7f7fe700 1 -- 172.21.15.64:0/3096486772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9400bbd0 con 0x1c9f790 2023-09-27T23:14:09.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.610+0000 7f9d7f7fe700 1 -- 172.21.15.64:0/3096486772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94007610 con 0x1c9f790 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3096486772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 msgr2=0x1ed28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 0x1ed28d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c068fd0 tx=0x7f9d94018000 comp rx=0 tx=0).stop 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3096486772 shutdown_connections 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c9f790 0x1ed28d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1ed2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3096486772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c54ae0 0x1ed1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3096486772 >> 172.21.15.64:0/3096486772 conn(0x1c54eb0 msgr2=0x1e8e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3096486772 shutdown_connections 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3096486772 wait complete. 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- start start 2023-09-27T23:14:09.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 Processor -- start 2023-09-27T23:14:09.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1e8e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1eb8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dc1d50 con 0x1c54ae0 2023-09-27T23:14:09.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d9a970 con 0x1c9f790 2023-09-27T23:14:09.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e89dd0 con 0x1c54710 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da6a1c740 10 client.0 _ll_register_callbacks cb 0x1e45b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36712/0 (socket says 172.21.15.64:36712) 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 -- 172.21.15.64:0/3641356079 learned_addr learned my addr 172.21.15.64:0/3641356079 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da250f700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1e8e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da2d10700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1eb8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 -- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 msgr2=0x1e8e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1e8e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 -- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 msgr2=0x1eb8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1eb8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.614+0000 7f9da1d0e700 1 -- 172.21.15.64:0/3641356079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c0aca20 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb7c00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0135d0 tx=0x7f9d8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba54d10d439c0fd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 1 -- 172.21.15.64:0/3641356079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c013d70 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 1 -- 172.21.15.64:0/3641356079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9d8c01b070 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d9acc0 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 1 -- 172.21.15.64:0/3641356079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d8c021620 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 10 client.4857 fetch_fsmap learned FSMap version 8 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x1dca370 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 1 -- 172.21.15.64:0/3641356079 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d9acc0 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 1 -- 172.21.15.64:0/3641356079 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f9d8c021780 con 0x1c54ae0 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 10 client.4857 fetch_fsmap finished waiting for FSMap version 8 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 10 client.4857 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-27T23:14:09.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 20 client.4857 populate_metadata read hostname 'smithi064' 2023-09-27T23:14:09.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1ede6b0 0x1ee0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 4 client.4857 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:14:09.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 --> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1d9e3e0 con 0x1ede6b0 2023-09-27T23:14:09.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da250f700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1ede6b0 0x1ee0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 10 client.4857 ms_handle_connect on v2:172.21.15.123:6834/2280995923 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da250f700 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1ede6b0 0x1ee0a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986e1ead698a0e8d server_cookie=f61183d6978aae0b in_seq=0 out_seq=0 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 1 -- 172.21.15.64:0/3641356079 <== mds.0 v2:172.21.15.123:6834/2280995923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1d9e3e0 con 0x1ede6b0 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9d7cff9700 10 client.4857 handle_command_reply: tid=0 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.618+0000 7f9da6a1c740 1 client.4857 shutdown 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 msgr2=0x1eb7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb7c00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0135d0 tx=0x7f9d8c014000 comp rx=0 tx=0).stop 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 shutdown_connections 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1ede6b0 0x1ee0a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1eb8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb7c00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3641356079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54710 0x1e8e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9d7cff9700 0 client.4857 ms_handle_reset on v2:172.21.15.123:6834/2280995923 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 >> 172.21.15.64:0/3641356079 conn(0x1c54eb0 msgr2=0x1e86ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 shutdown_connections 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3641356079 wait complete. 2023-09-27T23:14:09.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.622+0000 7f9da6a1c740 20 client.4857 trim_cache size 0 max 16384 2023-09-27T23:14:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.638+0000 7f9da6a1c740 1 Processor -- start 2023-09-27T23:14:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.638+0000 7f9da6a1c740 1 -- start start 2023-09-27T23:14:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 0x1ee0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54ae0 0x1ee0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1ee1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1daac90 con 0x1c54710 2023-09-27T23:14:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d9cb80 con 0x1c9f790 2023-09-27T23:14:09.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e233b0 con 0x1c54ae0 2023-09-27T23:14:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 0x1ee0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 0x1ee0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36716/0 (socket says 172.21.15.64:36716) 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 -- 172.21.15.64:0/3840618214 learned_addr learned my addr 172.21.15.64:0/3840618214 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da1d0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54ae0 0x1ee0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da2d10700 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1ee1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 -- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54ae0 msgr2=0x1ee0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54ae0 0x1ee0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 -- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 msgr2=0x1ee1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1ee1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 -- 172.21.15.64:0/3840618214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c1597c0 con 0x1c54710 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da250f700 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 0x1ee0a90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d98050fc0 tx=0x7f9d9807d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9077b1f882adfb10 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9d7ffff700 1 -- 172.21.15.64:0/3840618214 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98054100 con 0x1c54710 2023-09-27T23:14:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9d7ffff700 1 -- 172.21.15.64:0/3840618214 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d980542c0 con 0x1c54710 2023-09-27T23:14:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9d7ffff700 1 -- 172.21.15.64:0/3840618214 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98058010 con 0x1c54710 2023-09-27T23:14:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3840618214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 msgr2=0x1ee0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 0x1ee0a90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d98050fc0 tx=0x7f9d9807d040 comp rx=0 tx=0).stop 2023-09-27T23:14:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3840618214 shutdown_connections 2023-09-27T23:14:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1c9f790 0x1ee1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c54ae0 0x1ee0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/3840618214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54710 0x1ee0a90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3840618214 >> 172.21.15.64:0/3840618214 conn(0x1c54eb0 msgr2=0x1e87760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3840618214 shutdown_connections 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- 172.21.15.64:0/3840618214 wait complete. 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 -- start start 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.642+0000 7f9da6a1c740 1 Processor -- start 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 0x1eb3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb3840 0x1eb3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1df9890 con 0x1c54ae0 2023-09-27T23:14:09.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ccd7c0 con 0x1eb3840 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dfcef0 con 0x1c9f790 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 10 client.0 _ll_register_callbacks cb 0x1d180e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da1d0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 0x1eb3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da1d0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 0x1eb3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:39960/0 (socket says 172.21.15.64:39960) 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36724/0 (socket says 172.21.15.64:36724) 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da2d10700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb3840 0x1eb3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da1d0e700 1 -- 172.21.15.64:0/2639982446 learned_addr learned my addr 172.21.15.64:0/2639982446 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 -- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 msgr2=0x1eb3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 0x1eb3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 -- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb3840 msgr2=0x1eb3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb3840 0x1eb3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 -- 172.21.15.64:0/2639982446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c0ac010 con 0x1c54ae0 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 0x1eb3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da250f700 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb2dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d9804d530 tx=0x7f9d98067000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc836102d8a1f623 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 1 -- 172.21.15.64:0/2639982446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98058010 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 1 -- 172.21.15.64:0/2639982446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9d9804f3d0 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d9400bbd0 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 1 -- 172.21.15.64:0/2639982446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d9807d040 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 10 client.4863 fetch_fsmap learned FSMap version 8 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x1db0d80 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 1 -- 172.21.15.64:0/2639982446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9400bbd0 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 1 -- 172.21.15.64:0/2639982446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f9d980512d0 con 0x1c54ae0 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 10 client.4863 fetch_fsmap finished waiting for FSMap version 8 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 10 client.4863 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-27T23:14:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 20 client.4863 populate_metadata read hostname 'smithi064' 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1f31110 0x1f334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 4 client.4863 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 --> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1dd0410 con 0x1f31110 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1f31110 0x1f334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9da1d0e700 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1f31110 0x1f334f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fea39d855504d89 server_cookie=1ccacb17188c1a7b in_seq=0 out_seq=0 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.646+0000 7f9d7cff9700 10 client.4863 ms_handle_connect on v2:172.21.15.123:6834/2280995923 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9d7cff9700 1 -- 172.21.15.64:0/2639982446 <== mds.0 v2:172.21.15.123:6834/2280995923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x1dd0410 con 0x1f31110 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9d7cff9700 10 client.4863 handle_command_reply: tid=0 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 client.4863 shutdown 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 msgr2=0x1eb2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb2dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d9804d530 tx=0x7f9d98067000 comp rx=0 tx=0).stop 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 shutdown_connections 2023-09-27T23:14:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:6834/2280995923,v1:172.21.15.123:6835/2280995923] conn(0x1f31110 0x1f334f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb3840 0x1eb3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9d7cff9700 0 client.4863 ms_handle_reset on v2:172.21.15.123:6834/2280995923 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1c9f790 0x1eb3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 --2- 172.21.15.64:0/2639982446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1c54ae0 0x1eb2dc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 >> 172.21.15.64:0/2639982446 conn(0x1c54eb0 msgr2=0x1e87760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 shutdown_connections 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 1 -- 172.21.15.64:0/2639982446 wait complete. 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da6a1c740 20 client.4863 trim_cache size 0 max 16384 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:14:09.657 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4299, 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 24.886838492999999, 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 8, 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 29, 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 29, 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 29.190103254 2023-09-27T23:14:09.658 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:14:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d80060950 msgr2=0x7f9d80062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d80060950 0x7f9d80062e00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c16d660 tx=0x7f9d8c007d50 comp rx=0 tx=0).stop 2023-09-27T23:14:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 msgr2=0x7f9d9c0abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:09.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 0x7f9d9c0abad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800ec80 tx=0x7f9d980091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 shutdown_connections 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d80060950 0x7f9d80062e00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9c15aa10 0x7f9d9c150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c06d280 0x7f9d9c150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 --2- 172.21.15.64:0/1503487046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d9c068160 0x7f9d9c0abad0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 >> 172.21.15.64:0/1503487046 conn(0x7f9d9c06c080 msgr2=0x7f9d9c0c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 shutdown_connections 2023-09-27T23:14:09.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:09.650+0000 7f9da3511700 1 -- 172.21.15.64:0/1503487046 wait complete. 2023-09-27T23:14:09.672 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-27T23:14:09.684 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-27T23:14:09.684 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-09-27T23:14:09.684 INFO:tasks.ceph_fuse:client.0 config is {} 2023-09-27T23:14:09.684 INFO:tasks.ceph_fuse:client.2 config is {} 2023-09-27T23:14:09.684 INFO:tasks.ceph_fuse:client.1 config is {} 2023-09-27T23:14:09.684 INFO:tasks.ceph_fuse:client.3 config is {} 2023-09-27T23:14:09.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:09.685 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:14:09.691 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-27T23:14:09.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:09.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-27T23:14:09.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:09.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-09-27T23:14:09.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:09.866 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-09-27T23:14:09.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:09.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-09-27T23:14:09.892 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-09-27T23:14:09.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:14:09.893 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-27T23:14:09.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:14:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 Processor -- start 2023-09-27T23:14:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 -- start start 2023-09-27T23:14:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800ab910 0x7f7a80159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a800670e0 0x7f7a800abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a800cd430 0x7f7a800c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a800ac220 con 0x7f7a800670e0 2023-09-27T23:14:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a800c1910 con 0x7f7a800cd430 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a84859700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80159cf0 con 0x7f7a800ab910 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a800670e0 0x7f7a800abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a800670e0 0x7f7a800abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36728/0 (socket says 172.21.15.64:36728) 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 -- 172.21.15.64:0/3126000419 learned_addr learned my addr 172.21.15.64:0/3126000419 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7f7fe700 1 --2- 172.21.15.64:0/3126000419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800ab910 0x7f7a80159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7ffff700 1 --2- 172.21.15.64:0/3126000419 >> v2:172.21.15.123:3300/0 conn(0x7f7a800cd430 0x7f7a800c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 -- 172.21.15.64:0/3126000419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800ab910 msgr2=0x7f7a80159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 --2- 172.21.15.64:0/3126000419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800ab910 0x7f7a80159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 -- 172.21.15.64:0/3126000419 >> v2:172.21.15.123:3300/0 conn(0x7f7a800cd430 msgr2=0x7f7a800c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 --2- 172.21.15.64:0/3126000419 >> v2:172.21.15.123:3300/0 conn(0x7f7a800cd430 0x7f7a800c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 -- 172.21.15.64:0/3126000419 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80159fd0 con 0x7f7a800670e0 2023-09-27T23:14:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7ffff700 1 --2- 172.21.15.64:0/3126000419 >> v2:172.21.15.123:3300/0 conn(0x7f7a800cd430 0x7f7a800c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7effd700 1 --2- 172.21.15.64:0/3126000419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800abce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a800642f0 tx=0x7f7a68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82724e27b30b1f44 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7e7fc700 1 -- 172.21.15.64:0/3126000419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68009e70 con 0x7f7a800670e0 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7e7fc700 1 -- 172.21.15.64:0/3126000419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800b860 con 0x7f7a800670e0 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.010+0000 7f7a7e7fc700 1 -- 172.21.15.64:0/3126000419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800bc00 con 0x7f7a800670e0 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- 172.21.15.64:0/3126000419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 msgr2=0x7f7a800abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- 172.21.15.64:0/3126000419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800abce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a800642f0 tx=0x7f7a68009250 comp rx=0 tx=0).stop 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- 172.21.15.64:0/3126000419 shutdown_connections 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- 172.21.15.64:0/3126000419 >> v2:172.21.15.123:3300/0 conn(0x7f7a800cd430 0x7f7a800c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- 172.21.15.64:0/3126000419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800abce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- 172.21.15.64:0/3126000419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800ab910 0x7f7a80159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- 172.21.15.64:0/3126000419 >> 172.21.15.64:0/3126000419 conn(0x7f7a8006c080 msgr2=0x7f7a800ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- 172.21.15.64:0/3126000419 shutdown_connections 2023-09-27T23:14:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- 172.21.15.64:0/3126000419 wait complete. 2023-09-27T23:14:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 Processor -- start 2023-09-27T23:14:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- start start 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a8016a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800cd430 0x7f7a800afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80166f50 con 0x7f7a800670e0 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800ae760 con 0x7f7a800ab910 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a84859700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800aea40 con 0x7f7a800cd430 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36734/0 (socket says 172.21.15.64:36734) 2023-09-27T23:14:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 -- 172.21.15.64:0/1102706886 learned_addr learned my addr 172.21.15.64:0/1102706886 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7effd700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a8016a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7ffff700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800cd430 0x7f7a800afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 -- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800cd430 msgr2=0x7f7a800afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800cd430 0x7f7a800afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 -- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 msgr2=0x7f7a8016a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a8016a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 -- 172.21.15.64:0/1102706886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680094f0 con 0x7f7a800670e0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.014+0000 7f7a7f7fe700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800c61b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009f10 tx=0x7f7a7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800cc80 con 0x7f7a800670e0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7800f040 con 0x7f7a800670e0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a800aed20 con 0x7f7a800670e0 2023-09-27T23:14:10.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78014480 con 0x7f7a800670e0 2023-09-27T23:14:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a800c66f0 con 0x7f7a800670e0 2023-09-27T23:14:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8004be40 con 0x7f7a800670e0 2023-09-27T23:14:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7a78014830 con 0x7f7a800670e0 2023-09-27T23:14:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a777fe700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060980 0x7f7a60062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7a7807e370 con 0x7f7a800670e0 2023-09-27T23:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a7effd700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060980 0x7f7a60062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.018+0000 7f7a7effd700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060980 0x7f7a60062e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a800642f0 tx=0x7f7a68017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.026+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a7804e380 con 0x7f7a800670e0 2023-09-27T23:14:10.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.226+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7a80068cc0 con 0x7f7a800670e0 2023-09-27T23:14:10.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.226+0000 7f7a777fe700 1 -- 172.21.15.64:0/1102706886 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7a780519a0 con 0x7f7a800670e0 2023-09-27T23:14:10.231 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060980 msgr2=0x7f7a60062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060980 0x7f7a60062e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a800642f0 tx=0x7f7a68017000 comp rx=0 tx=0).stop 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 msgr2=0x7f7a800c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800c61b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009f10 tx=0x7f7a7800b040 comp rx=0 tx=0).stop 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 shutdown_connections 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060980 0x7f7a60062e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800cd430 0x7f7a800afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a8016a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 --2- 172.21.15.64:0/1102706886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a800c61b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 >> 172.21.15.64:0/1102706886 conn(0x7f7a8006c080 msgr2=0x7f7a800c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 shutdown_connections 2023-09-27T23:14:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:10.230+0000 7f7a84859700 1 -- 172.21.15.64:0/1102706886 wait complete. 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:14:10.246 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:14:10.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.247 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:14:10.251 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:10.252 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42534sec preferred_lft 42534sec 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:14:10.253 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:10.254 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-27T23:14:10.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-27T23:14:10.254 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:10.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.304 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:14:10.316 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:14:10.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.318 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:14:10.324 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:14:10.324 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:14:10.324 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:14:10.324 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:14:10.324 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:14:10.324 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:14:10.325 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:14:10.325 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:14:10.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.325 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:10.325 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:10.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:14:10.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:14:10.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:14:10.325 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:10.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.363 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:14:10.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.370 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-09-27T23:14:10.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.419 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:10.419 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:10.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:14:10.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-27T23:14:10.419 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:10.448 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-27T23:14:10.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.448 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-27T23:14:10.449 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:10.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:10.505 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:10.505 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:10.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:14:10.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2023-09-27T23:14:10.505 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:10.531 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:14:10.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:10.531 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:10.538 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:14:10.538 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:10.538 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:10.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:10.606 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:10.606 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:10.621 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:10.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:10.622 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:10.626 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:14:10.626 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:14:10.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:10.690 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:10.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:10.703 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:10.704 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:10.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:10.704 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:10.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:14:10.758+0000 7f0fa3bdf380 -1 init, newargv = 0x55c2f9aadce0 newargc=15 2023-09-27T23:14:10.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[41657]: starting ceph client 2023-09-27T23:14:10.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[41657]: starting fuse 2023-09-27T23:14:11.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:11.874 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:11.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:11.891 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:11.892 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:11.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:11.892 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:11.899 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:14:11.899 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:14:11.900 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:14:11.901 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:14:12.028 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.41657.asok 2023-09-27T23:14:12.212 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41657.asok 2023-09-27T23:14:12.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:12.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41657.asok status 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:14:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "41657", 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4872, 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4872 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1051093716 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1051093716 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4872 192.168.0.1:0/1051093716", 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/1051093716", 2023-09-27T23:14:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:14:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2023-09-27T23:14:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 29, 2023-09-27T23:14:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:14:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:14:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:14:12.315 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:14:12.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:14:12.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2bad0700 1 Processor -- start 2023-09-27T23:14:12.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2bad0700 1 -- start start 2023-09-27T23:14:12.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2bad0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e24140710 0x7f0e240c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:12.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2bad0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2406e150 con 0x7f0e24068800 2023-09-27T23:14:12.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2bad0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2406d540 con 0x7f0e24140ae0 2023-09-27T23:14:12.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2bad0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2406d820 con 0x7f0e24140710 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2986c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e24140710 0x7f0e240c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2a06d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0e24068800 0x7f0e240ac250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45056/0 (socket says 172.21.15.64:45056) 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2a06d700 1 -- 172.21.15.64:0/1289616694 learned_addr learned my addr 172.21.15.64:0/1289616694 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2886a700 1 -- 172.21.15.64:0/1289616694 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609948368 0 0) 0x7f0e2406e150 con 0x7f0e24068800 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2886a700 1 -- 172.21.15.64:0/1289616694 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c003570 con 0x7f0e24068800 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2986c700 1 -- 172.21.15.64:0/1289616694 >> v1:172.21.15.123:6789/0 conn(0x7f0e24140ae0 legacy=0x7f0e240680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2986c700 1 -- 172.21.15.64:0/1289616694 >> v1:172.21.15.64:6789/0 conn(0x7f0e24068800 legacy=0x7f0e240ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2986c700 1 -- 172.21.15.64:0/1289616694 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2406db00 con 0x7f0e24140710 2023-09-27T23:14:12.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.426+0000 7f0e2986c700 1 --2- 172.21.15.64:0/1289616694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24140710 0x7f0e240c70a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009e60 tx=0x7f0e14009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d825430b8c2cfae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2886a700 1 -- 172.21.15.64:0/1289616694 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400e070 con 0x7f0e24140710 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2886a700 1 -- 172.21.15.64:0/1289616694 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400bb60 con 0x7f0e24140710 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2886a700 1 -- 172.21.15.64:0/1289616694 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14012430 con 0x7f0e24140710 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1289616694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24140710 msgr2=0x7f0e240c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1289616694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24140710 0x7f0e240c70a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009e60 tx=0x7f0e14009780 comp rx=0 tx=0).stop 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1289616694 shutdown_connections 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1289616694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24140710 0x7f0e240c70a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1289616694 >> 172.21.15.64:0/1289616694 conn(0x7f0e240c8850 msgr2=0x7f0e240c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1289616694 shutdown_connections 2023-09-27T23:14:12.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1289616694 wait complete. 2023-09-27T23:14:12.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 Processor -- start 2023-09-27T23:14:12.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- start start 2023-09-27T23:14:12.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24068800 0x7f0e241305a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:12.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 0x7f0e240ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:12.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e24140ae0 0x7f0e240ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2415a640 con 0x7f0e24140710 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2415a1b0 con 0x7f0e24068800 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.430+0000 7f0e2bad0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24158500 con 0x7f0e24140ae0 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 0x7f0e240ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 0x7f0e240ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36748/0 (socket says 172.21.15.64:36748) 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2986c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24068800 0x7f0e241305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 -- 172.21.15.64:0/1766000902 learned_addr learned my addr 172.21.15.64:0/1766000902 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2a06d700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e24140ae0 0x7f0e240ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 -- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e24140ae0 msgr2=0x7f0e240ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e24140ae0 0x7f0e240ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 -- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24068800 msgr2=0x7f0e241305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24068800 0x7f0e241305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 -- 172.21.15.64:0/1766000902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009a20 con 0x7f0e24140710 2023-09-27T23:14:12.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2906b700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 0x7f0e240ad220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c790 tx=0x7f0e18007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18011590 con 0x7f0e24140710 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e18011710 con 0x7f0e24140710 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e24159880 con 0x7f0e24140710 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18011ad0 con 0x7f0e24140710 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e24159e40 con 0x7f0e24140710 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2415a340 con 0x7f0e24140710 2023-09-27T23:14:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.434+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f0e1801b3b0 con 0x7f0e24140710 2023-09-27T23:14:12.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.438+0000 7f0e21ffb700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e10060980 0x7f0e10062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:12.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.438+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0e1807c440 con 0x7f0e24140710 2023-09-27T23:14:12.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.438+0000 7f0e2986c700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e10060980 0x7f0e10062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:12.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.438+0000 7f0e2986c700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e10060980 0x7f0e10062e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e140097c0 tx=0x7f0e14019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:12.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.442+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0e1804c3d0 con 0x7f0e24140710 2023-09-27T23:14:12.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e24071120 con 0x7f0e24140710 2023-09-27T23:14:12.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e21ffb700 1 -- 172.21.15.64:0/1766000902 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0e18017070 con 0x7f0e24140710 2023-09-27T23:14:12.646 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-27T23:14:12.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e10060980 msgr2=0x7f0e10062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e10060980 0x7f0e10062e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e140097c0 tx=0x7f0e14019000 comp rx=0 tx=0).stop 2023-09-27T23:14:12.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 msgr2=0x7f0e240ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:12.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 0x7f0e240ad220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c790 tx=0x7f0e18007f40 comp rx=0 tx=0).stop 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 shutdown_connections 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e10060980 0x7f0e10062e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e24140ae0 0x7f0e240ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e24140710 0x7f0e240ad220 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 --2- 172.21.15.64:0/1766000902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e24068800 0x7f0e241305a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 >> 172.21.15.64:0/1766000902 conn(0x7f0e240c8850 msgr2=0x7f0e240c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 shutdown_connections 2023-09-27T23:14:12.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:12.642+0000 7f0e2bad0700 1 -- 172.21.15.64:0/1766000902 wait complete. 2023-09-27T23:14:12.660 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:14:12.661 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-09-27T23:14:12.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.661 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42531sec preferred_lft 42531sec 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:14:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.670 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:14:12.723 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:14:12.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.724 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:14:12.733 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:14:12.733 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:14:12.733 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:14:12.733 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:14:12.733 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:14:12.733 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:14:12.734 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:14:12.734 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:14:12.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.734 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:12.734 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:12.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:14:12.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:14:12.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:14:12.734 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:12.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.770 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:14:12.776 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:14:12.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.776 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-09-27T23:14:12.829 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-09-27T23:14:12.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.830 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:12.830 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:12.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:14:12.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-09-27T23:14:12.831 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:12.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.856 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 42:a1:1a:f0:31:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::40a1:1aff:fef0:31f4/64 scope link 2023-09-27T23:14:12.872 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-09-27T23:14:12.933 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:12.933 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 42:a1:1a:f0:31:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::40a1:1aff:fef0:31f4/64 scope link 2023-09-27T23:14:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:12.935 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-09-27T23:14:12.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-09-27T23:14:12.936 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:12.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:12.994 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:12.994 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:12.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2023-09-27T23:14:12.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2023-09-27T23:14:12.994 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:13.021 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-09-27T23:14:13.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:13.021 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:13.071 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-09-27T23:14:13.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:13.071 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:13.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:13.093 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:13.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:13.152 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:13.153 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:13.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:13.154 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:13.161 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:14:13.161 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-09-27T23:14:13.161 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-09-27T23:14:13.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:13.179 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:13.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:13.234 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:13.236 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:13.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:13.236 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:13.241 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:14:13.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:2023-09-27T23:14:13.242+0000 7fe393628380 -1 init, newargv = 0x55578aa4d4a0 newargc=15 2023-09-27T23:14:13.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[41796]: starting ceph client 2023-09-27T23:14:13.256 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[41796]: starting fuse 2023-09-27T23:14:14.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:14.361 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:14.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:14.420 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:14.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:14.421 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:14.425 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:14:14.426 INFO:teuthology.orchestra.run.smithi064.stdout:53 2023-09-27T23:14:14.426 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-09-27T23:14:14.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:14:14.426 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:14:14.426 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:14:14.426 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:14:14.426 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:14:14.427 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.2")) 2023-09-27T23:14:14.428 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:14:14.512 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.2.41796.asok 2023-09-27T23:14:14.696 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.41796.asok 2023-09-27T23:14:14.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:14.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.41796.asok status 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "2", 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:14:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "41796", 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4887, 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4887 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1436653902 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1436653902 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4887 192.168.0.2:0/1436653902", 2023-09-27T23:14:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.2:0/1436653902", 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 8, 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 29, 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:14:14.836 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:14:14.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:14:14.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 Processor -- start 2023-09-27T23:14:14.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 -- start start 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc155960 0x7f4adc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4adc06de20 0x7f4adc0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4adc06c7c0 0x7f4adc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc06d990 con 0x7f4adc06c7c0 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc15a570 con 0x7f4adc06de20 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adbfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc06cd80 con 0x7f4adc155960 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4adc06c7c0 0x7f4adc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4adc06c7c0 0x7f4adc0c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60472/0 (socket says 172.21.15.64:60472) 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 -- 172.21.15.64:0/2671547399 learned_addr learned my addr 172.21.15.64:0/2671547399 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4ada7fc700 1 --2- 172.21.15.64:0/2671547399 >> v2:172.21.15.123:3300/0 conn(0x7f4adc06de20 0x7f4adc0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adaffd700 1 --2- 172.21.15.64:0/2671547399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc155960 0x7f4adc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 -- 172.21.15.64:0/2671547399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc155960 msgr2=0x7f4adc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 --2- 172.21.15.64:0/2671547399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc155960 0x7f4adc15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 -- 172.21.15.64:0/2671547399 >> v2:172.21.15.123:3300/0 conn(0x7f4adc06de20 msgr2=0x7f4adc0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 --2- 172.21.15.64:0/2671547399 >> v2:172.21.15.123:3300/0 conn(0x7f4adc06de20 0x7f4adc0c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 -- 172.21.15.64:0/2671547399 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc06d060 con 0x7f4adc06c7c0 2023-09-27T23:14:14.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.946+0000 7f4adb7fe700 1 --2- 172.21.15.64:0/2671547399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc06c7c0 0x7f4adc0c7d30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4adc06c5a0 tx=0x7f4ad4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc7ba7ebc3064e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4ad9ffb700 1 -- 172.21.15.64:0/2671547399 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400e070 con 0x7f4adc06c7c0 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4ad9ffb700 1 -- 172.21.15.64:0/2671547399 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400b970 con 0x7f4adc06c7c0 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 -- 172.21.15.64:0/2671547399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc06c7c0 msgr2=0x7f4adc0c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4ad9ffb700 1 -- 172.21.15.64:0/2671547399 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40092e0 con 0x7f4adc06c7c0 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 --2- 172.21.15.64:0/2671547399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc06c7c0 0x7f4adc0c7d30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4adc06c5a0 tx=0x7f4ad4009840 comp rx=0 tx=0).stop 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 -- 172.21.15.64:0/2671547399 shutdown_connections 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 --2- 172.21.15.64:0/2671547399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc06c7c0 0x7f4adc0c7d30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 --2- 172.21.15.64:0/2671547399 >> v2:172.21.15.123:3300/0 conn(0x7f4adc06de20 0x7f4adc0c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 --2- 172.21.15.64:0/2671547399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc155960 0x7f4adc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 -- 172.21.15.64:0/2671547399 >> 172.21.15.64:0/2671547399 conn(0x7f4adc067bb0 msgr2=0x7f4adc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 -- 172.21.15.64:0/2671547399 shutdown_connections 2023-09-27T23:14:14.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 -- 172.21.15.64:0/2671547399 wait complete. 2023-09-27T23:14:14.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 Processor -- start 2023-09-27T23:14:14.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.950+0000 7f4adbfff700 1 -- start start 2023-09-27T23:14:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4adc06c7c0 0x7f4adc141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc06de20 0x7f4adc14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 0x7f4adc0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc145450 con 0x7f4adc155960 2023-09-27T23:14:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc144840 con 0x7f4adc06c7c0 2023-09-27T23:14:14.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc144b20 con 0x7f4adc06de20 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 0x7f4adc0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 0x7f4adc0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60486/0 (socket says 172.21.15.64:60486) 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 -- 172.21.15.64:0/1341414428 learned_addr learned my addr 172.21.15.64:0/1341414428 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4ada7fc700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc06de20 0x7f4adc14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adaffd700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4adc06c7c0 0x7f4adc141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 -- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc06de20 msgr2=0x7f4adc14aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc06de20 0x7f4adc14aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 -- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4adc06c7c0 msgr2=0x7f4adc141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4adc06c7c0 0x7f4adc141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 -- 172.21.15.64:0/1341414428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4009ae0 con 0x7f4adc155960 2023-09-27T23:14:14.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adb7fe700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 0x7f4adc0ad200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4adc064040 tx=0x7f4ad4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:14.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4008ad0 con 0x7f4adc155960 2023-09-27T23:14:14.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4008c70 con 0x7f4adc155960 2023-09-27T23:14:14.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc144e00 con 0x7f4adc155960 2023-09-27T23:14:14.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4026460 con 0x7f4adc155960 2023-09-27T23:14:14.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc141ad0 con 0x7f4adc155960 2023-09-27T23:14:14.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.954+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc145600 con 0x7f4adc155960 2023-09-27T23:14:14.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.958+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4ad4003040 con 0x7f4adc155960 2023-09-27T23:14:14.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.958+0000 7f4ad2ffd700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4abc060950 0x7f4abc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:14.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.958+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4ad407f420 con 0x7f4adc155960 2023-09-27T23:14:14.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.958+0000 7f4adaffd700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4abc060950 0x7f4abc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:14.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.958+0000 7f4adaffd700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4abc060950 0x7f4abc062e00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009d10 tx=0x7f4acc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:14.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:14.966+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4ad404f440 con 0x7f4adc155960 2023-09-27T23:14:15.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.158+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4adc04be40 con 0x7f4adc155960 2023-09-27T23:14:15.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4ad2ffd700 1 -- 172.21.15.64:0/1341414428 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4ad4052a60 con 0x7f4adc155960 2023-09-27T23:14:15.165 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4abc060950 msgr2=0x7f4abc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4abc060950 0x7f4abc062e00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009d10 tx=0x7f4acc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 msgr2=0x7f4adc0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 0x7f4adc0ad200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4adc064040 tx=0x7f4ad4018000 comp rx=0 tx=0).stop 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 shutdown_connections 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4abc060950 0x7f4abc062e00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4adc155960 0x7f4adc0ad200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4adc06de20 0x7f4adc14aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 --2- 172.21.15.64:0/1341414428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4adc06c7c0 0x7f4adc141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.162+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 >> 172.21.15.64:0/1341414428 conn(0x7f4adc067bb0 msgr2=0x7f4adc0c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.166+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 shutdown_connections 2023-09-27T23:14:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:15.166+0000 7f4adbfff700 1 -- 172.21.15.64:0/1341414428 wait complete. 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:14:15.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:14:15.185 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-09-27T23:14:15.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.185 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:15.193 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42532sec preferred_lft 42532sec 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:15.194 INFO:teuthology.orchestra.run.smithi123.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:14:15.195 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 02:42:a2:ab:3d:0e brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:15.195 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:14:15.195 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:15.195 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-27T23:14:15.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.195 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:15.195 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:15.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2023-09-27T23:14:15.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2023-09-27T23:14:15.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2023-09-27T23:14:15.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-27T23:14:15.196 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:15.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.284 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:14:15.294 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-09-27T23:14:15.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.296 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:14:15.318 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:14:15.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.319 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:15.319 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:15.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:14:15.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:14:15.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:14:15.319 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:15.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.359 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-09-27T23:14:15.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.367 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-09-27T23:14:15.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.414 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:15.414 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:15.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-27T23:14:15.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-09-27T23:14:15.415 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:15.447 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-09-27T23:14:15.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-09-27T23:14:15.448 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:15.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:15.511 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:15.511 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:15.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-09-27T23:14:15.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2023-09-27T23:14:15.511 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:15.536 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-09-27T23:14:15.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:15.536 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:15.557 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-09-27T23:14:15.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:15.557 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:15.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-09-27T23:14:15.580 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:15.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:15.595 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:15.596 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:15.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:15.597 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-09-27T23:14:15.603 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:14:15.603 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-09-27T23:14:15.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-09-27T23:14:15.664 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:15.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:15.675 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:15.676 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:15.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:15.677 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-09-27T23:14:15.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:2023-09-27T23:14:15.728+0000 7ff64e745380 -1 init, newargv = 0x55d10c08a2f0 newargc=15 2023-09-27T23:14:15.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[39439]: starting ceph client 2023-09-27T23:14:15.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[39439]: starting fuse 2023-09-27T23:14:16.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-09-27T23:14:16.850 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:16.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:16.907 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:16.908 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:16.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:16.909 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-09-27T23:14:16.915 INFO:teuthology.orchestra.run.smithi123.stdout:52 2023-09-27T23:14:16.915 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> import re 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:16.916 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> return f 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:14:16.917 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:16.918 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.1")) 2023-09-27T23:14:16.918 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-09-27T23:14:17.002 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.1.39439.asok 2023-09-27T23:14:17.185 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39439.asok 2023-09-27T23:14:17.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:17.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39439.asok status 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "1", 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "39439", 2023-09-27T23:14:17.276 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4894, 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4894 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:14:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 130691593 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 130691593 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4894 192.168.0.1:0/130691593", 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/130691593", 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 8, 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 29, 2023-09-27T23:14:17.278 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:14:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2023-09-27T23:14:17.279 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2023-09-27T23:14:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-27T23:14:17.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:14:17.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 Processor -- start 2023-09-27T23:14:17.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- start start 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3fc0ab910 0x7fb3fc159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0670e0 0x7fb3fc0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3fc0cd430 0x7fb3fc0c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0ac220 con 0x7fb3fc0cd430 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0c1910 con 0x7fb3fc0ab910 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc159cf0 con 0x7fb3fc0670e0 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3fc0cd430 0x7fb3fc0c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3fc0cd430 0x7fb3fc0c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60498/0 (socket says 172.21.15.64:60498) 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 -- 172.21.15.64:0/2644338636 learned_addr learned my addr 172.21.15.64:0/2644338636 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3faffd700 1 --2- 172.21.15.64:0/2644338636 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0670e0 0x7fb3fc0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fb7fe700 1 --2- 172.21.15.64:0/2644338636 >> v2:172.21.15.123:3300/0 conn(0x7fb3fc0ab910 0x7fb3fc159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 -- 172.21.15.64:0/2644338636 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0670e0 msgr2=0x7fb3fc0abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 --2- 172.21.15.64:0/2644338636 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0670e0 0x7fb3fc0abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 -- 172.21.15.64:0/2644338636 >> v2:172.21.15.123:3300/0 conn(0x7fb3fc0ab910 msgr2=0x7fb3fc159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 --2- 172.21.15.64:0/2644338636 >> v2:172.21.15.123:3300/0 conn(0x7fb3fc0ab910 0x7fb3fc159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 -- 172.21.15.64:0/2644338636 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc159fd0 con 0x7fb3fc0cd430 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fbfff700 1 --2- 172.21.15.64:0/2644338636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0cd430 0x7fb3fc0c2f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc063e30 tx=0x7fb3f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7665a7943e5d178 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fa7fc700 1 -- 172.21.15.64:0/2644338636 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400e070 con 0x7fb3fc0cd430 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fa7fc700 1 -- 172.21.15.64:0/2644338636 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f400ba50 con 0x7fb3fc0cd430 2023-09-27T23:14:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb3fa7fc700 1 -- 172.21.15.64:0/2644338636 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40092e0 con 0x7fb3fc0cd430 2023-09-27T23:14:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- 172.21.15.64:0/2644338636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0cd430 msgr2=0x7fb3fc0c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- 172.21.15.64:0/2644338636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0cd430 0x7fb3fc0c2f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc063e30 tx=0x7fb3f4009840 comp rx=0 tx=0).stop 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- 172.21.15.64:0/2644338636 shutdown_connections 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- 172.21.15.64:0/2644338636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0cd430 0x7fb3fc0c2f80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- 172.21.15.64:0/2644338636 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0670e0 0x7fb3fc0abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 --2- 172.21.15.64:0/2644338636 >> v2:172.21.15.123:3300/0 conn(0x7fb3fc0ab910 0x7fb3fc159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- 172.21.15.64:0/2644338636 >> 172.21.15.64:0/2644338636 conn(0x7fb3fc06c080 msgr2=0x7fb3fc0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- 172.21.15.64:0/2644338636 shutdown_connections 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.402+0000 7fb400ce7700 1 -- 172.21.15.64:0/2644338636 wait complete. 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 Processor -- start 2023-09-27T23:14:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 -- start start 2023-09-27T23:14:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3fc0670e0 0x7fb3fc13d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 0x7fb3fc1415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0cd430 0x7fb3fc158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b0c60 con 0x7fb3fc0ab910 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc141e80 con 0x7fb3fc0670e0 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc142160 con 0x7fb3fc0cd430 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 0x7fb3fc1415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 0x7fb3fc1415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60514/0 (socket says 172.21.15.64:60514) 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 -- 172.21.15.64:0/1151790074 learned_addr learned my addr 172.21.15.64:0/1151790074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3fb7fe700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3fc0670e0 0x7fb3fc13d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3fbfff700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0cd430 0x7fb3fc158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 -- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0cd430 msgr2=0x7fb3fc158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0cd430 0x7fb3fc158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 -- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3fc0670e0 msgr2=0x7fb3fc13d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3fc0670e0 0x7fb3fc13d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 -- 172.21.15.64:0/1151790074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009ae0 con 0x7fb3fc0ab910 2023-09-27T23:14:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3faffd700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 0x7fb3fc1415c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00d860 tx=0x7fb3ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0125f0 con 0x7fb3fc0ab910 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec012770 con 0x7fb3fc0ab910 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec012b30 con 0x7fb3fc0ab910 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc142440 con 0x7fb3fc0ab910 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.406+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc149c80 con 0x7fb3fc0ab910 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.410+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc04be40 con 0x7fb3fc0ab910 2023-09-27T23:14:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.410+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb3ec013480 con 0x7fb3fc0ab910 2023-09-27T23:14:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.410+0000 7fb3e37fe700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb3e8060950 0x7fb3e8062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.410+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb3ec07d4b0 con 0x7fb3fc0ab910 2023-09-27T23:14:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.410+0000 7fb3fb7fe700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb3e8060950 0x7fb3e8062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.410+0000 7fb3fb7fe700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb3e8060950 0x7fb3e8062e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0006fd0 tx=0x7fb3f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:17.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:14:17.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:14:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.422+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb3ec04d440 con 0x7fb3fc0ab910 2023-09-27T23:14:17.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.626+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb3fc068cc0 con 0x7fb3fc0ab910 2023-09-27T23:14:17.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.626+0000 7fb3e37fe700 1 -- 172.21.15.64:0/1151790074 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb3ec050a60 con 0x7fb3fc0ab910 2023-09-27T23:14:17.631 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-27T23:14:17.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb3e8060950 msgr2=0x7fb3e8062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:17.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb3e8060950 0x7fb3e8062e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0006fd0 tx=0x7fb3f0008040 comp rx=0 tx=0).stop 2023-09-27T23:14:17.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 msgr2=0x7fb3fc1415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:17.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 0x7fb3fc1415c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00d860 tx=0x7fb3ec008040 comp rx=0 tx=0).stop 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 shutdown_connections 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb3e8060950 0x7fb3e8062e00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3fc0cd430 0x7fb3fc158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3fc0ab910 0x7fb3fc1415c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 --2- 172.21.15.64:0/1151790074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3fc0670e0 0x7fb3fc13d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 >> 172.21.15.64:0/1151790074 conn(0x7fb3fc06c080 msgr2=0x7fb3fc14c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 shutdown_connections 2023-09-27T23:14:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:17.630+0000 7fb400ce7700 1 -- 172.21.15.64:0/1151790074 wait complete. 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:14:17.645 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-09-27T23:14:17.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.646 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-09-27T23:14:17.653 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42530sec preferred_lft 42530sec 2023-09-27T23:14:17.654 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 02:42:a2:ab:3d:0e brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 7e:b7:79:e2:c6:a8 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::840d:3fff:fe8a:1c54/64 scope link 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:14:17.655 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 7e:b7:79:e2:c6:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-27T23:14:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::7cb7:79ff:fee2:c6a8/64 scope link 2023-09-27T23:14:17.656 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.656 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:14:17.714 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-09-27T23:14:17.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.716 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-09-27T23:14:17.725 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-09-27T23:14:17.725 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:14:17.725 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:14:17.725 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:14:17.725 INFO:teuthology.orchestra.run.smithi123.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:14:17.726 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:14:17.726 INFO:teuthology.orchestra.run.smithi123.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:14:17.726 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:14:17.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.726 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:17.726 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:17.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:14:17.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:14:17.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:14:17.726 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:17.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.767 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-09-27T23:14:17.774 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-27T23:14:17.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.775 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-09-27T23:14:17.822 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-09-27T23:14:17.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.822 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:17.822 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:17.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-27T23:14:17.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-09-27T23:14:17.823 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:17.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-09-27T23:14:17.867 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:17.867 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 72:89:ed:dd:2f:de brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::7089:edff:fedd:2fde/64 scope link 2023-09-27T23:14:17.868 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.869 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 72:89:ed:dd:2f:de brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-27T23:14:17.880 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-27T23:14:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::7089:edff:fedd:2fde/64 scope link 2023-09-27T23:14:17.881 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:17.881 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-09-27T23:14:17.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-09-27T23:14:17.882 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:17.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:17.975 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-09-27T23:14:17.976 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-09-27T23:14:17.976 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2023-09-27T23:14:17.976 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2023-09-27T23:14:17.976 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-09-27T23:14:18.001 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-09-27T23:14:18.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:18.001 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:18.007 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-09-27T23:14:18.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:18.007 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:18.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-09-27T23:14:18.072 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:18.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:18.087 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:18.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:18.087 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-09-27T23:14:18.092 INFO:teuthology.orchestra.run.smithi123.stdout:52 2023-09-27T23:14:18.092 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-09-27T23:14:18.092 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-09-27T23:14:18.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-09-27T23:14:18.154 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:18.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:18.162 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:18.163 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:18.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:18.163 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-09-27T23:14:18.218 INFO:teuthology.orchestra.run.smithi123.stdout:52 2023-09-27T23:14:18.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:2023-09-27T23:14:18.212+0000 7fbc9f8a4380 -1 init, newargv = 0x557afacf50f0 newargc=15 2023-09-27T23:14:18.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:ceph-fuse[39555]: starting ceph client 2023-09-27T23:14:18.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:ceph-fuse[39555]: starting fuse 2023-09-27T23:14:19.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-09-27T23:14:19.338 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:19.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:19.350 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:19.351 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:19.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.351 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-09-27T23:14:19.401 INFO:teuthology.orchestra.run.smithi123.stdout:52 2023-09-27T23:14:19.401 INFO:teuthology.orchestra.run.smithi123.stdout:53 2023-09-27T23:14:19.402 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> import re 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2023-09-27T23:14:19.402 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2023-09-27T23:14:19.403 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2023-09-27T23:14:19.404 DEBUG:teuthology.orchestra.run.smithi123:> return f 2023-09-27T23:14:19.404 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:14:19.404 DEBUG:teuthology.orchestra.run.smithi123:> 2023-09-27T23:14:19.404 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.3")) 2023-09-27T23:14:19.404 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-09-27T23:14:19.486 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.3.39555.asok 2023-09-27T23:14:19.672 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39555.asok 2023-09-27T23:14:19.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39555.asok status 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "3", 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "39555", 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:19.767 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4909, 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4909 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 655621378 2023-09-27T23:14:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 655621378 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4909 192.168.0.2:0/655621378", 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.2:0/655621378", 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 8, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 29, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2023-09-27T23:14:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2023-09-27T23:14:19.770 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-27T23:14:19.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.776 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:14:19.776 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:19.784 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:14:19.784 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:19.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:19.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.798 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:14:19.798 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:19.804 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:14:19.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:19.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:19.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.818 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2023-09-27T23:14:19.818 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:19.826 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2023-09-27T23:14:19.826 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:19.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:19.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.886 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2023-09-27T23:14:19.887 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:19.893 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2023-09-27T23:14:19.893 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:19.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:19.893 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:19.908 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-09-27T23:14:19.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:20.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaf4a3c700 1 Processor -- start 2023-09-27T23:14:20.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaf4a3c700 1 -- start start 2023-09-27T23:14:20.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaf4a3c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf015d330 0x7fdaf006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaf4a3c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf006b470 con 0x7fdaf00c6cd0 2023-09-27T23:14:20.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaf4a3c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf006e110 con 0x7fdaf00c6900 2023-09-27T23:14:20.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaf4a3c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015d7b0 con 0x7fdaf015d330 2023-09-27T23:14:20.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaeeffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdaf00c6cd0 0x7fdaf015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60068/0 (socket says 172.21.15.64:60068) 2023-09-27T23:14:20.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaeeffd700 1 -- 172.21.15.64:0/293115523 learned_addr learned my addr 172.21.15.64:0/293115523 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:20.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.038+0000 7fdaeffff700 1 --2- 172.21.15.64:0/293115523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf015d330 0x7fdaf006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaee7fc700 1 -- 172.21.15.64:0/293115523 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470865049 0 0) 0x7fdaf006b470 con 0x7fdaf00c6cd0 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaee7fc700 1 -- 172.21.15.64:0/293115523 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdadc003570 con 0x7fdaf00c6cd0 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaeffff700 1 -- 172.21.15.64:0/293115523 >> v1:172.21.15.123:6789/0 conn(0x7fdaf00c6900 legacy=0x7fdaf006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaee7fc700 1 -- 172.21.15.64:0/293115523 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333005586 0 0) 0x7fdaf006e110 con 0x7fdaf00c6900 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaeffff700 1 -- 172.21.15.64:0/293115523 >> v1:172.21.15.64:6789/0 conn(0x7fdaf00c6cd0 legacy=0x7fdaf015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaeffff700 1 -- 172.21.15.64:0/293115523 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0161d10 con 0x7fdaf015d330 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaeffff700 1 --2- 172.21.15.64:0/293115523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf015d330 0x7fdaf006b900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009e90 tx=0x7fdae8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e077d296b3699b63 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaee7fc700 1 -- 172.21.15.64:0/293115523 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800e070 con 0x7fdaf015d330 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaee7fc700 1 -- 172.21.15.64:0/293115523 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae800bc30 con 0x7fdaf015d330 2023-09-27T23:14:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaee7fc700 1 -- 172.21.15.64:0/293115523 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8012430 con 0x7fdaf015d330 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/293115523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf015d330 msgr2=0x7fdaf006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/293115523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf015d330 0x7fdaf006b900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009e90 tx=0x7fdae8009780 comp rx=0 tx=0).stop 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/293115523 shutdown_connections 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/293115523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf015d330 0x7fdaf006b900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/293115523 >> 172.21.15.64:0/293115523 conn(0x7fdaf0067bb0 msgr2=0x7fdaf00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/293115523 shutdown_connections 2023-09-27T23:14:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/293115523 wait complete. 2023-09-27T23:14:20.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.042+0000 7fdaf4a3c700 1 Processor -- start 2023-09-27T23:14:20.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- start start 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 0x7fdaf00af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf00c6cd0 0x7fdaf00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf015d330 0x7fdaf0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015f8c0 con 0x7fdaf00c6900 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015ecb0 con 0x7fdaf015d330 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015ef90 con 0x7fdaf00c6cd0 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 0x7fdaf00af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 0x7fdaf00af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60518/0 (socket says 172.21.15.64:60518) 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 -- 172.21.15.64:0/1472250429 learned_addr learned my addr 172.21.15.64:0/1472250429 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:20.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaeeffd700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf00c6cd0 0x7fdaf00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaeffff700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf015d330 0x7fdaf0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 -- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf00c6cd0 msgr2=0x7fdaf00b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf00c6cd0 0x7fdaf00b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 -- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf015d330 msgr2=0x7fdaf0153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf015d330 0x7fdaf0153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 -- 172.21.15.64:0/1472250429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009a20 con 0x7fdaf00c6900 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaeeffd700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf00c6cd0 0x7fdaf00b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaef7fe700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 0x7fdaf00af920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdae000c7e0 tx=0x7fdae0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0012700 con 0x7fdaf00c6900 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0012880 con 0x7fdaf00c6900 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf015f270 con 0x7fdaf00c6900 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0012c40 con 0x7fdaf00c6900 2023-09-27T23:14:20.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf01544a0 con 0x7fdaf00c6900 2023-09-27T23:14:20.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.046+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf015f6f0 con 0x7fdaf00c6900 2023-09-27T23:14:20.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.050+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fdae001b400 con 0x7fdaf00c6900 2023-09-27T23:14:20.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.050+0000 7fdae77fe700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad0060950 0x7fdad0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.050+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdae007cdc0 con 0x7fdaf00c6900 2023-09-27T23:14:20.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.050+0000 7fdaeeffd700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad0060950 0x7fdad0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.050+0000 7fdaeeffd700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad0060950 0x7fdad0062e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0170a70 tx=0x7fdad8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.054+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdae004ccb0 con 0x7fdaf00c6900 2023-09-27T23:14:20.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.218+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf004be40 con 0x7fdaf00c6900 2023-09-27T23:14:20.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdae77fe700 1 -- 172.21.15.64:0/1472250429 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fdae00502d0 con 0x7fdaf00c6900 2023-09-27T23:14:20.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:20.226 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad0060950 msgr2=0x7fdad0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad0060950 0x7fdad0062e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0170a70 tx=0x7fdad8009040 comp rx=0 tx=0).stop 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 msgr2=0x7fdaf00af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 0x7fdaf00af920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdae000c7e0 tx=0x7fdae0007df0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 shutdown_connections 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdad0060950 0x7fdad0062e00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf015d330 0x7fdaf0153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdaf00c6cd0 0x7fdaf00b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 --2- 172.21.15.64:0/1472250429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaf00c6900 0x7fdaf00af920 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.222+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 >> 172.21.15.64:0/1472250429 conn(0x7fdaf0067bb0 msgr2=0x7fdaf00ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.226+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 shutdown_connections 2023-09-27T23:14:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.226+0000 7fdaf4a3c700 1 -- 172.21.15.64:0/1472250429 wait complete. 2023-09-27T23:14:20.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:20.241 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.346+0000 7f016b59e700 1 Processor -- start 2023-09-27T23:14:20.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.346+0000 7f016b59e700 1 -- start start 2023-09-27T23:14:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.346+0000 7f016b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06baa0 0x7f016c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.346+0000 7f016b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f016c0adab0 0x7f016c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c15ae30 con 0x7f016c154780 2023-09-27T23:14:20.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0abf70 con 0x7f016c0adab0 2023-09-27T23:14:20.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c154b50 con 0x7f016c06baa0 2023-09-27T23:14:20.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f016c0adab0 0x7f016c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f0169d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06baa0 0x7f016c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f016c0adab0 0x7f016c0adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:59736/0 (socket says 172.21.15.64:59736) 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 -- 172.21.15.64:0/1632784788 learned_addr learned my addr 172.21.15.64:0/1632784788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016a59c700 1 --1- 172.21.15.64:0/1632784788 >> v1:172.21.15.123:6789/0 conn(0x7f016c154780 0x7f016c157b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37626/0 (socket says 172.21.15.64:37626) 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 -- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06baa0 msgr2=0x7f016c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 --2- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06baa0 0x7f016c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 -- 172.21.15.64:0/1632784788 >> v1:172.21.15.123:6789/0 conn(0x7f016c154780 legacy=0x7f016c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 -- 172.21.15.64:0/1632784788 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0ac210 con 0x7f016c0adab0 2023-09-27T23:14:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016959a700 1 -- 172.21.15.64:0/1632784788 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614684161 0 0) 0x7f016c15ae30 con 0x7f016c154780 2023-09-27T23:14:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016ad9d700 1 --2- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0adab0 0x7f016c0adf10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0164009f20 tx=0x7f0164009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f49d14f41724b794 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016959a700 1 -- 172.21.15.64:0/1632784788 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400e070 con 0x7f016c0adab0 2023-09-27T23:14:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016959a700 1 -- 172.21.15.64:0/1632784788 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016400b9d0 con 0x7f016c0adab0 2023-09-27T23:14:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016959a700 1 -- 172.21.15.64:0/1632784788 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640092e0 con 0x7f016c0adab0 2023-09-27T23:14:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0adab0 msgr2=0x7f016c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 --2- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0adab0 0x7f016c0adf10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0164009f20 tx=0x7f0164009840 comp rx=0 tx=0).stop 2023-09-27T23:14:20.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- 172.21.15.64:0/1632784788 shutdown_connections 2023-09-27T23:14:20.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 --2- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0adab0 0x7f016c0adf10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 --2- 172.21.15.64:0/1632784788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06baa0 0x7f016c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- 172.21.15.64:0/1632784788 >> 172.21.15.64:0/1632784788 conn(0x7f016c067b50 msgr2=0x7f016c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- 172.21.15.64:0/1632784788 shutdown_connections 2023-09-27T23:14:20.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.350+0000 7f016b59e700 1 -- 172.21.15.64:0/1632784788 wait complete. 2023-09-27T23:14:20.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 Processor -- start 2023-09-27T23:14:20.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 -- start start 2023-09-27T23:14:20.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 0x7f016c14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0adab0 0x7f016c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c154780 0x7f016c151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c14f8a0 con 0x7f016c06baa0 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c150700 con 0x7f016c154780 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c150b90 con 0x7f016c0adab0 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 0x7f016c14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c154780 0x7f016c151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 0x7f016c14b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60534/0 (socket says 172.21.15.64:60534) 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 -- 172.21.15.64:0/3221720422 learned_addr learned my addr 172.21.15.64:0/3221720422 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:20.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f0169d9b700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0adab0 0x7f016c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 -- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0adab0 msgr2=0x7f016c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0adab0 0x7f016c0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 -- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c154780 msgr2=0x7f016c151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c154780 0x7f016c151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 -- 172.21.15.64:0/3221720422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164009ae0 con 0x7f016c06baa0 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016a59c700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 0x7f016c14b4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c5b0 tx=0x7f015c001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c005a70 con 0x7f016c06baa0 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c005bd0 con 0x7f016c06baa0 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c14faf0 con 0x7f016c06baa0 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c0070d0 con 0x7f016c06baa0 2023-09-27T23:14:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.354+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c1500b0 con 0x7f016c06baa0 2023-09-27T23:14:20.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.358+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c15a2d0 con 0x7f016c06baa0 2023-09-27T23:14:20.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.358+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f015c007480 con 0x7f016c06baa0 2023-09-27T23:14:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.358+0000 7f01627fc700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f014c060950 0x7f014c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.358+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f015c07c5b0 con 0x7f016c06baa0 2023-09-27T23:14:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.358+0000 7f0169d9b700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f014c060950 0x7f014c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.362+0000 7f0169d9b700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f014c060950 0x7f014c062e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f016c153be0 tx=0x7f01540091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.366+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f016c15a2d0 con 0x7f016c06baa0 2023-09-27T23:14:20.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f016c15a2d0 con 0x7f016c06baa0 2023-09-27T23:14:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f01627fc700 1 -- 172.21.15.64:0/3221720422 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f015c017020 con 0x7f016c06baa0 2023-09-27T23:14:20.534 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f014c060950 msgr2=0x7f014c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f014c060950 0x7f014c062e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f016c153be0 tx=0x7f01540091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 msgr2=0x7f016c14b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 0x7f016c14b4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c5b0 tx=0x7f015c001040 comp rx=0 tx=0).stop 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 shutdown_connections 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f014c060950 0x7f014c062e00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c154780 0x7f016c151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0adab0 0x7f016c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 --2- 172.21.15.64:0/3221720422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f016c06baa0 0x7f016c14b4d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 >> 172.21.15.64:0/3221720422 conn(0x7f016c067b50 msgr2=0x7f016c06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 shutdown_connections 2023-09-27T23:14:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.530+0000 7f016b59e700 1 -- 172.21.15.64:0/3221720422 wait complete. 2023-09-27T23:14:20.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5629b62700 1 Processor -- start 2023-09-27T23:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5629b62700 1 -- start start 2023-09-27T23:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5629b62700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f562406baa0 0x7f562415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5629b62700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562415ae30 con 0x7f5624154780 2023-09-27T23:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5629b62700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56240abf70 con 0x7f56240adab0 2023-09-27T23:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5629b62700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5624154b50 con 0x7f562406baa0 2023-09-27T23:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5628b60700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5624154780 0x7f5624157b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60072/0 (socket says 172.21.15.64:60072) 2023-09-27T23:14:20.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5628b60700 1 -- 172.21.15.64:0/3050780644 learned_addr learned my addr 172.21.15.64:0/3050780644 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5623fff700 1 --2- 172.21.15.64:0/3050780644 >> v2:172.21.15.123:3300/0 conn(0x7f562406baa0 0x7f562415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f56237fe700 1 -- 172.21.15.64:0/3050780644 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537772305 0 0) 0x7f562415ae30 con 0x7f5624154780 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f56237fe700 1 -- 172.21.15.64:0/3050780644 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5610003570 con 0x7f5624154780 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5623fff700 1 -- 172.21.15.64:0/3050780644 >> v1:172.21.15.123:6789/0 conn(0x7f56240adab0 legacy=0x7f56240adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5623fff700 1 -- 172.21.15.64:0/3050780644 >> v1:172.21.15.64:6789/0 conn(0x7f5624154780 legacy=0x7f5624157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5623fff700 1 -- 172.21.15.64:0/3050780644 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240ac210 con 0x7f562406baa0 2023-09-27T23:14:20.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f5623fff700 1 --2- 172.21.15.64:0/3050780644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f562415b2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5614009e60 tx=0x7f5614009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0471a4d3719f52 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f56237fe700 1 -- 172.21.15.64:0/3050780644 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400e070 con 0x7f562406baa0 2023-09-27T23:14:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f56237fe700 1 -- 172.21.15.64:0/3050780644 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561400bb60 con 0x7f562406baa0 2023-09-27T23:14:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.654+0000 7f56237fe700 1 -- 172.21.15.64:0/3050780644 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614012430 con 0x7f562406baa0 2023-09-27T23:14:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- 172.21.15.64:0/3050780644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 msgr2=0x7f562415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 --2- 172.21.15.64:0/3050780644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f562415b2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5614009e60 tx=0x7f5614009780 comp rx=0 tx=0).stop 2023-09-27T23:14:20.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- 172.21.15.64:0/3050780644 shutdown_connections 2023-09-27T23:14:20.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 --2- 172.21.15.64:0/3050780644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f562415b2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- 172.21.15.64:0/3050780644 >> 172.21.15.64:0/3050780644 conn(0x7f5624067b50 msgr2=0x7f56240cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- 172.21.15.64:0/3050780644 shutdown_connections 2023-09-27T23:14:20.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- 172.21.15.64:0/3050780644 wait complete. 2023-09-27T23:14:20.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 Processor -- start 2023-09-27T23:14:20.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- start start 2023-09-27T23:14:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f56241505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 0x7f5624166da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5624154780 0x7f56241672e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5624156760 con 0x7f5624154780 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5624167820 con 0x7f562406baa0 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629b62700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5624155ec0 con 0x7f56240adab0 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5623fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 0x7f5624166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5629361700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5624154780 0x7f56241672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5628b60700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f56241505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5623fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 0x7f5624166da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49714/0 (socket says 172.21.15.64:49714) 2023-09-27T23:14:20.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.658+0000 7f5623fff700 1 -- 172.21.15.64:0/4128234582 learned_addr learned my addr 172.21.15.64:0/4128234582 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5623fff700 1 -- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 msgr2=0x7f56241505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5623fff700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f56241505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5623fff700 1 -- 172.21.15.64:0/4128234582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5624154780 msgr2=0x7f56241672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5623fff700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5624154780 0x7f56241672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5623fff700 1 -- 172.21.15.64:0/4128234582 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614009a20 con 0x7f56240adab0 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5623fff700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 0x7f5624166da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56140097c0 tx=0x7f5614019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614012c90 con 0x7f56240adab0 2023-09-27T23:14:20.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614012e10 con 0x7f56240adab0 2023-09-27T23:14:20.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56241561a0 con 0x7f56240adab0 2023-09-27T23:14:20.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140276c0 con 0x7f56240adab0 2023-09-27T23:14:20.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56241680e0 con 0x7f56240adab0 2023-09-27T23:14:20.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5624168300 con 0x7f56240adab0 2023-09-27T23:14:20.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f5614024070 con 0x7f56240adab0 2023-09-27T23:14:20.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5620ff9700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5604060950 0x7f5604062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.662+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f561407f650 con 0x7f56240adab0 2023-09-27T23:14:20.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.666+0000 7f5628b60700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5604060950 0x7f5604062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.666+0000 7f5628b60700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5604060950 0x7f5604062e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f561c007fd0 tx=0x7f561c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.674+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f561404f5e0 con 0x7f56240adab0 2023-09-27T23:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.842+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f562404be40 con 0x7f56240adab0 2023-09-27T23:14:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.842+0000 7f5620ff9700 1 -- 172.21.15.64:0/4128234582 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f561402c020 con 0x7f56240adab0 2023-09-27T23:14:20.847 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:20.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5604060950 msgr2=0x7f5604062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5604060950 0x7f5604062e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f561c007fd0 tx=0x7f561c009040 comp rx=0 tx=0).stop 2023-09-27T23:14:20.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 msgr2=0x7f5624166da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 0x7f5624166da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56140097c0 tx=0x7f5614019000 comp rx=0 tx=0).stop 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 shutdown_connections 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5604060950 0x7f5604062e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5624154780 0x7f56241672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56240adab0 0x7f5624166da0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 --2- 172.21.15.64:0/4128234582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f562406baa0 0x7f56241505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 >> 172.21.15.64:0/4128234582 conn(0x7f5624067b50 msgr2=0x7f5624151850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 shutdown_connections 2023-09-27T23:14:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.846+0000 7f5629b62700 1 -- 172.21.15.64:0/4128234582 wait complete. 2023-09-27T23:14:20.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.970+0000 7fb1f9f97700 1 Processor -- start 2023-09-27T23:14:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.970+0000 7fb1f9f97700 1 -- start start 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f9f97700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f415a270 0x7fb1f40684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f9f97700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f4067790 con 0x7fb1f4159790 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f9f97700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f4068030 con 0x7fb1f40b1ed0 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f9f97700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40b1200 con 0x7fb1f415a270 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb1f4159790 0x7fb1f40b17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60086/0 (socket says 172.21.15.64:60086) 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f37fe700 1 -- 172.21.15.64:0/3871071513 learned_addr learned my addr 172.21.15.64:0/3871071513 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f3fff700 1 --2- 172.21.15.64:0/3871071513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f415a270 0x7fb1f40684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784814769 0 0) 0x7fb1f4067790 con 0x7fb1f4159790 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e0003570 con 0x7fb1f4159790 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939839073 0 0) 0x7fb1f4068030 con 0x7fb1f40b1ed0 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f4067790 con 0x7fb1f40b1ed0 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f3fff700 1 -- 172.21.15.64:0/3871071513 >> v1:172.21.15.123:6789/0 conn(0x7fb1f40b1ed0 legacy=0x7fb1f4159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f3fff700 1 -- 172.21.15.64:0/3871071513 >> v1:172.21.15.64:6789/0 conn(0x7fb1f4159790 legacy=0x7fb1f40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f3fff700 1 -- 172.21.15.64:0/3871071513 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40679e0 con 0x7fb1f415a270 2023-09-27T23:14:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f3fff700 1 --2- 172.21.15.64:0/3871071513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f415a270 0x7fb1f40684c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009e90 tx=0x7fb1ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfe7d15a593f24 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00e070 con 0x7fb1f415a270 2023-09-27T23:14:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec00bb80 con 0x7fb1f415a270 2023-09-27T23:14:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.974+0000 7fb1f2ffd700 1 -- 172.21.15.64:0/3871071513 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec012430 con 0x7fb1f415a270 2023-09-27T23:14:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 -- 172.21.15.64:0/3871071513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f415a270 msgr2=0x7fb1f40684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/3871071513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f415a270 0x7fb1f40684c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009e90 tx=0x7fb1ec009780 comp rx=0 tx=0).stop 2023-09-27T23:14:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 -- 172.21.15.64:0/3871071513 shutdown_connections 2023-09-27T23:14:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/3871071513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f415a270 0x7fb1f40684c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 -- 172.21.15.64:0/3871071513 >> 172.21.15.64:0/3871071513 conn(0x7fb1f40cdef0 msgr2=0x7fb1f40ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 -- 172.21.15.64:0/3871071513 shutdown_connections 2023-09-27T23:14:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 -- 172.21.15.64:0/3871071513 wait complete. 2023-09-27T23:14:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 Processor -- start 2023-09-27T23:14:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.978+0000 7fb1f9f97700 1 -- start start 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f40b1ed0 0x7fb1f415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f4159790 0x7fb1f415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f415a270 0x7fb1f414d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f415c540 con 0x7fb1f4159790 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f415af60 con 0x7fb1f415a270 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f415b240 con 0x7fb1f40b1ed0 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f4159790 0x7fb1f415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f40b1ed0 0x7fb1f415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f40b1ed0 0x7fb1f415c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49722/0 (socket says 172.21.15.64:49722) 2023-09-27T23:14:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f37fe700 1 -- 172.21.15.64:0/2066407335 learned_addr learned my addr 172.21.15.64:0/2066407335 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f3fff700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f415a270 0x7fb1f414d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 -- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f40b1ed0 msgr2=0x7fb1f415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f40b1ed0 0x7fb1f415c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 -- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f415a270 msgr2=0x7fb1f414d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f415a270 0x7fb1f414d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 -- 172.21.15.64:0/2066407335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec009a20 con 0x7fb1f4159790 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1eaffd700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f4159790 0x7fb1f415a730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40b22d0 tx=0x7fb1dc007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc011590 con 0x7fb1f4159790 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0116f0 con 0x7fb1f4159790 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f415b520 con 0x7fb1f4159790 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc011ab0 con 0x7fb1f4159790 2023-09-27T23:14:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f415bae0 con 0x7fb1f4159790 2023-09-27T23:14:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.982+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f415bd00 con 0x7fb1f4159790 2023-09-27T23:14:20.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.986+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb1dc01c460 con 0x7fb1f4159790 2023-09-27T23:14:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.986+0000 7fb1ebfff700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1d4060950 0x7fb1d4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.986+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb1dc07cd70 con 0x7fb1f4159790 2023-09-27T23:14:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.986+0000 7fb1f37fe700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1d4060950 0x7fb1d4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:20.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.986+0000 7fb1f37fe700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1d4060950 0x7fb1d4062e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40058d0 tx=0x7fb1e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:20.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:20.990+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb1dc04cd00 con 0x7fb1f4159790 2023-09-27T23:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.150+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1f4071120 con 0x7fb1f4159790 2023-09-27T23:14:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1ebfff700 1 -- 172.21.15.64:0/2066407335 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1dc050320 con 0x7fb1f4159790 2023-09-27T23:14:21.156 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1d4060950 msgr2=0x7fb1d4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1d4060950 0x7fb1d4062e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40058d0 tx=0x7fb1e4009040 comp rx=0 tx=0).stop 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f4159790 msgr2=0x7fb1f415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f4159790 0x7fb1f415a730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40b22d0 tx=0x7fb1dc007d50 comp rx=0 tx=0).stop 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 shutdown_connections 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb1d4060950 0x7fb1d4062e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f415a270 0x7fb1f414d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f4159790 0x7fb1f415a730 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 --2- 172.21.15.64:0/2066407335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1f40b1ed0 0x7fb1f415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 >> 172.21.15.64:0/2066407335 conn(0x7fb1f40cdef0 msgr2=0x7fb1f4156560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 shutdown_connections 2023-09-27T23:14:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.154+0000 7fb1f9f97700 1 -- 172.21.15.64:0/2066407335 wait complete. 2023-09-27T23:14:21.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:21.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.286+0000 7f1d7b59e700 1 Processor -- start 2023-09-27T23:14:21.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.286+0000 7f1d7b59e700 1 -- start start 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.286+0000 7f1d7b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d7c150200 0x7f1d7c0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0af250 con 0x7f1d7c0ae770 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0ca8a0 con 0x7f1d7c0ace30 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0af530 con 0x7f1d7c150200 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d79d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d7c0ae770 0x7f1d7c0ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60090/0 (socket says 172.21.15.64:60090) 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d79d9b700 1 -- 172.21.15.64:0/1371791928 learned_addr learned my addr 172.21.15.64:0/1371791928 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:21.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7a59c700 1 --2- 172.21.15.64:0/1371791928 >> v2:172.21.15.123:3300/0 conn(0x7f1d7c150200 0x7f1d7c0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52485402 0 0) 0x7f1d7c0af250 con 0x7f1d7c0ae770 2023-09-27T23:14:21.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d64003570 con 0x7f1d7c0ae770 2023-09-27T23:14:21.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155429914 0 0) 0x7f1d7c0ca8a0 con 0x7f1d7c0ace30 2023-09-27T23:14:21.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d7c0af250 con 0x7f1d7c0ace30 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3327577661 0 0) 0x7f1d64003570 con 0x7f1d7c0ae770 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7a59c700 1 -- 172.21.15.64:0/1371791928 >> v1:172.21.15.123:6789/0 conn(0x7f1d7c0ace30 legacy=0x7f1d7c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7a59c700 1 -- 172.21.15.64:0/1371791928 >> v1:172.21.15.64:6789/0 conn(0x7f1d7c0ae770 legacy=0x7f1d7c0ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7a59c700 1 -- 172.21.15.64:0/1371791928 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0cab80 con 0x7f1d7c150200 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7a59c700 1 --2- 172.21.15.64:0/1371791928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c0c7170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d700098e0 tx=0x7f1d700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1212b14eca59a13 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c001470 con 0x7f1d7c150200 2023-09-27T23:14:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d70011420 con 0x7f1d7c150200 2023-09-27T23:14:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7959a700 1 -- 172.21.15.64:0/1371791928 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700117c0 con 0x7f1d7c150200 2023-09-27T23:14:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- 172.21.15.64:0/1371791928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 msgr2=0x7f1d7c0c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/1371791928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c0c7170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d700098e0 tx=0x7f1d700091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- 172.21.15.64:0/1371791928 shutdown_connections 2023-09-27T23:14:21.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/1371791928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c0c7170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- 172.21.15.64:0/1371791928 >> 172.21.15.64:0/1371791928 conn(0x7f1d7c0ccb30 msgr2=0x7f1d7c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:21.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- 172.21.15.64:0/1371791928 shutdown_connections 2023-09-27T23:14:21.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.290+0000 7f1d7b59e700 1 -- 172.21.15.64:0/1371791928 wait complete. 2023-09-27T23:14:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 Processor -- start 2023-09-27T23:14:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 -- start start 2023-09-27T23:14:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d7c0ace30 0x7f1d7c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 0x7f1d7c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c159de0 con 0x7f1d7c0ae770 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c15c660 con 0x7f1d7c150200 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d7b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c15b5c0 con 0x7f1d7c0ace30 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d79d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 0x7f1d7c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.294+0000 7f1d79d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 0x7f1d7c1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60562/0 (socket says 172.21.15.64:60562) 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 -- 172.21.15.64:0/3618082065 learned_addr learned my addr 172.21.15.64:0/3618082065 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d7ad9d700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d7a59c700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d7c0ace30 0x7f1d7c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 -- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d7c0ace30 msgr2=0x7f1d7c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d7c0ace30 0x7f1d7c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 -- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 msgr2=0x7f1d7c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 -- 172.21.15.64:0/3618082065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d70009420 con 0x7f1d7c0ae770 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d79d9b700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 0x7f1d7c1598a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00cb90 tx=0x7f1d6c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00cea0 con 0x7f1d7c0ae770 2023-09-27T23:14:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c007e90 con 0x7f1d7c0ae770 2023-09-27T23:14:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c15b8a0 con 0x7f1d7c0ae770 2023-09-27T23:14:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0075d0 con 0x7f1d7c0ae770 2023-09-27T23:14:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c15bee0 con 0x7f1d7c0ae770 2023-09-27T23:14:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.298+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c04be40 con 0x7f1d7c0ae770 2023-09-27T23:14:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.302+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f1d6c004a40 con 0x7f1d7c0ae770 2023-09-27T23:14:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.302+0000 7f1d6a7fc700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d5c060950 0x7f1d5c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.302+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1d6c07c770 con 0x7f1d7c0ae770 2023-09-27T23:14:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.302+0000 7f1d7a59c700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d5c060950 0x7f1d5c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.302+0000 7f1d7a59c700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d5c060950 0x7f1d5c062e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d700093f0 tx=0x7f1d70016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.306+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1d6c04c780 con 0x7f1d7c0ae770 2023-09-27T23:14:21.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.466+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d7c06cbf0 con 0x7f1d7c0ae770 2023-09-27T23:14:21.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d6a7fc700 1 -- 172.21.15.64:0/3618082065 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d6c018020 con 0x7f1d7c0ae770 2023-09-27T23:14:21.473 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:21.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d5c060950 msgr2=0x7f1d5c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d5c060950 0x7f1d5c062e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d700093f0 tx=0x7f1d70016000 comp rx=0 tx=0).stop 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 msgr2=0x7f1d7c1598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 0x7f1d7c1598a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00cb90 tx=0x7f1d6c001b50 comp rx=0 tx=0).stop 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 shutdown_connections 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d5c060950 0x7f1d5c062e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d7c150200 0x7f1d7c069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7c0ae770 0x7f1d7c1598a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 --2- 172.21.15.64:0/3618082065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d7c0ace30 0x7f1d7c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 >> 172.21.15.64:0/3618082065 conn(0x7f1d7c0ccb30 msgr2=0x7f1d7c0c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 shutdown_connections 2023-09-27T23:14:21.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.470+0000 7f1d7b59e700 1 -- 172.21.15.64:0/3618082065 wait complete. 2023-09-27T23:14:21.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:21.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.594+0000 7fa0beaee700 1 Processor -- start 2023-09-27T23:14:21.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.594+0000 7fa0beaee700 1 -- start start 2023-09-27T23:14:21.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0b806de00 0x7fa0b80c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806c7c0 0x7fa0b815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b806d970 con 0x7fa0b80c69a0 2023-09-27T23:14:21.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b806cd60 con 0x7fa0b806de00 2023-09-27T23:14:21.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b806d040 con 0x7fa0b806c7c0 2023-09-27T23:14:21.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0b806de00 0x7fa0b80c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0b806de00 0x7fa0b80c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60564/0 (socket says 172.21.15.64:60564) 2023-09-27T23:14:21.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 -- 172.21.15.64:0/3572996563 learned_addr learned my addr 172.21.15.64:0/3572996563 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:21.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0be2ed700 1 --2- 172.21.15.64:0/3572996563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806c7c0 0x7fa0b815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 -- 172.21.15.64:0/3572996563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806c7c0 msgr2=0x7fa0b815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:21.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 --2- 172.21.15.64:0/3572996563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806c7c0 0x7fa0b815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 -- 172.21.15.64:0/3572996563 >> v1:172.21.15.123:6789/0 conn(0x7fa0b80c69a0 legacy=0x7fa0b806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 -- 172.21.15.64:0/3572996563 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b806d320 con 0x7fa0b806de00 2023-09-27T23:14:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bd2eb700 1 --2- 172.21.15.64:0/3572996563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b806de00 0x7fa0b80c6d70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0099b0 tx=0x7fa0ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ae7e780f2a5bc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bcaea700 1 -- 172.21.15.64:0/3572996563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac009ea0 con 0x7fa0b806de00 2023-09-27T23:14:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bcaea700 1 -- 172.21.15.64:0/3572996563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00b860 con 0x7fa0b806de00 2023-09-27T23:14:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0bcaea700 1 -- 172.21.15.64:0/3572996563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00bc00 con 0x7fa0b806de00 2023-09-27T23:14:21.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- 172.21.15.64:0/3572996563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b806de00 msgr2=0x7fa0b80c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3572996563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b806de00 0x7fa0b80c6d70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0099b0 tx=0x7fa0ac009250 comp rx=0 tx=0).stop 2023-09-27T23:14:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- 172.21.15.64:0/3572996563 shutdown_connections 2023-09-27T23:14:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3572996563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806c7c0 0x7fa0b815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3572996563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b806de00 0x7fa0b80c6d70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- 172.21.15.64:0/3572996563 >> 172.21.15.64:0/3572996563 conn(0x7fa0b8067b50 msgr2=0x7fa0b80cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- 172.21.15.64:0/3572996563 shutdown_connections 2023-09-27T23:14:21.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.598+0000 7fa0beaee700 1 -- 172.21.15.64:0/3572996563 wait complete. 2023-09-27T23:14:21.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.602+0000 7fa0beaee700 1 Processor -- start 2023-09-27T23:14:21.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.602+0000 7fa0beaee700 1 -- start start 2023-09-27T23:14:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b806c7c0 0x7fa0b816d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 0x7fa0b815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 0x7fa0b80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80b0380 con 0x7fa0b80c69a0 2023-09-27T23:14:21.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b815caa0 con 0x7fa0b806c7c0 2023-09-27T23:14:21.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b815cd80 con 0x7fa0b806de00 2023-09-27T23:14:21.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0bd2eb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 0x7fa0b815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 0x7fa0b80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0bd2eb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 0x7fa0b815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49754/0 (socket says 172.21.15.64:49754) 2023-09-27T23:14:21.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 0x7fa0b80af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60568/0 (socket says 172.21.15.64:60568) 2023-09-27T23:14:21.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0bdaec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b806c7c0 0x7fa0b816d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0bd2eb700 1 -- 172.21.15.64:0/3503207506 learned_addr learned my addr 172.21.15.64:0/3503207506 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:21.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 -- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 msgr2=0x7fa0b815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 0x7fa0b815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 -- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b806c7c0 msgr2=0x7fa0b816d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b806c7c0 0x7fa0b816d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 -- 172.21.15.64:0/3503207506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0094f0 con 0x7fa0b80c69a0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0bd2eb700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 0x7fa0b815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0bdaec700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b806c7c0 0x7fa0b816d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0be2ed700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 0x7fa0b80af8b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009e50 tx=0x7fa0b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000caa0 con 0x7fa0b80c69a0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b000f040 con 0x7fa0b80c69a0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b815d060 con 0x7fa0b80c69a0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000ce50 con 0x7fa0b80c69a0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b80aec00 con 0x7fa0b80c69a0 2023-09-27T23:14:21.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.606+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804be40 con 0x7fa0b80c69a0 2023-09-27T23:14:21.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.610+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fa0b0014690 con 0x7fa0b80c69a0 2023-09-27T23:14:21.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.610+0000 7fa0a9ffb700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa09c060950 0x7fa09c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.610+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa0b007dcc0 con 0x7fa0b80c69a0 2023-09-27T23:14:21.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.610+0000 7fa0bdaec700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa09c060950 0x7fa09c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.610+0000 7fa0bdaec700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa09c060950 0x7fa09c062e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001d20 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.622+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa0b00516e0 con 0x7fa0b80c69a0 2023-09-27T23:14:21.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.778+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa0b80ae230 con 0x7fa0b80c69a0 2023-09-27T23:14:21.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0a9ffb700 1 -- 172.21.15.64:0/3503207506 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa0b0018020 con 0x7fa0b80c69a0 2023-09-27T23:14:21.785 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:21.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa09c060950 msgr2=0x7fa09c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa09c060950 0x7fa09c062e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001d20 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2023-09-27T23:14:21.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 msgr2=0x7fa0b80af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 0x7fa0b80af8b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009e50 tx=0x7fa0b000b040 comp rx=0 tx=0).stop 2023-09-27T23:14:21.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 shutdown_connections 2023-09-27T23:14:21.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa09c060950 0x7fa09c062e00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b80c69a0 0x7fa0b80af8b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0b806de00 0x7fa0b815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 --2- 172.21.15.64:0/3503207506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b806c7c0 0x7fa0b816d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 >> 172.21.15.64:0/3503207506 conn(0x7fa0b8067b50 msgr2=0x7fa0b815ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:21.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 shutdown_connections 2023-09-27T23:14:21.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.782+0000 7fa0beaee700 1 -- 172.21.15.64:0/3503207506 wait complete. 2023-09-27T23:14:21.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 Processor -- start 2023-09-27T23:14:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 -- start start 2023-09-27T23:14:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe084156c90 0x7fe0840cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe08415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840cd5a0 con 0x7fe0840cda30 2023-09-27T23:14:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe084157470 con 0x7fe084156c90 2023-09-27T23:14:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe08a49a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840cc990 con 0x7fe084157060 2023-09-27T23:14:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe084156c90 0x7fe0840cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe084156c90 0x7fe0840cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60584/0 (socket says 172.21.15.64:60584) 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 -- 172.21.15.64:0/576164225 learned_addr learned my addr 172.21.15.64:0/576164225 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe088a37700 1 --2- 172.21.15.64:0/576164225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe08415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 -- 172.21.15.64:0/576164225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 msgr2=0x7fe08415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 --2- 172.21.15.64:0/576164225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe08415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 -- 172.21.15.64:0/576164225 >> v1:172.21.15.123:6789/0 conn(0x7fe0840cda30 legacy=0x7fe0840c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.902+0000 7fe083fff700 1 -- 172.21.15.64:0/576164225 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840ccc70 con 0x7fe084156c90 2023-09-27T23:14:21.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe083fff700 1 --2- 172.21.15.64:0/576164225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe0840cc3b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0099b0 tx=0x7fe06c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6d10d0b123ea61 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe082ffd700 1 -- 172.21.15.64:0/576164225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c009ea0 con 0x7fe084156c90 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe082ffd700 1 -- 172.21.15.64:0/576164225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c00b860 con 0x7fe084156c90 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe082ffd700 1 -- 172.21.15.64:0/576164225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00bc00 con 0x7fe084156c90 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 -- 172.21.15.64:0/576164225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 msgr2=0x7fe0840cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 --2- 172.21.15.64:0/576164225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe0840cc3b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0099b0 tx=0x7fe06c009250 comp rx=0 tx=0).stop 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 -- 172.21.15.64:0/576164225 shutdown_connections 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 --2- 172.21.15.64:0/576164225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe08415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 --2- 172.21.15.64:0/576164225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe0840cc3b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 -- 172.21.15.64:0/576164225 >> 172.21.15.64:0/576164225 conn(0x7fe08406c7c0 msgr2=0x7fe08406cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:21.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 -- 172.21.15.64:0/576164225 shutdown_connections 2023-09-27T23:14:21.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 -- 172.21.15.64:0/576164225 wait complete. 2023-09-27T23:14:21.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 Processor -- start 2023-09-27T23:14:21.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.906+0000 7fe08a49a700 1 -- start start 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0840cda30 0x7fe0840b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe084154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe0840c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08416c820 con 0x7fe084156c90 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840b3140 con 0x7fe0840cda30 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08416ca70 con 0x7fe084157060 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe084154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe084154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60594/0 (socket says 172.21.15.64:60594) 2023-09-27T23:14:21.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 -- 172.21.15.64:0/2389658164 learned_addr learned my addr 172.21.15.64:0/2389658164 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:21.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe088a37700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe0840c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe083fff700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0840cda30 0x7fe0840b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 -- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 msgr2=0x7fe0840c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe0840c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 -- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0840cda30 msgr2=0x7fe0840b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0840cda30 0x7fe0840b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 -- 172.21.15.64:0/2389658164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c0094f0 con 0x7fe084156c90 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe0837fe700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe084154b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c790 tx=0x7fe074007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074011590 con 0x7fe084156c90 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe074011710 con 0x7fe084156c90 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08416cd50 con 0x7fe084156c90 2023-09-27T23:14:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074011ad0 con 0x7fe084156c90 2023-09-27T23:14:21.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.910+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08416d310 con 0x7fe084156c90 2023-09-27T23:14:21.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.914+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe084071120 con 0x7fe084156c90 2023-09-27T23:14:21.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.914+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe07401b3b0 con 0x7fe084156c90 2023-09-27T23:14:21.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.914+0000 7fe07bfff700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe064060950 0x7fe064062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:21.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.914+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe07407c5f0 con 0x7fe084156c90 2023-09-27T23:14:21.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.914+0000 7fe083fff700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe064060950 0x7fe064062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.914+0000 7fe083fff700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe064060950 0x7fe064062e00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0099b0 tx=0x7fe06c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:21.922+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe07404c4e0 con 0x7fe084156c90 2023-09-27T23:14:22.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.094+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0840ccf50 con 0x7fe084156c90 2023-09-27T23:14:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.094+0000 7fe07bfff700 1 -- 172.21.15.64:0/2389658164 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fe074017090 con 0x7fe084156c90 2023-09-27T23:14:22.100 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:22.100 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:22.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe064060950 msgr2=0x7fe064062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe064060950 0x7fe064062e00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0099b0 tx=0x7fe06c017000 comp rx=0 tx=0).stop 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 msgr2=0x7fe084154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe084154b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c790 tx=0x7fe074007f40 comp rx=0 tx=0).stop 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 shutdown_connections 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe064060950 0x7fe064062e00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe084157060 0x7fe0840c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084156c90 0x7fe084154b40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 --2- 172.21.15.64:0/2389658164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0840cda30 0x7fe0840b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 >> 172.21.15.64:0/2389658164 conn(0x7fe08406c7c0 msgr2=0x7fe0840af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 shutdown_connections 2023-09-27T23:14:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.098+0000 7fe08a49a700 1 -- 172.21.15.64:0/2389658164 wait complete. 2023-09-27T23:14:22.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:22.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.230+0000 7f73ebfff700 1 Processor -- start 2023-09-27T23:14:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.230+0000 7f73ebfff700 1 -- start start 2023-09-27T23:14:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73ec0c69a0 0x7f73ec0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f73ec06c7c0 0x7f73ec15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ec06d970 con 0x7f73ec06de00 2023-09-27T23:14:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ec06cd60 con 0x7f73ec0c69a0 2023-09-27T23:14:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ec06d040 con 0x7f73ec06c7c0 2023-09-27T23:14:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73ec0c69a0 0x7f73ec0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73ec0c69a0 0x7f73ec0c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60598/0 (socket says 172.21.15.64:60598) 2023-09-27T23:14:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 -- 172.21.15.64:0/2451230725 learned_addr learned my addr 172.21.15.64:0/2451230725 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eb7fe700 1 --2- 172.21.15.64:0/2451230725 >> v2:172.21.15.123:3300/0 conn(0x7f73ec06c7c0 0x7f73ec15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 -- 172.21.15.64:0/2451230725 >> v2:172.21.15.123:3300/0 conn(0x7f73ec06c7c0 msgr2=0x7f73ec15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 --2- 172.21.15.64:0/2451230725 >> v2:172.21.15.123:3300/0 conn(0x7f73ec06c7c0 0x7f73ec15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 -- 172.21.15.64:0/2451230725 >> v1:172.21.15.123:6789/0 conn(0x7f73ec06de00 legacy=0x7f73ec06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 -- 172.21.15.64:0/2451230725 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec06d320 con 0x7f73ec0c69a0 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73eaffd700 1 --2- 172.21.15.64:0/2451230725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec0c69a0 0x7f73ec0c6d70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0c8360 tx=0x7f73d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774cdbb350a61f01 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ea7fc700 1 -- 172.21.15.64:0/2451230725 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d400b860 con 0x7f73ec0c69a0 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ea7fc700 1 -- 172.21.15.64:0/2451230725 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d400b9c0 con 0x7f73ec0c69a0 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ea7fc700 1 -- 172.21.15.64:0/2451230725 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d400bd60 con 0x7f73ec0c69a0 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- 172.21.15.64:0/2451230725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec0c69a0 msgr2=0x7f73ec0c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 --2- 172.21.15.64:0/2451230725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec0c69a0 0x7f73ec0c6d70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0c8360 tx=0x7f73d4009250 comp rx=0 tx=0).stop 2023-09-27T23:14:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- 172.21.15.64:0/2451230725 shutdown_connections 2023-09-27T23:14:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 --2- 172.21.15.64:0/2451230725 >> v2:172.21.15.123:3300/0 conn(0x7f73ec06c7c0 0x7f73ec15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 --2- 172.21.15.64:0/2451230725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec0c69a0 0x7f73ec0c6d70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- 172.21.15.64:0/2451230725 >> 172.21.15.64:0/2451230725 conn(0x7f73ec067b50 msgr2=0x7f73ec0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- 172.21.15.64:0/2451230725 shutdown_connections 2023-09-27T23:14:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.234+0000 7f73ebfff700 1 -- 172.21.15.64:0/2451230725 wait complete. 2023-09-27T23:14:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 Processor -- start 2023-09-27T23:14:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 -- start start 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73ec06c7c0 0x7f73ec0c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 0x7f73ec0c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec16d690 0x7f73ec0c1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec15e270 con 0x7f73ec06de00 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec15d660 con 0x7f73ec16d690 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec15d940 con 0x7f73ec06c7c0 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 0x7f73ec0c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 0x7f73ec0c0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60602/0 (socket says 172.21.15.64:60602) 2023-09-27T23:14:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 -- 172.21.15.64:0/487004721 learned_addr learned my addr 172.21.15.64:0/487004721 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73eaffd700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73ec06c7c0 0x7f73ec0c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73eb7fe700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec16d690 0x7f73ec0c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 -- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73ec06c7c0 msgr2=0x7f73ec0c09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73ec06c7c0 0x7f73ec0c09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 -- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec16d690 msgr2=0x7f73ec0c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec16d690 0x7f73ec0c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 -- 172.21.15.64:0/487004721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d40094f0 con 0x7f73ec06de00 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e3fff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 0x7f73ec0c0f10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c4e0 tx=0x7f73dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc011590 con 0x7f73ec06de00 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec15dc20 con 0x7f73ec06de00 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc011710 con 0x7f73ec06de00 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc011ad0 con 0x7f73ec06de00 2023-09-27T23:14:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.238+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec15d330 con 0x7f73ec06de00 2023-09-27T23:14:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.242+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec15e420 con 0x7f73ec06de00 2023-09-27T23:14:22.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.242+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f73dc01b3b0 con 0x7f73ec06de00 2023-09-27T23:14:22.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.242+0000 7f73e2ffd700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73cc060950 0x7f73cc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.242+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f73dc07c570 con 0x7f73ec06de00 2023-09-27T23:14:22.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.242+0000 7f73eaffd700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73cc060950 0x7f73cc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.246+0000 7f73eaffd700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73cc060950 0x7f73cc062e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0c8360 tx=0x7f73d4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.254+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f73ec15e420 con 0x7f73ec06de00 2023-09-27T23:14:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.418+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73ec04be40 con 0x7f73ec06de00 2023-09-27T23:14:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.418+0000 7f73e2ffd700 1 -- 172.21.15.64:0/487004721 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73dc0170a0 con 0x7f73ec06de00 2023-09-27T23:14:22.422 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73cc060950 msgr2=0x7f73cc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73cc060950 0x7f73cc062e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0c8360 tx=0x7f73d4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 msgr2=0x7f73ec0c0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 0x7f73ec0c0f10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c4e0 tx=0x7f73dc00c7f0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 shutdown_connections 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73cc060950 0x7f73cc062e00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec16d690 0x7f73ec0c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec06de00 0x7f73ec0c0f10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 --2- 172.21.15.64:0/487004721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73ec06c7c0 0x7f73ec0c09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 >> 172.21.15.64:0/487004721 conn(0x7f73ec067b50 msgr2=0x7f73ec0ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 shutdown_connections 2023-09-27T23:14:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.422+0000 7f73ebfff700 1 -- 172.21.15.64:0/487004721 wait complete. 2023-09-27T23:14:22.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 Processor -- start 2023-09-27T23:14:22.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 -- start start 2023-09-27T23:14:22.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35f80cc1e0 0x7f35f80c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35f80cc5f0 0x7f35f80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f806d750 con 0x7f35f8158fd0 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f80cbaf0 con 0x7f35f80cc5f0 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35fcad6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f806cb40 con 0x7f35f80cc1e0 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f35f8158fd0 0x7f35f806dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60104/0 (socket says 172.21.15.64:60104) 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f77fe700 1 -- 172.21.15.64:0/786356976 learned_addr learned my addr 172.21.15.64:0/786356976 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f7fff700 1 --2- 172.21.15.64:0/786356976 >> v2:172.21.15.64:3300/0 conn(0x7f35f80cc5f0 0x7f35f80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f6ffd700 1 --2- 172.21.15.64:0/786356976 >> v2:172.21.15.123:3300/0 conn(0x7f35f80cc1e0 0x7f35f80c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f7fff700 1 -- 172.21.15.64:0/786356976 >> v2:172.21.15.123:3300/0 conn(0x7f35f80cc1e0 msgr2=0x7f35f80c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f7fff700 1 --2- 172.21.15.64:0/786356976 >> v2:172.21.15.123:3300/0 conn(0x7f35f80cc1e0 0x7f35f80c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f7fff700 1 -- 172.21.15.64:0/786356976 >> v1:172.21.15.64:6789/0 conn(0x7f35f8158fd0 legacy=0x7f35f806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f67fc700 1 -- 172.21.15.64:0/786356976 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547625195 0 0) 0x7f35f806d750 con 0x7f35f8158fd0 2023-09-27T23:14:22.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f7fff700 1 -- 172.21.15.64:0/786356976 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f806ce20 con 0x7f35f80cc5f0 2023-09-27T23:14:22.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f7fff700 1 --2- 172.21.15.64:0/786356976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc5f0 0x7f35f80c8960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35f806b390 tx=0x7f35f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c023a7bdccaf3b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f67fc700 1 -- 172.21.15.64:0/786356976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f000e070 con 0x7f35f80cc5f0 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f67fc700 1 -- 172.21.15.64:0/786356976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f000bb60 con 0x7f35f80cc5f0 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.542+0000 7f35f67fc700 1 -- 172.21.15.64:0/786356976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0012430 con 0x7f35f80cc5f0 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 -- 172.21.15.64:0/786356976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc5f0 msgr2=0x7f35f80c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 --2- 172.21.15.64:0/786356976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc5f0 0x7f35f80c8960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35f806b390 tx=0x7f35f0009780 comp rx=0 tx=0).stop 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 -- 172.21.15.64:0/786356976 shutdown_connections 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 --2- 172.21.15.64:0/786356976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc5f0 0x7f35f80c8960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 --2- 172.21.15.64:0/786356976 >> v2:172.21.15.123:3300/0 conn(0x7f35f80cc1e0 0x7f35f80c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 -- 172.21.15.64:0/786356976 >> 172.21.15.64:0/786356976 conn(0x7f35f80cdc00 msgr2=0x7f35f80ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 -- 172.21.15.64:0/786356976 shutdown_connections 2023-09-27T23:14:22.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 -- 172.21.15.64:0/786356976 wait complete. 2023-09-27T23:14:22.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 Processor -- start 2023-09-27T23:14:22.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.546+0000 7f35fcad6700 1 -- start start 2023-09-27T23:14:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 0x7f35f80afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35f80cc5f0 0x7f35f80caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8158fd0 0x7f35f8169b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f80c9750 con 0x7f35f80cc1e0 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f816a070 con 0x7f35f8158fd0 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f80caaf0 con 0x7f35f80cc5f0 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 0x7f35f80afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f6ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35f80cc5f0 0x7f35f80caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 0x7f35f80afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60620/0 (socket says 172.21.15.64:60620) 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8158fd0 0x7f35f8169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 -- 172.21.15.64:0/1502339554 learned_addr learned my addr 172.21.15.64:0/1502339554 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 -- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35f80cc5f0 msgr2=0x7f35f80caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35f80cc5f0 0x7f35f80caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 -- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8158fd0 msgr2=0x7f35f8169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8158fd0 0x7f35f8169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 -- 172.21.15.64:0/1502339554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0009a20 con 0x7f35f80cc1e0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35f77fe700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 0x7f35f80afa70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35e800c5b0 tx=0x7f35e8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8005b30 con 0x7f35f80cc1e0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e8005c90 con 0x7f35f80cc1e0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f80c9ee0 con 0x7f35f80cc1e0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8007300 con 0x7f35f80cc1e0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.550+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f80ca4a0 con 0x7f35f80cc1e0 2023-09-27T23:14:22.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.554+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f804be40 con 0x7f35f80cc1e0 2023-09-27T23:14:22.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.554+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f35e801b3b0 con 0x7f35f80cc1e0 2023-09-27T23:14:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.554+0000 7f35df7fe700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f35e4060950 0x7f35e4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.554+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f35e807c580 con 0x7f35f80cc1e0 2023-09-27T23:14:22.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.554+0000 7f35f6ffd700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f35e4060950 0x7f35e4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.558+0000 7f35f6ffd700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f35e4060950 0x7f35e4062e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35f80affb0 tx=0x7f35ec0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.562+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f35e804c510 con 0x7f35f80cc1e0 2023-09-27T23:14:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.722+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35f806d100 con 0x7f35f80cc1e0 2023-09-27T23:14:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.722+0000 7f35df7fe700 1 -- 172.21.15.64:0/1502339554 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35e804fb30 con 0x7f35f80cc1e0 2023-09-27T23:14:22.727 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:22.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f35e4060950 msgr2=0x7f35e4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f35e4060950 0x7f35e4062e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35f80affb0 tx=0x7f35ec0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 msgr2=0x7f35f80afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 0x7f35f80afa70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35e800c5b0 tx=0x7f35e8001070 comp rx=0 tx=0).stop 2023-09-27T23:14:22.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 shutdown_connections 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f35e4060950 0x7f35e4062e00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f8158fd0 0x7f35f8169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35f80cc5f0 0x7f35f80caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 --2- 172.21.15.64:0/1502339554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f80cc1e0 0x7f35f80afa70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 >> 172.21.15.64:0/1502339554 conn(0x7f35f80cdc00 msgr2=0x7f35f81548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 shutdown_connections 2023-09-27T23:14:22.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.726+0000 7f35fcad6700 1 -- 172.21.15.64:0/1502339554 wait complete. 2023-09-27T23:14:22.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:22.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.850+0000 7f0cb5f39700 1 Processor -- start 2023-09-27T23:14:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.850+0000 7f0cb5f39700 1 -- start start 2023-09-27T23:14:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0cb0142680 0x7f0cb00c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb00c8f40 0x7f0cb006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb00cc7e0 con 0x7f0cb0142a50 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00cbbf0 con 0x7f0cb0142680 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00cbeb0 con 0x7f0cb00c8f40 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5738700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0cb0142a50 0x7f0cb00ca7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60120/0 (socket says 172.21.15.64:60120) 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5738700 1 -- 172.21.15.64:0/2043773876 learned_addr learned my addr 172.21.15.64:0/2043773876 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caffff700 1 --2- 172.21.15.64:0/2043773876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb00c8f40 0x7f0cb006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb4f37700 1 --2- 172.21.15.64:0/2043773876 >> v2:172.21.15.123:3300/0 conn(0x7f0cb0142680 0x7f0cb00c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caf7fe700 1 -- 172.21.15.64:0/2043773876 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687318556 0 0) 0x7f0cb00cc7e0 con 0x7f0cb0142a50 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caf7fe700 1 -- 172.21.15.64:0/2043773876 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ca0003570 con 0x7f0cb0142a50 2023-09-27T23:14:22.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caffff700 1 -- 172.21.15.64:0/2043773876 >> v2:172.21.15.123:3300/0 conn(0x7f0cb0142680 msgr2=0x7f0cb00c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caffff700 1 --2- 172.21.15.64:0/2043773876 >> v2:172.21.15.123:3300/0 conn(0x7f0cb0142680 0x7f0cb00c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caffff700 1 -- 172.21.15.64:0/2043773876 >> v1:172.21.15.64:6789/0 conn(0x7f0cb0142a50 legacy=0x7f0cb00ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caffff700 1 -- 172.21.15.64:0/2043773876 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00cc190 con 0x7f0cb00c8f40 2023-09-27T23:14:22.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caffff700 1 --2- 172.21.15.64:0/2043773876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb00c8f40 0x7f0cb006d050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0098e0 tx=0x7f0c9c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f20831482b0d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caf7fe700 1 -- 172.21.15.64:0/2043773876 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c011630 con 0x7f0cb00c8f40 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caf7fe700 1 -- 172.21.15.64:0/2043773876 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c011790 con 0x7f0cb00c8f40 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0caf7fe700 1 -- 172.21.15.64:0/2043773876 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c011b30 con 0x7f0cb00c8f40 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- 172.21.15.64:0/2043773876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb00c8f40 msgr2=0x7f0cb006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/2043773876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb00c8f40 0x7f0cb006d050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0098e0 tx=0x7f0c9c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- 172.21.15.64:0/2043773876 shutdown_connections 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/2043773876 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb00c8f40 0x7f0cb006d050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/2043773876 >> v2:172.21.15.123:3300/0 conn(0x7f0cb0142680 0x7f0cb00c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- 172.21.15.64:0/2043773876 >> 172.21.15.64:0/2043773876 conn(0x7f0cb00ccb30 msgr2=0x7f0cb00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- 172.21.15.64:0/2043773876 shutdown_connections 2023-09-27T23:14:22.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.854+0000 7f0cb5f39700 1 -- 172.21.15.64:0/2043773876 wait complete. 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 Processor -- start 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 -- start start 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cb00c8f40 0x7f0cb00ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb0142680 0x7f0cb00672f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb0142a50 0x7f0cb0067830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00a96c0 con 0x7f0cb00c8f40 2023-09-27T23:14:22.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb016dc90 con 0x7f0cb0142680 2023-09-27T23:14:22.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00b05e0 con 0x7f0cb0142a50 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb4f37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cb00c8f40 0x7f0cb00ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb4f37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cb00c8f40 0x7f0cb00ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60622/0 (socket says 172.21.15.64:60622) 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb4f37700 1 -- 172.21.15.64:0/3200389250 learned_addr learned my addr 172.21.15.64:0/3200389250 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5738700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb0142a50 0x7f0cb0067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb0142680 0x7f0cb00672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 -- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb0142a50 msgr2=0x7f0cb0067830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb0142a50 0x7f0cb0067830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 -- 172.21.15.64:0/3200389250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cb00c8f40 msgr2=0x7f0cb00ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cb00c8f40 0x7f0cb00ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 -- 172.21.15.64:0/3200389250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c009420 con 0x7f0cb0142680 2023-09-27T23:14:22.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0caffff700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb0142680 0x7f0cb00672f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00ad410 tx=0x7f0c9c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c011900 con 0x7f0cb0142680 2023-09-27T23:14:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb00ab710 con 0x7f0cb0142680 2023-09-27T23:14:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb00aff90 con 0x7f0cb0142680 2023-09-27T23:14:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c0233f0 con 0x7f0cb0142680 2023-09-27T23:14:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.858+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c023560 con 0x7f0cb0142680 2023-09-27T23:14:22.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.862+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb004be40 con 0x7f0cb0142680 2023-09-27T23:14:22.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.862+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f0c9c023950 con 0x7f0cb0142680 2023-09-27T23:14:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.862+0000 7f0cacff9700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c94060950 0x7f0c94062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.862+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0c9c07e470 con 0x7f0cb0142680 2023-09-27T23:14:22.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.862+0000 7f0cb4f37700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c94060950 0x7f0c94062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:22.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.862+0000 7f0cb4f37700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c94060950 0x7f0c94062e00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009e70 tx=0x7f0ca40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:22.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:22.870+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0c9c04e410 con 0x7f0cb0142680 2023-09-27T23:14:23.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.034+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0cb00b0720 con 0x7f0cb0142680 2023-09-27T23:14:23.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.034+0000 7f0cacff9700 1 -- 172.21.15.64:0/3200389250 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c9c051a30 con 0x7f0cb0142680 2023-09-27T23:14:23.039 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c94060950 msgr2=0x7f0c94062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c94060950 0x7f0c94062e00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009e70 tx=0x7f0ca40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb0142680 msgr2=0x7f0cb00672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb0142680 0x7f0cb00672f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00ad410 tx=0x7f0c9c017000 comp rx=0 tx=0).stop 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 shutdown_connections 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c94060950 0x7f0c94062e00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0cb0142a50 0x7f0cb0067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb0142680 0x7f0cb00672f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 --2- 172.21.15.64:0/3200389250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cb00c8f40 0x7f0cb00ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 >> 172.21.15.64:0/3200389250 conn(0x7f0cb00ccb30 msgr2=0x7f0cb00c1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 shutdown_connections 2023-09-27T23:14:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.038+0000 7f0cb5f39700 1 -- 172.21.15.64:0/3200389250 wait complete. 2023-09-27T23:14:23.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 Processor -- start 2023-09-27T23:14:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 -- start start 2023-09-27T23:14:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd38155520 0x7efd380c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd380ae6c0 0x7efd3806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380ad020 con 0x7efd380c7670 2023-09-27T23:14:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380c81e0 con 0x7efd380ae6c0 2023-09-27T23:14:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e901700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380aca10 con 0x7efd38155520 2023-09-27T23:14:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d0fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd380ae6c0 0x7efd3806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e100700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efd380c7670 0x7efd38159100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60126/0 (socket says 172.21.15.64:60126) 2023-09-27T23:14:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3e100700 1 -- 172.21.15.64:0/1085280255 learned_addr learned my addr 172.21.15.64:0/1085280255 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d8ff700 1 --2- 172.21.15.64:0/1085280255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd38155520 0x7efd380c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d0fe700 1 -- 172.21.15.64:0/1085280255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd38155520 msgr2=0x7efd380c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d0fe700 1 --2- 172.21.15.64:0/1085280255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd38155520 0x7efd380c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d0fe700 1 -- 172.21.15.64:0/1085280255 >> v1:172.21.15.64:6789/0 conn(0x7efd380c7670 legacy=0x7efd38159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3c8fd700 1 -- 172.21.15.64:0/1085280255 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030198390 0 0) 0x7efd380ad020 con 0x7efd380c7670 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d0fe700 1 -- 172.21.15.64:0/1085280255 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380c7c20 con 0x7efd380ae6c0 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3d0fe700 1 --2- 172.21.15.64:0/1085280255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd3806ae20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d9d0 tx=0x7efd280080e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa796ebd0ac88cb1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.166+0000 7efd3c8fd700 1 -- 172.21.15.64:0/1085280255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800abc0 con 0x7efd380ae6c0 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3c8fd700 1 -- 172.21.15.64:0/1085280255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2800ad20 con 0x7efd380ae6c0 2023-09-27T23:14:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3c8fd700 1 -- 172.21.15.64:0/1085280255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28015640 con 0x7efd380ae6c0 2023-09-27T23:14:23.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 -- 172.21.15.64:0/1085280255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 msgr2=0x7efd3806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 --2- 172.21.15.64:0/1085280255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd3806ae20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d9d0 tx=0x7efd280080e0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 -- 172.21.15.64:0/1085280255 shutdown_connections 2023-09-27T23:14:23.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 --2- 172.21.15.64:0/1085280255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd3806ae20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 --2- 172.21.15.64:0/1085280255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd38155520 0x7efd380c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 -- 172.21.15.64:0/1085280255 >> 172.21.15.64:0/1085280255 conn(0x7efd380c8cd0 msgr2=0x7efd380c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 -- 172.21.15.64:0/1085280255 shutdown_connections 2023-09-27T23:14:23.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 -- 172.21.15.64:0/1085280255 wait complete. 2023-09-27T23:14:23.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 Processor -- start 2023-09-27T23:14:23.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.170+0000 7efd3e901700 1 -- start start 2023-09-27T23:14:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd381504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd380c7670 0x7efd380ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd38155520 0x7efd380cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380a8860 con 0x7efd380ae6c0 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38168f40 con 0x7efd38155520 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380a9870 con 0x7efd380c7670 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd381504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd381504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60636/0 (socket says 172.21.15.64:60636) 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 -- 172.21.15.64:0/515336325 learned_addr learned my addr 172.21.15.64:0/515336325 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d0fe700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd380c7670 0x7efd380ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e100700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd38155520 0x7efd380cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 -- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd380c7670 msgr2=0x7efd380ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd380c7670 0x7efd380ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 -- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd38155520 msgr2=0x7efd380cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd38155520 0x7efd380cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 -- 172.21.15.64:0/515336325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2800d510 con 0x7efd380ae6c0 2023-09-27T23:14:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3d8ff700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd381504d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd3400e930 tx=0x7efd340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34013680 con 0x7efd380ae6c0 2023-09-27T23:14:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340137e0 con 0x7efd380ae6c0 2023-09-27T23:14:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380a8ab0 con 0x7efd380ae6c0 2023-09-27T23:14:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34013b80 con 0x7efd380ae6c0 2023-09-27T23:14:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380a9070 con 0x7efd380ae6c0 2023-09-27T23:14:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.174+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3804be40 con 0x7efd380ae6c0 2023-09-27T23:14:23.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.178+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7efd3400c3e0 con 0x7efd380ae6c0 2023-09-27T23:14:23.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.178+0000 7efd2dffb700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efd1c060950 0x7efd1c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.178+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7efd3407d2a0 con 0x7efd380ae6c0 2023-09-27T23:14:23.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.178+0000 7efd3d0fe700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efd1c060950 0x7efd1c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.178+0000 7efd3d0fe700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efd1c060950 0x7efd1c062e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd380aea90 tx=0x7efd28019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.186+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efd3404d190 con 0x7efd380ae6c0 2023-09-27T23:14:23.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.346+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd380c7f00 con 0x7efd380ae6c0 2023-09-27T23:14:23.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd2dffb700 1 -- 172.21.15.64:0/515336325 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd340507b0 con 0x7efd380ae6c0 2023-09-27T23:14:23.353 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:23.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efd1c060950 msgr2=0x7efd1c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efd1c060950 0x7efd1c062e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd380aea90 tx=0x7efd28019000 comp rx=0 tx=0).stop 2023-09-27T23:14:23.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 msgr2=0x7efd381504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd381504d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd3400e930 tx=0x7efd340091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 shutdown_connections 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efd1c060950 0x7efd1c062e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd38155520 0x7efd380cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd380c7670 0x7efd380ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 --2- 172.21.15.64:0/515336325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd380ae6c0 0x7efd381504d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 >> 172.21.15.64:0/515336325 conn(0x7efd380c8cd0 msgr2=0x7efd3806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 shutdown_connections 2023-09-27T23:14:23.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.350+0000 7efd3e901700 1 -- 172.21.15.64:0/515336325 wait complete. 2023-09-27T23:14:23.373 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:23.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.478+0000 7f8bb5b9f700 1 Processor -- start 2023-09-27T23:14:23.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.478+0000 7f8bb5b9f700 1 -- start start 2023-09-27T23:14:23.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb00ab6d0 0x7f8bb00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00ac390 con 0x7f8bb00b1440 2023-09-27T23:14:23.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00ac670 con 0x7f8bb006ae20 2023-09-27T23:14:23.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00ac950 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb539e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8bb00b1440 0x7f8bb00b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60140/0 (socket says 172.21.15.64:60140) 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb539e700 1 -- 172.21.15.64:0/2393371555 learned_addr learned my addr 172.21.15.64:0/2393371555 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baffff700 1 --2- 172.21.15.64:0/2393371555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb00ab6d0 0x7f8bb00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baf7fe700 1 -- 172.21.15.64:0/2393371555 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106778985 0 0) 0x7f8bb00ac390 con 0x7f8bb00b1440 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baf7fe700 1 -- 172.21.15.64:0/2393371555 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b98003570 con 0x7f8bb00b1440 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baffff700 1 -- 172.21.15.64:0/2393371555 >> v1:172.21.15.123:6789/0 conn(0x7f8bb006ae20 legacy=0x7f8bb006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baffff700 1 -- 172.21.15.64:0/2393371555 >> v1:172.21.15.64:6789/0 conn(0x7f8bb00b1440 legacy=0x7f8bb00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baffff700 1 -- 172.21.15.64:0/2393371555 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00acc30 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baffff700 1 --2- 172.21.15.64:0/2393371555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb00ab6d0 0x7f8bb00abb20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009910 tx=0x7f8ba00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c107b1aebf1a28 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baf7fe700 1 -- 172.21.15.64:0/2393371555 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000b680 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baf7fe700 1 -- 172.21.15.64:0/2393371555 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba000b7e0 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8baf7fe700 1 -- 172.21.15.64:0/2393371555 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000bba0 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/2393371555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb00ab6d0 msgr2=0x7f8bb00abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/2393371555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb00ab6d0 0x7f8bb00abb20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009910 tx=0x7f8ba00091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/2393371555 shutdown_connections 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/2393371555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb00ab6d0 0x7f8bb00abb20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/2393371555 >> 172.21.15.64:0/2393371555 conn(0x7f8bb00c8cd0 msgr2=0x7f8bb00c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/2393371555 shutdown_connections 2023-09-27T23:14:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.482+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/2393371555 wait complete. 2023-09-27T23:14:23.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 Processor -- start 2023-09-27T23:14:23.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 -- start start 2023-09-27T23:14:23.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb006ae20 0x7f8bb01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb00ab6d0 0x7f8bb00c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb00b1440 0x7f8bb00ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00c2930 con 0x7f8bb00b1440 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00c1d20 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00c2000 con 0x7f8bb006ae20 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb539e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb00b1440 0x7f8bb00ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb539e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb00b1440 0x7f8bb00ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60640/0 (socket says 172.21.15.64:60640) 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb539e700 1 -- 172.21.15.64:0/793985228 learned_addr learned my addr 172.21.15.64:0/793985228 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb4b9d700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb006ae20 0x7f8bb01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb00ab6d0 0x7f8bb00c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 -- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb006ae20 msgr2=0x7f8bb01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb006ae20 0x7f8bb01545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 -- 172.21.15.64:0/793985228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb00b1440 msgr2=0x7f8bb00ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb00b1440 0x7f8bb00ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 -- 172.21.15.64:0/793985228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0009420 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8baffff700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb00ab6d0 0x7f8bb00c13b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0016000 tx=0x7f8ba0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000bb60 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0000c90 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb00c22e0 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba001a660 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.486+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb00c2dc0 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.490+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb004be40 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.490+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8ba0044a50 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.490+0000 7f8bacff9700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8b9c060950 0x7f8b9c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.490+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8ba007ee90 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.490+0000 7f8bb4b9d700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8b9c060950 0x7f8b9c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.494+0000 7f8bb4b9d700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8b9c060950 0x7f8b9c062e00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40039e0 tx=0x7f8ba4007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.498+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8ba004eea0 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.662+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8bb00c2140 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bacff9700 1 -- 172.21.15.64:0/793985228 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8bb00c2140 con 0x7f8bb00ab6d0 2023-09-27T23:14:23.669 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8b9c060950 msgr2=0x7f8b9c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8b9c060950 0x7f8b9c062e00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40039e0 tx=0x7f8ba4007ec0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb00ab6d0 msgr2=0x7f8bb00c13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb00ab6d0 0x7f8bb00c13b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0016000 tx=0x7f8ba0017000 comp rx=0 tx=0).stop 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 shutdown_connections 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8b9c060950 0x7f8b9c062e00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bb00b1440 0x7f8bb00ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb00ab6d0 0x7f8bb00c13b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 --2- 172.21.15.64:0/793985228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bb006ae20 0x7f8bb01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 >> 172.21.15.64:0/793985228 conn(0x7f8bb00c8cd0 msgr2=0x7f8bb00b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 shutdown_connections 2023-09-27T23:14:23.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.666+0000 7f8bb5b9f700 1 -- 172.21.15.64:0/793985228 wait complete. 2023-09-27T23:14:23.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:23.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 Processor -- start 2023-09-27T23:14:23.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 -- start start 2023-09-27T23:14:23.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf000c6900 0x7fbf000c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbf0006b900 0x7fbf0006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0006b470 con 0x7fbf0015cc20 2023-09-27T23:14:23.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0015d7b0 con 0x7fbf0006b900 2023-09-27T23:14:23.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf07253700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00161d10 con 0x7fbf000c6900 2023-09-27T23:14:23.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf06a52700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbf0015cc20 0x7fbf0015d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60152/0 (socket says 172.21.15.64:60152) 2023-09-27T23:14:23.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf06a52700 1 -- 172.21.15.64:0/4150814662 learned_addr learned my addr 172.21.15.64:0/4150814662 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:23.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf05a50700 1 --2- 172.21.15.64:0/4150814662 >> v2:172.21.15.123:3300/0 conn(0x7fbf0006b900 0x7fbf0006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf06251700 1 --2- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf000c6900 0x7fbf000c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf0524f700 1 -- 172.21.15.64:0/4150814662 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010717514 0 0) 0x7fbf0006b470 con 0x7fbf0015cc20 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf0524f700 1 -- 172.21.15.64:0/4150814662 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeec003570 con 0x7fbf0015cc20 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf05a50700 1 -- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf000c6900 msgr2=0x7fbf000c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf05a50700 1 --2- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf000c6900 0x7fbf000c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf05a50700 1 -- 172.21.15.64:0/4150814662 >> v1:172.21.15.64:6789/0 conn(0x7fbf0015cc20 legacy=0x7fbf0015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf05a50700 1 -- 172.21.15.64:0/4150814662 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0006bd70 con 0x7fbf0006b900 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.790+0000 7fbf05a50700 1 --2- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0006da00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009910 tx=0x7fbef40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d717423f16290b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf0524f700 1 -- 172.21.15.64:0/4150814662 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4011630 con 0x7fbf0006b900 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf0524f700 1 -- 172.21.15.64:0/4150814662 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4011790 con 0x7fbf0006b900 2023-09-27T23:14:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf0524f700 1 -- 172.21.15.64:0/4150814662 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4011b10 con 0x7fbf0006b900 2023-09-27T23:14:23.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 -- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 msgr2=0x7fbf0006da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 --2- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0006da00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009910 tx=0x7fbef40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 -- 172.21.15.64:0/4150814662 shutdown_connections 2023-09-27T23:14:23.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 --2- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0006da00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 --2- 172.21.15.64:0/4150814662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf000c6900 0x7fbf000c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 -- 172.21.15.64:0/4150814662 >> 172.21.15.64:0/4150814662 conn(0x7fbf00067bb0 msgr2=0x7fbf000cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 -- 172.21.15.64:0/4150814662 shutdown_connections 2023-09-27T23:14:23.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.794+0000 7fbf07253700 1 -- 172.21.15.64:0/4150814662 wait complete. 2023-09-27T23:14:23.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 Processor -- start 2023-09-27T23:14:23.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 -- start start 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 0x7fbf000b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf0015cc20 0x7fbf00160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000af840 con 0x7fbf000c6900 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000afb20 con 0x7fbf0006b900 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf07253700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000afe00 con 0x7fbf0015cc20 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 0x7fbf000b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 0x7fbf000b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60650/0 (socket says 172.21.15.64:60650) 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 -- 172.21.15.64:0/1686981503 learned_addr learned my addr 172.21.15.64:0/1686981503 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf06251700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf06a52700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf0015cc20 0x7fbf00160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 -- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf0015cc20 msgr2=0x7fbf00160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf0015cc20 0x7fbf00160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 -- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 msgr2=0x7fbf0014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0014ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 -- 172.21.15.64:0/1686981503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4009420 con 0x7fbf000c6900 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.798+0000 7fbf05a50700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 0x7fbf000b1260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf00064040 tx=0x7fbef4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4011630 con 0x7fbf000c6900 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4011900 con 0x7fbf000c6900 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf000b00e0 con 0x7fbf000c6900 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef401a660 con 0x7fbf000c6900 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0014e300 con 0x7fbf000c6900 2023-09-27T23:14:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0004be40 con 0x7fbf000c6900 2023-09-27T23:14:23.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbef401aa90 con 0x7fbf000c6900 2023-09-27T23:14:23.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbef27fc700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbee4060950 0x7fbee4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:23.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbef407ed60 con 0x7fbf000c6900 2023-09-27T23:14:23.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.802+0000 7fbf06251700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbee4060950 0x7fbee4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:23.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.806+0000 7fbf06251700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbee4060950 0x7fbee4062e00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009db0 tx=0x7fbefc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:23.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.810+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbef404ec50 con 0x7fbf000c6900 2023-09-27T23:14:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.970+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf0006a580 con 0x7fbf000c6900 2023-09-27T23:14:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.970+0000 7fbef27fc700 1 -- 172.21.15.64:0/1686981503 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fbef4015020 con 0x7fbf000c6900 2023-09-27T23:14:23.977 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:23.977 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbee4060950 msgr2=0x7fbee4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbee4060950 0x7fbee4062e00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009db0 tx=0x7fbefc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 msgr2=0x7fbf000b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 0x7fbf000b1260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf00064040 tx=0x7fbef4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 shutdown_connections 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbee4060950 0x7fbee4062e00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf0015cc20 0x7fbf00160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf000c6900 0x7fbf000b1260 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 --2- 172.21.15.64:0/1686981503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf0006b900 0x7fbf0014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 >> 172.21.15.64:0/1686981503 conn(0x7fbf00067bb0 msgr2=0x7fbf000ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 shutdown_connections 2023-09-27T23:14:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:23.974+0000 7fbf07253700 1 -- 172.21.15.64:0/1686981503 wait complete. 2023-09-27T23:14:23.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:23.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.098+0000 7fbd76daa700 1 Processor -- start 2023-09-27T23:14:24.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- start start 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd700ac8f0 0x7fbd701557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd70156780 con 0x7fbd700acf30 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd70158f90 con 0x7fbd700ad300 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd701532a0 con 0x7fbd700ac8f0 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd765a9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd700ac8f0 0x7fbd701557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd75da8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbd700acf30 0x7fbd70159420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60156/0 (socket says 172.21.15.64:60156) 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd765a9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd700ac8f0 0x7fbd701557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60662/0 (socket says 172.21.15.64:60662) 2023-09-27T23:14:24.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd75da8700 1 -- 172.21.15.64:0/1376507302 learned_addr learned my addr 172.21.15.64:0/1376507302 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216413855 0 0) 0x7fbd70156780 con 0x7fbd700acf30 2023-09-27T23:14:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5c003570 con 0x7fbd700acf30 2023-09-27T23:14:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068323566 0 0) 0x7fbd70158f90 con 0x7fbd700ad300 2023-09-27T23:14:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd70156780 con 0x7fbd700ad300 2023-09-27T23:14:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd765a9700 1 -- 172.21.15.64:0/1376507302 >> v1:172.21.15.123:6789/0 conn(0x7fbd700ad300 legacy=0x7fbd700ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd765a9700 1 -- 172.21.15.64:0/1376507302 >> v1:172.21.15.64:6789/0 conn(0x7fbd700acf30 legacy=0x7fbd70159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201882278 0 0) 0x7fbd5c003570 con 0x7fbd700acf30 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd765a9700 1 -- 172.21.15.64:0/1376507302 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70155ee0 con 0x7fbd700ac8f0 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd765a9700 1 --2- 172.21.15.64:0/1376507302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd701557d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009e60 tx=0x7fbd68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b0053dbf9202fd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800e070 con 0x7fbd700ac8f0 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6800b7a0 con 0x7fbd700ac8f0 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd74da6700 1 -- 172.21.15.64:0/1376507302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c001470 con 0x7fbd700ac8f0 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- 172.21.15.64:0/1376507302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 msgr2=0x7fbd701557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 --2- 172.21.15.64:0/1376507302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd701557d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009e60 tx=0x7fbd68009780 comp rx=0 tx=0).stop 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- 172.21.15.64:0/1376507302 shutdown_connections 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 --2- 172.21.15.64:0/1376507302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd701557d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- 172.21.15.64:0/1376507302 >> 172.21.15.64:0/1376507302 conn(0x7fbd70067b50 msgr2=0x7fbd700cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- 172.21.15.64:0/1376507302 shutdown_connections 2023-09-27T23:14:24.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.102+0000 7fbd76daa700 1 -- 172.21.15.64:0/1376507302 wait complete. 2023-09-27T23:14:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 Processor -- start 2023-09-27T23:14:24.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 -- start start 2023-09-27T23:14:24.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd700b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd700acf30 0x7fbd700c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd700ad300 0x7fbd701487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700ca3f0 con 0x7fbd700ac8f0 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70148d00 con 0x7fbd700ad300 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd76daa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700b0790 con 0x7fbd700acf30 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd75da8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd700b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd75da8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd700b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60678/0 (socket says 172.21.15.64:60678) 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd75da8700 1 -- 172.21.15.64:0/3979397569 learned_addr learned my addr 172.21.15.64:0/3979397569 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd755a7700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd700acf30 0x7fbd700c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.106+0000 7fbd765a9700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd700ad300 0x7fbd701487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd75da8700 1 -- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd700acf30 msgr2=0x7fbd700c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd75da8700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd700acf30 0x7fbd700c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd75da8700 1 -- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd700ad300 msgr2=0x7fbd701487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd75da8700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd700ad300 0x7fbd701487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd75da8700 1 -- 172.21.15.64:0/3979397569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68009a20 con 0x7fbd700ac8f0 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd765a9700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd700ad300 0x7fbd701487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd755a7700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd700acf30 0x7fbd700c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd75da8700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd700b00d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cb60 tx=0x7fbd6c001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c007e90 con 0x7fbd700ac8f0 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c007360 con 0x7fbd700ac8f0 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd700b0a70 con 0x7fbd700ac8f0 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c004830 con 0x7fbd700ac8f0 2023-09-27T23:14:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd700b1030 con 0x7fbd700ac8f0 2023-09-27T23:14:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7004be40 con 0x7fbd700ac8f0 2023-09-27T23:14:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.110+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbd6c004ba0 con 0x7fbd700ac8f0 2023-09-27T23:14:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.114+0000 7fbd61ffb700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbd54060950 0x7fbd54062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.114+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbd6c07c920 con 0x7fbd700ac8f0 2023-09-27T23:14:24.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.114+0000 7fbd755a7700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbd54060950 0x7fbd54062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.114+0000 7fbd755a7700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbd54060950 0x7fbd54062e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd70152bd0 tx=0x7fbd64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.122+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbd6c04c930 con 0x7fbd700ac8f0 2023-09-27T23:14:24.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd700ae340 con 0x7fbd700ac8f0 2023-09-27T23:14:24.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd61ffb700 1 -- 172.21.15.64:0/3979397569 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd6c0180a0 con 0x7fbd700ac8f0 2023-09-27T23:14:24.294 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:24.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbd54060950 msgr2=0x7fbd54062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbd54060950 0x7fbd54062e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd70152bd0 tx=0x7fbd64009040 comp rx=0 tx=0).stop 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 msgr2=0x7fbd700b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd700b00d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cb60 tx=0x7fbd6c001ad0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 shutdown_connections 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbd54060950 0x7fbd54062e00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd700ad300 0x7fbd701487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd700acf30 0x7fbd700c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 --2- 172.21.15.64:0/3979397569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd700ac8f0 0x7fbd700b00d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.290+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 >> 172.21.15.64:0/3979397569 conn(0x7fbd70067b50 msgr2=0x7fbd700ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.294+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 shutdown_connections 2023-09-27T23:14:24.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.294+0000 7fbd76daa700 1 -- 172.21.15.64:0/3979397569 wait complete. 2023-09-27T23:14:24.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:24.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.422+0000 7feecfc48700 1 Processor -- start 2023-09-27T23:14:24.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.422+0000 7feecfc48700 1 -- start start 2023-09-27T23:14:24.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feec80ab870 0x7feec80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec80b08d0 0x7feec8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feec8159fb0 0x7feec8156470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec815a3f0 con 0x7feec8159fb0 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec80b0e40 con 0x7feec80ab870 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec80b1120 con 0x7feec80b08d0 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feec8159fb0 0x7feec8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecd1e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec80b08d0 0x7feec8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feec8159fb0 0x7feec8156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60690/0 (socket says 172.21.15.64:60690) 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 -- 172.21.15.64:0/3379049336 learned_addr learned my addr 172.21.15.64:0/3379049336 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:24.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecd9e4700 1 --2- 172.21.15.64:0/3379049336 >> v2:172.21.15.123:3300/0 conn(0x7feec80ab870 0x7feec80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 -- 172.21.15.64:0/3379049336 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec80b08d0 msgr2=0x7feec8159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 --2- 172.21.15.64:0/3379049336 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec80b08d0 0x7feec8159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 -- 172.21.15.64:0/3379049336 >> v2:172.21.15.123:3300/0 conn(0x7feec80ab870 msgr2=0x7feec80afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 --2- 172.21.15.64:0/3379049336 >> v2:172.21.15.123:3300/0 conn(0x7feec80ab870 0x7feec80afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 -- 172.21.15.64:0/3379049336 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80b1400 con 0x7feec8159fb0 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feece1e5700 1 --2- 172.21.15.64:0/3379049336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec8159fb0 0x7feec8156470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feec8063780 tx=0x7feec0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793523dd3adf75b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecc9e2700 1 -- 172.21.15.64:0/3379049336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000e070 con 0x7feec8159fb0 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecc9e2700 1 -- 172.21.15.64:0/3379049336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec000ba20 con 0x7feec8159fb0 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecc9e2700 1 -- 172.21.15.64:0/3379049336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00092e0 con 0x7feec8159fb0 2023-09-27T23:14:24.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- 172.21.15.64:0/3379049336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec8159fb0 msgr2=0x7feec8156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- 172.21.15.64:0/3379049336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec8159fb0 0x7feec8156470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feec8063780 tx=0x7feec0009840 comp rx=0 tx=0).stop 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- 172.21.15.64:0/3379049336 shutdown_connections 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- 172.21.15.64:0/3379049336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec8159fb0 0x7feec8156470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- 172.21.15.64:0/3379049336 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec80b08d0 0x7feec8159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 --2- 172.21.15.64:0/3379049336 >> v2:172.21.15.123:3300/0 conn(0x7feec80ab870 0x7feec80afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- 172.21.15.64:0/3379049336 >> 172.21.15.64:0/3379049336 conn(0x7feec80cc580 msgr2=0x7feec80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- 172.21.15.64:0/3379049336 shutdown_connections 2023-09-27T23:14:24.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.426+0000 7feecfc48700 1 -- 172.21.15.64:0/3379049336 wait complete. 2023-09-27T23:14:24.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 Processor -- start 2023-09-27T23:14:24.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 -- start start 2023-09-27T23:14:24.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec80ab870 0x7feec80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 0x7feec813e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec8159fb0 0x7feec80ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec813e760 con 0x7feec80b08d0 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec80ae560 con 0x7feec80ab870 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecfc48700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec80ad950 con 0x7feec8159fb0 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 0x7feec813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 0x7feec813e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60696/0 (socket says 172.21.15.64:60696) 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 -- 172.21.15.64:0/325360823 learned_addr learned my addr 172.21.15.64:0/325360823 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd9e4700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec80ab870 0x7feec80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feece1e5700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec8159fb0 0x7feec80ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 -- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec8159fb0 msgr2=0x7feec80ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec8159fb0 0x7feec80ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 -- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec80ab870 msgr2=0x7feec80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec80ab870 0x7feec80cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.430+0000 7feecd1e3700 1 -- 172.21.15.64:0/325360823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0009ae0 con 0x7feec80b08d0 2023-09-27T23:14:24.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feecd1e3700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 0x7feec813e220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d860 tx=0x7feeb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb80125f0 con 0x7feec80b08d0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8012750 con 0x7feec80b08d0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec80adc30 con 0x7feec80b08d0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8012b10 con 0x7feec80b08d0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec80ae1f0 con 0x7feec80b08d0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec80ae6f0 con 0x7feec80b08d0 2023-09-27T23:14:24.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7feeb801e460 con 0x7feec80b08d0 2023-09-27T23:14:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feebdffb700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feeac060950 0x7feeac062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7feeb807d760 con 0x7feec80b08d0 2023-09-27T23:14:24.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.434+0000 7feecd9e4700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feeac060950 0x7feeac062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.438+0000 7feecd9e4700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feeac060950 0x7feeac062e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d10 tx=0x7feec40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.442+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7feeb80510e0 con 0x7feec80b08d0 2023-09-27T23:14:24.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.602+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feec8071120 con 0x7feec80b08d0 2023-09-27T23:14:24.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feebdffb700 1 -- 172.21.15.64:0/325360823 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7feeb8050c70 con 0x7feec80b08d0 2023-09-27T23:14:24.609 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feeac060950 msgr2=0x7feeac062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feeac060950 0x7feeac062e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d10 tx=0x7feec40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 msgr2=0x7feec813e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 0x7feec813e220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d860 tx=0x7feeb8008040 comp rx=0 tx=0).stop 2023-09-27T23:14:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 shutdown_connections 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feeac060950 0x7feeac062e00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feec8159fb0 0x7feec80ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec80b08d0 0x7feec813e220 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 --2- 172.21.15.64:0/325360823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec80ab870 0x7feec80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 >> 172.21.15.64:0/325360823 conn(0x7feec80cc580 msgr2=0x7feec8154900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 shutdown_connections 2023-09-27T23:14:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.606+0000 7feecfc48700 1 -- 172.21.15.64:0/325360823 wait complete. 2023-09-27T23:14:24.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:24.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff5499b700 1 Processor -- start 2023-09-27T23:14:24.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff5499b700 1 -- start start 2023-09-27T23:14:24.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff5499b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff50158e20 0x7eff500ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff5499b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5006d5e0 con 0x7eff500cb970 2023-09-27T23:14:24.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff5499b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff500cbdf0 con 0x7eff500ca2c0 2023-09-27T23:14:24.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff5499b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff500cc0d0 con 0x7eff50158e20 2023-09-27T23:14:24.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff4dd9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7eff500cb970 0x7eff5006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40002/0 (socket says 172.21.15.64:40002) 2023-09-27T23:14:24.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff4dd9b700 1 -- 172.21.15.64:0/579730480 learned_addr learned my addr 172.21.15.64:0/579730480 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:24.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff4e59c700 1 --2- 172.21.15.64:0/579730480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff50158e20 0x7eff500ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923685775 0 0) 0x7eff5006d5e0 con 0x7eff500cb970 2023-09-27T23:14:24.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff38003570 con 0x7eff500cb970 2023-09-27T23:14:24.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.730+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989261724 0 0) 0x7eff500cbdf0 con 0x7eff500ca2c0 2023-09-27T23:14:24.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff5006d5e0 con 0x7eff500ca2c0 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215816730 0 0) 0x7eff38003570 con 0x7eff500cb970 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4e59c700 1 -- 172.21.15.64:0/579730480 >> v1:172.21.15.123:6789/0 conn(0x7eff500ca2c0 legacy=0x7eff5006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4e59c700 1 -- 172.21.15.64:0/579730480 >> v1:172.21.15.64:6789/0 conn(0x7eff500cb970 legacy=0x7eff5006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4e59c700 1 -- 172.21.15.64:0/579730480 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff500cc3b0 con 0x7eff50158e20 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4e59c700 1 --2- 172.21.15.64:0/579730480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff50158e20 0x7eff500ce5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7eff44009910 tx=0x7eff440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8988754b1579a14c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40001470 con 0x7eff50158e20 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff44011420 con 0x7eff50158e20 2023-09-27T23:14:24.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff4d59a700 1 -- 172.21.15.64:0/579730480 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff440117a0 con 0x7eff50158e20 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 -- 172.21.15.64:0/579730480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff50158e20 msgr2=0x7eff500ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 --2- 172.21.15.64:0/579730480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff50158e20 0x7eff500ce5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7eff44009910 tx=0x7eff440091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 -- 172.21.15.64:0/579730480 shutdown_connections 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 --2- 172.21.15.64:0/579730480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff50158e20 0x7eff500ce5c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 -- 172.21.15.64:0/579730480 >> 172.21.15.64:0/579730480 conn(0x7eff50067bb0 msgr2=0x7eff500d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 -- 172.21.15.64:0/579730480 shutdown_connections 2023-09-27T23:14:24.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 -- 172.21.15.64:0/579730480 wait complete. 2023-09-27T23:14:24.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 Processor -- start 2023-09-27T23:14:24.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.734+0000 7eff5499b700 1 -- start start 2023-09-27T23:14:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 0x7eff50151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50158e20 0x7eff500b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff501550c0 0x7eff500c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff500b60e0 con 0x7eff500ca2c0 2023-09-27T23:14:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff50152d10 con 0x7eff50158e20 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff500b5840 con 0x7eff501550c0 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 0x7eff50151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 0x7eff50151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33240/0 (socket says 172.21.15.64:33240) 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 -- 172.21.15.64:0/2799915131 learned_addr learned my addr 172.21.15.64:0/2799915131 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4dd9b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50158e20 0x7eff500b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4ed9d700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff501550c0 0x7eff500c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 -- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff501550c0 msgr2=0x7eff500c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff501550c0 0x7eff500c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 -- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50158e20 msgr2=0x7eff500b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50158e20 0x7eff500b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 -- 172.21.15.64:0/2799915131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44009420 con 0x7eff500ca2c0 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4dd9b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50158e20 0x7eff500b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff4e59c700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 0x7eff50151450 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff44009f00 tx=0x7eff44016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4401cbd0 con 0x7eff500ca2c0 2023-09-27T23:14:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4401cd30 con 0x7eff500ca2c0 2023-09-27T23:14:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4401cea0 con 0x7eff500ca2c0 2023-09-27T23:14:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff500b5b20 con 0x7eff500ca2c0 2023-09-27T23:14:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff50152fb0 con 0x7eff500ca2c0 2023-09-27T23:14:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7eff4401a910 con 0x7eff500ca2c0 2023-09-27T23:14:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.738+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff501531a0 con 0x7eff500ca2c0 2023-09-27T23:14:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.742+0000 7eff3e7fc700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff30060950 0x7eff30062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.742+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7eff4407ea70 con 0x7eff500ca2c0 2023-09-27T23:14:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.742+0000 7eff4dd9b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff30060950 0x7eff30062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.742+0000 7eff4dd9b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff30060950 0x7eff30062e00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff40001ed0 tx=0x7eff40000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:24.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.750+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff4404e960 con 0x7eff500ca2c0 2023-09-27T23:14:24.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff50071120 con 0x7eff500ca2c0 2023-09-27T23:14:24.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff3e7fc700 1 -- 172.21.15.64:0/2799915131 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff44051f80 con 0x7eff500ca2c0 2023-09-27T23:14:24.913 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff30060950 msgr2=0x7eff30062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff30060950 0x7eff30062e00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff40001ed0 tx=0x7eff40000bc0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 msgr2=0x7eff50151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 0x7eff50151450 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff44009f00 tx=0x7eff44016000 comp rx=0 tx=0).stop 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 shutdown_connections 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff30060950 0x7eff30062e00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff501550c0 0x7eff500c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50158e20 0x7eff500b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 --2- 172.21.15.64:0/2799915131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff500ca2c0 0x7eff50151450 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:24.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 >> 172.21.15.64:0/2799915131 conn(0x7eff50067bb0 msgr2=0x7eff500c4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:24.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 shutdown_connections 2023-09-27T23:14:24.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:24.910+0000 7eff5499b700 1 -- 172.21.15.64:0/2799915131 wait complete. 2023-09-27T23:14:24.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:25.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.038+0000 7fe6f359e700 1 Processor -- start 2023-09-27T23:14:25.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.038+0000 7fe6f359e700 1 -- start start 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe6ec0dc660 0x7fe6ec0e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec0e36a0 con 0x7fe6ec057820 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec053920 con 0x7fe6ec058b80 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ec038be0 con 0x7fe6ec0dc660 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f2d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe6ec057820 0x7fe6ec006430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40012/0 (socket says 172.21.15.64:40012) 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f2d9d700 1 -- 172.21.15.64:0/3263008798 learned_addr learned my addr 172.21.15.64:0/3263008798 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 --2- 172.21.15.64:0/3263008798 >> v2:172.21.15.123:3300/0 conn(0x7fe6ec0dc660 0x7fe6ec0e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404451416 0 0) 0x7fe6f406f9a0 con 0x7fe6ec057820 2023-09-27T23:14:25.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e0003570 con 0x7fe6ec057820 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034376763 0 0) 0x7fe6ec053920 con 0x7fe6ec058b80 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f406f9a0 con 0x7fe6ec058b80 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286395903 0 0) 0x7fe6e0003570 con 0x7fe6ec057820 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 >> v2:172.21.15.123:3300/0 conn(0x7fe6ec0dc660 msgr2=0x7fe6ec0e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 --2- 172.21.15.64:0/3263008798 >> v2:172.21.15.123:3300/0 conn(0x7fe6ec0dc660 0x7fe6ec0e3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 >> v1:172.21.15.123:6789/0 conn(0x7fe6ec058b80 legacy=0x7fe6ec057110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec053c00 con 0x7fe6ec057820 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fe6f404b600 con 0x7fe6ec057820 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 >> v1:172.21.15.64:6789/0 conn(0x7fe6ec057820 legacy=0x7fe6ec006430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6e0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6e0004810 0x7fe6e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e00051c0 0x7fe6e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec038be0 con 0x7fe6e0003bf0 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000bf40 con 0x7fe6e00051c0 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000c100 con 0x7fe6e0004810 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6e0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f2d9d700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e00051c0 0x7fe6e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f1d9b700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6e0004810 0x7fe6e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 -- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6e0004810 msgr2=0x7fe6e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:25.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6e0004810 0x7fe6e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 -- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e00051c0 msgr2=0x7fe6e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e00051c0 0x7fe6e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f2d9d700 1 -- 172.21.15.64:0/3263008798 reap_dead start 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 -- 172.21.15.64:0/3263008798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0003570 con 0x7fe6e0003bf0 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f259c700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6e0003fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006f00 tx=0x7fe6e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cafa3ae0b89b626 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400e710 con 0x7fe6e0003bf0 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400e870 con 0x7fe6e0003bf0 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f159a700 1 -- 172.21.15.64:0/3263008798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ec30 con 0x7fe6e0003bf0 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- 172.21.15.64:0/3263008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 msgr2=0x7fe6e0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6e0003fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006f00 tx=0x7fe6e40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- 172.21.15.64:0/3263008798 shutdown_connections 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 --2- 172.21.15.64:0/3263008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6e0003fe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- 172.21.15.64:0/3263008798 >> 172.21.15.64:0/3263008798 conn(0x7fe6ec00e7c0 msgr2=0x7fe6ec00ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- 172.21.15.64:0/3263008798 shutdown_connections 2023-09-27T23:14:25.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.042+0000 7fe6f359e700 1 -- 172.21.15.64:0/3263008798 wait complete. 2023-09-27T23:14:25.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 Processor -- start 2023-09-27T23:14:25.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 -- start start 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6ec03ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec03a6d0 0x7fe6ec0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6ec03bf50 0x7fe6ec03c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec050800 con 0x7fe6e0003bf0 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec03b710 con 0x7fe6ec03a6d0 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec03ae70 con 0x7fe6ec03bf50 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.046+0000 7fe6f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6ec03ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6ec03ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33246/0 (socket says 172.21.15.64:33246) 2023-09-27T23:14:25.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 -- 172.21.15.64:0/2347576552 learned_addr learned my addr 172.21.15.64:0/2347576552 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f2d9d700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6ec03bf50 0x7fe6ec03c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f1d9b700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec03a6d0 0x7fe6ec0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 -- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6ec03bf50 msgr2=0x7fe6ec03c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6ec03bf50 0x7fe6ec03c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 -- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec03a6d0 msgr2=0x7fe6ec0c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec03a6d0 0x7fe6ec0c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 -- 172.21.15.64:0/2347576552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec03b150 con 0x7fe6e0003bf0 2023-09-27T23:14:25.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f1d9b700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec03a6d0 0x7fe6ec0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f259c700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6ec03ea10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e4b0 tx=0x7fe6e401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4022070 con 0x7fe6e0003bf0 2023-09-27T23:14:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400e9e0 con 0x7fe6e0003bf0 2023-09-27T23:14:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec03b430 con 0x7fe6e0003bf0 2023-09-27T23:14:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400e710 con 0x7fe6e0003bf0 2023-09-27T23:14:25.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec03ab40 con 0x7fe6e0003bf0 2023-09-27T23:14:25.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec0054b0 con 0x7fe6e0003bf0 2023-09-27T23:14:25.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.050+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe6e401c450 con 0x7fe6e0003bf0 2023-09-27T23:14:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.054+0000 7fe6ea7fc700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe6d4060980 0x7fe6d4062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.054+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe6e407ed40 con 0x7fe6e0003bf0 2023-09-27T23:14:25.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.054+0000 7fe6f1d9b700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe6d4060980 0x7fe6d4062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.054+0000 7fe6f1d9b700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe6d4060980 0x7fe6d4062e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0e2920 tx=0x7fe6dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.062+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe6e404ecd0 con 0x7fe6e0003bf0 2023-09-27T23:14:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.218+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6ec056ef0 con 0x7fe6e0003bf0 2023-09-27T23:14:25.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.218+0000 7fe6ea7fc700 1 -- 172.21.15.64:0/2347576552 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6e40522f0 con 0x7fe6e0003bf0 2023-09-27T23:14:25.223 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe6d4060980 msgr2=0x7fe6d4062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe6d4060980 0x7fe6d4062e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0e2920 tx=0x7fe6dc009040 comp rx=0 tx=0).stop 2023-09-27T23:14:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 msgr2=0x7fe6ec03ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6ec03ea10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e4b0 tx=0x7fe6e401a000 comp rx=0 tx=0).stop 2023-09-27T23:14:25.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 shutdown_connections 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe6d4060980 0x7fe6d4062e30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6ec03bf50 0x7fe6ec03c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec03a6d0 0x7fe6ec0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 --2- 172.21.15.64:0/2347576552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6e0003bf0 0x7fe6ec03ea10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 >> 172.21.15.64:0/2347576552 conn(0x7fe6ec00e7c0 msgr2=0x7fe6ec00fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 shutdown_connections 2023-09-27T23:14:25.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.222+0000 7fe6f359e700 1 -- 172.21.15.64:0/2347576552 wait complete. 2023-09-27T23:14:25.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:25.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343f59e700 1 Processor -- start 2023-09-27T23:14:25.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343f59e700 1 -- start start 2023-09-27T23:14:25.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34400ce2d0 0x7f34400ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34400cbbb0 con 0x7f34400cc040 2023-09-27T23:14:25.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344006dc90 con 0x7f34400caa10 2023-09-27T23:14:25.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34400cafa0 con 0x7f34400ce2d0 2023-09-27T23:14:25.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f34400cc040 0x7f344006a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40028/0 (socket says 172.21.15.64:40028) 2023-09-27T23:14:25.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343ed9d700 1 -- 172.21.15.64:0/1830737744 learned_addr learned my addr 172.21.15.64:0/1830737744 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:25.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343e59c700 1 --2- 172.21.15.64:0/1830737744 >> v2:172.21.15.123:3300/0 conn(0x7f34400ce2d0 0x7f34400ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.358+0000 7f343d59a700 1 -- 172.21.15.64:0/1830737744 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224865088 0 0) 0x7f34400cbbb0 con 0x7f34400cc040 2023-09-27T23:14:25.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343d59a700 1 -- 172.21.15.64:0/1830737744 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3428003570 con 0x7f34400cc040 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343e59c700 1 -- 172.21.15.64:0/1830737744 >> v1:172.21.15.123:6789/0 conn(0x7f34400caa10 legacy=0x7f3440158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343e59c700 1 -- 172.21.15.64:0/1830737744 >> v1:172.21.15.64:6789/0 conn(0x7f34400cc040 legacy=0x7f344006a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343d59a700 1 -- 172.21.15.64:0/1830737744 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758995310 0 0) 0x7f3428003570 con 0x7f34400cc040 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343e59c700 1 -- 172.21.15.64:0/1830737744 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400cb280 con 0x7f34400ce2d0 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343e59c700 1 --2- 172.21.15.64:0/1830737744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400ce6a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34340098e0 tx=0x7f34340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af891310e263f729 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343d59a700 1 -- 172.21.15.64:0/1830737744 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34380017f0 con 0x7f34400ce2d0 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343d59a700 1 -- 172.21.15.64:0/1830737744 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434011420 con 0x7f34400ce2d0 2023-09-27T23:14:25.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343d59a700 1 -- 172.21.15.64:0/1830737744 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340117c0 con 0x7f34400ce2d0 2023-09-27T23:14:25.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 -- 172.21.15.64:0/1830737744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 msgr2=0x7f34400ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 --2- 172.21.15.64:0/1830737744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400ce6a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34340098e0 tx=0x7f34340091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 -- 172.21.15.64:0/1830737744 shutdown_connections 2023-09-27T23:14:25.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 --2- 172.21.15.64:0/1830737744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400ce6a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 -- 172.21.15.64:0/1830737744 >> 172.21.15.64:0/1830737744 conn(0x7f3440067bb0 msgr2=0x7f34400cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 -- 172.21.15.64:0/1830737744 shutdown_connections 2023-09-27T23:14:25.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 -- 172.21.15.64:0/1830737744 wait complete. 2023-09-27T23:14:25.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 Processor -- start 2023-09-27T23:14:25.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.362+0000 7f343f59e700 1 -- start start 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34400caa10 0x7f34400c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 0x7f344014c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34400c9230 con 0x7f34400cc040 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34400c9c00 con 0x7f34400ce2d0 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34400c88b0 con 0x7f34400caa10 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 0x7f344014c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 0x7f344014c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33256/0 (socket says 172.21.15.64:33256) 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 -- 172.21.15.64:0/3479977945 learned_addr learned my addr 172.21.15.64:0/3479977945 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:25.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343e59c700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34400caa10 0x7f34400c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343ed9d700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 -- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34400caa10 msgr2=0x7f34400c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34400caa10 0x7f34400c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 -- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 msgr2=0x7f34400c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 -- 172.21.15.64:0/3479977945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009420 con 0x7f34400cc040 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343dd9b700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 0x7f344014c840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f343000c590 tx=0x7f343000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430011590 con 0x7f34400cc040 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3430011710 con 0x7f34400cc040 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34400c7980 con 0x7f34400cc040 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430011ad0 con 0x7f34400cc040 2023-09-27T23:14:25.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34400c8be0 con 0x7f34400cc040 2023-09-27T23:14:25.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.366+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344004be40 con 0x7f34400cc040 2023-09-27T23:14:25.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.370+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f343001b3b0 con 0x7f34400cc040 2023-09-27T23:14:25.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.370+0000 7f342e7fc700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3420060950 0x7f3420062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.370+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f343007c640 con 0x7f34400cc040 2023-09-27T23:14:25.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.370+0000 7f343e59c700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3420060950 0x7f3420062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.370+0000 7f343e59c700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3420060950 0x7f3420062e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34340091e0 tx=0x7f3434016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.378+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f343004c5d0 con 0x7f34400cc040 2023-09-27T23:14:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.530+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f34400681f0 con 0x7f34400cc040 2023-09-27T23:14:25.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.530+0000 7f342e7fc700 1 -- 172.21.15.64:0/3479977945 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f34300170a0 con 0x7f34400cc040 2023-09-27T23:14:25.536 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3420060950 msgr2=0x7f3420062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3420060950 0x7f3420062e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34340091e0 tx=0x7f3434016000 comp rx=0 tx=0).stop 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 msgr2=0x7f344014c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 0x7f344014c840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f343000c590 tx=0x7f343000c8a0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 shutdown_connections 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3420060950 0x7f3420062e00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34400ce2d0 0x7f34400c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34400cc040 0x7f344014c840 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 --2- 172.21.15.64:0/3479977945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34400caa10 0x7f34400c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 >> 172.21.15.64:0/3479977945 conn(0x7f3440067bb0 msgr2=0x7f34400ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 shutdown_connections 2023-09-27T23:14:25.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.534+0000 7f343f59e700 1 -- 172.21.15.64:0/3479977945 wait complete. 2023-09-27T23:14:25.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 Processor -- start 2023-09-27T23:14:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- start start 2023-09-27T23:14:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbde415d320 0x7fbde415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde40aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde40c7190 con 0x7fbde406d690 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde40c1b50 con 0x7fbde415d320 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40c1e30 con 0x7fbde40c15b0 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbde415d320 0x7fbde415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbde415d320 0x7fbde415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33268/0 (socket says 172.21.15.64:33268) 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 -- 172.21.15.64:0/722421922 learned_addr learned my addr 172.21.15.64:0/722421922 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde27fc700 1 --2- 172.21.15.64:0/722421922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde40aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 -- 172.21.15.64:0/722421922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 msgr2=0x7fbde40aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 --2- 172.21.15.64:0/722421922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde40aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 -- 172.21.15.64:0/722421922 >> v1:172.21.15.64:6789/0 conn(0x7fbde406d690 legacy=0x7fbde40680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 -- 172.21.15.64:0/722421922 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40c6e20 con 0x7fbde415d320 2023-09-27T23:14:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde2ffd700 1 --2- 172.21.15.64:0/722421922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde415d6f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40099b0 tx=0x7fbdd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244c51798bc16227 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde1ffb700 1 -- 172.21.15.64:0/722421922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400b860 con 0x7fbde415d320 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde1ffb700 1 -- 172.21.15.64:0/722421922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd400b9c0 con 0x7fbde415d320 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde1ffb700 1 -- 172.21.15.64:0/722421922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400bd60 con 0x7fbde415d320 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- 172.21.15.64:0/722421922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 msgr2=0x7fbde415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 --2- 172.21.15.64:0/722421922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde415d6f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40099b0 tx=0x7fbdd4009250 comp rx=0 tx=0).stop 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- 172.21.15.64:0/722421922 shutdown_connections 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 --2- 172.21.15.64:0/722421922 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde40aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 --2- 172.21.15.64:0/722421922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde415d6f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- 172.21.15.64:0/722421922 >> 172.21.15.64:0/722421922 conn(0x7fbde40c8850 msgr2=0x7fbde40c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- 172.21.15.64:0/722421922 shutdown_connections 2023-09-27T23:14:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.662+0000 7fbde96e3700 1 -- 172.21.15.64:0/722421922 wait complete. 2023-09-27T23:14:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 Processor -- start 2023-09-27T23:14:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 -- start start 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde406d690 0x7fbde4146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde416e2b0 con 0x7fbde415d320 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde416d9a0 con 0x7fbde406d690 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde96e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4171d10 con 0x7fbde40c15b0 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde406cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33280/0 (socket says 172.21.15.64:33280) 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 -- 172.21.15.64:0/2515107666 learned_addr learned my addr 172.21.15.64:0/2515107666 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde27fc700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde2ffd700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde406d690 0x7fbde4146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 -- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 msgr2=0x7fbde406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 -- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde406d690 msgr2=0x7fbde4146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde406d690 0x7fbde4146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 -- 172.21.15.64:0/2515107666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40094f0 con 0x7fbde415d320 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde27fc700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbde37fe700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde406cd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c560 tx=0x7fbdd8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.666+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8011590 con 0x7fbde415d320 2023-09-27T23:14:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80116f0 con 0x7fbde415d320 2023-09-27T23:14:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde416dc60 con 0x7fbde415d320 2023-09-27T23:14:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8011ab0 con 0x7fbde415d320 2023-09-27T23:14:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde4147430 con 0x7fbde415d320 2023-09-27T23:14:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde416e460 con 0x7fbde415d320 2023-09-27T23:14:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbdd801b3b0 con 0x7fbde415d320 2023-09-27T23:14:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbdcaffd700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbdd0060950 0x7fbdd0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbdd807c670 con 0x7fbde415d320 2023-09-27T23:14:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.670+0000 7fbde2ffd700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbdd0060950 0x7fbdd0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.674+0000 7fbde2ffd700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbdd0060950 0x7fbdd0062e00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40099b0 tx=0x7fbdd4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.682+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbde416e460 con 0x7fbde415d320 2023-09-27T23:14:25.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.842+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbde41475c0 con 0x7fbde415d320 2023-09-27T23:14:25.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbdcaffd700 1 -- 172.21.15.64:0/2515107666 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fbdd8017090 con 0x7fbde415d320 2023-09-27T23:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:13:49.156173+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:25.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbdd0060950 msgr2=0x7fbdd0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbdd0060950 0x7fbdd0062e00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40099b0 tx=0x7fbdd4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:25.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 msgr2=0x7fbde406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde406cd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c560 tx=0x7fbdd8007ef0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 shutdown_connections 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbdd0060950 0x7fbdd0062e00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde415d320 0x7fbde406cd00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbde40c15b0 0x7fbde406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 --2- 172.21.15.64:0/2515107666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbde406d690 0x7fbde4146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 >> 172.21.15.64:0/2515107666 conn(0x7fbde40c8850 msgr2=0x7fbde415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 shutdown_connections 2023-09-27T23:14:25.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.846+0000 7fbde96e3700 1 -- 172.21.15.64:0/2515107666 wait complete. 2023-09-27T23:14:25.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:25.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.966+0000 7f8fd1fe8700 1 Processor -- start 2023-09-27T23:14:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.966+0000 7f8fd1fe8700 1 -- start start 2023-09-27T23:14:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fd1fe8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fd1fe8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8fcc13d2f0 0x7f8fcc0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fd1fe8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc0cc0e0 con 0x7f8fcc06b060 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fd1fe8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cb4d0 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fd1fe8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cb7b0 con 0x7f8fcc13cf20 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcbfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8fcc06b060 0x7f8fcc15a9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40056/0 (socket says 172.21.15.64:40056) 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcbfff700 1 -- 172.21.15.64:0/2329061263 learned_addr learned my addr 172.21.15.64:0/2329061263 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fc2ffd700 1 --2- 172.21.15.64:0/2329061263 >> v2:172.21.15.123:3300/0 conn(0x7f8fcc13d2f0 0x7f8fcc0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcb7fe700 1 --2- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcaffd700 1 -- 172.21.15.64:0/2329061263 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497118021 0 0) 0x7f8fcc0cc0e0 con 0x7f8fcc06b060 2023-09-27T23:14:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcaffd700 1 -- 172.21.15.64:0/2329061263 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb8003570 con 0x7f8fcc06b060 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fc2ffd700 1 -- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 msgr2=0x7f8fcc0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcaffd700 1 -- 172.21.15.64:0/2329061263 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790787394 0 0) 0x7f8fb8003570 con 0x7f8fcc06b060 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fc2ffd700 1 --2- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fc2ffd700 1 -- 172.21.15.64:0/2329061263 >> v1:172.21.15.64:6789/0 conn(0x7f8fcc06b060 legacy=0x7f8fcc15a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fc2ffd700 1 -- 172.21.15.64:0/2329061263 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0cba90 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fc2ffd700 1 --2- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc0c85d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40098e0 tx=0x7f8fb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e477e4d164df4ae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcaffd700 1 -- 172.21.15.64:0/2329061263 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40017f0 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcaffd700 1 -- 172.21.15.64:0/2329061263 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4011420 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.970+0000 7f8fcaffd700 1 -- 172.21.15.64:0/2329061263 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40117c0 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc13d2f0 msgr2=0x7f8fcc0c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc0c85d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40098e0 tx=0x7f8fb40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/2329061263 shutdown_connections 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc0c85d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/2329061263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/2329061263 >> 172.21.15.64:0/2329061263 conn(0x7f8fcc0cc580 msgr2=0x7f8fcc0cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/2329061263 shutdown_connections 2023-09-27T23:14:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/2329061263 wait complete. 2023-09-27T23:14:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 Processor -- start 2023-09-27T23:14:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- start start 2023-09-27T23:14:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc06b060 0x7f8fcc0a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc159870 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0aa530 con 0x7f8fcc06b060 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0a9c90 con 0x7f8fcc13cf20 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33284/0 (socket says 172.21.15.64:33284) 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 -- 172.21.15.64:0/1137898708 learned_addr learned my addr 172.21.15.64:0/1137898708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcb7fe700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc06b060 0x7f8fcc0a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fc2ffd700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 -- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 msgr2=0x7f8fcc0c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 -- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc06b060 msgr2=0x7f8fcc0a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc06b060 0x7f8fcc0a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 -- 172.21.15.64:0/1137898708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4009420 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fc2ffd700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fcbfff700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc155740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4001560 tx=0x7f8fc4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004dc0 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc400a6a0 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc0a9f70 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400aa60 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.974+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc155c80 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.978+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc15dbf0 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.978+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8fc4007150 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.978+0000 7f8fc3fff700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8fac060950 0x7f8fac062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.978+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8fc407c950 con 0x7f8fcc13d2f0 2023-09-27T23:14:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.978+0000 7f8fcb7fe700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8fac060950 0x7f8fac062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.982+0000 7f8fcb7fe700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8fac060950 0x7f8fac062e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009db0 tx=0x7f8fbc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:25.986+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8fcc15dbf0 con 0x7f8fcc13d2f0 2023-09-27T23:14:26.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fcc15dbf0 con 0x7f8fcc13d2f0 2023-09-27T23:14:26.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fc3fff700 1 -- 172.21.15.64:0/1137898708 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fcc15dbf0 con 0x7f8fcc13d2f0 2023-09-27T23:14:26.158 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:26.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8fac060950 msgr2=0x7f8fac062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8fac060950 0x7f8fac062e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009db0 tx=0x7f8fbc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 msgr2=0x7f8fcc155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc155740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4001560 tx=0x7f8fc4001640 comp rx=0 tx=0).stop 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 shutdown_connections 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8fac060950 0x7f8fac062e00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fcc13d2f0 0x7f8fcc155740 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fcc13cf20 0x7f8fcc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 --2- 172.21.15.64:0/1137898708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc06b060 0x7f8fcc0a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 >> 172.21.15.64:0/1137898708 conn(0x7f8fcc0cc580 msgr2=0x7f8fcc15d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 shutdown_connections 2023-09-27T23:14:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.154+0000 7f8fd1fe8700 1 -- 172.21.15.64:0/1137898708 wait complete. 2023-09-27T23:14:26.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:26.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.274+0000 7fd926ccc700 1 Processor -- start 2023-09-27T23:14:26.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 -- start start 2023-09-27T23:14:26.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd920158b00 0x7fd920158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd92006bb50 0x7fd92015d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92006b6c0 con 0x7fd92006a4f0 2023-09-27T23:14:26.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92015d880 con 0x7fd92006bb50 2023-09-27T23:14:26.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92006aab0 con 0x7fd920158b00 2023-09-27T23:14:26.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd92006bb50 0x7fd92015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9264cb700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd92006a4f0 0x7fd9200cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40058/0 (socket says 172.21.15.64:40058) 2023-09-27T23:14:26.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9264cb700 1 -- 172.21.15.64:0/2075291948 learned_addr learned my addr 172.21.15.64:0/2075291948 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:26.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd92006bb50 0x7fd92015d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33300/0 (socket says 172.21.15.64:33300) 2023-09-27T23:14:26.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd925cca700 1 --2- 172.21.15.64:0/2075291948 >> v2:172.21.15.123:3300/0 conn(0x7fd920158b00 0x7fd920158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 -- 172.21.15.64:0/2075291948 >> v2:172.21.15.123:3300/0 conn(0x7fd920158b00 msgr2=0x7fd920158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:26.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 --2- 172.21.15.64:0/2075291948 >> v2:172.21.15.123:3300/0 conn(0x7fd920158b00 0x7fd920158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 -- 172.21.15.64:0/2075291948 >> v1:172.21.15.64:6789/0 conn(0x7fd92006a4f0 legacy=0x7fd9200cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd924cc8700 1 -- 172.21.15.64:0/2075291948 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343701065 0 0) 0x7fd92006b6c0 con 0x7fd92006a4f0 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 -- 172.21.15.64:0/2075291948 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92006ad90 con 0x7fd92006bb50 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd9254c9700 1 --2- 172.21.15.64:0/2075291948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92006bb50 0x7fd92015d340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd92006a2e0 tx=0x7fd9100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be00d327d84d51a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd924cc8700 1 -- 172.21.15.64:0/2075291948 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000b6b0 con 0x7fd92006bb50 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd924cc8700 1 -- 172.21.15.64:0/2075291948 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91000b810 con 0x7fd92006bb50 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd924cc8700 1 -- 172.21.15.64:0/2075291948 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000bbb0 con 0x7fd92006bb50 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 -- 172.21.15.64:0/2075291948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92006bb50 msgr2=0x7fd92015d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.278+0000 7fd926ccc700 1 --2- 172.21.15.64:0/2075291948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92006bb50 0x7fd92015d340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd92006a2e0 tx=0x7fd9100091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- 172.21.15.64:0/2075291948 shutdown_connections 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 --2- 172.21.15.64:0/2075291948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92006bb50 0x7fd92015d340 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 --2- 172.21.15.64:0/2075291948 >> v2:172.21.15.123:3300/0 conn(0x7fd920158b00 0x7fd920158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- 172.21.15.64:0/2075291948 >> 172.21.15.64:0/2075291948 conn(0x7fd920067bb0 msgr2=0x7fd9200cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:26.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- 172.21.15.64:0/2075291948 shutdown_connections 2023-09-27T23:14:26.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- 172.21.15.64:0/2075291948 wait complete. 2023-09-27T23:14:26.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 Processor -- start 2023-09-27T23:14:26.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- start start 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd92006a4f0 0x7fd9200c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd92006bb50 0x7fd9200b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 0x7fd9200b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920144300 con 0x7fd920158b00 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200c9de0 con 0x7fd92006bb50 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd926ccc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9201436f0 con 0x7fd92006a4f0 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd9264cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 0x7fd9200b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd9264cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 0x7fd9200b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33312/0 (socket says 172.21.15.64:33312) 2023-09-27T23:14:26.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd925cca700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd92006a4f0 0x7fd9200c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd9264cb700 1 -- 172.21.15.64:0/1563498863 learned_addr learned my addr 172.21.15.64:0/1563498863 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:26.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.282+0000 7fd9254c9700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd92006bb50 0x7fd9200b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd9264cb700 1 -- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd92006a4f0 msgr2=0x7fd9200c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd9264cb700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd92006a4f0 0x7fd9200c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd9264cb700 1 -- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd92006bb50 msgr2=0x7fd9200b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd9264cb700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd92006bb50 0x7fd9200b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd9264cb700 1 -- 172.21.15.64:0/1563498863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009420 con 0x7fd920158b00 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd925cca700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd92006a4f0 0x7fd9200c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd9264cb700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 0x7fd9200b32a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd91800ccc0 tx=0x7fd918001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800a8a0 con 0x7fd920158b00 2023-09-27T23:14:26.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91800aa00 con 0x7fd920158b00 2023-09-27T23:14:26.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800adc0 con 0x7fd920158b00 2023-09-27T23:14:26.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9201439d0 con 0x7fd920158b00 2023-09-27T23:14:26.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920143f90 con 0x7fd920158b00 2023-09-27T23:14:26.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.286+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9201444b0 con 0x7fd920158b00 2023-09-27T23:14:26.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.290+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fd918007300 con 0x7fd920158b00 2023-09-27T23:14:26.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.290+0000 7fd915ffb700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd904060950 0x7fd904062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.290+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fd91807d4c0 con 0x7fd920158b00 2023-09-27T23:14:26.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.290+0000 7fd925cca700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd904060950 0x7fd904062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.290+0000 7fd925cca700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd904060950 0x7fd904062e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd91c006fd0 tx=0x7fd91c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.298+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd9201444b0 con 0x7fd920158b00 2023-09-27T23:14:26.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.462+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd9201444b0 con 0x7fd920158b00 2023-09-27T23:14:26.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.462+0000 7fd915ffb700 1 -- 172.21.15.64:0/1563498863 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd9201444b0 con 0x7fd920158b00 2023-09-27T23:14:26.466 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:26.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd904060950 msgr2=0x7fd904062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd904060950 0x7fd904062e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd91c006fd0 tx=0x7fd91c008040 comp rx=0 tx=0).stop 2023-09-27T23:14:26.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 msgr2=0x7fd9200b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 0x7fd9200b32a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd91800ccc0 tx=0x7fd918001a80 comp rx=0 tx=0).stop 2023-09-27T23:14:26.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 shutdown_connections 2023-09-27T23:14:26.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd904060950 0x7fd904062e00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd920158b00 0x7fd9200b32a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd92006bb50 0x7fd9200b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 --2- 172.21.15.64:0/1563498863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd92006a4f0 0x7fd9200c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 >> 172.21.15.64:0/1563498863 conn(0x7fd920067bb0 msgr2=0x7fd9200ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:26.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 shutdown_connections 2023-09-27T23:14:26.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.466+0000 7fd926ccc700 1 -- 172.21.15.64:0/1563498863 wait complete. 2023-09-27T23:14:26.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:26.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 Processor -- start 2023-09-27T23:14:26.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 -- start start 2023-09-27T23:14:26.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7410158c90 0x7f74100caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f741006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741006c850 con 0x7f741006aeb0 2023-09-27T23:14:26.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741006b470 con 0x7f7410158c90 2023-09-27T23:14:26.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7414d6f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741006b750 con 0x7f7410159770 2023-09-27T23:14:26.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7406ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f741006aeb0 0x7f7410159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40064/0 (socket says 172.21.15.64:40064) 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f7406ffd700 1 -- 172.21.15.64:0/2946016573 learned_addr learned my addr 172.21.15.64:0/2946016573 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740ffff700 1 --2- 172.21.15.64:0/2946016573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f741006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740f7fe700 1 --2- 172.21.15.64:0/2946016573 >> v2:172.21.15.123:3300/0 conn(0x7f7410158c90 0x7f74100caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740effd700 1 -- 172.21.15.64:0/2946016573 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186778433 0 0) 0x7f741006c850 con 0x7f741006aeb0 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740effd700 1 -- 172.21.15.64:0/2946016573 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73fc003570 con 0x7f741006aeb0 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740ffff700 1 -- 172.21.15.64:0/2946016573 >> v2:172.21.15.123:3300/0 conn(0x7f7410158c90 msgr2=0x7f74100caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740ffff700 1 --2- 172.21.15.64:0/2946016573 >> v2:172.21.15.123:3300/0 conn(0x7f7410158c90 0x7f74100caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740ffff700 1 -- 172.21.15.64:0/2946016573 >> v1:172.21.15.64:6789/0 conn(0x7f741006aeb0 legacy=0x7f7410159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.594+0000 7f740ffff700 1 -- 172.21.15.64:0/2946016573 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741006ba30 con 0x7f7410159770 2023-09-27T23:14:26.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f740ffff700 1 --2- 172.21.15.64:0/2946016573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f741006cce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7408009e60 tx=0x7f7408009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e8dde17eeac7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f740effd700 1 -- 172.21.15.64:0/2946016573 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800e070 con 0x7f7410159770 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f740effd700 1 -- 172.21.15.64:0/2946016573 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740800bab0 con 0x7f7410159770 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f740effd700 1 -- 172.21.15.64:0/2946016573 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800be30 con 0x7f7410159770 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 -- 172.21.15.64:0/2946016573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 msgr2=0x7f741006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 --2- 172.21.15.64:0/2946016573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f741006cce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7408009e60 tx=0x7f7408009780 comp rx=0 tx=0).stop 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 -- 172.21.15.64:0/2946016573 shutdown_connections 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 --2- 172.21.15.64:0/2946016573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f741006cce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 --2- 172.21.15.64:0/2946016573 >> v2:172.21.15.123:3300/0 conn(0x7f7410158c90 0x7f74100caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 -- 172.21.15.64:0/2946016573 >> 172.21.15.64:0/2946016573 conn(0x7f7410067bb0 msgr2=0x7f74100cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 -- 172.21.15.64:0/2946016573 shutdown_connections 2023-09-27T23:14:26.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 -- 172.21.15.64:0/2946016573 wait complete. 2023-09-27T23:14:26.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 Processor -- start 2023-09-27T23:14:26.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.598+0000 7f7414d6f700 1 -- start start 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 0x7f7410155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7410158c90 0x7f7410156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f7410157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7410143de0 con 0x7f741006aeb0 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74100b5690 con 0x7f7410158c90 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74100b4a80 con 0x7f7410159770 2023-09-27T23:14:26.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 0x7f7410155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 0x7f7410155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33324/0 (socket says 172.21.15.64:33324) 2023-09-27T23:14:26.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 -- 172.21.15.64:0/692267088 learned_addr learned my addr 172.21.15.64:0/692267088 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:26.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7406ffd700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7410158c90 0x7f7410156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740ffff700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f7410157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 -- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 msgr2=0x7f7410157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f7410157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 -- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7410158c90 msgr2=0x7f7410156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7410158c90 0x7f7410156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 -- 172.21.15.64:0/692267088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408009a20 con 0x7f741006aeb0 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f740f7fe700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 0x7f7410155190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f740000ec80 tx=0x7f74000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000c7f0 con 0x7f741006aeb0 2023-09-27T23:14:26.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740000c950 con 0x7f741006aeb0 2023-09-27T23:14:26.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74100b4d60 con 0x7f741006aeb0 2023-09-27T23:14:26.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000ccf0 con 0x7f741006aeb0 2023-09-27T23:14:26.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.602+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74100b5320 con 0x7f741006aeb0 2023-09-27T23:14:26.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.606+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74100c9f40 con 0x7f741006aeb0 2023-09-27T23:14:26.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.606+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7400013680 con 0x7f741006aeb0 2023-09-27T23:14:26.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.606+0000 7f7407fff700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73f0060950 0x7f73f0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.606+0000 7f7406ffd700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73f0060950 0x7f73f0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.606+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f740007d400 con 0x7f741006aeb0 2023-09-27T23:14:26.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.610+0000 7f7406ffd700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73f0060950 0x7f73f0062e00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f741016cbc0 tx=0x7f73f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.618+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f74100c9f40 con 0x7f741006aeb0 2023-09-27T23:14:26.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.778+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74100c9f40 con 0x7f741006aeb0 2023-09-27T23:14:26.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.778+0000 7f7407fff700 1 -- 172.21.15.64:0/692267088 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74100c9f40 con 0x7f741006aeb0 2023-09-27T23:14:26.784 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73f0060950 msgr2=0x7f73f0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73f0060950 0x7f73f0062e00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f741016cbc0 tx=0x7f73f8009040 comp rx=0 tx=0).stop 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 msgr2=0x7f7410155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 0x7f7410155190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f740000ec80 tx=0x7f74000091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 shutdown_connections 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f73f0060950 0x7f73f0062e00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7410159770 0x7f7410157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7410158c90 0x7f7410156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 --2- 172.21.15.64:0/692267088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f741006aeb0 0x7f7410155190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 >> 172.21.15.64:0/692267088 conn(0x7f7410067bb0 msgr2=0x7f74100cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:26.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 shutdown_connections 2023-09-27T23:14:26.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.782+0000 7f7414d6f700 1 -- 172.21.15.64:0/692267088 wait complete. 2023-09-27T23:14:26.798 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.906+0000 7f507359e700 1 Processor -- start 2023-09-27T23:14:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.906+0000 7f507359e700 1 -- start start 2023-09-27T23:14:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5074158c90 0x7f50740caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f5074159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507406c850 con 0x7f50741595a0 2023-09-27T23:14:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507406b470 con 0x7f5074158c90 2023-09-27T23:14:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507406b750 con 0x7f507406aeb0 2023-09-27T23:14:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5072d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f50741595a0 0x7f507415c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40066/0 (socket says 172.21.15.64:40066) 2023-09-27T23:14:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5072d9d700 1 -- 172.21.15.64:0/2271590473 learned_addr learned my addr 172.21.15.64:0/2271590473 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5071d9b700 1 --2- 172.21.15.64:0/2271590473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f5074159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507259c700 1 --2- 172.21.15.64:0/2271590473 >> v2:172.21.15.123:3300/0 conn(0x7f5074158c90 0x7f50740caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507159a700 1 -- 172.21.15.64:0/2271590473 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600622338 0 0) 0x7f507406c850 con 0x7f50741595a0 2023-09-27T23:14:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507159a700 1 -- 172.21.15.64:0/2271590473 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505c003570 con 0x7f50741595a0 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5071d9b700 1 -- 172.21.15.64:0/2271590473 >> v2:172.21.15.123:3300/0 conn(0x7f5074158c90 msgr2=0x7f50740caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5071d9b700 1 --2- 172.21.15.64:0/2271590473 >> v2:172.21.15.123:3300/0 conn(0x7f5074158c90 0x7f50740caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5071d9b700 1 -- 172.21.15.64:0/2271590473 >> v1:172.21.15.64:6789/0 conn(0x7f50741595a0 legacy=0x7f507415c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5071d9b700 1 -- 172.21.15.64:0/2271590473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507406ba30 con 0x7f507406aeb0 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f5071d9b700 1 --2- 172.21.15.64:0/2271590473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f5074159060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50640098e0 tx=0x7f50640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a431ea270166c40 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507159a700 1 -- 172.21.15.64:0/2271590473 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064011630 con 0x7f507406aeb0 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507159a700 1 -- 172.21.15.64:0/2271590473 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064011790 con 0x7f507406aeb0 2023-09-27T23:14:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507159a700 1 -- 172.21.15.64:0/2271590473 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064011b30 con 0x7f507406aeb0 2023-09-27T23:14:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- 172.21.15.64:0/2271590473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 msgr2=0x7f5074159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 --2- 172.21.15.64:0/2271590473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f5074159060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50640098e0 tx=0x7f50640091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- 172.21.15.64:0/2271590473 shutdown_connections 2023-09-27T23:14:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 --2- 172.21.15.64:0/2271590473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f5074159060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 --2- 172.21.15.64:0/2271590473 >> v2:172.21.15.123:3300/0 conn(0x7f5074158c90 0x7f50740caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- 172.21.15.64:0/2271590473 >> 172.21.15.64:0/2271590473 conn(0x7f5074067bb0 msgr2=0x7f50740cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:26.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- 172.21.15.64:0/2271590473 shutdown_connections 2023-09-27T23:14:26.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.910+0000 7f507359e700 1 -- 172.21.15.64:0/2271590473 wait complete. 2023-09-27T23:14:26.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 Processor -- start 2023-09-27T23:14:26.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 -- start start 2023-09-27T23:14:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f50740c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 0x7f5074154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50741595a0 0x7f5074154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c9630 con 0x7f5074158c90 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c8a20 con 0x7f50741595a0 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c8d00 con 0x7f507406aeb0 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 0x7f5074154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 0x7f5074154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33338/0 (socket says 172.21.15.64:33338) 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 -- 172.21.15.64:0/4250980309 learned_addr learned my addr 172.21.15.64:0/4250980309 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f507259c700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f50740c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5072d9d700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50741595a0 0x7f5074154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 -- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 msgr2=0x7f50740c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f50740c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 -- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50741595a0 msgr2=0x7f5074154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50741595a0 0x7f5074154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:26.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 -- 172.21.15.64:0/4250980309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064009420 con 0x7f5074158c90 2023-09-27T23:14:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.914+0000 7f5071d9b700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 0x7f5074154940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f507406e2e0 tx=0x7f5064017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064011b10 con 0x7f5074158c90 2023-09-27T23:14:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50640253f0 con 0x7f5074158c90 2023-09-27T23:14:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50740c8fe0 con 0x7f5074158c90 2023-09-27T23:14:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064025790 con 0x7f5074158c90 2023-09-27T23:14:26.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5074153650 con 0x7f5074158c90 2023-09-27T23:14:26.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507404be40 con 0x7f5074158c90 2023-09-27T23:14:26.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f5064025b80 con 0x7f5074158c90 2023-09-27T23:14:26.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.918+0000 7f50627fc700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5054060950 0x7f5054062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:26.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.922+0000 7f507259c700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5054060950 0x7f5054062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:26.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.922+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f506407ea10 con 0x7f5074158c90 2023-09-27T23:14:26.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.922+0000 7f507259c700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5054060950 0x7f5054062e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5068009cb0 tx=0x7f5068008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:26.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:26.930+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f506404e960 con 0x7f5074158c90 2023-09-27T23:14:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.094+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f507406bd10 con 0x7f5074158c90 2023-09-27T23:14:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.094+0000 7f50627fc700 1 -- 172.21.15.64:0/4250980309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5064051f80 con 0x7f5074158c90 2023-09-27T23:14:27.098 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.094+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5054060950 msgr2=0x7f5054062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.094+0000 7f507359e700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5054060950 0x7f5054062e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5068009cb0 tx=0x7f5068008040 comp rx=0 tx=0).stop 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.094+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 msgr2=0x7f5074154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.094+0000 7f507359e700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 0x7f5074154940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f507406e2e0 tx=0x7f5064017000 comp rx=0 tx=0).stop 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 shutdown_connections 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5054060950 0x7f5054062e00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50741595a0 0x7f5074154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5074158c90 0x7f5074154940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 --2- 172.21.15.64:0/4250980309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f507406aeb0 0x7f50740c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 >> 172.21.15.64:0/4250980309 conn(0x7f5074067bb0 msgr2=0x7f50740afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:27.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 shutdown_connections 2023-09-27T23:14:27.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.098+0000 7f507359e700 1 -- 172.21.15.64:0/4250980309 wait complete. 2023-09-27T23:14:27.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.214+0000 7f51c952f700 1 Processor -- start 2023-09-27T23:14:27.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.214+0000 7f51c952f700 1 -- start start 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c952f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51c4155520 0x7f51c40c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c952f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51c40ae6c0 0x7f51c406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c952f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c406b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c952f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c40ad020 con 0x7f51c40ae6c0 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c952f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c40c81e0 con 0x7f51c4155520 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c952f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40aca10 con 0x7f51c40c7670 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51c40ae6c0 0x7f51c406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c8d2e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c406b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51c4155520 0x7f51c40c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51c40ae6c0 0x7f51c406ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33350/0 (socket says 172.21.15.64:33350) 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 -- 172.21.15.64:0/1976796892 learned_addr learned my addr 172.21.15.64:0/1976796892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 -- 172.21.15.64:0/1976796892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 msgr2=0x7f51c406b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 --2- 172.21.15.64:0/1976796892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c406b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 -- 172.21.15.64:0/1976796892 >> v2:172.21.15.123:3300/0 conn(0x7f51c4155520 msgr2=0x7f51c40c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 --2- 172.21.15.64:0/1976796892 >> v2:172.21.15.123:3300/0 conn(0x7f51c4155520 0x7f51c40c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 -- 172.21.15.64:0/1976796892 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40c7c20 con 0x7f51c40ae6c0 2023-09-27T23:14:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51bbfff700 1 --2- 172.21.15.64:0/1976796892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c406ae20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51c40c8b80 tx=0x7f51ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18095616b9c101f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c37fe700 1 -- 172.21.15.64:0/1976796892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00b6d0 con 0x7f51c40ae6c0 2023-09-27T23:14:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c37fe700 1 -- 172.21.15.64:0/1976796892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac00b830 con 0x7f51c40ae6c0 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.218+0000 7f51c37fe700 1 -- 172.21.15.64:0/1976796892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00bbf0 con 0x7f51c40ae6c0 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- 172.21.15.64:0/1976796892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 msgr2=0x7f51c406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- 172.21.15.64:0/1976796892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c406ae20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51c40c8b80 tx=0x7f51ac009250 comp rx=0 tx=0).stop 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- 172.21.15.64:0/1976796892 shutdown_connections 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- 172.21.15.64:0/1976796892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c406b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- 172.21.15.64:0/1976796892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c406ae20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- 172.21.15.64:0/1976796892 >> v2:172.21.15.123:3300/0 conn(0x7f51c4155520 0x7f51c40c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- 172.21.15.64:0/1976796892 >> 172.21.15.64:0/1976796892 conn(0x7f51c40c8cd0 msgr2=0x7f51c40c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- 172.21.15.64:0/1976796892 shutdown_connections 2023-09-27T23:14:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- 172.21.15.64:0/1976796892 wait complete. 2023-09-27T23:14:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 Processor -- start 2023-09-27T23:14:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- start start 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c4162b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c40c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4155520 0x7f51c41558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4163070 con 0x7f51c40ae6c0 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b0160 con 0x7f51c4155520 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.222+0000 7f51c952f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b0440 con 0x7f51c40c7670 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c4162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c4162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33354/0 (socket says 172.21.15.64:33354) 2023-09-27T23:14:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 -- 172.21.15.64:0/2199108621 learned_addr learned my addr 172.21.15.64:0/2199108621 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51bbfff700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c40c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c8d2e700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4155520 0x7f51c41558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 -- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 msgr2=0x7f51c40c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c40c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 -- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4155520 msgr2=0x7f51c41558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4155520 0x7f51c41558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 -- 172.21.15.64:0/2199108621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0094f0 con 0x7f51c40ae6c0 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c8d2e700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4155520 0x7f51c41558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c3fff700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c4162b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51b400fa10 tx=0x7f51b40080e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400d750 con 0x7f51c40ae6c0 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b400d8b0 con 0x7f51c40ae6c0 2023-09-27T23:14:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c40b0720 con 0x7f51c40ae6c0 2023-09-27T23:14:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c40c2c40 con 0x7f51c40ae6c0 2023-09-27T23:14:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400dc30 con 0x7f51c40ae6c0 2023-09-27T23:14:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c404be40 con 0x7f51c40ae6c0 2023-09-27T23:14:27.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.226+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f51b4012670 con 0x7f51c40ae6c0 2023-09-27T23:14:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.230+0000 7f51c0ff9700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a4060950 0x7f51a4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.230+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f51b4018030 con 0x7f51c40ae6c0 2023-09-27T23:14:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.230+0000 7f51bbfff700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a4060950 0x7f51a4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.230+0000 7f51bbfff700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a4060950 0x7f51a4062e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51c406ac20 tx=0x7f51ac017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:27.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.238+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f51b404d710 con 0x7f51c40ae6c0 2023-09-27T23:14:27.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.398+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51c40c7f00 con 0x7f51c40ae6c0 2023-09-27T23:14:27.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.398+0000 7f51c0ff9700 1 -- 172.21.15.64:0/2199108621 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v31) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51b4050d30 con 0x7f51c40ae6c0 2023-09-27T23:14:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a4060950 msgr2=0x7f51a4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a4060950 0x7f51a4062e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51c406ac20 tx=0x7f51ac017000 comp rx=0 tx=0).stop 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 msgr2=0x7f51c4162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c4162b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51b400fa10 tx=0x7f51b40080e0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 shutdown_connections 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a4060950 0x7f51a4062e00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4155520 0x7f51c41558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51c40c7670 0x7f51c40c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 --2- 172.21.15.64:0/2199108621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c40ae6c0 0x7f51c4162b30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 >> 172.21.15.64:0/2199108621 conn(0x7f51c40c8cd0 msgr2=0x7f51c40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 shutdown_connections 2023-09-27T23:14:27.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.402+0000 7f51c952f700 1 -- 172.21.15.64:0/2199108621 wait complete. 2023-09-27T23:14:27.429 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-09-27T23:14:27.429 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:14:27.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-09-27T23:14:27.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.530+0000 7f59ddf70700 1 Processor -- start 2023-09-27T23:14:27.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.530+0000 7f59ddf70700 1 -- start start 2023-09-27T23:14:27.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59d815d320 0x7f59d815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59d80c15b0 0x7f59d80aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d80c7190 con 0x7f59d806d690 2023-09-27T23:14:27.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d80c1b50 con 0x7f59d815d320 2023-09-27T23:14:27.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d80c1e30 con 0x7f59d80c15b0 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59d815d320 0x7f59d815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59d815d320 0x7f59d815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33364/0 (socket says 172.21.15.64:33364) 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 -- 172.21.15.64:0/2558455343 learned_addr learned my addr 172.21.15.64:0/2558455343 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ceffd700 1 --2- 172.21.15.64:0/2558455343 >> v2:172.21.15.123:3300/0 conn(0x7f59d80c15b0 0x7f59d80aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 -- 172.21.15.64:0/2558455343 >> v2:172.21.15.123:3300/0 conn(0x7f59d80c15b0 msgr2=0x7f59d80aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 --2- 172.21.15.64:0/2558455343 >> v2:172.21.15.123:3300/0 conn(0x7f59d80c15b0 0x7f59d80aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 -- 172.21.15.64:0/2558455343 >> v1:172.21.15.123:6789/0 conn(0x7f59d806d690 legacy=0x7f59d80680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 -- 172.21.15.64:0/2558455343 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d80c6e20 con 0x7f59d815d320 2023-09-27T23:14:27.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d77fe700 1 --2- 172.21.15.64:0/2558455343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d815d320 0x7f59d815d6f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f59d806f530 tx=0x7f59c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5018628da17c3e28 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:27.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d6ffd700 1 -- 172.21.15.64:0/2558455343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8009e70 con 0x7f59d815d320 2023-09-27T23:14:27.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d6ffd700 1 -- 172.21.15.64:0/2558455343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c800b860 con 0x7f59d815d320 2023-09-27T23:14:27.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59d6ffd700 1 -- 172.21.15.64:0/2558455343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800bc00 con 0x7f59d815d320 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- 172.21.15.64:0/2558455343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d815d320 msgr2=0x7f59d815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 --2- 172.21.15.64:0/2558455343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d815d320 0x7f59d815d6f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f59d806f530 tx=0x7f59c8009250 comp rx=0 tx=0).stop 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- 172.21.15.64:0/2558455343 shutdown_connections 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 --2- 172.21.15.64:0/2558455343 >> v2:172.21.15.123:3300/0 conn(0x7f59d80c15b0 0x7f59d80aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 --2- 172.21.15.64:0/2558455343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d815d320 0x7f59d815d6f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- 172.21.15.64:0/2558455343 >> 172.21.15.64:0/2558455343 conn(0x7f59d80c8850 msgr2=0x7f59d80c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- 172.21.15.64:0/2558455343 shutdown_connections 2023-09-27T23:14:27.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.534+0000 7f59ddf70700 1 -- 172.21.15.64:0/2558455343 wait complete. 2023-09-27T23:14:27.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 Processor -- start 2023-09-27T23:14:27.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 -- start start 2023-09-27T23:14:27.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 0x7f59d8170fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59d80c15b0 0x7f59d81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59d815d320 0x7f59d8156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d8157a10 con 0x7f59d806d690 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d81566c0 con 0x7f59d80c15b0 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ddf70700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d81716a0 con 0x7f59d815d320 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 0x7f59d8170fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 0x7f59d8170fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33366/0 (socket says 172.21.15.64:33366) 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 -- 172.21.15.64:0/934240031 learned_addr learned my addr 172.21.15.64:0/934240031 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59ceffd700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59d80c15b0 0x7f59d81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d7fff700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59d815d320 0x7f59d8156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 -- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59d815d320 msgr2=0x7f59d8156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59d815d320 0x7f59d8156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 -- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59d80c15b0 msgr2=0x7f59d81574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:27.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59d80c15b0 0x7f59d81574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:27.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d77fe700 1 -- 172.21.15.64:0/934240031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80094f0 con 0x7f59d806d690 2023-09-27T23:14:27.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.538+0000 7f59d7fff700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59d815d320 0x7f59d8156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:27.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59d77fe700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 0x7f59d8170fa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f59d806a2b0 tx=0x7f59c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:27.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800b9d0 con 0x7f59d806d690 2023-09-27T23:14:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c800bb30 con 0x7f59d806d690 2023-09-27T23:14:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d8171980 con 0x7f59d806d690 2023-09-27T23:14:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8025630 con 0x7f59d806d690 2023-09-27T23:14:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d80af100 con 0x7f59d806d690 2023-09-27T23:14:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d8071120 con 0x7f59d806d690 2023-09-27T23:14:27.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f59c8023070 con 0x7f59d806d690 2023-09-27T23:14:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59cffff700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59b8060950 0x7f59b8062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f59c807eac0 con 0x7f59d806d690 2023-09-27T23:14:27.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.542+0000 7f59ceffd700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59b8060950 0x7f59b8062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:27.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.546+0000 7f59ceffd700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59b8060950 0x7f59b8062e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59d8160bd0 tx=0x7f59c00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.550+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f59c80524a0 con 0x7f59d806d690 2023-09-27T23:14:27.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:27.770+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f59d80642a0 con 0x7f59d806d690 2023-09-27T23:14:28.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.062+0000 7f59cffff700 1 -- 172.21.15.64:0/934240031 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f59c8052030 con 0x7f59d806d690 2023-09-27T23:14:28.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.066+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59b8060950 msgr2=0x7f59b8062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.066+0000 7f59ddf70700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59b8060950 0x7f59b8062e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59d8160bd0 tx=0x7f59c00091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.066+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 msgr2=0x7f59d8170fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.066+0000 7f59ddf70700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 0x7f59d8170fa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f59d806a2b0 tx=0x7f59c8017000 comp rx=0 tx=0).stop 2023-09-27T23:14:28.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 shutdown_connections 2023-09-27T23:14:28.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59b8060950 0x7f59b8062e00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59d815d320 0x7f59d8156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59d80c15b0 0x7f59d81574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 --2- 172.21.15.64:0/934240031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d806d690 0x7f59d8170fa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 >> 172.21.15.64:0/934240031 conn(0x7f59d80c8850 msgr2=0x7f59d815b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:28.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 shutdown_connections 2023-09-27T23:14:28.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.070+0000 7f59ddf70700 1 -- 172.21.15.64:0/934240031 wait complete. 2023-09-27T23:14:28.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:14:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.202+0000 7f64156b2700 1 Processor -- start 2023-09-27T23:14:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.202+0000 7f64156b2700 1 -- start start 2023-09-27T23:14:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f64156b2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64100acea0 0x7f641006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f64156b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64100cbbe0 0x7f64100c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f64156b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f64156b2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64100ccd20 con 0x7f64100cbbe0 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f64156b2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64100c8df0 con 0x7f64100acea0 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f64156b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641006cec0 con 0x7f64100c8990 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64100cbbe0 0x7f64100c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64100cbbe0 0x7f64100c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33374/0 (socket says 172.21.15.64:33374) 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 -- 172.21.15.64:0/1442788281 learned_addr learned my addr 172.21.15.64:0/1442788281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f640ffff700 1 --2- 172.21.15.64:0/1442788281 >> v2:172.21.15.123:3300/0 conn(0x7f64100acea0 0x7f641006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6414eb1700 1 --2- 172.21.15.64:0/1442788281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 -- 172.21.15.64:0/1442788281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 msgr2=0x7f64100ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 --2- 172.21.15.64:0/1442788281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 -- 172.21.15.64:0/1442788281 >> v2:172.21.15.123:3300/0 conn(0x7f64100acea0 msgr2=0x7f641006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:28.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 --2- 172.21.15.64:0/1442788281 >> v2:172.21.15.123:3300/0 conn(0x7f64100acea0 0x7f641006c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 -- 172.21.15.64:0/1442788281 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641006d1a0 con 0x7f64100cbbe0 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.206+0000 7f6407fff700 1 --2- 172.21.15.64:0/1442788281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100cbbe0 0x7f64100c8450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f641006ba40 tx=0x7f63f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24be5e8189643eec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f640f7fe700 1 -- 172.21.15.64:0/1442788281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f800b640 con 0x7f64100cbbe0 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f640f7fe700 1 -- 172.21.15.64:0/1442788281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f800b7a0 con 0x7f64100cbbe0 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f640f7fe700 1 -- 172.21.15.64:0/1442788281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f800bb60 con 0x7f64100cbbe0 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 -- 172.21.15.64:0/1442788281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100cbbe0 msgr2=0x7f64100c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 --2- 172.21.15.64:0/1442788281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100cbbe0 0x7f64100c8450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f641006ba40 tx=0x7f63f8009250 comp rx=0 tx=0).stop 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 -- 172.21.15.64:0/1442788281 shutdown_connections 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 --2- 172.21.15.64:0/1442788281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 --2- 172.21.15.64:0/1442788281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100cbbe0 0x7f64100c8450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 --2- 172.21.15.64:0/1442788281 >> v2:172.21.15.123:3300/0 conn(0x7f64100acea0 0x7f641006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 -- 172.21.15.64:0/1442788281 >> 172.21.15.64:0/1442788281 conn(0x7f64100cdc00 msgr2=0x7f64100ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:28.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 -- 172.21.15.64:0/1442788281 shutdown_connections 2023-09-27T23:14:28.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.210+0000 7f64156b2700 1 -- 172.21.15.64:0/1442788281 wait complete. 2023-09-27T23:14:28.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 Processor -- start 2023-09-27T23:14:28.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 -- start start 2023-09-27T23:14:28.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 0x7f6410155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64101594b0 0x7f64100b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6410159940 con 0x7f64100acea0 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64100c6290 con 0x7f64101594b0 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641016ad50 con 0x7f64100c8990 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 0x7f6410155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 0x7f6410155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33384/0 (socket says 172.21.15.64:33384) 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 -- 172.21.15.64:0/3097571430 learned_addr learned my addr 172.21.15.64:0/3097571430 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f6407fff700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f6414eb1700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64101594b0 0x7f64100b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 -- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 msgr2=0x7f64100b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 -- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64101594b0 msgr2=0x7f64100b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64101594b0 0x7f64100b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 -- 172.21.15.64:0/3097571430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80094f0 con 0x7f64100acea0 2023-09-27T23:14:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640ffff700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 0x7f6410155740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f640000ecb0 tx=0x7f64000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640000c850 con 0x7f64100acea0 2023-09-27T23:14:28.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640000c9d0 con 0x7f64100acea0 2023-09-27T23:14:28.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641016b1e0 con 0x7f64100acea0 2023-09-27T23:14:28.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.214+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640000cd90 con 0x7f64100acea0 2023-09-27T23:14:28.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.218+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641016a700 con 0x7f64100acea0 2023-09-27T23:14:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.218+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641004be40 con 0x7f64100acea0 2023-09-27T23:14:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.218+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6400013680 con 0x7f64100acea0 2023-09-27T23:14:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.218+0000 7f640cff9700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f63f0060950 0x7f63f0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.218+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f640007d420 con 0x7f64100acea0 2023-09-27T23:14:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.218+0000 7f6407fff700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f63f0060950 0x7f63f0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.222+0000 7f6407fff700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f63f0060950 0x7f63f0062e00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f641006ba40 tx=0x7f63f8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.226+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f640004d350 con 0x7f64100acea0 2023-09-27T23:14:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.386+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f64100aca10 con 0x7f64100acea0 2023-09-27T23:14:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.398+0000 7f640cff9700 1 -- 172.21.15.64:0/3097571430 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6400050970 con 0x7f64100acea0 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.398+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f63f0060950 msgr2=0x7f63f0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f63f0060950 0x7f63f0062e00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f641006ba40 tx=0x7f63f8017000 comp rx=0 tx=0).stop 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 msgr2=0x7f6410155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 0x7f6410155740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f640000ecb0 tx=0x7f64000091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 shutdown_connections 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f63f0060950 0x7f63f0062e00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64101594b0 0x7f64100b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64100c8990 0x7f64100b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 --2- 172.21.15.64:0/3097571430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64100acea0 0x7f6410155740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 >> 172.21.15.64:0/3097571430 conn(0x7f64100cdc00 msgr2=0x7f64101564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 shutdown_connections 2023-09-27T23:14:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.402+0000 7f64156b2700 1 -- 172.21.15.64:0/3097571430 wait complete. 2023-09-27T23:14:28.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:14:28.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:14:28.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:14:28.461 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:14:28.461 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:14:28.461 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:14:28.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:28.497 INFO:teuthology.orchestra.run:waiting for 900 2023-09-27T23:14:28.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:14:28.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:28.498 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:14:28.504 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2023-09-27T23:14:28.504 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:14:28.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-09-27T23:14:28.531 INFO:teuthology.orchestra.run:waiting for 900 2023-09-27T23:14:28.532 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:14:28.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:28.532 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:14:28.556 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:14:28.556 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:14:28.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-09-27T23:14:28.573 INFO:teuthology.orchestra.run:waiting for 900 2023-09-27T23:14:28.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:14:28.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:28.574 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:14:28.627 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2023-09-27T23:14:28.627 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:14:28.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-09-27T23:14:28.644 INFO:teuthology.orchestra.run:waiting for 900 2023-09-27T23:14:28.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:14:28.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:28.645 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:14:28.694 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:14:28.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:28.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f703325f700 1 Processor -- start 2023-09-27T23:14:28.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f703325f700 1 -- start start 2023-09-27T23:14:28.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f703325f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 0x7f702c067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f703325f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c068c70 con 0x7f702c158f70 2023-09-27T23:14:28.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f703325f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c068060 con 0x7f702c159340 2023-09-27T23:14:28.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f703325f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702c068340 con 0x7f702c069100 2023-09-27T23:14:28.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f702c158f70 0x7f702c0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40080/0 (socket says 172.21.15.64:40080) 2023-09-27T23:14:28.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 -- 172.21.15.64:0/1051950822 learned_addr learned my addr 172.21.15.64:0/1051950822 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865481637 0 0) 0x7f702c068c70 con 0x7f702c158f70 2023-09-27T23:14:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f700c003570 con 0x7f702c158f70 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702bfff700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 0x7f702c067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285678102 0 0) 0x7f700c003570 con 0x7f702c158f70 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 msgr2=0x7f702c067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 0x7f702c067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 >> v1:172.21.15.123:6789/0 conn(0x7f702c159340 legacy=0x7f702c0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c068620 con 0x7f702c158f70 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f70140017f0 con 0x7f702c158f70 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 >> v1:172.21.15.64:6789/0 conn(0x7f702c158f70 legacy=0x7f702c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700c003a90 0x7f700c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 0x7f700c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f700c004da0 0x7f700c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c068340 con 0x7f700c003a90 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00bc20 con 0x7f700c004da0 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00bde0 con 0x7f702c069100 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700c003a90 0x7f700c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f702bfff700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 0x7f700c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f70317fc700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f700c004da0 0x7f700c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 -- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 msgr2=0x7f700c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f702c069100 0x7f700c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 -- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f700c004da0 msgr2=0x7f700c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f700c004da0 0x7f700c0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.806+0000 7f7030ffb700 1 -- 172.21.15.64:0/1051950822 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c003570 con 0x7f700c003a90 2023-09-27T23:14:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f7030ffb700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700c003a90 0x7f700c003ea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f701400a900 tx=0x7f70140021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc279ecb49d9fca server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140070d0 con 0x7f700c003a90 2023-09-27T23:14:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014007250 con 0x7f700c003a90 2023-09-27T23:14:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f702b7fe700 1 -- 172.21.15.64:0/1051950822 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014007610 con 0x7f700c003a90 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 -- 172.21.15.64:0/1051950822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700c003a90 msgr2=0x7f700c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 --2- 172.21.15.64:0/1051950822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f700c003a90 0x7f700c003ea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f701400a900 tx=0x7f70140021d0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f70317fc700 1 -- 172.21.15.64:0/1051950822 reap_dead start 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 -- 172.21.15.64:0/1051950822 shutdown_connections 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 -- 172.21.15.64:0/1051950822 >> 172.21.15.64:0/1051950822 conn(0x7f702c0ccc40 msgr2=0x7f702c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 -- 172.21.15.64:0/1051950822 shutdown_connections 2023-09-27T23:14:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 -- 172.21.15.64:0/1051950822 wait complete. 2023-09-27T23:14:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 Processor -- start 2023-09-27T23:14:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.810+0000 7f703325f700 1 -- start start 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f700c003a90 0x7f702c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f702c14ab80 0x7f702c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f702c14aff0 0x7f702c0b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0b0ea0 con 0x7f702c14ab80 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0b26e0 con 0x7f702c14aff0 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0b1db0 con 0x7f700c003a90 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f702bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f702c14ab80 0x7f702c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f700c003a90 0x7f702c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f702bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f702c14ab80 0x7f702c0afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33404/0 (socket says 172.21.15.64:33404) 2023-09-27T23:14:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f70317fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f702c14aff0 0x7f702c0b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f702bfff700 1 -- 172.21.15.64:0/3229083383 learned_addr learned my addr 172.21.15.64:0/3229083383 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 -- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f702c14aff0 msgr2=0x7f702c0b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f702c14aff0 0x7f702c0b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 -- 172.21.15.64:0/3229083383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f702c14ab80 msgr2=0x7f702c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f702c14ab80 0x7f702c0afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 -- 172.21.15.64:0/3229083383 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0b2090 con 0x7f700c003a90 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7030ffb700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f700c003a90 0x7f702c0b5a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f700c0057f0 tx=0x7f7014016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140070d0 con 0x7f700c003a90 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70140017b0 con 0x7f700c003a90 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c0b2370 con 0x7f700c003a90 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014007980 con 0x7f700c003a90 2023-09-27T23:14:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.814+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c0b0770 con 0x7f700c003a90 2023-09-27T23:14:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.818+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c0b30c0 con 0x7f700c003a90 2023-09-27T23:14:28.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.818+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7014024070 con 0x7f700c003a90 2023-09-27T23:14:28.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.818+0000 7f7028ff9700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7010060950 0x7f7010062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:28.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.818+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f701407f9c0 con 0x7f700c003a90 2023-09-27T23:14:28.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.818+0000 7f702bfff700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7010060950 0x7f7010062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:28.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.818+0000 7f702bfff700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7010060950 0x7f7010062e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f702c15fb70 tx=0x7f701c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:28.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:28.830+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f701404fa50 con 0x7f700c003a90 2023-09-27T23:14:29.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.030+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f702c071120 con 0x7f700c003a90 2023-09-27T23:14:29.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f7028ff9700 1 -- 172.21.15.64:0/3229083383 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f70140075d0 con 0x7f700c003a90 2023-09-27T23:14:29.037 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:29.037 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:29.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7010060950 msgr2=0x7f7010062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7010060950 0x7f7010062e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f702c15fb70 tx=0x7f701c008040 comp rx=0 tx=0).stop 2023-09-27T23:14:29.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f700c003a90 msgr2=0x7f702c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f700c003a90 0x7f702c0b5a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f700c0057f0 tx=0x7f7014016000 comp rx=0 tx=0).stop 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 shutdown_connections 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7010060950 0x7f7010062e00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f702c14aff0 0x7f702c0b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f702c14ab80 0x7f702c0afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 --2- 172.21.15.64:0/3229083383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f700c003a90 0x7f702c0b5a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 >> 172.21.15.64:0/3229083383 conn(0x7f702c0ccc40 msgr2=0x7f702c14dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 shutdown_connections 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.034+0000 7f703325f700 1 -- 172.21.15.64:0/3229083383 wait complete. 2023-09-27T23:14:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:29.050 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa00759e700 1 Processor -- start 2023-09-27T23:14:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa00759e700 1 -- start start 2023-09-27T23:14:29.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa00759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7fa00806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa00759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00806b730 con 0x7fa0081591f0 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa00759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00806ab20 con 0x7fa0080ce810 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa00759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806ae00 con 0x7fa008158e20 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa005d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa0081591f0 0x7fa0080ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40094/0 (socket says 172.21.15.64:40094) 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.150+0000 7fa005d9b700 1 -- 172.21.15.64:0/2520298790 learned_addr learned my addr 172.21.15.64:0/2520298790 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00659c700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7fa00806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113714992 0 0) 0x7fa00806b730 con 0x7fa0081591f0 2023-09-27T23:14:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ff0003570 con 0x7fa0081591f0 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888194644 0 0) 0x7fa00806ab20 con 0x7fa0080ce810 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa00806b730 con 0x7fa0080ce810 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096248291 0 0) 0x7f9ff0003570 con 0x7fa0081591f0 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 msgr2=0x7fa00806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7fa00806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 >> v1:172.21.15.123:6789/0 conn(0x7fa0080ce810 legacy=0x7fa0080ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00806b0e0 con 0x7fa0081591f0 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f9ff8001470 con 0x7fa0081591f0 2023-09-27T23:14:29.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 >> v1:172.21.15.64:6789/0 conn(0x7fa0081591f0 legacy=0x7fa0080ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff0003c20 0x7f9ff0003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7f9ff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7f9ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806ae00 con 0x7f9ff0004810 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff000bb10 con 0x7f9ff0003c20 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff000bca0 con 0x7fa008158e20 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7f9ff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00659c700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff0003c20 0x7f9ff0003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa006d9d700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7f9ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 -- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 msgr2=0x7f9ff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:29.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7f9ff00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 -- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff0003c20 msgr2=0x7f9ff0003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff0003c20 0x7f9ff0003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 -- 172.21.15.64:0/2520298790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0003570 con 0x7f9ff0004810 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa006d9d700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008158e20 0x7f9ff00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa005d9b700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7f9ff0004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa0081595c0 tx=0x7f9ff8007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b3a50de96436ef server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800ce90 con 0x7f9ff0004810 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff8005920 con 0x7f9ff0004810 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00559a700 1 -- 172.21.15.64:0/2520298790 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8005ce0 con 0x7f9ff0004810 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00759e700 1 -- 172.21.15.64:0/2520298790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 msgr2=0x7f9ff0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00759e700 1 --2- 172.21.15.64:0/2520298790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7f9ff0004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa0081595c0 tx=0x7f9ff8007b60 comp rx=0 tx=0).stop 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa006d9d700 1 -- 172.21.15.64:0/2520298790 reap_dead start 2023-09-27T23:14:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00759e700 1 -- 172.21.15.64:0/2520298790 shutdown_connections 2023-09-27T23:14:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00759e700 1 -- 172.21.15.64:0/2520298790 >> 172.21.15.64:0/2520298790 conn(0x7fa008067bb0 msgr2=0x7fa0080cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00759e700 1 -- 172.21.15.64:0/2520298790 shutdown_connections 2023-09-27T23:14:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.154+0000 7fa00759e700 1 -- 172.21.15.64:0/2520298790 wait complete. 2023-09-27T23:14:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 Processor -- start 2023-09-27T23:14:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 -- start start 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7fa0080c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0081496c0 0x7fa00814b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008149b30 0x7fa00814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080c82c0 con 0x7f9ff0004810 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080c9120 con 0x7fa008149b30 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080c8510 con 0x7fa0081496c0 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.158+0000 7fa00659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7fa0080c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7fa0080c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33412/0 (socket says 172.21.15.64:33412) 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 -- 172.21.15.64:0/4201476879 learned_addr learned my addr 172.21.15.64:0/4201476879 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa005d9b700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0081496c0 0x7fa00814b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa006d9d700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008149b30 0x7fa00814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 -- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0081496c0 msgr2=0x7fa00814b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0081496c0 0x7fa00814b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 -- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008149b30 msgr2=0x7fa00814b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008149b30 0x7fa00814b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 -- 172.21.15.64:0/4201476879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0080c87f0 con 0x7f9ff0004810 2023-09-27T23:14:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa006d9d700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008149b30 0x7fa00814b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00659c700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7fa0080c95b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc011d70 tx=0x7f9ffc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00fac0 con 0x7f9ff0004810 2023-09-27T23:14:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc00fc20 con 0x7f9ff0004810 2023-09-27T23:14:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0080c8ad0 con 0x7f9ff0004810 2023-09-27T23:14:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc014440 con 0x7f9ff0004810 2023-09-27T23:14:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.162+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0080c9af0 con 0x7f9ff0004810 2023-09-27T23:14:29.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.166+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f9ffc014870 con 0x7f9ff0004810 2023-09-27T23:14:29.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.166+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00804be40 con 0x7f9ff0004810 2023-09-27T23:14:29.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.166+0000 7f9ff67fc700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fe8060980 0x7f9fe8062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.166+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f9ffc018030 con 0x7f9ff0004810 2023-09-27T23:14:29.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.166+0000 7fa005d9b700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fe8060980 0x7f9fe8062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.166+0000 7fa005d9b700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fe8060980 0x7f9fe8062e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa00816cbd0 tx=0x7f9ff8015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.174+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9ffc04da70 con 0x7f9ff0004810 2023-09-27T23:14:29.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.334+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa00806b3c0 con 0x7f9ff0004810 2023-09-27T23:14:29.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7f9ff67fc700 1 -- 172.21.15.64:0/4201476879 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f9ffc022090 con 0x7f9ff0004810 2023-09-27T23:14:29.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:29.341 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:27.041505+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.35","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:29.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fe8060980 msgr2=0x7f9fe8062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fe8060980 0x7f9fe8062e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa00816cbd0 tx=0x7f9ff8015000 comp rx=0 tx=0).stop 2023-09-27T23:14:29.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 msgr2=0x7fa0080c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7fa0080c95b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc011d70 tx=0x7f9ffc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 shutdown_connections 2023-09-27T23:14:29.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fe8060980 0x7f9fe8062e30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008149b30 0x7fa00814b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0081496c0 0x7fa00814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 --2- 172.21.15.64:0/4201476879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0004810 0x7fa0080c95b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 >> 172.21.15.64:0/4201476879 conn(0x7fa008067bb0 msgr2=0x7fa0080cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 shutdown_connections 2023-09-27T23:14:29.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.338+0000 7fa00759e700 1 -- 172.21.15.64:0/4201476879 wait complete. 2023-09-27T23:14:29.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:13:43.256567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:13:43.753338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-27T23:13:46.825262+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:14:29.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb05aeeb700 1 Processor -- start 2023-09-27T23:14:29.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb05aeeb700 1 -- start start 2023-09-27T23:14:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb05aeeb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb0540cecd0 0x7fb05406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb05aeeb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cbdf0 con 0x7fb0540cb970 2023-09-27T23:14:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb05aeeb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cc0d0 con 0x7fb054158e20 2023-09-27T23:14:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb05aeeb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540cc3b0 con 0x7fb0540cecd0 2023-09-27T23:14:29.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb04bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb0540cb970 0x7fb05406acc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40102/0 (socket says 172.21.15.64:40102) 2023-09-27T23:14:29.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb04bfff700 1 -- 172.21.15.64:0/838367030 learned_addr learned my addr 172.21.15.64:0/838367030 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:29.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb059488700 1 --2- 172.21.15.64:0/838367030 >> v2:172.21.15.123:3300/0 conn(0x7fb0540cecd0 0x7fb05406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641637287 0 0) 0x7fb0540cbdf0 con 0x7fb0540cb970 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb03c003570 con 0x7fb0540cb970 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072261352 0 0) 0x7fb0540cc0d0 con 0x7fb054158e20 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0540cbdf0 con 0x7fb054158e20 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb059488700 1 -- 172.21.15.64:0/838367030 >> v1:172.21.15.123:6789/0 conn(0x7fb054158e20 legacy=0x7fb0540ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb059488700 1 -- 172.21.15.64:0/838367030 >> v1:172.21.15.64:6789/0 conn(0x7fb0540cb970 legacy=0x7fb05406acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.458+0000 7fb059488700 1 -- 172.21.15.64:0/838367030 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540cc690 con 0x7fb0540cecd0 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb059488700 1 --2- 172.21.15.64:0/838367030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0540cecd0 0x7fb05406b3d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009e90 tx=0x7fb04c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2adb557c2025d91a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00e070 con 0x7fb0540cecd0 2023-09-27T23:14:29.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c00bb90 con 0x7fb0540cecd0 2023-09-27T23:14:29.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb04b7fe700 1 -- 172.21.15.64:0/838367030 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c012430 con 0x7fb0540cecd0 2023-09-27T23:14:29.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 -- 172.21.15.64:0/838367030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0540cecd0 msgr2=0x7fb05406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/838367030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0540cecd0 0x7fb05406b3d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009e90 tx=0x7fb04c009780 comp rx=0 tx=0).stop 2023-09-27T23:14:29.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 -- 172.21.15.64:0/838367030 shutdown_connections 2023-09-27T23:14:29.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/838367030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0540cecd0 0x7fb05406b3d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 -- 172.21.15.64:0/838367030 >> 172.21.15.64:0/838367030 conn(0x7fb054067bb0 msgr2=0x7fb0540d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 -- 172.21.15.64:0/838367030 shutdown_connections 2023-09-27T23:14:29.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 -- 172.21.15.64:0/838367030 wait complete. 2023-09-27T23:14:29.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 Processor -- start 2023-09-27T23:14:29.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.462+0000 7fb05aeeb700 1 -- start start 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0540cecd0 0x7fb05414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb054158e20 0x7fb054155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb054156410 0x7fb0540c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054156890 con 0x7fb054156410 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540c9d30 con 0x7fb054158e20 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05414fff0 con 0x7fb0540cecd0 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb059488700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb054156410 0x7fb0540c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb059488700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb054156410 0x7fb0540c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33420/0 (socket says 172.21.15.64:33420) 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0540cecd0 0x7fb05414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb059488700 1 -- 172.21.15.64:0/68922916 learned_addr learned my addr 172.21.15.64:0/68922916 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:29.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb04bfff700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb054158e20 0x7fb054155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 -- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb054158e20 msgr2=0x7fb054155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb054158e20 0x7fb054155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 -- 172.21.15.64:0/68922916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb054156410 msgr2=0x7fb0540c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb054156410 0x7fb0540c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 -- 172.21.15.64:0/68922916 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c009a20 con 0x7fb0540cecd0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb058c87700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0540cecd0 0x7fb05414bc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c590 tx=0x7fb050007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050011590 con 0x7fb0540cecd0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0500116f0 con 0x7fb0540cecd0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540c9120 con 0x7fb0540cecd0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050011ab0 con 0x7fb0540cecd0 2023-09-27T23:14:29.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540c96e0 con 0x7fb0540cecd0 2023-09-27T23:14:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.466+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054071120 con 0x7fb0540cecd0 2023-09-27T23:14:29.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.470+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb05001c460 con 0x7fb0540cecd0 2023-09-27T23:14:29.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.470+0000 7fb048ff9700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb040060980 0x7fb040062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.470+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fb05007d3f0 con 0x7fb0540cecd0 2023-09-27T23:14:29.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.470+0000 7fb04bfff700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb040060980 0x7fb040062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.470+0000 7fb04bfff700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb040060980 0x7fb040062e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb05416c650 tx=0x7fb044009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.482+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb05004d320 con 0x7fb0540cecd0 2023-09-27T23:14:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.646+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb05406a7a0 con 0x7fb0540cecd0 2023-09-27T23:14:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.646+0000 7fb048ff9700 1 -- 172.21.15.64:0/68922916 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb050050940 con 0x7fb0540cecd0 2023-09-27T23:14:29.650 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb040060980 msgr2=0x7fb040062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb040060980 0x7fb040062e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb05416c650 tx=0x7fb044009040 comp rx=0 tx=0).stop 2023-09-27T23:14:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0540cecd0 msgr2=0x7fb05414bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0540cecd0 0x7fb05414bc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c590 tx=0x7fb050007d50 comp rx=0 tx=0).stop 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 shutdown_connections 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb040060980 0x7fb040062e30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb054156410 0x7fb0540c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb054158e20 0x7fb054155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 --2- 172.21.15.64:0/68922916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0540cecd0 0x7fb05414bc90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 >> 172.21.15.64:0/68922916 conn(0x7fb054067bb0 msgr2=0x7fb05414caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 shutdown_connections 2023-09-27T23:14:29.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.650+0000 7fb05aeeb700 1 -- 172.21.15.64:0/68922916 wait complete. 2023-09-27T23:14:29.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.782+0000 7fc0aeec2700 1 Processor -- start 2023-09-27T23:14:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.782+0000 7fc0aeec2700 1 -- start start 2023-09-27T23:14:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0a813f6a0 0x7fc0a80ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a80c7780 0x7fc0a80c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a80adc60 con 0x7fc0a813f2d0 2023-09-27T23:14:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a806d7d0 con 0x7fc0a813f6a0 2023-09-27T23:14:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80c8140 con 0x7fc0a80c7780 2023-09-27T23:14:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0a813f6a0 0x7fc0a80ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0a813f6a0 0x7fc0a80ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33430/0 (socket says 172.21.15.64:33430) 2023-09-27T23:14:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/2630356870 learned_addr learned my addr 172.21.15.64:0/2630356870 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0adec0700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc0a813f2d0 0x7fc0a815a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40108/0 (socket says 172.21.15.64:40108) 2023-09-27T23:14:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ae6c1700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a80c7780 0x7fc0a80c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/2630356870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a80c7780 msgr2=0x7fc0a80c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a80c7780 0x7fc0a80c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/2630356870 >> v1:172.21.15.64:6789/0 conn(0x7fc0a813f2d0 legacy=0x7fc0a815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/2630356870 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a80ad050 con 0x7fc0a813f6a0 2023-09-27T23:14:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ae6c1700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a80c7780 0x7fc0a80c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0ad6bf700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f6a0 0x7fc0a80ac9b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc0980099e0 tx=0x7fc098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95afd87c409eaea server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0acebe700 1 -- 172.21.15.64:0/2630356870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800b820 con 0x7fc0a813f6a0 2023-09-27T23:14:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0acebe700 1 -- 172.21.15.64:0/2630356870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09800b980 con 0x7fc0a813f6a0 2023-09-27T23:14:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0acebe700 1 -- 172.21.15.64:0/2630356870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800bd20 con 0x7fc0a813f6a0 2023-09-27T23:14:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- 172.21.15.64:0/2630356870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f6a0 msgr2=0x7fc0a80ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f6a0 0x7fc0a80ac9b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc0980099e0 tx=0x7fc098009250 comp rx=0 tx=0).stop 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- 172.21.15.64:0/2630356870 shutdown_connections 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a80c7780 0x7fc0a80c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/2630356870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f6a0 0x7fc0a80ac9b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- 172.21.15.64:0/2630356870 >> 172.21.15.64:0/2630356870 conn(0x7fc0a8067b50 msgr2=0x7fc0a80cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- 172.21.15.64:0/2630356870 shutdown_connections 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.786+0000 7fc0aeec2700 1 -- 172.21.15.64:0/2630356870 wait complete. 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 Processor -- start 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 -- start start 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a80c7780 0x7fc0a8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 0x7fc0a80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a813f6a0 0x7fc0a816bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a815a580 con 0x7fc0a813f2d0 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a8159970 con 0x7fc0a80c7780 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a8159c50 con 0x7fc0a813f6a0 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 0x7fc0a80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 0x7fc0a80ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33438/0 (socket says 172.21.15.64:33438) 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/1954616635 learned_addr learned my addr 172.21.15.64:0/1954616635 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0adec0700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a80c7780 0x7fc0a8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ae6c1700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a813f6a0 0x7fc0a816bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a813f6a0 msgr2=0x7fc0a816bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a813f6a0 0x7fc0a816bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a80c7780 msgr2=0x7fc0a8159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a80c7780 0x7fc0a8159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 -- 172.21.15.64:0/1954616635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980094f0 con 0x7fc0a813f2d0 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0ad6bf700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 0x7fc0a80ca970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc0980090b0 tx=0x7fc098017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098011d70 con 0x7fc0a813f2d0 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc098026450 con 0x7fc0a813f2d0 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a8159f30 con 0x7fc0a813f2d0 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098026810 con 0x7fc0a813f2d0 2023-09-27T23:14:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.790+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a81595b0 con 0x7fc0a813f2d0 2023-09-27T23:14:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.794+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a804be40 con 0x7fc0a813f2d0 2023-09-27T23:14:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.794+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fc098026bc0 con 0x7fc0a813f2d0 2023-09-27T23:14:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.794+0000 7fc09dffb700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc08c060950 0x7fc08c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.794+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fc09807ed10 con 0x7fc0a813f2d0 2023-09-27T23:14:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.798+0000 7fc0adec0700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc08c060950 0x7fc08c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.798+0000 7fc0adec0700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc08c060950 0x7fc08c062e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80cb660 tx=0x7fc0a4007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.802+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc098084020 con 0x7fc0a813f2d0 2023-09-27T23:14:29.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.966+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc0a806ba60 con 0x7fc0a813f2d0 2023-09-27T23:14:29.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.966+0000 7fc09dffb700 1 -- 172.21.15.64:0/1954616635 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc09804ec40 con 0x7fc0a813f2d0 2023-09-27T23:14:29.971 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc08c060950 msgr2=0x7fc08c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc08c060950 0x7fc08c062e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80cb660 tx=0x7fc0a4007eb0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 msgr2=0x7fc0a80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 0x7fc0a80ca970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc0980090b0 tx=0x7fc098017000 comp rx=0 tx=0).stop 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 shutdown_connections 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc08c060950 0x7fc08c062e00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a813f6a0 0x7fc0a816bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0a813f2d0 0x7fc0a80ca970 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 --2- 172.21.15.64:0/1954616635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a80c7780 0x7fc0a8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 >> 172.21.15.64:0/1954616635 conn(0x7fc0a8067b50 msgr2=0x7fc0a80c6150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 shutdown_connections 2023-09-27T23:14:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:29.970+0000 7fc0aeec2700 1 -- 172.21.15.64:0/1954616635 wait complete. 2023-09-27T23:14:29.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:14:30.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 Processor -- start 2023-09-27T23:14:30.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 -- start start 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9490150420 0x7f94900c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9490156e80 0x7f94900ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94900af4c0 con 0x7f94900aecb0 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94900cae80 con 0x7f9490150420 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9495073700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94900af1e0 con 0x7f9490156e80 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9490150420 0x7f94900c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9494872700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f94900aecb0 0x7f94900ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40112/0 (socket says 172.21.15.64:40112) 2023-09-27T23:14:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f9494872700 1 -- 172.21.15.64:0/3589026700 learned_addr learned my addr 172.21.15.64:0/3589026700 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:30.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948f7fe700 1 --2- 172.21.15.64:0/3589026700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9490156e80 0x7f94900ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948ffff700 1 -- 172.21.15.64:0/3589026700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9490156e80 msgr2=0x7f94900ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948ffff700 1 --2- 172.21.15.64:0/3589026700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9490156e80 0x7f94900ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948ffff700 1 -- 172.21.15.64:0/3589026700 >> v1:172.21.15.64:6789/0 conn(0x7f94900aecb0 legacy=0x7f94900ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948ffff700 1 -- 172.21.15.64:0/3589026700 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900ca830 con 0x7f9490150420 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948effd700 1 -- 172.21.15.64:0/3589026700 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989307534 0 0) 0x7f94900af4c0 con 0x7f94900aecb0 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948ffff700 1 --2- 172.21.15.64:0/3589026700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f948000bb90 tx=0x7f948000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e98e7f65950b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948effd700 1 -- 172.21.15.64:0/3589026700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000db40 con 0x7f9490150420 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948effd700 1 -- 172.21.15.64:0/3589026700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948000dca0 con 0x7f9490150420 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.094+0000 7f948effd700 1 -- 172.21.15.64:0/3589026700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800196b0 con 0x7f9490150420 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- 172.21.15.64:0/3589026700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 msgr2=0x7f94900c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 --2- 172.21.15.64:0/3589026700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f948000bb90 tx=0x7f948000b430 comp rx=0 tx=0).stop 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- 172.21.15.64:0/3589026700 shutdown_connections 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 --2- 172.21.15.64:0/3589026700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9490156e80 0x7f94900ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 --2- 172.21.15.64:0/3589026700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- 172.21.15.64:0/3589026700 >> 172.21.15.64:0/3589026700 conn(0x7f94900ccb30 msgr2=0x7f94900ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- 172.21.15.64:0/3589026700 shutdown_connections 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- 172.21.15.64:0/3589026700 wait complete. 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 Processor -- start 2023-09-27T23:14:30.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- start start 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94900aecb0 0x7f949014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9490156e80 0x7f94900c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949014d9e0 con 0x7f9490150420 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490158a50 con 0x7f9490156e80 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9495073700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9490158d30 con 0x7f94900aecb0 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f948f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f948f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33456/0 (socket says 172.21.15.64:33456) 2023-09-27T23:14:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f948f7fe700 1 -- 172.21.15.64:0/3716949758 learned_addr learned my addr 172.21.15.64:0/3716949758 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f9494872700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9490156e80 0x7f94900c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f948ffff700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94900aecb0 0x7f949014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f948f7fe700 1 -- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94900aecb0 msgr2=0x7f949014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.098+0000 7f948f7fe700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94900aecb0 0x7f949014d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f948f7fe700 1 -- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9490156e80 msgr2=0x7f94900c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f948f7fe700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9490156e80 0x7f94900c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f948f7fe700 1 -- 172.21.15.64:0/3716949758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948000b6d0 con 0x7f9490150420 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f948ffff700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94900aecb0 0x7f949014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f948f7fe700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9484009e70 tx=0x7f948400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400cb50 con 0x7f9490150420 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948400f040 con 0x7f9490150420 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9490159010 con 0x7f9490150420 2023-09-27T23:14:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484014480 con 0x7f9490150420 2023-09-27T23:14:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94900afe70 con 0x7f9490150420 2023-09-27T23:14:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94900b00f0 con 0x7f9490150420 2023-09-27T23:14:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f9484014830 con 0x7f9490150420 2023-09-27T23:14:30.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.102+0000 7f9477fff700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f947c060950 0x7f947c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.106+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f948407dae0 con 0x7f9490150420 2023-09-27T23:14:30.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.106+0000 7f948ffff700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f947c060950 0x7f947c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.106+0000 7f948ffff700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f947c060950 0x7f947c062e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f948000b6a0 tx=0x7f948001b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.114+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f948404da30 con 0x7f9490150420 2023-09-27T23:14:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.270+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f949004be40 con 0x7f9490150420 2023-09-27T23:14:30.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9477fff700 1 -- 172.21.15.64:0/3716949758 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9484051050 con 0x7f9490150420 2023-09-27T23:14:30.277 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f947c060950 msgr2=0x7f947c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f947c060950 0x7f947c062e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f948000b6a0 tx=0x7f948001b000 comp rx=0 tx=0).stop 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 msgr2=0x7f94900c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9484009e70 tx=0x7f948400b040 comp rx=0 tx=0).stop 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 shutdown_connections 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f947c060950 0x7f947c062e00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9490156e80 0x7f94900c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9490150420 0x7f94900c7bf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 --2- 172.21.15.64:0/3716949758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94900aecb0 0x7f949014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 >> 172.21.15.64:0/3716949758 conn(0x7f94900ccb30 msgr2=0x7f94900cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 shutdown_connections 2023-09-27T23:14:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.274+0000 7f9495073700 1 -- 172.21.15.64:0/3716949758 wait complete. 2023-09-27T23:14:30.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:14:30.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 Processor -- start 2023-09-27T23:14:30.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 -- start start 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3040acf30 0x7fb3041557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb30406d1d0 0x7fb3040ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb304156780 con 0x7fb3040ad300 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3041532a0 con 0x7fb3040acf30 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb303fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb304158f90 con 0x7fb30406d1d0 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb3037fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb30406d1d0 0x7fb3040ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb3027fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb3040ad300 0x7fb304159420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:57346/0 (socket says 172.21.15.64:57346) 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb302ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3040acf30 0x7fb3041557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb3027fc700 1 -- 172.21.15.64:0/1276047782 learned_addr learned my addr 172.21.15.64:0/1276047782 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb301ffb700 1 -- 172.21.15.64:0/1276047782 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044707169 0 0) 0x7fb304156780 con 0x7fb3040ad300 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb301ffb700 1 -- 172.21.15.64:0/1276047782 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ec003570 con 0x7fb3040ad300 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.402+0000 7fb3037fe700 1 -- 172.21.15.64:0/1276047782 >> v2:172.21.15.123:3300/0 conn(0x7fb3040acf30 msgr2=0x7fb3041557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb3037fe700 1 --2- 172.21.15.64:0/1276047782 >> v2:172.21.15.123:3300/0 conn(0x7fb3040acf30 0x7fb3041557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb3037fe700 1 -- 172.21.15.64:0/1276047782 >> v1:172.21.15.123:6789/0 conn(0x7fb3040ad300 legacy=0x7fb304159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb3037fe700 1 -- 172.21.15.64:0/1276047782 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30406d650 con 0x7fb30406d1d0 2023-09-27T23:14:30.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb302ffd700 1 --2- 172.21.15.64:0/1276047782 >> v2:172.21.15.123:3300/0 conn(0x7fb3040acf30 0x7fb3041557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:30.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb3037fe700 1 --2- 172.21.15.64:0/1276047782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb30406d1d0 0x7fb3040ac1e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009e60 tx=0x7fb2fc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694c5fad850b24e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb301ffb700 1 -- 172.21.15.64:0/1276047782 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00e070 con 0x7fb30406d1d0 2023-09-27T23:14:30.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb301ffb700 1 -- 172.21.15.64:0/1276047782 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc00bab0 con 0x7fb30406d1d0 2023-09-27T23:14:30.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb301ffb700 1 -- 172.21.15.64:0/1276047782 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00be70 con 0x7fb30406d1d0 2023-09-27T23:14:30.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 -- 172.21.15.64:0/1276047782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb30406d1d0 msgr2=0x7fb3040ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 --2- 172.21.15.64:0/1276047782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb30406d1d0 0x7fb3040ac1e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009e60 tx=0x7fb2fc009780 comp rx=0 tx=0).stop 2023-09-27T23:14:30.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 -- 172.21.15.64:0/1276047782 shutdown_connections 2023-09-27T23:14:30.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 --2- 172.21.15.64:0/1276047782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb30406d1d0 0x7fb3040ac1e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 --2- 172.21.15.64:0/1276047782 >> v2:172.21.15.123:3300/0 conn(0x7fb3040acf30 0x7fb3041557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 -- 172.21.15.64:0/1276047782 >> 172.21.15.64:0/1276047782 conn(0x7fb304067b50 msgr2=0x7fb3040cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 -- 172.21.15.64:0/1276047782 shutdown_connections 2023-09-27T23:14:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.406+0000 7fb303fff700 1 -- 172.21.15.64:0/1276047782 wait complete. 2023-09-27T23:14:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 Processor -- start 2023-09-27T23:14:30.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 -- start start 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb30406d1d0 0x7fb304152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3040acf30 0x7fb30413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 0x7fb3041533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb304156640 con 0x7fb3040ad300 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb304156920 con 0x7fb30406d1d0 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb304153c40 con 0x7fb3040acf30 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 0x7fb3041533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 0x7fb3041533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33472/0 (socket says 172.21.15.64:33472) 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb302ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb30406d1d0 0x7fb304152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 -- 172.21.15.64:0/3503616004 learned_addr learned my addr 172.21.15.64:0/3503616004 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3027fc700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3040acf30 0x7fb30413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 -- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3040acf30 msgr2=0x7fb30413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3040acf30 0x7fb30413f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 -- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb30406d1d0 msgr2=0x7fb304152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb30406d1d0 0x7fb304152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 -- 172.21.15.64:0/3503616004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f4009420 con 0x7fb3040ad300 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb3037fe700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 0x7fb3041533f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00b610 tx=0x7fb2fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc012d70 con 0x7fb3040ad300 2023-09-27T23:14:30.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc009a20 con 0x7fb3040ad300 2023-09-27T23:14:30.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc027470 con 0x7fb3040ad300 2023-09-27T23:14:30.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc027830 con 0x7fb3040ad300 2023-09-27T23:14:30.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.410+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb304154200 con 0x7fb3040ad300 2023-09-27T23:14:30.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.414+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb304156ab0 con 0x7fb3040ad300 2023-09-27T23:14:30.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.414+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb2fc024070 con 0x7fb3040ad300 2023-09-27T23:14:30.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.414+0000 7fb2eaffd700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb2f0060950 0x7fb2f0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.414+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fb2fc07f7c0 con 0x7fb3040ad300 2023-09-27T23:14:30.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.414+0000 7fb302ffd700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb2f0060950 0x7fb2f0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.414+0000 7fb302ffd700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb2f0060950 0x7fb2f0062e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb304154c90 tx=0x7fb2f40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.418+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb2fc04f710 con 0x7fb3040ad300 2023-09-27T23:14:30.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.586+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb30404be40 con 0x7fb3040ad300 2023-09-27T23:14:30.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.586+0000 7fb2eaffd700 1 -- 172.21.15.64:0/3503616004 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2fc02c020 con 0x7fb3040ad300 2023-09-27T23:14:30.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:14:30.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb2f0060950 msgr2=0x7fb2f0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb2f0060950 0x7fb2f0062e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb304154c90 tx=0x7fb2f40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 msgr2=0x7fb3041533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 0x7fb3041533f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00b610 tx=0x7fb2fc019000 comp rx=0 tx=0).stop 2023-09-27T23:14:30.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 shutdown_connections 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb2f0060950 0x7fb2f0062e00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3040ad300 0x7fb3041533f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3040acf30 0x7fb30413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 --2- 172.21.15.64:0/3503616004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb30406d1d0 0x7fb304152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 >> 172.21.15.64:0/3503616004 conn(0x7fb304067b50 msgr2=0x7fb3040c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 shutdown_connections 2023-09-27T23:14:30.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.590+0000 7fb303fff700 1 -- 172.21.15.64:0/3503616004 wait complete. 2023-09-27T23:14:30.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-27T23:14:30.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 Processor -- start 2023-09-27T23:14:30.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 -- start start 2023-09-27T23:14:30.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f618c150200 0x7f618c0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f618c0ae770 0x7f618c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c0af250 con 0x7f618c0ac720 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0af530 con 0x7f618c150200 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6192579700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0ca8a0 con 0x7f618c0ae770 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f618c150200 0x7f618c0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f618c150200 0x7f618c0c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33474/0 (socket says 172.21.15.64:33474) 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 -- 172.21.15.64:0/347978902 learned_addr learned my addr 172.21.15.64:0/347978902 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6190d76700 1 --2- 172.21.15.64:0/347978902 >> v2:172.21.15.123:3300/0 conn(0x7f618c0ae770 0x7f618c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 -- 172.21.15.64:0/347978902 >> v2:172.21.15.123:3300/0 conn(0x7f618c0ae770 msgr2=0x7f618c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 --2- 172.21.15.64:0/347978902 >> v2:172.21.15.123:3300/0 conn(0x7f618c0ae770 0x7f618c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 -- 172.21.15.64:0/347978902 >> v1:172.21.15.123:6789/0 conn(0x7f618c0ac720 legacy=0x7f618c0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:30.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 -- 172.21.15.64:0/347978902 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0cab80 con 0x7f618c150200 2023-09-27T23:14:30.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6190d76700 1 --2- 172.21.15.64:0/347978902 >> v2:172.21.15.123:3300/0 conn(0x7f618c0ae770 0x7f618c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:30.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6191577700 1 --2- 172.21.15.64:0/347978902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c150200 0x7f618c0c7170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f618c06b3d0 tx=0x7f6188009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759e76e096a4ff49 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6183fff700 1 -- 172.21.15.64:0/347978902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800b820 con 0x7f618c150200 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6183fff700 1 -- 172.21.15.64:0/347978902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618800b980 con 0x7f618c150200 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.718+0000 7f6183fff700 1 -- 172.21.15.64:0/347978902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800bd20 con 0x7f618c150200 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 -- 172.21.15.64:0/347978902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c150200 msgr2=0x7f618c0c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 --2- 172.21.15.64:0/347978902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c150200 0x7f618c0c7170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f618c06b3d0 tx=0x7f6188009250 comp rx=0 tx=0).stop 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 -- 172.21.15.64:0/347978902 shutdown_connections 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 --2- 172.21.15.64:0/347978902 >> v2:172.21.15.123:3300/0 conn(0x7f618c0ae770 0x7f618c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 --2- 172.21.15.64:0/347978902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c150200 0x7f618c0c7170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 -- 172.21.15.64:0/347978902 >> 172.21.15.64:0/347978902 conn(0x7f618c0ccb30 msgr2=0x7f618c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 -- 172.21.15.64:0/347978902 shutdown_connections 2023-09-27T23:14:30.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 -- 172.21.15.64:0/347978902 wait complete. 2023-09-27T23:14:30.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 Processor -- start 2023-09-27T23:14:30.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.722+0000 7f6192579700 1 -- start start 2023-09-27T23:14:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 0x7f618c16c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f618c0ae770 0x7f618c168960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f618c150200 0x7f618c169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0c5b70 con 0x7f618c0ac720 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1691b0 con 0x7f618c150200 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c169490 con 0x7f618c0ae770 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 0x7f618c16c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 0x7f618c16c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33490/0 (socket says 172.21.15.64:33490) 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 -- 172.21.15.64:0/1159457999 learned_addr learned my addr 172.21.15.64:0/1159457999 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:30.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191d78700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f618c150200 0x7f618c169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6190d76700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f618c0ae770 0x7f618c168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 -- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f618c0ae770 msgr2=0x7f618c168960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f618c0ae770 0x7f618c168960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 -- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f618c150200 msgr2=0x7f618c169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f618c150200 0x7f618c169a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 -- 172.21.15.64:0/1159457999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880094f0 con 0x7f618c0ac720 2023-09-27T23:14:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6191577700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 0x7f618c16c9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61880030c0 tx=0x7f6188016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188011d00 con 0x7f618c0ac720 2023-09-27T23:14:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6188020070 con 0x7f618c0ac720 2023-09-27T23:14:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618801b6a0 con 0x7f618c0ac720 2023-09-27T23:14:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c169770 con 0x7f618c0ac720 2023-09-27T23:14:30.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.726+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c169f90 con 0x7f618c0ac720 2023-09-27T23:14:30.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.730+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c04be40 con 0x7f618c0ac720 2023-09-27T23:14:30.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.730+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6188022030 con 0x7f618c0ac720 2023-09-27T23:14:30.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.730+0000 7f61817fa700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6170060950 0x7f6170062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:30.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.730+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f618807eff0 con 0x7f618c0ac720 2023-09-27T23:14:30.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.730+0000 7f6190d76700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6170060950 0x7f6170062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:30.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.730+0000 7f6190d76700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6170060950 0x7f6170062e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f618c14fbc0 tx=0x7f617c006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:30.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.738+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f618804ef40 con 0x7f618c0ac720 2023-09-27T23:14:30.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.898+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f618c04be40 con 0x7f618c0ac720 2023-09-27T23:14:30.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f61817fa700 1 -- 172.21.15.64:0/1159457999 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v31) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6188052560 con 0x7f618c0ac720 2023-09-27T23:14:30.905 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:14:30.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6170060950 msgr2=0x7f6170062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6170060950 0x7f6170062e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f618c14fbc0 tx=0x7f617c006f90 comp rx=0 tx=0).stop 2023-09-27T23:14:30.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 msgr2=0x7f618c16c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 0x7f618c16c9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61880030c0 tx=0x7f6188016000 comp rx=0 tx=0).stop 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 shutdown_connections 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6170060950 0x7f6170062e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f618c150200 0x7f618c169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f618c0ae770 0x7f618c168960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 --2- 172.21.15.64:0/1159457999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c0ac720 0x7f618c16c9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 >> 172.21.15.64:0/1159457999 conn(0x7f618c0ccb30 msgr2=0x7f618c0c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:30.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 shutdown_connections 2023-09-27T23:14:30.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:30.902+0000 7f6192579700 1 -- 172.21.15.64:0/1159457999 wait complete. 2023-09-27T23:14:30.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:31.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 Processor -- start 2023-09-27T23:14:31.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- start start 2023-09-27T23:14:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310815c2d0 0x7f31081591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3108159710 0x7f310806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 0x7f310815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310815c6a0 con 0x7f310806d590 2023-09-27T23:14:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3108159b60 con 0x7f3108159710 2023-09-27T23:14:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310806da10 con 0x7f310815c2d0 2023-09-27T23:14:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310ca36700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 0x7f310815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310ca36700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 0x7f310815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33498/0 (socket says 172.21.15.64:33498) 2023-09-27T23:14:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310ca36700 1 -- 172.21.15.64:0/3992222685 learned_addr learned my addr 172.21.15.64:0/3992222685 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 --2- 172.21.15.64:0/3992222685 >> v2:172.21.15.123:3300/0 conn(0x7f3108159710 0x7f310806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f3107fff700 1 --2- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310815c2d0 0x7f31081591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 -- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310815c2d0 msgr2=0x7f31081591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 --2- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310815c2d0 0x7f31081591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 -- 172.21.15.64:0/3992222685 >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 msgr2=0x7f310815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 --2- 172.21.15.64:0/3992222685 >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 0x7f310815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 -- 172.21.15.64:0/3992222685 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310815a780 con 0x7f3108159710 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310ca36700 1 --2- 172.21.15.64:0/3992222685 >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 0x7f310815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f30fffff700 1 --2- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3108159710 0x7f310806d050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f310806ad30 tx=0x7f30f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ad9b0dd0896019 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f31077fe700 1 -- 172.21.15.64:0/3992222685 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0011630 con 0x7f3108159710 2023-09-27T23:14:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f31077fe700 1 -- 172.21.15.64:0/3992222685 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0011790 con 0x7f3108159710 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f31077fe700 1 -- 172.21.15.64:0/3992222685 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0011b30 con 0x7f3108159710 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3108159710 msgr2=0x7f310806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3108159710 0x7f310806d050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f310806ad30 tx=0x7f30f00091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- 172.21.15.64:0/3992222685 shutdown_connections 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- 172.21.15.64:0/3992222685 >> v2:172.21.15.64:3300/0 conn(0x7f310806d590 0x7f310815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3108159710 0x7f310806d050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 --2- 172.21.15.64:0/3992222685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310815c2d0 0x7f31081591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- 172.21.15.64:0/3992222685 >> 172.21.15.64:0/3992222685 conn(0x7f310806c0d0 msgr2=0x7f31080ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- 172.21.15.64:0/3992222685 shutdown_connections 2023-09-27T23:14:31.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.022+0000 7f310d237700 1 -- 172.21.15.64:0/3992222685 wait complete. 2023-09-27T23:14:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 Processor -- start 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 -- start start 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310806d590 0x7f310816c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 0x7f3108145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f310815c2d0 0x7f3108146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31080c4250 con 0x7f3108159710 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31080c4530 con 0x7f310815c2d0 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31080c4810 con 0x7f310806d590 2023-09-27T23:14:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 0x7f3108145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 0x7f3108145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33500/0 (socket says 172.21.15.64:33500) 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 -- 172.21.15.64:0/2376151533 learned_addr learned my addr 172.21.15.64:0/2376151533 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310ca36700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f310815c2d0 0x7f3108146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f3107fff700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310806d590 0x7f310816c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 -- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310806d590 msgr2=0x7f310816c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310806d590 0x7f310816c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 -- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f310815c2d0 msgr2=0x7f3108146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f310815c2d0 0x7f3108146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 -- 172.21.15.64:0/2376151533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3100009ae0 con 0x7f3108159710 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f30fffff700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 0x7f3108145ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f310815c7e0 tx=0x7f30f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0011900 con 0x7f3108159710 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0011aa0 con 0x7f3108159710 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f0009420 con 0x7f3108159710 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0026680 con 0x7f3108159710 2023-09-27T23:14:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.026+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31080c4dd0 con 0x7f3108159710 2023-09-27T23:14:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.030+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31080c5020 con 0x7f3108159710 2023-09-27T23:14:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.030+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f30f0023030 con 0x7f3108159710 2023-09-27T23:14:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.030+0000 7f3104ff9700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f30e8060950 0x7f30e8062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.030+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f30f007ea90 con 0x7f3108159710 2023-09-27T23:14:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.030+0000 7f3107fff700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f30e8060950 0x7f30e8062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.034+0000 7f3107fff700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f30e8060950 0x7f30e8062e00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006fd0 tx=0x7f30f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.038+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f30f004e9c0 con 0x7f3108159710 2023-09-27T23:14:31.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.238+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f310804be40 con 0x7f3108159710 2023-09-27T23:14:31.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.238+0000 7f3104ff9700 1 -- 172.21.15.64:0/2376151533 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7f30f002e3a0 con 0x7f3108159710 2023-09-27T23:14:31.244 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:31.244 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f30e8060950 msgr2=0x7f30e8062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f30e8060950 0x7f30e8062e00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006fd0 tx=0x7f30f8008040 comp rx=0 tx=0).stop 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 msgr2=0x7f3108145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 0x7f3108145ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f310815c7e0 tx=0x7f30f0017000 comp rx=0 tx=0).stop 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 shutdown_connections 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f30e8060950 0x7f30e8062e00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f310815c2d0 0x7f3108146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3108159710 0x7f3108145ee0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 --2- 172.21.15.64:0/2376151533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f310806d590 0x7f310816c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 >> 172.21.15.64:0/2376151533 conn(0x7f310806c0d0 msgr2=0x7f31080cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:31.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 shutdown_connections 2023-09-27T23:14:31.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.242+0000 7f310d237700 1 -- 172.21.15.64:0/2376151533 wait complete. 2023-09-27T23:14:31.248 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:31.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 Processor -- start 2023-09-27T23:14:31.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 -- start start 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a180b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a1806be10 0x7f0a18158d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a180af9c0 0x7f0a18154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180af530 con 0x7f0a1806be10 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a18159660 con 0x7f0a180af9c0 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1f418700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18159940 con 0x7f0a180b0810 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1dc15700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a1806be10 0x7f0a18158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1dc15700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a1806be10 0x7f0a18158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33502/0 (socket says 172.21.15.64:33502) 2023-09-27T23:14:31.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1dc15700 1 -- 172.21.15.64:0/1586607690 learned_addr learned my addr 172.21.15.64:0/1586607690 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1ec17700 1 --2- 172.21.15.64:0/1586607690 >> v2:172.21.15.123:3300/0 conn(0x7f0a180af9c0 0x7f0a18154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.366+0000 7f0a1e416700 1 --2- 172.21.15.64:0/1586607690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a180b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1dc15700 1 -- 172.21.15.64:0/1586607690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 msgr2=0x7f0a180b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1dc15700 1 --2- 172.21.15.64:0/1586607690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a180b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1dc15700 1 -- 172.21.15.64:0/1586607690 >> v2:172.21.15.123:3300/0 conn(0x7f0a180af9c0 msgr2=0x7f0a18154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1dc15700 1 --2- 172.21.15.64:0/1586607690 >> v2:172.21.15.123:3300/0 conn(0x7f0a180af9c0 0x7f0a18154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1dc15700 1 -- 172.21.15.64:0/1586607690 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18159c20 con 0x7f0a1806be10 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1ec17700 1 --2- 172.21.15.64:0/1586607690 >> v2:172.21.15.123:3300/0 conn(0x7f0a180af9c0 0x7f0a18154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:31.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1dc15700 1 --2- 172.21.15.64:0/1586607690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a18158d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0a080099b0 tx=0x7f0a08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96678e2e77e9a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1d414700 1 -- 172.21.15.64:0/1586607690 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800b8b0 con 0x7f0a1806be10 2023-09-27T23:14:31.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1d414700 1 -- 172.21.15.64:0/1586607690 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0800ba10 con 0x7f0a1806be10 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1d414700 1 -- 172.21.15.64:0/1586607690 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800bdb0 con 0x7f0a1806be10 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 -- 172.21.15.64:0/1586607690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 msgr2=0x7f0a18158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 --2- 172.21.15.64:0/1586607690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a18158d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0a080099b0 tx=0x7f0a08009250 comp rx=0 tx=0).stop 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 -- 172.21.15.64:0/1586607690 shutdown_connections 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 --2- 172.21.15.64:0/1586607690 >> v2:172.21.15.123:3300/0 conn(0x7f0a180af9c0 0x7f0a18154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 --2- 172.21.15.64:0/1586607690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a18158d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 --2- 172.21.15.64:0/1586607690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a180b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 -- 172.21.15.64:0/1586607690 >> 172.21.15.64:0/1586607690 conn(0x7f0a18067bb0 msgr2=0x7f0a180add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 -- 172.21.15.64:0/1586607690 shutdown_connections 2023-09-27T23:14:31.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.370+0000 7f0a1f418700 1 -- 172.21.15.64:0/1586607690 wait complete. 2023-09-27T23:14:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 Processor -- start 2023-09-27T23:14:31.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 -- start start 2023-09-27T23:14:31.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a1814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a180af9c0 0x7f0a181493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a181498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1814bbe0 con 0x7f0a1806be10 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18149e20 con 0x7f0a180af9c0 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1f418700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1814a0b0 con 0x7f0a180b0810 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1e416700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a1814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1e416700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a1814aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33510/0 (socket says 172.21.15.64:33510) 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1e416700 1 -- 172.21.15.64:0/2622277022 learned_addr learned my addr 172.21.15.64:0/2622277022 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1dc15700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a180af9c0 0x7f0a181493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.374+0000 7f0a1ec17700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a181498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1e416700 1 -- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 msgr2=0x7f0a181498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1e416700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a181498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1e416700 1 -- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a180af9c0 msgr2=0x7f0a181493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1e416700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a180af9c0 0x7f0a181493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1e416700 1 -- 172.21.15.64:0/2622277022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080094f0 con 0x7f0a1806be10 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1e416700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a1814aba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400fdf0 tx=0x7f0a14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400fe20 con 0x7f0a1806be10 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14009aa0 con 0x7f0a1806be10 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14009e60 con 0x7f0a1806be10 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1814a390 con 0x7f0a1806be10 2023-09-27T23:14:31.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1814b870 con 0x7f0a1806be10 2023-09-27T23:14:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1804be40 con 0x7f0a1806be10 2023-09-27T23:14:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f0a14011660 con 0x7f0a1806be10 2023-09-27T23:14:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.378+0000 7f0a0e7fc700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09fc060950 0x7f09fc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.382+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f0a1407f700 con 0x7f0a1806be10 2023-09-27T23:14:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.382+0000 7f0a1dc15700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09fc060950 0x7f09fc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.382+0000 7f0a1dc15700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09fc060950 0x7f09fc062e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a080099b0 tx=0x7f0a08017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.390+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0a140530e0 con 0x7f0a1806be10 2023-09-27T23:14:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.550+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a180af1c0 con 0x7f0a1806be10 2023-09-27T23:14:31.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.550+0000 7f0a0e7fc700 1 -- 172.21.15.64:0/2622277022 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f0a14052c70 con 0x7f0a1806be10 2023-09-27T23:14:31.555 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:31.555 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:27.041505+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.35","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:31.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09fc060950 msgr2=0x7f09fc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09fc060950 0x7f09fc062e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a080099b0 tx=0x7f0a08017000 comp rx=0 tx=0).stop 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 msgr2=0x7f0a1814aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a1814aba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400fdf0 tx=0x7f0a14008040 comp rx=0 tx=0).stop 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 shutdown_connections 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09fc060950 0x7f09fc062e00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a180b0810 0x7f0a181498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a180af9c0 0x7f0a181493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 --2- 172.21.15.64:0/2622277022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a1806be10 0x7f0a1814aba0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 >> 172.21.15.64:0/2622277022 conn(0x7f0a18067bb0 msgr2=0x7f0a18158060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:31.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 shutdown_connections 2023-09-27T23:14:31.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.554+0000 7f0a1f418700 1 -- 172.21.15.64:0/2622277022 wait complete. 2023-09-27T23:14:31.570 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:14:31.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:31.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.674+0000 7fe4a6b24700 1 Processor -- start 2023-09-27T23:14:31.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.674+0000 7fe4a6b24700 1 -- start start 2023-09-27T23:14:31.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe4a00af9c0 0x7fe4a0154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a00af530 con 0x7fe4a00b0810 2023-09-27T23:14:31.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a0159470 con 0x7fe4a00af9c0 2023-09-27T23:14:31.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0159660 con 0x7fe4a006be10 2023-09-27T23:14:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a5b22700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe4a00b0810 0x7fe4a0158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40118/0 (socket says 172.21.15.64:40118) 2023-09-27T23:14:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a5b22700 1 -- 172.21.15.64:0/3787372344 learned_addr learned my addr 172.21.15.64:0/3787372344 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:31.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a5321700 1 --2- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6323700 1 --2- 172.21.15.64:0/3787372344 >> v2:172.21.15.123:3300/0 conn(0x7fe4a00af9c0 0x7fe4a0154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a4b20700 1 -- 172.21.15.64:0/3787372344 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840301606 0 0) 0x7fe4a00af530 con 0x7fe4a00b0810 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a4b20700 1 -- 172.21.15.64:0/3787372344 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48c003570 con 0x7fe4a00b0810 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6323700 1 -- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 msgr2=0x7fe4a00b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6323700 1 --2- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a00b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6323700 1 -- 172.21.15.64:0/3787372344 >> v1:172.21.15.64:6789/0 conn(0x7fe4a00b0810 legacy=0x7fe4a0158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6323700 1 -- 172.21.15.64:0/3787372344 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0159940 con 0x7fe4a00af9c0 2023-09-27T23:14:31.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6323700 1 --2- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00af9c0 0x7fe4a0154b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe498009e90 tx=0x7fe498009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61539a566a264aef server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a4b20700 1 -- 172.21.15.64:0/3787372344 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800e070 con 0x7fe4a00af9c0 2023-09-27T23:14:31.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a4b20700 1 -- 172.21.15.64:0/3787372344 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49800bbd0 con 0x7fe4a00af9c0 2023-09-27T23:14:31.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a4b20700 1 -- 172.21.15.64:0/3787372344 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498012430 con 0x7fe4a00af9c0 2023-09-27T23:14:31.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00af9c0 msgr2=0x7fe4a0154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00af9c0 0x7fe4a0154b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe498009e90 tx=0x7fe498009780 comp rx=0 tx=0).stop 2023-09-27T23:14:31.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- 172.21.15.64:0/3787372344 shutdown_connections 2023-09-27T23:14:31.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00af9c0 0x7fe4a0154b20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/3787372344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- 172.21.15.64:0/3787372344 >> 172.21.15.64:0/3787372344 conn(0x7fe4a0067bb0 msgr2=0x7fe4a00add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:31.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- 172.21.15.64:0/3787372344 shutdown_connections 2023-09-27T23:14:31.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.678+0000 7fe4a6b24700 1 -- 172.21.15.64:0/3787372344 wait complete. 2023-09-27T23:14:31.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 Processor -- start 2023-09-27T23:14:31.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 -- start start 2023-09-27T23:14:31.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a00af9c0 0x7fe4a014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00b0810 0x7fe4a015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a015e9b0 con 0x7fe4a00af9c0 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0160e30 con 0x7fe4a00b0810 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6b24700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0160b50 con 0x7fe4a006be10 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a5321700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a00af9c0 0x7fe4a014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a5321700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a00af9c0 0x7fe4a014da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33526/0 (socket says 172.21.15.64:33526) 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a5321700 1 -- 172.21.15.64:0/4281786593 learned_addr learned my addr 172.21.15.64:0/4281786593 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a5b22700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.682+0000 7fe4a6323700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00b0810 0x7fe4a015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a5b22700 1 -- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00b0810 msgr2=0x7fe4a015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a5b22700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00b0810 0x7fe4a015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a5b22700 1 -- 172.21.15.64:0/4281786593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a00af9c0 msgr2=0x7fe4a014da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a5b22700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a00af9c0 0x7fe4a014da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a5b22700 1 -- 172.21.15.64:0/4281786593 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498009a20 con 0x7fe4a006be10 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a6323700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00b0810 0x7fe4a015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a5b22700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a0157760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c590 tx=0x7fe49c007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c005b30 con 0x7fe4a006be10 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a015e360 con 0x7fe4a006be10 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c005cb0 con 0x7fe4a006be10 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c007320 con 0x7fe4a006be10 2023-09-27T23:14:31.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a0157ca0 con 0x7fe4a006be10 2023-09-27T23:14:31.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.686+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a004be40 con 0x7fe4a006be10 2023-09-27T23:14:31.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.690+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fe49c033b50 con 0x7fe4a006be10 2023-09-27T23:14:31.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.690+0000 7fe495ffb700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe484060980 0x7fe484062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:31.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.690+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fe49c07c5f0 con 0x7fe4a006be10 2023-09-27T23:14:31.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.690+0000 7fe4a5321700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe484060980 0x7fe484062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:31.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.690+0000 7fe4a5321700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe484060980 0x7fe484062e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4a014e5a0 tx=0x7fe4900091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:31.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.698+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe49c04c4a0 con 0x7fe4a006be10 2023-09-27T23:14:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.894+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4a0159c20 con 0x7fe4a006be10 2023-09-27T23:14:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.894+0000 7fe495ffb700 1 -- 172.21.15.64:0/4281786593 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7fe49c04fac0 con 0x7fe4a006be10 2023-09-27T23:14:31.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:31.900 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe484060980 msgr2=0x7fe484062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe484060980 0x7fe484062e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4a014e5a0 tx=0x7fe4900091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 msgr2=0x7fe4a0157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a0157760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c590 tx=0x7fe49c007d50 comp rx=0 tx=0).stop 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 shutdown_connections 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe484060980 0x7fe484062e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a00b0810 0x7fe4a015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a00af9c0 0x7fe4a014da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 --2- 172.21.15.64:0/4281786593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006be10 0x7fe4a0157760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 >> 172.21.15.64:0/4281786593 conn(0x7fe4a0067bb0 msgr2=0x7fe4a0158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 shutdown_connections 2023-09-27T23:14:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:31.898+0000 7fe4a6b24700 1 -- 172.21.15.64:0/4281786593 wait complete. 2023-09-27T23:14:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:31.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 Processor -- start 2023-09-27T23:14:32.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 -- start start 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbfb80c6900 0x7fbfb80c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb806b900 0x7fbfb806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb806b470 con 0x7fbfb815cc20 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb815d7b0 con 0x7fbfb80c6900 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb7fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8161d10 con 0x7fbfb806b900 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbfb815cc20 0x7fbfb815d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40130/0 (socket says 172.21.15.64:40130) 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb77fe700 1 -- 172.21.15.64:0/3463366108 learned_addr learned my addr 172.21.15.64:0/3463366108 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb806b900 0x7fbfb806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/3463366108 >> v2:172.21.15.123:3300/0 conn(0x7fbfb80c6900 0x7fbfb80c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171590707 0 0) 0x7fbfb806b470 con 0x7fbfb815cc20 2023-09-27T23:14:32.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa4003570 con 0x7fbfb815cc20 2023-09-27T23:14:32.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933676932 0 0) 0x7fbfa4003570 con 0x7fbfb815cc20 2023-09-27T23:14:32.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb806b900 msgr2=0x7fbfb806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:32.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb806b900 0x7fbfb806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 >> v2:172.21.15.123:3300/0 conn(0x7fbfb80c6900 msgr2=0x7fbfb80c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 --2- 172.21.15.64:0/3463366108 >> v2:172.21.15.123:3300/0 conn(0x7fbfb80c6900 0x7fbfb80c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb806bd70 con 0x7fbfb815cc20 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fbfb00017f0 con 0x7fbfb815cc20 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 >> v1:172.21.15.64:6789/0 conn(0x7fbfb815cc20 legacy=0x7fbfb815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfa4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfa40051c0 0x7fbfa400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb815d7b0 con 0x7fbfa4003a60 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8161d10 con 0x7fbfa4004810 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400bfa0 con 0x7fbfa40051c0 2023-09-27T23:14:32.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb806b900 0x7fbfb806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfa4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb77fe700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfa40051c0 0x7fbfa400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfa40051c0 msgr2=0x7fbfa400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfa40051c0 0x7fbfa400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4004810 msgr2=0x7fbfa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/3463366108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4003570 con 0x7fbfa4003a60 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.022+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4004810 0x7fbfa4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfa4003e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009c00 tx=0x7fbfa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57891a8df639b76f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800e710 con 0x7fbfa4003a60 2023-09-27T23:14:32.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800e870 con 0x7fbfa4003a60 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb5ffb700 1 -- 172.21.15.64:0/3463366108 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800ec30 con 0x7fbfa4003a60 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 -- 172.21.15.64:0/3463366108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 msgr2=0x7fbfa4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/3463366108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfa4003e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009c00 tx=0x7fbfa80091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb77fe700 1 -- 172.21.15.64:0/3463366108 reap_dead start 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 -- 172.21.15.64:0/3463366108 shutdown_connections 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 -- 172.21.15.64:0/3463366108 >> 172.21.15.64:0/3463366108 conn(0x7fbfb8067bb0 msgr2=0x7fbfb80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 -- 172.21.15.64:0/3463366108 shutdown_connections 2023-09-27T23:14:32.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 -- 172.21.15.64:0/3463366108 wait complete. 2023-09-27T23:14:32.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 Processor -- start 2023-09-27T23:14:32.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.026+0000 7fbfb7fff700 1 -- start start 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfb80c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb815e890 0x7fbfb80b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb815fea0 0x7fbfb816db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb815ee00 con 0x7fbfa4003a60 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb815f0e0 con 0x7fbfb815e890 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb815f3c0 con 0x7fbfb815fea0 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfb80c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfb80c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33546/0 (socket says 172.21.15.64:33546) 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/1868000823 learned_addr learned my addr 172.21.15.64:0/1868000823 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb77fe700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb815fea0 0x7fbfb816db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb815e890 0x7fbfb80b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb815fea0 msgr2=0x7fbfb816db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb815fea0 0x7fbfb816db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb815e890 msgr2=0x7fbfb80b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb815e890 0x7fbfb80b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 -- 172.21.15.64:0/1868000823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb815f6a0 con 0x7fbfa4003a60 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb6ffd700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfb80c4da0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e4b0 tx=0x7fbfa801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8022070 con 0x7fbfa4003a60 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800e9e0 con 0x7fbfa4003a60 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb815f980 con 0x7fbfa4003a60 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8027470 con 0x7fbfa4003a60 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.030+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb815ee00 con 0x7fbfa4003a60 2023-09-27T23:14:32.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.034+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb815fb10 con 0x7fbfa4003a60 2023-09-27T23:14:32.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.034+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbfa801c450 con 0x7fbfa4003a60 2023-09-27T23:14:32.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.034+0000 7fbfaeffd700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf98060950 0x7fbf98062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.034+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fbfa807ebe0 con 0x7fbfa4003a60 2023-09-27T23:14:32.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.034+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf98060950 0x7fbf98062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.034+0000 7fbfb67fc700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf98060950 0x7fbf98062e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00093f0 tx=0x7fbfa0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.042+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbfa804eb30 con 0x7fbfa4003a60 2023-09-27T23:14:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.238+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfb804be40 con 0x7fbfa4003a60 2023-09-27T23:14:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.238+0000 7fbfaeffd700 1 -- 172.21.15.64:0/1868000823 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4596 (secure 0 0 0) 0x7fbfa802c090 con 0x7fbfa4003a60 2023-09-27T23:14:32.243 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:32.244 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1136774655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1136774655},{"type":"v1","addr":"172.21.15.123:6837","nonce":1136774655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/3170856699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3170856699},{"type":"v1","addr":"172.21.15.64:6834","nonce":3170856699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/487926962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":487926962},{"type":"v1","addr":"172.21.15.64:6835","nonce":487926962}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:13:44.760383+0000","modified":"2023-09-27T23:13:56.848999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/2280995923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2280995923},{"type":"v1","addr":"172.21.15.123:6835","nonce":2280995923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf98060950 msgr2=0x7fbf98062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf98060950 0x7fbf98062e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00093f0 tx=0x7fbfa0008040 comp rx=0 tx=0).stop 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 msgr2=0x7fbfb80c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfb80c4da0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e4b0 tx=0x7fbfa801a000 comp rx=0 tx=0).stop 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 shutdown_connections 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbf98060950 0x7fbf98062e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfb815fea0 0x7fbfb816db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb815e890 0x7fbfb80b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 --2- 172.21.15.64:0/1868000823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa4003a60 0x7fbfb80c4da0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 >> 172.21.15.64:0/1868000823 conn(0x7fbfb8067bb0 msgr2=0x7fbfb815bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 shutdown_connections 2023-09-27T23:14:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.242+0000 7fbfb7fff700 1 -- 172.21.15.64:0/1868000823 wait complete. 2023-09-27T23:14:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-27T23:14:32.258 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.358+0000 7f11b9f25700 1 Processor -- start 2023-09-27T23:14:32.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.358+0000 7f11b9f25700 1 -- start start 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b9f25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11b40670e0 0x7f11b4159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b9f25700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f11b40abce0 0x7f11b40c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b9f25700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b40c1910 con 0x7f11b40ab910 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b9f25700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b4159cf0 con 0x7f11b40670e0 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b9f25700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b4159fd0 con 0x7f11b40abce0 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11b40670e0 0x7f11b4159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11b40670e0 0x7f11b4159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33552/0 (socket says 172.21.15.64:33552) 2023-09-27T23:14:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 -- 172.21.15.64:0/2337828610 learned_addr learned my addr 172.21.15.64:0/2337828610 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b9724700 1 --2- 172.21.15.64:0/2337828610 >> v2:172.21.15.123:3300/0 conn(0x7f11b40abce0 0x7f11b40c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 -- 172.21.15.64:0/2337828610 >> v2:172.21.15.123:3300/0 conn(0x7f11b40abce0 msgr2=0x7f11b40c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 --2- 172.21.15.64:0/2337828610 >> v2:172.21.15.123:3300/0 conn(0x7f11b40abce0 0x7f11b40c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 -- 172.21.15.64:0/2337828610 >> v1:172.21.15.123:6789/0 conn(0x7f11b40ab910 legacy=0x7f11b40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 -- 172.21.15.64:0/2337828610 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b415a2b0 con 0x7f11b40670e0 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b3fff700 1 --2- 172.21.15.64:0/2337828610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b4159710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f11b40c1880 tx=0x7f11a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8bcc9c603b05d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b37fe700 1 -- 172.21.15.64:0/2337828610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0009e70 con 0x7f11b40670e0 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b37fe700 1 -- 172.21.15.64:0/2337828610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a000b860 con 0x7f11b40670e0 2023-09-27T23:14:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.362+0000 7f11b37fe700 1 -- 172.21.15.64:0/2337828610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000bc00 con 0x7f11b40670e0 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- 172.21.15.64:0/2337828610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 msgr2=0x7f11b4159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 --2- 172.21.15.64:0/2337828610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b4159710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f11b40c1880 tx=0x7f11a0009250 comp rx=0 tx=0).stop 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- 172.21.15.64:0/2337828610 shutdown_connections 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 --2- 172.21.15.64:0/2337828610 >> v2:172.21.15.123:3300/0 conn(0x7f11b40abce0 0x7f11b40c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 --2- 172.21.15.64:0/2337828610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b4159710 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- 172.21.15.64:0/2337828610 >> 172.21.15.64:0/2337828610 conn(0x7f11b406c080 msgr2=0x7f11b40ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- 172.21.15.64:0/2337828610 shutdown_connections 2023-09-27T23:14:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- 172.21.15.64:0/2337828610 wait complete. 2023-09-27T23:14:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 Processor -- start 2023-09-27T23:14:32.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- start start 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11b40ab910 0x7f11b40c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40abce0 0x7f11b40c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40af920 con 0x7f11b40670e0 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40afc00 con 0x7f11b40abce0 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b9f25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40afee0 con 0x7f11b40ab910 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.366+0000 7f11b8f23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b40b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33558/0 (socket says 172.21.15.64:33558) 2023-09-27T23:14:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 -- 172.21.15.64:0/1924635281 learned_addr learned my addr 172.21.15.64:0/1924635281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b3fff700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11b40ab910 0x7f11b40c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b9724700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40abce0 0x7f11b40c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 -- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11b40ab910 msgr2=0x7f11b40c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11b40ab910 0x7f11b40c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 -- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40abce0 msgr2=0x7f11b40c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40abce0 0x7f11b40c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 -- 172.21.15.64:0/1924635281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00094f0 con 0x7f11b40670e0 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b3fff700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11b40ab910 0x7f11b40c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b9724700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40abce0 0x7f11b40c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b8f23700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b40b2080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e9d0 tx=0x7f11a800c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8001890 con 0x7f11b40670e0 2023-09-27T23:14:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b40b01c0 con 0x7f11b40670e0 2023-09-27T23:14:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a80019f0 con 0x7f11b40670e0 2023-09-27T23:14:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b40c6b70 con 0x7f11b40670e0 2023-09-27T23:14:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8001d70 con 0x7f11b40670e0 2023-09-27T23:14:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b404be40 con 0x7f11b40670e0 2023-09-27T23:14:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f11a8017030 con 0x7f11b40670e0 2023-09-27T23:14:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b0ff9700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1198060950 0x7f1198062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f11a8015070 con 0x7f11b40670e0 2023-09-27T23:14:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.370+0000 7f11b3fff700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1198060950 0x7f1198062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.374+0000 7f11b3fff700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1198060950 0x7f1198062e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11b40c1880 tx=0x7f11a0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.382+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f11a804f4c0 con 0x7f11b40670e0 2023-09-27T23:14:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.546+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11b40afa60 con 0x7f11b40670e0 2023-09-27T23:14:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.546+0000 7f11b0ff9700 1 -- 172.21.15.64:0/1924635281 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f11b40afa60 con 0x7f11b40670e0 2023-09-27T23:14:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:32.552 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":31,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:27.041505+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:13:43.256567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-27T23:13:43.753338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-09-27T23:13:46.825262+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.35","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1198060950 msgr2=0x7f1198062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1198060950 0x7f1198062e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11b40c1880 tx=0x7f11a0017000 comp rx=0 tx=0).stop 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 msgr2=0x7f11b40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b40b2080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e9d0 tx=0x7f11a800c300 comp rx=0 tx=0).stop 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 shutdown_connections 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1198060950 0x7f1198062e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11b40abce0 0x7f11b40c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11b40ab910 0x7f11b40c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 --2- 172.21.15.64:0/1924635281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b40670e0 0x7f11b40b2080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 >> 172.21.15.64:0/1924635281 conn(0x7f11b406c080 msgr2=0x7f11b40cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 shutdown_connections 2023-09-27T23:14:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.550+0000 7f11b9f25700 1 -- 172.21.15.64:0/1924635281 wait complete. 2023-09-27T23:14:32.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:14:32.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 Processor -- start 2023-09-27T23:14:32.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 -- start start 2023-09-27T23:14:32.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f678406a580 0x7f678406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f678406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678406c7c0 con 0x7f67840b1410 2023-09-27T23:14:32.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678406af40 con 0x7f678406a580 2023-09-27T23:14:32.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678b40f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678406cd80 con 0x7f678406d210 2023-09-27T23:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.678+0000 7f678ac0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f678406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f6789c0c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f678406a580 0x7f678406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678ac0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f678406d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:55938/0 (socket says 172.21.15.64:55938) 2023-09-27T23:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678ac0e700 1 -- 172.21.15.64:0/2266296383 learned_addr learned my addr 172.21.15.64:0/2266296383 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:32.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678a40d700 1 --1- 172.21.15.64:0/2266296383 >> v1:172.21.15.123:6789/0 conn(0x7f67840b1410 0x7f67840ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:57378/0 (socket says 172.21.15.64:57378) 2023-09-27T23:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f6789c0c700 1 -- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 msgr2=0x7f678406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f6789c0c700 1 --2- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f678406d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f6789c0c700 1 -- 172.21.15.64:0/2266296383 >> v1:172.21.15.123:6789/0 conn(0x7f67840b1410 legacy=0x7f67840ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f6789c0c700 1 -- 172.21.15.64:0/2266296383 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678406b8f0 con 0x7f678406a580 2023-09-27T23:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678940b700 1 -- 172.21.15.64:0/2266296383 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217501049 0 0) 0x7f678406c7c0 con 0x7f67840b1410 2023-09-27T23:14:32.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f6789c0c700 1 --2- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678406a580 0x7f678406aa00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67780098e0 tx=0x7f67780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8a6fb0eb2c0be78 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678940b700 1 -- 172.21.15.64:0/2266296383 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677800b640 con 0x7f678406a580 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678940b700 1 -- 172.21.15.64:0/2266296383 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677800b7a0 con 0x7f678406a580 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678940b700 1 -- 172.21.15.64:0/2266296383 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677800bb40 con 0x7f678406a580 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 -- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678406a580 msgr2=0x7f678406aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 --2- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678406a580 0x7f678406aa00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67780098e0 tx=0x7f67780091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 -- 172.21.15.64:0/2266296383 shutdown_connections 2023-09-27T23:14:32.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 --2- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f678406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 --2- 172.21.15.64:0/2266296383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678406a580 0x7f678406aa00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 -- 172.21.15.64:0/2266296383 >> 172.21.15.64:0/2266296383 conn(0x7f6784067bb0 msgr2=0x7f67840cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:32.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 -- 172.21.15.64:0/2266296383 shutdown_connections 2023-09-27T23:14:32.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 -- 172.21.15.64:0/2266296383 wait complete. 2023-09-27T23:14:32.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 Processor -- start 2023-09-27T23:14:32.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.682+0000 7f678b40f700 1 -- start start 2023-09-27T23:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 0x7f678415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f6784155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67840b1410 0x7f67840c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840aaba0 con 0x7f678406a580 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678415b810 con 0x7f67840b1410 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67840c73b0 con 0x7f678406d210 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 0x7f678415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 0x7f678415b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33570/0 (socket says 172.21.15.64:33570) 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 -- 172.21.15.64:0/809421312 learned_addr learned my addr 172.21.15.64:0/809421312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f6789c0c700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f6784155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678ac0e700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67840b1410 0x7f67840c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 -- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 msgr2=0x7f6784155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f6784155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 -- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67840b1410 msgr2=0x7f67840c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67840b1410 0x7f67840c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 -- 172.21.15.64:0/809421312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778009420 con 0x7f678406a580 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678a40d700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 0x7f678415b2d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f678000c5e0 tx=0x7f6780001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780005a70 con 0x7f678406a580 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780005bd0 con 0x7f678406a580 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6784155ca0 con 0x7f678406a580 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800070d0 con 0x7f678406a580 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.686+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67840c6a80 con 0x7f678406a580 2023-09-27T23:14:32.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.690+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678404be40 con 0x7f678406a580 2023-09-27T23:14:32.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.690+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6780007480 con 0x7f678406a580 2023-09-27T23:14:32.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.690+0000 7f67767fc700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6768060980 0x7f6768062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:32.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.690+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f678007c5f0 con 0x7f678406a580 2023-09-27T23:14:32.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.690+0000 7f6789c0c700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6768060980 0x7f6768062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.694+0000 7f6789c0c700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6768060980 0x7f6768062e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67780093f0 tx=0x7f6778017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:32.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.698+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f678004c540 con 0x7f678406a580 2023-09-27T23:14:32.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:32.898+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f67840cc340 con 0x7f678406a580 2023-09-27T23:14:33.088 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:14:33.082+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:14:33.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.090+0000 7f67767fc700 1 -- 172.21.15.64:0/809421312 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f678004fb60 con 0x7f678406a580 2023-09-27T23:14:33.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6768060980 msgr2=0x7f6768062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6768060980 0x7f6768062e30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67780093f0 tx=0x7f6778017000 comp rx=0 tx=0).stop 2023-09-27T23:14:33.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 msgr2=0x7f678415b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 0x7f678415b2d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f678000c5e0 tx=0x7f6780001040 comp rx=0 tx=0).stop 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 shutdown_connections 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6768060980 0x7f6768062e30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67840b1410 0x7f67840c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f678406d210 0x7f6784155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 --2- 172.21.15.64:0/809421312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f678406a580 0x7f678415b2d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 >> 172.21.15.64:0/809421312 conn(0x7f6784067bb0 msgr2=0x7f6784156510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 shutdown_connections 2023-09-27T23:14:33.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.094+0000 7f678b40f700 1 -- 172.21.15.64:0/809421312 wait complete. 2023-09-27T23:14:33.102 INFO:tasks.ceph.mds.d.smithi123.stderr: -3305> 2023-09-27T23:13:46.485+0000 7feda7fdc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:14:33.102 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:14:33.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:14:33.128 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:14:33.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.226+0000 7f7cba3af700 1 Processor -- start 2023-09-27T23:14:33.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.226+0000 7f7cba3af700 1 -- start start 2023-09-27T23:14:33.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cba3af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cb40c63f0 0x7f7cb40cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cba3af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40c9db0 con 0x7f7cb406db40 2023-09-27T23:14:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cba3af700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40cb2e0 con 0x7f7cb406df10 2023-09-27T23:14:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cba3af700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40ca950 con 0x7f7cb40c63f0 2023-09-27T23:14:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7cb406db40 0x7f7cb40ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40146/0 (socket says 172.21.15.64:40146) 2023-09-27T23:14:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb3fff700 1 -- 172.21.15.64:0/2526080948 learned_addr learned my addr 172.21.15.64:0/2526080948 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb37fe700 1 --2- 172.21.15.64:0/2526080948 >> v2:172.21.15.123:3300/0 conn(0x7f7cb40c63f0 0x7f7cb40cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929697361 0 0) 0x7f7cb40c9db0 con 0x7f7cb406db40 2023-09-27T23:14:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca0003570 con 0x7f7cb406db40 2023-09-27T23:14:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795887769 0 0) 0x7f7ca0003570 con 0x7f7cb406db40 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 >> v2:172.21.15.123:3300/0 conn(0x7f7cb40c63f0 msgr2=0x7f7cb40cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 --2- 172.21.15.64:0/2526080948 >> v2:172.21.15.123:3300/0 conn(0x7f7cb40c63f0 0x7f7cb40cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 >> v1:172.21.15.123:6789/0 conn(0x7f7cb406df10 legacy=0x7f7cb4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40cb5c0 con 0x7f7cb406db40 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb37fe700 1 --2- 172.21.15.64:0/2526080948 >> v2:172.21.15.123:3300/0 conn(0x7f7cb40c63f0 0x7f7cb40cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f7ca4001470 con 0x7f7cb406db40 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 >> v1:172.21.15.64:6789/0 conn(0x7f7cb406db40 legacy=0x7f7cb40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca0003a60 0x7f7ca0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ca0004810 0x7f7ca0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ca00051c0 0x7f7ca000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40ca950 con 0x7f7ca00051c0 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca000c0d0 con 0x7f7ca0004810 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca000c290 con 0x7f7ca0003a60 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb3fff700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca0003a60 0x7f7ca0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ca00051c0 0x7f7ca000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb37fe700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ca0004810 0x7f7ca0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 -- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca0003a60 msgr2=0x7f7ca0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca0003a60 0x7f7ca0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 -- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ca0004810 msgr2=0x7f7ca0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ca0004810 0x7f7ca0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 -- 172.21.15.64:0/2526080948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0003570 con 0x7f7ca00051c0 2023-09-27T23:14:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb3fff700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca0003a60 0x7f7ca0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 -- 172.21.15.64:0/2526080948 reap_dead start 2023-09-27T23:14:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb894c700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ca00051c0 0x7f7ca000ba60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7cac001f90 tx=0x7f7cac001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7f760ca042e1cc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac001950 con 0x7f7ca00051c0 2023-09-27T23:14:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac00c8b0 con 0x7f7ca00051c0 2023-09-27T23:14:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.230+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/2526080948 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00cc50 con 0x7f7ca00051c0 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 -- 172.21.15.64:0/2526080948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ca00051c0 msgr2=0x7f7ca000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ca00051c0 0x7f7ca000ba60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7cac001f90 tx=0x7f7cac001640 comp rx=0 tx=0).stop 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 -- 172.21.15.64:0/2526080948 shutdown_connections 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 --2- 172.21.15.64:0/2526080948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ca00051c0 0x7f7ca000ba60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 -- 172.21.15.64:0/2526080948 >> 172.21.15.64:0/2526080948 conn(0x7f7cb4067bb0 msgr2=0x7f7cb40cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 -- 172.21.15.64:0/2526080948 shutdown_connections 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 -- 172.21.15.64:0/2526080948 wait complete. 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 Processor -- start 2023-09-27T23:14:33.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.234+0000 7f7cba3af700 1 -- start start 2023-09-27T23:14:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca00051c0 0x7f7cb40c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb41437e0 0x7f7cb4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 0x7f7cb4166bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415a6c0 con 0x7f7cb415a250 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c2d80 con 0x7f7cb41437e0 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c3620 con 0x7f7ca00051c0 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 0x7f7cb4166bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 0x7f7cb4166bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33588/0 (socket says 172.21.15.64:33588) 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 -- 172.21.15.64:0/628389007 learned_addr learned my addr 172.21.15.64:0/628389007 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb3fff700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca00051c0 0x7f7cb40c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb37fe700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb41437e0 0x7f7cb4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 -- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca00051c0 msgr2=0x7f7cb40c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca00051c0 0x7f7cb40c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 -- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb41437e0 msgr2=0x7f7cb4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb41437e0 0x7f7cb4159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 -- 172.21.15.64:0/628389007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb41678e0 con 0x7f7cb415a250 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cb894c700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 0x7f7cb4166bb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40b1e70 tx=0x7f7cac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac001950 con 0x7f7cb415a250 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac00cea0 con 0x7f7cb415a250 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac01d850 con 0x7f7cb415a250 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40c2fd0 con 0x7f7cb415a250 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb4143c50 con 0x7f7cb415a250 2023-09-27T23:14:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.238+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb40c3ff0 con 0x7f7cb415a250 2023-09-27T23:14:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.242+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7cac019070 con 0x7f7cb415a250 2023-09-27T23:14:33.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.242+0000 7f7cabfff700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c94060980 0x7f7c94062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:33.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.242+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6953+0+0 (secure 0 0 0) 0x7f7cac07e4c0 con 0x7f7cb415a250 2023-09-27T23:14:33.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.242+0000 7f7cb3fff700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c94060980 0x7f7c94062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.242+0000 7f7cb3fff700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c94060980 0x7f7c94062e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4007cd0 tx=0x7f7ca4001e50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:33.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.250+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7cac04e3c0 con 0x7f7cb415a250 2023-09-27T23:14:33.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:33.446+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7cb4071120 con 0x7f7cb415a250 2023-09-27T23:14:34.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.110+0000 7f7cabfff700 1 -- 172.21.15.64:0/628389007 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7cac004030 con 0x7f7cb415a250 2023-09-27T23:14:34.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c94060980 msgr2=0x7f7c94062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c94060980 0x7f7c94062e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4007cd0 tx=0x7f7ca4001e50 comp rx=0 tx=0).stop 2023-09-27T23:14:34.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 msgr2=0x7f7cb4166bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 0x7f7cb4166bb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40b1e70 tx=0x7f7cac015000 comp rx=0 tx=0).stop 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 shutdown_connections 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7c94060980 0x7f7c94062e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb415a250 0x7f7cb4166bb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb41437e0 0x7f7cb4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 --2- 172.21.15.64:0/628389007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ca00051c0 0x7f7cb40c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 >> 172.21.15.64:0/628389007 conn(0x7f7cb4067bb0 msgr2=0x7f7cb40c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 shutdown_connections 2023-09-27T23:14:34.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.114+0000 7f7cba3af700 1 -- 172.21.15.64:0/628389007 wait complete. 2023-09-27T23:14:34.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:14:34.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.258+0000 7ffb3f3da700 1 Processor -- start 2023-09-27T23:14:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.258+0000 7ffb3f3da700 1 -- start start 2023-09-27T23:14:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb3806d1f0 0x7ffb3806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb38158710 con 0x7ffb381544e0 2023-09-27T23:14:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb380c7ab0 con 0x7ffb381581f0 2023-09-27T23:14:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb38155020 con 0x7ffb3806d1f0 2023-09-27T23:14:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d176700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb3806d1f0 0x7ffb3806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d977700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffb381544e0 0x7ffb38154910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40156/0 (socket says 172.21.15.64:40156) 2023-09-27T23:14:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d977700 1 -- 172.21.15.64:0/19192762 learned_addr learned my addr 172.21.15.64:0/19192762 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:34.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d176700 1 -- 172.21.15.64:0/19192762 >> v1:172.21.15.123:6789/0 conn(0x7ffb381581f0 legacy=0x7ffb380c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb2ffff700 1 -- 172.21.15.64:0/19192762 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270054106 0 0) 0x7ffb380c7ab0 con 0x7ffb381581f0 2023-09-27T23:14:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d176700 1 -- 172.21.15.64:0/19192762 >> v1:172.21.15.64:6789/0 conn(0x7ffb381544e0 legacy=0x7ffb38154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d176700 1 -- 172.21.15.64:0/19192762 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380c7d90 con 0x7ffb3806d1f0 2023-09-27T23:14:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3d176700 1 --2- 172.21.15.64:0/19192762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb3806d1f0 0x7ffb3806d5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009910 tx=0x7ffb280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978989932f2aa61f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb2ffff700 1 -- 172.21.15.64:0/19192762 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28009eb0 con 0x7ffb3806d1f0 2023-09-27T23:14:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb2ffff700 1 -- 172.21.15.64:0/19192762 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2800b760 con 0x7ffb3806d1f0 2023-09-27T23:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb2ffff700 1 -- 172.21.15.64:0/19192762 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2800bb00 con 0x7ffb3806d1f0 2023-09-27T23:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- 172.21.15.64:0/19192762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb3806d1f0 msgr2=0x7ffb3806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/19192762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb3806d1f0 0x7ffb3806d5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009910 tx=0x7ffb280091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- 172.21.15.64:0/19192762 shutdown_connections 2023-09-27T23:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/19192762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb3806d1f0 0x7ffb3806d5c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- 172.21.15.64:0/19192762 >> 172.21.15.64:0/19192762 conn(0x7ffb380ccc20 msgr2=0x7ffb380cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- 172.21.15.64:0/19192762 shutdown_connections 2023-09-27T23:14:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.262+0000 7ffb3f3da700 1 -- 172.21.15.64:0/19192762 wait complete. 2023-09-27T23:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 Processor -- start 2023-09-27T23:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 -- start start 2023-09-27T23:14:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb3806d1f0 0x7ffb38151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb381544e0 0x7ffb380c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 0x7ffb38150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c6b50 con 0x7ffb381581f0 2023-09-27T23:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c4bb0 con 0x7ffb381544e0 2023-09-27T23:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3f3da700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c6590 con 0x7ffb3806d1f0 2023-09-27T23:14:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3d977700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 0x7ffb38150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3d977700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 0x7ffb38150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33608/0 (socket says 172.21.15.64:33608) 2023-09-27T23:14:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3d977700 1 -- 172.21.15.64:0/709330896 learned_addr learned my addr 172.21.15.64:0/709330896 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3d176700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb3806d1f0 0x7ffb38151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.266+0000 7ffb3c975700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb381544e0 0x7ffb380c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:34.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3d977700 1 -- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb3806d1f0 msgr2=0x7ffb38151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3d977700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb3806d1f0 0x7ffb38151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3d977700 1 -- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb381544e0 msgr2=0x7ffb380c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3d977700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb381544e0 0x7ffb380c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3d977700 1 -- 172.21.15.64:0/709330896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28009420 con 0x7ffb381581f0 2023-09-27T23:14:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3d977700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 0x7ffb38150520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffb30002230 tx=0x7ffb30001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000aa80 con 0x7ffb381581f0 2023-09-27T23:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3000abe0 con 0x7ffb381581f0 2023-09-27T23:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb380c6870 con 0x7ffb381581f0 2023-09-27T23:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300070f0 con 0x7ffb381581f0 2023-09-27T23:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb38150a60 con 0x7ffb381581f0 2023-09-27T23:14:34.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ffb300074a0 con 0x7ffb381581f0 2023-09-27T23:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb38152750 con 0x7ffb381581f0 2023-09-27T23:14:34.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb2d7fa700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb1c060950 0x7ffb1c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.270+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6953+0+0 (secure 0 0 0) 0x7ffb3007d610 con 0x7ffb381581f0 2023-09-27T23:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.274+0000 7ffb3d176700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb1c060950 0x7ffb1c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.274+0000 7ffb3d176700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb1c060950 0x7ffb1c062e00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb280093f0 tx=0x7ffb28016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:34.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.278+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ffb38152750 con 0x7ffb381581f0 2023-09-27T23:14:34.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:34.442+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb38152750 con 0x7ffb381581f0 2023-09-27T23:14:35.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.126+0000 7ffb2d7fa700 1 -- 172.21.15.64:0/709330896 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v33) v1 ==== 181+0+0 (secure 0 0 0) 0x7ffb38152750 con 0x7ffb381581f0 2023-09-27T23:14:35.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb1c060950 msgr2=0x7ffb1c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb1c060950 0x7ffb1c062e00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb280093f0 tx=0x7ffb28016000 comp rx=0 tx=0).stop 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 msgr2=0x7ffb38150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 0x7ffb38150520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffb30002230 tx=0x7ffb30001b90 comp rx=0 tx=0).stop 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 shutdown_connections 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb1c060950 0x7ffb1c062e00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb381581f0 0x7ffb38150520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb381544e0 0x7ffb380c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 --2- 172.21.15.64:0/709330896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb3806d1f0 0x7ffb38151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 >> 172.21.15.64:0/709330896 conn(0x7ffb380ccc20 msgr2=0x7ffb38154910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 shutdown_connections 2023-09-27T23:14:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.130+0000 7ffb3f3da700 1 -- 172.21.15.64:0/709330896 wait complete. 2023-09-27T23:14:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:14:35.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:14:35.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 Processor -- start 2023-09-27T23:14:35.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 -- start start 2023-09-27T23:14:35.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb93c06af50 0x7fb93c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb93c06d6b0 0x7fb93c06db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93c0cc140 con 0x7fb93c06d6b0 2023-09-27T23:14:35.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93c0cb3f0 con 0x7fb93c06af50 2023-09-27T23:14:35.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb942ade700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c06d220 con 0x7fb93c0aedc0 2023-09-27T23:14:35.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb93c06d6b0 0x7fb93c06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb93c06d6b0 0x7fb93c06db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51156/0 (socket says 172.21.15.64:51156) 2023-09-27T23:14:35.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 -- 172.21.15.64:0/3994731448 learned_addr learned my addr 172.21.15.64:0/3994731448 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:35.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb941adc700 1 --2- 172.21.15.64:0/3994731448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9412db700 1 --2- 172.21.15.64:0/3994731448 >> v2:172.21.15.123:3300/0 conn(0x7fb93c06af50 0x7fb93c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 -- 172.21.15.64:0/3994731448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 msgr2=0x7fb93c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 --2- 172.21.15.64:0/3994731448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 -- 172.21.15.64:0/3994731448 >> v2:172.21.15.123:3300/0 conn(0x7fb93c06af50 msgr2=0x7fb93c0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 --2- 172.21.15.64:0/3994731448 >> v2:172.21.15.123:3300/0 conn(0x7fb93c06af50 0x7fb93c0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.254+0000 7fb9422dd700 1 -- 172.21.15.64:0/3994731448 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c0cb190 con 0x7fb93c06d6b0 2023-09-27T23:14:35.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb9422dd700 1 --2- 172.21.15.64:0/3994731448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06d6b0 0x7fb93c06db30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb93c068210 tx=0x7fb934009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febc6a808cfaa3b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb940ada700 1 -- 172.21.15.64:0/3994731448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93400e070 con 0x7fb93c06d6b0 2023-09-27T23:14:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb940ada700 1 -- 172.21.15.64:0/3994731448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93400b970 con 0x7fb93c06d6b0 2023-09-27T23:14:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb940ada700 1 -- 172.21.15.64:0/3994731448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340092e0 con 0x7fb93c06d6b0 2023-09-27T23:14:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 -- 172.21.15.64:0/3994731448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06d6b0 msgr2=0x7fb93c06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:35.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 --2- 172.21.15.64:0/3994731448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06d6b0 0x7fb93c06db30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb93c068210 tx=0x7fb934009840 comp rx=0 tx=0).stop 2023-09-27T23:14:35.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 -- 172.21.15.64:0/3994731448 shutdown_connections 2023-09-27T23:14:35.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 --2- 172.21.15.64:0/3994731448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06d6b0 0x7fb93c06db30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 --2- 172.21.15.64:0/3994731448 >> v2:172.21.15.123:3300/0 conn(0x7fb93c06af50 0x7fb93c0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 --2- 172.21.15.64:0/3994731448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 -- 172.21.15.64:0/3994731448 >> 172.21.15.64:0/3994731448 conn(0x7fb93c0cc760 msgr2=0x7fb93c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 -- 172.21.15.64:0/3994731448 shutdown_connections 2023-09-27T23:14:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 -- 172.21.15.64:0/3994731448 wait complete. 2023-09-27T23:14:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.258+0000 7fb942ade700 1 Processor -- start 2023-09-27T23:14:35.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 -- start start 2023-09-27T23:14:35.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 0x7fb93c151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb93c06d6b0 0x7fb93c0c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0c7ef0 con 0x7fb93c06af50 2023-09-27T23:14:35.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0b5320 con 0x7fb93c06d6b0 2023-09-27T23:14:35.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0c75c0 con 0x7fb93c0aedc0 2023-09-27T23:14:35.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 0x7fb93c151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 0x7fb93c151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51160/0 (socket says 172.21.15.64:51160) 2023-09-27T23:14:35.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 -- 172.21.15.64:0/1338753281 learned_addr learned my addr 172.21.15.64:0/1338753281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:35.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb9422dd700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb9412db700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb93c06d6b0 0x7fb93c0c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 -- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 msgr2=0x7fb93c0c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:35.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 -- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb93c06d6b0 msgr2=0x7fb93c0c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:35.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb93c06d6b0 0x7fb93c0c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:35.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 -- 172.21.15.64:0/1338753281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb934009ae0 con 0x7fb93c06af50 2023-09-27T23:14:35.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb941adc700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 0x7fb93c151500 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb93800ec80 tx=0x7fb9380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:35.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800c850 con 0x7fb93c06af50 2023-09-27T23:14:35.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93800c9d0 con 0x7fb93c06af50 2023-09-27T23:14:35.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c0c78a0 con 0x7fb93c06af50 2023-09-27T23:14:35.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800cd90 con 0x7fb93c06af50 2023-09-27T23:14:35.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.262+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c151a40 con 0x7fb93c06af50 2023-09-27T23:14:35.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.266+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c04be40 con 0x7fb93c06af50 2023-09-27T23:14:35.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.266+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb938013680 con 0x7fb93c06af50 2023-09-27T23:14:35.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.266+0000 7fb931ffb700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb920060950 0x7fb920062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:35.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.266+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6515+0+0 (secure 0 0 0) 0x7fb93807d2f0 con 0x7fb93c06af50 2023-09-27T23:14:35.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.266+0000 7fb9412db700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb920060950 0x7fb920062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:35.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.266+0000 7fb9412db700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb920060950 0x7fb920062e00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0b59f0 tx=0x7fb92c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:35.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.274+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb93804d3a0 con 0x7fb93c06af50 2023-09-27T23:14:35.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:35.434+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb93c06f3d0 con 0x7fb93c06af50 2023-09-27T23:14:36.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.134+0000 7fb931ffb700 1 -- 172.21.15.64:0/1338753281 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v34) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb9380509c0 con 0x7fb93c06af50 2023-09-27T23:14:36.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb920060950 msgr2=0x7fb920062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:36.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb920060950 0x7fb920062e00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0b59f0 tx=0x7fb92c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 msgr2=0x7fb93c151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:36.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 0x7fb93c151500 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb93800ec80 tx=0x7fb9380091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 shutdown_connections 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb920060950 0x7fb920062e00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb93c0aedc0 0x7fb93c0c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb93c06d6b0 0x7fb93c0c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 --2- 172.21.15.64:0/1338753281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb93c06af50 0x7fb93c151500 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 >> 172.21.15.64:0/1338753281 conn(0x7fb93c0cc760 msgr2=0x7fb93c0ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 shutdown_connections 2023-09-27T23:14:36.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.146+0000 7fb942ade700 1 -- 172.21.15.64:0/1338753281 wait complete. 2023-09-27T23:14:36.155 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:14:36.168 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-09-27T23:14:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 Processor -- start 2023-09-27T23:14:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 -- start start 2023-09-27T23:14:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c881568e0 0x7f0c880c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:36.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c880c84a0 0x7f0c880ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880c8a00 con 0x7f0c88156cb0 2023-09-27T23:14:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880afa50 con 0x7f0c880c84a0 2023-09-27T23:14:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c87fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880c92a0 con 0x7f0c881568e0 2023-09-27T23:14:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c880c84a0 0x7f0c880ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c880c84a0 0x7f0c880ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51164/0 (socket says 172.21.15.64:51164) 2023-09-27T23:14:36.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 -- 172.21.15.64:0/2248669212 learned_addr learned my addr 172.21.15.64:0/2248669212 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c86ffd700 1 --2- 172.21.15.64:0/2248669212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c881568e0 0x7f0c880c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 -- 172.21.15.64:0/2248669212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c881568e0 msgr2=0x7f0c880c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 --2- 172.21.15.64:0/2248669212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c881568e0 0x7f0c880c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 -- 172.21.15.64:0/2248669212 >> v1:172.21.15.123:6789/0 conn(0x7f0c88156cb0 legacy=0x7f0c8806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 -- 172.21.15.64:0/2248669212 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880c8ce0 con 0x7f0c880c84a0 2023-09-27T23:14:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.274+0000 7f0c867fc700 1 --2- 172.21.15.64:0/2248669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c880c84a0 0x7f0c880ae800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0099b0 tx=0x7f0c7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112bd17be2dd551c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c85ffb700 1 -- 172.21.15.64:0/2248669212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c009ea0 con 0x7f0c880c84a0 2023-09-27T23:14:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c85ffb700 1 -- 172.21.15.64:0/2248669212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00b860 con 0x7f0c880c84a0 2023-09-27T23:14:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c85ffb700 1 -- 172.21.15.64:0/2248669212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00bc00 con 0x7f0c880c84a0 2023-09-27T23:14:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 -- 172.21.15.64:0/2248669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c880c84a0 msgr2=0x7f0c880ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2248669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c880c84a0 0x7f0c880ae800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0099b0 tx=0x7f0c7c009250 comp rx=0 tx=0).stop 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 -- 172.21.15.64:0/2248669212 shutdown_connections 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2248669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c880c84a0 0x7f0c880ae800 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2248669212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c881568e0 0x7f0c880c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 -- 172.21.15.64:0/2248669212 >> 172.21.15.64:0/2248669212 conn(0x7f0c8806b5b0 msgr2=0x7f0c8806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 -- 172.21.15.64:0/2248669212 shutdown_connections 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.278+0000 7f0c87fff700 1 -- 172.21.15.64:0/2248669212 wait complete. 2023-09-27T23:14:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 Processor -- start 2023-09-27T23:14:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 -- start start 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c880c84a0 0x7f0c880c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c881568e0 0x7f0c880c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 0x7f0c88154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880c2e00 con 0x7f0c88156cb0 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880c14a0 con 0x7f0c881568e0 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c87fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880c1780 con 0x7f0c880c84a0 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 0x7f0c88154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c86ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c880c84a0 0x7f0c880c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:36.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c867fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c881568e0 0x7f0c880c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 0x7f0c88154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51178/0 (socket says 172.21.15.64:51178) 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c86ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c880c84a0 0x7f0c880c0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38528/0 (socket says 172.21.15.64:38528) 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 -- 172.21.15.64:0/2001671298 learned_addr learned my addr 172.21.15.64:0/2001671298 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 -- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c880c84a0 msgr2=0x7f0c880c0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c880c84a0 0x7f0c880c0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 -- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c881568e0 msgr2=0x7f0c880c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c881568e0 0x7f0c880c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 -- 172.21.15.64:0/2001671298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0094f0 con 0x7f0c88156cb0 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c877fe700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 0x7f0c88154410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c8e0 tx=0x7f0c80001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.282+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80012630 con 0x7f0c88156cb0 2023-09-27T23:14:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c880c1a60 con 0x7f0c88156cb0 2023-09-27T23:14:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c80012790 con 0x7f0c88156cb0 2023-09-27T23:14:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80012b50 con 0x7f0c88156cb0 2023-09-27T23:14:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c880c2020 con 0x7f0c88156cb0 2023-09-27T23:14:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8804be40 con 0x7f0c88156cb0 2023-09-27T23:14:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f0c8001a3a0 con 0x7f0c88156cb0 2023-09-27T23:14:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c6effd700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c74060950 0x7f0c74062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.286+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6043+0+0 (secure 0 0 0) 0x7f0c8007d060 con 0x7f0c88156cb0 2023-09-27T23:14:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.290+0000 7f0c86ffd700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c74060950 0x7f0c74062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:36.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.290+0000 7f0c86ffd700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c74060950 0x7f0c74062e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009db0 tx=0x7f0c780091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:36.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.294+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0c80050d80 con 0x7f0c88156cb0 2023-09-27T23:14:36.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:36.454+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c880c9030 con 0x7f0c88156cb0 2023-09-27T23:14:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.142+0000 7f0c6effd700 1 -- 172.21.15.64:0/2001671298 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v35) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0c80050910 con 0x7f0c88156cb0 2023-09-27T23:14:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c74060950 msgr2=0x7f0c74062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c74060950 0x7f0c74062e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009db0 tx=0x7f0c780091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 msgr2=0x7f0c88154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 0x7f0c88154410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c8e0 tx=0x7f0c80001710 comp rx=0 tx=0).stop 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 shutdown_connections 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0c74060950 0x7f0c74062e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c88156cb0 0x7f0c88154410 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c881568e0 0x7f0c880c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 --2- 172.21.15.64:0/2001671298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c880c84a0 0x7f0c880c0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 >> 172.21.15.64:0/2001671298 conn(0x7f0c8806b5b0 msgr2=0x7f0c88067380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:37.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 shutdown_connections 2023-09-27T23:14:37.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.150+0000 7f0c87fff700 1 -- 172.21.15.64:0/2001671298 wait complete. 2023-09-27T23:14:37.155 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data_ec' removed 2023-09-27T23:14:37.168 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:14:37.168 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:14:37.168 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:14:37.169 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:14:37.169 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:14:37.169 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:14:37.169 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:14:37.169 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:14:37.169 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:14:37.169 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:14:37.170 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:14:37.170 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:14:37.170 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:14:37.170 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:14:37.170 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:14:37.170 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:14:37.181 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:14:37.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:14:37.183 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:14:37.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:14:37.184 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:14:37.185 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:37.185 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:14:37.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:14:37.186 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:14:37.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:14:37.187 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:14:37.188 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:14:37.188 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:37.188 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:37.189 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:14:37.189 DEBUG:teuthology.parallel:result is None 2023-09-27T23:14:37.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:14:37.250 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:14:37.255 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:14:37.256 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:14:37.271 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:14:37.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.302+0000 7fbe3ffff700 1 Processor -- start 2023-09-27T23:14:37.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.302+0000 7fbe3ffff700 1 -- start start 2023-09-27T23:14:37.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.302+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe40155990 0x7fbe400c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.302+0000 7fbe3ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbe400b1580 0x7fbe4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.302+0000 7fbe3ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4006cf00 con 0x7fbe400b0670 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.302+0000 7fbe3ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe400ae7e0 con 0x7fbe400b1580 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400cbc50 con 0x7fbe40155990 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbe400b1580 0x7fbe4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe37fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbe400b0670 0x7fbe400b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51580/0 (socket says 172.21.15.64:51580) 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe37fff700 1 -- 172.21.15.64:0/1099318678 learned_addr learned my addr 172.21.15.64:0/1099318678 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:37.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3effd700 1 --2- 172.21.15.64:0/1099318678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe40155990 0x7fbe400c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/1099318678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe40155990 msgr2=0x7fbe400c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3f7fe700 1 --2- 172.21.15.64:0/1099318678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe40155990 0x7fbe400c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/1099318678 >> v1:172.21.15.64:6789/0 conn(0x7fbe400b0670 legacy=0x7fbe400b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/1099318678 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe400b01d0 con 0x7fbe400b1580 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3f7fe700 1 --2- 172.21.15.64:0/1099318678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b1580 0x7fbe4006d390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009e60 tx=0x7fbe38009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2dcf6d4264c3b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3e7fc700 1 -- 172.21.15.64:0/1099318678 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800e070 con 0x7fbe400b1580 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3e7fc700 1 -- 172.21.15.64:0/1099318678 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3800bb60 con 0x7fbe400b1580 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3e7fc700 1 -- 172.21.15.64:0/1099318678 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38012430 con 0x7fbe400b1580 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- 172.21.15.64:0/1099318678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b1580 msgr2=0x7fbe4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:37.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/1099318678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b1580 0x7fbe4006d390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009e60 tx=0x7fbe38009780 comp rx=0 tx=0).stop 2023-09-27T23:14:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- 172.21.15.64:0/1099318678 shutdown_connections 2023-09-27T23:14:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/1099318678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b1580 0x7fbe4006d390 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/1099318678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe40155990 0x7fbe400c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- 172.21.15.64:0/1099318678 >> 172.21.15.64:0/1099318678 conn(0x7fbe40067bb0 msgr2=0x7fbe400add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- 172.21.15.64:0/1099318678 shutdown_connections 2023-09-27T23:14:37.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- 172.21.15.64:0/1099318678 wait complete. 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 Processor -- start 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.306+0000 7fbe3ffff700 1 -- start start 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe400b0670 0x7fbe40169310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe400b1580 0x7fbe400a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 0x7fbe400adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4016c4f0 con 0x7fbe40155990 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400a5240 con 0x7fbe400b1580 2023-09-27T23:14:37.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400a4f60 con 0x7fbe400b0670 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 0x7fbe400adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 0x7fbe400adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51214/0 (socket says 172.21.15.64:51214) 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/2425803262 learned_addr learned my addr 172.21.15.64:0/2425803262 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3effd700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe400b0670 0x7fbe40169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe37fff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe400b1580 0x7fbe400a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe400b0670 msgr2=0x7fbe40169310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe400b0670 0x7fbe40169310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe400b1580 msgr2=0x7fbe400a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe400b1580 0x7fbe400a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 -- 172.21.15.64:0/2425803262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe38009a20 con 0x7fbe40155990 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3f7fe700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 0x7fbe400adcf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbe38013000 tx=0x7fbe38019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:37.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38012e60 con 0x7fbe40155990 2023-09-27T23:14:37.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38027470 con 0x7fbe40155990 2023-09-27T23:14:37.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4016bea0 con 0x7fbe40155990 2023-09-27T23:14:37.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38027830 con 0x7fbe40155990 2023-09-27T23:14:37.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe400a4c70 con 0x7fbe40155990 2023-09-27T23:14:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4016c6a0 con 0x7fbe40155990 2023-09-27T23:14:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fbe38024070 con 0x7fbe40155990 2023-09-27T23:14:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe36ffd700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe20060980 0x7fbe20062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fbe3807f2b0 con 0x7fbe40155990 2023-09-27T23:14:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.310+0000 7fbe3effd700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe20060980 0x7fbe20062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:37.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.314+0000 7fbe3effd700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe20060980 0x7fbe20062e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009db0 tx=0x7fbe300091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.318+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbe4016c6a0 con 0x7fbe40155990 2023-09-27T23:14:37.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:37.486+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbe4016c6a0 con 0x7fbe40155990 2023-09-27T23:14:38.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.306+0000 7fbe36ffd700 1 -- 172.21.15.64:0/2425803262 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbe4016c6a0 con 0x7fbe40155990 2023-09-27T23:14:38.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe20060980 msgr2=0x7fbe20062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe20060980 0x7fbe20062e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009db0 tx=0x7fbe300091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 msgr2=0x7fbe400adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 0x7fbe400adcf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbe38013000 tx=0x7fbe38019000 comp rx=0 tx=0).stop 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 shutdown_connections 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbe20060980 0x7fbe20062e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe40155990 0x7fbe400adcf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe400b1580 0x7fbe400a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 --2- 172.21.15.64:0/2425803262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe400b0670 0x7fbe40169310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 >> 172.21.15.64:0/2425803262 conn(0x7fbe40067bb0 msgr2=0x7fbe400cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 shutdown_connections 2023-09-27T23:14:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.310+0000 7fbe3ffff700 1 -- 172.21.15.64:0/2425803262 wait complete. 2023-09-27T23:14:38.317 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:14:38.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:14:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.442+0000 7f09f3fff700 1 Processor -- start 2023-09-27T23:14:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.442+0000 7f09f3fff700 1 -- start start 2023-09-27T23:14:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09f4155960 0x7f09f415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 0x7f09f40aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f406d990 con 0x7f09f406de20 2023-09-27T23:14:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f3fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f415a570 con 0x7f09f4155960 2023-09-27T23:14:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f406cd80 con 0x7f09f40c7f00 2023-09-27T23:14:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 0x7f09f40aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09f4155960 0x7f09f415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 0x7f09f40aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38596/0 (socket says 172.21.15.64:38596) 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09f4155960 0x7f09f415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:39220/0 (socket says 172.21.15.64:39220) 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f27fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f09f406de20 0x7f09f40c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37800/0 (socket says 172.21.15.64:37800) 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 -- 172.21.15.64:0/3252649599 learned_addr learned my addr 172.21.15.64:0/3252649599 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f1ffb700 1 -- 172.21.15.64:0/3252649599 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903387502 0 0) 0x7f09f406d990 con 0x7f09f406de20 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 -- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 msgr2=0x7f09f40aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 --2- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 0x7f09f40aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 -- 172.21.15.64:0/3252649599 >> v1:172.21.15.123:6789/0 conn(0x7f09f406de20 legacy=0x7f09f40c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 -- 172.21.15.64:0/3252649599 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f406d060 con 0x7f09f4155960 2023-09-27T23:14:38.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f37fe700 1 --2- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 0x7f09f40aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:38.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f2ffd700 1 --2- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f4155960 0x7f09f415a030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f09e40099b0 tx=0x7f09e4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa26693ff70275a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f1ffb700 1 -- 172.21.15.64:0/3252649599 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4009ea0 con 0x7f09f4155960 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f1ffb700 1 -- 172.21.15.64:0/3252649599 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e400b860 con 0x7f09f4155960 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.446+0000 7f09f1ffb700 1 -- 172.21.15.64:0/3252649599 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e400bc00 con 0x7f09f4155960 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 -- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f4155960 msgr2=0x7f09f415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 --2- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f4155960 0x7f09f415a030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f09e40099b0 tx=0x7f09e4009250 comp rx=0 tx=0).stop 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 -- 172.21.15.64:0/3252649599 shutdown_connections 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 --2- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c7f00 0x7f09f40aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 --2- 172.21.15.64:0/3252649599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f4155960 0x7f09f415a030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 -- 172.21.15.64:0/3252649599 >> 172.21.15.64:0/3252649599 conn(0x7f09f4067bb0 msgr2=0x7f09f40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 -- 172.21.15.64:0/3252649599 shutdown_connections 2023-09-27T23:14:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 -- 172.21.15.64:0/3252649599 wait complete. 2023-09-27T23:14:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 Processor -- start 2023-09-27T23:14:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.450+0000 7f09f3fff700 1 -- start start 2023-09-27T23:14:38.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f406de20 0x7f09f415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 0x7f09f40b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f4155960 0x7f09f414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40af840 con 0x7f09f40c7f00 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f415d770 con 0x7f09f406de20 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40b19a0 con 0x7f09f4155960 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 0x7f09f40b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 0x7f09f40b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51226/0 (socket says 172.21.15.64:51226) 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 -- 172.21.15.64:0/2724277153 learned_addr learned my addr 172.21.15.64:0/2724277153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f37fe700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f4155960 0x7f09f414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f2ffd700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f406de20 0x7f09f415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 -- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f4155960 msgr2=0x7f09f414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f4155960 0x7f09f414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 -- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f406de20 msgr2=0x7f09f415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f406de20 0x7f09f415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 -- 172.21.15.64:0/2724277153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009a20 con 0x7f09f40c7f00 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f27fc700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 0x7f09f40b1460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0011e0 tx=0x7f09dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc005a70 con 0x7f09f40c7f00 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc005bd0 con 0x7f09f40c7f00 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e40094f0 con 0x7f09f40c7f00 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0070d0 con 0x7f09f40c7f00 2023-09-27T23:14:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.454+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f40af280 con 0x7f09f40c7f00 2023-09-27T23:14:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.458+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f415d930 con 0x7f09f40c7f00 2023-09-27T23:14:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.458+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f09dc007480 con 0x7f09f40c7f00 2023-09-27T23:14:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.458+0000 7f09eaffd700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060950 0x7f09d4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.458+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f09dc07c0f0 con 0x7f09f40c7f00 2023-09-27T23:14:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.458+0000 7f09f2ffd700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060950 0x7f09d4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.458+0000 7f09f2ffd700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060950 0x7f09d4062e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f09e40099b0 tx=0x7f09e4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.462+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f09dc04c650 con 0x7f09f40c7f00 2023-09-27T23:14:38.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.682+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f09f404be40 con 0x7f09f40c7f00 2023-09-27T23:14:38.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.682+0000 7f09eaffd700 1 -- 172.21.15.64:0/2724277153 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f09dc017090 con 0x7f09f40c7f00 2023-09-27T23:14:38.687 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:38.687 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:14:38.690 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.691 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.692 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.693 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.694 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.696 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.697 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:14:38.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:14:38.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2023-09-27T23:14:38.701 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAQtxRlepcyOxAASh9V6vkoFIIplux4kMPc7g==", 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAQtxRlnakyOxAAua57wrUcSeiPArP6HfXnDA==", 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.702 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAQtxRlHLcyOxAAxsCw3231N0/ijkLYYfdoBQ==", 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAQtxRljcMyOxAAMKUB8cQdTKPnkbsOra7w6A==", 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2023-09-27T23:14:38.703 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAQtxRl4tAyOxAAkZ6uFJg9Uh6ectaZ7ad7Bg==", 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAQtxRl094yOxAAXjWN7cdG/Ke2EOYjBCDUMA==", 2023-09-27T23:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:14:38.706 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060950 msgr2=0x7f09d4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060950 0x7f09d4062e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f09e40099b0 tx=0x7f09e4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 msgr2=0x7f09f40b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 0x7f09f40b1460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0011e0 tx=0x7f09dc007e10 comp rx=0 tx=0).stop 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 shutdown_connections 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060950 0x7f09d4062e00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f4155960 0x7f09f414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c7f00 0x7f09f40b1460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 --2- 172.21.15.64:0/2724277153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f406de20 0x7f09f415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 >> 172.21.15.64:0/2724277153 conn(0x7f09f4067bb0 msgr2=0x7f09f40c5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 shutdown_connections 2023-09-27T23:14:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.686+0000 7f09f3fff700 1 -- 172.21.15.64:0/2724277153 wait complete. 2023-09-27T23:14:38.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-09-27T23:14:38.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 Processor -- start 2023-09-27T23:14:38.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 -- start start 2023-09-27T23:14:38.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6f340c6900 0x7f6f340c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3406da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3406b470 con 0x7f6f3415cc20 2023-09-27T23:14:38.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3415d7b0 con 0x7f6f340c6900 2023-09-27T23:14:38.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.818+0000 7f6f38822700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f34161d10 con 0x7f6f3406b900 2023-09-27T23:14:38.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f3259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6f340c6900 0x7f6f340c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6f340c6900 0x7f6f340c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:39250/0 (socket says 172.21.15.64:39250) 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f3259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3406da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38614/0 (socket says 172.21.15.64:38614) 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 -- 172.21.15.64:0/2623109249 learned_addr learned my addr 172.21.15.64:0/2623109249 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f3359e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6f3415cc20 0x7f6f3415d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37810/0 (socket says 172.21.15.64:37810) 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 -- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 msgr2=0x7f6f3406da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 --2- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3406da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 -- 172.21.15.64:0/2623109249 >> v1:172.21.15.123:6789/0 conn(0x7f6f3415cc20 legacy=0x7f6f3415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f31d9b700 1 -- 172.21.15.64:0/2623109249 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294858113 0 0) 0x7f6f3406b470 con 0x7f6f3415cc20 2023-09-27T23:14:38.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 -- 172.21.15.64:0/2623109249 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3406bd70 con 0x7f6f340c6900 2023-09-27T23:14:38.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f3259c700 1 --2- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3406da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f32d9d700 1 --2- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f340c6cd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099e0 tx=0x7f6f24009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce36309b56c31f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f31d9b700 1 -- 172.21.15.64:0/2623109249 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400b820 con 0x7f6f340c6900 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f31d9b700 1 -- 172.21.15.64:0/2623109249 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2400b980 con 0x7f6f340c6900 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f31d9b700 1 -- 172.21.15.64:0/2623109249 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400bd20 con 0x7f6f340c6900 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 -- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 msgr2=0x7f6f340c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 --2- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f340c6cd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099e0 tx=0x7f6f24009250 comp rx=0 tx=0).stop 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 -- 172.21.15.64:0/2623109249 shutdown_connections 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 --2- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3406da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 --2- 172.21.15.64:0/2623109249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f340c6cd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 -- 172.21.15.64:0/2623109249 >> 172.21.15.64:0/2623109249 conn(0x7f6f34067bb0 msgr2=0x7f6f340cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:38.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 -- 172.21.15.64:0/2623109249 shutdown_connections 2023-09-27T23:14:38.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.822+0000 7f6f38822700 1 -- 172.21.15.64:0/2623109249 wait complete. 2023-09-27T23:14:38.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 Processor -- start 2023-09-27T23:14:38.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 -- start start 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3414eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f3415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3415cc20 0x7f6f3415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3415f0e0 con 0x7f6f3415cc20 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3415a070 con 0x7f6f340c6900 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f38822700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3415e490 con 0x7f6f3406b900 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3415cc20 0x7f6f3415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3415cc20 0x7f6f3415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51240/0 (socket says 172.21.15.64:51240) 2023-09-27T23:14:38.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f3415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f32d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3414eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3359e700 1 -- 172.21.15.64:0/1486847961 learned_addr learned my addr 172.21.15.64:0/1486847961 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 -- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 msgr2=0x7f6f3414eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3414eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 -- 172.21.15.64:0/1486847961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3415cc20 msgr2=0x7f6f3415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3415cc20 0x7f6f3415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 -- 172.21.15.64:0/1486847961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c009420 con 0x7f6f340c6900 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f3259c700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f3415d8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00ec80 tx=0x7f6f1c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.826+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00c850 con 0x7f6f340c6900 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c00c9b0 con 0x7f6f340c6900 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f240094f0 con 0x7f6f340c6900 2023-09-27T23:14:38.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00cd70 con 0x7f6f340c6900 2023-09-27T23:14:38.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3415ea90 con 0x7f6f340c6900 2023-09-27T23:14:38.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3415f250 con 0x7f6f340c6900 2023-09-27T23:14:38.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f6f1c013680 con 0x7f6f340c6900 2023-09-27T23:14:38.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f2affd700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6f14060950 0x7f6f14062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:38.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6f1c07ce90 con 0x7f6f340c6900 2023-09-27T23:14:38.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.830+0000 7f6f32d9d700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6f14060950 0x7f6f14062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:38.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.834+0000 7f6f32d9d700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6f14060950 0x7f6f14062e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099e0 tx=0x7f6f24017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:38.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:38.842+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6f1c04d300 con 0x7f6f340c6900 2023-09-27T23:14:39.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.062+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f6f3404be40 con 0x7f6f340c6900 2023-09-27T23:14:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f2affd700 1 -- 172.21.15.64:0/1486847961 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6f1c050920 con 0x7f6f340c6900 2023-09-27T23:14:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6f14060950 msgr2=0x7f6f14062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6f14060950 0x7f6f14062e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099e0 tx=0x7f6f24017000 comp rx=0 tx=0).stop 2023-09-27T23:14:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 msgr2=0x7f6f3415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f3415d8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00ec80 tx=0x7f6f1c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 shutdown_connections 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6f14060950 0x7f6f14062e00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f3415cc20 0x7f6f3415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f340c6900 0x7f6f3415d8f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 --2- 172.21.15.64:0/1486847961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f3406b900 0x7f6f3414eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 >> 172.21.15.64:0/1486847961 conn(0x7f6f34067bb0 msgr2=0x7f6f3415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 shutdown_connections 2023-09-27T23:14:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.074+0000 7f6f38822700 1 -- 172.21.15.64:0/1486847961 wait complete. 2023-09-27T23:14:39.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-09-27T23:14:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 Processor -- start 2023-09-27T23:14:39.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 -- start start 2023-09-27T23:14:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2d4155990 0x7fd2d40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2d40b0670 0x7fd2d40b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d406cf00 con 0x7fd2d40b13b0 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40ae7e0 con 0x7fd2d40b0670 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2da4b3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40cbc50 con 0x7fd2d4155990 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2d8cb0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2d40b0670 0x7fd2d40b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2d9cb2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd2d40b13b0 0x7fd2d406d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51596/0 (socket says 172.21.15.64:51596) 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2d94b1700 1 --2- 172.21.15.64:0/1725317924 >> v2:172.21.15.123:3300/0 conn(0x7fd2d4155990 0x7fd2d40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.194+0000 7fd2d9cb2700 1 -- 172.21.15.64:0/1725317924 learned_addr learned my addr 172.21.15.64:0/1725317924 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2d94b1700 1 -- 172.21.15.64:0/1725317924 >> v2:172.21.15.64:3300/0 conn(0x7fd2d40b0670 msgr2=0x7fd2d40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2d94b1700 1 --2- 172.21.15.64:0/1725317924 >> v2:172.21.15.64:3300/0 conn(0x7fd2d40b0670 0x7fd2d40b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2d94b1700 1 -- 172.21.15.64:0/1725317924 >> v1:172.21.15.64:6789/0 conn(0x7fd2d40b13b0 legacy=0x7fd2d406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2d94b1700 1 -- 172.21.15.64:0/1725317924 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40b17d0 con 0x7fd2d4155990 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2cbfff700 1 -- 172.21.15.64:0/1725317924 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064663985 0 0) 0x7fd2d406cf00 con 0x7fd2d40b13b0 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2d8cb0700 1 --2- 172.21.15.64:0/1725317924 >> v2:172.21.15.64:3300/0 conn(0x7fd2d40b0670 0x7fd2d40b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2d94b1700 1 --2- 172.21.15.64:0/1725317924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d4155990 0x7fd2d40c97d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2d406f0c0 tx=0x7fd2d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a05a6a4a9e898ab5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2cbfff700 1 -- 172.21.15.64:0/1725317924 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0011630 con 0x7fd2d4155990 2023-09-27T23:14:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2cbfff700 1 -- 172.21.15.64:0/1725317924 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0011790 con 0x7fd2d4155990 2023-09-27T23:14:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2cbfff700 1 -- 172.21.15.64:0/1725317924 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0011b30 con 0x7fd2d4155990 2023-09-27T23:14:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 -- 172.21.15.64:0/1725317924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d4155990 msgr2=0x7fd2d40c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/1725317924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d4155990 0x7fd2d40c97d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2d406f0c0 tx=0x7fd2d00091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 -- 172.21.15.64:0/1725317924 shutdown_connections 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/1725317924 >> v2:172.21.15.64:3300/0 conn(0x7fd2d40b0670 0x7fd2d40b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/1725317924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d4155990 0x7fd2d40c97d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 -- 172.21.15.64:0/1725317924 >> 172.21.15.64:0/1725317924 conn(0x7fd2d4067bb0 msgr2=0x7fd2d40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 -- 172.21.15.64:0/1725317924 shutdown_connections 2023-09-27T23:14:39.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.198+0000 7fd2da4b3700 1 -- 172.21.15.64:0/1725317924 wait complete. 2023-09-27T23:14:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 Processor -- start 2023-09-27T23:14:39.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 -- start start 2023-09-27T23:14:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 0x7fd2d4166360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 0x7fd2d4164f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2d4155990 0x7fd2d40ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4165ed0 con 0x7fd2d40b13b0 2023-09-27T23:14:39.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4156a90 con 0x7fd2d40b0670 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4156d70 con 0x7fd2d4155990 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 0x7fd2d4164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d94b1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 0x7fd2d4166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d94b1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 0x7fd2d4166360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:39272/0 (socket says 172.21.15.64:39272) 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d94b1700 1 -- 172.21.15.64:0/2927756253 learned_addr learned my addr 172.21.15.64:0/2927756253 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 0x7fd2d4164f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51258/0 (socket says 172.21.15.64:51258) 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d9cb2700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2d4155990 0x7fd2d40ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 -- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2d4155990 msgr2=0x7fd2d40ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2d4155990 0x7fd2d40ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 -- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 msgr2=0x7fd2d4166360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 0x7fd2d4166360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 -- 172.21.15.64:0/2927756253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009420 con 0x7fd2d40b13b0 2023-09-27T23:14:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d94b1700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 0x7fd2d4166360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2d8cb0700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 0x7fd2d4164f80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400ec80 tx=0x7fd2c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400c850 con 0x7fd2d40b13b0 2023-09-27T23:14:39.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d0009420 con 0x7fd2d40b13b0 2023-09-27T23:14:39.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400c9b0 con 0x7fd2d40b13b0 2023-09-27T23:14:39.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400cd70 con 0x7fd2d40b13b0 2023-09-27T23:14:39.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.202+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d4165b60 con 0x7fd2d40b13b0 2023-09-27T23:14:39.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.206+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fd2c4013680 con 0x7fd2d40b13b0 2023-09-27T23:14:39.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.206+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d4166080 con 0x7fd2d40b13b0 2023-09-27T23:14:39.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.206+0000 7fd2c97fa700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd2b8060950 0x7fd2b8062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.206+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd2c407cec0 con 0x7fd2d40b13b0 2023-09-27T23:14:39.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.206+0000 7fd2d94b1700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd2b8060950 0x7fd2b8062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.206+0000 7fd2d94b1700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd2b8060950 0x7fd2b8062e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00093c0 tx=0x7fd2d0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.214+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd2d4166080 con 0x7fd2d40b13b0 2023-09-27T23:14:39.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.434+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd2d404be40 con 0x7fd2d40b13b0 2023-09-27T23:14:39.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.442+0000 7fd2c97fa700 1 -- 172.21.15.64:0/2927756253 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd2c404d350 con 0x7fd2d40b13b0 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.442+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd2b8060950 msgr2=0x7fd2b8062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.442+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd2b8060950 0x7fd2b8062e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00093c0 tx=0x7fd2d0017000 comp rx=0 tx=0).stop 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.442+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 msgr2=0x7fd2d4164f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.442+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 0x7fd2d4164f80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400ec80 tx=0x7fd2c40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 shutdown_connections 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd2b8060950 0x7fd2b8062e00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2d4155990 0x7fd2d40ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2d40b13b0 0x7fd2d4164f80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 --2- 172.21.15.64:0/2927756253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d40b0670 0x7fd2d4166360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 >> 172.21.15.64:0/2927756253 conn(0x7fd2d4067bb0 msgr2=0x7fd2d4154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:39.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 shutdown_connections 2023-09-27T23:14:39.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.446+0000 7fd2da4b3700 1 -- 172.21.15.64:0/2927756253 wait complete. 2023-09-27T23:14:39.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-09-27T23:14:39.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 Processor -- start 2023-09-27T23:14:39.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 -- start start 2023-09-27T23:14:39.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f42bc155960 0x7f42bc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42bc0c7f00 0x7f42bc0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc06d990 con 0x7f42bc06de20 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc15a570 con 0x7f42bc0c7f00 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.570+0000 7f42c1381700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc06cd80 con 0x7f42bc155960 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42bc0c7f00 0x7f42bc0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42bc0c7f00 0x7f42bc0aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51262/0 (socket says 172.21.15.64:51262) 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 -- 172.21.15.64:0/3760074394 learned_addr learned my addr 172.21.15.64:0/3760074394 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42bbfff700 1 --2- 172.21.15.64:0/3760074394 >> v2:172.21.15.123:3300/0 conn(0x7f42bc155960 0x7f42bc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 -- 172.21.15.64:0/3760074394 >> v2:172.21.15.123:3300/0 conn(0x7f42bc155960 msgr2=0x7f42bc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 --2- 172.21.15.64:0/3760074394 >> v2:172.21.15.123:3300/0 conn(0x7f42bc155960 0x7f42bc15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 -- 172.21.15.64:0/3760074394 >> v1:172.21.15.123:6789/0 conn(0x7f42bc06de20 legacy=0x7f42bc0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 -- 172.21.15.64:0/3760074394 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc06d060 con 0x7f42bc0c7f00 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c0b80700 1 --2- 172.21.15.64:0/3760074394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc0c7f00 0x7f42bc0aca10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f42bc064070 tx=0x7f42b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520d9d5aaf76266c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42baffd700 1 -- 172.21.15.64:0/3760074394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b400e070 con 0x7f42bc0c7f00 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42baffd700 1 -- 172.21.15.64:0/3760074394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b400ba50 con 0x7f42bc0c7f00 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42baffd700 1 -- 172.21.15.64:0/3760074394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b40092e0 con 0x7f42bc0c7f00 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 -- 172.21.15.64:0/3760074394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc0c7f00 msgr2=0x7f42bc0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 --2- 172.21.15.64:0/3760074394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc0c7f00 0x7f42bc0aca10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f42bc064070 tx=0x7f42b4009840 comp rx=0 tx=0).stop 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 -- 172.21.15.64:0/3760074394 shutdown_connections 2023-09-27T23:14:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 --2- 172.21.15.64:0/3760074394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc0c7f00 0x7f42bc0aca10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 --2- 172.21.15.64:0/3760074394 >> v2:172.21.15.123:3300/0 conn(0x7f42bc155960 0x7f42bc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 -- 172.21.15.64:0/3760074394 >> 172.21.15.64:0/3760074394 conn(0x7f42bc067bb0 msgr2=0x7f42bc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:39.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 -- 172.21.15.64:0/3760074394 shutdown_connections 2023-09-27T23:14:39.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 -- 172.21.15.64:0/3760074394 wait complete. 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 Processor -- start 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.574+0000 7f42c1381700 1 -- start start 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 0x7f42bc155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42bc0c7f00 0x7f42bc169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42bc155960 0x7f42bc15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc16a3c0 con 0x7f42bc06de20 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc156f90 con 0x7f42bc0c7f00 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc15b850 con 0x7f42bc155960 2023-09-27T23:14:39.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 0x7f42bc155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 0x7f42bc155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51274/0 (socket says 172.21.15.64:51274) 2023-09-27T23:14:39.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 -- 172.21.15.64:0/1334842505 learned_addr learned my addr 172.21.15.64:0/1334842505 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:39.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bb7fe700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42bc0c7f00 0x7f42bc169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c0b80700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42bc155960 0x7f42bc15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 -- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42bc155960 msgr2=0x7f42bc15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42bc155960 0x7f42bc15b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 -- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42bc0c7f00 msgr2=0x7f42bc169e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42bc0c7f00 0x7f42bc169e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 -- 172.21.15.64:0/1334842505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b4009ae0 con 0x7f42bc06de20 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bb7fe700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42bc0c7f00 0x7f42bc169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42bbfff700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 0x7f42bc155d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00d860 tx=0x7f42ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0125f0 con 0x7f42bc06de20 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac012750 con 0x7f42bc06de20 2023-09-27T23:14:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc156380 con 0x7f42bc06de20 2023-09-27T23:14:39.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac012b10 con 0x7f42bc06de20 2023-09-27T23:14:39.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc156940 con 0x7f42bc06de20 2023-09-27T23:14:39.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.578+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc04be40 con 0x7f42bc06de20 2023-09-27T23:14:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.582+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f42ac01e460 con 0x7f42bc06de20 2023-09-27T23:14:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.582+0000 7f42b3fff700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f429c060950 0x7f429c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.582+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f42ac07d210 con 0x7f42bc06de20 2023-09-27T23:14:39.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.582+0000 7f42bb7fe700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f429c060950 0x7f429c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.582+0000 7f42bb7fe700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f429c060950 0x7f429c062e00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42bc14fa50 tx=0x7f42a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.590+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f42ac051130 con 0x7f42bc06de20 2023-09-27T23:14:39.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.806+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f42bc06d340 con 0x7f42bc06de20 2023-09-27T23:14:39.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.814+0000 7f42b3fff700 1 -- 172.21.15.64:0/1334842505 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f42ac050cc0 con 0x7f42bc06de20 2023-09-27T23:14:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f429c060950 msgr2=0x7f429c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f429c060950 0x7f429c062e00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42bc14fa50 tx=0x7f42a4009040 comp rx=0 tx=0).stop 2023-09-27T23:14:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 msgr2=0x7f42bc155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 0x7f42bc155d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00d860 tx=0x7f42ac008040 comp rx=0 tx=0).stop 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 shutdown_connections 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f429c060950 0x7f429c062e00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42bc155960 0x7f42bc15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42bc0c7f00 0x7f42bc169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 --2- 172.21.15.64:0/1334842505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42bc06de20 0x7f42bc155d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 >> 172.21.15.64:0/1334842505 conn(0x7f42bc067bb0 msgr2=0x7f42bc150000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 shutdown_connections 2023-09-27T23:14:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.818+0000 7f42c1381700 1 -- 172.21.15.64:0/1334842505 wait complete. 2023-09-27T23:14:39.836 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-09-27T23:14:39.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 Processor -- start 2023-09-27T23:14:39.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- start start 2023-09-27T23:14:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38ec06dc00 0x7f38ec06b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38ec06baa0 0x7f38ec0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec06b0d0 con 0x7f38ec158cc0 2023-09-27T23:14:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec06bec0 con 0x7f38ec06dc00 2023-09-27T23:14:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b4790 con 0x7f38ec06baa0 2023-09-27T23:14:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38ec06dc00 0x7f38ec06b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f38ec06dc00 0x7f38ec06b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51282/0 (socket says 172.21.15.64:51282) 2023-09-27T23:14:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 -- 172.21.15.64:0/4190427052 learned_addr learned my addr 172.21.15.64:0/4190427052 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38ebfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f38ec158cc0 0x7f38ec06d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51612/0 (socket says 172.21.15.64:51612) 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f0880700 1 --2- 172.21.15.64:0/4190427052 >> v2:172.21.15.123:3300/0 conn(0x7f38ec06baa0 0x7f38ec0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 -- 172.21.15.64:0/4190427052 >> v2:172.21.15.123:3300/0 conn(0x7f38ec06baa0 msgr2=0x7f38ec0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 --2- 172.21.15.64:0/4190427052 >> v2:172.21.15.123:3300/0 conn(0x7f38ec06baa0 0x7f38ec0b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 -- 172.21.15.64:0/4190427052 >> v1:172.21.15.64:6789/0 conn(0x7f38ec158cc0 legacy=0x7f38ec06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 -- 172.21.15.64:0/4190427052 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec06a7e0 con 0x7f38ec06dc00 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eb7fe700 1 --2- 172.21.15.64:0/4190427052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec06b560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06b040 tx=0x7f38d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142a795f90634a71 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eaffd700 1 -- 172.21.15.64:0/4190427052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400b860 con 0x7f38ec06dc00 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eaffd700 1 -- 172.21.15.64:0/4190427052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d400b9c0 con 0x7f38ec06dc00 2023-09-27T23:14:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38eaffd700 1 -- 172.21.15.64:0/4190427052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400bd60 con 0x7f38ec06dc00 2023-09-27T23:14:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- 172.21.15.64:0/4190427052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 msgr2=0x7f38ec06b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 --2- 172.21.15.64:0/4190427052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec06b560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06b040 tx=0x7f38d4009250 comp rx=0 tx=0).stop 2023-09-27T23:14:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- 172.21.15.64:0/4190427052 shutdown_connections 2023-09-27T23:14:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 --2- 172.21.15.64:0/4190427052 >> v2:172.21.15.123:3300/0 conn(0x7f38ec06baa0 0x7f38ec0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 --2- 172.21.15.64:0/4190427052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec06b560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.942+0000 7f38f22e3700 1 -- 172.21.15.64:0/4190427052 >> 172.21.15.64:0/4190427052 conn(0x7f38ec067bb0 msgr2=0x7f38ec0cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- 172.21.15.64:0/4190427052 shutdown_connections 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- 172.21.15.64:0/4190427052 wait complete. 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 Processor -- start 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- start start 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38ec06baa0 0x7f38ec15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38ec158cc0 0x7f38ec154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec155330 con 0x7f38ec06dc00 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec155610 con 0x7f38ec158cc0 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec1558f0 con 0x7f38ec06baa0 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51290/0 (socket says 172.21.15.64:51290) 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 -- 172.21.15.64:0/3064810118 learned_addr learned my addr 172.21.15.64:0/3064810118 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f0880700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38ec158cc0 0x7f38ec154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38ebfff700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38ec06baa0 0x7f38ec15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 -- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38ec06baa0 msgr2=0x7f38ec15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38ec06baa0 0x7f38ec15d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 -- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38ec158cc0 msgr2=0x7f38ec154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38ec158cc0 0x7f38ec154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 -- 172.21.15.64:0/3064810118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40094f0 con 0x7f38ec06dc00 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38eb7fe700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec154620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009290 tx=0x7f38d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4026450 con 0x7f38ec06dc00 2023-09-27T23:14:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d40265b0 con 0x7f38ec06dc00 2023-09-27T23:14:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4026970 con 0x7f38ec06dc00 2023-09-27T23:14:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec155bd0 con 0x7f38ec06dc00 2023-09-27T23:14:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.946+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec15dde0 con 0x7f38ec06dc00 2023-09-27T23:14:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.950+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec15e020 con 0x7f38ec06dc00 2023-09-27T23:14:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.950+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f38d401a420 con 0x7f38ec06dc00 2023-09-27T23:14:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.950+0000 7f38e3fff700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38cc060950 0x7f38cc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.950+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f38d407ea00 con 0x7f38ec06dc00 2023-09-27T23:14:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.950+0000 7f38ebfff700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38cc060950 0x7f38cc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:39.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.950+0000 7f38ebfff700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38cc060950 0x7f38cc062e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000c00 tx=0x7f38dc007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:39.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:39.958+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f38d404ef10 con 0x7f38ec06dc00 2023-09-27T23:14:40.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.174+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f38ec071120 con 0x7f38ec06dc00 2023-09-27T23:14:40.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38e3fff700 1 -- 172.21.15.64:0/3064810118 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f38d4052530 con 0x7f38ec06dc00 2023-09-27T23:14:40.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38cc060950 msgr2=0x7f38cc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38cc060950 0x7f38cc062e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000c00 tx=0x7f38dc007e70 comp rx=0 tx=0).stop 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 msgr2=0x7f38ec154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec154620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009290 tx=0x7f38d4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 shutdown_connections 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38cc060950 0x7f38cc062e00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38ec158cc0 0x7f38ec154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38ec06dc00 0x7f38ec154620 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 --2- 172.21.15.64:0/3064810118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38ec06baa0 0x7f38ec15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 >> 172.21.15.64:0/3064810118 conn(0x7f38ec067bb0 msgr2=0x7f38ec151290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:40.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 shutdown_connections 2023-09-27T23:14:40.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.182+0000 7f38f22e3700 1 -- 172.21.15.64:0/3064810118 wait complete. 2023-09-27T23:14:40.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-09-27T23:14:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.302+0000 7fc8df385700 1 Processor -- start 2023-09-27T23:14:40.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.302+0000 7fc8df385700 1 -- start start 2023-09-27T23:14:40.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc8d80c9720 0x7fc8d80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8d80b2030 0x7fc8d80ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d80b2490 con 0x7fc8d80c9350 2023-09-27T23:14:40.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d80ad9b0 con 0x7fc8d80c9720 2023-09-27T23:14:40.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d80adc90 con 0x7fc8d80b2030 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc8d80c9720 0x7fc8d80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc8d80c9720 0x7fc8d80b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51296/0 (socket says 172.21.15.64:51296) 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 -- 172.21.15.64:0/384212707 learned_addr learned my addr 172.21.15.64:0/384212707 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dd922700 1 --2- 172.21.15.64:0/384212707 >> v2:172.21.15.123:3300/0 conn(0x7fc8d80b2030 0x7fc8d80ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 -- 172.21.15.64:0/384212707 >> v2:172.21.15.123:3300/0 conn(0x7fc8d80b2030 msgr2=0x7fc8d80ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 --2- 172.21.15.64:0/384212707 >> v2:172.21.15.123:3300/0 conn(0x7fc8d80b2030 0x7fc8d80ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 -- 172.21.15.64:0/384212707 >> v1:172.21.15.123:6789/0 conn(0x7fc8d80c9350 legacy=0x7fc8d80afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 -- 172.21.15.64:0/384212707 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80c9b70 con 0x7fc8d80c9720 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8dc920700 1 --2- 172.21.15.64:0/384212707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9720 0x7fc8d80b1af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8d814d570 tx=0x7fc8c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d65194fdb2df2fd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8cffff700 1 -- 172.21.15.64:0/384212707 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800b820 con 0x7fc8d80c9720 2023-09-27T23:14:40.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8cffff700 1 -- 172.21.15.64:0/384212707 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c800b980 con 0x7fc8d80c9720 2023-09-27T23:14:40.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8cffff700 1 -- 172.21.15.64:0/384212707 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800bd20 con 0x7fc8d80c9720 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- 172.21.15.64:0/384212707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9720 msgr2=0x7fc8d80b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 --2- 172.21.15.64:0/384212707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9720 0x7fc8d80b1af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8d814d570 tx=0x7fc8c8009250 comp rx=0 tx=0).stop 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- 172.21.15.64:0/384212707 shutdown_connections 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 --2- 172.21.15.64:0/384212707 >> v2:172.21.15.123:3300/0 conn(0x7fc8d80b2030 0x7fc8d80ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 --2- 172.21.15.64:0/384212707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9720 0x7fc8d80b1af0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- 172.21.15.64:0/384212707 >> 172.21.15.64:0/384212707 conn(0x7fc8d8069310 msgr2=0x7fc8d80696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- 172.21.15.64:0/384212707 shutdown_connections 2023-09-27T23:14:40.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- 172.21.15.64:0/384212707 wait complete. 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 Processor -- start 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.306+0000 7fc8df385700 1 -- start start 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8d80b2030 0x7fc8d80c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 0x7fc8d816cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8d80c9720 0x7fc8d8155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d814d650 con 0x7fc8d80c9350 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d8158230 con 0x7fc8d80c9720 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d81588f0 con 0x7fc8d80b2030 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 0x7fc8d816cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 0x7fc8d816cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51308/0 (socket says 172.21.15.64:51308) 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 -- 172.21.15.64:0/60773551 learned_addr learned my addr 172.21.15.64:0/60773551 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:40.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dd121700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8d80b2030 0x7fc8d80c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dd922700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8d80c9720 0x7fc8d8155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 -- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8d80b2030 msgr2=0x7fc8d80c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8d80b2030 0x7fc8d80c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 -- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8d80c9720 msgr2=0x7fc8d8155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8d80c9720 0x7fc8d8155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 -- 172.21.15.64:0/60773551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80094f0 con 0x7fc8d80c9350 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8dc920700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 0x7fc8d816cd40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc8d806ec70 tx=0x7fc8c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800bbc0 con 0x7fc8d80c9350 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8020070 con 0x7fc8d80c9350 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d8158bd0 con 0x7fc8d80c9350 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c801b6a0 con 0x7fc8d80c9350 2023-09-27T23:14:40.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d8159190 con 0x7fc8d80c9350 2023-09-27T23:14:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fc8c8022030 con 0x7fc8d80c9350 2023-09-27T23:14:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8cd7fa700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8bc060950 0x7fc8bc062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc8c807eae0 con 0x7fc8d80c9350 2023-09-27T23:14:40.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.310+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d80b2490 con 0x7fc8d80c9350 2023-09-27T23:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.314+0000 7fc8dd121700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8bc060950 0x7fc8bc062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.314+0000 7fc8dd121700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8bc060950 0x7fc8bc062e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4003870 tx=0x7fc8d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:40.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.322+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc8d80b2490 con 0x7fc8d80c9350 2023-09-27T23:14:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.538+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc8d80b2490 con 0x7fc8d80c9350 2023-09-27T23:14:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.546+0000 7fc8cd7fa700 1 -- 172.21.15.64:0/60773551 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc8d80b2490 con 0x7fc8d80c9350 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8bc060950 msgr2=0x7fc8bc062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8bc060950 0x7fc8bc062e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4003870 tx=0x7fc8d4009040 comp rx=0 tx=0).stop 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 msgr2=0x7fc8d816cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 0x7fc8d816cd40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc8d806ec70 tx=0x7fc8c8016000 comp rx=0 tx=0).stop 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 shutdown_connections 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8bc060950 0x7fc8bc062e00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8d80c9720 0x7fc8d8155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8d80c9350 0x7fc8d816cd40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 --2- 172.21.15.64:0/60773551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8d80b2030 0x7fc8d80c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 >> 172.21.15.64:0/60773551 conn(0x7fc8d8069310 msgr2=0x7fc8d814c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 shutdown_connections 2023-09-27T23:14:40.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.550+0000 7fc8df385700 1 -- 172.21.15.64:0/60773551 wait complete. 2023-09-27T23:14:40.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-09-27T23:14:40.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.674+0000 7f3659be1700 1 Processor -- start 2023-09-27T23:14:40.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.674+0000 7f3659be1700 1 -- start start 2023-09-27T23:14:40.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36540acc30 0x7f36540ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f36540c8280 0x7f36540c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36540cc920 con 0x7f36540ad600 2023-09-27T23:14:40.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36540c90a0 con 0x7f36540acc30 2023-09-27T23:14:40.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36540c9290 con 0x7f36540c8280 2023-09-27T23:14:40.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36540acc30 0x7f36540ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36540acc30 0x7f36540ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51314/0 (socket says 172.21.15.64:51314) 2023-09-27T23:14:40.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 -- 172.21.15.64:0/4158296943 learned_addr learned my addr 172.21.15.64:0/4158296943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:40.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3653fff700 1 --2- 172.21.15.64:0/4158296943 >> v2:172.21.15.123:3300/0 conn(0x7f36540c8280 0x7f36540c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 -- 172.21.15.64:0/4158296943 >> v2:172.21.15.123:3300/0 conn(0x7f36540c8280 msgr2=0x7f36540c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 --2- 172.21.15.64:0/4158296943 >> v2:172.21.15.123:3300/0 conn(0x7f36540c8280 0x7f36540c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 -- 172.21.15.64:0/4158296943 >> v1:172.21.15.123:6789/0 conn(0x7f36540ad600 legacy=0x7f365415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 -- 172.21.15.64:0/4158296943 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540c86d0 con 0x7f36540acc30 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3658bdf700 1 --2- 172.21.15.64:0/4158296943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540acc30 0x7f36540ad000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3654087020 tx=0x7f3648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141ec8019141ff1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f36537fe700 1 -- 172.21.15.64:0/4158296943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648009e70 con 0x7f36540acc30 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f36537fe700 1 -- 172.21.15.64:0/4158296943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364800b8f0 con 0x7f36540acc30 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f36537fe700 1 -- 172.21.15.64:0/4158296943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364800bc90 con 0x7f36540acc30 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- 172.21.15.64:0/4158296943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540acc30 msgr2=0x7f36540ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 --2- 172.21.15.64:0/4158296943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540acc30 0x7f36540ad000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3654087020 tx=0x7f3648009250 comp rx=0 tx=0).stop 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- 172.21.15.64:0/4158296943 shutdown_connections 2023-09-27T23:14:40.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 --2- 172.21.15.64:0/4158296943 >> v2:172.21.15.123:3300/0 conn(0x7f36540c8280 0x7f36540c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 --2- 172.21.15.64:0/4158296943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540acc30 0x7f36540ad000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- 172.21.15.64:0/4158296943 >> 172.21.15.64:0/4158296943 conn(0x7f36540cdbd0 msgr2=0x7f36540cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:40.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- 172.21.15.64:0/4158296943 shutdown_connections 2023-09-27T23:14:40.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.678+0000 7f3659be1700 1 -- 172.21.15.64:0/4158296943 wait complete. 2023-09-27T23:14:40.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 Processor -- start 2023-09-27T23:14:40.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 -- start start 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 0x7f365414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36540c8280 0x7f3654158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3654168170 0x7f3654152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365416b7c0 con 0x7f36540ad600 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540b0c40 con 0x7f36540c8280 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3659be1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365416abb0 con 0x7f3654168170 2023-09-27T23:14:40.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 0x7f365414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 0x7f365414a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51322/0 (socket says 172.21.15.64:51322) 2023-09-27T23:14:40.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3653fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36540c8280 0x7f3654158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 -- 172.21.15.64:0/2381343466 learned_addr learned my addr 172.21.15.64:0/2381343466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:40.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f36593e0700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3654168170 0x7f3654152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 -- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3654168170 msgr2=0x7f3654152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3654168170 0x7f3654152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 -- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36540c8280 msgr2=0x7f3654158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36540c8280 0x7f3654158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 -- 172.21.15.64:0/2381343466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480094f0 con 0x7f36540ad600 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3653fff700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36540c8280 0x7f3654158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.682+0000 7f3658bdf700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 0x7f365414a470 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3648006fd0 tx=0x7f3648017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364800bc70 con 0x7f36540ad600 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364800be40 con 0x7f36540ad600 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365416ae90 con 0x7f36540ad600 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648024670 con 0x7f36540ad600 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365416b450 con 0x7f36540ad600 2023-09-27T23:14:40.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365404be40 con 0x7f36540ad600 2023-09-27T23:14:40.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.686+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f364801a420 con 0x7f36540ad600 2023-09-27T23:14:40.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.690+0000 7f3650ff9700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3640060950 0x7f3640062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:40.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.690+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f364807e120 con 0x7f36540ad600 2023-09-27T23:14:40.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.690+0000 7f3653fff700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3640060950 0x7f3640062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:40.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.690+0000 7f3653fff700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3640060950 0x7f3640062e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36540aee90 tx=0x7f36440091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.698+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f364804e510 con 0x7f36540ad600 2023-09-27T23:14:40.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.914+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f36540cb870 con 0x7f36540ad600 2023-09-27T23:14:40.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3650ff9700 1 -- 172.21.15.64:0/2381343466 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3648051b30 con 0x7f36540ad600 2023-09-27T23:14:40.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3640060950 msgr2=0x7f3640062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3640060950 0x7f3640062e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36540aee90 tx=0x7f36440091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 msgr2=0x7f365414a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 0x7f365414a470 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3648006fd0 tx=0x7f3648017000 comp rx=0 tx=0).stop 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 shutdown_connections 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3640060950 0x7f3640062e00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3654168170 0x7f3654152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36540c8280 0x7f3654158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 --2- 172.21.15.64:0/2381343466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36540ad600 0x7f365414a470 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 >> 172.21.15.64:0/2381343466 conn(0x7f36540cdbd0 msgr2=0x7f3654154490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 shutdown_connections 2023-09-27T23:14:40.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:40.922+0000 7f3659be1700 1 -- 172.21.15.64:0/2381343466 wait complete. 2023-09-27T23:14:40.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:41.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 Processor -- start 2023-09-27T23:14:41.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- start start 2023-09-27T23:14:41.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7bf015cf70 0x7f7bf00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7bf015d340 0x7f7bf006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf006cf40 con 0x7f7bf00cbd00 2023-09-27T23:14:41.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf00b0890 con 0x7f7bf015cf70 2023-09-27T23:14:41.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf00b21d0 con 0x7f7bf015d340 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7bf015cf70 0x7f7bf00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7bf015cf70 0x7f7bf00afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51334/0 (socket says 172.21.15.64:51334) 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 -- 172.21.15.64:0/652949759 learned_addr learned my addr 172.21.15.64:0/652949759 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bedd9b700 1 --2- 172.21.15.64:0/652949759 >> v2:172.21.15.123:3300/0 conn(0x7f7bf015d340 0x7f7bf006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 -- 172.21.15.64:0/652949759 >> v2:172.21.15.123:3300/0 conn(0x7f7bf015d340 msgr2=0x7f7bf006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 --2- 172.21.15.64:0/652949759 >> v2:172.21.15.123:3300/0 conn(0x7f7bf015d340 0x7f7bf006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 -- 172.21.15.64:0/652949759 >> v1:172.21.15.64:6789/0 conn(0x7f7bf00cbd00 legacy=0x7f7bf0160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 -- 172.21.15.64:0/652949759 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf00b0330 con 0x7f7bf015cf70 2023-09-27T23:14:41.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bee59c700 1 --2- 172.21.15.64:0/652949759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015cf70 0x7f7bf00afdf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7bf006c3a0 tx=0x7f7bd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38206454ccb53c71 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bed59a700 1 -- 172.21.15.64:0/652949759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800b860 con 0x7f7bf015cf70 2023-09-27T23:14:41.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bed59a700 1 -- 172.21.15.64:0/652949759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd800b9c0 con 0x7f7bf015cf70 2023-09-27T23:14:41.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bed59a700 1 -- 172.21.15.64:0/652949759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800bd60 con 0x7f7bf015cf70 2023-09-27T23:14:41.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- 172.21.15.64:0/652949759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015cf70 msgr2=0x7f7bf00afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 --2- 172.21.15.64:0/652949759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015cf70 0x7f7bf00afdf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7bf006c3a0 tx=0x7f7bd8009250 comp rx=0 tx=0).stop 2023-09-27T23:14:41.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- 172.21.15.64:0/652949759 shutdown_connections 2023-09-27T23:14:41.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 --2- 172.21.15.64:0/652949759 >> v2:172.21.15.123:3300/0 conn(0x7f7bf015d340 0x7f7bf006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 --2- 172.21.15.64:0/652949759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015cf70 0x7f7bf00afdf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- 172.21.15.64:0/652949759 >> 172.21.15.64:0/652949759 conn(0x7f7bf0067bb0 msgr2=0x7f7bf00add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- 172.21.15.64:0/652949759 shutdown_connections 2023-09-27T23:14:41.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.050+0000 7f7bef59e700 1 -- 172.21.15.64:0/652949759 wait complete. 2023-09-27T23:14:41.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 Processor -- start 2023-09-27T23:14:41.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 -- start start 2023-09-27T23:14:41.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bf00cbd00 0x7f7bf0171730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015d340 0x7f7bf015af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf015b4d0 0x7f7bf0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf0171c70 con 0x7f7bf015d340 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015e850 con 0x7f7bf015b4d0 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bef59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015d7b0 con 0x7f7bf00cbd00 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.054+0000 7f7bedd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015d340 0x7f7bf015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7bedd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015d340 0x7f7bf015af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51336/0 (socket says 172.21.15.64:51336) 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7bedd9b700 1 -- 172.21.15.64:0/1254716997 learned_addr learned my addr 172.21.15.64:0/1254716997 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf015b4d0 0x7f7bf0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7bee59c700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bf00cbd00 0x7f7bf0171730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 -- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bf00cbd00 msgr2=0x7f7bf0171730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bf00cbd00 0x7f7bf0171730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 -- 172.21.15.64:0/1254716997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015d340 msgr2=0x7f7bf015af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015d340 0x7f7bf015af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 -- 172.21.15.64:0/1254716997 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80094f0 con 0x7f7bf015b4d0 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7beed9d700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf015b4d0 0x7f7bf0156840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c830 tx=0x7f7be8001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8012630 con 0x7f7bf015b4d0 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be8012790 con 0x7f7bf015b4d0 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf015da90 con 0x7f7bf015b4d0 2023-09-27T23:14:41.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8012b50 con 0x7f7bf015b4d0 2023-09-27T23:14:41.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf015de20 con 0x7f7bf015b4d0 2023-09-27T23:14:41.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.058+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf015dc20 con 0x7f7bf015b4d0 2023-09-27T23:14:41.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.062+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7be801a3a0 con 0x7f7bf015b4d0 2023-09-27T23:14:41.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.062+0000 7f7be67fc700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7bd0060950 0x7f7bd0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.062+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7be807cdf0 con 0x7f7bf015b4d0 2023-09-27T23:14:41.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.062+0000 7f7bee59c700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7bd0060950 0x7f7bd0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.062+0000 7f7bee59c700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7bd0060950 0x7f7bd0062e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bf006c3a0 tx=0x7f7bd8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.070+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7be804d1e0 con 0x7f7bf015b4d0 2023-09-27T23:14:41.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.238+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf00b05b0 con 0x7f7bf015b4d0 2023-09-27T23:14:41.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.238+0000 7f7be67fc700 1 -- 172.21.15.64:0/1254716997 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f7be8050800 con 0x7f7bf015b4d0 2023-09-27T23:14:41.244 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:41.244 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:38.300890+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7bd0060950 msgr2=0x7f7bd0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7bd0060950 0x7f7bd0062e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bf006c3a0 tx=0x7f7bd8017000 comp rx=0 tx=0).stop 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf015b4d0 msgr2=0x7f7bf0156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf015b4d0 0x7f7bf0156840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c830 tx=0x7f7be8001690 comp rx=0 tx=0).stop 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 shutdown_connections 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7bd0060950 0x7f7bd0062e00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf015b4d0 0x7f7bf0156840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bf015d340 0x7f7bf015af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 --2- 172.21.15.64:0/1254716997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bf00cbd00 0x7f7bf0171730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 >> 172.21.15.64:0/1254716997 conn(0x7f7bf0067bb0 msgr2=0x7f7bf00cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 shutdown_connections 2023-09-27T23:14:41.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.242+0000 7f7bef59e700 1 -- 172.21.15.64:0/1254716997 wait complete. 2023-09-27T23:14:41.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:14:41.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:14:41.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 Processor -- start 2023-09-27T23:14:41.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 -- start start 2023-09-27T23:14:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83300ce2d0 0x7f83300ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83300cc040 0x7f8330159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83300cbbb0 con 0x7f83300caa10 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833006dc90 con 0x7f83300cc040 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83300cafa0 con 0x7f83300ce2d0 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83300cc040 0x7f8330159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83300cc040 0x7f8330159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51346/0 (socket says 172.21.15.64:51346) 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 -- 172.21.15.64:0/2013315518 learned_addr learned my addr 172.21.15.64:0/2013315518 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832dd9b700 1 --1- 172.21.15.64:0/2013315518 >> v1:172.21.15.64:6789/0 conn(0x7f83300caa10 0x7f8330158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51630/0 (socket says 172.21.15.64:51630) 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832e59c700 1 --2- 172.21.15.64:0/2013315518 >> v2:172.21.15.123:3300/0 conn(0x7f83300ce2d0 0x7f83300ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 -- 172.21.15.64:0/2013315518 >> v2:172.21.15.123:3300/0 conn(0x7f83300ce2d0 msgr2=0x7f83300ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 --2- 172.21.15.64:0/2013315518 >> v2:172.21.15.123:3300/0 conn(0x7f83300ce2d0 0x7f83300ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 -- 172.21.15.64:0/2013315518 >> v1:172.21.15.64:6789/0 conn(0x7f83300caa10 legacy=0x7f8330158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 -- 172.21.15.64:0/2013315518 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300cb280 con 0x7f83300cc040 2023-09-27T23:14:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832ed9d700 1 --2- 172.21.15.64:0/2013315518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300cc040 0x7f8330159010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f833006eb00 tx=0x7f8328009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e90381502757ac server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832d59a700 1 -- 172.21.15.64:0/2013315518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800e070 con 0x7f83300cc040 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832d59a700 1 -- 172.21.15.64:0/2013315518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832800ba20 con 0x7f83300cc040 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.374+0000 7f832d59a700 1 -- 172.21.15.64:0/2013315518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280092e0 con 0x7f83300cc040 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 -- 172.21.15.64:0/2013315518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300cc040 msgr2=0x7f8330159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 --2- 172.21.15.64:0/2013315518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300cc040 0x7f8330159010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f833006eb00 tx=0x7f8328009840 comp rx=0 tx=0).stop 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 -- 172.21.15.64:0/2013315518 shutdown_connections 2023-09-27T23:14:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 --2- 172.21.15.64:0/2013315518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300cc040 0x7f8330159010 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 --2- 172.21.15.64:0/2013315518 >> v2:172.21.15.123:3300/0 conn(0x7f83300ce2d0 0x7f83300ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 -- 172.21.15.64:0/2013315518 >> 172.21.15.64:0/2013315518 conn(0x7f8330067bb0 msgr2=0x7f83300cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 -- 172.21.15.64:0/2013315518 shutdown_connections 2023-09-27T23:14:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 -- 172.21.15.64:0/2013315518 wait complete. 2023-09-27T23:14:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.378+0000 7f832f59e700 1 Processor -- start 2023-09-27T23:14:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 -- start start 2023-09-27T23:14:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 0x7f83300c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300cc040 0x7f8330154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300ce2d0 0x7f833015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8330159cf0 con 0x7f83300caa10 2023-09-27T23:14:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300c95d0 con 0x7f83300ce2d0 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83301593c0 con 0x7f83300cc040 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 0x7f83300c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 0x7f83300c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51348/0 (socket says 172.21.15.64:51348) 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 -- 172.21.15.64:0/2367809034 learned_addr learned my addr 172.21.15.64:0/2367809034 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832ed9d700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300ce2d0 0x7f833015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832dd9b700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300cc040 0x7f8330154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 -- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300cc040 msgr2=0x7f8330154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300cc040 0x7f8330154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 -- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300ce2d0 msgr2=0x7f833015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300ce2d0 0x7f833015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 -- 172.21.15.64:0/2367809034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328009ae0 con 0x7f83300caa10 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832ed9d700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300ce2d0 0x7f833015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832dd9b700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300cc040 0x7f8330154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832e59c700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 0x7f83300c9090 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f832000e950 tx=0x7f83200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320009530 con 0x7f83300caa10 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832000c850 con 0x7f83300caa10 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83301596a0 con 0x7f83300caa10 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000cc10 con 0x7f83300caa10 2023-09-27T23:14:41.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.382+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83301590b0 con 0x7f83300caa10 2023-09-27T23:14:41.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.386+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83301550f0 con 0x7f83300caa10 2023-09-27T23:14:41.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.386+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8320013470 con 0x7f83300caa10 2023-09-27T23:14:41.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.386+0000 7f83267fc700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8310060980 0x7f8310062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.386+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f832007cdf0 con 0x7f83300caa10 2023-09-27T23:14:41.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.386+0000 7f832dd9b700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8310060980 0x7f8310062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.386+0000 7f832dd9b700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8310060980 0x7f8310062e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83300b1ee0 tx=0x7f8318009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.394+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f832004d280 con 0x7f83300caa10 2023-09-27T23:14:41.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.550+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f833004be40 con 0x7f83300caa10 2023-09-27T23:14:41.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f83267fc700 1 -- 172.21.15.64:0/2367809034 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83200508a0 con 0x7f83300caa10 2023-09-27T23:14:41.557 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:14:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8310060980 msgr2=0x7f8310062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8310060980 0x7f8310062e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83300b1ee0 tx=0x7f8318009040 comp rx=0 tx=0).stop 2023-09-27T23:14:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 msgr2=0x7f83300c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 0x7f83300c9090 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f832000e950 tx=0x7f83200091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 shutdown_connections 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8310060980 0x7f8310062e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83300ce2d0 0x7f833015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83300cc040 0x7f8330154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 --2- 172.21.15.64:0/2367809034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83300caa10 0x7f83300c9090 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 >> 172.21.15.64:0/2367809034 conn(0x7f8330067bb0 msgr2=0x7f83300c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 shutdown_connections 2023-09-27T23:14:41.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.554+0000 7f832f59e700 1 -- 172.21.15.64:0/2367809034 wait complete. 2023-09-27T23:14:41.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:14:41.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 Processor -- start 2023-09-27T23:14:41.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 -- start start 2023-09-27T23:14:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19780acf30 0x7f19781557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f197806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19780cc0d0 0x7f1978159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19781532a0 con 0x7f19780acf30 2023-09-27T23:14:41.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1978156780 con 0x7f19780cc0d0 2023-09-27T23:14:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197da48700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1978158f90 con 0x7f19780ad300 2023-09-27T23:14:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197ca46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19780acf30 0x7f19781557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197ca46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19780acf30 0x7f19781557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51358/0 (socket says 172.21.15.64:51358) 2023-09-27T23:14:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197ca46700 1 -- 172.21.15.64:0/2595727984 learned_addr learned my addr 172.21.15.64:0/2595727984 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f197d247700 1 --2- 172.21.15.64:0/2595727984 >> v2:172.21.15.123:3300/0 conn(0x7f19780cc0d0 0x7f1978159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.682+0000 7f1977fff700 1 --2- 172.21.15.64:0/2595727984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f197806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197ca46700 1 -- 172.21.15.64:0/2595727984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 msgr2=0x7f197806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197ca46700 1 --2- 172.21.15.64:0/2595727984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f197806d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197ca46700 1 -- 172.21.15.64:0/2595727984 >> v2:172.21.15.123:3300/0 conn(0x7f19780cc0d0 msgr2=0x7f1978159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197ca46700 1 --2- 172.21.15.64:0/2595727984 >> v2:172.21.15.123:3300/0 conn(0x7f19780cc0d0 0x7f1978159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197ca46700 1 -- 172.21.15.64:0/2595727984 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978155ee0 con 0x7f19780acf30 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197d247700 1 --2- 172.21.15.64:0/2595727984 >> v2:172.21.15.123:3300/0 conn(0x7f19780cc0d0 0x7f1978159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197ca46700 1 --2- 172.21.15.64:0/2595727984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780acf30 0x7f19781557d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f19780b1c20 tx=0x7f1968009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c562abec8c8312d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f19777fe700 1 -- 172.21.15.64:0/2595727984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800b640 con 0x7f19780acf30 2023-09-27T23:14:41.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f19777fe700 1 -- 172.21.15.64:0/2595727984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196800b7a0 con 0x7f19780acf30 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f19777fe700 1 -- 172.21.15.64:0/2595727984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800bb60 con 0x7f19780acf30 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 -- 172.21.15.64:0/2595727984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780acf30 msgr2=0x7f19781557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 --2- 172.21.15.64:0/2595727984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780acf30 0x7f19781557d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f19780b1c20 tx=0x7f1968009250 comp rx=0 tx=0).stop 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 -- 172.21.15.64:0/2595727984 shutdown_connections 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 --2- 172.21.15.64:0/2595727984 >> v2:172.21.15.123:3300/0 conn(0x7f19780cc0d0 0x7f1978159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 --2- 172.21.15.64:0/2595727984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f197806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 --2- 172.21.15.64:0/2595727984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780acf30 0x7f19781557d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 -- 172.21.15.64:0/2595727984 >> 172.21.15.64:0/2595727984 conn(0x7f1978067b50 msgr2=0x7f19780cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 -- 172.21.15.64:0/2595727984 shutdown_connections 2023-09-27T23:14:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 -- 172.21.15.64:0/2595727984 wait complete. 2023-09-27T23:14:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 Processor -- start 2023-09-27T23:14:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.686+0000 7f197da48700 1 -- start start 2023-09-27T23:14:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 0x7f19781407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f1978144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 0x7f19780b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1978145cd0 con 0x7f19780cc0d0 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1978145150 con 0x7f19780acf30 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1978145430 con 0x7f19780ad300 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 0x7f19780b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197ca46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 0x7f19781407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197ca46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 0x7f19781407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:39394/0 (socket says 172.21.15.64:39394) 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197ca46700 1 -- 172.21.15.64:0/1121446216 learned_addr learned my addr 172.21.15.64:0/1121446216 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 0x7f19780b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51368/0 (socket says 172.21.15.64:51368) 2023-09-27T23:14:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f1977fff700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f1978144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 -- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 msgr2=0x7f1978144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f1978144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 -- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 msgr2=0x7f19781407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 0x7f19781407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 -- 172.21.15.64:0/1121446216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970009a20 con 0x7f19780cc0d0 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197ca46700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 0x7f19781407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197d247700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 0x7f19780b0db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1970009f10 tx=0x7f197000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000cb70 con 0x7f19780cc0d0 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000f040 con 0x7f19780cc0d0 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19680094f0 con 0x7f19780cc0d0 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970014480 con 0x7f19780cc0d0 2023-09-27T23:14:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781459f0 con 0x7f19780cc0d0 2023-09-27T23:14:41.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.690+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19781568c0 con 0x7f19780cc0d0 2023-09-27T23:14:41.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.694+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f1970003020 con 0x7f19780cc0d0 2023-09-27T23:14:41.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.694+0000 7f1974ff9700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f195c060950 0x7f195c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.694+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f197007d4c0 con 0x7f19780cc0d0 2023-09-27T23:14:41.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.694+0000 7f197ca46700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f195c060950 0x7f195c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.694+0000 7f197ca46700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f195c060950 0x7f195c062e00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1968003080 tx=0x7f1968017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.702+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f19781568c0 con 0x7f19780cc0d0 2023-09-27T23:14:41.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.866+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f19781568c0 con 0x7f19780cc0d0 2023-09-27T23:14:41.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.866+0000 7f1974ff9700 1 -- 172.21.15.64:0/1121446216 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f19781568c0 con 0x7f19780cc0d0 2023-09-27T23:14:41.871 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:14:41.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f195c060950 msgr2=0x7f195c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f195c060950 0x7f195c062e00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1968003080 tx=0x7f1968017000 comp rx=0 tx=0).stop 2023-09-27T23:14:41.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 msgr2=0x7f19780b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 0x7f19780b0db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1970009f10 tx=0x7f197000b040 comp rx=0 tx=0).stop 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 shutdown_connections 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f195c060950 0x7f195c062e00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19780cc0d0 0x7f19780b0db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19780ad300 0x7f1978144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 --2- 172.21.15.64:0/1121446216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19780acf30 0x7f19781407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 >> 172.21.15.64:0/1121446216 conn(0x7f1978067b50 msgr2=0x7f19780c7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 shutdown_connections 2023-09-27T23:14:41.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.870+0000 7f197da48700 1 -- 172.21.15.64:0/1121446216 wait complete. 2023-09-27T23:14:41.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:14:41.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 Processor -- start 2023-09-27T23:14:41.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 -- start start 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53000c6900 0x7f53000c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f530006b900 0x7f530006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530006b470 con 0x7f530015cc20 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530015d790 con 0x7f530006b900 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5306282700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5300161d10 con 0x7f53000c6900 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f530006b900 0x7f530006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f530006b900 0x7f530006da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51374/0 (socket says 172.21.15.64:51374) 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 -- 172.21.15.64:0/691047223 learned_addr learned my addr 172.21.15.64:0/691047223 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5305280700 1 --2- 172.21.15.64:0/691047223 >> v2:172.21.15.123:3300/0 conn(0x7f53000c6900 0x7f53000c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 -- 172.21.15.64:0/691047223 >> v2:172.21.15.123:3300/0 conn(0x7f53000c6900 msgr2=0x7f53000c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:41.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 --2- 172.21.15.64:0/691047223 >> v2:172.21.15.123:3300/0 conn(0x7f53000c6900 0x7f53000c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 -- 172.21.15.64:0/691047223 >> v1:172.21.15.123:6789/0 conn(0x7f530015cc20 legacy=0x7f530015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 -- 172.21.15.64:0/691047223 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530006bd70 con 0x7f530006b900 2023-09-27T23:14:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f5304a7f700 1 --2- 172.21.15.64:0/691047223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f530006b900 0x7f530006da00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f530006c570 tx=0x7f52fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1228e3722e04cc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f52f7fff700 1 -- 172.21.15.64:0/691047223 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00b860 con 0x7f530006b900 2023-09-27T23:14:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f52f7fff700 1 -- 172.21.15.64:0/691047223 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc00b9c0 con 0x7f530006b900 2023-09-27T23:14:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.986+0000 7f52f7fff700 1 -- 172.21.15.64:0/691047223 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00bd60 con 0x7f530006b900 2023-09-27T23:14:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/691047223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f530006b900 msgr2=0x7f530006da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 --2- 172.21.15.64:0/691047223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f530006b900 0x7f530006da00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f530006c570 tx=0x7f52fc009250 comp rx=0 tx=0).stop 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/691047223 shutdown_connections 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 --2- 172.21.15.64:0/691047223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f530006b900 0x7f530006da00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 --2- 172.21.15.64:0/691047223 >> v2:172.21.15.123:3300/0 conn(0x7f53000c6900 0x7f53000c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/691047223 >> 172.21.15.64:0/691047223 conn(0x7f5300067bb0 msgr2=0x7f53000cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/691047223 shutdown_connections 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/691047223 wait complete. 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 Processor -- start 2023-09-27T23:14:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- start start 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f530006b900 0x7f530014ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 0x7f53000c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f530015cc20 0x7f53000c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530014b1c0 con 0x7f53000c6900 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000afd40 con 0x7f530015cc20 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000af130 con 0x7f530006b900 2023-09-27T23:14:41.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 0x7f53000c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 0x7f53000c0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51380/0 (socket says 172.21.15.64:51380) 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 -- 172.21.15.64:0/1832934795 learned_addr learned my addr 172.21.15.64:0/1832934795 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5305280700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f530006b900 0x7f530014ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5305a81700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f530015cc20 0x7f53000c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 -- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f530006b900 msgr2=0x7f530014ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f530006b900 0x7f530014ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 -- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f530015cc20 msgr2=0x7f53000c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f530015cc20 0x7f53000c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 -- 172.21.15.64:0/1832934795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0094f0 con 0x7f53000c6900 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5305280700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f530006b900 0x7f530014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5304a7f700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 0x7f53000c0860 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009290 tx=0x7f52fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc026450 con 0x7f53000c6900 2023-09-27T23:14:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0265b0 con 0x7f53000c6900 2023-09-27T23:14:41.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53000af410 con 0x7f53000c6900 2023-09-27T23:14:41.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc026950 con 0x7f53000c6900 2023-09-27T23:14:41.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.990+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53000af9d0 con 0x7f53000c6900 2023-09-27T23:14:41.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.994+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000ae440 con 0x7f53000c6900 2023-09-27T23:14:41.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.994+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f52fc01a420 con 0x7f53000c6900 2023-09-27T23:14:42.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.994+0000 7f52f57fa700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f52e4060950 0x7f52e4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.994+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f52fc07eae0 con 0x7f53000c6900 2023-09-27T23:14:42.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.994+0000 7f5305280700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f52e4060950 0x7f52e4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:41.998+0000 7f5305280700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f52e4060950 0x7f52e4062e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f52f0006fd0 tx=0x7f52f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:42.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.002+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f53000ae440 con 0x7f53000c6900 2023-09-27T23:14:42.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.166+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f53000ae440 con 0x7f53000c6900 2023-09-27T23:14:42.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f52f57fa700 1 -- 172.21.15.64:0/1832934795 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+126 (secure 0 0 0) 0x7f53000ae440 con 0x7f53000c6900 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:14:42.174 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:42.175 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:14:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f52e4060950 msgr2=0x7f52e4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f52e4060950 0x7f52e4062e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f52f0006fd0 tx=0x7f52f0008040 comp rx=0 tx=0).stop 2023-09-27T23:14:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 msgr2=0x7f53000c0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 0x7f53000c0860 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009290 tx=0x7f52fc017000 comp rx=0 tx=0).stop 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 shutdown_connections 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f52e4060950 0x7f52e4062e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f530015cc20 0x7f53000c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53000c6900 0x7f53000c0860 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 --2- 172.21.15.64:0/1832934795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f530006b900 0x7f530014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 >> 172.21.15.64:0/1832934795 conn(0x7f5300067bb0 msgr2=0x7f53000cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 shutdown_connections 2023-09-27T23:14:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.170+0000 7f5306282700 1 -- 172.21.15.64:0/1832934795 wait complete. 2023-09-27T23:14:42.193 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:14:42.193 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:14:42.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 Processor -- start 2023-09-27T23:14:42.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 -- start start 2023-09-27T23:14:42.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59a8158b30 0x7f59a80ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a80c7a00 0x7f59a815d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a815d870 con 0x7f59a8158f00 2023-09-27T23:14:42.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a80cacc0 con 0x7f59a8158b30 2023-09-27T23:14:42.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.298+0000 7f59ae867700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a80c7f70 con 0x7f59a80c7a00 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae066700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f59a8158f00 0x7f59a80c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37878/0 (socket says 172.21.15.64:37878) 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae066700 1 -- 172.21.15.64:0/496592849 learned_addr learned my addr 172.21.15.64:0/496592849 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad064700 1 --2- 172.21.15.64:0/496592849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a80c7a00 0x7f59a815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad865700 1 --2- 172.21.15.64:0/496592849 >> v2:172.21.15.123:3300/0 conn(0x7f59a8158b30 0x7f59a80ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ac863700 1 -- 172.21.15.64:0/496592849 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163433121 0 0) 0x7f59a0001490 con 0x7f59a8158f00 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ac863700 1 -- 172.21.15.64:0/496592849 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5994003570 con 0x7f59a8158f00 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad064700 1 -- 172.21.15.64:0/496592849 >> v2:172.21.15.123:3300/0 conn(0x7f59a8158b30 msgr2=0x7f59a80ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad064700 1 --2- 172.21.15.64:0/496592849 >> v2:172.21.15.123:3300/0 conn(0x7f59a8158b30 0x7f59a80ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad064700 1 -- 172.21.15.64:0/496592849 >> v1:172.21.15.123:6789/0 conn(0x7f59a8158f00 legacy=0x7f59a80c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:42.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad064700 1 -- 172.21.15.64:0/496592849 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a80caf10 con 0x7f59a80c7a00 2023-09-27T23:14:42.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ad064700 1 --2- 172.21.15.64:0/496592849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a80c7a00 0x7f59a815d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59980098e0 tx=0x7f59980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b3dd4d883ef17d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ac863700 1 -- 172.21.15.64:0/496592849 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998011630 con 0x7f59a80c7a00 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ac863700 1 -- 172.21.15.64:0/496592849 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998011790 con 0x7f59a80c7a00 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ac863700 1 -- 172.21.15.64:0/496592849 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998011b30 con 0x7f59a80c7a00 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 -- 172.21.15.64:0/496592849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a80c7a00 msgr2=0x7f59a815d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 --2- 172.21.15.64:0/496592849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a80c7a00 0x7f59a815d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59980098e0 tx=0x7f59980091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 -- 172.21.15.64:0/496592849 shutdown_connections 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 --2- 172.21.15.64:0/496592849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a80c7a00 0x7f59a815d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 --2- 172.21.15.64:0/496592849 >> v2:172.21.15.123:3300/0 conn(0x7f59a8158b30 0x7f59a80ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 -- 172.21.15.64:0/496592849 >> 172.21.15.64:0/496592849 conn(0x7f59a806b5b0 msgr2=0x7f59a806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 -- 172.21.15.64:0/496592849 shutdown_connections 2023-09-27T23:14:42.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.302+0000 7f59ae867700 1 -- 172.21.15.64:0/496592849 wait complete. 2023-09-27T23:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 Processor -- start 2023-09-27T23:14:42.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 -- start start 2023-09-27T23:14:42.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 0x7f59a8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a8158b30 0x7f59a80c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59a8158f00 0x7f59a80c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a815bc30 con 0x7f59a80c7a00 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8156250 con 0x7f59a8158f00 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae867700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a81567a0 con 0x7f59a8158b30 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ad865700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 0x7f59a8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ad865700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 0x7f59a8155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51390/0 (socket says 172.21.15.64:51390) 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ad865700 1 -- 172.21.15.64:0/1764430933 learned_addr learned my addr 172.21.15.64:0/1764430933 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ad064700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a8158b30 0x7f59a80c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ae066700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59a8158f00 0x7f59a80c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.306+0000 7f59ad865700 1 -- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a8158b30 msgr2=0x7f59a80c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:42.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ad865700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a8158b30 0x7f59a80c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ad865700 1 -- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59a8158f00 msgr2=0x7f59a80c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ad865700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59a8158f00 0x7f59a80c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ad865700 1 -- 172.21.15.64:0/1764430933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998009420 con 0x7f59a80c7a00 2023-09-27T23:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ae066700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59a8158f00 0x7f59a80c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ad865700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 0x7f59a8155d10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ec80 tx=0x7f59a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:42.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400c850 con 0x7f59a80c7a00 2023-09-27T23:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400c9b0 con 0x7f59a80c7a00 2023-09-27T23:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a80c4170 con 0x7f59a80c7a00 2023-09-27T23:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400cd50 con 0x7f59a80c7a00 2023-09-27T23:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a815b8c0 con 0x7f59a80c7a00 2023-09-27T23:14:42.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.310+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a804be40 con 0x7f59a80c7a00 2023-09-27T23:14:42.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.314+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f59a4013680 con 0x7f59a80c7a00 2023-09-27T23:14:42.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.314+0000 7f599dffb700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f598c060950 0x7f598c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:42.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.314+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f59a407cf80 con 0x7f59a80c7a00 2023-09-27T23:14:42.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.314+0000 7f59ad064700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f598c060950 0x7f598c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:42.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.314+0000 7f59ad064700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f598c060950 0x7f598c062e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59980091e0 tx=0x7f5998017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:42.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.322+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f59a40acbb0 con 0x7f59a80c7a00 2023-09-27T23:14:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:42.482+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f59a80ca870 con 0x7f59a80c7a00 2023-09-27T23:14:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.190+0000 7f599dffb700 1 -- 172.21.15.64:0/1764430933 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f59a40170b0 con 0x7f59a80c7a00 2023-09-27T23:14:43.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f598c060950 msgr2=0x7f598c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:43.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f598c060950 0x7f598c062e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59980091e0 tx=0x7f5998017000 comp rx=0 tx=0).stop 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 msgr2=0x7f59a8155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 0x7f59a8155d10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ec80 tx=0x7f59a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 shutdown_connections 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f598c060950 0x7f598c062e00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59a8158f00 0x7f59a80c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a8158b30 0x7f59a80c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 --2- 172.21.15.64:0/1764430933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a80c7a00 0x7f59a8155d10 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ec80 tx=0x7f59a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.194+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 >> 172.21.15.64:0/1764430933 conn(0x7f59a806b5b0 msgr2=0x7f59a81573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:43.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.198+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 shutdown_connections 2023-09-27T23:14:43.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.198+0000 7f59ae867700 1 -- 172.21.15.64:0/1764430933 wait complete. 2023-09-27T23:14:43.205 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:14:43.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:14:43.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.330+0000 7f7923fff700 1 Processor -- start 2023-09-27T23:14:43.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.330+0000 7f7923fff700 1 -- start start 2023-09-27T23:14:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7923fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7923fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79240ab6d0 0x7f79240abb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7923fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792406b730 con 0x7f79240b1440 2023-09-27T23:14:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7923fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240ac390 con 0x7f79240ab6d0 2023-09-27T23:14:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7923fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79240ac670 con 0x7f792406ae20 2023-09-27T23:14:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792406b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38728/0 (socket says 172.21.15.64:38728) 2023-09-27T23:14:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 -- 172.21.15.64:0/2246295892 learned_addr learned my addr 172.21.15.64:0/2246295892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f79227fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79240ab6d0 0x7f79240abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7921ffb700 1 -- 172.21.15.64:0/2246295892 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764269598 0 0) 0x7f792406b730 con 0x7f79240b1440 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7921ffb700 1 -- 172.21.15.64:0/2246295892 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c003570 con 0x7f79240b1440 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 -- 172.21.15.64:0/2246295892 >> v2:172.21.15.123:3300/0 conn(0x7f79240ab6d0 msgr2=0x7f79240abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 --2- 172.21.15.64:0/2246295892 >> v2:172.21.15.123:3300/0 conn(0x7f79240ab6d0 0x7f79240abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 -- 172.21.15.64:0/2246295892 >> v1:172.21.15.123:6789/0 conn(0x7f79240b1440 legacy=0x7f79240b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 -- 172.21.15.64:0/2246295892 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240ac950 con 0x7f792406ae20 2023-09-27T23:14:43.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f79227fc700 1 --2- 172.21.15.64:0/2246295892 >> v2:172.21.15.123:3300/0 conn(0x7f79240ab6d0 0x7f79240abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7922ffd700 1 --2- 172.21.15.64:0/2246295892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792406b1f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79140099b0 tx=0x7f7914009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be15ef18abb3b866 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7921ffb700 1 -- 172.21.15.64:0/2246295892 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791400b680 con 0x7f792406ae20 2023-09-27T23:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7921ffb700 1 -- 172.21.15.64:0/2246295892 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791400b7e0 con 0x7f792406ae20 2023-09-27T23:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.334+0000 7f7921ffb700 1 -- 172.21.15.64:0/2246295892 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791400bb80 con 0x7f792406ae20 2023-09-27T23:14:43.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 -- 172.21.15.64:0/2246295892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 msgr2=0x7f792406b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 --2- 172.21.15.64:0/2246295892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792406b1f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79140099b0 tx=0x7f7914009250 comp rx=0 tx=0).stop 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 -- 172.21.15.64:0/2246295892 shutdown_connections 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 --2- 172.21.15.64:0/2246295892 >> v2:172.21.15.123:3300/0 conn(0x7f79240ab6d0 0x7f79240abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 --2- 172.21.15.64:0/2246295892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792406b1f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 -- 172.21.15.64:0/2246295892 >> 172.21.15.64:0/2246295892 conn(0x7f79240c8cd0 msgr2=0x7f79240c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 -- 172.21.15.64:0/2246295892 shutdown_connections 2023-09-27T23:14:43.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 -- 172.21.15.64:0/2246295892 wait complete. 2023-09-27T23:14:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 Processor -- start 2023-09-27T23:14:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.338+0000 7f7923fff700 1 -- start start 2023-09-27T23:14:43.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:43.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79240ab6d0 0x7f7924157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 0x7f7924145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924155380 con 0x7f79240b1440 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924158d20 con 0x7f79240ab6d0 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7924158110 con 0x7f792406ae20 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 0x7f7924145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 0x7f7924145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51394/0 (socket says 172.21.15.64:51394) 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 -- 172.21.15.64:0/1425899811 learned_addr learned my addr 172.21.15.64:0/1425899811 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7922ffd700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79227fc700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79240ab6d0 0x7f7924157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 -- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 msgr2=0x7f792415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 -- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79240ab6d0 msgr2=0x7f7924157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:43.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79240ab6d0 0x7f7924157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:43.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 -- 172.21.15.64:0/1425899811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918009420 con 0x7f79240b1440 2023-09-27T23:14:43.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7922ffd700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:43.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79227fc700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79240ab6d0 0x7f7924157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:43.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f79237fe700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 0x7f7924145390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c910 tx=0x7f791c0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c00a7e0 con 0x7f79240b1440 2023-09-27T23:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c00a960 con 0x7f79240b1440 2023-09-27T23:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79140094f0 con 0x7f79240b1440 2023-09-27T23:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c00ad20 con 0x7f79240b1440 2023-09-27T23:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.342+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79241589b0 con 0x7f79240b1440 2023-09-27T23:14:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.346+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7924158eb0 con 0x7f79240b1440 2023-09-27T23:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.346+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f791c012630 con 0x7f79240b1440 2023-09-27T23:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.346+0000 7f790affd700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7910060950 0x7f7910062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:43.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.346+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f791c07d1c0 con 0x7f79240b1440 2023-09-27T23:14:43.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.346+0000 7f7922ffd700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7910060950 0x7f7910062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:43.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.346+0000 7f7922ffd700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7910060950 0x7f7910062e00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79140099b0 tx=0x7f7914017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:43.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.354+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f791c04d4e0 con 0x7f79240b1440 2023-09-27T23:14:43.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:43.514+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f792404be40 con 0x7f79240b1440 2023-09-27T23:14:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.194+0000 7f790affd700 1 -- 172.21.15.64:0/1425899811 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) v1 ==== 170+0+0 (secure 0 0 0) 0x7f791c056020 con 0x7f79240b1440 2023-09-27T23:14:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7910060950 msgr2=0x7f7910062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7910060950 0x7f7910062e00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79140099b0 tx=0x7f7914017000 comp rx=0 tx=0).stop 2023-09-27T23:14:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 msgr2=0x7f7924145390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 0x7f7924145390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c910 tx=0x7f791c0027a0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 shutdown_connections 2023-09-27T23:14:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7910060950 0x7f7910062e00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79240b1440 0x7f7924145390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79240ab6d0 0x7f7924157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 --2- 172.21.15.64:0/1425899811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f792406ae20 0x7f792415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 >> 172.21.15.64:0/1425899811 conn(0x7f79240c8cd0 msgr2=0x7f79240b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 shutdown_connections 2023-09-27T23:14:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.218+0000 7f7923fff700 1 -- 172.21.15.64:0/1425899811 wait complete. 2023-09-27T23:14:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:14:44.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:14:44.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.338+0000 7f4b75f68700 1 Processor -- start 2023-09-27T23:14:44.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.338+0000 7f4b75f68700 1 -- start start 2023-09-27T23:14:44.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b75f68700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 0x7f4b700c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b75f68700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7006a7a0 con 0x7f4b7006ac30 2023-09-27T23:14:44.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b75f68700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b700c7630 con 0x7f4b7006c7c0 2023-09-27T23:14:44.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b75f68700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7006d4b0 con 0x7f4b700c6d20 2023-09-27T23:14:44.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4b7006ac30 0x7f4b7006b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51634/0 (socket says 172.21.15.64:51634) 2023-09-27T23:14:44.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2469136259 learned_addr learned my addr 172.21.15.64:0/2469136259 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:44.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6f7fe700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 0x7f4b700c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677631804 0 0) 0x7f4b7006a7a0 con 0x7f4b7006ac30 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b5c003570 con 0x7f4b7006ac30 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719580106 0 0) 0x7f4b700c7630 con 0x7f4b7006c7c0 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b7006a7a0 con 0x7f4b7006c7c0 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657720817 0 0) 0x7f4b5c003570 con 0x7f4b7006ac30 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 msgr2=0x7f4b700c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 0x7f4b700c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 >> v1:172.21.15.123:6789/0 conn(0x7f4b7006c7c0 legacy=0x7f4b7006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7006d790 con 0x7f4b7006ac30 2023-09-27T23:14:44.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f4b680017f0 con 0x7f4b7006ac30 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 >> v1:172.21.15.64:6789/0 conn(0x7f4b7006ac30 legacy=0x7f4b7006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b5c003c20 0x7f4b5c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 0x7f4b5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5c004d50 0x7f4b5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7006d4b0 con 0x7f4b5c004d50 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c00bb40 con 0x7f4b5c003c20 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c00bd00 con 0x7f4b700c6d20 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5c004d50 0x7f4b5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6f7fe700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b5c003c20 0x7f4b5c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6effd700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 0x7f4b5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 msgr2=0x7f4b5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:44.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b700c6d20 0x7f4b5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b5c003c20 msgr2=0x7f4b5c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b5c003c20 0x7f4b5c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2469136259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c003570 con 0x7f4b5c004d50 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5c004d50 0x7f4b5c0051c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800a8f0 tx=0x7f4b68002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579e72dbe8952811 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680070d0 con 0x7f4b5c004d50 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68007230 con 0x7f4b5c004d50 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.342+0000 7f4b6e7fc700 1 -- 172.21.15.64:0/2469136259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680075f0 con 0x7f4b5c004d50 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- 172.21.15.64:0/2469136259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5c004d50 msgr2=0x7f4b5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2469136259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5c004d50 0x7f4b5c0051c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800a8f0 tx=0x7f4b68002190 comp rx=0 tx=0).stop 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2469136259 reap_dead start 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- 172.21.15.64:0/2469136259 shutdown_connections 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- 172.21.15.64:0/2469136259 >> 172.21.15.64:0/2469136259 conn(0x7f4b70067bb0 msgr2=0x7f4b700cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- 172.21.15.64:0/2469136259 shutdown_connections 2023-09-27T23:14:44.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- 172.21.15.64:0/2469136259 wait complete. 2023-09-27T23:14:44.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 Processor -- start 2023-09-27T23:14:44.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- start start 2023-09-27T23:14:44.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b70166d50 0x7f4b7016b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b70167120 0x7f4b700adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 0x7f4b701560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7016baf0 con 0x7f4b70155c90 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700c2d80 con 0x7f4b70167120 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b75f68700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700c35e0 con 0x7f4b70166d50 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 0x7f4b701560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 0x7f4b701560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51408/0 (socket says 172.21.15.64:51408) 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b70167120 0x7f4b700adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2129234500 learned_addr learned my addr 172.21.15.64:0/2129234500 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6f7fe700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b70166d50 0x7f4b7016b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b70166d50 msgr2=0x7f4b7016b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b70166d50 0x7f4b7016b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b70167120 msgr2=0x7f4b700adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b70167120 0x7f4b700adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 -- 172.21.15.64:0/2129234500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700ae090 con 0x7f4b70155c90 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.346+0000 7f4b6ffff700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 0x7f4b701560e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b68012000 tx=0x7f4b68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68023070 con 0x7f4b70155c90 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68001eb0 con 0x7f4b70155c90 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b70156620 con 0x7f4b70155c90 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680070d0 con 0x7f4b70155c90 2023-09-27T23:14:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b700c2f90 con 0x7f4b70155c90 2023-09-27T23:14:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b700c3770 con 0x7f4b70155c90 2023-09-27T23:14:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4b6801e430 con 0x7f4b70155c90 2023-09-27T23:14:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b677fe700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b50060970 0x7f4b50062e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.350+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f4b6807f1f0 con 0x7f4b70155c90 2023-09-27T23:14:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.354+0000 7f4b6f7fe700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b50060970 0x7f4b50062e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.354+0000 7f4b6f7fe700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b50060970 0x7f4b50062e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009db0 tx=0x7f4b60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:44.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.358+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4b6804f390 con 0x7f4b70155c90 2023-09-27T23:14:44.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.558+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4b70071120 con 0x7f4b70155c90 2023-09-27T23:14:44.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.578+0000 7f4b677fe700 1 -- 172.21.15.64:0/2129234500 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4b680075b0 con 0x7f4b70155c90 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b50060970 msgr2=0x7f4b50062e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b50060970 0x7f4b50062e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009db0 tx=0x7f4b60008040 comp rx=0 tx=0).stop 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 msgr2=0x7f4b701560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 0x7f4b701560e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b68012000 tx=0x7f4b68016000 comp rx=0 tx=0).stop 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 shutdown_connections 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4b50060970 0x7f4b50062e20 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009db0 tx=0x7f4b60008040 comp rx=0 tx=0).stop 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b70155c90 0x7f4b701560e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b70167120 0x7f4b700adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 --2- 172.21.15.64:0/2129234500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b70166d50 0x7f4b7016b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 >> 172.21.15.64:0/2129234500 conn(0x7f4b70067bb0 msgr2=0x7f4b70153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 shutdown_connections 2023-09-27T23:14:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.586+0000 7f4b75f68700 1 -- 172.21.15.64:0/2129234500 wait complete. 2023-09-27T23:14:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 6 and data pool 7 2023-09-27T23:14:44.616 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:44.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 Processor -- start 2023-09-27T23:14:44.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 -- start start 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c7c0af9c0 0x7f9c7c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c7c0af530 con 0x7f9c7c06be10 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7c159470 con 0x7f9c7c0af9c0 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c159660 con 0x7f9c7c0b0810 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c7c0af9c0 0x7f9c7c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7a59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c7c0af9c0 0x7f9c7c154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57076/0 (socket says 172.21.15.64:57076) 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7a59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c0b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:46786/0 (socket says 172.21.15.64:46786) 2023-09-27T23:14:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 -- 172.21.15.64:0/2323764463 learned_addr learned my addr 172.21.15.64:0/2323764463 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:44.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 -- 172.21.15.64:0/2323764463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 msgr2=0x7f9c7c0b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:44.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7959a700 1 -- 172.21.15.64:0/2323764463 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391101498 0 0) 0x7f9c7c0af530 con 0x7f9c7c06be10 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 --2- 172.21.15.64:0/2323764463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c0b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 -- 172.21.15.64:0/2323764463 >> v1:172.21.15.123:6789/0 conn(0x7f9c7c06be10 legacy=0x7f9c7c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.710+0000 7f9c7ad9d700 1 -- 172.21.15.64:0/2323764463 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c159940 con 0x7f9c7c0af9c0 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7ad9d700 1 --2- 172.21.15.64:0/2323764463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c154b20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009f50 tx=0x7f9c74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7999eca56a98dd51 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7959a700 1 -- 172.21.15.64:0/2323764463 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400e070 con 0x7f9c7c0af9c0 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7959a700 1 -- 172.21.15.64:0/2323764463 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7400ba50 con 0x7f9c7c0af9c0 2023-09-27T23:14:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7959a700 1 -- 172.21.15.64:0/2323764463 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740092e0 con 0x7f9c7c0af9c0 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 -- 172.21.15.64:0/2323764463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c0af9c0 msgr2=0x7f9c7c154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/2323764463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c154b20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009f50 tx=0x7f9c74009840 comp rx=0 tx=0).stop 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 -- 172.21.15.64:0/2323764463 shutdown_connections 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/2323764463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c154b20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/2323764463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 -- 172.21.15.64:0/2323764463 >> 172.21.15.64:0/2323764463 conn(0x7f9c7c067bb0 msgr2=0x7f9c7c0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 -- 172.21.15.64:0/2323764463 shutdown_connections 2023-09-27T23:14:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 -- 172.21.15.64:0/2323764463 wait complete. 2023-09-27T23:14:44.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 Processor -- start 2023-09-27T23:14:44.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.714+0000 7f9c7b59e700 1 -- start start 2023-09-27T23:14:44.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 0x7f9c7c14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c0cc080 con 0x7f9c7c06be10 2023-09-27T23:14:44.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c0cb1d0 con 0x7f9c7c0af9c0 2023-09-27T23:14:44.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c0cb470 con 0x7f9c7c0b0810 2023-09-27T23:14:44.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 0x7f9c7c14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 0x7f9c7c14a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57078/0 (socket says 172.21.15.64:57078) 2023-09-27T23:14:44.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 -- 172.21.15.64:0/3759774021 learned_addr learned my addr 172.21.15.64:0/3759774021 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:44.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c79d9b700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7ad9d700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 -- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 msgr2=0x7f9c7c150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 -- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0af9c0 msgr2=0x7f9c7c14fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c14fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 -- 172.21.15.64:0/3759774021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74009ae0 con 0x7f9c7c06be10 2023-09-27T23:14:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7a59c700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 0x7f9c7c14a8f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00e9a0 tx=0x7f9c6c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00a620 con 0x7f9c7c06be10 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c00c740 con 0x7f9c7c06be10 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7c0cb750 con 0x7f9c7c06be10 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00cac0 con 0x7f9c7c06be10 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7c0cbd10 con 0x7f9c7c06be10 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.718+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7c04be40 con 0x7f9c7c06be10 2023-09-27T23:14:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.722+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f9c6c0113f0 con 0x7f9c7c06be10 2023-09-27T23:14:44.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.722+0000 7f9c6a7fc700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9c5c060950 0x7f9c5c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:44.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.722+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9c6c015030 con 0x7f9c7c06be10 2023-09-27T23:14:44.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.722+0000 7f9c79d9b700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9c5c060950 0x7f9c5c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:44.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.722+0000 7f9c79d9b700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9c5c060950 0x7f9c5c062e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c15ff70 tx=0x7f9c70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:44.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.726+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9c6c04d590 con 0x7f9c7c06be10 2023-09-27T23:14:44.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.890+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c7c159c20 con 0x7f9c7c06be10 2023-09-27T23:14:44.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.890+0000 7f9c6a7fc700 1 -- 172.21.15.64:0/3759774021 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f9c6c050bb0 con 0x7f9c7c06be10 2023-09-27T23:14:44.897 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:44.897 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:44.562121+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:14:42.525604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-09-27T23:14:43.561136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9c5c060950 msgr2=0x7f9c5c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9c5c060950 0x7f9c5c062e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c15ff70 tx=0x7f9c70009040 comp rx=0 tx=0).stop 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 msgr2=0x7f9c7c14a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 0x7f9c7c14a8f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00e9a0 tx=0x7f9c6c00a330 comp rx=0 tx=0).stop 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 shutdown_connections 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9c5c060950 0x7f9c5c062e00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c7c0b0810 0x7f9c7c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c7c0af9c0 0x7f9c7c14fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 --2- 172.21.15.64:0/3759774021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c7c06be10 0x7f9c7c14a8f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:44.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 >> 172.21.15.64:0/3759774021 conn(0x7f9c7c067bb0 msgr2=0x7f9c7c153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:44.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 shutdown_connections 2023-09-27T23:14:44.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:44.894+0000 7f9c7b59e700 1 -- 172.21.15.64:0/3759774021 wait complete. 2023-09-27T23:14:44.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:45.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.014+0000 7f5f6490e700 1 Processor -- start 2023-09-27T23:14:45.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.014+0000 7f5f6490e700 1 -- start start 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6cd0 0x7f5f6006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f6015cc20 0x7f5f6015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6006b470 con 0x7f5f600c6900 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6006be40 con 0x7f5f6015cc20 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6006e110 con 0x7f5f600c6cd0 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f6015cc20 0x7f5f6015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5f6015cc20 0x7f5f6015d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57084/0 (socket says 172.21.15.64:57084) 2023-09-27T23:14:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 -- 172.21.15.64:0/1222722834 learned_addr learned my addr 172.21.15.64:0/1222722834 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:45.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5e59c700 1 --2- 172.21.15.64:0/1222722834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6cd0 0x7f5f6006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 -- 172.21.15.64:0/1222722834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6cd0 msgr2=0x7f5f6006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:45.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 --2- 172.21.15.64:0/1222722834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6cd0 0x7f5f6006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 -- 172.21.15.64:0/1222722834 >> v1:172.21.15.123:6789/0 conn(0x7f5f600c6900 legacy=0x7f5f6006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 -- 172.21.15.64:0/1222722834 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f60161d10 con 0x7f5f6015cc20 2023-09-27T23:14:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5f59e700 1 --2- 172.21.15.64:0/1222722834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015d0a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009f50 tx=0x7f5f58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fa0c4946939b70 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5dd9b700 1 -- 172.21.15.64:0/1222722834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800e070 con 0x7f5f6015cc20 2023-09-27T23:14:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5dd9b700 1 -- 172.21.15.64:0/1222722834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5800b9c0 con 0x7f5f6015cc20 2023-09-27T23:14:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f5dd9b700 1 -- 172.21.15.64:0/1222722834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f580092e0 con 0x7f5f6015cc20 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- 172.21.15.64:0/1222722834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 msgr2=0x7f5f6015d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 --2- 172.21.15.64:0/1222722834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015d0a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009f50 tx=0x7f5f58009840 comp rx=0 tx=0).stop 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- 172.21.15.64:0/1222722834 shutdown_connections 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 --2- 172.21.15.64:0/1222722834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015d0a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 --2- 172.21.15.64:0/1222722834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6cd0 0x7f5f6006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- 172.21.15.64:0/1222722834 >> 172.21.15.64:0/1222722834 conn(0x7f5f60067bb0 msgr2=0x7f5f600cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- 172.21.15.64:0/1222722834 shutdown_connections 2023-09-27T23:14:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.018+0000 7f5f6490e700 1 -- 172.21.15.64:0/1222722834 wait complete. 2023-09-27T23:14:45.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 Processor -- start 2023-09-27T23:14:45.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 -- start start 2023-09-27T23:14:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6900 0x7f5f60149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f600c6cd0 0x7f5f600af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600b0200 con 0x7f5f6015cc20 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600afc40 con 0x7f5f600c6cd0 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f6490e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600aff20 con 0x7f5f600c6900 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57094/0 (socket says 172.21.15.64:57094) 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 -- 172.21.15.64:0/3448245202 learned_addr learned my addr 172.21.15.64:0/3448245202 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5e59c700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f600c6cd0 0x7f5f600af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5ed9d700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6900 0x7f5f60149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 -- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6900 msgr2=0x7f5f60149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6900 0x7f5f60149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 -- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f600c6cd0 msgr2=0x7f5f600af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f600c6cd0 0x7f5f600af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 -- 172.21.15.64:0/3448245202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58009ae0 con 0x7f5f6015cc20 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.022+0000 7f5f5f59e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015f240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009880 tx=0x7f5f58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58008ba0 con 0x7f5f6015cc20 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f58008d00 con 0x7f5f6015cc20 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6015d0c0 con 0x7f5f6015cc20 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58025690 con 0x7f5f6015cc20 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6014a010 con 0x7f5f6015cc20 2023-09-27T23:14:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6004be40 con 0x7f5f6015cc20 2023-09-27T23:14:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f5f58003040 con 0x7f5f6015cc20 2023-09-27T23:14:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f56ffd700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5f40060980 0x7f5f40062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5f5807eaa0 con 0x7f5f6015cc20 2023-09-27T23:14:45.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.026+0000 7f5f5ed9d700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5f40060980 0x7f5f40062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.030+0000 7f5f5ed9d700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5f40060980 0x7f5f40062e30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f50003a80 tx=0x7f5f50007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.034+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5f5804ebe0 con 0x7f5f6015cc20 2023-09-27T23:14:45.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.194+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f6006a580 con 0x7f5f6015cc20 2023-09-27T23:14:45.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f56ffd700 1 -- 172.21.15.64:0/3448245202 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f5f58052200 con 0x7f5f6015cc20 2023-09-27T23:14:45.202 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:45.202 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":43,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:44.562121+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:14:42.525604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-09-27T23:14:43.561136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5f40060980 msgr2=0x7f5f40062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5f40060980 0x7f5f40062e30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f50003a80 tx=0x7f5f50007ec0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 msgr2=0x7f5f6015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015f240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009880 tx=0x7f5f58018000 comp rx=0 tx=0).stop 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 shutdown_connections 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5f40060980 0x7f5f40062e30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6015cc20 0x7f5f6015f240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f600c6cd0 0x7f5f600af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 --2- 172.21.15.64:0/3448245202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f600c6900 0x7f5f60149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.198+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 >> 172.21.15.64:0/3448245202 conn(0x7f5f60067bb0 msgr2=0x7f5f600abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.202+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 shutdown_connections 2023-09-27T23:14:45.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.202+0000 7f5f6490e700 1 -- 172.21.15.64:0/3448245202 wait complete. 2023-09-27T23:14:45.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:14:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 Processor -- start 2023-09-27T23:14:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 -- start start 2023-09-27T23:14:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1440158e20 0x7f14400ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14400cb970 0x7f14400ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400cbdf0 con 0x7f14400ca2c0 2023-09-27T23:14:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400cc0d0 con 0x7f1440158e20 2023-09-27T23:14:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f14448af700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400cc3b0 con 0x7f14400cb970 2023-09-27T23:14:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f14400ca2c0 0x7f144006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51542/0 (socket says 172.21.15.64:51542) 2023-09-27T23:14:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1440158e20 0x7f14400ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143ed9d700 1 -- 172.21.15.64:0/981374832 learned_addr learned my addr 172.21.15.64:0/981374832 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f1437fff700 1 --2- 172.21.15.64:0/981374832 >> v2:172.21.15.123:3300/0 conn(0x7f14400cb970 0x7f14400ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143e59c700 1 -- 172.21.15.64:0/981374832 >> v2:172.21.15.123:3300/0 conn(0x7f14400cb970 msgr2=0x7f14400ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143e59c700 1 --2- 172.21.15.64:0/981374832 >> v2:172.21.15.123:3300/0 conn(0x7f14400cb970 0x7f14400ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143e59c700 1 -- 172.21.15.64:0/981374832 >> v1:172.21.15.64:6789/0 conn(0x7f14400ca2c0 legacy=0x7f144006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143e59c700 1 -- 172.21.15.64:0/981374832 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400cc690 con 0x7f1440158e20 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143dd9b700 1 -- 172.21.15.64:0/981374832 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448372321 0 0) 0x7f14400cbdf0 con 0x7f14400ca2c0 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.330+0000 7f143e59c700 1 --2- 172.21.15.64:0/981374832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1440158e20 0x7f14400ce5c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f14300098e0 tx=0x7f14300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eb6a17db595322 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f143dd9b700 1 -- 172.21.15.64:0/981374832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430011630 con 0x7f1440158e20 2023-09-27T23:14:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f143dd9b700 1 -- 172.21.15.64:0/981374832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430011790 con 0x7f1440158e20 2023-09-27T23:14:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f143dd9b700 1 -- 172.21.15.64:0/981374832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430011b30 con 0x7f1440158e20 2023-09-27T23:14:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 -- 172.21.15.64:0/981374832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1440158e20 msgr2=0x7f14400ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 --2- 172.21.15.64:0/981374832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1440158e20 0x7f14400ce5c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f14300098e0 tx=0x7f14300091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 -- 172.21.15.64:0/981374832 shutdown_connections 2023-09-27T23:14:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 --2- 172.21.15.64:0/981374832 >> v2:172.21.15.123:3300/0 conn(0x7f14400cb970 0x7f14400ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 --2- 172.21.15.64:0/981374832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1440158e20 0x7f14400ce5c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 -- 172.21.15.64:0/981374832 >> 172.21.15.64:0/981374832 conn(0x7f1440067bb0 msgr2=0x7f14400d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 -- 172.21.15.64:0/981374832 shutdown_connections 2023-09-27T23:14:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.334+0000 7f14448af700 1 -- 172.21.15.64:0/981374832 wait complete. 2023-09-27T23:14:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 Processor -- start 2023-09-27T23:14:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 -- start start 2023-09-27T23:14:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14400ca2c0 0x7f144014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1440158e20 0x7f14400c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 0x7f14400ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400ce810 con 0x7f14400cde50 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14401595c0 con 0x7f14400ca2c0 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400b3cf0 con 0x7f1440158e20 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 0x7f14400ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 0x7f14400ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57116/0 (socket says 172.21.15.64:57116) 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 -- 172.21.15.64:0/3422712430 learned_addr learned my addr 172.21.15.64:0/3422712430 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143e59c700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14400ca2c0 0x7f144014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f1437fff700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1440158e20 0x7f14400c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 -- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1440158e20 msgr2=0x7f14400c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1440158e20 0x7f14400c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 -- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14400ca2c0 msgr2=0x7f144014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14400ca2c0 0x7f144014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 -- 172.21.15.64:0/3422712430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1430009420 con 0x7f14400cde50 2023-09-27T23:14:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f143ed9d700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 0x7f14400ce2d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f143800cc90 tx=0x7f1438001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800a870 con 0x7f14400cde50 2023-09-27T23:14:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800a9d0 con 0x7f14400cde50 2023-09-27T23:14:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144014d190 con 0x7f14400cde50 2023-09-27T23:14:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800ad90 con 0x7f14400cde50 2023-09-27T23:14:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.338+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14400b33c0 con 0x7f14400cde50 2023-09-27T23:14:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.342+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400b3ea0 con 0x7f14400cde50 2023-09-27T23:14:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.342+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f1438007300 con 0x7f14400cde50 2023-09-27T23:14:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.342+0000 7f14367fc700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1420060980 0x7f1420062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.342+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f143807d280 con 0x7f14400cde50 2023-09-27T23:14:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.342+0000 7f143e59c700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1420060980 0x7f1420062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.342+0000 7f143e59c700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1420060980 0x7f1420062e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f14300091e0 tx=0x7f1430017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.350+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f14400b3ea0 con 0x7f14400cde50 2023-09-27T23:14:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.546+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f14400b3e60 con 0x7f14400cde50 2023-09-27T23:14:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.578+0000 7f14367fc700 1 -- 172.21.15.64:0/3422712430 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f143804d440 con 0x7f14400cde50 2023-09-27T23:14:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1420060980 msgr2=0x7f1420062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1420060980 0x7f1420062e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f14300091e0 tx=0x7f1430017000 comp rx=0 tx=0).stop 2023-09-27T23:14:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 msgr2=0x7f14400ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 0x7f14400ce2d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f143800cc90 tx=0x7f1438001a20 comp rx=0 tx=0).stop 2023-09-27T23:14:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 shutdown_connections 2023-09-27T23:14:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1420060980 0x7f1420062e30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14400cde50 0x7f14400ce2d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1440158e20 0x7f14400c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 --2- 172.21.15.64:0/3422712430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14400ca2c0 0x7f144014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 >> 172.21.15.64:0/3422712430 conn(0x7f1440067bb0 msgr2=0x7f144014ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 shutdown_connections 2023-09-27T23:14:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.582+0000 7f14448af700 1 -- 172.21.15.64:0/3422712430 wait complete. 2023-09-27T23:14:45.597 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:14:45.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:14:45.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 Processor -- start 2023-09-27T23:14:45.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 -- start start 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a280c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a280ace30 0x7f7a280aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a280af250 con 0x7f7a280ae770 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a280af530 con 0x7f7a280ace30 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.702+0000 7f7a2edd0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280ca8a0 con 0x7f7a28150200 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a280ace30 0x7f7a280aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2ddce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a280c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a280ace30 0x7f7a280aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57128/0 (socket says 172.21.15.64:57128) 2023-09-27T23:14:45.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 -- 172.21.15.64:0/781540261 learned_addr learned my addr 172.21.15.64:0/781540261 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 -- 172.21.15.64:0/781540261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 msgr2=0x7f7a280c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 --2- 172.21.15.64:0/781540261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a280c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 -- 172.21.15.64:0/781540261 >> v1:172.21.15.123:6789/0 conn(0x7f7a280ae770 legacy=0x7f7a280ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 -- 172.21.15.64:0/781540261 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a280cab80 con 0x7f7a280ace30 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2e5cf700 1 --2- 172.21.15.64:0/781540261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ace30 0x7f7a280aebe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009f20 tx=0x7f7a20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130dfdda2af24436 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2cdcc700 1 -- 172.21.15.64:0/781540261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000e070 con 0x7f7a280ace30 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2cdcc700 1 -- 172.21.15.64:0/781540261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2000b9d0 con 0x7f7a280ace30 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2cdcc700 1 -- 172.21.15.64:0/781540261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200092e0 con 0x7f7a280ace30 2023-09-27T23:14:45.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 -- 172.21.15.64:0/781540261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ace30 msgr2=0x7f7a280aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/781540261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ace30 0x7f7a280aebe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009f20 tx=0x7f7a20009840 comp rx=0 tx=0).stop 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 -- 172.21.15.64:0/781540261 shutdown_connections 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/781540261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ace30 0x7f7a280aebe0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/781540261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a280c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 -- 172.21.15.64:0/781540261 >> 172.21.15.64:0/781540261 conn(0x7f7a280ccb30 msgr2=0x7f7a280ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 -- 172.21.15.64:0/781540261 shutdown_connections 2023-09-27T23:14:45.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.706+0000 7f7a2edd0700 1 -- 172.21.15.64:0/781540261 wait complete. 2023-09-27T23:14:45.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 Processor -- start 2023-09-27T23:14:45.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 -- start start 2023-09-27T23:14:45.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a280ace30 0x7f7a280b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 0x7f7a28068130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a2814dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280672e0 con 0x7f7a280ae770 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a28068670 con 0x7f7a280ace30 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2edd0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2814e4e0 con 0x7f7a28150200 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 0x7f7a28068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 0x7f7a28068130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57134/0 (socket says 172.21.15.64:57134) 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 -- 172.21.15.64:0/354854793 learned_addr learned my addr 172.21.15.64:0/354854793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2e5cf700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a2814dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2ddce700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a280ace30 0x7f7a280b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 -- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 msgr2=0x7f7a2814dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a2814dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 -- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a280ace30 msgr2=0x7f7a280b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:45.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a280ace30 0x7f7a280b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:45.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 -- 172.21.15.64:0/354854793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a20009ae0 con 0x7f7a280ae770 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a2d5cd700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 0x7f7a28068130 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c590 tx=0x7f7a1800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.710+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011590 con 0x7f7a280ae770 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18011710 con 0x7f7a280ae770 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011ad0 con 0x7f7a280ae770 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a28168cb0 con 0x7f7a280ae770 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a28067930 con 0x7f7a280ae770 2023-09-27T23:14:45.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a280674a0 con 0x7f7a280ae770 2023-09-27T23:14:45.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f7a1801b3b0 con 0x7f7a280ae770 2023-09-27T23:14:45.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a1dffb700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a0c060950 0x7f7a0c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:45.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.714+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7a1807c460 con 0x7f7a280ae770 2023-09-27T23:14:45.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.718+0000 7f7a2ddce700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a0c060950 0x7f7a0c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:45.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.718+0000 7f7a2ddce700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a0c060950 0x7f7a0c062e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009d10 tx=0x7f7a240091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:45.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.722+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a1804c500 con 0x7f7a280ae770 2023-09-27T23:14:45.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:45.922+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a2804be40 con 0x7f7a280ae770 2023-09-27T23:14:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.774+0000 7f7a1dffb700 1 -- 172.21.15.64:0/354854793 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a180170c0 con 0x7f7a280ae770 2023-09-27T23:14:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a0c060950 msgr2=0x7f7a0c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a0c060950 0x7f7a0c062e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009d10 tx=0x7f7a240091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 msgr2=0x7f7a28068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 0x7f7a28068130 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c590 tx=0x7f7a1800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 shutdown_connections 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a0c060950 0x7f7a0c062e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a28150200 0x7f7a2814dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a280ae770 0x7f7a28068130 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 --2- 172.21.15.64:0/354854793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a280ace30 0x7f7a280b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 >> 172.21.15.64:0/354854793 conn(0x7f7a280ccb30 msgr2=0x7f7a2814f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 shutdown_connections 2023-09-27T23:14:46.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.778+0000 7f7a2edd0700 1 -- 172.21.15.64:0/354854793 wait complete. 2023-09-27T23:14:46.800 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:14:46.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e8856700 1 Processor -- start 2023-09-27T23:14:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e8856700 1 -- start start 2023-09-27T23:14:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e8856700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65e4155990 0x7f65e40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e8856700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e406cf00 con 0x7f65e40b1580 2023-09-27T23:14:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e8856700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40b01d0 con 0x7f65e40b0670 2023-09-27T23:14:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e8856700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40ae7e0 con 0x7f65e4155990 2023-09-27T23:14:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f65e40b1580 0x7f65e406d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51544/0 (socket says 172.21.15.64:51544) 2023-09-27T23:14:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e3fff700 1 -- 172.21.15.64:0/863790513 learned_addr learned my addr 172.21.15.64:0/863790513 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e37fe700 1 --2- 172.21.15.64:0/863790513 >> v2:172.21.15.123:3300/0 conn(0x7f65e4155990 0x7f65e40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e2ffd700 1 -- 172.21.15.64:0/863790513 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256091149 0 0) 0x7f65e406cf00 con 0x7f65e40b1580 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e2ffd700 1 -- 172.21.15.64:0/863790513 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d0003570 con 0x7f65e40b1580 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e37fe700 1 -- 172.21.15.64:0/863790513 >> v1:172.21.15.123:6789/0 conn(0x7f65e40b0670 legacy=0x7f65e40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e37fe700 1 -- 172.21.15.64:0/863790513 >> v1:172.21.15.64:6789/0 conn(0x7f65e40b1580 legacy=0x7f65e406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.914+0000 7f65e37fe700 1 -- 172.21.15.64:0/863790513 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e406c8f0 con 0x7f65e4155990 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e37fe700 1 --2- 172.21.15.64:0/863790513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e4155990 0x7f65e40c97d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098e0 tx=0x7f65d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4877afe4b19c167b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e2ffd700 1 -- 172.21.15.64:0/863790513 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400b640 con 0x7f65e4155990 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e2ffd700 1 -- 172.21.15.64:0/863790513 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400b7a0 con 0x7f65e4155990 2023-09-27T23:14:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e2ffd700 1 -- 172.21.15.64:0/863790513 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400bb40 con 0x7f65e4155990 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- 172.21.15.64:0/863790513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e4155990 msgr2=0x7f65e40c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 --2- 172.21.15.64:0/863790513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e4155990 0x7f65e40c97d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098e0 tx=0x7f65d40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- 172.21.15.64:0/863790513 shutdown_connections 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 --2- 172.21.15.64:0/863790513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e4155990 0x7f65e40c97d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- 172.21.15.64:0/863790513 >> 172.21.15.64:0/863790513 conn(0x7f65e4067bb0 msgr2=0x7f65e40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- 172.21.15.64:0/863790513 shutdown_connections 2023-09-27T23:14:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- 172.21.15.64:0/863790513 wait complete. 2023-09-27T23:14:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 Processor -- start 2023-09-27T23:14:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- start start 2023-09-27T23:14:46.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65e40b0670 0x7f65e40c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e40b1580 0x7f65e40c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 0x7f65e40c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40c1f70 con 0x7f65e4155990 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40c1c40 con 0x7f65e40b1580 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.918+0000 7f65e8856700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40c78d0 con 0x7f65e40b0670 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 0x7f65e40c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 0x7f65e40c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57138/0 (socket says 172.21.15.64:57138) 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 -- 172.21.15.64:0/3983816509 learned_addr learned my addr 172.21.15.64:0/3983816509 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e37fe700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65e40b0670 0x7f65e40c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65dbfff700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e40b1580 0x7f65e40c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 -- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65e40b0670 msgr2=0x7f65e40c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65e40b0670 0x7f65e40c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 -- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e40b1580 msgr2=0x7f65e40c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e40b1580 0x7f65e40c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 -- 172.21.15.64:0/3983816509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009420 con 0x7f65e4155990 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e3fff700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 0x7f65e40c3d10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cc40 tx=0x7f65dc001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00a8b0 con 0x7f65e4155990 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc00aa10 con 0x7f65e4155990 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40c7bb0 con 0x7f65e4155990 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00ad90 con 0x7f65e4155990 2023-09-27T23:14:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40c75b0 con 0x7f65e4155990 2023-09-27T23:14:46.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.922+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e404be40 con 0x7f65e4155990 2023-09-27T23:14:46.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.926+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f65dc007300 con 0x7f65e4155990 2023-09-27T23:14:46.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.926+0000 7f65db7fe700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65c4060950 0x7f65c4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:46.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.926+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f65dc07d330 con 0x7f65e4155990 2023-09-27T23:14:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.926+0000 7f65e37fe700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65c4060950 0x7f65c4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.926+0000 7f65e37fe700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65c4060950 0x7f65c4062e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65d40093f0 tx=0x7f65d4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:46.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:46.934+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f65dc04d460 con 0x7f65e4155990 2023-09-27T23:14:47.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.134+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e406df10 con 0x7f65e4155990 2023-09-27T23:14:47.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65db7fe700 1 -- 172.21.15.64:0/3983816509 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f65dc050a80 con 0x7f65e4155990 2023-09-27T23:14:47.141 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:47.141 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:14:44.562884+0000","modified":"2023-09-27T23:14:46.768950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/974610030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":974610030},{"type":"v1","addr":"172.21.15.64:6835","nonce":974610030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-09-27T23:14:47.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65c4060950 msgr2=0x7f65c4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65c4060950 0x7f65c4062e00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65d40093f0 tx=0x7f65d4017000 comp rx=0 tx=0).stop 2023-09-27T23:14:47.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 msgr2=0x7f65e40c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 0x7f65e40c3d10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cc40 tx=0x7f65dc001aa0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 shutdown_connections 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65c4060950 0x7f65c4062e00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65e4155990 0x7f65e40c3d10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65e40b1580 0x7f65e40c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 --2- 172.21.15.64:0/3983816509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65e40b0670 0x7f65e40c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 >> 172.21.15.64:0/3983816509 conn(0x7f65e4067bb0 msgr2=0x7f65e40ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 shutdown_connections 2023-09-27T23:14:47.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.138+0000 7f65e8856700 1 -- 172.21.15.64:0/3983816509 wait complete. 2023-09-27T23:14:47.145 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2023-09-27T23:14:47.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:14:47.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.254+0000 7ffa749e4700 1 Processor -- start 2023-09-27T23:14:47.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.254+0000 7ffa749e4700 1 -- start start 2023-09-27T23:14:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa749e4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffa7006d280 0x7ffa7006d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa749e4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700ad070 con 0x7ffa70068160 2023-09-27T23:14:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa749e4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700ac780 con 0x7ffa7015aa10 2023-09-27T23:14:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa749e4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa70068e90 con 0x7ffa7006d280 2023-09-27T23:14:47.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa70068160 0x7ffa700685c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51546/0 (socket says 172.21.15.64:51546) 2023-09-27T23:14:47.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6effd700 1 -- 172.21.15.64:0/735837878 learned_addr learned my addr 172.21.15.64:0/735837878 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:47.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6f7fe700 1 --2- 172.21.15.64:0/735837878 >> v2:172.21.15.123:3300/0 conn(0x7ffa7006d280 0x7ffa7006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6e7fc700 1 -- 172.21.15.64:0/735837878 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349563532 0 0) 0x7ffa700ad070 con 0x7ffa70068160 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6e7fc700 1 -- 172.21.15.64:0/735837878 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58003570 con 0x7ffa70068160 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6f7fe700 1 -- 172.21.15.64:0/735837878 >> v1:172.21.15.123:6789/0 conn(0x7ffa7015aa10 legacy=0x7ffa7015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6f7fe700 1 -- 172.21.15.64:0/735837878 >> v1:172.21.15.64:6789/0 conn(0x7ffa70068160 legacy=0x7ffa700685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6f7fe700 1 -- 172.21.15.64:0/735837878 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700aca20 con 0x7ffa7006d280 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6f7fe700 1 --2- 172.21.15.64:0/735837878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7006d280 0x7ffa7006d650 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa600098e0 tx=0x7ffa600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d251bb829ea2eda server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6e7fc700 1 -- 172.21.15.64:0/735837878 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000b6d0 con 0x7ffa7006d280 2023-09-27T23:14:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6e7fc700 1 -- 172.21.15.64:0/735837878 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000b830 con 0x7ffa7006d280 2023-09-27T23:14:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa6e7fc700 1 -- 172.21.15.64:0/735837878 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000bbb0 con 0x7ffa7006d280 2023-09-27T23:14:47.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa749e4700 1 -- 172.21.15.64:0/735837878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7006d280 msgr2=0x7ffa7006d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.258+0000 7ffa749e4700 1 --2- 172.21.15.64:0/735837878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7006d280 0x7ffa7006d650 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa600098e0 tx=0x7ffa600091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- 172.21.15.64:0/735837878 shutdown_connections 2023-09-27T23:14:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 --2- 172.21.15.64:0/735837878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7006d280 0x7ffa7006d650 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- 172.21.15.64:0/735837878 >> 172.21.15.64:0/735837878 conn(0x7ffa7006c080 msgr2=0x7ffa700ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- 172.21.15.64:0/735837878 shutdown_connections 2023-09-27T23:14:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- 172.21.15.64:0/735837878 wait complete. 2023-09-27T23:14:47.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 Processor -- start 2023-09-27T23:14:47.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- start start 2023-09-27T23:14:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa70068160 0x7ffa70167480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 0x7ffa701567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7015aa10 0x7ffa70156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70166950 con 0x7ffa7006d280 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa701597c0 con 0x7ffa7015aa10 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa749e4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa701679c0 con 0x7ffa70068160 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 0x7ffa701567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 0x7ffa701567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57140/0 (socket says 172.21.15.64:57140) 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 -- 172.21.15.64:0/143087625 learned_addr learned my addr 172.21.15.64:0/143087625 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6ffff700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7015aa10 0x7ffa70156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6f7fe700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa70068160 0x7ffa70167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 -- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa70068160 msgr2=0x7ffa70167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa70068160 0x7ffa70167480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 -- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7015aa10 msgr2=0x7ffa70156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7015aa10 0x7ffa70156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 -- 172.21.15.64:0/143087625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009420 con 0x7ffa7006d280 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6ffff700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7015aa10 0x7ffa70156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.262+0000 7ffa6effd700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 0x7ffa701567e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c7f0 tx=0x7ffa64007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64005b30 con 0x7ffa7006d280 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64005c90 con 0x7ffa7006d280 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64007340 con 0x7ffa7006d280 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa70154130 con 0x7ffa7006d280 2023-09-27T23:14:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa70158e90 con 0x7ffa7006d280 2023-09-27T23:14:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ffa6401b400 con 0x7ffa7006d280 2023-09-27T23:14:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70159950 con 0x7ffa7006d280 2023-09-27T23:14:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa577fe700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa5c060950 0x7ffa5c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ffa6407cac0 con 0x7ffa7006d280 2023-09-27T23:14:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa6f7fe700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa5c060950 0x7ffa5c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.266+0000 7ffa6f7fe700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa5c060950 0x7ffa5c062e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa600093f0 tx=0x7ffa60017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.274+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ffa6404cc80 con 0x7ffa7006d280 2023-09-27T23:14:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.434+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa7004be40 con 0x7ffa7006d280 2023-09-27T23:14:47.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.434+0000 7ffa577fe700 1 -- 172.21.15.64:0/143087625 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17071 (secure 0 0 0) 0x7ffa6407ce60 con 0x7ffa7006d280 2023-09-27T23:14:47.438 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:47.438 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:14:45.566000+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:14:42.525604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-09-27T23:14:43.561136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa5c060950 msgr2=0x7ffa5c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa5c060950 0x7ffa5c062e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa600093f0 tx=0x7ffa60017000 comp rx=0 tx=0).stop 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 msgr2=0x7ffa701567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 0x7ffa701567e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c7f0 tx=0x7ffa64007e00 comp rx=0 tx=0).stop 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 shutdown_connections 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffa5c060950 0x7ffa5c062e00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa7015aa10 0x7ffa70156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa7006d280 0x7ffa701567e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 --2- 172.21.15.64:0/143087625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa70068160 0x7ffa70167480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 >> 172.21.15.64:0/143087625 conn(0x7ffa7006c080 msgr2=0x7ffa700cb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 shutdown_connections 2023-09-27T23:14:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.438+0000 7ffa749e4700 1 -- 172.21.15.64:0/143087625 wait complete. 2023-09-27T23:14:47.459 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:14:47.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:47.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:14:47.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:14:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.566+0000 7f8f95680700 1 Processor -- start 2023-09-27T23:14:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.566+0000 7f8f95680700 1 -- start start 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f9006a540 0x7f8f9006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f900cc470 0x7f8f900cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900ca5a0 con 0x7f8f90158e70 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9006dac0 con 0x7f8f9006a540 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9006d220 con 0x7f8f900cc470 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f9006a540 0x7f8f9006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f9006a540 0x7f8f9006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57152/0 (socket says 172.21.15.64:57152) 2023-09-27T23:14:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/3613261777 learned_addr learned my addr 172.21.15.64:0/3613261777 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8ffff700 1 --1- 172.21.15.64:0/3613261777 >> v1:172.21.15.64:6789/0 conn(0x7f8f90158e70 0x7f8f900afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51552/0 (socket says 172.21.15.64:51552) 2023-09-27T23:14:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f94e7f700 1 --2- 172.21.15.64:0/3613261777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f900cc470 0x7f8f900cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/3613261777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f900cc470 msgr2=0x7f8f900cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 --2- 172.21.15.64:0/3613261777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f900cc470 0x7f8f900cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/3613261777 >> v1:172.21.15.64:6789/0 conn(0x7f8f90158e70 legacy=0x7f8f900afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/3613261777 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9006d500 con 0x7f8f9006a540 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8f7fe700 1 --2- 172.21.15.64:0/3613261777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f9006a540 0x7f8f9006c9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8f800099b0 tx=0x7f8f80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16779dd63458bc34 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8effd700 1 -- 172.21.15.64:0/3613261777 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80009ea0 con 0x7f8f9006a540 2023-09-27T23:14:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8effd700 1 -- 172.21.15.64:0/3613261777 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8000b860 con 0x7f8f9006a540 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f8effd700 1 -- 172.21.15.64:0/3613261777 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000bc20 con 0x7f8f9006a540 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- 172.21.15.64:0/3613261777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f9006a540 msgr2=0x7f8f9006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 --2- 172.21.15.64:0/3613261777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f9006a540 0x7f8f9006c9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8f800099b0 tx=0x7f8f80009250 comp rx=0 tx=0).stop 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- 172.21.15.64:0/3613261777 shutdown_connections 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 --2- 172.21.15.64:0/3613261777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f900cc470 0x7f8f900cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 --2- 172.21.15.64:0/3613261777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f9006a540 0x7f8f9006c9d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- 172.21.15.64:0/3613261777 >> 172.21.15.64:0/3613261777 conn(0x7f8f900670e0 msgr2=0x7f8f900674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- 172.21.15.64:0/3613261777 shutdown_connections 2023-09-27T23:14:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.570+0000 7f8f95680700 1 -- 172.21.15.64:0/3613261777 wait complete. 2023-09-27T23:14:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 Processor -- start 2023-09-27T23:14:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 -- start start 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f9006a540 0x7f8f901690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 0x7f8f900b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f90158e70 0x7f8f90156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900b58b0 con 0x7f8f900cc470 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900b4ca0 con 0x7f8f90158e70 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900b4f80 con 0x7f8f9006a540 2023-09-27T23:14:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 0x7f8f900b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 0x7f8f900b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57168/0 (socket says 172.21.15.64:57168) 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/1540041154 learned_addr learned my addr 172.21.15.64:0/1540041154 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8ffff700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f9006a540 0x7f8f901690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f94e7f700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f90158e70 0x7f8f90156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f9006a540 msgr2=0x7f8f901690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f9006a540 0x7f8f901690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f90158e70 msgr2=0x7f8f90156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f90158e70 0x7f8f90156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 -- 172.21.15.64:0/1540041154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800094f0 con 0x7f8f900cc470 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8ffff700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f9006a540 0x7f8f901690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f94e7f700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f90158e70 0x7f8f90156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f8f7fe700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 0x7f8f900b43a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8f80006fd0 tx=0x7f8f80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8001f070 con 0x7f8f900cc470 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80000bc0 con 0x7f8f900cc470 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f900b5260 con 0x7f8f900cc470 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80025420 con 0x7f8f900cc470 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.574+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f90169600 con 0x7f8f900cc470 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.578+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9004be40 con 0x7f8f900cc470 2023-09-27T23:14:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.578+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8f80023070 con 0x7f8f900cc470 2023-09-27T23:14:47.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.578+0000 7f8f77fff700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f7c060980 0x7f8f7c062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.578+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8f8007e220 con 0x7f8f900cc470 2023-09-27T23:14:47.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.578+0000 7f8f8ffff700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f7c060980 0x7f8f7c062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.578+0000 7f8f8ffff700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f7c060980 0x7f8f7c062e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f84007fd0 tx=0x7f8f84007e10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.586+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8f8004e360 con 0x7f8f900cc470 2023-09-27T23:14:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.742+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f9006bb10 con 0x7f8f7c060980 2023-09-27T23:14:47.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.758+0000 7f8f77fff700 1 -- 172.21.15.64:0/1540041154 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296003 (secure 0 0 0) 0x7f8f9006bb10 con 0x7f8f7c060980 2023-09-27T23:14:47.766 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:47.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f7c060980 msgr2=0x7f8f7c062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f7c060980 0x7f8f7c062e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f84007fd0 tx=0x7f8f84007e10 comp rx=0 tx=0).stop 2023-09-27T23:14:47.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 msgr2=0x7f8f900b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 0x7f8f900b43a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8f80006fd0 tx=0x7f8f80017000 comp rx=0 tx=0).stop 2023-09-27T23:14:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 shutdown_connections 2023-09-27T23:14:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f7c060980 0x7f8f7c062e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f90158e70 0x7f8f90156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f900cc470 0x7f8f900b43a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.762+0000 7f8f95680700 1 --2- 172.21.15.64:0/1540041154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f9006a540 0x7f8f901690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.766+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 >> 172.21.15.64:0/1540041154 conn(0x7f8f900670e0 msgr2=0x7f8f900b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.766+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 shutdown_connections 2023-09-27T23:14:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.766+0000 7f8f95680700 1 -- 172.21.15.64:0/1540041154 wait complete. 2023-09-27T23:14:47.770 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:47.786 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-09-27T23:14:47.167854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591926,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":169,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":70596,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":65728,"kb_avail":754904124,"statfs":{"total":773094113280,"available":773021822976,"internally_reserved":0,"allocated":4919296,"data_stored":2788920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67305472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001295"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604058+0000","last_change":"2023-09-27T23:14:44.585114+0000","last_active":"2023-09-27T23:14:44.604058+0000","last_peered":"2023-09-27T23:14:44.604058+0000","last_clean":"2023-09-27T23:14:44.604058+0000","last_became_active":"2023-09-27T23:14:44.584800+0000","last_became_peered":"2023-09-27T23:14:44.584800+0000","last_unstale":"2023-09-27T23:14:44.604058+0000","last_undegraded":"2023-09-27T23:14:44.604058+0000","last_fullsized":"2023-09-27T23:14:44.604058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:32.813292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605484+0000","last_change":"2023-09-27T23:14:44.591703+0000","last_active":"2023-09-27T23:14:44.605484+0000","last_peered":"2023-09-27T23:14:44.605484+0000","last_clean":"2023-09-27T23:14:44.605484+0000","last_became_active":"2023-09-27T23:14:44.591432+0000","last_became_peered":"2023-09-27T23:14:44.591432+0000","last_unstale":"2023-09-27T23:14:44.605484+0000","last_undegraded":"2023-09-27T23:14:44.605484+0000","last_fullsized":"2023-09-27T23:14:44.605484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:05:55.604974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604052+0000","last_change":"2023-09-27T23:14:44.585552+0000","last_active":"2023-09-27T23:14:44.604052+0000","last_peered":"2023-09-27T23:14:44.604052+0000","last_clean":"2023-09-27T23:14:44.604052+0000","last_became_active":"2023-09-27T23:14:44.585234+0000","last_became_peered":"2023-09-27T23:14:44.585234+0000","last_unstale":"2023-09-27T23:14:44.604052+0000","last_undegraded":"2023-09-27T23:14:44.604052+0000","last_fullsized":"2023-09-27T23:14:44.604052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:21:55.504453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604266+0000","last_change":"2023-09-27T23:14:44.592678+0000","last_active":"2023-09-27T23:14:44.604266+0000","last_peered":"2023-09-27T23:14:44.604266+0000","last_clean":"2023-09-27T23:14:44.604266+0000","last_became_active":"2023-09-27T23:14:44.592380+0000","last_became_peered":"2023-09-27T23:14:44.592380+0000","last_unstale":"2023-09-27T23:14:44.604266+0000","last_undegraded":"2023-09-27T23:14:44.604266+0000","last_fullsized":"2023-09-27T23:14:44.604266+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:25.182304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601442+0000","last_change":"2023-09-27T23:14:44.596994+0000","last_active":"2023-09-27T23:14:44.601442+0000","last_peered":"2023-09-27T23:14:44.601442+0000","last_clean":"2023-09-27T23:14:44.601442+0000","last_became_active":"2023-09-27T23:14:44.596700+0000","last_became_peered":"2023-09-27T23:14:44.596700+0000","last_unstale":"2023-09-27T23:14:44.601442+0000","last_undegraded":"2023-09-27T23:14:44.601442+0000","last_fullsized":"2023-09-27T23:14:44.601442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:23.475703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609839+0000","last_change":"2023-09-27T23:14:44.217297+0000","last_active":"2023-09-27T23:14:44.609839+0000","last_peered":"2023-09-27T23:14:44.609839+0000","last_clean":"2023-09-27T23:14:44.609839+0000","last_became_active":"2023-09-27T23:14:44.216935+0000","last_became_peered":"2023-09-27T23:14:44.216935+0000","last_unstale":"2023-09-27T23:14:44.609839+0000","last_undegraded":"2023-09-27T23:14:44.609839+0000","last_fullsized":"2023-09-27T23:14:44.609839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:34.357991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600805+0000","last_change":"2023-09-27T23:14:44.594912+0000","last_active":"2023-09-27T23:14:44.600805+0000","last_peered":"2023-09-27T23:14:44.600805+0000","last_clean":"2023-09-27T23:14:44.600805+0000","last_became_active":"2023-09-27T23:14:44.594632+0000","last_became_peered":"2023-09-27T23:14:44.594632+0000","last_unstale":"2023-09-27T23:14:44.600805+0000","last_undegraded":"2023-09-27T23:14:44.600805+0000","last_fullsized":"2023-09-27T23:14:44.600805+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:00.409802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.612265+0000","last_change":"2023-09-27T23:14:44.213685+0000","last_active":"2023-09-27T23:14:44.612265+0000","last_peered":"2023-09-27T23:14:44.612265+0000","last_clean":"2023-09-27T23:14:44.612265+0000","last_became_active":"2023-09-27T23:14:44.213204+0000","last_became_peered":"2023-09-27T23:14:44.213204+0000","last_unstale":"2023-09-27T23:14:44.612265+0000","last_undegraded":"2023-09-27T23:14:44.612265+0000","last_fullsized":"2023-09-27T23:14:44.612265+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:51:19.929143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603875+0000","last_change":"2023-09-27T23:14:44.579511+0000","last_active":"2023-09-27T23:14:44.603875+0000","last_peered":"2023-09-27T23:14:44.603875+0000","last_clean":"2023-09-27T23:14:44.603875+0000","last_became_active":"2023-09-27T23:14:44.579172+0000","last_became_peered":"2023-09-27T23:14:44.579172+0000","last_unstale":"2023-09-27T23:14:44.603875+0000","last_undegraded":"2023-09-27T23:14:44.603875+0000","last_fullsized":"2023-09-27T23:14:44.603875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:48.765761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608309+0000","last_change":"2023-09-27T23:14:44.222653+0000","last_active":"2023-09-27T23:14:44.608309+0000","last_peered":"2023-09-27T23:14:44.608309+0000","last_clean":"2023-09-27T23:14:44.608309+0000","last_became_active":"2023-09-27T23:14:44.222336+0000","last_became_peered":"2023-09-27T23:14:44.222336+0000","last_unstale":"2023-09-27T23:14:44.608309+0000","last_undegraded":"2023-09-27T23:14:44.608309+0000","last_fullsized":"2023-09-27T23:14:44.608309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:45:57.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.602876+0000","last_change":"2023-09-27T23:14:44.600767+0000","last_active":"2023-09-27T23:14:44.602876+0000","last_peered":"2023-09-27T23:14:44.602876+0000","last_clean":"2023-09-27T23:14:44.602876+0000","last_became_active":"2023-09-27T23:14:44.600469+0000","last_became_peered":"2023-09-27T23:14:44.600469+0000","last_unstale":"2023-09-27T23:14:44.602876+0000","last_undegraded":"2023-09-27T23:14:44.602876+0000","last_fullsized":"2023-09-27T23:14:44.602876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:14.250239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604205+0000","last_change":"2023-09-27T23:14:44.590185+0000","last_active":"2023-09-27T23:14:44.604205+0000","last_peered":"2023-09-27T23:14:44.604205+0000","last_clean":"2023-09-27T23:14:44.604205+0000","last_became_active":"2023-09-27T23:14:44.589892+0000","last_became_peered":"2023-09-27T23:14:44.589892+0000","last_unstale":"2023-09-27T23:14:44.604205+0000","last_undegraded":"2023-09-27T23:14:44.604205+0000","last_fullsized":"2023-09-27T23:14:44.604205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:27.397171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.414355+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:14:37.414355+0000","last_peered":"2023-09-27T23:14:37.414355+0000","last_clean":"2023-09-27T23:14:37.414355+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:14:37.414355+0000","last_undegraded":"2023-09-27T23:14:37.414355+0000","last_fullsized":"2023-09-27T23:14:37.414355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589429+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:14:44.589429+0000","last_peered":"2023-09-27T23:14:44.589429+0000","last_clean":"2023-09-27T23:14:44.589429+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:14:44.589429+0000","last_undegraded":"2023-09-27T23:14:44.589429+0000","last_fullsized":"2023-09-27T23:14:44.589429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589480+0000","last_change":"2023-09-27T23:14:44.221449+0000","last_active":"2023-09-27T23:14:44.589480+0000","last_peered":"2023-09-27T23:14:44.589480+0000","last_clean":"2023-09-27T23:14:44.589480+0000","last_became_active":"2023-09-27T23:14:44.221090+0000","last_became_peered":"2023-09-27T23:14:44.221090+0000","last_unstale":"2023-09-27T23:14:44.589480+0000","last_undegraded":"2023-09-27T23:14:44.589480+0000","last_fullsized":"2023-09-27T23:14:44.589480+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:17:40.184736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296871+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:14:37.296871+0000","last_peered":"2023-09-27T23:14:37.296871+0000","last_clean":"2023-09-27T23:14:37.296871+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:14:37.296871+0000","last_undegraded":"2023-09-27T23:14:37.296871+0000","last_fullsized":"2023-09-27T23:14:37.296871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589240+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:14:44.589240+0000","last_peered":"2023-09-27T23:14:44.589240+0000","last_clean":"2023-09-27T23:14:44.589240+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:14:44.589240+0000","last_undegraded":"2023-09-27T23:14:44.589240+0000","last_fullsized":"2023-09-27T23:14:44.589240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.570569+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:14:44.570569+0000","last_peered":"2023-09-27T23:14:44.570569+0000","last_clean":"2023-09-27T23:14:44.570569+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:14:44.570569+0000","last_undegraded":"2023-09-27T23:14:44.570569+0000","last_fullsized":"2023-09-27T23:14:44.570569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589024+0000","last_change":"2023-09-27T23:14:44.212342+0000","last_active":"2023-09-27T23:14:44.589024+0000","last_peered":"2023-09-27T23:14:44.589024+0000","last_clean":"2023-09-27T23:14:44.589024+0000","last_became_active":"2023-09-27T23:14:44.211923+0000","last_became_peered":"2023-09-27T23:14:44.211923+0000","last_unstale":"2023-09-27T23:14:44.589024+0000","last_undegraded":"2023-09-27T23:14:44.589024+0000","last_fullsized":"2023-09-27T23:14:44.589024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:54:46.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296909+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:14:37.296909+0000","last_peered":"2023-09-27T23:14:37.296909+0000","last_clean":"2023-09-27T23:14:37.296909+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:14:37.296909+0000","last_undegraded":"2023-09-27T23:14:37.296909+0000","last_fullsized":"2023-09-27T23:14:37.296909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296920+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:14:37.296920+0000","last_peered":"2023-09-27T23:14:37.296920+0000","last_clean":"2023-09-27T23:14:37.296920+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:14:37.296920+0000","last_undegraded":"2023-09-27T23:14:37.296920+0000","last_fullsized":"2023-09-27T23:14:37.296920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":10,"acting":10,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.97799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.079}]},{"osd":2,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.016}]},{"osd":3,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.94799999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.878},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.1399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.042}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8452,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363388,"statfs":{"total":96636764160,"available":96628109312,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.90700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0409999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.165}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.1000000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.29}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.141}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.034},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.085}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1679999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.044},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.329}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.0069999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.97899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.1819999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.006}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.129}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.048}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.341},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.2370000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":507904,"data_stored":218009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93700000000000006}]},{"osd":1,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.052}]},{"osd":3,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.1040000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.23},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.091}]},{"osd":6,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.294},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.4790000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.998},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3120000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.157}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4259999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.157,"5min":6.157,"15min":6.157},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":5.649,"5min":5.649,"15min":5.649},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"last":0.503},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.282},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.014}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:47.793 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.934+0000 7fb0a6fa3700 1 Processor -- start 2023-09-27T23:14:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.934+0000 7fb0a6fa3700 1 -- start start 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb0a015d0b0 0x7fb0a0163d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a0068dc0 con 0x7fb0a0158d40 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a0068490 con 0x7fb0a00cb920 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00cc910 con 0x7fb0a015d0b0 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a67a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb0a015d0b0 0x7fb0a0163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a5fa1700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb0a0158d40 0x7fb0a00cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51566/0 (socket says 172.21.15.64:51566) 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a67a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb0a015d0b0 0x7fb0a0163d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57174/0 (socket says 172.21.15.64:57174) 2023-09-27T23:14:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a5fa1700 1 -- 172.21.15.64:0/505737595 learned_addr learned my addr 172.21.15.64:0/505737595 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a67a2700 1 -- 172.21.15.64:0/505737595 >> v1:172.21.15.123:6789/0 conn(0x7fb0a00cb920 legacy=0x7fb0a015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a67a2700 1 -- 172.21.15.64:0/505737595 >> v1:172.21.15.64:6789/0 conn(0x7fb0a0158d40 legacy=0x7fb0a00cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a67a2700 1 -- 172.21.15.64:0/505737595 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0068770 con 0x7fb0a015d0b0 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a67a2700 1 --2- 172.21.15.64:0/505737595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a0163d40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb0900098e0 tx=0x7fb0900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371ce93a39b5163e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a4f9f700 1 -- 172.21.15.64:0/505737595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090009e80 con 0x7fb0a015d0b0 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a4f9f700 1 -- 172.21.15.64:0/505737595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09000b860 con 0x7fb0a015d0b0 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a4f9f700 1 -- 172.21.15.64:0/505737595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000bbe0 con 0x7fb0a015d0b0 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/505737595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 msgr2=0x7fb0a0163d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/505737595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a0163d40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb0900098e0 tx=0x7fb0900091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/505737595 shutdown_connections 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/505737595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a0163d40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/505737595 >> 172.21.15.64:0/505737595 conn(0x7fb0a00cdbc0 msgr2=0x7fb0a00cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/505737595 shutdown_connections 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.938+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/505737595 wait complete. 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 Processor -- start 2023-09-27T23:14:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 -- start start 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a00cb920 0x7fb0a00c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0a0158d40 0x7fb0a0159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a00c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0159a80 con 0x7fb0a015d0b0 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0159d60 con 0x7fb0a00cb920 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015a040 con 0x7fb0a0158d40 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a00c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a5fa1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a00cb920 0x7fb0a00c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a00c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57184/0 (socket says 172.21.15.64:57184) 2023-09-27T23:14:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a57a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0a0158d40 0x7fb0a0159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 -- 172.21.15.64:0/4090753600 learned_addr learned my addr 172.21.15.64:0/4090753600 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 -- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0a0158d40 msgr2=0x7fb0a0159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0a0158d40 0x7fb0a0159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 -- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a00cb920 msgr2=0x7fb0a00c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a00cb920 0x7fb0a00c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 -- 172.21.15.64:0/4090753600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009420 con 0x7fb0a015d0b0 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a67a2700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a00c9aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb090013000 tx=0x7fb090017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000bba0 con 0x7fb0a015d0b0 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090025420 con 0x7fb0a015d0b0 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900257e0 con 0x7fb0a015d0b0 2023-09-27T23:14:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a015ad00 con 0x7fb0a015d0b0 2023-09-27T23:14:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.942+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a00c51b0 con 0x7fb0a015d0b0 2023-09-27T23:14:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.946+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a015b180 con 0x7fb0a015d0b0 2023-09-27T23:14:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.946+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb090023070 con 0x7fb0a015d0b0 2023-09-27T23:14:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.946+0000 7fb095ffb700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060950 0x7fb08c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.946+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb09007e7e0 con 0x7fb0a015d0b0 2023-09-27T23:14:47.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.946+0000 7fb0a5fa1700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060950 0x7fb08c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:47.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.950+0000 7fb0a5fa1700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060950 0x7fb08c062e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0980020c0 tx=0x7fb098001a10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:47.954+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb09004e880 con 0x7fb0a015d0b0 2023-09-27T23:14:48.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.110+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0a004be40 con 0x7fb08c060950 2023-09-27T23:14:48.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.130+0000 7fb095ffb700 1 -- 172.21.15.64:0/4090753600 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296003 (secure 0 0 0) 0x7fb0a004be40 con 0x7fb08c060950 2023-09-27T23:14:48.138 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060950 msgr2=0x7fb08c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060950 0x7fb08c062e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0980020c0 tx=0x7fb098001a10 comp rx=0 tx=0).stop 2023-09-27T23:14:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 msgr2=0x7fb0a00c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a00c9aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb090013000 tx=0x7fb090017000 comp rx=0 tx=0).stop 2023-09-27T23:14:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 shutdown_connections 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060950 0x7fb08c062e00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0a015d0b0 0x7fb0a00c9aa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0a0158d40 0x7fb0a0159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 --2- 172.21.15.64:0/4090753600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a00cb920 0x7fb0a00c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 >> 172.21.15.64:0/4090753600 conn(0x7fb0a00cdbc0 msgr2=0x7fb0a0152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 shutdown_connections 2023-09-27T23:14:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.138+0000 7fb0a6fa3700 1 -- 172.21.15.64:0/4090753600 wait complete. 2023-09-27T23:14:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:48.158 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-09-27T23:14:47.167854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591926,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":169,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":70596,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":65728,"kb_avail":754904124,"statfs":{"total":773094113280,"available":773021822976,"internally_reserved":0,"allocated":4919296,"data_stored":2788920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67305472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001295"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604058+0000","last_change":"2023-09-27T23:14:44.585114+0000","last_active":"2023-09-27T23:14:44.604058+0000","last_peered":"2023-09-27T23:14:44.604058+0000","last_clean":"2023-09-27T23:14:44.604058+0000","last_became_active":"2023-09-27T23:14:44.584800+0000","last_became_peered":"2023-09-27T23:14:44.584800+0000","last_unstale":"2023-09-27T23:14:44.604058+0000","last_undegraded":"2023-09-27T23:14:44.604058+0000","last_fullsized":"2023-09-27T23:14:44.604058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:32.813292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605484+0000","last_change":"2023-09-27T23:14:44.591703+0000","last_active":"2023-09-27T23:14:44.605484+0000","last_peered":"2023-09-27T23:14:44.605484+0000","last_clean":"2023-09-27T23:14:44.605484+0000","last_became_active":"2023-09-27T23:14:44.591432+0000","last_became_peered":"2023-09-27T23:14:44.591432+0000","last_unstale":"2023-09-27T23:14:44.605484+0000","last_undegraded":"2023-09-27T23:14:44.605484+0000","last_fullsized":"2023-09-27T23:14:44.605484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:05:55.604974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604052+0000","last_change":"2023-09-27T23:14:44.585552+0000","last_active":"2023-09-27T23:14:44.604052+0000","last_peered":"2023-09-27T23:14:44.604052+0000","last_clean":"2023-09-27T23:14:44.604052+0000","last_became_active":"2023-09-27T23:14:44.585234+0000","last_became_peered":"2023-09-27T23:14:44.585234+0000","last_unstale":"2023-09-27T23:14:44.604052+0000","last_undegraded":"2023-09-27T23:14:44.604052+0000","last_fullsized":"2023-09-27T23:14:44.604052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:21:55.504453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604266+0000","last_change":"2023-09-27T23:14:44.592678+0000","last_active":"2023-09-27T23:14:44.604266+0000","last_peered":"2023-09-27T23:14:44.604266+0000","last_clean":"2023-09-27T23:14:44.604266+0000","last_became_active":"2023-09-27T23:14:44.592380+0000","last_became_peered":"2023-09-27T23:14:44.592380+0000","last_unstale":"2023-09-27T23:14:44.604266+0000","last_undegraded":"2023-09-27T23:14:44.604266+0000","last_fullsized":"2023-09-27T23:14:44.604266+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:25.182304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601442+0000","last_change":"2023-09-27T23:14:44.596994+0000","last_active":"2023-09-27T23:14:44.601442+0000","last_peered":"2023-09-27T23:14:44.601442+0000","last_clean":"2023-09-27T23:14:44.601442+0000","last_became_active":"2023-09-27T23:14:44.596700+0000","last_became_peered":"2023-09-27T23:14:44.596700+0000","last_unstale":"2023-09-27T23:14:44.601442+0000","last_undegraded":"2023-09-27T23:14:44.601442+0000","last_fullsized":"2023-09-27T23:14:44.601442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:23.475703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609839+0000","last_change":"2023-09-27T23:14:44.217297+0000","last_active":"2023-09-27T23:14:44.609839+0000","last_peered":"2023-09-27T23:14:44.609839+0000","last_clean":"2023-09-27T23:14:44.609839+0000","last_became_active":"2023-09-27T23:14:44.216935+0000","last_became_peered":"2023-09-27T23:14:44.216935+0000","last_unstale":"2023-09-27T23:14:44.609839+0000","last_undegraded":"2023-09-27T23:14:44.609839+0000","last_fullsized":"2023-09-27T23:14:44.609839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:34.357991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600805+0000","last_change":"2023-09-27T23:14:44.594912+0000","last_active":"2023-09-27T23:14:44.600805+0000","last_peered":"2023-09-27T23:14:44.600805+0000","last_clean":"2023-09-27T23:14:44.600805+0000","last_became_active":"2023-09-27T23:14:44.594632+0000","last_became_peered":"2023-09-27T23:14:44.594632+0000","last_unstale":"2023-09-27T23:14:44.600805+0000","last_undegraded":"2023-09-27T23:14:44.600805+0000","last_fullsized":"2023-09-27T23:14:44.600805+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:00.409802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.612265+0000","last_change":"2023-09-27T23:14:44.213685+0000","last_active":"2023-09-27T23:14:44.612265+0000","last_peered":"2023-09-27T23:14:44.612265+0000","last_clean":"2023-09-27T23:14:44.612265+0000","last_became_active":"2023-09-27T23:14:44.213204+0000","last_became_peered":"2023-09-27T23:14:44.213204+0000","last_unstale":"2023-09-27T23:14:44.612265+0000","last_undegraded":"2023-09-27T23:14:44.612265+0000","last_fullsized":"2023-09-27T23:14:44.612265+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:51:19.929143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603875+0000","last_change":"2023-09-27T23:14:44.579511+0000","last_active":"2023-09-27T23:14:44.603875+0000","last_peered":"2023-09-27T23:14:44.603875+0000","last_clean":"2023-09-27T23:14:44.603875+0000","last_became_active":"2023-09-27T23:14:44.579172+0000","last_became_peered":"2023-09-27T23:14:44.579172+0000","last_unstale":"2023-09-27T23:14:44.603875+0000","last_undegraded":"2023-09-27T23:14:44.603875+0000","last_fullsized":"2023-09-27T23:14:44.603875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:48.765761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608309+0000","last_change":"2023-09-27T23:14:44.222653+0000","last_active":"2023-09-27T23:14:44.608309+0000","last_peered":"2023-09-27T23:14:44.608309+0000","last_clean":"2023-09-27T23:14:44.608309+0000","last_became_active":"2023-09-27T23:14:44.222336+0000","last_became_peered":"2023-09-27T23:14:44.222336+0000","last_unstale":"2023-09-27T23:14:44.608309+0000","last_undegraded":"2023-09-27T23:14:44.608309+0000","last_fullsized":"2023-09-27T23:14:44.608309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:45:57.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.602876+0000","last_change":"2023-09-27T23:14:44.600767+0000","last_active":"2023-09-27T23:14:44.602876+0000","last_peered":"2023-09-27T23:14:44.602876+0000","last_clean":"2023-09-27T23:14:44.602876+0000","last_became_active":"2023-09-27T23:14:44.600469+0000","last_became_peered":"2023-09-27T23:14:44.600469+0000","last_unstale":"2023-09-27T23:14:44.602876+0000","last_undegraded":"2023-09-27T23:14:44.602876+0000","last_fullsized":"2023-09-27T23:14:44.602876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:14.250239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604205+0000","last_change":"2023-09-27T23:14:44.590185+0000","last_active":"2023-09-27T23:14:44.604205+0000","last_peered":"2023-09-27T23:14:44.604205+0000","last_clean":"2023-09-27T23:14:44.604205+0000","last_became_active":"2023-09-27T23:14:44.589892+0000","last_became_peered":"2023-09-27T23:14:44.589892+0000","last_unstale":"2023-09-27T23:14:44.604205+0000","last_undegraded":"2023-09-27T23:14:44.604205+0000","last_fullsized":"2023-09-27T23:14:44.604205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:27.397171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.414355+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:14:37.414355+0000","last_peered":"2023-09-27T23:14:37.414355+0000","last_clean":"2023-09-27T23:14:37.414355+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:14:37.414355+0000","last_undegraded":"2023-09-27T23:14:37.414355+0000","last_fullsized":"2023-09-27T23:14:37.414355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589429+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:14:44.589429+0000","last_peered":"2023-09-27T23:14:44.589429+0000","last_clean":"2023-09-27T23:14:44.589429+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:14:44.589429+0000","last_undegraded":"2023-09-27T23:14:44.589429+0000","last_fullsized":"2023-09-27T23:14:44.589429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589480+0000","last_change":"2023-09-27T23:14:44.221449+0000","last_active":"2023-09-27T23:14:44.589480+0000","last_peered":"2023-09-27T23:14:44.589480+0000","last_clean":"2023-09-27T23:14:44.589480+0000","last_became_active":"2023-09-27T23:14:44.221090+0000","last_became_peered":"2023-09-27T23:14:44.221090+0000","last_unstale":"2023-09-27T23:14:44.589480+0000","last_undegraded":"2023-09-27T23:14:44.589480+0000","last_fullsized":"2023-09-27T23:14:44.589480+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:17:40.184736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296871+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:14:37.296871+0000","last_peered":"2023-09-27T23:14:37.296871+0000","last_clean":"2023-09-27T23:14:37.296871+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:14:37.296871+0000","last_undegraded":"2023-09-27T23:14:37.296871+0000","last_fullsized":"2023-09-27T23:14:37.296871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589240+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:14:44.589240+0000","last_peered":"2023-09-27T23:14:44.589240+0000","last_clean":"2023-09-27T23:14:44.589240+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:14:44.589240+0000","last_undegraded":"2023-09-27T23:14:44.589240+0000","last_fullsized":"2023-09-27T23:14:44.589240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.570569+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:14:44.570569+0000","last_peered":"2023-09-27T23:14:44.570569+0000","last_clean":"2023-09-27T23:14:44.570569+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:14:44.570569+0000","last_undegraded":"2023-09-27T23:14:44.570569+0000","last_fullsized":"2023-09-27T23:14:44.570569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589024+0000","last_change":"2023-09-27T23:14:44.212342+0000","last_active":"2023-09-27T23:14:44.589024+0000","last_peered":"2023-09-27T23:14:44.589024+0000","last_clean":"2023-09-27T23:14:44.589024+0000","last_became_active":"2023-09-27T23:14:44.211923+0000","last_became_peered":"2023-09-27T23:14:44.211923+0000","last_unstale":"2023-09-27T23:14:44.589024+0000","last_undegraded":"2023-09-27T23:14:44.589024+0000","last_fullsized":"2023-09-27T23:14:44.589024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:54:46.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296909+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:14:37.296909+0000","last_peered":"2023-09-27T23:14:37.296909+0000","last_clean":"2023-09-27T23:14:37.296909+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:14:37.296909+0000","last_undegraded":"2023-09-27T23:14:37.296909+0000","last_fullsized":"2023-09-27T23:14:37.296909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296920+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:14:37.296920+0000","last_peered":"2023-09-27T23:14:37.296920+0000","last_clean":"2023-09-27T23:14:37.296920+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:14:37.296920+0000","last_undegraded":"2023-09-27T23:14:37.296920+0000","last_fullsized":"2023-09-27T23:14:37.296920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":10,"acting":10,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.97799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.079}]},{"osd":2,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.016}]},{"osd":3,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.94799999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.878},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.1399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.042}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8452,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363388,"statfs":{"total":96636764160,"available":96628109312,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.90700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0409999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.165}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.1000000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.29}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.141}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.034},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.085}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1679999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.044},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.329}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.0069999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.97899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.1819999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.006}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.129}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.048}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.341},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.2370000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":507904,"data_stored":218009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93700000000000006}]},{"osd":1,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.052}]},{"osd":3,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.1040000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.23},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.091}]},{"osd":6,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.294},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.4790000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.998},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3120000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.157}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4259999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.157,"5min":6.157,"15min":6.157},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":5.649,"5min":5.649,"15min":5.649},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"last":0.503},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.282},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.014}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:48.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:48.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.274+0000 7f489759e700 1 Processor -- start 2023-09-27T23:14:48.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.274+0000 7f489759e700 1 -- start start 2023-09-27T23:14:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.274+0000 7f489759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4898155960 0x7f489815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.274+0000 7f489759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489806d990 con 0x7f48980c7f00 2023-09-27T23:14:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.274+0000 7f489759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489815a570 con 0x7f489806de20 2023-09-27T23:14:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.274+0000 7f489759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489806cd80 con 0x7f4898155960 2023-09-27T23:14:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4898155960 0x7f489815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4898155960 0x7f489815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57186/0 (socket says 172.21.15.64:57186) 2023-09-27T23:14:48.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 -- 172.21.15.64:0/3803288538 learned_addr learned my addr 172.21.15.64:0/3803288538 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:48.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 -- 172.21.15.64:0/3803288538 >> v1:172.21.15.123:6789/0 conn(0x7f489806de20 legacy=0x7f48980c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 -- 172.21.15.64:0/3803288538 >> v1:172.21.15.64:6789/0 conn(0x7f48980c7f00 legacy=0x7f48980aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 -- 172.21.15.64:0/3803288538 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489806d060 con 0x7f4898155960 2023-09-27T23:14:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489659c700 1 --2- 172.21.15.64:0/3803288538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4898155960 0x7f489815a030 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f488c0099b0 tx=0x7f488c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7ada838d56d234 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489559a700 1 -- 172.21.15.64:0/3803288538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00b860 con 0x7f4898155960 2023-09-27T23:14:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489559a700 1 -- 172.21.15.64:0/3803288538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00b9c0 con 0x7f4898155960 2023-09-27T23:14:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489559a700 1 -- 172.21.15.64:0/3803288538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00bd60 con 0x7f4898155960 2023-09-27T23:14:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 -- 172.21.15.64:0/3803288538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4898155960 msgr2=0x7f489815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 --2- 172.21.15.64:0/3803288538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4898155960 0x7f489815a030 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f488c0099b0 tx=0x7f488c009250 comp rx=0 tx=0).stop 2023-09-27T23:14:48.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 -- 172.21.15.64:0/3803288538 shutdown_connections 2023-09-27T23:14:48.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 --2- 172.21.15.64:0/3803288538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4898155960 0x7f489815a030 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 -- 172.21.15.64:0/3803288538 >> 172.21.15.64:0/3803288538 conn(0x7f4898067bb0 msgr2=0x7f48980cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:48.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 -- 172.21.15.64:0/3803288538 shutdown_connections 2023-09-27T23:14:48.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 -- 172.21.15.64:0/3803288538 wait complete. 2023-09-27T23:14:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 Processor -- start 2023-09-27T23:14:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.278+0000 7f489759e700 1 -- start start 2023-09-27T23:14:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489806de20 0x7f4898132270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 0x7f48981355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48981327b0 0x7f4898135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4898146b60 con 0x7f48980c7f00 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4898146e40 con 0x7f48981327b0 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4898147120 con 0x7f489806de20 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 0x7f48981355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 0x7f48981355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57194/0 (socket says 172.21.15.64:57194) 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 -- 172.21.15.64:0/2434700106 learned_addr learned my addr 172.21.15.64:0/2434700106 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4896d9d700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48981327b0 0x7f4898135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489659c700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489806de20 0x7f4898132270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 -- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489806de20 msgr2=0x7f4898132270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489806de20 0x7f4898132270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 -- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48981327b0 msgr2=0x7f4898135b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48981327b0 0x7f4898135b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 -- 172.21.15.64:0/2434700106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0094f0 con 0x7f48980c7f00 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f4895d9b700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 0x7f48981355e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f488800c590 tx=0x7f488800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888011590 con 0x7f48980c7f00 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888011710 con 0x7f48980c7f00 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4898147400 con 0x7f48980c7f00 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888011ad0 con 0x7f48980c7f00 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48980ae8e0 con 0x7f48980c7f00 2023-09-27T23:14:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.282+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4898147900 con 0x7f48980c7f00 2023-09-27T23:14:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.286+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f488801b3b0 con 0x7f48980c7f00 2023-09-27T23:14:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.286+0000 7f48867fc700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4878060950 0x7f4878062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.286+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f488807c420 con 0x7f48980c7f00 2023-09-27T23:14:48.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.286+0000 7f489659c700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4878060950 0x7f4878062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.286+0000 7f489659c700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4878060950 0x7f4878062e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f488c0099b0 tx=0x7f488c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.294+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f488804c560 con 0x7f48980c7f00 2023-09-27T23:14:48.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.454+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f489804be40 con 0x7f4878060950 2023-09-27T23:14:48.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.466+0000 7f48867fc700 1 -- 172.21.15.64:0/2434700106 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296003 (secure 0 0 0) 0x7f489804be40 con 0x7f4878060950 2023-09-27T23:14:48.476 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:48.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4878060950 msgr2=0x7f4878062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4878060950 0x7f4878062e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f488c0099b0 tx=0x7f488c017000 comp rx=0 tx=0).stop 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 msgr2=0x7f48981355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 0x7f48981355e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f488800c590 tx=0x7f488800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 shutdown_connections 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4878060950 0x7f4878062e00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48981327b0 0x7f4898135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c7f00 0x7f48981355e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 --2- 172.21.15.64:0/2434700106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489806de20 0x7f4898132270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 >> 172.21.15.64:0/2434700106 conn(0x7f4898067bb0 msgr2=0x7f48980c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 shutdown_connections 2023-09-27T23:14:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.474+0000 7f489759e700 1 -- 172.21.15.64:0/2434700106 wait complete. 2023-09-27T23:14:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:48.494 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-09-27T23:14:47.167854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591926,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":169,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":70596,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":65728,"kb_avail":754904124,"statfs":{"total":773094113280,"available":773021822976,"internally_reserved":0,"allocated":4919296,"data_stored":2788920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67305472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001295"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604058+0000","last_change":"2023-09-27T23:14:44.585114+0000","last_active":"2023-09-27T23:14:44.604058+0000","last_peered":"2023-09-27T23:14:44.604058+0000","last_clean":"2023-09-27T23:14:44.604058+0000","last_became_active":"2023-09-27T23:14:44.584800+0000","last_became_peered":"2023-09-27T23:14:44.584800+0000","last_unstale":"2023-09-27T23:14:44.604058+0000","last_undegraded":"2023-09-27T23:14:44.604058+0000","last_fullsized":"2023-09-27T23:14:44.604058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:32.813292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605484+0000","last_change":"2023-09-27T23:14:44.591703+0000","last_active":"2023-09-27T23:14:44.605484+0000","last_peered":"2023-09-27T23:14:44.605484+0000","last_clean":"2023-09-27T23:14:44.605484+0000","last_became_active":"2023-09-27T23:14:44.591432+0000","last_became_peered":"2023-09-27T23:14:44.591432+0000","last_unstale":"2023-09-27T23:14:44.605484+0000","last_undegraded":"2023-09-27T23:14:44.605484+0000","last_fullsized":"2023-09-27T23:14:44.605484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:05:55.604974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604052+0000","last_change":"2023-09-27T23:14:44.585552+0000","last_active":"2023-09-27T23:14:44.604052+0000","last_peered":"2023-09-27T23:14:44.604052+0000","last_clean":"2023-09-27T23:14:44.604052+0000","last_became_active":"2023-09-27T23:14:44.585234+0000","last_became_peered":"2023-09-27T23:14:44.585234+0000","last_unstale":"2023-09-27T23:14:44.604052+0000","last_undegraded":"2023-09-27T23:14:44.604052+0000","last_fullsized":"2023-09-27T23:14:44.604052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:21:55.504453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604266+0000","last_change":"2023-09-27T23:14:44.592678+0000","last_active":"2023-09-27T23:14:44.604266+0000","last_peered":"2023-09-27T23:14:44.604266+0000","last_clean":"2023-09-27T23:14:44.604266+0000","last_became_active":"2023-09-27T23:14:44.592380+0000","last_became_peered":"2023-09-27T23:14:44.592380+0000","last_unstale":"2023-09-27T23:14:44.604266+0000","last_undegraded":"2023-09-27T23:14:44.604266+0000","last_fullsized":"2023-09-27T23:14:44.604266+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:25.182304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601442+0000","last_change":"2023-09-27T23:14:44.596994+0000","last_active":"2023-09-27T23:14:44.601442+0000","last_peered":"2023-09-27T23:14:44.601442+0000","last_clean":"2023-09-27T23:14:44.601442+0000","last_became_active":"2023-09-27T23:14:44.596700+0000","last_became_peered":"2023-09-27T23:14:44.596700+0000","last_unstale":"2023-09-27T23:14:44.601442+0000","last_undegraded":"2023-09-27T23:14:44.601442+0000","last_fullsized":"2023-09-27T23:14:44.601442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:23.475703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609839+0000","last_change":"2023-09-27T23:14:44.217297+0000","last_active":"2023-09-27T23:14:44.609839+0000","last_peered":"2023-09-27T23:14:44.609839+0000","last_clean":"2023-09-27T23:14:44.609839+0000","last_became_active":"2023-09-27T23:14:44.216935+0000","last_became_peered":"2023-09-27T23:14:44.216935+0000","last_unstale":"2023-09-27T23:14:44.609839+0000","last_undegraded":"2023-09-27T23:14:44.609839+0000","last_fullsized":"2023-09-27T23:14:44.609839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:34.357991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600805+0000","last_change":"2023-09-27T23:14:44.594912+0000","last_active":"2023-09-27T23:14:44.600805+0000","last_peered":"2023-09-27T23:14:44.600805+0000","last_clean":"2023-09-27T23:14:44.600805+0000","last_became_active":"2023-09-27T23:14:44.594632+0000","last_became_peered":"2023-09-27T23:14:44.594632+0000","last_unstale":"2023-09-27T23:14:44.600805+0000","last_undegraded":"2023-09-27T23:14:44.600805+0000","last_fullsized":"2023-09-27T23:14:44.600805+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:00.409802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.612265+0000","last_change":"2023-09-27T23:14:44.213685+0000","last_active":"2023-09-27T23:14:44.612265+0000","last_peered":"2023-09-27T23:14:44.612265+0000","last_clean":"2023-09-27T23:14:44.612265+0000","last_became_active":"2023-09-27T23:14:44.213204+0000","last_became_peered":"2023-09-27T23:14:44.213204+0000","last_unstale":"2023-09-27T23:14:44.612265+0000","last_undegraded":"2023-09-27T23:14:44.612265+0000","last_fullsized":"2023-09-27T23:14:44.612265+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:51:19.929143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603875+0000","last_change":"2023-09-27T23:14:44.579511+0000","last_active":"2023-09-27T23:14:44.603875+0000","last_peered":"2023-09-27T23:14:44.603875+0000","last_clean":"2023-09-27T23:14:44.603875+0000","last_became_active":"2023-09-27T23:14:44.579172+0000","last_became_peered":"2023-09-27T23:14:44.579172+0000","last_unstale":"2023-09-27T23:14:44.603875+0000","last_undegraded":"2023-09-27T23:14:44.603875+0000","last_fullsized":"2023-09-27T23:14:44.603875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:48.765761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608309+0000","last_change":"2023-09-27T23:14:44.222653+0000","last_active":"2023-09-27T23:14:44.608309+0000","last_peered":"2023-09-27T23:14:44.608309+0000","last_clean":"2023-09-27T23:14:44.608309+0000","last_became_active":"2023-09-27T23:14:44.222336+0000","last_became_peered":"2023-09-27T23:14:44.222336+0000","last_unstale":"2023-09-27T23:14:44.608309+0000","last_undegraded":"2023-09-27T23:14:44.608309+0000","last_fullsized":"2023-09-27T23:14:44.608309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:45:57.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.602876+0000","last_change":"2023-09-27T23:14:44.600767+0000","last_active":"2023-09-27T23:14:44.602876+0000","last_peered":"2023-09-27T23:14:44.602876+0000","last_clean":"2023-09-27T23:14:44.602876+0000","last_became_active":"2023-09-27T23:14:44.600469+0000","last_became_peered":"2023-09-27T23:14:44.600469+0000","last_unstale":"2023-09-27T23:14:44.602876+0000","last_undegraded":"2023-09-27T23:14:44.602876+0000","last_fullsized":"2023-09-27T23:14:44.602876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:14.250239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604205+0000","last_change":"2023-09-27T23:14:44.590185+0000","last_active":"2023-09-27T23:14:44.604205+0000","last_peered":"2023-09-27T23:14:44.604205+0000","last_clean":"2023-09-27T23:14:44.604205+0000","last_became_active":"2023-09-27T23:14:44.589892+0000","last_became_peered":"2023-09-27T23:14:44.589892+0000","last_unstale":"2023-09-27T23:14:44.604205+0000","last_undegraded":"2023-09-27T23:14:44.604205+0000","last_fullsized":"2023-09-27T23:14:44.604205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:27.397171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.414355+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:14:37.414355+0000","last_peered":"2023-09-27T23:14:37.414355+0000","last_clean":"2023-09-27T23:14:37.414355+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:14:37.414355+0000","last_undegraded":"2023-09-27T23:14:37.414355+0000","last_fullsized":"2023-09-27T23:14:37.414355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589429+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:14:44.589429+0000","last_peered":"2023-09-27T23:14:44.589429+0000","last_clean":"2023-09-27T23:14:44.589429+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:14:44.589429+0000","last_undegraded":"2023-09-27T23:14:44.589429+0000","last_fullsized":"2023-09-27T23:14:44.589429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589480+0000","last_change":"2023-09-27T23:14:44.221449+0000","last_active":"2023-09-27T23:14:44.589480+0000","last_peered":"2023-09-27T23:14:44.589480+0000","last_clean":"2023-09-27T23:14:44.589480+0000","last_became_active":"2023-09-27T23:14:44.221090+0000","last_became_peered":"2023-09-27T23:14:44.221090+0000","last_unstale":"2023-09-27T23:14:44.589480+0000","last_undegraded":"2023-09-27T23:14:44.589480+0000","last_fullsized":"2023-09-27T23:14:44.589480+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:17:40.184736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296871+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:14:37.296871+0000","last_peered":"2023-09-27T23:14:37.296871+0000","last_clean":"2023-09-27T23:14:37.296871+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:14:37.296871+0000","last_undegraded":"2023-09-27T23:14:37.296871+0000","last_fullsized":"2023-09-27T23:14:37.296871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589240+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:14:44.589240+0000","last_peered":"2023-09-27T23:14:44.589240+0000","last_clean":"2023-09-27T23:14:44.589240+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:14:44.589240+0000","last_undegraded":"2023-09-27T23:14:44.589240+0000","last_fullsized":"2023-09-27T23:14:44.589240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.570569+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:14:44.570569+0000","last_peered":"2023-09-27T23:14:44.570569+0000","last_clean":"2023-09-27T23:14:44.570569+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:14:44.570569+0000","last_undegraded":"2023-09-27T23:14:44.570569+0000","last_fullsized":"2023-09-27T23:14:44.570569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589024+0000","last_change":"2023-09-27T23:14:44.212342+0000","last_active":"2023-09-27T23:14:44.589024+0000","last_peered":"2023-09-27T23:14:44.589024+0000","last_clean":"2023-09-27T23:14:44.589024+0000","last_became_active":"2023-09-27T23:14:44.211923+0000","last_became_peered":"2023-09-27T23:14:44.211923+0000","last_unstale":"2023-09-27T23:14:44.589024+0000","last_undegraded":"2023-09-27T23:14:44.589024+0000","last_fullsized":"2023-09-27T23:14:44.589024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:54:46.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296909+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:14:37.296909+0000","last_peered":"2023-09-27T23:14:37.296909+0000","last_clean":"2023-09-27T23:14:37.296909+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:14:37.296909+0000","last_undegraded":"2023-09-27T23:14:37.296909+0000","last_fullsized":"2023-09-27T23:14:37.296909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296920+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:14:37.296920+0000","last_peered":"2023-09-27T23:14:37.296920+0000","last_clean":"2023-09-27T23:14:37.296920+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:14:37.296920+0000","last_undegraded":"2023-09-27T23:14:37.296920+0000","last_fullsized":"2023-09-27T23:14:37.296920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":10,"acting":10,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.97799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.079}]},{"osd":2,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.016}]},{"osd":3,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.94799999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.878},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.1399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.042}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8452,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363388,"statfs":{"total":96636764160,"available":96628109312,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.90700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0409999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.165}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.1000000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.29}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.141}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.034},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.085}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1679999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.044},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.329}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.0069999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.97899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.1819999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.006}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.129}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.048}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.341},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.2370000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":507904,"data_stored":218009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93700000000000006}]},{"osd":1,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.052}]},{"osd":3,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.1040000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.23},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.091}]},{"osd":6,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.294},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.4790000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.998},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3120000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.157}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4259999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.157,"5min":6.157,"15min":6.157},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":5.649,"5min":5.649,"15min":5.649},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"last":0.503},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.282},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.014}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-09-27T23:14:48.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-09-27T23:14:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-09-27T23:14:48.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-09-27T23:14:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-09-27T23:14:48.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-09-27T23:14:48.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-09-27T23:14:48.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-09-27T23:14:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-09-27T23:14:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-09-27T23:14:48.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-09-27T23:14:48.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:44.186626+0000', 'last_change': '2023-09-27T23:14:44.186626+0000', 'last_active': '2023-09-27T23:14:44.186626+0000', 'last_peered': '2023-09-27T23:14:44.186626+0000', 'last_clean': '2023-09-27T23:14:44.186626+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:44.186626+0000', 'last_undegraded': '2023-09-27T23:14:44.186626+0000', 'last_fullsized': '2023-09-27T23:14:44.186626+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:44.186626+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-09-27T23:14:48.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:14:43.175985+0000', 'last_change': '2023-09-27T23:14:43.175985+0000', 'last_active': '2023-09-27T23:14:43.175985+0000', 'last_peered': '2023-09-27T23:14:43.175985+0000', 'last_clean': '2023-09-27T23:14:43.175985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:14:43.175985+0000', 'last_undegraded': '2023-09-27T23:14:43.175985+0000', 'last_fullsized': '2023-09-27T23:14:43.175985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'last_clean_scrub_stamp': '2023-09-27T23:14:43.175985+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:14:48.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:14:48.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.618+0000 7ff852028700 1 Processor -- start 2023-09-27T23:14:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.618+0000 7ff852028700 1 -- start start 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff84c0ce3d0 0x7ff84c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff84c06b8c0 0x7ff84c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84c15da00 con 0x7ff84c1589c0 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84c06a540 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84c06a820 con 0x7ff84c06b8c0 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851026700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff84c0ce3d0 0x7ff84c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851827700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff84c1589c0 0x7ff84c158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51588/0 (socket says 172.21.15.64:51588) 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851827700 1 -- 172.21.15.64:0/1495869285 learned_addr learned my addr 172.21.15.64:0/1495869285 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff850825700 1 --2- 172.21.15.64:0/1495869285 >> v2:172.21.15.123:3300/0 conn(0x7ff84c06b8c0 0x7ff84c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff843fff700 1 -- 172.21.15.64:0/1495869285 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516421760 0 0) 0x7ff84c15da00 con 0x7ff84c1589c0 2023-09-27T23:14:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff843fff700 1 -- 172.21.15.64:0/1495869285 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff838003570 con 0x7ff84c1589c0 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851026700 1 -- 172.21.15.64:0/1495869285 >> v2:172.21.15.123:3300/0 conn(0x7ff84c06b8c0 msgr2=0x7ff84c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851026700 1 --2- 172.21.15.64:0/1495869285 >> v2:172.21.15.123:3300/0 conn(0x7ff84c06b8c0 0x7ff84c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851026700 1 -- 172.21.15.64:0/1495869285 >> v1:172.21.15.64:6789/0 conn(0x7ff84c1589c0 legacy=0x7ff84c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851026700 1 -- 172.21.15.64:0/1495869285 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c06ab00 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff851026700 1 --2- 172.21.15.64:0/1495869285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c0ce7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff84c064350 tx=0x7ff8480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883b8b30d97264cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff843fff700 1 -- 172.21.15.64:0/1495869285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848011630 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff843fff700 1 -- 172.21.15.64:0/1495869285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8480117b0 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff843fff700 1 -- 172.21.15.64:0/1495869285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848011b70 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- 172.21.15.64:0/1495869285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 msgr2=0x7ff84c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 --2- 172.21.15.64:0/1495869285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c0ce7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff84c064350 tx=0x7ff8480091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- 172.21.15.64:0/1495869285 shutdown_connections 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 --2- 172.21.15.64:0/1495869285 >> v2:172.21.15.123:3300/0 conn(0x7ff84c06b8c0 0x7ff84c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 --2- 172.21.15.64:0/1495869285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c0ce7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- 172.21.15.64:0/1495869285 >> 172.21.15.64:0/1495869285 conn(0x7ff84c067bb0 msgr2=0x7ff84c0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- 172.21.15.64:0/1495869285 shutdown_connections 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.622+0000 7ff852028700 1 -- 172.21.15.64:0/1495869285 wait complete. 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.626+0000 7ff852028700 1 Processor -- start 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.626+0000 7ff852028700 1 -- start start 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.626+0000 7ff852028700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff84c06b8c0 0x7ff84c0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.626+0000 7ff852028700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c13ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1589c0 0x7ff84c15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c8600 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c88e0 con 0x7ff84c1589c0 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c8bc0 con 0x7ff84c06b8c0 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c13ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c13ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57214/0 (socket says 172.21.15.64:57214) 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 -- 172.21.15.64:0/2740301619 learned_addr learned my addr 172.21.15.64:0/2740301619 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff851026700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff84c06b8c0 0x7ff84c0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff851827700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1589c0 0x7ff84c15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 -- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff84c06b8c0 msgr2=0x7ff84c0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff84c06b8c0 0x7ff84c0c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 -- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1589c0 msgr2=0x7ff84c15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1589c0 0x7ff84c15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 -- 172.21.15.64:0/2740301619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff848009420 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff851026700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff84c06b8c0 0x7ff84c0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff850825700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c13ef00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00ec80 tx=0x7ff83c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00c9d0 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c00cb30 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c0c8ea0 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c013470 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c0c9be0 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.630+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c04be40 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.634+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7ff83c013820 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.634+0000 7ff8417fa700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff830060950 0x7ff830062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.634+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff83c07d300 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.634+0000 7ff851026700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff830060950 0x7ff830062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.634+0000 7ff851026700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff830060950 0x7ff830062e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff84c064350 tx=0x7ff848017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.638+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff83c04d440 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff84c06ade0 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff8417fa700 1 -- 172.21.15.64:0/2740301619 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1507 (secure 0 0 0) 0x7ff83c050a60 con 0x7ff84c0ce3d0 2023-09-27T23:14:48.866 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:48.866 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":87,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":82},{"state_name":"active+clean","count":23}],"num_pgs":105,"num_pools":4,"num_objects":8,"data_bytes":591926,"bytes_used":72290304,"bytes_avail":773021822976,"bytes_total":773094113280,"unknown_pgs_ratio":0.78095239400863647,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5211}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-09-27T23:14:33.164129+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:14:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff830060950 msgr2=0x7ff830062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff830060950 0x7ff830062e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff84c064350 tx=0x7ff848017000 comp rx=0 tx=0).stop 2023-09-27T23:14:48.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 msgr2=0x7ff84c13ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c13ef00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00ec80 tx=0x7ff83c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 shutdown_connections 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff830060950 0x7ff830062e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff84c1589c0 0x7ff84c15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff84c0ce3d0 0x7ff84c13ef00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 --2- 172.21.15.64:0/2740301619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff84c06b8c0 0x7ff84c0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.862+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 >> 172.21.15.64:0/2740301619 conn(0x7ff84c067bb0 msgr2=0x7ff84c0c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.866+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 shutdown_connections 2023-09-27T23:14:48.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.866+0000 7ff852028700 1 -- 172.21.15.64:0/2740301619 wait complete. 2023-09-27T23:14:48.880 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:14:48.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:48.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 Processor -- start 2023-09-27T23:14:48.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- start start 2023-09-27T23:14:48.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f58f415d320 0x7f58f415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f40c15b0 0x7f58f40aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40c7190 con 0x7f58f406d690 2023-09-27T23:14:48.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40c1b50 con 0x7f58f415d320 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c1e30 con 0x7f58f40c15b0 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f96f3700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58f406d690 0x7f58f40680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51598/0 (socket says 172.21.15.64:51598) 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f96f3700 1 -- 172.21.15.64:0/1672633064 learned_addr learned my addr 172.21.15.64:0/1672633064 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f3fff700 1 --2- 172.21.15.64:0/1672633064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f40c15b0 0x7f58f40aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f8ef2700 1 --2- 172.21.15.64:0/1672633064 >> v2:172.21.15.123:3300/0 conn(0x7f58f415d320 0x7f58f415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f37fe700 1 -- 172.21.15.64:0/1672633064 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323733925 0 0) 0x7f58f40c7190 con 0x7f58f406d690 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f37fe700 1 -- 172.21.15.64:0/1672633064 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003570 con 0x7f58f406d690 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f3fff700 1 -- 172.21.15.64:0/1672633064 >> v2:172.21.15.123:3300/0 conn(0x7f58f415d320 msgr2=0x7f58f415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f3fff700 1 --2- 172.21.15.64:0/1672633064 >> v2:172.21.15.123:3300/0 conn(0x7f58f415d320 0x7f58f415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f3fff700 1 -- 172.21.15.64:0/1672633064 >> v1:172.21.15.64:6789/0 conn(0x7f58f406d690 legacy=0x7f58f40680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f3fff700 1 -- 172.21.15.64:0/1672633064 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40c6e20 con 0x7f58f40c15b0 2023-09-27T23:14:48.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f8ef2700 1 --2- 172.21.15.64:0/1672633064 >> v2:172.21.15.123:3300/0 conn(0x7f58f415d320 0x7f58f415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:48.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f3fff700 1 --2- 172.21.15.64:0/1672633064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f40c15b0 0x7f58f40aeba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58e40098e0 tx=0x7f58e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7f4ce9812c001 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f37fe700 1 -- 172.21.15.64:0/1672633064 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4011630 con 0x7f58f40c15b0 2023-09-27T23:14:48.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f37fe700 1 -- 172.21.15.64:0/1672633064 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e4011790 con 0x7f58f40c15b0 2023-09-27T23:14:48.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58f37fe700 1 -- 172.21.15.64:0/1672633064 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4011b10 con 0x7f58f40c15b0 2023-09-27T23:14:48.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- 172.21.15.64:0/1672633064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f40c15b0 msgr2=0x7f58f40aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 --2- 172.21.15.64:0/1672633064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f40c15b0 0x7f58f40aeba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58e40098e0 tx=0x7f58e40091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- 172.21.15.64:0/1672633064 shutdown_connections 2023-09-27T23:14:48.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 --2- 172.21.15.64:0/1672633064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f40c15b0 0x7f58f40aeba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 --2- 172.21.15.64:0/1672633064 >> v2:172.21.15.123:3300/0 conn(0x7f58f415d320 0x7f58f415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- 172.21.15.64:0/1672633064 >> 172.21.15.64:0/1672633064 conn(0x7f58f40c8850 msgr2=0x7f58f40c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:48.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- 172.21.15.64:0/1672633064 shutdown_connections 2023-09-27T23:14:48.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.982+0000 7f58fb156700 1 -- 172.21.15.64:0/1672633064 wait complete. 2023-09-27T23:14:48.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 Processor -- start 2023-09-27T23:14:48.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 -- start start 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 0x7f58f406a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f40c15b0 0x7f58f406adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 0x7f58f40b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b1140 con 0x7f58f415d320 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c3b90 con 0x7f58f40c15b0 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58fb156700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b0e60 con 0x7f58f406d690 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 0x7f58f40b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f8ef2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 0x7f58f406a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f8ef2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 0x7f58f406a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:46950/0 (socket says 172.21.15.64:46950) 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 0x7f58f40b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57220/0 (socket says 172.21.15.64:57220) 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f8ef2700 1 -- 172.21.15.64:0/3611492858 learned_addr learned my addr 172.21.15.64:0/3611492858 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f3fff700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f40c15b0 0x7f58f406adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 -- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 msgr2=0x7f58f406a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 0x7f58f406a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 -- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f40c15b0 msgr2=0x7f58f406adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f40c15b0 0x7f58f406adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 -- 172.21.15.64:0/3611492858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009420 con 0x7f58f415d320 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f8ef2700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 0x7f58f406a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:48.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f3fff700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f40c15b0 0x7f58f406adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f96f3700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 0x7f58f40b00f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001560 tx=0x7f58ec001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.986+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00a8b0 con 0x7f58f415d320 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.990+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec00aa10 con 0x7f58f415d320 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.990+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e4009420 con 0x7f58f415d320 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.990+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00add0 con 0x7f58f415d320 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.990+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f40c4110 con 0x7f58f415d320 2023-09-27T23:14:48.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.990+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4071120 con 0x7f58f415d320 2023-09-27T23:14:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.990+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f58ec007300 con 0x7f58f415d320 2023-09-27T23:14:48.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.994+0000 7f58f0ff9700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58d4060950 0x7f58d4062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:48.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.994+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f58ec07ddb0 con 0x7f58f415d320 2023-09-27T23:14:48.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.994+0000 7f58f8ef2700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58d4060950 0x7f58d4062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:48.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.994+0000 7f58f8ef2700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58d4060950 0x7f58d4062e00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58f406b3f0 tx=0x7f58dc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:49.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:48.998+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f58ec04df70 con 0x7f58f415d320 2023-09-27T23:14:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.154+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58f40642a0 con 0x7f58d4060950 2023-09-27T23:14:49.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.170+0000 7f58f0ff9700 1 -- 172.21.15.64:0/3611492858 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296003 (secure 0 0 0) 0x7f58f40642a0 con 0x7f58d4060950 2023-09-27T23:14:49.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58d4060950 msgr2=0x7f58d4062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58d4060950 0x7f58d4062e00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58f406b3f0 tx=0x7f58dc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 msgr2=0x7f58f40b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 0x7f58f40b00f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001560 tx=0x7f58ec001a00 comp rx=0 tx=0).stop 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 shutdown_connections 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58d4060950 0x7f58d4062e00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f415d320 0x7f58f40b00f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f40c15b0 0x7f58f406adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 --2- 172.21.15.64:0/3611492858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f406d690 0x7f58f406a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 >> 172.21.15.64:0/3611492858 conn(0x7f58f40c8850 msgr2=0x7f58f40ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 shutdown_connections 2023-09-27T23:14:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:49.174+0000 7f58fb156700 1 -- 172.21.15.64:0/3611492858 wait complete. 2023-09-27T23:14:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:49.193 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-09-27T23:14:47.167854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591926,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":169,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":70596,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":65728,"kb_avail":754904124,"statfs":{"total":773094113280,"available":773021822976,"internally_reserved":0,"allocated":4919296,"data_stored":2788920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67305472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001295"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604058+0000","last_change":"2023-09-27T23:14:44.585114+0000","last_active":"2023-09-27T23:14:44.604058+0000","last_peered":"2023-09-27T23:14:44.604058+0000","last_clean":"2023-09-27T23:14:44.604058+0000","last_became_active":"2023-09-27T23:14:44.584800+0000","last_became_peered":"2023-09-27T23:14:44.584800+0000","last_unstale":"2023-09-27T23:14:44.604058+0000","last_undegraded":"2023-09-27T23:14:44.604058+0000","last_fullsized":"2023-09-27T23:14:44.604058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:32.813292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605484+0000","last_change":"2023-09-27T23:14:44.591703+0000","last_active":"2023-09-27T23:14:44.605484+0000","last_peered":"2023-09-27T23:14:44.605484+0000","last_clean":"2023-09-27T23:14:44.605484+0000","last_became_active":"2023-09-27T23:14:44.591432+0000","last_became_peered":"2023-09-27T23:14:44.591432+0000","last_unstale":"2023-09-27T23:14:44.605484+0000","last_undegraded":"2023-09-27T23:14:44.605484+0000","last_fullsized":"2023-09-27T23:14:44.605484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:05:55.604974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604052+0000","last_change":"2023-09-27T23:14:44.585552+0000","last_active":"2023-09-27T23:14:44.604052+0000","last_peered":"2023-09-27T23:14:44.604052+0000","last_clean":"2023-09-27T23:14:44.604052+0000","last_became_active":"2023-09-27T23:14:44.585234+0000","last_became_peered":"2023-09-27T23:14:44.585234+0000","last_unstale":"2023-09-27T23:14:44.604052+0000","last_undegraded":"2023-09-27T23:14:44.604052+0000","last_fullsized":"2023-09-27T23:14:44.604052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:21:55.504453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604266+0000","last_change":"2023-09-27T23:14:44.592678+0000","last_active":"2023-09-27T23:14:44.604266+0000","last_peered":"2023-09-27T23:14:44.604266+0000","last_clean":"2023-09-27T23:14:44.604266+0000","last_became_active":"2023-09-27T23:14:44.592380+0000","last_became_peered":"2023-09-27T23:14:44.592380+0000","last_unstale":"2023-09-27T23:14:44.604266+0000","last_undegraded":"2023-09-27T23:14:44.604266+0000","last_fullsized":"2023-09-27T23:14:44.604266+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:25.182304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601442+0000","last_change":"2023-09-27T23:14:44.596994+0000","last_active":"2023-09-27T23:14:44.601442+0000","last_peered":"2023-09-27T23:14:44.601442+0000","last_clean":"2023-09-27T23:14:44.601442+0000","last_became_active":"2023-09-27T23:14:44.596700+0000","last_became_peered":"2023-09-27T23:14:44.596700+0000","last_unstale":"2023-09-27T23:14:44.601442+0000","last_undegraded":"2023-09-27T23:14:44.601442+0000","last_fullsized":"2023-09-27T23:14:44.601442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:23.475703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609839+0000","last_change":"2023-09-27T23:14:44.217297+0000","last_active":"2023-09-27T23:14:44.609839+0000","last_peered":"2023-09-27T23:14:44.609839+0000","last_clean":"2023-09-27T23:14:44.609839+0000","last_became_active":"2023-09-27T23:14:44.216935+0000","last_became_peered":"2023-09-27T23:14:44.216935+0000","last_unstale":"2023-09-27T23:14:44.609839+0000","last_undegraded":"2023-09-27T23:14:44.609839+0000","last_fullsized":"2023-09-27T23:14:44.609839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:34.357991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600805+0000","last_change":"2023-09-27T23:14:44.594912+0000","last_active":"2023-09-27T23:14:44.600805+0000","last_peered":"2023-09-27T23:14:44.600805+0000","last_clean":"2023-09-27T23:14:44.600805+0000","last_became_active":"2023-09-27T23:14:44.594632+0000","last_became_peered":"2023-09-27T23:14:44.594632+0000","last_unstale":"2023-09-27T23:14:44.600805+0000","last_undegraded":"2023-09-27T23:14:44.600805+0000","last_fullsized":"2023-09-27T23:14:44.600805+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:00.409802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.612265+0000","last_change":"2023-09-27T23:14:44.213685+0000","last_active":"2023-09-27T23:14:44.612265+0000","last_peered":"2023-09-27T23:14:44.612265+0000","last_clean":"2023-09-27T23:14:44.612265+0000","last_became_active":"2023-09-27T23:14:44.213204+0000","last_became_peered":"2023-09-27T23:14:44.213204+0000","last_unstale":"2023-09-27T23:14:44.612265+0000","last_undegraded":"2023-09-27T23:14:44.612265+0000","last_fullsized":"2023-09-27T23:14:44.612265+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:51:19.929143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603875+0000","last_change":"2023-09-27T23:14:44.579511+0000","last_active":"2023-09-27T23:14:44.603875+0000","last_peered":"2023-09-27T23:14:44.603875+0000","last_clean":"2023-09-27T23:14:44.603875+0000","last_became_active":"2023-09-27T23:14:44.579172+0000","last_became_peered":"2023-09-27T23:14:44.579172+0000","last_unstale":"2023-09-27T23:14:44.603875+0000","last_undegraded":"2023-09-27T23:14:44.603875+0000","last_fullsized":"2023-09-27T23:14:44.603875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:48.765761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608309+0000","last_change":"2023-09-27T23:14:44.222653+0000","last_active":"2023-09-27T23:14:44.608309+0000","last_peered":"2023-09-27T23:14:44.608309+0000","last_clean":"2023-09-27T23:14:44.608309+0000","last_became_active":"2023-09-27T23:14:44.222336+0000","last_became_peered":"2023-09-27T23:14:44.222336+0000","last_unstale":"2023-09-27T23:14:44.608309+0000","last_undegraded":"2023-09-27T23:14:44.608309+0000","last_fullsized":"2023-09-27T23:14:44.608309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:45:57.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.602876+0000","last_change":"2023-09-27T23:14:44.600767+0000","last_active":"2023-09-27T23:14:44.602876+0000","last_peered":"2023-09-27T23:14:44.602876+0000","last_clean":"2023-09-27T23:14:44.602876+0000","last_became_active":"2023-09-27T23:14:44.600469+0000","last_became_peered":"2023-09-27T23:14:44.600469+0000","last_unstale":"2023-09-27T23:14:44.602876+0000","last_undegraded":"2023-09-27T23:14:44.602876+0000","last_fullsized":"2023-09-27T23:14:44.602876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:14.250239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604205+0000","last_change":"2023-09-27T23:14:44.590185+0000","last_active":"2023-09-27T23:14:44.604205+0000","last_peered":"2023-09-27T23:14:44.604205+0000","last_clean":"2023-09-27T23:14:44.604205+0000","last_became_active":"2023-09-27T23:14:44.589892+0000","last_became_peered":"2023-09-27T23:14:44.589892+0000","last_unstale":"2023-09-27T23:14:44.604205+0000","last_undegraded":"2023-09-27T23:14:44.604205+0000","last_fullsized":"2023-09-27T23:14:44.604205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:27.397171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.414355+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:14:37.414355+0000","last_peered":"2023-09-27T23:14:37.414355+0000","last_clean":"2023-09-27T23:14:37.414355+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:14:37.414355+0000","last_undegraded":"2023-09-27T23:14:37.414355+0000","last_fullsized":"2023-09-27T23:14:37.414355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589429+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:14:44.589429+0000","last_peered":"2023-09-27T23:14:44.589429+0000","last_clean":"2023-09-27T23:14:44.589429+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:14:44.589429+0000","last_undegraded":"2023-09-27T23:14:44.589429+0000","last_fullsized":"2023-09-27T23:14:44.589429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589480+0000","last_change":"2023-09-27T23:14:44.221449+0000","last_active":"2023-09-27T23:14:44.589480+0000","last_peered":"2023-09-27T23:14:44.589480+0000","last_clean":"2023-09-27T23:14:44.589480+0000","last_became_active":"2023-09-27T23:14:44.221090+0000","last_became_peered":"2023-09-27T23:14:44.221090+0000","last_unstale":"2023-09-27T23:14:44.589480+0000","last_undegraded":"2023-09-27T23:14:44.589480+0000","last_fullsized":"2023-09-27T23:14:44.589480+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:17:40.184736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296871+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:14:37.296871+0000","last_peered":"2023-09-27T23:14:37.296871+0000","last_clean":"2023-09-27T23:14:37.296871+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:14:37.296871+0000","last_undegraded":"2023-09-27T23:14:37.296871+0000","last_fullsized":"2023-09-27T23:14:37.296871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589240+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:14:44.589240+0000","last_peered":"2023-09-27T23:14:44.589240+0000","last_clean":"2023-09-27T23:14:44.589240+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:14:44.589240+0000","last_undegraded":"2023-09-27T23:14:44.589240+0000","last_fullsized":"2023-09-27T23:14:44.589240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.570569+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:14:44.570569+0000","last_peered":"2023-09-27T23:14:44.570569+0000","last_clean":"2023-09-27T23:14:44.570569+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:14:44.570569+0000","last_undegraded":"2023-09-27T23:14:44.570569+0000","last_fullsized":"2023-09-27T23:14:44.570569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589024+0000","last_change":"2023-09-27T23:14:44.212342+0000","last_active":"2023-09-27T23:14:44.589024+0000","last_peered":"2023-09-27T23:14:44.589024+0000","last_clean":"2023-09-27T23:14:44.589024+0000","last_became_active":"2023-09-27T23:14:44.211923+0000","last_became_peered":"2023-09-27T23:14:44.211923+0000","last_unstale":"2023-09-27T23:14:44.589024+0000","last_undegraded":"2023-09-27T23:14:44.589024+0000","last_fullsized":"2023-09-27T23:14:44.589024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:54:46.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296909+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:14:37.296909+0000","last_peered":"2023-09-27T23:14:37.296909+0000","last_clean":"2023-09-27T23:14:37.296909+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:14:37.296909+0000","last_undegraded":"2023-09-27T23:14:37.296909+0000","last_fullsized":"2023-09-27T23:14:37.296909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:44.186626+0000","last_change":"2023-09-27T23:14:44.186626+0000","last_active":"2023-09-27T23:14:44.186626+0000","last_peered":"2023-09-27T23:14:44.186626+0000","last_clean":"2023-09-27T23:14:44.186626+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:44.186626+0000","last_undegraded":"2023-09-27T23:14:44.186626+0000","last_fullsized":"2023-09-27T23:14:44.186626+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:14:43.175985+0000","last_change":"2023-09-27T23:14:43.175985+0000","last_active":"2023-09-27T23:14:43.175985+0000","last_peered":"2023-09-27T23:14:43.175985+0000","last_clean":"2023-09-27T23:14:43.175985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:14:43.175985+0000","last_undegraded":"2023-09-27T23:14:43.175985+0000","last_fullsized":"2023-09-27T23:14:43.175985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.295313+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:14:37.295313+0000","last_peered":"2023-09-27T23:14:37.295313+0000","last_clean":"2023-09-27T23:14:37.295313+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:14:37.295313+0000","last_undegraded":"2023-09-27T23:14:37.295313+0000","last_fullsized":"2023-09-27T23:14:37.295313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-09-27T23:14:37.296920+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:14:37.296920+0000","last_peered":"2023-09-27T23:14:37.296920+0000","last_clean":"2023-09-27T23:14:37.296920+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:14:37.296920+0000","last_undegraded":"2023-09-27T23:14:37.296920+0000","last_fullsized":"2023-09-27T23:14:37.296920+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1558,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":10,"acting":10,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.97799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.079}]},{"osd":2,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.016}]},{"osd":3,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.94799999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.878},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.1399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.042}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8452,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363388,"statfs":{"total":96636764160,"available":96628109312,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.90700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0409999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.165}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.1000000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.29}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.141}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.034},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.085}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1679999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.044},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.329}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.0069999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.97899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.1819999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8516,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363324,"statfs":{"total":96636764160,"available":96628043776,"internally_reserved":0,"allocated":454656,"data_stored":195609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.006}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.129}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.048}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.341},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.2370000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":507904,"data_stored":218009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93700000000000006}]},{"osd":1,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.052}]},{"osd":3,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.1040000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.23},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.091}]},{"osd":6,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.294},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.4790000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":785977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.998},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3120000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.157}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4259999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.157,"5min":6.157,"15min":6.157},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":5.649,"5min":5.649,"15min":5.649},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"last":0.503},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.282},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.014}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:52.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:14:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.310+0000 7f851ad2b700 1 Processor -- start 2023-09-27T23:14:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.310+0000 7f851ad2b700 1 -- start start 2023-09-27T23:14:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8514069100 0x7f8514067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8514068c70 con 0x7f8514158f70 2023-09-27T23:14:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8514068060 con 0x7f8514159340 2023-09-27T23:14:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8514068340 con 0x7f8514069100 2023-09-27T23:14:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8514069100 0x7f8514067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8514069100 0x7f8514067980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57226/0 (socket says 172.21.15.64:57226) 2023-09-27T23:14:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 -- 172.21.15.64:0/819101446 learned_addr learned my addr 172.21.15.64:0/819101446 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8518ac7700 1 --1- 172.21.15.64:0/819101446 >> v1:172.21.15.64:6789/0 conn(0x7f8514158f70 0x7f85140cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51600/0 (socket says 172.21.15.64:51600) 2023-09-27T23:14:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 -- 172.21.15.64:0/819101446 >> v1:172.21.15.123:6789/0 conn(0x7f8514159340 legacy=0x7f85140cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 -- 172.21.15.64:0/819101446 >> v1:172.21.15.64:6789/0 conn(0x7f8514158f70 legacy=0x7f85140cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 -- 172.21.15.64:0/819101446 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514068620 con 0x7f8514069100 2023-09-27T23:14:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f85137fe700 1 -- 172.21.15.64:0/819101446 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361549031 0 0) 0x7f8514068c70 con 0x7f8514158f70 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f8513fff700 1 --2- 172.21.15.64:0/819101446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514069100 0x7f8514067980 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85000099e0 tx=0x7f8500009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564b59d680c21a9e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f85137fe700 1 -- 172.21.15.64:0/819101446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850000b820 con 0x7f8514069100 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f85137fe700 1 -- 172.21.15.64:0/819101446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850000b980 con 0x7f8514069100 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f85137fe700 1 -- 172.21.15.64:0/819101446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850000bd20 con 0x7f8514069100 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- 172.21.15.64:0/819101446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514069100 msgr2=0x7f8514067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 --2- 172.21.15.64:0/819101446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514069100 0x7f8514067980 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85000099e0 tx=0x7f8500009250 comp rx=0 tx=0).stop 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- 172.21.15.64:0/819101446 shutdown_connections 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 --2- 172.21.15.64:0/819101446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514069100 0x7f8514067980 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- 172.21.15.64:0/819101446 >> 172.21.15.64:0/819101446 conn(0x7f85140ccc40 msgr2=0x7f85140cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- 172.21.15.64:0/819101446 shutdown_connections 2023-09-27T23:14:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.314+0000 7f851ad2b700 1 -- 172.21.15.64:0/819101446 wait complete. 2023-09-27T23:14:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 Processor -- start 2023-09-27T23:14:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 -- start start 2023-09-27T23:14:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8514069100 0x7f85140c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 0x7f8514169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8514159340 0x7f8514151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514169d80 con 0x7f8514158f70 2023-09-27T23:14:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140afd70 con 0x7f8514159340 2023-09-27T23:14:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f851ad2b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8514151e70 con 0x7f8514069100 2023-09-27T23:14:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 0x7f8514169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 0x7f8514169840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57236/0 (socket says 172.21.15.64:57236) 2023-09-27T23:14:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 -- 172.21.15.64:0/937963094 learned_addr learned my addr 172.21.15.64:0/937963094 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f85192c8700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8514159340 0x7f8514151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8518ac7700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8514069100 0x7f85140c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 -- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8514069100 msgr2=0x7f85140c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8514069100 0x7f85140c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 -- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8514159340 msgr2=0x7f8514151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8514159340 0x7f8514151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 -- 172.21.15.64:0/937963094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000094f0 con 0x7f8514158f70 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.318+0000 7f8513fff700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 0x7f8514169840 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85000094c0 tx=0x7f8500017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001f070 con 0x7f8514158f70 2023-09-27T23:14:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850000bed0 con 0x7f8514158f70 2023-09-27T23:14:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8514169fc0 con 0x7f8514158f70 2023-09-27T23:14:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850000bd20 con 0x7f8514158f70 2023-09-27T23:14:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851416a580 con 0x7f8514158f70 2023-09-27T23:14:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8514071120 con 0x7f8514158f70 2023-09-27T23:14:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8500023070 con 0x7f8514158f70 2023-09-27T23:14:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8510ff9700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f84f8060980 0x7f84f8062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f850007e300 con 0x7f8514158f70 2023-09-27T23:14:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.322+0000 7f8518ac7700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f84f8060980 0x7f84f8062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.326+0000 7f8518ac7700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f84f8060980 0x7f84f8062e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f851416cd40 tx=0x7f8508007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.330+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f850004e440 con 0x7f8514158f70 2023-09-27T23:14:52.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.494+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85140afb60 con 0x7f84f8060980 2023-09-27T23:14:52.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.510+0000 7f8510ff9700 1 -- 172.21.15.64:0/937963094 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308164 (secure 0 0 0) 0x7f8508004c10 con 0x7f84f8060980 2023-09-27T23:14:52.521 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:52.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f84f8060980 msgr2=0x7f84f8062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f84f8060980 0x7f84f8062e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f851416cd40 tx=0x7f8508007d30 comp rx=0 tx=0).stop 2023-09-27T23:14:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 msgr2=0x7f8514169840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 0x7f8514169840 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85000094c0 tx=0x7f8500017000 comp rx=0 tx=0).stop 2023-09-27T23:14:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 shutdown_connections 2023-09-27T23:14:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f84f8060980 0x7f84f8062e30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8514159340 0x7f8514151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514158f70 0x7f8514169840 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 --2- 172.21.15.64:0/937963094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8514069100 0x7f85140c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 >> 172.21.15.64:0/937963094 conn(0x7f85140ccc40 msgr2=0x7f85140c59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 shutdown_connections 2023-09-27T23:14:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.518+0000 7f851ad2b700 1 -- 172.21.15.64:0/937963094 wait complete. 2023-09-27T23:14:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:14:52.537 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-09-27T23:14:51.171311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71852,"kb_used_data":5100,"kb_used_omap":0,"kb_used_meta":66688,"kb_avail":754902868,"statfs":{"total":773094113280,"available":773020536832,"internally_reserved":0,"allocated":5222400,"data_stored":2937839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68288512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.978085"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600600+0000","last_change":"2023-09-27T23:14:44.595070+0000","last_active":"2023-09-27T23:14:44.600600+0000","last_peered":"2023-09-27T23:14:44.600600+0000","last_clean":"2023-09-27T23:14:44.600600+0000","last_became_active":"2023-09-27T23:14:44.594806+0000","last_became_peered":"2023-09-27T23:14:44.594806+0000","last_unstale":"2023-09-27T23:14:44.600600+0000","last_undegraded":"2023-09-27T23:14:44.600600+0000","last_fullsized":"2023-09-27T23:14:44.600600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:20.747262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.587238+0000","last_change":"2023-09-27T23:14:44.587238+0000","last_active":"2023-09-27T23:14:44.587238+0000","last_peered":"2023-09-27T23:14:44.587238+0000","last_clean":"2023-09-27T23:14:44.587238+0000","last_became_active":"2023-09-27T23:14:44.586881+0000","last_became_peered":"2023-09-27T23:14:44.586881+0000","last_unstale":"2023-09-27T23:14:44.587238+0000","last_undegraded":"2023-09-27T23:14:44.587238+0000","last_fullsized":"2023-09-27T23:14:44.587238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:04.591094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604137+0000","last_change":"2023-09-27T23:14:44.604137+0000","last_active":"2023-09-27T23:14:44.604137+0000","last_peered":"2023-09-27T23:14:44.604137+0000","last_clean":"2023-09-27T23:14:44.604137+0000","last_became_active":"2023-09-27T23:14:44.603806+0000","last_became_peered":"2023-09-27T23:14:44.603806+0000","last_unstale":"2023-09-27T23:14:44.604137+0000","last_undegraded":"2023-09-27T23:14:44.604137+0000","last_fullsized":"2023-09-27T23:14:44.604137+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:21:24.044809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.610950+0000","last_change":"2023-09-27T23:14:44.605182+0000","last_active":"2023-09-27T23:14:44.610950+0000","last_peered":"2023-09-27T23:14:44.610950+0000","last_clean":"2023-09-27T23:14:44.610950+0000","last_became_active":"2023-09-27T23:14:44.604913+0000","last_became_peered":"2023-09-27T23:14:44.604913+0000","last_unstale":"2023-09-27T23:14:44.610950+0000","last_undegraded":"2023-09-27T23:14:44.610950+0000","last_fullsized":"2023-09-27T23:14:44.610950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:15.834897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.586422+0000","last_change":"2023-09-27T23:14:44.586422+0000","last_active":"2023-09-27T23:14:44.586422+0000","last_peered":"2023-09-27T23:14:44.586422+0000","last_clean":"2023-09-27T23:14:44.586422+0000","last_became_active":"2023-09-27T23:14:44.585049+0000","last_became_peered":"2023-09-27T23:14:44.585049+0000","last_unstale":"2023-09-27T23:14:44.586422+0000","last_undegraded":"2023-09-27T23:14:44.586422+0000","last_fullsized":"2023-09-27T23:14:44.586422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:21:46.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596443+0000","last_change":"2023-09-27T23:14:44.590243+0000","last_active":"2023-09-27T23:14:44.596443+0000","last_peered":"2023-09-27T23:14:44.596443+0000","last_clean":"2023-09-27T23:14:44.596443+0000","last_became_active":"2023-09-27T23:14:44.590010+0000","last_became_peered":"2023-09-27T23:14:44.590010+0000","last_unstale":"2023-09-27T23:14:44.596443+0000","last_undegraded":"2023-09-27T23:14:44.596443+0000","last_fullsized":"2023-09-27T23:14:44.596443+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:20:59.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600560+0000","last_change":"2023-09-27T23:14:44.592360+0000","last_active":"2023-09-27T23:14:44.600560+0000","last_peered":"2023-09-27T23:14:44.600560+0000","last_clean":"2023-09-27T23:14:44.600560+0000","last_became_active":"2023-09-27T23:14:44.591998+0000","last_became_peered":"2023-09-27T23:14:44.591998+0000","last_unstale":"2023-09-27T23:14:44.600560+0000","last_undegraded":"2023-09-27T23:14:44.600560+0000","last_fullsized":"2023-09-27T23:14:44.600560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:31.108718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.611002+0000","last_change":"2023-09-27T23:14:44.605796+0000","last_active":"2023-09-27T23:14:44.611002+0000","last_peered":"2023-09-27T23:14:44.611002+0000","last_clean":"2023-09-27T23:14:44.611002+0000","last_became_active":"2023-09-27T23:14:44.605344+0000","last_became_peered":"2023-09-27T23:14:44.605344+0000","last_unstale":"2023-09-27T23:14:44.611002+0000","last_undegraded":"2023-09-27T23:14:44.611002+0000","last_fullsized":"2023-09-27T23:14:44.611002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:22.030239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601705+0000","last_change":"2023-09-27T23:14:44.598911+0000","last_active":"2023-09-27T23:14:44.601705+0000","last_peered":"2023-09-27T23:14:44.601705+0000","last_clean":"2023-09-27T23:14:44.601705+0000","last_became_active":"2023-09-27T23:14:44.593390+0000","last_became_peered":"2023-09-27T23:14:44.593390+0000","last_unstale":"2023-09-27T23:14:44.601705+0000","last_undegraded":"2023-09-27T23:14:44.601705+0000","last_fullsized":"2023-09-27T23:14:44.601705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:21.392076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.611028+0000","last_change":"2023-09-27T23:14:44.605574+0000","last_active":"2023-09-27T23:14:44.611028+0000","last_peered":"2023-09-27T23:14:44.611028+0000","last_clean":"2023-09-27T23:14:44.611028+0000","last_became_active":"2023-09-27T23:14:44.605305+0000","last_became_peered":"2023-09-27T23:14:44.605305+0000","last_unstale":"2023-09-27T23:14:44.611028+0000","last_undegraded":"2023-09-27T23:14:44.611028+0000","last_fullsized":"2023-09-27T23:14:44.611028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:17:27.750005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601729+0000","last_change":"2023-09-27T23:14:44.594195+0000","last_active":"2023-09-27T23:14:44.601729+0000","last_peered":"2023-09-27T23:14:44.601729+0000","last_clean":"2023-09-27T23:14:44.601729+0000","last_became_active":"2023-09-27T23:14:44.593943+0000","last_became_peered":"2023-09-27T23:14:44.593943+0000","last_unstale":"2023-09-27T23:14:44.601729+0000","last_undegraded":"2023-09-27T23:14:44.601729+0000","last_fullsized":"2023-09-27T23:14:44.601729+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:05:41.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.595452+0000","last_change":"2023-09-27T23:14:44.595452+0000","last_active":"2023-09-27T23:14:44.595452+0000","last_peered":"2023-09-27T23:14:44.595452+0000","last_clean":"2023-09-27T23:14:44.595452+0000","last_became_active":"2023-09-27T23:14:44.594828+0000","last_became_peered":"2023-09-27T23:14:44.594828+0000","last_unstale":"2023-09-27T23:14:44.595452+0000","last_undegraded":"2023-09-27T23:14:44.595452+0000","last_fullsized":"2023-09-27T23:14:44.595452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:20.747262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604058+0000","last_change":"2023-09-27T23:14:44.585114+0000","last_active":"2023-09-27T23:14:44.604058+0000","last_peered":"2023-09-27T23:14:44.604058+0000","last_clean":"2023-09-27T23:14:44.604058+0000","last_became_active":"2023-09-27T23:14:44.584800+0000","last_became_peered":"2023-09-27T23:14:44.584800+0000","last_unstale":"2023-09-27T23:14:44.604058+0000","last_undegraded":"2023-09-27T23:14:44.604058+0000","last_fullsized":"2023-09-27T23:14:44.604058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:32.813292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601855+0000","last_change":"2023-09-27T23:14:44.593508+0000","last_active":"2023-09-27T23:14:44.601855+0000","last_peered":"2023-09-27T23:14:44.601855+0000","last_clean":"2023-09-27T23:14:44.601855+0000","last_became_active":"2023-09-27T23:14:44.593188+0000","last_became_peered":"2023-09-27T23:14:44.593188+0000","last_unstale":"2023-09-27T23:14:44.601855+0000","last_undegraded":"2023-09-27T23:14:44.601855+0000","last_fullsized":"2023-09-27T23:14:44.601855+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:31.108718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596482+0000","last_change":"2023-09-27T23:14:44.590955+0000","last_active":"2023-09-27T23:14:44.596482+0000","last_peered":"2023-09-27T23:14:44.596482+0000","last_clean":"2023-09-27T23:14:44.596482+0000","last_became_active":"2023-09-27T23:14:44.590541+0000","last_became_peered":"2023-09-27T23:14:44.590541+0000","last_unstale":"2023-09-27T23:14:44.596482+0000","last_undegraded":"2023-09-27T23:14:44.596482+0000","last_fullsized":"2023-09-27T23:14:44.596482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:08:10.960550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.597571+0000","last_change":"2023-09-27T23:14:44.597571+0000","last_active":"2023-09-27T23:14:44.597571+0000","last_peered":"2023-09-27T23:14:44.597571+0000","last_clean":"2023-09-27T23:14:44.597571+0000","last_became_active":"2023-09-27T23:14:44.597212+0000","last_became_peered":"2023-09-27T23:14:44.597212+0000","last_unstale":"2023-09-27T23:14:44.597571+0000","last_undegraded":"2023-09-27T23:14:44.597571+0000","last_fullsized":"2023-09-27T23:14:44.597571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:58:50.283087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.594236+0000","last_change":"2023-09-27T23:14:44.594236+0000","last_active":"2023-09-27T23:14:44.594236+0000","last_peered":"2023-09-27T23:14:44.594236+0000","last_clean":"2023-09-27T23:14:44.594236+0000","last_became_active":"2023-09-27T23:14:44.592063+0000","last_became_peered":"2023-09-27T23:14:44.592063+0000","last_unstale":"2023-09-27T23:14:44.594236+0000","last_undegraded":"2023-09-27T23:14:44.594236+0000","last_fullsized":"2023-09-27T23:14:44.594236+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:22.030239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.587158+0000","last_change":"2023-09-27T23:14:44.587158+0000","last_active":"2023-09-27T23:14:44.587158+0000","last_peered":"2023-09-27T23:14:44.587158+0000","last_clean":"2023-09-27T23:14:44.587158+0000","last_became_active":"2023-09-27T23:14:44.586849+0000","last_became_peered":"2023-09-27T23:14:44.586849+0000","last_unstale":"2023-09-27T23:14:44.587158+0000","last_undegraded":"2023-09-27T23:14:44.587158+0000","last_fullsized":"2023-09-27T23:14:44.587158+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:03.359592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596001+0000","last_change":"2023-09-27T23:14:44.596001+0000","last_active":"2023-09-27T23:14:44.596001+0000","last_peered":"2023-09-27T23:14:44.596001+0000","last_clean":"2023-09-27T23:14:44.596001+0000","last_became_active":"2023-09-27T23:14:44.593443+0000","last_became_peered":"2023-09-27T23:14:44.593443+0000","last_unstale":"2023-09-27T23:14:44.596001+0000","last_undegraded":"2023-09-27T23:14:44.596001+0000","last_fullsized":"2023-09-27T23:14:44.596001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:27.397171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605484+0000","last_change":"2023-09-27T23:14:44.591703+0000","last_active":"2023-09-27T23:14:44.605484+0000","last_peered":"2023-09-27T23:14:44.605484+0000","last_clean":"2023-09-27T23:14:44.605484+0000","last_became_active":"2023-09-27T23:14:44.591432+0000","last_became_peered":"2023-09-27T23:14:44.591432+0000","last_unstale":"2023-09-27T23:14:44.605484+0000","last_undegraded":"2023-09-27T23:14:44.605484+0000","last_fullsized":"2023-09-27T23:14:44.605484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:05:55.604974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.591157+0000","last_change":"2023-09-27T23:14:44.586619+0000","last_active":"2023-09-27T23:14:44.591157+0000","last_peered":"2023-09-27T23:14:44.591157+0000","last_clean":"2023-09-27T23:14:44.591157+0000","last_became_active":"2023-09-27T23:14:44.586339+0000","last_became_peered":"2023-09-27T23:14:44.586339+0000","last_unstale":"2023-09-27T23:14:44.591157+0000","last_undegraded":"2023-09-27T23:14:44.591157+0000","last_fullsized":"2023-09-27T23:14:44.591157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:03.359592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601838+0000","last_change":"2023-09-27T23:14:44.592419+0000","last_active":"2023-09-27T23:14:44.601838+0000","last_peered":"2023-09-27T23:14:44.601838+0000","last_clean":"2023-09-27T23:14:44.601838+0000","last_became_active":"2023-09-27T23:14:44.591971+0000","last_became_peered":"2023-09-27T23:14:44.591971+0000","last_unstale":"2023-09-27T23:14:44.601838+0000","last_undegraded":"2023-09-27T23:14:44.601838+0000","last_fullsized":"2023-09-27T23:14:44.601838+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:13:09.738456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589145+0000","last_change":"2023-09-27T23:14:44.589145+0000","last_active":"2023-09-27T23:14:44.589145+0000","last_peered":"2023-09-27T23:14:44.589145+0000","last_clean":"2023-09-27T23:14:44.589145+0000","last_became_active":"2023-09-27T23:14:44.588539+0000","last_became_peered":"2023-09-27T23:14:44.588539+0000","last_unstale":"2023-09-27T23:14:44.589145+0000","last_undegraded":"2023-09-27T23:14:44.589145+0000","last_fullsized":"2023-09-27T23:14:44.589145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:51:20.939784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.590359+0000","last_change":"2023-09-27T23:14:44.590359+0000","last_active":"2023-09-27T23:14:44.590359+0000","last_peered":"2023-09-27T23:14:44.590359+0000","last_clean":"2023-09-27T23:14:44.590359+0000","last_became_active":"2023-09-27T23:14:44.589960+0000","last_became_peered":"2023-09-27T23:14:44.589960+0000","last_unstale":"2023-09-27T23:14:44.590359+0000","last_undegraded":"2023-09-27T23:14:44.590359+0000","last_fullsized":"2023-09-27T23:14:44.590359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:12:34.470004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596846+0000","last_change":"2023-09-27T23:14:44.596846+0000","last_active":"2023-09-27T23:14:44.596846+0000","last_peered":"2023-09-27T23:14:44.596846+0000","last_clean":"2023-09-27T23:14:44.596846+0000","last_became_active":"2023-09-27T23:14:44.596535+0000","last_became_peered":"2023-09-27T23:14:44.596535+0000","last_unstale":"2023-09-27T23:14:44.596846+0000","last_undegraded":"2023-09-27T23:14:44.596846+0000","last_fullsized":"2023-09-27T23:14:44.596846+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:52:06.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604052+0000","last_change":"2023-09-27T23:14:44.585552+0000","last_active":"2023-09-27T23:14:44.604052+0000","last_peered":"2023-09-27T23:14:44.604052+0000","last_clean":"2023-09-27T23:14:44.604052+0000","last_became_active":"2023-09-27T23:14:44.585234+0000","last_became_peered":"2023-09-27T23:14:44.585234+0000","last_unstale":"2023-09-27T23:14:44.604052+0000","last_undegraded":"2023-09-27T23:14:44.604052+0000","last_fullsized":"2023-09-27T23:14:44.604052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:21:55.504453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.588040+0000","last_change":"2023-09-27T23:14:44.588040+0000","last_active":"2023-09-27T23:14:44.588040+0000","last_peered":"2023-09-27T23:14:44.588040+0000","last_clean":"2023-09-27T23:14:44.588040+0000","last_became_active":"2023-09-27T23:14:44.587745+0000","last_became_peered":"2023-09-27T23:14:44.587745+0000","last_unstale":"2023-09-27T23:14:44.588040+0000","last_undegraded":"2023-09-27T23:14:44.588040+0000","last_fullsized":"2023-09-27T23:14:44.588040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:31:38.435688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.595633+0000","last_change":"2023-09-27T23:14:44.595633+0000","last_active":"2023-09-27T23:14:44.595633+0000","last_peered":"2023-09-27T23:14:44.595633+0000","last_clean":"2023-09-27T23:14:44.595633+0000","last_became_active":"2023-09-27T23:14:44.595181+0000","last_became_peered":"2023-09-27T23:14:44.595181+0000","last_unstale":"2023-09-27T23:14:44.595633+0000","last_undegraded":"2023-09-27T23:14:44.595633+0000","last_fullsized":"2023-09-27T23:14:44.595633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:21:20.987178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596349+0000","last_change":"2023-09-27T23:14:44.596349+0000","last_active":"2023-09-27T23:14:44.596349+0000","last_peered":"2023-09-27T23:14:44.596349+0000","last_clean":"2023-09-27T23:14:44.596349+0000","last_became_active":"2023-09-27T23:14:44.593239+0000","last_became_peered":"2023-09-27T23:14:44.593239+0000","last_unstale":"2023-09-27T23:14:44.596349+0000","last_undegraded":"2023-09-27T23:14:44.596349+0000","last_fullsized":"2023-09-27T23:14:44.596349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:04:49.511322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.590868+0000","last_change":"2023-09-27T23:14:44.590868+0000","last_active":"2023-09-27T23:14:44.590868+0000","last_peered":"2023-09-27T23:14:44.590868+0000","last_clean":"2023-09-27T23:14:44.590868+0000","last_became_active":"2023-09-27T23:14:44.587882+0000","last_became_peered":"2023-09-27T23:14:44.587882+0000","last_unstale":"2023-09-27T23:14:44.590868+0000","last_undegraded":"2023-09-27T23:14:44.590868+0000","last_fullsized":"2023-09-27T23:14:44.590868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:08:10.960550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.591228+0000","last_change":"2023-09-27T23:14:44.583613+0000","last_active":"2023-09-27T23:14:44.591228+0000","last_peered":"2023-09-27T23:14:44.591228+0000","last_clean":"2023-09-27T23:14:44.591228+0000","last_became_active":"2023-09-27T23:14:44.583065+0000","last_became_peered":"2023-09-27T23:14:44.583065+0000","last_unstale":"2023-09-27T23:14:44.591228+0000","last_undegraded":"2023-09-27T23:14:44.591228+0000","last_fullsized":"2023-09-27T23:14:44.591228+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:17:27.750005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604266+0000","last_change":"2023-09-27T23:14:44.592678+0000","last_active":"2023-09-27T23:14:44.604266+0000","last_peered":"2023-09-27T23:14:44.604266+0000","last_clean":"2023-09-27T23:14:44.604266+0000","last_became_active":"2023-09-27T23:14:44.592380+0000","last_became_peered":"2023-09-27T23:14:44.592380+0000","last_unstale":"2023-09-27T23:14:44.604266+0000","last_undegraded":"2023-09-27T23:14:44.604266+0000","last_fullsized":"2023-09-27T23:14:44.604266+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:25.182304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601442+0000","last_change":"2023-09-27T23:14:44.596994+0000","last_active":"2023-09-27T23:14:44.601442+0000","last_peered":"2023-09-27T23:14:44.601442+0000","last_clean":"2023-09-27T23:14:44.601442+0000","last_became_active":"2023-09-27T23:14:44.596700+0000","last_became_peered":"2023-09-27T23:14:44.596700+0000","last_unstale":"2023-09-27T23:14:44.601442+0000","last_undegraded":"2023-09-27T23:14:44.601442+0000","last_fullsized":"2023-09-27T23:14:44.601442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:23.475703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.606935+0000","last_change":"2023-09-27T23:14:44.224159+0000","last_active":"2023-09-27T23:14:44.606935+0000","last_peered":"2023-09-27T23:14:44.606935+0000","last_clean":"2023-09-27T23:14:44.606935+0000","last_became_active":"2023-09-27T23:14:44.223820+0000","last_became_peered":"2023-09-27T23:14:44.223820+0000","last_unstale":"2023-09-27T23:14:44.606935+0000","last_undegraded":"2023-09-27T23:14:44.606935+0000","last_fullsized":"2023-09-27T23:14:44.606935+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:20:31.872309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.587031+0000","last_change":"2023-09-27T23:14:44.587031+0000","last_active":"2023-09-27T23:14:44.587031+0000","last_peered":"2023-09-27T23:14:44.587031+0000","last_clean":"2023-09-27T23:14:44.587031+0000","last_became_active":"2023-09-27T23:14:44.586742+0000","last_became_peered":"2023-09-27T23:14:44.586742+0000","last_unstale":"2023-09-27T23:14:44.587031+0000","last_undegraded":"2023-09-27T23:14:44.587031+0000","last_fullsized":"2023-09-27T23:14:44.587031+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:18.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.582455+0000","last_change":"2023-09-27T23:14:44.220394+0000","last_active":"2023-09-27T23:14:44.582455+0000","last_peered":"2023-09-27T23:14:44.582455+0000","last_clean":"2023-09-27T23:14:44.582455+0000","last_became_active":"2023-09-27T23:14:44.220070+0000","last_became_peered":"2023-09-27T23:14:44.220070+0000","last_unstale":"2023-09-27T23:14:44.582455+0000","last_undegraded":"2023-09-27T23:14:44.582455+0000","last_fullsized":"2023-09-27T23:14:44.582455+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:03.580454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.588865+0000","last_change":"2023-09-27T23:14:44.588865+0000","last_active":"2023-09-27T23:14:44.588865+0000","last_peered":"2023-09-27T23:14:44.588865+0000","last_clean":"2023-09-27T23:14:44.588865+0000","last_became_active":"2023-09-27T23:14:44.588585+0000","last_became_peered":"2023-09-27T23:14:44.588585+0000","last_unstale":"2023-09-27T23:14:44.588865+0000","last_undegraded":"2023-09-27T23:14:44.588865+0000","last_fullsized":"2023-09-27T23:14:44.588865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:31.108718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609839+0000","last_change":"2023-09-27T23:14:44.217297+0000","last_active":"2023-09-27T23:14:44.609839+0000","last_peered":"2023-09-27T23:14:44.609839+0000","last_clean":"2023-09-27T23:14:44.609839+0000","last_became_active":"2023-09-27T23:14:44.216935+0000","last_became_peered":"2023-09-27T23:14:44.216935+0000","last_unstale":"2023-09-27T23:14:44.609839+0000","last_undegraded":"2023-09-27T23:14:44.609839+0000","last_fullsized":"2023-09-27T23:14:44.609839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:34.357991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600805+0000","last_change":"2023-09-27T23:14:44.594912+0000","last_active":"2023-09-27T23:14:44.600805+0000","last_peered":"2023-09-27T23:14:44.600805+0000","last_clean":"2023-09-27T23:14:44.600805+0000","last_became_active":"2023-09-27T23:14:44.594632+0000","last_became_peered":"2023-09-27T23:14:44.594632+0000","last_unstale":"2023-09-27T23:14:44.600805+0000","last_undegraded":"2023-09-27T23:14:44.600805+0000","last_fullsized":"2023-09-27T23:14:44.600805+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:00.409802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609314+0000","last_change":"2023-09-27T23:14:44.212557+0000","last_active":"2023-09-27T23:14:44.609314+0000","last_peered":"2023-09-27T23:14:44.609314+0000","last_clean":"2023-09-27T23:14:44.609314+0000","last_became_active":"2023-09-27T23:14:44.212250+0000","last_became_peered":"2023-09-27T23:14:44.212250+0000","last_unstale":"2023-09-27T23:14:44.609314+0000","last_undegraded":"2023-09-27T23:14:44.609314+0000","last_fullsized":"2023-09-27T23:14:44.609314+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:24.171663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.586373+0000","last_change":"2023-09-27T23:14:44.586373+0000","last_active":"2023-09-27T23:14:44.586373+0000","last_peered":"2023-09-27T23:14:44.586373+0000","last_clean":"2023-09-27T23:14:44.586373+0000","last_became_active":"2023-09-27T23:14:44.585975+0000","last_became_peered":"2023-09-27T23:14:44.585975+0000","last_unstale":"2023-09-27T23:14:44.586373+0000","last_undegraded":"2023-09-27T23:14:44.586373+0000","last_fullsized":"2023-09-27T23:14:44.586373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:06:14.444225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605299+0000","last_change":"2023-09-27T23:14:44.216444+0000","last_active":"2023-09-27T23:14:44.605299+0000","last_peered":"2023-09-27T23:14:44.605299+0000","last_clean":"2023-09-27T23:14:44.605299+0000","last_became_active":"2023-09-27T23:14:44.216141+0000","last_became_peered":"2023-09-27T23:14:44.216141+0000","last_unstale":"2023-09-27T23:14:44.605299+0000","last_undegraded":"2023-09-27T23:14:44.605299+0000","last_fullsized":"2023-09-27T23:14:44.605299+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:56:53.180933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.611066+0000","last_change":"2023-09-27T23:14:44.606155+0000","last_active":"2023-09-27T23:14:44.611066+0000","last_peered":"2023-09-27T23:14:44.611066+0000","last_clean":"2023-09-27T23:14:44.611066+0000","last_became_active":"2023-09-27T23:14:44.605874+0000","last_became_peered":"2023-09-27T23:14:44.605874+0000","last_unstale":"2023-09-27T23:14:44.611066+0000","last_undegraded":"2023-09-27T23:14:44.611066+0000","last_fullsized":"2023-09-27T23:14:44.611066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:57:04.488573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.582780+0000","last_change":"2023-09-27T23:14:44.212483+0000","last_active":"2023-09-27T23:14:44.582780+0000","last_peered":"2023-09-27T23:14:44.582780+0000","last_clean":"2023-09-27T23:14:44.582780+0000","last_became_active":"2023-09-27T23:14:44.212177+0000","last_became_peered":"2023-09-27T23:14:44.212177+0000","last_unstale":"2023-09-27T23:14:44.582780+0000","last_undegraded":"2023-09-27T23:14:44.582780+0000","last_fullsized":"2023-09-27T23:14:44.582780+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:21:19.976537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.610983+0000","last_change":"2023-09-27T23:14:44.605608+0000","last_active":"2023-09-27T23:14:44.610983+0000","last_peered":"2023-09-27T23:14:44.610983+0000","last_clean":"2023-09-27T23:14:44.610983+0000","last_became_active":"2023-09-27T23:14:44.605009+0000","last_became_peered":"2023-09-27T23:14:44.605009+0000","last_unstale":"2023-09-27T23:14:44.610983+0000","last_undegraded":"2023-09-27T23:14:44.610983+0000","last_fullsized":"2023-09-27T23:14:44.610983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:21:24.044809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.211955+0000","last_change":"2023-09-27T23:14:44.210329+0000","last_active":"2023-09-27T23:14:44.211955+0000","last_peered":"2023-09-27T23:14:44.211955+0000","last_clean":"2023-09-27T23:14:44.211955+0000","last_became_active":"2023-09-27T23:14:44.210063+0000","last_became_peered":"2023-09-27T23:14:44.210063+0000","last_unstale":"2023-09-27T23:14:44.211955+0000","last_undegraded":"2023-09-27T23:14:44.211955+0000","last_fullsized":"2023-09-27T23:14:44.211955+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:47.755120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589317+0000","last_change":"2023-09-27T23:14:44.589317+0000","last_active":"2023-09-27T23:14:44.589317+0000","last_peered":"2023-09-27T23:14:44.589317+0000","last_clean":"2023-09-27T23:14:44.589317+0000","last_became_active":"2023-09-27T23:14:44.588877+0000","last_became_peered":"2023-09-27T23:14:44.588877+0000","last_unstale":"2023-09-27T23:14:44.589317+0000","last_undegraded":"2023-09-27T23:14:44.589317+0000","last_fullsized":"2023-09-27T23:14:44.589317+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:25.078836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.585446+0000","last_change":"2023-09-27T23:14:44.223100+0000","last_active":"2023-09-27T23:14:44.585446+0000","last_peered":"2023-09-27T23:14:44.585446+0000","last_clean":"2023-09-27T23:14:44.585446+0000","last_became_active":"2023-09-27T23:14:44.222538+0000","last_became_peered":"2023-09-27T23:14:44.222538+0000","last_unstale":"2023-09-27T23:14:44.585446+0000","last_undegraded":"2023-09-27T23:14:44.585446+0000","last_fullsized":"2023-09-27T23:14:44.585446+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:26.386530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.587195+0000","last_change":"2023-09-27T23:14:44.587195+0000","last_active":"2023-09-27T23:14:44.587195+0000","last_peered":"2023-09-27T23:14:44.587195+0000","last_clean":"2023-09-27T23:14:44.587195+0000","last_became_active":"2023-09-27T23:14:44.586844+0000","last_became_peered":"2023-09-27T23:14:44.586844+0000","last_unstale":"2023-09-27T23:14:44.587195+0000","last_undegraded":"2023-09-27T23:14:44.587195+0000","last_fullsized":"2023-09-27T23:14:44.587195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:03.025897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.218562+0000","last_change":"2023-09-27T23:14:44.214482+0000","last_active":"2023-09-27T23:14:44.218562+0000","last_peered":"2023-09-27T23:14:44.218562+0000","last_clean":"2023-09-27T23:14:44.218562+0000","last_became_active":"2023-09-27T23:14:44.214104+0000","last_became_peered":"2023-09-27T23:14:44.214104+0000","last_unstale":"2023-09-27T23:14:44.218562+0000","last_undegraded":"2023-09-27T23:14:44.218562+0000","last_fullsized":"2023-09-27T23:14:44.218562+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:57:23.081247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603151+0000","last_change":"2023-09-27T23:14:44.597237+0000","last_active":"2023-09-27T23:14:44.603151+0000","last_peered":"2023-09-27T23:14:44.603151+0000","last_clean":"2023-09-27T23:14:44.603151+0000","last_became_active":"2023-09-27T23:14:44.596866+0000","last_became_peered":"2023-09-27T23:14:44.596866+0000","last_unstale":"2023-09-27T23:14:44.603151+0000","last_undegraded":"2023-09-27T23:14:44.603151+0000","last_fullsized":"2023-09-27T23:14:44.603151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:21:46.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.587579+0000","last_change":"2023-09-27T23:14:44.216052+0000","last_active":"2023-09-27T23:14:44.587579+0000","last_peered":"2023-09-27T23:14:44.587579+0000","last_clean":"2023-09-27T23:14:44.587579+0000","last_became_active":"2023-09-27T23:14:44.213827+0000","last_became_peered":"2023-09-27T23:14:44.213827+0000","last_unstale":"2023-09-27T23:14:44.587579+0000","last_undegraded":"2023-09-27T23:14:44.587579+0000","last_fullsized":"2023-09-27T23:14:44.587579+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:17.911634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.610949+0000","last_change":"2023-09-27T23:14:44.607261+0000","last_active":"2023-09-27T23:14:44.610949+0000","last_peered":"2023-09-27T23:14:44.610949+0000","last_clean":"2023-09-27T23:14:44.610949+0000","last_became_active":"2023-09-27T23:14:44.606953+0000","last_became_peered":"2023-09-27T23:14:44.606953+0000","last_unstale":"2023-09-27T23:14:44.610949+0000","last_undegraded":"2023-09-27T23:14:44.610949+0000","last_fullsized":"2023-09-27T23:14:44.610949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:22.030239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.606338+0000","last_change":"2023-09-27T23:14:44.222752+0000","last_active":"2023-09-27T23:14:44.606338+0000","last_peered":"2023-09-27T23:14:44.606338+0000","last_clean":"2023-09-27T23:14:44.606338+0000","last_became_active":"2023-09-27T23:14:44.222414+0000","last_became_peered":"2023-09-27T23:14:44.222414+0000","last_unstale":"2023-09-27T23:14:44.606338+0000","last_undegraded":"2023-09-27T23:14:44.606338+0000","last_fullsized":"2023-09-27T23:14:44.606338+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:05:54.594333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.600615+0000","last_change":"2023-09-27T23:14:44.593177+0000","last_active":"2023-09-27T23:14:44.600615+0000","last_peered":"2023-09-27T23:14:44.600615+0000","last_clean":"2023-09-27T23:14:44.600615+0000","last_became_active":"2023-09-27T23:14:44.592883+0000","last_became_peered":"2023-09-27T23:14:44.592883+0000","last_unstale":"2023-09-27T23:14:44.600615+0000","last_undegraded":"2023-09-27T23:14:44.600615+0000","last_fullsized":"2023-09-27T23:14:44.600615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:04.591094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.612265+0000","last_change":"2023-09-27T23:14:44.213685+0000","last_active":"2023-09-27T23:14:44.612265+0000","last_peered":"2023-09-27T23:14:44.612265+0000","last_clean":"2023-09-27T23:14:44.612265+0000","last_became_active":"2023-09-27T23:14:44.213204+0000","last_became_peered":"2023-09-27T23:14:44.213204+0000","last_unstale":"2023-09-27T23:14:44.612265+0000","last_undegraded":"2023-09-27T23:14:44.612265+0000","last_fullsized":"2023-09-27T23:14:44.612265+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:51:19.929143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.593595+0000","last_change":"2023-09-27T23:14:44.593595+0000","last_active":"2023-09-27T23:14:44.593595+0000","last_peered":"2023-09-27T23:14:44.593595+0000","last_clean":"2023-09-27T23:14:44.593595+0000","last_became_active":"2023-09-27T23:14:44.592952+0000","last_became_peered":"2023-09-27T23:14:44.592952+0000","last_unstale":"2023-09-27T23:14:44.593595+0000","last_undegraded":"2023-09-27T23:14:44.593595+0000","last_fullsized":"2023-09-27T23:14:44.593595+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:34:56.101733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605652+0000","last_change":"2023-09-27T23:14:44.222945+0000","last_active":"2023-09-27T23:14:44.605652+0000","last_peered":"2023-09-27T23:14:44.605652+0000","last_clean":"2023-09-27T23:14:44.605652+0000","last_became_active":"2023-09-27T23:14:44.222674+0000","last_became_peered":"2023-09-27T23:14:44.222674+0000","last_unstale":"2023-09-27T23:14:44.605652+0000","last_undegraded":"2023-09-27T23:14:44.605652+0000","last_fullsized":"2023-09-27T23:14:44.605652+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:21:45.511051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.592524+0000","last_change":"2023-09-27T23:14:44.590124+0000","last_active":"2023-09-27T23:14:44.592524+0000","last_peered":"2023-09-27T23:14:44.592524+0000","last_clean":"2023-09-27T23:14:44.592524+0000","last_became_active":"2023-09-27T23:14:44.589868+0000","last_became_peered":"2023-09-27T23:14:44.589868+0000","last_unstale":"2023-09-27T23:14:44.592524+0000","last_undegraded":"2023-09-27T23:14:44.592524+0000","last_fullsized":"2023-09-27T23:14:44.592524+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:57:24.091888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.584235+0000","last_change":"2023-09-27T23:14:44.214013+0000","last_active":"2023-09-27T23:14:44.584235+0000","last_peered":"2023-09-27T23:14:44.584235+0000","last_clean":"2023-09-27T23:14:44.584235+0000","last_became_active":"2023-09-27T23:14:44.213729+0000","last_became_peered":"2023-09-27T23:14:44.213729+0000","last_unstale":"2023-09-27T23:14:44.584235+0000","last_undegraded":"2023-09-27T23:14:44.584235+0000","last_fullsized":"2023-09-27T23:14:44.584235+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:31.802652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608205+0000","last_change":"2023-09-27T23:14:44.604712+0000","last_active":"2023-09-27T23:14:44.608205+0000","last_peered":"2023-09-27T23:14:44.608205+0000","last_clean":"2023-09-27T23:14:44.608205+0000","last_became_active":"2023-09-27T23:14:44.604402+0000","last_became_peered":"2023-09-27T23:14:44.604402+0000","last_unstale":"2023-09-27T23:14:44.608205+0000","last_undegraded":"2023-09-27T23:14:44.608205+0000","last_fullsized":"2023-09-27T23:14:44.608205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:21:24.044809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.580627+0000","last_change":"2023-09-27T23:14:44.222891+0000","last_active":"2023-09-27T23:14:44.580627+0000","last_peered":"2023-09-27T23:14:44.580627+0000","last_clean":"2023-09-27T23:14:44.580627+0000","last_became_active":"2023-09-27T23:14:44.222574+0000","last_became_peered":"2023-09-27T23:14:44.222574+0000","last_unstale":"2023-09-27T23:14:44.580627+0000","last_undegraded":"2023-09-27T23:14:44.580627+0000","last_fullsized":"2023-09-27T23:14:44.580627+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:21:19.976537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.595293+0000","last_change":"2023-09-27T23:14:44.589946+0000","last_active":"2023-09-27T23:14:44.595293+0000","last_peered":"2023-09-27T23:14:44.595293+0000","last_clean":"2023-09-27T23:14:44.595293+0000","last_became_active":"2023-09-27T23:14:44.589206+0000","last_became_peered":"2023-09-27T23:14:44.589206+0000","last_unstale":"2023-09-27T23:14:44.595293+0000","last_undegraded":"2023-09-27T23:14:44.595293+0000","last_fullsized":"2023-09-27T23:14:44.595293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:57:24.091888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.612872+0000","last_change":"2023-09-27T23:14:44.222271+0000","last_active":"2023-09-27T23:14:44.612872+0000","last_peered":"2023-09-27T23:14:44.612872+0000","last_clean":"2023-09-27T23:14:44.612872+0000","last_became_active":"2023-09-27T23:14:44.221958+0000","last_became_peered":"2023-09-27T23:14:44.221958+0000","last_unstale":"2023-09-27T23:14:44.612872+0000","last_undegraded":"2023-09-27T23:14:44.612872+0000","last_fullsized":"2023-09-27T23:14:44.612872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:04:28.785126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603875+0000","last_change":"2023-09-27T23:14:44.579511+0000","last_active":"2023-09-27T23:14:44.603875+0000","last_peered":"2023-09-27T23:14:44.603875+0000","last_clean":"2023-09-27T23:14:44.603875+0000","last_became_active":"2023-09-27T23:14:44.579172+0000","last_became_peered":"2023-09-27T23:14:44.579172+0000","last_unstale":"2023-09-27T23:14:44.603875+0000","last_undegraded":"2023-09-27T23:14:44.603875+0000","last_fullsized":"2023-09-27T23:14:44.603875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:48.765761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608309+0000","last_change":"2023-09-27T23:14:44.222653+0000","last_active":"2023-09-27T23:14:44.608309+0000","last_peered":"2023-09-27T23:14:44.608309+0000","last_clean":"2023-09-27T23:14:44.608309+0000","last_became_active":"2023-09-27T23:14:44.222336+0000","last_became_peered":"2023-09-27T23:14:44.222336+0000","last_unstale":"2023-09-27T23:14:44.608309+0000","last_undegraded":"2023-09-27T23:14:44.608309+0000","last_fullsized":"2023-09-27T23:14:44.608309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:45:57.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601684+0000","last_change":"2023-09-27T23:14:44.593587+0000","last_active":"2023-09-27T23:14:44.601684+0000","last_peered":"2023-09-27T23:14:44.601684+0000","last_clean":"2023-09-27T23:14:44.601684+0000","last_became_active":"2023-09-27T23:14:44.592908+0000","last_became_peered":"2023-09-27T23:14:44.592908+0000","last_unstale":"2023-09-27T23:14:44.601684+0000","last_undegraded":"2023-09-27T23:14:44.601684+0000","last_fullsized":"2023-09-27T23:14:44.601684+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:31:38.435688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.603602+0000","last_change":"2023-09-27T23:14:44.210433+0000","last_active":"2023-09-27T23:14:44.603602+0000","last_peered":"2023-09-27T23:14:44.603602+0000","last_clean":"2023-09-27T23:14:44.603602+0000","last_became_active":"2023-09-27T23:14:44.209934+0000","last_became_peered":"2023-09-27T23:14:44.209934+0000","last_unstale":"2023-09-27T23:14:44.603602+0000","last_undegraded":"2023-09-27T23:14:44.603602+0000","last_fullsized":"2023-09-27T23:14:44.603602+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:19.736622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.590538+0000","last_change":"2023-09-27T23:14:44.590538+0000","last_active":"2023-09-27T23:14:44.590538+0000","last_peered":"2023-09-27T23:14:44.590538+0000","last_clean":"2023-09-27T23:14:44.590538+0000","last_became_active":"2023-09-27T23:14:44.590219+0000","last_became_peered":"2023-09-27T23:14:44.590219+0000","last_unstale":"2023-09-27T23:14:44.590538+0000","last_undegraded":"2023-09-27T23:14:44.590538+0000","last_fullsized":"2023-09-27T23:14:44.590538+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:14:24.583331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.573286+0000","last_change":"2023-09-27T23:14:44.217619+0000","last_active":"2023-09-27T23:14:44.573286+0000","last_peered":"2023-09-27T23:14:44.573286+0000","last_clean":"2023-09-27T23:14:44.573286+0000","last_became_active":"2023-09-27T23:14:44.217272+0000","last_became_peered":"2023-09-27T23:14:44.217272+0000","last_unstale":"2023-09-27T23:14:44.573286+0000","last_undegraded":"2023-09-27T23:14:44.573286+0000","last_fullsized":"2023-09-27T23:14:44.573286+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:24.171663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596611+0000","last_change":"2023-09-27T23:14:44.596611+0000","last_active":"2023-09-27T23:14:44.596611+0000","last_peered":"2023-09-27T23:14:44.596611+0000","last_clean":"2023-09-27T23:14:44.596611+0000","last_became_active":"2023-09-27T23:14:44.594054+0000","last_became_peered":"2023-09-27T23:14:44.594054+0000","last_unstale":"2023-09-27T23:14:44.596611+0000","last_undegraded":"2023-09-27T23:14:44.596611+0000","last_fullsized":"2023-09-27T23:14:44.596611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:36:25.182304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.578305+0000","last_change":"2023-09-27T23:14:44.221512+0000","last_active":"2023-09-27T23:14:44.578305+0000","last_peered":"2023-09-27T23:14:44.578305+0000","last_clean":"2023-09-27T23:14:44.578305+0000","last_became_active":"2023-09-27T23:14:44.221209+0000","last_became_peered":"2023-09-27T23:14:44.221209+0000","last_unstale":"2023-09-27T23:14:44.578305+0000","last_undegraded":"2023-09-27T23:14:44.578305+0000","last_fullsized":"2023-09-27T23:14:44.578305+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:19.736622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608227+0000","last_change":"2023-09-27T23:14:44.604881+0000","last_active":"2023-09-27T23:14:44.608227+0000","last_peered":"2023-09-27T23:14:44.608227+0000","last_clean":"2023-09-27T23:14:44.608227+0000","last_became_active":"2023-09-27T23:14:44.604519+0000","last_became_peered":"2023-09-27T23:14:44.604519+0000","last_unstale":"2023-09-27T23:14:44.608227+0000","last_undegraded":"2023-09-27T23:14:44.608227+0000","last_fullsized":"2023-09-27T23:14:44.608227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:03.025897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.605408+0000","last_change":"2023-09-27T23:14:44.216160+0000","last_active":"2023-09-27T23:14:44.605408+0000","last_peered":"2023-09-27T23:14:44.605408+0000","last_clean":"2023-09-27T23:14:44.605408+0000","last_became_active":"2023-09-27T23:14:44.215860+0000","last_became_peered":"2023-09-27T23:14:44.215860+0000","last_unstale":"2023-09-27T23:14:44.605408+0000","last_undegraded":"2023-09-27T23:14:44.605408+0000","last_fullsized":"2023-09-27T23:14:44.605408+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:19:52.964441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.602876+0000","last_change":"2023-09-27T23:14:44.600767+0000","last_active":"2023-09-27T23:14:44.602876+0000","last_peered":"2023-09-27T23:14:44.602876+0000","last_clean":"2023-09-27T23:14:44.602876+0000","last_became_active":"2023-09-27T23:14:44.600469+0000","last_became_peered":"2023-09-27T23:14:44.600469+0000","last_unstale":"2023-09-27T23:14:44.602876+0000","last_undegraded":"2023-09-27T23:14:44.602876+0000","last_fullsized":"2023-09-27T23:14:44.602876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:14.250239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.223490+0000","last_change":"2023-09-27T23:14:44.220507+0000","last_active":"2023-09-27T23:14:44.223490+0000","last_peered":"2023-09-27T23:14:44.223490+0000","last_clean":"2023-09-27T23:14:44.223490+0000","last_became_active":"2023-09-27T23:14:44.220186+0000","last_became_peered":"2023-09-27T23:14:44.220186+0000","last_unstale":"2023-09-27T23:14:44.223490+0000","last_undegraded":"2023-09-27T23:14:44.223490+0000","last_fullsized":"2023-09-27T23:14:44.223490+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:27:31.802652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.604205+0000","last_change":"2023-09-27T23:14:44.590185+0000","last_active":"2023-09-27T23:14:44.604205+0000","last_peered":"2023-09-27T23:14:44.604205+0000","last_clean":"2023-09-27T23:14:44.604205+0000","last_became_active":"2023-09-27T23:14:44.589892+0000","last_became_peered":"2023-09-27T23:14:44.589892+0000","last_unstale":"2023-09-27T23:14:44.604205+0000","last_undegraded":"2023-09-27T23:14:44.604205+0000","last_fullsized":"2023-09-27T23:14:44.604205+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:27.397171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.585515+0000","last_change":"2023-09-27T23:14:44.213095+0000","last_active":"2023-09-27T23:14:44.585515+0000","last_peered":"2023-09-27T23:14:44.585515+0000","last_clean":"2023-09-27T23:14:44.585515+0000","last_became_active":"2023-09-27T23:14:44.212756+0000","last_became_peered":"2023-09-27T23:14:44.212756+0000","last_unstale":"2023-09-27T23:14:44.585515+0000","last_undegraded":"2023-09-27T23:14:44.585515+0000","last_fullsized":"2023-09-27T23:14:44.585515+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:24.068195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601750+0000","last_change":"2023-09-27T23:14:44.593257+0000","last_active":"2023-09-27T23:14:44.601750+0000","last_peered":"2023-09-27T23:14:44.601750+0000","last_clean":"2023-09-27T23:14:44.601750+0000","last_became_active":"2023-09-27T23:14:44.592935+0000","last_became_peered":"2023-09-27T23:14:44.592935+0000","last_unstale":"2023-09-27T23:14:44.601750+0000","last_undegraded":"2023-09-27T23:14:44.601750+0000","last_fullsized":"2023-09-27T23:14:44.601750+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:20:32.882949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.586809+0000","last_change":"2023-09-27T23:14:44.213872+0000","last_active":"2023-09-27T23:14:44.586809+0000","last_peered":"2023-09-27T23:14:44.586809+0000","last_clean":"2023-09-27T23:14:44.586809+0000","last_became_active":"2023-09-27T23:14:44.213618+0000","last_became_peered":"2023-09-27T23:14:44.213618+0000","last_unstale":"2023-09-27T23:14:44.586809+0000","last_undegraded":"2023-09-27T23:14:44.586809+0000","last_fullsized":"2023-09-27T23:14:44.586809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:47.755120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.599952+0000","last_change":"2023-09-27T23:14:44.593468+0000","last_active":"2023-09-27T23:14:44.599952+0000","last_peered":"2023-09-27T23:14:44.599952+0000","last_clean":"2023-09-27T23:14:44.599952+0000","last_became_active":"2023-09-27T23:14:44.593160+0000","last_became_peered":"2023-09-27T23:14:44.593160+0000","last_unstale":"2023-09-27T23:14:44.599952+0000","last_undegraded":"2023-09-27T23:14:44.599952+0000","last_fullsized":"2023-09-27T23:14:44.599952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:15.834897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.585623+0000","last_change":"2023-09-27T23:14:44.217498+0000","last_active":"2023-09-27T23:14:44.585623+0000","last_peered":"2023-09-27T23:14:44.585623+0000","last_clean":"2023-09-27T23:14:44.585623+0000","last_became_active":"2023-09-27T23:14:44.217152+0000","last_became_peered":"2023-09-27T23:14:44.217152+0000","last_unstale":"2023-09-27T23:14:44.585623+0000","last_undegraded":"2023-09-27T23:14:44.585623+0000","last_fullsized":"2023-09-27T23:14:44.585623+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:26.386530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.217170+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:14:44.217170+0000","last_peered":"2023-09-27T23:14:44.217170+0000","last_clean":"2023-09-27T23:14:44.217170+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:14:44.217170+0000","last_undegraded":"2023-09-27T23:14:44.217170+0000","last_fullsized":"2023-09-27T23:14:44.217170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608242+0000","last_change":"2023-09-27T23:14:44.604964+0000","last_active":"2023-09-27T23:14:44.608242+0000","last_peered":"2023-09-27T23:14:44.608242+0000","last_clean":"2023-09-27T23:14:44.608242+0000","last_became_active":"2023-09-27T23:14:44.604443+0000","last_became_peered":"2023-09-27T23:14:44.604443+0000","last_unstale":"2023-09-27T23:14:44.608242+0000","last_undegraded":"2023-09-27T23:14:44.608242+0000","last_fullsized":"2023-09-27T23:14:44.608242+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:09:58.780457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.218442+0000","last_change":"2023-09-27T23:14:44.214382+0000","last_active":"2023-09-27T23:14:44.218442+0000","last_peered":"2023-09-27T23:14:44.218442+0000","last_clean":"2023-09-27T23:14:44.218442+0000","last_became_active":"2023-09-27T23:14:44.214006+0000","last_became_peered":"2023-09-27T23:14:44.214006+0000","last_unstale":"2023-09-27T23:14:44.218442+0000","last_undegraded":"2023-09-27T23:14:44.218442+0000","last_fullsized":"2023-09-27T23:14:44.218442+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:30.098077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589429+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:14:44.589429+0000","last_peered":"2023-09-27T23:14:44.589429+0000","last_clean":"2023-09-27T23:14:44.589429+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:14:44.589429+0000","last_undegraded":"2023-09-27T23:14:44.589429+0000","last_fullsized":"2023-09-27T23:14:44.589429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.591636+0000","last_change":"2023-09-27T23:14:44.591636+0000","last_active":"2023-09-27T23:14:44.591636+0000","last_peered":"2023-09-27T23:14:44.591636+0000","last_clean":"2023-09-27T23:14:44.591636+0000","last_became_active":"2023-09-27T23:14:44.591360+0000","last_became_peered":"2023-09-27T23:14:44.591360+0000","last_unstale":"2023-09-27T23:14:44.591636+0000","last_undegraded":"2023-09-27T23:14:44.591636+0000","last_fullsized":"2023-09-27T23:14:44.591636+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:27:55.086936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589480+0000","last_change":"2023-09-27T23:14:44.221449+0000","last_active":"2023-09-27T23:14:44.589480+0000","last_peered":"2023-09-27T23:14:44.589480+0000","last_clean":"2023-09-27T23:14:44.589480+0000","last_became_active":"2023-09-27T23:14:44.221090+0000","last_became_peered":"2023-09-27T23:14:44.221090+0000","last_unstale":"2023-09-27T23:14:44.589480+0000","last_undegraded":"2023-09-27T23:14:44.589480+0000","last_fullsized":"2023-09-27T23:14:44.589480+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:17:40.184736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.584627+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:14:44.584627+0000","last_peered":"2023-09-27T23:14:44.584627+0000","last_clean":"2023-09-27T23:14:44.584627+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:14:44.584627+0000","last_undegraded":"2023-09-27T23:14:44.584627+0000","last_fullsized":"2023-09-27T23:14:44.584627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.591932+0000","last_change":"2023-09-27T23:14:44.591932+0000","last_active":"2023-09-27T23:14:44.591932+0000","last_peered":"2023-09-27T23:14:44.591932+0000","last_clean":"2023-09-27T23:14:44.591932+0000","last_became_active":"2023-09-27T23:14:44.591617+0000","last_became_peered":"2023-09-27T23:14:44.591617+0000","last_unstale":"2023-09-27T23:14:44.591932+0000","last_undegraded":"2023-09-27T23:14:44.591932+0000","last_fullsized":"2023-09-27T23:14:44.591932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:20:59.587217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.585635+0000","last_change":"2023-09-27T23:14:44.223317+0000","last_active":"2023-09-27T23:14:44.585635+0000","last_peered":"2023-09-27T23:14:44.585635+0000","last_clean":"2023-09-27T23:14:44.585635+0000","last_became_active":"2023-09-27T23:14:44.222810+0000","last_became_peered":"2023-09-27T23:14:44.222810+0000","last_unstale":"2023-09-27T23:14:44.585635+0000","last_undegraded":"2023-09-27T23:14:44.585635+0000","last_fullsized":"2023-09-27T23:14:44.585635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:34:55.091093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589240+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:14:44.589240+0000","last_peered":"2023-09-27T23:14:44.589240+0000","last_clean":"2023-09-27T23:14:44.589240+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:14:44.589240+0000","last_undegraded":"2023-09-27T23:14:44.589240+0000","last_fullsized":"2023-09-27T23:14:44.589240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.596149+0000","last_change":"2023-09-27T23:14:44.596149+0000","last_active":"2023-09-27T23:14:44.596149+0000","last_peered":"2023-09-27T23:14:44.596149+0000","last_clean":"2023-09-27T23:14:44.596149+0000","last_became_active":"2023-09-27T23:14:44.593504+0000","last_became_peered":"2023-09-27T23:14:44.593504+0000","last_unstale":"2023-09-27T23:14:44.596149+0000","last_undegraded":"2023-09-27T23:14:44.596149+0000","last_fullsized":"2023-09-27T23:14:44.596149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:04:29.795767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.579399+0000","last_change":"2023-09-27T23:14:44.218418+0000","last_active":"2023-09-27T23:14:44.579399+0000","last_peered":"2023-09-27T23:14:44.579399+0000","last_clean":"2023-09-27T23:14:44.579399+0000","last_became_active":"2023-09-27T23:14:44.217861+0000","last_became_peered":"2023-09-27T23:14:44.217861+0000","last_unstale":"2023-09-27T23:14:44.579399+0000","last_undegraded":"2023-09-27T23:14:44.579399+0000","last_fullsized":"2023-09-27T23:14:44.579399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:30.098077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.202750+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:14:44.202750+0000","last_peered":"2023-09-27T23:14:44.202750+0000","last_clean":"2023-09-27T23:14:44.202750+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:14:44.202750+0000","last_undegraded":"2023-09-27T23:14:44.202750+0000","last_fullsized":"2023-09-27T23:14:44.202750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.601814+0000","last_change":"2023-09-27T23:14:44.599244+0000","last_active":"2023-09-27T23:14:44.601814+0000","last_peered":"2023-09-27T23:14:44.601814+0000","last_clean":"2023-09-27T23:14:44.601814+0000","last_became_active":"2023-09-27T23:14:44.598913+0000","last_became_peered":"2023-09-27T23:14:44.598913+0000","last_unstale":"2023-09-27T23:14:44.601814+0000","last_undegraded":"2023-09-27T23:14:44.601814+0000","last_fullsized":"2023-09-27T23:14:44.601814+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:28:19.129322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.584575+0000","last_change":"2023-09-27T23:14:44.220590+0000","last_active":"2023-09-27T23:14:44.584575+0000","last_peered":"2023-09-27T23:14:44.584575+0000","last_clean":"2023-09-27T23:14:44.584575+0000","last_became_active":"2023-09-27T23:14:44.220295+0000","last_became_peered":"2023-09-27T23:14:44.220295+0000","last_unstale":"2023-09-27T23:14:44.584575+0000","last_undegraded":"2023-09-27T23:14:44.584575+0000","last_fullsized":"2023-09-27T23:14:44.584575+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:17.911634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.570569+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:14:44.570569+0000","last_peered":"2023-09-27T23:14:44.570569+0000","last_clean":"2023-09-27T23:14:44.570569+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:14:44.570569+0000","last_undegraded":"2023-09-27T23:14:44.570569+0000","last_fullsized":"2023-09-27T23:14:44.570569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.609032+0000","last_change":"2023-09-27T23:14:44.604003+0000","last_active":"2023-09-27T23:14:44.609032+0000","last_peered":"2023-09-27T23:14:44.609032+0000","last_clean":"2023-09-27T23:14:44.609032+0000","last_became_active":"2023-09-27T23:14:44.603680+0000","last_became_peered":"2023-09-27T23:14:44.603680+0000","last_unstale":"2023-09-27T23:14:44.609032+0000","last_undegraded":"2023-09-27T23:14:44.609032+0000","last_fullsized":"2023-09-27T23:14:44.609032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:21:20.987178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.589024+0000","last_change":"2023-09-27T23:14:44.212342+0000","last_active":"2023-09-27T23:14:44.589024+0000","last_peered":"2023-09-27T23:14:44.589024+0000","last_clean":"2023-09-27T23:14:44.589024+0000","last_became_active":"2023-09-27T23:14:44.211923+0000","last_became_peered":"2023-09-27T23:14:44.211923+0000","last_unstale":"2023-09-27T23:14:44.589024+0000","last_undegraded":"2023-09-27T23:14:44.589024+0000","last_fullsized":"2023-09-27T23:14:44.589024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:54:46.815636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.586368+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:14:44.586368+0000","last_peered":"2023-09-27T23:14:44.586368+0000","last_clean":"2023-09-27T23:14:44.586368+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:14:44.586368+0000","last_undegraded":"2023-09-27T23:14:44.586368+0000","last_fullsized":"2023-09-27T23:14:44.586368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.592513+0000","last_change":"2023-09-27T23:14:44.589166+0000","last_active":"2023-09-27T23:14:44.592513+0000","last_peered":"2023-09-27T23:14:44.592513+0000","last_clean":"2023-09-27T23:14:44.592513+0000","last_became_active":"2023-09-27T23:14:44.588374+0000","last_became_peered":"2023-09-27T23:14:44.588374+0000","last_unstale":"2023-09-27T23:14:44.592513+0000","last_undegraded":"2023-09-27T23:14:44.592513+0000","last_fullsized":"2023-09-27T23:14:44.592513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:44.186626+0000","last_clean_scrub_stamp":"2023-09-27T23:14:44.186626+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:41:33.110991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.608502+0000","last_change":"2023-09-27T23:14:44.214968+0000","last_active":"2023-09-27T23:14:44.608502+0000","last_peered":"2023-09-27T23:14:44.608502+0000","last_clean":"2023-09-27T23:14:44.608502+0000","last_became_active":"2023-09-27T23:14:44.214545+0000","last_became_peered":"2023-09-27T23:14:44.214545+0000","last_unstale":"2023-09-27T23:14:44.608502+0000","last_undegraded":"2023-09-27T23:14:44.608502+0000","last_fullsized":"2023-09-27T23:14:44.608502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:14:43.175985+0000","last_clean_scrub_stamp":"2023-09-27T23:14:43.175985+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:05:26.386530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.202774+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:14:44.202774+0000","last_peered":"2023-09-27T23:14:44.202774+0000","last_clean":"2023-09-27T23:14:44.202774+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:14:44.202774+0000","last_undegraded":"2023-09-27T23:14:44.202774+0000","last_fullsized":"2023-09-27T23:14:44.202774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":184,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-09-27T23:14:44.586588+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:14:44.586588+0000","last_peered":"2023-09-27T23:14:44.586588+0000","last_clean":"2023-09-27T23:14:44.586588+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:14:44.586588+0000","last_undegraded":"2023-09-27T23:14:44.586588+0000","last_fullsized":"2023-09-27T23:14:44.586588+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9852,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361988,"statfs":{"total":96636764160,"available":96626675712,"internally_reserved":0,"allocated":1101824,"data_stored":813870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.2110000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.119}]},{"osd":2,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.02}]},{"osd":6,"last update":"Wed Sep 27 23:14:35 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.258}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":217095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.0129999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.93100000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.85099999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.038},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.1870000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.069},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.1399999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8684,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363156,"statfs":{"total":96636764160,"available":96627871744,"internally_reserved":0,"allocated":495616,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.0589999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.749},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.98599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.84699999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.94699999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.90600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.032},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.097}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.127},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.0840000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8756,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363084,"statfs":{"total":96636764160,"available":96627798016,"internally_reserved":0,"allocated":503808,"data_stored":217987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":1.1639999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.206},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.093}]},{"osd":3,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.026},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.2350000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.1359999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8768,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363072,"statfs":{"total":96636764160,"available":96627785728,"internally_reserved":0,"allocated":516096,"data_stored":224022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.85999999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.75},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.98099999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.236},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.377}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.3029999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.94599999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8568,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363272,"statfs":{"total":96636764160,"available":96627990528,"internally_reserved":0,"allocated":507904,"data_stored":218009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93700000000000006}]},{"osd":1,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.052}]},{"osd":3,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.1040000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.23},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.091}]},{"osd":6,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:14:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.294},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.4790000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9852,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361988,"statfs":{"total":96636764160,"available":96626675712,"internally_reserved":0,"allocated":1101824,"data_stored":813814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.998},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3120000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.157}]},{"osd":7,"last update":"Wed Sep 27 23:14:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4259999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.157,"5min":6.157,"15min":6.157},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":5.649,"5min":5.649,"15min":5.649},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"last":0.503},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.0499999999999998,"5min":4.0499999999999998,"15min":4.0499999999999998},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.282},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.5670000000000002,"5min":2.5670000000000002,"15min":2.5670000000000002},"last":1.014}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:14:52.544 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:14:52.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434d605700 1 Processor -- start 2023-09-27T23:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434d605700 1 -- start start 2023-09-27T23:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434d605700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f43481608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434d605700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480c68a0 con 0x7f434806db70 2023-09-27T23:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434d605700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480c6e60 con 0x7f4348155a90 2023-09-27T23:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434d605700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43480ae4d0 con 0x7f43480ae040 2023-09-27T23:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f43477fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f434806db70 0x7f43480ad930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51606/0 (socket says 172.21.15.64:51606) 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f43477fe700 1 -- 172.21.15.64:0/1984537760 learned_addr learned my addr 172.21.15.64:0/1984537760 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434ce04700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f43481608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826922791 0 0) 0x7f43480c68a0 con 0x7f434806db70 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4330003570 con 0x7f434806db70 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2724831176 0 0) 0x7f4330003570 con 0x7f434806db70 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 msgr2=0x7f43481608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f43481608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 >> v1:172.21.15.123:6789/0 conn(0x7f4348155a90 legacy=0x7f43480ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43480ae7b0 con 0x7f434806db70 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f433c001470 con 0x7f434806db70 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.662+0000 7f434ce04700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f43481608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 >> v1:172.21.15.64:6789/0 conn(0x7f434806db70 legacy=0x7f43480ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f4330003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4330003fd0 0x7f4330004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f43300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480ae4d0 con 0x7f4330004d50 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433000bba0 con 0x7f4330003fd0 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433000bd60 con 0x7f43480ae040 2023-09-27T23:14:52.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f43300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4347fff700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f4330003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f43477fe700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4330003fd0 0x7f4330004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 -- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 msgr2=0x7f4330003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43480ae040 0x7f4330003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 -- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4330003fd0 msgr2=0x7f4330004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4330003fd0 0x7f4330004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 -- 172.21.15.64:0/1984537760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009a20 con 0x7f4330004d50 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f43300051c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f434000e3c0 tx=0x7f434000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc189a0ff5f586c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:52.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340009c50 con 0x7f4330004d50 2023-09-27T23:14:52.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434000f040 con 0x7f4330004d50 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f4346ffd700 1 -- 172.21.15.64:0/1984537760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000ee60 con 0x7f4330004d50 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434d605700 1 -- 172.21.15.64:0/1984537760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 msgr2=0x7f43300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434d605700 1 --2- 172.21.15.64:0/1984537760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f43300051c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f434000e3c0 tx=0x7f434000b040 comp rx=0 tx=0).stop 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434ce04700 1 -- 172.21.15.64:0/1984537760 reap_dead start 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434d605700 1 -- 172.21.15.64:0/1984537760 shutdown_connections 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434d605700 1 -- 172.21.15.64:0/1984537760 >> 172.21.15.64:0/1984537760 conn(0x7f43480c8cd0 msgr2=0x7f43480c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434d605700 1 -- 172.21.15.64:0/1984537760 shutdown_connections 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.666+0000 7f434d605700 1 -- 172.21.15.64:0/1984537760 wait complete. 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 Processor -- start 2023-09-27T23:14:52.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 -- start start 2023-09-27T23:14:52.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f4348156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4348157fe0 0x7f4348158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434814f0d0 0x7f434814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434814fa80 con 0x7f4330004d50 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480af720 con 0x7f434814f0d0 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434d605700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43480af0f0 con 0x7f4348157fe0 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f4347fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f4348156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f4347fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f4348156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57258/0 (socket says 172.21.15.64:57258) 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f4347fff700 1 -- 172.21.15.64:0/2237246071 learned_addr learned my addr 172.21.15.64:0/2237246071 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f43477fe700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4348157fe0 0x7f4348158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f434ce04700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434814f0d0 0x7f434814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.670+0000 7f4347fff700 1 -- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4348157fe0 msgr2=0x7f4348158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f4347fff700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4348157fe0 0x7f4348158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f4347fff700 1 -- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434814f0d0 msgr2=0x7f434814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f4347fff700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434814f0d0 0x7f434814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f4347fff700 1 -- 172.21.15.64:0/2237246071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348158ac0 con 0x7f4330004d50 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f434ce04700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434814f0d0 0x7f434814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f4347fff700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f4348156720 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4338002ab0 tx=0x7f4338007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800f8d0 con 0x7f4330004d50 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800fa30 con 0x7f4330004d50 2023-09-27T23:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4348158da0 con 0x7f4330004d50 2023-09-27T23:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800fe30 con 0x7f4330004d50 2023-09-27T23:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434814eca0 con 0x7f4330004d50 2023-09-27T23:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43480ac840 con 0x7f4330004d50 2023-09-27T23:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f4338013600 con 0x7f4330004d50 2023-09-27T23:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f432ffff700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4334060950 0x7f4334062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.674+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f433807d530 con 0x7f4330004d50 2023-09-27T23:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.678+0000 7f43477fe700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4334060950 0x7f4334062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.678+0000 7f43477fe700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4334060950 0x7f4334062e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43480cbf60 tx=0x7f433c001da0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:52.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.686+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f43480ac840 con 0x7f4330004d50 2023-09-27T23:14:52.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.898+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f43480ac840 con 0x7f4330004d50 2023-09-27T23:14:52.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.906+0000 7f432ffff700 1 -- 172.21.15.64:0/2237246071 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f43480ac840 con 0x7f4330004d50 2023-09-27T23:14:52.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4334060950 msgr2=0x7f4334062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4334060950 0x7f4334062e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43480cbf60 tx=0x7f433c001da0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 msgr2=0x7f4348156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f4348156720 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4338002ab0 tx=0x7f4338007d10 comp rx=0 tx=0).stop 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 shutdown_connections 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4334060950 0x7f4334062e00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434814f0d0 0x7f434814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4348157fe0 0x7f4348158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 --2- 172.21.15.64:0/2237246071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4330004d50 0x7f4348156720 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:52.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 >> 172.21.15.64:0/2237246071 conn(0x7f43480c8cd0 msgr2=0x7f43480cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:52.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 shutdown_connections 2023-09-27T23:14:52.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:52.910+0000 7f434d605700 1 -- 172.21.15.64:0/2237246071 wait complete. 2023-09-27T23:14:52.916 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:14:52.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:14:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.030+0000 7fb65b402700 1 Processor -- start 2023-09-27T23:14:53.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.030+0000 7fb65b402700 1 -- start start 2023-09-27T23:14:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.030+0000 7fb65b402700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb654155960 0x7fb65415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.030+0000 7fb65b402700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65406d990 con 0x7fb65406de20 2023-09-27T23:14:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.030+0000 7fb65b402700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65415a570 con 0x7fb6540c7f00 2023-09-27T23:14:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65406cd80 con 0x7fb654155960 2023-09-27T23:14:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb65406de20 0x7fb6540c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51618/0 (socket says 172.21.15.64:51618) 2023-09-27T23:14:53.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 -- 172.21.15.64:0/651397364 learned_addr learned my addr 172.21.15.64:0/651397364 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65a400700 1 --2- 172.21.15.64:0/651397364 >> v2:172.21.15.64:3300/0 conn(0x7fb654155960 0x7fb65415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611925768 0 0) 0x7fb65406d990 con 0x7fb65406de20 2023-09-27T23:14:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63c003570 con 0x7fb65406de20 2023-09-27T23:14:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2919629536 0 0) 0x7fb63c003570 con 0x7fb65406de20 2023-09-27T23:14:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 >> v2:172.21.15.64:3300/0 conn(0x7fb654155960 msgr2=0x7fb65415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 --2- 172.21.15.64:0/651397364 >> v2:172.21.15.64:3300/0 conn(0x7fb654155960 0x7fb65415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 >> v1:172.21.15.123:6789/0 conn(0x7fb6540c7f00 legacy=0x7fb6540aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65406d060 con 0x7fb65406de20 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fb650001470 con 0x7fb65406de20 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65a400700 1 --2- 172.21.15.64:0/651397364 >> v2:172.21.15.64:3300/0 conn(0x7fb654155960 0x7fb65415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 >> v1:172.21.15.64:6789/0 conn(0x7fb65406de20 legacy=0x7fb6540c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63c003a60 0x7fb63c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c004400 0x7fb63c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65406cd80 con 0x7fb63c004400 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c004400 0x7fb63c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c00bfd0 con 0x7fb63c003a60 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c00c190 con 0x7fb63c0050b0 2023-09-27T23:14:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65a400700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63c003a60 0x7fb63c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65ac01700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 -- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb63c0050b0 msgr2=0x7fb63c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 -- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63c003a60 msgr2=0x7fb63c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63c003a60 0x7fb63c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 -- 172.21.15.64:0/651397364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c003570 con 0x7fb63c004400 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65ac01700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65a400700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63c003a60 0x7fb63c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65ac01700 1 -- 172.21.15.64:0/651397364 reap_dead start 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb659bff700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c004400 0x7fb63c0048b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb650005370 tx=0x7fb65000ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c025bb9a59eb5e19 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650005920 con 0x7fb63c004400 2023-09-27T23:14:53.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650005a80 con 0x7fb63c004400 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb6593fe700 1 -- 172.21.15.64:0/651397364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650005e20 con 0x7fb63c004400 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 -- 172.21.15.64:0/651397364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c004400 msgr2=0x7fb63c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c004400 0x7fb63c0048b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb650005370 tx=0x7fb65000ade0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 -- 172.21.15.64:0/651397364 shutdown_connections 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 --2- 172.21.15.64:0/651397364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c004400 0x7fb63c0048b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 -- 172.21.15.64:0/651397364 >> 172.21.15.64:0/651397364 conn(0x7fb654067bb0 msgr2=0x7fb6540cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 -- 172.21.15.64:0/651397364 shutdown_connections 2023-09-27T23:14:53.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.034+0000 7fb65b402700 1 -- 172.21.15.64:0/651397364 wait complete. 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 Processor -- start 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 -- start start 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 0x7fb65414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb654143770 0x7fb6540c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6540c51b0 0x7fb6540aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540c5600 con 0x7fb654143770 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654150d40 con 0x7fb6541433a0 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65b402700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb654151020 con 0x7fb6540c51b0 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb659bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb654143770 0x7fb6540c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 0x7fb65414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 0x7fb65414e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:54098/0 (socket says 172.21.15.64:54098) 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65a400700 1 -- 172.21.15.64:0/3032274894 learned_addr learned my addr 172.21.15.64:0/3032274894 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.038+0000 7fb65ac01700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6540c51b0 0x7fb6540aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65a400700 1 -- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6540c51b0 msgr2=0x7fb6540aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65a400700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6540c51b0 0x7fb6540aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65a400700 1 -- 172.21.15.64:0/3032274894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb654143770 msgr2=0x7fb6540c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65a400700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb654143770 0x7fb6540c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65a400700 1 -- 172.21.15.64:0/3032274894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654151300 con 0x7fb6541433a0 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb659bff700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb654143770 0x7fb6540c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:53.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65a400700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 0x7fb65414e500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb644011890 tx=0x7fb644008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644015670 con 0x7fb6541433a0 2023-09-27T23:14:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440157d0 con 0x7fb6541433a0 2023-09-27T23:14:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6541515e0 con 0x7fb6541433a0 2023-09-27T23:14:53.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644015b90 con 0x7fb6541433a0 2023-09-27T23:14:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65414ea40 con 0x7fb6541433a0 2023-09-27T23:14:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65404be40 con 0x7fb6541433a0 2023-09-27T23:14:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.042+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7fb644013410 con 0x7fb6541433a0 2023-09-27T23:14:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.046+0000 7fb64a7fc700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb640060950 0x7fb640062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.046+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb64407d7e0 con 0x7fb6541433a0 2023-09-27T23:14:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.046+0000 7fb659bff700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb640060950 0x7fb640062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.046+0000 7fb659bff700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb640060950 0x7fb640062e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb65000ae20 tx=0x7fb650018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.054+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb64404d880 con 0x7fb6541433a0 2023-09-27T23:14:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.258+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6540af200 con 0x7fb6541433a0 2023-09-27T23:14:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.258+0000 7fb64a7fc700 1 -- 172.21.15.64:0/3032274894 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fb644050ea0 con 0x7fb6541433a0 2023-09-27T23:14:53.264 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:14:53.264 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:14:44.562884+0000","modified":"2023-09-27T23:14:46.768950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/974610030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":974610030},{"type":"v1","addr":"172.21.15.64:6835","nonce":974610030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-09-27T23:14:53.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb640060950 msgr2=0x7fb640062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb640060950 0x7fb640062e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb65000ae20 tx=0x7fb650018000 comp rx=0 tx=0).stop 2023-09-27T23:14:53.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 msgr2=0x7fb65414e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 0x7fb65414e500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb644011890 tx=0x7fb644008040 comp rx=0 tx=0).stop 2023-09-27T23:14:53.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 shutdown_connections 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb640060950 0x7fb640062e00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6540c51b0 0x7fb6540aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb654143770 0x7fb6540c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 --2- 172.21.15.64:0/3032274894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6541433a0 0x7fb65414e500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 >> 172.21.15.64:0/3032274894 conn(0x7fb654067bb0 msgr2=0x7fb6540c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 shutdown_connections 2023-09-27T23:14:53.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.262+0000 7fb65b402700 1 -- 172.21.15.64:0/3032274894 wait complete. 2023-09-27T23:14:53.270 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2023-09-27T23:14:53.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:14:44.562884+0000', 'modified': '2023-09-27T23:14:46.768950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5211}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5211': {'gid': 5211, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/974610030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 974610030}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 974610030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:14:53.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:14:53.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-27T23:14:53.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.802+0000 7feb4bfff700 1 Processor -- start 2023-09-27T23:14:53.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.802+0000 7feb4bfff700 1 -- start start 2023-09-27T23:14:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb4c0b0810 0x7feb4c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0af530 con 0x7feb4c0af9c0 2023-09-27T23:14:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c159660 con 0x7feb4c0b0810 2023-09-27T23:14:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c159940 con 0x7feb4c06be10 2023-09-27T23:14:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7feb4c0af9c0 0x7feb4c154b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51620/0 (socket says 172.21.15.64:51620) 2023-09-27T23:14:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4b7fe700 1 -- 172.21.15.64:0/2897107667 learned_addr learned my addr 172.21.15.64:0/2897107667 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb427fc700 1 --2- 172.21.15.64:0/2897107667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4affd700 1 --2- 172.21.15.64:0/2897107667 >> v2:172.21.15.123:3300/0 conn(0x7feb4c0b0810 0x7feb4c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4a7fc700 1 -- 172.21.15.64:0/2897107667 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924544081 0 0) 0x7feb4c0af530 con 0x7feb4c0af9c0 2023-09-27T23:14:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb4a7fc700 1 -- 172.21.15.64:0/2897107667 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb38003570 con 0x7feb4c0af9c0 2023-09-27T23:14:53.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb427fc700 1 -- 172.21.15.64:0/2897107667 >> v2:172.21.15.123:3300/0 conn(0x7feb4c0b0810 msgr2=0x7feb4c0b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb427fc700 1 --2- 172.21.15.64:0/2897107667 >> v2:172.21.15.123:3300/0 conn(0x7feb4c0b0810 0x7feb4c0b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb427fc700 1 -- 172.21.15.64:0/2897107667 >> v1:172.21.15.64:6789/0 conn(0x7feb4c0af9c0 legacy=0x7feb4c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb427fc700 1 -- 172.21.15.64:0/2897107667 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c159c20 con 0x7feb4c06be10 2023-09-27T23:14:53.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.806+0000 7feb427fc700 1 --2- 172.21.15.64:0/2897107667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c158d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb34009910 tx=0x7feb340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=836500b72a922c38 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4a7fc700 1 -- 172.21.15.64:0/2897107667 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34011630 con 0x7feb4c06be10 2023-09-27T23:14:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4a7fc700 1 -- 172.21.15.64:0/2897107667 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34011790 con 0x7feb4c06be10 2023-09-27T23:14:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4a7fc700 1 -- 172.21.15.64:0/2897107667 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34011b30 con 0x7feb4c06be10 2023-09-27T23:14:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 -- 172.21.15.64:0/2897107667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 msgr2=0x7feb4c158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2897107667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c158d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb34009910 tx=0x7feb340091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 -- 172.21.15.64:0/2897107667 shutdown_connections 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2897107667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c158d60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2897107667 >> v2:172.21.15.123:3300/0 conn(0x7feb4c0b0810 0x7feb4c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 -- 172.21.15.64:0/2897107667 >> 172.21.15.64:0/2897107667 conn(0x7feb4c067bb0 msgr2=0x7feb4c0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 -- 172.21.15.64:0/2897107667 shutdown_connections 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.810+0000 7feb4bfff700 1 -- 172.21.15.64:0/2897107667 wait complete. 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 Processor -- start 2023-09-27T23:14:53.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 -- start start 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb4c0af9c0 0x7feb4c16c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0b0810 0x7feb4c16c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1557a0 con 0x7feb4c0af9c0 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c6df0 con 0x7feb4c0b0810 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c6550 con 0x7feb4c06be10 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb4c0af9c0 0x7feb4c16c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb4c0af9c0 0x7feb4c16c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57300/0 (socket says 172.21.15.64:57300) 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb427fc700 1 -- 172.21.15.64:0/2292548053 learned_addr learned my addr 172.21.15.64:0/2292548053 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0b0810 0x7feb4c16c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4affd700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 -- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 msgr2=0x7feb4c0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c0c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 -- 172.21.15.64:0/2292548053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb4c0af9c0 msgr2=0x7feb4c16c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb4c0af9c0 0x7feb4c16c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 -- 172.21.15.64:0/2292548053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34009420 con 0x7feb4c0b0810 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0b0810 0x7feb4c16c6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb4400cbc0 tx=0x7feb44001a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb437fe700 1 -- 172.21.15.64:0/2292548053 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400a7e0 con 0x7feb4c0b0810 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb437fe700 1 -- 172.21.15.64:0/2292548053 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4400a940 con 0x7feb4c0b0810 2023-09-27T23:14:53.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c0c6830 con 0x7feb4c0b0810 2023-09-27T23:14:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb437fe700 1 -- 172.21.15.64:0/2292548053 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400acc0 con 0x7feb4c0b0810 2023-09-27T23:14:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.814+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c0a4670 con 0x7feb4c0b0810 2023-09-27T23:14:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.818+0000 7feb437fe700 1 -- 172.21.15.64:0/2292548053 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7feb44007300 con 0x7feb4c0b0810 2023-09-27T23:14:53.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.818+0000 7feb437fe700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb2c060950 0x7feb2c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.818+0000 7feb437fe700 1 -- 172.21.15.64:0/2292548053 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7feb4407d250 con 0x7feb4c0b0810 2023-09-27T23:14:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.818+0000 7feb4affd700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb2c060950 0x7feb2c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.818+0000 7feb4affd700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb2c060950 0x7feb2c062e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb4c157530 tx=0x7feb3c00a330 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 Processor -- start 2023-09-27T23:14:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 -- start start 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x2a08390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x2a088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2935f30 con 0x278a710 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2931ed0 con 0x27d5790 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x28f80b0 con 0x278aae0 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x2a08390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x2a08390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57304/0 (socket says 172.21.15.64:57304) 2023-09-27T23:14:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 -- 172.21.15.64:0/445387309 learned_addr learned my addr 172.21.15.64:0/445387309 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb4affd700 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x2a088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 -- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 msgr2=0x2a07e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a07e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 -- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 msgr2=0x2a088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x2a088d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 -- 172.21.15.64:0/445387309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27d69e0 con 0x278a710 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb427fc700 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x2a08390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feb34009f00 tx=0x7feb34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fda6d0dfaf318f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb2bfff700 1 -- 172.21.15.64:0/445387309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34021070 con 0x278a710 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb2bfff700 1 -- 172.21.15.64:0/445387309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34000bc0 con 0x278a710 2023-09-27T23:14:53.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.822+0000 7feb2bfff700 1 -- 172.21.15.64:0/445387309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34011b10 con 0x278a710 2023-09-27T23:14:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 -- 172.21.15.64:0/445387309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 msgr2=0x2a08390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x2a08390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feb34009f00 tx=0x7feb34017000 comp rx=0 tx=0).stop 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 -- 172.21.15.64:0/445387309 shutdown_connections 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x2a088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x2a08390 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 --2- 172.21.15.64:0/445387309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 -- 172.21.15.64:0/445387309 >> 172.21.15.64:0/445387309 conn(0x278aeb0 msgr2=0x29c4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 -- 172.21.15.64:0/445387309 shutdown_connections 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 -- 172.21.15.64:0/445387309 wait complete. 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 -- start start 2023-09-27T23:14:53.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.826+0000 7feb53c5d740 1 Processor -- start 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29edc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278aae0 0x29ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x27d5790 0x29ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x28f7d50 con 0x278a710 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x28d0970 con 0x278aae0 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x28fa920 con 0x27d5790 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29edc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29edc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57318/0 (socket says 172.21.15.64:57318) 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 -- 172.21.15.64:0/3403529420 learned_addr learned my addr 172.21.15.64:0/3403529420 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb427fc700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278aae0 0x29ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 10 client.0 _ll_register_callbacks cb 0x297bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:14:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x27d5790 0x29ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 -- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x27d5790 msgr2=0x29ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x27d5790 0x29ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 -- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278aae0 msgr2=0x29ee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278aae0 0x29ee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 -- 172.21.15.64:0/3403529420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c159c20 con 0x278a710 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb427fc700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278aae0 0x29ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb4affd700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29edc00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb3c013620 tx=0x7feb3c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dda1406f954a56 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb297fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb297fa700 1 -- 172.21.15.64:0/3403529420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c013e90 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb297fa700 1 -- 172.21.15.64:0/3403529420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7feb3c021640 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28d0cc0 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb297fa700 1 -- 172.21.15.64:0/3403529420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb3c021800 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 10 client.5421 fetch_fsmap learned FSMap version 14 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x2900370 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb297fa700 1 -- 172.21.15.64:0/3403529420 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28d0cc0 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb297fa700 1 -- 172.21.15.64:0/3403529420 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 14) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feb3c021960 con 0x278a710 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 10 client.5421 fetch_fsmap finished waiting for FSMap version 14 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 10 client.5421 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 20 client.5421 populate_metadata read hostname 'smithi064' 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x2a148c0 0x2a16ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 4 client.5421 mds_command: new command op to 5211 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:14:53.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 --> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x28d43e0 con 0x2a148c0 2023-09-27T23:14:53.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.830+0000 7feb427fc700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x2a148c0 0x2a16ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb297fa700 10 client.5421 ms_handle_connect on v2:172.21.15.64:6834/974610030 2023-09-27T23:14:53.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb427fc700 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x2a148c0 0x2a16ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e1be6a01b39e8a server_cookie=80633a35e37a8333 in_seq=0 out_seq=0 2023-09-27T23:14:53.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb297fa700 1 -- 172.21.15.64:0/3403529420 <== mds.0 v2:172.21.15.64:6834/974610030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x28d43e0 con 0x2a148c0 2023-09-27T23:14:53.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb297fa700 10 client.5421 handle_command_reply: tid=0 2023-09-27T23:14:53.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 client.5421 shutdown 2023-09-27T23:14:53.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 msgr2=0x29edc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29edc00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb3c013620 tx=0x7feb3c018000 comp rx=0 tx=0).stop 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 shutdown_connections 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x27d5790 0x29ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x2a148c0 0x2a16ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278aae0 0x29ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 --2- 172.21.15.64:0/3403529420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29edc00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 >> 172.21.15.64:0/3403529420 conn(0x278aeb0 msgr2=0x29c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb297fa700 0 client.5421 ms_handle_reset on v2:172.21.15.64:6834/974610030 2023-09-27T23:14:53.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 shutdown_connections 2023-09-27T23:14:53.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 1 -- 172.21.15.64:0/3403529420 wait complete. 2023-09-27T23:14:53.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.834+0000 7feb53c5d740 20 client.5421 trim_cache size 0 max 16384 2023-09-27T23:14:53.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 Processor -- start 2023-09-27T23:14:53.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 -- start start 2023-09-27T23:14:53.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x29c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x29e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x28037c0 con 0x278a710 2023-09-27T23:14:53.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x29359d0 con 0x27d5790 2023-09-27T23:14:53.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x292f890 con 0x278aae0 2023-09-27T23:14:53.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57322/0 (socket says 172.21.15.64:57322) 2023-09-27T23:14:53.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 -- 172.21.15.64:0/1000174690 learned_addr learned my addr 172.21.15.64:0/1000174690 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:53.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb427fc700 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x29c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x29e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 -- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 msgr2=0x29c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x29c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 -- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 msgr2=0x29e56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x29e56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.854+0000 7feb4affd700 1 -- 172.21.15.64:0/1000174690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0cbdc0 con 0x278a710 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29c3960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feb3c013590 tx=0x7feb3c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb5ff89c46860d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb40ff9700 1 -- 172.21.15.64:0/1000174690 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c011c70 con 0x278a710 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb40ff9700 1 -- 172.21.15.64:0/1000174690 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c013c10 con 0x278a710 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb40ff9700 1 -- 172.21.15.64:0/1000174690 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c013d80 con 0x278a710 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- 172.21.15.64:0/1000174690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 msgr2=0x29c3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29c3960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feb3c013590 tx=0x7feb3c01b000 comp rx=0 tx=0).stop 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- 172.21.15.64:0/1000174690 shutdown_connections 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x27d5790 0x29e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x29c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- 172.21.15.64:0/1000174690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x278a710 0x29c3960 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- 172.21.15.64:0/1000174690 >> 172.21.15.64:0/1000174690 conn(0x278aeb0 msgr2=0x29c2240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- 172.21.15.64:0/1000174690 shutdown_connections 2023-09-27T23:14:53.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- 172.21.15.64:0/1000174690 wait complete. 2023-09-27T23:14:53.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- start start 2023-09-27T23:14:53.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 Processor -- start 2023-09-27T23:14:53.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278a710 0x2a60cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a61210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 0x29e8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2936490 con 0x27d5790 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2930d60 con 0x278a710 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x299fbb0 con 0x278aae0 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb53c5d740 10 client.0 _ll_register_callbacks cb 0x284e0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 0x29e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278a710 0x2a60cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 0x29e8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57332/0 (socket says 172.21.15.64:57332) 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4b7fe700 1 -- 172.21.15.64:0/2121940442 learned_addr learned my addr 172.21.15.64:0/2121940442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb427fc700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a61210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 -- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 msgr2=0x2a61210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a61210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 -- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 msgr2=0x29e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 0x29e8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 -- 172.21.15.64:0/2121940442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2900370 con 0x278a710 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4b7fe700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 0x29e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.858+0000 7feb4affd700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278a710 0x2a60cd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb3c03d000 tx=0x7feb3c037040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872e383a3a2444f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 1 -- 172.21.15.64:0/2121940442 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c013d80 con 0x278a710 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 1 -- 172.21.15.64:0/2121940442 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7feb3c021850 con 0x278a710 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb34000bc0 con 0x278a710 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 1 -- 172.21.15.64:0/2121940442 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb3c02a460 con 0x278a710 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 10 client.5428 fetch_fsmap learned FSMap version 14 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x28e6d80 con 0x278a710 2023-09-27T23:14:53.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 1 -- 172.21.15.64:0/2121940442 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34000bc0 con 0x278a710 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 1 -- 172.21.15.64:0/2121940442 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 14) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feb3c02a5c0 con 0x278a710 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 10 client.5428 fetch_fsmap finished waiting for FSMap version 14 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 10 client.5428 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 20 client.5428 populate_metadata read hostname 'smithi064' 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x29e8c20 0x29e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 4 client.5428 mds_command: new command op to 5211 tid=0[{"prefix": "status"}] 2023-09-27T23:14:53.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 --> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2906410 con 0x29e8c20 2023-09-27T23:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb427fc700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x29e8c20 0x29e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 10 client.5428 ms_handle_connect on v2:172.21.15.64:6834/974610030 2023-09-27T23:14:53.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb427fc700 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x29e8c20 0x29e8ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4031507b4a717cff server_cookie=9efc4d3390725ba8 in_seq=0 out_seq=0 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 1 -- 172.21.15.64:0/2121940442 <== mds.0 v2:172.21.15.64:6834/974610030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x2906410 con 0x29e8c20 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 10 client.5428 handle_command_reply: tid=0 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 client.5428 shutdown 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278a710 msgr2=0x2a60cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278a710 0x2a60cd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb3c03d000 tx=0x7feb3c037040 comp rx=0 tx=0).stop 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 shutdown_connections 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x27d5790 0x29e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x278aae0 0x2a61210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.64:6834/974610030,v1:172.21.15.64:6835/974610030] conn(0x29e8c20 0x29e8ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 --2- 172.21.15.64:0/2121940442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x278a710 0x2a60cd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 >> 172.21.15.64:0/2121940442 conn(0x278aeb0 msgr2=0x29ee3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb297fa700 0 client.5428 ms_handle_reset on v2:172.21.15.64:6834/974610030 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 shutdown_connections 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 1 -- 172.21.15.64:0/2121940442 wait complete. 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb53c5d740 20 client.5428 trim_cache size 0 max 16384 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:14:53.871 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5211, 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.299219484, 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 14, 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 44, 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 44, 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 16.593985325999999 2023-09-27T23:14:53.872 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb2c060950 msgr2=0x7feb2c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb2c060950 0x7feb2c062e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb4c157530 tx=0x7feb3c00a330 comp rx=0 tx=0).stop 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0b0810 msgr2=0x7feb4c16c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0b0810 0x7feb4c16c6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb4400cbc0 tx=0x7feb44001a20 comp rx=0 tx=0).stop 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 shutdown_connections 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb2c060950 0x7feb2c062e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0b0810 0x7feb4c16c6c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb4c0af9c0 0x7feb4c16c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 --2- 172.21.15.64:0/2292548053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c06be10 0x7feb4c0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.862+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 >> 172.21.15.64:0/2292548053 conn(0x7feb4c067bb0 msgr2=0x7feb4c0c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.866+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 shutdown_connections 2023-09-27T23:14:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.866+0000 7feb4bfff700 1 -- 172.21.15.64:0/2292548053 wait complete. 2023-09-27T23:14:53.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:14:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 Processor -- start 2023-09-27T23:14:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- start start 2023-09-27T23:14:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58a00c15b0 0x7f58a015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a00c7190 con 0x7f58a015d320 2023-09-27T23:14:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a00c1b50 con 0x7f58a00aeba0 2023-09-27T23:14:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00aefd0 con 0x7f58a00c15b0 2023-09-27T23:14:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5d02700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58a00c15b0 0x7f58a015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a6503700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58a015d320 0x7f58a006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51626/0 (socket says 172.21.15.64:51626) 2023-09-27T23:14:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a6503700 1 -- 172.21.15.64:0/3284158976 learned_addr learned my addr 172.21.15.64:0/3284158976 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5d02700 1 -- 172.21.15.64:0/3284158976 >> v1:172.21.15.123:6789/0 conn(0x7f58a00aeba0 legacy=0x7f58a00680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5d02700 1 -- 172.21.15.64:0/3284158976 >> v1:172.21.15.64:6789/0 conn(0x7f58a015d320 legacy=0x7f58a006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5d02700 1 -- 172.21.15.64:0/3284158976 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00c1e30 con 0x7f58a00c15b0 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5d02700 1 --2- 172.21.15.64:0/3284158976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00c15b0 0x7f58a015d6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5898009e60 tx=0x7f5898009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e82d063a275c6b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5501700 1 -- 172.21.15.64:0/3284158976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800e070 con 0x7f58a00c15b0 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5501700 1 -- 172.21.15.64:0/3284158976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589800bc30 con 0x7f58a00c15b0 2023-09-27T23:14:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a5501700 1 -- 172.21.15.64:0/3284158976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898012430 con 0x7f58a00c15b0 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- 172.21.15.64:0/3284158976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00c15b0 msgr2=0x7f58a015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 --2- 172.21.15.64:0/3284158976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00c15b0 0x7f58a015d6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5898009e60 tx=0x7f5898009780 comp rx=0 tx=0).stop 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- 172.21.15.64:0/3284158976 shutdown_connections 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 --2- 172.21.15.64:0/3284158976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00c15b0 0x7f58a015d6f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- 172.21.15.64:0/3284158976 >> 172.21.15.64:0/3284158976 conn(0x7f58a00c8850 msgr2=0x7f58a00c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- 172.21.15.64:0/3284158976 shutdown_connections 2023-09-27T23:14:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.994+0000 7f58a8767700 1 -- 172.21.15.64:0/3284158976 wait complete. 2023-09-27T23:14:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 Processor -- start 2023-09-27T23:14:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 -- start start 2023-09-27T23:14:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 0x7f58a016eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58a00c15b0 0x7f58a0148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:54.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a015d320 0x7f58a016d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00b0a20 con 0x7f58a00aeba0 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00c3150 con 0x7f58a015d320 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a8767700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a016e4d0 con 0x7f58a00c15b0 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a6503700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 0x7f58a016eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a6503700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 0x7f58a016eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57346/0 (socket says 172.21.15.64:57346) 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a6503700 1 -- 172.21.15.64:0/733629293 learned_addr learned my addr 172.21.15.64:0/733629293 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a5d02700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58a00c15b0 0x7f58a0148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:53.998+0000 7f58a6d04700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a015d320 0x7f58a016d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a6503700 1 -- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58a00c15b0 msgr2=0x7f58a0148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a6503700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58a00c15b0 0x7f58a0148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a6503700 1 -- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a015d320 msgr2=0x7f58a016d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a6503700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a015d320 0x7f58a016d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a6503700 1 -- 172.21.15.64:0/733629293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898009a20 con 0x7f58a00aeba0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a6503700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 0x7f58a016eac0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c7c0 tx=0x7f589c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c005b30 con 0x7f58a00aeba0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c005c90 con 0x7f58a00aeba0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c004a80 con 0x7f58a00aeba0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a016e7b0 con 0x7f58a00aeba0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00b06b0 con 0x7f58a00aeba0 2023-09-27T23:14:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a00c2a10 con 0x7f58a00aeba0 2023-09-27T23:14:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f589c0070d0 con 0x7f58a00aeba0 2023-09-27T23:14:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58967fc700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f588c060950 0x7f588c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.002+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f589c07c340 con 0x7f58a00aeba0 2023-09-27T23:14:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.006+0000 7f58a5d02700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f588c060950 0x7f588c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.006+0000 7f58a5d02700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f588c060950 0x7f588c062e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5898009e60 tx=0x7f5898019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.010+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f589c049bb0 con 0x7f58a00aeba0 2023-09-27T23:14:54.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.210+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58a0071120 con 0x7f58a00aeba0 2023-09-27T23:14:54.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.210+0000 7f58967fc700 1 -- 172.21.15.64:0/733629293 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f58a00c2a10 con 0x7f58a00aeba0 2023-09-27T23:14:54.214 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f588c060950 msgr2=0x7f588c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f588c060950 0x7f588c062e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5898009e60 tx=0x7f5898019000 comp rx=0 tx=0).stop 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 msgr2=0x7f58a016eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 0x7f58a016eac0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c7c0 tx=0x7f589c007f70 comp rx=0 tx=0).stop 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 shutdown_connections 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f588c060950 0x7f588c062e00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58a015d320 0x7f58a016d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58a00c15b0 0x7f58a0148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 --2- 172.21.15.64:0/733629293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a00aeba0 0x7f58a016eac0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:54.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 >> 172.21.15.64:0/733629293 conn(0x7f58a00c8850 msgr2=0x7f58a015b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:54.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 shutdown_connections 2023-09-27T23:14:54.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:54.214+0000 7f58a8767700 1 -- 172.21.15.64:0/733629293 wait complete. 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:14:54.229 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:14:54.230 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:14:54.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:54.230 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:14:54.237 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:54.238 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42490sec preferred_lft 42490sec 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:54.239 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:14:54.240 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:14:54.241 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:14:54.241 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:14:54.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:54.241 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:14:54.255 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:14:54.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:54.257 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:14:54.265 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:14:54.266 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:14:54.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:54.267 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:14:54.267 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:14:54.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:14:54.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:14:54.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:14:54.267 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:14:54.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:54.305 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:14:54.359 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:14:54.359 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:14:54.359 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:14:54.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:14:54.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:14:54.373 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:14:54.373 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:54.374 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:54.381 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:14:54.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:14:54.381 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:54.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:54.404 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:54.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:54.465 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:54.466 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:54.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:54.466 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:54.473 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:14:54.474 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:14:54.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:54.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:54.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:54.503 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:54.504 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:54.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:54.504 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:54.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:14:54.554+0000 7f1bab6aa380 -1 init, newargv = 0x55638a890a00 newargc=15 2023-09-27T23:14:54.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[44038]: starting ceph client 2023-09-27T23:14:54.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[44038]: starting fuse 2023-09-27T23:14:55.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:14:55.631 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:14:55.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:14:55.687 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:14:55.688 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:14:55.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:55.689 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:14:55.696 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:14:55.696 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:14:55.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:14:55.696 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:14:55.697 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:55.698 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:14:55.699 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:14:55.699 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:14:55.699 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:14:55.699 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:14:55.699 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:14:55.699 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:14:55.700 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:14:55.700 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:14:55.700 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:14:55.700 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:14:55.782 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.44038.asok 2023-09-27T23:14:55.968 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44038.asok 2023-09-27T23:14:55.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:55.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44038.asok status 2023-09-27T23:14:56.067 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:14:56.067 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:14:56.067 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:14:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:14:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:14:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:14:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:14:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "44038", 2023-09-27T23:14:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:14:56.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:56.069 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:14:56.069 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:14:56.069 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5442, 2023-09-27T23:14:56.069 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:14:56.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:14:56.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:14:56.070 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5442 2023-09-27T23:14:56.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:56.070 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:14:56.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2227474541 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:14:56.071 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:14:56.072 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2227474541 2023-09-27T23:14:56.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:14:56.072 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5442 192.168.0.1:0/2227474541", 2023-09-27T23:14:56.072 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2227474541", 2023-09-27T23:14:56.072 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:14:56.072 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 14, 2023-09-27T23:14:56.073 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 44, 2023-09-27T23:14:56.073 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:14:56.073 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:14:56.073 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:14:56.073 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:14:56.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:56.078 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:14:56.078 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:56.086 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:14:56.087 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:56.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:56.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:14:56.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:14:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.210+0000 7f3b3ab74700 1 Processor -- start 2023-09-27T23:14:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.210+0000 7f3b3ab74700 1 -- start start 2023-09-27T23:14:56.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b3406de20 0x7f3b340c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3406d990 con 0x7f3b3406c7c0 2023-09-27T23:14:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3406cd80 con 0x7f3b34155960 2023-09-27T23:14:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3406d060 con 0x7f3b3406de20 2023-09-27T23:14:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b3406de20 0x7f3b340c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b3406de20 0x7f3b340c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55144/0 (socket says 172.21.15.64:55144) 2023-09-27T23:14:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 -- 172.21.15.64:0/2619228978 learned_addr learned my addr 172.21.15.64:0/2619228978 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 -- 172.21.15.64:0/2619228978 >> v1:172.21.15.123:6789/0 conn(0x7f3b34155960 legacy=0x7f3b3415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 -- 172.21.15.64:0/2619228978 >> v1:172.21.15.64:6789/0 conn(0x7f3b3406c7c0 legacy=0x7f3b340aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 -- 172.21.15.64:0/2619228978 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3406d340 con 0x7f3b3406de20 2023-09-27T23:14:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b39371700 1 --2- 172.21.15.64:0/2619228978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406de20 0x7f3b340c77f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000d1a0 tx=0x7f3b3000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56949d4a1373b4cc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b38b70700 1 -- 172.21.15.64:0/2619228978 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30010070 con 0x7f3b3406de20 2023-09-27T23:14:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b38b70700 1 -- 172.21.15.64:0/2619228978 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3000de80 con 0x7f3b3406de20 2023-09-27T23:14:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b38b70700 1 -- 172.21.15.64:0/2619228978 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000b320 con 0x7f3b3406de20 2023-09-27T23:14:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- 172.21.15.64:0/2619228978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406de20 msgr2=0x7f3b340c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/2619228978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406de20 0x7f3b340c77f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000d1a0 tx=0x7f3b3000ba50 comp rx=0 tx=0).stop 2023-09-27T23:14:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- 172.21.15.64:0/2619228978 shutdown_connections 2023-09-27T23:14:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/2619228978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406de20 0x7f3b340c77f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- 172.21.15.64:0/2619228978 >> 172.21.15.64:0/2619228978 conn(0x7f3b34067bb0 msgr2=0x7f3b340cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:56.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- 172.21.15.64:0/2619228978 shutdown_connections 2023-09-27T23:14:56.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.214+0000 7f3b3ab74700 1 -- 172.21.15.64:0/2619228978 wait complete. 2023-09-27T23:14:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 Processor -- start 2023-09-27T23:14:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 -- start start 2023-09-27T23:14:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 0x7f3b340a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b3406de20 0x7f3b34153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b34155960 0x7f3b3414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340ab1f0 con 0x7f3b3406c7c0 2023-09-27T23:14:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340aa5e0 con 0x7f3b34155960 2023-09-27T23:14:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340aa8c0 con 0x7f3b3406de20 2023-09-27T23:14:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 0x7f3b340a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 0x7f3b340a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55146/0 (socket says 172.21.15.64:55146) 2023-09-27T23:14:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 -- 172.21.15.64:0/672263212 learned_addr learned my addr 172.21.15.64:0/672263212 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39371700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b3406de20 0x7f3b34153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3a373700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b34155960 0x7f3b3414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 -- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b3406de20 msgr2=0x7f3b34153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b3406de20 0x7f3b34153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 -- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b34155960 msgr2=0x7f3b3414d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b34155960 0x7f3b3414d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 -- 172.21.15.64:0/672263212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3000bcf0 con 0x7f3b3406c7c0 2023-09-27T23:14:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b39b72700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 0x7f3b340a4e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b5e0 tx=0x7f3b24009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400f510 con 0x7f3b3406c7c0 2023-09-27T23:14:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2400f670 con 0x7f3b3406c7c0 2023-09-27T23:14:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b340aaba0 con 0x7f3b3406c7c0 2023-09-27T23:14:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400fa30 con 0x7f3b3406c7c0 2023-09-27T23:14:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.218+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b340a5390 con 0x7f3b3406c7c0 2023-09-27T23:14:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.222+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34068740 con 0x7f3b3406c7c0 2023-09-27T23:14:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.222+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f3b2401a400 con 0x7f3b3406c7c0 2023-09-27T23:14:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.222+0000 7f3b29ffb700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3b18060950 0x7f3b18062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.222+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3b2407ca40 con 0x7f3b3406c7c0 2023-09-27T23:14:56.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.222+0000 7f3b39371700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3b18060950 0x7f3b18062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.222+0000 7f3b39371700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3b18060950 0x7f3b18062e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000ba90 tx=0x7f3b30019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:56.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.234+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3b34068740 con 0x7f3b3406c7c0 2023-09-27T23:14:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.410+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b340ab330 con 0x7f3b18060950 2023-09-27T23:14:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.414+0000 7f3b29ffb700 1 -- 172.21.15.64:0/672263212 <== mgr.4108 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3b30001060 con 0x7f3b18060950 2023-09-27T23:14:56.419 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:14:56.419 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:14:56.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:14:56.419 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:14:56.420 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:14:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3b18060950 msgr2=0x7f3b18062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3b18060950 0x7f3b18062e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000ba90 tx=0x7f3b30019000 comp rx=0 tx=0).stop 2023-09-27T23:14:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 msgr2=0x7f3b340a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 0x7f3b340a4e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b5e0 tx=0x7f3b24009120 comp rx=0 tx=0).stop 2023-09-27T23:14:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 shutdown_connections 2023-09-27T23:14:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3b18060950 0x7f3b18062e00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b34155960 0x7f3b3414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b3406de20 0x7f3b34153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 --2- 172.21.15.64:0/672263212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b3406c7c0 0x7f3b340a4e50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 >> 172.21.15.64:0/672263212 conn(0x7f3b34067bb0 msgr2=0x7f3b3414e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 shutdown_connections 2023-09-27T23:14:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.418+0000 7f3b3ab74700 1 -- 172.21.15.64:0/672263212 wait complete. 2023-09-27T23:14:56.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:14:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.546+0000 7f8165a85700 1 Processor -- start 2023-09-27T23:14:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.546+0000 7f8165a85700 1 -- start start 2023-09-27T23:14:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.546+0000 7f8165a85700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f81600c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f816006c7c0 0x7f816015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816006d970 con 0x7f816006de00 2023-09-27T23:14:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816006cd60 con 0x7f816006c7c0 2023-09-27T23:14:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816006d040 con 0x7f81600c69a0 2023-09-27T23:14:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f816006c7c0 0x7f816015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f816006c7c0 0x7f816015cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55152/0 (socket says 172.21.15.64:55152) 2023-09-27T23:14:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 -- 172.21.15.64:0/2620888632 learned_addr learned my addr 172.21.15.64:0/2620888632 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8164a83700 1 --2- 172.21.15.64:0/2620888632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f81600c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 -- 172.21.15.64:0/2620888632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 msgr2=0x7f81600c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 --2- 172.21.15.64:0/2620888632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f81600c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 -- 172.21.15.64:0/2620888632 >> v1:172.21.15.123:6789/0 conn(0x7f816006de00 legacy=0x7f816006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 -- 172.21.15.64:0/2620888632 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816006d320 con 0x7f816006c7c0 2023-09-27T23:14:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165284700 1 --2- 172.21.15.64:0/2620888632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816015cba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f81580099e0 tx=0x7f8158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2ba12dc4183254 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f815f7fe700 1 -- 172.21.15.64:0/2620888632 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800b820 con 0x7f816006c7c0 2023-09-27T23:14:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f815f7fe700 1 -- 172.21.15.64:0/2620888632 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815800b980 con 0x7f816006c7c0 2023-09-27T23:14:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f815f7fe700 1 -- 172.21.15.64:0/2620888632 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800bd20 con 0x7f816006c7c0 2023-09-27T23:14:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- 172.21.15.64:0/2620888632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 msgr2=0x7f816015cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 --2- 172.21.15.64:0/2620888632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816015cba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f81580099e0 tx=0x7f8158009250 comp rx=0 tx=0).stop 2023-09-27T23:14:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- 172.21.15.64:0/2620888632 shutdown_connections 2023-09-27T23:14:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 --2- 172.21.15.64:0/2620888632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816015cba0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 --2- 172.21.15.64:0/2620888632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f81600c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- 172.21.15.64:0/2620888632 >> 172.21.15.64:0/2620888632 conn(0x7f8160067b50 msgr2=0x7f81600cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- 172.21.15.64:0/2620888632 shutdown_connections 2023-09-27T23:14:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.550+0000 7f8165a85700 1 -- 172.21.15.64:0/2620888632 wait complete. 2023-09-27T23:14:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 Processor -- start 2023-09-27T23:14:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 -- start start 2023-09-27T23:14:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816016c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816006de00 0x7f816015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f816015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816015dad0 con 0x7f816006c7c0 2023-09-27T23:14:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81600b01e0 con 0x7f816006de00 2023-09-27T23:14:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816015d1a0 con 0x7f81600c69a0 2023-09-27T23:14:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816016c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816016c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55164/0 (socket says 172.21.15.64:55164) 2023-09-27T23:14:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 -- 172.21.15.64:0/1473233794 learned_addr learned my addr 172.21.15.64:0/1473233794 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f815ffff700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816006de00 0x7f816015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165284700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f816015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 -- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 msgr2=0x7f816015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f816015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 -- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816006de00 msgr2=0x7f816015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816006de00 0x7f816015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 -- 172.21.15.64:0/1473233794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580094f0 con 0x7f816006c7c0 2023-09-27T23:14:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f815ffff700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816006de00 0x7f816015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8164a83700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816016c660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f815000e950 tx=0x7f81500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815000c850 con 0x7f816006c7c0 2023-09-27T23:14:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815000c9b0 con 0x7f816006c7c0 2023-09-27T23:14:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815000cdb0 con 0x7f816006c7c0 2023-09-27T23:14:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816015d480 con 0x7f816006c7c0 2023-09-27T23:14:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.554+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81601467a0 con 0x7f816006c7c0 2023-09-27T23:14:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.558+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816004be40 con 0x7f816006c7c0 2023-09-27T23:14:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.558+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 80259+0+0 (secure 0 0 0) 0x7f8150013680 con 0x7f816006c7c0 2023-09-27T23:14:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.558+0000 7f815cff9700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f814c060950 0x7f814c062e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.558+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f815007d200 con 0x7f816006c7c0 2023-09-27T23:14:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.558+0000 7f815ffff700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f814c060950 0x7f814c062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.558+0000 7f815ffff700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f814c060950 0x7f814c062e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f816015a820 tx=0x7f8154009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.566+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f815004d340 con 0x7f816006c7c0 2023-09-27T23:14:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.762+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f81600c61f0 con 0x7f816006c7c0 2023-09-27T23:14:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.930+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 6) v1 ==== 80276+0+0 (secure 0 0 0) 0x7f815004d710 con 0x7f816006c7c0 2023-09-27T23:14:56.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.934+0000 7f815cff9700 1 -- 172.21.15.64:0/1473233794 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8150050dd0 con 0x7f816006c7c0 2023-09-27T23:14:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f814c060950 msgr2=0x7f814c062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f814c060950 0x7f814c062e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f816015a820 tx=0x7f8154009040 comp rx=0 tx=0).stop 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 msgr2=0x7f816016c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816016c660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f815000e950 tx=0x7f81500091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 shutdown_connections 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f814c060950 0x7f814c062e00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81600c69a0 0x7f816015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816006de00 0x7f816015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 --2- 172.21.15.64:0/1473233794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f816006c7c0 0x7f816016c660 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 >> 172.21.15.64:0/1473233794 conn(0x7f8160067b50 msgr2=0x7f816015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 shutdown_connections 2023-09-27T23:14:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:56.938+0000 7f8165a85700 1 -- 172.21.15.64:0/1473233794 wait complete. 2023-09-27T23:14:56.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:14:57.052 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:57.048+0000 7f808f72e000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:14:57.053 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:57.050+0000 7fe31daf9000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:14:57.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.062+0000 7f6751b49700 1 Processor -- start 2023-09-27T23:14:57.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.062+0000 7f6751b49700 1 -- start start 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f674c06b2c0 0x7f674c06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f674c06d210 0x7f674c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c06c7c0 con 0x7f674c0afbd0 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c0b0760 con 0x7f674c06d210 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c06cd80 con 0x7f674c06b2c0 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f674c06d210 0x7f674c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f674c06d210 0x7f674c06d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55188/0 (socket says 172.21.15.64:55188) 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 -- 172.21.15.64:0/1694775497 learned_addr learned my addr 172.21.15.64:0/1694775497 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:57.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674bfff700 1 --1- 172.21.15.64:0/1694775497 >> v1:172.21.15.64:6789/0 conn(0x7f674c0afbd0 0x7f674c0b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50632/0 (socket says 172.21.15.64:50632) 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674b7fe700 1 --2- 172.21.15.64:0/1694775497 >> v2:172.21.15.123:3300/0 conn(0x7f674c06b2c0 0x7f674c06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 -- 172.21.15.64:0/1694775497 >> v2:172.21.15.123:3300/0 conn(0x7f674c06b2c0 msgr2=0x7f674c06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 --2- 172.21.15.64:0/1694775497 >> v2:172.21.15.123:3300/0 conn(0x7f674c06b2c0 0x7f674c06b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 -- 172.21.15.64:0/1694775497 >> v1:172.21.15.64:6789/0 conn(0x7f674c0afbd0 legacy=0x7f674c0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 -- 172.21.15.64:0/1694775497 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0b0940 con 0x7f674c06d210 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674affd700 1 --2- 172.21.15.64:0/1694775497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06d210 0x7f674c06d680 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f674c06aaa0 tx=0x7f6740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a6e2d60a53d84d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674a7fc700 1 -- 172.21.15.64:0/1694775497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674000b860 con 0x7f674c06d210 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674a7fc700 1 -- 172.21.15.64:0/1694775497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674000b9c0 con 0x7f674c06d210 2023-09-27T23:14:57.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f674a7fc700 1 -- 172.21.15.64:0/1694775497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674000bd60 con 0x7f674c06d210 2023-09-27T23:14:57.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- 172.21.15.64:0/1694775497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06d210 msgr2=0x7f674c06d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 --2- 172.21.15.64:0/1694775497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06d210 0x7f674c06d680 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f674c06aaa0 tx=0x7f6740009250 comp rx=0 tx=0).stop 2023-09-27T23:14:57.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- 172.21.15.64:0/1694775497 shutdown_connections 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 --2- 172.21.15.64:0/1694775497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06d210 0x7f674c06d680 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 --2- 172.21.15.64:0/1694775497 >> v2:172.21.15.123:3300/0 conn(0x7f674c06b2c0 0x7f674c06b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- 172.21.15.64:0/1694775497 >> 172.21.15.64:0/1694775497 conn(0x7f674c0cc580 msgr2=0x7f674c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- 172.21.15.64:0/1694775497 shutdown_connections 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.066+0000 7f6751b49700 1 -- 172.21.15.64:0/1694775497 wait complete. 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 Processor -- start 2023-09-27T23:14:57.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 -- start start 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 0x7f674c14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f674c06d210 0x7f674c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f674c0afbd0 0x7f674c15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c15d3f0 con 0x7f674c06b2c0 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c15c540 con 0x7f674c0afbd0 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c15c7e0 con 0x7f674c06d210 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 0x7f674c14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 0x7f674c14e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55194/0 (socket says 172.21.15.64:55194) 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 -- 172.21.15.64:0/1987067994 learned_addr learned my addr 172.21.15.64:0/1987067994 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:57.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674affd700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f674c06d210 0x7f674c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674bfff700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f674c0afbd0 0x7f674c15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 -- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f674c06d210 msgr2=0x7f674c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f674c06d210 0x7f674c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 -- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f674c0afbd0 msgr2=0x7f674c15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f674c0afbd0 0x7f674c15f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 -- 172.21.15.64:0/1987067994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400094f0 con 0x7f674c06b2c0 2023-09-27T23:14:57.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f674b7fe700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 0x7f674c14e590 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f673c00d890 tx=0x7f673c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:57.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0125f0 con 0x7f674c06b2c0 2023-09-27T23:14:57.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c012750 con 0x7f674c06b2c0 2023-09-27T23:14:57.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c15cac0 con 0x7f674c06b2c0 2023-09-27T23:14:57.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c012af0 con 0x7f674c06b2c0 2023-09-27T23:14:57.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.070+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c15d080 con 0x7f674c06b2c0 2023-09-27T23:14:57.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.074+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c0aee60 con 0x7f674c06b2c0 2023-09-27T23:14:57.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.074+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 80276+0+0 (secure 0 0 0) 0x7f673c01e460 con 0x7f674c06b2c0 2023-09-27T23:14:57.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.074+0000 7f67337fe700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6738060a70 0x7f6738062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.074+0000 7f674affd700 1 -- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6738060a70 msgr2=0x7f6738062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:14:57.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.074+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f673c07d5b0 con 0x7f674c06b2c0 2023-09-27T23:14:57.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.074+0000 7f674affd700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6738060a70 0x7f6738062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:14:57.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.086+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f674c0aee60 con 0x7f674c06b2c0 2023-09-27T23:14:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.250+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f674c0aee60 con 0x7f674c06b2c0 2023-09-27T23:14:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.258+0000 7f67337fe700 1 -- 172.21.15.64:0/1987067994 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f674c0aee60 con 0x7f674c06b2c0 2023-09-27T23:14:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6738060a70 msgr2=0x7f6738062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:14:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6738060a70 0x7f6738062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 msgr2=0x7f674c14e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 0x7f674c14e590 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f673c00d890 tx=0x7f673c008040 comp rx=0 tx=0).stop 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 shutdown_connections 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6738060a70 0x7f6738062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f674c0afbd0 0x7f674c15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f674c06d210 0x7f674c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 --2- 172.21.15.64:0/1987067994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f674c06b2c0 0x7f674c14e590 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 >> 172.21.15.64:0/1987067994 conn(0x7f674c0cc580 msgr2=0x7f674c150110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 shutdown_connections 2023-09-27T23:14:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.262+0000 7f6751b49700 1 -- 172.21.15.64:0/1987067994 wait complete. 2023-09-27T23:14:57.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:14:57.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 Processor -- start 2023-09-27T23:14:57.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 -- start start 2023-09-27T23:14:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f889c159fb0 0x7f889c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c15a3f0 con 0x7f889c0ab870 2023-09-27T23:14:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c0b0e40 con 0x7f889c159fb0 2023-09-27T23:14:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a38d0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0b1120 con 0x7f889c0b08d0 2023-09-27T23:14:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a166c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f889c0ab870 0x7f889c0afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50638/0 (socket says 172.21.15.64:50638) 2023-09-27T23:14:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a166c700 1 -- 172.21.15.64:0/3376513305 learned_addr learned my addr 172.21.15.64:0/3376513305 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:14:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a1e6d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f889c159fb0 0x7f889c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.386+0000 7f88a0e6b700 1 --2- 172.21.15.64:0/3376513305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a1e6d700 1 -- 172.21.15.64:0/3376513305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 msgr2=0x7f889c159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a1e6d700 1 --2- 172.21.15.64:0/3376513305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a1e6d700 1 -- 172.21.15.64:0/3376513305 >> v1:172.21.15.64:6789/0 conn(0x7f889c0ab870 legacy=0x7f889c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f8893fff700 1 -- 172.21.15.64:0/3376513305 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016476551 0 0) 0x7f889c15a3f0 con 0x7f889c0ab870 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a1e6d700 1 -- 172.21.15.64:0/3376513305 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0b1400 con 0x7f889c159fb0 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a1e6d700 1 --2- 172.21.15.64:0/3376513305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c06c7c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8894009e60 tx=0x7f8894009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770052bc106fe0a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:57.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f8893fff700 1 -- 172.21.15.64:0/3376513305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400e070 con 0x7f889c159fb0 2023-09-27T23:14:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f8893fff700 1 -- 172.21.15.64:0/3376513305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889400bb60 con 0x7f889c159fb0 2023-09-27T23:14:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f8893fff700 1 -- 172.21.15.64:0/3376513305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894012430 con 0x7f889c159fb0 2023-09-27T23:14:57.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 -- 172.21.15.64:0/3376513305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 msgr2=0x7f889c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3376513305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c06c7c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8894009e60 tx=0x7f8894009780 comp rx=0 tx=0).stop 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 -- 172.21.15.64:0/3376513305 shutdown_connections 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3376513305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c06c7c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3376513305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 -- 172.21.15.64:0/3376513305 >> 172.21.15.64:0/3376513305 conn(0x7f889c0cc580 msgr2=0x7f889c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 -- 172.21.15.64:0/3376513305 shutdown_connections 2023-09-27T23:14:57.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 -- 172.21.15.64:0/3376513305 wait complete. 2023-09-27T23:14:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 Processor -- start 2023-09-27T23:14:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.390+0000 7f88a38d0700 1 -- start start 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f889c0ab870 0x7f889c0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1633c0 con 0x7f889c159fb0 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0ae260 con 0x7f889c0ab870 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0ad3b0 con 0x7f889c0b08d0 2023-09-27T23:14:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a1e6d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:51688/0 (socket says 172.21.15.64:51688) 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 -- 172.21.15.64:0/3073272969 learned_addr learned my addr 172.21.15.64:0/3073272969 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a166c700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f889c0ab870 0x7f889c0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 -- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f889c0ab870 msgr2=0x7f889c0ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f889c0ab870 0x7f889c0ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 -- 172.21.15.64:0/3073272969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 msgr2=0x7f889c153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 -- 172.21.15.64:0/3073272969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009a20 con 0x7f889c0b08d0 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a166c700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f889c0ab870 0x7f889c0ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:57.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a0e6b700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c152cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e930 tx=0x7f888c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c013680 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c0137e0 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c0ad650 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c013b80 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c0adc10 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c071120 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 6) v1 ==== 80276+0+0 (secure 0 0 0) 0x7f888c00c460 con 0x7f889c0b08d0 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.394+0000 7f88917fa700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f88800609a0 0x7f8880062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.398+0000 7f88a166c700 1 -- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f88800609a0 msgr2=0x7f8880062e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:14:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.398+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f888c07d260 con 0x7f889c0b08d0 2023-09-27T23:14:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.398+0000 7f88a166c700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f88800609a0 0x7f8880062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:14:57.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.410+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f888c04d300 con 0x7f889c0b08d0 2023-09-27T23:14:57.476 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:57.471+0000 7f808f72e000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:14:57.509 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:57.506+0000 7fe31daf9000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:14:57.523 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:57.519+0000 7f808f72e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:14:57.560 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:57.558+0000 7fe31daf9000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:14:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.570+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f889c0cc2f0 con 0x7f889c0b08d0 2023-09-27T23:14:57.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.582+0000 7f88917fa700 1 -- 172.21.15.64:0/3073272969 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f888c050920 con 0x7f889c0b08d0 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f88800609a0 msgr2=0x7f8880062e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f88800609a0 0x7f8880062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 msgr2=0x7f889c152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c152cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e930 tx=0x7f888c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 shutdown_connections 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f88800609a0 0x7f8880062e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f889c159fb0 0x7f889c153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f889c0b08d0 0x7f889c152cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 --2- 172.21.15.64:0/3073272969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f889c0ab870 0x7f889c0ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 >> 172.21.15.64:0/3073272969 conn(0x7f889c0cc580 msgr2=0x7f889c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 shutdown_connections 2023-09-27T23:14:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.586+0000 7f88a38d0700 1 -- 172.21.15.64:0/3073272969 wait complete. 2023-09-27T23:14:57.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:57.601 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:14:57.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:14:57.614 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-09-27T23:14:57.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-09-27T23:14:57.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:57.623+0000 7f808f72e000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:14:57.675 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:57.670+0000 7fe31daf9000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:14:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 Processor -- start 2023-09-27T23:14:57.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- start start 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45780ae720 0x7f45780acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f457806ced0 0x7f4578159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45780c85d0 con 0x7f4578158ed0 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45780aeb70 con 0x7f45780ae720 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780ae290 con 0x7f457806ced0 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45780ae720 0x7f45780acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45780ae720 0x7f45780acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55222/0 (socket says 172.21.15.64:55222) 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 -- 172.21.15.64:0/2201001351 learned_addr learned my addr 172.21.15.64:0/2201001351 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:57.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4576d9d700 1 --2- 172.21.15.64:0/2201001351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f457806ced0 0x7f4578159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 -- 172.21.15.64:0/2201001351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f457806ced0 msgr2=0x7f4578159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 --2- 172.21.15.64:0/2201001351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f457806ced0 0x7f4578159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 -- 172.21.15.64:0/2201001351 >> v1:172.21.15.123:6789/0 conn(0x7f4578158ed0 legacy=0x7f457806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 -- 172.21.15.64:0/2201001351 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780ad430 con 0x7f45780ae720 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f4575d9b700 1 --2- 172.21.15.64:0/2201001351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45780ae720 0x7f45780acef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f456c0099b0 tx=0x7f456c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a945c9dda47b6d59 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457559a700 1 -- 172.21.15.64:0/2201001351 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c009ea0 con 0x7f45780ae720 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457559a700 1 -- 172.21.15.64:0/2201001351 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c00b860 con 0x7f45780ae720 2023-09-27T23:14:57.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457559a700 1 -- 172.21.15.64:0/2201001351 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c00bc00 con 0x7f45780ae720 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- 172.21.15.64:0/2201001351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45780ae720 msgr2=0x7f45780acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 --2- 172.21.15.64:0/2201001351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45780ae720 0x7f45780acef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f456c0099b0 tx=0x7f456c009250 comp rx=0 tx=0).stop 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- 172.21.15.64:0/2201001351 shutdown_connections 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 --2- 172.21.15.64:0/2201001351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f457806ced0 0x7f4578159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 --2- 172.21.15.64:0/2201001351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45780ae720 0x7f45780acef0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- 172.21.15.64:0/2201001351 >> 172.21.15.64:0/2201001351 conn(0x7f45780c8c00 msgr2=0x7f45780c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- 172.21.15.64:0/2201001351 shutdown_connections 2023-09-27T23:14:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.734+0000 7f457759e700 1 -- 172.21.15.64:0/2201001351 wait complete. 2023-09-27T23:14:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 Processor -- start 2023-09-27T23:14:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 -- start start 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 0x7f45780ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45780ae720 0x7f45780aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4578158ed0 0x7f4578168b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780af670 con 0x7f457806ced0 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780b15e0 con 0x7f4578158ed0 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780af480 con 0x7f45780ae720 2023-09-27T23:14:57.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 0x7f45780ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 0x7f45780ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55224/0 (socket says 172.21.15.64:55224) 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 -- 172.21.15.64:0/390097387 learned_addr learned my addr 172.21.15.64:0/390097387 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f4575d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45780ae720 0x7f45780aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f4576d9d700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4578158ed0 0x7f4578168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45780ae720 msgr2=0x7f45780aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45780ae720 0x7f45780aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4578158ed0 msgr2=0x7f4578168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4578158ed0 0x7f4578168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 -- 172.21.15.64:0/390097387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0094f0 con 0x7f457806ced0 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f4576d9d700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4578158ed0 0x7f4578168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.738+0000 7f457659c700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 0x7f45780ab4a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f456800c5c0 tx=0x7f4568007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:14:57.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568016070 con 0x7f457806ced0 2023-09-27T23:14:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568006820 con 0x7f457806ced0 2023-09-27T23:14:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45780b2070 con 0x7f457806ced0 2023-09-27T23:14:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568006bc0 con 0x7f457806ced0 2023-09-27T23:14:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45780b17d0 con 0x7f457806ced0 2023-09-27T23:14:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457804be40 con 0x7f457806ced0 2023-09-27T23:14:57.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 6) v1 ==== 80276+0+0 (secure 0 0 0) 0x7f4568006fd0 con 0x7f457806ced0 2023-09-27T23:14:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f45667fc700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:14:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f456807c7d0 con 0x7f457806ced0 2023-09-27T23:14:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f4575d9b700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 msgr2=0x7f4558062e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:14:57.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.742+0000 7f4575d9b700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:14:57.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.750+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f456804c910 con 0x7f457806ced0 2023-09-27T23:14:57.824 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:57.819+0000 7f808f72e000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:14:57.872 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:57.867+0000 7f808f72e000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:14:57.908 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:57.906+0000 7fe31daf9000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:14:57.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.942+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f457806cc90 con 0x7f45580609a0 2023-09-27T23:14:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.942+0000 7f4575d9b700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 msgr2=0x7f4558062e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:14:57.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:57.942+0000 7f4575d9b700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:14:57.961 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:57.958+0000 7fe31daf9000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:14:58.026 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.019+0000 7f808f72e000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:14:58.081 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.075+0000 7f808f72e000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:14:58.128 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:58.126+0000 7fe31daf9000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:14:58.139 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.131+0000 7f808f72e000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:14:58.191 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:58.186+0000 7fe31daf9000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:14:58.257 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:58.254+0000 7fe31daf9000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:14:58.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:58.346+0000 7f4575d9b700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 msgr2=0x7f4558062e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:14:58.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:58.346+0000 7f4575d9b700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:14:58.432 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.427+0000 7f808f72e000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:14:58.542 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.535+0000 7f808f72e000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:14:58.588 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:58.586+0000 7fe31daf9000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:14:58.713 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:58.710+0000 7fe31daf9000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:14:58.822 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.815+0000 7f808f72e000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:14:58.982 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:58.975+0000 7f808f72e000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:14:59.030 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.026+0000 7fe31daf9000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:14:59.045 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.039+0000 7f808f72e000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:14:59.096 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.091+0000 7f808f72e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:14:59.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:59.146+0000 7f4575d9b700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 msgr2=0x7f4558062e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:14:59.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:14:59.146+0000 7f4575d9b700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:14:59.151 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.143+0000 7f808f72e000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:14:59.198 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.194+0000 7fe31daf9000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:14:59.265 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.262+0000 7fe31daf9000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:14:59.319 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.314+0000 7fe31daf9000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:14:59.320 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.315+0000 7f808f72e000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:14:59.373 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.367+0000 7f808f72e000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:14:59.374 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.370+0000 7fe31daf9000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:14:59.434 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.427+0000 7f808f72e000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:14:59.549 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.546+0000 7fe31daf9000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:14:59.580 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.575+0000 7f808f72e000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:14:59.605 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.602+0000 7fe31daf9000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:14:59.639 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:14:59.631+0000 7f808f72e000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:14:59.668 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.666+0000 7fe31daf9000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:14:59.824 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.822+0000 7fe31daf9000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:14:59.887 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:14:59.882+0000 7fe31daf9000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:15:00.224 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.219+0000 7f808f72e000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:15:00.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:00.258+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 7) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f456804cce0 con 0x7f457806ced0 2023-09-27T23:15:00.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:00.258+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 msgr2=0x7f4558062e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:15:00.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:00.258+0000 7f45667fc700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:00.432 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f80528de700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.432 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f80528de700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.432 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f80528de700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.432 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f804f8d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.433 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f80528de700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.433 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f804f8d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.433 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f80528de700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.433 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f804f8d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.433 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f804f8d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.433 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:15:00.427+0000 7f804f8d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:15:00.522 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:15:00.518+0000 7fe31daf9000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:15:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.262+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 8) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f4568057120 con 0x7f457806ced0 2023-09-27T23:15:01.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.266+0000 7f45667fc700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.266+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f457806cc90 con 0x7f45580609a0 2023-09-27T23:15:01.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.266+0000 7f4575d9b700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.270+0000 7f4575d9b700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f456c0099b0 tx=0x7f456c017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:01.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.390+0000 7f45667fc700 1 -- 172.21.15.64:0/390097387 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f457806cc90 con 0x7f45580609a0 2023-09-27T23:15:01.395 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir0 2023-09-27T23:15:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 msgr2=0x7f4558062e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f456c0099b0 tx=0x7f456c017000 comp rx=0 tx=0).stop 2023-09-27T23:15:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 msgr2=0x7f45780ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 0x7f45780ab4a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f456800c5c0 tx=0x7f4568007f50 comp rx=0 tx=0).stop 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 shutdown_connections 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f45580609a0 0x7f4558062e50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4578158ed0 0x7f4578168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45780ae720 0x7f45780aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 --2- 172.21.15.64:0/390097387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f457806ced0 0x7f45780ab4a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 >> 172.21.15.64:0/390097387 conn(0x7f45780c8c00 msgr2=0x7f45780c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 shutdown_connections 2023-09-27T23:15:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.394+0000 7f457759e700 1 -- 172.21.15.64:0/390097387 wait complete. 2023-09-27T23:15:01.417 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-09-27T23:15:01.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:01.418 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-09-27T23:15:01.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-09-27T23:15:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 Processor -- start 2023-09-27T23:15:01.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 -- start start 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31c80cb130 0x7f31c80cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31c815c930 0x7f31c806aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c80b0ac0 con 0x7f31c806a6a0 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c806b030 con 0x7f31c80cb130 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31cdf56700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c806b210 con 0x7f31c815c930 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31c77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31c80cb130 0x7f31c80cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31c77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31c80cb130 0x7f31c80cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55234/0 (socket says 172.21.15.64:55234) 2023-09-27T23:15:01.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31c77fe700 1 -- 172.21.15.64:0/2117499733 learned_addr learned my addr 172.21.15.64:0/2117499733 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:01.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31c7fff700 1 --2- 172.21.15.64:0/2117499733 >> v2:172.21.15.123:3300/0 conn(0x7f31c815c930 0x7f31c806aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31c77fe700 1 -- 172.21.15.64:0/2117499733 >> v2:172.21.15.123:3300/0 conn(0x7f31c815c930 msgr2=0x7f31c806aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.546+0000 7f31c77fe700 1 --2- 172.21.15.64:0/2117499733 >> v2:172.21.15.123:3300/0 conn(0x7f31c815c930 0x7f31c806aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31c77fe700 1 -- 172.21.15.64:0/2117499733 >> v1:172.21.15.123:6789/0 conn(0x7f31c806a6a0 legacy=0x7f31c815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31c77fe700 1 -- 172.21.15.64:0/2117499733 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c80ca810 con 0x7f31c80cb130 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31c77fe700 1 --2- 172.21.15.64:0/2117499733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cb500 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f31c80cfc60 tx=0x7f31b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bd21b73cee6482 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31c67fc700 1 -- 172.21.15.64:0/2117499733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0009e70 con 0x7f31c80cb130 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31c67fc700 1 -- 172.21.15.64:0/2117499733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b000b860 con 0x7f31c80cb130 2023-09-27T23:15:01.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31c67fc700 1 -- 172.21.15.64:0/2117499733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b000bc00 con 0x7f31c80cb130 2023-09-27T23:15:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 -- 172.21.15.64:0/2117499733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 msgr2=0x7f31c80cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 --2- 172.21.15.64:0/2117499733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cb500 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f31c80cfc60 tx=0x7f31b0009250 comp rx=0 tx=0).stop 2023-09-27T23:15:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 -- 172.21.15.64:0/2117499733 shutdown_connections 2023-09-27T23:15:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 --2- 172.21.15.64:0/2117499733 >> v2:172.21.15.123:3300/0 conn(0x7f31c815c930 0x7f31c806aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 --2- 172.21.15.64:0/2117499733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cb500 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 -- 172.21.15.64:0/2117499733 >> 172.21.15.64:0/2117499733 conn(0x7f31c8067bb0 msgr2=0x7f31c80cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 -- 172.21.15.64:0/2117499733 shutdown_connections 2023-09-27T23:15:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 -- 172.21.15.64:0/2117499733 wait complete. 2023-09-27T23:15:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 Processor -- start 2023-09-27T23:15:01.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.550+0000 7f31cdf56700 1 -- start start 2023-09-27T23:15:01.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31c806a6a0 0x7f31c80c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c815c930 0x7f31c80cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c81435b0 con 0x7f31c80cb130 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8146220 con 0x7f31c806a6a0 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c81466b0 con 0x7f31c815c930 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55238/0 (socket says 172.21.15.64:55238) 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31c806a6a0 0x7f31c80c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 -- 172.21.15.64:0/3151925973 learned_addr learned my addr 172.21.15.64:0/3151925973 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c7fff700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c815c930 0x7f31c80cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 -- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c815c930 msgr2=0x7f31c80cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c815c930 0x7f31c80cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 -- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31c806a6a0 msgr2=0x7f31c80c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31c806a6a0 0x7f31c80c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 -- 172.21.15.64:0/3151925973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00094f0 con 0x7f31c80cb130 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31c6ffd700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cefc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f31b800c4e0 tx=0x7f31b800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8011590 con 0x7f31c80cb130 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b8011710 con 0x7f31c80cb130 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c8145610 con 0x7f31c80cb130 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8011ad0 con 0x7f31c80cb130 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.554+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c8145bd0 con 0x7f31c80cb130 2023-09-27T23:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.558+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c81463d0 con 0x7f31c80cb130 2023-09-27T23:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.558+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f31b801b3b0 con 0x7f31c80cb130 2023-09-27T23:15:01.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.558+0000 7f31bf7fe700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f31a80607f0 0x7f31a8062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:01.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.558+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f31b807c390 con 0x7f31c80cb130 2023-09-27T23:15:01.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.558+0000 7f31c77fe700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f31a80607f0 0x7f31a8062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:01.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.562+0000 7f31c77fe700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f31a80607f0 0x7f31a8062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31c80cfc60 tx=0x7f31b0017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:01.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.566+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f31b804c700 con 0x7f31c80cb130 2023-09-27T23:15:01.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.750+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f31c8071150 con 0x7f31a80607f0 2023-09-27T23:15:01.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.882+0000 7f31bf7fe700 1 -- 172.21.15.64:0/3151925973 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f31c8071150 con 0x7f31a80607f0 2023-09-27T23:15:01.887 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir1 2023-09-27T23:15:01.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f31a80607f0 msgr2=0x7f31a8062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f31a80607f0 0x7f31a8062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31c80cfc60 tx=0x7f31b0017000 comp rx=0 tx=0).stop 2023-09-27T23:15:01.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 msgr2=0x7f31c80cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:01.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cefc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f31b800c4e0 tx=0x7f31b800c7f0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 shutdown_connections 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f31a80607f0 0x7f31a8062ca0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c815c930 0x7f31c80cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31c80cb130 0x7f31c80cefc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 --2- 172.21.15.64:0/3151925973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31c806a6a0 0x7f31c80c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 >> 172.21.15.64:0/3151925973 conn(0x7f31c8067bb0 msgr2=0x7f31c81579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 shutdown_connections 2023-09-27T23:15:01.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:01.886+0000 7f31cdf56700 1 -- 172.21.15.64:0/3151925973 wait complete. 2023-09-27T23:15:01.907 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-09-27T23:15:01.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:01.908 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-09-27T23:15:01.918 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-09-27T23:15:02.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.034+0000 7fb11f2bf700 1 Processor -- start 2023-09-27T23:15:02.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.034+0000 7fb11f2bf700 1 -- start start 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1180aef60 0x7fb118156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11806beb0 con 0x7fb1181569e0 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11806cb80 con 0x7fb1180adf40 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11806ce60 con 0x7fb1180aef60 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1180aef60 0x7fb118156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1180aef60 0x7fb118156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55246/0 (socket says 172.21.15.64:55246) 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 -- 172.21.15.64:0/1102280363 learned_addr learned my addr 172.21.15.64:0/1102280363 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:02.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11e2bd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb1181569e0 0x7fb1180b1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50648/0 (socket says 172.21.15.64:50648) 2023-09-27T23:15:02.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 -- 172.21.15.64:0/1102280363 >> v1:172.21.15.123:6789/0 conn(0x7fb1180adf40 legacy=0x7fb1180ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 -- 172.21.15.64:0/1102280363 >> v1:172.21.15.64:6789/0 conn(0x7fb1181569e0 legacy=0x7fb1180b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 -- 172.21.15.64:0/1102280363 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11806d140 con 0x7fb1180aef60 2023-09-27T23:15:02.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11dabc700 1 --2- 172.21.15.64:0/1102280363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180aef60 0x7fb118156db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb1080099b0 tx=0x7fb108009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b63585441190ac server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11d2bb700 1 -- 172.21.15.64:0/1102280363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800b860 con 0x7fb1180aef60 2023-09-27T23:15:02.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11d2bb700 1 -- 172.21.15.64:0/1102280363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800b9c0 con 0x7fb1180aef60 2023-09-27T23:15:02.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11d2bb700 1 -- 172.21.15.64:0/1102280363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800bd60 con 0x7fb1180aef60 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1102280363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180aef60 msgr2=0x7fb118156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1102280363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180aef60 0x7fb118156db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb1080099b0 tx=0x7fb108009250 comp rx=0 tx=0).stop 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1102280363 shutdown_connections 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1102280363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180aef60 0x7fb118156db0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1102280363 >> 172.21.15.64:0/1102280363 conn(0x7fb1180c8c00 msgr2=0x7fb1180c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1102280363 shutdown_connections 2023-09-27T23:15:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.038+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1102280363 wait complete. 2023-09-27T23:15:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 Processor -- start 2023-09-27T23:15:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 -- start start 2023-09-27T23:15:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 0x7fb118154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1180aef60 0x7fb118151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1181569e0 0x7fb118157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb118152090 con 0x7fb1180adf40 2023-09-27T23:15:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180c4400 con 0x7fb1181569e0 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11f2bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb118155c70 con 0x7fb1180aef60 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 0x7fb118154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 0x7fb118154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55254/0 (socket says 172.21.15.64:55254) 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 -- 172.21.15.64:0/1073298046 learned_addr learned my addr 172.21.15.64:0/1073298046 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11dabc700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1180aef60 0x7fb118151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11eabe700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1181569e0 0x7fb118157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 -- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1180aef60 msgr2=0x7fb118151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1180aef60 0x7fb118151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 -- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1181569e0 msgr2=0x7fb118157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1181569e0 0x7fb118157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 -- 172.21.15.64:0/1073298046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080094f0 con 0x7fb1180adf40 2023-09-27T23:15:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb11e2bd700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 0x7fb118154d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c560 tx=0x7fb114007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.042+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114005a70 con 0x7fb1180adf40 2023-09-27T23:15:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114005bd0 con 0x7fb1180adf40 2023-09-27T23:15:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1181553d0 con 0x7fb1180adf40 2023-09-27T23:15:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1140070d0 con 0x7fb1180adf40 2023-09-27T23:15:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb118155990 con 0x7fb1180adf40 2023-09-27T23:15:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11804be40 con 0x7fb1180adf40 2023-09-27T23:15:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fb114007480 con 0x7fb1180adf40 2023-09-27T23:15:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb10e7fc700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb0fc0607f0 0x7fb0fc062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.046+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fb11407c310 con 0x7fb1180adf40 2023-09-27T23:15:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.050+0000 7fb11dabc700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb0fc0607f0 0x7fb0fc062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.050+0000 7fb11dabc700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb0fc0607f0 0x7fb0fc062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1080099b0 tx=0x7fb108017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.054+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb11404c600 con 0x7fb1180adf40 2023-09-27T23:15:02.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.234+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb11806d420 con 0x7fb0fc0607f0 2023-09-27T23:15:02.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.342+0000 7fb10e7fc700 1 -- 172.21.15.64:0/1073298046 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb11806d420 con 0x7fb0fc0607f0 2023-09-27T23:15:02.346 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir2 2023-09-27T23:15:02.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb0fc0607f0 msgr2=0x7fb0fc062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb0fc0607f0 0x7fb0fc062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1080099b0 tx=0x7fb108017000 comp rx=0 tx=0).stop 2023-09-27T23:15:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 msgr2=0x7fb118154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 0x7fb118154d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c560 tx=0x7fb114007f70 comp rx=0 tx=0).stop 2023-09-27T23:15:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 shutdown_connections 2023-09-27T23:15:02.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb0fc0607f0 0x7fb0fc062ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1181569e0 0x7fb118157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb1180aef60 0x7fb118151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 --2- 172.21.15.64:0/1073298046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1180adf40 0x7fb118154d40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 >> 172.21.15.64:0/1073298046 conn(0x7fb1180c8c00 msgr2=0x7fb1181523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:02.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 shutdown_connections 2023-09-27T23:15:02.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.346+0000 7fb11f2bf700 1 -- 172.21.15.64:0/1073298046 wait complete. 2023-09-27T23:15:02.370 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-09-27T23:15:02.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:02.370 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-09-27T23:15:02.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-09-27T23:15:02.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 Processor -- start 2023-09-27T23:15:02.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 -- start start 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f484806ed80 0x7f484806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f484806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484815d720 con 0x7f4848158da0 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48480ca290 con 0x7f484806ed80 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484d1af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48480ca4a0 con 0x7f484806df10 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f4847fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4848158da0 0x7f484806e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50650/0 (socket says 172.21.15.64:50650) 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f4847fff700 1 -- 172.21.15.64:0/1115149652 learned_addr learned my addr 172.21.15.64:0/1115149652 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f48477fe700 1 --2- 172.21.15.64:0/1115149652 >> v2:172.21.15.123:3300/0 conn(0x7f484806ed80 0x7f484806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484c9ae700 1 --2- 172.21.15.64:0/1115149652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f484806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f4846ffd700 1 -- 172.21.15.64:0/1115149652 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061493687 0 0) 0x7f484815d720 con 0x7f4848158da0 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f4846ffd700 1 -- 172.21.15.64:0/1115149652 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4834003570 con 0x7f4848158da0 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484c9ae700 1 -- 172.21.15.64:0/1115149652 >> v2:172.21.15.123:3300/0 conn(0x7f484806ed80 msgr2=0x7f484806a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484c9ae700 1 --2- 172.21.15.64:0/1115149652 >> v2:172.21.15.123:3300/0 conn(0x7f484806ed80 0x7f484806a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484c9ae700 1 -- 172.21.15.64:0/1115149652 >> v1:172.21.15.64:6789/0 conn(0x7f4848158da0 legacy=0x7f484806e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.498+0000 7f484c9ae700 1 -- 172.21.15.64:0/1115149652 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480ca780 con 0x7f484806df10 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484c9ae700 1 --2- 172.21.15.64:0/1115149652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f484806ad10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48400098e0 tx=0x7f48400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f07be410eacb45 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f4846ffd700 1 -- 172.21.15.64:0/1115149652 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840011630 con 0x7f484806df10 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f4846ffd700 1 -- 172.21.15.64:0/1115149652 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4840011790 con 0x7f484806df10 2023-09-27T23:15:02.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f4846ffd700 1 -- 172.21.15.64:0/1115149652 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840011b30 con 0x7f484806df10 2023-09-27T23:15:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 -- 172.21.15.64:0/1115149652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 msgr2=0x7f484806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 --2- 172.21.15.64:0/1115149652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f484806ad10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48400098e0 tx=0x7f48400091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 -- 172.21.15.64:0/1115149652 shutdown_connections 2023-09-27T23:15:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 --2- 172.21.15.64:0/1115149652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f484806ad10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 --2- 172.21.15.64:0/1115149652 >> v2:172.21.15.123:3300/0 conn(0x7f484806ed80 0x7f484806a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 -- 172.21.15.64:0/1115149652 >> 172.21.15.64:0/1115149652 conn(0x7f48480cc760 msgr2=0x7f48480ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 -- 172.21.15.64:0/1115149652 shutdown_connections 2023-09-27T23:15:02.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.502+0000 7f484d1af700 1 -- 172.21.15.64:0/1115149652 wait complete. 2023-09-27T23:15:02.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 Processor -- start 2023-09-27T23:15:02.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 -- start start 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f48481544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 0x7f48480b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4848158da0 0x7f48480b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c9530 con 0x7f484806ed80 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c8480 con 0x7f4848158da0 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48480b4460 con 0x7f484806df10 2023-09-27T23:15:02.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 0x7f48480b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 0x7f48480b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55256/0 (socket says 172.21.15.64:55256) 2023-09-27T23:15:02.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 -- 172.21.15.64:0/2780328366 learned_addr learned my addr 172.21.15.64:0/2780328366 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:02.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f4847fff700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f48481544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484c9ae700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4848158da0 0x7f48480b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 -- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 msgr2=0x7f48481544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f48481544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 -- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4848158da0 msgr2=0x7f48480b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4848158da0 0x7f48480b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 -- 172.21.15.64:0/2780328366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840009420 con 0x7f484806ed80 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f48477fe700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 0x7f48480b38c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f483000ec80 tx=0x7f48300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000c850 con 0x7f484806ed80 2023-09-27T23:15:02.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000c9b0 con 0x7f484806ed80 2023-09-27T23:15:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48480c8c00 con 0x7f484806ed80 2023-09-27T23:15:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000cd50 con 0x7f484806ed80 2023-09-27T23:15:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.506+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48480c91c0 con 0x7f484806ed80 2023-09-27T23:15:02.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.510+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48480c96a0 con 0x7f484806ed80 2023-09-27T23:15:02.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.510+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f4830013680 con 0x7f484806ed80 2023-09-27T23:15:02.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.510+0000 7f483ffff700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f48280607f0 0x7f4828062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.510+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f483007d0a0 con 0x7f484806ed80 2023-09-27T23:15:02.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.510+0000 7f4847fff700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f48280607f0 0x7f4828062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.514+0000 7f4847fff700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f48280607f0 0x7f4828062ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4838005c50 tx=0x7f4838009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.522+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f483004d2c0 con 0x7f484806ed80 2023-09-27T23:15:02.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.698+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f484804be40 con 0x7f48280607f0 2023-09-27T23:15:02.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.826+0000 7f483ffff700 1 -- 172.21.15.64:0/2780328366 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f484804be40 con 0x7f48280607f0 2023-09-27T23:15:02.829 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir3 2023-09-27T23:15:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f48280607f0 msgr2=0x7f4828062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f48280607f0 0x7f4828062ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4838005c50 tx=0x7f4838009040 comp rx=0 tx=0).stop 2023-09-27T23:15:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 msgr2=0x7f48480b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 0x7f48480b38c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f483000ec80 tx=0x7f48300091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 shutdown_connections 2023-09-27T23:15:02.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f48280607f0 0x7f4828062ca0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4848158da0 0x7f48480b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484806ed80 0x7f48480b38c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 --2- 172.21.15.64:0/2780328366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484806df10 0x7f48481544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 >> 172.21.15.64:0/2780328366 conn(0x7f48480cc760 msgr2=0x7f48480ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 shutdown_connections 2023-09-27T23:15:02.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.830+0000 7f484d1af700 1 -- 172.21.15.64:0/2780328366 wait complete. 2023-09-27T23:15:02.851 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-09-27T23:15:02.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:02.852 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-09-27T23:15:02.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-09-27T23:15:02.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 Processor -- start 2023-09-27T23:15:02.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 -- start start 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f40aedc0 0x7f51f40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51f40cac50 0x7f51f40af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f40cb3f0 con 0x7f51f406af50 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f40cc140 con 0x7f51f40cac50 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51fa064700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f406d220 con 0x7f51f40aedc0 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f8861700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f51f406af50 0x7f51f406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50652/0 (socket says 172.21.15.64:50652) 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f8861700 1 -- 172.21.15.64:0/2483781114 learned_addr learned my addr 172.21.15.64:0/2483781114 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f9863700 1 --2- 172.21.15.64:0/2483781114 >> v2:172.21.15.123:3300/0 conn(0x7f51f40cac50 0x7f51f40af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f9062700 1 --2- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f40aedc0 0x7f51f40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51ebfff700 1 -- 172.21.15.64:0/2483781114 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943931717 0 0) 0x7f51f40cb3f0 con 0x7f51f406af50 2023-09-27T23:15:02.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51ebfff700 1 -- 172.21.15.64:0/2483781114 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e0003570 con 0x7f51f406af50 2023-09-27T23:15:02.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f9863700 1 -- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f40aedc0 msgr2=0x7f51f40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f9863700 1 --2- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f40aedc0 0x7f51f40af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f9863700 1 -- 172.21.15.64:0/2483781114 >> v1:172.21.15.64:6789/0 conn(0x7f51f406af50 legacy=0x7f51f406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.986+0000 7f51f9863700 1 -- 172.21.15.64:0/2483781114 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f406b3a0 con 0x7f51f40cac50 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51f9863700 1 --2- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f40af930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009e60 tx=0x7f51ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf7b80428c582df9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51ebfff700 1 -- 172.21.15.64:0/2483781114 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00e070 con 0x7f51f40cac50 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51ebfff700 1 -- 172.21.15.64:0/2483781114 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00bab0 con 0x7f51f40cac50 2023-09-27T23:15:02.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51ebfff700 1 -- 172.21.15.64:0/2483781114 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00be70 con 0x7f51f40cac50 2023-09-27T23:15:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 -- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 msgr2=0x7f51f40af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 --2- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f40af930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009e60 tx=0x7f51ec009780 comp rx=0 tx=0).stop 2023-09-27T23:15:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 -- 172.21.15.64:0/2483781114 shutdown_connections 2023-09-27T23:15:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 --2- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f40af930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 --2- 172.21.15.64:0/2483781114 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f40aedc0 0x7f51f40af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 -- 172.21.15.64:0/2483781114 >> 172.21.15.64:0/2483781114 conn(0x7f51f40cc760 msgr2=0x7f51f40ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:02.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 -- 172.21.15.64:0/2483781114 shutdown_connections 2023-09-27T23:15:02.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.990+0000 7f51fa064700 1 -- 172.21.15.64:0/2483781114 wait complete. 2023-09-27T23:15:02.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 Processor -- start 2023-09-27T23:15:02.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 -- start start 2023-09-27T23:15:02.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f406af50 0x7f51f415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 0x7f51f415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f415e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:02.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c9870 con 0x7f51f40aedc0 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c4960 con 0x7f51f40cac50 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c3d50 con 0x7f51f406af50 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 0x7f51f415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 0x7f51f415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55262/0 (socket says 172.21.15.64:55262) 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 -- 172.21.15.64:0/3375476478 learned_addr learned my addr 172.21.15.64:0/3375476478 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f9062700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f406af50 0x7f51f415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f9863700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:02.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 -- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f406af50 msgr2=0x7f51f415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:02.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f406af50 0x7f51f415a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:02.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 -- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 msgr2=0x7f51f415e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f415e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 -- 172.21.15.64:0/3375476478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009a20 con 0x7f51f40aedc0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51f8861700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 0x7f51f415e110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c7c0 tx=0x7f51e4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4005b30 con 0x7f51f40aedc0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4005c90 con 0x7f51f40aedc0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f40c4030 con 0x7f51f40aedc0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4007300 con 0x7f51f40aedc0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.994+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f40c45f0 con 0x7f51f40aedc0 2023-09-27T23:15:03.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.998+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f40c4b10 con 0x7f51f40aedc0 2023-09-27T23:15:03.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.998+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f51e401b3b0 con 0x7f51f40aedc0 2023-09-27T23:15:03.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.998+0000 7f51e97fa700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51d80607f0 0x7f51d8062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.998+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f51e407c410 con 0x7f51f40aedc0 2023-09-27T23:15:03.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.998+0000 7f51f9062700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51d80607f0 0x7f51d8062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:02.998+0000 7f51f9062700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51d80607f0 0x7f51d8062ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009db0 tx=0x7f51f00091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.006+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f51f40c4b10 con 0x7f51f40aedc0 2023-09-27T23:15:03.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.186+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51f40c4ad0 con 0x7f51d80607f0 2023-09-27T23:15:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.274+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f51e404ca00 con 0x7f51f40aedc0 2023-09-27T23:15:03.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.298+0000 7f51e97fa700 1 -- 172.21.15.64:0/3375476478 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f51f40c4ad0 con 0x7f51d80607f0 2023-09-27T23:15:03.301 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir4 2023-09-27T23:15:03.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51d80607f0 msgr2=0x7f51d8062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51d80607f0 0x7f51d8062ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009db0 tx=0x7f51f00091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 msgr2=0x7f51f415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 0x7f51f415e110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c7c0 tx=0x7f51e4007e00 comp rx=0 tx=0).stop 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 shutdown_connections 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51d80607f0 0x7f51d8062ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f40cac50 0x7f51f415e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51f40aedc0 0x7f51f415e110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 --2- 172.21.15.64:0/3375476478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51f406af50 0x7f51f415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 >> 172.21.15.64:0/3375476478 conn(0x7f51f40cc760 msgr2=0x7f51f40ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 shutdown_connections 2023-09-27T23:15:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.302+0000 7f51fa064700 1 -- 172.21.15.64:0/3375476478 wait complete. 2023-09-27T23:15:03.321 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-09-27T23:15:03.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:03.322 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-09-27T23:15:03.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-09-27T23:15:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 Processor -- start 2023-09-27T23:15:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 -- start start 2023-09-27T23:15:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7b681558d0 0x7f7b681569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b680abe10 0x7f7b680ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7b680adc80 0x7f7b680c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b68156fa0 con 0x7f7b681558d0 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b680abc60 con 0x7f7b680adc80 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6d711700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680aeba0 con 0x7f7b680abe10 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7b681558d0 0x7f7b681569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7b681558d0 0x7f7b681569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55264/0 (socket says 172.21.15.64:55264) 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 -- 172.21.15.64:0/753491107 learned_addr learned my addr 172.21.15.64:0/753491107 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b6cf10700 1 --2- 172.21.15.64:0/753491107 >> v2:172.21.15.123:3300/0 conn(0x7f7b680adc80 0x7f7b680c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b5ffff700 1 --2- 172.21.15.64:0/753491107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b680abe10 0x7f7b680ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 -- 172.21.15.64:0/753491107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b680abe10 msgr2=0x7f7b680ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 --2- 172.21.15.64:0/753491107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b680abe10 0x7f7b680ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 -- 172.21.15.64:0/753491107 >> v2:172.21.15.123:3300/0 conn(0x7f7b680adc80 msgr2=0x7f7b680c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 --2- 172.21.15.64:0/753491107 >> v2:172.21.15.123:3300/0 conn(0x7f7b680adc80 0x7f7b680c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.454+0000 7f7b67fff700 1 -- 172.21.15.64:0/753491107 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680ac2b0 con 0x7f7b681558d0 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b67fff700 1 --2- 172.21.15.64:0/753491107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b681558d0 0x7f7b681569b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7b680cc960 tx=0x7f7b58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19e59b27faf9c31 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b677fe700 1 -- 172.21.15.64:0/753491107 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5800b640 con 0x7f7b681558d0 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b677fe700 1 -- 172.21.15.64:0/753491107 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5800b7a0 con 0x7f7b681558d0 2023-09-27T23:15:03.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b677fe700 1 -- 172.21.15.64:0/753491107 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5800bb60 con 0x7f7b681558d0 2023-09-27T23:15:03.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 -- 172.21.15.64:0/753491107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b681558d0 msgr2=0x7f7b681569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 --2- 172.21.15.64:0/753491107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b681558d0 0x7f7b681569b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7b680cc960 tx=0x7f7b58009250 comp rx=0 tx=0).stop 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 -- 172.21.15.64:0/753491107 shutdown_connections 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 --2- 172.21.15.64:0/753491107 >> v2:172.21.15.123:3300/0 conn(0x7f7b680adc80 0x7f7b680c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 --2- 172.21.15.64:0/753491107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b680abe10 0x7f7b680ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 --2- 172.21.15.64:0/753491107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b681558d0 0x7f7b681569b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 -- 172.21.15.64:0/753491107 >> 172.21.15.64:0/753491107 conn(0x7f7b6806b5b0 msgr2=0x7f7b6806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 -- 172.21.15.64:0/753491107 shutdown_connections 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 -- 172.21.15.64:0/753491107 wait complete. 2023-09-27T23:15:03.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 Processor -- start 2023-09-27T23:15:03.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.458+0000 7f7b6d711700 1 -- start start 2023-09-27T23:15:03.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 0x7f7b680c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 0x7f7b680c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b681558d0 0x7f7b6815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680c2e90 con 0x7f7b680adc80 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680b0320 con 0x7f7b680abe10 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680c2210 con 0x7f7b681558d0 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 0x7f7b680c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b67fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 0x7f7b680c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b67fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 0x7f7b680c1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:51908/0 (socket says 172.21.15.64:51908) 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b67fff700 1 -- 172.21.15.64:0/3979197161 learned_addr learned my addr 172.21.15.64:0/3979197161 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6cf10700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b681558d0 0x7f7b6815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 0x7f7b680c1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55268/0 (socket says 172.21.15.64:55268) 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 -- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b681558d0 msgr2=0x7f7b6815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b681558d0 0x7f7b6815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 -- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 msgr2=0x7f7b680c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 0x7f7b680c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 -- 172.21.15.64:0/3979197161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580094f0 con 0x7f7b680adc80 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b67fff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 0x7f7b680c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b5ffff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 0x7f7b680c1b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e900 tx=0x7f7b500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50013680 con 0x7f7b680adc80 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b500137e0 con 0x7f7b680adc80 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b680c24f0 con 0x7f7b680adc80 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50013ba0 con 0x7f7b680adc80 2023-09-27T23:15:03.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b680c2ab0 con 0x7f7b680adc80 2023-09-27T23:15:03.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.462+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6804be40 con 0x7f7b680adc80 2023-09-27T23:15:03.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.466+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f7b500253d0 con 0x7f7b680adc80 2023-09-27T23:15:03.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.466+0000 7f7b64ff9700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7b4c060b50 0x7f7b4c063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.466+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f7b5007dc60 con 0x7f7b680adc80 2023-09-27T23:15:03.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.466+0000 7f7b67fff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7b4c060b50 0x7f7b4c063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.466+0000 7f7b67fff700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7b4c060b50 0x7f7b4c063000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b58003080 tx=0x7f7b58017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.474+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7b5004da20 con 0x7f7b680adc80 2023-09-27T23:15:03.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.658+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b680c0230 con 0x7f7b4c060b50 2023-09-27T23:15:03.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.786+0000 7f7b64ff9700 1 -- 172.21.15.64:0/3979197161 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7b680c0230 con 0x7f7b4c060b50 2023-09-27T23:15:03.791 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir5 2023-09-27T23:15:03.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7b4c060b50 msgr2=0x7f7b4c063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7b4c060b50 0x7f7b4c063000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b58003080 tx=0x7f7b58017000 comp rx=0 tx=0).stop 2023-09-27T23:15:03.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 msgr2=0x7f7b680c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 0x7f7b680c1b70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e900 tx=0x7f7b500091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 shutdown_connections 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7b4c060b50 0x7f7b4c063000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b681558d0 0x7f7b6815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b680adc80 0x7f7b680c1b70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 --2- 172.21.15.64:0/3979197161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b680abe10 0x7f7b680c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 >> 172.21.15.64:0/3979197161 conn(0x7f7b6806b5b0 msgr2=0x7f7b68156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 shutdown_connections 2023-09-27T23:15:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.790+0000 7f7b6d711700 1 -- 172.21.15.64:0/3979197161 wait complete. 2023-09-27T23:15:03.812 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-09-27T23:15:03.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:03.812 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-09-27T23:15:03.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-09-27T23:15:03.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.942+0000 7fbbe72ec700 1 Processor -- start 2023-09-27T23:15:03.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.942+0000 7fbbe72ec700 1 -- start start 2023-09-27T23:15:03.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbe00cb130 0x7fbbe00cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbbe015c930 0x7fbbe006aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00b0ac0 con 0x7fbbe006a6a0 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe006b030 con 0x7fbbe015c930 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe006b210 con 0x7fbbe00cb130 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbbe015c930 0x7fbbe006aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbbe015c930 0x7fbbe006aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55272/0 (socket says 172.21.15.64:55272) 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 -- 172.21.15.64:0/3095626956 learned_addr learned my addr 172.21.15.64:0/3095626956 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:03.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5088700 1 --2- 172.21.15.64:0/3095626956 >> v2:172.21.15.123:3300/0 conn(0x7fbbe00cb130 0x7fbbe00cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 -- 172.21.15.64:0/3095626956 >> v2:172.21.15.123:3300/0 conn(0x7fbbe00cb130 msgr2=0x7fbbe00cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 --2- 172.21.15.64:0/3095626956 >> v2:172.21.15.123:3300/0 conn(0x7fbbe00cb130 0x7fbbe00cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 -- 172.21.15.64:0/3095626956 >> v1:172.21.15.123:6789/0 conn(0x7fbbe006a6a0 legacy=0x7fbbe015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 -- 172.21.15.64:0/3095626956 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00ca810 con 0x7fbbe015c930 2023-09-27T23:15:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe5889700 1 --2- 172.21.15.64:0/3095626956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe015c930 0x7fbbe006aaf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00694e0 tx=0x7fbbd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaef9cb086adca5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbd3fff700 1 -- 172.21.15.64:0/3095626956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800e070 con 0x7fbbe015c930 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbd3fff700 1 -- 172.21.15.64:0/3095626956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800ba20 con 0x7fbbe015c930 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbd3fff700 1 -- 172.21.15.64:0/3095626956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80092e0 con 0x7fbbe015c930 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- 172.21.15.64:0/3095626956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe015c930 msgr2=0x7fbbe006aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/3095626956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe015c930 0x7fbbe006aaf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00694e0 tx=0x7fbbd8009840 comp rx=0 tx=0).stop 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- 172.21.15.64:0/3095626956 shutdown_connections 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/3095626956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe015c930 0x7fbbe006aaf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/3095626956 >> v2:172.21.15.123:3300/0 conn(0x7fbbe00cb130 0x7fbbe00cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- 172.21.15.64:0/3095626956 >> 172.21.15.64:0/3095626956 conn(0x7fbbe0067bb0 msgr2=0x7fbbe00cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- 172.21.15.64:0/3095626956 shutdown_connections 2023-09-27T23:15:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.946+0000 7fbbe72ec700 1 -- 172.21.15.64:0/3095626956 wait complete. 2023-09-27T23:15:03.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.950+0000 7fbbe72ec700 1 Processor -- start 2023-09-27T23:15:03.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.950+0000 7fbbe72ec700 1 -- start start 2023-09-27T23:15:03.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe006a6a0 0x7fbbe0144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 0x7fbbe00b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe015c930 0x7fbbe00b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0144c60 con 0x7fbbe00cb130 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0144f40 con 0x7fbbe006a6a0 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0145220 con 0x7fbbe015c930 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 0x7fbbe00b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe5088700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe006a6a0 0x7fbbe0144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 0x7fbbe00b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55280/0 (socket says 172.21.15.64:55280) 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 -- 172.21.15.64:0/4092106196 learned_addr learned my addr 172.21.15.64:0/4092106196 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe5889700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe015c930 0x7fbbe00b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 -- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe015c930 msgr2=0x7fbbe00b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe015c930 0x7fbbe00b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 -- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe006a6a0 msgr2=0x7fbbe0144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe006a6a0 0x7fbbe0144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:03.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 -- 172.21.15.64:0/4092106196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009ae0 con 0x7fbbe00cb130 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe4887700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 0x7fbbe00b3740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c4e0 tx=0x7fbbdc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc011590 con 0x7fbbe00cb130 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc011710 con 0x7fbbe00cb130 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc011ad0 con 0x7fbbe00cb130 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe0145500 con 0x7fbbe00cb130 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.954+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01448d0 con 0x7fbbe00cb130 2023-09-27T23:15:03.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.958+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe0145a00 con 0x7fbbe00cb130 2023-09-27T23:15:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.958+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7fbbdc01b3b0 con 0x7fbbe00cb130 2023-09-27T23:15:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.958+0000 7fbbd17fa700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbbc4060b50 0x7fbbc4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.958+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fbbdc07c880 con 0x7fbbe00cb130 2023-09-27T23:15:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.958+0000 7fbbe5088700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbbc4060b50 0x7fbbc4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:03.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.958+0000 7fbbe5088700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbbc4060b50 0x7fbbc4063000 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009db0 tx=0x7fbbd40091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:03.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:03.966+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbbdc04c5a0 con 0x7fbbe00cb130 2023-09-27T23:15:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.150+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbe01450b0 con 0x7fbbc4060b50 2023-09-27T23:15:04.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.258+0000 7fbbd17fa700 1 -- 172.21.15.64:0/4092106196 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbbe01450b0 con 0x7fbbc4060b50 2023-09-27T23:15:04.263 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir6 2023-09-27T23:15:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.262+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbbc4060b50 msgr2=0x7fbbc4063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.262+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbbc4060b50 0x7fbbc4063000 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009db0 tx=0x7fbbd40091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.262+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 msgr2=0x7fbbe00b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.262+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 0x7fbbe00b3740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c4e0 tx=0x7fbbdc00c7f0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 shutdown_connections 2023-09-27T23:15:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbbc4060b50 0x7fbbc4063000 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe015c930 0x7fbbe00b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe00cb130 0x7fbbe00b3740 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 --2- 172.21.15.64:0/4092106196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe006a6a0 0x7fbbe0144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 >> 172.21.15.64:0/4092106196 conn(0x7fbbe0067bb0 msgr2=0x7fbbe015a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 shutdown_connections 2023-09-27T23:15:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.266+0000 7fbbe72ec700 1 -- 172.21.15.64:0/4092106196 wait complete. 2023-09-27T23:15:04.285 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-09-27T23:15:04.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:04.285 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-09-27T23:15:04.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-09-27T23:15:04.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 Processor -- start 2023-09-27T23:15:04.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 -- start start 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb0c158da0 0x7feb0c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb0c0caca0 0x7feb0c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c15d720 con 0x7feb0c06df10 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca290 con 0x7feb0c158da0 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1286b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca4a0 con 0x7feb0c0caca0 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb11869700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb0c158da0 0x7feb0c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb11068700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7feb0c06df10 0x7feb0c06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50660/0 (socket says 172.21.15.64:50660) 2023-09-27T23:15:04.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb11068700 1 -- 172.21.15.64:0/881962335 learned_addr learned my addr 172.21.15.64:0/881962335 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1206a700 1 --2- 172.21.15.64:0/881962335 >> v2:172.21.15.123:3300/0 conn(0x7feb0c0caca0 0x7feb0c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834512465 0 0) 0x7feb0c15d720 con 0x7feb0c06df10 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf8003570 con 0x7feb0c06df10 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 213015577 0 0) 0x7feaf8003570 con 0x7feb0c06df10 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 >> v2:172.21.15.123:3300/0 conn(0x7feb0c0caca0 msgr2=0x7feb0c06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 --2- 172.21.15.64:0/881962335 >> v2:172.21.15.123:3300/0 conn(0x7feb0c0caca0 0x7feb0c06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 >> v2:172.21.15.64:3300/0 conn(0x7feb0c158da0 msgr2=0x7feb0c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 --2- 172.21.15.64:0/881962335 >> v2:172.21.15.64:3300/0 conn(0x7feb0c158da0 0x7feb0c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0ca780 con 0x7feb0c06df10 2023-09-27T23:15:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb11869700 1 --2- 172.21.15.64:0/881962335 >> v2:172.21.15.64:3300/0 conn(0x7feb0c158da0 0x7feb0c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7feafc001470 con 0x7feb0c06df10 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 >> v1:172.21.15.64:6789/0 conn(0x7feb0c06df10 legacy=0x7feb0c06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf8003a60 0x7feaf8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf8004810 0x7feaf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf80051c0 0x7feaf800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca290 con 0x7feaf8003a60 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca4a0 con 0x7feaf8004810 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf800bfd0 con 0x7feaf80051c0 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb11869700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf8003a60 0x7feaf8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb11068700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf8004810 0x7feaf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.414+0000 7feb1206a700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf80051c0 0x7feaf800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb11068700 1 -- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf80051c0 msgr2=0x7feaf800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb11068700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf80051c0 0x7feaf800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb11068700 1 -- 172.21.15.64:0/881962335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf8003a60 msgr2=0x7feaf8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb11068700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaf8003a60 0x7feaf8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb11068700 1 -- 172.21.15.64:0/881962335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8003570 con 0x7feaf8004810 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1206a700 1 -- 172.21.15.64:0/881962335 reap_dead start 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb11068700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf8004810 0x7feaf8004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c390 tx=0x7feafc001e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba75fcb9c6710838 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cc60 con 0x7feaf8004810 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00cdc0 con 0x7feaf8004810 2023-09-27T23:15:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb10867700 1 -- 172.21.15.64:0/881962335 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc005b70 con 0x7feaf8004810 2023-09-27T23:15:04.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 -- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf8004810 msgr2=0x7feaf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf8004810 0x7feaf8004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c390 tx=0x7feafc001e90 comp rx=0 tx=0).stop 2023-09-27T23:15:04.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 -- 172.21.15.64:0/881962335 shutdown_connections 2023-09-27T23:15:04.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 --2- 172.21.15.64:0/881962335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf8004810 0x7feaf8004c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 -- 172.21.15.64:0/881962335 >> 172.21.15.64:0/881962335 conn(0x7feb0c0cc760 msgr2=0x7feb0c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:04.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 -- 172.21.15.64:0/881962335 shutdown_connections 2023-09-27T23:15:04.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.418+0000 7feb1286b700 1 -- 172.21.15.64:0/881962335 wait complete. 2023-09-27T23:15:04.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 Processor -- start 2023-09-27T23:15:04.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 -- start start 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8004810 0x7feb0c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 0x7feb0c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0c0b41a0 0x7feb0c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c5140 con 0x7feb0c0c40c0 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c4530 con 0x7feb0c0b41a0 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c4810 con 0x7feaf8004810 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 0x7feb0c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 0x7feb0c0b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55304/0 (socket says 172.21.15.64:55304) 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 -- 172.21.15.64:0/2989004647 learned_addr learned my addr 172.21.15.64:0/2989004647 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11869700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8004810 0x7feb0c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1206a700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0c0b41a0 0x7feb0c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 -- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8004810 msgr2=0x7feb0c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8004810 0x7feb0c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 -- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0c0b41a0 msgr2=0x7feb0c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0c0b41a0 0x7feb0c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 -- 172.21.15.64:0/2989004647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0c4af0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb11068700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 0x7feb0c0b3c60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cc60 con 0x7feb0c0c40c0 2023-09-27T23:15:04.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00c9b0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0c4dd0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0072c0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.422+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c0b55a0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.426+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c0c52f0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.426+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7feafc01e430 con 0x7feb0c0c40c0 2023-09-27T23:15:04.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.426+0000 7feb01ffb700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feaf0060b50 0x7feaf0063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.426+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7feafc07e2f0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.426+0000 7feb11869700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feaf0060b50 0x7feaf0063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.430+0000 7feb11869700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feaf0060b50 0x7feaf0063000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb08006f00 tx=0x7feb080091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.434+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7feb0c0c52f0 con 0x7feb0c0c40c0 2023-09-27T23:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.618+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0c04be40 con 0x7feaf0060b50 2023-09-27T23:15:04.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.742+0000 7feb01ffb700 1 -- 172.21.15.64:0/2989004647 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7feb0c04be40 con 0x7feaf0060b50 2023-09-27T23:15:04.746 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir7 2023-09-27T23:15:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feaf0060b50 msgr2=0x7feaf0063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feaf0060b50 0x7feaf0063000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb08006f00 tx=0x7feb080091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 msgr2=0x7feb0c0b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 0x7feb0c0b3c60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc015000 comp rx=0 tx=0).stop 2023-09-27T23:15:04.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 shutdown_connections 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feaf0060b50 0x7feaf0063000 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0c0b41a0 0x7feb0c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c0c40c0 0x7feb0c0b3c60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 --2- 172.21.15.64:0/2989004647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8004810 0x7feb0c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 >> 172.21.15.64:0/2989004647 conn(0x7feb0c0cc760 msgr2=0x7feb0c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 shutdown_connections 2023-09-27T23:15:04.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.746+0000 7feb1286b700 1 -- 172.21.15.64:0/2989004647 wait complete. 2023-09-27T23:15:04.766 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-09-27T23:15:04.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:04.767 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-09-27T23:15:04.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-09-27T23:15:04.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 Processor -- start 2023-09-27T23:15:04.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 -- start start 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3c0c0caca0 0x7f3c0c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0c15d720 con 0x7f3c0c06df10 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0c0ca290 con 0x7f3c0c0caca0 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12f64700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0ca4a0 con 0x7f3c0c158da0 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3c0c0caca0 0x7f3c0c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3c0c0caca0 0x7f3c0c06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39440/0 (socket says 172.21.15.64:39440) 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 -- 172.21.15.64:0/79312323 learned_addr learned my addr 172.21.15.64:0/79312323 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c11761700 1 --1- 172.21.15.64:0/79312323 >> v1:172.21.15.64:6789/0 conn(0x7f3c0c06df10 0x7f3c0c06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55574/0 (socket says 172.21.15.64:55574) 2023-09-27T23:15:04.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c11f62700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 -- 172.21.15.64:0/79312323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 msgr2=0x7f3c0c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 -- 172.21.15.64:0/79312323 >> v1:172.21.15.64:6789/0 conn(0x7f3c0c06df10 legacy=0x7f3c0c06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 -- 172.21.15.64:0/79312323 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c0ca780 con 0x7f3c0c0caca0 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c11f62700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c12763700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c06ebb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009f20 tx=0x7f3c04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999774dce697066 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c10f60700 1 -- 172.21.15.64:0/79312323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400e070 con 0x7f3c0c0caca0 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c10f60700 1 -- 172.21.15.64:0/79312323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0400ba80 con 0x7f3c0c0caca0 2023-09-27T23:15:04.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.946+0000 7f3c10f60700 1 -- 172.21.15.64:0/79312323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040092e0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 -- 172.21.15.64:0/79312323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 msgr2=0x7f3c0c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c06ebb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009f20 tx=0x7f3c04009840 comp rx=0 tx=0).stop 2023-09-27T23:15:04.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 -- 172.21.15.64:0/79312323 shutdown_connections 2023-09-27T23:15:04.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c06ebb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 --2- 172.21.15.64:0/79312323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 -- 172.21.15.64:0/79312323 >> 172.21.15.64:0/79312323 conn(0x7f3c0c0cc760 msgr2=0x7f3c0c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:04.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 -- 172.21.15.64:0/79312323 shutdown_connections 2023-09-27T23:15:04.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 -- 172.21.15.64:0/79312323 wait complete. 2023-09-27T23:15:04.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 Processor -- start 2023-09-27T23:15:04.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.950+0000 7f3c12f64700 1 -- start start 2023-09-27T23:15:04.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c0c06df10 0x7f3c0c0b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0c74f0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0b4db0 con 0x7f3c0c06df10 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0b5090 con 0x7f3c0c158da0 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39450/0 (socket says 172.21.15.64:39450) 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 -- 172.21.15.64:0/1853795731 learned_addr learned my addr 172.21.15.64:0/1853795731 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11f62700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c0c06df10 0x7f3c0c0b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12763700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 -- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 msgr2=0x7f3c0c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c15a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 -- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c0c06df10 msgr2=0x7f3c0c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c0c06df10 0x7f3c0c0b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:04.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 -- 172.21.15.64:0/1853795731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c04009ae0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12763700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:15:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c11761700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c155910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000c7c0 tx=0x7f3c00007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00005a70 con 0x7f3c0c0caca0 2023-09-27T23:15:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c00005bf0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c0b5370 con 0x7f3c0c0caca0 2023-09-27T23:15:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000070f0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c0b49c0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.954+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0c0c76a0 con 0x7f3c0c0caca0 2023-09-27T23:15:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.958+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f3c00007460 con 0x7f3c0c0caca0 2023-09-27T23:15:04.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.958+0000 7f3bfdffb700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bf0060b80 0x7f3bf0063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:04.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.958+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f3c0007cf10 con 0x7f3c0c0caca0 2023-09-27T23:15:04.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.958+0000 7f3c11f62700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bf0060b80 0x7f3bf0063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.958+0000 7f3c11f62700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bf0060b80 0x7f3bf0063030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c08006fd0 tx=0x7f3c08008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:04.966+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3c0c0c76a0 con 0x7f3c0c0caca0 2023-09-27T23:15:05.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.150+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c0c04be40 con 0x7f3bf0060b80 2023-09-27T23:15:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.286+0000 7f3bfdffb700 1 -- 172.21.15.64:0/1853795731 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3c0c04be40 con 0x7f3bf0060b80 2023-09-27T23:15:05.290 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir8 2023-09-27T23:15:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bf0060b80 msgr2=0x7f3bf0063030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bf0060b80 0x7f3bf0063030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c08006fd0 tx=0x7f3c08008040 comp rx=0 tx=0).stop 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 msgr2=0x7f3c0c155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c155910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000c7c0 tx=0x7f3c00007f70 comp rx=0 tx=0).stop 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 shutdown_connections 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bf0060b80 0x7f3bf0063030 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c0c158da0 0x7f3c0c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c0c0caca0 0x7f3c0c155910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 --2- 172.21.15.64:0/1853795731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c0c06df10 0x7f3c0c0b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 >> 172.21.15.64:0/1853795731 conn(0x7f3c0c0cc760 msgr2=0x7f3c0c0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 shutdown_connections 2023-09-27T23:15:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.290+0000 7f3c12f64700 1 -- 172.21.15.64:0/1853795731 wait complete. 2023-09-27T23:15:05.311 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-09-27T23:15:05.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:15:05.311 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-09-27T23:15:05.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-09-27T23:15:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 Processor -- start 2023-09-27T23:15:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 -- start start 2023-09-27T23:15:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5bc0abe40 0x7fd5bc158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc06bb70 0x7fd5bc06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc06d9b0 con 0x7fd5bc158b00 2023-09-27T23:15:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c85a0 con 0x7fd5bc0abe40 2023-09-27T23:15:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c2a90700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c67d0 con 0x7fd5bc06bb70 2023-09-27T23:15:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c128d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5bc0abe40 0x7fd5bc158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c1a8e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd5bc158b00 0x7fd5bc06d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55576/0 (socket says 172.21.15.64:55576) 2023-09-27T23:15:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c1a8e700 1 -- 172.21.15.64:0/163648595 learned_addr learned my addr 172.21.15.64:0/163648595 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:05.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c228f700 1 --2- 172.21.15.64:0/163648595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc06bb70 0x7fd5bc06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c128d700 1 -- 172.21.15.64:0/163648595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc06bb70 msgr2=0x7fd5bc06ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c128d700 1 --2- 172.21.15.64:0/163648595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc06bb70 0x7fd5bc06ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c128d700 1 -- 172.21.15.64:0/163648595 >> v1:172.21.15.64:6789/0 conn(0x7fd5bc158b00 legacy=0x7fd5bc06d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c128d700 1 -- 172.21.15.64:0/163648595 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0c6ab0 con 0x7fd5bc0abe40 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c0a8c700 1 -- 172.21.15.64:0/163648595 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068493464 0 0) 0x7fd5bc06d9b0 con 0x7fd5bc158b00 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.442+0000 7fd5c128d700 1 --2- 172.21.15.64:0/163648595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc158ed0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80098e0 tx=0x7fd5b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd089c032c5604ab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c0a8c700 1 -- 172.21.15.64:0/163648595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800b640 con 0x7fd5bc0abe40 2023-09-27T23:15:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c0a8c700 1 -- 172.21.15.64:0/163648595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b800b7a0 con 0x7fd5bc0abe40 2023-09-27T23:15:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c0a8c700 1 -- 172.21.15.64:0/163648595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800bb40 con 0x7fd5bc0abe40 2023-09-27T23:15:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 -- 172.21.15.64:0/163648595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 msgr2=0x7fd5bc158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/163648595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc158ed0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80098e0 tx=0x7fd5b80091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 -- 172.21.15.64:0/163648595 shutdown_connections 2023-09-27T23:15:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/163648595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc06bb70 0x7fd5bc06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/163648595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc158ed0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 -- 172.21.15.64:0/163648595 >> 172.21.15.64:0/163648595 conn(0x7fd5bc0c8c00 msgr2=0x7fd5bc0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 -- 172.21.15.64:0/163648595 shutdown_connections 2023-09-27T23:15:05.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 -- 172.21.15.64:0/163648595 wait complete. 2023-09-27T23:15:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 Processor -- start 2023-09-27T23:15:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.446+0000 7fd5c2a90700 1 -- start start 2023-09-27T23:15:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5bc06bb70 0x7fd5bc0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc0a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc158b00 0x7fd5bc0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc153940 con 0x7fd5bc0abe40 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0ad130 con 0x7fd5bc06bb70 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0ac520 con 0x7fd5bc158b00 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc0a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c1a8e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5bc06bb70 0x7fd5bc0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc0a9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39466/0 (socket says 172.21.15.64:39466) 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 -- 172.21.15.64:0/3379522718 learned_addr learned my addr 172.21.15.64:0/3379522718 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c228f700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc158b00 0x7fd5bc0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 -- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc158b00 msgr2=0x7fd5bc0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc158b00 0x7fd5bc0ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 -- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5bc06bb70 msgr2=0x7fd5bc0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5bc06bb70 0x7fd5bc0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 -- 172.21.15.64:0/3379522718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b8009420 con 0x7fd5bc0abe40 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c128d700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc0a9750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc06a910 tx=0x7fd5b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800bb20 con 0x7fd5bc0abe40 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8000c90 con 0x7fd5bc0abe40 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc0ac800 con 0x7fd5bc0abe40 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8011650 con 0x7fd5bc0abe40 2023-09-27T23:15:05.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.450+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc0acdc0 con 0x7fd5bc0abe40 2023-09-27T23:15:05.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.454+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7fd5b8011a00 con 0x7fd5bc0abe40 2023-09-27T23:15:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.454+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc04be40 con 0x7fd5bc0abe40 2023-09-27T23:15:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.454+0000 7fd5adffb700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd5a0060b50 0x7fd5a0063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.454+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fd5b807ea10 con 0x7fd5bc0abe40 2023-09-27T23:15:05.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.454+0000 7fd5c1a8e700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd5a0060b50 0x7fd5a0063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.454+0000 7fd5c1a8e700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd5a0060b50 0x7fd5a0063000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0007160 tx=0x7fd5b0007f50 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:05.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.462+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd5b804e730 con 0x7fd5bc0abe40 2023-09-27T23:15:05.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.638+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5bc0c6d90 con 0x7fd5a0060b50 2023-09-27T23:15:05.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.758+0000 7fd5adffb700 1 -- 172.21.15.64:0/3379522718 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd5bc0c6d90 con 0x7fd5a0060b50 2023-09-27T23:15:05.763 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/dir9 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd5a0060b50 msgr2=0x7fd5a0063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd5a0060b50 0x7fd5a0063000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0007160 tx=0x7fd5b0007f50 comp rx=0 tx=0).stop 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 msgr2=0x7fd5bc0a9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc0a9750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc06a910 tx=0x7fd5b8017000 comp rx=0 tx=0).stop 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 shutdown_connections 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd5a0060b50 0x7fd5a0063000 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5bc158b00 0x7fd5bc0ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5bc0abe40 0x7fd5bc0a9750 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 --2- 172.21.15.64:0/3379522718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5bc06bb70 0x7fd5bc0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 >> 172.21.15.64:0/3379522718 conn(0x7fd5bc0c8c00 msgr2=0x7fd5bc0cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:05.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 shutdown_connections 2023-09-27T23:15:05.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.762+0000 7fd5c2a90700 1 -- 172.21.15.64:0/3379522718 wait complete. 2023-09-27T23:15:05.784 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-09-27T23:15:05.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-09-27T23:15:05.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 Processor -- start 2023-09-27T23:15:05.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- start start 2023-09-27T23:15:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d4013f830 0x7f7d400ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d400aea30 0x7f7d400aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d400cc200 con 0x7f7d401563e0 2023-09-27T23:15:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d400cb160 con 0x7f7d400aea30 2023-09-27T23:15:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400cb440 con 0x7f7d4013f830 2023-09-27T23:15:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d400aea30 0x7f7d400aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d400aea30 0x7f7d400aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39482/0 (socket says 172.21.15.64:39482) 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 -- 172.21.15.64:0/4008425115 learned_addr learned my addr 172.21.15.64:0/4008425115 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3f7fe700 1 --2- 172.21.15.64:0/4008425115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d4013f830 0x7f7d400ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 -- 172.21.15.64:0/4008425115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d4013f830 msgr2=0x7f7d400ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 --2- 172.21.15.64:0/4008425115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d4013f830 0x7f7d400ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 -- 172.21.15.64:0/4008425115 >> v1:172.21.15.123:6789/0 conn(0x7f7d401563e0 legacy=0x7f7d40156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 -- 172.21.15.64:0/4008425115 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400cb720 con 0x7f7d400aea30 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3effd700 1 --2- 172.21.15.64:0/4008425115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d400aeea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d340099b0 tx=0x7f7d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211d51ee93675fb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3e7fc700 1 -- 172.21.15.64:0/4008425115 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34009ea0 con 0x7f7d400aea30 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3e7fc700 1 -- 172.21.15.64:0/4008425115 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400b860 con 0x7f7d400aea30 2023-09-27T23:15:05.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d3e7fc700 1 -- 172.21.15.64:0/4008425115 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400bc00 con 0x7f7d400aea30 2023-09-27T23:15:05.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- 172.21.15.64:0/4008425115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 msgr2=0x7f7d400aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 --2- 172.21.15.64:0/4008425115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d400aeea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d340099b0 tx=0x7f7d34009250 comp rx=0 tx=0).stop 2023-09-27T23:15:05.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- 172.21.15.64:0/4008425115 shutdown_connections 2023-09-27T23:15:05.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 --2- 172.21.15.64:0/4008425115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d400aeea0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 --2- 172.21.15.64:0/4008425115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d4013f830 0x7f7d400ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- 172.21.15.64:0/4008425115 >> 172.21.15.64:0/4008425115 conn(0x7f7d400ccc20 msgr2=0x7f7d400cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- 172.21.15.64:0/4008425115 shutdown_connections 2023-09-27T23:15:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.906+0000 7f7d45baa700 1 -- 172.21.15.64:0/4008425115 wait complete. 2023-09-27T23:15:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 Processor -- start 2023-09-27T23:15:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 -- start start 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d4013b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d4013f830 0x7f7d4016a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d401563e0 0x7f7d401688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40169f00 con 0x7f7d400aea30 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401692f0 con 0x7f7d4013f830 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d45baa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401695d0 con 0x7f7d401563e0 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d4013b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d4013b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39494/0 (socket says 172.21.15.64:39494) 2023-09-27T23:15:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3500331375 learned_addr learned my addr 172.21.15.64:0/3500331375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3effd700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d4013f830 0x7f7d4016a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d401563e0 0x7f7d401688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d401563e0 msgr2=0x7f7d401688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d401563e0 0x7f7d401688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d4013f830 msgr2=0x7f7d4016a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d4013f830 0x7f7d4016a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3500331375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340094f0 con 0x7f7d400aea30 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.910+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d401563e0 0x7f7d401688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d3f7fe700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d4013b820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000ec80 tx=0x7f7d300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000c850 con 0x7f7d400aea30 2023-09-27T23:15:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3000c9b0 con 0x7f7d400aea30 2023-09-27T23:15:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d401698b0 con 0x7f7d400aea30 2023-09-27T23:15:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000cd50 con 0x7f7d400aea30 2023-09-27T23:15:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4013bd60 con 0x7f7d400aea30 2023-09-27T23:15:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d40071120 con 0x7f7d400aea30 2023-09-27T23:15:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f7d30013680 con 0x7f7d400aea30 2023-09-27T23:15:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d277fe700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d2c060b50 0x7f7d2c063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f7d3007d760 con 0x7f7d400aea30 2023-09-27T23:15:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.914+0000 7f7d3effd700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d2c060b50 0x7f7d2c063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:05.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.918+0000 7f7d3effd700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d2c060b50 0x7f7d2c063000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7d340099b0 tx=0x7f7d34017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:05.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:05.922+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7d3004d480 con 0x7f7d400aea30 2023-09-27T23:15:06.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.110+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d4016a040 con 0x7f7d2c060b50 2023-09-27T23:15:06.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.206+0000 7f7d277fe700 1 -- 172.21.15.64:0/3500331375 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7d3400b9d0 con 0x7f7d2c060b50 2023-09-27T23:15:06.211 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-09-27T23:15:06.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d2c060b50 msgr2=0x7f7d2c063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d2c060b50 0x7f7d2c063000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7d340099b0 tx=0x7f7d34017000 comp rx=0 tx=0).stop 2023-09-27T23:15:06.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 msgr2=0x7f7d4013b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d4013b820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000ec80 tx=0x7f7d300091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 shutdown_connections 2023-09-27T23:15:06.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d2c060b50 0x7f7d2c063000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d401563e0 0x7f7d401688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d4013f830 0x7f7d4016a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 --2- 172.21.15.64:0/3500331375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400aea30 0x7f7d4013b820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 >> 172.21.15.64:0/3500331375 conn(0x7f7d400ccc20 msgr2=0x7f7d40062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:06.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 shutdown_connections 2023-09-27T23:15:06.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.214+0000 7f7d45baa700 1 -- 172.21.15.64:0/3500331375 wait complete. 2023-09-27T23:15:06.233 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-09-27T23:15:06.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-09-27T23:15:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 Processor -- start 2023-09-27T23:15:06.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 -- start start 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd00cb500 0x7efcd015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcd015c760 0x7efcd00b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd00b0ac0 con 0x7efcd00cb130 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd015cbe0 con 0x7efcd015c760 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd006b210 con 0x7efcd00cb500 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcd015c760 0x7efcd00b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcd015c760 0x7efcd00b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39508/0 (socket says 172.21.15.64:39508) 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 -- 172.21.15.64:0/481561962 learned_addr learned my addr 172.21.15.64:0/481561962 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efccf7fe700 1 --2- 172.21.15.64:0/481561962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd00cb500 0x7efcd015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 -- 172.21.15.64:0/481561962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd00cb500 msgr2=0x7efcd015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 --2- 172.21.15.64:0/481561962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd00cb500 0x7efcd015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcceffd700 1 -- 172.21.15.64:0/481561962 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442238323 0 0) 0x7efcd00b0ac0 con 0x7efcd00cb130 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 -- 172.21.15.64:0/481561962 >> v1:172.21.15.123:6789/0 conn(0x7efcd00cb130 legacy=0x7efcd006a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 -- 172.21.15.64:0/481561962 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00ca810 con 0x7efcd015c760 2023-09-27T23:15:06.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd4b17700 1 --2- 172.21.15.64:0/481561962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd015c760 0x7efcd00b0f50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f20 tx=0x7efcc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa9ff7232ddd8ca server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:06.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcceffd700 1 -- 172.21.15.64:0/481561962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800e070 con 0x7efcd015c760 2023-09-27T23:15:06.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcceffd700 1 -- 172.21.15.64:0/481561962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc800b9d0 con 0x7efcd015c760 2023-09-27T23:15:06.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcceffd700 1 -- 172.21.15.64:0/481561962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80092e0 con 0x7efcd015c760 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 -- 172.21.15.64:0/481561962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd015c760 msgr2=0x7efcd00b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.358+0000 7efcd657a700 1 --2- 172.21.15.64:0/481561962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd015c760 0x7efcd00b0f50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f20 tx=0x7efcc8009840 comp rx=0 tx=0).stop 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 -- 172.21.15.64:0/481561962 shutdown_connections 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 --2- 172.21.15.64:0/481561962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd015c760 0x7efcd00b0f50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 --2- 172.21.15.64:0/481561962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd00cb500 0x7efcd015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 -- 172.21.15.64:0/481561962 >> 172.21.15.64:0/481561962 conn(0x7efcd0067bb0 msgr2=0x7efcd00cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 -- 172.21.15.64:0/481561962 shutdown_connections 2023-09-27T23:15:06.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 -- 172.21.15.64:0/481561962 wait complete. 2023-09-27T23:15:06.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 Processor -- start 2023-09-27T23:15:06.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.362+0000 7efcd657a700 1 -- start start 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 0x7efcd01436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcd00cb500 0x7efcd00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd015c760 0x7efcd015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0157270 con 0x7efcd00cb130 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00b40d0 con 0x7efcd00cb500 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd015a680 con 0x7efcd015c760 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 0x7efcd01436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 0x7efcd01436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39518/0 (socket says 172.21.15.64:39518) 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 -- 172.21.15.64:0/3647022151 learned_addr learned my addr 172.21.15.64:0/3647022151 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccf7fe700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcd00cb500 0x7efcd00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd4b17700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd015c760 0x7efcd015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 -- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd015c760 msgr2=0x7efcd015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd015c760 0x7efcd015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 -- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcd00cb500 msgr2=0x7efcd00b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcd00cb500 0x7efcd00b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 -- 172.21.15.64:0/3647022151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8009ae0 con 0x7efcd00cb130 2023-09-27T23:15:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efccffff700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 0x7efcd01436e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efcc400c530 tx=0x7efcc400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4011590 con 0x7efcd00cb130 2023-09-27T23:15:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc4011710 con 0x7efcd00cb130 2023-09-27T23:15:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd015a920 con 0x7efcd00cb130 2023-09-27T23:15:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4011ad0 con 0x7efcd00cb130 2023-09-27T23:15:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd015aee0 con 0x7efcd00cb130 2023-09-27T23:15:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.366+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd00b3ea0 con 0x7efcd00cb130 2023-09-27T23:15:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.370+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7efcc401b3b0 con 0x7efcd00cb130 2023-09-27T23:15:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.370+0000 7efcb7fff700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efcbc060b80 0x7efcbc063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.370+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7efcc407c7f0 con 0x7efcd00cb130 2023-09-27T23:15:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.370+0000 7efccf7fe700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efcbc060b80 0x7efcbc063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.370+0000 7efccf7fe700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efcbc060b80 0x7efcbc063030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efcd016c5e0 tx=0x7efcc00091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.374+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efcd00b3ea0 con 0x7efcd00cb130 2023-09-27T23:15:06.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.566+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efcd00b3ea0 con 0x7efcbc060b80 2023-09-27T23:15:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.670+0000 7efcb7fff700 1 -- 172.21.15.64:0/3647022151 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efcd00b3ea0 con 0x7efcbc060b80 2023-09-27T23:15:06.675 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-09-27T23:15:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efcbc060b80 msgr2=0x7efcbc063030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efcbc060b80 0x7efcbc063030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efcd016c5e0 tx=0x7efcc00091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 msgr2=0x7efcd01436e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 0x7efcd01436e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efcc400c530 tx=0x7efcc400c840 comp rx=0 tx=0).stop 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 shutdown_connections 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efcbc060b80 0x7efcbc063030 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcd015c760 0x7efcd015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcd00cb500 0x7efcd00b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 --2- 172.21.15.64:0/3647022151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcd00cb130 0x7efcd01436e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 >> 172.21.15.64:0/3647022151 conn(0x7efcd0067bb0 msgr2=0x7efcd00b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 shutdown_connections 2023-09-27T23:15:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.674+0000 7efcd657a700 1 -- 172.21.15.64:0/3647022151 wait complete. 2023-09-27T23:15:06.696 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-09-27T23:15:06.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-09-27T23:15:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6961dd700 1 Processor -- start 2023-09-27T23:15:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6961dd700 1 -- start start 2023-09-27T23:15:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6961dd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6900b0eb0 0x7fd690161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6961dd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6900c8a00 con 0x7fd6901568e0 2023-09-27T23:15:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6961dd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6900c8ce0 con 0x7fd690156cb0 2023-09-27T23:15:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6961dd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900afa50 con 0x7fd6900b0eb0 2023-09-27T23:15:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6951db700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd6901568e0 0x7fd6900c7f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55580/0 (socket says 172.21.15.64:55580) 2023-09-27T23:15:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6951db700 1 -- 172.21.15.64:0/166094823 learned_addr learned my addr 172.21.15.64:0/166094823 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6959dc700 1 --2- 172.21.15.64:0/166094823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6900b0eb0 0x7fd690161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950256625 0 0) 0x7fd6900c8a00 con 0x7fd6901568e0 2023-09-27T23:15:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd67c003570 con 0x7fd6901568e0 2023-09-27T23:15:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964839663 0 0) 0x7fd6900c8ce0 con 0x7fd690156cb0 2023-09-27T23:15:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6900c8a00 con 0x7fd690156cb0 2023-09-27T23:15:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006498514 0 0) 0x7fd67c003570 con 0x7fd6901568e0 2023-09-27T23:15:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6959dc700 1 -- 172.21.15.64:0/166094823 >> v1:172.21.15.123:6789/0 conn(0x7fd690156cb0 legacy=0x7fd6900ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6959dc700 1 -- 172.21.15.64:0/166094823 >> v1:172.21.15.64:6789/0 conn(0x7fd6901568e0 legacy=0x7fd6900c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.814+0000 7fd6959dc700 1 -- 172.21.15.64:0/166094823 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900c8fc0 con 0x7fd6900b0eb0 2023-09-27T23:15:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6959dc700 1 --2- 172.21.15.64:0/166094823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6900b0eb0 0x7fd690161730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd688009e90 tx=0x7fd688009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481189db4e9e3ad3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800e070 con 0x7fd6900b0eb0 2023-09-27T23:15:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800b870 con 0x7fd6900b0eb0 2023-09-27T23:15:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd687fff700 1 -- 172.21.15.64:0/166094823 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c001470 con 0x7fd6900b0eb0 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 -- 172.21.15.64:0/166094823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6900b0eb0 msgr2=0x7fd690161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 --2- 172.21.15.64:0/166094823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6900b0eb0 0x7fd690161730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd688009e90 tx=0x7fd688009780 comp rx=0 tx=0).stop 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 -- 172.21.15.64:0/166094823 shutdown_connections 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 --2- 172.21.15.64:0/166094823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6900b0eb0 0x7fd690161730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 -- 172.21.15.64:0/166094823 >> 172.21.15.64:0/166094823 conn(0x7fd69006b5b0 msgr2=0x7fd69006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 -- 172.21.15.64:0/166094823 shutdown_connections 2023-09-27T23:15:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.818+0000 7fd6961dd700 1 -- 172.21.15.64:0/166094823 wait complete. 2023-09-27T23:15:06.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 Processor -- start 2023-09-27T23:15:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 -- start start 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 0x7fd6900bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6901568e0 0x7fd69015b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd690156cb0 0x7fd69015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900caa90 con 0x7fd6900b0eb0 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900c3320 con 0x7fd6901568e0 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900c35c0 con 0x7fd690156cb0 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 0x7fd6900bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 0x7fd6900bf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39526/0 (socket says 172.21.15.64:39526) 2023-09-27T23:15:06.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 -- 172.21.15.64:0/3156404552 learned_addr learned my addr 172.21.15.64:0/3156404552 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:06.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6949da700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6901568e0 0x7fd69015b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6959dc700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd690156cb0 0x7fd69015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 -- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd690156cb0 msgr2=0x7fd69015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd690156cb0 0x7fd69015bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 -- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6901568e0 msgr2=0x7fd69015b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6901568e0 0x7fd69015b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 -- 172.21.15.64:0/3156404552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009a20 con 0x7fd6900b0eb0 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6951db700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 0x7fd6900bf6a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00cb60 tx=0x7fd68c001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c007150 con 0x7fd6900b0eb0 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c0072b0 con 0x7fd6900b0eb0 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6900ca720 con 0x7fd6900b0eb0 2023-09-27T23:15:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c007670 con 0x7fd6900b0eb0 2023-09-27T23:15:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.822+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd690155e60 con 0x7fd6900b0eb0 2023-09-27T23:15:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.826+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6900c37e0 con 0x7fd6900b0eb0 2023-09-27T23:15:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.826+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7fd68c004a40 con 0x7fd6900b0eb0 2023-09-27T23:15:06.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.826+0000 7fd6857fa700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd674060b50 0x7fd674063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:06.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.826+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fd68c07ca80 con 0x7fd6900b0eb0 2023-09-27T23:15:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.826+0000 7fd6949da700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd674060b50 0x7fd674063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.830+0000 7fd6949da700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd674060b50 0x7fd674063000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd6901409f0 tx=0x7fd680007d50 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:06.838+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd68c04c8c0 con 0x7fd6900b0eb0 2023-09-27T23:15:07.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.022+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6900cabd0 con 0x7fd674060b50 2023-09-27T23:15:07.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.122+0000 7fd6857fa700 1 -- 172.21.15.64:0/3156404552 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd68000f5f0 con 0x7fd674060b50 2023-09-27T23:15:07.126 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-09-27T23:15:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd674060b50 msgr2=0x7fd674063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd674060b50 0x7fd674063000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd6901409f0 tx=0x7fd680007d50 comp rx=0 tx=0).stop 2023-09-27T23:15:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 msgr2=0x7fd6900bf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 0x7fd6900bf6a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00cb60 tx=0x7fd68c001ad0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 shutdown_connections 2023-09-27T23:15:07.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd674060b50 0x7fd674063000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd690156cb0 0x7fd69015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6901568e0 0x7fd69015b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 --2- 172.21.15.64:0/3156404552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6900b0eb0 0x7fd6900bf6a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 >> 172.21.15.64:0/3156404552 conn(0x7fd69006b5b0 msgr2=0x7fd6900c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 shutdown_connections 2023-09-27T23:15:07.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.126+0000 7fd6961dd700 1 -- 172.21.15.64:0/3156404552 wait complete. 2023-09-27T23:15:07.147 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-09-27T23:15:07.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-09-27T23:15:07.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed6031700 1 Processor -- start 2023-09-27T23:15:07.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed6031700 1 -- start start 2023-09-27T23:15:07.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed6031700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed0158da0 0x7fbed006e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed6031700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed015d720 con 0x7fbed006df10 2023-09-27T23:15:07.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed6031700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed00ca4a0 con 0x7fbed00caca0 2023-09-27T23:15:07.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed6031700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00ca290 con 0x7fbed0158da0 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed482e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbed006df10 0x7fbed006a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55588/0 (socket says 172.21.15.64:55588) 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed482e700 1 -- 172.21.15.64:0/4070669608 learned_addr learned my addr 172.21.15.64:0/4070669608 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed502f700 1 --2- 172.21.15.64:0/4070669608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed0158da0 0x7fbed006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbecffff700 1 -- 172.21.15.64:0/4070669608 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618314510 0 0) 0x7fbed015d720 con 0x7fbed006df10 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbecffff700 1 -- 172.21.15.64:0/4070669608 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb8003570 con 0x7fbed006df10 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbecffff700 1 -- 172.21.15.64:0/4070669608 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876643174 0 0) 0x7fbed00ca4a0 con 0x7fbed00caca0 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed502f700 1 -- 172.21.15.64:0/4070669608 >> v1:172.21.15.123:6789/0 conn(0x7fbed00caca0 legacy=0x7fbed006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed502f700 1 -- 172.21.15.64:0/4070669608 >> v1:172.21.15.64:6789/0 conn(0x7fbed006df10 legacy=0x7fbed006a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed502f700 1 -- 172.21.15.64:0/4070669608 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00ca780 con 0x7fbed0158da0 2023-09-27T23:15:07.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbed502f700 1 --2- 172.21.15.64:0/4070669608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed0158da0 0x7fbed006e670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009e90 tx=0x7fbec8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add6b9ed2138c503 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbecffff700 1 -- 172.21.15.64:0/4070669608 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800e070 con 0x7fbed0158da0 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbecffff700 1 -- 172.21.15.64:0/4070669608 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec800bc30 con 0x7fbed0158da0 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.274+0000 7fbecffff700 1 -- 172.21.15.64:0/4070669608 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012430 con 0x7fbed0158da0 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 -- 172.21.15.64:0/4070669608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed0158da0 msgr2=0x7fbed006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 --2- 172.21.15.64:0/4070669608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed0158da0 0x7fbed006e670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009e90 tx=0x7fbec8009780 comp rx=0 tx=0).stop 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 -- 172.21.15.64:0/4070669608 shutdown_connections 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 --2- 172.21.15.64:0/4070669608 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed0158da0 0x7fbed006e670 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 -- 172.21.15.64:0/4070669608 >> 172.21.15.64:0/4070669608 conn(0x7fbed00cc760 msgr2=0x7fbed00ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:07.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 -- 172.21.15.64:0/4070669608 shutdown_connections 2023-09-27T23:15:07.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.278+0000 7fbed6031700 1 -- 172.21.15.64:0/4070669608 wait complete. 2023-09-27T23:15:07.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 Processor -- start 2023-09-27T23:15:07.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 -- start start 2023-09-27T23:15:07.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed006df10 0x7fbed01582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 0x7fbed00c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed0158da0 0x7fbed00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed0158830 con 0x7fbed00caca0 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00c4310 con 0x7fbed0158da0 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed6031700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00c3700 con 0x7fbed006df10 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed482e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 0x7fbed00c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed482e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 0x7fbed00c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39540/0 (socket says 172.21.15.64:39540) 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed482e700 1 -- 172.21.15.64:0/2597810795 learned_addr learned my addr 172.21.15.64:0/2597810795 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed502f700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed006df10 0x7fbed01582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.282+0000 7fbed5830700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed0158da0 0x7fbed00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed482e700 1 -- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed006df10 msgr2=0x7fbed01582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed482e700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed006df10 0x7fbed01582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed482e700 1 -- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed0158da0 msgr2=0x7fbed00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed482e700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed0158da0 0x7fbed00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed482e700 1 -- 172.21.15.64:0/2597810795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009a20 con 0x7fbed00caca0 2023-09-27T23:15:07.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed482e700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 0x7fbed00c5170 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c7c0 tx=0x7fbec4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:07.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4005cd0 con 0x7fbed00caca0 2023-09-27T23:15:07.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4005e30 con 0x7fbed00caca0 2023-09-27T23:15:07.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4007320 con 0x7fbed00caca0 2023-09-27T23:15:07.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed00c39e0 con 0x7fbed00caca0 2023-09-27T23:15:07.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed00c3fa0 con 0x7fbed00caca0 2023-09-27T23:15:07.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed00c4480 con 0x7fbed00caca0 2023-09-27T23:15:07.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7fbec4007690 con 0x7fbed00caca0 2023-09-27T23:15:07.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbecd7fa700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbebc060b50 0x7fbebc063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fbec407cad0 con 0x7fbed00caca0 2023-09-27T23:15:07.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.286+0000 7fbed502f700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbebc060b50 0x7fbebc063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.290+0000 7fbed502f700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbebc060b50 0x7fbebc063000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009de0 tx=0x7fbec8019000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:07.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.294+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbec404c890 con 0x7fbed00caca0 2023-09-27T23:15:07.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.478+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbed004be40 con 0x7fbebc060b50 2023-09-27T23:15:07.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.582+0000 7fbecd7fa700 1 -- 172.21.15.64:0/2597810795 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbed004be40 con 0x7fbebc060b50 2023-09-27T23:15:07.586 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-09-27T23:15:07.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbebc060b50 msgr2=0x7fbebc063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbebc060b50 0x7fbebc063000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009de0 tx=0x7fbec8019000 comp rx=0 tx=0).stop 2023-09-27T23:15:07.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 msgr2=0x7fbed00c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 0x7fbed00c5170 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c7c0 tx=0x7fbec4007e00 comp rx=0 tx=0).stop 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 shutdown_connections 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fbebc060b50 0x7fbebc063000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed0158da0 0x7fbed00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed00caca0 0x7fbed00c5170 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 --2- 172.21.15.64:0/2597810795 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbed006df10 0x7fbed01582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 >> 172.21.15.64:0/2597810795 conn(0x7fbed00cc760 msgr2=0x7fbed01594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 shutdown_connections 2023-09-27T23:15:07.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.586+0000 7fbed6031700 1 -- 172.21.15.64:0/2597810795 wait complete. 2023-09-27T23:15:07.607 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-09-27T23:15:07.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-09-27T23:15:07.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 Processor -- start 2023-09-27T23:15:07.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 -- start start 2023-09-27T23:15:07.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c0aef60 0x7f633c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f633c0adf40 0x7f633c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c06beb0 con 0x7f633c1569e0 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c06cb80 con 0x7f633c0adf40 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6341e3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c06ce60 con 0x7f633c0aef60 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f634163e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f633c0adf40 0x7f633c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6340e3d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f633c1569e0 0x7f633c0b1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37562/0 (socket says 172.21.15.64:37562) 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f6340e3d700 1 -- 172.21.15.64:0/1095715442 learned_addr learned my addr 172.21.15.64:0/1095715442 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:07.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f633bfff700 1 --2- 172.21.15.64:0/1095715442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c0aef60 0x7f633c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f634163e700 1 -- 172.21.15.64:0/1095715442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c0aef60 msgr2=0x7f633c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:07.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f634163e700 1 --2- 172.21.15.64:0/1095715442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c0aef60 0x7f633c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f634163e700 1 -- 172.21.15.64:0/1095715442 >> v1:172.21.15.123:6789/0 conn(0x7f633c1569e0 legacy=0x7f633c0b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f633b7fe700 1 -- 172.21.15.64:0/1095715442 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695851785 0 0) 0x7f633c06beb0 con 0x7f633c1569e0 2023-09-27T23:15:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.726+0000 7f634163e700 1 -- 172.21.15.64:0/1095715442 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c06d140 con 0x7f633c0adf40 2023-09-27T23:15:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f634163e700 1 --2- 172.21.15.64:0/1095715442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c0ae3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e60 tx=0x7f6334009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3991ceb24380ae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f633b7fe700 1 -- 172.21.15.64:0/1095715442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400e070 con 0x7f633c0adf40 2023-09-27T23:15:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f633b7fe700 1 -- 172.21.15.64:0/1095715442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633400bb60 con 0x7f633c0adf40 2023-09-27T23:15:07.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f633b7fe700 1 -- 172.21.15.64:0/1095715442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334012430 con 0x7f633c0adf40 2023-09-27T23:15:07.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 -- 172.21.15.64:0/1095715442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 msgr2=0x7f633c0ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 --2- 172.21.15.64:0/1095715442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c0ae3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e60 tx=0x7f6334009780 comp rx=0 tx=0).stop 2023-09-27T23:15:07.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 -- 172.21.15.64:0/1095715442 shutdown_connections 2023-09-27T23:15:07.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 --2- 172.21.15.64:0/1095715442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c0ae3a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 --2- 172.21.15.64:0/1095715442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c0aef60 0x7f633c156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 -- 172.21.15.64:0/1095715442 >> 172.21.15.64:0/1095715442 conn(0x7f633c0c8c00 msgr2=0x7f633c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:07.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 -- 172.21.15.64:0/1095715442 shutdown_connections 2023-09-27T23:15:07.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 -- 172.21.15.64:0/1095715442 wait complete. 2023-09-27T23:15:07.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 Processor -- start 2023-09-27T23:15:07.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.730+0000 7f6341e3f700 1 -- start start 2023-09-27T23:15:07.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f633c0aef60 0x7f633c0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c1569e0 0x7f633c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0c3ab0 con 0x7f633c0adf40 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1532b0 con 0x7f633c0aef60 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0c31a0 con 0x7f633c1569e0 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39558/0 (socket says 172.21.15.64:39558) 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 -- 172.21.15.64:0/773862374 learned_addr learned my addr 172.21.15.64:0/773862374 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f634163e700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c1569e0 0x7f633c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f633bfff700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f633c0aef60 0x7f633c0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 -- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c1569e0 msgr2=0x7f633c0b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c1569e0 0x7f633c0b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 -- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f633c0aef60 msgr2=0x7f633c0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:07.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f633c0aef60 0x7f633c0c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:07.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 -- 172.21.15.64:0/773862374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334009a20 con 0x7f633c0adf40 2023-09-27T23:15:07.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6340e3d700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c152d70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f633000c7e0 tx=0x7f6330007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330005b30 con 0x7f633c0adf40 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330005c90 con 0x7f633c0adf40 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330007300 con 0x7f633c0adf40 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c0c5640 con 0x7f633c0adf40 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.734+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c0c3740 con 0x7f633c0adf40 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.738+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c04be40 con 0x7f633c0adf40 2023-09-27T23:15:07.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.738+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f633001b3b0 con 0x7f633c0adf40 2023-09-27T23:15:07.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.738+0000 7f6338ff9700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6320060b50 0x7f6320063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:07.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.738+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f633007c9a0 con 0x7f633c0adf40 2023-09-27T23:15:07.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.738+0000 7f633bfff700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6320060b50 0x7f6320063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:07.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.738+0000 7f633bfff700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6320060b50 0x7f6320063000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f633c0bfae0 tx=0x7f63280091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:07.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.746+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f633004c6c0 con 0x7f633c0adf40 2023-09-27T23:15:07.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:07.926+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f633c0c3bf0 con 0x7f6320060b50 2023-09-27T23:15:08.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.034+0000 7f6338ff9700 1 -- 172.21.15.64:0/773862374 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6328009ad0 con 0x7f6320060b50 2023-09-27T23:15:08.037 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6320060b50 msgr2=0x7f6320063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6320060b50 0x7f6320063000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f633c0bfae0 tx=0x7f63280091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 msgr2=0x7f633c152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c152d70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f633000c7e0 tx=0x7f6330007df0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 shutdown_connections 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6320060b50 0x7f6320063000 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f633c1569e0 0x7f633c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f633c0aef60 0x7f633c0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 --2- 172.21.15.64:0/773862374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f633c0adf40 0x7f633c152d70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 >> 172.21.15.64:0/773862374 conn(0x7f633c0c8c00 msgr2=0x7f633c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 shutdown_connections 2023-09-27T23:15:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.038+0000 7f6341e3f700 1 -- 172.21.15.64:0/773862374 wait complete. 2023-09-27T23:15:08.060 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-09-27T23:15:08.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-09-27T23:15:08.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 Processor -- start 2023-09-27T23:15:08.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 -- start start 2023-09-27T23:15:08.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f4406e0f0 0x7f3f440c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f4406e4c0 0x7f3f440c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f440ae270 0x7f3f440c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f44156140 con 0x7f3f4406e0f0 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f440c7130 con 0x7f3f440ae270 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f49ada700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440c7410 con 0x7f3f4406e4c0 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f4406e0f0 0x7f3f440c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f4406e0f0 0x7f3f440c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39560/0 (socket says 172.21.15.64:39560) 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 -- 172.21.15.64:0/4279366495 learned_addr learned my addr 172.21.15.64:0/4279366495 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f43fff700 1 --2- 172.21.15.64:0/4279366495 >> v2:172.21.15.123:3300/0 conn(0x7f3f440ae270 0x7f3f440c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f42ffd700 1 --2- 172.21.15.64:0/4279366495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f4406e4c0 0x7f3f440c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 -- 172.21.15.64:0/4279366495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f4406e4c0 msgr2=0x7f3f440c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 --2- 172.21.15.64:0/4279366495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f4406e4c0 0x7f3f440c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 -- 172.21.15.64:0/4279366495 >> v2:172.21.15.123:3300/0 conn(0x7f3f440ae270 msgr2=0x7f3f440c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 --2- 172.21.15.64:0/4279366495 >> v2:172.21.15.123:3300/0 conn(0x7f3f440ae270 0x7f3f440c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 -- 172.21.15.64:0/4279366495 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440c76f0 con 0x7f3f4406e0f0 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f437fe700 1 --2- 172.21.15.64:0/4279366495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f440c68e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3f4406c490 tx=0x7f3f34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68e3a18acdab0bd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f427fc700 1 -- 172.21.15.64:0/4279366495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34009e70 con 0x7f3f4406e0f0 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f427fc700 1 -- 172.21.15.64:0/4279366495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3400b860 con 0x7f3f4406e0f0 2023-09-27T23:15:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.182+0000 7f3f427fc700 1 -- 172.21.15.64:0/4279366495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3400bc00 con 0x7f3f4406e0f0 2023-09-27T23:15:08.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 -- 172.21.15.64:0/4279366495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 msgr2=0x7f3f440c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 --2- 172.21.15.64:0/4279366495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f440c68e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3f4406c490 tx=0x7f3f34009250 comp rx=0 tx=0).stop 2023-09-27T23:15:08.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 -- 172.21.15.64:0/4279366495 shutdown_connections 2023-09-27T23:15:08.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 --2- 172.21.15.64:0/4279366495 >> v2:172.21.15.123:3300/0 conn(0x7f3f440ae270 0x7f3f440c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 --2- 172.21.15.64:0/4279366495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f4406e4c0 0x7f3f440c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 --2- 172.21.15.64:0/4279366495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f440c68e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 -- 172.21.15.64:0/4279366495 >> 172.21.15.64:0/4279366495 conn(0x7f3f440c8db0 msgr2=0x7f3f440c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:08.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 -- 172.21.15.64:0/4279366495 shutdown_connections 2023-09-27T23:15:08.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 -- 172.21.15.64:0/4279366495 wait complete. 2023-09-27T23:15:08.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 Processor -- start 2023-09-27T23:15:08.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.186+0000 7f3f49ada700 1 -- start start 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f441485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f4406e4c0 0x7f3f440c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f440ae270 0x7f3f440b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440c1770 con 0x7f3f4406e0f0 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440c0b60 con 0x7f3f4406e4c0 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440c0e40 con 0x7f3f440ae270 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f441485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f441485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39562/0 (socket says 172.21.15.64:39562) 2023-09-27T23:15:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f437fe700 1 -- 172.21.15.64:0/1473123975 learned_addr learned my addr 172.21.15.64:0/1473123975 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f4406e4c0 0x7f3f440c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f43fff700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f440ae270 0x7f3f440b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 -- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f440ae270 msgr2=0x7f3f440b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f440ae270 0x7f3f440b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 -- 172.21.15.64:0/1473123975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 msgr2=0x7f3f441485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f441485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 -- 172.21.15.64:0/1473123975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340094f0 con 0x7f3f4406e4c0 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f437fe700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f441485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:15:08.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f42ffd700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f4406e4c0 0x7f3f440c2260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d890 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f380125f0 con 0x7f3f4406e4c0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38012770 con 0x7f3f4406e4c0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f440c1120 con 0x7f3f4406e4c0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38012b30 con 0x7f3f4406e4c0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.190+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f44156440 con 0x7f3f4406e4c0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.194+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f44071120 con 0x7f3f4406e4c0 2023-09-27T23:15:08.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.194+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f3f38013480 con 0x7f3f4406e4c0 2023-09-27T23:15:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.194+0000 7f3f2b7fe700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f30060b50 0x7f3f30063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.194+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f3f3807d840 con 0x7f3f4406e4c0 2023-09-27T23:15:08.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.194+0000 7f3f437fe700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f30060b50 0x7f3f30063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.194+0000 7f3f437fe700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f30060b50 0x7f3f30063000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009490 tx=0x7f3f34019000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:08.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.206+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3f38050ff0 con 0x7f3f4406e4c0 2023-09-27T23:15:08.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.398+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f440694f0 con 0x7f3f30060b50 2023-09-27T23:15:08.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.502+0000 7f3f2b7fe700 1 -- 172.21.15.64:0/1473123975 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3f440694f0 con 0x7f3f30060b50 2023-09-27T23:15:08.504 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-09-27T23:15:08.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f30060b50 msgr2=0x7f3f30063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f30060b50 0x7f3f30063000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009490 tx=0x7f3f34019000 comp rx=0 tx=0).stop 2023-09-27T23:15:08.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f4406e4c0 msgr2=0x7f3f440c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f4406e4c0 0x7f3f440c2260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d890 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 shutdown_connections 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f30060b50 0x7f3f30063000 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f440ae270 0x7f3f440b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f4406e4c0 0x7f3f440c2260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 --2- 172.21.15.64:0/1473123975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f4406e0f0 0x7f3f441485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 >> 172.21.15.64:0/1473123975 conn(0x7f3f440c8db0 msgr2=0x7f3f440ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 shutdown_connections 2023-09-27T23:15:08.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.506+0000 7f3f49ada700 1 -- 172.21.15.64:0/1473123975 wait complete. 2023-09-27T23:15:08.527 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-09-27T23:15:08.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-09-27T23:15:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.650+0000 7fb394a30700 1 Processor -- start 2023-09-27T23:15:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.650+0000 7fb394a30700 1 -- start start 2023-09-27T23:15:08.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb39015c930 0x7fb39006aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900b0ac0 con 0x7fb39006a6a0 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39006b030 con 0x7fb39015c930 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39006b210 con 0x7fb3900cb130 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb39015c930 0x7fb39006aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb39015c930 0x7fb39006aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39564/0 (socket says 172.21.15.64:39564) 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 -- 172.21.15.64:0/1031016306 learned_addr learned my addr 172.21.15.64:0/1031016306 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38e59c700 1 --2- 172.21.15.64:0/1031016306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 -- 172.21.15.64:0/1031016306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 msgr2=0x7fb3900cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 --2- 172.21.15.64:0/1031016306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 -- 172.21.15.64:0/1031016306 >> v1:172.21.15.123:6789/0 conn(0x7fb39006a6a0 legacy=0x7fb39015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 -- 172.21.15.64:0/1031016306 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900ca810 con 0x7fb39015c930 2023-09-27T23:15:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38ed9d700 1 --2- 172.21.15.64:0/1031016306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb39006aaf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb388009f20 tx=0x7fb388009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a9bf064dd1ee89 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38d59a700 1 -- 172.21.15.64:0/1031016306 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800e070 con 0x7fb39015c930 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38d59a700 1 -- 172.21.15.64:0/1031016306 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38800ba80 con 0x7fb39015c930 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb38d59a700 1 -- 172.21.15.64:0/1031016306 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880092e0 con 0x7fb39015c930 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- 172.21.15.64:0/1031016306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 msgr2=0x7fb39006aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 --2- 172.21.15.64:0/1031016306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb39006aaf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb388009f20 tx=0x7fb388009840 comp rx=0 tx=0).stop 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- 172.21.15.64:0/1031016306 shutdown_connections 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 --2- 172.21.15.64:0/1031016306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb39006aaf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 --2- 172.21.15.64:0/1031016306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- 172.21.15.64:0/1031016306 >> 172.21.15.64:0/1031016306 conn(0x7fb390067bb0 msgr2=0x7fb3900cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:08.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.654+0000 7fb394a30700 1 -- 172.21.15.64:0/1031016306 shutdown_connections 2023-09-27T23:15:08.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.658+0000 7fb394a30700 1 -- 172.21.15.64:0/1031016306 wait complete. 2023-09-27T23:15:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.658+0000 7fb394a30700 1 Processor -- start 2023-09-27T23:15:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.658+0000 7fb394a30700 1 -- start start 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39006a6a0 0x7fb39013e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb3900cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015da80 con 0x7fb39015c930 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015ce70 con 0x7fb39006a6a0 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015d150 con 0x7fb3900cb130 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb3900cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb3900cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39570/0 (socket says 172.21.15.64:39570) 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 -- 172.21.15.64:0/3853070394 learned_addr learned my addr 172.21.15.64:0/3853070394 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38e59c700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39006a6a0 0x7fb39013e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38dd9b700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 -- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 msgr2=0x7fb3900cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 -- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39006a6a0 msgr2=0x7fb39013e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39006a6a0 0x7fb39013e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 -- 172.21.15.64:0/3853070394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388009ae0 con 0x7fb39015c930 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb38ed9d700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb3900cef80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb39006f300 tx=0x7fb388018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880032f0 con 0x7fb39015c930 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388008d20 con 0x7fb39015c930 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39015d430 con 0x7fb39015c930 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38801c640 con 0x7fb39015c930 2023-09-27T23:15:08.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.662+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39013ef20 con 0x7fb39015c930 2023-09-27T23:15:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.666+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7fb388003040 con 0x7fb39015c930 2023-09-27T23:15:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.666+0000 7fb3867fc700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb370060b50 0x7fb370063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.666+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fb38807ed40 con 0x7fb39015c930 2023-09-27T23:15:08.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.666+0000 7fb38e59c700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb370060b50 0x7fb370063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:08.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.666+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39015dbf0 con 0x7fb39015c930 2023-09-27T23:15:08.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.666+0000 7fb38e59c700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb370060b50 0x7fb370063000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb380006fd0 tx=0x7fb380008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:08.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.674+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb38804e9f0 con 0x7fb39015c930 2023-09-27T23:15:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.854+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb390071120 con 0x7fb370060b50 2023-09-27T23:15:08.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.958+0000 7fb3867fc700 1 -- 172.21.15.64:0/3853070394 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb390071120 con 0x7fb370060b50 2023-09-27T23:15:08.960 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.958+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb370060b50 msgr2=0x7fb370063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.958+0000 7fb394a30700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb370060b50 0x7fb370063000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb380006fd0 tx=0x7fb380008040 comp rx=0 tx=0).stop 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 msgr2=0x7fb3900cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb3900cef80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb39006f300 tx=0x7fb388018000 comp rx=0 tx=0).stop 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 shutdown_connections 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb370060b50 0x7fb370063000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39015c930 0x7fb3900cef80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3900cb130 0x7fb3900cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 --2- 172.21.15.64:0/3853070394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39006a6a0 0x7fb39013e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 >> 172.21.15.64:0/3853070394 conn(0x7fb390067bb0 msgr2=0x7fb3900b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 shutdown_connections 2023-09-27T23:15:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:08.962+0000 7fb394a30700 1 -- 172.21.15.64:0/3853070394 wait complete. 2023-09-27T23:15:08.981 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-09-27T23:15:08.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-09-27T23:15:09.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.090+0000 7f61b6f1b700 1 Processor -- start 2023-09-27T23:15:09.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.090+0000 7f61b6f1b700 1 -- start start 2023-09-27T23:15:09.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b013f830 0x7f61b00ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61b00aea30 0x7f61b00aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b00cc200 con 0x7f61b01563e0 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b00cb160 con 0x7f61b00aea30 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00cb440 con 0x7f61b013f830 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b54b8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f61b01563e0 0x7f61b0156840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55600/0 (socket says 172.21.15.64:55600) 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b54b8700 1 -- 172.21.15.64:0/1476409875 learned_addr learned my addr 172.21.15.64:0/1476409875 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61affff700 1 --2- 172.21.15.64:0/1476409875 >> v2:172.21.15.123:3300/0 conn(0x7f61b00aea30 0x7f61b00aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b4cb7700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b013f830 0x7f61b00ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61af7fe700 1 -- 172.21.15.64:0/1476409875 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203880291 0 0) 0x7f61b00cc200 con 0x7f61b01563e0 2023-09-27T23:15:09.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61af7fe700 1 -- 172.21.15.64:0/1476409875 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a0003570 con 0x7f61b01563e0 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61affff700 1 -- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b013f830 msgr2=0x7f61b00ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61affff700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b013f830 0x7f61b00ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61affff700 1 -- 172.21.15.64:0/1476409875 >> v1:172.21.15.64:6789/0 conn(0x7f61b01563e0 legacy=0x7f61b0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61affff700 1 -- 172.21.15.64:0/1476409875 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00cb720 con 0x7f61b00aea30 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b4cb7700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b013f830 0x7f61b00ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61affff700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b00aea30 0x7f61b00aeea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f619c0098e0 tx=0x7f619c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8783db1da53fc593 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61af7fe700 1 -- 172.21.15.64:0/1476409875 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c011630 con 0x7f61b00aea30 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61af7fe700 1 -- 172.21.15.64:0/1476409875 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c011790 con 0x7f61b00aea30 2023-09-27T23:15:09.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61af7fe700 1 -- 172.21.15.64:0/1476409875 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c011b30 con 0x7f61b00aea30 2023-09-27T23:15:09.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 -- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b00aea30 msgr2=0x7f61b00aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.094+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b00aea30 0x7f61b00aeea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f619c0098e0 tx=0x7f619c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 -- 172.21.15.64:0/1476409875 shutdown_connections 2023-09-27T23:15:09.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b00aea30 0x7f61b00aeea0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/1476409875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b013f830 0x7f61b00ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 -- 172.21.15.64:0/1476409875 >> 172.21.15.64:0/1476409875 conn(0x7f61b00ccc20 msgr2=0x7f61b00cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:09.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 -- 172.21.15.64:0/1476409875 shutdown_connections 2023-09-27T23:15:09.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 -- 172.21.15.64:0/1476409875 wait complete. 2023-09-27T23:15:09.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 Processor -- start 2023-09-27T23:15:09.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.098+0000 7f61b6f1b700 1 -- start start 2023-09-27T23:15:09.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b00aea30 0x7f61b0069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b013f830 0x7f61b00c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 0x7f61b00b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00b0580 con 0x7f61b01563e0 2023-09-27T23:15:09.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00c4210 con 0x7f61b013f830 2023-09-27T23:15:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00c44b0 con 0x7f61b00aea30 2023-09-27T23:15:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 0x7f61b00b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 0x7f61b00b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39584/0 (socket says 172.21.15.64:39584) 2023-09-27T23:15:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 -- 172.21.15.64:0/3822720805 learned_addr learned my addr 172.21.15.64:0/3822720805 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61affff700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b013f830 0x7f61b00c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b4cb7700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b00aea30 0x7f61b0069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 -- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b00aea30 msgr2=0x7f61b0069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b00aea30 0x7f61b0069790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 -- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b013f830 msgr2=0x7f61b00c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b013f830 0x7f61b00c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 -- 172.21.15.64:0/3822720805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a400b6d0 con 0x7f61b01563e0 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b54b8700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 0x7f61b00b20c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f61a8001460 tx=0x7f61a8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:09.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800a870 con 0x7f61b01563e0 2023-09-27T23:15:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a800a9d0 con 0x7f61b01563e0 2023-09-27T23:15:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c009420 con 0x7f61b01563e0 2023-09-27T23:15:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800ad50 con 0x7f61b01563e0 2023-09-27T23:15:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.102+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b00c4a70 con 0x7f61b01563e0 2023-09-27T23:15:09.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.106+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b00b06f0 con 0x7f61b01563e0 2023-09-27T23:15:09.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.106+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f61a8007300 con 0x7f61b01563e0 2023-09-27T23:15:09.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.106+0000 7f61acff9700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6194060b50 0x7f6194063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.106+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f61a807e230 con 0x7f61b01563e0 2023-09-27T23:15:09.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.106+0000 7f61b4cb7700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6194060b50 0x7f6194063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.110+0000 7f61b4cb7700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6194060b50 0x7f6194063000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61a400bfd0 tx=0x7f61a400b430 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:09.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.110+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f61a804e0c0 con 0x7f61b01563e0 2023-09-27T23:15:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.298+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f61b0071120 con 0x7f6194060b50 2023-09-27T23:15:09.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.402+0000 7f61acff9700 1 -- 172.21.15.64:0/3822720805 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f61b0071120 con 0x7f6194060b50 2023-09-27T23:15:09.407 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-09-27T23:15:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6194060b50 msgr2=0x7f6194063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6194060b50 0x7f6194063000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61a400bfd0 tx=0x7f61a400b430 comp rx=0 tx=0).stop 2023-09-27T23:15:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 msgr2=0x7f61b00b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 0x7f61b00b20c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f61a8001460 tx=0x7f61a8001a40 comp rx=0 tx=0).stop 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 shutdown_connections 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6194060b50 0x7f6194063000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61b01563e0 0x7f61b00b20c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b013f830 0x7f61b00c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 --2- 172.21.15.64:0/3822720805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b00aea30 0x7f61b0069790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 >> 172.21.15.64:0/3822720805 conn(0x7f61b00ccc20 msgr2=0x7f61b00c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 shutdown_connections 2023-09-27T23:15:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.410+0000 7f61b6f1b700 1 -- 172.21.15.64:0/3822720805 wait complete. 2023-09-27T23:15:09.429 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-09-27T23:15:09.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-09-27T23:15:09.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 Processor -- start 2023-09-27T23:15:09.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 -- start start 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f28ec0687f0 0x7f28ec06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ec0ad3b0 con 0x7f28ec0b02b0 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec069120 con 0x7f28ec0687f0 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f1616700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec068c50 con 0x7f28ec158a10 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28eb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f28ec0b02b0 0x7f28ec0680e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55606/0 (socket says 172.21.15.64:55606) 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28eb7fe700 1 -- 172.21.15.64:0/1690554088 learned_addr learned my addr 172.21.15.64:0/1690554088 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28f0e15700 1 --2- 172.21.15.64:0/1690554088 >> v2:172.21.15.123:3300/0 conn(0x7f28ec0687f0 0x7f28ec06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28ebfff700 1 --2- 172.21.15.64:0/1690554088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28eaffd700 1 -- 172.21.15.64:0/1690554088 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191482216 0 0) 0x7f28ec0ad3b0 con 0x7f28ec0b02b0 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28eaffd700 1 -- 172.21.15.64:0/1690554088 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d4003570 con 0x7f28ec0b02b0 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28ebfff700 1 -- 172.21.15.64:0/1690554088 >> v2:172.21.15.123:3300/0 conn(0x7f28ec0687f0 msgr2=0x7f28ec06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28ebfff700 1 --2- 172.21.15.64:0/1690554088 >> v2:172.21.15.123:3300/0 conn(0x7f28ec0687f0 0x7f28ec06d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28eaffd700 1 -- 172.21.15.64:0/1690554088 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3287444193 0 0) 0x7f28d4003570 con 0x7f28ec0b02b0 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28ebfff700 1 -- 172.21.15.64:0/1690554088 >> v1:172.21.15.64:6789/0 conn(0x7f28ec0b02b0 legacy=0x7f28ec0680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28ebfff700 1 -- 172.21.15.64:0/1690554088 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec068e40 con 0x7f28ec158a10 2023-09-27T23:15:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28ebfff700 1 --2- 172.21.15.64:0/1690554088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec158de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0098e0 tx=0x7f28dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d0e2ff688ab35c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.554+0000 7f28eaffd700 1 -- 172.21.15.64:0/1690554088 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0001470 con 0x7f28ec158a10 2023-09-27T23:15:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28eaffd700 1 -- 172.21.15.64:0/1690554088 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc011420 con 0x7f28ec158a10 2023-09-27T23:15:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28eaffd700 1 -- 172.21.15.64:0/1690554088 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0117a0 con 0x7f28ec158a10 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 -- 172.21.15.64:0/1690554088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 msgr2=0x7f28ec158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 --2- 172.21.15.64:0/1690554088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec158de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0098e0 tx=0x7f28dc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 -- 172.21.15.64:0/1690554088 shutdown_connections 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 --2- 172.21.15.64:0/1690554088 >> v2:172.21.15.123:3300/0 conn(0x7f28ec0687f0 0x7f28ec06d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 --2- 172.21.15.64:0/1690554088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec158de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 -- 172.21.15.64:0/1690554088 >> 172.21.15.64:0/1690554088 conn(0x7f28ec0ce5e0 msgr2=0x7f28ec0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 -- 172.21.15.64:0/1690554088 shutdown_connections 2023-09-27T23:15:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 -- 172.21.15.64:0/1690554088 wait complete. 2023-09-27T23:15:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 Processor -- start 2023-09-27T23:15:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.558+0000 7f28f1616700 1 -- start start 2023-09-27T23:15:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec0687f0 0x7f28ec0c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 0x7f28ec0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0c2ac0 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec154850 con 0x7f28ec0687f0 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0c1e40 con 0x7f28ec158a10 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 0x7f28ec0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 0x7f28ec0b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39596/0 (socket says 172.21.15.64:39596) 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 -- 172.21.15.64:0/181579603 learned_addr learned my addr 172.21.15.64:0/181579603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28eb7fe700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28ebfff700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec0687f0 0x7f28ec0c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 -- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 msgr2=0x7f28ec0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec0b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 -- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec0687f0 msgr2=0x7f28ec0c17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec0687f0 0x7f28ec0c17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 -- 172.21.15.64:0/181579603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc009420 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f0e15700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 0x7f28ec0b0d20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009df0 tx=0x7f28e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:09.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400caa0 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e400f040 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec0c2120 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400ce70 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.562+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec0c26e0 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.566+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f28e4014690 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.566+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec0c2c00 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.566+0000 7f28d3fff700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28d8060b50 0x7f28d8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.566+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f28e407dd20 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.566+0000 7f28ebfff700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28d8060b50 0x7f28d8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.566+0000 7f28ebfff700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28d8060b50 0x7f28d8063000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0091e0 tx=0x7f28dc016000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:09.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.578+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f28ec0c2c00 con 0x7f28ec0c2ee0 2023-09-27T23:15:09.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.762+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f28ec0c2c00 con 0x7f28d8060b50 2023-09-27T23:15:09.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.870+0000 7f28d3fff700 1 -- 172.21.15.64:0/181579603 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f28ec0c2c00 con 0x7f28d8060b50 2023-09-27T23:15:09.873 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-09-27T23:15:09.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28d8060b50 msgr2=0x7f28d8063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28d8060b50 0x7f28d8063000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0091e0 tx=0x7f28dc016000 comp rx=0 tx=0).stop 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 msgr2=0x7f28ec0b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 0x7f28ec0b0d20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009df0 tx=0x7f28e400b040 comp rx=0 tx=0).stop 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 shutdown_connections 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28d8060b50 0x7f28d8063000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28ec0c2ee0 0x7f28ec0b0d20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28ec158a10 0x7f28ec0b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 --2- 172.21.15.64:0/181579603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec0687f0 0x7f28ec0c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:09.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 >> 172.21.15.64:0/181579603 conn(0x7f28ec0ce5e0 msgr2=0x7f28ec0af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 shutdown_connections 2023-09-27T23:15:09.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:09.874+0000 7f28f1616700 1 -- 172.21.15.64:0/181579603 wait complete. 2023-09-27T23:15:09.895 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-09-27T23:15:09.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-09-27T23:15:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.014+0000 7f64cc8a9700 1 Processor -- start 2023-09-27T23:15:10.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.014+0000 7f64cc8a9700 1 -- start start 2023-09-27T23:15:10.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64cc8a9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64c806e550 0x7f64c81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64cc8a9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c806e0c0 con 0x7f64c8069570 2023-09-27T23:15:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64cc8a9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c815a2c0 con 0x7f64c814fd20 2023-09-27T23:15:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64cc8a9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c815a5a0 con 0x7f64c806e550 2023-09-27T23:15:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c6ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f64c8069570 0x7f64c8159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55614/0 (socket says 172.21.15.64:55614) 2023-09-27T23:15:10.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c6ffd700 1 -- 172.21.15.64:0/489032630 learned_addr learned my addr 172.21.15.64:0/489032630 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:15:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c7fff700 1 --2- 172.21.15.64:0/489032630 >> v2:172.21.15.123:3300/0 conn(0x7f64c806e550 0x7f64c81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716607217 0 0) 0x7f64c806e0c0 con 0x7f64c8069570 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b0003570 con 0x7f64c8069570 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527099424 0 0) 0x7f64b0003570 con 0x7f64c8069570 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 >> v2:172.21.15.123:3300/0 conn(0x7f64c806e550 msgr2=0x7f64c81500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 --2- 172.21.15.64:0/489032630 >> v2:172.21.15.123:3300/0 conn(0x7f64c806e550 0x7f64c81500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 >> v1:172.21.15.123:6789/0 conn(0x7f64c814fd20 legacy=0x7f64c8068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c806da70 con 0x7f64c8069570 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f64bc001470 con 0x7f64c8069570 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c7fff700 1 --2- 172.21.15.64:0/489032630 >> v2:172.21.15.123:3300/0 conn(0x7f64c806e550 0x7f64c81500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 >> v1:172.21.15.64:6789/0 conn(0x7f64c8069570 legacy=0x7f64c8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b0003a90 0x7f64b0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0004810 0x7f64b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b00051c0 0x7f64b000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c815a5a0 con 0x7f64b0003a90 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b000e290 con 0x7f64b00051c0 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b000e450 con 0x7f64b0004810 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b0003a90 0x7f64b0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c7fff700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b00051c0 0x7f64b000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c6ffd700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0004810 0x7f64b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 -- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0004810 msgr2=0x7f64b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0004810 0x7f64b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 -- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b00051c0 msgr2=0x7f64b000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b00051c0 0x7f64b000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 -- 172.21.15.64:0/489032630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b0003570 con 0x7f64b0003a90 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c7fff700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b00051c0 0x7f64b000dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c7fff700 1 -- 172.21.15.64:0/489032630 reap_dead start 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c77fe700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b0003a90 0x7f64b0003e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001880 tx=0x7f64b8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebd91d9646fdd26 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8012420 con 0x7f64b0003a90 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80125a0 con 0x7f64b0003a90 2023-09-27T23:15:10.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.018+0000 7f64c67fc700 1 -- 172.21.15.64:0/489032630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8012960 con 0x7f64b0003a90 2023-09-27T23:15:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 -- 172.21.15.64:0/489032630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b0003a90 msgr2=0x7f64b0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b0003a90 0x7f64b0003e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001880 tx=0x7f64b8007d30 comp rx=0 tx=0).stop 2023-09-27T23:15:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 -- 172.21.15.64:0/489032630 shutdown_connections 2023-09-27T23:15:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/489032630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b0003a90 0x7f64b0003e60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 -- 172.21.15.64:0/489032630 >> 172.21.15.64:0/489032630 conn(0x7f64c80ccaa0 msgr2=0x7f64c80ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 -- 172.21.15.64:0/489032630 shutdown_connections 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 -- 172.21.15.64:0/489032630 wait complete. 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 Processor -- start 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 -- start start 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0003a90 0x7f64c814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c815ba20 0x7f64c815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.022+0000 7f64cc8a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 0x7f64c8169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64cc8a9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8169cc0 con 0x7f64c8169310 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64cc8a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c815b590 con 0x7f64c815ba20 2023-09-27T23:15:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64cc8a9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c815a6e0 con 0x7f64b0003a90 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 0x7f64c8169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 0x7f64c8169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39612/0 (socket says 172.21.15.64:39612) 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 -- 172.21.15.64:0/3100588061 learned_addr learned my addr 172.21.15.64:0/3100588061 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c6ffd700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c815ba20 0x7f64c815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c77fe700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0003a90 0x7f64c814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 -- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0003a90 msgr2=0x7f64c814d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0003a90 0x7f64c814d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 -- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c815ba20 msgr2=0x7f64c815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c815ba20 0x7f64c815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 -- 172.21.15.64:0/3100588061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c814d690 con 0x7f64c8169310 2023-09-27T23:15:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64c7fff700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 0x7f64c8169780 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f64c806ac20 tx=0x7f64c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000e070 con 0x7f64c8169310 2023-09-27T23:15:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c0011c50 con 0x7f64c8169310 2023-09-27T23:15:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c815a980 con 0x7f64c8169310 2023-09-27T23:15:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0016470 con 0x7f64c8169310 2023-09-27T23:15:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c815af40 con 0x7f64c8169310 2023-09-27T23:15:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.026+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c815b700 con 0x7f64c8169310 2023-09-27T23:15:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.030+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f64c0016820 con 0x7f64c8169310 2023-09-27T23:15:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.030+0000 7f64af7fe700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64b4060b80 0x7f64b4063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:15:10.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.030+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f64c007e010 con 0x7f64c8169310 2023-09-27T23:15:10.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.030+0000 7f64c77fe700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64b4060b80 0x7f64b4063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:15:10.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.030+0000 7f64c77fe700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64b4060b80 0x7f64b4063030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001010 tx=0x7f64b8015000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:15:10.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.034+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f64c004dcc0 con 0x7f64c8169310 2023-09-27T23:15:10.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.218+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f64c804be40 con 0x7f64b4060b80 2023-09-27T23:15:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.326+0000 7f64af7fe700 1 -- 172.21.15.64:0/3100588061 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f64c804be40 con 0x7f64b4060b80 2023-09-27T23:15:10.329 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-09-27T23:15:10.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64b4060b80 msgr2=0x7f64b4063030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64b4060b80 0x7f64b4063030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001010 tx=0x7f64b8015000 comp rx=0 tx=0).stop 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 msgr2=0x7f64c8169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 0x7f64c8169780 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f64c806ac20 tx=0x7f64c000b040 comp rx=0 tx=0).stop 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 shutdown_connections 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64b4060b80 0x7f64b4063030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c8169310 0x7f64c8169780 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c815ba20 0x7f64c815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 --2- 172.21.15.64:0/3100588061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b0003a90 0x7f64c814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 >> 172.21.15.64:0/3100588061 conn(0x7f64c80ccaa0 msgr2=0x7f64c80cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 shutdown_connections 2023-09-27T23:15:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:15:10.330+0000 7f64cc8a9700 1 -- 172.21.15.64:0/3100588061 wait complete. 2023-09-27T23:15:10.351 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-09-27T23:15:17.538 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:15:17.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:15:47.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:15:47.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:16:00.058 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:16:00.050+0000 7f80410bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:16:00.059 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:16:00.050+0000 7f80410bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:16:00.059 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:16:00.050+0000 7f80410bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:16:00.059 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:16:00.050+0000 7f80410bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:16:00.059 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:16:00.050+0000 7f80410bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:16:17.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:16:17.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:16:47.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:16:47.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:17:17.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:17:17.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:17:47.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:17:47.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:18:17.878 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:18:17.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:18:47.959 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:18:47.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:19:18.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:19:18.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:19:48.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:19:48.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:20:18.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:20:18.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:20:48.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:20:48.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:21:00.212 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:21:00.208+0000 7f80408ba700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:21:00.212 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:21:00.208+0000 7f80408ba700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:21:00.212 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:21:00.208+0000 7f80408ba700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:21:00.212 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:21:00.208+0000 7f80408ba700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:21:00.212 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:21:00.208+0000 7f80408ba700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:21:18.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:21:18.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:21:48.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:21:48.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:22:18.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:22:18.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:22:48.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:22:48.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:23:18.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:23:18.519 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:23:48.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:23:48.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:24:18.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:24:18.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:24:48.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:24:48.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:25:18.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:25:18.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:25:48.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:25:48.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:26:18.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:26:18.994 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:26:20.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-09-27T23:26:20.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.476+0000 7f2ad8808700 1 Processor -- start 2023-09-27T23:26:20.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.476+0000 7f2ad8808700 1 -- start start 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ed80 0x7f2ad406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad406b420 con 0x7f2ad406df10 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad415d720 con 0x7f2ad4158da0 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40ca290 con 0x7f2ad406ed80 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2ad406df10 0x7f2ad406ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33394/0 (socket says 172.21.15.64:33394) 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad3fff700 1 -- 172.21.15.64:0/1606817282 learned_addr learned my addr 172.21.15.64:0/1606817282 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:20.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/1606817282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ed80 0x7f2ad406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258053758 0 0) 0x7f2ad406b420 con 0x7f2ad406df10 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2abc003570 con 0x7f2ad406df10 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554168325 0 0) 0x7f2ad415d720 con 0x7f2ad4158da0 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad406b420 con 0x7f2ad4158da0 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/1606817282 >> v1:172.21.15.123:6789/0 conn(0x7f2ad4158da0 legacy=0x7f2ad406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/1606817282 >> v1:172.21.15.64:6789/0 conn(0x7f2ad406df10 legacy=0x7f2ad406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/1606817282 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad40ca4a0 con 0x7f2ad406ed80 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/1606817282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ed80 0x7f2ad406a7d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009910 tx=0x7f2ac40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0003f90a09ca6b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4011630 con 0x7f2ad406ed80 2023-09-27T23:26:20.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac4011790 con 0x7f2ad406ed80 2023-09-27T23:26:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1606817282 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4011b30 con 0x7f2ad406ed80 2023-09-27T23:26:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- 172.21.15.64:0/1606817282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ed80 msgr2=0x7f2ad406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 --2- 172.21.15.64:0/1606817282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ed80 0x7f2ad406a7d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009910 tx=0x7f2ac40091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- 172.21.15.64:0/1606817282 shutdown_connections 2023-09-27T23:26:20.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 --2- 172.21.15.64:0/1606817282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ed80 0x7f2ad406a7d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- 172.21.15.64:0/1606817282 >> 172.21.15.64:0/1606817282 conn(0x7f2ad40cc760 msgr2=0x7f2ad40ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- 172.21.15.64:0/1606817282 shutdown_connections 2023-09-27T23:26:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.480+0000 7f2ad8808700 1 -- 172.21.15.64:0/1606817282 wait complete. 2023-09-27T23:26:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 Processor -- start 2023-09-27T23:26:20.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 -- start start 2023-09-27T23:26:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406df10 0x7f2ad4155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 0x7f2ad40c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4158da0 0x7f2ad40c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41555d0 con 0x7f2ad406ed80 2023-09-27T23:26:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40c3690 con 0x7f2ad4158da0 2023-09-27T23:26:20.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40c3970 con 0x7f2ad406df10 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 0x7f2ad40c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 0x7f2ad40c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38032/0 (socket says 172.21.15.64:38032) 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/3260104251 learned_addr learned my addr 172.21.15.64:0/3260104251 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406df10 0x7f2ad4155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad3fff700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4158da0 0x7f2ad40c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406df10 msgr2=0x7f2ad4155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406df10 0x7f2ad4155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4158da0 msgr2=0x7f2ad40c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4158da0 0x7f2ad40c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/3260104251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4009420 con 0x7f2ad406ed80 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad3fff700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4158da0 0x7f2ad40c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 0x7f2ad40c76c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4068230 tx=0x7f2ac4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4000d10 con 0x7f2ad406ed80 2023-09-27T23:26:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac4011b10 con 0x7f2ad406ed80 2023-09-27T23:26:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4025630 con 0x7f2ad406ed80 2023-09-27T23:26:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad40c3c50 con 0x7f2ad406ed80 2023-09-27T23:26:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.484+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad40c4210 con 0x7f2ad406ed80 2023-09-27T23:26:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.488+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f2ac4025a60 con 0x7f2ad406ed80 2023-09-27T23:26:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.488+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad404be40 con 0x7f2ad406ed80 2023-09-27T23:26:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.488+0000 7f2abb7fe700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060b50 0x7f2ac0063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.488+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f2ac407ee00 con 0x7f2ad406ed80 2023-09-27T23:26:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.488+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060b50 0x7f2ac0063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.488+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060b50 0x7f2ac0063000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007fd0 tx=0x7f2ac8007d30 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.492+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2ac404eb80 con 0x7f2ad406ed80 2023-09-27T23:26:20.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.684+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ad406b200 con 0x7f2ac0060b50 2023-09-27T23:26:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.832+0000 7f2abb7fe700 1 -- 172.21.15.64:0/3260104251 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2ad406b200 con 0x7f2ac0060b50 2023-09-27T23:26:20.839 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-09-27T23:26:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060b50 msgr2=0x7f2ac0063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060b50 0x7f2ac0063000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007fd0 tx=0x7f2ac8007d30 comp rx=0 tx=0).stop 2023-09-27T23:26:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 msgr2=0x7f2ad40c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 0x7f2ad40c76c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4068230 tx=0x7f2ac4017000 comp rx=0 tx=0).stop 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 shutdown_connections 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060b50 0x7f2ac0063000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4158da0 0x7f2ad40c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad406ed80 0x7f2ad40c76c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 --2- 172.21.15.64:0/3260104251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406df10 0x7f2ad4155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 >> 172.21.15.64:0/3260104251 conn(0x7f2ad40cc760 msgr2=0x7f2ad40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 shutdown_connections 2023-09-27T23:26:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.840+0000 7f2ad8808700 1 -- 172.21.15.64:0/3260104251 wait complete. 2023-09-27T23:26:20.860 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-09-27T23:26:20.861 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-09-27T23:26:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 Processor -- start 2023-09-27T23:26:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 -- start start 2023-09-27T23:26:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc22c158a10 0x7fc22c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0687f0 0x7fc22c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c0ad3b0 con 0x7fc22c0b02b0 2023-09-27T23:26:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c069120 con 0x7fc22c158a10 2023-09-27T23:26:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c068e40 con 0x7fc22c0687f0 2023-09-27T23:26:20.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22affd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc22c158a10 0x7fc22c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0687f0 0x7fc22c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22a7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc22c0b02b0 0x7fc22c0680e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59758/0 (socket says 172.21.15.64:59758) 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22affd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc22c158a10 0x7fc22c158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:44254/0 (socket says 172.21.15.64:44254) 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22a7fc700 1 -- 172.21.15.64:0/3152226555 learned_addr learned my addr 172.21.15.64:0/3152226555 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22affd700 1 -- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0687f0 msgr2=0x7fc22c06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.980+0000 7fc22affd700 1 --2- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0687f0 0x7fc22c06d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22affd700 1 -- 172.21.15.64:0/3152226555 >> v1:172.21.15.123:6789/0 conn(0x7fc22c0b02b0 legacy=0x7fc22c0680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22affd700 1 -- 172.21.15.64:0/3152226555 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c0ad7c0 con 0x7fc22c158a10 2023-09-27T23:26:20.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc229ffb700 1 -- 172.21.15.64:0/3152226555 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210592233 0 0) 0x7fc22c0ad3b0 con 0x7fc22c0b02b0 2023-09-27T23:26:20.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22affd700 1 --2- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c158a10 0x7fc22c158de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009e60 tx=0x7fc21c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a276c5a1bd9b945 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:20.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc229ffb700 1 -- 172.21.15.64:0/3152226555 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00e070 con 0x7fc22c158a10 2023-09-27T23:26:20.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc229ffb700 1 -- 172.21.15.64:0/3152226555 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c00bb60 con 0x7fc22c158a10 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc229ffb700 1 -- 172.21.15.64:0/3152226555 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c012430 con 0x7fc22c158a10 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c158a10 msgr2=0x7fc22c158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c158a10 0x7fc22c158de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009e60 tx=0x7fc21c009780 comp rx=0 tx=0).stop 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- 172.21.15.64:0/3152226555 shutdown_connections 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0687f0 0x7fc22c06d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3152226555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c158a10 0x7fc22c158de0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- 172.21.15.64:0/3152226555 >> 172.21.15.64:0/3152226555 conn(0x7fc22c0ce5e0 msgr2=0x7fc22c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- 172.21.15.64:0/3152226555 shutdown_connections 2023-09-27T23:26:20.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- 172.21.15.64:0/3152226555 wait complete. 2023-09-27T23:26:20.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 Processor -- start 2023-09-27T23:26:20.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- start start 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c0687f0 0x7fc22c15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 0x7fc22c0c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0c1d50 0x7fc22c0c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0b2e50 con 0x7fc22c158a10 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c15ce10 con 0x7fc22c0687f0 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c154130 con 0x7fc22c0c1d50 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 0x7fc22c0c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 0x7fc22c0c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38042/0 (socket says 172.21.15.64:38042) 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c0687f0 0x7fc22c15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22a7fc700 1 -- 172.21.15.64:0/3126089081 learned_addr learned my addr 172.21.15.64:0/3126089081 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.984+0000 7fc22b7fe700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0c1d50 0x7fc22c0c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22affd700 1 -- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0c1d50 msgr2=0x7fc22c0c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22affd700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0c1d50 0x7fc22c0c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22affd700 1 -- 172.21.15.64:0/3126089081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 msgr2=0x7fc22c0c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22affd700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 0x7fc22c0c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22affd700 1 -- 172.21.15.64:0/3126089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c009a20 con 0x7fc22c0687f0 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22a7fc700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 0x7fc22c0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22affd700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c0687f0 0x7fc22c15c8d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0c9220 tx=0x7fc21c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c012c80 con 0x7fc22c0687f0 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c012de0 con 0x7fc22c0687f0 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0276c0 con 0x7fc22c0687f0 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c0b2250 con 0x7fc22c0687f0 2023-09-27T23:26:20.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c0b2790 con 0x7fc22c0687f0 2023-09-27T23:26:20.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.988+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c04be40 con 0x7fc22c0687f0 2023-09-27T23:26:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.992+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fc21c024070 con 0x7fc22c0687f0 2023-09-27T23:26:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.992+0000 7fc212ffd700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc218060b50 0x7fc218063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.992+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fc21c07f920 con 0x7fc22c0687f0 2023-09-27T23:26:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.992+0000 7fc22a7fc700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc218060b50 0x7fc218063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:20.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:20.992+0000 7fc22a7fc700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc218060b50 0x7fc218063000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc22c06b180 tx=0x7fc220007d90 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:21.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.000+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc21c04f6a0 con 0x7fc22c0687f0 2023-09-27T23:26:21.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.196+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc22c0640c0 con 0x7fc218060b50 2023-09-27T23:26:21.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.348+0000 7fc212ffd700 1 -- 172.21.15.64:0/3126089081 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc22c0640c0 con 0x7fc218060b50 2023-09-27T23:26:21.353 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-09-27T23:26:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc218060b50 msgr2=0x7fc218063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc218060b50 0x7fc218063000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc22c06b180 tx=0x7fc220007d90 comp rx=0 tx=0).stop 2023-09-27T23:26:21.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c0687f0 msgr2=0x7fc22c15c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c0687f0 0x7fc22c15c8d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0c9220 tx=0x7fc21c019000 comp rx=0 tx=0).stop 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 shutdown_connections 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc218060b50 0x7fc218063000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc22c0c1d50 0x7fc22c0c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc22c158a10 0x7fc22c0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 --2- 172.21.15.64:0/3126089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c0687f0 0x7fc22c15c8d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 >> 172.21.15.64:0/3126089081 conn(0x7fc22c0ce5e0 msgr2=0x7fc22c156f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 shutdown_connections 2023-09-27T23:26:21.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.352+0000 7fc22bfff700 1 -- 172.21.15.64:0/3126089081 wait complete. 2023-09-27T23:26:21.371 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-09-27T23:26:21.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-09-27T23:26:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.488+0000 7f6e52da0700 1 Processor -- start 2023-09-27T23:26:21.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.488+0000 7f6e52da0700 1 -- start start 2023-09-27T23:26:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e52da0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 0x7f6e4c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e52da0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e52da0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c0cc200 con 0x7f6e4c1563e0 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e52da0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c0cb160 con 0x7f6e4c13f830 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e52da0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0cb440 con 0x7f6e4c0aea30 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e50b3c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 0x7f6e4c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e50b3c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 0x7f6e4c0ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38050/0 (socket says 172.21.15.64:38050) 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e50b3c700 1 -- 172.21.15.64:0/1503096787 learned_addr learned my addr 172.21.15.64:0/1503096787 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e43fff700 1 --2- 172.21.15.64:0/1503096787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e437fe700 1 -- 172.21.15.64:0/1503096787 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781518299 0 0) 0x7f6e4c0cc200 con 0x7f6e4c1563e0 2023-09-27T23:26:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e437fe700 1 -- 172.21.15.64:0/1503096787 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e34003570 con 0x7f6e4c1563e0 2023-09-27T23:26:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e43fff700 1 -- 172.21.15.64:0/1503096787 >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 msgr2=0x7f6e4c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e43fff700 1 --2- 172.21.15.64:0/1503096787 >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 0x7f6e4c0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e43fff700 1 -- 172.21.15.64:0/1503096787 >> v1:172.21.15.123:6789/0 conn(0x7f6e4c1563e0 legacy=0x7f6e4c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e43fff700 1 -- 172.21.15.64:0/1503096787 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0cb720 con 0x7f6e4c0aea30 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e50b3c700 1 --2- 172.21.15.64:0/1503096787 >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 0x7f6e4c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e43fff700 1 --2- 172.21.15.64:0/1503096787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0aeed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0098e0 tx=0x7f6e3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6be6553a106d19 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e437fe700 1 -- 172.21.15.64:0/1503096787 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011630 con 0x7f6e4c0aea30 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e437fe700 1 -- 172.21.15.64:0/1503096787 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c011790 con 0x7f6e4c0aea30 2023-09-27T23:26:21.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.492+0000 7f6e437fe700 1 -- 172.21.15.64:0/1503096787 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011b30 con 0x7f6e4c0aea30 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 -- 172.21.15.64:0/1503096787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 msgr2=0x7f6e4c0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 --2- 172.21.15.64:0/1503096787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0aeed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0098e0 tx=0x7f6e3c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 -- 172.21.15.64:0/1503096787 shutdown_connections 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 --2- 172.21.15.64:0/1503096787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0aeed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 --2- 172.21.15.64:0/1503096787 >> v2:172.21.15.64:3300/0 conn(0x7f6e4c13f830 0x7f6e4c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 -- 172.21.15.64:0/1503096787 >> 172.21.15.64:0/1503096787 conn(0x7f6e4c0ccc20 msgr2=0x7f6e4c0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:21.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 -- 172.21.15.64:0/1503096787 shutdown_connections 2023-09-27T23:26:21.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.496+0000 7f6e52da0700 1 -- 172.21.15.64:0/1503096787 wait complete. 2023-09-27T23:26:21.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 Processor -- start 2023-09-27T23:26:21.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 -- start start 2023-09-27T23:26:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4c13f830 0x7f6e4c16d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 0x7f6e4c06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0aff00 con 0x7f6e4c1563e0 2023-09-27T23:26:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0b0130 con 0x7f6e4c13f830 2023-09-27T23:26:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0b0410 con 0x7f6e4c0aea30 2023-09-27T23:26:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 0x7f6e4c06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 0x7f6e4c06d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38060/0 (socket says 172.21.15.64:38060) 2023-09-27T23:26:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 -- 172.21.15.64:0/3120250204 learned_addr learned my addr 172.21.15.64:0/3120250204 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e43fff700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4c13f830 0x7f6e4c16d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e50b3c700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 -- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 msgr2=0x7f6e4c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 -- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4c13f830 msgr2=0x7f6e4c16d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4c13f830 0x7f6e4c16d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 -- 172.21.15.64:0/3120250204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480094f0 con 0x7f6e4c1563e0 2023-09-27T23:26:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e5133d700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 0x7f6e4c06d280 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c860 tx=0x7f6e44002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44002030 con 0x7f6e4c1563e0 2023-09-27T23:26:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e440021b0 con 0x7f6e4c1563e0 2023-09-27T23:26:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c009420 con 0x7f6e4c1563e0 2023-09-27T23:26:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44012630 con 0x7f6e4c1563e0 2023-09-27T23:26:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.500+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c16d2f0 con 0x7f6e4c1563e0 2023-09-27T23:26:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.504+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c071120 con 0x7f6e4c1563e0 2023-09-27T23:26:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.504+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f6e44012a20 con 0x7f6e4c1563e0 2023-09-27T23:26:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.504+0000 7f6e40ff9700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e38060b50 0x7f6e38063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.504+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f6e4407dbe0 con 0x7f6e4c1563e0 2023-09-27T23:26:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.504+0000 7f6e50b3c700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e38060b50 0x7f6e38063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.504+0000 7f6e50b3c700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e38060b50 0x7f6e38063000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006d20 tx=0x7f6e48009250 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:21.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.508+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6e4404d960 con 0x7f6e4c1563e0 2023-09-27T23:26:21.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.688+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e4c0b2ae0 con 0x7f6e38060b50 2023-09-27T23:26:21.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.848+0000 7f6e40ff9700 1 -- 172.21.15.64:0/3120250204 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6e4c0b2ae0 con 0x7f6e38060b50 2023-09-27T23:26:21.854 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-09-27T23:26:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e38060b50 msgr2=0x7f6e38063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e38060b50 0x7f6e38063000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006d20 tx=0x7f6e48009250 comp rx=0 tx=0).stop 2023-09-27T23:26:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 msgr2=0x7f6e4c06d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 0x7f6e4c06d280 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c860 tx=0x7f6e44002700 comp rx=0 tx=0).stop 2023-09-27T23:26:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 shutdown_connections 2023-09-27T23:26:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6e38060b50 0x7f6e38063000 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c1563e0 0x7f6e4c06d280 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4c13f830 0x7f6e4c16d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 --2- 172.21.15.64:0/3120250204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4c0aea30 0x7f6e4c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 >> 172.21.15.64:0/3120250204 conn(0x7f6e4c0ccc20 msgr2=0x7f6e4c062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 shutdown_connections 2023-09-27T23:26:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.852+0000 7f6e52da0700 1 -- 172.21.15.64:0/3120250204 wait complete. 2023-09-27T23:26:21.875 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-09-27T23:26:21.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-09-27T23:26:22.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493b59e700 1 Processor -- start 2023-09-27T23:26:22.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493b59e700 1 -- start start 2023-09-27T23:26:22.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f493c156990 0x7f493c06dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c0b0480 con 0x7f493c0adfc0 2023-09-27T23:26:22.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c06c7c0 con 0x7f493c156d60 2023-09-27T23:26:22.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c06d620 con 0x7f493c156990 2023-09-27T23:26:22.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493ad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f493c0adfc0 0x7f493c0c9970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33406/0 (socket says 172.21.15.64:33406) 2023-09-27T23:26:22.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493ad9d700 1 -- 172.21.15.64:0/333492335 learned_addr learned my addr 172.21.15.64:0/333492335 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:22.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493a59c700 1 --2- 172.21.15.64:0/333492335 >> v2:172.21.15.123:3300/0 conn(0x7f493c156990 0x7f493c06dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648293758 0 0) 0x7f493c0b0480 con 0x7f493c0adfc0 2023-09-27T23:26:22.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4928003570 con 0x7f493c0adfc0 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428173842 0 0) 0x7f493c06c7c0 con 0x7f493c156d60 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f493c0b0480 con 0x7f493c156d60 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355456868 0 0) 0x7f4928003570 con 0x7f493c0adfc0 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 >> v2:172.21.15.123:3300/0 conn(0x7f493c156990 msgr2=0x7f493c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 --2- 172.21.15.64:0/333492335 >> v2:172.21.15.123:3300/0 conn(0x7f493c156990 0x7f493c06dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 >> v1:172.21.15.123:6789/0 conn(0x7f493c156d60 legacy=0x7f493c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c06ca10 con 0x7f493c0adfc0 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f49340017f0 con 0x7f493c0adfc0 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f493a59c700 1 --2- 172.21.15.64:0/333492335 >> v2:172.21.15.123:3300/0 conn(0x7f493c156990 0x7f493c06dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 >> v1:172.21.15.64:6789/0 conn(0x7f493c0adfc0 legacy=0x7f493c0c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928003c40 0x7f4928004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4928004810 0x7f4928004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:21.996+0000 7f4939d9b700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4928005210 0x7f492800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c06d620 con 0x7f4928005210 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492800c130 con 0x7f4928003c40 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492800c2c0 con 0x7f4928004810 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493a59c700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928003c40 0x7f4928004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4928005210 0x7f492800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4931d9b700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4928004810 0x7f4928004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 -- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4928004810 msgr2=0x7f4928004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4928004810 0x7f4928004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 -- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928003c40 msgr2=0x7f4928004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928003c40 0x7f4928004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 -- 172.21.15.64:0/333492335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928003570 con 0x7f4928005210 2023-09-27T23:26:22.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 -- 172.21.15.64:0/333492335 reap_dead start 2023-09-27T23:26:22.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493ad9d700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4928005210 0x7f492800ba60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f493400a830 tx=0x7f493c06d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2f0101871cdf34 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340071a0 con 0x7f4928005210 2023-09-27T23:26:22.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934007300 con 0x7f4928005210 2023-09-27T23:26:22.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f4939d9b700 1 -- 172.21.15.64:0/333492335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340076c0 con 0x7f4928005210 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 -- 172.21.15.64:0/333492335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4928005210 msgr2=0x7f492800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4928005210 0x7f492800ba60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f493400a830 tx=0x7f493c06d760 comp rx=0 tx=0).stop 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 -- 172.21.15.64:0/333492335 shutdown_connections 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 --2- 172.21.15.64:0/333492335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4928005210 0x7f492800ba60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 -- 172.21.15.64:0/333492335 >> 172.21.15.64:0/333492335 conn(0x7f493c0ce5e0 msgr2=0x7f493c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 -- 172.21.15.64:0/333492335 shutdown_connections 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.000+0000 7f493b59e700 1 -- 172.21.15.64:0/333492335 wait complete. 2023-09-27T23:26:22.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 Processor -- start 2023-09-27T23:26:22.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 -- start start 2023-09-27T23:26:22.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928005210 0x7f493c0c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 0x7f493c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f493c155c70 0x7f493c068210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c1560e0 con 0x7f493c153970 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c5310 con 0x7f4928005210 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c068f00 con 0x7f493c155c70 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 0x7f493c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 0x7f493c155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38086/0 (socket says 172.21.15.64:38086) 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 -- 172.21.15.64:0/1511006053 learned_addr learned my addr 172.21.15.64:0/1511006053 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493a59c700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928005210 0x7f493c0c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f493ad9d700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f493c155c70 0x7f493c068210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 -- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f493c155c70 msgr2=0x7f493c068210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f493c155c70 0x7f493c068210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 -- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928005210 msgr2=0x7f493c0c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928005210 0x7f493c0c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 -- 172.21.15.64:0/1511006053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c0c4cc0 con 0x7f493c153970 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4931d9b700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 0x7f493c155730 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4924002b80 tx=0x7f4924007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.004+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400f8d0 con 0x7f493c153970 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400fa50 con 0x7f493c153970 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c0c4fa0 con 0x7f493c153970 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400fe10 con 0x7f493c153970 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c068750 con 0x7f493c153970 2023-09-27T23:26:22.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c0c4b20 con 0x7f493c153970 2023-09-27T23:26:22.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f4924006a30 con 0x7f493c153970 2023-09-27T23:26:22.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f4932ffd700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f491c060b80 0x7f491c063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f492407d700 con 0x7f493c153970 2023-09-27T23:26:22.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.008+0000 7f493a59c700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f491c060b80 0x7f491c063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.012+0000 7f493a59c700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f491c060b80 0x7f491c063030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f492c006f00 tx=0x7f492c0091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.016+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f493c0c4b20 con 0x7f493c153970 2023-09-27T23:26:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.200+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f493c0c4b20 con 0x7f491c060b80 2023-09-27T23:26:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.352+0000 7f4932ffd700 1 -- 172.21.15.64:0/1511006053 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f493c0c4b20 con 0x7f491c060b80 2023-09-27T23:26:22.356 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f491c060b80 msgr2=0x7f491c063030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f491c060b80 0x7f491c063030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f492c006f00 tx=0x7f492c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 msgr2=0x7f493c155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 0x7f493c155730 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4924002b80 tx=0x7f4924007d30 comp rx=0 tx=0).stop 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 shutdown_connections 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f491c060b80 0x7f491c063030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f493c155c70 0x7f493c068210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f493c153970 0x7f493c155730 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 --2- 172.21.15.64:0/1511006053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4928005210 0x7f493c0c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 >> 172.21.15.64:0/1511006053 conn(0x7f493c0ce5e0 msgr2=0x7f493c0cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:22.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 shutdown_connections 2023-09-27T23:26:22.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.356+0000 7f493b59e700 1 -- 172.21.15.64:0/1511006053 wait complete. 2023-09-27T23:26:22.377 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-09-27T23:26:22.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-09-27T23:26:22.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.492+0000 7f554aaa8700 1 Processor -- start 2023-09-27T23:26:22.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.492+0000 7f554aaa8700 1 -- start start 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f554aaa8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55440ae920 0x7f5544159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f554aaa8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f554415a240 0x7f5544159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f554aaa8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55440ae490 con 0x7f5544155a40 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f554aaa8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440ad880 con 0x7f55440ae920 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f554aaa8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55440adb60 con 0x7f554415a240 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55440ae920 0x7f5544159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55440ae920 0x7f5544159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38090/0 (socket says 172.21.15.64:38090) 2023-09-27T23:26:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 -- 172.21.15.64:0/2368940977 learned_addr learned my addr 172.21.15.64:0/2368940977 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f554a2a7700 1 --2- 172.21.15.64:0/2368940977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f554415a240 0x7f5544159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 -- 172.21.15.64:0/2368940977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f554415a240 msgr2=0x7f5544159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 --2- 172.21.15.64:0/2368940977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f554415a240 0x7f5544159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 -- 172.21.15.64:0/2368940977 >> v1:172.21.15.123:6789/0 conn(0x7f5544155a40 legacy=0x7f55440ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 -- 172.21.15.64:0/2368940977 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440ade40 con 0x7f55440ae920 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f55492a5700 1 --2- 172.21.15.64:0/2368940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544159620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f55340099b0 tx=0x7f5534009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec399f57c5359a3c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f5548aa4700 1 -- 172.21.15.64:0/2368940977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534009ea0 con 0x7f55440ae920 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f5548aa4700 1 -- 172.21.15.64:0/2368940977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553400b860 con 0x7f55440ae920 2023-09-27T23:26:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.496+0000 7f5548aa4700 1 -- 172.21.15.64:0/2368940977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553400bc00 con 0x7f55440ae920 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 -- 172.21.15.64:0/2368940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 msgr2=0x7f5544159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 --2- 172.21.15.64:0/2368940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544159620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f55340099b0 tx=0x7f5534009250 comp rx=0 tx=0).stop 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 -- 172.21.15.64:0/2368940977 shutdown_connections 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 --2- 172.21.15.64:0/2368940977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f554415a240 0x7f5544159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 --2- 172.21.15.64:0/2368940977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544159620 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 -- 172.21.15.64:0/2368940977 >> 172.21.15.64:0/2368940977 conn(0x7f55440c8c00 msgr2=0x7f55440c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 -- 172.21.15.64:0/2368940977 shutdown_connections 2023-09-27T23:26:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 -- 172.21.15.64:0/2368940977 wait complete. 2023-09-27T23:26:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 Processor -- start 2023-09-27T23:26:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.500+0000 7f554aaa8700 1 -- start start 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5544155a40 0x7f554414f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554415a240 0x7f55440b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554414f720 con 0x7f55440ae920 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55441538e0 con 0x7f554415a240 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5544152cd0 con 0x7f5544155a40 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38100/0 (socket says 172.21.15.64:38100) 2023-09-27T23:26:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 -- 172.21.15.64:0/1451465209 learned_addr learned my addr 172.21.15.64:0/1451465209 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554a2a7700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554415a240 0x7f55440b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f55492a5700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5544155a40 0x7f554414f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 -- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5544155a40 msgr2=0x7f554414f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5544155a40 0x7f554414f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 -- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554415a240 msgr2=0x7f55440b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554415a240 0x7f55440b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 -- 172.21.15.64:0/1451465209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55340094f0 con 0x7f55440ae920 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5549aa6700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544151200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f554000c590 tx=0x7f554000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540011590 con 0x7f55440ae920 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55400116f0 con 0x7f55440ae920 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5544152fb0 con 0x7f55440ae920 2023-09-27T23:26:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540011ab0 con 0x7f55440ae920 2023-09-27T23:26:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5544153570 con 0x7f55440ae920 2023-09-27T23:26:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5544166e70 con 0x7f55440ae920 2023-09-27T23:26:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f554001b3b0 con 0x7f55440ae920 2023-09-27T23:26:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.504+0000 7f5539ffb700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5528060b50 0x7f5528063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.508+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f554007c9f0 con 0x7f55440ae920 2023-09-27T23:26:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.508+0000 7f55492a5700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5528060b50 0x7f5528063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.508+0000 7f55492a5700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5528060b50 0x7f5528063000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55340099b0 tx=0x7f5534017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.512+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f554004c770 con 0x7f55440ae920 2023-09-27T23:26:22.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.696+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f554404be40 con 0x7f5528060b50 2023-09-27T23:26:22.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.848+0000 7f5539ffb700 1 -- 172.21.15.64:0/1451465209 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f554404be40 con 0x7f5528060b50 2023-09-27T23:26:22.853 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-09-27T23:26:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5528060b50 msgr2=0x7f5528063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5528060b50 0x7f5528063000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55340099b0 tx=0x7f5534017000 comp rx=0 tx=0).stop 2023-09-27T23:26:22.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 msgr2=0x7f5544151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544151200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f554000c590 tx=0x7f554000c950 comp rx=0 tx=0).stop 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 shutdown_connections 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5528060b50 0x7f5528063000 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554415a240 0x7f55440b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5544155a40 0x7f554414f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 --2- 172.21.15.64:0/1451465209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55440ae920 0x7f5544151200 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 >> 172.21.15.64:0/1451465209 conn(0x7f55440c8c00 msgr2=0x7f55440cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 shutdown_connections 2023-09-27T23:26:22.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.852+0000 7f554aaa8700 1 -- 172.21.15.64:0/1451465209 wait complete. 2023-09-27T23:26:22.871 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-09-27T23:26:22.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-09-27T23:26:22.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.988+0000 7f51ac8f6700 1 Processor -- start 2023-09-27T23:26:22.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.988+0000 7f51ac8f6700 1 -- start start 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a8158740 0x7f51a806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51a806d760 0x7f51a80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a806ded0 con 0x7f51a806e7b0 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a80c82f0 con 0x7f51a806d760 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c7fa0 con 0x7f51a8158740 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51a806d760 0x7f51a80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51a806d760 0x7f51a80ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38110/0 (socket says 172.21.15.64:38110) 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 -- 172.21.15.64:0/3840145977 learned_addr learned my addr 172.21.15.64:0/3840145977 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a6d9d700 1 --2- 172.21.15.64:0/3840145977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a8158740 0x7f51a806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:22.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 -- 172.21.15.64:0/3840145977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a8158740 msgr2=0x7f51a806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 --2- 172.21.15.64:0/3840145977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a8158740 0x7f51a806e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 -- 172.21.15.64:0/3840145977 >> v1:172.21.15.123:6789/0 conn(0x7f51a806e7b0 legacy=0x7f51a806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 -- 172.21.15.64:0/3840145977 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80c70f0 con 0x7f51a806d760 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a759e700 1 --2- 172.21.15.64:0/3840145977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806d760 0x7f51a80ac8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c0d0 tx=0x7f51a000a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8693545dcac4911c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a5d9b700 1 -- 172.21.15.64:0/3840145977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000f070 con 0x7f51a806d760 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a5d9b700 1 -- 172.21.15.64:0/3840145977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a000a2e0 con 0x7f51a806d760 2023-09-27T23:26:22.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51a5d9b700 1 -- 172.21.15.64:0/3840145977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000cd80 con 0x7f51a806d760 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- 172.21.15.64:0/3840145977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806d760 msgr2=0x7f51a80ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/3840145977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806d760 0x7f51a80ac8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c0d0 tx=0x7f51a000a950 comp rx=0 tx=0).stop 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- 172.21.15.64:0/3840145977 shutdown_connections 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/3840145977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806d760 0x7f51a80ac8e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/3840145977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a8158740 0x7f51a806e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- 172.21.15.64:0/3840145977 >> 172.21.15.64:0/3840145977 conn(0x7f51a80c8c00 msgr2=0x7f51a80c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- 172.21.15.64:0/3840145977 shutdown_connections 2023-09-27T23:26:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.992+0000 7f51ac8f6700 1 -- 172.21.15.64:0/3840145977 wait complete. 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 Processor -- start 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 -- start start 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a806d760 0x7f51a80c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 0x7f51a80ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a8158740 0x7f51a8168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8155f90 con 0x7f51a806e7b0 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80af290 con 0x7f51a8158740 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80af570 con 0x7f51a806d760 2023-09-27T23:26:23.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 0x7f51a80ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 0x7f51a80ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38118/0 (socket says 172.21.15.64:38118) 2023-09-27T23:26:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 -- 172.21.15.64:0/2013928213 learned_addr learned my addr 172.21.15.64:0/2013928213 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a759e700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a8158740 0x7f51a8168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a6d9d700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a806d760 0x7f51a80c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 -- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a806d760 msgr2=0x7f51a80c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a806d760 0x7f51a80c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 -- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a8158740 msgr2=0x7f51a8168dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a8158740 0x7f51a8168dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 -- 172.21.15.64:0/2013928213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a000abf0 con 0x7f51a806e7b0 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51a659c700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 0x7f51a80ca290 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f519800c590 tx=0x7f519800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198011590 con 0x7f51a806e7b0 2023-09-27T23:26:23.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5198011710 con 0x7f51a806e7b0 2023-09-27T23:26:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80af850 con 0x7f51a806e7b0 2023-09-27T23:26:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198011ad0 con 0x7f51a806e7b0 2023-09-27T23:26:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:22.996+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80aef40 con 0x7f51a806e7b0 2023-09-27T23:26:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.000+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8169f70 con 0x7f51a806e7b0 2023-09-27T23:26:23.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.000+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f519801b3b0 con 0x7f51a806e7b0 2023-09-27T23:26:23.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.000+0000 7f518effd700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5194060b50 0x7f5194063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.000+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f519807c980 con 0x7f51a806e7b0 2023-09-27T23:26:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.000+0000 7f51a6d9d700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5194060b50 0x7f5194063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.004+0000 7f51a6d9d700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5194060b50 0x7f5194063000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51a8169db0 tx=0x7f519c0091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:23.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.008+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f51a8169f70 con 0x7f51a806e7b0 2023-09-27T23:26:23.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.192+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51a8169f70 con 0x7f5194060b50 2023-09-27T23:26:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.356+0000 7f518effd700 1 -- 172.21.15.64:0/2013928213 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f51a8169f70 con 0x7f5194060b50 2023-09-27T23:26:23.362 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5194060b50 msgr2=0x7f5194063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5194060b50 0x7f5194063000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51a8169db0 tx=0x7f519c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 msgr2=0x7f51a80ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 0x7f51a80ca290 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f519800c590 tx=0x7f519800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 shutdown_connections 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5194060b50 0x7f5194063000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a8158740 0x7f51a8168dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51a806e7b0 0x7f51a80ca290 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 --2- 172.21.15.64:0/2013928213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51a806d760 0x7f51a80c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 >> 172.21.15.64:0/2013928213 conn(0x7f51a80c8c00 msgr2=0x7f51a8062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 shutdown_connections 2023-09-27T23:26:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.360+0000 7f51ac8f6700 1 -- 172.21.15.64:0/2013928213 wait complete. 2023-09-27T23:26:23.382 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-09-27T23:26:23.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-09-27T23:26:23.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.500+0000 7fad85187700 1 Processor -- start 2023-09-27T23:26:23.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad85187700 1 -- start start 2023-09-27T23:26:23.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad85187700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad80158740 0x7fad8006e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad85187700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad8006e7b0 0x7fad8006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad85187700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8006ded0 con 0x7fad80158b10 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad85187700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800c82f0 con 0x7fad80158740 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad85187700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c7fa0 con 0x7fad8006e7b0 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad80158740 0x7fad8006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad80158740 0x7fad8006e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38126/0 (socket says 172.21.15.64:38126) 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 -- 172.21.15.64:0/2728987851 learned_addr learned my addr 172.21.15.64:0/2728987851 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad777fe700 1 --2- 172.21.15.64:0/2728987851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad8006e7b0 0x7fad8006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 -- 172.21.15.64:0/2728987851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad8006e7b0 msgr2=0x7fad8006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 --2- 172.21.15.64:0/2728987851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad8006e7b0 0x7fad8006d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 -- 172.21.15.64:0/2728987851 >> v1:172.21.15.123:6789/0 conn(0x7fad80158b10 legacy=0x7fad8006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 -- 172.21.15.64:0/2728987851 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800c70f0 con 0x7fad80158740 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7ffff700 1 --2- 172.21.15.64:0/2728987851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158740 0x7fad8006e270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fad7000bc40 tx=0x7fad7000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158c758d9ac1bb28 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.504+0000 7fad7f7fe700 1 -- 172.21.15.64:0/2728987851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000da90 con 0x7fad80158740 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad7f7fe700 1 -- 172.21.15.64:0/2728987851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000dbf0 con 0x7fad80158740 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad7f7fe700 1 -- 172.21.15.64:0/2728987851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70017410 con 0x7fad80158740 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 -- 172.21.15.64:0/2728987851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158740 msgr2=0x7fad8006e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 --2- 172.21.15.64:0/2728987851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158740 0x7fad8006e270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fad7000bc40 tx=0x7fad7000b4e0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 -- 172.21.15.64:0/2728987851 shutdown_connections 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 --2- 172.21.15.64:0/2728987851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad8006e7b0 0x7fad8006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 --2- 172.21.15.64:0/2728987851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158740 0x7fad8006e270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 -- 172.21.15.64:0/2728987851 >> 172.21.15.64:0/2728987851 conn(0x7fad800c8c00 msgr2=0x7fad800c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:23.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 -- 172.21.15.64:0/2728987851 shutdown_connections 2023-09-27T23:26:23.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 -- 172.21.15.64:0/2728987851 wait complete. 2023-09-27T23:26:23.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 Processor -- start 2023-09-27T23:26:23.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.508+0000 7fad85187700 1 -- start start 2023-09-27T23:26:23.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 0x7fad801557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad80158740 0x7fad800c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158b10 0x7fad800aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80155ce0 con 0x7fad80158b10 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c5720 con 0x7fad8006e7b0 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c5a00 con 0x7fad80158740 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad84986700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158b10 0x7fad800aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 0x7fad801557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad777fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad80158740 0x7fad800c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 0x7fad801557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:44334/0 (socket says 172.21.15.64:44334) 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 -- 172.21.15.64:0/2907526352 learned_addr learned my addr 172.21.15.64:0/2907526352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad84986700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158b10 0x7fad800aaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38140/0 (socket says 172.21.15.64:38140) 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 -- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad80158740 msgr2=0x7fad800c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad80158740 0x7fad800c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 -- 172.21.15.64:0/2907526352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158b10 msgr2=0x7fad800aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158b10 0x7fad800aaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 -- 172.21.15.64:0/2907526352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7000b780 con 0x7fad8006e7b0 2023-09-27T23:26:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad777fe700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad80158740 0x7fad800c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:23.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7ffff700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 0x7fad801557a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d6d0 tx=0x7fad70019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70022610 con 0x7fad8006e7b0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70022770 con 0x7fad8006e7b0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800c5ce0 con 0x7fad8006e7b0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70022b10 con 0x7fad8006e7b0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad800c62a0 con 0x7fad8006e7b0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.512+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8004be40 con 0x7fad8006e7b0 2023-09-27T23:26:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.516+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fad70015070 con 0x7fad8006e7b0 2023-09-27T23:26:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.516+0000 7fad7cff9700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fad60060b50 0x7fad60063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.516+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fad70081830 con 0x7fad8006e7b0 2023-09-27T23:26:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.516+0000 7fad777fe700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fad60060b50 0x7fad60063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.516+0000 7fad777fe700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fad60060b50 0x7fad60063000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad8006da60 tx=0x7fad680091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:23.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.528+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fad70051480 con 0x7fad8006e7b0 2023-09-27T23:26:23.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.712+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fad800c7390 con 0x7fad60060b50 2023-09-27T23:26:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.864+0000 7fad7cff9700 1 -- 172.21.15.64:0/2907526352 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fad800c7390 con 0x7fad60060b50 2023-09-27T23:26:23.868 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-09-27T23:26:23.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fad60060b50 msgr2=0x7fad60063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fad60060b50 0x7fad60063000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad8006da60 tx=0x7fad680091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 msgr2=0x7fad801557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 0x7fad801557a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d6d0 tx=0x7fad70019000 comp rx=0 tx=0).stop 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 shutdown_connections 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fad60060b50 0x7fad60063000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80158b10 0x7fad800aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad80158740 0x7fad800c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 --2- 172.21.15.64:0/2907526352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad8006e7b0 0x7fad801557a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 >> 172.21.15.64:0/2907526352 conn(0x7fad800c8c00 msgr2=0x7fad800b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 shutdown_connections 2023-09-27T23:26:23.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:23.868+0000 7fad85187700 1 -- 172.21.15.64:0/2907526352 wait complete. 2023-09-27T23:26:23.890 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-09-27T23:26:23.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-09-27T23:26:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 Processor -- start 2023-09-27T23:26:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 -- start start 2023-09-27T23:26:24.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f0c0cb130 0x7f3f0c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f0c15c930 0x7f3f0c06aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0b0ac0 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c06b030 con 0x7f3f0c15c930 2023-09-27T23:26:24.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f12446700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c06b210 con 0x7f3f0c0cb130 2023-09-27T23:26:24.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f0c15c930 0x7f3f0c06aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f0c15c930 0x7f3f0c06aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38144/0 (socket says 172.21.15.64:38144) 2023-09-27T23:26:24.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 -- 172.21.15.64:0/1014506948 learned_addr learned my addr 172.21.15.64:0/1014506948 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f0bfff700 1 --2- 172.21.15.64:0/1014506948 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c0cb130 0x7f3f0c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f0affd700 1 -- 172.21.15.64:0/1014506948 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379031976 0 0) 0x7f3f0c0b0ac0 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 -- 172.21.15.64:0/1014506948 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c0cb130 msgr2=0x7f3f0c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 --2- 172.21.15.64:0/1014506948 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c0cb130 0x7f3f0c0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 -- 172.21.15.64:0/1014506948 >> v1:172.21.15.123:6789/0 conn(0x7f3f0c06a6a0 legacy=0x7f3f0c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 -- 172.21.15.64:0/1014506948 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0ca810 con 0x7f3f0c15c930 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f109e3700 1 --2- 172.21.15.64:0/1014506948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c15c930 0x7f3f0c06aaf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0694e0 tx=0x7f3f04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da5a093f30ddf7f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f0affd700 1 -- 172.21.15.64:0/1014506948 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400e070 con 0x7f3f0c15c930 2023-09-27T23:26:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f0affd700 1 -- 172.21.15.64:0/1014506948 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0400ba20 con 0x7f3f0c15c930 2023-09-27T23:26:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.008+0000 7f3f0affd700 1 -- 172.21.15.64:0/1014506948 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040092e0 con 0x7f3f0c15c930 2023-09-27T23:26:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 -- 172.21.15.64:0/1014506948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c15c930 msgr2=0x7f3f0c06aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 --2- 172.21.15.64:0/1014506948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c15c930 0x7f3f0c06aaf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0694e0 tx=0x7f3f04009840 comp rx=0 tx=0).stop 2023-09-27T23:26:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 -- 172.21.15.64:0/1014506948 shutdown_connections 2023-09-27T23:26:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 --2- 172.21.15.64:0/1014506948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c15c930 0x7f3f0c06aaf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 --2- 172.21.15.64:0/1014506948 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c0cb130 0x7f3f0c0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 -- 172.21.15.64:0/1014506948 >> 172.21.15.64:0/1014506948 conn(0x7f3f0c067bb0 msgr2=0x7f3f0c0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 -- 172.21.15.64:0/1014506948 shutdown_connections 2023-09-27T23:26:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 -- 172.21.15.64:0/1014506948 wait complete. 2023-09-27T23:26:24.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 Processor -- start 2023-09-27T23:26:24.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.012+0000 7f3f12446700 1 -- start start 2023-09-27T23:26:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 0x7f3f0c0d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c0cb130 0x7f3f0c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c15c930 0x7f3f0c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c151d50 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c157050 con 0x7f3f0c0cb130 2023-09-27T23:26:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c157330 con 0x7f3f0c15c930 2023-09-27T23:26:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 0x7f3f0c0d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 0x7f3f0c0d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38154/0 (socket says 172.21.15.64:38154) 2023-09-27T23:26:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c0cb130 0x7f3f0c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 -- 172.21.15.64:0/2101441584 learned_addr learned my addr 172.21.15.64:0/2101441584 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f109e3700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c15c930 0x7f3f0c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 -- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c15c930 msgr2=0x7f3f0c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c15c930 0x7f3f0c0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 -- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c0cb130 msgr2=0x7f3f0c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c0cb130 0x7f3f0c0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 -- 172.21.15.64:0/2101441584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04009ae0 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0b7fe700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c0cb130 0x7f3f0c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f0bfff700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 0x7f3f0c0d03b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00ec80 tx=0x7f3efc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc009690 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00c850 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c157610 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00cc10 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.016+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c157bd0 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.020+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c071120 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.020+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f3efc013470 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.020+0000 7f3ef3fff700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef8060b50 0x7f3ef8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.020+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f3efc07d5d0 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.020+0000 7f3f0b7fe700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef8060b50 0x7f3ef8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.020+0000 7f3f0b7fe700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef8060b50 0x7f3ef8063000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c157f80 tx=0x7f3f00009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.028+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3efc050de0 con 0x7f3f0c06a6a0 2023-09-27T23:26:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.212+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f0c15a4b0 con 0x7f3ef8060b50 2023-09-27T23:26:24.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.376+0000 7f3ef3fff700 1 -- 172.21.15.64:0/2101441584 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3f0c15a4b0 con 0x7f3ef8060b50 2023-09-27T23:26:24.380 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.376+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef8060b50 msgr2=0x7f3ef8063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.376+0000 7f3f12446700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef8060b50 0x7f3ef8063000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c157f80 tx=0x7f3f00009040 comp rx=0 tx=0).stop 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.376+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 msgr2=0x7f3f0c0d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.376+0000 7f3f12446700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 0x7f3f0c0d03b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00ec80 tx=0x7f3efc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 shutdown_connections 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef8060b50 0x7f3ef8063000 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c15c930 0x7f3f0c0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c0cb130 0x7f3f0c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 --2- 172.21.15.64:0/2101441584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c06a6a0 0x7f3f0c0d03b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 >> 172.21.15.64:0/2101441584 conn(0x7f3f0c067bb0 msgr2=0x7f3f0c0b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 shutdown_connections 2023-09-27T23:26:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.380+0000 7f3f12446700 1 -- 172.21.15.64:0/2101441584 wait complete. 2023-09-27T23:26:24.400 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-09-27T23:26:24.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-09-27T23:26:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 Processor -- start 2023-09-27T23:26:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 -- start start 2023-09-27T23:26:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efdf0158a10 0x7efdf0158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf00b02b0 0x7efdf00680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf00ad3b0 con 0x7efdf0068620 2023-09-27T23:26:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf0069120 con 0x7efdf0158a10 2023-09-27T23:26:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6812700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf0068e40 con 0x7efdf00b02b0 2023-09-27T23:26:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6011700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efdf0068620 0x7efdf006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33412/0 (socket says 172.21.15.64:33412) 2023-09-27T23:26:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6011700 1 -- 172.21.15.64:0/4018946132 learned_addr learned my addr 172.21.15.64:0/4018946132 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf5810700 1 --2- 172.21.15.64:0/4018946132 >> v2:172.21.15.123:3300/0 conn(0x7efdf0158a10 0x7efdf0158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf500f700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf00b02b0 0x7efdf00680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399708863 0 0) 0x7efdf00ad3b0 con 0x7efdf0068620 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efddc003570 con 0x7efdf0068620 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845529513 0 0) 0x7efddc003570 con 0x7efdf0068620 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf00b02b0 msgr2=0x7efdf00680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf00b02b0 0x7efdf00680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 >> v2:172.21.15.123:3300/0 conn(0x7efdf0158a10 msgr2=0x7efdf0158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 --2- 172.21.15.64:0/4018946132 >> v2:172.21.15.123:3300/0 conn(0x7efdf0158a10 0x7efdf0158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00ad7c0 con 0x7efdf0068620 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7efde80017f0 con 0x7efdf0068620 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 >> v1:172.21.15.64:6789/0 conn(0x7efdf0068620 legacy=0x7efdf006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efddc003a60 0x7efddc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efddc0043b0 0x7efddc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efddc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf0069120 con 0x7efddc004ee0 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf0068e40 con 0x7efddc0043b0 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0056d0 con 0x7efddc003a60 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf6011700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efddc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf500f700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efddc0043b0 0x7efddc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.516+0000 7efdf5810700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efddc003a60 0x7efddc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 -- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efddc003a60 msgr2=0x7efddc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efddc003a60 0x7efddc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 -- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efddc0043b0 msgr2=0x7efddc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efddc0043b0 0x7efddc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 -- 172.21.15.64:0/4018946132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc003570 con 0x7efddc004ee0 2023-09-27T23:26:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efddc00ba60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efde800a970 tx=0x7efde8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc9d5a3a4ae83de server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80070d0 con 0x7efddc004ee0 2023-09-27T23:26:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8007270 con 0x7efddc004ee0 2023-09-27T23:26:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf480e700 1 -- 172.21.15.64:0/4018946132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8007630 con 0x7efddc004ee0 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 -- 172.21.15.64:0/4018946132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 msgr2=0x7efddc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 --2- 172.21.15.64:0/4018946132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efddc00ba60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efde800a970 tx=0x7efde8002210 comp rx=0 tx=0).stop 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6011700 1 -- 172.21.15.64:0/4018946132 reap_dead start 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 -- 172.21.15.64:0/4018946132 shutdown_connections 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 -- 172.21.15.64:0/4018946132 >> 172.21.15.64:0/4018946132 conn(0x7efdf00ce5e0 msgr2=0x7efdf00ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 -- 172.21.15.64:0/4018946132 shutdown_connections 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 -- 172.21.15.64:0/4018946132 wait complete. 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.520+0000 7efdf6812700 1 Processor -- start 2023-09-27T23:26:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 -- start start 2023-09-27T23:26:24.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efdf015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf0149f50 0x7efdf00a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf015d250 0x7efdf015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf015dc00 con 0x7efddc004ee0 2023-09-27T23:26:24.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf00b1040 con 0x7efdf015d250 2023-09-27T23:26:24.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf00b1320 con 0x7efdf0149f50 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efdf015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efdf015cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38182/0 (socket says 172.21.15.64:38182) 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 -- 172.21.15.64:0/3623935732 learned_addr learned my addr 172.21.15.64:0/3623935732 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6011700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf015d250 0x7efdf015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf500f700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf0149f50 0x7efdf00a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 -- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf0149f50 msgr2=0x7efdf00a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf0149f50 0x7efdf00a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 -- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf015d250 msgr2=0x7efdf015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf015d250 0x7efdf015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 -- 172.21.15.64:0/3623935732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00b1600 con 0x7efddc004ee0 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf5810700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efdf015cb60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efde4006ac0 tx=0x7efde40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4011ec0 con 0x7efddc004ee0 2023-09-27T23:26:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde400f9d0 con 0x7efddc004ee0 2023-09-27T23:26:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf00b18e0 con 0x7efddc004ee0 2023-09-27T23:26:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400fd50 con 0x7efddc004ee0 2023-09-27T23:26:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.524+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf014a3a0 con 0x7efddc004ee0 2023-09-27T23:26:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.528+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7efde4014650 con 0x7efddc004ee0 2023-09-27T23:26:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.528+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf015d0a0 con 0x7efddc004ee0 2023-09-27T23:26:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.528+0000 7efde1ffb700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efdd4060b50 0x7efdd4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.528+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7efde4018030 con 0x7efddc004ee0 2023-09-27T23:26:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.528+0000 7efdf500f700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efdd4060b50 0x7efdd4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.528+0000 7efdf500f700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efdd4060b50 0x7efdd4063000 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdf0067240 tx=0x7efdec0091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:24.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.532+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efde404db80 con 0x7efddc004ee0 2023-09-27T23:26:24.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.716+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efdf004be40 con 0x7efdd4060b50 2023-09-27T23:26:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.876+0000 7efde1ffb700 1 -- 172.21.15.64:0/3623935732 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7efdf004be40 con 0x7efdd4060b50 2023-09-27T23:26:24.881 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efdd4060b50 msgr2=0x7efdd4063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efdd4060b50 0x7efdd4063000 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdf0067240 tx=0x7efdec0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 msgr2=0x7efdf015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efdf015cb60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efde4006ac0 tx=0x7efde40091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 shutdown_connections 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efdd4060b50 0x7efdd4063000 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf015d250 0x7efdf015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdf0149f50 0x7efdf00a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 --2- 172.21.15.64:0/3623935732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efddc004ee0 0x7efdf015cb60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 >> 172.21.15.64:0/3623935732 conn(0x7efdf00ce5e0 msgr2=0x7efdf00ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 shutdown_connections 2023-09-27T23:26:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:24.880+0000 7efdf6812700 1 -- 172.21.15.64:0/3623935732 wait complete. 2023-09-27T23:26:24.902 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-09-27T23:26:24.903 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-09-27T23:26:25.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 Processor -- start 2023-09-27T23:26:25.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 -- start start 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69b0158b00 0x7f69b0158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f69b00abe40 0x7f69b006d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b006d9b0 con 0x7f69b006bb70 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00c85a0 con 0x7f69b0158b00 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b63e9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00c67d0 con 0x7f69b00abe40 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b53e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69b0158b00 0x7f69b0158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b5be8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f69b006bb70 0x7f69b006ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52744/0 (socket says 172.21.15.64:52744) 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b53e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69b0158b00 0x7f69b0158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34296/0 (socket says 172.21.15.64:34296) 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b5be8700 1 -- 172.21.15.64:0/351170486 learned_addr learned my addr 172.21.15.64:0/351170486 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:25.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b4be6700 1 --2- 172.21.15.64:0/351170486 >> v2:172.21.15.123:3300/0 conn(0x7f69b00abe40 0x7f69b006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b53e7700 1 -- 172.21.15.64:0/351170486 >> v2:172.21.15.123:3300/0 conn(0x7f69b00abe40 msgr2=0x7f69b006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:25.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b53e7700 1 --2- 172.21.15.64:0/351170486 >> v2:172.21.15.123:3300/0 conn(0x7f69b00abe40 0x7f69b006d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69a7fff700 1 -- 172.21.15.64:0/351170486 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465100877 0 0) 0x7f69b006d9b0 con 0x7f69b006bb70 2023-09-27T23:26:25.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b53e7700 1 -- 172.21.15.64:0/351170486 >> v1:172.21.15.64:6789/0 conn(0x7f69b006bb70 legacy=0x7f69b006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.024+0000 7f69b53e7700 1 -- 172.21.15.64:0/351170486 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b00c6ab0 con 0x7f69b0158b00 2023-09-27T23:26:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b53e7700 1 --2- 172.21.15.64:0/351170486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b0158b00 0x7f69b0158ed0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f69b006a560 tx=0x7f69ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c0d3fde43b55fb server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69a7fff700 1 -- 172.21.15.64:0/351170486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac011630 con 0x7f69b0158b00 2023-09-27T23:26:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69a7fff700 1 -- 172.21.15.64:0/351170486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac011790 con 0x7f69b0158b00 2023-09-27T23:26:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69a7fff700 1 -- 172.21.15.64:0/351170486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac011b30 con 0x7f69b0158b00 2023-09-27T23:26:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 -- 172.21.15.64:0/351170486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b0158b00 msgr2=0x7f69b0158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 --2- 172.21.15.64:0/351170486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b0158b00 0x7f69b0158ed0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f69b006a560 tx=0x7f69ac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 -- 172.21.15.64:0/351170486 shutdown_connections 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 --2- 172.21.15.64:0/351170486 >> v2:172.21.15.123:3300/0 conn(0x7f69b00abe40 0x7f69b006d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 --2- 172.21.15.64:0/351170486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b0158b00 0x7f69b0158ed0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 -- 172.21.15.64:0/351170486 >> 172.21.15.64:0/351170486 conn(0x7f69b00c8c00 msgr2=0x7f69b00c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 -- 172.21.15.64:0/351170486 shutdown_connections 2023-09-27T23:26:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.028+0000 7f69b63e9700 1 -- 172.21.15.64:0/351170486 wait complete. 2023-09-27T23:26:25.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 Processor -- start 2023-09-27T23:26:25.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 -- start start 2023-09-27T23:26:25.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b006bb70 0x7f69b014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 0x7f69b00b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69b0158b00 0x7f69b00b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00a8860 con 0x7f69b00abe40 2023-09-27T23:26:25.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00a8de0 con 0x7f69b006bb70 2023-09-27T23:26:25.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b014df10 con 0x7f69b0158b00 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 0x7f69b00b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 0x7f69b00b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34308/0 (socket says 172.21.15.64:34308) 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 -- 172.21.15.64:0/49548171 learned_addr learned my addr 172.21.15.64:0/49548171 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b53e7700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b006bb70 0x7f69b014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b5be8700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69b0158b00 0x7f69b00b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 -- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69b0158b00 msgr2=0x7f69b00b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69b0158b00 0x7f69b00b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 -- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b006bb70 msgr2=0x7f69b014d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b006bb70 0x7f69b014d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 -- 172.21.15.64:0/49548171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac009420 con 0x7f69b00abe40 2023-09-27T23:26:25.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b4be6700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 0x7f69b00b1520 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f69a000ec80 tx=0x7f69a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a000c850 con 0x7f69b00abe40 2023-09-27T23:26:25.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a000c9b0 con 0x7f69b00abe40 2023-09-27T23:26:25.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b0150c00 con 0x7f69b00abe40 2023-09-27T23:26:25.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a000cd50 con 0x7f69b00abe40 2023-09-27T23:26:25.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.032+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b00a8a70 con 0x7f69b00abe40 2023-09-27T23:26:25.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.036+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b004be40 con 0x7f69b00abe40 2023-09-27T23:26:25.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.036+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f69a0013680 con 0x7f69b00abe40 2023-09-27T23:26:25.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.036+0000 7f69a57fa700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6994060b50 0x7f6994063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.036+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f69a007d680 con 0x7f69b00abe40 2023-09-27T23:26:25.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.036+0000 7f69b53e7700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6994060b50 0x7f6994063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.040+0000 7f69b53e7700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6994060b50 0x7f6994063000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0093f0 tx=0x7f69ac017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.048+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f69a004d400 con 0x7f69b00abe40 2023-09-27T23:26:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.232+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f69b00c6d90 con 0x7f6994060b50 2023-09-27T23:26:25.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.384+0000 7f69a57fa700 1 -- 172.21.15.64:0/49548171 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f69b00c6d90 con 0x7f6994060b50 2023-09-27T23:26:25.391 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-09-27T23:26:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6994060b50 msgr2=0x7f6994063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6994060b50 0x7f6994063000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0093f0 tx=0x7f69ac017000 comp rx=0 tx=0).stop 2023-09-27T23:26:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 msgr2=0x7f69b00b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 0x7f69b00b1520 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f69a000ec80 tx=0x7f69a00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 shutdown_connections 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6994060b50 0x7f6994063000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69b0158b00 0x7f69b00b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69b00abe40 0x7f69b00b1520 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 --2- 172.21.15.64:0/49548171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b006bb70 0x7f69b014d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 >> 172.21.15.64:0/49548171 conn(0x7f69b00c8c00 msgr2=0x7f69b00ac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 shutdown_connections 2023-09-27T23:26:25.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.388+0000 7f69b63e9700 1 -- 172.21.15.64:0/49548171 wait complete. 2023-09-27T23:26:25.412 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-09-27T23:26:25.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:25.412 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_01_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:25.429 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:25.430 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:25.430 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:25.430 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_01_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_01_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:25.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-09-27T23:26:25.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205bfff700 1 Processor -- start 2023-09-27T23:26:25.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205bfff700 1 -- start start 2023-09-27T23:26:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f205c158b30 0x7f205c0ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0cacc0 con 0x7f205c0c9880 2023-09-27T23:26:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0c7f70 con 0x7f205c0c7a00 2023-09-27T23:26:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0caf10 con 0x7f205c158b30 2023-09-27T23:26:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205affd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f205c158b30 0x7f205c0ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f205c0c9880 0x7f205c0cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52748/0 (socket says 172.21.15.64:52748) 2023-09-27T23:26:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205affd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f205c158b30 0x7f205c0ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34310/0 (socket says 172.21.15.64:34310) 2023-09-27T23:26:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205b7fe700 1 -- 172.21.15.64:0/3280568221 learned_addr learned my addr 172.21.15.64:0/3280568221 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205affd700 1 -- 172.21.15.64:0/3280568221 >> v1:172.21.15.123:6789/0 conn(0x7f205c0c7a00 legacy=0x7f205c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f2059ffb700 1 -- 172.21.15.64:0/3280568221 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156744952 0 0) 0x7f205c0cacc0 con 0x7f205c0c9880 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205affd700 1 -- 172.21.15.64:0/3280568221 >> v1:172.21.15.64:6789/0 conn(0x7f205c0c9880 legacy=0x7f205c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205affd700 1 -- 172.21.15.64:0/3280568221 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0cb1f0 con 0x7f205c158b30 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f205affd700 1 --2- 172.21.15.64:0/3280568221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c158b30 0x7f205c0ac600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f204c009890 tx=0x7f204c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bcf5a49854f9bc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f2059ffb700 1 -- 172.21.15.64:0/3280568221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c011630 con 0x7f205c158b30 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f2059ffb700 1 -- 172.21.15.64:0/3280568221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c011790 con 0x7f205c158b30 2023-09-27T23:26:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.592+0000 7f2059ffb700 1 -- 172.21.15.64:0/3280568221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c011b30 con 0x7f205c158b30 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 -- 172.21.15.64:0/3280568221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c158b30 msgr2=0x7f205c0ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 --2- 172.21.15.64:0/3280568221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c158b30 0x7f205c0ac600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f204c009890 tx=0x7f204c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 -- 172.21.15.64:0/3280568221 shutdown_connections 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 --2- 172.21.15.64:0/3280568221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c158b30 0x7f205c0ac600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 -- 172.21.15.64:0/3280568221 >> 172.21.15.64:0/3280568221 conn(0x7f205c06b5b0 msgr2=0x7f205c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 -- 172.21.15.64:0/3280568221 shutdown_connections 2023-09-27T23:26:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.596+0000 7f205bfff700 1 -- 172.21.15.64:0/3280568221 wait complete. 2023-09-27T23:26:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 Processor -- start 2023-09-27T23:26:25.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 -- start start 2023-09-27T23:26:25.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 0x7f205c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f205c0c9880 0x7f205c16c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205c158b30 0x7f205c16cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0c1af0 con 0x7f205c0c7a00 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c15cd60 con 0x7f205c158b30 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c15d040 con 0x7f205c0c9880 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 0x7f205c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 0x7f205c0b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34316/0 (socket says 172.21.15.64:34316) 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 -- 172.21.15.64:0/2791371039 learned_addr learned my addr 172.21.15.64:0/2791371039 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205b7fe700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205c158b30 0x7f205c16cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205a7fc700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f205c0c9880 0x7f205c16c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 -- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f205c0c9880 msgr2=0x7f205c16c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f205c0c9880 0x7f205c16c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 -- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205c158b30 msgr2=0x7f205c16cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205c158b30 0x7f205c16cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 -- 172.21.15.64:0/2791371039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c009420 con 0x7f205c0c7a00 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205a7fc700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f205c0c9880 0x7f205c16c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205affd700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 0x7f205c0b0920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f204c0030c0 tx=0x7f204c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c011b10 con 0x7f205c0c7a00 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c020070 con 0x7f205c0c7a00 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c15d320 con 0x7f205c0c7a00 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0276b0 con 0x7f205c0c7a00 2023-09-27T23:26:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.600+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c0b25b0 con 0x7f205c0c7a00 2023-09-27T23:26:25.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.604+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c04be40 con 0x7f205c0c7a00 2023-09-27T23:26:25.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.604+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f204c027aa0 con 0x7f205c0c7a00 2023-09-27T23:26:25.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.604+0000 7f2042ffd700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2048060b50 0x7f2048063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.604+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f204c022070 con 0x7f205c0c7a00 2023-09-27T23:26:25.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.604+0000 7f205a7fc700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2048060b50 0x7f2048063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.604+0000 7f205a7fc700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2048060b50 0x7f2048063000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f205c0c3090 tx=0x7f2050009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.612+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f204c04f280 con 0x7f205c0c7a00 2023-09-27T23:26:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.796+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f205c0ca870 con 0x7f2048060b50 2023-09-27T23:26:25.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.812+0000 7f2042ffd700 1 -- 172.21.15.64:0/2791371039 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f205c0ca870 con 0x7f2048060b50 2023-09-27T23:26:25.818 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:25.818 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:01", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:25.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2048060b50 msgr2=0x7f2048063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2048060b50 0x7f2048063000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f205c0c3090 tx=0x7f2050009040 comp rx=0 tx=0).stop 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 msgr2=0x7f205c0b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 0x7f205c0b0920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f204c0030c0 tx=0x7f204c016000 comp rx=0 tx=0).stop 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 shutdown_connections 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2048060b50 0x7f2048063000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205c158b30 0x7f205c16cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f205c0c9880 0x7f205c16c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 --2- 172.21.15.64:0/2791371039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f205c0c7a00 0x7f205c0b0920 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 >> 172.21.15.64:0/2791371039 conn(0x7f205c06b5b0 msgr2=0x7f205c0acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:25.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 shutdown_connections 2023-09-27T23:26:25.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.816+0000 7f205bfff700 1 -- 172.21.15.64:0/2791371039 wait complete. 2023-09-27T23:26:25.833 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:01", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:25.834 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:01', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:25.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:25.834 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-09-27T23:26:25.849 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_01_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:25.851 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_01_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:25.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-09-27T23:26:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 Processor -- start 2023-09-27T23:26:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 -- start start 2023-09-27T23:26:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa413f830 0x7fefa40ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefa40c7780 0x7fefa40c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa40cc200 con 0x7fefa40aea30 2023-09-27T23:26:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa40cb160 con 0x7fefa40c7780 2023-09-27T23:26:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefabab4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40cb440 con 0x7fefa413f830 2023-09-27T23:26:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefa40c7780 0x7fefa40c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefa40c7780 0x7fefa40c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34326/0 (socket says 172.21.15.64:34326) 2023-09-27T23:26:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 -- 172.21.15.64:0/3235850727 learned_addr learned my addr 172.21.15.64:0/3235850727 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefa9850700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa413f830 0x7fefa40ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefa884e700 1 -- 172.21.15.64:0/3235850727 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219969099 0 0) 0x7fefa40cc200 con 0x7fefa40aea30 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefa884e700 1 -- 172.21.15.64:0/3235850727 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef90003570 con 0x7fefa40aea30 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 -- 172.21.15.64:0/3235850727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa413f830 msgr2=0x7fefa40ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa413f830 0x7fefa40ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 -- 172.21.15.64:0/3235850727 >> v1:172.21.15.123:6789/0 conn(0x7fefa40aea30 legacy=0x7fefa40aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 -- 172.21.15.64:0/3235850727 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40cb720 con 0x7fefa40c7780 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefa9850700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa413f830 0x7fefa40ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.964+0000 7fefaa051700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40c7780 0x7fefa40c7be0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00bc50 tx=0x7fef9c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83249a8da3e4484 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefa884e700 1 -- 172.21.15.64:0/3235850727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00db40 con 0x7fefa40c7780 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefa884e700 1 -- 172.21.15.64:0/3235850727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00dca0 con 0x7fefa40c7780 2023-09-27T23:26:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefa884e700 1 -- 172.21.15.64:0/3235850727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c019690 con 0x7fefa40c7780 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 -- 172.21.15.64:0/3235850727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40c7780 msgr2=0x7fefa40c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40c7780 0x7fefa40c7be0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00bc50 tx=0x7fef9c00b4f0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 -- 172.21.15.64:0/3235850727 shutdown_connections 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40c7780 0x7fefa40c7be0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 --2- 172.21.15.64:0/3235850727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa413f830 0x7fefa40ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 -- 172.21.15.64:0/3235850727 >> 172.21.15.64:0/3235850727 conn(0x7fefa40ccc20 msgr2=0x7fefa40cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 -- 172.21.15.64:0/3235850727 shutdown_connections 2023-09-27T23:26:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 -- 172.21.15.64:0/3235850727 wait complete. 2023-09-27T23:26:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 Processor -- start 2023-09-27T23:26:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.968+0000 7fefabab4700 1 -- start start 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40aea30 0x7fefa40b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40c7780 0x7fefa413c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 0x7fefa40c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40b1b70 con 0x7fefa413f830 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40b0f60 con 0x7fefa40aea30 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40b1240 con 0x7fefa40c7780 2023-09-27T23:26:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 0x7fefa40c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 0x7fefa40c2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34328/0 (socket says 172.21.15.64:34328) 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 -- 172.21.15.64:0/1896338750 learned_addr learned my addr 172.21.15.64:0/1896338750 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefa904f700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40c7780 0x7fefa413c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefa9850700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40aea30 0x7fefa40b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 -- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40c7780 msgr2=0x7fefa413c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40c7780 0x7fefa413c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 -- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40aea30 msgr2=0x7fefa40b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40aea30 0x7fefa40b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 -- 172.21.15.64:0/1896338750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009420 con 0x7fefa413f830 2023-09-27T23:26:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefaa051700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 0x7fefa40c2cf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef9c006fd0 tx=0x7fef9c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0107d0 con 0x7fefa413f830 2023-09-27T23:26:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c01e050 con 0x7fefa413f830 2023-09-27T23:26:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c00b790 con 0x7fefa413f830 2023-09-27T23:26:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c010b80 con 0x7fefa413f830 2023-09-27T23:26:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.972+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa40b1800 con 0x7fefa413f830 2023-09-27T23:26:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.976+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa40b1ce0 con 0x7fefa413f830 2023-09-27T23:26:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.976+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fef9c024470 con 0x7fefa413f830 2023-09-27T23:26:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.976+0000 7fef95ffb700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060b50 0x7fef88063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.976+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fef9c014070 con 0x7fefa413f830 2023-09-27T23:26:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.976+0000 7fefa9850700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060b50 0x7fef88063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.976+0000 7fefa9850700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060b50 0x7fef88063000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009f40 tx=0x7fefa00091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:25.984+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fef9c051910 con 0x7fefa413f830 2023-09-27T23:26:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.164+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fefa4071120 con 0x7fef88060b50 2023-09-27T23:26:26.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.180+0000 7fef95ffb700 1 -- 172.21.15.64:0/1896338750 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fefa4071120 con 0x7fef88060b50 2023-09-27T23:26:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:26.188 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:01", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060b50 msgr2=0x7fef88063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060b50 0x7fef88063000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009f40 tx=0x7fefa00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 msgr2=0x7fefa40c2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 0x7fefa40c2cf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef9c006fd0 tx=0x7fef9c01b000 comp rx=0 tx=0).stop 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 shutdown_connections 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060b50 0x7fef88063000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa413f830 0x7fefa40c2cf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40c7780 0x7fefa413c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 --2- 172.21.15.64:0/1896338750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40aea30 0x7fefa40b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 >> 172.21.15.64:0/1896338750 conn(0x7fefa40ccc20 msgr2=0x7fefa40c7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 shutdown_connections 2023-09-27T23:26:26.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.184+0000 7fefabab4700 1 -- 172.21.15.64:0/1896338750 wait complete. 2023-09-27T23:26:26.208 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:01", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:26.208 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:01', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:26.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:26.209 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_01_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:26.225 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_01_UTC 2023-09-27T23:26:26.226 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_01_UTC 2023-09-27T23:26:26.226 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:26.226 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:26.226 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_01_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_01_UTC', 'scheduled-2023-09-27-23_24_01_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:26.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-09-27T23:26:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 Processor -- start 2023-09-27T23:26:26.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 -- start start 2023-09-27T23:26:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0dc40cb130 0x7f0dc40cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0dc406a6a0 0x7f0dc406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc40b0ac0 con 0x7f0dc415c220 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc406b030 con 0x7f0dc40cb130 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dcbbc8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc406b210 con 0x7f0dc406a6a0 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dc9964700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0dc40cb130 0x7f0dc40cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dc9964700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0dc40cb130 0x7f0dc40cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34342/0 (socket says 172.21.15.64:34342) 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.384+0000 7f0dc9964700 1 -- 172.21.15.64:0/3134803751 learned_addr learned my addr 172.21.15.64:0/3134803751 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc9163700 1 --2- 172.21.15.64:0/3134803751 >> v2:172.21.15.123:3300/0 conn(0x7f0dc406a6a0 0x7f0dc406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc9964700 1 -- 172.21.15.64:0/3134803751 >> v2:172.21.15.123:3300/0 conn(0x7f0dc406a6a0 msgr2=0x7f0dc406aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc9964700 1 --2- 172.21.15.64:0/3134803751 >> v2:172.21.15.123:3300/0 conn(0x7f0dc406a6a0 0x7f0dc406aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc9964700 1 -- 172.21.15.64:0/3134803751 >> v1:172.21.15.123:6789/0 conn(0x7f0dc415c220 legacy=0x7f0dc406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc9964700 1 -- 172.21.15.64:0/3134803751 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc40ca810 con 0x7f0dc40cb130 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc9964700 1 --2- 172.21.15.64:0/3134803751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc40cb500 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40cc930 tx=0x7f0db8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e117955e22440032 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc8962700 1 -- 172.21.15.64:0/3134803751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8009e70 con 0x7f0dc40cb130 2023-09-27T23:26:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc8962700 1 -- 172.21.15.64:0/3134803751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800b860 con 0x7f0dc40cb130 2023-09-27T23:26:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dc8962700 1 -- 172.21.15.64:0/3134803751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800bc00 con 0x7f0dc40cb130 2023-09-27T23:26:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/3134803751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 msgr2=0x7f0dc40cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/3134803751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc40cb500 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40cc930 tx=0x7f0db8009250 comp rx=0 tx=0).stop 2023-09-27T23:26:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/3134803751 shutdown_connections 2023-09-27T23:26:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/3134803751 >> v2:172.21.15.123:3300/0 conn(0x7f0dc406a6a0 0x7f0dc406aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/3134803751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc40cb500 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/3134803751 >> 172.21.15.64:0/3134803751 conn(0x7f0dc4067bb0 msgr2=0x7f0dc40cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/3134803751 shutdown_connections 2023-09-27T23:26:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.388+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/3134803751 wait complete. 2023-09-27T23:26:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 Processor -- start 2023-09-27T23:26:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 -- start start 2023-09-27T23:26:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc406a6a0 0x7f0dc416cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc415c220 0x7f0dc40b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc414fbe0 con 0x7f0dc40cb130 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc416d440 con 0x7f0dc406a6a0 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4156100 con 0x7f0dc415c220 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc414ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34352/0 (socket says 172.21.15.64:34352) 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 -- 172.21.15.64:0/2983146838 learned_addr learned my addr 172.21.15.64:0/2983146838 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9964700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc406a6a0 0x7f0dc416cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dca165700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc415c220 0x7f0dc40b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 -- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc415c220 msgr2=0x7f0dc40b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc415c220 0x7f0dc40b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 -- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc406a6a0 msgr2=0x7f0dc416cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc406a6a0 0x7f0dc416cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 -- 172.21.15.64:0/2983146838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80094f0 con 0x7f0dc40cb130 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dca165700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc415c220 0x7f0dc40b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dc9163700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc414ef20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000e930 tx=0x7f0dc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0013680 con 0x7f0dc40cb130 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc00137e0 con 0x7f0dc40cb130 2023-09-27T23:26:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc414fe80 con 0x7f0dc40cb130 2023-09-27T23:26:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0013b80 con 0x7f0dc40cb130 2023-09-27T23:26:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.392+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc41563e0 con 0x7f0dc40cb130 2023-09-27T23:26:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.396+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc4071120 con 0x7f0dc40cb130 2023-09-27T23:26:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.396+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f0dc000c460 con 0x7f0dc40cb130 2023-09-27T23:26:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.396+0000 7f0db5ffb700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0da8060b50 0x7f0da8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.396+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f0dc007d620 con 0x7f0dc40cb130 2023-09-27T23:26:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.396+0000 7f0dc9964700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0da8060b50 0x7f0da8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.396+0000 7f0dc9964700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0da8060b50 0x7f0da8063000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40cc930 tx=0x7f0db8017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.404+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0dc004d3a0 con 0x7f0dc40cb130 2023-09-27T23:26:26.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.588+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dc40cbe90 con 0x7f0da8060b50 2023-09-27T23:26:26.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.604+0000 7f0db5ffb700 1 -- 172.21.15.64:0/2983146838 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0dc40cbe90 con 0x7f0da8060b50 2023-09-27T23:26:26.611 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:26.611 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:02", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0da8060b50 msgr2=0x7f0da8063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0da8060b50 0x7f0da8063000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40cc930 tx=0x7f0db8017000 comp rx=0 tx=0).stop 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 msgr2=0x7f0dc414ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc414ef20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000e930 tx=0x7f0dc00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 shutdown_connections 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0da8060b50 0x7f0da8063000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc415c220 0x7f0dc40b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc40cb130 0x7f0dc414ef20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 --2- 172.21.15.64:0/2983146838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc406a6a0 0x7f0dc416cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 >> 172.21.15.64:0/2983146838 conn(0x7f0dc4067bb0 msgr2=0x7f0dc40cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:26.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 shutdown_connections 2023-09-27T23:26:26.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.608+0000 7f0dcbbc8700 1 -- 172.21.15.64:0/2983146838 wait complete. 2023-09-27T23:26:26.627 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:02", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:26.628 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:02', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:26.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:26.628 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_01_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:26.644 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:26.645 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:26.646 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_01_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_01_UTC', 'scheduled-2023-09-27-23_26_01_UTC'] 2023-09-27T23:26:26.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-09-27T23:26:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93fd5b3700 1 Processor -- start 2023-09-27T23:26:26.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93fd5b3700 1 -- start start 2023-09-27T23:26:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93fd5b3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f93f80aea30 0x7f93f80aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93fd5b3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f80caf00 con 0x7f93f813f830 2023-09-27T23:26:26.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93fd5b3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f80cc200 con 0x7f93f80c7780 2023-09-27T23:26:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93fd5b3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f80cb160 con 0x7f93f80aea30 2023-09-27T23:26:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f6ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f93f813f830 0x7f93f80ca7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52756/0 (socket says 172.21.15.64:52756) 2023-09-27T23:26:26.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f6ffd700 1 -- 172.21.15.64:0/1378629655 learned_addr learned my addr 172.21.15.64:0/1378629655 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f67fc700 1 --2- 172.21.15.64:0/1378629655 >> v2:172.21.15.123:3300/0 conn(0x7f93f80aea30 0x7f93f80aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f5ffb700 1 -- 172.21.15.64:0/1378629655 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814793949 0 0) 0x7f93f80caf00 con 0x7f93f813f830 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f5ffb700 1 -- 172.21.15.64:0/1378629655 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e0003570 con 0x7f93f813f830 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f5ffb700 1 -- 172.21.15.64:0/1378629655 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4163384149 0 0) 0x7f93e0003570 con 0x7f93f813f830 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f67fc700 1 -- 172.21.15.64:0/1378629655 >> v1:172.21.15.123:6789/0 conn(0x7f93f80c7780 legacy=0x7f93f80c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f67fc700 1 -- 172.21.15.64:0/1378629655 >> v1:172.21.15.64:6789/0 conn(0x7f93f813f830 legacy=0x7f93f80ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f67fc700 1 -- 172.21.15.64:0/1378629655 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80cb440 con 0x7f93f80aea30 2023-09-27T23:26:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f67fc700 1 --2- 172.21.15.64:0/1378629655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80aea30 0x7f93f80aeed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0098e0 tx=0x7f93ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12571e4de1b10103 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f5ffb700 1 -- 172.21.15.64:0/1378629655 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8001470 con 0x7f93f80aea30 2023-09-27T23:26:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f5ffb700 1 -- 172.21.15.64:0/1378629655 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec011420 con 0x7f93f80aea30 2023-09-27T23:26:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.760+0000 7f93f5ffb700 1 -- 172.21.15.64:0/1378629655 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0117a0 con 0x7f93f80aea30 2023-09-27T23:26:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- 172.21.15.64:0/1378629655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80aea30 msgr2=0x7f93f80aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/1378629655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80aea30 0x7f93f80aeed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0098e0 tx=0x7f93ec0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- 172.21.15.64:0/1378629655 shutdown_connections 2023-09-27T23:26:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/1378629655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80aea30 0x7f93f80aeed0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- 172.21.15.64:0/1378629655 >> 172.21.15.64:0/1378629655 conn(0x7f93f80ccc20 msgr2=0x7f93f80cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- 172.21.15.64:0/1378629655 shutdown_connections 2023-09-27T23:26:26.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- 172.21.15.64:0/1378629655 wait complete. 2023-09-27T23:26:26.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 Processor -- start 2023-09-27T23:26:26.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- start start 2023-09-27T23:26:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93f80aea30 0x7f93f8155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80c7780 0x7f93f8068520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 0x7f93f8067a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f813bca0 con 0x7f93f813f830 2023-09-27T23:26:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f8067580 con 0x7f93f80c7780 2023-09-27T23:26:26.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93fd5b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f8155bc0 con 0x7f93f80aea30 2023-09-27T23:26:26.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 0x7f93f8067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93f67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80c7780 0x7f93f8068520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 0x7f93f8067a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34362/0 (socket says 172.21.15.64:34362) 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93f77fe700 1 -- 172.21.15.64:0/2825476828 learned_addr learned my addr 172.21.15.64:0/2825476828 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.764+0000 7f93f6ffd700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93f80aea30 0x7f93f8155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93f77fe700 1 -- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93f80aea30 msgr2=0x7f93f8155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93f77fe700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93f80aea30 0x7f93f8155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93f77fe700 1 -- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80c7780 msgr2=0x7f93f8068520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93f77fe700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80c7780 0x7f93f8068520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93f77fe700 1 -- 172.21.15.64:0/2825476828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec009420 con 0x7f93f813f830 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93f77fe700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 0x7f93f8067a10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c870 tx=0x7f93f0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0012630 con 0x7f93f813f830 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f00127b0 con 0x7f93f813f830 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f813b6e0 con 0x7f93f813f830 2023-09-27T23:26:26.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0012b70 con 0x7f93f813f830 2023-09-27T23:26:26.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f8067210 con 0x7f93f813f830 2023-09-27T23:26:26.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f8067710 con 0x7f93f813f830 2023-09-27T23:26:26.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.768+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f93f0003030 con 0x7f93f813f830 2023-09-27T23:26:26.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.772+0000 7f93deffd700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f93e4060b50 0x7f93e4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:26.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.772+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f93f007d250 con 0x7f93f813f830 2023-09-27T23:26:26.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.772+0000 7f93f6ffd700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f93e4060b50 0x7f93e4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:26.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.772+0000 7f93f6ffd700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f93e4060b50 0x7f93e4063000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93e8001620 tx=0x7f93e8000bc0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:26.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.776+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f93f004cfd0 con 0x7f93f813f830 2023-09-27T23:26:26.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.960+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93f8071120 con 0x7f93e4060b50 2023-09-27T23:26:26.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.980+0000 7f93deffd700 1 -- 172.21.15.64:0/2825476828 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f93f8071120 con 0x7f93e4060b50 2023-09-27T23:26:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:26.986 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:02", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f93e4060b50 msgr2=0x7f93e4063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f93e4060b50 0x7f93e4063000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93e8001620 tx=0x7f93e8000bc0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 msgr2=0x7f93f8067a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 0x7f93f8067a10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c870 tx=0x7f93f0001670 comp rx=0 tx=0).stop 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 shutdown_connections 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f93e4060b50 0x7f93e4063000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93f813f830 0x7f93f8067a10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93f80c7780 0x7f93f8068520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 --2- 172.21.15.64:0/2825476828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93f80aea30 0x7f93f8155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 >> 172.21.15.64:0/2825476828 conn(0x7f93f80ccc20 msgr2=0x7f93f80ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 shutdown_connections 2023-09-27T23:26:26.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:26.984+0000 7f93fd5b3700 1 -- 172.21.15.64:0/2825476828 wait complete. 2023-09-27T23:26:27.003 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:02", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:27.003 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:02', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:27.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:27.004 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-09-27T23:26:27.020 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:27.020 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_01_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_01_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:27.021 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:27.022 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_01_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_01_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_01_UTC'] 2023-09-27T23:26:27.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-09-27T23:26:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 Processor -- start 2023-09-27T23:26:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- start start 2023-09-27T23:26:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09e8156640 0x7f09e8156a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09e80c8f30 0x7f09e806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e806c7c0 con 0x7f09e80c8ae0 2023-09-27T23:26:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e806ca10 con 0x7f09e80c8f30 2023-09-27T23:26:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e806ccf0 con 0x7f09e8156640 2023-09-27T23:26:27.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09e80c8f30 0x7f09e806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09e80c8f30 0x7f09e806d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34376/0 (socket says 172.21.15.64:34376) 2023-09-27T23:26:27.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 -- 172.21.15.64:0/3770218602 learned_addr learned my addr 172.21.15.64:0/3770218602 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:27.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09e7fff700 1 --2- 172.21.15.64:0/3770218602 >> v2:172.21.15.123:3300/0 conn(0x7f09e8156640 0x7f09e8156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 -- 172.21.15.64:0/3770218602 >> v2:172.21.15.123:3300/0 conn(0x7f09e8156640 msgr2=0x7f09e8156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 --2- 172.21.15.64:0/3770218602 >> v2:172.21.15.123:3300/0 conn(0x7f09e8156640 0x7f09e8156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 -- 172.21.15.64:0/3770218602 >> v1:172.21.15.123:6789/0 conn(0x7f09e80c8ae0 legacy=0x7f09e8157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 -- 172.21.15.64:0/3770218602 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e806cfd0 con 0x7f09e80c8f30 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ecd4d700 1 --2- 172.21.15.64:0/3770218602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e806d7d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f09e8068060 tx=0x7f09e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a2bc6a8f76486a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09e6ffd700 1 -- 172.21.15.64:0/3770218602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000e070 con 0x7f09e80c8f30 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09e6ffd700 1 -- 172.21.15.64:0/3770218602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e000ba20 con 0x7f09e80c8f30 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09e6ffd700 1 -- 172.21.15.64:0/3770218602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00092e0 con 0x7f09e80c8f30 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- 172.21.15.64:0/3770218602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 msgr2=0x7f09e806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 --2- 172.21.15.64:0/3770218602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e806d7d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f09e8068060 tx=0x7f09e0009840 comp rx=0 tx=0).stop 2023-09-27T23:26:27.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- 172.21.15.64:0/3770218602 shutdown_connections 2023-09-27T23:26:27.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 --2- 172.21.15.64:0/3770218602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e806d7d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 --2- 172.21.15.64:0/3770218602 >> v2:172.21.15.123:3300/0 conn(0x7f09e8156640 0x7f09e8156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- 172.21.15.64:0/3770218602 >> 172.21.15.64:0/3770218602 conn(0x7f09e8067bb0 msgr2=0x7f09e80c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:27.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- 172.21.15.64:0/3770218602 shutdown_connections 2023-09-27T23:26:27.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.184+0000 7f09ed54e700 1 -- 172.21.15.64:0/3770218602 wait complete. 2023-09-27T23:26:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 Processor -- start 2023-09-27T23:26:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 -- start start 2023-09-27T23:26:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e80c8ae0 0x7f09e814a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e8156640 0x7f09e814f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81538c0 con 0x7f09e80c8f30 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8152a10 con 0x7f09e8156640 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ed54e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8153d50 con 0x7f09e80c8ae0 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e81522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34390/0 (socket says 172.21.15.64:34390) 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 -- 172.21.15.64:0/473907749 learned_addr learned my addr 172.21.15.64:0/473907749 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ecd4d700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e8156640 0x7f09e814f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e7fff700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e80c8ae0 0x7f09e814a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 -- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e80c8ae0 msgr2=0x7f09e814a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e80c8ae0 0x7f09e814a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 -- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e8156640 msgr2=0x7f09e814f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e8156640 0x7f09e814f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09e77fe700 1 -- 172.21.15.64:0/473907749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009ae0 con 0x7f09e80c8f30 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.188+0000 7f09ecd4d700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e8156640 0x7f09e814f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09e77fe700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e81522c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c4e0 tx=0x7f09dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc011590 con 0x7f09e80c8f30 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc011710 con 0x7f09e80c8f30 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e8152cb0 con 0x7f09e80c8f30 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc011ad0 con 0x7f09e80c8f30 2023-09-27T23:26:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e8153270 con 0x7f09e80c8f30 2023-09-27T23:26:27.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e8153a70 con 0x7f09e80c8f30 2023-09-27T23:26:27.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f09dc01b3b0 con 0x7f09e80c8f30 2023-09-27T23:26:27.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.192+0000 7f09cffff700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060b50 0x7f09d4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.196+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f09dc07c940 con 0x7f09e80c8f30 2023-09-27T23:26:27.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.196+0000 7f09e7fff700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060b50 0x7f09d4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.196+0000 7f09e7fff700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060b50 0x7f09d4063000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009ca0 tx=0x7f09d8008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.204+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f09e8153a70 con 0x7f09e80c8f30 2023-09-27T23:26:27.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.384+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09e8152800 con 0x7f09d4060b50 2023-09-27T23:26:27.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09cffff700 1 -- 172.21.15.64:0/473907749 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f09e8152800 con 0x7f09d4060b50 2023-09-27T23:26:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:27.409 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:03", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060b50 msgr2=0x7f09d4063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060b50 0x7f09d4063000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009ca0 tx=0x7f09d8008040 comp rx=0 tx=0).stop 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 msgr2=0x7f09e81522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e81522c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c4e0 tx=0x7f09dc00c8a0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 shutdown_connections 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d4060b50 0x7f09d4063000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e8156640 0x7f09e814f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09e80c8f30 0x7f09e81522c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 --2- 172.21.15.64:0/473907749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e80c8ae0 0x7f09e814a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 >> 172.21.15.64:0/473907749 conn(0x7f09e8067bb0 msgr2=0x7f09e80cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 shutdown_connections 2023-09-27T23:26:27.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.404+0000 7f09ed54e700 1 -- 172.21.15.64:0/473907749 wait complete. 2023-09-27T23:26:27.425 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:03", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:27.425 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:03', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:27.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:27.426 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-09-27T23:26:27.440 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_01_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:27.442 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_01_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_01_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:27.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-09-27T23:26:27.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.552+0000 7f4a7b59e700 1 Processor -- start 2023-09-27T23:26:27.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.552+0000 7f4a7b59e700 1 -- start start 2023-09-27T23:26:27.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a7c158c20 0x7f4a7c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a7c0cb5b0 0x7f4a7c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7c0c8380 con 0x7f4a7c0c8d50 2023-09-27T23:26:27.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c0c77b0 con 0x7f4a7c158c20 2023-09-27T23:26:27.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c0c7a50 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a7c158c20 0x7f4a7c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a7c158c20 0x7f4a7c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34404/0 (socket says 172.21.15.64:34404) 2023-09-27T23:26:27.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 -- 172.21.15.64:0/2833996208 learned_addr learned my addr 172.21.15.64:0/2833996208 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:27.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7ad9d700 1 --2- 172.21.15.64:0/2833996208 >> v2:172.21.15.123:3300/0 conn(0x7f4a7c0cb5b0 0x7f4a7c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 -- 172.21.15.64:0/2833996208 >> v2:172.21.15.123:3300/0 conn(0x7f4a7c0cb5b0 msgr2=0x7f4a7c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:27.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 --2- 172.21.15.64:0/2833996208 >> v2:172.21.15.123:3300/0 conn(0x7f4a7c0cb5b0 0x7f4a7c06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 -- 172.21.15.64:0/2833996208 >> v1:172.21.15.123:6789/0 conn(0x7f4a7c0c8d50 legacy=0x7f4a7c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 -- 172.21.15.64:0/2833996208 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0c7d30 con 0x7f4a7c158c20 2023-09-27T23:26:27.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7a59c700 1 --2- 172.21.15.64:0/2833996208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c158c20 0x7f4a7c0c8810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0caa20 tx=0x7f4a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c091bb0c3d1481 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a79d9b700 1 -- 172.21.15.64:0/2833996208 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00b860 con 0x7f4a7c158c20 2023-09-27T23:26:27.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a79d9b700 1 -- 172.21.15.64:0/2833996208 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00b9c0 con 0x7f4a7c158c20 2023-09-27T23:26:27.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a79d9b700 1 -- 172.21.15.64:0/2833996208 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00bd60 con 0x7f4a7c158c20 2023-09-27T23:26:27.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 -- 172.21.15.64:0/2833996208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c158c20 msgr2=0x7f4a7c0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/2833996208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c158c20 0x7f4a7c0c8810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0caa20 tx=0x7f4a6c009250 comp rx=0 tx=0).stop 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 -- 172.21.15.64:0/2833996208 shutdown_connections 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.556+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/2833996208 >> v2:172.21.15.123:3300/0 conn(0x7f4a7c0cb5b0 0x7f4a7c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/2833996208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c158c20 0x7f4a7c0c8810 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- 172.21.15.64:0/2833996208 >> 172.21.15.64:0/2833996208 conn(0x7f4a7c06b600 msgr2=0x7f4a7c06ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- 172.21.15.64:0/2833996208 shutdown_connections 2023-09-27T23:26:27.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- 172.21.15.64:0/2833996208 wait complete. 2023-09-27T23:26:27.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 Processor -- start 2023-09-27T23:26:27.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- start start 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c0c8d50 0x7f4a7c1484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 0x7f4a7c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a7c158c20 0x7f4a7c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0c6bc0 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c159860 con 0x7f4a7c0c8d50 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.560+0000 7f4a7b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0c68e0 con 0x7f4a7c158c20 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 0x7f4a7c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a7ad9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a7c158c20 0x7f4a7c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a7a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c0c8d50 0x7f4a7c1484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 0x7f4a7c0c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34410/0 (socket says 172.21.15.64:34410) 2023-09-27T23:26:27.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 -- 172.21.15.64:0/1242376074 learned_addr learned my addr 172.21.15.64:0/1242376074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 -- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a7c158c20 msgr2=0x7f4a7c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a7c158c20 0x7f4a7c159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 -- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c0c8d50 msgr2=0x7f4a7c1484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c0c8d50 0x7f4a7c1484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 -- 172.21.15.64:0/1242376074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0094f0 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a71d9b700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 0x7f4a7c0c5f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c4e0 tx=0x7f4a6400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64011590 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a64011710 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7c159210 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64011ad0 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c148a10 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.564+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7c159070 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.568+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f4a6401b3b0 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.568+0000 7f4a72ffd700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a5c060b50 0x7f4a5c063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.568+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f4a6407c7e0 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.568+0000 7f4a7a59c700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a5c060b50 0x7f4a5c063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.568+0000 7f4a7a59c700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a5c060b50 0x7f4a5c063000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0caa20 tx=0x7f4a6c017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.576+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4a7c159070 con 0x7f4a7c0cb5b0 2023-09-27T23:26:27.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.756+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a7c159070 con 0x7f4a5c060b50 2023-09-27T23:26:27.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.772+0000 7f4a72ffd700 1 -- 172.21.15.64:0/1242376074 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4a7c159070 con 0x7f4a5c060b50 2023-09-27T23:26:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:27.779 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:03", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a5c060b50 msgr2=0x7f4a5c063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a5c060b50 0x7f4a5c063000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0caa20 tx=0x7f4a6c017000 comp rx=0 tx=0).stop 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 msgr2=0x7f4a7c0c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 0x7f4a7c0c5f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c4e0 tx=0x7f4a6400c7f0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 shutdown_connections 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a5c060b50 0x7f4a5c063000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a7c158c20 0x7f4a7c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a7c0cb5b0 0x7f4a7c0c5f70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 --2- 172.21.15.64:0/1242376074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c0c8d50 0x7f4a7c1484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 >> 172.21.15.64:0/1242376074 conn(0x7f4a7c06b600 msgr2=0x7f4a7c0cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:27.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 shutdown_connections 2023-09-27T23:26:27.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.776+0000 7f4a7b59e700 1 -- 172.21.15.64:0/1242376074 wait complete. 2023-09-27T23:26:27.795 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:03", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:27.795 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:03', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:27.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:27.795 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-09-27T23:26:27.812 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_01_UTC 2023-09-27T23:26:27.812 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:27.812 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:27.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:27.814 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_01_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:27.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-09-27T23:26:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.972+0000 7f3923151700 1 Processor -- start 2023-09-27T23:26:27.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.972+0000 7f3923151700 1 -- start start 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f391c06af50 0x7f391c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391c0cb3f0 con 0x7f391c06d6b0 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c0cc140 con 0x7f391c06af50 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c06d220 con 0x7f391c0aedc0 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f391c06af50 0x7f391c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f391c06af50 0x7f391c0cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34424/0 (socket says 172.21.15.64:34424) 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 -- 172.21.15.64:0/535993786 learned_addr learned my addr 172.21.15.64:0/535993786 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:27.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392214f700 1 --2- 172.21.15.64:0/535993786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 -- 172.21.15.64:0/535993786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 msgr2=0x7f391c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 --2- 172.21.15.64:0/535993786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 -- 172.21.15.64:0/535993786 >> v1:172.21.15.123:6789/0 conn(0x7f391c06d6b0 legacy=0x7f391c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 -- 172.21.15.64:0/535993786 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c0cb190 con 0x7f391c06af50 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392194e700 1 --2- 172.21.15.64:0/535993786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c0cac50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f39180099b0 tx=0x7f3918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab7e55d1387a369 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392114d700 1 -- 172.21.15.64:0/535993786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918009ea0 con 0x7f391c06af50 2023-09-27T23:26:27.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392114d700 1 -- 172.21.15.64:0/535993786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800b860 con 0x7f391c06af50 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f392114d700 1 -- 172.21.15.64:0/535993786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800bc00 con 0x7f391c06af50 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- 172.21.15.64:0/535993786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 msgr2=0x7f391c0cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 --2- 172.21.15.64:0/535993786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c0cac50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f39180099b0 tx=0x7f3918009250 comp rx=0 tx=0).stop 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- 172.21.15.64:0/535993786 shutdown_connections 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 --2- 172.21.15.64:0/535993786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c0cac50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 --2- 172.21.15.64:0/535993786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- 172.21.15.64:0/535993786 >> 172.21.15.64:0/535993786 conn(0x7f391c0cc760 msgr2=0x7f391c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- 172.21.15.64:0/535993786 shutdown_connections 2023-09-27T23:26:27.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.976+0000 7f3923151700 1 -- 172.21.15.64:0/535993786 wait complete. 2023-09-27T23:26:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 Processor -- start 2023-09-27T23:26:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 -- start start 2023-09-27T23:26:27.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c15c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f391c06d6b0 0x7f391c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0c3750 con 0x7f391c06af50 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0c39a0 con 0x7f391c06d6b0 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3923151700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0c3c80 con 0x7f391c0aedc0 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f392214f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f392214f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34430/0 (socket says 172.21.15.64:34430) 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f392214f700 1 -- 172.21.15.64:0/3981123398 learned_addr learned my addr 172.21.15.64:0/3981123398 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f392194e700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f391c06d6b0 0x7f391c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.980+0000 7f3922950700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f392214f700 1 -- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 msgr2=0x7f391c14cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f392214f700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c14cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f392214f700 1 -- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f391c06d6b0 msgr2=0x7f391c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f392214f700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f391c06d6b0 0x7f391c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f392214f700 1 -- 172.21.15.64:0/3981123398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39180094f0 con 0x7f391c06af50 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f392214f700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c15c160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f390c00ec80 tx=0x7f390c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00c850 con 0x7f391c06af50 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c00c9b0 con 0x7f391c06af50 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c0c3f60 con 0x7f391c06af50 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00cd50 con 0x7f391c06af50 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c15c6a0 con 0x7f391c06af50 2023-09-27T23:26:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c06d3d0 con 0x7f391c06af50 2023-09-27T23:26:27.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.984+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f390c013680 con 0x7f391c06af50 2023-09-27T23:26:27.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.988+0000 7f39127fc700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3900060b50 0x7f3900063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:27.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.988+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f390c07d680 con 0x7f391c06af50 2023-09-27T23:26:27.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.988+0000 7f392194e700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3900060b50 0x7f3900063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:27.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.988+0000 7f392194e700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3900060b50 0x7f3900063000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39180099b0 tx=0x7f3918017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:27.996+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f391c06d3d0 con 0x7f391c06af50 2023-09-27T23:26:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.180+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f391c06d3d0 con 0x7f3900060b50 2023-09-27T23:26:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.196+0000 7f39127fc700 1 -- 172.21.15.64:0/3981123398 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f391c06d3d0 con 0x7f3900060b50 2023-09-27T23:26:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:28.203 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:04", "first": "2023-09-27T23:16:01", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3900060b50 msgr2=0x7f3900063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3900060b50 0x7f3900063000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39180099b0 tx=0x7f3918017000 comp rx=0 tx=0).stop 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 msgr2=0x7f391c15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c15c160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f390c00ec80 tx=0x7f390c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 shutdown_connections 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3900060b50 0x7f3900063000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f391c0aedc0 0x7f391c14cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f391c06d6b0 0x7f391c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 --2- 172.21.15.64:0/3981123398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391c06af50 0x7f391c15c160 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 >> 172.21.15.64:0/3981123398 conn(0x7f391c0cc760 msgr2=0x7f391c15ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 shutdown_connections 2023-09-27T23:26:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.200+0000 7f3923151700 1 -- 172.21.15.64:0/3981123398 wait complete. 2023-09-27T23:26:28.222 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:04", "first": "2023-09-27T23:16:01", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:28.222 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:04', 'first': '2023-09-27T23:16:01', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:28.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:28.222 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:28.238 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:28.239 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:28.239 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_01_UTC 2023-09-27T23:26:28.239 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:28.239 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:28.240 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_01_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_01_UTC', 'scheduled-2023-09-27-23_25_01_UTC', 'scheduled-2023-09-27-23_26_01_UTC'] 2023-09-27T23:26:28.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-09-27T23:26:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.356+0000 7fc2dec8b700 1 Processor -- start 2023-09-27T23:26:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.356+0000 7fc2dec8b700 1 -- start start 2023-09-27T23:26:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2d80ae270 0x7fc2d80ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc2d80aef60 0x7fc2d80b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d806d7b0 con 0x7fc2d8156720 2023-09-27T23:26:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d80b2060 con 0x7fc2d80ae270 2023-09-27T23:26:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d806cba0 con 0x7fc2d80aef60 2023-09-27T23:26:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2ddc89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2d80ae270 0x7fc2d80ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2de48a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc2d8156720 0x7fc2d8156ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52766/0 (socket says 172.21.15.64:52766) 2023-09-27T23:26:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2de48a700 1 -- 172.21.15.64:0/1189996912 learned_addr learned my addr 172.21.15.64:0/1189996912 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dd488700 1 --2- 172.21.15.64:0/1189996912 >> v2:172.21.15.123:3300/0 conn(0x7fc2d80aef60 0x7fc2d80b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dcc87700 1 -- 172.21.15.64:0/1189996912 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919372929 0 0) 0x7fc2d806d7b0 con 0x7fc2d8156720 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dcc87700 1 -- 172.21.15.64:0/1189996912 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c4003570 con 0x7fc2d8156720 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2ddc89700 1 -- 172.21.15.64:0/1189996912 >> v2:172.21.15.123:3300/0 conn(0x7fc2d80aef60 msgr2=0x7fc2d80b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2ddc89700 1 --2- 172.21.15.64:0/1189996912 >> v2:172.21.15.123:3300/0 conn(0x7fc2d80aef60 0x7fc2d80b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2ddc89700 1 -- 172.21.15.64:0/1189996912 >> v1:172.21.15.64:6789/0 conn(0x7fc2d8156720 legacy=0x7fc2d8156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2ddc89700 1 -- 172.21.15.64:0/1189996912 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d806ce80 con 0x7fc2d80ae270 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2ddc89700 1 --2- 172.21.15.64:0/1189996912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d80ae270 0x7fc2d80ae640 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40098e0 tx=0x7fc2d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c7f9f248b6ba4a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dcc87700 1 -- 172.21.15.64:0/1189996912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011630 con 0x7fc2d80ae270 2023-09-27T23:26:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dcc87700 1 -- 172.21.15.64:0/1189996912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4011790 con 0x7fc2d80ae270 2023-09-27T23:26:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dcc87700 1 -- 172.21.15.64:0/1189996912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011b30 con 0x7fc2d80ae270 2023-09-27T23:26:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- 172.21.15.64:0/1189996912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d80ae270 msgr2=0x7fc2d80ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/1189996912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d80ae270 0x7fc2d80ae640 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40098e0 tx=0x7fc2d40091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- 172.21.15.64:0/1189996912 shutdown_connections 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/1189996912 >> v2:172.21.15.123:3300/0 conn(0x7fc2d80aef60 0x7fc2d80b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/1189996912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d80ae270 0x7fc2d80ae640 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- 172.21.15.64:0/1189996912 >> 172.21.15.64:0/1189996912 conn(0x7fc2d80c8c00 msgr2=0x7fc2d80c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- 172.21.15.64:0/1189996912 shutdown_connections 2023-09-27T23:26:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.360+0000 7fc2dec8b700 1 -- 172.21.15.64:0/1189996912 wait complete. 2023-09-27T23:26:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 Processor -- start 2023-09-27T23:26:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 -- start start 2023-09-27T23:26:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2d80ae270 0x7fc2d80c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d80aef60 0x7fc2d80c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 0x7fc2d80c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8153980 con 0x7fc2d8156720 2023-09-27T23:26:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8152d70 con 0x7fc2d80aef60 2023-09-27T23:26:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dec8b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8153050 con 0x7fc2d80ae270 2023-09-27T23:26:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2de48a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 0x7fc2d80c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2ddc89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2d80ae270 0x7fc2d80c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2dd488700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d80aef60 0x7fc2d80c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2de48a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 0x7fc2d80c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34452/0 (socket says 172.21.15.64:34452) 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.364+0000 7fc2de48a700 1 -- 172.21.15.64:0/2748747758 learned_addr learned my addr 172.21.15.64:0/2748747758 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2de48a700 1 -- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2d80ae270 msgr2=0x7fc2d80c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2de48a700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2d80ae270 0x7fc2d80c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2de48a700 1 -- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d80aef60 msgr2=0x7fc2d80c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2de48a700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d80aef60 0x7fc2d80c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2de48a700 1 -- 172.21.15.64:0/2748747758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4009420 con 0x7fc2d8156720 2023-09-27T23:26:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2de48a700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 0x7fc2d80c36e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0001460 tx=0x7fc2d0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000a870 con 0x7fc2d8156720 2023-09-27T23:26:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000a9d0 con 0x7fc2d8156720 2023-09-27T23:26:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d8153330 con 0x7fc2d8156720 2023-09-27T23:26:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000ad50 con 0x7fc2d8156720 2023-09-27T23:26:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d814dc80 con 0x7fc2d8156720 2023-09-27T23:26:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.368+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d8153b30 con 0x7fc2d8156720 2023-09-27T23:26:28.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.372+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fc2d0007300 con 0x7fc2d8156720 2023-09-27T23:26:28.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.372+0000 7fc2cdffb700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2bc060b50 0x7fc2bc063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.372+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fc2d007d820 con 0x7fc2d8156720 2023-09-27T23:26:28.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.372+0000 7fc2ddc89700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2bc060b50 0x7fc2bc063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.372+0000 7fc2ddc89700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2bc060b50 0x7fc2bc063000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40093f0 tx=0x7fc2d4017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.380+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc2d8153b30 con 0x7fc2d8156720 2023-09-27T23:26:28.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.568+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2d804be40 con 0x7fc2bc060b50 2023-09-27T23:26:28.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2cdffb700 1 -- 172.21.15.64:0/2748747758 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc2d804be40 con 0x7fc2bc060b50 2023-09-27T23:26:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:28.596 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:04", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:28.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2bc060b50 msgr2=0x7fc2bc063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2bc060b50 0x7fc2bc063000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40093f0 tx=0x7fc2d4017000 comp rx=0 tx=0).stop 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 msgr2=0x7fc2d80c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 0x7fc2d80c36e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0001460 tx=0x7fc2d0001a40 comp rx=0 tx=0).stop 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 shutdown_connections 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2bc060b50 0x7fc2bc063000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d8156720 0x7fc2d80c36e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2d80aef60 0x7fc2d80c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 --2- 172.21.15.64:0/2748747758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2d80ae270 0x7fc2d80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 >> 172.21.15.64:0/2748747758 conn(0x7fc2d80c8c00 msgr2=0x7fc2d80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 shutdown_connections 2023-09-27T23:26:28.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.592+0000 7fc2dec8b700 1 -- 172.21.15.64:0/2748747758 wait complete. 2023-09-27T23:26:28.615 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:04", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:28.615 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:04', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:28.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:28.615 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-09-27T23:26:28.628 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_01_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_01_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:28.630 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_01_UTC', 'scheduled-2023-09-27-23_17_00_UTC', 'scheduled-2023-09-27-23_18_01_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:28.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-09-27T23:26:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.740+0000 7f9ef0d86700 1 Processor -- start 2023-09-27T23:26:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.740+0000 7f9ef0d86700 1 -- start start 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9ef0d86700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9eec06df10 0x7f9eec06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9ef0d86700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec06b420 con 0x7f9eec158da0 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9ef0d86700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec15d720 con 0x7f9eec06ed80 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9ef0d86700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec0ca290 con 0x7f9eec06df10 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9eec158da0 0x7f9eec06e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52770/0 (socket says 172.21.15.64:52770) 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeb7fe700 1 -- 172.21.15.64:0/1915447313 learned_addr learned my addr 172.21.15.64:0/1915447313 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eebfff700 1 --2- 172.21.15.64:0/1915447313 >> v2:172.21.15.123:3300/0 conn(0x7f9eec06df10 0x7f9eec06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeaffd700 1 -- 172.21.15.64:0/1915447313 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351586842 0 0) 0x7f9eec06b420 con 0x7f9eec158da0 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeaffd700 1 -- 172.21.15.64:0/1915447313 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed8003570 con 0x7f9eec158da0 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eebfff700 1 -- 172.21.15.64:0/1915447313 >> v1:172.21.15.123:6789/0 conn(0x7f9eec06ed80 legacy=0x7f9eec06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eebfff700 1 -- 172.21.15.64:0/1915447313 >> v1:172.21.15.64:6789/0 conn(0x7f9eec158da0 legacy=0x7f9eec06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eebfff700 1 -- 172.21.15.64:0/1915447313 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0ca4a0 con 0x7f9eec06df10 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eebfff700 1 --2- 172.21.15.64:0/1915447313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06df10 0x7f9eec06aee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009e60 tx=0x7f9ee4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0238b4c7aa76a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeaffd700 1 -- 172.21.15.64:0/1915447313 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400e070 con 0x7f9eec06df10 2023-09-27T23:26:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeaffd700 1 -- 172.21.15.64:0/1915447313 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee400bb60 con 0x7f9eec06df10 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.744+0000 7f9eeaffd700 1 -- 172.21.15.64:0/1915447313 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4012430 con 0x7f9eec06df10 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1915447313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06df10 msgr2=0x7f9eec06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1915447313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06df10 0x7f9eec06aee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009e60 tx=0x7f9ee4009780 comp rx=0 tx=0).stop 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1915447313 shutdown_connections 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1915447313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06df10 0x7f9eec06aee0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1915447313 >> 172.21.15.64:0/1915447313 conn(0x7f9eec0cc760 msgr2=0x7f9eec0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1915447313 shutdown_connections 2023-09-27T23:26:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1915447313 wait complete. 2023-09-27T23:26:28.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 Processor -- start 2023-09-27T23:26:28.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- start start 2023-09-27T23:26:28.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 0x7f9eec0c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06ed80 0x7f9eec1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9eec158da0 0x7f9eec0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0c6ab0 con 0x7f9eec06df10 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0c6560 con 0x7f9eec06ed80 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.748+0000 7f9ef0d86700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0c3aa0 con 0x7f9eec158da0 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9eeb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 0x7f9eec0c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06ed80 0x7f9eec1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9eeb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 0x7f9eec0c3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34458/0 (socket says 172.21.15.64:34458) 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9eeb7fe700 1 -- 172.21.15.64:0/1091997902 learned_addr learned my addr 172.21.15.64:0/1091997902 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9eebfff700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9eec158da0 0x7f9eec0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 -- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9eec158da0 msgr2=0x7f9eec0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9eec158da0 0x7f9eec0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 -- 172.21.15.64:0/1091997902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 msgr2=0x7f9eec0c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 0x7f9eec0c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 -- 172.21.15.64:0/1091997902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4009a20 con 0x7f9eec06ed80 2023-09-27T23:26:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9eeb7fe700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 0x7f9eec0c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:28.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee3fff700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06ed80 0x7f9eec1559c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c5c0 tx=0x7f9ed4007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4011590 con 0x7f9eec06ed80 2023-09-27T23:26:28.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4011710 con 0x7f9eec06ed80 2023-09-27T23:26:28.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec0c67d0 con 0x7f9eec06ed80 2023-09-27T23:26:28.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4011ad0 con 0x7f9eec06ed80 2023-09-27T23:26:28.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec155f00 con 0x7f9eec06ed80 2023-09-27T23:26:28.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.752+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec04be40 con 0x7f9eec06ed80 2023-09-27T23:26:28.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.756+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f9ed401b3b0 con 0x7f9eec06ed80 2023-09-27T23:26:28.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.756+0000 7f9ee37fe700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9ecc060b50 0x7f9ecc063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:28.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.756+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f9ed407c840 con 0x7f9eec06ed80 2023-09-27T23:26:28.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.756+0000 7f9eeb7fe700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9ecc060b50 0x7f9ecc063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:28.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.756+0000 7f9eeb7fe700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9ecc060b50 0x7f9ecc063000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9edc001e30 tx=0x7f9edc009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:28.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.764+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9ed404c520 con 0x7f9eec06ed80 2023-09-27T23:26:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.948+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9eec158950 con 0x7f9ecc060b50 2023-09-27T23:26:28.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.964+0000 7f9ee37fe700 1 -- 172.21.15.64:0/1091997902 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f9eec158950 con 0x7f9ecc060b50 2023-09-27T23:26:28.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:28.971 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:05", "first": "2023-09-27T23:16:01", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:28.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9ecc060b50 msgr2=0x7f9ecc063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9ecc060b50 0x7f9ecc063000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9edc001e30 tx=0x7f9edc009040 comp rx=0 tx=0).stop 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06ed80 msgr2=0x7f9eec1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06ed80 0x7f9eec1559c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c5c0 tx=0x7f9ed4007ec0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 shutdown_connections 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9ecc060b50 0x7f9ecc063000 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9eec158da0 0x7f9eec0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec06ed80 0x7f9eec1559c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 --2- 172.21.15.64:0/1091997902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eec06df10 0x7f9eec0c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 >> 172.21.15.64:0/1091997902 conn(0x7f9eec0cc760 msgr2=0x7f9eec06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 shutdown_connections 2023-09-27T23:26:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:28.968+0000 7f9ef0d86700 1 -- 172.21.15.64:0/1091997902 wait complete. 2023-09-27T23:26:28.991 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:05", "first": "2023-09-27T23:16:01", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:28.991 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:05', 'first': '2023-09-27T23:16:01', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:28.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:28.991 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-09-27T23:26:29.007 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:29.007 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:29.008 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:29.009 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_16_00_UTC', 'scheduled-2023-09-27-23_17_01_UTC', 'scheduled-2023-09-27-23_18_00_UTC', 'scheduled-2023-09-27-23_19_00_UTC', 'scheduled-2023-09-27-23_20_00_UTC', 'scheduled-2023-09-27-23_21_00_UTC', 'scheduled-2023-09-27-23_22_00_UTC', 'scheduled-2023-09-27-23_23_00_UTC', 'scheduled-2023-09-27-23_24_00_UTC', 'scheduled-2023-09-27-23_25_00_UTC', 'scheduled-2023-09-27-23_26_00_UTC'] 2023-09-27T23:26:29.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-09-27T23:26:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.176+0000 7f593a945700 1 Processor -- start 2023-09-27T23:26:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.176+0000 7f593a945700 1 -- start start 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59340abe40 0x7f5934158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59340c85a0 con 0x7f5934158b00 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593406d9b0 con 0x7f593406bb70 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59340c67d0 con 0x7f59340abe40 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939142700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59340abe40 0x7f5934158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939943700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5934158b00 0x7f593406d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52778/0 (socket says 172.21.15.64:52778) 2023-09-27T23:26:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939142700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59340abe40 0x7f5934158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34462/0 (socket says 172.21.15.64:34462) 2023-09-27T23:26:29.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939943700 1 -- 172.21.15.64:0/788385413 learned_addr learned my addr 172.21.15.64:0/788385413 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:29.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939142700 1 -- 172.21.15.64:0/788385413 >> v1:172.21.15.123:6789/0 conn(0x7f593406bb70 legacy=0x7f593406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939142700 1 -- 172.21.15.64:0/788385413 >> v1:172.21.15.64:6789/0 conn(0x7f5934158b00 legacy=0x7f593406d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939142700 1 -- 172.21.15.64:0/788385413 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340c6ab0 con 0x7f59340abe40 2023-09-27T23:26:29.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5939142700 1 --2- 172.21.15.64:0/788385413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59340abe40 0x7f5934158ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f59300098e0 tx=0x7f59300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fa80ef4a67a22e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:29.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5938941700 1 -- 172.21.15.64:0/788385413 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930009e80 con 0x7f59340abe40 2023-09-27T23:26:29.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5938941700 1 -- 172.21.15.64:0/788385413 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593000b860 con 0x7f59340abe40 2023-09-27T23:26:29.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f5938941700 1 -- 172.21.15.64:0/788385413 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000bbe0 con 0x7f59340abe40 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 -- 172.21.15.64:0/788385413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59340abe40 msgr2=0x7f5934158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 --2- 172.21.15.64:0/788385413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59340abe40 0x7f5934158ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f59300098e0 tx=0x7f59300091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 -- 172.21.15.64:0/788385413 shutdown_connections 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 --2- 172.21.15.64:0/788385413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59340abe40 0x7f5934158ed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.180+0000 7f593a945700 1 -- 172.21.15.64:0/788385413 >> 172.21.15.64:0/788385413 conn(0x7f59340c8c00 msgr2=0x7f59340c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.184+0000 7f593a945700 1 -- 172.21.15.64:0/788385413 shutdown_connections 2023-09-27T23:26:29.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.184+0000 7f593a945700 1 -- 172.21.15.64:0/788385413 wait complete. 2023-09-27T23:26:29.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.184+0000 7f593a945700 1 Processor -- start 2023-09-27T23:26:29.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.184+0000 7f593a945700 1 -- start start 2023-09-27T23:26:29.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593406bb70 0x7f59340b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59340abe40 0x7f59340aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 0x7f59340aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593416c980 con 0x7f5934158b00 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593416d1e0 con 0x7f593406bb70 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59340ab490 con 0x7f59340abe40 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 0x7f59340aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 0x7f59340aaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34478/0 (socket says 172.21.15.64:34478) 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f5939943700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593406bb70 0x7f59340b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 -- 172.21.15.64:0/2799156166 learned_addr learned my addr 172.21.15.64:0/2799156166 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:29.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f5939142700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59340abe40 0x7f59340aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 -- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59340abe40 msgr2=0x7f59340aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59340abe40 0x7f59340aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 -- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593406bb70 msgr2=0x7f59340b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593406bb70 0x7f59340b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 -- 172.21.15.64:0/2799156166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930009420 con 0x7f5934158b00 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a144700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 0x7f59340aaf50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f592c00c8e0 tx=0x7f592c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c012630 con 0x7f5934158b00 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c012790 con 0x7f5934158b00 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59340c5e00 con 0x7f5934158b00 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c012b50 con 0x7f5934158b00 2023-09-27T23:26:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.188+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593416ce70 con 0x7f5934158b00 2023-09-27T23:26:29.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.192+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593416d370 con 0x7f5934158b00 2023-09-27T23:26:29.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.192+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f592c01a3a0 con 0x7f5934158b00 2023-09-27T23:26:29.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.192+0000 7f5929ffb700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5918060b50 0x7f5918063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.192+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f592c07d600 con 0x7f5934158b00 2023-09-27T23:26:29.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.192+0000 7f5939943700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5918060b50 0x7f5918063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.192+0000 7f5939943700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5918060b50 0x7f5918063000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5924001950 tx=0x7f5924009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.200+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f592c04d380 con 0x7f5934158b00 2023-09-27T23:26:29.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.384+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f593404be40 con 0x7f5918060b50 2023-09-27T23:26:29.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.400+0000 7f5929ffb700 1 -- 172.21.15.64:0/2799156166 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f593404be40 con 0x7f5918060b50 2023-09-27T23:26:29.407 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:29.408 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:05", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5918060b50 msgr2=0x7f5918063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5918060b50 0x7f5918063000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5924001950 tx=0x7f5924009040 comp rx=0 tx=0).stop 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 msgr2=0x7f59340aaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 0x7f59340aaf50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f592c00c8e0 tx=0x7f592c001710 comp rx=0 tx=0).stop 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 shutdown_connections 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5918060b50 0x7f5918063000 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5934158b00 0x7f59340aaf50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59340abe40 0x7f59340aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 --2- 172.21.15.64:0/2799156166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593406bb70 0x7f59340b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 >> 172.21.15.64:0/2799156166 conn(0x7f59340c8c00 msgr2=0x7f5934157830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 shutdown_connections 2023-09-27T23:26:29.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.404+0000 7f593a945700 1 -- 172.21.15.64:0/2799156166 wait complete. 2023-09-27T23:26:29.427 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:15:05", "first": "2023-09-27T23:16:00", "last": "2023-09-27T23:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-09-27T23:26:29.427 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:15:05', 'first': '2023-09-27T23:16:00', 'last': '2023-09-27T23:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-09-27T23:26:29.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-09-27T23:26:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 Processor -- start 2023-09-27T23:26:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 -- start start 2023-09-27T23:26:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5afc0ad9f0 0x7f5afc0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5afc0c8db0 0x7f5afc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc06b730 con 0x7f5afc0c8db0 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc15b680 con 0x7f5afc0ad9f0 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b02401700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0c9230 con 0x7f5afc06ade0 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5afc0c8db0 0x7f5afc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5afc0c8db0 0x7f5afc15b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34488/0 (socket says 172.21.15.64:34488) 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 -- 172.21.15.64:0/3083944595 learned_addr learned my addr 172.21.15.64:0/3083944595 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b013ff700 1 --2- 172.21.15.64:0/3083944595 >> v2:172.21.15.123:3300/0 conn(0x7f5afc0ad9f0 0x7f5afc0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b01c00700 1 --2- 172.21.15.64:0/3083944595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 -- 172.21.15.64:0/3083944595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 msgr2=0x7f5afc06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 --2- 172.21.15.64:0/3083944595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc06b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 -- 172.21.15.64:0/3083944595 >> v2:172.21.15.123:3300/0 conn(0x7f5afc0ad9f0 msgr2=0x7f5afc0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 --2- 172.21.15.64:0/3083944595 >> v2:172.21.15.123:3300/0 conn(0x7f5afc0ad9f0 0x7f5afc0addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 -- 172.21.15.64:0/3083944595 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc06da90 con 0x7f5afc0c8db0 2023-09-27T23:26:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5b00bfe700 1 --2- 172.21.15.64:0/3083944595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc15b140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5afc069830 tx=0x7f5aec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfcb2729a5eec41 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5af3fff700 1 -- 172.21.15.64:0/3083944595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00b640 con 0x7f5afc0c8db0 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5af3fff700 1 -- 172.21.15.64:0/3083944595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00b7a0 con 0x7f5afc0c8db0 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.544+0000 7f5af3fff700 1 -- 172.21.15.64:0/3083944595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00bb40 con 0x7f5afc0c8db0 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- 172.21.15.64:0/3083944595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 msgr2=0x7f5afc15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- 172.21.15.64:0/3083944595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc15b140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5afc069830 tx=0x7f5aec009250 comp rx=0 tx=0).stop 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- 172.21.15.64:0/3083944595 shutdown_connections 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- 172.21.15.64:0/3083944595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc06b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- 172.21.15.64:0/3083944595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc15b140 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- 172.21.15.64:0/3083944595 >> v2:172.21.15.123:3300/0 conn(0x7f5afc0ad9f0 0x7f5afc0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- 172.21.15.64:0/3083944595 >> 172.21.15.64:0/3083944595 conn(0x7f5afc067bb0 msgr2=0x7f5afc0c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- 172.21.15.64:0/3083944595 shutdown_connections 2023-09-27T23:26:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- 172.21.15.64:0/3083944595 wait complete. 2023-09-27T23:26:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 Processor -- start 2023-09-27T23:26:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- start start 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5afc0ad9f0 0x7f5afc13e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc13ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0a9780 con 0x7f5afc0c8db0 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0aac00 con 0x7f5afc0ad9f0 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.548+0000 7f5b02401700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0aa2d0 con 0x7f5afc06ade0 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc13ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b013ff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc13ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34502/0 (socket says 172.21.15.64:34502) 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 -- 172.21.15.64:0/2152909115 learned_addr learned my addr 172.21.15.64:0/2152909115 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b00bfe700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5afc0ad9f0 0x7f5afc13e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 -- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 msgr2=0x7f5afc156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 -- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5afc0ad9f0 msgr2=0x7f5afc13e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5afc0ad9f0 0x7f5afc13e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 -- 172.21.15.64:0/2152909115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0094f0 con 0x7f5afc0c8db0 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b01c00700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc13ee30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009e90 tx=0x7f5af400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af400cb70 con 0x7f5afc0c8db0 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af400f040 con 0x7f5afc0c8db0 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc0aa5b0 con 0x7f5afc0c8db0 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4014480 con 0x7f5afc0c8db0 2023-09-27T23:26:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc156d60 con 0x7f5afc0c8db0 2023-09-27T23:26:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc0aad70 con 0x7f5afc0c8db0 2023-09-27T23:26:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.552+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f5af4014830 con 0x7f5afc0c8db0 2023-09-27T23:26:29.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.556+0000 7f5af17fa700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5ae0060b50 0x7f5ae0063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:29.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.556+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f5af407df10 con 0x7f5afc0c8db0 2023-09-27T23:26:29.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.556+0000 7f5b013ff700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5ae0060b50 0x7f5ae0063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:29.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.556+0000 7f5b013ff700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5ae0060b50 0x7f5ae0063000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009d20 tx=0x7f5af80091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:29.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.560+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5af404db80 con 0x7f5afc0c8db0 2023-09-27T23:26:29.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.744+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5afc04be40 con 0x7f5ae0060b50 2023-09-27T23:26:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.928+0000 7f5af17fa700 1 -- 172.21.15.64:0/2152909115 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f5afc04be40 con 0x7f5ae0060b50 2023-09-27T23:26:29.933 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5ae0060b50 msgr2=0x7f5ae0063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5ae0060b50 0x7f5ae0063000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009d20 tx=0x7f5af80091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 msgr2=0x7f5afc13ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc13ee30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009e90 tx=0x7f5af400b040 comp rx=0 tx=0).stop 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 shutdown_connections 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5ae0060b50 0x7f5ae0063000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5afc0c8db0 0x7f5afc13ee30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5afc0ad9f0 0x7f5afc13e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 --2- 172.21.15.64:0/2152909115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5afc06ade0 0x7f5afc156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 >> 172.21.15.64:0/2152909115 conn(0x7f5afc067bb0 msgr2=0x7f5afc13c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 shutdown_connections 2023-09-27T23:26:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:29.932+0000 7f5b02401700 1 -- 172.21.15.64:0/2152909115 wait complete. 2023-09-27T23:26:29.953 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-09-27T23:26:29.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:29.953 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-09-27T23:26:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_01_UTC 2023-09-27T23:26:29.969 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:29.970 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:29.970 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:29.970 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:29.970 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:29.970 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:29.971 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:29.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:29.971 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:29.997 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:29.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:29.998 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:30.067 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:30.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.068 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:30.094 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:30.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.095 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:30.121 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_20_01_UTC 2023-09-27T23:26:30.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.121 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_20_01_UTC) 2023-09-27T23:26:30.190 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:30.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.190 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:30.219 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:30.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.219 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:30.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:30.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.249 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:30.278 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:30.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.278 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:30.309 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:30.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.309 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_25_01_UTC) 2023-09-27T23:26:30.338 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:30.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.339 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:30.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.410 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-09-27T23:26:30.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-09-27T23:26:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 Processor -- start 2023-09-27T23:26:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 -- start start 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0c40cb130 0x7fc0c40cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc0c406a6a0 0x7fc0c406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c40b0ac0 con 0x7fc0c415c220 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c406b030 con 0x7fc0c40cb130 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0cc762700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c406b210 con 0x7fc0c406a6a0 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0ca4fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0c40cb130 0x7fc0c40cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0ca4fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0c40cb130 0x7fc0c40cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34506/0 (socket says 172.21.15.64:34506) 2023-09-27T23:26:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0ca4fe700 1 -- 172.21.15.64:0/2913785867 learned_addr learned my addr 172.21.15.64:0/2913785867 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:30.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.544+0000 7fc0c9cfd700 1 --2- 172.21.15.64:0/2913785867 >> v2:172.21.15.123:3300/0 conn(0x7fc0c406a6a0 0x7fc0c406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0c94fc700 1 -- 172.21.15.64:0/2913785867 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559871292 0 0) 0x7fc0c40b0ac0 con 0x7fc0c415c220 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0c94fc700 1 -- 172.21.15.64:0/2913785867 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b0003570 con 0x7fc0c415c220 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0ca4fe700 1 -- 172.21.15.64:0/2913785867 >> v2:172.21.15.123:3300/0 conn(0x7fc0c406a6a0 msgr2=0x7fc0c406aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0ca4fe700 1 --2- 172.21.15.64:0/2913785867 >> v2:172.21.15.123:3300/0 conn(0x7fc0c406a6a0 0x7fc0c406aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0ca4fe700 1 -- 172.21.15.64:0/2913785867 >> v1:172.21.15.123:6789/0 conn(0x7fc0c415c220 legacy=0x7fc0c406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0ca4fe700 1 -- 172.21.15.64:0/2913785867 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40ca810 con 0x7fc0c40cb130 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0c9cfd700 1 --2- 172.21.15.64:0/2913785867 >> v2:172.21.15.123:3300/0 conn(0x7fc0c406a6a0 0x7fc0c406aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0ca4fe700 1 --2- 172.21.15.64:0/2913785867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c40cb130 0x7fc0c40cb500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40cc930 tx=0x7fc0b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab33db77064e1233 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0c94fc700 1 -- 172.21.15.64:0/2913785867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400b640 con 0x7fc0c40cb130 2023-09-27T23:26:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0c94fc700 1 -- 172.21.15.64:0/2913785867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b400b7a0 con 0x7fc0c40cb130 2023-09-27T23:26:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0c94fc700 1 -- 172.21.15.64:0/2913785867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400bb60 con 0x7fc0c40cb130 2023-09-27T23:26:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 -- 172.21.15.64:0/2913785867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c40cb130 msgr2=0x7fc0c40cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2913785867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c40cb130 0x7fc0c40cb500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40cc930 tx=0x7fc0b4009250 comp rx=0 tx=0).stop 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 -- 172.21.15.64:0/2913785867 shutdown_connections 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2913785867 >> v2:172.21.15.123:3300/0 conn(0x7fc0c406a6a0 0x7fc0c406aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2913785867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c40cb130 0x7fc0c40cb500 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 -- 172.21.15.64:0/2913785867 >> 172.21.15.64:0/2913785867 conn(0x7fc0c4067bb0 msgr2=0x7fc0c40cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 -- 172.21.15.64:0/2913785867 shutdown_connections 2023-09-27T23:26:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.548+0000 7fc0cc762700 1 -- 172.21.15.64:0/2913785867 wait complete. 2023-09-27T23:26:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 Processor -- start 2023-09-27T23:26:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 -- start start 2023-09-27T23:26:30.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0c406a6a0 0x7fc0c41502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0c40cb130 0x7fc0c4155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 0x7fc0c414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c414fe10 con 0x7fc0c415c220 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c414fae0 con 0x7fc0c406a6a0 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cc762700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40b4380 con 0x7fc0c40cb130 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cacff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 0x7fc0c414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cacff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 0x7fc0c414ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34514/0 (socket says 172.21.15.64:34514) 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0cacff700 1 -- 172.21.15.64:0/2761268863 learned_addr learned my addr 172.21.15.64:0/2761268863 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0ca4fe700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0c406a6a0 0x7fc0c41502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:30.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.552+0000 7fc0c9cfd700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0c40cb130 0x7fc0c4155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:30.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cacff700 1 -- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0c40cb130 msgr2=0x7fc0c4155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:30.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cacff700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0c40cb130 0x7fc0c4155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cacff700 1 -- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0c406a6a0 msgr2=0x7fc0c41502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cacff700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0c406a6a0 0x7fc0c41502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cacff700 1 -- 172.21.15.64:0/2761268863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0009420 con 0x7fc0c415c220 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cacff700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 0x7fc0c414ee00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00c910 tx=0x7fc0bc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc012630 con 0x7fc0c415c220 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc0127b0 con 0x7fc0c415c220 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b40094f0 con 0x7fc0c415c220 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc012b70 con 0x7fc0c415c220 2023-09-27T23:26:30.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c41507e0 con 0x7fc0c415c220 2023-09-27T23:26:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c414ff80 con 0x7fc0c415c220 2023-09-27T23:26:30.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.556+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fc0bc01b400 con 0x7fc0c415c220 2023-09-27T23:26:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.560+0000 7fc0ba7fc700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc0a8060b50 0x7fc0a8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.560+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fc0bc07d760 con 0x7fc0c415c220 2023-09-27T23:26:30.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.560+0000 7fc0ca4fe700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc0a8060b50 0x7fc0a8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.560+0000 7fc0ca4fe700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc0a8060b50 0x7fc0a8063000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40cc930 tx=0x7fc0b4017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:30.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.568+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc0bc04d4e0 con 0x7fc0c415c220 2023-09-27T23:26:30.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.752+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0c40ce360 con 0x7fc0a8060b50 2023-09-27T23:26:30.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.928+0000 7fc0ba7fc700 1 -- 172.21.15.64:0/2761268863 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc0c40ce360 con 0x7fc0a8060b50 2023-09-27T23:26:30.933 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-09-27T23:26:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc0a8060b50 msgr2=0x7fc0a8063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc0a8060b50 0x7fc0a8063000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40cc930 tx=0x7fc0b4017000 comp rx=0 tx=0).stop 2023-09-27T23:26:30.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 msgr2=0x7fc0c414ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 0x7fc0c414ee00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00c910 tx=0x7fc0bc0027a0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 shutdown_connections 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc0a8060b50 0x7fc0a8063000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0c415c220 0x7fc0c414ee00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0c40cb130 0x7fc0c4155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 --2- 172.21.15.64:0/2761268863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0c406a6a0 0x7fc0c41502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 >> 172.21.15.64:0/2761268863 conn(0x7fc0c4067bb0 msgr2=0x7fc0c40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 shutdown_connections 2023-09-27T23:26:30.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:30.932+0000 7fc0cc762700 1 -- 172.21.15.64:0/2761268863 wait complete. 2023-09-27T23:26:30.955 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-09-27T23:26:30.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.955 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_01_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:30.979 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:30.980 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:30.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:30.980 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:31.004 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:31.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.005 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:31.072 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:31.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.073 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:31.095 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:31.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.096 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:31.121 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:31.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.121 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:31.189 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:31.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.189 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:31.213 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:31.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.213 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:31.235 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_23_01_UTC 2023-09-27T23:26:31.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.235 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_23_01_UTC) 2023-09-27T23:26:31.306 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:31.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.306 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:31.334 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:31.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.334 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:31.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:31.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.360 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:31.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.385 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-09-27T23:26:31.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-09-27T23:26:31.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 Processor -- start 2023-09-27T23:26:31.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 -- start start 2023-09-27T23:26:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f757c0cb500 0x7f757c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0b0ac0 con 0x7f757c15c760 2023-09-27T23:26:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c06b210 con 0x7f757c0cb130 2023-09-27T23:26:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c15cbe0 con 0x7f757c0cb500 2023-09-27T23:26:31.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f758144e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f757c0cb500 0x7f757c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7582450700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f757c15c760 0x7f757c0b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52784/0 (socket says 172.21.15.64:52784) 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7582450700 1 -- 172.21.15.64:0/3034115493 learned_addr learned my addr 172.21.15.64:0/3034115493 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f758144e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f757c0cb500 0x7f757c15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34526/0 (socket says 172.21.15.64:34526) 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f758144e700 1 -- 172.21.15.64:0/3034115493 >> v1:172.21.15.123:6789/0 conn(0x7f757c0cb130 legacy=0x7f757c06a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f758144e700 1 -- 172.21.15.64:0/3034115493 >> v1:172.21.15.64:6789/0 conn(0x7f757c15c760 legacy=0x7f757c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f758144e700 1 -- 172.21.15.64:0/3034115493 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0ca810 con 0x7f757c0cb500 2023-09-27T23:26:31.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f758144e700 1 --2- 172.21.15.64:0/3034115493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb500 0x7f757c15c220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098e0 tx=0x7f756c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512691eab1d2871f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7580c4d700 1 -- 172.21.15.64:0/3034115493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c009e80 con 0x7f757c0cb500 2023-09-27T23:26:31.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7580c4d700 1 -- 172.21.15.64:0/3034115493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00b860 con 0x7f757c0cb500 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7580c4d700 1 -- 172.21.15.64:0/3034115493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00bbe0 con 0x7f757c0cb500 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 -- 172.21.15.64:0/3034115493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb500 msgr2=0x7f757c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.520+0000 7f7583eb3700 1 --2- 172.21.15.64:0/3034115493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb500 0x7f757c15c220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098e0 tx=0x7f756c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- 172.21.15.64:0/3034115493 shutdown_connections 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 --2- 172.21.15.64:0/3034115493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb500 0x7f757c15c220 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- 172.21.15.64:0/3034115493 >> 172.21.15.64:0/3034115493 conn(0x7f757c067bb0 msgr2=0x7f757c0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- 172.21.15.64:0/3034115493 shutdown_connections 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- 172.21.15.64:0/3034115493 wait complete. 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 Processor -- start 2023-09-27T23:26:31.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- start start 2023-09-27T23:26:31.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 0x7f757c14e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:31.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f757c0cb500 0x7f757c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:31.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f757c15c760 0x7f757c14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c16ae80 con 0x7f757c0cb130 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c16b650 con 0x7f757c15c760 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7583eb3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c157300 con 0x7f757c0cb500 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7581c4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 0x7f757c14e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7581c4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 0x7f757c14e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34528/0 (socket says 172.21.15.64:34528) 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.524+0000 7f7581c4f700 1 -- 172.21.15.64:0/1235923933 learned_addr learned my addr 172.21.15.64:0/1235923933 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f758144e700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f757c0cb500 0x7f757c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7582450700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f757c15c760 0x7f757c14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7581c4f700 1 -- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f757c0cb500 msgr2=0x7f757c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7581c4f700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f757c0cb500 0x7f757c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7581c4f700 1 -- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f757c15c760 msgr2=0x7f757c14e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7581c4f700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f757c15c760 0x7f757c14e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7581c4f700 1 -- 172.21.15.64:0/1235923933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009420 con 0x7f757c0cb130 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f758144e700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f757c0cb500 0x7f757c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7582450700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f757c15c760 0x7f757c14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7581c4f700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 0x7f757c14e820 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f757800c590 tx=0x7f757800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578011590 con 0x7f757c0cb130 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75780116f0 con 0x7f757c0cb130 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c14e610 con 0x7f757c0cb130 2023-09-27T23:26:31.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578011ab0 con 0x7f757c0cb130 2023-09-27T23:26:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c16b370 con 0x7f757c0cb130 2023-09-27T23:26:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c071120 con 0x7f757c0cb130 2023-09-27T23:26:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.528+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f757801b3b0 con 0x7f757c0cb130 2023-09-27T23:26:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.532+0000 7f7571ffb700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7560060b50 0x7f7560063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.532+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f757807ca30 con 0x7f757c0cb130 2023-09-27T23:26:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.532+0000 7f758144e700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7560060b50 0x7f7560063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.532+0000 7f758144e700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7560060b50 0x7f7560063000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f756c0093f0 tx=0x7f756c017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:31.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.536+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f757804c7b0 con 0x7f757c0cb130 2023-09-27T23:26:31.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.716+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f757c157440 con 0x7f7560060b50 2023-09-27T23:26:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.896+0000 7f7571ffb700 1 -- 172.21.15.64:0/1235923933 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f756c000ce0 con 0x7f7560060b50 2023-09-27T23:26:31.901 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-09-27T23:26:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7560060b50 msgr2=0x7f7560063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7560060b50 0x7f7560063000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f756c0093f0 tx=0x7f756c017000 comp rx=0 tx=0).stop 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 msgr2=0x7f757c14e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 0x7f757c14e820 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f757800c590 tx=0x7f757800c950 comp rx=0 tx=0).stop 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 shutdown_connections 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7560060b50 0x7f7560063000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f757c15c760 0x7f757c14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f757c0cb500 0x7f757c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 --2- 172.21.15.64:0/1235923933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f757c0cb130 0x7f757c14e820 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 >> 172.21.15.64:0/1235923933 conn(0x7f757c067bb0 msgr2=0x7f757c0cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 shutdown_connections 2023-09-27T23:26:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:31.900+0000 7f7583eb3700 1 -- 172.21.15.64:0/1235923933 wait complete. 2023-09-27T23:26:31.922 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-09-27T23:26:31.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.922 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-09-27T23:26:31.947 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_01_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_01_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_01_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:31.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:31.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.949 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:31.973 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:31.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.974 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:31.998 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_18_01_UTC 2023-09-27T23:26:31.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:31.998 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_18_01_UTC) 2023-09-27T23:26:32.021 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:32.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.021 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:32.046 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:32.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.046 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:32.114 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:32.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.114 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:32.137 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:32.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.137 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:32.157 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_23_01_UTC 2023-09-27T23:26:32.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.157 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_23_01_UTC) 2023-09-27T23:26:32.182 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_24_01_UTC 2023-09-27T23:26:32.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.182 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_24_01_UTC) 2023-09-27T23:26:32.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:32.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.249 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:32.273 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:32.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.274 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:32.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.299 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-09-27T23:26:32.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-09-27T23:26:32.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 Processor -- start 2023-09-27T23:26:32.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 -- start start 2023-09-27T23:26:32.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb4c80cb130 0x7fb4c80cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:32.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb4c815c930 0x7fb4c806aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c80b0ac0 con 0x7fb4c806a6a0 2023-09-27T23:26:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c806b030 con 0x7fb4c80cb130 2023-09-27T23:26:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4cd244700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c806b210 con 0x7fb4c815c930 2023-09-27T23:26:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb4c80cb130 0x7fb4c80cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb4c806a6a0 0x7fb4c815c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52788/0 (socket says 172.21.15.64:52788) 2023-09-27T23:26:32.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c659c700 1 -- 172.21.15.64:0/2597450408 learned_addr learned my addr 172.21.15.64:0/2597450408 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c759e700 1 --2- 172.21.15.64:0/2597450408 >> v2:172.21.15.123:3300/0 conn(0x7fb4c815c930 0x7fb4c806aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c6d9d700 1 -- 172.21.15.64:0/2597450408 >> v2:172.21.15.123:3300/0 conn(0x7fb4c815c930 msgr2=0x7fb4c806aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c6d9d700 1 --2- 172.21.15.64:0/2597450408 >> v2:172.21.15.123:3300/0 conn(0x7fb4c815c930 0x7fb4c806aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c6d9d700 1 -- 172.21.15.64:0/2597450408 >> v1:172.21.15.64:6789/0 conn(0x7fb4c806a6a0 legacy=0x7fb4c815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c5d9b700 1 -- 172.21.15.64:0/2597450408 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236438449 0 0) 0x7fb4c80b0ac0 con 0x7fb4c806a6a0 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c6d9d700 1 -- 172.21.15.64:0/2597450408 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c80ca810 con 0x7fb4c80cb130 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c759e700 1 --2- 172.21.15.64:0/2597450408 >> v2:172.21.15.123:3300/0 conn(0x7fb4c815c930 0x7fb4c806aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c6d9d700 1 --2- 172.21.15.64:0/2597450408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c80cb130 0x7fb4c80cb500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80cfc60 tx=0x7fb4b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e446318dfc4aa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c5d9b700 1 -- 172.21.15.64:0/2597450408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800b6b0 con 0x7fb4c80cb130 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c5d9b700 1 -- 172.21.15.64:0/2597450408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b800b810 con 0x7fb4c80cb130 2023-09-27T23:26:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.428+0000 7fb4c5d9b700 1 -- 172.21.15.64:0/2597450408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800bbb0 con 0x7fb4c80cb130 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 -- 172.21.15.64:0/2597450408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c80cb130 msgr2=0x7fb4c80cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 --2- 172.21.15.64:0/2597450408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c80cb130 0x7fb4c80cb500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80cfc60 tx=0x7fb4b80091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 -- 172.21.15.64:0/2597450408 shutdown_connections 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 --2- 172.21.15.64:0/2597450408 >> v2:172.21.15.123:3300/0 conn(0x7fb4c815c930 0x7fb4c806aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 --2- 172.21.15.64:0/2597450408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c80cb130 0x7fb4c80cb500 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 -- 172.21.15.64:0/2597450408 >> 172.21.15.64:0/2597450408 conn(0x7fb4c8067bb0 msgr2=0x7fb4c80cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:32.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 -- 172.21.15.64:0/2597450408 shutdown_connections 2023-09-27T23:26:32.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 -- 172.21.15.64:0/2597450408 wait complete. 2023-09-27T23:26:32.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 Processor -- start 2023-09-27T23:26:32.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.432+0000 7fb4cd244700 1 -- start start 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c806a6a0 0x7fb4c80b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 0x7fb4c814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4c81597d0 0x7fb4c806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c815b5b0 con 0x7fb4c815c930 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80b3d90 con 0x7fb4c806a6a0 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c815a870 con 0x7fb4c81597d0 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 0x7fb4c814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:32.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 0x7fb4c814e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34538/0 (socket says 172.21.15.64:34538) 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 -- 172.21.15.64:0/640492814 learned_addr learned my addr 172.21.15.64:0/640492814 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c6d9d700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c806a6a0 0x7fb4c80b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c759e700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4c81597d0 0x7fb4c806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 -- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4c81597d0 msgr2=0x7fb4c806cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4c81597d0 0x7fb4c806cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 -- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c806a6a0 msgr2=0x7fb4c80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c806a6a0 0x7fb4c80b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 -- 172.21.15.64:0/640492814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8009420 con 0x7fb4c815c930 2023-09-27T23:26:32.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c759e700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4c81597d0 0x7fb4c806cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:32.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4c659c700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 0x7fb4c814e010 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00c7e0 tx=0x7fb4bc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:32.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc005b30 con 0x7fb4c815c930 2023-09-27T23:26:32.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc005c90 con 0x7fb4c815c930 2023-09-27T23:26:32.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c814e550 con 0x7fb4c815c930 2023-09-27T23:26:32.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0072e0 con 0x7fb4c815c930 2023-09-27T23:26:32.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.436+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c8159f40 con 0x7fb4c815c930 2023-09-27T23:26:32.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.440+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fb4bc0076d0 con 0x7fb4c815c930 2023-09-27T23:26:32.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.440+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c815a9e0 con 0x7fb4c815c930 2023-09-27T23:26:32.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.440+0000 7fb4aeffd700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4b4060b50 0x7fb4b4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:32.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.440+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fb4bc07c8a0 con 0x7fb4c815c930 2023-09-27T23:26:32.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.440+0000 7fb4c6d9d700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4b4060b50 0x7fb4b4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:32.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.440+0000 7fb4c6d9d700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4b4060b50 0x7fb4b4063000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80634c0 tx=0x7fb4b8017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:32.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.448+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb4bc049bb0 con 0x7fb4c815c930 2023-09-27T23:26:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.632+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4c8071120 con 0x7fb4b4060b50 2023-09-27T23:26:32.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.808+0000 7fb4aeffd700 1 -- 172.21.15.64:0/640492814 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb4c8071120 con 0x7fb4b4060b50 2023-09-27T23:26:32.812 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-09-27T23:26:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4b4060b50 msgr2=0x7fb4b4063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4b4060b50 0x7fb4b4063000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80634c0 tx=0x7fb4b8017000 comp rx=0 tx=0).stop 2023-09-27T23:26:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 msgr2=0x7fb4c814e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:32.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 0x7fb4c814e010 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00c7e0 tx=0x7fb4bc007df0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 shutdown_connections 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4b4060b50 0x7fb4b4063000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4c81597d0 0x7fb4c806cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4c815c930 0x7fb4c814e010 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 --2- 172.21.15.64:0/640492814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c806a6a0 0x7fb4c80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 >> 172.21.15.64:0/640492814 conn(0x7fb4c8067bb0 msgr2=0x7fb4c80cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 shutdown_connections 2023-09-27T23:26:32.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:32.812+0000 7fb4cd244700 1 -- 172.21.15.64:0/640492814 wait complete. 2023-09-27T23:26:32.835 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-09-27T23:26:32.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.835 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_01_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:32.862 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:32.863 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:32.863 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:32.863 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:32.863 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:32.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.864 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:32.889 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:32.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.890 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:32.917 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:32.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.917 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:32.940 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:32.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:32.940 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:33.011 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:33.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.011 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:33.037 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_21_01_UTC 2023-09-27T23:26:33.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.038 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_21_01_UTC) 2023-09-27T23:26:33.061 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:33.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.061 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:33.083 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:33.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.084 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:33.170 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:33.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.170 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:33.198 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:33.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.198 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_25_01_UTC) 2023-09-27T23:26:33.224 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:33.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.224 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-09-27-23_26_01_UTC) 2023-09-27T23:26:33.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.249 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-09-27T23:26:33.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-09-27T23:26:33.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 Processor -- start 2023-09-27T23:26:33.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 -- start start 2023-09-27T23:26:33.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b6806b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:33.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b68154f90 0x7f6b6806a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:33.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680af100 con 0x7f6b6806aac0 2023-09-27T23:26:33.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6806af60 con 0x7f6b68154f90 2023-09-27T23:26:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b67fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680aead0 con 0x7f6b68154bc0 2023-09-27T23:26:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b667fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b68154f90 0x7f6b6806a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b667fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b68154f90 0x7f6b6806a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34544/0 (socket says 172.21.15.64:34544) 2023-09-27T23:26:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b667fc700 1 -- 172.21.15.64:0/2343109102 learned_addr learned my addr 172.21.15.64:0/2343109102 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.456+0000 7f6b66ffd700 1 --2- 172.21.15.64:0/2343109102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b6806b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:33.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b667fc700 1 -- 172.21.15.64:0/2343109102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 msgr2=0x7f6b6806b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b667fc700 1 --2- 172.21.15.64:0/2343109102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b6806b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b667fc700 1 -- 172.21.15.64:0/2343109102 >> v1:172.21.15.123:6789/0 conn(0x7f6b6806aac0 legacy=0x7f6b6806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b667fc700 1 -- 172.21.15.64:0/2343109102 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6806bc40 con 0x7f6b68154f90 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b667fc700 1 --2- 172.21.15.64:0/2343109102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68154f90 0x7f6b6806a580 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007480 tx=0x7f6b5800ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a181f437af8bb166 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b65ffb700 1 -- 172.21.15.64:0/2343109102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800f070 con 0x7f6b68154f90 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b65ffb700 1 -- 172.21.15.64:0/2343109102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58008bf0 con 0x7f6b68154f90 2023-09-27T23:26:33.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b65ffb700 1 -- 172.21.15.64:0/2343109102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800a0d0 con 0x7f6b68154f90 2023-09-27T23:26:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 -- 172.21.15.64:0/2343109102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68154f90 msgr2=0x7f6b6806a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 --2- 172.21.15.64:0/2343109102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68154f90 0x7f6b6806a580 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007480 tx=0x7f6b5800ac80 comp rx=0 tx=0).stop 2023-09-27T23:26:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 -- 172.21.15.64:0/2343109102 shutdown_connections 2023-09-27T23:26:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 --2- 172.21.15.64:0/2343109102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68154f90 0x7f6b6806a580 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 --2- 172.21.15.64:0/2343109102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b6806b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 -- 172.21.15.64:0/2343109102 >> 172.21.15.64:0/2343109102 conn(0x7f6b68067bb0 msgr2=0x7f6b680c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:33.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.460+0000 7f6b67fff700 1 -- 172.21.15.64:0/2343109102 shutdown_connections 2023-09-27T23:26:33.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 -- 172.21.15.64:0/2343109102 wait complete. 2023-09-27T23:26:33.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 Processor -- start 2023-09-27T23:26:33.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 -- start start 2023-09-27T23:26:33.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b6806aac0 0x7f6b6814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:33.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b68150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:33.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 0x7f6b68151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:33.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68157640 con 0x7f6b6806aac0 2023-09-27T23:26:33.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68156a30 con 0x7f6b68157ad0 2023-09-27T23:26:33.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b67fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68156d10 con 0x7f6b68154bc0 2023-09-27T23:26:33.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b677fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 0x7f6b68151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:33.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b677fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 0x7f6b68151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:51684/0 (socket says 172.21.15.64:51684) 2023-09-27T23:26:33.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b677fe700 1 -- 172.21.15.64:0/3051988829 learned_addr learned my addr 172.21.15.64:0/3051988829 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:33.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b667fc700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b68150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:33.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.464+0000 7f6b66ffd700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b6806aac0 0x7f6b6814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b667fc700 1 -- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 msgr2=0x7f6b68151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b667fc700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 0x7f6b68151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b667fc700 1 -- 172.21.15.64:0/3051988829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b6806aac0 msgr2=0x7f6b6814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b667fc700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b6806aac0 0x7f6b6814e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b667fc700 1 -- 172.21.15.64:0/3051988829 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58007040 con 0x7f6b68154bc0 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b66ffd700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b6806aac0 0x7f6b6814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b677fe700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 0x7f6b68151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:33.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b667fc700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b68150d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b6806e2a0 tx=0x7f6b58019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:33.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800a810 con 0x7f6b68154bc0 2023-09-27T23:26:33.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b580078a0 con 0x7f6b68154bc0 2023-09-27T23:26:33.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58007c60 con 0x7f6b68154bc0 2023-09-27T23:26:33.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68156ff0 con 0x7f6b68154bc0 2023-09-27T23:26:33.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.468+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b68155d20 con 0x7f6b68154bc0 2023-09-27T23:26:33.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.472+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f6b58008250 con 0x7f6b68154bc0 2023-09-27T23:26:33.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.472+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b681577b0 con 0x7f6b68154bc0 2023-09-27T23:26:33.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.472+0000 7f6b4effd700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b54060c20 0x7f6b540630d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:33.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.472+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7550+0+0 (secure 0 0 0) 0x7f6b5807ecb0 con 0x7f6b68154bc0 2023-09-27T23:26:33.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.472+0000 7f6b66ffd700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b54060c20 0x7f6b540630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:33.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.472+0000 7f6b66ffd700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b54060c20 0x7f6b540630d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b60007900 tx=0x7f6b600097e0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:33.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.480+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6b5804e5b0 con 0x7f6b68154bc0 2023-09-27T23:26:33.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.700+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b6804be40 con 0x7f6b54060c20 2023-09-27T23:26:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.868+0000 7f6b4effd700 1 -- 172.21.15.64:0/3051988829 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6b6804be40 con 0x7f6b54060c20 2023-09-27T23:26:33.874 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b54060c20 msgr2=0x7f6b540630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b54060c20 0x7f6b540630d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b60007900 tx=0x7f6b600097e0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 msgr2=0x7f6b68150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b68150d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b6806e2a0 tx=0x7f6b58019000 comp rx=0 tx=0).stop 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 shutdown_connections 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b54060c20 0x7f6b540630d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b68157ad0 0x7f6b68151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b68154bc0 0x7f6b68150d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 --2- 172.21.15.64:0/3051988829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b6806aac0 0x7f6b6814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 >> 172.21.15.64:0/3051988829 conn(0x7f6b68067bb0 msgr2=0x7f6b68150160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 shutdown_connections 2023-09-27T23:26:33.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:33.872+0000 7f6b67fff700 1 -- 172.21.15.64:0/3051988829 wait complete. 2023-09-27T23:26:33.893 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-09-27T23:26:33.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.893 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-09-27T23:26:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_01_UTC 2023-09-27T23:26:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:33.919 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:33.920 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_01_UTC 2023-09-27T23:26:33.920 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:33.920 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:33.920 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:33.920 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:33.921 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:33.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.921 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:33.946 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:33.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.946 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:33.970 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:33.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:33.970 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:34.038 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_19_01_UTC 2023-09-27T23:26:34.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.038 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_19_01_UTC) 2023-09-27T23:26:34.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:34.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.063 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:34.089 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:34.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.089 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:34.116 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_22_01_UTC 2023-09-27T23:26:34.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.117 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_22_01_UTC) 2023-09-27T23:26:34.143 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:34.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.143 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:34.205 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:34.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.206 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:34.238 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:34.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.238 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:34.268 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:34.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.269 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-09-27-23_26_01_UTC) 2023-09-27T23:26:34.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.290 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-09-27T23:26:34.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-09-27T23:26:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833ffff700 1 Processor -- start 2023-09-27T23:26:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833ffff700 1 -- start start 2023-09-27T23:26:34.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f834006ac90 0x7f8340160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834006be40 con 0x7f83401587c0 2023-09-27T23:26:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8340158f00 con 0x7f83401583f0 2023-09-27T23:26:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83401591e0 con 0x7f834006ac90 2023-09-27T23:26:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f83367fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f83401587c0 0x7f834006a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52792/0 (socket says 172.21.15.64:52792) 2023-09-27T23:26:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f833f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f834006ac90 0x7f8340160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.408+0000 7f83367fc700 1 -- 172.21.15.64:0/1981542203 learned_addr learned my addr 172.21.15.64:0/1981542203 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833e7fc700 1 -- 172.21.15.64:0/1981542203 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698094626 0 0) 0x7f834006be40 con 0x7f83401587c0 2023-09-27T23:26:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833e7fc700 1 -- 172.21.15.64:0/1981542203 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f832c003570 con 0x7f83401587c0 2023-09-27T23:26:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833f7fe700 1 -- 172.21.15.64:0/1981542203 >> v1:172.21.15.123:6789/0 conn(0x7f83401583f0 legacy=0x7f834006b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833f7fe700 1 -- 172.21.15.64:0/1981542203 >> v1:172.21.15.64:6789/0 conn(0x7f83401587c0 legacy=0x7f834006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833f7fe700 1 -- 172.21.15.64:0/1981542203 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83401594c0 con 0x7f834006ac90 2023-09-27T23:26:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833f7fe700 1 --2- 172.21.15.64:0/1981542203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340160100 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f833800d0d0 tx=0x7f833800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd62cd256d8ce081 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833e7fc700 1 -- 172.21.15.64:0/1981542203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338010070 con 0x7f834006ac90 2023-09-27T23:26:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833e7fc700 1 -- 172.21.15.64:0/1981542203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833800db30 con 0x7f834006ac90 2023-09-27T23:26:34.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833e7fc700 1 -- 172.21.15.64:0/1981542203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338016400 con 0x7f834006ac90 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 -- 172.21.15.64:0/1981542203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 msgr2=0x7f8340160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 --2- 172.21.15.64:0/1981542203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340160100 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f833800d0d0 tx=0x7f833800b990 comp rx=0 tx=0).stop 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 -- 172.21.15.64:0/1981542203 shutdown_connections 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 --2- 172.21.15.64:0/1981542203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340160100 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 -- 172.21.15.64:0/1981542203 >> 172.21.15.64:0/1981542203 conn(0x7f8340067bb0 msgr2=0x7f83400c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 -- 172.21.15.64:0/1981542203 shutdown_connections 2023-09-27T23:26:34.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.412+0000 7f833ffff700 1 -- 172.21.15.64:0/1981542203 wait complete. 2023-09-27T23:26:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 Processor -- start 2023-09-27T23:26:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 -- start start 2023-09-27T23:26:34.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340169d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83401583f0 0x7f834016a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83401587c0 0x7f83401567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400af6f0 con 0x7f834006ac90 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8340156d00 con 0x7f83401587c0 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400aeae0 con 0x7f83401583f0 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340169d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340169d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34572/0 (socket says 172.21.15.64:34572) 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 -- 172.21.15.64:0/921726605 learned_addr learned my addr 172.21.15.64:0/921726605 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f83367fc700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83401583f0 0x7f834016a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833f7fe700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83401587c0 0x7f83401567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 -- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83401583f0 msgr2=0x7f834016a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83401583f0 0x7f834016a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 -- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83401587c0 msgr2=0x7f83401567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83401587c0 0x7f83401567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 -- 172.21.15.64:0/921726605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833800bc30 con 0x7f834006ac90 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833effd700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340169d60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f833000c7c0 tx=0x7f8330007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330011590 con 0x7f834006ac90 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330011710 con 0x7f834006ac90 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83400aedc0 con 0x7f834006ac90 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330011ad0 con 0x7f834006ac90 2023-09-27T23:26:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.416+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400af380 con 0x7f834006ac90 2023-09-27T23:26:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.420+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83400c9a20 con 0x7f834006ac90 2023-09-27T23:26:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.420+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f833001b3b0 con 0x7f834006ac90 2023-09-27T23:26:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.420+0000 7f83377fe700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8320060b50 0x7f8320063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.420+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7310+0+0 (secure 0 0 0) 0x7f833007cd40 con 0x7f834006ac90 2023-09-27T23:26:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.420+0000 7f83367fc700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8320060b50 0x7f8320063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.420+0000 7f83367fc700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8320060b50 0x7f8320063000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f834016a2a0 tx=0x7f8328009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.428+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f83400c9a20 con 0x7f834006ac90 2023-09-27T23:26:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.608+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f83400af830 con 0x7f8320060b50 2023-09-27T23:26:34.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.776+0000 7f83377fe700 1 -- 172.21.15.64:0/921726605 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8328005900 con 0x7f8320060b50 2023-09-27T23:26:34.783 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-09-27T23:26:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8320060b50 msgr2=0x7f8320063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8320060b50 0x7f8320063000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f834016a2a0 tx=0x7f8328009040 comp rx=0 tx=0).stop 2023-09-27T23:26:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 msgr2=0x7f8340169d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340169d60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f833000c7c0 tx=0x7f8330007ef0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 shutdown_connections 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8320060b50 0x7f8320063000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83401587c0 0x7f83401567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83401583f0 0x7f834016a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 --2- 172.21.15.64:0/921726605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f834006ac90 0x7f8340169d60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 >> 172.21.15.64:0/921726605 conn(0x7f8340067bb0 msgr2=0x7f83400c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 shutdown_connections 2023-09-27T23:26:34.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:34.784+0000 7f833ffff700 1 -- 172.21.15.64:0/921726605 wait complete. 2023-09-27T23:26:34.805 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-09-27T23:26:34.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.805 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-09-27T23:26:34.828 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_01_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:34.829 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:34.830 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:34.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.831 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:34.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:34.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.855 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_17_01_UTC) 2023-09-27T23:26:34.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:34.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.880 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:34.952 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:34.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.952 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:34.978 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_20_01_UTC 2023-09-27T23:26:34.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:34.978 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_20_01_UTC) 2023-09-27T23:26:35.003 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:35.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.003 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:35.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:35.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.027 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:35.098 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:35.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.098 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:35.119 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:35.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.120 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:35.147 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:35.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.147 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:35.216 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:35.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.216 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:35.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.248 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-09-27T23:26:35.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-09-27T23:26:35.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bd866700 1 Processor -- start 2023-09-27T23:26:35.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bd866700 1 -- start start 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bd866700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62b80af190 0x7f62b806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bd866700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b80cb3f0 con 0x7f62b80aedc0 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bd866700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b806d220 con 0x7f62b80cac50 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bd866700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b80cc140 con 0x7f62b80af190 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bc864700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f62b80aedc0 0x7f62b806d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55286/0 (socket says 172.21.15.64:55286) 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62bc864700 1 -- 172.21.15.64:0/1382889037 learned_addr learned my addr 172.21.15.64:0/1382889037 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.384+0000 7f62b7fff700 1 --2- 172.21.15.64:0/1382889037 >> v2:172.21.15.64:3300/0 conn(0x7f62b80af190 0x7f62b806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b77fe700 1 -- 172.21.15.64:0/1382889037 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756537541 0 0) 0x7f62b80cb3f0 con 0x7f62b80aedc0 2023-09-27T23:26:35.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b77fe700 1 -- 172.21.15.64:0/1382889037 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a0003570 con 0x7f62b80aedc0 2023-09-27T23:26:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b7fff700 1 -- 172.21.15.64:0/1382889037 >> v1:172.21.15.123:6789/0 conn(0x7f62b80cac50 legacy=0x7f62b80af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b7fff700 1 -- 172.21.15.64:0/1382889037 >> v1:172.21.15.64:6789/0 conn(0x7f62b80aedc0 legacy=0x7f62b806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b7fff700 1 -- 172.21.15.64:0/1382889037 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b806c8f0 con 0x7f62b80af190 2023-09-27T23:26:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b7fff700 1 --2- 172.21.15.64:0/1382889037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80af190 0x7f62b806af50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0098e0 tx=0x7f62ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f008900bc577f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b77fe700 1 -- 172.21.15.64:0/1382889037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00b640 con 0x7f62b80af190 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b77fe700 1 -- 172.21.15.64:0/1382889037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00b7a0 con 0x7f62b80af190 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62b77fe700 1 -- 172.21.15.64:0/1382889037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00bb60 con 0x7f62b80af190 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 -- 172.21.15.64:0/1382889037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80af190 msgr2=0x7f62b806af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 --2- 172.21.15.64:0/1382889037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80af190 0x7f62b806af50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0098e0 tx=0x7f62ac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 -- 172.21.15.64:0/1382889037 shutdown_connections 2023-09-27T23:26:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 --2- 172.21.15.64:0/1382889037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80af190 0x7f62b806af50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 -- 172.21.15.64:0/1382889037 >> 172.21.15.64:0/1382889037 conn(0x7f62b80cc760 msgr2=0x7f62b80ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 -- 172.21.15.64:0/1382889037 shutdown_connections 2023-09-27T23:26:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 -- 172.21.15.64:0/1382889037 wait complete. 2023-09-27T23:26:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 Processor -- start 2023-09-27T23:26:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.388+0000 7f62bd866700 1 -- start start 2023-09-27T23:26:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62b80aedc0 0x7f62b8159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62b80af190 0x7f62b80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:35.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 0x7f62b80c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80af610 con 0x7f62b80cac50 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80c4ca0 con 0x7f62b80af190 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80b3d40 con 0x7f62b80aedc0 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 0x7f62b80c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 0x7f62b80c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32898/0 (socket says 172.21.15.64:32898) 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 -- 172.21.15.64:0/1206706130 learned_addr learned my addr 172.21.15.64:0/1206706130 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62b7fff700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62b80af190 0x7f62b80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bc864700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62b80aedc0 0x7f62b8159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 -- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62b80aedc0 msgr2=0x7f62b8159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62b80aedc0 0x7f62b8159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 -- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62b80af190 msgr2=0x7f62b80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62b80af190 0x7f62b80c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 -- 172.21.15.64:0/1206706130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac009420 con 0x7f62b80cac50 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62b7fff700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62b80af190 0x7f62b80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd065700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 0x7f62b80c7240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c560 tx=0x7f62a8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8011590 con 0x7f62b80cac50 2023-09-27T23:26:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a80116f0 con 0x7f62b80cac50 2023-09-27T23:26:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b80c78e0 con 0x7f62b80cac50 2023-09-27T23:26:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8011ab0 con 0x7f62b80cac50 2023-09-27T23:26:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b80c7ea0 con 0x7f62b80cac50 2023-09-27T23:26:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.392+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b80c4a70 con 0x7f62b80cac50 2023-09-27T23:26:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.396+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f62a801b3b0 con 0x7f62b80cac50 2023-09-27T23:26:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.396+0000 7f62b4ff9700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a4060b50 0x7f62a4063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.396+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7454+0+0 (secure 0 0 0) 0x7f62a807cb90 con 0x7f62b80cac50 2023-09-27T23:26:35.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.396+0000 7f62bc864700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a4060b50 0x7f62a4063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:35.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.396+0000 7f62bc864700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a4060b50 0x7f62a4063000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005c50 tx=0x7f62b0009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:35.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.404+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f62b80c4a70 con 0x7f62b80cac50 2023-09-27T23:26:35.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.588+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b804bec0 con 0x7f62a4060b50 2023-09-27T23:26:35.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.768+0000 7f62b4ff9700 1 -- 172.21.15.64:0/1206706130 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f62b804bec0 con 0x7f62a4060b50 2023-09-27T23:26:35.776 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-09-27T23:26:35.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a4060b50 msgr2=0x7f62a4063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a4060b50 0x7f62a4063000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005c50 tx=0x7f62b0009040 comp rx=0 tx=0).stop 2023-09-27T23:26:35.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 msgr2=0x7f62b80c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:35.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 0x7f62b80c7240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c560 tx=0x7f62a8007ef0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 shutdown_connections 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a4060b50 0x7f62a4063000 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b80cac50 0x7f62b80c7240 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62b80af190 0x7f62b80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 --2- 172.21.15.64:0/1206706130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62b80aedc0 0x7f62b8159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 >> 172.21.15.64:0/1206706130 conn(0x7f62b80cc760 msgr2=0x7f62b815e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 shutdown_connections 2023-09-27T23:26:35.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:35.776+0000 7f62bd866700 1 -- 172.21.15.64:0/1206706130 wait complete. 2023-09-27T23:26:35.796 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-09-27T23:26:35.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.797 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-09-27T23:26:35.824 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_01_UTC 2023-09-27T23:26:35.824 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:35.824 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:35.824 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:35.824 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:35.825 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:35.825 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:35.825 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:35.825 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:35.825 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:35.825 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:35.826 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_16_01_UTC 2023-09-27T23:26:35.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.826 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_16_01_UTC) 2023-09-27T23:26:35.851 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:35.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.851 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:35.875 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:35.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.875 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:35.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:35.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.943 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:35.967 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:35.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.967 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:35.991 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:35.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:35.992 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:36.061 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:36.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.062 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:36.083 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:36.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.084 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:36.109 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:36.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.109 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:36.132 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:36.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.133 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:36.210 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:36.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.211 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:36.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.244 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-09-27T23:26:36.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-09-27T23:26:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 Processor -- start 2023-09-27T23:26:36.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 -- start start 2023-09-27T23:26:36.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f27d4158ed0 0x7f27d40ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:36.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40acef0 0x7f27d406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:36.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d40c85d0 con 0x7f27d406cd00 2023-09-27T23:26:36.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d40ae290 con 0x7f27d4158ed0 2023-09-27T23:26:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27da0cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40ad680 con 0x7f27d40acef0 2023-09-27T23:26:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27d98cb700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f27d406cd00 0x7f27d40c6730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55296/0 (socket says 172.21.15.64:55296) 2023-09-27T23:26:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27d98cb700 1 -- 172.21.15.64:0/1542450036 learned_addr learned my addr 172.21.15.64:0/1542450036 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27d90ca700 1 --2- 172.21.15.64:0/1542450036 >> v2:172.21.15.123:3300/0 conn(0x7f27d4158ed0 0x7f27d40ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:36.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27d88c9700 1 --2- 172.21.15.64:0/1542450036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40acef0 0x7f27d406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:36.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27cbfff700 1 -- 172.21.15.64:0/1542450036 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145279510 0 0) 0x7f27d40c85d0 con 0x7f27d406cd00 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.476+0000 7f27cbfff700 1 -- 172.21.15.64:0/1542450036 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c0003570 con 0x7f27d406cd00 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27d88c9700 1 -- 172.21.15.64:0/1542450036 >> v2:172.21.15.123:3300/0 conn(0x7f27d4158ed0 msgr2=0x7f27d40ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27d88c9700 1 --2- 172.21.15.64:0/1542450036 >> v2:172.21.15.123:3300/0 conn(0x7f27d4158ed0 0x7f27d40ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27d88c9700 1 -- 172.21.15.64:0/1542450036 >> v1:172.21.15.64:6789/0 conn(0x7f27d406cd00 legacy=0x7f27d40c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27d88c9700 1 -- 172.21.15.64:0/1542450036 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40ad960 con 0x7f27d40acef0 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27d90ca700 1 --2- 172.21.15.64:0/1542450036 >> v2:172.21.15.123:3300/0 conn(0x7f27d4158ed0 0x7f27d40ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:36.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27d88c9700 1 --2- 172.21.15.64:0/1542450036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40acef0 0x7f27d406c7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27c40098e0 tx=0x7f27c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96dae3181d7d5338 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27cbfff700 1 -- 172.21.15.64:0/1542450036 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4011630 con 0x7f27d40acef0 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27cbfff700 1 -- 172.21.15.64:0/1542450036 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4011790 con 0x7f27d40acef0 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27cbfff700 1 -- 172.21.15.64:0/1542450036 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4011b30 con 0x7f27d40acef0 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 -- 172.21.15.64:0/1542450036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40acef0 msgr2=0x7f27d406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 --2- 172.21.15.64:0/1542450036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40acef0 0x7f27d406c7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27c40098e0 tx=0x7f27c40091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 -- 172.21.15.64:0/1542450036 shutdown_connections 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 --2- 172.21.15.64:0/1542450036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d40acef0 0x7f27d406c7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 --2- 172.21.15.64:0/1542450036 >> v2:172.21.15.123:3300/0 conn(0x7f27d4158ed0 0x7f27d40ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 -- 172.21.15.64:0/1542450036 >> 172.21.15.64:0/1542450036 conn(0x7f27d40c8c00 msgr2=0x7f27d40c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 -- 172.21.15.64:0/1542450036 shutdown_connections 2023-09-27T23:26:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.480+0000 7f27da0cc700 1 -- 172.21.15.64:0/1542450036 wait complete. 2023-09-27T23:26:36.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 Processor -- start 2023-09-27T23:26:36.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 -- start start 2023-09-27T23:26:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d406cd00 0x7f27d40af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40acef0 0x7f27d40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 0x7f27d414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40afe40 con 0x7f27d4158ed0 2023-09-27T23:26:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40cb9d0 con 0x7f27d40acef0 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40a5c70 con 0x7f27d406cd00 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 0x7f27d414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d88c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40acef0 0x7f27d40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d90ca700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d406cd00 0x7f27d40af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 0x7f27d414d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32900/0 (socket says 172.21.15.64:32900) 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d88c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40acef0 0x7f27d40a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:53090/0 (socket says 172.21.15.64:53090) 2023-09-27T23:26:36.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 -- 172.21.15.64:0/2335261268 learned_addr learned my addr 172.21.15.64:0/2335261268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:36.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 -- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d406cd00 msgr2=0x7f27d40af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d406cd00 0x7f27d40af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 -- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40acef0 msgr2=0x7f27d40a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40acef0 0x7f27d40a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 -- 172.21.15.64:0/2335261268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0009420 con 0x7f27d4158ed0 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27d98cb700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 0x7f27d414d4f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f27cc001460 tx=0x7f27cc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc00a890 con 0x7f27d4158ed0 2023-09-27T23:26:36.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc00a9f0 con 0x7f27d4158ed0 2023-09-27T23:26:36.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.484+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c4009420 con 0x7f27d4158ed0 2023-09-27T23:26:36.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc00ad70 con 0x7f27d4158ed0 2023-09-27T23:26:36.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d40b0610 con 0x7f27d4158ed0 2023-09-27T23:26:36.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d404be40 con 0x7f27d4158ed0 2023-09-27T23:26:36.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f27cc007300 con 0x7f27d4158ed0 2023-09-27T23:26:36.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27c97fa700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27b8060b50 0x7f27b8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:36.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f27cc07db50 con 0x7f27d4158ed0 2023-09-27T23:26:36.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.488+0000 7f27d90ca700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27b8060b50 0x7f27b8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:36.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.492+0000 7f27d90ca700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27b8060b50 0x7f27b8063000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009ea0 tx=0x7f27d00091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:36.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.500+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f27cc04d5c0 con 0x7f27d4158ed0 2023-09-27T23:26:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.680+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f27d40adc40 con 0x7f27b8060b50 2023-09-27T23:26:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.860+0000 7f27c97fa700 1 -- 172.21.15.64:0/2335261268 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f27d40adc40 con 0x7f27b8060b50 2023-09-27T23:26:36.866 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27b8060b50 msgr2=0x7f27b8063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27b8060b50 0x7f27b8063000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009ea0 tx=0x7f27d00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 msgr2=0x7f27d414d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 0x7f27d414d4f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f27cc001460 tx=0x7f27cc001ae0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 shutdown_connections 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f27b8060b50 0x7f27b8063000 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27d4158ed0 0x7f27d414d4f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d40acef0 0x7f27d40a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 --2- 172.21.15.64:0/2335261268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27d406cd00 0x7f27d40af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 >> 172.21.15.64:0/2335261268 conn(0x7f27d40c8c00 msgr2=0x7f27d40c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 shutdown_connections 2023-09-27T23:26:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:36.864+0000 7f27da0cc700 1 -- 172.21.15.64:0/2335261268 wait complete. 2023-09-27T23:26:36.888 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-09-27T23:26:36.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.888 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-09-27T23:26:36.914 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:36.914 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:36.914 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_01_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:36.915 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:36.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:36.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.916 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:36.943 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:36.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:36.944 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_17_01_UTC) 2023-09-27T23:26:37.018 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:37.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.018 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:37.043 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:37.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.044 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:37.068 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:37.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.069 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:37.137 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:37.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.137 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:37.161 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:37.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.161 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:37.187 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:37.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.187 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:37.256 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_24_01_UTC 2023-09-27T23:26:37.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.256 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_24_01_UTC) 2023-09-27T23:26:37.293 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_25_01_UTC 2023-09-27T23:26:37.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.293 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_25_01_UTC) 2023-09-27T23:26:37.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_26_01_UTC 2023-09-27T23:26:37.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.316 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-09-27-23_26_01_UTC) 2023-09-27T23:26:37.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.340 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-09-27T23:26:37.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-09-27T23:26:37.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 Processor -- start 2023-09-27T23:26:37.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 -- start start 2023-09-27T23:26:37.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9780ae270 0x7fb9780ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:37.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9780aef60 0x7fb9780b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97806d7b0 con 0x7fb978156720 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9780b2060 con 0x7fb9780aef60 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb977fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97806cba0 con 0x7fb9780ae270 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb9767fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9780aef60 0x7fb9780b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb9767fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9780aef60 0x7fb9780b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32916/0 (socket says 172.21.15.64:32916) 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb9767fc700 1 -- 172.21.15.64:0/2964084481 learned_addr learned my addr 172.21.15.64:0/2964084481 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.500+0000 7fb976ffd700 1 --2- 172.21.15.64:0/2964084481 >> v2:172.21.15.123:3300/0 conn(0x7fb9780ae270 0x7fb9780ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb9767fc700 1 -- 172.21.15.64:0/2964084481 >> v2:172.21.15.123:3300/0 conn(0x7fb9780ae270 msgr2=0x7fb9780ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb9767fc700 1 --2- 172.21.15.64:0/2964084481 >> v2:172.21.15.123:3300/0 conn(0x7fb9780ae270 0x7fb9780ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb9767fc700 1 -- 172.21.15.64:0/2964084481 >> v1:172.21.15.123:6789/0 conn(0x7fb978156720 legacy=0x7fb978156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb9767fc700 1 -- 172.21.15.64:0/2964084481 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97806ce80 con 0x7fb9780aef60 2023-09-27T23:26:37.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb976ffd700 1 --2- 172.21.15.64:0/2964084481 >> v2:172.21.15.123:3300/0 conn(0x7fb9780ae270 0x7fb9780ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb9767fc700 1 --2- 172.21.15.64:0/2964084481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9780aef60 0x7fb9780b1b20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0099b0 tx=0x7fb96c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fdd6a8ca3cdab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb975ffb700 1 -- 172.21.15.64:0/2964084481 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00b820 con 0x7fb9780aef60 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb975ffb700 1 -- 172.21.15.64:0/2964084481 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c00b980 con 0x7fb9780aef60 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb975ffb700 1 -- 172.21.15.64:0/2964084481 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00bd20 con 0x7fb9780aef60 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 -- 172.21.15.64:0/2964084481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9780aef60 msgr2=0x7fb9780b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 --2- 172.21.15.64:0/2964084481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9780aef60 0x7fb9780b1b20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0099b0 tx=0x7fb96c009250 comp rx=0 tx=0).stop 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 -- 172.21.15.64:0/2964084481 shutdown_connections 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 --2- 172.21.15.64:0/2964084481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9780aef60 0x7fb9780b1b20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 --2- 172.21.15.64:0/2964084481 >> v2:172.21.15.123:3300/0 conn(0x7fb9780ae270 0x7fb9780ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 -- 172.21.15.64:0/2964084481 >> 172.21.15.64:0/2964084481 conn(0x7fb9780c8c00 msgr2=0x7fb9780c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:37.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 -- 172.21.15.64:0/2964084481 shutdown_connections 2023-09-27T23:26:37.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 -- 172.21.15.64:0/2964084481 wait complete. 2023-09-27T23:26:37.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 Processor -- start 2023-09-27T23:26:37.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.504+0000 7fb977fff700 1 -- start start 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9780ae270 0x7fb97814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9780aef60 0x7fb9780c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 0x7fb9780c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97816a0a0 con 0x7fb978156720 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97816cb10 con 0x7fb9780aef60 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97816cdf0 con 0x7fb9780ae270 2023-09-27T23:26:37.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 0x7fb9780c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 0x7fb9780c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32932/0 (socket says 172.21.15.64:32932) 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 -- 172.21.15.64:0/2174299014 learned_addr learned my addr 172.21.15.64:0/2174299014 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9767fc700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9780aef60 0x7fb9780c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb976ffd700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9780ae270 0x7fb97814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 -- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9780ae270 msgr2=0x7fb97814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9780ae270 0x7fb97814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 -- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9780aef60 msgr2=0x7fb9780c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9780aef60 0x7fb9780c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 -- 172.21.15.64:0/2174299014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0094f0 con 0x7fb978156720 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9767fc700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9780aef60 0x7fb9780c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:37.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb9777fe700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 0x7fb9780c4ec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c4c0 tx=0x7fb970007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:37.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970011590 con 0x7fb978156720 2023-09-27T23:26:37.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9700116f0 con 0x7fb978156720 2023-09-27T23:26:37.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97816d0d0 con 0x7fb978156720 2023-09-27T23:26:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970011ab0 con 0x7fb978156720 2023-09-27T23:26:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97816d5f0 con 0x7fb978156720 2023-09-27T23:26:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.508+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97816c970 con 0x7fb978156720 2023-09-27T23:26:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.512+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7fb97001b3b0 con 0x7fb978156720 2023-09-27T23:26:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.512+0000 7fb95effd700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb964060b50 0x7fb964063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:37.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.512+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fb97007cbc0 con 0x7fb978156720 2023-09-27T23:26:37.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.512+0000 7fb976ffd700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb964060b50 0x7fb964063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:37.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.512+0000 7fb976ffd700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb964060b50 0x7fb964063000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb968007680 tx=0x7fb9680098f0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:37.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.524+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb97816c970 con 0x7fb978156720 2023-09-27T23:26:37.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.732+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb97816c970 con 0x7fb964060b50 2023-09-27T23:26:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.904+0000 7fb95effd700 1 -- 172.21.15.64:0/2174299014 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb97816c970 con 0x7fb964060b50 2023-09-27T23:26:37.909 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-09-27T23:26:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb964060b50 msgr2=0x7fb964063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb964060b50 0x7fb964063000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb968007680 tx=0x7fb9680098f0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 msgr2=0x7fb9780c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 0x7fb9780c4ec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c4c0 tx=0x7fb970007f00 comp rx=0 tx=0).stop 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 shutdown_connections 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb964060b50 0x7fb964063000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb978156720 0x7fb9780c4ec0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9780aef60 0x7fb9780c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 --2- 172.21.15.64:0/2174299014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9780ae270 0x7fb97814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 >> 172.21.15.64:0/2174299014 conn(0x7fb9780c8c00 msgr2=0x7fb9780b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:37.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 shutdown_connections 2023-09-27T23:26:37.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:37.908+0000 7fb977fff700 1 -- 172.21.15.64:0/2174299014 wait complete. 2023-09-27T23:26:37.929 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-09-27T23:26:37.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.930 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-09-27T23:26:37.955 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_01_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_01_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:37.956 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:37.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_16_01_UTC 2023-09-27T23:26:37.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.957 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_16_01_UTC) 2023-09-27T23:26:37.983 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_17_00_UTC 2023-09-27T23:26:37.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:37.984 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_17_00_UTC) 2023-09-27T23:26:38.009 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_18_01_UTC 2023-09-27T23:26:38.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_18_01_UTC) 2023-09-27T23:26:38.077 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:38.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.078 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:38.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:38.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.104 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:38.130 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:38.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.130 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:38.155 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:38.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.155 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:38.181 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:38.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.181 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:38.260 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:38.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.261 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:38.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:38.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.286 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:38.303 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:38.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.303 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:38.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.320 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-09-27T23:26:38.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-09-27T23:26:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 Processor -- start 2023-09-27T23:26:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 -- start start 2023-09-27T23:26:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98ec158da0 0x7f98ec06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec06df10 0x7f98ec06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98ec0caca0 0x7f98ec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec0ca290 con 0x7f98ec0caca0 2023-09-27T23:26:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec15d720 con 0x7f98ec158da0 2023-09-27T23:26:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f2f10700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec0ca4a0 con 0x7f98ec06df10 2023-09-27T23:26:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98ec0caca0 0x7f98ec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98ec0caca0 0x7f98ec06a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32946/0 (socket says 172.21.15.64:32946) 2023-09-27T23:26:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 -- 172.21.15.64:0/2584584777 learned_addr learned my addr 172.21.15.64:0/2584584777 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f1f0e700 1 --2- 172.21.15.64:0/2584584777 >> v2:172.21.15.123:3300/0 conn(0x7f98ec158da0 0x7f98ec06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f170d700 1 --2- 172.21.15.64:0/2584584777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec06df10 0x7f98ec06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 -- 172.21.15.64:0/2584584777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec06df10 msgr2=0x7f98ec06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 --2- 172.21.15.64:0/2584584777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec06df10 0x7f98ec06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 -- 172.21.15.64:0/2584584777 >> v2:172.21.15.123:3300/0 conn(0x7f98ec158da0 msgr2=0x7f98ec06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 --2- 172.21.15.64:0/2584584777 >> v2:172.21.15.123:3300/0 conn(0x7f98ec158da0 0x7f98ec06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 -- 172.21.15.64:0/2584584777 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec0ca780 con 0x7f98ec0caca0 2023-09-27T23:26:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.500+0000 7f98f270f700 1 --2- 172.21.15.64:0/2584584777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec0caca0 0x7f98ec06a7d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f98ec06a310 tx=0x7f98e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1f8728c34871f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f0f0c700 1 -- 172.21.15.64:0/2584584777 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8009e70 con 0x7f98ec0caca0 2023-09-27T23:26:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f0f0c700 1 -- 172.21.15.64:0/2584584777 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e800b860 con 0x7f98ec0caca0 2023-09-27T23:26:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f0f0c700 1 -- 172.21.15.64:0/2584584777 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e800bc00 con 0x7f98ec0caca0 2023-09-27T23:26:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 -- 172.21.15.64:0/2584584777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec0caca0 msgr2=0x7f98ec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 --2- 172.21.15.64:0/2584584777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec0caca0 0x7f98ec06a7d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f98ec06a310 tx=0x7f98e8009250 comp rx=0 tx=0).stop 2023-09-27T23:26:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 -- 172.21.15.64:0/2584584777 shutdown_connections 2023-09-27T23:26:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 --2- 172.21.15.64:0/2584584777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec0caca0 0x7f98ec06a7d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 --2- 172.21.15.64:0/2584584777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec06df10 0x7f98ec06ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 --2- 172.21.15.64:0/2584584777 >> v2:172.21.15.123:3300/0 conn(0x7f98ec158da0 0x7f98ec06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 -- 172.21.15.64:0/2584584777 >> 172.21.15.64:0/2584584777 conn(0x7f98ec0cc760 msgr2=0x7f98ec0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 -- 172.21.15.64:0/2584584777 shutdown_connections 2023-09-27T23:26:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.504+0000 7f98f2f10700 1 -- 172.21.15.64:0/2584584777 wait complete. 2023-09-27T23:26:38.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 Processor -- start 2023-09-27T23:26:38.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 -- start start 2023-09-27T23:26:38.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98ec06df10 0x7f98ec1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec0caca0 0x7f98ec0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 0x7f98ec0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec0c4aa0 con 0x7f98ec158da0 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec0c3bd0 con 0x7f98ec06df10 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec0c3e20 con 0x7f98ec0caca0 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 0x7f98ec0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f1f0e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98ec06df10 0x7f98ec1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 0x7f98ec0c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32952/0 (socket says 172.21.15.64:32952) 2023-09-27T23:26:38.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 -- 172.21.15.64:0/3981014993 learned_addr learned my addr 172.21.15.64:0/3981014993 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:38.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f170d700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec0caca0 0x7f98ec0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 -- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec0caca0 msgr2=0x7f98ec0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec0caca0 0x7f98ec0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 -- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98ec06df10 msgr2=0x7f98ec1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98ec06df10 0x7f98ec1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 -- 172.21.15.64:0/3981014993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80094f0 con 0x7f98ec158da0 2023-09-27T23:26:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f270f700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 0x7f98ec0c66a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98ec068230 tx=0x7f98e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e800b9d0 con 0x7f98ec158da0 2023-09-27T23:26:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e800bb70 con 0x7f98ec158da0 2023-09-27T23:26:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec0c4100 con 0x7f98ec158da0 2023-09-27T23:26:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8025670 con 0x7f98ec158da0 2023-09-27T23:26:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.508+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec0c46c0 con 0x7f98ec158da0 2023-09-27T23:26:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.512+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f98e8023070 con 0x7f98ec158da0 2023-09-27T23:26:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.512+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec0c4be0 con 0x7f98ec158da0 2023-09-27T23:26:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.512+0000 7f98e1ffb700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f98d0060b80 0x7f98d0063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:38.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.512+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7310+0+0 (secure 0 0 0) 0x7f98e807ebe0 con 0x7f98ec158da0 2023-09-27T23:26:38.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.512+0000 7f98f1f0e700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f98d0060b80 0x7f98d0063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:38.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.512+0000 7f98f1f0e700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f98d0060b80 0x7f98d0063030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009fd0 tx=0x7f98e400b040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.520+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f98e804e710 con 0x7f98ec158da0 2023-09-27T23:26:38.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.744+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f98ec04be40 con 0x7f98d0060b80 2023-09-27T23:26:38.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.912+0000 7f98e1ffb700 1 -- 172.21.15.64:0/3981014993 <== mgr.5452 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f98ec04be40 con 0x7f98d0060b80 2023-09-27T23:26:38.919 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-09-27T23:26:38.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.916+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f98d0060b80 msgr2=0x7f98d0063030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:38.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.916+0000 7f98f2f10700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f98d0060b80 0x7f98d0063030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009fd0 tx=0x7f98e400b040 comp rx=0 tx=0).stop 2023-09-27T23:26:38.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 msgr2=0x7f98ec0c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:38.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 0x7f98ec0c66a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98ec068230 tx=0x7f98e8017000 comp rx=0 tx=0).stop 2023-09-27T23:26:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 shutdown_connections 2023-09-27T23:26:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f98d0060b80 0x7f98d0063030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98ec158da0 0x7f98ec0c66a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98ec0caca0 0x7f98ec0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 --2- 172.21.15.64:0/3981014993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98ec06df10 0x7f98ec1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:38.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 >> 172.21.15.64:0/3981014993 conn(0x7f98ec0cc760 msgr2=0x7f98ec0670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:38.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 shutdown_connections 2023-09-27T23:26:38.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:38.920+0000 7f98f2f10700 1 -- 172.21.15.64:0/3981014993 wait complete. 2023-09-27T23:26:38.940 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-09-27T23:26:38.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.940 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-09-27T23:26:38.967 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:38.967 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:38.967 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:38.967 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:38.967 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:38.968 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:38.968 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:38.968 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:38.968 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:38.968 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:38.968 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:38.969 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_16_00_UTC 2023-09-27T23:26:38.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.970 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_16_00_UTC) 2023-09-27T23:26:38.995 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_17_01_UTC 2023-09-27T23:26:38.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:38.995 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_17_01_UTC) 2023-09-27T23:26:39.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_18_00_UTC 2023-09-27T23:26:39.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.062 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_18_00_UTC) 2023-09-27T23:26:39.088 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_19_00_UTC 2023-09-27T23:26:39.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.089 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_19_00_UTC) 2023-09-27T23:26:39.114 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_20_00_UTC 2023-09-27T23:26:39.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.115 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_20_00_UTC) 2023-09-27T23:26:39.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_21_00_UTC 2023-09-27T23:26:39.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.183 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_21_00_UTC) 2023-09-27T23:26:39.207 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_22_00_UTC 2023-09-27T23:26:39.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.208 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_22_00_UTC) 2023-09-27T23:26:39.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_23_00_UTC 2023-09-27T23:26:39.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.249 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_23_00_UTC) 2023-09-27T23:26:39.310 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_24_00_UTC 2023-09-27T23:26:39.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.310 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_24_00_UTC) 2023-09-27T23:26:39.333 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_25_00_UTC 2023-09-27T23:26:39.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.333 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_25_00_UTC) 2023-09-27T23:26:39.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_26_00_UTC 2023-09-27T23:26:39.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.357 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-09-27-23_26_00_UTC) 2023-09-27T23:26:39.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:26:39.425 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-09-27T23:26:39.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:26:39.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.540+0000 7f3568348700 1 Processor -- start 2023-09-27T23:26:39.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.540+0000 7f3568348700 1 -- start start 2023-09-27T23:26:39.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3560158e20 0x7f35600ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35600cb970 0x7f35600ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35600cbdf0 con 0x7f3560158e20 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35600cc0d0 con 0x7f35600cb970 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35600cc3b0 con 0x7f35600ca2c0 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3560158e20 0x7f35600ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3560158e20 0x7f35600ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32954/0 (socket says 172.21.15.64:32954) 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 -- 172.21.15.64:0/2576283427 learned_addr learned my addr 172.21.15.64:0/2576283427 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35658e3700 1 --2- 172.21.15.64:0/2576283427 >> v2:172.21.15.123:3300/0 conn(0x7f35600cb970 0x7f35600ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 -- 172.21.15.64:0/2576283427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 msgr2=0x7f356006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35668e5700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356006c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 -- 172.21.15.64:0/2576283427 >> v2:172.21.15.123:3300/0 conn(0x7f35600cb970 msgr2=0x7f35600ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 --2- 172.21.15.64:0/2576283427 >> v2:172.21.15.123:3300/0 conn(0x7f35600cb970 0x7f35600ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 -- 172.21.15.64:0/2576283427 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600cc690 con 0x7f3560158e20 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35658e3700 1 --2- 172.21.15.64:0/2576283427 >> v2:172.21.15.123:3300/0 conn(0x7f35600cb970 0x7f35600ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:26:39.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35668e5700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:26:39.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35660e4700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f35600ce5c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3558009f20 tx=0x7f3558009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fc1031d729b070 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:39.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35650e2700 1 -- 172.21.15.64:0/2576283427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800e070 con 0x7f3560158e20 2023-09-27T23:26:39.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35650e2700 1 -- 172.21.15.64:0/2576283427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355800ba40 con 0x7f3560158e20 2023-09-27T23:26:39.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f35650e2700 1 -- 172.21.15.64:0/2576283427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580092e0 con 0x7f3560158e20 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- 172.21.15.64:0/2576283427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 msgr2=0x7f35600ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f35600ce5c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3558009f20 tx=0x7f3558009840 comp rx=0 tx=0).stop 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- 172.21.15.64:0/2576283427 shutdown_connections 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- 172.21.15.64:0/2576283427 >> v2:172.21.15.123:3300/0 conn(0x7f35600cb970 0x7f35600ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 --2- 172.21.15.64:0/2576283427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f35600ce5c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- 172.21.15.64:0/2576283427 >> 172.21.15.64:0/2576283427 conn(0x7f3560067bb0 msgr2=0x7f35600d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- 172.21.15.64:0/2576283427 shutdown_connections 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.544+0000 7f3568348700 1 -- 172.21.15.64:0/2576283427 wait complete. 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 Processor -- start 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 -- start start 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f3560159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356014fb70 0x7f356015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356015da50 con 0x7f3560158e20 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356015f260 con 0x7f356014fb70 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356015e650 con 0x7f35600ca2c0 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35658e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f3560159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356015e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:48332/0 (socket says 172.21.15.64:48332) 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 -- 172.21.15.64:0/1411419464 learned_addr learned my addr 172.21.15.64:0/1411419464 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35668e5700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356014fb70 0x7f356015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 -- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356014fb70 msgr2=0x7f356015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356014fb70 0x7f356015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 -- 172.21.15.64:0/1411419464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 msgr2=0x7f3560159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f3560159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 -- 172.21.15.64:0/1411419464 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3558009ae0 con 0x7f35600ca2c0 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35658e3700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f3560159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35660e4700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356015e070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3560069a80 tx=0x7f3558018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:39.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558008930 con 0x7f35600ca2c0 2023-09-27T23:26:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558008a90 con 0x7f35600ca2c0 2023-09-27T23:26:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356015e930 con 0x7f35600ca2c0 2023-09-27T23:26:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558008e50 con 0x7f35600ca2c0 2023-09-27T23:26:39.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.548+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356015eef0 con 0x7f35600ca2c0 2023-09-27T23:26:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.552+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35600cffb0 con 0x7f35600ca2c0 2023-09-27T23:26:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.552+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 12) v1 ==== 80447+0+0 (secure 0 0 0) 0x7f3558003040 con 0x7f35600ca2c0 2023-09-27T23:26:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.552+0000 7f35567fc700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060b50 0x7f3544063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:39.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.552+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7150+0+0 (secure 0 0 0) 0x7f355807f630 con 0x7f35600ca2c0 2023-09-27T23:26:39.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.552+0000 7f35658e3700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060b50 0x7f3544063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:39.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.552+0000 7f35658e3700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060b50 0x7f3544063000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35600c5110 tx=0x7f3550006f90 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.564+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f355804f040 con 0x7f35600ca2c0 2023-09-27T23:26:39.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:39.768+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f35600cffb0 con 0x7f35600ca2c0 2023-09-27T23:26:40.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.232+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f355804f410 con 0x7f35600ca2c0 2023-09-27T23:26:40.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.236+0000 7f35567fc700 1 -- 172.21.15.64:0/1411419464 <== mon.2 v2:172.21.15.123:3301/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v13) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3558052ad0 con 0x7f35600ca2c0 2023-09-27T23:26:40.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060b50 msgr2=0x7f3544063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060b50 0x7f3544063000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35600c5110 tx=0x7f3550006f90 comp rx=0 tx=0).stop 2023-09-27T23:26:40.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 msgr2=0x7f356015e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356015e070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3560069a80 tx=0x7f3558018000 comp rx=0 tx=0).stop 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 shutdown_connections 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060b50 0x7f3544063000 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35600c5110 tx=0x7f3550006f90 comp rx=0 tx=0).stop 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356014fb70 0x7f356015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3560158e20 0x7f3560159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 --2- 172.21.15.64:0/1411419464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600ca2c0 0x7f356015e070 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 >> 172.21.15.64:0/1411419464 conn(0x7f3560067bb0 msgr2=0x7f35600c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 shutdown_connections 2023-09-27T23:26:40.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.240+0000 7f3568348700 1 -- 172.21.15.64:0/1411419464 wait complete. 2023-09-27T23:26:40.260 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:26:40.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:26:40.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:26:40.286 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:26:40.286 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:26:40.286 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:26:40.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:26:40.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[44038]: fuse finished with error 0 and tester_r 0 2023-09-27T23:26:40.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:26:40.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:26:40.330 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:26:40.376 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:40.372+0000 7f654729b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:26:40.389 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:40.386+0000 7f8007463000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:26:40.462 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:26:40.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:26:40.462 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:26:40.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:26:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:26:40.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:26:40.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:26:40.468 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:26:40.508 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:26:40.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:26:40.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:26:40.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:26:40.509 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:26:40.518 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:26:40.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:26:40.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:26:40.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 Processor -- start 2023-09-27T23:26:40.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- start start 2023-09-27T23:26:40.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806d8b0 0x7f6cf806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80afad0 con 0x7f6cf806ad70 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80afdb0 con 0x7f6cf806e1d0 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf806b750 con 0x7f6cf806d8b0 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf5d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6cf806ad70 0x7f6cf814f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55316/0 (socket says 172.21.15.64:55316) 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf5d9b700 1 -- 172.21.15.64:0/1055563875 learned_addr learned my addr 172.21.15.64:0/1055563875 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf6d9d700 1 --2- 172.21.15.64:0/1055563875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806d8b0 0x7f6cf806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557401501 0 0) 0x7f6cf80afad0 con 0x7f6cf806ad70 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce0003570 con 0x7f6cf806ad70 2023-09-27T23:26:40.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856653246 0 0) 0x7f6cf80afdb0 con 0x7f6cf806e1d0 2023-09-27T23:26:40.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf80afad0 con 0x7f6cf806e1d0 2023-09-27T23:26:40.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf6d9d700 1 -- 172.21.15.64:0/1055563875 >> v1:172.21.15.123:6789/0 conn(0x7f6cf806e1d0 legacy=0x7f6cf8159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf6d9d700 1 -- 172.21.15.64:0/1055563875 >> v1:172.21.15.64:6789/0 conn(0x7f6cf806ad70 legacy=0x7f6cf814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 562386841 0 0) 0x7f6ce0003570 con 0x7f6cf806ad70 2023-09-27T23:26:40.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf6d9d700 1 -- 172.21.15.64:0/1055563875 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80b0090 con 0x7f6cf806d8b0 2023-09-27T23:26:40.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf6d9d700 1 --2- 172.21.15.64:0/1055563875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806d8b0 0x7f6cf806b210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009e90 tx=0x7f6cf0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e5bb909765e2ae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:40.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf000e070 con 0x7f6cf806d8b0 2023-09-27T23:26:40.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf000b870 con 0x7f6cf806d8b0 2023-09-27T23:26:40.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf559a700 1 -- 172.21.15.64:0/1055563875 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec001470 con 0x7f6cf806d8b0 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- 172.21.15.64:0/1055563875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806d8b0 msgr2=0x7f6cf806b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 --2- 172.21.15.64:0/1055563875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806d8b0 0x7f6cf806b210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009e90 tx=0x7f6cf0009780 comp rx=0 tx=0).stop 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- 172.21.15.64:0/1055563875 shutdown_connections 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 --2- 172.21.15.64:0/1055563875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806d8b0 0x7f6cf806b210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- 172.21.15.64:0/1055563875 >> 172.21.15.64:0/1055563875 conn(0x7f6cf8067bb0 msgr2=0x7f6cf80add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- 172.21.15.64:0/1055563875 shutdown_connections 2023-09-27T23:26:40.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.628+0000 7f6cf759e700 1 -- 172.21.15.64:0/1055563875 wait complete. 2023-09-27T23:26:40.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 Processor -- start 2023-09-27T23:26:40.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 -- start start 2023-09-27T23:26:40.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806ad70 0x7f6cf8140d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 0x7f6cf8169400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf806e1d0 0x7f6cf81556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8141270 con 0x7f6cf806d8b0 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8154820 con 0x7f6cf806e1d0 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8153c10 con 0x7f6cf806ad70 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 0x7f6cf8169400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 0x7f6cf8169400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32972/0 (socket says 172.21.15.64:32972) 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 -- 172.21.15.64:0/619201718 learned_addr learned my addr 172.21.15.64:0/619201718 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf6d9d700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf806e1d0 0x7f6cf81556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf659c700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806ad70 0x7f6cf8140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 -- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806ad70 msgr2=0x7f6cf8140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806ad70 0x7f6cf8140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 -- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf806e1d0 msgr2=0x7f6cf81556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf806e1d0 0x7f6cf81556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 -- 172.21.15.64:0/619201718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0009a20 con 0x7f6cf806d8b0 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf659c700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806ad70 0x7f6cf8140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf5d9b700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 0x7f6cf8169400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00cb70 tx=0x7f6cec001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec007e90 con 0x7f6cf806d8b0 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec007360 con 0x7f6cf806d8b0 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec004830 con 0x7f6cf806d8b0 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf8153ef0 con 0x7f6cf806d8b0 2023-09-27T23:26:40.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.632+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf81544b0 con 0x7f6cf806d8b0 2023-09-27T23:26:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.636+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf815d880 con 0x7f6cf806d8b0 2023-09-27T23:26:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.636+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f6cec004ba0 con 0x7f6cf806d8b0 2023-09-27T23:26:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.636+0000 7f6ce67fc700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 0x7f6cd8063080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.636+0000 7f6cf659c700 1 -- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 msgr2=0x7f6cd8063080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.636+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f6cec07cbf0 con 0x7f6cf806d8b0 2023-09-27T23:26:40.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.636+0000 7f6cf659c700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 0x7f6cd8063080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.644+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6cec04abb0 con 0x7f6cf806d8b0 2023-09-27T23:26:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.840+0000 7f6cf659c700 1 -- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 msgr2=0x7f6cd8063080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.840+0000 7f6cf659c700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 0x7f6cd8063080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:26:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.840+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cf804be40 con 0x7f6cf806d8b0 2023-09-27T23:26:40.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.840+0000 7f6ce67fc700 1 -- 172.21.15.64:0/619201718 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f6cec018070 con 0x7f6cf806d8b0 2023-09-27T23:26:40.846 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:40.846 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:14:44.562884+0000","modified":"2023-09-27T23:14:46.768950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/974610030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":974610030},{"type":"v1","addr":"172.21.15.64:6835","nonce":974610030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-09-27T23:26:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 msgr2=0x7f6cd8063080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 0x7f6cd8063080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 msgr2=0x7f6cf8169400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 0x7f6cf8169400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00cb70 tx=0x7f6cec001b50 comp rx=0 tx=0).stop 2023-09-27T23:26:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 shutdown_connections 2023-09-27T23:26:40.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6cd8060bd0 0x7f6cd8063080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf806e1d0 0x7f6cf81556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf806d8b0 0x7f6cf8169400 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 --2- 172.21.15.64:0/619201718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6cf806ad70 0x7f6cf8140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 >> 172.21.15.64:0/619201718 conn(0x7f6cf8067bb0 msgr2=0x7f6cf8159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 shutdown_connections 2023-09-27T23:26:40.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.844+0000 7f6cf759e700 1 -- 172.21.15.64:0/619201718 wait complete. 2023-09-27T23:26:40.851 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2023-09-27T23:26:40.860 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:26:40.861 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:40.854+0000 7f8007463000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:26:40.882 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:40.876+0000 7f654729b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:26:40.909 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:40.906+0000 7f8007463000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:26:40.934 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:40.928+0000 7f654729b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:26:40.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6f59e700 1 Processor -- start 2023-09-27T23:26:40.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6f59e700 1 -- start start 2023-09-27T23:26:40.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700ab910 0x7f0b70159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700c1910 con 0x7f0b700abce0 2023-09-27T23:26:40.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b70159cf0 con 0x7f0b700670e0 2023-09-27T23:26:40.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70159fd0 con 0x7f0b700ab910 2023-09-27T23:26:40.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0b700abce0 0x7f0b700c2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55320/0 (socket says 172.21.15.64:55320) 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6ed9d700 1 -- 172.21.15.64:0/3396890263 learned_addr learned my addr 172.21.15.64:0/3396890263 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6e59c700 1 --2- 172.21.15.64:0/3396890263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700ab910 0x7f0b70159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848589986 0 0) 0x7f0b700c1910 con 0x7f0b700abce0 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b5c003570 con 0x7f0b700abce0 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055419187 0 0) 0x7f0b70159cf0 con 0x7f0b700670e0 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b700c1910 con 0x7f0b700670e0 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6e59c700 1 -- 172.21.15.64:0/3396890263 >> v1:172.21.15.123:6789/0 conn(0x7f0b700670e0 legacy=0x7f0b700cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6e59c700 1 -- 172.21.15.64:0/3396890263 >> v1:172.21.15.64:6789/0 conn(0x7f0b700abce0 legacy=0x7f0b700c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6e59c700 1 -- 172.21.15.64:0/3396890263 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7015a2b0 con 0x7f0b700ab910 2023-09-27T23:26:40.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.956+0000 7f0b6e59c700 1 --2- 172.21.15.64:0/3396890263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700ab910 0x7f0b70159710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009e90 tx=0x7f0b68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3272ca7552c95d83 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:40.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800e070 con 0x7f0b700ab910 2023-09-27T23:26:40.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800bb80 con 0x7f0b700ab910 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6d59a700 1 -- 172.21.15.64:0/3396890263 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68012430 con 0x7f0b700ab910 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 -- 172.21.15.64:0/3396890263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700ab910 msgr2=0x7f0b70159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/3396890263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700ab910 0x7f0b70159710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009e90 tx=0x7f0b68009780 comp rx=0 tx=0).stop 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 -- 172.21.15.64:0/3396890263 shutdown_connections 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/3396890263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700ab910 0x7f0b70159710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 -- 172.21.15.64:0/3396890263 >> 172.21.15.64:0/3396890263 conn(0x7f0b7006c080 msgr2=0x7f0b700ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 -- 172.21.15.64:0/3396890263 shutdown_connections 2023-09-27T23:26:40.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 -- 172.21.15.64:0/3396890263 wait complete. 2023-09-27T23:26:40.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 Processor -- start 2023-09-27T23:26:40.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.960+0000 7f0b6f59e700 1 -- start start 2023-09-27T23:26:40.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b700670e0 0x7f0b70155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 0x7f0b70150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700abce0 0x7f0b7014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7015bad0 con 0x7f0b700ab910 2023-09-27T23:26:40.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c5290 con 0x7f0b700670e0 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70150690 con 0x7f0b700abce0 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 0x7f0b70150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b700670e0 0x7f0b70155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 0x7f0b70150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32986/0 (socket says 172.21.15.64:32986) 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 -- 172.21.15.64:0/743134367 learned_addr learned my addr 172.21.15.64:0/743134367 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6ed9d700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700abce0 0x7f0b7014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 -- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700abce0 msgr2=0x7f0b7014f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700abce0 0x7f0b7014f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 -- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b700670e0 msgr2=0x7f0b70155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b700670e0 0x7f0b70155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 -- 172.21.15.64:0/743134367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68009a20 con 0x7f0b700ab910 2023-09-27T23:26:40.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6dd9b700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 0x7f0b70150000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800c5b0 tx=0x7f0b58007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58010510 con 0x7f0b700ab910 2023-09-27T23:26:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58010670 con 0x7f0b700ab910 2023-09-27T23:26:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b70150970 con 0x7f0b700ab910 2023-09-27T23:26:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58010a30 con 0x7f0b700ab910 2023-09-27T23:26:40.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7014fd50 con 0x7f0b700ab910 2023-09-27T23:26:40.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7004be40 con 0x7f0b700ab910 2023-09-27T23:26:40.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.964+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f0b58025430 con 0x7f0b700ab910 2023-09-27T23:26:40.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.968+0000 7f0b667fc700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b50060b00 0x7f0b50062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:40.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.968+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f0b5807dbc0 con 0x7f0b700ab910 2023-09-27T23:26:40.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.968+0000 7f0b6e59c700 1 -- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b50060b00 msgr2=0x7f0b50062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:40.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.968+0000 7f0b6e59c700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b50060b00 0x7f0b50062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:40.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:40.972+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0b5804d810 con 0x7f0b700ab910 2023-09-27T23:26:41.014 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.010+0000 7f8007463000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:26:41.045 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.040+0000 7f654729b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:26:41.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.152+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b700c53d0 con 0x7f0b700ab910 2023-09-27T23:26:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b667fc700 1 -- 172.21.15.64:0/743134367 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+19454 (secure 0 0 0) 0x7f0b58050e30 con 0x7f0b700ab910 2023-09-27T23:26:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:41.162 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":56,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:41.040566+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:14:42.525604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-09-27T23:14:43.561136+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/1007482582":"2023-09-28T23:15:00.247118+0000","172.21.15.123:6833/33228":"2023-09-28T23:15:00.247118+0000","172.21.15.123:6832/33228":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/3145850360":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/439915678":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/118144335":"2023-09-28T23:15:00.247118+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":2},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":30,"length":1},{"begin":32,"length":1},{"begin":34,"length":1},{"begin":37,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":44,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":53,"length":2},{"begin":56,"length":1},{"begin":60,"length":1},{"begin":62,"length":2},{"begin":67,"length":3},{"begin":71,"length":1},{"begin":73,"length":5},{"begin":79,"length":1},{"begin":82,"length":2},{"begin":86,"length":2},{"begin":89,"length":1},{"begin":91,"length":4},{"begin":96,"length":2},{"begin":100,"length":1},{"begin":104,"length":2},{"begin":107,"length":1},{"begin":109,"length":2},{"begin":162,"length":59}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":53,"length":1},{"begin":67,"length":1},{"begin":74,"length":1},{"begin":89,"length":1},{"begin":96,"length":1},{"begin":216,"length":5}]}],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":2,"length":3},{"begin":7,"length":3},{"begin":12,"length":3},{"begin":16,"length":1},{"begin":18,"length":1},{"begin":21,"length":1},{"begin":23,"length":2},{"begin":26,"length":1},{"begin":28,"length":2},{"begin":31,"length":1},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":38,"length":2},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":45,"length":2},{"begin":48,"length":1},{"begin":50,"length":3},{"begin":55,"length":1},{"begin":57,"length":3},{"begin":61,"length":1},{"begin":64,"length":3},{"begin":70,"length":1},{"begin":72,"length":1},{"begin":78,"length":1},{"begin":80,"length":2},{"begin":84,"length":2},{"begin":88,"length":1},{"begin":90,"length":1},{"begin":95,"length":1},{"begin":98,"length":2},{"begin":101,"length":2},{"begin":106,"length":1},{"begin":108,"length":1},{"begin":111,"length":51}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:26:41.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b50060b00 msgr2=0x7f0b50062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:41.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b50060b00 0x7f0b50062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 msgr2=0x7f0b70150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 0x7f0b70150000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800c5b0 tx=0x7f0b58007e10 comp rx=0 tx=0).stop 2023-09-27T23:26:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 shutdown_connections 2023-09-27T23:26:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b50060b00 0x7f0b50062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b700abce0 0x7f0b7014f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b700ab910 0x7f0b70150000 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 --2- 172.21.15.64:0/743134367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b700670e0 0x7f0b70155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 >> 172.21.15.64:0/743134367 conn(0x7f0b7006c080 msgr2=0x7f0b70156130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 shutdown_connections 2023-09-27T23:26:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.156+0000 7f0b6f59e700 1 -- 172.21.15.64:0/743134367 wait complete. 2023-09-27T23:26:41.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:14:42.525604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:14:43.561136+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:26:41.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:26:41.229 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.226+0000 7f8007463000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:26:41.261 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.256+0000 7f654729b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:26:41.282 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.278+0000 7f8007463000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:26:41.314 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.308+0000 7f654729b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:26:41.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 Processor -- start 2023-09-27T23:26:41.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- start start 2023-09-27T23:26:41.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad80ae040 0x7f3ad81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80c68a0 con 0x7f3ad8155a90 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80c6e60 con 0x7f3ad806db70 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80ae4d0 con 0x7f3ad80ae040 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3ad8155a90 0x7f3ad80ad220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55330/0 (socket says 172.21.15.64:55330) 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad77fe700 1 -- 172.21.15.64:0/1291293694 learned_addr learned my addr 172.21.15.64:0/1291293694 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad7fff700 1 --2- 172.21.15.64:0/1291293694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad80ae040 0x7f3ad81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014912383 0 0) 0x7f3ad80c68a0 con 0x7f3ad8155a90 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac0003570 con 0x7f3ad8155a90 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759367472 0 0) 0x7f3ad80c6e60 con 0x7f3ad806db70 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ad80c68a0 con 0x7f3ad806db70 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3278609939 0 0) 0x7f3ac0003570 con 0x7f3ad8155a90 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1291293694 >> v1:172.21.15.123:6789/0 conn(0x7f3ad806db70 legacy=0x7f3ad80ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1291293694 >> v1:172.21.15.64:6789/0 conn(0x7f3ad8155a90 legacy=0x7f3ad80ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1291293694 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80ae7b0 con 0x7f3ad80ae040 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad7fff700 1 --2- 172.21.15.64:0/1291293694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad80ae040 0x7f3ad81608a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009910 tx=0x7f3acc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b122d20805a007f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00017f0 con 0x7f3ad80ae040 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc011420 con 0x7f3ad80ae040 2023-09-27T23:26:41.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3ad67fc700 1 -- 172.21.15.64:0/1291293694 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0117c0 con 0x7f3ad80ae040 2023-09-27T23:26:41.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- 172.21.15.64:0/1291293694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad80ae040 msgr2=0x7f3ad81608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 --2- 172.21.15.64:0/1291293694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad80ae040 0x7f3ad81608a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009910 tx=0x7f3acc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- 172.21.15.64:0/1291293694 shutdown_connections 2023-09-27T23:26:41.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 --2- 172.21.15.64:0/1291293694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad80ae040 0x7f3ad81608a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- 172.21.15.64:0/1291293694 >> 172.21.15.64:0/1291293694 conn(0x7f3ad80c8cd0 msgr2=0x7f3ad80c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:41.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- 172.21.15.64:0/1291293694 shutdown_connections 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.332+0000 7f3adc805700 1 -- 172.21.15.64:0/1291293694 wait complete. 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 Processor -- start 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- start start 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad806db70 0x7f3ad80c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad80ae040 0x7f3ad80af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 0x7f3ad80af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b0950 con 0x7f3ad8155a90 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c5ad0 con 0x7f3ad80ae040 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80afcd0 con 0x7f3ad806db70 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 0x7f3ad80af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad77fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad806db70 0x7f3ad80c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 0x7f3ad80af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32992/0 (socket says 172.21.15.64:32992) 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1938827147 learned_addr learned my addr 172.21.15.64:0/1938827147 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad6ffd700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad80ae040 0x7f3ad80af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad806db70 msgr2=0x7f3ad80c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad806db70 0x7f3ad80c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad80ae040 msgr2=0x7f3ad80af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad80ae040 0x7f3ad80af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 -- 172.21.15.64:0/1938827147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc009420 con 0x7f3ad8155a90 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3ad7fff700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 0x7f3ad80af630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0030c0 tx=0x7f3acc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc01cd10 con 0x7f3ad8155a90 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc01ce70 con 0x7f3ad8155a90 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad80affb0 con 0x7f3ad8155a90 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc01a660 con 0x7f3ad8155a90 2023-09-27T23:26:41.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad80b0570 con 0x7f3ad8155a90 2023-09-27T23:26:41.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad804be40 con 0x7f3ad8155a90 2023-09-27T23:26:41.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.336+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f3acc01aa50 con 0x7f3ad8155a90 2023-09-27T23:26:41.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.340+0000 7f3abf7fe700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ac4060b00 0x7f3ac4062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.340+0000 7f3ad77fe700 1 -- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ac4060b00 msgr2=0x7f3ac4062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:41.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.340+0000 7f3ad77fe700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ac4060b00 0x7f3ac4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:41.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.340+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7946+0+0 (secure 0 0 0) 0x7f3acc07f600 con 0x7f3ad8155a90 2023-09-27T23:26:41.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.344+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3acc04eea0 con 0x7f3ad8155a90 2023-09-27T23:26:41.460 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.458+0000 7f8007463000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:26:41.478 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.472+0000 7f654729b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:26:41.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.512+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ad80c6af0 con 0x7f3ad8155a90 2023-09-27T23:26:41.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.512+0000 7f3abf7fe700 1 -- 172.21.15.64:0/1938827147 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3acc019ed0 con 0x7f3ad8155a90 2023-09-27T23:26:41.519 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:26:41.519 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.514+0000 7f8007463000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:26:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ac4060b00 msgr2=0x7f3ac4062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ac4060b00 0x7f3ac4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 msgr2=0x7f3ad80af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 0x7f3ad80af630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0030c0 tx=0x7f3acc016000 comp rx=0 tx=0).stop 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 shutdown_connections 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ac4060b00 0x7f3ac4062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ad8155a90 0x7f3ad80af630 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad80ae040 0x7f3ad80af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 --2- 172.21.15.64:0/1938827147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ad806db70 0x7f3ad80c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 >> 172.21.15.64:0/1938827147 conn(0x7f3ad80c8cd0 msgr2=0x7f3ad8068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 shutdown_connections 2023-09-27T23:26:41.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.516+0000 7f3adc805700 1 -- 172.21.15.64:0/1938827147 wait complete. 2023-09-27T23:26:41.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:26:41.538 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.532+0000 7f654729b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:26:41.589 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.586+0000 7f8007463000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:26:41.609 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.604+0000 7f654729b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:26:41.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.640+0000 7f365359e700 1 Processor -- start 2023-09-27T23:26:41.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.640+0000 7f365359e700 1 -- start start 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f365406de00 0x7f36540c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f365406c7c0 0x7f365415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365406cd60 con 0x7f36540c69a0 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365406d040 con 0x7f365406c7c0 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365406d320 con 0x7f365406de00 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f365406c7c0 0x7f365415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f365406c7c0 0x7f365415cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33002/0 (socket says 172.21.15.64:33002) 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 -- 172.21.15.64:0/2127085618 learned_addr learned my addr 172.21.15.64:0/2127085618 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:41.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3651d9b700 1 --2- 172.21.15.64:0/2127085618 >> v2:172.21.15.123:3300/0 conn(0x7f365406de00 0x7f36540c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 -- 172.21.15.64:0/2127085618 >> v2:172.21.15.123:3300/0 conn(0x7f365406de00 msgr2=0x7f36540c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:41.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 --2- 172.21.15.64:0/2127085618 >> v2:172.21.15.123:3300/0 conn(0x7f365406de00 0x7f36540c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 -- 172.21.15.64:0/2127085618 >> v1:172.21.15.64:6789/0 conn(0x7f36540c69a0 legacy=0x7f365406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 -- 172.21.15.64:0/2127085618 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365406d600 con 0x7f365406c7c0 2023-09-27T23:26:41.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f3652d9d700 1 --2- 172.21.15.64:0/2127085618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f365415cba0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f36540635f0 tx=0x7f364c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbd56e9883652c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:41.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365159a700 1 -- 172.21.15.64:0/2127085618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00e070 con 0x7f365406c7c0 2023-09-27T23:26:41.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.644+0000 7f365159a700 1 -- 172.21.15.64:0/2127085618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c00ba50 con 0x7f365406c7c0 2023-09-27T23:26:41.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365159a700 1 -- 172.21.15.64:0/2127085618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00e070 con 0x7f365406c7c0 2023-09-27T23:26:41.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 -- 172.21.15.64:0/2127085618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 msgr2=0x7f365415cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 --2- 172.21.15.64:0/2127085618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f365415cba0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f36540635f0 tx=0x7f364c009840 comp rx=0 tx=0).stop 2023-09-27T23:26:41.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 -- 172.21.15.64:0/2127085618 shutdown_connections 2023-09-27T23:26:41.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 --2- 172.21.15.64:0/2127085618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f365415cba0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 --2- 172.21.15.64:0/2127085618 >> v2:172.21.15.123:3300/0 conn(0x7f365406de00 0x7f36540c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 -- 172.21.15.64:0/2127085618 >> 172.21.15.64:0/2127085618 conn(0x7f3654067bb0 msgr2=0x7f36540cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:41.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 -- 172.21.15.64:0/2127085618 shutdown_connections 2023-09-27T23:26:41.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.648+0000 7f365359e700 1 -- 172.21.15.64:0/2127085618 wait complete. 2023-09-27T23:26:41.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 Processor -- start 2023-09-27T23:26:41.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 -- start start 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f3654157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f365406de00 0x7f365414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36540c69a0 0x7f3654147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36541582f0 con 0x7f365406c7c0 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3654148ed0 con 0x7f365406de00 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36541482c0 con 0x7f36540c69a0 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f3654157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f3654157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33018/0 (socket says 172.21.15.64:33018) 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f3651d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f365406de00 0x7f365414a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 -- 172.21.15.64:0/840163327 learned_addr learned my addr 172.21.15.64:0/840163327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f3652d9d700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36540c69a0 0x7f3654147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 -- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36540c69a0 msgr2=0x7f3654147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36540c69a0 0x7f3654147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 -- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f365406de00 msgr2=0x7f365414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f365406de00 0x7f365414a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 -- 172.21.15.64:0/840163327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c009ae0 con 0x7f365406c7c0 2023-09-27T23:26:41.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365259c700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f3654157db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f364400c590 tx=0x7f3644007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:41.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644011590 con 0x7f365406c7c0 2023-09-27T23:26:41.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.652+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36541485a0 con 0x7f365406c7c0 2023-09-27T23:26:41.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3654148b60 con 0x7f365406c7c0 2023-09-27T23:26:41.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644011730 con 0x7f365406c7c0 2023-09-27T23:26:41.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644011af0 con 0x7f365406c7c0 2023-09-27T23:26:41.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36380052b0 con 0x7f365406c7c0 2023-09-27T23:26:41.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f3644012420 con 0x7f365406c7c0 2023-09-27T23:26:41.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f36427fc700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3634060b00 0x7f3634062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7946+0+0 (secure 0 0 0) 0x7f364407cab0 con 0x7f365406c7c0 2023-09-27T23:26:41.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f3651d9b700 1 -- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3634060b00 msgr2=0x7f3634062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:41.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.656+0000 7f3651d9b700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3634060b00 0x7f3634062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:41.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.664+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3644049bb0 con 0x7f365406c7c0 2023-09-27T23:26:41.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.816+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3638005ae0 con 0x7f365406c7c0 2023-09-27T23:26:41.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.816+0000 7f36427fc700 1 -- 172.21.15.64:0/840163327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3644017070 con 0x7f365406c7c0 2023-09-27T23:26:41.821 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:26:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3634060b00 msgr2=0x7f3634062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3634060b00 0x7f3634062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 msgr2=0x7f3654157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f3654157db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f364400c590 tx=0x7f3644007f30 comp rx=0 tx=0).stop 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 shutdown_connections 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3634060b00 0x7f3634062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36540c69a0 0x7f3654147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f365406de00 0x7f365414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 --2- 172.21.15.64:0/840163327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365406c7c0 0x7f3654157db0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 >> 172.21.15.64:0/840163327 conn(0x7f3654067bb0 msgr2=0x7f365415ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 shutdown_connections 2023-09-27T23:26:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.820+0000 7f365359e700 1 -- 172.21.15.64:0/840163327 wait complete. 2023-09-27T23:26:41.841 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:26:41.898 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:41.894+0000 7f8007463000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:26:41.933 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:41.924+0000 7f654729b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:26:41.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe75264e700 1 Processor -- start 2023-09-27T23:26:41.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe75264e700 1 -- start start 2023-09-27T23:26:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe75264e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe74c06b690 0x7fe74c0afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe75264e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c06c7c0 con 0x7fe74c0b0110 2023-09-27T23:26:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe75264e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c06cd80 con 0x7fe74c06b2c0 2023-09-27T23:26:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe75264e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c0b0940 con 0x7fe74c06b690 2023-09-27T23:26:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe750beb700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe74c0b0110 0x7fe74c158d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55354/0 (socket says 172.21.15.64:55354) 2023-09-27T23:26:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe750beb700 1 -- 172.21.15.64:0/2933215025 learned_addr learned my addr 172.21.15.64:0/2933215025 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:41.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe7437fe700 1 --2- 172.21.15.64:0/2933215025 >> v2:172.21.15.123:3300/0 conn(0x7fe74c06b690 0x7fe74c0afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe74b7fe700 1 -- 172.21.15.64:0/2933215025 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668514132 0 0) 0x7fe74c06c7c0 con 0x7fe74c0b0110 2023-09-27T23:26:41.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe74b7fe700 1 -- 172.21.15.64:0/2933215025 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe738003570 con 0x7fe74c0b0110 2023-09-27T23:26:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe7437fe700 1 -- 172.21.15.64:0/2933215025 >> v1:172.21.15.123:6789/0 conn(0x7fe74c06b2c0 legacy=0x7fe74c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe7437fe700 1 -- 172.21.15.64:0/2933215025 >> v1:172.21.15.64:6789/0 conn(0x7fe74c0b0110 legacy=0x7fe74c158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe74b7fe700 1 -- 172.21.15.64:0/2933215025 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2142918495 0 0) 0x7fe738003570 con 0x7fe74c0b0110 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe7437fe700 1 -- 172.21.15.64:0/2933215025 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c0b0c20 con 0x7fe74c06b690 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe7437fe700 1 --2- 172.21.15.64:0/2933215025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0afbd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe7340098e0 tx=0x7fe7340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c934e4aa3d44f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe74b7fe700 1 -- 172.21.15.64:0/2933215025 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440017f0 con 0x7fe74c06b690 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe74b7fe700 1 -- 172.21.15.64:0/2933215025 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe734011420 con 0x7fe74c06b690 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.940+0000 7fe74b7fe700 1 -- 172.21.15.64:0/2933215025 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340117a0 con 0x7fe74c06b690 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- 172.21.15.64:0/2933215025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 msgr2=0x7fe74c0afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 --2- 172.21.15.64:0/2933215025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0afbd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe7340098e0 tx=0x7fe7340091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- 172.21.15.64:0/2933215025 shutdown_connections 2023-09-27T23:26:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 --2- 172.21.15.64:0/2933215025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0afbd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- 172.21.15.64:0/2933215025 >> 172.21.15.64:0/2933215025 conn(0x7fe74c0cc580 msgr2=0x7fe74c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- 172.21.15.64:0/2933215025 shutdown_connections 2023-09-27T23:26:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- 172.21.15.64:0/2933215025 wait complete. 2023-09-27T23:26:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 Processor -- start 2023-09-27T23:26:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- start start 2023-09-27T23:26:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe74c06b2c0 0x7fe74c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 0x7fe74c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c151e00 con 0x7fe74c0b0110 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c14d540 con 0x7fe74c06b690 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe75264e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c14d820 con 0x7fe74c06b2c0 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe750beb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 0x7fe74c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.944+0000 7fe750beb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 0x7fe74c151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33030/0 (socket says 172.21.15.64:33030) 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 -- 172.21.15.64:0/595807540 learned_addr learned my addr 172.21.15.64:0/595807540 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe7437fe700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe74bfff700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe74c06b2c0 0x7fe74c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 -- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe74c06b2c0 msgr2=0x7fe74c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe74c06b2c0 0x7fe74c0aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 -- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 msgr2=0x7fe74c0aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 -- 172.21.15.64:0/595807540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe734009420 con 0x7fe74c0b0110 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe7437fe700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe750beb700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 0x7fe74c151140 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe74400cb90 tx=0x7fe7440015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744004dc0 con 0x7fe74c0b0110 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400a5d0 con 0x7fe74c0b0110 2023-09-27T23:26:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c14db00 con 0x7fe74c0b0110 2023-09-27T23:26:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400a990 con 0x7fe74c0b0110 2023-09-27T23:26:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c14ce40 con 0x7fe74c0b0110 2023-09-27T23:26:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c14d0c0 con 0x7fe74c0b0110 2023-09-27T23:26:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7fe74400ad40 con 0x7fe74c0b0110 2023-09-27T23:26:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.948+0000 7fe748ff9700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe72c060b00 0x7fe72c062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.952+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7946+0+0 (secure 0 0 0) 0x7fe74407d000 con 0x7fe74c0b0110 2023-09-27T23:26:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.952+0000 7fe74bfff700 1 -- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe72c060b00 msgr2=0x7fe72c062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.952+0000 7fe74bfff700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe72c060b00 0x7fe72c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:41.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:41.960+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe74404c8a0 con 0x7fe74c0b0110 2023-09-27T23:26:42.010 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.006+0000 7f8007463000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:26:42.042 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.036+0000 7f654729b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:26:42.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.124+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe74c071120 con 0x7fe74c0b0110 2023-09-27T23:26:42.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe748ff9700 1 -- 172.21.15.64:0/595807540 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe744018020 con 0x7fe74c0b0110 2023-09-27T23:26:42.132 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:26:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe72c060b00 msgr2=0x7fe72c062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe72c060b00 0x7fe72c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 msgr2=0x7fe74c151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 0x7fe74c151140 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe74400cb90 tx=0x7fe7440015b0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 shutdown_connections 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe72c060b00 0x7fe72c062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe74c0b0110 0x7fe74c151140 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe74c06b690 0x7fe74c0aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 --2- 172.21.15.64:0/595807540 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe74c06b2c0 0x7fe74c0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 >> 172.21.15.64:0/595807540 conn(0x7fe74c0cc580 msgr2=0x7fe74c0a8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 shutdown_connections 2023-09-27T23:26:42.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.128+0000 7fe75264e700 1 -- 172.21.15.64:0/595807540 wait complete. 2023-09-27T23:26:42.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:26:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49da58700 1 Processor -- start 2023-09-27T23:26:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49da58700 1 -- start start 2023-09-27T23:26:42.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49da58700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff498142680 0x7ff4980c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49da58700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980cc7e0 con 0x7ff49806d760 2023-09-27T23:26:42.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49da58700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980cbbf0 con 0x7ff4980c8f40 2023-09-27T23:26:42.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49da58700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980cbeb0 con 0x7ff498142680 2023-09-27T23:26:42.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49ca56700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff498142680 0x7ff4980c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49d257700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff49806d760 0x7ff4980ca7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55370/0 (socket says 172.21.15.64:55370) 2023-09-27T23:26:42.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49d257700 1 -- 172.21.15.64:0/2080950874 learned_addr learned my addr 172.21.15.64:0/2080950874 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:42.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff4977fe700 1 -- 172.21.15.64:0/2080950874 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223296682 0 0) 0x7ff4980cc7e0 con 0x7ff49806d760 2023-09-27T23:26:42.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff4977fe700 1 -- 172.21.15.64:0/2080950874 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff488003570 con 0x7ff49806d760 2023-09-27T23:26:42.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49ca56700 1 -- 172.21.15.64:0/2080950874 >> v1:172.21.15.123:6789/0 conn(0x7ff4980c8f40 legacy=0x7ff49806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49ca56700 1 -- 172.21.15.64:0/2080950874 >> v1:172.21.15.64:6789/0 conn(0x7ff49806d760 legacy=0x7ff4980ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49ca56700 1 -- 172.21.15.64:0/2080950874 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980cc190 con 0x7ff498142680 2023-09-27T23:26:42.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff49ca56700 1 --2- 172.21.15.64:0/2080950874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff498142680 0x7ff4980c8a00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0098e0 tx=0x7ff48c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c523fc1f666e18aa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:42.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff4977fe700 1 -- 172.21.15.64:0/2080950874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00b640 con 0x7ff498142680 2023-09-27T23:26:42.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff4977fe700 1 -- 172.21.15.64:0/2080950874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c00b7a0 con 0x7ff498142680 2023-09-27T23:26:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.248+0000 7ff4977fe700 1 -- 172.21.15.64:0/2080950874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00bb20 con 0x7ff498142680 2023-09-27T23:26:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 -- 172.21.15.64:0/2080950874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff498142680 msgr2=0x7ff4980c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 --2- 172.21.15.64:0/2080950874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff498142680 0x7ff4980c8a00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0098e0 tx=0x7ff48c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 -- 172.21.15.64:0/2080950874 shutdown_connections 2023-09-27T23:26:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 --2- 172.21.15.64:0/2080950874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff498142680 0x7ff4980c8a00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 -- 172.21.15.64:0/2080950874 >> 172.21.15.64:0/2080950874 conn(0x7ff4980ccb30 msgr2=0x7ff4980ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 -- 172.21.15.64:0/2080950874 shutdown_connections 2023-09-27T23:26:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 -- 172.21.15.64:0/2080950874 wait complete. 2023-09-27T23:26:42.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 Processor -- start 2023-09-27T23:26:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.252+0000 7ff49da58700 1 -- start start 2023-09-27T23:26:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff49806d760 0x7ff4980680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 0x7ff4980c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff498142680 0x7ff4980c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980c5180 con 0x7ff4980c8f40 2023-09-27T23:26:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980687b0 con 0x7ff498142680 2023-09-27T23:26:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff498068a90 con 0x7ff49806d760 2023-09-27T23:26:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 0x7ff4980c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 0x7ff4980c18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33048/0 (socket says 172.21.15.64:33048) 2023-09-27T23:26:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 -- 172.21.15.64:0/1292889055 learned_addr learned my addr 172.21.15.64:0/1292889055 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:42.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49d257700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff498142680 0x7ff4980c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49ca56700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff49806d760 0x7ff4980680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 -- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff49806d760 msgr2=0x7ff4980680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff49806d760 0x7ff4980680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 -- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff498142680 msgr2=0x7ff4980c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff498142680 0x7ff4980c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 -- 172.21.15.64:0/1292889055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c009420 con 0x7ff4980c8f40 2023-09-27T23:26:42.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff497fff700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 0x7ff4980c18d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c7c0 tx=0x7ff484007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484011590 con 0x7ff4980c8f40 2023-09-27T23:26:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4840116f0 con 0x7ff4980c8f40 2023-09-27T23:26:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff498068d70 con 0x7ff4980c8f40 2023-09-27T23:26:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484011ab0 con 0x7ff4980c8f40 2023-09-27T23:26:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff498169e00 con 0x7ff4980c8f40 2023-09-27T23:26:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980691f0 con 0x7ff4980c8f40 2023-09-27T23:26:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.256+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7ff48401b3b0 con 0x7ff4980c8f40 2023-09-27T23:26:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.260+0000 7ff494ff9700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff47c060b30 0x7ff47c062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.260+0000 7ff49ca56700 1 -- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff47c060b30 msgr2=0x7ff47c062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.260+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7310+0+0 (secure 0 0 0) 0x7ff48407cab0 con 0x7ff4980c8f40 2023-09-27T23:26:42.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.260+0000 7ff49ca56700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff47c060b30 0x7ff47c062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.264+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff48404c5d0 con 0x7ff4980c8f40 2023-09-27T23:26:42.297 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.294+0000 7f8007463000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:26:42.343 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.336+0000 7f654729b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:26:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.432+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff49804be40 con 0x7ff4980c8f40 2023-09-27T23:26:42.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.432+0000 7ff494ff9700 1 -- 172.21.15.64:0/1292889055 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff484017020 con 0x7ff4980c8f40 2023-09-27T23:26:42.439 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:26:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff47c060b30 msgr2=0x7ff47c062fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff47c060b30 0x7ff47c062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 msgr2=0x7ff4980c18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 0x7ff4980c18d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c7c0 tx=0x7ff484007f50 comp rx=0 tx=0).stop 2023-09-27T23:26:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 shutdown_connections 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff47c060b30 0x7ff47c062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff498142680 0x7ff4980c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4980c8f40 0x7ff4980c18d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 --2- 172.21.15.64:0/1292889055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff49806d760 0x7ff4980680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 >> 172.21.15.64:0/1292889055 conn(0x7ff4980ccb30 msgr2=0x7ff4980c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 shutdown_connections 2023-09-27T23:26:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.436+0000 7ff49da58700 1 -- 172.21.15.64:0/1292889055 wait complete. 2023-09-27T23:26:42.453 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:26:42.464 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.458+0000 7f8007463000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:26:42.514 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.508+0000 7f654729b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:26:42.528 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.522+0000 7f8007463000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:26:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 Processor -- start 2023-09-27T23:26:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 -- start start 2023-09-27T23:26:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2044156770 0x7f2044156b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20440c8b50 0x7f20440c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f204406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204406b730 con 0x7f20440c8b50 2023-09-27T23:26:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20440ca8c0 con 0x7f2044156770 2023-09-27T23:26:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20440cb700 con 0x7f204406ad70 2023-09-27T23:26:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20440c8b50 0x7f20440c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20440c8b50 0x7f20440c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33058/0 (socket says 172.21.15.64:33058) 2023-09-27T23:26:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 -- 172.21.15.64:0/3029809990 learned_addr learned my addr 172.21.15.64:0/3029809990 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2042d9d700 1 --2- 172.21.15.64:0/3029809990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f204406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204259c700 1 --2- 172.21.15.64:0/3029809990 >> v2:172.21.15.123:3300/0 conn(0x7f2044156770 0x7f2044156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 -- 172.21.15.64:0/3029809990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 msgr2=0x7f204406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 --2- 172.21.15.64:0/3029809990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f204406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 -- 172.21.15.64:0/3029809990 >> v2:172.21.15.123:3300/0 conn(0x7f2044156770 msgr2=0x7f2044156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:42.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 --2- 172.21.15.64:0/3029809990 >> v2:172.21.15.123:3300/0 conn(0x7f2044156770 0x7f2044156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 -- 172.21.15.64:0/3029809990 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204406bd20 con 0x7f20440c8b50 2023-09-27T23:26:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f2041d9b700 1 --2- 172.21.15.64:0/3029809990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440c8b50 0x7f20440c8fb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f204406bf10 tx=0x7f2038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f472deaf751fe3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204159a700 1 -- 172.21.15.64:0/3029809990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800b820 con 0x7f20440c8b50 2023-09-27T23:26:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204159a700 1 -- 172.21.15.64:0/3029809990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800b980 con 0x7f20440c8b50 2023-09-27T23:26:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.548+0000 7f204159a700 1 -- 172.21.15.64:0/3029809990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800bd40 con 0x7f20440c8b50 2023-09-27T23:26:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/3029809990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440c8b50 msgr2=0x7f20440c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- 172.21.15.64:0/3029809990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440c8b50 0x7f20440c8fb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f204406bf10 tx=0x7f2038009250 comp rx=0 tx=0).stop 2023-09-27T23:26:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/3029809990 shutdown_connections 2023-09-27T23:26:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- 172.21.15.64:0/3029809990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f204406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- 172.21.15.64:0/3029809990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20440c8b50 0x7f20440c8fb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- 172.21.15.64:0/3029809990 >> v2:172.21.15.123:3300/0 conn(0x7f2044156770 0x7f2044156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/3029809990 >> 172.21.15.64:0/3029809990 conn(0x7f2044067bb0 msgr2=0x7f20440add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/3029809990 shutdown_connections 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/3029809990 wait complete. 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 Processor -- start 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- start start 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f20440c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20440c8b50 0x7f2044168d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 0x7f2044158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044153120 con 0x7f2044156770 2023-09-27T23:26:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20440c15e0 con 0x7f20440c8b50 2023-09-27T23:26:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20440c1860 con 0x7f204406ad70 2023-09-27T23:26:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 0x7f2044158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 0x7f2044158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33062/0 (socket says 172.21.15.64:33062) 2023-09-27T23:26:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 -- 172.21.15.64:0/2421560932 learned_addr learned my addr 172.21.15.64:0/2421560932 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204259c700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f20440c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2041d9b700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20440c8b50 0x7f2044168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 -- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 msgr2=0x7f20440c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f20440c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 -- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20440c8b50 msgr2=0x7f2044168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20440c8b50 0x7f2044168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 -- 172.21.15.64:0/2421560932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380094f0 con 0x7f2044156770 2023-09-27T23:26:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204259c700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f20440c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f2042d9d700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 0x7f2044158310 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f203c007f50 tx=0x7f203c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00ec20 con 0x7f2044156770 2023-09-27T23:26:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20440c1b40 con 0x7f2044156770 2023-09-27T23:26:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2044169280 con 0x7f2044156770 2023-09-27T23:26:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c0099c0 con 0x7f2044156770 2023-09-27T23:26:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.552+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c01e460 con 0x7f2044156770 2023-09-27T23:26:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.556+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2044169470 con 0x7f2044156770 2023-09-27T23:26:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.556+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f203c01e7d0 con 0x7f2044156770 2023-09-27T23:26:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.556+0000 7f20327fc700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 0x7f2024062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.556+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7310+0+0 (secure 0 0 0) 0x7f203c07dcd0 con 0x7f2044156770 2023-09-27T23:26:42.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.556+0000 7f204259c700 1 -- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 msgr2=0x7f2024062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:42.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.556+0000 7f204259c700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 0x7f2024062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:42.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.560+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f203c04d7f0 con 0x7f2044156770 2023-09-27T23:26:42.579 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.574+0000 7f8007463000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:26:42.582 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.576+0000 7f654729b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:26:42.632 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.626+0000 7f8007463000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:26:42.637 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.632+0000 7f654729b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:26:42.693 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.688+0000 7f654729b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:26:42.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.760+0000 7f204259c700 1 -- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 msgr2=0x7f2024062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:42.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.760+0000 7f204259c700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 0x7f2024062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:26:42.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.760+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20441532d0 con 0x7f2044156770 2023-09-27T23:26:42.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.760+0000 7f20327fc700 1 -- 172.21.15.64:0/2421560932 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f20441532d0 con 0x7f2044156770 2023-09-27T23:26:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:42.767 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:14:44.562884+0000","modified":"2023-09-27T23:14:46.768950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/974610030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":974610030},{"type":"v1","addr":"172.21.15.64:6835","nonce":974610030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 msgr2=0x7f2024062fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 0x7f2024062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 msgr2=0x7f2044158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 0x7f2044158310 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f203c007f50 tx=0x7f203c009860 comp rx=0 tx=0).stop 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 shutdown_connections 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2024060b30 0x7f2024062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2044156770 0x7f2044158310 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20440c8b50 0x7f2044168d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 --2- 172.21.15.64:0/2421560932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f204406ad70 0x7f20440c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 >> 172.21.15.64:0/2421560932 conn(0x7f2044067bb0 msgr2=0x7f20441527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:42.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 shutdown_connections 2023-09-27T23:26:42.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.764+0000 7f204359e700 1 -- 172.21.15.64:0/2421560932 wait complete. 2023-09-27T23:26:42.771 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2023-09-27T23:26:42.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:26:42.805 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.802+0000 7f8007463000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:26:42.858 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.854+0000 7f8007463000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:26:42.871 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.868+0000 7f654729b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:26:42.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.876+0000 7f90266f3700 1 Processor -- start 2023-09-27T23:26:42.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.876+0000 7f90266f3700 1 -- start start 2023-09-27T23:26:42.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 0x7f902006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200c92c0 con 0x7f90200cbdb0 2023-09-27T23:26:42.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200c8450 con 0x7f902006d1f0 2023-09-27T23:26:42.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c86b0 con 0x7f90200c2d30 2023-09-27T23:26:42.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9017fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90200cbdb0 0x7f90201605e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55374/0 (socket says 172.21.15.64:55374) 2023-09-27T23:26:42.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9017fff700 1 -- 172.21.15.64:0/1407442598 learned_addr learned my addr 172.21.15.64:0/1407442598 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 0x7f902006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195001390 0 0) 0x7f90200c92c0 con 0x7f90200cbdb0 2023-09-27T23:26:42.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c003570 con 0x7f90200cbdb0 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3101763773 0 0) 0x7f900c003570 con 0x7f90200cbdb0 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 msgr2=0x7f902006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 0x7f902006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 >> v1:172.21.15.123:6789/0 conn(0x7f902006d1f0 legacy=0x7f90200c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200c8990 con 0x7f90200cbdb0 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f9008001470 con 0x7f90200cbdb0 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 >> v1:172.21.15.64:6789/0 conn(0x7f90200cbdb0 legacy=0x7f90201605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 0x7f900c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f900c004010 0x7f900c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f900c004d50 0x7f900c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c86b0 con 0x7f900c004d50 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00bbf0 con 0x7f900c004010 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00bdb0 con 0x7f90200c2d30 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f900c004d50 0x7f900c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901ffff700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 0x7f900c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9017fff700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f900c004010 0x7f900c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 -- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 msgr2=0x7f900c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c2d30 0x7f900c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 -- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f900c004010 msgr2=0x7f900c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f900c004010 0x7f900c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 -- 172.21.15.64:0/1407442598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c003570 con 0x7f900c004d50 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f900c004d50 0x7f900c0051c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f90180094d0 tx=0x7f901800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25d3f0be63a73ae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800ea90 con 0x7f900c004d50 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901800f040 con 0x7f900c004d50 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f901f7fe700 1 -- 172.21.15.64:0/1407442598 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800ee20 con 0x7f900c004d50 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- 172.21.15.64:0/1407442598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f900c004d50 msgr2=0x7f900c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 --2- 172.21.15.64:0/1407442598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f900c004d50 0x7f900c0051c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f90180094d0 tx=0x7f901800b040 comp rx=0 tx=0).stop 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f9024c90700 1 -- 172.21.15.64:0/1407442598 reap_dead start 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- 172.21.15.64:0/1407442598 shutdown_connections 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- 172.21.15.64:0/1407442598 >> 172.21.15.64:0/1407442598 conn(0x7f90200cd9c0 msgr2=0x7f90200c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- 172.21.15.64:0/1407442598 shutdown_connections 2023-09-27T23:26:42.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.880+0000 7f90266f3700 1 -- 172.21.15.64:0/1407442598 wait complete. 2023-09-27T23:26:42.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 Processor -- start 2023-09-27T23:26:42.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 -- start start 2023-09-27T23:26:42.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902015c270 0x7f9020162220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 0x7f90200c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c6660 0x7f902015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c6ae0 con 0x7f9020162760 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902016dff0 con 0x7f902015c270 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9020162c20 con 0x7f90200c6660 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 0x7f90200c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 0x7f90200c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33088/0 (socket says 172.21.15.64:33088) 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 -- 172.21.15.64:0/904404334 learned_addr learned my addr 172.21.15.64:0/904404334 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f901ffff700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902015c270 0x7f9020162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9024c90700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c6660 0x7f902015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 -- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c6660 msgr2=0x7f902015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c6660 0x7f902015dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 -- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902015c270 msgr2=0x7f9020162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902015c270 0x7f9020162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 -- 172.21.15.64:0/904404334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902016d9a0 con 0x7f9020162760 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f901ffff700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902015c270 0x7f9020162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9024c90700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c6660 0x7f902015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f9017fff700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 0x7f90200c6120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f90080073c0 tx=0x7f9008001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008018030 con 0x7f9020162760 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900800f8d0 con 0x7f9020162760 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902016dc80 con 0x7f9020162760 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900800fc70 con 0x7f9020162760 2023-09-27T23:26:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.884+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902015c640 con 0x7f9020162760 2023-09-27T23:26:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.888+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9020071120 con 0x7f9020162760 2023-09-27T23:26:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.888+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f900801b070 con 0x7f9020162760 2023-09-27T23:26:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.888+0000 7f901cff9700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9000060b00 0x7f9000062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.888+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7310+0+0 (secure 0 0 0) 0x7f9008019070 con 0x7f9020162760 2023-09-27T23:26:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.888+0000 7f901ffff700 1 -- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9000060b00 msgr2=0x7f9000062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.888+0000 7f901ffff700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9000060b00 0x7f9000062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:42.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:42.896+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f900804d1a0 con 0x7f9020162760 2023-09-27T23:26:42.920 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:42.918+0000 7f8007463000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:26:42.930 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.924+0000 7f654729b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:26:42.996 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:42.992+0000 7f654729b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:26:43.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.056+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f902016e130 con 0x7f9020162760 2023-09-27T23:26:43.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.056+0000 7f901cff9700 1 -- 172.21.15.64:0/904404334 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18496 (secure 0 0 0) 0x7f90080507c0 con 0x7f9020162760 2023-09-27T23:26:43.063 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:43.063 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":57,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:42.050608+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:14:42.525604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-09-27T23:14:43.561136+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/1007482582":"2023-09-28T23:15:00.247118+0000","172.21.15.123:6833/33228":"2023-09-28T23:15:00.247118+0000","172.21.15.123:6832/33228":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/3145850360":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/439915678":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/118144335":"2023-09-28T23:15:00.247118+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":2},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":30,"length":1},{"begin":32,"length":1},{"begin":34,"length":1},{"begin":37,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":44,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":53,"length":2},{"begin":56,"length":1},{"begin":60,"length":1},{"begin":62,"length":2},{"begin":67,"length":3},{"begin":71,"length":1},{"begin":73,"length":5},{"begin":79,"length":1},{"begin":82,"length":2},{"begin":86,"length":2},{"begin":89,"length":1},{"begin":91,"length":4},{"begin":96,"length":2},{"begin":100,"length":1},{"begin":103,"length":3},{"begin":107,"length":1},{"begin":109,"length":2},{"begin":162,"length":60}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:26:43.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9000060b00 msgr2=0x7f9000062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:43.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9000060b00 0x7f9000062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 msgr2=0x7f90200c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 0x7f90200c6120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f90080073c0 tx=0x7f9008001e50 comp rx=0 tx=0).stop 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 shutdown_connections 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9000060b00 0x7f9000062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90200c6660 0x7f902015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9020162760 0x7f90200c6120 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 --2- 172.21.15.64:0/904404334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f902015c270 0x7f9020162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 >> 172.21.15.64:0/904404334 conn(0x7f90200cd9c0 msgr2=0x7f90200c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 shutdown_connections 2023-09-27T23:26:43.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.060+0000 7f90266f3700 1 -- 172.21.15.64:0/904404334 wait complete. 2023-09-27T23:26:43.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.062+0000 7f8007463000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:26:43.081 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:26:43.081 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:26:43.125 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.122+0000 7f8007463000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:26:43.178 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:43.172+0000 7f654729b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:26:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b936d700 1 Processor -- start 2023-09-27T23:26:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b936d700 1 -- start start 2023-09-27T23:26:43.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b936d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f28b4158f70 0x7f28b4067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b936d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b4068c70 con 0x7f28b4069100 2023-09-27T23:26:43.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b936d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b4068060 con 0x7f28b4159340 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b936d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b4068340 con 0x7f28b4158f70 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b27fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f28b4069100 0x7f28b40cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55390/0 (socket says 172.21.15.64:55390) 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b27fc700 1 -- 172.21.15.64:0/5735741 learned_addr learned my addr 172.21.15.64:0/5735741 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/5735741 >> v2:172.21.15.123:3300/0 conn(0x7f28b4158f70 0x7f28b4067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074142647 0 0) 0x7f28b4068c70 con 0x7f28b4069100 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f289c003570 con 0x7f28b4069100 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902286695 0 0) 0x7f289c003570 con 0x7f28b4069100 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 >> v2:172.21.15.123:3300/0 conn(0x7f28b4158f70 msgr2=0x7f28b4067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 --2- 172.21.15.64:0/5735741 >> v2:172.21.15.123:3300/0 conn(0x7f28b4158f70 0x7f28b4067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 >> v1:172.21.15.123:6789/0 conn(0x7f28b4159340 legacy=0x7f28b40cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4068620 con 0x7f28b4069100 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f28a8001470 con 0x7f28b4069100 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 >> v1:172.21.15.64:6789/0 conn(0x7f28b4069100 legacy=0x7f28b40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c003a60 0x7f289c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c004810 0x7f289c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c005210 0x7f289c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4068340 con 0x7f289c003a60 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c00c1b0 con 0x7f289c004810 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c00c370 con 0x7f289c005210 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c003a60 0x7f289c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b27fc700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c004810 0x7f289c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b37fe700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c005210 0x7f289c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 -- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c005210 msgr2=0x7f289c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c005210 0x7f289c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 -- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c004810 msgr2=0x7f289c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f289c004810 0x7f289c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b2ffd700 1 -- 172.21.15.64:0/5735741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c003570 con 0x7f289c003a60 2023-09-27T23:26:43.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.180+0000 7f28b37fe700 1 -- 172.21.15.64:0/5735741 reap_dead start 2023-09-27T23:26:43.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c003a60 0x7f289c003e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006f00 tx=0x7f28a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c19daa1ba5dbc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400e710 con 0x7f289c003a60 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a400e870 con 0x7f289c003a60 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b1ffb700 1 -- 172.21.15.64:0/5735741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400ec30 con 0x7f289c003a60 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- 172.21.15.64:0/5735741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c003a60 msgr2=0x7f289c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c003a60 0x7f289c003e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006f00 tx=0x7f28a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- 172.21.15.64:0/5735741 shutdown_connections 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 --2- 172.21.15.64:0/5735741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c003a60 0x7f289c003e70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- 172.21.15.64:0/5735741 >> 172.21.15.64:0/5735741 conn(0x7f28b40ccc40 msgr2=0x7f28b40cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- 172.21.15.64:0/5735741 shutdown_connections 2023-09-27T23:26:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- 172.21.15.64:0/5735741 wait complete. 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 Processor -- start 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- start start 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c003a60 0x7f28b414dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 0x7f28b415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b415f780 0x7f28b40c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b415fc20 con 0x7f28b416d5a0 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b415c520 con 0x7f28b415f780 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b936d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b415c800 con 0x7f289c003a60 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.184+0000 7f28b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 0x7f28b415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 0x7f28b415f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33118/0 (socket says 172.21.15.64:33118) 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 -- 172.21.15.64:0/4016393662 learned_addr learned my addr 172.21.15.64:0/4016393662 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c003a60 0x7f28b414dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b37fe700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b415f780 0x7f28b40c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 -- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c003a60 msgr2=0x7f28b414dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c003a60 0x7f28b414dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 -- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b415f780 msgr2=0x7f28b40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b415f780 0x7f28b40c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 -- 172.21.15.64:0/4016393662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b415cae0 con 0x7f28b416d5a0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b27fc700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 0x7f28b415f240 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f28a8001050 tx=0x7f28a8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8018030 con 0x7f28b416d5a0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a800f6c0 con 0x7f28b416d5a0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b415cdc0 con 0x7f28b416d5a0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a800fa60 con 0x7f28b416d5a0 2023-09-27T23:26:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b416da10 con 0x7f28b416d5a0 2023-09-27T23:26:43.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b4071120 con 0x7f28b416d5a0 2023-09-27T23:26:43.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f28a801b070 con 0x7f28b416d5a0 2023-09-27T23:26:43.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f289affd700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 0x7f28a0062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7310+0+0 (secure 0 0 0) 0x7f28a8019070 con 0x7f28b416d5a0 2023-09-27T23:26:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b2ffd700 1 -- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 msgr2=0x7f28a0062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.188+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 0x7f28a0062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:43.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.196+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f28a804cee0 con 0x7f28b416d5a0 2023-09-27T23:26:43.245 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:43.240+0000 7f654729b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:26:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.392+0000 7f28b2ffd700 1 -- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 msgr2=0x7f28a0062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.392+0000 7f28b2ffd700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 0x7f28a0062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:26:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.396+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28b4068900 con 0x7f28b416d5a0 2023-09-27T23:26:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f289affd700 1 -- 172.21.15.64:0/4016393662 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f28a8050500 con 0x7f28b416d5a0 2023-09-27T23:26:43.404 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:43.404 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:14:44.562884+0000","modified":"2023-09-27T23:14:46.768950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/974610030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":974610030},{"type":"v1","addr":"172.21.15.64:6835","nonce":974610030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 msgr2=0x7f28a0062fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 0x7f28a0062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 msgr2=0x7f28b415f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 0x7f28b415f240 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f28a8001050 tx=0x7f28a8001e50 comp rx=0 tx=0).stop 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 shutdown_connections 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28a0060b30 0x7f28a0062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b415f780 0x7f28b40c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28b416d5a0 0x7f28b415f240 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 --2- 172.21.15.64:0/4016393662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f289c003a60 0x7f28b414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 >> 172.21.15.64:0/4016393662 conn(0x7f28b40ccc40 msgr2=0x7f28b40c5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 shutdown_connections 2023-09-27T23:26:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.400+0000 7f28b936d700 1 -- 172.21.15.64:0/4016393662 wait complete. 2023-09-27T23:26:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2023-09-27T23:26:43.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:26:43.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 Processor -- start 2023-09-27T23:26:43.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- start start 2023-09-27T23:26:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880b0110 0x7f4e880b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e880ccb40 con 0x7f4e88153290 2023-09-27T23:26:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e880cbf30 con 0x7f4e880ccfd0 2023-09-27T23:26:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880cc210 con 0x7f4e880b0110 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8ea9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e88153290 0x7f4e881536d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55392/0 (socket says 172.21.15.64:55392) 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8e29a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880b0110 0x7f4e880b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8ea9b700 1 -- 172.21.15.64:0/715010173 learned_addr learned my addr 172.21.15.64:0/715010173 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8d298700 1 -- 172.21.15.64:0/715010173 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328745606 0 0) 0x7f4e880ccb40 con 0x7f4e88153290 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8d298700 1 -- 172.21.15.64:0/715010173 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e70003570 con 0x7f4e88153290 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8e29a700 1 -- 172.21.15.64:0/715010173 >> v1:172.21.15.123:6789/0 conn(0x7f4e880ccfd0 legacy=0x7f4e8815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8e29a700 1 -- 172.21.15.64:0/715010173 >> v1:172.21.15.64:6789/0 conn(0x7f4e88153290 legacy=0x7f4e881536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8e29a700 1 -- 172.21.15.64:0/715010173 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880cc4f0 con 0x7f4e880b0110 2023-09-27T23:26:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8e29a700 1 --2- 172.21.15.64:0/715010173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880b0110 0x7f4e880b04e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009e90 tx=0x7f4e80009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796db66fbe13d12d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8d298700 1 -- 172.21.15.64:0/715010173 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000e070 con 0x7f4e880b0110 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8d298700 1 -- 172.21.15.64:0/715010173 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8000bc30 con 0x7f4e880b0110 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e8d298700 1 -- 172.21.15.64:0/715010173 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80012430 con 0x7f4e880b0110 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- 172.21.15.64:0/715010173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880b0110 msgr2=0x7f4e880b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 --2- 172.21.15.64:0/715010173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880b0110 0x7f4e880b04e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009e90 tx=0x7f4e80009780 comp rx=0 tx=0).stop 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- 172.21.15.64:0/715010173 shutdown_connections 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 --2- 172.21.15.64:0/715010173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880b0110 0x7f4e880b04e0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009e90 tx=0x7f4e80009780 comp rx=0 tx=0).stop 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- 172.21.15.64:0/715010173 >> 172.21.15.64:0/715010173 conn(0x7f4e8806c7c0 msgr2=0x7f4e8806cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- 172.21.15.64:0/715010173 shutdown_connections 2023-09-27T23:26:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.516+0000 7f4e904fe700 1 -- 172.21.15.64:0/715010173 wait complete. 2023-09-27T23:26:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 Processor -- start 2023-09-27T23:26:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 -- start start 2023-09-27T23:26:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ccfd0 0x7f4e88067af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 0x7f4e880ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880ac820 0x7f4e88168a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88068a20 con 0x7f4e88153290 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880a8e60 con 0x7f4e880ccfd0 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88069cd0 con 0x7f4e880ac820 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 0x7f4e880ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 0x7f4e880ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33128/0 (socket says 172.21.15.64:33128) 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 -- 172.21.15.64:0/1355450463 learned_addr learned my addr 172.21.15.64:0/1355450463 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8ea9b700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880ac820 0x7f4e88168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8e29a700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ccfd0 0x7f4e88067af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 -- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880ac820 msgr2=0x7f4e88168a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880ac820 0x7f4e88168a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 -- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ccfd0 msgr2=0x7f4e88067af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ccfd0 0x7f4e88067af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 -- 172.21.15.64:0/1355450463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80009a20 con 0x7f4e88153290 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e8da99700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 0x7f4e880ac2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c5c0 tx=0x7f4e7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c011590 con 0x7f4e88153290 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c011710 con 0x7f4e88153290 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c011ad0 con 0x7f4e88153290 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.520+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e88068180 con 0x7f4e88153290 2023-09-27T23:26:43.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e88068740 con 0x7f4e88153290 2023-09-27T23:26:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88071120 con 0x7f4e88153290 2023-09-27T23:26:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 80430+0+0 (secure 0 0 0) 0x7f4e7c01b3b0 con 0x7f4e88153290 2023-09-27T23:26:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e867fc700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 0x7f4e74062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e8e29a700 1 -- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 msgr2=0x7f4e74062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7310+0+0 (secure 0 0 0) 0x7f4e7c07ca80 con 0x7f4e88153290 2023-09-27T23:26:43.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.524+0000 7f4e8e29a700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 0x7f4e74062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:26:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.532+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4e7c04c5a0 con 0x7f4e88153290 2023-09-27T23:26:43.715 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.710+0000 7f8007463000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:26:43.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e8e29a700 1 -- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 msgr2=0x7f4e74062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:26:43.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e8e29a700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 0x7f4e74062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:26:43.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e880a8fa0 con 0x7f4e88153290 2023-09-27T23:26:43.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e867fc700 1 -- 172.21.15.64:0/1355450463 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f4e7c04fbc0 con 0x7f4e88153290 2023-09-27T23:26:43.733 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:43.733 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:14:44.562884+0000","modified":"2023-09-27T23:14:46.768950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/974610030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":974610030},{"type":"v1","addr":"172.21.15.64:6835","nonce":974610030}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 msgr2=0x7f4e74062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 0x7f4e74062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 msgr2=0x7f4e880ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 0x7f4e880ac2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c5c0 tx=0x7f4e7c00c8d0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 shutdown_connections 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4e74060b00 0x7f4e74062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e880ac820 0x7f4e88168a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e88153290 0x7f4e880ac2e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 --2- 172.21.15.64:0/1355450463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ccfd0 0x7f4e88067af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 >> 172.21.15.64:0/1355450463 conn(0x7f4e8806c7c0 msgr2=0x7f4e880c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:43.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 shutdown_connections 2023-09-27T23:26:43.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.728+0000 7f4e904fe700 1 -- 172.21.15.64:0/1355450463 wait complete. 2023-09-27T23:26:43.737 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2023-09-27T23:26:43.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd0ea0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd0ea0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd0ea0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd0ea0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd0ea0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd46a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.810 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd46a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.811 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd46a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.811 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd46a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.811 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:26:43.806+0000 7f7fd46a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:26:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 Processor -- start 2023-09-27T23:26:43.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 -- start start 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 0x7f2c2006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c2006b2b0 0x7f2c200af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c200ca8c0 con 0x7f2c20156770 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2006b730 con 0x7f2c2006b2b0 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c254c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c200cb700 con 0x7f2c20156b40 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2c20156770 0x7f2c200c8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55398/0 (socket says 172.21.15.64:55398) 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1ffff700 1 -- 172.21.15.64:0/583205975 learned_addr learned my addr 172.21.15.64:0/583205975 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c24cc8700 1 --2- 172.21.15.64:0/583205975 >> v2:172.21.15.123:3300/0 conn(0x7f2c2006b2b0 0x7f2c200af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1f7fe700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 0x7f2c2006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539081630 0 0) 0x7f2c200ca8c0 con 0x7f2c20156770 2023-09-27T23:26:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c08003570 con 0x7f2c20156770 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2209097312 0 0) 0x7f2c08003570 con 0x7f2c20156770 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 msgr2=0x7f2c2006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.856+0000 7f2c1effd700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 0x7f2c2006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 >> v2:172.21.15.123:3300/0 conn(0x7f2c2006b2b0 msgr2=0x7f2c200af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 --2- 172.21.15.64:0/583205975 >> v2:172.21.15.123:3300/0 conn(0x7f2c2006b2b0 0x7f2c200af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c20156f70 con 0x7f2c20156770 2023-09-27T23:26:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f2c10001470 con 0x7f2c20156770 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 >> v1:172.21.15.64:6789/0 conn(0x7f2c20156770 legacy=0x7f2c200c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08003a60 0x7f2c08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08004810 0x7f2c08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 0x7f2c080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2006b730 con 0x7f2c08004810 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c200cb700 con 0x7f2c08003a60 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0800ba90 con 0x7f2c20156b40 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c24cc8700 1 --2- 172.21.15.64:0/583205975 >> v2:172.21.15.123:3300/0 conn(0x7f2c2006b2b0 0x7f2c200af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08004810 0x7f2c08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1ffff700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08003a60 0x7f2c08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c24cc8700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 0x7f2c080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 -- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 msgr2=0x7f2c080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c20156b40 0x7f2c080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 -- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08003a60 msgr2=0x7f2c08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08003a60 0x7f2c08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 -- 172.21.15.64:0/583205975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c08003570 con 0x7f2c08004810 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1ffff700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08003a60 0x7f2c08003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1f7fe700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08004810 0x7f2c08004c80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009c00 tx=0x7f2c140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffac49171137ffb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e6b0 con 0x7f2c08004810 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1400e810 con 0x7f2c08004810 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c1effd700 1 -- 172.21.15.64:0/583205975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400ebd0 con 0x7f2c08004810 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c254c9700 1 -- 172.21.15.64:0/583205975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08004810 msgr2=0x7f2c08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c254c9700 1 --2- 172.21.15.64:0/583205975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08004810 0x7f2c08004c80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009c00 tx=0x7f2c140091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c24cc8700 1 -- 172.21.15.64:0/583205975 reap_dead start 2023-09-27T23:26:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c254c9700 1 -- 172.21.15.64:0/583205975 shutdown_connections 2023-09-27T23:26:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c254c9700 1 -- 172.21.15.64:0/583205975 >> 172.21.15.64:0/583205975 conn(0x7f2c20067bb0 msgr2=0x7f2c200add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c254c9700 1 -- 172.21.15.64:0/583205975 shutdown_connections 2023-09-27T23:26:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.860+0000 7f2c254c9700 1 -- 172.21.15.64:0/583205975 wait complete. 2023-09-27T23:26:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 Processor -- start 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 -- start start 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08004810 0x7f2c20150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2015b1c0 0x7f2c200cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 0x7f2c2014a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c20151c10 con 0x7f2c200ccd20 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c20151930 con 0x7f2c08004810 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2016a910 con 0x7f2c2015b1c0 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 0x7f2c2014a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 0x7f2c2014a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33154/0 (socket says 172.21.15.64:33154) 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 -- 172.21.15.64:0/1911986665 learned_addr learned my addr 172.21.15.64:0/1911986665 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c1f7fe700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2015b1c0 0x7f2c200cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c1ffff700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08004810 0x7f2c20150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 -- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2015b1c0 msgr2=0x7f2c200cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2015b1c0 0x7f2c200cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 -- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08004810 msgr2=0x7f2c20150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08004810 0x7f2c20150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 -- 172.21.15.64:0/1911986665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2016abf0 con 0x7f2c200ccd20 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c24cc8700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 0x7f2c2014a950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c200ca3d0 tx=0x7f2c1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800e070 con 0x7f2c200ccd20 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c18011c50 con 0x7f2c200ccd20 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2016aed0 con 0x7f2c200ccd20 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18016470 con 0x7f2c200ccd20 2023-09-27T23:26:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.864+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2016b3f0 con 0x7f2c200ccd20 2023-09-27T23:26:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.868+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2016a770 con 0x7f2c200ccd20 2023-09-27T23:26:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.868+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 14) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f2c18016840 con 0x7f2c200ccd20 2023-09-27T23:26:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.868+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7567+0+0 (secure 0 0 0) 0x7f2c1807ded0 con 0x7f2c200ccd20 2023-09-27T23:26:43.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:43.876+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2c2016a770 con 0x7f2c200ccd20 2023-09-27T23:26:43.926 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:26:43.920+0000 7f654729b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:26:44.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.036+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c2016a770 con 0x7f2c200ccd20 2023-09-27T23:26:44.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.036+0000 7f2c07fff700 1 -- 172.21.15.64:0/1911986665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+18866 (secure 0 0 0) 0x7f2c2016a770 con 0x7f2c200ccd20 2023-09-27T23:26:44.043 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:44.043 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":58,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:43.735988+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:14:42.525604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-09-27T23:14:43.561136+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/993105349":"2023-09-28T23:26:43.735819+0000","172.21.15.123:0/3294479245":"2023-09-28T23:26:43.735819+0000","172.21.15.123:0/2396637360":"2023-09-28T23:26:43.735819+0000","172.21.15.123:0/1314743155":"2023-09-28T23:26:43.735819+0000","172.21.15.123:0/3285093195":"2023-09-28T23:26:43.735819+0000","172.21.15.123:0/2522320671":"2023-09-28T23:26:43.735819+0000","172.21.15.123:0/118144335":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/439915678":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/3145850360":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/3436933931":"2023-09-28T23:26:43.735819+0000","172.21.15.123:6832/33228":"2023-09-28T23:15:00.247118+0000","172.21.15.123:6833/33228":"2023-09-28T23:15:00.247118+0000","172.21.15.123:0/1007482582":"2023-09-28T23:15:00.247118+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":2},{"begin":22,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":30,"length":1},{"begin":32,"length":1},{"begin":34,"length":1},{"begin":37,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":44,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":53,"length":2},{"begin":56,"length":1},{"begin":60,"length":1},{"begin":62,"length":2},{"begin":67,"length":3},{"begin":71,"length":1},{"begin":73,"length":5},{"begin":79,"length":1},{"begin":82,"length":2},{"begin":86,"length":2},{"begin":89,"length":1},{"begin":91,"length":4},{"begin":96,"length":2},{"begin":100,"length":1},{"begin":103,"length":3},{"begin":107,"length":1},{"begin":109,"length":2},{"begin":162,"length":60}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 msgr2=0x7f2c2014a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 0x7f2c2014a950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c200ca3d0 tx=0x7f2c1800b040 comp rx=0 tx=0).stop 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 shutdown_connections 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c200ccd20 0x7f2c2014a950 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2015b1c0 0x7f2c200cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 --2- 172.21.15.64:0/1911986665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c08004810 0x7f2c20150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 >> 172.21.15.64:0/1911986665 conn(0x7f2c20067bb0 msgr2=0x7f2c200c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 shutdown_connections 2023-09-27T23:26:44.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.040+0000 7f2c254c9700 1 -- 172.21.15.64:0/1911986665 wait complete. 2023-09-27T23:26:44.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:26:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.156+0000 7f2b32344700 1 Processor -- start 2023-09-27T23:26:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.156+0000 7f2b32344700 1 -- start start 2023-09-27T23:26:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b2c140420 0x7f2b2c15b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2b2c06c7c0 0x7f2b2c0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c1407f0 0x7f2b2c0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0ae280 con 0x7f2b2c140420 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c06d7d0 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c06cc50 con 0x7f2b2c1407f0 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b2c140420 0x7f2b2c15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2b2c06c7c0 0x7f2b2c0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2b2c06c7c0 0x7f2b2c0b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:53332/0 (socket says 172.21.15.64:53332) 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 -- 172.21.15.64:0/1943121541 learned_addr learned my addr 172.21.15.64:0/1943121541 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b308e1700 1 --2- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c1407f0 0x7f2b2c0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 -- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c1407f0 msgr2=0x7f2b2c0b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 --2- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c1407f0 0x7f2b2c0b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 -- 172.21.15.64:0/1943121541 >> v2:172.21.15.64:3300/0 conn(0x7f2b2c140420 msgr2=0x7f2b2c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 --2- 172.21.15.64:0/1943121541 >> v2:172.21.15.64:3300/0 conn(0x7f2b2c140420 0x7f2b2c15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 -- 172.21.15.64:0/1943121541 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c06cf30 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2b7fe700 1 --2- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c0b0cd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c06a1f0 tx=0x7f2b14009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533d1453d1103843 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2affd700 1 -- 172.21.15.64:0/1943121541 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400e070 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2affd700 1 -- 172.21.15.64:0/1943121541 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1400b970 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b2affd700 1 -- 172.21.15.64:0/1943121541 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140092e0 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c06c7c0 msgr2=0x7f2b2c0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:44.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c0b0cd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c06a1f0 tx=0x7f2b14009840 comp rx=0 tx=0).stop 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- 172.21.15.64:0/1943121541 shutdown_connections 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c1407f0 0x7f2b2c0b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- 172.21.15.64:0/1943121541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c0b0cd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 --2- 172.21.15.64:0/1943121541 >> v2:172.21.15.64:3300/0 conn(0x7f2b2c140420 0x7f2b2c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- 172.21.15.64:0/1943121541 >> 172.21.15.64:0/1943121541 conn(0x7f2b2c067b60 msgr2=0x7f2b2c0ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- 172.21.15.64:0/1943121541 shutdown_connections 2023-09-27T23:26:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.160+0000 7f2b32344700 1 -- 172.21.15.64:0/1943121541 wait complete. 2023-09-27T23:26:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 Processor -- start 2023-09-27T23:26:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 -- start start 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c140420 0x7f2b2c13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c1407f0 0x7f2b2c13e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c09e0 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8180 con 0x7f2b2c1407f0 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8460 con 0x7f2b2c140420 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33174/0 (socket says 172.21.15.64:33174) 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c140420 0x7f2b2c13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b308e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c1407f0 0x7f2b2c13e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 -- 172.21.15.64:0/3515334292 learned_addr learned my addr 172.21.15.64:0/3515334292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 -- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c140420 msgr2=0x7f2b2c13dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c140420 0x7f2b2c13dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 -- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c1407f0 msgr2=0x7f2b2c13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c1407f0 0x7f2b2c13e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 -- 172.21.15.64:0/3515334292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b14009ae0 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b308e1700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c1407f0 0x7f2b2c13e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2b7fe700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c140420 0x7f2b2c13dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b2bfff700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c1585c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00ec80 tx=0x7f2b1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00c850 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00c9b0 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00cd50 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c0c0c80 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.164+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c158b00 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.168+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c071120 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.168+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 14) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f2b1c013680 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.168+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7567+0+0 (secure 0 0 0) 0x7f2b1c07d200 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.176+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2b1c04d260 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.380+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2b2c06d210 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.752+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 15) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f2b1c04cd10 con 0x7f2b2c06c7c0 2023-09-27T23:26:44.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.752+0000 7f2b23fff700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2b0c0671b0 0x7f2b0c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:44.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.752+0000 7f2b2b7fe700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2b0c0671b0 0x7f2b0c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:44.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:44.760+0000 7f2b2b7fe700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2b0c0671b0 0x7f2b0c069590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009880 tx=0x7f2b14018000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:45.228 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:26:45.224+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:26:45.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.244+0000 7f2b23fff700 1 -- 172.21.15.64:0/3515334292 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2b1c039c50 con 0x7f2b2c06c7c0 2023-09-27T23:26:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2b0c0671b0 msgr2=0x7f2b0c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2b0c0671b0 0x7f2b0c069590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009880 tx=0x7f2b14018000 comp rx=0 tx=0).stop 2023-09-27T23:26:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 msgr2=0x7f2b2c1585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c1585c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00ec80 tx=0x7f2b1c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 shutdown_connections 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2b0c0671b0 0x7f2b0c069590 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b2c1407f0 0x7f2b2c13e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b2c140420 0x7f2b2c13dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 --2- 172.21.15.64:0/3515334292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b2c06c7c0 0x7f2b2c1585c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 >> 172.21.15.64:0/3515334292 conn(0x7f2b2c067b60 msgr2=0x7f2b2c13f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 shutdown_connections 2023-09-27T23:26:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.248+0000 7f2b32344700 1 -- 172.21.15.64:0/3515334292 wait complete. 2023-09-27T23:26:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:26:45.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:26:45.287 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:26:45.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.372+0000 7f259f734700 1 Processor -- start 2023-09-27T23:26:45.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.372+0000 7f259f734700 1 -- start start 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.372+0000 7f259f734700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f259815aa20 0x7f259806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.372+0000 7f259f734700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25980ac8f0 con 0x7f25980ca320 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.372+0000 7f259f734700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25980ca750 con 0x7f2598156eb0 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.372+0000 7f259f734700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2598158290 con 0x7f259815aa20 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259dcd1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f259815aa20 0x7f259806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259cccf700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f25980ca320 0x7f25980cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57154/0 (socket says 172.21.15.64:57154) 2023-09-27T23:26:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259cccf700 1 -- 172.21.15.64:0/1630949480 learned_addr learned my addr 172.21.15.64:0/1630949480 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f2597fff700 1 -- 172.21.15.64:0/1630949480 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729601201 0 0) 0x7f25980ac8f0 con 0x7f25980ca320 2023-09-27T23:26:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259dcd1700 1 -- 172.21.15.64:0/1630949480 >> v1:172.21.15.123:6789/0 conn(0x7f2598156eb0 legacy=0x7f25980acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259dcd1700 1 -- 172.21.15.64:0/1630949480 >> v1:172.21.15.64:6789/0 conn(0x7f25980ca320 legacy=0x7f25980cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259dcd1700 1 -- 172.21.15.64:0/1630949480 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259815b120 con 0x7f259815aa20 2023-09-27T23:26:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259dcd1700 1 --2- 172.21.15.64:0/1630949480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f259806d050 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f25880098e0 tx=0x7f25880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff58d77446270d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f2597fff700 1 -- 172.21.15.64:0/1630949480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588009e80 con 0x7f259815aa20 2023-09-27T23:26:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f2597fff700 1 -- 172.21.15.64:0/1630949480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258800b860 con 0x7f259815aa20 2023-09-27T23:26:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f2597fff700 1 -- 172.21.15.64:0/1630949480 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800bbe0 con 0x7f259815aa20 2023-09-27T23:26:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 -- 172.21.15.64:0/1630949480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 msgr2=0x7f259806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 --2- 172.21.15.64:0/1630949480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f259806d050 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f25880098e0 tx=0x7f25880091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 -- 172.21.15.64:0/1630949480 shutdown_connections 2023-09-27T23:26:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 --2- 172.21.15.64:0/1630949480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f259806d050 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 -- 172.21.15.64:0/1630949480 >> 172.21.15.64:0/1630949480 conn(0x7f25980cd8f0 msgr2=0x7f25980c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 -- 172.21.15.64:0/1630949480 shutdown_connections 2023-09-27T23:26:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.376+0000 7f259f734700 1 -- 172.21.15.64:0/1630949480 wait complete. 2023-09-27T23:26:45.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 Processor -- start 2023-09-27T23:26:45.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 -- start start 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25980ca320 0x7f2598069b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2598156eb0 0x7f2598158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f25980b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259806a0d0 con 0x7f259815aa20 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980c7590 con 0x7f25980ca320 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598158da0 con 0x7f2598156eb0 2023-09-27T23:26:45.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f25980b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f25980b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60996/0 (socket says 172.21.15.64:60996) 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 -- 172.21.15.64:0/3786375906 learned_addr learned my addr 172.21.15.64:0/3786375906 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259d4d0700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25980ca320 0x7f2598069b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259cccf700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2598156eb0 0x7f2598158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 -- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2598156eb0 msgr2=0x7f2598158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2598156eb0 0x7f2598158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 -- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25980ca320 msgr2=0x7f2598069b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25980ca320 0x7f2598069b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:45.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 -- 172.21.15.64:0/3786375906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588009420 con 0x7f259815aa20 2023-09-27T23:26:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259dcd1700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f25980b1760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f259806e130 tx=0x7f2588017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258801f070 con 0x7f259815aa20 2023-09-27T23:26:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258800baa0 con 0x7f259815aa20 2023-09-27T23:26:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588025670 con 0x7f259815aa20 2023-09-27T23:26:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2598159080 con 0x7f259815aa20 2023-09-27T23:26:45.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.380+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2598159640 con 0x7f259815aa20 2023-09-27T23:26:45.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.384+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f2588023070 con 0x7f259815aa20 2023-09-27T23:26:45.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.384+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25980c7700 con 0x7f259815aa20 2023-09-27T23:26:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.384+0000 7f25957fa700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f25840607d0 0x7f2584062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.384+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7653+0+0 (secure 0 0 0) 0x7f258807e660 con 0x7f259815aa20 2023-09-27T23:26:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.384+0000 7f259d4d0700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f25840607d0 0x7f2584062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:45.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.388+0000 7f259d4d0700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f25840607d0 0x7f2584062c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2590007fd0 tx=0x7f25900016f0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:45.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.392+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f258804e550 con 0x7f259815aa20 2023-09-27T23:26:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:45.592+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2598071120 con 0x7f259815aa20 2023-09-27T23:26:46.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.244+0000 7f25957fa700 1 -- 172.21.15.64:0/3786375906 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2588051b70 con 0x7f259815aa20 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f25840607d0 msgr2=0x7f2584062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f25840607d0 0x7f2584062c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2590007fd0 tx=0x7f25900016f0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 msgr2=0x7f25980b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f25980b1760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f259806e130 tx=0x7f2588017000 comp rx=0 tx=0).stop 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 shutdown_connections 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f25840607d0 0x7f2584062c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f259815aa20 0x7f25980b1760 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2598156eb0 0x7f2598158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 --2- 172.21.15.64:0/3786375906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25980ca320 0x7f2598069b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 >> 172.21.15.64:0/3786375906 conn(0x7f25980cd8f0 msgr2=0x7f25980b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:46.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.248+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 shutdown_connections 2023-09-27T23:26:46.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.252+0000 7f259f734700 1 -- 172.21.15.64:0/3786375906 wait complete. 2023-09-27T23:26:46.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:26:46.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.392+0000 7f597b6be700 1 Processor -- start 2023-09-27T23:26:46.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.392+0000 7f597b6be700 1 -- start start 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59740cb500 0x7f597415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59740b0ac0 con 0x7f59740cb130 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597406b210 con 0x7f597415c760 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597415cbe0 con 0x7f59740cb500 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f5978c59700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59740cb500 0x7f597415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597945a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f59740cb130 0x7f597406a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57162/0 (socket says 172.21.15.64:57162) 2023-09-27T23:26:46.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597945a700 1 -- 172.21.15.64:0/1775736639 learned_addr learned my addr 172.21.15.64:0/1775736639 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f5978c59700 1 -- 172.21.15.64:0/1775736639 >> v1:172.21.15.123:6789/0 conn(0x7f597415c760 legacy=0x7f59740b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f5978c59700 1 -- 172.21.15.64:0/1775736639 >> v1:172.21.15.64:6789/0 conn(0x7f59740cb130 legacy=0x7f597406a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f5978c59700 1 -- 172.21.15.64:0/1775736639 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740ca810 con 0x7f59740cb500 2023-09-27T23:26:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f5978c59700 1 --2- 172.21.15.64:0/1775736639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb500 0x7f597415c220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59700098e0 tx=0x7f59700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bc5c9fbdd89070b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f596bfff700 1 -- 172.21.15.64:0/1775736639 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970009e80 con 0x7f59740cb500 2023-09-27T23:26:46.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f596bfff700 1 -- 172.21.15.64:0/1775736639 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597000b860 con 0x7f59740cb500 2023-09-27T23:26:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f596bfff700 1 -- 172.21.15.64:0/1775736639 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597000bbe0 con 0x7f59740cb500 2023-09-27T23:26:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- 172.21.15.64:0/1775736639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb500 msgr2=0x7f597415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 --2- 172.21.15.64:0/1775736639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb500 0x7f597415c220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59700098e0 tx=0x7f59700091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- 172.21.15.64:0/1775736639 shutdown_connections 2023-09-27T23:26:46.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 --2- 172.21.15.64:0/1775736639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb500 0x7f597415c220 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- 172.21.15.64:0/1775736639 >> 172.21.15.64:0/1775736639 conn(0x7f5974067bb0 msgr2=0x7f59740cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:46.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- 172.21.15.64:0/1775736639 shutdown_connections 2023-09-27T23:26:46.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.396+0000 7f597b6be700 1 -- 172.21.15.64:0/1775736639 wait complete. 2023-09-27T23:26:46.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 Processor -- start 2023-09-27T23:26:46.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 -- start start 2023-09-27T23:26:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb130 0x7f59741517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59740cb500 0x7f597414ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 0x7f597414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5974159690 con 0x7f597415c760 2023-09-27T23:26:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5974159970 con 0x7f59740cb130 2023-09-27T23:26:46.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597b6be700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59740b55a0 con 0x7f59740cb500 2023-09-27T23:26:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 0x7f597414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 0x7f597414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32776/0 (socket says 172.21.15.64:32776) 2023-09-27T23:26:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 -- 172.21.15.64:0/1479760511 learned_addr learned my addr 172.21.15.64:0/1479760511 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5978c59700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59740cb500 0x7f597414ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:46.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f597945a700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb130 0x7f59741517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 -- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59740cb500 msgr2=0x7f597414ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59740cb500 0x7f597414ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 -- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb130 msgr2=0x7f59741517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb130 0x7f59741517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 -- 172.21.15.64:0/1479760511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009420 con 0x7f597415c760 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.400+0000 7f5979c5b700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 0x7f597414ef70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c840 tx=0x7f596c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:46.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c012630 con 0x7f597415c760 2023-09-27T23:26:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c0127b0 con 0x7f597415c760 2023-09-27T23:26:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59740b5880 con 0x7f597415c760 2023-09-27T23:26:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c012b70 con 0x7f597415c760 2023-09-27T23:26:46.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5974151ce0 con 0x7f597415c760 2023-09-27T23:26:46.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597406a280 con 0x7f597415c760 2023-09-27T23:26:46.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.404+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f596c01a3a0 con 0x7f597415c760 2023-09-27T23:26:46.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.408+0000 7f59697fa700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59580607d0 0x7f5958062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:46.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.408+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7653+0+0 (secure 0 0 0) 0x7f596c07d290 con 0x7f597415c760 2023-09-27T23:26:46.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.408+0000 7f597945a700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59580607d0 0x7f5958062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:46.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.408+0000 7f597945a700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59580607d0 0x7f5958062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5964007130 tx=0x7f5964007eb0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.416+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f597406a280 con 0x7f597415c760 2023-09-27T23:26:46.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.580+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f597406a280 con 0x7f597415c760 2023-09-27T23:26:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:46.764+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f596c0560d0 con 0x7f597415c760 2023-09-27T23:26:47.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.248+0000 7f59697fa700 1 -- 172.21.15.64:0/1479760511 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) v1 ==== 181+0+0 (secure 0 0 0) 0x7f596c050c10 con 0x7f597415c760 2023-09-27T23:26:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.252+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59580607d0 msgr2=0x7f5958062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.252+0000 7f597b6be700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59580607d0 0x7f5958062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5964007130 tx=0x7f5964007eb0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.252+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 msgr2=0x7f597414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:47.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.252+0000 7f597b6be700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 0x7f597414ef70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c840 tx=0x7f596c001670 comp rx=0 tx=0).stop 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 shutdown_connections 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59580607d0 0x7f5958062c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f597415c760 0x7f597414ef70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59740cb500 0x7f597414ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 --2- 172.21.15.64:0/1479760511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59740cb130 0x7f59741517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 >> 172.21.15.64:0/1479760511 conn(0x7f5974067bb0 msgr2=0x7f59740cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:47.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 shutdown_connections 2023-09-27T23:26:47.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.256+0000 7f597b6be700 1 -- 172.21.15.64:0/1479760511 wait complete. 2023-09-27T23:26:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:26:47.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:26:47.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 Processor -- start 2023-09-27T23:26:47.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 -- start start 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0af190 0x7fdf3c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf3c0cac50 0x7fdf3c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0cb3f0 con 0x7fdf3c0aedc0 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0cc140 con 0x7fdf3c0cac50 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf409f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0af5e0 con 0x7fdf3c0af190 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf3bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf3c0cac50 0x7fdf3c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf3bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf3c0cac50 0x7fdf3c0af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32786/0 (socket says 172.21.15.64:32786) 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf3b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdf3c0aedc0 0x7fdf3c06d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57168/0 (socket says 172.21.15.64:57168) 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf3bfff700 1 -- 172.21.15.64:0/1574647295 learned_addr learned my addr 172.21.15.64:0/1574647295 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.384+0000 7fdf32ffd700 1 --2- 172.21.15.64:0/1574647295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0af190 0x7fdf3c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3bfff700 1 -- 172.21.15.64:0/1574647295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0af190 msgr2=0x7fdf3c06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3bfff700 1 --2- 172.21.15.64:0/1574647295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0af190 0x7fdf3c06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3bfff700 1 -- 172.21.15.64:0/1574647295 >> v1:172.21.15.64:6789/0 conn(0x7fdf3c0aedc0 legacy=0x7fdf3c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3affd700 1 -- 172.21.15.64:0/1574647295 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225868807 0 0) 0x7fdf3c0cb3f0 con 0x7fdf3c0aedc0 2023-09-27T23:26:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3bfff700 1 -- 172.21.15.64:0/1574647295 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c06d220 con 0x7fdf3c0cac50 2023-09-27T23:26:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3bfff700 1 --2- 172.21.15.64:0/1574647295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c0af930 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009f20 tx=0x7fdf34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37a54d3bac86ab2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3affd700 1 -- 172.21.15.64:0/1574647295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400e070 con 0x7fdf3c0cac50 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3affd700 1 -- 172.21.15.64:0/1574647295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3400ba80 con 0x7fdf3c0cac50 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf3affd700 1 -- 172.21.15.64:0/1574647295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340092e0 con 0x7fdf3c0cac50 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 -- 172.21.15.64:0/1574647295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0cac50 msgr2=0x7fdf3c0af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 --2- 172.21.15.64:0/1574647295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c0af930 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009f20 tx=0x7fdf34009840 comp rx=0 tx=0).stop 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 -- 172.21.15.64:0/1574647295 shutdown_connections 2023-09-27T23:26:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 --2- 172.21.15.64:0/1574647295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c0af930 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 --2- 172.21.15.64:0/1574647295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0af190 0x7fdf3c06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 -- 172.21.15.64:0/1574647295 >> 172.21.15.64:0/1574647295 conn(0x7fdf3c0cc760 msgr2=0x7fdf3c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 -- 172.21.15.64:0/1574647295 shutdown_connections 2023-09-27T23:26:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.388+0000 7fdf409f8700 1 -- 172.21.15.64:0/1574647295 wait complete. 2023-09-27T23:26:47.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 Processor -- start 2023-09-27T23:26:47.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 -- start start 2023-09-27T23:26:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0aedc0 0x7fdf3c0b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 0x7fdf3c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c168850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0c7a60 con 0x7fdf3c0af190 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c169310 con 0x7fdf3c0cac50 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf409f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1695f0 con 0x7fdf3c0aedc0 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 0x7fdf3c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 0x7fdf3c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32790/0 (socket says 172.21.15.64:32790) 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 -- 172.21.15.64:0/366794265 learned_addr learned my addr 172.21.15.64:0/366794265 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf3b7fe700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0aedc0 0x7fdf3c0b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf3bfff700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 -- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0aedc0 msgr2=0x7fdf3c0b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0aedc0 0x7fdf3c0b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 -- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0cac50 msgr2=0x7fdf3c168850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c168850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 -- 172.21.15.64:0/366794265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009ae0 con 0x7fdf3c0af190 2023-09-27T23:26:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf3bfff700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c168850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:47.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.392+0000 7fdf32ffd700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 0x7fdf3c0b3890 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d890 tx=0x7fdf24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:47.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240125f0 con 0x7fdf3c0af190 2023-09-27T23:26:47.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24012750 con 0x7fdf3c0af190 2023-09-27T23:26:47.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24012ad0 con 0x7fdf3c0af190 2023-09-27T23:26:47.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0c7ce0 con 0x7fdf3c0af190 2023-09-27T23:26:47.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0b2fe0 con 0x7fdf3c0af190 2023-09-27T23:26:47.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c04be40 con 0x7fdf3c0af190 2023-09-27T23:26:47.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.396+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fdf2401e460 con 0x7fdf3c0af190 2023-09-27T23:26:47.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.400+0000 7fdf33fff700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf20060a70 0x7fdf20062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:47.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.400+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6643+0+0 (secure 0 0 0) 0x7fdf2407d860 con 0x7fdf3c0af190 2023-09-27T23:26:47.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.400+0000 7fdf3b7fe700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf20060a70 0x7fdf20062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:47.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.400+0000 7fdf3b7fe700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf20060a70 0x7fdf20062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c007fd0 tx=0x7fdf2c007d50 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.404+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdf2404d780 con 0x7fdf3c0af190 2023-09-27T23:26:47.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:47.564+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf3c06c8f0 con 0x7fdf3c0af190 2023-09-27T23:26:48.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.260+0000 7fdf33fff700 1 -- 172.21.15.64:0/366794265 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf24050da0 con 0x7fdf3c0af190 2023-09-27T23:26:48.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf20060a70 msgr2=0x7fdf20062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf20060a70 0x7fdf20062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c007fd0 tx=0x7fdf2c007d50 comp rx=0 tx=0).stop 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 msgr2=0x7fdf3c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 0x7fdf3c0b3890 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d890 tx=0x7fdf24008040 comp rx=0 tx=0).stop 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 shutdown_connections 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdf20060a70 0x7fdf20062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf3c0cac50 0x7fdf3c168850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf3c0af190 0x7fdf3c0b3890 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 --2- 172.21.15.64:0/366794265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf3c0aedc0 0x7fdf3c0b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 >> 172.21.15.64:0/366794265 conn(0x7fdf3c0cc760 msgr2=0x7fdf3c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:48.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 shutdown_connections 2023-09-27T23:26:48.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.268+0000 7fdf409f8700 1 -- 172.21.15.64:0/366794265 wait complete. 2023-09-27T23:26:48.276 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:26:48.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:26:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.388+0000 7ff5af7c9700 1 Processor -- start 2023-09-27T23:26:48.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.388+0000 7ff5af7c9700 1 -- start start 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5a815c8f0 0x7ff5a80a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80c6ab0 con 0x7ff5a80ca240 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a806b240 con 0x7ff5a815c520 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a806b520 con 0x7ff5a815c8f0 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5add66700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff5a80ca240 0x7ff5a80ca630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57174/0 (socket says 172.21.15.64:57174) 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5add66700 1 -- 172.21.15.64:0/3087254799 learned_addr learned my addr 172.21.15.64:0/3087254799 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:48.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5acd64700 1 --2- 172.21.15.64:0/3087254799 >> v2:172.21.15.123:3300/0 conn(0x7ff5a815c8f0 0x7ff5a80a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff59ffff700 1 -- 172.21.15.64:0/3087254799 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598258275 0 0) 0x7ff5a80c6ab0 con 0x7ff5a80ca240 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff59ffff700 1 -- 172.21.15.64:0/3087254799 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff594003570 con 0x7ff5a80ca240 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5acd64700 1 -- 172.21.15.64:0/3087254799 >> v1:172.21.15.123:6789/0 conn(0x7ff5a815c520 legacy=0x7ff5a806a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5acd64700 1 -- 172.21.15.64:0/3087254799 >> v1:172.21.15.64:6789/0 conn(0x7ff5a80ca240 legacy=0x7ff5a80ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5acd64700 1 -- 172.21.15.64:0/3087254799 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80c6d50 con 0x7ff5a815c8f0 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5acd64700 1 --2- 172.21.15.64:0/3087254799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80a8850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40098e0 tx=0x7ff5a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0dcfb5374aba14 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff59ffff700 1 -- 172.21.15.64:0/3087254799 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400b6d0 con 0x7ff5a815c8f0 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff59ffff700 1 -- 172.21.15.64:0/3087254799 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a400b830 con 0x7ff5a815c8f0 2023-09-27T23:26:48.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff59ffff700 1 -- 172.21.15.64:0/3087254799 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400bbd0 con 0x7ff5a815c8f0 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 -- 172.21.15.64:0/3087254799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 msgr2=0x7ff5a80a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/3087254799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80a8850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40098e0 tx=0x7ff5a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 -- 172.21.15.64:0/3087254799 shutdown_connections 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/3087254799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80a8850 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.392+0000 7ff5af7c9700 1 -- 172.21.15.64:0/3087254799 >> 172.21.15.64:0/3087254799 conn(0x7ff5a8067bb0 msgr2=0x7ff5a80cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.396+0000 7ff5af7c9700 1 -- 172.21.15.64:0/3087254799 shutdown_connections 2023-09-27T23:26:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.396+0000 7ff5af7c9700 1 -- 172.21.15.64:0/3087254799 wait complete. 2023-09-27T23:26:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.396+0000 7ff5af7c9700 1 Processor -- start 2023-09-27T23:26:48.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.396+0000 7ff5af7c9700 1 -- start start 2023-09-27T23:26:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 0x7ff5a8156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5a815c520 0x7ff5a814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b1320 con 0x7ff5a80ca240 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8157400 con 0x7ff5a815c8f0 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a81567f0 con 0x7ff5a815c520 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 0x7ff5a8156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 0x7ff5a8156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32794/0 (socket says 172.21.15.64:32794) 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5acd64700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5a815c520 0x7ff5a814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 -- 172.21.15.64:0/545650567 learned_addr learned my addr 172.21.15.64:0/545650567 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5add66700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 -- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5a815c520 msgr2=0x7ff5a814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5a815c520 0x7ff5a814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 -- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 msgr2=0x7ff5a80b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 -- 172.21.15.64:0/545650567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4009420 con 0x7ff5a80ca240 2023-09-27T23:26:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5ad565700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 0x7ff5a8156160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c590 tx=0x7ff59800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598011590 con 0x7ff5a80ca240 2023-09-27T23:26:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598011710 con 0x7ff5a80ca240 2023-09-27T23:26:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a8156ad0 con 0x7ff5a80ca240 2023-09-27T23:26:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598011ad0 con 0x7ff5a80ca240 2023-09-27T23:26:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8157090 con 0x7ff5a80ca240 2023-09-27T23:26:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff59801b3b0 con 0x7ff5a80ca240 2023-09-27T23:26:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.400+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a8071120 con 0x7ff5a80ca240 2023-09-27T23:26:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.404+0000 7ff59d7fa700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff58c060a70 0x7ff58c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.404+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7ff59807c4c0 con 0x7ff5a80ca240 2023-09-27T23:26:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.404+0000 7ff5acd64700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff58c060a70 0x7ff58c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.404+0000 7ff5acd64700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff58c060a70 0x7ff58c062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40093f0 tx=0x7ff5a4017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:48.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.408+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff59804c5c0 con 0x7ff5a80ca240 2023-09-27T23:26:48.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.572+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff5a80ca760 con 0x7ff5a80ca240 2023-09-27T23:26:48.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff59d7fa700 1 -- 172.21.15.64:0/545650567 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v13) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff5980170b0 con 0x7ff5a80ca240 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff58c060a70 msgr2=0x7ff58c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff58c060a70 0x7ff58c062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40093f0 tx=0x7ff5a4017000 comp rx=0 tx=0).stop 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 msgr2=0x7ff5a8156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 0x7ff5a8156160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c590 tx=0x7ff59800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 shutdown_connections 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff58c060a70 0x7ff58c062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a815c8f0 0x7ff5a80b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5a815c520 0x7ff5a814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 --2- 172.21.15.64:0/545650567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a80ca240 0x7ff5a8156160 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 >> 172.21.15.64:0/545650567 conn(0x7ff5a8067bb0 msgr2=0x7ff5a80c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:48.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 shutdown_connections 2023-09-27T23:26:48.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.584+0000 7ff5af7c9700 1 -- 172.21.15.64:0/545650567 wait complete. 2023-09-27T23:26:48.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:26:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 Processor -- start 2023-09-27T23:26:48.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 -- start start 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9d8150200 0x7fa9d80c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d80aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d80af250 con 0x7fa9d80ae770 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d80af530 con 0x7fa9d8150200 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dda00700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80ca8a0 con 0x7fa9d80ace30 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9d8150200 0x7fa9d80c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9d8150200 0x7fa9d80c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32798/0 (socket says 172.21.15.64:32798) 2023-09-27T23:26:48.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 -- 172.21.15.64:0/159976898 learned_addr learned my addr 172.21.15.64:0/159976898 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dd1ff700 1 --2- 172.21.15.64:0/159976898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d80aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 -- 172.21.15.64:0/159976898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 msgr2=0x7fa9d80aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 --2- 172.21.15.64:0/159976898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d80aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 -- 172.21.15.64:0/159976898 >> v1:172.21.15.123:6789/0 conn(0x7fa9d80ae770 legacy=0x7fa9d80ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 -- 172.21.15.64:0/159976898 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80cab80 con 0x7fa9d8150200 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.716+0000 7fa9dc9fe700 1 --2- 172.21.15.64:0/159976898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d8150200 0x7fa9d80c7170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009f20 tx=0x7fa9c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438e69d0e75193a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9d77fe700 1 -- 172.21.15.64:0/159976898 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800e070 con 0x7fa9d8150200 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9d77fe700 1 -- 172.21.15.64:0/159976898 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c800ba80 con 0x7fa9d8150200 2023-09-27T23:26:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9d77fe700 1 -- 172.21.15.64:0/159976898 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80092e0 con 0x7fa9d8150200 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 -- 172.21.15.64:0/159976898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d8150200 msgr2=0x7fa9d80c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 --2- 172.21.15.64:0/159976898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d8150200 0x7fa9d80c7170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009f20 tx=0x7fa9c8009840 comp rx=0 tx=0).stop 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 -- 172.21.15.64:0/159976898 shutdown_connections 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 --2- 172.21.15.64:0/159976898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d80aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 --2- 172.21.15.64:0/159976898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d8150200 0x7fa9d80c7170 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 -- 172.21.15.64:0/159976898 >> 172.21.15.64:0/159976898 conn(0x7fa9d80ccb30 msgr2=0x7fa9d80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:48.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 -- 172.21.15.64:0/159976898 shutdown_connections 2023-09-27T23:26:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 -- 172.21.15.64:0/159976898 wait complete. 2023-09-27T23:26:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 Processor -- start 2023-09-27T23:26:48.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.720+0000 7fa9dda00700 1 -- start start 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 0x7fa9d81659b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8150200 0x7fa9d813a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d814ca30 con 0x7fa9d80ae770 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8165ef0 con 0x7fa9d8150200 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d814d950 con 0x7fa9d80ace30 2023-09-27T23:26:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 0x7fa9d81659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 0x7fa9d81659b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32800/0 (socket says 172.21.15.64:32800) 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 -- 172.21.15.64:0/1576645525 learned_addr learned my addr 172.21.15.64:0/1576645525 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dc9fe700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dd1ff700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8150200 0x7fa9d813a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 -- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 msgr2=0x7fa9d814c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d814c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 -- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8150200 msgr2=0x7fa9d813a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8150200 0x7fa9d813a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 -- 172.21.15.64:0/1576645525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c8009ae0 con 0x7fa9d80ae770 2023-09-27T23:26:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d7fff700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 0x7fa9d81659b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c590 tx=0x7fa9cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc011590 con 0x7fa9d80ae770 2023-09-27T23:26:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0116f0 con 0x7fa9d80ae770 2023-09-27T23:26:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d813ac60 con 0x7fa9d80ae770 2023-09-27T23:26:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc011ab0 con 0x7fa9d80ae770 2023-09-27T23:26:48.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d814d300 con 0x7fa9d80ae770 2023-09-27T23:26:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d814dae0 con 0x7fa9d80ae770 2023-09-27T23:26:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.724+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa9cc01b3b0 con 0x7fa9d80ae770 2023-09-27T23:26:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.728+0000 7fa9d4ff9700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9c4060a70 0x7fa9c4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.728+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fa9cc07c6b0 con 0x7fa9d80ae770 2023-09-27T23:26:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.728+0000 7fa9dc9fe700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9c4060a70 0x7fa9c4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:48.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.728+0000 7fa9dc9fe700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9c4060a70 0x7fa9c4062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009880 tx=0x7fa9c8018000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:48.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.736+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa9cc04c710 con 0x7fa9d80ae770 2023-09-27T23:26:48.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.896+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa9d804be40 con 0x7fa9d80ae770 2023-09-27T23:26:48.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.900+0000 7fa9d4ff9700 1 -- 172.21.15.64:0/1576645525 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa9cc04fd30 con 0x7fa9d80ae770 2023-09-27T23:26:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9c4060a70 msgr2=0x7fa9c4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9c4060a70 0x7fa9c4062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009880 tx=0x7fa9c8018000 comp rx=0 tx=0).stop 2023-09-27T23:26:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 msgr2=0x7fa9d81659b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 0x7fa9d81659b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c590 tx=0x7fa9cc00c950 comp rx=0 tx=0).stop 2023-09-27T23:26:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 shutdown_connections 2023-09-27T23:26:48.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa9c4060a70 0x7fa9c4062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8150200 0x7fa9d813a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d80ae770 0x7fa9d81659b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 --2- 172.21.15.64:0/1576645525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d80ace30 0x7fa9d814c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:48.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 >> 172.21.15.64:0/1576645525 conn(0x7fa9d80ccb30 msgr2=0x7fa9d80c22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:48.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 shutdown_connections 2023-09-27T23:26:48.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:48.904+0000 7fa9dda00700 1 -- 172.21.15.64:0/1576645525 wait complete. 2023-09-27T23:26:48.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:26:49.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.028+0000 7f115759e700 1 Processor -- start 2023-09-27T23:26:49.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.028+0000 7f115759e700 1 -- start start 2023-09-27T23:26:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.028+0000 7f115759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f115806d760 0x7f11580ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.028+0000 7f115759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580cc7e0 con 0x7f1158142680 2023-09-27T23:26:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.028+0000 7f115759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580cbbf0 con 0x7f1158142a50 2023-09-27T23:26:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.028+0000 7f115759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580cbeb0 con 0x7f115806d760 2023-09-27T23:26:49.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1158142680 0x7f11580c8a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57186/0 (socket says 172.21.15.64:57186) 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115659c700 1 -- 172.21.15.64:0/2338234569 learned_addr learned my addr 172.21.15.64:0/2338234569 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f1156d9d700 1 --2- 172.21.15.64:0/2338234569 >> v2:172.21.15.123:3300/0 conn(0x7f115806d760 0x7f11580ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307855359 0 0) 0x7f11580cc7e0 con 0x7f1158142680 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1140003570 con 0x7f1158142680 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004044265 0 0) 0x7f11580cbbf0 con 0x7f1158142a50 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11580cc7e0 con 0x7f1158142a50 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 770900631 0 0) 0x7f1140003570 con 0x7f1158142680 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f1156d9d700 1 -- 172.21.15.64:0/2338234569 >> v1:172.21.15.123:6789/0 conn(0x7f1158142a50 legacy=0x7f115806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f1156d9d700 1 -- 172.21.15.64:0/2338234569 >> v1:172.21.15.64:6789/0 conn(0x7f1158142680 legacy=0x7f11580c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f1156d9d700 1 -- 172.21.15.64:0/2338234569 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580cc190 con 0x7f115806d760 2023-09-27T23:26:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f1156d9d700 1 --2- 172.21.15.64:0/2338234569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f115806d760 0x7f11580ca7e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1150009e60 tx=0x7f1150009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7812eda46515a276 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:49.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000e070 con 0x7f115806d760 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000b7a0 con 0x7f115806d760 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115559a700 1 -- 172.21.15.64:0/2338234569 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148001470 con 0x7f115806d760 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 -- 172.21.15.64:0/2338234569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f115806d760 msgr2=0x7f11580ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 --2- 172.21.15.64:0/2338234569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f115806d760 0x7f11580ca7e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1150009e60 tx=0x7f1150009780 comp rx=0 tx=0).stop 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 -- 172.21.15.64:0/2338234569 shutdown_connections 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 --2- 172.21.15.64:0/2338234569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f115806d760 0x7f11580ca7e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 -- 172.21.15.64:0/2338234569 >> 172.21.15.64:0/2338234569 conn(0x7f11580ccb30 msgr2=0x7f11580ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 -- 172.21.15.64:0/2338234569 shutdown_connections 2023-09-27T23:26:49.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 -- 172.21.15.64:0/2338234569 wait complete. 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 Processor -- start 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.032+0000 7f115759e700 1 -- start start 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 0x7f11581586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1158142680 0x7f11580670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1158142a50 0x7f11580aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158169990 con 0x7f1158142680 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580a9430 con 0x7f1158142a50 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580a9710 con 0x7f115806d760 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 0x7f11581586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 0x7f11581586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:54750/0 (socket says 172.21.15.64:54750) 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 -- 172.21.15.64:0/3312987064 learned_addr learned my addr 172.21.15.64:0/3312987064 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f1155d9b700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1158142680 0x7f11580670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f1156d9d700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1158142a50 0x7f11580aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 -- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1158142a50 msgr2=0x7f11580aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1158142a50 0x7f11580aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 -- 172.21.15.64:0/3312987064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1158142680 msgr2=0x7f11580670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1158142680 0x7f11580670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 -- 172.21.15.64:0/3312987064 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150009a20 con 0x7f115806d760 2023-09-27T23:26:49.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115659c700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 0x7f11581586d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f114800cb90 tx=0x7f1148001ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:49.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148007150 con 0x7f115806d760 2023-09-27T23:26:49.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11480072d0 con 0x7f115806d760 2023-09-27T23:26:49.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11580a9cd0 con 0x7f115806d760 2023-09-27T23:26:49.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148007690 con 0x7f115806d760 2023-09-27T23:26:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11580aa290 con 0x7f115806d760 2023-09-27T23:26:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11580aa4b0 con 0x7f115806d760 2023-09-27T23:26:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.036+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1148004a40 con 0x7f115806d760 2023-09-27T23:26:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.040+0000 7f11467fc700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1138060a70 0x7f1138062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.040+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f114807c420 con 0x7f115806d760 2023-09-27T23:26:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.040+0000 7f1155d9b700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1138060a70 0x7f1138062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.040+0000 7f1155d9b700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1138060a70 0x7f1138062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11580b2190 tx=0x7f114c007d50 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:49.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.048+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f114804c520 con 0x7f115806d760 2023-09-27T23:26:49.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:26:49.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:26:49.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.208+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f115804be40 con 0x7f115806d760 2023-09-27T23:26:49.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f11467fc700 1 -- 172.21.15.64:0/3312987064 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f114804fb40 con 0x7f115806d760 2023-09-27T23:26:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1138060a70 msgr2=0x7f1138062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1138060a70 0x7f1138062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11580b2190 tx=0x7f114c007d50 comp rx=0 tx=0).stop 2023-09-27T23:26:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 msgr2=0x7f11581586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 0x7f11581586d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f114800cb90 tx=0x7f1148001ad0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 shutdown_connections 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1138060a70 0x7f1138062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1158142a50 0x7f11580aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1158142680 0x7f11580670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 --2- 172.21.15.64:0/3312987064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f115806d760 0x7f11581586d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 >> 172.21.15.64:0/3312987064 conn(0x7f11580ccb30 msgr2=0x7f115813c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 shutdown_connections 2023-09-27T23:26:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.224+0000 7f115759e700 1 -- 172.21.15.64:0/3312987064 wait complete. 2023-09-27T23:26:49.245 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-09-27T23:26:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.344+0000 7f1b12dc1700 1 Processor -- start 2023-09-27T23:26:49.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.344+0000 7f1b12dc1700 1 -- start start 2023-09-27T23:26:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b12dc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c06a540 0x7f1b0c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b12dc1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 0x7f1b0c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b12dc1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0ca5a0 con 0x7f1b0c158e70 2023-09-27T23:26:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b12dc1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c06dac0 con 0x7f1b0c0cc470 2023-09-27T23:26:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b12dc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c06d220 con 0x7f1b0c06a540 2023-09-27T23:26:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b125c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 0x7f1b0c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b115be700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c06a540 0x7f1b0c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b125c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 0x7f1b0c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32818/0 (socket says 172.21.15.64:32818) 2023-09-27T23:26:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b11dbf700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1b0c158e70 0x7f1b0c0afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:38802/0 (socket says 172.21.15.64:38802) 2023-09-27T23:26:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b125c0700 1 -- 172.21.15.64:0/1740357250 learned_addr learned my addr 172.21.15.64:0/1740357250 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b115be700 1 -- 172.21.15.64:0/1740357250 >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 msgr2=0x7f1b0c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b10dbd700 1 -- 172.21.15.64:0/1740357250 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229569178 0 0) 0x7f1b0c0ca5a0 con 0x7f1b0c158e70 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b115be700 1 --2- 172.21.15.64:0/1740357250 >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 0x7f1b0c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b115be700 1 -- 172.21.15.64:0/1740357250 >> v1:172.21.15.123:6789/0 conn(0x7f1b0c158e70 legacy=0x7f1b0c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b115be700 1 -- 172.21.15.64:0/1740357250 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c06d500 con 0x7f1b0c06a540 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b125c0700 1 --2- 172.21.15.64:0/1740357250 >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 0x7f1b0c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b115be700 1 --2- 172.21.15.64:0/1740357250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c06a540 0x7f1b0c06c9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b080098e0 tx=0x7f1b080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2182066238daa3ff server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:49.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b10dbd700 1 -- 172.21.15.64:0/1740357250 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800b640 con 0x7f1b0c06a540 2023-09-27T23:26:49.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b10dbd700 1 -- 172.21.15.64:0/1740357250 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0800b7a0 con 0x7f1b0c06a540 2023-09-27T23:26:49.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.348+0000 7f1b10dbd700 1 -- 172.21.15.64:0/1740357250 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800bb40 con 0x7f1b0c06a540 2023-09-27T23:26:49.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- 172.21.15.64:0/1740357250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c06a540 msgr2=0x7f1b0c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/1740357250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c06a540 0x7f1b0c06c9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b080098e0 tx=0x7f1b080091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- 172.21.15.64:0/1740357250 shutdown_connections 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/1740357250 >> v2:172.21.15.64:3300/0 conn(0x7f1b0c0cc470 0x7f1b0c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/1740357250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c06a540 0x7f1b0c06c9d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- 172.21.15.64:0/1740357250 >> 172.21.15.64:0/1740357250 conn(0x7f1b0c0670e0 msgr2=0x7f1b0c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- 172.21.15.64:0/1740357250 shutdown_connections 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- 172.21.15.64:0/1740357250 wait complete. 2023-09-27T23:26:49.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 Processor -- start 2023-09-27T23:26:49.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- start start 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 0x7f1b0c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c0cc470 0x7f1b0c1490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0c158e70 0x7f1b0c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c156bb0 con 0x7f1b0c06a540 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0c7b20 con 0x7f1b0c158e70 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b12dc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0c6f10 con 0x7f1b0c0cc470 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b11dbf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 0x7f1b0c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b11dbf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 0x7f1b0c15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32832/0 (socket says 172.21.15.64:32832) 2023-09-27T23:26:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.352+0000 7f1b11dbf700 1 -- 172.21.15.64:0/2579624755 learned_addr learned my addr 172.21.15.64:0/2579624755 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b115be700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c0cc470 0x7f1b0c1490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b125c0700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0c158e70 0x7f1b0c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b11dbf700 1 -- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c0cc470 msgr2=0x7f1b0c1490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b11dbf700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c0cc470 0x7f1b0c1490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b11dbf700 1 -- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0c158e70 msgr2=0x7f1b0c156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b11dbf700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0c158e70 0x7f1b0c156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b11dbf700 1 -- 172.21.15.64:0/2579624755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009ae0 con 0x7f1b0c06a540 2023-09-27T23:26:49.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b125c0700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0c158e70 0x7f1b0c156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b11dbf700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 0x7f1b0c15bac0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0011e0 tx=0x7f1afc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005a70 con 0x7f1b0c06a540 2023-09-27T23:26:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc005bd0 con 0x7f1b0c06a540 2023-09-27T23:26:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b08009420 con 0x7f1b0c06a540 2023-09-27T23:26:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0070d0 con 0x7f1b0c06a540 2023-09-27T23:26:49.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0c7550 con 0x7f1b0c06a540 2023-09-27T23:26:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c04be40 con 0x7f1b0c06a540 2023-09-27T23:26:49.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.356+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1afc007480 con 0x7f1b0c06a540 2023-09-27T23:26:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.360+0000 7f1b01ffb700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1af0060a70 0x7f1af0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.360+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f1afc07c480 con 0x7f1b0c06a540 2023-09-27T23:26:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.360+0000 7f1b115be700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1af0060a70 0x7f1af0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.360+0000 7f1b115be700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1af0060a70 0x7f1af0062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800a010 tx=0x7f1b08017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:49.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.364+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1afc049bb0 con 0x7f1b0c06a540 2023-09-27T23:26:49.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:49.596+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f1b0c0c7870 con 0x7f1b0c06a540 2023-09-27T23:26:50.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.280+0000 7f1b01ffb700 1 -- 172.21.15.64:0/2579624755 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v470) v1 ==== 158+0+0 (secure 0 0 0) 0x7f1afc04c630 con 0x7f1b0c06a540 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1af0060a70 msgr2=0x7f1af0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1af0060a70 0x7f1af0062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800a010 tx=0x7f1b08017000 comp rx=0 tx=0).stop 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 msgr2=0x7f1b0c15bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 0x7f1b0c15bac0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0011e0 tx=0x7f1afc007e10 comp rx=0 tx=0).stop 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 shutdown_connections 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1af0060a70 0x7f1af0062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0c158e70 0x7f1b0c156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b0c0cc470 0x7f1b0c1490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 --2- 172.21.15.64:0/2579624755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c06a540 0x7f1b0c15bac0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 >> 172.21.15.64:0/2579624755 conn(0x7f1b0c0670e0 msgr2=0x7f1b0c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 shutdown_connections 2023-09-27T23:26:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.284+0000 7f1b12dc1700 1 -- 172.21.15.64:0/2579624755 wait complete. 2023-09-27T23:26:50.304 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-09-27T23:26:50.304 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-09-27T23:26:50.304 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:26:50.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-09-27T23:26:50.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.408+0000 7f3567f1c700 1 Processor -- start 2023-09-27T23:26:50.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.408+0000 7f3567f1c700 1 -- start start 2023-09-27T23:26:50.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f356015ad00 0x7f356015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:50.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35600cd1c0 con 0x7f35600cdaa0 2023-09-27T23:26:50.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35600cc5b0 con 0x7f35600cd650 2023-09-27T23:26:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35600cc890 con 0x7f356015ad00 2023-09-27T23:26:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f35664b9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f35600cdaa0 0x7f3560156a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57198/0 (socket says 172.21.15.64:57198) 2023-09-27T23:26:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f35664b9700 1 -- 172.21.15.64:0/2939936988 learned_addr learned my addr 172.21.15.64:0/2939936988 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3565cb8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f356015ad00 0x7f356015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3564cb6700 1 -- 172.21.15.64:0/2939936988 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756315684 0 0) 0x7f35600cd1c0 con 0x7f35600cdaa0 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3564cb6700 1 -- 172.21.15.64:0/2939936988 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f354c003570 con 0x7f35600cdaa0 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3565cb8700 1 -- 172.21.15.64:0/2939936988 >> v1:172.21.15.123:6789/0 conn(0x7f35600cd650 legacy=0x7f35600c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3565cb8700 1 -- 172.21.15.64:0/2939936988 >> v1:172.21.15.64:6789/0 conn(0x7f35600cdaa0 legacy=0x7f3560156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3565cb8700 1 -- 172.21.15.64:0/2939936988 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600ccb70 con 0x7f356015ad00 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3565cb8700 1 --2- 172.21.15.64:0/2939936988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f356015b0d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f355c0098e0 tx=0x7f355c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fec29ef13ca79b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:50.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3564cb6700 1 -- 172.21.15.64:0/2939936988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00b640 con 0x7f356015ad00 2023-09-27T23:26:50.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3564cb6700 1 -- 172.21.15.64:0/2939936988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c00b7a0 con 0x7f356015ad00 2023-09-27T23:26:50.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3564cb6700 1 -- 172.21.15.64:0/2939936988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00bb20 con 0x7f356015ad00 2023-09-27T23:26:50.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- 172.21.15.64:0/2939936988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 msgr2=0x7f356015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 --2- 172.21.15.64:0/2939936988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f356015b0d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f355c0098e0 tx=0x7f355c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- 172.21.15.64:0/2939936988 shutdown_connections 2023-09-27T23:26:50.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 --2- 172.21.15.64:0/2939936988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f356015b0d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- 172.21.15.64:0/2939936988 >> 172.21.15.64:0/2939936988 conn(0x7f35600ce000 msgr2=0x7f35600ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:50.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.412+0000 7f3567f1c700 1 -- 172.21.15.64:0/2939936988 shutdown_connections 2023-09-27T23:26:50.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.416+0000 7f3567f1c700 1 -- 172.21.15.64:0/2939936988 wait complete. 2023-09-27T23:26:50.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.416+0000 7f3567f1c700 1 Processor -- start 2023-09-27T23:26:50.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.416+0000 7f3567f1c700 1 -- start start 2023-09-27T23:26:50.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600cd650 0x7f35600c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35600cdaa0 0x7f35600c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f35601673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356016aa80 con 0x7f356015ad00 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600c55e0 con 0x7f35600cdaa0 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35600c58c0 con 0x7f35600cd650 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f35601673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f35601673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32848/0 (socket says 172.21.15.64:32848) 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 -- 172.21.15.64:0/1683669204 learned_addr learned my addr 172.21.15.64:0/1683669204 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35654b7700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35600cdaa0 0x7f35600c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3565cb8700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600cd650 0x7f35600c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:50.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 -- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600cd650 msgr2=0x7f35600c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600cd650 0x7f35600c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 -- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35600cdaa0 msgr2=0x7f35600c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35600cdaa0 0x7f35600c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 -- 172.21.15.64:0/1683669204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c009420 con 0x7f356015ad00 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f35664b9700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f35601673b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3558001460 tx=0x7f3558001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800ce80 con 0x7f356015ad00 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355800ab70 con 0x7f356015ad00 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35600c5ba0 con 0x7f356015ad00 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580070f0 con 0x7f356015ad00 2023-09-27T23:26:50.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35600c6f80 con 0x7f356015ad00 2023-09-27T23:26:50.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356016abf0 con 0x7f356015ad00 2023-09-27T23:26:50.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.420+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3558007520 con 0x7f356015ad00 2023-09-27T23:26:50.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.424+0000 7f3555ffb700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060a70 0x7f3544062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:50.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.424+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f355807d590 con 0x7f356015ad00 2023-09-27T23:26:50.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.424+0000 7f3565cb8700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060a70 0x7f3544062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:50.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.424+0000 7f3565cb8700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060a70 0x7f3544062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f355c0093f0 tx=0x7f355c017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:50.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.428+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f356016abf0 con 0x7f356015ad00 2023-09-27T23:26:50.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:50.644+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f356016abc0 con 0x7f356015ad00 2023-09-27T23:26:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.284+0000 7f3555ffb700 1 -- 172.21.15.64:0/1683669204 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v471) v1 ==== 157+0+0 (secure 0 0 0) 0x7f355804d680 con 0x7f356015ad00 2023-09-27T23:26:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060a70 msgr2=0x7f3544062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060a70 0x7f3544062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f355c0093f0 tx=0x7f355c017000 comp rx=0 tx=0).stop 2023-09-27T23:26:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 msgr2=0x7f35601673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f35601673b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3558001460 tx=0x7f3558001ae0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 shutdown_connections 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3544060a70 0x7f3544062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f356015ad00 0x7f35601673b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35600cdaa0 0x7f35600c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 --2- 172.21.15.64:0/1683669204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35600cd650 0x7f35600c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 >> 172.21.15.64:0/1683669204 conn(0x7f35600ce000 msgr2=0x7f3560150740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 shutdown_connections 2023-09-27T23:26:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.288+0000 7f3567f1c700 1 -- 172.21.15.64:0/1683669204 wait complete. 2023-09-27T23:26:51.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:26:51.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.428+0000 7f8d77193700 1 Processor -- start 2023-09-27T23:26:51.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.428+0000 7f8d77193700 1 -- start start 2023-09-27T23:26:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d70158e20 0x7f8d700ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700cb970 0x7f8d700ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700cbdf0 con 0x7f8d700ca2c0 2023-09-27T23:26:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700cc0d0 con 0x7f8d70158e20 2023-09-27T23:26:51.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700cc3b0 con 0x7f8d700cb970 2023-09-27T23:26:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d6ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700cb970 0x7f8d700ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d70158e20 0x7f8d700ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d70158e20 0x7f8d700ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:43678/0 (socket says 172.21.15.64:43678) 2023-09-27T23:26:51.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 -- 172.21.15.64:0/598253783 learned_addr learned my addr 172.21.15.64:0/598253783 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d75730700 1 --1- 172.21.15.64:0/598253783 >> v1:172.21.15.123:6789/0 conn(0x7f8d700ca2c0 0x7f8d7006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:38824/0 (socket says 172.21.15.64:38824) 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 -- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700cb970 msgr2=0x7f8d700ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 --2- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700cb970 0x7f8d700ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 -- 172.21.15.64:0/598253783 >> v1:172.21.15.123:6789/0 conn(0x7f8d700ca2c0 legacy=0x7f8d7006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d6f7fe700 1 -- 172.21.15.64:0/598253783 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833955713 0 0) 0x7f8d700cbdf0 con 0x7f8d700ca2c0 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 -- 172.21.15.64:0/598253783 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700cc690 con 0x7f8d70158e20 2023-09-27T23:26:51.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d6ffff700 1 --2- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700cb970 0x7f8d700ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:26:51.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d74f2f700 1 --2- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d700ce5c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8d640099b0 tx=0x7f8d64009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a37533fc398e194 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:51.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d6f7fe700 1 -- 172.21.15.64:0/598253783 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64009ea0 con 0x7f8d70158e20 2023-09-27T23:26:51.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d6f7fe700 1 -- 172.21.15.64:0/598253783 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400b860 con 0x7f8d70158e20 2023-09-27T23:26:51.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d6f7fe700 1 -- 172.21.15.64:0/598253783 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400bc00 con 0x7f8d70158e20 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 -- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 msgr2=0x7f8d700ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 --2- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d700ce5c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8d640099b0 tx=0x7f8d64009250 comp rx=0 tx=0).stop 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 -- 172.21.15.64:0/598253783 shutdown_connections 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 --2- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700cb970 0x7f8d700ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 --2- 172.21.15.64:0/598253783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d700ce5c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.432+0000 7f8d77193700 1 -- 172.21.15.64:0/598253783 >> 172.21.15.64:0/598253783 conn(0x7f8d70067bb0 msgr2=0x7f8d700d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.436+0000 7f8d77193700 1 -- 172.21.15.64:0/598253783 shutdown_connections 2023-09-27T23:26:51.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.436+0000 7f8d77193700 1 -- 172.21.15.64:0/598253783 wait complete. 2023-09-27T23:26:51.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.436+0000 7f8d77193700 1 Processor -- start 2023-09-27T23:26:51.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.436+0000 7f8d77193700 1 -- start start 2023-09-27T23:26:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700ca2c0 0x7f8d7014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d70159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 0x7f8d7014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70169ac0 con 0x7f8d7015a450 2023-09-27T23:26:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015aa80 con 0x7f8d70158e20 2023-09-27T23:26:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701594d0 con 0x7f8d700ca2c0 2023-09-27T23:26:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 0x7f8d7014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d74f2f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700ca2c0 0x7f8d7014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 0x7f8d7014f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32864/0 (socket says 172.21.15.64:32864) 2023-09-27T23:26:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 -- 172.21.15.64:0/3511658259 learned_addr learned my addr 172.21.15.64:0/3511658259 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d6ffff700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d70159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 -- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700ca2c0 msgr2=0x7f8d7014c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700ca2c0 0x7f8d7014c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 -- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 msgr2=0x7f8d70159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d70159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 -- 172.21.15.64:0/3511658259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60009420 con 0x7f8d7015a450 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d75730700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 0x7f8d7014f750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800cd30 tx=0x7f8d68001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800ce90 con 0x7f8d7015a450 2023-09-27T23:26:51.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68005df0 con 0x7f8d7015a450 2023-09-27T23:26:51.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800ab00 con 0x7f8d7015a450 2023-09-27T23:26:51.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d640094f0 con 0x7f8d7015a450 2023-09-27T23:26:51.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.440+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d70168fe0 con 0x7f8d7015a450 2023-09-27T23:26:51.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.444+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d70071120 con 0x7f8d7015a450 2023-09-27T23:26:51.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.444+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8d680070f0 con 0x7f8d7015a450 2023-09-27T23:26:51.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.444+0000 7f8d6cff9700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8d5c060a70 0x7f8d5c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.444+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f8d6807d160 con 0x7f8d7015a450 2023-09-27T23:26:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.444+0000 7f8d74f2f700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8d5c060a70 0x7f8d5c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.444+0000 7f8d74f2f700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8d5c060a70 0x7f8d5c062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d640099b0 tx=0x7f8d64017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:51.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.452+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8d6804d240 con 0x7f8d7015a450 2023-09-27T23:26:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.620+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8d7006a7a0 con 0x7f8d7015a450 2023-09-27T23:26:51.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.624+0000 7f8d6cff9700 1 -- 172.21.15.64:0/3511658259 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8d68050860 con 0x7f8d7015a450 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8d5c060a70 msgr2=0x7f8d5c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8d5c060a70 0x7f8d5c062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d640099b0 tx=0x7f8d64017000 comp rx=0 tx=0).stop 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 msgr2=0x7f8d7014f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 0x7f8d7014f750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800cd30 tx=0x7f8d68001b10 comp rx=0 tx=0).stop 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 shutdown_connections 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8d5c060a70 0x7f8d5c062f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7015a450 0x7f8d7014f750 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70158e20 0x7f8d70159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 --2- 172.21.15.64:0/3511658259 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d700ca2c0 0x7f8d7014c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 >> 172.21.15.64:0/3511658259 conn(0x7f8d70067bb0 msgr2=0x7f8d70156800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:51.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 shutdown_connections 2023-09-27T23:26:51.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.628+0000 7f8d77193700 1 -- 172.21.15.64:0/3511658259 wait complete. 2023-09-27T23:26:51.650 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:26:51.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:26:51.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:26:51.688 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:26:51.688 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:26:51.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:26:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.796+0000 7eff5e836700 1 Processor -- start 2023-09-27T23:26:51.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.796+0000 7eff5e836700 1 -- start start 2023-09-27T23:26:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e836700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff580c6900 0x7eff580c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e836700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 0x7eff5815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e836700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5806b470 con 0x7eff5806b900 2023-09-27T23:26:51.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e836700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5806e110 con 0x7eff580c6900 2023-09-27T23:26:51.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e836700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff58161d10 con 0x7eff5815cc20 2023-09-27T23:26:51.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5d033700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7eff5806b900 0x7eff5806da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57208/0 (socket says 172.21.15.64:57208) 2023-09-27T23:26:51.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5d033700 1 -- 172.21.15.64:0/1473045678 learned_addr learned my addr 172.21.15.64:0/1473045678 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 0x7eff5815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99753333 0 0) 0x7eff5806b470 con 0x7eff5806b900 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff44003570 con 0x7eff5806b900 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5d834700 1 --2- 172.21.15.64:0/1473045678 >> v2:172.21.15.123:3300/0 conn(0x7eff580c6900 0x7eff580c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 12096731 0 0) 0x7eff44003570 con 0x7eff5806b900 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 msgr2=0x7eff5815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 0x7eff5815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 >> v2:172.21.15.123:3300/0 conn(0x7eff580c6900 msgr2=0x7eff580c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 --2- 172.21.15.64:0/1473045678 >> v2:172.21.15.123:3300/0 conn(0x7eff580c6900 0x7eff580c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5806bd70 con 0x7eff5806b900 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7eff4c001470 con 0x7eff5806b900 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 >> v1:172.21.15.64:6789/0 conn(0x7eff5806b900 legacy=0x7eff5806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 0x7eff44003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440040b0 0x7eff44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44004d50 0x7eff440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5806e110 con 0x7eff44004d50 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff58161d10 con 0x7eff440040b0 2023-09-27T23:26:51.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4400bb60 con 0x7eff5815cc20 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44004d50 0x7eff440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5d834700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 0x7eff44003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5d033700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440040b0 0x7eff44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 -- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 msgr2=0x7eff44003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5815cc20 0x7eff44003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 -- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440040b0 msgr2=0x7eff44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440040b0 0x7eff44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 -- 172.21.15.64:0/1473045678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44003570 con 0x7eff44004d50 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5e035700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44004d50 0x7eff440051c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7eff500094d0 tx=0x7eff5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7fcab461adc438 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:51.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000ea90 con 0x7eff44004d50 2023-09-27T23:26:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000f040 con 0x7eff44004d50 2023-09-27T23:26:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.800+0000 7eff5c832700 1 -- 172.21.15.64:0/1473045678 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000ee20 con 0x7eff44004d50 2023-09-27T23:26:51.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- 172.21.15.64:0/1473045678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44004d50 msgr2=0x7eff440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 --2- 172.21.15.64:0/1473045678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44004d50 0x7eff440051c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7eff500094d0 tx=0x7eff5000b040 comp rx=0 tx=0).stop 2023-09-27T23:26:51.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e035700 1 -- 172.21.15.64:0/1473045678 reap_dead start 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- 172.21.15.64:0/1473045678 shutdown_connections 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- 172.21.15.64:0/1473045678 >> 172.21.15.64:0/1473045678 conn(0x7eff58067bb0 msgr2=0x7eff580cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- 172.21.15.64:0/1473045678 shutdown_connections 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- 172.21.15.64:0/1473045678 wait complete. 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 Processor -- start 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- start start 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 0x7eff580ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff58159030 0x7eff5816d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5816dcd0 0x7eff5816f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5816e9a0 con 0x7eff58147780 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580add10 con 0x7eff58159030 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.804+0000 7eff5e836700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580adff0 con 0x7eff5816dcd0 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 0x7eff580ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 0x7eff580ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32892/0 (socket says 172.21.15.64:32892) 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 -- 172.21.15.64:0/1239394309 learned_addr learned my addr 172.21.15.64:0/1239394309 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d033700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff58159030 0x7eff5816d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5e035700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5816dcd0 0x7eff5816f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 -- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5816dcd0 msgr2=0x7eff5816f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5816dcd0 0x7eff5816f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 -- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff58159030 msgr2=0x7eff5816d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff58159030 0x7eff5816d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 -- 172.21.15.64:0/1239394309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580ae2d0 con 0x7eff58147780 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5e035700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5816dcd0 0x7eff5816f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5d834700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 0x7eff580ad400 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7eff54006ac0 tx=0x7eff540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400f940 con 0x7eff58147780 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5400faa0 con 0x7eff58147780 2023-09-27T23:26:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400fe20 con 0x7eff58147780 2023-09-27T23:26:51.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5816ebf0 con 0x7eff58147780 2023-09-27T23:26:51.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5816e140 con 0x7eff58147780 2023-09-27T23:26:51.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.808+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5816ee30 con 0x7eff58147780 2023-09-27T23:26:51.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.812+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7eff54014650 con 0x7eff58147780 2023-09-27T23:26:51.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.812+0000 7eff49ffb700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff3c060a70 0x7eff3c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:51.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.812+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6171+0+0 (secure 0 0 0) 0x7eff54018030 con 0x7eff58147780 2023-09-27T23:26:51.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.812+0000 7eff5d033700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff3c060a70 0x7eff3c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.812+0000 7eff5d033700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff3c060a70 0x7eff3c062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000e50 tx=0x7eff4c001dc0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:51.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:51.816+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff5816ee30 con 0x7eff58147780 2023-09-27T23:26:52.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.012+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff5816ee30 con 0x7eff58147780 2023-09-27T23:26:52.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff49ffb700 1 -- 172.21.15.64:0/1239394309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3535 (secure 0 0 0) 0x7eff5816ee30 con 0x7eff58147780 2023-09-27T23:26:52.021 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:52.021 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1885365785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1885365785},{"type":"v1","addr":"172.21.15.123:6837","nonce":1885365785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5197,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1839542091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1839542091},{"type":"v1","addr":"172.21.15.123:6835","nonce":1839542091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2781506640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2781506640},{"type":"v1","addr":"172.21.15.64:6833","nonce":2781506640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":6012,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2919449849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2919449849},{"type":"v1","addr":"172.21.15.64:6835","nonce":2919449849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[]} 2023-09-27T23:26:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff3c060a70 msgr2=0x7eff3c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff3c060a70 0x7eff3c062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000e50 tx=0x7eff4c001dc0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 msgr2=0x7eff580ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 0x7eff580ad400 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7eff54006ac0 tx=0x7eff540091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 shutdown_connections 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7eff3c060a70 0x7eff3c062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5816dcd0 0x7eff5816f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff58159030 0x7eff5816d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 --2- 172.21.15.64:0/1239394309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff58147780 0x7eff580ad400 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 >> 172.21.15.64:0/1239394309 conn(0x7eff58067bb0 msgr2=0x7eff58159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 shutdown_connections 2023-09-27T23:26:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.016+0000 7eff5e836700 1 -- 172.21.15.64:0/1239394309 wait complete. 2023-09-27T23:26:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2023-09-27T23:26:52.036 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:26:52.136 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:26:52.136 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:26:52.137 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:26:52.137 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:26:52.137 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:26:52.137 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:26:52.137 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:26:52.137 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:26:52.138 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:26:52.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:26:52.138 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:26:52.138 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:26:52.138 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:26:52.138 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:26:52.138 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:26:52.152 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:26:52.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:26:52.153 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:26:52.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:26:52.154 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:26:52.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:26:52.158 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:26:52.159 DEBUG:teuthology.parallel:result is None 2023-09-27T23:26:52.160 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:26:52.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:26:52.160 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:26:52.161 DEBUG:teuthology.parallel:result is None 2023-09-27T23:26:52.201 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:26:52.202 DEBUG:teuthology.parallel:result is None 2023-09-27T23:26:52.203 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:26:52.203 DEBUG:teuthology.parallel:result is None 2023-09-27T23:26:52.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:26:52.227 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:26:52.229 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:26:52.278 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:26:52.279 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:26:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 Processor -- start 2023-09-27T23:26:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 -- start start 2023-09-27T23:26:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380ab910 0x7f5138159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51380670e0 0x7f51380abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51380c1910 con 0x7f51380cd430 2023-09-27T23:26:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51380ac220 con 0x7f51380670e0 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f5137fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5138159cf0 con 0x7f51380ab910 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.308+0000 7f512e7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51380670e0 0x7f51380abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f51380670e0 0x7f51380abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32934/0 (socket says 172.21.15.64:32934) 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 -- 172.21.15.64:0/3045385913 learned_addr learned my addr 172.21.15.64:0/3045385913 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5136ffd700 1 --2- 172.21.15.64:0/3045385913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380ab910 0x7f5138159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 -- 172.21.15.64:0/3045385913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380ab910 msgr2=0x7f5138159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 --2- 172.21.15.64:0/3045385913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380ab910 0x7f5138159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 -- 172.21.15.64:0/3045385913 >> v1:172.21.15.123:6789/0 conn(0x7f51380cd430 legacy=0x7f51380c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 -- 172.21.15.64:0/3045385913 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138159fd0 con 0x7f51380670e0 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f512e7fc700 1 --2- 172.21.15.64:0/3045385913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380670e0 0x7f51380abce0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f51200099b0 tx=0x7f5120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ea2d6fbef3d9bf server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f51367fc700 1 -- 172.21.15.64:0/3045385913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000b860 con 0x7f51380670e0 2023-09-27T23:26:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f51367fc700 1 -- 172.21.15.64:0/3045385913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512000b9c0 con 0x7f51380670e0 2023-09-27T23:26:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f51367fc700 1 -- 172.21.15.64:0/3045385913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000bd60 con 0x7f51380670e0 2023-09-27T23:26:52.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 -- 172.21.15.64:0/3045385913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380670e0 msgr2=0x7f51380abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 --2- 172.21.15.64:0/3045385913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380670e0 0x7f51380abce0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f51200099b0 tx=0x7f5120009250 comp rx=0 tx=0).stop 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 -- 172.21.15.64:0/3045385913 shutdown_connections 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 --2- 172.21.15.64:0/3045385913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380670e0 0x7f51380abce0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 --2- 172.21.15.64:0/3045385913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380ab910 0x7f5138159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 -- 172.21.15.64:0/3045385913 >> 172.21.15.64:0/3045385913 conn(0x7f513806c080 msgr2=0x7f51380ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 -- 172.21.15.64:0/3045385913 shutdown_connections 2023-09-27T23:26:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 -- 172.21.15.64:0/3045385913 wait complete. 2023-09-27T23:26:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 Processor -- start 2023-09-27T23:26:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.312+0000 7f5137fff700 1 -- start start 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380670e0 0x7f51380ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 0x7f51380cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51380cd430 0x7f51380c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380cc750 con 0x7f51380ab910 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380cb900 con 0x7f51380cd430 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51380cbbd0 con 0x7f51380670e0 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 0x7f51380cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 0x7f51380cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32948/0 (socket says 172.21.15.64:32948) 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 -- 172.21.15.64:0/1527597834 learned_addr learned my addr 172.21.15.64:0/1527597834 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5136ffd700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380670e0 0x7f51380ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f51377fe700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51380cd430 0x7f51380c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 -- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380670e0 msgr2=0x7f51380ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380670e0 0x7f51380ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 -- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51380cd430 msgr2=0x7f51380c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51380cd430 0x7f51380c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 -- 172.21.15.64:0/1527597834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51200094f0 con 0x7f51380ab910 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512e7fc700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 0x7f51380cb3c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5120009290 tx=0x7f5120019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512001e070 con 0x7f51380ab910 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120019c30 con 0x7f51380ab910 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51380cbeb0 con 0x7f51380ab910 2023-09-27T23:26:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120010470 con 0x7f51380ab910 2023-09-27T23:26:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51380cc470 con 0x7f51380ab910 2023-09-27T23:26:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.316+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51380af8f0 con 0x7f51380ab910 2023-09-27T23:26:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.320+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5120022070 con 0x7f51380ab910 2023-09-27T23:26:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.320+0000 7f512f7fe700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5118060a70 0x7f5118062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.320+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f5120014070 con 0x7f51380ab910 2023-09-27T23:26:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.320+0000 7f5136ffd700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5118060a70 0x7f5118062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.320+0000 7f5136ffd700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5118060a70 0x7f5118062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5128006fd0 tx=0x7f5128008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:52.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.328+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f51380af8f0 con 0x7f51380ab910 2023-09-27T23:26:52.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:52.500+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f51380af8f0 con 0x7f51380ab910 2023-09-27T23:26:53.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.316+0000 7f512f7fe700 1 -- 172.21.15.64:0/1527597834 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7f51380af8f0 con 0x7f51380ab910 2023-09-27T23:26:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5118060a70 msgr2=0x7f5118062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5118060a70 0x7f5118062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5128006fd0 tx=0x7f5128008040 comp rx=0 tx=0).stop 2023-09-27T23:26:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 msgr2=0x7f51380cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 0x7f51380cb3c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5120009290 tx=0x7f5120019000 comp rx=0 tx=0).stop 2023-09-27T23:26:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 shutdown_connections 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5118060a70 0x7f5118062f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51380cd430 0x7f51380c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51380ab910 0x7f51380cb3c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 --2- 172.21.15.64:0/1527597834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51380670e0 0x7f51380ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 >> 172.21.15.64:0/1527597834 conn(0x7f513806c080 msgr2=0x7f5138151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 shutdown_connections 2023-09-27T23:26:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.324+0000 7f5137fff700 1 -- 172.21.15.64:0/1527597834 wait complete. 2023-09-27T23:26:53.326 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:26:53.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:26:53.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.495+0000 7f1aa58b5700 1 Processor -- start 2023-09-27T23:26:53.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.495+0000 7f1aa58b5700 1 -- start start 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1aa00cc470 0x7f1aa00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa00ca5a0 con 0x7f1aa006a540 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa006dac0 con 0x7f1aa0158e70 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa006d220 con 0x7f1aa00cc470 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1aa006a540 0x7f1aa006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:57240/0 (socket says 172.21.15.64:57240) 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9ffff700 1 -- 172.21.15.64:0/2383585094 learned_addr learned my addr 172.21.15.64:0/2383585094 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa50b4700 1 --2- 172.21.15.64:0/2383585094 >> v2:172.21.15.123:3300/0 conn(0x7f1aa00cc470 0x7f1aa00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9f7fe700 1 -- 172.21.15.64:0/2383585094 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735250004 0 0) 0x7f1aa00ca5a0 con 0x7f1aa006a540 2023-09-27T23:26:53.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9f7fe700 1 -- 172.21.15.64:0/2383585094 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a88003570 con 0x7f1aa006a540 2023-09-27T23:26:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa50b4700 1 -- 172.21.15.64:0/2383585094 >> v1:172.21.15.123:6789/0 conn(0x7f1aa0158e70 legacy=0x7f1aa00afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9f7fe700 1 -- 172.21.15.64:0/2383585094 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188907356 0 0) 0x7f1a88003570 con 0x7f1aa006a540 2023-09-27T23:26:53.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa50b4700 1 -- 172.21.15.64:0/2383585094 >> v1:172.21.15.64:6789/0 conn(0x7f1aa006a540 legacy=0x7f1aa006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa50b4700 1 -- 172.21.15.64:0/2383585094 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa006d500 con 0x7f1aa00cc470 2023-09-27T23:26:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa50b4700 1 --2- 172.21.15.64:0/2383585094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa00cc470 0x7f1aa00cc890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009e60 tx=0x7f1a98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2886d67b36bb9d6c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9f7fe700 1 -- 172.21.15.64:0/2383585094 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800e070 con 0x7f1aa00cc470 2023-09-27T23:26:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9f7fe700 1 -- 172.21.15.64:0/2383585094 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9800b850 con 0x7f1aa00cc470 2023-09-27T23:26:53.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1a9f7fe700 1 -- 172.21.15.64:0/2383585094 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90001470 con 0x7f1aa00cc470 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- 172.21.15.64:0/2383585094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa00cc470 msgr2=0x7f1aa00cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/2383585094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa00cc470 0x7f1aa00cc890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009e60 tx=0x7f1a98009780 comp rx=0 tx=0).stop 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- 172.21.15.64:0/2383585094 shutdown_connections 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/2383585094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa00cc470 0x7f1aa00cc890 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- 172.21.15.64:0/2383585094 >> 172.21.15.64:0/2383585094 conn(0x7f1aa00670e0 msgr2=0x7f1aa00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- 172.21.15.64:0/2383585094 shutdown_connections 2023-09-27T23:26:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.499+0000 7f1aa58b5700 1 -- 172.21.15.64:0/2383585094 wait complete. 2023-09-27T23:26:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 Processor -- start 2023-09-27T23:26:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 -- start start 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa006a540 0x7f1aa00c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 0x7f1aa00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1aa0158e70 0x7f1aa00c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00c7b20 con 0x7f1aa00cc470 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00c6f10 con 0x7f1aa006a540 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa58b5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00c71f0 con 0x7f1aa0158e70 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 0x7f1aa00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 0x7f1aa00c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32950/0 (socket says 172.21.15.64:32950) 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 -- 172.21.15.64:0/1378494046 learned_addr learned my addr 172.21.15.64:0/1378494046 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:53.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa48b3700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa006a540 0x7f1aa00c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1aa50b4700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1aa0158e70 0x7f1aa00c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 -- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1aa0158e70 msgr2=0x7f1aa00c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1aa0158e70 0x7f1aa00c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 -- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa006a540 msgr2=0x7f1aa00c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa006a540 0x7f1aa00c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.503+0000 7f1a9ffff700 1 -- 172.21.15.64:0/1378494046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a98009a20 con 0x7f1aa00cc470 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9ffff700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 0x7f1aa00c7fb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000cb90 tx=0x7f1a90001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90007150 con 0x7f1aa00cc470 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a900072d0 con 0x7f1aa00cc470 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa00c74d0 con 0x7f1aa00cc470 2023-09-27T23:26:53.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90007690 con 0x7f1aa00cc470 2023-09-27T23:26:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa00c9890 con 0x7f1aa00cc470 2023-09-27T23:26:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa004be40 con 0x7f1aa00cc470 2023-09-27T23:26:53.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1a90004a40 con 0x7f1aa00cc470 2023-09-27T23:26:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9cff9700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1a8c060a70 0x7f1a8c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1a9007c390 con 0x7f1aa00cc470 2023-09-27T23:26:53.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.507+0000 7f1aa48b3700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1a8c060a70 0x7f1a8c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.511+0000 7f1aa48b3700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1a8c060a70 0x7f1a8c062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a94003a80 tx=0x7f1a94007ec0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:53.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.515+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1a9004c790 con 0x7f1aa00cc470 2023-09-27T23:26:53.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.739+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1aa006bb10 con 0x7f1aa00cc470 2023-09-27T23:26:53.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1a9cff9700 1 -- 172.21.15.64:0/1378494046 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1a9004fdb0 con 0x7f1aa00cc470 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:26:53.745 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.747 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.748 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.749 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.750 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:26:53.753 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:26:53.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:26:53.756 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:53.758 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:26:53.759 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1a8c060a70 msgr2=0x7f1a8c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1a8c060a70 0x7f1a8c062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a94003a80 tx=0x7f1a94007ec0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 msgr2=0x7f1aa00c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 0x7f1aa00c7fb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000cb90 tx=0x7f1a90001a30 comp rx=0 tx=0).stop 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 shutdown_connections 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1a8c060a70 0x7f1a8c062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1aa0158e70 0x7f1aa00c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa00cc470 0x7f1aa00c7fb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 --2- 172.21.15.64:0/1378494046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa006a540 0x7f1aa00c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 >> 172.21.15.64:0/1378494046 conn(0x7f1aa00670e0 msgr2=0x7f1aa0158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 shutdown_connections 2023-09-27T23:26:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.743+0000 7f1aa58b5700 1 -- 172.21.15.64:0/1378494046 wait complete. 2023-09-27T23:26:53.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:26:53.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 Processor -- start 2023-09-27T23:26:53.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 -- start start 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f56c40c6900 0x7f56c40c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c406da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56c415cc20 0x7f56c415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c406b470 con 0x7f56c415cc20 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c4161d10 con 0x7f56c40c6900 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c406bd70 con 0x7f56c406b900 2023-09-27T23:26:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56c415cc20 0x7f56c415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56c415cc20 0x7f56c415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32966/0 (socket says 172.21.15.64:32966) 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 -- 172.21.15.64:0/2443605623 learned_addr learned my addr 172.21.15.64:0/2443605623 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca0df700 1 --2- 172.21.15.64:0/2443605623 >> v2:172.21.15.123:3300/0 conn(0x7f56c40c6900 0x7f56c40c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56c98de700 1 --2- 172.21.15.64:0/2443605623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 -- 172.21.15.64:0/2443605623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 msgr2=0x7f56c406da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 --2- 172.21.15.64:0/2443605623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c406da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 -- 172.21.15.64:0/2443605623 >> v2:172.21.15.123:3300/0 conn(0x7f56c40c6900 msgr2=0x7f56c40c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 --2- 172.21.15.64:0/2443605623 >> v2:172.21.15.123:3300/0 conn(0x7f56c40c6900 0x7f56c40c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 -- 172.21.15.64:0/2443605623 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c406a580 con 0x7f56c415cc20 2023-09-27T23:26:53.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56ca8e0700 1 --2- 172.21.15.64:0/2443605623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009f20 tx=0x7f56bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf287d4879d321ba server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:53.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56c90dd700 1 -- 172.21.15.64:0/2443605623 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00e070 con 0x7f56c415cc20 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56c90dd700 1 -- 172.21.15.64:0/2443605623 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc00b930 con 0x7f56c415cc20 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56c90dd700 1 -- 172.21.15.64:0/2443605623 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0092e0 con 0x7f56c415cc20 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 -- 172.21.15.64:0/2443605623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 msgr2=0x7f56c415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.871+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/2443605623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009f20 tx=0x7f56bc009840 comp rx=0 tx=0).stop 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- 172.21.15.64:0/2443605623 shutdown_connections 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/2443605623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d0a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/2443605623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c406da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/2443605623 >> v2:172.21.15.123:3300/0 conn(0x7f56c40c6900 0x7f56c40c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- 172.21.15.64:0/2443605623 >> 172.21.15.64:0/2443605623 conn(0x7f56c4067bb0 msgr2=0x7f56c40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- 172.21.15.64:0/2443605623 shutdown_connections 2023-09-27T23:26:53.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- 172.21.15.64:0/2443605623 wait complete. 2023-09-27T23:26:53.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 Processor -- start 2023-09-27T23:26:53.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- start start 2023-09-27T23:26:53.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c41442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c6900 0x7f56c415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415a1c0 con 0x7f56c415cc20 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415b900 con 0x7f56c40c6900 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56cb0e1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4160090 con 0x7f56c406b900 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56ca8e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56ca0df700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c41442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56ca8e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32970/0 (socket says 172.21.15.64:32970) 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56ca8e0700 1 -- 172.21.15.64:0/1707834998 learned_addr learned my addr 172.21.15.64:0/1707834998 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:53.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.875+0000 7f56c98de700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c6900 0x7f56c415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ca8e0700 1 -- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 msgr2=0x7f56c41442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ca8e0700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c41442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ca8e0700 1 -- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c6900 msgr2=0x7f56c415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:53.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ca8e0700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c6900 0x7f56c415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:53.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ca8e0700 1 -- 172.21.15.64:0/1707834998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc009ae0 con 0x7f56c415cc20 2023-09-27T23:26:53.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ca8e0700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f56c4064040 tx=0x7f56bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:53.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc008ad0 con 0x7f56c415cc20 2023-09-27T23:26:53.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc008c30 con 0x7f56c415cc20 2023-09-27T23:26:53.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4160370 con 0x7f56c415cc20 2023-09-27T23:26:53.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc025460 con 0x7f56c415cc20 2023-09-27T23:26:53.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c4160930 con 0x7f56c415cc20 2023-09-27T23:26:53.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f56bc003040 con 0x7f56c415cc20 2023-09-27T23:26:53.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.879+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c415a370 con 0x7f56c415cc20 2023-09-27T23:26:53.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.883+0000 7f56ba7fc700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56a8060a70 0x7f56a8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:53.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.883+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f56bc07e9b0 con 0x7f56c415cc20 2023-09-27T23:26:53.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.883+0000 7f56ca0df700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56a8060a70 0x7f56a8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:53.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.883+0000 7f56ca0df700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56a8060a70 0x7f56a8062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f56c40b1a80 tx=0x7f56c00091a0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:53.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:53.891+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f56c415a370 con 0x7f56c415cc20 2023-09-27T23:26:54.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.051+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56c415a370 con 0x7f56c415cc20 2023-09-27T23:26:54.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.051+0000 7f56ba7fc700 1 -- 172.21.15.64:0/1707834998 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f56c415a370 con 0x7f56c415cc20 2023-09-27T23:26:54.054 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:54.054 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":67,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:53.311728+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56a8060a70 msgr2=0x7f56a8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56a8060a70 0x7f56a8062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f56c40b1a80 tx=0x7f56c00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 msgr2=0x7f56c415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f56c4064040 tx=0x7f56bc018000 comp rx=0 tx=0).stop 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 shutdown_connections 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f56a8060a70 0x7f56a8062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56c415cc20 0x7f56c415d560 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56c40c6900 0x7f56c415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 --2- 172.21.15.64:0/1707834998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56c406b900 0x7f56c41442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 >> 172.21.15.64:0/1707834998 conn(0x7f56c4067bb0 msgr2=0x7f56c40c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 shutdown_connections 2023-09-27T23:26:54.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.055+0000 7f56cb0e1700 1 -- 172.21.15.64:0/1707834998 wait complete. 2023-09-27T23:26:54.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:26:54.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:26:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.171+0000 7f416f0bf700 1 Processor -- start 2023-09-27T23:26:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.171+0000 7f416f0bf700 1 -- start start 2023-09-27T23:26:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41680c63f0 0x7f41680cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806df10 0x7f4168159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680c9db0 con 0x7f416806db40 2023-09-27T23:26:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680ca950 con 0x7f41680c63f0 2023-09-27T23:26:54.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41680cb2e0 con 0x7f416806df10 2023-09-27T23:26:54.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41680c63f0 0x7f41680cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41680c63f0 0x7f41680cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32984/0 (socket says 172.21.15.64:32984) 2023-09-27T23:26:54.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 -- 172.21.15.64:0/4079618557 learned_addr learned my addr 172.21.15.64:0/4079618557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:54.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416d65c700 1 --2- 172.21.15.64:0/4079618557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806df10 0x7f4168159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 -- 172.21.15.64:0/4079618557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806df10 msgr2=0x7f4168159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 --2- 172.21.15.64:0/4079618557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806df10 0x7f4168159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 -- 172.21.15.64:0/4079618557 >> v1:172.21.15.123:6789/0 conn(0x7f416806db40 legacy=0x7f41680ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 -- 172.21.15.64:0/4079618557 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680cb5c0 con 0x7f41680c63f0 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f4167fff700 1 --2- 172.21.15.64:0/4079618557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41680c63f0 0x7f41680cc3e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4160009f20 tx=0x7f4160009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5df6107613baa6a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f41677fe700 1 -- 172.21.15.64:0/4079618557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000e070 con 0x7f41680c63f0 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f41677fe700 1 -- 172.21.15.64:0/4079618557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416000b9d0 con 0x7f41680c63f0 2023-09-27T23:26:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f41677fe700 1 -- 172.21.15.64:0/4079618557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600092e0 con 0x7f41680c63f0 2023-09-27T23:26:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- 172.21.15.64:0/4079618557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41680c63f0 msgr2=0x7f41680cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4079618557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41680c63f0 0x7f41680cc3e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4160009f20 tx=0x7f4160009840 comp rx=0 tx=0).stop 2023-09-27T23:26:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- 172.21.15.64:0/4079618557 shutdown_connections 2023-09-27T23:26:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4079618557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806df10 0x7f4168159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4079618557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41680c63f0 0x7f41680cc3e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- 172.21.15.64:0/4079618557 >> 172.21.15.64:0/4079618557 conn(0x7f4168067bb0 msgr2=0x7f41680cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- 172.21.15.64:0/4079618557 shutdown_connections 2023-09-27T23:26:54.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.175+0000 7f416f0bf700 1 -- 172.21.15.64:0/4079618557 wait complete. 2023-09-27T23:26:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.179+0000 7f416f0bf700 1 Processor -- start 2023-09-27T23:26:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.179+0000 7f416f0bf700 1 -- start start 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806db40 0x7f4168145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 0x7f41680b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41680c63f0 0x7f416806a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168153f20 con 0x7f416806df10 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168154d80 con 0x7f41680c63f0 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4168154170 con 0x7f416806db40 2023-09-27T23:26:54.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 0x7f41680b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 0x7f41680b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32990/0 (socket says 172.21.15.64:32990) 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 -- 172.21.15.64:0/4112653162 learned_addr learned my addr 172.21.15.64:0/4112653162 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416ce5b700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806db40 0x7f4168145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416d65c700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41680c63f0 0x7f416806a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 -- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806db40 msgr2=0x7f4168145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806db40 0x7f4168145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 -- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41680c63f0 msgr2=0x7f416806a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41680c63f0 0x7f416806a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 -- 172.21.15.64:0/4112653162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009ae0 con 0x7f416806df10 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4167fff700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 0x7f41680b1260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f416000b610 tx=0x7f4160018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600088f0 con 0x7f416806df10 2023-09-27T23:26:54.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160008a50 con 0x7f416806df10 2023-09-27T23:26:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4168154450 con 0x7f416806df10 2023-09-27T23:26:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160008e10 con 0x7f416806df10 2023-09-27T23:26:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4168154a10 con 0x7f416806df10 2023-09-27T23:26:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.183+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168071120 con 0x7f416806df10 2023-09-27T23:26:54.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.187+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4160003040 con 0x7f416806df10 2023-09-27T23:26:54.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.187+0000 7f4164ff9700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4148060a70 0x7f4148062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.187+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f416007ebc0 con 0x7f416806df10 2023-09-27T23:26:54.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.187+0000 7f416ce5b700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4148060a70 0x7f4148062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.187+0000 7f416ce5b700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4148060a70 0x7f4148062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f4158007ea0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.195+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f416004eee0 con 0x7f416806df10 2023-09-27T23:26:54.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.359+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41680cc210 con 0x7f416806df10 2023-09-27T23:26:54.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.359+0000 7f4164ff9700 1 -- 172.21.15.64:0/4112653162 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41600210a0 con 0x7f416806df10 2023-09-27T23:26:54.362 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4148060a70 msgr2=0x7f4148062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4148060a70 0x7f4148062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f4158007ea0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 msgr2=0x7f41680b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 0x7f41680b1260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f416000b610 tx=0x7f4160018000 comp rx=0 tx=0).stop 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 shutdown_connections 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4148060a70 0x7f4148062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41680c63f0 0x7f416806a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f416806df10 0x7f41680b1260 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 --2- 172.21.15.64:0/4112653162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f416806db40 0x7f4168145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 >> 172.21.15.64:0/4112653162 conn(0x7f4168067bb0 msgr2=0x7f4168157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 shutdown_connections 2023-09-27T23:26:54.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.363+0000 7f416f0bf700 1 -- 172.21.15.64:0/4112653162 wait complete. 2023-09-27T23:26:54.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:26:54.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 Processor -- start 2023-09-27T23:26:54.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 -- start start 2023-09-27T23:26:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c155520 0x7f4d6c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d6c06b530 0x7f4d6c0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0ad020 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c0c81e0 con 0x7f4d6c06b530 2023-09-27T23:26:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d70fcf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0aca10 con 0x7f4d6c155520 2023-09-27T23:26:54.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d6c06b530 0x7f4d6c0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d6c06b530 0x7f4d6c0ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33004/0 (socket says 172.21.15.64:33004) 2023-09-27T23:26:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 -- 172.21.15.64:0/2471806695 learned_addr learned my addr 172.21.15.64:0/2471806695 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:54.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/2471806695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c155520 0x7f4d6c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 -- 172.21.15.64:0/2471806695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c155520 msgr2=0x7f4d6c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 --2- 172.21.15.64:0/2471806695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c155520 0x7f4d6c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 -- 172.21.15.64:0/2471806695 >> v1:172.21.15.123:6789/0 conn(0x7f4d6c0ae6c0 legacy=0x7f4d6c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.487+0000 7f4d6bfff700 1 -- 172.21.15.64:0/2471806695 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0c7c20 con 0x7f4d6c06b530 2023-09-27T23:26:54.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d6bfff700 1 --2- 172.21.15.64:0/2471806695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c06b530 0x7f4d6c0ad4b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c0d0 tx=0x7f4d6400a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1026728c57f4425 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2471806695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400f070 con 0x7f4d6c06b530 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2471806695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6400cdd0 con 0x7f4d6c06b530 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d6a7fc700 1 -- 172.21.15.64:0/2471806695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400a320 con 0x7f4d6c06b530 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 -- 172.21.15.64:0/2471806695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c06b530 msgr2=0x7f4d6c0ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/2471806695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c06b530 0x7f4d6c0ad4b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c0d0 tx=0x7f4d6400a950 comp rx=0 tx=0).stop 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 -- 172.21.15.64:0/2471806695 shutdown_connections 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/2471806695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c06b530 0x7f4d6c0ad4b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/2471806695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c155520 0x7f4d6c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 -- 172.21.15.64:0/2471806695 >> 172.21.15.64:0/2471806695 conn(0x7f4d6c0c8cd0 msgr2=0x7f4d6c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 -- 172.21.15.64:0/2471806695 shutdown_connections 2023-09-27T23:26:54.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 -- 172.21.15.64:0/2471806695 wait complete. 2023-09-27T23:26:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 Processor -- start 2023-09-27T23:26:54.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.491+0000 7f4d70fcf700 1 -- start start 2023-09-27T23:26:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c06b530 0x7f4d6c1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 0x7f4d6c0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c155520 0x7f4d6c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c153810 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0cba40 con 0x7f4d6c155520 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0cbed0 con 0x7f4d6c06b530 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 0x7f4d6c0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 0x7f4d6c0ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33020/0 (socket says 172.21.15.64:33020) 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 -- 172.21.15.64:0/192479119 learned_addr learned my addr 172.21.15.64:0/192479119 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c06b530 0x7f4d6c1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6bfff700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c155520 0x7f4d6c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 -- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c06b530 msgr2=0x7f4d6c1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c06b530 0x7f4d6c1532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 -- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c155520 msgr2=0x7f4d6c156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c155520 0x7f4d6c156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 -- 172.21.15.64:0/192479119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6400abf0 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d6affd700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 0x7f4d6c0ca450 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e9d0 tx=0x7f4d6000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600015f0 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60001770 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60001b30 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c0cae30 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.495+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c0cb3f0 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.499+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c0cbbd0 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.499+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4d60017030 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.499+0000 7f4d537fe700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d58060a70 0x7f4d58062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.499+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f4d60015070 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.499+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d58060a70 0x7f4d58062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.499+0000 7f4d6b7fe700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d58060a70 0x7f4d58062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c1589f0 tx=0x7f4d5c0091a0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.507+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4d6004f3a0 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.663+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d6c04be40 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.663+0000 7f4d537fe700 1 -- 172.21.15.64:0/192479119 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v67) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d600529c0 con 0x7f4d6c0ae6c0 2023-09-27T23:26:54.665 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d58060a70 msgr2=0x7f4d58062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d58060a70 0x7f4d58062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c1589f0 tx=0x7f4d5c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 msgr2=0x7f4d6c0ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 0x7f4d6c0ca450 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e9d0 tx=0x7f4d6000c270 comp rx=0 tx=0).stop 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 shutdown_connections 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d58060a70 0x7f4d58062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6c155520 0x7f4d6c156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d6c0ae6c0 0x7f4d6c0ca450 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 --2- 172.21.15.64:0/192479119 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d6c06b530 0x7f4d6c1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 >> 172.21.15.64:0/192479119 conn(0x7f4d6c0c8cd0 msgr2=0x7f4d6c06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 shutdown_connections 2023-09-27T23:26:54.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.667+0000 7f4d70fcf700 1 -- 172.21.15.64:0/192479119 wait complete. 2023-09-27T23:26:54.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:26:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.787+0000 7f5b796fb700 1 Processor -- start 2023-09-27T23:26:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.787+0000 7f5b796fb700 1 -- start start 2023-09-27T23:26:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b74159c10 0x7f5b74159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5b7413fb40 0x7f5b740ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b740c22e0 con 0x7f5b7415a690 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b740cc6d0 con 0x7f5b7413fb40 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740cc8e0 con 0x7f5b74159c10 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b72ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b74159c10 0x7f5b74159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b6bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5b7415a690 0x7f5b7413f430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:54132/0 (socket says 172.21.15.64:54132) 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b6bfff700 1 -- 172.21.15.64:0/1577836894 learned_addr learned my addr 172.21.15.64:0/1577836894 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b737fe700 1 --2- 172.21.15.64:0/1577836894 >> v2:172.21.15.123:3300/0 conn(0x7f5b7413fb40 0x7f5b740ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b727fc700 1 -- 172.21.15.64:0/1577836894 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188998902 0 0) 0x7f5b740c22e0 con 0x7f5b7415a690 2023-09-27T23:26:54.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b727fc700 1 -- 172.21.15.64:0/1577836894 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b60003570 con 0x7f5b7415a690 2023-09-27T23:26:54.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b72ffd700 1 -- 172.21.15.64:0/1577836894 >> v2:172.21.15.123:3300/0 conn(0x7f5b7413fb40 msgr2=0x7f5b740ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:54.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b72ffd700 1 --2- 172.21.15.64:0/1577836894 >> v2:172.21.15.123:3300/0 conn(0x7f5b7413fb40 0x7f5b740ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b72ffd700 1 -- 172.21.15.64:0/1577836894 >> v1:172.21.15.123:6789/0 conn(0x7f5b7415a690 legacy=0x7f5b7413f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b72ffd700 1 -- 172.21.15.64:0/1577836894 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740ccbc0 con 0x7f5b74159c10 2023-09-27T23:26:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b72ffd700 1 --2- 172.21.15.64:0/1577836894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b74159c10 0x7f5b74159fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098e0 tx=0x7f5b640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe9059d173cb16b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b727fc700 1 -- 172.21.15.64:0/1577836894 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64011630 con 0x7f5b74159c10 2023-09-27T23:26:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b727fc700 1 -- 172.21.15.64:0/1577836894 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64011790 con 0x7f5b74159c10 2023-09-27T23:26:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b727fc700 1 -- 172.21.15.64:0/1577836894 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64011b30 con 0x7f5b74159c10 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- 172.21.15.64:0/1577836894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b74159c10 msgr2=0x7f5b74159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 --2- 172.21.15.64:0/1577836894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b74159c10 0x7f5b74159fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098e0 tx=0x7f5b640091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- 172.21.15.64:0/1577836894 shutdown_connections 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 --2- 172.21.15.64:0/1577836894 >> v2:172.21.15.123:3300/0 conn(0x7f5b7413fb40 0x7f5b740ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 --2- 172.21.15.64:0/1577836894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b74159c10 0x7f5b74159fe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- 172.21.15.64:0/1577836894 >> 172.21.15.64:0/1577836894 conn(0x7f5b740cd9c0 msgr2=0x7f5b740c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- 172.21.15.64:0/1577836894 shutdown_connections 2023-09-27T23:26:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.791+0000 7f5b796fb700 1 -- 172.21.15.64:0/1577836894 wait complete. 2023-09-27T23:26:54.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 Processor -- start 2023-09-27T23:26:54.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 -- start start 2023-09-27T23:26:54.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7413fb40 0x7f5b74159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 0x7f5b7413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7415a690 0x7f5b74154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740c94e0 con 0x7f5b74159c10 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7413b940 con 0x7f5b7413fb40 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b796fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7413bba0 con 0x7f5b7415a690 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 0x7f5b7413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 0x7f5b7413d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49018/0 (socket says 172.21.15.64:49018) 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b6bfff700 1 -- 172.21.15.64:0/81267668 learned_addr learned my addr 172.21.15.64:0/81267668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b72ffd700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7413fb40 0x7f5b74159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.795+0000 7f5b737fe700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7415a690 0x7f5b74154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6bfff700 1 -- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7415a690 msgr2=0x7f5b74154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6bfff700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7415a690 0x7f5b74154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6bfff700 1 -- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7413fb40 msgr2=0x7f5b74159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6bfff700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7413fb40 0x7f5b74159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6bfff700 1 -- 172.21.15.64:0/81267668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009420 con 0x7f5b74159c10 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6bfff700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 0x7f5b7413d230 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c590 tx=0x7f5b5c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c005b30 con 0x7f5b74159c10 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c005cb0 con 0x7f5b74159c10 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b740c9730 con 0x7f5b74159c10 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c007320 con 0x7f5b74159c10 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b74165120 con 0x7f5b74159c10 2023-09-27T23:26:54.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b74071120 con 0x7f5b74159c10 2023-09-27T23:26:54.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.799+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5b5c01b3b0 con 0x7f5b74159c10 2023-09-27T23:26:54.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.803+0000 7f5b6affd700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5b54060a70 0x7f5b54062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:54.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.803+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f5b5c07c1f0 con 0x7f5b74159c10 2023-09-27T23:26:54.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.803+0000 7f5b72ffd700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5b54060a70 0x7f5b54062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:54.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.803+0000 7f5b72ffd700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5b54060a70 0x7f5b54062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098e0 tx=0x7f5b64017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:54.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.807+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5b5c04c570 con 0x7f5b74159c10 2023-09-27T23:26:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.975+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5b740c1c90 con 0x7f5b74159c10 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.975+0000 7f5b6affd700 1 -- 172.21.15.64:0/81267668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5b5c017090 con 0x7f5b74159c10 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:26:54.978 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:26:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:26:54.979 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:26:54.979 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:26:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5b54060a70 msgr2=0x7f5b54062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5b54060a70 0x7f5b54062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098e0 tx=0x7f5b64017000 comp rx=0 tx=0).stop 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 msgr2=0x7f5b7413d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 0x7f5b7413d230 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c590 tx=0x7f5b5c007e00 comp rx=0 tx=0).stop 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 shutdown_connections 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5b54060a70 0x7f5b54062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7415a690 0x7f5b74154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b74159c10 0x7f5b7413d230 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 --2- 172.21.15.64:0/81267668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7413fb40 0x7f5b74159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 >> 172.21.15.64:0/81267668 conn(0x7f5b740cd9c0 msgr2=0x7f5b740c2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 shutdown_connections 2023-09-27T23:26:54.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:54.979+0000 7f5b796fb700 1 -- 172.21.15.64:0/81267668 wait complete. 2023-09-27T23:26:54.992 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:26:54.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:26:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.107+0000 7fb6a1175700 1 Processor -- start 2023-09-27T23:26:55.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.107+0000 7fb6a1175700 1 -- start start 2023-09-27T23:26:55.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb69c158da0 0x7fb69c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c06df10 0x7fb69c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c15d720 con 0x7fb69c0caca0 2023-09-27T23:26:55.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0ca290 con 0x7fb69c158da0 2023-09-27T23:26:55.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0ca4a0 con 0x7fb69c06df10 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb69c158da0 0x7fb69c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a0974700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb69c0caca0 0x7fb69c06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:54138/0 (socket says 172.21.15.64:54138) 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a0974700 1 -- 172.21.15.64:0/159711037 learned_addr learned my addr 172.21.15.64:0/159711037 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69b7fe700 1 --2- 172.21.15.64:0/159711037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c06df10 0x7fb69c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69affd700 1 -- 172.21.15.64:0/159711037 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529159335 0 0) 0x7fb69c15d720 con 0x7fb69c0caca0 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69affd700 1 -- 172.21.15.64:0/159711037 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb67c003570 con 0x7fb69c0caca0 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69b7fe700 1 -- 172.21.15.64:0/159711037 >> v2:172.21.15.123:3300/0 conn(0x7fb69c158da0 msgr2=0x7fb69c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69b7fe700 1 --2- 172.21.15.64:0/159711037 >> v2:172.21.15.123:3300/0 conn(0x7fb69c158da0 0x7fb69c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69b7fe700 1 -- 172.21.15.64:0/159711037 >> v1:172.21.15.123:6789/0 conn(0x7fb69c0caca0 legacy=0x7fb69c06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69b7fe700 1 -- 172.21.15.64:0/159711037 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0ca780 con 0x7fb69c06df10 2023-09-27T23:26:55.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69b7fe700 1 --2- 172.21.15.64:0/159711037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c06df10 0x7fb69c06ebb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6840098e0 tx=0x7fb6840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4bf996eeafbf82 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:55.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69affd700 1 -- 172.21.15.64:0/159711037 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684011630 con 0x7fb69c06df10 2023-09-27T23:26:55.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69affd700 1 -- 172.21.15.64:0/159711037 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684011790 con 0x7fb69c06df10 2023-09-27T23:26:55.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb69affd700 1 -- 172.21.15.64:0/159711037 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684011b10 con 0x7fb69c06df10 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- 172.21.15.64:0/159711037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c06df10 msgr2=0x7fb69c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 --2- 172.21.15.64:0/159711037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c06df10 0x7fb69c06ebb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6840098e0 tx=0x7fb6840091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- 172.21.15.64:0/159711037 shutdown_connections 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 --2- 172.21.15.64:0/159711037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c06df10 0x7fb69c06ebb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 --2- 172.21.15.64:0/159711037 >> v2:172.21.15.123:3300/0 conn(0x7fb69c158da0 0x7fb69c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- 172.21.15.64:0/159711037 >> 172.21.15.64:0/159711037 conn(0x7fb69c0cc760 msgr2=0x7fb69c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:55.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- 172.21.15.64:0/159711037 shutdown_connections 2023-09-27T23:26:55.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.111+0000 7fb6a1175700 1 -- 172.21.15.64:0/159711037 wait complete. 2023-09-27T23:26:55.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 Processor -- start 2023-09-27T23:26:55.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 -- start start 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 0x7fb69c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c0caca0 0x7fb69c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb69c158da0 0x7fb69c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c152cb0 con 0x7fb69c06df10 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c152f90 con 0x7fb69c158da0 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a1175700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c153270 con 0x7fb69c0caca0 2023-09-27T23:26:55.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 0x7fb69c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 0x7fb69c157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49032/0 (socket says 172.21.15.64:49032) 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 -- 172.21.15.64:0/3849281263 learned_addr learned my addr 172.21.15.64:0/3849281263 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb6a0974700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb69c158da0 0x7fb69c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69b7fe700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c0caca0 0x7fb69c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 -- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c0caca0 msgr2=0x7fb69c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c0caca0 0x7fb69c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 -- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb69c158da0 msgr2=0x7fb69c154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb69c158da0 0x7fb69c154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 -- 172.21.15.64:0/3849281263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684009420 con 0x7fb69c06df10 2023-09-27T23:26:55.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb69bfff700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 0x7fb69c157410 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00e930 tx=0x7fb68c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.115+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c013680 con 0x7fb69c06df10 2023-09-27T23:26:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c153550 con 0x7fb69c06df10 2023-09-27T23:26:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c0137e0 con 0x7fb69c06df10 2023-09-27T23:26:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c013b80 con 0x7fb69c06df10 2023-09-27T23:26:55.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c1525c0 con 0x7fb69c06df10 2023-09-27T23:26:55.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c04be40 con 0x7fb69c06df10 2023-09-27T23:26:55.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb68c00c3e0 con 0x7fb69c06df10 2023-09-27T23:26:55.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb693fff700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb680060a70 0x7fb680062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.119+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fb68c07cef0 con 0x7fb69c06df10 2023-09-27T23:26:55.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.123+0000 7fb69b7fe700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb680060a70 0x7fb680062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.123+0000 7fb69b7fe700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb680060a70 0x7fb680062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6840091e0 tx=0x7fb684017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:55.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.127+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb68c04d270 con 0x7fb69c06df10 2023-09-27T23:26:55.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.295+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb69c06f360 con 0x7fb69c06df10 2023-09-27T23:26:55.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.395+0000 7fb693fff700 1 -- 172.21.15.64:0/3849281263 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb68c050890 con 0x7fb69c06df10 2023-09-27T23:26:55.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb680060a70 msgr2=0x7fb680062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb680060a70 0x7fb680062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6840091e0 tx=0x7fb684017000 comp rx=0 tx=0).stop 2023-09-27T23:26:55.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 msgr2=0x7fb69c157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 0x7fb69c157410 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00e930 tx=0x7fb68c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 shutdown_connections 2023-09-27T23:26:55.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb680060a70 0x7fb680062f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb69c158da0 0x7fb69c154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb69c0caca0 0x7fb69c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 --2- 172.21.15.64:0/3849281263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69c06df10 0x7fb69c157410 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 >> 172.21.15.64:0/3849281263 conn(0x7fb69c0cc760 msgr2=0x7fb69c0670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 shutdown_connections 2023-09-27T23:26:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.403+0000 7fb6a1175700 1 -- 172.21.15.64:0/3849281263 wait complete. 2023-09-27T23:26:55.408 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:26:55.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:26:55.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafe5a62700 1 Processor -- start 2023-09-27T23:26:55.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafe5a62700 1 -- start start 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafe5a62700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafe00cb970 0x7fafe006acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafe5a62700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe00cbdf0 con 0x7fafe0158e20 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafe5a62700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe00cc0d0 con 0x7fafe00cecd0 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafe5a62700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe00cc3b0 con 0x7fafe00cb970 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafdeffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafe00cb970 0x7fafe006acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafdf7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fafe0158e20 0x7fafe00ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48010/0 (socket says 172.21.15.64:48010) 2023-09-27T23:26:55.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafdeffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafe00cb970 0x7fafe006acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49048/0 (socket says 172.21.15.64:49048) 2023-09-27T23:26:55.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.531+0000 7fafdf7fe700 1 -- 172.21.15.64:0/2285022570 learned_addr learned my addr 172.21.15.64:0/2285022570 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafde7fc700 1 -- 172.21.15.64:0/2285022570 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536724181 0 0) 0x7fafe00cbdf0 con 0x7fafe0158e20 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafde7fc700 1 -- 172.21.15.64:0/2285022570 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc8003570 con 0x7fafe0158e20 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafdeffd700 1 -- 172.21.15.64:0/2285022570 >> v1:172.21.15.123:6789/0 conn(0x7fafe00cecd0 legacy=0x7fafe006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafdeffd700 1 -- 172.21.15.64:0/2285022570 >> v1:172.21.15.64:6789/0 conn(0x7fafe0158e20 legacy=0x7fafe00ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafdeffd700 1 -- 172.21.15.64:0/2285022570 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe00cc690 con 0x7fafe00cb970 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafdeffd700 1 --2- 172.21.15.64:0/2285022570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe00cb970 0x7fafe006acc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009910 tx=0x7fafd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97df1ec3133fd9b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafde7fc700 1 -- 172.21.15.64:0/2285022570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000b640 con 0x7fafe00cb970 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafde7fc700 1 -- 172.21.15.64:0/2285022570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd000b7a0 con 0x7fafe00cb970 2023-09-27T23:26:55.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafde7fc700 1 -- 172.21.15.64:0/2285022570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000bb60 con 0x7fafe00cb970 2023-09-27T23:26:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 -- 172.21.15.64:0/2285022570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe00cb970 msgr2=0x7fafe006acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2285022570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe00cb970 0x7fafe006acc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009910 tx=0x7fafd00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 -- 172.21.15.64:0/2285022570 shutdown_connections 2023-09-27T23:26:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2285022570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe00cb970 0x7fafe006acc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 -- 172.21.15.64:0/2285022570 >> 172.21.15.64:0/2285022570 conn(0x7fafe0067bb0 msgr2=0x7fafe00d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 -- 172.21.15.64:0/2285022570 shutdown_connections 2023-09-27T23:26:55.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 -- 172.21.15.64:0/2285022570 wait complete. 2023-09-27T23:26:55.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 Processor -- start 2023-09-27T23:26:55.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.535+0000 7fafe5a62700 1 -- start start 2023-09-27T23:26:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafe00cecd0 0x7fafe0169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 0x7fafe016a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafe015cfe0 0x7fafe00c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe016a6c0 con 0x7fafe0158e20 2023-09-27T23:26:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe00c55e0 con 0x7fafe00cecd0 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe00c4fd0 con 0x7fafe015cfe0 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 0x7fafe016a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 0x7fafe016a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49052/0 (socket says 172.21.15.64:49052) 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 -- 172.21.15.64:0/2258823174 learned_addr learned my addr 172.21.15.64:0/2258823174 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdffff700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafe015cfe0 0x7fafe00c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdf7fe700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafe00cecd0 0x7fafe0169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 -- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafe015cfe0 msgr2=0x7fafe00c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafe015cfe0 0x7fafe00c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 -- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafe00cecd0 msgr2=0x7fafe0169c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafe00cecd0 0x7fafe0169c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 -- 172.21.15.64:0/2258823174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0009420 con 0x7fafe0158e20 2023-09-27T23:26:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafdeffd700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 0x7fafe016a180 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fafd00093f0 tx=0x7fafd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:55.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd001f070 con 0x7fafe0158e20 2023-09-27T23:26:55.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0000bc0 con 0x7fafe0158e20 2023-09-27T23:26:55.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe016a8f0 con 0x7fafe0158e20 2023-09-27T23:26:55.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000bb60 con 0x7fafe0158e20 2023-09-27T23:26:55.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe016abd0 con 0x7fafe0158e20 2023-09-27T23:26:55.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.539+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe00c5750 con 0x7fafe0158e20 2023-09-27T23:26:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.543+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fafd0023070 con 0x7fafe0158e20 2023-09-27T23:26:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.543+0000 7fafc77fe700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fafcc060a70 0x7fafcc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.543+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5896+0+0 (secure 0 0 0) 0x7fafd007e830 con 0x7fafe0158e20 2023-09-27T23:26:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.543+0000 7fafdf7fe700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fafcc060a70 0x7fafcc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:55.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.543+0000 7fafdf7fe700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fafcc060a70 0x7fafcc062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fafe00ce6e0 tx=0x7fafd4009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:55.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.551+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fafd004eaa0 con 0x7fafe0158e20 2023-09-27T23:26:55.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:55.707+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fafe0071120 con 0x7fafe0158e20 2023-09-27T23:26:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.411+0000 7fafc77fe700 1 -- 172.21.15.64:0/2258823174 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) v1 ==== 170+0+0 (secure 0 0 0) 0x7fafd00150d0 con 0x7fafe0158e20 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fafcc060a70 msgr2=0x7fafcc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fafcc060a70 0x7fafcc062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fafe00ce6e0 tx=0x7fafd4009040 comp rx=0 tx=0).stop 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 msgr2=0x7fafe016a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 0x7fafe016a180 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fafd00093f0 tx=0x7fafd0017000 comp rx=0 tx=0).stop 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 shutdown_connections 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fafcc060a70 0x7fafcc062f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafe015cfe0 0x7fafe00c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafe0158e20 0x7fafe016a180 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 --2- 172.21.15.64:0/2258823174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafe00cecd0 0x7fafe0169c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 >> 172.21.15.64:0/2258823174 conn(0x7fafe0067bb0 msgr2=0x7fafe00cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:56.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.427+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 shutdown_connections 2023-09-27T23:26:56.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.431+0000 7fafe5a62700 1 -- 172.21.15.64:0/2258823174 wait complete. 2023-09-27T23:26:56.434 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:26:56.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:26:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 Processor -- start 2023-09-27T23:26:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 -- start start 2023-09-27T23:26:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb510155520 0x7fb5100c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb51006b530 0x7fb5100ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100ad020 con 0x7fb5100ae6c0 2023-09-27T23:26:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100c81e0 con 0x7fb51006b530 2023-09-27T23:26:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb515ad9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100aca10 con 0x7fb510155520 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb5152d8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb51006b530 0x7fb5100ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb514ad7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb510155520 0x7fb5100c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb507fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb5100ae6c0 0x7fb51006ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:54166/0 (socket says 172.21.15.64:54166) 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb5152d8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb51006b530 0x7fb5100ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:43570/0 (socket says 172.21.15.64:43570) 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb514ad7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb510155520 0x7fb5100c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:33078/0 (socket says 172.21.15.64:33078) 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.559+0000 7fb507fff700 1 -- 172.21.15.64:0/293257903 learned_addr learned my addr 172.21.15.64:0/293257903 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5152d8700 1 -- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb510155520 msgr2=0x7fb5100c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5152d8700 1 --2- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb510155520 0x7fb5100c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5152d8700 1 -- 172.21.15.64:0/293257903 >> v1:172.21.15.123:6789/0 conn(0x7fb5100ae6c0 legacy=0x7fb51006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5152d8700 1 -- 172.21.15.64:0/293257903 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100c7c20 con 0x7fb51006b530 2023-09-27T23:26:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5152d8700 1 --2- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5100ad4b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb50800af70 tx=0x7fb50800a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e934aad4dca821eb server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5077fe700 1 -- 172.21.15.64:0/293257903 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800f070 con 0x7fb51006b530 2023-09-27T23:26:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5077fe700 1 -- 172.21.15.64:0/293257903 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800a2e0 con 0x7fb51006b530 2023-09-27T23:26:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb5077fe700 1 -- 172.21.15.64:0/293257903 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800cd00 con 0x7fb51006b530 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 -- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 msgr2=0x7fb5100ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 --2- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5100ad4b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb50800af70 tx=0x7fb50800a890 comp rx=0 tx=0).stop 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 -- 172.21.15.64:0/293257903 shutdown_connections 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 --2- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5100ad4b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 --2- 172.21.15.64:0/293257903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb510155520 0x7fb5100c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 -- 172.21.15.64:0/293257903 >> 172.21.15.64:0/293257903 conn(0x7fb5100c8cd0 msgr2=0x7fb5100c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 -- 172.21.15.64:0/293257903 shutdown_connections 2023-09-27T23:26:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.563+0000 7fb515ad9700 1 -- 172.21.15.64:0/293257903 wait complete. 2023-09-27T23:26:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 Processor -- start 2023-09-27T23:26:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 -- start start 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5101532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5100ae6c0 0x7fb5100ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 0x7fb510156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510153810 con 0x7fb510155520 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100cba40 con 0x7fb51006b530 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.567+0000 7fb515ad9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100cbed0 con 0x7fb5100ae6c0 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 0x7fb510156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb507fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5100ae6c0 0x7fb5100ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb514ad7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5101532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 0x7fb510156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49064/0 (socket says 172.21.15.64:49064) 2023-09-27T23:26:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 -- 172.21.15.64:0/1608299627 learned_addr learned my addr 172.21.15.64:0/1608299627 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 -- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5100ae6c0 msgr2=0x7fb5100ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5100ae6c0 0x7fb5100ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 -- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 msgr2=0x7fb5101532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5101532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 -- 172.21.15.64:0/1608299627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50800ab30 con 0x7fb510155520 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb5152d8700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 0x7fb510156300 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb508013000 tx=0x7fb508019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508009bf0 con 0x7fb510155520 2023-09-27T23:26:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb508009d70 con 0x7fb510155520 2023-09-27T23:26:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5100cae30 con 0x7fb510155520 2023-09-27T23:26:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50801e670 con 0x7fb510155520 2023-09-27T23:26:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5100cb3f0 con 0x7fb510155520 2023-09-27T23:26:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5100cbbd0 con 0x7fb510155520 2023-09-27T23:26:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.571+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb508003040 con 0x7fb510155520 2023-09-27T23:26:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.575+0000 7fb504ff9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4fc060a70 0x7fb4fc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.575+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fb50807fd70 con 0x7fb510155520 2023-09-27T23:26:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.575+0000 7fb514ad7700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4fc060a70 0x7fb4fc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.575+0000 7fb514ad7700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4fc060a70 0x7fb4fc062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb5101589f0 tx=0x7fb50c0091a0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.579+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb50804fcf0 con 0x7fb510155520 2023-09-27T23:26:56.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.775+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb51004be40 con 0x7fb510155520 2023-09-27T23:26:56.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.791+0000 7fb504ff9700 1 -- 172.21.15.64:0/1608299627 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v18) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb508053310 con 0x7fb510155520 2023-09-27T23:26:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4fc060a70 msgr2=0x7fb4fc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4fc060a70 0x7fb4fc062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb5101589f0 tx=0x7fb50c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 msgr2=0x7fb510156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 0x7fb510156300 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb508013000 tx=0x7fb508019000 comp rx=0 tx=0).stop 2023-09-27T23:26:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 shutdown_connections 2023-09-27T23:26:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb4fc060a70 0x7fb4fc062f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510155520 0x7fb510156300 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5100ae6c0 0x7fb5100ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 --2- 172.21.15.64:0/1608299627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb51006b530 0x7fb5101532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 >> 172.21.15.64:0/1608299627 conn(0x7fb5100c8cd0 msgr2=0x7fb51006dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:56.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 shutdown_connections 2023-09-27T23:26:56.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.799+0000 7fb515ad9700 1 -- 172.21.15.64:0/1608299627 wait complete. 2023-09-27T23:26:56.807 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 8 and data pool 9 2023-09-27T23:26:56.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:26:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 Processor -- start 2023-09-27T23:26:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 -- start start 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12c015d320 0x7f12c015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c00c15b0 0x7f12c00aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12c006d690 0x7f12c00680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c00c7190 con 0x7f12c006d690 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c00c1b50 con 0x7f12c015d320 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c742a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00c1e30 con 0x7f12c00c15b0 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12c006d690 0x7f12c00680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12c006d690 0x7f12c00680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49072/0 (socket says 172.21.15.64:49072) 2023-09-27T23:26:56.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c51c6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12c015d320 0x7f12c015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 -- 172.21.15.64:0/1142033417 learned_addr learned my addr 172.21.15.64:0/1142033417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:56.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c49c5700 1 --2- 172.21.15.64:0/1142033417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c00c15b0 0x7f12c00aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 -- 172.21.15.64:0/1142033417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c00c15b0 msgr2=0x7f12c00aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 --2- 172.21.15.64:0/1142033417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c00c15b0 0x7f12c00aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 -- 172.21.15.64:0/1142033417 >> v2:172.21.15.123:3300/0 conn(0x7f12c015d320 msgr2=0x7f12c015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 --2- 172.21.15.64:0/1142033417 >> v2:172.21.15.123:3300/0 conn(0x7f12c015d320 0x7f12c015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 -- 172.21.15.64:0/1142033417 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00c6e20 con 0x7f12c006d690 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12c59c7700 1 --2- 172.21.15.64:0/1142033417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c00680f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f12c00cbd40 tx=0x7f12b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7682015ac86ff9f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12b7fff700 1 -- 172.21.15.64:0/1142033417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000b640 con 0x7f12c006d690 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12b7fff700 1 -- 172.21.15.64:0/1142033417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b000b7a0 con 0x7f12c006d690 2023-09-27T23:26:56.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.927+0000 7f12b7fff700 1 -- 172.21.15.64:0/1142033417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000bb60 con 0x7f12c006d690 2023-09-27T23:26:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 -- 172.21.15.64:0/1142033417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 msgr2=0x7f12c00680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 --2- 172.21.15.64:0/1142033417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c00680f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f12c00cbd40 tx=0x7f12b0009250 comp rx=0 tx=0).stop 2023-09-27T23:26:56.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 -- 172.21.15.64:0/1142033417 shutdown_connections 2023-09-27T23:26:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 --2- 172.21.15.64:0/1142033417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c00680f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 --2- 172.21.15.64:0/1142033417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c00c15b0 0x7f12c00aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 --2- 172.21.15.64:0/1142033417 >> v2:172.21.15.123:3300/0 conn(0x7f12c015d320 0x7f12c015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 -- 172.21.15.64:0/1142033417 >> 172.21.15.64:0/1142033417 conn(0x7f12c00c8850 msgr2=0x7f12c00c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 -- 172.21.15.64:0/1142033417 shutdown_connections 2023-09-27T23:26:56.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 -- 172.21.15.64:0/1142033417 wait complete. 2023-09-27T23:26:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.931+0000 7f12c742a700 1 Processor -- start 2023-09-27T23:26:56.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 -- start start 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c00c15b0 0x7f12c015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c015d320 0x7f12c00af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b12b0 con 0x7f12c006d690 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b06a0 con 0x7f12c00c15b0 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b0980 con 0x7f12c015d320 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c015f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49080/0 (socket says 172.21.15.64:49080) 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 -- 172.21.15.64:0/401387352 learned_addr learned my addr 172.21.15.64:0/401387352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c59c7700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c015d320 0x7f12c00af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c49c5700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c00c15b0 0x7f12c015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 -- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c015d320 msgr2=0x7f12c00af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c015d320 0x7f12c00af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 -- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c00c15b0 msgr2=0x7f12c015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c00c15b0 0x7f12c015fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 -- 172.21.15.64:0/401387352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b00094f0 con 0x7f12c006d690 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c51c6700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c015f8f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00ec80 tx=0x7f12bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00c850 con 0x7f12c006d690 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc00c9b0 con 0x7f12c006d690 2023-09-27T23:26:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c00b0c60 con 0x7f12c006d690 2023-09-27T23:26:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00cd50 con 0x7f12c006d690 2023-09-27T23:26:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.935+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c014a280 con 0x7f12c006d690 2023-09-27T23:26:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.939+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c0071120 con 0x7f12c006d690 2023-09-27T23:26:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.939+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f12bc013680 con 0x7f12c006d690 2023-09-27T23:26:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.939+0000 7f12b57fa700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f12ac060a70 0x7f12ac062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.939+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f12bc07d3f0 con 0x7f12c006d690 2023-09-27T23:26:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.939+0000 7f12c49c5700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f12ac060a70 0x7f12ac062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.939+0000 7f12c49c5700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f12ac060a70 0x7f12ac062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009fd0 tx=0x7f12b800b040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:56.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:56.947+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f12bc04d420 con 0x7f12c006d690 2023-09-27T23:26:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.111+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12c00642a0 con 0x7f12c006d690 2023-09-27T23:26:57.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.111+0000 7f12b57fa700 1 -- 172.21.15.64:0/401387352 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f12bc050a40 con 0x7f12c006d690 2023-09-27T23:26:57.115 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:57.115 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":70,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:56.778770+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:26:55.337700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-27T23:26:55.749823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:26:57.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f12ac060a70 msgr2=0x7f12ac062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f12ac060a70 0x7f12ac062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009fd0 tx=0x7f12b800b040 comp rx=0 tx=0).stop 2023-09-27T23:26:57.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 msgr2=0x7f12c015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c015f8f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00ec80 tx=0x7f12bc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 shutdown_connections 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f12ac060a70 0x7f12ac062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12c015d320 0x7f12c00af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c00c15b0 0x7f12c015fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 --2- 172.21.15.64:0/401387352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c006d690 0x7f12c015f8f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 >> 172.21.15.64:0/401387352 conn(0x7f12c00c8850 msgr2=0x7f12c006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 shutdown_connections 2023-09-27T23:26:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.115+0000 7f12c742a700 1 -- 172.21.15.64:0/401387352 wait complete. 2023-09-27T23:26:57.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:26:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.239+0000 7fd62e6f4700 1 Processor -- start 2023-09-27T23:26:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.239+0000 7fd62e6f4700 1 -- start start 2023-09-27T23:26:57.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628159340 0x7fd628067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628068c70 con 0x7fd628158f70 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628068060 con 0x7fd628069100 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd628068340 con 0x7fd628159340 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd627fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd628158f70 0x7fd6280cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48014/0 (socket says 172.21.15.64:48014) 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd627fff700 1 -- 172.21.15.64:0/534772906 learned_addr learned my addr 172.21.15.64:0/534772906 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62cc91700 1 --2- 172.21.15.64:0/534772906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628159340 0x7fd628067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070586640 0 0) 0x7fd628068060 con 0x7fd628069100 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd610003570 con 0x7fd628069100 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218444794 0 0) 0x7fd628068c70 con 0x7fd628158f70 2023-09-27T23:26:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd628068060 con 0x7fd628158f70 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62cc91700 1 -- 172.21.15.64:0/534772906 >> v1:172.21.15.123:6789/0 conn(0x7fd628069100 legacy=0x7fd6280cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62cc91700 1 -- 172.21.15.64:0/534772906 >> v1:172.21.15.64:6789/0 conn(0x7fd628158f70 legacy=0x7fd6280cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62cc91700 1 -- 172.21.15.64:0/534772906 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd628068620 con 0x7fd628159340 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62cc91700 1 --2- 172.21.15.64:0/534772906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628159340 0x7fd628067980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd620009910 tx=0x7fd6200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c64384cbde0063 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620011630 con 0x7fd628159340 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd620011790 con 0x7fd628159340 2023-09-27T23:26:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd626ffd700 1 -- 172.21.15.64:0/534772906 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620011b10 con 0x7fd628159340 2023-09-27T23:26:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- 172.21.15.64:0/534772906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628159340 msgr2=0x7fd628067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/534772906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628159340 0x7fd628067980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd620009910 tx=0x7fd6200091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- 172.21.15.64:0/534772906 shutdown_connections 2023-09-27T23:26:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/534772906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628159340 0x7fd628067980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- 172.21.15.64:0/534772906 >> 172.21.15.64:0/534772906 conn(0x7fd6280ccc40 msgr2=0x7fd6280cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- 172.21.15.64:0/534772906 shutdown_connections 2023-09-27T23:26:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.243+0000 7fd62e6f4700 1 -- 172.21.15.64:0/534772906 wait complete. 2023-09-27T23:26:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 Processor -- start 2023-09-27T23:26:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 -- start start 2023-09-27T23:26:57.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628069100 0x7fd628155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 0x7fd6280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd628159340 0x7fd6280b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280b59f0 con 0x7fd628158f70 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280b0f80 con 0x7fd628159340 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.247+0000 7fd62e6f4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd628155590 con 0x7fd628069100 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 0x7fd6280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd627fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628069100 0x7fd628155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 0x7fd6280b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49082/0 (socket says 172.21.15.64:49082) 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 -- 172.21.15.64:0/1396008178 learned_addr learned my addr 172.21.15.64:0/1396008178 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd62cc91700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd628159340 0x7fd6280b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 -- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628069100 msgr2=0x7fd628155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628069100 0x7fd628155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 -- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd628159340 msgr2=0x7fd6280b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd628159340 0x7fd6280b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 -- 172.21.15.64:0/1396008178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd620009420 con 0x7fd628158f70 2023-09-27T23:26:57.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd6277fe700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 0x7fd6280b4dc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c940 tx=0x7fd61c002730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c012630 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c012790 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c012b50 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280b5710 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6280b0c10 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6280b0790 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.251+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd61c01b400 con 0x7fd628158f70 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.255+0000 7fd60ffff700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd614060a70 0x7fd614062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.255+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd61c07d470 con 0x7fd628158f70 2023-09-27T23:26:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.255+0000 7fd627fff700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd614060a70 0x7fd614062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.255+0000 7fd627fff700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd614060a70 0x7fd614062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd618007fd0 tx=0x7fd618009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:57.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.259+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd6280b0790 con 0x7fd628158f70 2023-09-27T23:26:57.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.431+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6280b0790 con 0x7fd628158f70 2023-09-27T23:26:57.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.431+0000 7fd60ffff700 1 -- 172.21.15.64:0/1396008178 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17084 (secure 0 0 0) 0x7fd6280b0790 con 0x7fd628158f70 2023-09-27T23:26:57.435 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:26:57.435 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":70,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:56.778770+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:26:55.337700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-27T23:26:55.749823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:26:57.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd614060a70 msgr2=0x7fd614062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd614060a70 0x7fd614062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd618007fd0 tx=0x7fd618009040 comp rx=0 tx=0).stop 2023-09-27T23:26:57.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 msgr2=0x7fd6280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 0x7fd6280b4dc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c940 tx=0x7fd61c002730 comp rx=0 tx=0).stop 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 shutdown_connections 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd614060a70 0x7fd614062f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd628159340 0x7fd6280b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd628158f70 0x7fd6280b4dc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 --2- 172.21.15.64:0/1396008178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd628069100 0x7fd628155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 >> 172.21.15.64:0/1396008178 conn(0x7fd6280ccc40 msgr2=0x7fd6280c5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 shutdown_connections 2023-09-27T23:26:57.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.435+0000 7fd62e6f4700 1 -- 172.21.15.64:0/1396008178 wait complete. 2023-09-27T23:26:57.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:26:57.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34130e1700 1 Processor -- start 2023-09-27T23:26:57.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34130e1700 1 -- start start 2023-09-27T23:26:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34130e1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f340c154780 0x7f340c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34130e1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c15ae30 con 0x7f340c06baa0 2023-09-27T23:26:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34130e1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0abf70 con 0x7f340c0adab0 2023-09-27T23:26:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34130e1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c154b50 con 0x7f340c154780 2023-09-27T23:26:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34120df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f340c154780 0x7f340c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34118de700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f340c06baa0 0x7f340c157b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48016/0 (socket says 172.21.15.64:48016) 2023-09-27T23:26:57.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34118de700 1 -- 172.21.15.64:0/4120570151 learned_addr learned my addr 172.21.15.64:0/4120570151 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:57.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34110dd700 1 -- 172.21.15.64:0/4120570151 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413196820 0 0) 0x7f340c15ae30 con 0x7f340c06baa0 2023-09-27T23:26:57.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34120df700 1 -- 172.21.15.64:0/4120570151 >> v1:172.21.15.123:6789/0 conn(0x7f340c0adab0 legacy=0x7f340c0adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34120df700 1 -- 172.21.15.64:0/4120570151 >> v1:172.21.15.64:6789/0 conn(0x7f340c06baa0 legacy=0x7f340c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.563+0000 7f34120df700 1 -- 172.21.15.64:0/4120570151 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0ac210 con 0x7f340c154780 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34120df700 1 --2- 172.21.15.64:0/4120570151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c15b2c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0098e0 tx=0x7f33fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fad46fd7370d7f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34110dd700 1 -- 172.21.15.64:0/4120570151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc009e80 con 0x7f340c154780 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34110dd700 1 -- 172.21.15.64:0/4120570151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc00b860 con 0x7f340c154780 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34110dd700 1 -- 172.21.15.64:0/4120570151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00bbe0 con 0x7f340c154780 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 -- 172.21.15.64:0/4120570151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 msgr2=0x7f340c15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 --2- 172.21.15.64:0/4120570151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c15b2c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0098e0 tx=0x7f33fc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 -- 172.21.15.64:0/4120570151 shutdown_connections 2023-09-27T23:26:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 --2- 172.21.15.64:0/4120570151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c15b2c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 -- 172.21.15.64:0/4120570151 >> 172.21.15.64:0/4120570151 conn(0x7f340c067b50 msgr2=0x7f340c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 -- 172.21.15.64:0/4120570151 shutdown_connections 2023-09-27T23:26:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 -- 172.21.15.64:0/4120570151 wait complete. 2023-09-27T23:26:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 Processor -- start 2023-09-27T23:26:57.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.567+0000 7f34130e1700 1 -- start start 2023-09-27T23:26:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f340c06baa0 0x7f340c16a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f340c0adab0 0x7f340c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c1692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c16c720 con 0x7f340c154780 2023-09-27T23:26:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c5fd0 con 0x7f340c06baa0 2023-09-27T23:26:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c62b0 con 0x7f340c0adab0 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c1692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c1692e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49088/0 (socket says 172.21.15.64:49088) 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 -- 172.21.15.64:0/610646560 learned_addr learned my addr 172.21.15.64:0/610646560 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34118de700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f340c0adab0 0x7f340c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34120df700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f340c06baa0 0x7f340c16a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 -- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f340c0adab0 msgr2=0x7f340c147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f340c0adab0 0x7f340c147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 -- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f340c06baa0 msgr2=0x7f340c16a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f340c06baa0 0x7f340c16a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 -- 172.21.15.64:0/610646560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc009420 con 0x7f340c154780 2023-09-27T23:26:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34120df700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f340c06baa0 0x7f340c16a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:26:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34128e0700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c1692e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f340400c590 tx=0x7f3404007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404011590 con 0x7f340c154780 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34040116f0 con 0x7f340c154780 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c14fc30 con 0x7f340c154780 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404011ab0 con 0x7f340c154780 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c169e60 con 0x7f340c154780 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.571+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c04be40 con 0x7f340c154780 2023-09-27T23:26:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.575+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f340401b3b0 con 0x7f340c154780 2023-09-27T23:26:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.575+0000 7f340a7fc700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f33f8060a70 0x7f33f8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.575+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f340407c5f0 con 0x7f340c154780 2023-09-27T23:26:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.575+0000 7f34120df700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f33f8060a70 0x7f33f8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:57.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.575+0000 7f34120df700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f33f8060a70 0x7f33f8062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33fc003080 tx=0x7f33fc017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:57.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.583+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f340404c620 con 0x7f340c154780 2023-09-27T23:26:57.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:57.779+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f340c0c63f0 con 0x7f340c154780 2023-09-27T23:26:58.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.791+0000 7f340a7fc700 1 -- 172.21.15.64:0/610646560 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7f34040170b0 con 0x7f340c154780 2023-09-27T23:26:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f33f8060a70 msgr2=0x7f33f8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f33f8060a70 0x7f33f8062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33fc003080 tx=0x7f33fc017000 comp rx=0 tx=0).stop 2023-09-27T23:26:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 msgr2=0x7f340c1692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c1692e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f340400c590 tx=0x7f3404007ef0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 shutdown_connections 2023-09-27T23:26:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f33f8060a70 0x7f33f8062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340c154780 0x7f340c1692e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f340c0adab0 0x7f340c147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 --2- 172.21.15.64:0/610646560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f340c06baa0 0x7f340c16a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 >> 172.21.15.64:0/610646560 conn(0x7f340c067b50 msgr2=0x7f340c0c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 shutdown_connections 2023-09-27T23:26:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.795+0000 7f34130e1700 1 -- 172.21.15.64:0/610646560 wait complete. 2023-09-27T23:26:58.816 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:26:58.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:26:58.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.935+0000 7f0b04a93700 1 Processor -- start 2023-09-27T23:26:58.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.935+0000 7f0b04a93700 1 -- start start 2023-09-27T23:26:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b00150200 0x7f0b000c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b000ae770 0x7f0b000aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b000af250 con 0x7f0b000ac720 2023-09-27T23:26:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b000af530 con 0x7f0b000ae770 2023-09-27T23:26:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000ca8a0 con 0x7f0b00150200 2023-09-27T23:26:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b000ae770 0x7f0b000aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b000ae770 0x7f0b000aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49098/0 (socket says 172.21.15.64:49098) 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 -- 172.21.15.64:0/3788116881 learned_addr learned my addr 172.21.15.64:0/3788116881 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afffff700 1 --1- 172.21.15.64:0/3788116881 >> v1:172.21.15.64:6789/0 conn(0x7f0b000ac720 0x7f0b000cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48018/0 (socket says 172.21.15.64:48018) 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0aff7fe700 1 --2- 172.21.15.64:0/3788116881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b00150200 0x7f0b000c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 -- 172.21.15.64:0/3788116881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b00150200 msgr2=0x7f0b000c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 --2- 172.21.15.64:0/3788116881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b00150200 0x7f0b000c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 -- 172.21.15.64:0/3788116881 >> v1:172.21.15.64:6789/0 conn(0x7f0b000ac720 legacy=0x7f0b000cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 -- 172.21.15.64:0/3788116881 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b000cab80 con 0x7f0b000ae770 2023-09-27T23:26:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afe7fc700 1 -- 172.21.15.64:0/3788116881 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383702958 0 0) 0x7f0b000af250 con 0x7f0b000ac720 2023-09-27T23:26:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afeffd700 1 --2- 172.21.15.64:0/3788116881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b000ae770 0x7f0b000aebe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0af40099b0 tx=0x7f0af4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70dd4d59deef51 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afe7fc700 1 -- 172.21.15.64:0/3788116881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4009ea0 con 0x7f0b000ae770 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afe7fc700 1 -- 172.21.15.64:0/3788116881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af400b860 con 0x7f0b000ae770 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0afe7fc700 1 -- 172.21.15.64:0/3788116881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400bc00 con 0x7f0b000ae770 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- 172.21.15.64:0/3788116881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b000ae770 msgr2=0x7f0b000aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 --2- 172.21.15.64:0/3788116881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b000ae770 0x7f0b000aebe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0af40099b0 tx=0x7f0af4009250 comp rx=0 tx=0).stop 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- 172.21.15.64:0/3788116881 shutdown_connections 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 --2- 172.21.15.64:0/3788116881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b000ae770 0x7f0b000aebe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 --2- 172.21.15.64:0/3788116881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b00150200 0x7f0b000c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- 172.21.15.64:0/3788116881 >> 172.21.15.64:0/3788116881 conn(0x7f0b000ccb30 msgr2=0x7f0b000ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- 172.21.15.64:0/3788116881 shutdown_connections 2023-09-27T23:26:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.939+0000 7f0b04a93700 1 -- 172.21.15.64:0/3788116881 wait complete. 2023-09-27T23:26:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 Processor -- start 2023-09-27T23:26:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 -- start start 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b000ac720 0x7f0b000a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b000ae770 0x7f0b0014ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 0x7f0b0014b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000a8850 con 0x7f0b00150200 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0014b8e0 con 0x7f0b000ae770 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0b04a93700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000a8aa0 con 0x7f0b000ac720 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 0x7f0b0014b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 0x7f0b0014b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49112/0 (socket says 172.21.15.64:49112) 2023-09-27T23:26:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 -- 172.21.15.64:0/195876466 learned_addr learned my addr 172.21.15.64:0/195876466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0aff7fe700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b000ac720 0x7f0b000a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:58.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afeffd700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b000ae770 0x7f0b0014ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 -- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b000ac720 msgr2=0x7f0b000a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b000ac720 0x7f0b000a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 -- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b000ae770 msgr2=0x7f0b0014ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b000ae770 0x7f0b0014ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.943+0000 7f0afffff700 1 -- 172.21.15.64:0/195876466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40094f0 con 0x7f0b00150200 2023-09-27T23:26:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0afffff700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 0x7f0b0014b3a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c910 tx=0x7f0af80018e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8004da0 con 0x7f0b00150200 2023-09-27T23:26:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af80060f0 con 0x7f0b00150200 2023-09-27T23:26:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b000a8d80 con 0x7f0b00150200 2023-09-27T23:26:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af80064b0 con 0x7f0b00150200 2023-09-27T23:26:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b000a9340 con 0x7f0b00150200 2023-09-27T23:26:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b000a9530 con 0x7f0b00150200 2023-09-27T23:26:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0af80068e0 con 0x7f0b00150200 2023-09-27T23:26:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0ae77fe700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0aec060a70 0x7f0aec062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.947+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0af807cdf0 con 0x7f0b00150200 2023-09-27T23:26:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.951+0000 7f0aff7fe700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0aec060a70 0x7f0aec062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.951+0000 7f0aff7fe700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0aec060a70 0x7f0aec062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009d10 tx=0x7f0af00091a0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:58.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:58.955+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0af804cea0 con 0x7f0b00150200 2023-09-27T23:26:59.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.151+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0b0004be40 con 0x7f0b00150200 2023-09-27T23:26:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.803+0000 7f0ae77fe700 1 -- 172.21.15.64:0/195876466 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0af80504c0 con 0x7f0b00150200 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0aec060a70 msgr2=0x7f0aec062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0aec060a70 0x7f0aec062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009d10 tx=0x7f0af00091a0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 msgr2=0x7f0b0014b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 0x7f0b0014b3a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c910 tx=0x7f0af80018e0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 shutdown_connections 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0aec060a70 0x7f0aec062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00150200 0x7f0b0014b3a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b000ae770 0x7f0b0014ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 --2- 172.21.15.64:0/195876466 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b000ac720 0x7f0b000a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 >> 172.21.15.64:0/195876466 conn(0x7f0b000ccb30 msgr2=0x7f0b000c1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:59.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 shutdown_connections 2023-09-27T23:26:59.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.807+0000 7f0b04a93700 1 -- 172.21.15.64:0/195876466 wait complete. 2023-09-27T23:26:59.825 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:26:59.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:26:59.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 Processor -- start 2023-09-27T23:26:59.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 -- start start 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a9806d760 0x7f7a981591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 0x7f7a980c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a98159f90 con 0x7f7a98158df0 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9815a270 con 0x7f7a9806d760 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c7930 con 0x7f7a98159700 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7a98158df0 0x7f7a9806d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48026/0 (socket says 172.21.15.64:48026) 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9659c700 1 -- 172.21.15.64:0/3046780231 learned_addr learned my addr 172.21.15.64:0/3046780231 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a96d9d700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 0x7f7a980c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a95d9b700 1 --2- 172.21.15.64:0/3046780231 >> v2:172.21.15.123:3300/0 conn(0x7f7a9806d760 0x7f7a981591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35319602 0 0) 0x7f7a98159f90 con 0x7f7a98158df0 2023-09-27T23:26:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a78003570 con 0x7f7a98158df0 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 161860465 0 0) 0x7f7a78003570 con 0x7f7a98158df0 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 msgr2=0x7f7a980c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 0x7f7a980c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 >> v2:172.21.15.123:3300/0 conn(0x7f7a9806d760 msgr2=0x7f7a981591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 --2- 172.21.15.64:0/3046780231 >> v2:172.21.15.123:3300/0 conn(0x7f7a9806d760 0x7f7a981591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980c7c10 con 0x7f7a98158df0 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f7a880017f0 con 0x7f7a98158df0 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 >> v1:172.21.15.64:6789/0 conn(0x7f7a98158df0 legacy=0x7f7a9806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a78003a60 0x7f7a78003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a78004810 0x7f7a78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 0x7f7a780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9815a270 con 0x7f7a78003a60 2023-09-27T23:26:59.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c7930 con 0x7f7a78004810 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7800bae0 con 0x7f7a98159700 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a9659c700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a78003a60 0x7f7a78003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a95d9b700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a78004810 0x7f7a78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.939+0000 7f7a96d9d700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 0x7f7a780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9659c700 1 -- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 msgr2=0x7f7a780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9659c700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a98159700 0x7f7a780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9659c700 1 -- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a78004810 msgr2=0x7f7a78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9659c700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a78004810 0x7f7a78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9659c700 1 -- 172.21.15.64:0/3046780231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78003570 con 0x7f7a78003a60 2023-09-27T23:26:59.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9659c700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a78003a60 0x7f7a78003e70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800a970 tx=0x7f7a88002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e1cd734b6ae7b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88012440 con 0x7f7a78003a60 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a880125c0 con 0x7f7a78003a60 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9559a700 1 -- 172.21.15.64:0/3046780231 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88012980 con 0x7f7a78003a60 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9759e700 1 -- 172.21.15.64:0/3046780231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a78003a60 msgr2=0x7f7a78003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9759e700 1 --2- 172.21.15.64:0/3046780231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a78003a60 0x7f7a78003e70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800a970 tx=0x7f7a88002210 comp rx=0 tx=0).stop 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a96d9d700 1 -- 172.21.15.64:0/3046780231 reap_dead start 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9759e700 1 -- 172.21.15.64:0/3046780231 shutdown_connections 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9759e700 1 -- 172.21.15.64:0/3046780231 >> 172.21.15.64:0/3046780231 conn(0x7f7a9806c080 msgr2=0x7f7a980ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9759e700 1 -- 172.21.15.64:0/3046780231 shutdown_connections 2023-09-27T23:26:59.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.943+0000 7f7a9759e700 1 -- 172.21.15.64:0/3046780231 wait complete. 2023-09-27T23:26:59.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 Processor -- start 2023-09-27T23:26:59.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 -- start start 2023-09-27T23:26:59.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a98169a30 0x7f7a980b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 0x7f7a9816a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a980c6130 0x7f7a980c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98148e90 con 0x7f7a98169e00 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980b21f0 con 0x7f7a98169a30 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980b1820 con 0x7f7a980c6130 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 0x7f7a9816a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 0x7f7a9816a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49130/0 (socket says 172.21.15.64:49130) 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 -- 172.21.15.64:0/26902847 learned_addr learned my addr 172.21.15.64:0/26902847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a9659c700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a98169a30 0x7f7a980b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a96d9d700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a980c6130 0x7f7a980c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 -- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a980c6130 msgr2=0x7f7a980c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:59.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a980c6130 0x7f7a980c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 -- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a98169a30 msgr2=0x7f7a980b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:26:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a98169a30 0x7f7a980b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:26:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.947+0000 7f7a95d9b700 1 -- 172.21.15.64:0/26902847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980b0c10 con 0x7f7a98169e00 2023-09-27T23:26:59.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a95d9b700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 0x7f7a9816a270 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7a80011b50 tx=0x7f7a800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000f8e0 con 0x7f7a98169e00 2023-09-27T23:26:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8000fa40 con 0x7f7a98169e00 2023-09-27T23:26:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a980b0ef0 con 0x7f7a98169e00 2023-09-27T23:26:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000fdc0 con 0x7f7a98169e00 2023-09-27T23:26:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a980b14b0 con 0x7f7a98169e00 2023-09-27T23:26:59.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a980b19b0 con 0x7f7a98169e00 2023-09-27T23:26:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7a80014650 con 0x7f7a98169e00 2023-09-27T23:26:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a8e7fc700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a7c060aa0 0x7f7a7c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:26:59.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.951+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7a80018030 con 0x7f7a98169e00 2023-09-27T23:26:59.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.955+0000 7f7a9659c700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a7c060aa0 0x7f7a7c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:26:59.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.955+0000 7f7a9659c700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a7c060aa0 0x7f7a7c062f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a9806afd0 tx=0x7f7a88015000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:26:59.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:26:59.959+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a8004d910 con 0x7f7a98169e00 2023-09-27T23:27:00.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.159+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a9804be40 con 0x7f7a98169e00 2023-09-27T23:27:00.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a8e7fc700 1 -- 172.21.15.64:0/26902847 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f7a80050f30 con 0x7f7a98169e00 2023-09-27T23:27:00.164 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:00.164 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:26:56.779553+0000","modified":"2023-09-27T23:26:59.790212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2178143597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2178143597},{"type":"v1","addr":"172.21.15.64:6835","nonce":2178143597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-09-27T23:27:00.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a7c060aa0 msgr2=0x7f7a7c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a7c060aa0 0x7f7a7c062f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a9806afd0 tx=0x7f7a88015000 comp rx=0 tx=0).stop 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 msgr2=0x7f7a9816a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 0x7f7a9816a270 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7a80011b50 tx=0x7f7a800091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 shutdown_connections 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a7c060aa0 0x7f7a7c062f50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a980c6130 0x7f7a980c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a98169e00 0x7f7a9816a270 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 --2- 172.21.15.64:0/26902847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a98169a30 0x7f7a980b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 >> 172.21.15.64:0/26902847 conn(0x7f7a9806c080 msgr2=0x7f7a980cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:00.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 shutdown_connections 2023-09-27T23:27:00.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.163+0000 7f7a9759e700 1 -- 172.21.15.64:0/26902847 wait complete. 2023-09-27T23:27:00.168 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-09-27T23:27:00.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:27:00.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 Processor -- start 2023-09-27T23:27:00.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 -- start start 2023-09-27T23:27:00.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 0x7f825c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f825c06de20 0x7f825c0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c06d990 con 0x7f825c155960 2023-09-27T23:27:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c15a570 con 0x7f825c06de20 2023-09-27T23:27:00.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826253e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c06cd80 con 0x7f825c06c7c0 2023-09-27T23:27:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826153c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 0x7f825c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826153c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 0x7f825c15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49138/0 (socket says 172.21.15.64:49138) 2023-09-27T23:27:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 --2- 172.21.15.64:0/1358438676 >> v2:172.21.15.123:3300/0 conn(0x7f825c06de20 0x7f825c0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826153c700 1 -- 172.21.15.64:0/1358438676 learned_addr learned my addr 172.21.15.64:0/1358438676 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8261d3d700 1 --2- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 -- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 msgr2=0x7f825c0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 --2- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 -- 172.21.15.64:0/1358438676 >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 msgr2=0x7f825c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 --2- 172.21.15.64:0/1358438676 >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 0x7f825c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 -- 172.21.15.64:0/1358438676 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c06d060 con 0x7f825c06de20 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f826153c700 1 --2- 172.21.15.64:0/1358438676 >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 0x7f825c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8260d3b700 1 --2- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c0c77f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f825c06bf60 tx=0x7f824c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924731a7b6ffb427 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8253fff700 1 -- 172.21.15.64:0/1358438676 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00f6b0 con 0x7f825c06de20 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8253fff700 1 -- 172.21.15.64:0/1358438676 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00f830 con 0x7f825c06de20 2023-09-27T23:27:00.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.287+0000 7f8253fff700 1 -- 172.21.15.64:0/1358438676 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00fbf0 con 0x7f825c06de20 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 -- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 msgr2=0x7f825c0c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 --2- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c0c77f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f825c06bf60 tx=0x7f824c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 -- 172.21.15.64:0/1358438676 shutdown_connections 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 --2- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 --2- 172.21.15.64:0/1358438676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c0c77f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 --2- 172.21.15.64:0/1358438676 >> v2:172.21.15.64:3300/0 conn(0x7f825c155960 0x7f825c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 -- 172.21.15.64:0/1358438676 >> 172.21.15.64:0/1358438676 conn(0x7f825c067bb0 msgr2=0x7f825c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 -- 172.21.15.64:0/1358438676 shutdown_connections 2023-09-27T23:27:00.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 -- 172.21.15.64:0/1358438676 wait complete. 2023-09-27T23:27:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 Processor -- start 2023-09-27T23:27:00.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.291+0000 7f826253e700 1 -- start start 2023-09-27T23:27:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f825c155960 0x7f825c1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c14fb20 con 0x7f825c155960 2023-09-27T23:27:00.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c14e320 con 0x7f825c06de20 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c14e600 con 0x7f825c06c7c0 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f8261d3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f825c155960 0x7f825c1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c14dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:33170/0 (socket says 172.21.15.64:33170) 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 -- 172.21.15.64:0/4139365836 learned_addr learned my addr 172.21.15.64:0/4139365836 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f8260d3b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 -- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 msgr2=0x7f825c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c14f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 -- 172.21.15.64:0/4139365836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f825c155960 msgr2=0x7f825c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f825c155960 0x7f825c1535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 -- 172.21.15.64:0/4139365836 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580094f0 con 0x7f825c06c7c0 2023-09-27T23:27:00.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826153c700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c14dc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f825800ec50 tx=0x7f8258009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800c850 con 0x7f825c06c7c0 2023-09-27T23:27:00.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825800c9b0 con 0x7f825c06c7c0 2023-09-27T23:27:00.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824c009420 con 0x7f825c06c7c0 2023-09-27T23:27:00.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800cd30 con 0x7f825c06c7c0 2023-09-27T23:27:00.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c14ebc0 con 0x7f825c06c7c0 2023-09-27T23:27:00.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.295+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c04be40 con 0x7f825c06c7c0 2023-09-27T23:27:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.299+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8258013680 con 0x7f825c06c7c0 2023-09-27T23:27:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.299+0000 7f82517fa700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8240060a70 0x7f8240062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.299+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f825807d3d0 con 0x7f825c06c7c0 2023-09-27T23:27:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.299+0000 7f8260d3b700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8240060a70 0x7f8240062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.299+0000 7f8260d3b700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8240060a70 0x7f8240062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f825c06bf60 tx=0x7f824c019000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.307+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f825804d360 con 0x7f825c06c7c0 2023-09-27T23:27:00.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.475+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f825c06d340 con 0x7f825c06c7c0 2023-09-27T23:27:00.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.475+0000 7f82517fa700 1 -- 172.21.15.64:0/4139365836 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17071 (secure 0 0 0) 0x7f8258050980 con 0x7f825c06c7c0 2023-09-27T23:27:00.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:00.479 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":71,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:26:57.783526+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:26:55.337700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-27T23:26:55.749823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:27:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8240060a70 msgr2=0x7f8240062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8240060a70 0x7f8240062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f825c06bf60 tx=0x7f824c019000 comp rx=0 tx=0).stop 2023-09-27T23:27:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 msgr2=0x7f825c14dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c14dc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f825800ec50 tx=0x7f8258009250 comp rx=0 tx=0).stop 2023-09-27T23:27:00.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 shutdown_connections 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8240060a70 0x7f8240062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f825c155960 0x7f825c1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f825c06de20 0x7f825c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 --2- 172.21.15.64:0/4139365836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f825c06c7c0 0x7f825c14dc90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 >> 172.21.15.64:0/4139365836 conn(0x7f825c067bb0 msgr2=0x7f825c0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 shutdown_connections 2023-09-27T23:27:00.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.479+0000 7f826253e700 1 -- 172.21.15.64:0/4139365836 wait complete. 2023-09-27T23:27:00.498 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:27:00.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:27:00.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.599+0000 7f09fa570700 1 Processor -- start 2023-09-27T23:27:00.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.599+0000 7f09fa570700 1 -- start start 2023-09-27T23:27:00.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09f40acc30 0x7f09f40ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c8280 0x7f09f40c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f40cc920 con 0x7f09f40ad600 2023-09-27T23:27:00.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f40c90a0 con 0x7f09f40acc30 2023-09-27T23:27:00.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40c9290 con 0x7f09f40c8280 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f9d6f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f09f40ad600 0x7f09f415a6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48032/0 (socket says 172.21.15.64:48032) 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f9d6f700 1 -- 172.21.15.64:0/1278238467 learned_addr learned my addr 172.21.15.64:0/1278238467 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f956e700 1 --2- 172.21.15.64:0/1278238467 >> v2:172.21.15.123:3300/0 conn(0x7f09f40acc30 0x7f09f40ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f8d6d700 1 --2- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c8280 0x7f09f40c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f956e700 1 -- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c8280 msgr2=0x7f09f40c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f956e700 1 --2- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c8280 0x7f09f40c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09ebfff700 1 -- 172.21.15.64:0/1278238467 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415753358 0 0) 0x7f09f40cc920 con 0x7f09f40ad600 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f956e700 1 -- 172.21.15.64:0/1278238467 >> v1:172.21.15.64:6789/0 conn(0x7f09f40ad600 legacy=0x7f09f415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f956e700 1 -- 172.21.15.64:0/1278238467 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40c86d0 con 0x7f09f40acc30 2023-09-27T23:27:00.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09f956e700 1 --2- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40ad000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09f00098e0 tx=0x7f09f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d9e7999e82d1a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09ebfff700 1 -- 172.21.15.64:0/1278238467 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000b6d0 con 0x7f09f40acc30 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09ebfff700 1 -- 172.21.15.64:0/1278238467 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f000b830 con 0x7f09f40acc30 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09ebfff700 1 -- 172.21.15.64:0/1278238467 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000bbd0 con 0x7f09f40acc30 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 msgr2=0x7f09f40ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 --2- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40ad000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09f00098e0 tx=0x7f09f00091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- 172.21.15.64:0/1278238467 shutdown_connections 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 --2- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40c8280 0x7f09f40c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 --2- 172.21.15.64:0/1278238467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40ad000 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- 172.21.15.64:0/1278238467 >> 172.21.15.64:0/1278238467 conn(0x7f09f40cdbd0 msgr2=0x7f09f40cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:00.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- 172.21.15.64:0/1278238467 shutdown_connections 2023-09-27T23:27:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.603+0000 7f09fa570700 1 -- 172.21.15.64:0/1278238467 wait complete. 2023-09-27T23:27:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 Processor -- start 2023-09-27T23:27:00.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 -- start start 2023-09-27T23:27:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40ad600 0x7f09f415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 0x7f09f415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f416b570 con 0x7f09f40c8280 2023-09-27T23:27:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40c5b40 con 0x7f09f40acc30 2023-09-27T23:27:00.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f416a7b0 con 0x7f09f40ad600 2023-09-27T23:27:00.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 0x7f09f415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 0x7f09f415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49162/0 (socket says 172.21.15.64:49162) 2023-09-27T23:27:00.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 -- 172.21.15.64:0/308308060 learned_addr learned my addr 172.21.15.64:0/308308060 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:00.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f8d6d700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40ad600 0x7f09f415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f956e700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 -- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40ad600 msgr2=0x7f09f415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40ad600 0x7f09f415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 -- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 msgr2=0x7f09f40c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 -- 172.21.15.64:0/308308060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f0009420 con 0x7f09f40c8280 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09f9d6f700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 0x7f09f415a8e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00cc90 tx=0x7f09ec001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00a8b0 con 0x7f09f40c8280 2023-09-27T23:27:00.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec00aa10 con 0x7f09f40c8280 2023-09-27T23:27:00.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f416aa90 con 0x7f09f40c8280 2023-09-27T23:27:00.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00add0 con 0x7f09f40c8280 2023-09-27T23:27:00.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.607+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f416b050 con 0x7f09f40c8280 2023-09-27T23:27:00.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.611+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f416b270 con 0x7f09f40c8280 2023-09-27T23:27:00.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.611+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f09ec007300 con 0x7f09f40c8280 2023-09-27T23:27:00.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.611+0000 7f09e97fa700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d8060a70 0x7f09d8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.611+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f09ec07d540 con 0x7f09f40c8280 2023-09-27T23:27:00.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.611+0000 7f09f956e700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d8060a70 0x7f09d8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.611+0000 7f09f956e700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d8060a70 0x7f09d8062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09f00093f0 tx=0x7f09f0017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.619+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f09ec04d4d0 con 0x7f09f40c8280 2023-09-27T23:27:00.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.771+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09f404be40 con 0x7f09d8060a70 2023-09-27T23:27:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.791+0000 7f09e97fa700 1 -- 172.21.15.64:0/308308060 <== mgr.5402 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295732 (secure 0 0 0) 0x7f09f404be40 con 0x7f09d8060a70 2023-09-27T23:27:00.797 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:00.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d8060a70 msgr2=0x7f09d8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d8060a70 0x7f09d8062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09f00093f0 tx=0x7f09f0017000 comp rx=0 tx=0).stop 2023-09-27T23:27:00.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 msgr2=0x7f09f415a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 0x7f09f415a8e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00cc90 tx=0x7f09ec001a80 comp rx=0 tx=0).stop 2023-09-27T23:27:00.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 shutdown_connections 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f09d8060a70 0x7f09d8062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09f40c8280 0x7f09f415a8e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09f40ad600 0x7f09f415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 --2- 172.21.15.64:0/308308060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f40acc30 0x7f09f40c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 >> 172.21.15.64:0/308308060 conn(0x7f09f40cdbd0 msgr2=0x7f09f414af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 shutdown_connections 2023-09-27T23:27:00.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.799+0000 7f09fa570700 1 -- 172.21.15.64:0/308308060 wait complete. 2023-09-27T23:27:00.803 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:27:00.816 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-27T23:26:59.764594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":202,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":193268,"kb_used_data":7476,"kb_used_omap":0,"kb_used_meta":185728,"kb_avail":754781452,"statfs":{"total":773094113280,"available":772896206848,"internally_reserved":0,"allocated":7655424,"data_stored":4410976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":190185472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367720"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.814138+0000","last_change":"2023-09-27T23:26:56.811633+0000","last_active":"2023-09-27T23:26:56.814138+0000","last_peered":"2023-09-27T23:26:56.814138+0000","last_clean":"2023-09-27T23:26:56.814138+0000","last_became_active":"2023-09-27T23:26:56.811088+0000","last_became_peered":"2023-09-27T23:26:56.811088+0000","last_unstale":"2023-09-27T23:26:56.814138+0000","last_undegraded":"2023-09-27T23:26:56.814138+0000","last_fullsized":"2023-09-27T23:26:56.814138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:46.867145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800866+0000","last_change":"2023-09-27T23:26:56.796931+0000","last_active":"2023-09-27T23:26:56.800866+0000","last_peered":"2023-09-27T23:26:56.800866+0000","last_clean":"2023-09-27T23:26:56.800866+0000","last_became_active":"2023-09-27T23:26:56.796639+0000","last_became_peered":"2023-09-27T23:26:56.796639+0000","last_unstale":"2023-09-27T23:26:56.800866+0000","last_undegraded":"2023-09-27T23:26:56.800866+0000","last_fullsized":"2023-09-27T23:26:56.800866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:13:35.626367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824033+0000","last_change":"2023-09-27T23:26:56.812859+0000","last_active":"2023-09-27T23:26:56.824033+0000","last_peered":"2023-09-27T23:26:56.824033+0000","last_clean":"2023-09-27T23:26:56.824033+0000","last_became_active":"2023-09-27T23:26:56.811956+0000","last_became_peered":"2023-09-27T23:26:56.811956+0000","last_unstale":"2023-09-27T23:26:56.824033+0000","last_undegraded":"2023-09-27T23:26:56.824033+0000","last_fullsized":"2023-09-27T23:26:56.824033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:29.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816795+0000","last_change":"2023-09-27T23:26:56.805621+0000","last_active":"2023-09-27T23:26:56.816795+0000","last_peered":"2023-09-27T23:26:56.816795+0000","last_clean":"2023-09-27T23:26:56.816795+0000","last_became_active":"2023-09-27T23:26:56.805331+0000","last_became_peered":"2023-09-27T23:26:56.805331+0000","last_unstale":"2023-09-27T23:26:56.816795+0000","last_undegraded":"2023-09-27T23:26:56.816795+0000","last_fullsized":"2023-09-27T23:26:56.816795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:05.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810612+0000","last_change":"2023-09-27T23:26:56.805480+0000","last_active":"2023-09-27T23:26:56.810612+0000","last_peered":"2023-09-27T23:26:56.810612+0000","last_clean":"2023-09-27T23:26:56.810612+0000","last_became_active":"2023-09-27T23:26:56.805168+0000","last_became_peered":"2023-09-27T23:26:56.805168+0000","last_unstale":"2023-09-27T23:26:56.810612+0000","last_undegraded":"2023-09-27T23:26:56.810612+0000","last_fullsized":"2023-09-27T23:26:56.810612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:41.801806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818742+0000","last_change":"2023-09-27T23:26:56.814879+0000","last_active":"2023-09-27T23:26:56.818742+0000","last_peered":"2023-09-27T23:26:56.818742+0000","last_clean":"2023-09-27T23:26:56.818742+0000","last_became_active":"2023-09-27T23:26:56.814574+0000","last_became_peered":"2023-09-27T23:26:56.814574+0000","last_unstale":"2023-09-27T23:26:56.818742+0000","last_undegraded":"2023-09-27T23:26:56.818742+0000","last_fullsized":"2023-09-27T23:26:56.818742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:38.070621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.802160+0000","last_change":"2023-09-27T23:26:56.800806+0000","last_active":"2023-09-27T23:26:56.802160+0000","last_peered":"2023-09-27T23:26:56.802160+0000","last_clean":"2023-09-27T23:26:56.802160+0000","last_became_active":"2023-09-27T23:26:56.800324+0000","last_became_peered":"2023-09-27T23:26:56.800324+0000","last_unstale":"2023-09-27T23:26:56.802160+0000","last_undegraded":"2023-09-27T23:26:56.802160+0000","last_fullsized":"2023-09-27T23:26:56.802160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:58.847869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819184+0000","last_change":"2023-09-27T23:26:56.817217+0000","last_active":"2023-09-27T23:26:56.819184+0000","last_peered":"2023-09-27T23:26:56.819184+0000","last_clean":"2023-09-27T23:26:56.819184+0000","last_became_active":"2023-09-27T23:26:56.816937+0000","last_became_peered":"2023-09-27T23:26:56.816937+0000","last_unstale":"2023-09-27T23:26:56.819184+0000","last_undegraded":"2023-09-27T23:26:56.819184+0000","last_fullsized":"2023-09-27T23:26:56.819184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:17:55.307710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824218+0000","last_change":"2023-09-27T23:26:56.814262+0000","last_active":"2023-09-27T23:26:56.824218+0000","last_peered":"2023-09-27T23:26:56.824218+0000","last_clean":"2023-09-27T23:26:56.824218+0000","last_became_active":"2023-09-27T23:26:56.813928+0000","last_became_peered":"2023-09-27T23:26:56.813928+0000","last_unstale":"2023-09-27T23:26:56.824218+0000","last_undegraded":"2023-09-27T23:26:56.824218+0000","last_fullsized":"2023-09-27T23:26:56.824218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:49.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824186+0000","last_change":"2023-09-27T23:26:56.816043+0000","last_active":"2023-09-27T23:26:56.824186+0000","last_peered":"2023-09-27T23:26:56.824186+0000","last_clean":"2023-09-27T23:26:56.824186+0000","last_became_active":"2023-09-27T23:26:56.815712+0000","last_became_peered":"2023-09-27T23:26:56.815712+0000","last_unstale":"2023-09-27T23:26:56.824186+0000","last_undegraded":"2023-09-27T23:26:56.824186+0000","last_fullsized":"2023-09-27T23:26:56.824186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:54.503767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817510+0000","last_change":"2023-09-27T23:26:56.433280+0000","last_active":"2023-09-27T23:26:56.817510+0000","last_peered":"2023-09-27T23:26:56.817510+0000","last_clean":"2023-09-27T23:26:56.817510+0000","last_became_active":"2023-09-27T23:26:56.432946+0000","last_became_peered":"2023-09-27T23:26:56.432946+0000","last_unstale":"2023-09-27T23:26:56.817510+0000","last_undegraded":"2023-09-27T23:26:56.817510+0000","last_fullsized":"2023-09-27T23:26:56.817510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:05:40.954906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824360+0000","last_change":"2023-09-27T23:26:56.815852+0000","last_active":"2023-09-27T23:26:56.824360+0000","last_peered":"2023-09-27T23:26:56.824360+0000","last_clean":"2023-09-27T23:26:56.824360+0000","last_became_active":"2023-09-27T23:26:56.815565+0000","last_became_peered":"2023-09-27T23:26:56.815565+0000","last_unstale":"2023-09-27T23:26:56.824360+0000","last_undegraded":"2023-09-27T23:26:56.824360+0000","last_fullsized":"2023-09-27T23:26:56.824360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:21:59.389594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799681+0000","last_change":"2023-09-27T23:26:56.431567+0000","last_active":"2023-09-27T23:26:56.799681+0000","last_peered":"2023-09-27T23:26:56.799681+0000","last_clean":"2023-09-27T23:26:56.799681+0000","last_became_active":"2023-09-27T23:26:56.431206+0000","last_became_peered":"2023-09-27T23:26:56.431206+0000","last_unstale":"2023-09-27T23:26:56.799681+0000","last_undegraded":"2023-09-27T23:26:56.799681+0000","last_fullsized":"2023-09-27T23:26:56.799681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:52:27.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824213+0000","last_change":"2023-09-27T23:26:56.811986+0000","last_active":"2023-09-27T23:26:56.824213+0000","last_peered":"2023-09-27T23:26:56.824213+0000","last_clean":"2023-09-27T23:26:56.824213+0000","last_became_active":"2023-09-27T23:26:56.811704+0000","last_became_peered":"2023-09-27T23:26:56.811704+0000","last_unstale":"2023-09-27T23:26:56.824213+0000","last_undegraded":"2023-09-27T23:26:56.824213+0000","last_fullsized":"2023-09-27T23:26:56.824213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:10:10.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.797203+0000","last_change":"2023-09-27T23:26:56.439391+0000","last_active":"2023-09-27T23:26:56.797203+0000","last_peered":"2023-09-27T23:26:56.797203+0000","last_clean":"2023-09-27T23:26:56.797203+0000","last_became_active":"2023-09-27T23:26:56.439021+0000","last_became_peered":"2023-09-27T23:26:56.439021+0000","last_unstale":"2023-09-27T23:26:56.797203+0000","last_undegraded":"2023-09-27T23:26:56.797203+0000","last_fullsized":"2023-09-27T23:26:56.797203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:59.110216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":285,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643461+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:26:52.643461+0000","last_peered":"2023-09-27T23:26:52.643461+0000","last_clean":"2023-09-27T23:26:52.643461+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:26:52.643461+0000","last_undegraded":"2023-09-27T23:26:52.643461+0000","last_fullsized":"2023-09-27T23:26:52.643461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":139,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.311206+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:26:52.311206+0000","last_peered":"2023-09-27T23:26:52.311206+0000","last_clean":"2023-09-27T23:26:52.311206+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:26:52.311206+0000","last_undegraded":"2023-09-27T23:26:52.311206+0000","last_fullsized":"2023-09-27T23:26:52.311206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798523+0000","last_change":"2023-09-27T23:26:56.428852+0000","last_active":"2023-09-27T23:26:56.798523+0000","last_peered":"2023-09-27T23:26:56.798523+0000","last_clean":"2023-09-27T23:26:56.798523+0000","last_became_active":"2023-09-27T23:26:56.428486+0000","last_became_peered":"2023-09-27T23:26:56.428486+0000","last_unstale":"2023-09-27T23:26:56.798523+0000","last_undegraded":"2023-09-27T23:26:56.798523+0000","last_fullsized":"2023-09-27T23:26:56.798523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:23:54.572803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643321+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:26:52.643321+0000","last_peered":"2023-09-27T23:26:52.643321+0000","last_clean":"2023-09-27T23:26:52.643321+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:26:52.643321+0000","last_undegraded":"2023-09-27T23:26:52.643321+0000","last_fullsized":"2023-09-27T23:26:52.643321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.787944+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:26:56.787944+0000","last_peered":"2023-09-27T23:26:56.787944+0000","last_clean":"2023-09-27T23:26:56.787944+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:26:56.787944+0000","last_undegraded":"2023-09-27T23:26:56.787944+0000","last_fullsized":"2023-09-27T23:26:56.787944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.311130+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:26:52.311130+0000","last_peered":"2023-09-27T23:26:52.311130+0000","last_clean":"2023-09-27T23:26:52.311130+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:26:52.311130+0000","last_undegraded":"2023-09-27T23:26:52.311130+0000","last_fullsized":"2023-09-27T23:26:52.311130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799914+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:26:56.799914+0000","last_peered":"2023-09-27T23:26:56.799914+0000","last_clean":"2023-09-27T23:26:56.799914+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:26:56.799914+0000","last_undegraded":"2023-09-27T23:26:56.799914+0000","last_fullsized":"2023-09-27T23:26:56.799914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643342+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:26:52.643342+0000","last_peered":"2023-09-27T23:26:52.643342+0000","last_clean":"2023-09-27T23:26:52.643342+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:26:52.643342+0000","last_undegraded":"2023-09-27T23:26:52.643342+0000","last_fullsized":"2023-09-27T23:26:52.643342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799800+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:26:56.799800+0000","last_peered":"2023-09-27T23:26:56.799800+0000","last_clean":"2023-09-27T23:26:56.799800+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:26:56.799800+0000","last_undegraded":"2023-09-27T23:26:56.799800+0000","last_fullsized":"2023-09-27T23:26:56.799800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:53.216004+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:26:53.216004+0000","last_peered":"2023-09-27T23:26:53.216004+0000","last_clean":"2023-09-27T23:26:53.216004+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:26:53.216004+0000","last_undegraded":"2023-09-27T23:26:53.216004+0000","last_fullsized":"2023-09-27T23:26:53.216004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509602,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24404,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":23040,"kb_avail":94347436,"statfs":{"total":96636764160,"available":96611774464,"internally_reserved":0,"allocated":1388544,"data_stored":987179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23592960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63700000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.232,"5min":1.363,"15min":1.7949999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.94199999999999995,"15min":0.88500000000000001},"min":{"1min":0.63,"5min":0.58199999999999996,"15min":0.47199999999999998},"max":{"1min":1.2210000000000001,"5min":1.363,"15min":1.363},"last":1.712}]},{"osd":1,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.89900000000000002,"15min":0.90000000000000002},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.92600000000000005,"15min":0.93200000000000005},"min":{"1min":0.67600000000000005,"5min":0.66500000000000004,"15min":0.56899999999999995},"max":{"1min":1.1479999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.7589999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91300000000000003,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.62,"15min":0.61299999999999999},"max":{"1min":1.1899999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":2.681},{"interface":"front","average":{"1min":1.0609999999999999,"5min":0.98999999999999999,"15min":0.96699999999999997},"min":{"1min":0.71899999999999997,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.5920000000000001},"last":1.2929999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.94999999999999996},"min":{"1min":0.72299999999999998,"5min":0.66300000000000003,"15min":0.65100000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.421},"last":0.995},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0840000000000001,"15min":1.044},"min":{"1min":0.79500000000000004,"5min":0.75,"15min":0.622},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.4339999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89100000000000001,"15min":0.91200000000000003},"min":{"1min":0.628,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.3879999999999999},"last":2.6230000000000002},{"interface":"front","average":{"1min":0.997,"5min":0.98299999999999998,"15min":0.98799999999999999},"min":{"1min":0.84699999999999998,"5min":0.67600000000000005,"15min":0.59599999999999997},"max":{"1min":1.1930000000000001,"5min":1.3080000000000001,"15min":1.4099999999999999},"last":2.8879999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.034,"15min":1.0569999999999999},"min":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.48999999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.5880000000000001},"last":1.391},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.0960000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.64100000000000001},"max":{"1min":1.395,"5min":1.51,"15min":1.51},"last":2.7280000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1200000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.78200000000000003},"max":{"1min":1.468,"5min":1.498,"15min":1.498},"last":1.806},{"interface":"front","average":{"1min":1.139,"5min":1.1399999999999999,"15min":1.133},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.73899999999999999},"max":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":2.548}]}]},{"osd":6,"up_from":15,"seq":64424509601,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20304,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":19520,"kb_avail":94351536,"statfs":{"total":96636764160,"available":96615972864,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19988480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.68899999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.327,"15min":1.327},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.079,"5min":1.026,"15min":1.0289999999999999},"min":{"1min":0.81999999999999995,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.649},"last":0.83899999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.54700000000000004},"max":{"1min":1.2889999999999999,"5min":1.323,"15min":1.323},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1739999999999999,"5min":1.2509999999999999,"15min":1.3939999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.88},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.56799999999999995},"max":{"1min":1.0820000000000001,"5min":1.353,"15min":1.397},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60499999999999998},"max":{"1min":1.2589999999999999,"5min":1.367,"15min":1.4259999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.58999999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.734},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.84599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"max":{"1min":1.2290000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.97199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.93100000000000005},"min":{"1min":0.64900000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":1.292,"5min":1.335,"15min":1.544},"last":0.874},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.61899999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.24,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":1.0269999999999999,"5min":0.83999999999999997,"15min":0.65900000000000003},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.095},{"interface":"front","average":{"1min":1.153,"5min":1.167,"15min":1.165},"min":{"1min":0.92800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3660000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1200000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509601,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30160,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":29376,"kb_avail":94341680,"statfs":{"total":96636764160,"available":96605880320,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30081024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.151,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.2929999999999999,"15min":1.359},"last":1.3440000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.50700000000000001},"max":{"1min":1.2969999999999999,"5min":1.3109999999999999,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.88500000000000001,"15min":0.876},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.234,"15min":1.4179999999999999},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.93000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3859999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"max":{"1min":1.27,"5min":1.4730000000000001,"15min":1.5740000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91900000000000004,"15min":0.97199999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.214,"5min":1.238,"15min":1.5640000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.008,"5min":1.024,"15min":1.014},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3819999999999999,"15min":1.954},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1599999999999999,"5min":1.2829999999999999,"15min":1.5349999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.032,"15min":1.046},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.304,"5min":1.3939999999999999,"15min":1.704},"last":1.399}]},{"osd":6,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.02,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.323,"5min":1.5580000000000001,"15min":1.6319999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.3779999999999999,"15min":1.837},"last":1.3069999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1479999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3,"5min":1.496,"15min":1.7230000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.204,"15min":1.1950000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.4319999999999999,"5min":1.633,"15min":2.0529999999999999},"last":1.425}]}]},{"osd":4,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":20624,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94351216,"statfs":{"total":96636764160,"available":96615645184,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.92100000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.57099999999999995,"15min":0.53700000000000003},"max":{"1min":1.256,"5min":1.27,"15min":1.3080000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90000000000000002,"15min":0.876},"min":{"1min":0.68500000000000005,"5min":0.63600000000000001,"15min":0.53800000000000003},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.3320000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.877,"15min":0.88800000000000001},"min":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.3380000000000001,"15min":1.3919999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95999999999999996,"15min":0.92400000000000004},"min":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.58199999999999996},"max":{"1min":1.171,"5min":1.228,"15min":1.329},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91500000000000004,"15min":0.90500000000000003},"min":{"1min":0.68300000000000005,"5min":0.63600000000000001,"15min":0.621},"max":{"1min":1.2090000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.268},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95199999999999996,"15min":0.98199999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.522},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92500000000000004,"15min":0.92700000000000005},"min":{"1min":0.72799999999999998,"5min":0.66100000000000003,"15min":0.60099999999999998},"max":{"1min":1.167,"5min":1.26,"15min":1.373},"last":1.006},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.012},"min":{"1min":0.77800000000000002,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.2929999999999999,"5min":1.5129999999999999,"15min":1.575},"last":1.0469999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89200000000000002,"15min":0.92000000000000004},"min":{"1min":0.626,"5min":0.60199999999999998,"15min":0.57599999999999996},"max":{"1min":1.2729999999999999,"5min":1.409,"15min":1.627},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.022,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.67200000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3220000000000001,"5min":1.5489999999999999,"15min":2.2730000000000001},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0249999999999999,"15min":1.0489999999999999},"min":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.60099999999999998},"max":{"1min":1.4059999999999999,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.369},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.149,"15min":1.1619999999999999},"min":{"1min":0.97499999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.478,"5min":1.478,"15min":1.8620000000000001},"last":1.137}]},{"osd":7,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0820000000000001,"15min":1.0940000000000001},"min":{"1min":0.83799999999999997,"5min":0.76500000000000001,"15min":0.64000000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.726},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1499999999999999,"15min":1.1759999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78500000000000003},"max":{"1min":1.5069999999999999,"5min":1.859,"15min":2.532},"last":1.4319999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21200,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350640,"statfs":{"total":96636764160,"available":96615055360,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.30199999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.746},"min":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.35299999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.337},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.39800000000000002},"max":{"1min":0.95299999999999996,"5min":1.224,"15min":1.2869999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.72199999999999998,"5min":0.61699999999999999,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.3180000000000001,"15min":1.3939999999999999},"last":0.75}]},{"osd":2,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.88300000000000001,"15min":0.85699999999999998},"min":{"1min":0.78200000000000003,"5min":0.65700000000000003,"15min":0.36699999999999999},"max":{"1min":1.1479999999999999,"5min":1.288,"15min":1.353},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91600000000000004},"min":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.45700000000000002},"max":{"1min":1.256,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.76000000000000001,"5min":0.67700000000000005,"15min":0.66100000000000003},"max":{"1min":1.175,"5min":1.466,"15min":1.5349999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.628},"max":{"1min":1.2030000000000001,"5min":1.3680000000000001,"15min":1.4950000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.91300000000000003,"5min":0.83099999999999996,"15min":0.626},"max":{"1min":1.3120000000000001,"5min":1.468,"15min":1.571},"last":1.02},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1479999999999999},"min":{"1min":0.96399999999999997,"5min":0.76700000000000002,"15min":0.58399999999999996},"max":{"1min":1.47,"5min":1.51,"15min":1.764},"last":1.252}]},{"osd":6,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.119,"15min":1.137},"min":{"1min":0.92600000000000005,"5min":0.76900000000000002,"15min":0.72799999999999998},"max":{"1min":1.407,"5min":1.4610000000000001,"15min":1.6970000000000001},"last":1.169},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":1.0269999999999999,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.3140000000000001,"5min":1.5800000000000001,"15min":1.8620000000000001},"last":1.1180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.246,"15min":1.2290000000000001},"min":{"1min":1.0289999999999999,"5min":0.86799999999999999,"15min":0.78700000000000003},"max":{"1min":1.5640000000000001,"5min":1.627,"15min":1.655},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.2},"min":{"1min":1.054,"5min":0.89500000000000002,"15min":0.73099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.671},"last":1.083}]}]},{"osd":2,"up_from":15,"seq":64424509603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20040,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":19200,"kb_avail":94351800,"statfs":{"total":96636764160,"available":96616243200,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19660800},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.68700000000000006},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":1.133,"15min":1.7370000000000001},"last":1.093},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.214,"15min":1.6080000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.79500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":1.1950000000000001,"15min":1.8069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.82799999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.3120000000000001,"15min":1.651},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.4850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.58099999999999996},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.8280000000000001},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.751,"5min":0.751,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":1.585,"15min":1.7849999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0629999999999999,"15min":1.097},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69499999999999995},"max":{"1min":1.337,"5min":1.4590000000000001,"15min":2.601},"last":0.91900000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.024,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.77400000000000002,"15min":0.71099999999999997},"max":{"1min":1.3029999999999999,"5min":1.403,"15min":2.7320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.107,"5min":1.1299999999999999,"15min":1.1559999999999999},"min":{"1min":0.84999999999999998,"5min":0.76700000000000002,"15min":0.747},"max":{"1min":1.375,"5min":1.496,"15min":2.6760000000000002},"last":1.5149999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1579999999999999,"15min":1.1910000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6560000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1559999999999999,"15min":1.1779999999999999},"min":{"1min":0.871,"5min":0.84099999999999997,"15min":0.71699999999999997},"max":{"1min":1.417,"5min":1.552,"15min":2.6230000000000002},"last":1.377}]},{"osd":7,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.218,"15min":1.2490000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.76600000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80000000000000004},"max":{"1min":1.3999999999999999,"5min":1.6200000000000001,"15min":2.7749999999999999},"last":1.282}]}]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35140,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":34304,"kb_avail":94336700,"statfs":{"total":96636764160,"available":96600780800,"internally_reserved":0,"allocated":847872,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35127296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.503,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.3560000000000001,"15min":3.798},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":1.151,"15min":2.3330000000000002},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.27500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":6.157},"last":1.179},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.879,"15min":0.97499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":5.649},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1919999999999999,"5min":1.323,"15min":6.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.294,"15min":1.9370000000000001},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.173,"5min":1.3340000000000001,"15min":1.863},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.133,"15min":1.1579999999999999},"min":{"1min":0.749,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.381,"5min":1.579,"15min":4.0499999999999998},"last":1.294}]},{"osd":5,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.65100000000000002},"max":{"1min":1.476,"5min":1.524,"15min":2.7040000000000002},"last":0.999},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.0960000000000001},"min":{"1min":0.80900000000000005,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.365,"5min":1.53,"15min":2.2509999999999999},"last":1.365}]},{"osd":6,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.234,"15min":1.2470000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.587,"5min":1.6919999999999999,"15min":3.9260000000000002},"last":1.22},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1910000000000001,"15min":1.1890000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.76400000000000001},"max":{"1min":1.556,"5min":1.5900000000000001,"15min":2.79},"last":1.127}]},{"osd":7,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.218},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.64800000000000002},"max":{"1min":1.532,"5min":1.7210000000000001,"15min":2.843},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.429,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.429}]}]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21396,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350444,"statfs":{"total":96636764160,"available":96614854656,"internally_reserved":0,"allocated":1388544,"data_stored":987179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0229999999999999,"15min":1.901},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81000000000000005,"15min":0.78400000000000003},"min":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.54700000000000004},"max":{"1min":1.0669999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73399999999999999,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.51200000000000001,"15min":0.39600000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.60299999999999998,"5min":0.56999999999999995,"15min":0.41499999999999998},"max":{"1min":1.069,"5min":1.1020000000000001,"15min":1.335},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85999999999999999,"15min":0.85299999999999998},"min":{"1min":0.67700000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":1.1719999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.89500000000000002,"15min":0.85999999999999999},"min":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.149,"5min":1.286,"15min":1.3400000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.79800000000000004,"5min":0.71099999999999997,"15min":0.59199999999999997},"max":{"1min":1.2569999999999999,"5min":1.4590000000000001,"15min":2.2519999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0629999999999999},"min":{"1min":0.78800000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5589999999999999,"5min":1.587,"15min":1.952},"last":0.98999999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0860000000000001,"15min":1.0920000000000001},"min":{"1min":0.85499999999999998,"5min":0.76300000000000001,"15min":0.71899999999999997},"max":{"1min":1.208,"5min":1.78,"15min":1.986},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1479999999999999,"15min":1.153},"min":{"1min":0.92900000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.4199999999999999,"5min":1.5780000000000001,"15min":2.0710000000000002},"last":1.3440000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1879999999999999,"15min":1.1619999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.4750000000000001,"5min":1.5940000000000001,"15min":1.7729999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.228,"15min":1.2070000000000001},"min":{"1min":1.012,"5min":0.90800000000000003,"15min":0.61199999999999999},"max":{"1min":1.6140000000000001,"5min":1.8300000000000001,"15min":2.1850000000000001},"last":1.306}]},{"osd":7,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2270000000000001,"15min":1.23},"min":{"1min":0.90600000000000003,"5min":0.85999999999999999,"15min":0.71999999999999997},"max":{"1min":1.55,"5min":1.7090000000000001,"15min":2.3130000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":1.054,"5min":0.97499999999999998,"15min":0.66400000000000003},"max":{"1min":1.4239999999999999,"5min":1.5269999999999999,"15min":2.3540000000000001},"last":1.407}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:27:00.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:27:00.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.931+0000 7f0b3c8d6700 1 Processor -- start 2023-09-27T23:27:00.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.931+0000 7f0b3c8d6700 1 -- start start 2023-09-27T23:27:00.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b3815bd00 0x7f0b3815c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3806b2c0 con 0x7f0b380cb0e0 2023-09-27T23:27:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3806e0f0 con 0x7f0b380cad10 2023-09-27T23:27:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3806a6b0 con 0x7f0b3815bd00 2023-09-27T23:27:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b2ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0b380cb0e0 0x7f0b3806d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48046/0 (socket says 172.21.15.64:48046) 2023-09-27T23:27:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b2ffff700 1 -- 172.21.15.64:0/2672212621 learned_addr learned my addr 172.21.15.64:0/2672212621 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:00.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3759e700 1 --2- 172.21.15.64:0/2672212621 >> v2:172.21.15.123:3300/0 conn(0x7f0b3815bd00 0x7f0b3815c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3659c700 1 -- 172.21.15.64:0/2672212621 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491290633 0 0) 0x7f0b3806b2c0 con 0x7f0b380cb0e0 2023-09-27T23:27:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3659c700 1 -- 172.21.15.64:0/2672212621 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b24003570 con 0x7f0b380cb0e0 2023-09-27T23:27:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3759e700 1 -- 172.21.15.64:0/2672212621 >> v1:172.21.15.123:6789/0 conn(0x7f0b380cad10 legacy=0x7f0b3806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3759e700 1 -- 172.21.15.64:0/2672212621 >> v1:172.21.15.64:6789/0 conn(0x7f0b380cb0e0 legacy=0x7f0b3806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3759e700 1 -- 172.21.15.64:0/2672212621 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3806a990 con 0x7f0b3815bd00 2023-09-27T23:27:00.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3759e700 1 --2- 172.21.15.64:0/2672212621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b3815bd00 0x7f0b3815c150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009e60 tx=0x7f0b30009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa598ff51290b7bf server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3659c700 1 -- 172.21.15.64:0/2672212621 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000e070 con 0x7f0b3815bd00 2023-09-27T23:27:00.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3659c700 1 -- 172.21.15.64:0/2672212621 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000bb60 con 0x7f0b3815bd00 2023-09-27T23:27:00.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3659c700 1 -- 172.21.15.64:0/2672212621 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30012430 con 0x7f0b3815bd00 2023-09-27T23:27:00.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2672212621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b3815bd00 msgr2=0x7f0b3815c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2672212621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b3815bd00 0x7f0b3815c150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009e60 tx=0x7f0b30009780 comp rx=0 tx=0).stop 2023-09-27T23:27:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2672212621 shutdown_connections 2023-09-27T23:27:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2672212621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b3815bd00 0x7f0b3815c150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2672212621 >> 172.21.15.64:0/2672212621 conn(0x7f0b38067bb0 msgr2=0x7f0b380cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.935+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2672212621 shutdown_connections 2023-09-27T23:27:00.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.939+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2672212621 wait complete. 2023-09-27T23:27:00.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.939+0000 7f0b3c8d6700 1 Processor -- start 2023-09-27T23:27:00.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.939+0000 7f0b3c8d6700 1 -- start start 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 0x7f0b38146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b380cb0e0 0x7f0b38146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b3815bd00 0x7f0b381470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3814a060 con 0x7f0b380cb0e0 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381475e0 con 0x7f0b380cad10 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b38147870 con 0x7f0b3815bd00 2023-09-27T23:27:00.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b380cb0e0 0x7f0b38146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 0x7f0b38146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 0x7f0b38146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:43710/0 (socket says 172.21.15.64:43710) 2023-09-27T23:27:00.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 -- 172.21.15.64:0/2523512252 learned_addr learned my addr 172.21.15.64:0/2523512252 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:00.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b3815bd00 0x7f0b381470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 -- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b3815bd00 msgr2=0x7f0b381470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b3815bd00 0x7f0b381470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 -- 172.21.15.64:0/2523512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b380cb0e0 msgr2=0x7f0b38146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b380cb0e0 0x7f0b38146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 -- 172.21.15.64:0/2523512252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30009a20 con 0x7f0b380cad10 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b36d9d700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 0x7f0b38146500 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c590 tx=0x7f0b28007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28011590 con 0x7f0b380cad10 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28011730 con 0x7f0b380cad10 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28011af0 con 0x7f0b380cad10 2023-09-27T23:27:00.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b38147b50 con 0x7f0b380cad10 2023-09-27T23:27:00.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.943+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b38148190 con 0x7f0b380cad10 2023-09-27T23:27:00.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.947+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3804be40 con 0x7f0b380cad10 2023-09-27T23:27:00.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.947+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0b28012420 con 0x7f0b380cad10 2023-09-27T23:27:00.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.947+0000 7f0b2effd700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b18060a70 0x7f0b18062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:00.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.947+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0b2807c320 con 0x7f0b380cad10 2023-09-27T23:27:00.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.947+0000 7f0b2ffff700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b18060a70 0x7f0b18062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:00.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.947+0000 7f0b2ffff700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b18060a70 0x7f0b18062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cf430 tx=0x7f0b20009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:00.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:00.959+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0b28049bb0 con 0x7f0b380cad10 2023-09-27T23:27:01.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.119+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b3806b7f0 con 0x7f0b18060a70 2023-09-27T23:27:01.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.139+0000 7f0b2effd700 1 -- 172.21.15.64:0/2523512252 <== mgr.5402 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295732 (secure 0 0 0) 0x7f0b3806b7f0 con 0x7f0b18060a70 2023-09-27T23:27:01.144 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b18060a70 msgr2=0x7f0b18062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b18060a70 0x7f0b18062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cf430 tx=0x7f0b20009040 comp rx=0 tx=0).stop 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 msgr2=0x7f0b38146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 0x7f0b38146500 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c590 tx=0x7f0b28007f30 comp rx=0 tx=0).stop 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 shutdown_connections 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0b18060a70 0x7f0b18062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b3815bd00 0x7f0b381470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b380cb0e0 0x7f0b38146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 --2- 172.21.15.64:0/2523512252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b380cad10 0x7f0b38146500 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 >> 172.21.15.64:0/2523512252 conn(0x7f0b38067bb0 msgr2=0x7f0b380ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:01.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.143+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 shutdown_connections 2023-09-27T23:27:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.147+0000 7f0b3c8d6700 1 -- 172.21.15.64:0/2523512252 wait complete. 2023-09-27T23:27:01.149 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:27:01.162 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-27T23:26:59.764594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":202,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":193268,"kb_used_data":7476,"kb_used_omap":0,"kb_used_meta":185728,"kb_avail":754781452,"statfs":{"total":773094113280,"available":772896206848,"internally_reserved":0,"allocated":7655424,"data_stored":4410976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":190185472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367720"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.814138+0000","last_change":"2023-09-27T23:26:56.811633+0000","last_active":"2023-09-27T23:26:56.814138+0000","last_peered":"2023-09-27T23:26:56.814138+0000","last_clean":"2023-09-27T23:26:56.814138+0000","last_became_active":"2023-09-27T23:26:56.811088+0000","last_became_peered":"2023-09-27T23:26:56.811088+0000","last_unstale":"2023-09-27T23:26:56.814138+0000","last_undegraded":"2023-09-27T23:26:56.814138+0000","last_fullsized":"2023-09-27T23:26:56.814138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:46.867145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800866+0000","last_change":"2023-09-27T23:26:56.796931+0000","last_active":"2023-09-27T23:26:56.800866+0000","last_peered":"2023-09-27T23:26:56.800866+0000","last_clean":"2023-09-27T23:26:56.800866+0000","last_became_active":"2023-09-27T23:26:56.796639+0000","last_became_peered":"2023-09-27T23:26:56.796639+0000","last_unstale":"2023-09-27T23:26:56.800866+0000","last_undegraded":"2023-09-27T23:26:56.800866+0000","last_fullsized":"2023-09-27T23:26:56.800866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:13:35.626367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824033+0000","last_change":"2023-09-27T23:26:56.812859+0000","last_active":"2023-09-27T23:26:56.824033+0000","last_peered":"2023-09-27T23:26:56.824033+0000","last_clean":"2023-09-27T23:26:56.824033+0000","last_became_active":"2023-09-27T23:26:56.811956+0000","last_became_peered":"2023-09-27T23:26:56.811956+0000","last_unstale":"2023-09-27T23:26:56.824033+0000","last_undegraded":"2023-09-27T23:26:56.824033+0000","last_fullsized":"2023-09-27T23:26:56.824033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:29.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816795+0000","last_change":"2023-09-27T23:26:56.805621+0000","last_active":"2023-09-27T23:26:56.816795+0000","last_peered":"2023-09-27T23:26:56.816795+0000","last_clean":"2023-09-27T23:26:56.816795+0000","last_became_active":"2023-09-27T23:26:56.805331+0000","last_became_peered":"2023-09-27T23:26:56.805331+0000","last_unstale":"2023-09-27T23:26:56.816795+0000","last_undegraded":"2023-09-27T23:26:56.816795+0000","last_fullsized":"2023-09-27T23:26:56.816795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:05.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810612+0000","last_change":"2023-09-27T23:26:56.805480+0000","last_active":"2023-09-27T23:26:56.810612+0000","last_peered":"2023-09-27T23:26:56.810612+0000","last_clean":"2023-09-27T23:26:56.810612+0000","last_became_active":"2023-09-27T23:26:56.805168+0000","last_became_peered":"2023-09-27T23:26:56.805168+0000","last_unstale":"2023-09-27T23:26:56.810612+0000","last_undegraded":"2023-09-27T23:26:56.810612+0000","last_fullsized":"2023-09-27T23:26:56.810612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:41.801806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818742+0000","last_change":"2023-09-27T23:26:56.814879+0000","last_active":"2023-09-27T23:26:56.818742+0000","last_peered":"2023-09-27T23:26:56.818742+0000","last_clean":"2023-09-27T23:26:56.818742+0000","last_became_active":"2023-09-27T23:26:56.814574+0000","last_became_peered":"2023-09-27T23:26:56.814574+0000","last_unstale":"2023-09-27T23:26:56.818742+0000","last_undegraded":"2023-09-27T23:26:56.818742+0000","last_fullsized":"2023-09-27T23:26:56.818742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:38.070621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.802160+0000","last_change":"2023-09-27T23:26:56.800806+0000","last_active":"2023-09-27T23:26:56.802160+0000","last_peered":"2023-09-27T23:26:56.802160+0000","last_clean":"2023-09-27T23:26:56.802160+0000","last_became_active":"2023-09-27T23:26:56.800324+0000","last_became_peered":"2023-09-27T23:26:56.800324+0000","last_unstale":"2023-09-27T23:26:56.802160+0000","last_undegraded":"2023-09-27T23:26:56.802160+0000","last_fullsized":"2023-09-27T23:26:56.802160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:58.847869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819184+0000","last_change":"2023-09-27T23:26:56.817217+0000","last_active":"2023-09-27T23:26:56.819184+0000","last_peered":"2023-09-27T23:26:56.819184+0000","last_clean":"2023-09-27T23:26:56.819184+0000","last_became_active":"2023-09-27T23:26:56.816937+0000","last_became_peered":"2023-09-27T23:26:56.816937+0000","last_unstale":"2023-09-27T23:26:56.819184+0000","last_undegraded":"2023-09-27T23:26:56.819184+0000","last_fullsized":"2023-09-27T23:26:56.819184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:17:55.307710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824218+0000","last_change":"2023-09-27T23:26:56.814262+0000","last_active":"2023-09-27T23:26:56.824218+0000","last_peered":"2023-09-27T23:26:56.824218+0000","last_clean":"2023-09-27T23:26:56.824218+0000","last_became_active":"2023-09-27T23:26:56.813928+0000","last_became_peered":"2023-09-27T23:26:56.813928+0000","last_unstale":"2023-09-27T23:26:56.824218+0000","last_undegraded":"2023-09-27T23:26:56.824218+0000","last_fullsized":"2023-09-27T23:26:56.824218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:49.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824186+0000","last_change":"2023-09-27T23:26:56.816043+0000","last_active":"2023-09-27T23:26:56.824186+0000","last_peered":"2023-09-27T23:26:56.824186+0000","last_clean":"2023-09-27T23:26:56.824186+0000","last_became_active":"2023-09-27T23:26:56.815712+0000","last_became_peered":"2023-09-27T23:26:56.815712+0000","last_unstale":"2023-09-27T23:26:56.824186+0000","last_undegraded":"2023-09-27T23:26:56.824186+0000","last_fullsized":"2023-09-27T23:26:56.824186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:54.503767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817510+0000","last_change":"2023-09-27T23:26:56.433280+0000","last_active":"2023-09-27T23:26:56.817510+0000","last_peered":"2023-09-27T23:26:56.817510+0000","last_clean":"2023-09-27T23:26:56.817510+0000","last_became_active":"2023-09-27T23:26:56.432946+0000","last_became_peered":"2023-09-27T23:26:56.432946+0000","last_unstale":"2023-09-27T23:26:56.817510+0000","last_undegraded":"2023-09-27T23:26:56.817510+0000","last_fullsized":"2023-09-27T23:26:56.817510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:05:40.954906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824360+0000","last_change":"2023-09-27T23:26:56.815852+0000","last_active":"2023-09-27T23:26:56.824360+0000","last_peered":"2023-09-27T23:26:56.824360+0000","last_clean":"2023-09-27T23:26:56.824360+0000","last_became_active":"2023-09-27T23:26:56.815565+0000","last_became_peered":"2023-09-27T23:26:56.815565+0000","last_unstale":"2023-09-27T23:26:56.824360+0000","last_undegraded":"2023-09-27T23:26:56.824360+0000","last_fullsized":"2023-09-27T23:26:56.824360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:21:59.389594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799681+0000","last_change":"2023-09-27T23:26:56.431567+0000","last_active":"2023-09-27T23:26:56.799681+0000","last_peered":"2023-09-27T23:26:56.799681+0000","last_clean":"2023-09-27T23:26:56.799681+0000","last_became_active":"2023-09-27T23:26:56.431206+0000","last_became_peered":"2023-09-27T23:26:56.431206+0000","last_unstale":"2023-09-27T23:26:56.799681+0000","last_undegraded":"2023-09-27T23:26:56.799681+0000","last_fullsized":"2023-09-27T23:26:56.799681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:52:27.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824213+0000","last_change":"2023-09-27T23:26:56.811986+0000","last_active":"2023-09-27T23:26:56.824213+0000","last_peered":"2023-09-27T23:26:56.824213+0000","last_clean":"2023-09-27T23:26:56.824213+0000","last_became_active":"2023-09-27T23:26:56.811704+0000","last_became_peered":"2023-09-27T23:26:56.811704+0000","last_unstale":"2023-09-27T23:26:56.824213+0000","last_undegraded":"2023-09-27T23:26:56.824213+0000","last_fullsized":"2023-09-27T23:26:56.824213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:10:10.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.797203+0000","last_change":"2023-09-27T23:26:56.439391+0000","last_active":"2023-09-27T23:26:56.797203+0000","last_peered":"2023-09-27T23:26:56.797203+0000","last_clean":"2023-09-27T23:26:56.797203+0000","last_became_active":"2023-09-27T23:26:56.439021+0000","last_became_peered":"2023-09-27T23:26:56.439021+0000","last_unstale":"2023-09-27T23:26:56.797203+0000","last_undegraded":"2023-09-27T23:26:56.797203+0000","last_fullsized":"2023-09-27T23:26:56.797203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:59.110216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":285,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643461+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:26:52.643461+0000","last_peered":"2023-09-27T23:26:52.643461+0000","last_clean":"2023-09-27T23:26:52.643461+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:26:52.643461+0000","last_undegraded":"2023-09-27T23:26:52.643461+0000","last_fullsized":"2023-09-27T23:26:52.643461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":139,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.311206+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:26:52.311206+0000","last_peered":"2023-09-27T23:26:52.311206+0000","last_clean":"2023-09-27T23:26:52.311206+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:26:52.311206+0000","last_undegraded":"2023-09-27T23:26:52.311206+0000","last_fullsized":"2023-09-27T23:26:52.311206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798523+0000","last_change":"2023-09-27T23:26:56.428852+0000","last_active":"2023-09-27T23:26:56.798523+0000","last_peered":"2023-09-27T23:26:56.798523+0000","last_clean":"2023-09-27T23:26:56.798523+0000","last_became_active":"2023-09-27T23:26:56.428486+0000","last_became_peered":"2023-09-27T23:26:56.428486+0000","last_unstale":"2023-09-27T23:26:56.798523+0000","last_undegraded":"2023-09-27T23:26:56.798523+0000","last_fullsized":"2023-09-27T23:26:56.798523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:23:54.572803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643321+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:26:52.643321+0000","last_peered":"2023-09-27T23:26:52.643321+0000","last_clean":"2023-09-27T23:26:52.643321+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:26:52.643321+0000","last_undegraded":"2023-09-27T23:26:52.643321+0000","last_fullsized":"2023-09-27T23:26:52.643321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.787944+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:26:56.787944+0000","last_peered":"2023-09-27T23:26:56.787944+0000","last_clean":"2023-09-27T23:26:56.787944+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:26:56.787944+0000","last_undegraded":"2023-09-27T23:26:56.787944+0000","last_fullsized":"2023-09-27T23:26:56.787944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.311130+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:26:52.311130+0000","last_peered":"2023-09-27T23:26:52.311130+0000","last_clean":"2023-09-27T23:26:52.311130+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:26:52.311130+0000","last_undegraded":"2023-09-27T23:26:52.311130+0000","last_fullsized":"2023-09-27T23:26:52.311130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799914+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:26:56.799914+0000","last_peered":"2023-09-27T23:26:56.799914+0000","last_clean":"2023-09-27T23:26:56.799914+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:26:56.799914+0000","last_undegraded":"2023-09-27T23:26:56.799914+0000","last_fullsized":"2023-09-27T23:26:56.799914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643342+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:26:52.643342+0000","last_peered":"2023-09-27T23:26:52.643342+0000","last_clean":"2023-09-27T23:26:52.643342+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:26:52.643342+0000","last_undegraded":"2023-09-27T23:26:52.643342+0000","last_fullsized":"2023-09-27T23:26:52.643342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799800+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:26:56.799800+0000","last_peered":"2023-09-27T23:26:56.799800+0000","last_clean":"2023-09-27T23:26:56.799800+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:26:56.799800+0000","last_undegraded":"2023-09-27T23:26:56.799800+0000","last_fullsized":"2023-09-27T23:26:56.799800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:53.216004+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:26:53.216004+0000","last_peered":"2023-09-27T23:26:53.216004+0000","last_clean":"2023-09-27T23:26:53.216004+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:26:53.216004+0000","last_undegraded":"2023-09-27T23:26:53.216004+0000","last_fullsized":"2023-09-27T23:26:53.216004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509602,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24404,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":23040,"kb_avail":94347436,"statfs":{"total":96636764160,"available":96611774464,"internally_reserved":0,"allocated":1388544,"data_stored":987179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23592960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63700000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.232,"5min":1.363,"15min":1.7949999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.94199999999999995,"15min":0.88500000000000001},"min":{"1min":0.63,"5min":0.58199999999999996,"15min":0.47199999999999998},"max":{"1min":1.2210000000000001,"5min":1.363,"15min":1.363},"last":1.712}]},{"osd":1,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.89900000000000002,"15min":0.90000000000000002},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.92600000000000005,"15min":0.93200000000000005},"min":{"1min":0.67600000000000005,"5min":0.66500000000000004,"15min":0.56899999999999995},"max":{"1min":1.1479999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.7589999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91300000000000003,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.62,"15min":0.61299999999999999},"max":{"1min":1.1899999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":2.681},{"interface":"front","average":{"1min":1.0609999999999999,"5min":0.98999999999999999,"15min":0.96699999999999997},"min":{"1min":0.71899999999999997,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.5920000000000001},"last":1.2929999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.94999999999999996},"min":{"1min":0.72299999999999998,"5min":0.66300000000000003,"15min":0.65100000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.421},"last":0.995},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0840000000000001,"15min":1.044},"min":{"1min":0.79500000000000004,"5min":0.75,"15min":0.622},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.4339999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89100000000000001,"15min":0.91200000000000003},"min":{"1min":0.628,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.3879999999999999},"last":2.6230000000000002},{"interface":"front","average":{"1min":0.997,"5min":0.98299999999999998,"15min":0.98799999999999999},"min":{"1min":0.84699999999999998,"5min":0.67600000000000005,"15min":0.59599999999999997},"max":{"1min":1.1930000000000001,"5min":1.3080000000000001,"15min":1.4099999999999999},"last":2.8879999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.034,"15min":1.0569999999999999},"min":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.48999999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.5880000000000001},"last":1.391},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.0960000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.64100000000000001},"max":{"1min":1.395,"5min":1.51,"15min":1.51},"last":2.7280000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1200000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.78200000000000003},"max":{"1min":1.468,"5min":1.498,"15min":1.498},"last":1.806},{"interface":"front","average":{"1min":1.139,"5min":1.1399999999999999,"15min":1.133},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.73899999999999999},"max":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":2.548}]}]},{"osd":6,"up_from":15,"seq":64424509601,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20304,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":19520,"kb_avail":94351536,"statfs":{"total":96636764160,"available":96615972864,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19988480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.68899999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.327,"15min":1.327},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.079,"5min":1.026,"15min":1.0289999999999999},"min":{"1min":0.81999999999999995,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.649},"last":0.83899999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.54700000000000004},"max":{"1min":1.2889999999999999,"5min":1.323,"15min":1.323},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1739999999999999,"5min":1.2509999999999999,"15min":1.3939999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.88},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.56799999999999995},"max":{"1min":1.0820000000000001,"5min":1.353,"15min":1.397},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60499999999999998},"max":{"1min":1.2589999999999999,"5min":1.367,"15min":1.4259999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.58999999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.734},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.84599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"max":{"1min":1.2290000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.97199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.93100000000000005},"min":{"1min":0.64900000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":1.292,"5min":1.335,"15min":1.544},"last":0.874},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.61899999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.24,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":1.0269999999999999,"5min":0.83999999999999997,"15min":0.65900000000000003},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.095},{"interface":"front","average":{"1min":1.153,"5min":1.167,"15min":1.165},"min":{"1min":0.92800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3660000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1200000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509601,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30160,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":29376,"kb_avail":94341680,"statfs":{"total":96636764160,"available":96605880320,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30081024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.151,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.2929999999999999,"15min":1.359},"last":1.3440000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.50700000000000001},"max":{"1min":1.2969999999999999,"5min":1.3109999999999999,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.88500000000000001,"15min":0.876},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.234,"15min":1.4179999999999999},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.93000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3859999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"max":{"1min":1.27,"5min":1.4730000000000001,"15min":1.5740000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91900000000000004,"15min":0.97199999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.214,"5min":1.238,"15min":1.5640000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.008,"5min":1.024,"15min":1.014},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3819999999999999,"15min":1.954},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1599999999999999,"5min":1.2829999999999999,"15min":1.5349999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.032,"15min":1.046},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.304,"5min":1.3939999999999999,"15min":1.704},"last":1.399}]},{"osd":6,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.02,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.323,"5min":1.5580000000000001,"15min":1.6319999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.3779999999999999,"15min":1.837},"last":1.3069999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1479999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3,"5min":1.496,"15min":1.7230000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.204,"15min":1.1950000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.4319999999999999,"5min":1.633,"15min":2.0529999999999999},"last":1.425}]}]},{"osd":4,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":20624,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94351216,"statfs":{"total":96636764160,"available":96615645184,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.92100000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.57099999999999995,"15min":0.53700000000000003},"max":{"1min":1.256,"5min":1.27,"15min":1.3080000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90000000000000002,"15min":0.876},"min":{"1min":0.68500000000000005,"5min":0.63600000000000001,"15min":0.53800000000000003},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.3320000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.877,"15min":0.88800000000000001},"min":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.3380000000000001,"15min":1.3919999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95999999999999996,"15min":0.92400000000000004},"min":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.58199999999999996},"max":{"1min":1.171,"5min":1.228,"15min":1.329},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91500000000000004,"15min":0.90500000000000003},"min":{"1min":0.68300000000000005,"5min":0.63600000000000001,"15min":0.621},"max":{"1min":1.2090000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.268},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95199999999999996,"15min":0.98199999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.522},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92500000000000004,"15min":0.92700000000000005},"min":{"1min":0.72799999999999998,"5min":0.66100000000000003,"15min":0.60099999999999998},"max":{"1min":1.167,"5min":1.26,"15min":1.373},"last":1.006},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.012},"min":{"1min":0.77800000000000002,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.2929999999999999,"5min":1.5129999999999999,"15min":1.575},"last":1.0469999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89200000000000002,"15min":0.92000000000000004},"min":{"1min":0.626,"5min":0.60199999999999998,"15min":0.57599999999999996},"max":{"1min":1.2729999999999999,"5min":1.409,"15min":1.627},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.022,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.67200000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3220000000000001,"5min":1.5489999999999999,"15min":2.2730000000000001},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0249999999999999,"15min":1.0489999999999999},"min":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.60099999999999998},"max":{"1min":1.4059999999999999,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.369},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.149,"15min":1.1619999999999999},"min":{"1min":0.97499999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.478,"5min":1.478,"15min":1.8620000000000001},"last":1.137}]},{"osd":7,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0820000000000001,"15min":1.0940000000000001},"min":{"1min":0.83799999999999997,"5min":0.76500000000000001,"15min":0.64000000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.726},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1499999999999999,"15min":1.1759999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78500000000000003},"max":{"1min":1.5069999999999999,"5min":1.859,"15min":2.532},"last":1.4319999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21200,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350640,"statfs":{"total":96636764160,"available":96615055360,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.30199999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.746},"min":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.35299999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.337},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.39800000000000002},"max":{"1min":0.95299999999999996,"5min":1.224,"15min":1.2869999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.72199999999999998,"5min":0.61699999999999999,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.3180000000000001,"15min":1.3939999999999999},"last":0.75}]},{"osd":2,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.88300000000000001,"15min":0.85699999999999998},"min":{"1min":0.78200000000000003,"5min":0.65700000000000003,"15min":0.36699999999999999},"max":{"1min":1.1479999999999999,"5min":1.288,"15min":1.353},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91600000000000004},"min":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.45700000000000002},"max":{"1min":1.256,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.76000000000000001,"5min":0.67700000000000005,"15min":0.66100000000000003},"max":{"1min":1.175,"5min":1.466,"15min":1.5349999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.628},"max":{"1min":1.2030000000000001,"5min":1.3680000000000001,"15min":1.4950000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.91300000000000003,"5min":0.83099999999999996,"15min":0.626},"max":{"1min":1.3120000000000001,"5min":1.468,"15min":1.571},"last":1.02},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1479999999999999},"min":{"1min":0.96399999999999997,"5min":0.76700000000000002,"15min":0.58399999999999996},"max":{"1min":1.47,"5min":1.51,"15min":1.764},"last":1.252}]},{"osd":6,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.119,"15min":1.137},"min":{"1min":0.92600000000000005,"5min":0.76900000000000002,"15min":0.72799999999999998},"max":{"1min":1.407,"5min":1.4610000000000001,"15min":1.6970000000000001},"last":1.169},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":1.0269999999999999,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.3140000000000001,"5min":1.5800000000000001,"15min":1.8620000000000001},"last":1.1180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.246,"15min":1.2290000000000001},"min":{"1min":1.0289999999999999,"5min":0.86799999999999999,"15min":0.78700000000000003},"max":{"1min":1.5640000000000001,"5min":1.627,"15min":1.655},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.2},"min":{"1min":1.054,"5min":0.89500000000000002,"15min":0.73099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.671},"last":1.083}]}]},{"osd":2,"up_from":15,"seq":64424509603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20040,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":19200,"kb_avail":94351800,"statfs":{"total":96636764160,"available":96616243200,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19660800},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.68700000000000006},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":1.133,"15min":1.7370000000000001},"last":1.093},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.214,"15min":1.6080000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.79500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":1.1950000000000001,"15min":1.8069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.82799999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.3120000000000001,"15min":1.651},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.4850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.58099999999999996},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.8280000000000001},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.751,"5min":0.751,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":1.585,"15min":1.7849999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0629999999999999,"15min":1.097},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69499999999999995},"max":{"1min":1.337,"5min":1.4590000000000001,"15min":2.601},"last":0.91900000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.024,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.77400000000000002,"15min":0.71099999999999997},"max":{"1min":1.3029999999999999,"5min":1.403,"15min":2.7320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.107,"5min":1.1299999999999999,"15min":1.1559999999999999},"min":{"1min":0.84999999999999998,"5min":0.76700000000000002,"15min":0.747},"max":{"1min":1.375,"5min":1.496,"15min":2.6760000000000002},"last":1.5149999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1579999999999999,"15min":1.1910000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6560000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1559999999999999,"15min":1.1779999999999999},"min":{"1min":0.871,"5min":0.84099999999999997,"15min":0.71699999999999997},"max":{"1min":1.417,"5min":1.552,"15min":2.6230000000000002},"last":1.377}]},{"osd":7,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.218,"15min":1.2490000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.76600000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80000000000000004},"max":{"1min":1.3999999999999999,"5min":1.6200000000000001,"15min":2.7749999999999999},"last":1.282}]}]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35140,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":34304,"kb_avail":94336700,"statfs":{"total":96636764160,"available":96600780800,"internally_reserved":0,"allocated":847872,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35127296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.503,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.3560000000000001,"15min":3.798},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":1.151,"15min":2.3330000000000002},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.27500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":6.157},"last":1.179},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.879,"15min":0.97499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":5.649},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1919999999999999,"5min":1.323,"15min":6.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.294,"15min":1.9370000000000001},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.173,"5min":1.3340000000000001,"15min":1.863},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.133,"15min":1.1579999999999999},"min":{"1min":0.749,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.381,"5min":1.579,"15min":4.0499999999999998},"last":1.294}]},{"osd":5,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.65100000000000002},"max":{"1min":1.476,"5min":1.524,"15min":2.7040000000000002},"last":0.999},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.0960000000000001},"min":{"1min":0.80900000000000005,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.365,"5min":1.53,"15min":2.2509999999999999},"last":1.365}]},{"osd":6,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.234,"15min":1.2470000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.587,"5min":1.6919999999999999,"15min":3.9260000000000002},"last":1.22},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1910000000000001,"15min":1.1890000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.76400000000000001},"max":{"1min":1.556,"5min":1.5900000000000001,"15min":2.79},"last":1.127}]},{"osd":7,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.218},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.64800000000000002},"max":{"1min":1.532,"5min":1.7210000000000001,"15min":2.843},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.429,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.429}]}]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21396,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350444,"statfs":{"total":96636764160,"available":96614854656,"internally_reserved":0,"allocated":1388544,"data_stored":987179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0229999999999999,"15min":1.901},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81000000000000005,"15min":0.78400000000000003},"min":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.54700000000000004},"max":{"1min":1.0669999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73399999999999999,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.51200000000000001,"15min":0.39600000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.60299999999999998,"5min":0.56999999999999995,"15min":0.41499999999999998},"max":{"1min":1.069,"5min":1.1020000000000001,"15min":1.335},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85999999999999999,"15min":0.85299999999999998},"min":{"1min":0.67700000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":1.1719999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.89500000000000002,"15min":0.85999999999999999},"min":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.149,"5min":1.286,"15min":1.3400000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.79800000000000004,"5min":0.71099999999999997,"15min":0.59199999999999997},"max":{"1min":1.2569999999999999,"5min":1.4590000000000001,"15min":2.2519999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0629999999999999},"min":{"1min":0.78800000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5589999999999999,"5min":1.587,"15min":1.952},"last":0.98999999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0860000000000001,"15min":1.0920000000000001},"min":{"1min":0.85499999999999998,"5min":0.76300000000000001,"15min":0.71899999999999997},"max":{"1min":1.208,"5min":1.78,"15min":1.986},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1479999999999999,"15min":1.153},"min":{"1min":0.92900000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.4199999999999999,"5min":1.5780000000000001,"15min":2.0710000000000002},"last":1.3440000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1879999999999999,"15min":1.1619999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.4750000000000001,"5min":1.5940000000000001,"15min":1.7729999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.228,"15min":1.2070000000000001},"min":{"1min":1.012,"5min":0.90800000000000003,"15min":0.61199999999999999},"max":{"1min":1.6140000000000001,"5min":1.8300000000000001,"15min":2.1850000000000001},"last":1.306}]},{"osd":7,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2270000000000001,"15min":1.23},"min":{"1min":0.90600000000000003,"5min":0.85999999999999999,"15min":0.71999999999999997},"max":{"1min":1.55,"5min":1.7090000000000001,"15min":2.3130000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":1.054,"5min":0.97499999999999998,"15min":0.66400000000000003},"max":{"1min":1.4239999999999999,"5min":1.5269999999999999,"15min":2.3540000000000001},"last":1.407}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:27:01.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:27:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.275+0000 7f7f69f88700 1 Processor -- start 2023-09-27T23:27:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.275+0000 7f7f69f88700 1 -- start start 2023-09-27T23:27:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f64155990 0x7f7f640c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f640b0670 0x7f7f640b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6406cf00 con 0x7f7f640b13b0 2023-09-27T23:27:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f640ae7e0 con 0x7f7f640b0670 2023-09-27T23:27:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640cbc50 con 0x7f7f64155990 2023-09-27T23:27:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f640b0670 0x7f7f640b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f640b0670 0x7f7f640b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49180/0 (socket says 172.21.15.64:49180) 2023-09-27T23:27:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 -- 172.21.15.64:0/509007756 learned_addr learned my addr 172.21.15.64:0/509007756 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:01.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f68f86700 1 --2- 172.21.15.64:0/509007756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f64155990 0x7f7f640c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 -- 172.21.15.64:0/509007756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f64155990 msgr2=0x7f7f640c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 --2- 172.21.15.64:0/509007756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f64155990 0x7f7f640c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 -- 172.21.15.64:0/509007756 >> v1:172.21.15.123:6789/0 conn(0x7f7f640b13b0 legacy=0x7f7f6406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 -- 172.21.15.64:0/509007756 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640b01d0 con 0x7f7f640b0670 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f63fff700 1 --2- 172.21.15.64:0/509007756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f640b0670 0x7f7f640b0e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7f580099b0 tx=0x7f7f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be95a9efad5fa6e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f637fe700 1 -- 172.21.15.64:0/509007756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58009ea0 con 0x7f7f640b0670 2023-09-27T23:27:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f637fe700 1 -- 172.21.15.64:0/509007756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5800b860 con 0x7f7f640b0670 2023-09-27T23:27:01.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f637fe700 1 -- 172.21.15.64:0/509007756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800bc00 con 0x7f7f640b0670 2023-09-27T23:27:01.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- 172.21.15.64:0/509007756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f640b0670 msgr2=0x7f7f640b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 --2- 172.21.15.64:0/509007756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f640b0670 0x7f7f640b0e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7f580099b0 tx=0x7f7f58009250 comp rx=0 tx=0).stop 2023-09-27T23:27:01.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- 172.21.15.64:0/509007756 shutdown_connections 2023-09-27T23:27:01.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 --2- 172.21.15.64:0/509007756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f640b0670 0x7f7f640b0e70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 --2- 172.21.15.64:0/509007756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f64155990 0x7f7f640c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- 172.21.15.64:0/509007756 >> 172.21.15.64:0/509007756 conn(0x7f7f64067bb0 msgr2=0x7f7f640add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:01.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- 172.21.15.64:0/509007756 shutdown_connections 2023-09-27T23:27:01.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.279+0000 7f7f69f88700 1 -- 172.21.15.64:0/509007756 wait complete. 2023-09-27T23:27:01.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 Processor -- start 2023-09-27T23:27:01.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 -- start start 2023-09-27T23:27:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f640b0670 0x7f7f64167760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f640b13b0 0x7f7f6406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 0x7f7f6406b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640c39d0 con 0x7f7f64155990 2023-09-27T23:27:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64167ca0 con 0x7f7f640b13b0 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69f88700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640c1f90 con 0x7f7f640b0670 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 0x7f7f6406b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 0x7f7f6406b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49196/0 (socket says 172.21.15.64:49196) 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 -- 172.21.15.64:0/4162937705 learned_addr learned my addr 172.21.15.64:0/4162937705 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f68f86700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f640b0670 0x7f7f64167760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f63fff700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f640b13b0 0x7f7f6406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 -- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f640b0670 msgr2=0x7f7f64167760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f640b0670 0x7f7f64167760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 -- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f640b13b0 msgr2=0x7f7f6406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f640b13b0 0x7f7f6406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 -- 172.21.15.64:0/4162937705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f580094f0 con 0x7f7f64155990 2023-09-27T23:27:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.283+0000 7f7f69787700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 0x7f7f6406b2b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c8e0 tx=0x7f7f5c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00ce80 con 0x7f7f64155990 2023-09-27T23:27:01.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00a7e0 con 0x7f7f64155990 2023-09-27T23:27:01.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f640c2270 con 0x7f7f64155990 2023-09-27T23:27:01.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00ab80 con 0x7f7f64155990 2023-09-27T23:27:01.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f640c2830 con 0x7f7f64155990 2023-09-27T23:27:01.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f641512f0 con 0x7f7f64155990 2023-09-27T23:27:01.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7f5c012420 con 0x7f7f64155990 2023-09-27T23:27:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f60ff9700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f50060a70 0x7f7f50062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.287+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7f5c07d440 con 0x7f7f64155990 2023-09-27T23:27:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.291+0000 7f7f68f86700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f50060a70 0x7f7f50062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.291+0000 7f7f68f86700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f50060a70 0x7f7f50062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f64150a20 tx=0x7f7f54008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.295+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7f641512f0 con 0x7f7f64155990 2023-09-27T23:27:01.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.451+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f641512f0 con 0x7f7f50060a70 2023-09-27T23:27:01.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.471+0000 7f7f60ff9700 1 -- 172.21.15.64:0/4162937705 <== mgr.5402 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295732 (secure 0 0 0) 0x7f7f641512f0 con 0x7f7f50060a70 2023-09-27T23:27:01.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f50060a70 msgr2=0x7f7f50062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f50060a70 0x7f7f50062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f64150a20 tx=0x7f7f54008040 comp rx=0 tx=0).stop 2023-09-27T23:27:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 msgr2=0x7f7f6406b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 0x7f7f6406b2b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c8e0 tx=0x7f7f5c001710 comp rx=0 tx=0).stop 2023-09-27T23:27:01.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 shutdown_connections 2023-09-27T23:27:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f50060a70 0x7f7f50062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f64155990 0x7f7f6406b2b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f640b13b0 0x7f7f6406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 --2- 172.21.15.64:0/4162937705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f640b0670 0x7f7f64167760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 >> 172.21.15.64:0/4162937705 conn(0x7f7f64067bb0 msgr2=0x7f7f640cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:01.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 shutdown_connections 2023-09-27T23:27:01.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.475+0000 7f7f69f88700 1 -- 172.21.15.64:0/4162937705 wait complete. 2023-09-27T23:27:01.485 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:27:01.491 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-27T23:26:59.764594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":202,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":193268,"kb_used_data":7476,"kb_used_omap":0,"kb_used_meta":185728,"kb_avail":754781452,"statfs":{"total":773094113280,"available":772896206848,"internally_reserved":0,"allocated":7655424,"data_stored":4410976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":190185472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367720"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.814138+0000","last_change":"2023-09-27T23:26:56.811633+0000","last_active":"2023-09-27T23:26:56.814138+0000","last_peered":"2023-09-27T23:26:56.814138+0000","last_clean":"2023-09-27T23:26:56.814138+0000","last_became_active":"2023-09-27T23:26:56.811088+0000","last_became_peered":"2023-09-27T23:26:56.811088+0000","last_unstale":"2023-09-27T23:26:56.814138+0000","last_undegraded":"2023-09-27T23:26:56.814138+0000","last_fullsized":"2023-09-27T23:26:56.814138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:46.867145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800866+0000","last_change":"2023-09-27T23:26:56.796931+0000","last_active":"2023-09-27T23:26:56.800866+0000","last_peered":"2023-09-27T23:26:56.800866+0000","last_clean":"2023-09-27T23:26:56.800866+0000","last_became_active":"2023-09-27T23:26:56.796639+0000","last_became_peered":"2023-09-27T23:26:56.796639+0000","last_unstale":"2023-09-27T23:26:56.800866+0000","last_undegraded":"2023-09-27T23:26:56.800866+0000","last_fullsized":"2023-09-27T23:26:56.800866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:13:35.626367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824033+0000","last_change":"2023-09-27T23:26:56.812859+0000","last_active":"2023-09-27T23:26:56.824033+0000","last_peered":"2023-09-27T23:26:56.824033+0000","last_clean":"2023-09-27T23:26:56.824033+0000","last_became_active":"2023-09-27T23:26:56.811956+0000","last_became_peered":"2023-09-27T23:26:56.811956+0000","last_unstale":"2023-09-27T23:26:56.824033+0000","last_undegraded":"2023-09-27T23:26:56.824033+0000","last_fullsized":"2023-09-27T23:26:56.824033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:29.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816795+0000","last_change":"2023-09-27T23:26:56.805621+0000","last_active":"2023-09-27T23:26:56.816795+0000","last_peered":"2023-09-27T23:26:56.816795+0000","last_clean":"2023-09-27T23:26:56.816795+0000","last_became_active":"2023-09-27T23:26:56.805331+0000","last_became_peered":"2023-09-27T23:26:56.805331+0000","last_unstale":"2023-09-27T23:26:56.816795+0000","last_undegraded":"2023-09-27T23:26:56.816795+0000","last_fullsized":"2023-09-27T23:26:56.816795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:05.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810612+0000","last_change":"2023-09-27T23:26:56.805480+0000","last_active":"2023-09-27T23:26:56.810612+0000","last_peered":"2023-09-27T23:26:56.810612+0000","last_clean":"2023-09-27T23:26:56.810612+0000","last_became_active":"2023-09-27T23:26:56.805168+0000","last_became_peered":"2023-09-27T23:26:56.805168+0000","last_unstale":"2023-09-27T23:26:56.810612+0000","last_undegraded":"2023-09-27T23:26:56.810612+0000","last_fullsized":"2023-09-27T23:26:56.810612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:41.801806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818742+0000","last_change":"2023-09-27T23:26:56.814879+0000","last_active":"2023-09-27T23:26:56.818742+0000","last_peered":"2023-09-27T23:26:56.818742+0000","last_clean":"2023-09-27T23:26:56.818742+0000","last_became_active":"2023-09-27T23:26:56.814574+0000","last_became_peered":"2023-09-27T23:26:56.814574+0000","last_unstale":"2023-09-27T23:26:56.818742+0000","last_undegraded":"2023-09-27T23:26:56.818742+0000","last_fullsized":"2023-09-27T23:26:56.818742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:38.070621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.802160+0000","last_change":"2023-09-27T23:26:56.800806+0000","last_active":"2023-09-27T23:26:56.802160+0000","last_peered":"2023-09-27T23:26:56.802160+0000","last_clean":"2023-09-27T23:26:56.802160+0000","last_became_active":"2023-09-27T23:26:56.800324+0000","last_became_peered":"2023-09-27T23:26:56.800324+0000","last_unstale":"2023-09-27T23:26:56.802160+0000","last_undegraded":"2023-09-27T23:26:56.802160+0000","last_fullsized":"2023-09-27T23:26:56.802160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:58.847869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819184+0000","last_change":"2023-09-27T23:26:56.817217+0000","last_active":"2023-09-27T23:26:56.819184+0000","last_peered":"2023-09-27T23:26:56.819184+0000","last_clean":"2023-09-27T23:26:56.819184+0000","last_became_active":"2023-09-27T23:26:56.816937+0000","last_became_peered":"2023-09-27T23:26:56.816937+0000","last_unstale":"2023-09-27T23:26:56.819184+0000","last_undegraded":"2023-09-27T23:26:56.819184+0000","last_fullsized":"2023-09-27T23:26:56.819184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:17:55.307710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824218+0000","last_change":"2023-09-27T23:26:56.814262+0000","last_active":"2023-09-27T23:26:56.824218+0000","last_peered":"2023-09-27T23:26:56.824218+0000","last_clean":"2023-09-27T23:26:56.824218+0000","last_became_active":"2023-09-27T23:26:56.813928+0000","last_became_peered":"2023-09-27T23:26:56.813928+0000","last_unstale":"2023-09-27T23:26:56.824218+0000","last_undegraded":"2023-09-27T23:26:56.824218+0000","last_fullsized":"2023-09-27T23:26:56.824218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:49.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824186+0000","last_change":"2023-09-27T23:26:56.816043+0000","last_active":"2023-09-27T23:26:56.824186+0000","last_peered":"2023-09-27T23:26:56.824186+0000","last_clean":"2023-09-27T23:26:56.824186+0000","last_became_active":"2023-09-27T23:26:56.815712+0000","last_became_peered":"2023-09-27T23:26:56.815712+0000","last_unstale":"2023-09-27T23:26:56.824186+0000","last_undegraded":"2023-09-27T23:26:56.824186+0000","last_fullsized":"2023-09-27T23:26:56.824186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:54.503767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817510+0000","last_change":"2023-09-27T23:26:56.433280+0000","last_active":"2023-09-27T23:26:56.817510+0000","last_peered":"2023-09-27T23:26:56.817510+0000","last_clean":"2023-09-27T23:26:56.817510+0000","last_became_active":"2023-09-27T23:26:56.432946+0000","last_became_peered":"2023-09-27T23:26:56.432946+0000","last_unstale":"2023-09-27T23:26:56.817510+0000","last_undegraded":"2023-09-27T23:26:56.817510+0000","last_fullsized":"2023-09-27T23:26:56.817510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:05:40.954906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824360+0000","last_change":"2023-09-27T23:26:56.815852+0000","last_active":"2023-09-27T23:26:56.824360+0000","last_peered":"2023-09-27T23:26:56.824360+0000","last_clean":"2023-09-27T23:26:56.824360+0000","last_became_active":"2023-09-27T23:26:56.815565+0000","last_became_peered":"2023-09-27T23:26:56.815565+0000","last_unstale":"2023-09-27T23:26:56.824360+0000","last_undegraded":"2023-09-27T23:26:56.824360+0000","last_fullsized":"2023-09-27T23:26:56.824360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:21:59.389594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799681+0000","last_change":"2023-09-27T23:26:56.431567+0000","last_active":"2023-09-27T23:26:56.799681+0000","last_peered":"2023-09-27T23:26:56.799681+0000","last_clean":"2023-09-27T23:26:56.799681+0000","last_became_active":"2023-09-27T23:26:56.431206+0000","last_became_peered":"2023-09-27T23:26:56.431206+0000","last_unstale":"2023-09-27T23:26:56.799681+0000","last_undegraded":"2023-09-27T23:26:56.799681+0000","last_fullsized":"2023-09-27T23:26:56.799681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:52:27.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824213+0000","last_change":"2023-09-27T23:26:56.811986+0000","last_active":"2023-09-27T23:26:56.824213+0000","last_peered":"2023-09-27T23:26:56.824213+0000","last_clean":"2023-09-27T23:26:56.824213+0000","last_became_active":"2023-09-27T23:26:56.811704+0000","last_became_peered":"2023-09-27T23:26:56.811704+0000","last_unstale":"2023-09-27T23:26:56.824213+0000","last_undegraded":"2023-09-27T23:26:56.824213+0000","last_fullsized":"2023-09-27T23:26:56.824213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:10:10.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.797203+0000","last_change":"2023-09-27T23:26:56.439391+0000","last_active":"2023-09-27T23:26:56.797203+0000","last_peered":"2023-09-27T23:26:56.797203+0000","last_clean":"2023-09-27T23:26:56.797203+0000","last_became_active":"2023-09-27T23:26:56.439021+0000","last_became_peered":"2023-09-27T23:26:56.439021+0000","last_unstale":"2023-09-27T23:26:56.797203+0000","last_undegraded":"2023-09-27T23:26:56.797203+0000","last_fullsized":"2023-09-27T23:26:56.797203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:59.110216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":285,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643461+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:26:52.643461+0000","last_peered":"2023-09-27T23:26:52.643461+0000","last_clean":"2023-09-27T23:26:52.643461+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:26:52.643461+0000","last_undegraded":"2023-09-27T23:26:52.643461+0000","last_fullsized":"2023-09-27T23:26:52.643461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":139,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.311206+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:26:52.311206+0000","last_peered":"2023-09-27T23:26:52.311206+0000","last_clean":"2023-09-27T23:26:52.311206+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:26:52.311206+0000","last_undegraded":"2023-09-27T23:26:52.311206+0000","last_fullsized":"2023-09-27T23:26:52.311206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798523+0000","last_change":"2023-09-27T23:26:56.428852+0000","last_active":"2023-09-27T23:26:56.798523+0000","last_peered":"2023-09-27T23:26:56.798523+0000","last_clean":"2023-09-27T23:26:56.798523+0000","last_became_active":"2023-09-27T23:26:56.428486+0000","last_became_peered":"2023-09-27T23:26:56.428486+0000","last_unstale":"2023-09-27T23:26:56.798523+0000","last_undegraded":"2023-09-27T23:26:56.798523+0000","last_fullsized":"2023-09-27T23:26:56.798523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:23:54.572803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643321+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:26:52.643321+0000","last_peered":"2023-09-27T23:26:52.643321+0000","last_clean":"2023-09-27T23:26:52.643321+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:26:52.643321+0000","last_undegraded":"2023-09-27T23:26:52.643321+0000","last_fullsized":"2023-09-27T23:26:52.643321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.787944+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:26:56.787944+0000","last_peered":"2023-09-27T23:26:56.787944+0000","last_clean":"2023-09-27T23:26:56.787944+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:26:56.787944+0000","last_undegraded":"2023-09-27T23:26:56.787944+0000","last_fullsized":"2023-09-27T23:26:56.787944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.311130+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:26:52.311130+0000","last_peered":"2023-09-27T23:26:52.311130+0000","last_clean":"2023-09-27T23:26:52.311130+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:26:52.311130+0000","last_undegraded":"2023-09-27T23:26:52.311130+0000","last_fullsized":"2023-09-27T23:26:52.311130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799914+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:26:56.799914+0000","last_peered":"2023-09-27T23:26:56.799914+0000","last_clean":"2023-09-27T23:26:56.799914+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:26:56.799914+0000","last_undegraded":"2023-09-27T23:26:56.799914+0000","last_fullsized":"2023-09-27T23:26:56.799914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:52.643342+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:26:52.643342+0000","last_peered":"2023-09-27T23:26:52.643342+0000","last_clean":"2023-09-27T23:26:52.643342+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:26:52.643342+0000","last_undegraded":"2023-09-27T23:26:52.643342+0000","last_fullsized":"2023-09-27T23:26:52.643342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799800+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:26:56.799800+0000","last_peered":"2023-09-27T23:26:56.799800+0000","last_clean":"2023-09-27T23:26:56.799800+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:26:56.799800+0000","last_undegraded":"2023-09-27T23:26:56.799800+0000","last_fullsized":"2023-09-27T23:26:56.799800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:56.392452+0000","last_change":"2023-09-27T23:26:56.392452+0000","last_active":"2023-09-27T23:26:56.392452+0000","last_peered":"2023-09-27T23:26:56.392452+0000","last_clean":"2023-09-27T23:26:56.392452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:56.392452+0000","last_undegraded":"2023-09-27T23:26:56.392452+0000","last_fullsized":"2023-09-27T23:26:56.392452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:26:55.387939+0000","last_change":"2023-09-27T23:26:55.387939+0000","last_active":"2023-09-27T23:26:55.387939+0000","last_peered":"2023-09-27T23:26:55.387939+0000","last_clean":"2023-09-27T23:26:55.387939+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:26:55.387939+0000","last_undegraded":"2023-09-27T23:26:55.387939+0000","last_fullsized":"2023-09-27T23:26:55.387939+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-09-27T23:26:53.216004+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:26:53.216004+0000","last_peered":"2023-09-27T23:26:53.216004+0000","last_clean":"2023-09-27T23:26:53.216004+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:26:53.216004+0000","last_undegraded":"2023-09-27T23:26:53.216004+0000","last_fullsized":"2023-09-27T23:26:53.216004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509602,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24404,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":23040,"kb_avail":94347436,"statfs":{"total":96636764160,"available":96611774464,"internally_reserved":0,"allocated":1388544,"data_stored":987179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23592960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63700000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.232,"5min":1.363,"15min":1.7949999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.94199999999999995,"15min":0.88500000000000001},"min":{"1min":0.63,"5min":0.58199999999999996,"15min":0.47199999999999998},"max":{"1min":1.2210000000000001,"5min":1.363,"15min":1.363},"last":1.712}]},{"osd":1,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.89900000000000002,"15min":0.90000000000000002},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.92600000000000005,"15min":0.93200000000000005},"min":{"1min":0.67600000000000005,"5min":0.66500000000000004,"15min":0.56899999999999995},"max":{"1min":1.1479999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.7589999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91300000000000003,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.62,"15min":0.61299999999999999},"max":{"1min":1.1899999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":2.681},{"interface":"front","average":{"1min":1.0609999999999999,"5min":0.98999999999999999,"15min":0.96699999999999997},"min":{"1min":0.71899999999999997,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.5920000000000001},"last":1.2929999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.94999999999999996},"min":{"1min":0.72299999999999998,"5min":0.66300000000000003,"15min":0.65100000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.421},"last":0.995},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0840000000000001,"15min":1.044},"min":{"1min":0.79500000000000004,"5min":0.75,"15min":0.622},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.4339999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89100000000000001,"15min":0.91200000000000003},"min":{"1min":0.628,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.3879999999999999},"last":2.6230000000000002},{"interface":"front","average":{"1min":0.997,"5min":0.98299999999999998,"15min":0.98799999999999999},"min":{"1min":0.84699999999999998,"5min":0.67600000000000005,"15min":0.59599999999999997},"max":{"1min":1.1930000000000001,"5min":1.3080000000000001,"15min":1.4099999999999999},"last":2.8879999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.034,"15min":1.0569999999999999},"min":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.48999999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.5880000000000001},"last":1.391},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.0960000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.64100000000000001},"max":{"1min":1.395,"5min":1.51,"15min":1.51},"last":2.7280000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:25:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1200000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.78200000000000003},"max":{"1min":1.468,"5min":1.498,"15min":1.498},"last":1.806},{"interface":"front","average":{"1min":1.139,"5min":1.1399999999999999,"15min":1.133},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.73899999999999999},"max":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":2.548}]}]},{"osd":6,"up_from":15,"seq":64424509601,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20304,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":19520,"kb_avail":94351536,"statfs":{"total":96636764160,"available":96615972864,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19988480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.68899999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.327,"15min":1.327},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.079,"5min":1.026,"15min":1.0289999999999999},"min":{"1min":0.81999999999999995,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.649},"last":0.83899999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.54700000000000004},"max":{"1min":1.2889999999999999,"5min":1.323,"15min":1.323},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1739999999999999,"5min":1.2509999999999999,"15min":1.3939999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.88},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.56799999999999995},"max":{"1min":1.0820000000000001,"5min":1.353,"15min":1.397},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60499999999999998},"max":{"1min":1.2589999999999999,"5min":1.367,"15min":1.4259999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.58999999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.734},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.84599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"max":{"1min":1.2290000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.97199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.93100000000000005},"min":{"1min":0.64900000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":1.292,"5min":1.335,"15min":1.544},"last":0.874},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.61899999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.24,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":1.0269999999999999,"5min":0.83999999999999997,"15min":0.65900000000000003},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.095},{"interface":"front","average":{"1min":1.153,"5min":1.167,"15min":1.165},"min":{"1min":0.92800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3660000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1200000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509601,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30160,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":29376,"kb_avail":94341680,"statfs":{"total":96636764160,"available":96605880320,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30081024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.151,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.2929999999999999,"15min":1.359},"last":1.3440000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.50700000000000001},"max":{"1min":1.2969999999999999,"5min":1.3109999999999999,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.88500000000000001,"15min":0.876},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.234,"15min":1.4179999999999999},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.93000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3859999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"max":{"1min":1.27,"5min":1.4730000000000001,"15min":1.5740000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91900000000000004,"15min":0.97199999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.214,"5min":1.238,"15min":1.5640000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.008,"5min":1.024,"15min":1.014},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3819999999999999,"15min":1.954},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1599999999999999,"5min":1.2829999999999999,"15min":1.5349999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.032,"15min":1.046},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.304,"5min":1.3939999999999999,"15min":1.704},"last":1.399}]},{"osd":6,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.02,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.323,"5min":1.5580000000000001,"15min":1.6319999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.3779999999999999,"15min":1.837},"last":1.3069999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1479999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3,"5min":1.496,"15min":1.7230000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.204,"15min":1.1950000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.4319999999999999,"5min":1.633,"15min":2.0529999999999999},"last":1.425}]}]},{"osd":4,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":20624,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":19840,"kb_avail":94351216,"statfs":{"total":96636764160,"available":96615645184,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20316160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.92100000000000004,"15min":0.88900000000000001},"min":{"1min":0.61099999999999999,"5min":0.57099999999999995,"15min":0.53700000000000003},"max":{"1min":1.256,"5min":1.27,"15min":1.3080000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90000000000000002,"15min":0.876},"min":{"1min":0.68500000000000005,"5min":0.63600000000000001,"15min":0.53800000000000003},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.3320000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.877,"15min":0.88800000000000001},"min":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.3380000000000001,"15min":1.3919999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95999999999999996,"15min":0.92400000000000004},"min":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.58199999999999996},"max":{"1min":1.171,"5min":1.228,"15min":1.329},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91500000000000004,"15min":0.90500000000000003},"min":{"1min":0.68300000000000005,"5min":0.63600000000000001,"15min":0.621},"max":{"1min":1.2090000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.268},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95199999999999996,"15min":0.98199999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.522},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92500000000000004,"15min":0.92700000000000005},"min":{"1min":0.72799999999999998,"5min":0.66100000000000003,"15min":0.60099999999999998},"max":{"1min":1.167,"5min":1.26,"15min":1.373},"last":1.006},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.012},"min":{"1min":0.77800000000000002,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.2929999999999999,"5min":1.5129999999999999,"15min":1.575},"last":1.0469999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89200000000000002,"15min":0.92000000000000004},"min":{"1min":0.626,"5min":0.60199999999999998,"15min":0.57599999999999996},"max":{"1min":1.2729999999999999,"5min":1.409,"15min":1.627},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.022,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.67200000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3220000000000001,"5min":1.5489999999999999,"15min":2.2730000000000001},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0249999999999999,"15min":1.0489999999999999},"min":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.60099999999999998},"max":{"1min":1.4059999999999999,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.369},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.149,"15min":1.1619999999999999},"min":{"1min":0.97499999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.478,"5min":1.478,"15min":1.8620000000000001},"last":1.137}]},{"osd":7,"last update":"Wed Sep 27 23:25:54 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0820000000000001,"15min":1.0940000000000001},"min":{"1min":0.83799999999999997,"5min":0.76500000000000001,"15min":0.64000000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.726},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1499999999999999,"15min":1.1759999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78500000000000003},"max":{"1min":1.5069999999999999,"5min":1.859,"15min":2.532},"last":1.4319999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21200,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350640,"statfs":{"total":96636764160,"available":96615055360,"internally_reserved":0,"allocated":794624,"data_stored":396811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.30199999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.746},"min":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.35299999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.337},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.39800000000000002},"max":{"1min":0.95299999999999996,"5min":1.224,"15min":1.2869999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.72199999999999998,"5min":0.61699999999999999,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.3180000000000001,"15min":1.3939999999999999},"last":0.75}]},{"osd":2,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.88300000000000001,"15min":0.85699999999999998},"min":{"1min":0.78200000000000003,"5min":0.65700000000000003,"15min":0.36699999999999999},"max":{"1min":1.1479999999999999,"5min":1.288,"15min":1.353},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91600000000000004},"min":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.45700000000000002},"max":{"1min":1.256,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.76000000000000001,"5min":0.67700000000000005,"15min":0.66100000000000003},"max":{"1min":1.175,"5min":1.466,"15min":1.5349999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.628},"max":{"1min":1.2030000000000001,"5min":1.3680000000000001,"15min":1.4950000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.91300000000000003,"5min":0.83099999999999996,"15min":0.626},"max":{"1min":1.3120000000000001,"5min":1.468,"15min":1.571},"last":1.02},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1479999999999999},"min":{"1min":0.96399999999999997,"5min":0.76700000000000002,"15min":0.58399999999999996},"max":{"1min":1.47,"5min":1.51,"15min":1.764},"last":1.252}]},{"osd":6,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.119,"15min":1.137},"min":{"1min":0.92600000000000005,"5min":0.76900000000000002,"15min":0.72799999999999998},"max":{"1min":1.407,"5min":1.4610000000000001,"15min":1.6970000000000001},"last":1.169},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":1.0269999999999999,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.3140000000000001,"5min":1.5800000000000001,"15min":1.8620000000000001},"last":1.1180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.246,"15min":1.2290000000000001},"min":{"1min":1.0289999999999999,"5min":0.86799999999999999,"15min":0.78700000000000003},"max":{"1min":1.5640000000000001,"5min":1.627,"15min":1.655},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.2},"min":{"1min":1.054,"5min":0.89500000000000002,"15min":0.73099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.671},"last":1.083}]}]},{"osd":2,"up_from":15,"seq":64424509603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20040,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":19200,"kb_avail":94351800,"statfs":{"total":96636764160,"available":96616243200,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19660800},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.68700000000000006},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":1.133,"15min":1.7370000000000001},"last":1.093},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.214,"15min":1.6080000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.79500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":1.1950000000000001,"15min":1.8069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.82799999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.3120000000000001,"15min":1.651},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.4850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.58099999999999996},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.8280000000000001},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.751,"5min":0.751,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":1.585,"15min":1.7849999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0629999999999999,"15min":1.097},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69499999999999995},"max":{"1min":1.337,"5min":1.4590000000000001,"15min":2.601},"last":0.91900000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.024,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.77400000000000002,"15min":0.71099999999999997},"max":{"1min":1.3029999999999999,"5min":1.403,"15min":2.7320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.107,"5min":1.1299999999999999,"15min":1.1559999999999999},"min":{"1min":0.84999999999999998,"5min":0.76700000000000002,"15min":0.747},"max":{"1min":1.375,"5min":1.496,"15min":2.6760000000000002},"last":1.5149999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1579999999999999,"15min":1.1910000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6560000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1559999999999999,"15min":1.1779999999999999},"min":{"1min":0.871,"5min":0.84099999999999997,"15min":0.71699999999999997},"max":{"1min":1.417,"5min":1.552,"15min":2.6230000000000002},"last":1.377}]},{"osd":7,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.218,"15min":1.2490000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.76600000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80000000000000004},"max":{"1min":1.3999999999999999,"5min":1.6200000000000001,"15min":2.7749999999999999},"last":1.282}]}]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35140,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":34304,"kb_avail":94336700,"statfs":{"total":96636764160,"available":96600780800,"internally_reserved":0,"allocated":847872,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35127296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.503,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.3560000000000001,"15min":3.798},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":1.151,"15min":2.3330000000000002},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.27500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":6.157},"last":1.179},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.879,"15min":0.97499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":5.649},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1919999999999999,"5min":1.323,"15min":6.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.294,"15min":1.9370000000000001},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.173,"5min":1.3340000000000001,"15min":1.863},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.133,"15min":1.1579999999999999},"min":{"1min":0.749,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.381,"5min":1.579,"15min":4.0499999999999998},"last":1.294}]},{"osd":5,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.65100000000000002},"max":{"1min":1.476,"5min":1.524,"15min":2.7040000000000002},"last":0.999},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.0960000000000001},"min":{"1min":0.80900000000000005,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.365,"5min":1.53,"15min":2.2509999999999999},"last":1.365}]},{"osd":6,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.234,"15min":1.2470000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.587,"5min":1.6919999999999999,"15min":3.9260000000000002},"last":1.22},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1910000000000001,"15min":1.1890000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.76400000000000001},"max":{"1min":1.556,"5min":1.5900000000000001,"15min":2.79},"last":1.127}]},{"osd":7,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.218},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.64800000000000002},"max":{"1min":1.532,"5min":1.7210000000000001,"15min":2.843},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.429,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.429}]}]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21396,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350444,"statfs":{"total":96636764160,"available":96614854656,"internally_reserved":0,"allocated":1388544,"data_stored":987179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0229999999999999,"15min":1.901},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81000000000000005,"15min":0.78400000000000003},"min":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.54700000000000004},"max":{"1min":1.0669999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73399999999999999,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.51200000000000001,"15min":0.39600000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.60299999999999998,"5min":0.56999999999999995,"15min":0.41499999999999998},"max":{"1min":1.069,"5min":1.1020000000000001,"15min":1.335},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85999999999999999,"15min":0.85299999999999998},"min":{"1min":0.67700000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":1.1719999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.89500000000000002,"15min":0.85999999999999999},"min":{"1min":0.70099999999999996,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.149,"5min":1.286,"15min":1.3400000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.79800000000000004,"5min":0.71099999999999997,"15min":0.59199999999999997},"max":{"1min":1.2569999999999999,"5min":1.4590000000000001,"15min":2.2519999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.05,"15min":1.0629999999999999},"min":{"1min":0.78800000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5589999999999999,"5min":1.587,"15min":1.952},"last":0.98999999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0860000000000001,"15min":1.0920000000000001},"min":{"1min":0.85499999999999998,"5min":0.76300000000000001,"15min":0.71899999999999997},"max":{"1min":1.208,"5min":1.78,"15min":1.986},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1479999999999999,"15min":1.153},"min":{"1min":0.92900000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.4199999999999999,"5min":1.5780000000000001,"15min":2.0710000000000002},"last":1.3440000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1879999999999999,"15min":1.1619999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.4750000000000001,"5min":1.5940000000000001,"15min":1.7729999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.228,"15min":1.2070000000000001},"min":{"1min":1.012,"5min":0.90800000000000003,"15min":0.61199999999999999},"max":{"1min":1.6140000000000001,"5min":1.8300000000000001,"15min":2.1850000000000001},"last":1.306}]},{"osd":7,"last update":"Wed Sep 27 23:25:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2270000000000001,"15min":1.23},"min":{"1min":0.90600000000000003,"5min":0.85999999999999999,"15min":0.71999999999999997},"max":{"1min":1.55,"5min":1.7090000000000001,"15min":2.3130000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":1.054,"5min":0.97499999999999998,"15min":0.66400000000000003},"max":{"1min":1.4239999999999999,"5min":1.5269999999999999,"15min":2.3540000000000001},"last":1.407}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:27:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-09-27T23:27:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-09-27T23:27:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-09-27T23:27:01.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-09-27T23:27:01.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-09-27T23:27:01.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-09-27T23:27:01.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-09-27T23:27:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-09-27T23:27:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-09-27T23:27:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-09-27T23:27:01.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-09-27T23:27:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-09-27T23:27:01.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-09-27T23:27:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-09-27T23:27:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-09-27T23:27:01.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-09-27T23:27:01.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:56.392452+0000', 'last_change': '2023-09-27T23:26:56.392452+0000', 'last_active': '2023-09-27T23:26:56.392452+0000', 'last_peered': '2023-09-27T23:26:56.392452+0000', 'last_clean': '2023-09-27T23:26:56.392452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:56.392452+0000', 'last_undegraded': '2023-09-27T23:26:56.392452+0000', 'last_fullsized': '2023-09-27T23:26:56.392452+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:56.392452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-09-27T23:27:01.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:26:55.387939+0000', 'last_change': '2023-09-27T23:26:55.387939+0000', 'last_active': '2023-09-27T23:26:55.387939+0000', 'last_peered': '2023-09-27T23:26:55.387939+0000', 'last_clean': '2023-09-27T23:26:55.387939+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:26:55.387939+0000', 'last_undegraded': '2023-09-27T23:26:55.387939+0000', 'last_fullsized': '2023-09-27T23:26:55.387939+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'last_clean_scrub_stamp': '2023-09-27T23:26:55.387939+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:27:01.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:27:01.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.623+0000 7f5e9759e700 1 Processor -- start 2023-09-27T23:27:01.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.623+0000 7f5e9759e700 1 -- start start 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e9806d760 0x7f5e981591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e98159700 0x7f5e980c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e98159f90 con 0x7f5e98158df0 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9815a270 con 0x7f5e9806d760 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e980c7930 con 0x7f5e98159700 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e9806d760 0x7f5e981591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e9806d760 0x7f5e981591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49200/0 (socket says 172.21.15.64:49200) 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 -- 172.21.15.64:0/2351884233 learned_addr learned my addr 172.21.15.64:0/2351884233 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e96d9d700 1 --2- 172.21.15.64:0/2351884233 >> v2:172.21.15.123:3300/0 conn(0x7f5e98159700 0x7f5e980c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 -- 172.21.15.64:0/2351884233 >> v2:172.21.15.123:3300/0 conn(0x7f5e98159700 msgr2=0x7f5e980c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:01.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 --2- 172.21.15.64:0/2351884233 >> v2:172.21.15.123:3300/0 conn(0x7f5e98159700 0x7f5e980c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 -- 172.21.15.64:0/2351884233 >> v1:172.21.15.123:6789/0 conn(0x7f5e98158df0 legacy=0x7f5e9806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 -- 172.21.15.64:0/2351884233 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980c7c10 con 0x7f5e9806d760 2023-09-27T23:27:01.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e95d9b700 1 --2- 172.21.15.64:0/2351884233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9806d760 0x7f5e981591c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5e980c2d00 tx=0x7f5e80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bb6172e3090c9e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9559a700 1 -- 172.21.15.64:0/2351884233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000e070 con 0x7f5e9806d760 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9559a700 1 -- 172.21.15.64:0/2351884233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8000ba50 con 0x7f5e9806d760 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9559a700 1 -- 172.21.15.64:0/2351884233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800092e0 con 0x7f5e9806d760 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- 172.21.15.64:0/2351884233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9806d760 msgr2=0x7f5e981591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 --2- 172.21.15.64:0/2351884233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9806d760 0x7f5e981591c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5e980c2d00 tx=0x7f5e80009840 comp rx=0 tx=0).stop 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- 172.21.15.64:0/2351884233 shutdown_connections 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 --2- 172.21.15.64:0/2351884233 >> v2:172.21.15.123:3300/0 conn(0x7f5e98159700 0x7f5e980c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 --2- 172.21.15.64:0/2351884233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e9806d760 0x7f5e981591c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- 172.21.15.64:0/2351884233 >> 172.21.15.64:0/2351884233 conn(0x7f5e9806c080 msgr2=0x7f5e980ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- 172.21.15.64:0/2351884233 shutdown_connections 2023-09-27T23:27:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.627+0000 7f5e9759e700 1 -- 172.21.15.64:0/2351884233 wait complete. 2023-09-27T23:27:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 Processor -- start 2023-09-27T23:27:01.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 -- start start 2023-09-27T23:27:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9806d760 0x7f5e980b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e98158df0 0x7f5e981566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 0x7f5e981581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980aab80 con 0x7f5e98159700 2023-09-27T23:27:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980a88f0 con 0x7f5e9806d760 2023-09-27T23:27:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980b1f10 con 0x7f5e98158df0 2023-09-27T23:27:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 0x7f5e981581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 0x7f5e981581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49210/0 (socket says 172.21.15.64:49210) 2023-09-27T23:27:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e9659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9806d760 0x7f5e980b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 -- 172.21.15.64:0/150124558 learned_addr learned my addr 172.21.15.64:0/150124558 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:01.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e95d9b700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e98158df0 0x7f5e981566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 -- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e98158df0 msgr2=0x7f5e981566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e98158df0 0x7f5e981566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 -- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9806d760 msgr2=0x7f5e980b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9806d760 0x7f5e980b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 -- 172.21.15.64:0/150124558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e80009ae0 con 0x7f5e98159700 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e96d9d700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 0x7f5e981581d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000d890 tx=0x7f5e90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e900125f0 con 0x7f5e98159700 2023-09-27T23:27:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.631+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90012770 con 0x7f5e98159700 2023-09-27T23:27:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e980aad90 con 0x7f5e98159700 2023-09-27T23:27:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90012b30 con 0x7f5e98159700 2023-09-27T23:27:01.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e980ab630 con 0x7f5e98159700 2023-09-27T23:27:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e980ab180 con 0x7f5e98159700 2023-09-27T23:27:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5e9001e460 con 0x7f5e98159700 2023-09-27T23:27:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e8e7fc700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e78060aa0 0x7f5e78062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.635+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5e9007d5f0 con 0x7f5e98159700 2023-09-27T23:27:01.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.639+0000 7f5e9659c700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e78060aa0 0x7f5e78062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.639+0000 7f5e9659c700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e78060aa0 0x7f5e78062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e88003a80 tx=0x7f5e88007ec0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.647+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5e900510b0 con 0x7f5e98159700 2023-09-27T23:27:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.867+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5e9804be40 con 0x7f5e98159700 2023-09-27T23:27:01.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.867+0000 7f5e8e7fc700 1 -- 172.21.15.64:0/150124558 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3282 (secure 0 0 0) 0x7f5e90050c40 con 0x7f5e98159700 2023-09-27T23:27:01.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:01.871 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":820,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":71,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":80},{"state_name":"active+clean","count":25}],"num_pgs":105,"num_pools":4,"num_objects":3,"data_bytes":590368,"bytes_used":197906432,"bytes_avail":772896206848,"bytes_total":773094113280,"unknown_pgs_ratio":0.76190477609634399,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":21,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":6096}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-09-27T23:26:45.759878+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:27:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e78060aa0 msgr2=0x7f5e78062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e78060aa0 0x7f5e78062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e88003a80 tx=0x7f5e88007ec0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 msgr2=0x7f5e981581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 0x7f5e981581d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000d890 tx=0x7f5e90008040 comp rx=0 tx=0).stop 2023-09-27T23:27:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 shutdown_connections 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e78060aa0 0x7f5e78062f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e98159700 0x7f5e981581d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e98158df0 0x7f5e981566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 --2- 172.21.15.64:0/150124558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9806d760 0x7f5e980b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 >> 172.21.15.64:0/150124558 conn(0x7f5e9806c080 msgr2=0x7f5e9806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 shutdown_connections 2023-09-27T23:27:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.871+0000 7f5e9759e700 1 -- 172.21.15.64:0/150124558 wait complete. 2023-09-27T23:27:01.885 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:27:01.886 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:27:01.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 Processor -- start 2023-09-27T23:27:01.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 -- start start 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42ec140710 0x7f42ec0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec068800 0x7f42ec0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec06e150 con 0x7f42ec140ae0 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec06d540 con 0x7f42ec140710 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.991+0000 7f42f171d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec06d820 con 0x7f42ec068800 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eaffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42ec140710 0x7f42ec0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42ea7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f42ec140ae0 0x7f42ec0680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:54222/0 (socket says 172.21.15.64:54222) 2023-09-27T23:27:01.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42ea7fc700 1 -- 172.21.15.64:0/2261785920 learned_addr learned my addr 172.21.15.64:0/2261785920 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:01.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eb7fe700 1 --2- 172.21.15.64:0/2261785920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec068800 0x7f42ec0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:01.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42e9ffb700 1 -- 172.21.15.64:0/2261785920 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605734104 0 0) 0x7f42ec06e150 con 0x7f42ec140ae0 2023-09-27T23:27:01.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42e9ffb700 1 -- 172.21.15.64:0/2261785920 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d8003570 con 0x7f42ec140ae0 2023-09-27T23:27:01.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eaffd700 1 -- 172.21.15.64:0/2261785920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec068800 msgr2=0x7f42ec0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:01.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eaffd700 1 --2- 172.21.15.64:0/2261785920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec068800 0x7f42ec0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eaffd700 1 -- 172.21.15.64:0/2261785920 >> v1:172.21.15.123:6789/0 conn(0x7f42ec140ae0 legacy=0x7f42ec0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eaffd700 1 -- 172.21.15.64:0/2261785920 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec06db00 con 0x7f42ec140710 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42eaffd700 1 --2- 172.21.15.64:0/2261785920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0c70a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f42d40098e0 tx=0x7f42d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7617d754563dbb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42e9ffb700 1 -- 172.21.15.64:0/2261785920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4011630 con 0x7f42ec140710 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42e9ffb700 1 -- 172.21.15.64:0/2261785920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4011790 con 0x7f42ec140710 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42e9ffb700 1 -- 172.21.15.64:0/2261785920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4011b10 con 0x7f42ec140710 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 -- 172.21.15.64:0/2261785920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 msgr2=0x7f42ec0c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 --2- 172.21.15.64:0/2261785920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0c70a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f42d40098e0 tx=0x7f42d40091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 -- 172.21.15.64:0/2261785920 shutdown_connections 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 --2- 172.21.15.64:0/2261785920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec068800 0x7f42ec0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 --2- 172.21.15.64:0/2261785920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0c70a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 -- 172.21.15.64:0/2261785920 >> 172.21.15.64:0/2261785920 conn(0x7f42ec0c8850 msgr2=0x7f42ec0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:02.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 -- 172.21.15.64:0/2261785920 shutdown_connections 2023-09-27T23:27:02.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.995+0000 7f42f171d700 1 -- 172.21.15.64:0/2261785920 wait complete. 2023-09-27T23:27:02.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 Processor -- start 2023-09-27T23:27:02.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 -- start start 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ec068800 0x7f42ec159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec140ae0 0x7f42ec0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0c2580 con 0x7f42ec140710 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0c1900 con 0x7f42ec068800 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0c1be0 con 0x7f42ec140ae0 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49234/0 (socket says 172.21.15.64:49234) 2023-09-27T23:27:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 -- 172.21.15.64:0/1086908278 learned_addr learned my addr 172.21.15.64:0/1086908278 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42eaffd700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ec068800 0x7f42ec159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42eb7fe700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec140ae0 0x7f42ec0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 -- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec140ae0 msgr2=0x7f42ec0aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec140ae0 0x7f42ec0aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 -- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ec068800 msgr2=0x7f42ec159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ec068800 0x7f42ec159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 -- 172.21.15.64:0/1086908278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d4009420 con 0x7f42ec140710 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42ea7fc700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0b11d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0011e0 tx=0x7f42dc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc005b30 con 0x7f42ec140710 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc005c90 con 0x7f42ec140710 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc007300 con 0x7f42ec140710 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec0c1ec0 con 0x7f42ec140710 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:01.999+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec15a370 con 0x7f42ec140710 2023-09-27T23:27:02.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.003+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec0a8d80 con 0x7f42ec140710 2023-09-27T23:27:02.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.003+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f42dc007670 con 0x7f42ec140710 2023-09-27T23:27:02.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.003+0000 7f42e2ffd700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f42cc060a70 0x7f42cc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.003+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f42dc07c4f0 con 0x7f42ec140710 2023-09-27T23:27:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.003+0000 7f42eaffd700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f42cc060a70 0x7f42cc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:02.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.007+0000 7f42eaffd700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f42cc060a70 0x7f42cc062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42d40091e0 tx=0x7f42d4017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:02.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.011+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f42ec0a8d80 con 0x7f42ec140710 2023-09-27T23:27:02.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.167+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42ec0a8d80 con 0x7f42cc060a70 2023-09-27T23:27:02.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.187+0000 7f42e2ffd700 1 -- 172.21.15.64:0/1086908278 <== mgr.5402 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307360 (secure 0 0 0) 0x7f42ec0a8d80 con 0x7f42cc060a70 2023-09-27T23:27:02.193 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:02.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f42cc060a70 msgr2=0x7f42cc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:02.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f42cc060a70 0x7f42cc062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42d40091e0 tx=0x7f42d4017000 comp rx=0 tx=0).stop 2023-09-27T23:27:02.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 msgr2=0x7f42ec0b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0b11d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0011e0 tx=0x7f42dc007df0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 shutdown_connections 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f42cc060a70 0x7f42cc062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42ec140ae0 0x7f42ec0aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42ec140710 0x7f42ec0b11d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 --2- 172.21.15.64:0/1086908278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ec068800 0x7f42ec159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 >> 172.21.15.64:0/1086908278 conn(0x7f42ec0c8850 msgr2=0x7f42ec13ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 shutdown_connections 2023-09-27T23:27:02.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:02.191+0000 7f42f171d700 1 -- 172.21.15.64:0/1086908278 wait complete. 2023-09-27T23:27:02.197 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:27:02.208 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:27:01.765352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":194504,"kb_used_data":7816,"kb_used_omap":0,"kb_used_meta":186624,"kb_avail":754780216,"statfs":{"total":773094113280,"available":772894941184,"internally_reserved":0,"allocated":8003584,"data_stored":4582260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":191102976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001961"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.814138+0000","last_change":"2023-09-27T23:26:56.811633+0000","last_active":"2023-09-27T23:26:56.814138+0000","last_peered":"2023-09-27T23:26:56.814138+0000","last_clean":"2023-09-27T23:26:56.814138+0000","last_became_active":"2023-09-27T23:26:56.811088+0000","last_became_peered":"2023-09-27T23:26:56.811088+0000","last_unstale":"2023-09-27T23:26:56.814138+0000","last_undegraded":"2023-09-27T23:26:56.814138+0000","last_fullsized":"2023-09-27T23:26:56.814138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:46.867145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822270+0000","last_change":"2023-09-27T23:26:56.818433+0000","last_active":"2023-09-27T23:26:56.822270+0000","last_peered":"2023-09-27T23:26:56.822270+0000","last_clean":"2023-09-27T23:26:56.822270+0000","last_became_active":"2023-09-27T23:26:56.818166+0000","last_became_peered":"2023-09-27T23:26:56.818166+0000","last_unstale":"2023-09-27T23:26:56.822270+0000","last_undegraded":"2023-09-27T23:26:56.822270+0000","last_fullsized":"2023-09-27T23:26:56.822270+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:43.997454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.829130+0000","last_change":"2023-09-27T23:26:56.821988+0000","last_active":"2023-09-27T23:26:56.829130+0000","last_peered":"2023-09-27T23:26:56.829130+0000","last_clean":"2023-09-27T23:26:56.829130+0000","last_became_active":"2023-09-27T23:26:56.821689+0000","last_became_peered":"2023-09-27T23:26:56.821689+0000","last_unstale":"2023-09-27T23:26:56.829130+0000","last_undegraded":"2023-09-27T23:26:56.829130+0000","last_fullsized":"2023-09-27T23:26:56.829130+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:44.053616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810217+0000","last_change":"2023-09-27T23:26:56.804550+0000","last_active":"2023-09-27T23:26:56.810217+0000","last_peered":"2023-09-27T23:26:56.810217+0000","last_clean":"2023-09-27T23:26:56.810217+0000","last_became_active":"2023-09-27T23:26:56.802913+0000","last_became_peered":"2023-09-27T23:26:56.802913+0000","last_unstale":"2023-09-27T23:26:56.810217+0000","last_undegraded":"2023-09-27T23:26:56.810217+0000","last_fullsized":"2023-09-27T23:26:56.810217+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:20:40.278734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800866+0000","last_change":"2023-09-27T23:26:56.796931+0000","last_active":"2023-09-27T23:26:56.800866+0000","last_peered":"2023-09-27T23:26:56.800866+0000","last_clean":"2023-09-27T23:26:56.800866+0000","last_became_active":"2023-09-27T23:26:56.796639+0000","last_became_peered":"2023-09-27T23:26:56.796639+0000","last_unstale":"2023-09-27T23:26:56.800866+0000","last_undegraded":"2023-09-27T23:26:56.800866+0000","last_fullsized":"2023-09-27T23:26:56.800866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:13:35.626367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827692+0000","last_change":"2023-09-27T23:26:56.809034+0000","last_active":"2023-09-27T23:26:56.827692+0000","last_peered":"2023-09-27T23:26:56.827692+0000","last_clean":"2023-09-27T23:26:56.827692+0000","last_became_active":"2023-09-27T23:26:56.808019+0000","last_became_peered":"2023-09-27T23:26:56.808019+0000","last_unstale":"2023-09-27T23:26:56.827692+0000","last_undegraded":"2023-09-27T23:26:56.827692+0000","last_fullsized":"2023-09-27T23:26:56.827692+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:06.599201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.820477+0000","last_change":"2023-09-27T23:26:56.810828+0000","last_active":"2023-09-27T23:26:56.820477+0000","last_peered":"2023-09-27T23:26:56.820477+0000","last_clean":"2023-09-27T23:26:56.820477+0000","last_became_active":"2023-09-27T23:26:56.810510+0000","last_became_peered":"2023-09-27T23:26:56.810510+0000","last_unstale":"2023-09-27T23:26:56.820477+0000","last_undegraded":"2023-09-27T23:26:56.820477+0000","last_fullsized":"2023-09-27T23:26:56.820477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:37:25.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822053+0000","last_change":"2023-09-27T23:26:56.810760+0000","last_active":"2023-09-27T23:26:56.822053+0000","last_peered":"2023-09-27T23:26:56.822053+0000","last_clean":"2023-09-27T23:26:56.822053+0000","last_became_active":"2023-09-27T23:26:56.810479+0000","last_became_peered":"2023-09-27T23:26:56.810479+0000","last_unstale":"2023-09-27T23:26:56.822053+0000","last_undegraded":"2023-09-27T23:26:56.822053+0000","last_fullsized":"2023-09-27T23:26:56.822053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:15:14.077484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.832378+0000","last_change":"2023-09-27T23:26:56.830036+0000","last_active":"2023-09-27T23:26:56.832378+0000","last_peered":"2023-09-27T23:26:56.832378+0000","last_clean":"2023-09-27T23:26:56.832378+0000","last_became_active":"2023-09-27T23:26:56.829721+0000","last_became_peered":"2023-09-27T23:26:56.829721+0000","last_unstale":"2023-09-27T23:26:56.832378+0000","last_undegraded":"2023-09-27T23:26:56.832378+0000","last_fullsized":"2023-09-27T23:26:56.832378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:06.599201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824033+0000","last_change":"2023-09-27T23:26:56.812859+0000","last_active":"2023-09-27T23:26:56.824033+0000","last_peered":"2023-09-27T23:26:56.824033+0000","last_clean":"2023-09-27T23:26:56.824033+0000","last_became_active":"2023-09-27T23:26:56.811956+0000","last_became_peered":"2023-09-27T23:26:56.811956+0000","last_unstale":"2023-09-27T23:26:56.824033+0000","last_undegraded":"2023-09-27T23:26:56.824033+0000","last_fullsized":"2023-09-27T23:26:56.824033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:29.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816795+0000","last_change":"2023-09-27T23:26:56.805621+0000","last_active":"2023-09-27T23:26:56.816795+0000","last_peered":"2023-09-27T23:26:56.816795+0000","last_clean":"2023-09-27T23:26:56.816795+0000","last_became_active":"2023-09-27T23:26:56.805331+0000","last_became_peered":"2023-09-27T23:26:56.805331+0000","last_unstale":"2023-09-27T23:26:56.816795+0000","last_undegraded":"2023-09-27T23:26:56.816795+0000","last_fullsized":"2023-09-27T23:26:56.816795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:05.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816125+0000","last_change":"2023-09-27T23:26:56.814235+0000","last_active":"2023-09-27T23:26:56.816125+0000","last_peered":"2023-09-27T23:26:56.816125+0000","last_clean":"2023-09-27T23:26:56.816125+0000","last_became_active":"2023-09-27T23:26:56.813878+0000","last_became_peered":"2023-09-27T23:26:56.813878+0000","last_unstale":"2023-09-27T23:26:56.816125+0000","last_undegraded":"2023-09-27T23:26:56.816125+0000","last_fullsized":"2023-09-27T23:26:56.816125+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:38.417239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831279+0000","last_change":"2023-09-27T23:26:56.829620+0000","last_active":"2023-09-27T23:26:56.831279+0000","last_peered":"2023-09-27T23:26:56.831279+0000","last_clean":"2023-09-27T23:26:56.831279+0000","last_became_active":"2023-09-27T23:26:56.829263+0000","last_became_peered":"2023-09-27T23:26:56.829263+0000","last_unstale":"2023-09-27T23:26:56.831279+0000","last_undegraded":"2023-09-27T23:26:56.831279+0000","last_fullsized":"2023-09-27T23:26:56.831279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:12.064989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.820204+0000","last_change":"2023-09-27T23:26:56.809406+0000","last_active":"2023-09-27T23:26:56.820204+0000","last_peered":"2023-09-27T23:26:56.820204+0000","last_clean":"2023-09-27T23:26:56.820204+0000","last_became_active":"2023-09-27T23:26:56.809041+0000","last_became_peered":"2023-09-27T23:26:56.809041+0000","last_unstale":"2023-09-27T23:26:56.820204+0000","last_undegraded":"2023-09-27T23:26:56.820204+0000","last_fullsized":"2023-09-27T23:26:56.820204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:15:14.077484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822441+0000","last_change":"2023-09-27T23:26:56.819439+0000","last_active":"2023-09-27T23:26:56.822441+0000","last_peered":"2023-09-27T23:26:56.822441+0000","last_clean":"2023-09-27T23:26:56.822441+0000","last_became_active":"2023-09-27T23:26:56.819086+0000","last_became_peered":"2023-09-27T23:26:56.819086+0000","last_unstale":"2023-09-27T23:26:56.822441+0000","last_undegraded":"2023-09-27T23:26:56.822441+0000","last_fullsized":"2023-09-27T23:26:56.822441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:44:25.686309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831468+0000","last_change":"2023-09-27T23:26:56.822019+0000","last_active":"2023-09-27T23:26:56.831468+0000","last_peered":"2023-09-27T23:26:56.831468+0000","last_clean":"2023-09-27T23:26:56.831468+0000","last_became_active":"2023-09-27T23:26:56.821498+0000","last_became_peered":"2023-09-27T23:26:56.821498+0000","last_unstale":"2023-09-27T23:26:56.831468+0000","last_undegraded":"2023-09-27T23:26:56.831468+0000","last_fullsized":"2023-09-27T23:26:56.831468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:38.823773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823344+0000","last_change":"2023-09-27T23:26:56.809825+0000","last_active":"2023-09-27T23:26:56.823344+0000","last_peered":"2023-09-27T23:26:56.823344+0000","last_clean":"2023-09-27T23:26:56.823344+0000","last_became_active":"2023-09-27T23:26:56.809496+0000","last_became_peered":"2023-09-27T23:26:56.809496+0000","last_unstale":"2023-09-27T23:26:56.823344+0000","last_undegraded":"2023-09-27T23:26:56.823344+0000","last_fullsized":"2023-09-27T23:26:56.823344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:26:04.894120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830665+0000","last_change":"2023-09-27T23:26:56.826954+0000","last_active":"2023-09-27T23:26:56.830665+0000","last_peered":"2023-09-27T23:26:56.830665+0000","last_clean":"2023-09-27T23:26:56.830665+0000","last_became_active":"2023-09-27T23:26:56.825620+0000","last_became_peered":"2023-09-27T23:26:56.825620+0000","last_unstale":"2023-09-27T23:26:56.830665+0000","last_undegraded":"2023-09-27T23:26:56.830665+0000","last_fullsized":"2023-09-27T23:26:56.830665+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:33:30.240296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810612+0000","last_change":"2023-09-27T23:26:56.805480+0000","last_active":"2023-09-27T23:26:56.810612+0000","last_peered":"2023-09-27T23:26:56.810612+0000","last_clean":"2023-09-27T23:26:56.810612+0000","last_became_active":"2023-09-27T23:26:56.805168+0000","last_became_peered":"2023-09-27T23:26:56.805168+0000","last_unstale":"2023-09-27T23:26:56.810612+0000","last_undegraded":"2023-09-27T23:26:56.810612+0000","last_fullsized":"2023-09-27T23:26:56.810612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:41.801806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827369+0000","last_change":"2023-09-27T23:26:56.824083+0000","last_active":"2023-09-27T23:26:56.827369+0000","last_peered":"2023-09-27T23:26:56.827369+0000","last_clean":"2023-09-27T23:26:56.827369+0000","last_became_active":"2023-09-27T23:26:56.823794+0000","last_became_peered":"2023-09-27T23:26:56.823794+0000","last_unstale":"2023-09-27T23:26:56.827369+0000","last_undegraded":"2023-09-27T23:26:56.827369+0000","last_fullsized":"2023-09-27T23:26:56.827369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:32.765279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830531+0000","last_change":"2023-09-27T23:26:56.824788+0000","last_active":"2023-09-27T23:26:56.830531+0000","last_peered":"2023-09-27T23:26:56.830531+0000","last_clean":"2023-09-27T23:26:56.830531+0000","last_became_active":"2023-09-27T23:26:56.822995+0000","last_became_peered":"2023-09-27T23:26:56.822995+0000","last_unstale":"2023-09-27T23:26:56.830531+0000","last_undegraded":"2023-09-27T23:26:56.830531+0000","last_fullsized":"2023-09-27T23:26:56.830531+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:12:30.183845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822214+0000","last_change":"2023-09-27T23:26:56.815124+0000","last_active":"2023-09-27T23:26:56.822214+0000","last_peered":"2023-09-27T23:26:56.822214+0000","last_clean":"2023-09-27T23:26:56.822214+0000","last_became_active":"2023-09-27T23:26:56.814844+0000","last_became_peered":"2023-09-27T23:26:56.814844+0000","last_unstale":"2023-09-27T23:26:56.822214+0000","last_undegraded":"2023-09-27T23:26:56.822214+0000","last_fullsized":"2023-09-27T23:26:56.822214+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:07:07.711075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830575+0000","last_change":"2023-09-27T23:26:56.825713+0000","last_active":"2023-09-27T23:26:56.830575+0000","last_peered":"2023-09-27T23:26:56.830575+0000","last_clean":"2023-09-27T23:26:56.830575+0000","last_became_active":"2023-09-27T23:26:56.825227+0000","last_became_peered":"2023-09-27T23:26:56.825227+0000","last_unstale":"2023-09-27T23:26:56.830575+0000","last_undegraded":"2023-09-27T23:26:56.830575+0000","last_fullsized":"2023-09-27T23:26:56.830575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:39.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.815977+0000","last_change":"2023-09-27T23:26:56.811719+0000","last_active":"2023-09-27T23:26:56.815977+0000","last_peered":"2023-09-27T23:26:56.815977+0000","last_clean":"2023-09-27T23:26:56.815977+0000","last_became_active":"2023-09-27T23:26:56.811372+0000","last_became_peered":"2023-09-27T23:26:56.811372+0000","last_unstale":"2023-09-27T23:26:56.815977+0000","last_undegraded":"2023-09-27T23:26:56.815977+0000","last_fullsized":"2023-09-27T23:26:56.815977+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:48:43.034321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818742+0000","last_change":"2023-09-27T23:26:56.814879+0000","last_active":"2023-09-27T23:26:56.818742+0000","last_peered":"2023-09-27T23:26:56.818742+0000","last_clean":"2023-09-27T23:26:56.818742+0000","last_became_active":"2023-09-27T23:26:56.814574+0000","last_became_peered":"2023-09-27T23:26:56.814574+0000","last_unstale":"2023-09-27T23:26:56.818742+0000","last_undegraded":"2023-09-27T23:26:56.818742+0000","last_fullsized":"2023-09-27T23:26:56.818742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:38.070621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819381+0000","last_change":"2023-09-27T23:26:56.817033+0000","last_active":"2023-09-27T23:26:56.819381+0000","last_peered":"2023-09-27T23:26:56.819381+0000","last_clean":"2023-09-27T23:26:56.819381+0000","last_became_active":"2023-09-27T23:26:56.816540+0000","last_became_peered":"2023-09-27T23:26:56.816540+0000","last_unstale":"2023-09-27T23:26:56.819381+0000","last_undegraded":"2023-09-27T23:26:56.819381+0000","last_fullsized":"2023-09-27T23:26:56.819381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:12:30.183845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831552+0000","last_change":"2023-09-27T23:26:56.825743+0000","last_active":"2023-09-27T23:26:56.831552+0000","last_peered":"2023-09-27T23:26:56.831552+0000","last_clean":"2023-09-27T23:26:56.831552+0000","last_became_active":"2023-09-27T23:26:56.823231+0000","last_became_peered":"2023-09-27T23:26:56.823231+0000","last_unstale":"2023-09-27T23:26:56.831552+0000","last_undegraded":"2023-09-27T23:26:56.831552+0000","last_fullsized":"2023-09-27T23:26:56.831552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:34:23.130491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.802160+0000","last_change":"2023-09-27T23:26:56.800806+0000","last_active":"2023-09-27T23:26:56.802160+0000","last_peered":"2023-09-27T23:26:56.802160+0000","last_clean":"2023-09-27T23:26:56.802160+0000","last_became_active":"2023-09-27T23:26:56.800324+0000","last_became_peered":"2023-09-27T23:26:56.800324+0000","last_unstale":"2023-09-27T23:26:56.802160+0000","last_undegraded":"2023-09-27T23:26:56.802160+0000","last_fullsized":"2023-09-27T23:26:56.802160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:58.847869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.808016+0000","last_change":"2023-09-27T23:26:56.805693+0000","last_active":"2023-09-27T23:26:56.808016+0000","last_peered":"2023-09-27T23:26:56.808016+0000","last_clean":"2023-09-27T23:26:56.808016+0000","last_became_active":"2023-09-27T23:26:56.805419+0000","last_became_peered":"2023-09-27T23:26:56.805419+0000","last_unstale":"2023-09-27T23:26:56.808016+0000","last_undegraded":"2023-09-27T23:26:56.808016+0000","last_fullsized":"2023-09-27T23:26:56.808016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:06.599201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830495+0000","last_change":"2023-09-27T23:26:56.825150+0000","last_active":"2023-09-27T23:26:56.830495+0000","last_peered":"2023-09-27T23:26:56.830495+0000","last_clean":"2023-09-27T23:26:56.830495+0000","last_became_active":"2023-09-27T23:26:56.822922+0000","last_became_peered":"2023-09-27T23:26:56.822922+0000","last_unstale":"2023-09-27T23:26:56.830495+0000","last_undegraded":"2023-09-27T23:26:56.830495+0000","last_fullsized":"2023-09-27T23:26:56.830495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:20:40.278734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819184+0000","last_change":"2023-09-27T23:26:56.817217+0000","last_active":"2023-09-27T23:26:56.819184+0000","last_peered":"2023-09-27T23:26:56.819184+0000","last_clean":"2023-09-27T23:26:56.819184+0000","last_became_active":"2023-09-27T23:26:56.816937+0000","last_became_peered":"2023-09-27T23:26:56.816937+0000","last_unstale":"2023-09-27T23:26:56.819184+0000","last_undegraded":"2023-09-27T23:26:56.819184+0000","last_fullsized":"2023-09-27T23:26:56.819184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:17:55.307710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831445+0000","last_change":"2023-09-27T23:26:56.821769+0000","last_active":"2023-09-27T23:26:56.831445+0000","last_peered":"2023-09-27T23:26:56.831445+0000","last_clean":"2023-09-27T23:26:56.831445+0000","last_became_active":"2023-09-27T23:26:56.818198+0000","last_became_peered":"2023-09-27T23:26:56.818198+0000","last_unstale":"2023-09-27T23:26:56.831445+0000","last_undegraded":"2023-09-27T23:26:56.831445+0000","last_fullsized":"2023-09-27T23:26:56.831445+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:44:25.686309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.832315+0000","last_change":"2023-09-27T23:26:56.828745+0000","last_active":"2023-09-27T23:26:56.832315+0000","last_peered":"2023-09-27T23:26:56.832315+0000","last_clean":"2023-09-27T23:26:56.832315+0000","last_became_active":"2023-09-27T23:26:56.828288+0000","last_became_peered":"2023-09-27T23:26:56.828288+0000","last_unstale":"2023-09-27T23:26:56.832315+0000","last_undegraded":"2023-09-27T23:26:56.832315+0000","last_fullsized":"2023-09-27T23:26:56.832315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:37:25.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"70'4","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828290+0000","last_change":"2023-09-27T23:26:56.436638+0000","last_active":"2023-09-27T23:26:56.828290+0000","last_peered":"2023-09-27T23:26:56.828290+0000","last_clean":"2023-09-27T23:26:56.828290+0000","last_became_active":"2023-09-27T23:26:56.436325+0000","last_became_peered":"2023-09-27T23:26:56.436325+0000","last_unstale":"2023-09-27T23:26:56.828290+0000","last_undegraded":"2023-09-27T23:26:56.828290+0000","last_fullsized":"2023-09-27T23:26:56.828290+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:20:39.274221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824218+0000","last_change":"2023-09-27T23:26:56.814262+0000","last_active":"2023-09-27T23:26:56.824218+0000","last_peered":"2023-09-27T23:26:56.824218+0000","last_clean":"2023-09-27T23:26:56.824218+0000","last_became_active":"2023-09-27T23:26:56.813928+0000","last_became_peered":"2023-09-27T23:26:56.813928+0000","last_unstale":"2023-09-27T23:26:56.824218+0000","last_undegraded":"2023-09-27T23:26:56.824218+0000","last_fullsized":"2023-09-27T23:26:56.824218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:49.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823715+0000","last_change":"2023-09-27T23:26:56.429097+0000","last_active":"2023-09-27T23:26:56.823715+0000","last_peered":"2023-09-27T23:26:56.823715+0000","last_clean":"2023-09-27T23:26:56.823715+0000","last_became_active":"2023-09-27T23:26:56.428639+0000","last_became_peered":"2023-09-27T23:26:56.428639+0000","last_unstale":"2023-09-27T23:26:56.823715+0000","last_undegraded":"2023-09-27T23:26:56.823715+0000","last_fullsized":"2023-09-27T23:26:56.823715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:10:29.314492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831569+0000","last_change":"2023-09-27T23:26:56.822307+0000","last_active":"2023-09-27T23:26:56.831569+0000","last_peered":"2023-09-27T23:26:56.831569+0000","last_clean":"2023-09-27T23:26:56.831569+0000","last_became_active":"2023-09-27T23:26:56.821304+0000","last_became_peered":"2023-09-27T23:26:56.821304+0000","last_unstale":"2023-09-27T23:26:56.831569+0000","last_undegraded":"2023-09-27T23:26:56.831569+0000","last_fullsized":"2023-09-27T23:26:56.831569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:56.689414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828509+0000","last_change":"2023-09-27T23:26:56.436484+0000","last_active":"2023-09-27T23:26:56.828509+0000","last_peered":"2023-09-27T23:26:56.828509+0000","last_clean":"2023-09-27T23:26:56.828509+0000","last_became_active":"2023-09-27T23:26:56.436220+0000","last_became_peered":"2023-09-27T23:26:56.436220+0000","last_unstale":"2023-09-27T23:26:56.828509+0000","last_undegraded":"2023-09-27T23:26:56.828509+0000","last_fullsized":"2023-09-27T23:26:56.828509+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:38.217423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828010+0000","last_change":"2023-09-27T23:26:56.816908+0000","last_active":"2023-09-27T23:26:56.828010+0000","last_peered":"2023-09-27T23:26:56.828010+0000","last_clean":"2023-09-27T23:26:56.828010+0000","last_became_active":"2023-09-27T23:26:56.813414+0000","last_became_peered":"2023-09-27T23:26:56.813414+0000","last_unstale":"2023-09-27T23:26:56.828010+0000","last_undegraded":"2023-09-27T23:26:56.828010+0000","last_fullsized":"2023-09-27T23:26:56.828010+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:41:15.113008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823492+0000","last_change":"2023-09-27T23:26:56.425814+0000","last_active":"2023-09-27T23:26:56.823492+0000","last_peered":"2023-09-27T23:26:56.823492+0000","last_clean":"2023-09-27T23:26:56.823492+0000","last_became_active":"2023-09-27T23:26:56.423551+0000","last_became_peered":"2023-09-27T23:26:56.423551+0000","last_unstale":"2023-09-27T23:26:56.823492+0000","last_undegraded":"2023-09-27T23:26:56.823492+0000","last_fullsized":"2023-09-27T23:26:56.823492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:40.797293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.825718+0000","last_change":"2023-09-27T23:26:56.821293+0000","last_active":"2023-09-27T23:26:56.825718+0000","last_peered":"2023-09-27T23:26:56.825718+0000","last_clean":"2023-09-27T23:26:56.825718+0000","last_became_active":"2023-09-27T23:26:56.821019+0000","last_became_peered":"2023-09-27T23:26:56.821019+0000","last_unstale":"2023-09-27T23:26:56.825718+0000","last_undegraded":"2023-09-27T23:26:56.825718+0000","last_fullsized":"2023-09-27T23:26:56.825718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:10:10.206062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.808720+0000","last_change":"2023-09-27T23:26:56.426151+0000","last_active":"2023-09-27T23:26:56.808720+0000","last_peered":"2023-09-27T23:26:56.808720+0000","last_clean":"2023-09-27T23:26:56.808720+0000","last_became_active":"2023-09-27T23:26:56.425674+0000","last_became_peered":"2023-09-27T23:26:56.425674+0000","last_unstale":"2023-09-27T23:26:56.808720+0000","last_undegraded":"2023-09-27T23:26:56.808720+0000","last_fullsized":"2023-09-27T23:26:56.808720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:19.710475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831735+0000","last_change":"2023-09-27T23:26:56.828424+0000","last_active":"2023-09-27T23:26:56.831735+0000","last_peered":"2023-09-27T23:26:56.831735+0000","last_clean":"2023-09-27T23:26:56.831735+0000","last_became_active":"2023-09-27T23:26:56.828044+0000","last_became_peered":"2023-09-27T23:26:56.828044+0000","last_unstale":"2023-09-27T23:26:56.831735+0000","last_undegraded":"2023-09-27T23:26:56.831735+0000","last_fullsized":"2023-09-27T23:26:56.831735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:00:51.597357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827750+0000","last_change":"2023-09-27T23:26:56.436705+0000","last_active":"2023-09-27T23:26:56.827750+0000","last_peered":"2023-09-27T23:26:56.827750+0000","last_clean":"2023-09-27T23:26:56.827750+0000","last_became_active":"2023-09-27T23:26:56.436367+0000","last_became_peered":"2023-09-27T23:26:56.436367+0000","last_unstale":"2023-09-27T23:26:56.827750+0000","last_undegraded":"2023-09-27T23:26:56.827750+0000","last_fullsized":"2023-09-27T23:26:56.827750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:10:29.314492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824529+0000","last_change":"2023-09-27T23:26:56.823238+0000","last_active":"2023-09-27T23:26:56.824529+0000","last_peered":"2023-09-27T23:26:56.824529+0000","last_clean":"2023-09-27T23:26:56.824529+0000","last_became_active":"2023-09-27T23:26:56.822535+0000","last_became_peered":"2023-09-27T23:26:56.822535+0000","last_unstale":"2023-09-27T23:26:56.824529+0000","last_undegraded":"2023-09-27T23:26:56.824529+0000","last_fullsized":"2023-09-27T23:26:56.824529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:46.173544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796531+0000","last_change":"2023-09-27T23:26:56.426213+0000","last_active":"2023-09-27T23:26:56.796531+0000","last_peered":"2023-09-27T23:26:56.796531+0000","last_clean":"2023-09-27T23:26:56.796531+0000","last_became_active":"2023-09-27T23:26:56.425900+0000","last_became_peered":"2023-09-27T23:26:56.425900+0000","last_unstale":"2023-09-27T23:26:56.796531+0000","last_undegraded":"2023-09-27T23:26:56.796531+0000","last_fullsized":"2023-09-27T23:26:56.796531+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:38.377307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823517+0000","last_change":"2023-09-27T23:26:56.812947+0000","last_active":"2023-09-27T23:26:56.823517+0000","last_peered":"2023-09-27T23:26:56.823517+0000","last_clean":"2023-09-27T23:26:56.823517+0000","last_became_active":"2023-09-27T23:26:56.810262+0000","last_became_peered":"2023-09-27T23:26:56.810262+0000","last_unstale":"2023-09-27T23:26:56.823517+0000","last_undegraded":"2023-09-27T23:26:56.823517+0000","last_fullsized":"2023-09-27T23:26:56.823517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:34:23.130491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.803567+0000","last_change":"2023-09-27T23:26:56.431639+0000","last_active":"2023-09-27T23:26:56.803567+0000","last_peered":"2023-09-27T23:26:56.803567+0000","last_clean":"2023-09-27T23:26:56.803567+0000","last_became_active":"2023-09-27T23:26:56.431313+0000","last_became_peered":"2023-09-27T23:26:56.431313+0000","last_unstale":"2023-09-27T23:26:56.803567+0000","last_undegraded":"2023-09-27T23:26:56.803567+0000","last_fullsized":"2023-09-27T23:26:56.803567+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:06:14.449464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822470+0000","last_change":"2023-09-27T23:26:56.819969+0000","last_active":"2023-09-27T23:26:56.822470+0000","last_peered":"2023-09-27T23:26:56.822470+0000","last_clean":"2023-09-27T23:26:56.822470+0000","last_became_active":"2023-09-27T23:26:56.819674+0000","last_became_peered":"2023-09-27T23:26:56.819674+0000","last_unstale":"2023-09-27T23:26:56.822470+0000","last_undegraded":"2023-09-27T23:26:56.822470+0000","last_fullsized":"2023-09-27T23:26:56.822470+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:10:10.206062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.806649+0000","last_change":"2023-09-27T23:26:56.433705+0000","last_active":"2023-09-27T23:26:56.806649+0000","last_peered":"2023-09-27T23:26:56.806649+0000","last_clean":"2023-09-27T23:26:56.806649+0000","last_became_active":"2023-09-27T23:26:56.433345+0000","last_became_peered":"2023-09-27T23:26:56.433345+0000","last_unstale":"2023-09-27T23:26:56.806649+0000","last_undegraded":"2023-09-27T23:26:56.806649+0000","last_fullsized":"2023-09-27T23:26:56.806649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:40.797293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830278+0000","last_change":"2023-09-27T23:26:56.824751+0000","last_active":"2023-09-27T23:26:56.830278+0000","last_peered":"2023-09-27T23:26:56.830278+0000","last_clean":"2023-09-27T23:26:56.830278+0000","last_became_active":"2023-09-27T23:26:56.822551+0000","last_became_peered":"2023-09-27T23:26:56.822551+0000","last_unstale":"2023-09-27T23:26:56.830278+0000","last_undegraded":"2023-09-27T23:26:56.830278+0000","last_fullsized":"2023-09-27T23:26:56.830278+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:17:04.288794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.809813+0000","last_change":"2023-09-27T23:26:56.428912+0000","last_active":"2023-09-27T23:26:56.809813+0000","last_peered":"2023-09-27T23:26:56.809813+0000","last_clean":"2023-09-27T23:26:56.809813+0000","last_became_active":"2023-09-27T23:26:56.428319+0000","last_became_peered":"2023-09-27T23:26:56.428319+0000","last_unstale":"2023-09-27T23:26:56.809813+0000","last_undegraded":"2023-09-27T23:26:56.809813+0000","last_fullsized":"2023-09-27T23:26:56.809813+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:26.414570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824186+0000","last_change":"2023-09-27T23:26:56.816043+0000","last_active":"2023-09-27T23:26:56.824186+0000","last_peered":"2023-09-27T23:26:56.824186+0000","last_clean":"2023-09-27T23:26:56.824186+0000","last_became_active":"2023-09-27T23:26:56.815712+0000","last_became_peered":"2023-09-27T23:26:56.815712+0000","last_unstale":"2023-09-27T23:26:56.824186+0000","last_undegraded":"2023-09-27T23:26:56.824186+0000","last_fullsized":"2023-09-27T23:26:56.824186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:54.503767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"70'3","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.829079+0000","last_change":"2023-09-27T23:26:56.430635+0000","last_active":"2023-09-27T23:26:56.829079+0000","last_peered":"2023-09-27T23:26:56.829079+0000","last_clean":"2023-09-27T23:26:56.829079+0000","last_became_active":"2023-09-27T23:26:56.430216+0000","last_became_peered":"2023-09-27T23:26:56.430216+0000","last_unstale":"2023-09-27T23:26:56.829079+0000","last_undegraded":"2023-09-27T23:26:56.829079+0000","last_fullsized":"2023-09-27T23:26:56.829079+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:45.862631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830558+0000","last_change":"2023-09-27T23:26:56.825066+0000","last_active":"2023-09-27T23:26:56.830558+0000","last_peered":"2023-09-27T23:26:56.830558+0000","last_clean":"2023-09-27T23:26:56.830558+0000","last_became_active":"2023-09-27T23:26:56.824694+0000","last_became_peered":"2023-09-27T23:26:56.824694+0000","last_unstale":"2023-09-27T23:26:56.830558+0000","last_undegraded":"2023-09-27T23:26:56.830558+0000","last_fullsized":"2023-09-27T23:26:56.830558+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:57:09.552899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"70'6","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.833685+0000","last_change":"2023-09-27T23:26:56.438412+0000","last_active":"2023-09-27T23:26:56.833685+0000","last_peered":"2023-09-27T23:26:56.833685+0000","last_clean":"2023-09-27T23:26:56.833685+0000","last_became_active":"2023-09-27T23:26:56.438097+0000","last_became_peered":"2023-09-27T23:26:56.438097+0000","last_unstale":"2023-09-27T23:26:56.833685+0000","last_undegraded":"2023-09-27T23:26:56.833685+0000","last_fullsized":"2023-09-27T23:26:56.833685+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:59:09.616338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.821231+0000","last_change":"2023-09-27T23:26:56.818999+0000","last_active":"2023-09-27T23:26:56.821231+0000","last_peered":"2023-09-27T23:26:56.821231+0000","last_clean":"2023-09-27T23:26:56.821231+0000","last_became_active":"2023-09-27T23:26:56.818507+0000","last_became_peered":"2023-09-27T23:26:56.818507+0000","last_unstale":"2023-09-27T23:26:56.821231+0000","last_undegraded":"2023-09-27T23:26:56.821231+0000","last_fullsized":"2023-09-27T23:26:56.821231+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:39.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823136+0000","last_change":"2023-09-27T23:26:56.429881+0000","last_active":"2023-09-27T23:26:56.823136+0000","last_peered":"2023-09-27T23:26:56.823136+0000","last_clean":"2023-09-27T23:26:56.823136+0000","last_became_active":"2023-09-27T23:26:56.429578+0000","last_became_peered":"2023-09-27T23:26:56.429578+0000","last_unstale":"2023-09-27T23:26:56.823136+0000","last_undegraded":"2023-09-27T23:26:56.823136+0000","last_fullsized":"2023-09-27T23:26:56.823136+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:38.217423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828858+0000","last_change":"2023-09-27T23:26:56.818767+0000","last_active":"2023-09-27T23:26:56.828858+0000","last_peered":"2023-09-27T23:26:56.828858+0000","last_clean":"2023-09-27T23:26:56.828858+0000","last_became_active":"2023-09-27T23:26:56.818428+0000","last_became_peered":"2023-09-27T23:26:56.818428+0000","last_unstale":"2023-09-27T23:26:56.828858+0000","last_undegraded":"2023-09-27T23:26:56.828858+0000","last_fullsized":"2023-09-27T23:26:56.828858+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:27.419083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798785+0000","last_change":"2023-09-27T23:26:56.424428+0000","last_active":"2023-09-27T23:26:56.798785+0000","last_peered":"2023-09-27T23:26:56.798785+0000","last_clean":"2023-09-27T23:26:56.798785+0000","last_became_active":"2023-09-27T23:26:56.424118+0000","last_became_peered":"2023-09-27T23:26:56.424118+0000","last_unstale":"2023-09-27T23:26:56.798785+0000","last_undegraded":"2023-09-27T23:26:56.798785+0000","last_fullsized":"2023-09-27T23:26:56.798785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:18:10.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.832373+0000","last_change":"2023-09-27T23:26:56.829945+0000","last_active":"2023-09-27T23:26:56.832373+0000","last_peered":"2023-09-27T23:26:56.832373+0000","last_clean":"2023-09-27T23:26:56.832373+0000","last_became_active":"2023-09-27T23:26:56.829652+0000","last_became_peered":"2023-09-27T23:26:56.829652+0000","last_unstale":"2023-09-27T23:26:56.832373+0000","last_undegraded":"2023-09-27T23:26:56.832373+0000","last_fullsized":"2023-09-27T23:26:56.832373+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:56.689414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796611+0000","last_change":"2023-09-27T23:26:56.425629+0000","last_active":"2023-09-27T23:26:56.796611+0000","last_peered":"2023-09-27T23:26:56.796611+0000","last_clean":"2023-09-27T23:26:56.796611+0000","last_became_active":"2023-09-27T23:26:56.423579+0000","last_became_peered":"2023-09-27T23:26:56.423579+0000","last_unstale":"2023-09-27T23:26:56.796611+0000","last_undegraded":"2023-09-27T23:26:56.796611+0000","last_fullsized":"2023-09-27T23:26:56.796611+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:37.066108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830694+0000","last_change":"2023-09-27T23:26:56.827492+0000","last_active":"2023-09-27T23:26:56.830694+0000","last_peered":"2023-09-27T23:26:56.830694+0000","last_clean":"2023-09-27T23:26:56.830694+0000","last_became_active":"2023-09-27T23:26:56.826988+0000","last_became_peered":"2023-09-27T23:26:56.826988+0000","last_unstale":"2023-09-27T23:26:56.830694+0000","last_undegraded":"2023-09-27T23:26:56.830694+0000","last_fullsized":"2023-09-27T23:26:56.830694+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:39.221936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"70'8","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828947+0000","last_change":"2023-09-27T23:26:56.429262+0000","last_active":"2023-09-27T23:26:56.828947+0000","last_peered":"2023-09-27T23:26:56.828947+0000","last_clean":"2023-09-27T23:26:56.828947+0000","last_became_active":"2023-09-27T23:26:56.428964+0000","last_became_peered":"2023-09-27T23:26:56.428964+0000","last_unstale":"2023-09-27T23:26:56.828947+0000","last_undegraded":"2023-09-27T23:26:56.828947+0000","last_fullsized":"2023-09-27T23:26:56.828947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:37.066108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810177+0000","last_change":"2023-09-27T23:26:56.803168+0000","last_active":"2023-09-27T23:26:56.810177+0000","last_peered":"2023-09-27T23:26:56.810177+0000","last_clean":"2023-09-27T23:26:56.810177+0000","last_became_active":"2023-09-27T23:26:56.802813+0000","last_became_peered":"2023-09-27T23:26:56.802813+0000","last_unstale":"2023-09-27T23:26:56.810177+0000","last_undegraded":"2023-09-27T23:26:56.810177+0000","last_fullsized":"2023-09-27T23:26:56.810177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:59.999313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817510+0000","last_change":"2023-09-27T23:26:56.433280+0000","last_active":"2023-09-27T23:26:56.817510+0000","last_peered":"2023-09-27T23:26:56.817510+0000","last_clean":"2023-09-27T23:26:56.817510+0000","last_became_active":"2023-09-27T23:26:56.432946+0000","last_became_peered":"2023-09-27T23:26:56.432946+0000","last_unstale":"2023-09-27T23:26:56.817510+0000","last_undegraded":"2023-09-27T23:26:56.817510+0000","last_fullsized":"2023-09-27T23:26:56.817510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:05:40.954906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816094+0000","last_change":"2023-09-27T23:26:56.813408+0000","last_active":"2023-09-27T23:26:56.816094+0000","last_peered":"2023-09-27T23:26:56.816094+0000","last_clean":"2023-09-27T23:26:56.816094+0000","last_became_active":"2023-09-27T23:26:56.813107+0000","last_became_peered":"2023-09-27T23:26:56.813107+0000","last_unstale":"2023-09-27T23:26:56.816094+0000","last_undegraded":"2023-09-27T23:26:56.816094+0000","last_fullsized":"2023-09-27T23:26:56.816094+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:44.053616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.812199+0000","last_change":"2023-09-27T23:26:56.431022+0000","last_active":"2023-09-27T23:26:56.812199+0000","last_peered":"2023-09-27T23:26:56.812199+0000","last_clean":"2023-09-27T23:26:56.812199+0000","last_became_active":"2023-09-27T23:26:56.430709+0000","last_became_peered":"2023-09-27T23:26:56.430709+0000","last_unstale":"2023-09-27T23:26:56.812199+0000","last_undegraded":"2023-09-27T23:26:56.812199+0000","last_fullsized":"2023-09-27T23:26:56.812199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:38.982539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824360+0000","last_change":"2023-09-27T23:26:56.815852+0000","last_active":"2023-09-27T23:26:56.824360+0000","last_peered":"2023-09-27T23:26:56.824360+0000","last_clean":"2023-09-27T23:26:56.824360+0000","last_became_active":"2023-09-27T23:26:56.815565+0000","last_became_peered":"2023-09-27T23:26:56.815565+0000","last_unstale":"2023-09-27T23:26:56.824360+0000","last_undegraded":"2023-09-27T23:26:56.824360+0000","last_fullsized":"2023-09-27T23:26:56.824360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:21:59.389594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798790+0000","last_change":"2023-09-27T23:26:56.425304+0000","last_active":"2023-09-27T23:26:56.798790+0000","last_peered":"2023-09-27T23:26:56.798790+0000","last_clean":"2023-09-27T23:26:56.798790+0000","last_became_active":"2023-09-27T23:26:56.425031+0000","last_became_peered":"2023-09-27T23:26:56.425031+0000","last_unstale":"2023-09-27T23:26:56.798790+0000","last_undegraded":"2023-09-27T23:26:56.798790+0000","last_fullsized":"2023-09-27T23:26:56.798790+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:45.293801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828029+0000","last_change":"2023-09-27T23:26:56.817529+0000","last_active":"2023-09-27T23:26:56.828029+0000","last_peered":"2023-09-27T23:26:56.828029+0000","last_clean":"2023-09-27T23:26:56.828029+0000","last_became_active":"2023-09-27T23:26:56.817162+0000","last_became_peered":"2023-09-27T23:26:56.817162+0000","last_unstale":"2023-09-27T23:26:56.828029+0000","last_undegraded":"2023-09-27T23:26:56.828029+0000","last_fullsized":"2023-09-27T23:26:56.828029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:07.457954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799350+0000","last_change":"2023-09-27T23:26:56.425530+0000","last_active":"2023-09-27T23:26:56.799350+0000","last_peered":"2023-09-27T23:26:56.799350+0000","last_clean":"2023-09-27T23:26:56.799350+0000","last_became_active":"2023-09-27T23:26:56.425247+0000","last_became_peered":"2023-09-27T23:26:56.425247+0000","last_unstale":"2023-09-27T23:26:56.799350+0000","last_undegraded":"2023-09-27T23:26:56.799350+0000","last_fullsized":"2023-09-27T23:26:56.799350+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:08:59.880116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822329+0000","last_change":"2023-09-27T23:26:56.819129+0000","last_active":"2023-09-27T23:26:56.822329+0000","last_peered":"2023-09-27T23:26:56.822329+0000","last_clean":"2023-09-27T23:26:56.822329+0000","last_became_active":"2023-09-27T23:26:56.818825+0000","last_became_peered":"2023-09-27T23:26:56.818825+0000","last_unstale":"2023-09-27T23:26:56.822329+0000","last_undegraded":"2023-09-27T23:26:56.822329+0000","last_fullsized":"2023-09-27T23:26:56.822329+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:35:01.738648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799681+0000","last_change":"2023-09-27T23:26:56.431567+0000","last_active":"2023-09-27T23:26:56.799681+0000","last_peered":"2023-09-27T23:26:56.799681+0000","last_clean":"2023-09-27T23:26:56.799681+0000","last_became_active":"2023-09-27T23:26:56.431206+0000","last_became_peered":"2023-09-27T23:26:56.431206+0000","last_unstale":"2023-09-27T23:26:56.799681+0000","last_undegraded":"2023-09-27T23:26:56.799681+0000","last_fullsized":"2023-09-27T23:26:56.799681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:52:27.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830622+0000","last_change":"2023-09-27T23:26:56.825789+0000","last_active":"2023-09-27T23:26:56.830622+0000","last_peered":"2023-09-27T23:26:56.830622+0000","last_clean":"2023-09-27T23:26:56.830622+0000","last_became_active":"2023-09-27T23:26:56.825516+0000","last_became_peered":"2023-09-27T23:26:56.825516+0000","last_unstale":"2023-09-27T23:26:56.830622+0000","last_undegraded":"2023-09-27T23:26:56.830622+0000","last_fullsized":"2023-09-27T23:26:56.830622+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:10:30.319005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.804419+0000","last_change":"2023-09-27T23:26:56.436059+0000","last_active":"2023-09-27T23:26:56.804419+0000","last_peered":"2023-09-27T23:26:56.804419+0000","last_clean":"2023-09-27T23:26:56.804419+0000","last_became_active":"2023-09-27T23:26:56.435772+0000","last_became_peered":"2023-09-27T23:26:56.435772+0000","last_unstale":"2023-09-27T23:26:56.804419+0000","last_undegraded":"2023-09-27T23:26:56.804419+0000","last_fullsized":"2023-09-27T23:26:56.804419+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:52.872997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824213+0000","last_change":"2023-09-27T23:26:56.811986+0000","last_active":"2023-09-27T23:26:56.824213+0000","last_peered":"2023-09-27T23:26:56.824213+0000","last_clean":"2023-09-27T23:26:56.824213+0000","last_became_active":"2023-09-27T23:26:56.811704+0000","last_became_peered":"2023-09-27T23:26:56.811704+0000","last_unstale":"2023-09-27T23:26:56.824213+0000","last_undegraded":"2023-09-27T23:26:56.824213+0000","last_fullsized":"2023-09-27T23:26:56.824213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:10:10.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.797203+0000","last_change":"2023-09-27T23:26:56.439391+0000","last_active":"2023-09-27T23:26:56.797203+0000","last_peered":"2023-09-27T23:26:56.797203+0000","last_clean":"2023-09-27T23:26:56.797203+0000","last_became_active":"2023-09-27T23:26:56.439021+0000","last_became_peered":"2023-09-27T23:26:56.439021+0000","last_unstale":"2023-09-27T23:26:56.797203+0000","last_undegraded":"2023-09-27T23:26:56.797203+0000","last_fullsized":"2023-09-27T23:26:56.797203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:59.110216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823537+0000","last_change":"2023-09-27T23:26:56.813169+0000","last_active":"2023-09-27T23:26:56.823537+0000","last_peered":"2023-09-27T23:26:56.823537+0000","last_clean":"2023-09-27T23:26:56.823537+0000","last_became_active":"2023-09-27T23:26:56.810577+0000","last_became_peered":"2023-09-27T23:26:56.810577+0000","last_unstale":"2023-09-27T23:26:56.823537+0000","last_undegraded":"2023-09-27T23:26:56.823537+0000","last_fullsized":"2023-09-27T23:26:56.823537+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:56.689414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.804070+0000","last_change":"2023-09-27T23:26:56.431077+0000","last_active":"2023-09-27T23:26:56.804070+0000","last_peered":"2023-09-27T23:26:56.804070+0000","last_clean":"2023-09-27T23:26:56.804070+0000","last_became_active":"2023-09-27T23:26:56.429844+0000","last_became_peered":"2023-09-27T23:26:56.429844+0000","last_unstale":"2023-09-27T23:26:56.804070+0000","last_undegraded":"2023-09-27T23:26:56.804070+0000","last_fullsized":"2023-09-27T23:26:56.804070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:38.982539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822074+0000","last_change":"2023-09-27T23:26:56.813632+0000","last_active":"2023-09-27T23:26:56.822074+0000","last_peered":"2023-09-27T23:26:56.822074+0000","last_clean":"2023-09-27T23:26:56.822074+0000","last_became_active":"2023-09-27T23:26:56.813352+0000","last_became_peered":"2023-09-27T23:26:56.813352+0000","last_unstale":"2023-09-27T23:26:56.822074+0000","last_undegraded":"2023-09-27T23:26:56.822074+0000","last_fullsized":"2023-09-27T23:26:56.822074+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:29:26.826938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798184+0000","last_change":"2023-09-27T23:26:56.429406+0000","last_active":"2023-09-27T23:26:56.798184+0000","last_peered":"2023-09-27T23:26:56.798184+0000","last_clean":"2023-09-27T23:26:56.798184+0000","last_became_active":"2023-09-27T23:26:56.429125+0000","last_became_peered":"2023-09-27T23:26:56.429125+0000","last_unstale":"2023-09-27T23:26:56.798184+0000","last_undegraded":"2023-09-27T23:26:56.798184+0000","last_fullsized":"2023-09-27T23:26:56.798184+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:40.797293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":289,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818096+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:26:56.818096+0000","last_peered":"2023-09-27T23:26:56.818096+0000","last_clean":"2023-09-27T23:26:56.818096+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:26:56.818096+0000","last_undegraded":"2023-09-27T23:26:56.818096+0000","last_fullsized":"2023-09-27T23:26:56.818096+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":139,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798953+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:26:56.798953+0000","last_peered":"2023-09-27T23:26:56.798953+0000","last_clean":"2023-09-27T23:26:56.798953+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:26:56.798953+0000","last_undegraded":"2023-09-27T23:26:56.798953+0000","last_fullsized":"2023-09-27T23:26:56.798953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831510+0000","last_change":"2023-09-27T23:26:56.822811+0000","last_active":"2023-09-27T23:26:56.831510+0000","last_peered":"2023-09-27T23:26:56.831510+0000","last_clean":"2023-09-27T23:26:56.831510+0000","last_became_active":"2023-09-27T23:26:56.822521+0000","last_became_peered":"2023-09-27T23:26:56.822521+0000","last_unstale":"2023-09-27T23:26:56.831510+0000","last_undegraded":"2023-09-27T23:26:56.831510+0000","last_fullsized":"2023-09-27T23:26:56.831510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:26:04.894120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798523+0000","last_change":"2023-09-27T23:26:56.428852+0000","last_active":"2023-09-27T23:26:56.798523+0000","last_peered":"2023-09-27T23:26:56.798523+0000","last_clean":"2023-09-27T23:26:56.798523+0000","last_became_active":"2023-09-27T23:26:56.428486+0000","last_became_peered":"2023-09-27T23:26:56.428486+0000","last_unstale":"2023-09-27T23:26:56.798523+0000","last_undegraded":"2023-09-27T23:26:56.798523+0000","last_fullsized":"2023-09-27T23:26:56.798523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:23:54.572803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817957+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:26:56.817957+0000","last_peered":"2023-09-27T23:26:56.817957+0000","last_clean":"2023-09-27T23:26:56.817957+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:26:56.817957+0000","last_undegraded":"2023-09-27T23:26:56.817957+0000","last_fullsized":"2023-09-27T23:26:56.817957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816256+0000","last_change":"2023-09-27T23:26:56.813832+0000","last_active":"2023-09-27T23:26:56.816256+0000","last_peered":"2023-09-27T23:26:56.816256+0000","last_clean":"2023-09-27T23:26:56.816256+0000","last_became_active":"2023-09-27T23:26:56.813282+0000","last_became_peered":"2023-09-27T23:26:56.813282+0000","last_unstale":"2023-09-27T23:26:56.816256+0000","last_undegraded":"2023-09-27T23:26:56.816256+0000","last_fullsized":"2023-09-27T23:26:56.816256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:39.221936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823181+0000","last_change":"2023-09-27T23:26:56.428421+0000","last_active":"2023-09-27T23:26:56.823181+0000","last_peered":"2023-09-27T23:26:56.823181+0000","last_clean":"2023-09-27T23:26:56.823181+0000","last_became_active":"2023-09-27T23:26:56.428110+0000","last_became_peered":"2023-09-27T23:26:56.428110+0000","last_unstale":"2023-09-27T23:26:56.823181+0000","last_undegraded":"2023-09-27T23:26:56.823181+0000","last_fullsized":"2023-09-27T23:26:56.823181+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:17:03.284281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.787944+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:26:56.787944+0000","last_peered":"2023-09-27T23:26:56.787944+0000","last_clean":"2023-09-27T23:26:56.787944+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:26:56.787944+0000","last_undegraded":"2023-09-27T23:26:56.787944+0000","last_fullsized":"2023-09-27T23:26:56.787944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827987+0000","last_change":"2023-09-27T23:26:56.818239+0000","last_active":"2023-09-27T23:26:56.827987+0000","last_peered":"2023-09-27T23:26:56.827987+0000","last_clean":"2023-09-27T23:26:56.827987+0000","last_became_active":"2023-09-27T23:26:56.817954+0000","last_became_peered":"2023-09-27T23:26:56.817954+0000","last_unstale":"2023-09-27T23:26:56.827987+0000","last_undegraded":"2023-09-27T23:26:56.827987+0000","last_fullsized":"2023-09-27T23:26:56.827987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:57:30.548112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796489+0000","last_change":"2023-09-27T23:26:56.434807+0000","last_active":"2023-09-27T23:26:56.796489+0000","last_peered":"2023-09-27T23:26:56.796489+0000","last_clean":"2023-09-27T23:26:56.796489+0000","last_became_active":"2023-09-27T23:26:56.434527+0000","last_became_peered":"2023-09-27T23:26:56.434527+0000","last_unstale":"2023-09-27T23:26:56.796489+0000","last_undegraded":"2023-09-27T23:26:56.796489+0000","last_fullsized":"2023-09-27T23:26:56.796489+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:18:10.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796677+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:26:56.796677+0000","last_peered":"2023-09-27T23:26:56.796677+0000","last_clean":"2023-09-27T23:26:56.796677+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:26:56.796677+0000","last_undegraded":"2023-09-27T23:26:56.796677+0000","last_fullsized":"2023-09-27T23:26:56.796677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824711+0000","last_change":"2023-09-27T23:26:56.819491+0000","last_active":"2023-09-27T23:26:56.824711+0000","last_peered":"2023-09-27T23:26:56.824711+0000","last_clean":"2023-09-27T23:26:56.824711+0000","last_became_active":"2023-09-27T23:26:56.819213+0000","last_became_peered":"2023-09-27T23:26:56.819213+0000","last_unstale":"2023-09-27T23:26:56.824711+0000","last_undegraded":"2023-09-27T23:26:56.824711+0000","last_fullsized":"2023-09-27T23:26:56.824711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:27.419083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827945+0000","last_change":"2023-09-27T23:26:56.434974+0000","last_active":"2023-09-27T23:26:56.827945+0000","last_peered":"2023-09-27T23:26:56.827945+0000","last_clean":"2023-09-27T23:26:56.827945+0000","last_became_active":"2023-09-27T23:26:56.434674+0000","last_became_peered":"2023-09-27T23:26:56.434674+0000","last_unstale":"2023-09-27T23:26:56.827945+0000","last_undegraded":"2023-09-27T23:26:56.827945+0000","last_fullsized":"2023-09-27T23:26:56.827945+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:43.049103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799914+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:26:56.799914+0000","last_peered":"2023-09-27T23:26:56.799914+0000","last_clean":"2023-09-27T23:26:56.799914+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:26:56.799914+0000","last_undegraded":"2023-09-27T23:26:56.799914+0000","last_fullsized":"2023-09-27T23:26:56.799914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828995+0000","last_change":"2023-09-27T23:26:56.824329+0000","last_active":"2023-09-27T23:26:56.828995+0000","last_peered":"2023-09-27T23:26:56.828995+0000","last_clean":"2023-09-27T23:26:56.828995+0000","last_became_active":"2023-09-27T23:26:56.822836+0000","last_became_peered":"2023-09-27T23:26:56.822836+0000","last_unstale":"2023-09-27T23:26:56.828995+0000","last_undegraded":"2023-09-27T23:26:56.828995+0000","last_fullsized":"2023-09-27T23:26:56.828995+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:38.417239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823413+0000","last_change":"2023-09-27T23:26:56.430433+0000","last_active":"2023-09-27T23:26:56.823413+0000","last_peered":"2023-09-27T23:26:56.823413+0000","last_clean":"2023-09-27T23:26:56.823413+0000","last_became_active":"2023-09-27T23:26:56.430173+0000","last_became_peered":"2023-09-27T23:26:56.430173+0000","last_unstale":"2023-09-27T23:26:56.823413+0000","last_undegraded":"2023-09-27T23:26:56.823413+0000","last_fullsized":"2023-09-27T23:26:56.823413+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:26.780386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.813850+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:26:56.813850+0000","last_peered":"2023-09-27T23:26:56.813850+0000","last_clean":"2023-09-27T23:26:56.813850+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:26:56.813850+0000","last_undegraded":"2023-09-27T23:26:56.813850+0000","last_fullsized":"2023-09-27T23:26:56.813850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830579+0000","last_change":"2023-09-27T23:26:56.825211+0000","last_active":"2023-09-27T23:26:56.830579+0000","last_peered":"2023-09-27T23:26:56.830579+0000","last_clean":"2023-09-27T23:26:56.830579+0000","last_became_active":"2023-09-27T23:26:56.824718+0000","last_became_peered":"2023-09-27T23:26:56.824718+0000","last_unstale":"2023-09-27T23:26:56.830579+0000","last_undegraded":"2023-09-27T23:26:56.830579+0000","last_fullsized":"2023-09-27T23:26:56.830579+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:27.784899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.820537+0000","last_change":"2023-09-27T23:26:56.436836+0000","last_active":"2023-09-27T23:26:56.820537+0000","last_peered":"2023-09-27T23:26:56.820537+0000","last_clean":"2023-09-27T23:26:56.820537+0000","last_became_active":"2023-09-27T23:26:56.436287+0000","last_became_peered":"2023-09-27T23:26:56.436287+0000","last_unstale":"2023-09-27T23:26:56.820537+0000","last_undegraded":"2023-09-27T23:26:56.820537+0000","last_fullsized":"2023-09-27T23:26:56.820537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:26.414570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799800+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:26:56.799800+0000","last_peered":"2023-09-27T23:26:56.799800+0000","last_clean":"2023-09-27T23:26:56.799800+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:26:56.799800+0000","last_undegraded":"2023-09-27T23:26:56.799800+0000","last_fullsized":"2023-09-27T23:26:56.799800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.821988+0000","last_change":"2023-09-27T23:26:56.816231+0000","last_active":"2023-09-27T23:26:56.821988+0000","last_peered":"2023-09-27T23:26:56.821988+0000","last_clean":"2023-09-27T23:26:56.821988+0000","last_became_active":"2023-09-27T23:26:56.815931+0000","last_became_peered":"2023-09-27T23:26:56.815931+0000","last_unstale":"2023-09-27T23:26:56.821988+0000","last_undegraded":"2023-09-27T23:26:56.821988+0000","last_fullsized":"2023-09-27T23:26:56.821988+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:38.823773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.829082+0000","last_change":"2023-09-27T23:26:56.424937+0000","last_active":"2023-09-27T23:26:56.829082+0000","last_peered":"2023-09-27T23:26:56.829082+0000","last_clean":"2023-09-27T23:26:56.829082+0000","last_became_active":"2023-09-27T23:26:56.423527+0000","last_became_peered":"2023-09-27T23:26:56.423527+0000","last_unstale":"2023-09-27T23:26:56.829082+0000","last_undegraded":"2023-09-27T23:26:56.829082+0000","last_fullsized":"2023-09-27T23:26:56.829082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:45.293801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800752+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:26:56.800752+0000","last_peered":"2023-09-27T23:26:56.800752+0000","last_clean":"2023-09-27T23:26:56.800752+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:26:56.800752+0000","last_undegraded":"2023-09-27T23:26:56.800752+0000","last_fullsized":"2023-09-27T23:26:56.800752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509603,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24652,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":23232,"kb_avail":94347188,"statfs":{"total":96636764160,"available":96611520512,"internally_reserved":0,"allocated":1445888,"data_stored":1015142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23789568},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93999999999999995,"15min":0.92700000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2190000000000001,"5min":1.363,"15min":1.7949999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.92200000000000004,"15min":0.88900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89100000000000001,"15min":0.90000000000000002},"min":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93300000000000005,"15min":0.92100000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":1.048},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.99299999999999999,"15min":0.96999999999999997},"min":{"1min":0.68000000000000005,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94699999999999995,"15min":0.94999999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2410000000000001,"5min":1.29,"15min":1.421},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.0800000000000001,"15min":1.048},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.622},"max":{"1min":1.546,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.94899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.90700000000000003,"15min":0.91200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.093},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.99399999999999999,"15min":0.99099999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.179}]},{"osd":5,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0389999999999999,"15min":1.052},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.4139999999999999,"5min":1.51,"15min":1.5880000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.105,"15min":1.1000000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.2390000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1339999999999999,"15min":1.129},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.1659999999999999,"15min":1.1399999999999999},"min":{"1min":0.878,"5min":0.85199999999999998,"15min":0.73899999999999999},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.288}]}]},{"osd":6,"up_from":15,"seq":64424509602,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20416,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":19584,"kb_avail":94351424,"statfs":{"total":96636764160,"available":96615858176,"internally_reserved":0,"allocated":843776,"data_stored":424614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20054016},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.68899999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.327,"15min":1.327},"last":0.997},{"interface":"front","average":{"1min":1.079,"5min":1.026,"15min":1.0289999999999999},"min":{"1min":0.81999999999999995,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.649},"last":1.264}]},{"osd":1,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.54700000000000004},"max":{"1min":1.2889999999999999,"5min":1.323,"15min":1.323},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1739999999999999,"5min":1.2509999999999999,"15min":1.3939999999999999},"last":1.159}]},{"osd":2,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.88},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.56799999999999995},"max":{"1min":1.0820000000000001,"5min":1.353,"15min":1.397},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60499999999999998},"max":{"1min":1.2589999999999999,"5min":1.367,"15min":1.4259999999999999},"last":1.083}]},{"osd":3,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.58999999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.734},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.84599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"max":{"1min":1.2290000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.1160000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.93100000000000005},"min":{"1min":0.64900000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":1.292,"5min":1.335,"15min":1.544},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.61899999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.04}]},{"osd":5,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.24,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.204,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":1.0269999999999999,"5min":0.83999999999999997,"15min":0.65900000000000003},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.1830000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.167,"15min":1.165},"min":{"1min":0.92800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3660000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2370000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30408,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":29568,"kb_avail":94341432,"statfs":{"total":96636764160,"available":96605626368,"internally_reserved":0,"allocated":851968,"data_stored":425238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30277632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.151,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.2929999999999999,"15min":1.359},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.50700000000000001},"max":{"1min":1.2969999999999999,"5min":1.3109999999999999,"15min":1.526},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.88500000000000001,"15min":0.876},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.234,"15min":1.4179999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.93000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3859999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"max":{"1min":1.27,"5min":1.4730000000000001,"15min":1.5740000000000001},"last":0.628}]},{"osd":3,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91900000000000004,"15min":0.97199999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.214,"5min":1.238,"15min":1.5640000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.024,"15min":1.014},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3819999999999999,"15min":1.954},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1599999999999999,"5min":1.2829999999999999,"15min":1.5349999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.032,"15min":1.046},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.304,"5min":1.3939999999999999,"15min":1.704},"last":1.286}]},{"osd":6,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.02,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.323,"5min":1.5580000000000001,"15min":1.6319999999999999},"last":1},{"interface":"front","average":{"1min":1.046,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.3779999999999999,"15min":1.837},"last":1.1519999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1479999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3,"5min":1.496,"15min":1.7230000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.204,"15min":1.1950000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.4319999999999999,"5min":1.633,"15min":2.0529999999999999},"last":1.083}]}]},{"osd":4,"up_from":15,"seq":64424509603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20812,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94351028,"statfs":{"total":96636764160,"available":96615452672,"internally_reserved":0,"allocated":856064,"data_stored":426150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.91100000000000003,"15min":0.88700000000000001},"min":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.53700000000000003},"max":{"1min":1.0529999999999999,"5min":1.27,"15min":1.3080000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.3320000000000001,"5min":1.45,"15min":1.45},"last":1.028}]},{"osd":1,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.181,"5min":1.2190000000000001,"15min":1.3919999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.95999999999999996,"15min":0.92400000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.139,"5min":1.2050000000000001,"15min":1.329},"last":0.84299999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91600000000000004,"15min":0.90900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93400000000000005,"15min":0.97099999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.153,"5min":1.3660000000000001,"15min":1.522},"last":1.25}]},{"osd":3,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92300000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0449999999999999,"5min":1.26,"15min":1.373},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.02,"15min":1.0129999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2989999999999999,"5min":1.5129999999999999,"15min":1.575},"last":0.96399999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.92400000000000004,"15min":0.92000000000000004},"min":{"1min":0.67400000000000004,"5min":0.60199999999999998,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.409,"15min":1.627},"last":1.355},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99199999999999999,"15min":1.016},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4019999999999999,"5min":1.5489999999999999,"15min":2.2730000000000001},"last":1.2090000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.054,"15min":1.0489999999999999},"min":{"1min":0.752,"5min":0.63900000000000001,"15min":0.60099999999999998},"max":{"1min":1.369,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.153,"15min":1.1579999999999999},"min":{"1min":0.83399999999999996,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.415,"5min":1.478,"15min":1.8620000000000001},"last":1.5169999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1000000000000001,"15min":1.093},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.64000000000000001},"max":{"1min":1.3280000000000001,"5min":1.627,"15min":1.726},"last":1.282},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1659999999999999,"15min":1.1719999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4319999999999999,"5min":1.859,"15min":2.532},"last":1.323}]}]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21448,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94350392,"statfs":{"total":96636764160,"available":96614801408,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.30199999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.746},"min":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.35299999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.337},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.39800000000000002},"max":{"1min":0.95299999999999996,"5min":1.224,"15min":1.2869999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.72199999999999998,"5min":0.61699999999999999,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.3180000000000001,"15min":1.3939999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.88300000000000001,"15min":0.85699999999999998},"min":{"1min":0.78200000000000003,"5min":0.65700000000000003,"15min":0.36699999999999999},"max":{"1min":1.1479999999999999,"5min":1.288,"15min":1.353},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91600000000000004},"min":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.45700000000000002},"max":{"1min":1.256,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.76000000000000001,"5min":0.67700000000000005,"15min":0.66100000000000003},"max":{"1min":1.175,"5min":1.466,"15min":1.5349999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.628},"max":{"1min":1.2030000000000001,"5min":1.3680000000000001,"15min":1.4950000000000001},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.91300000000000003,"5min":0.83099999999999996,"15min":0.626},"max":{"1min":1.3120000000000001,"5min":1.468,"15min":1.571},"last":1.087},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1479999999999999},"min":{"1min":0.96399999999999997,"5min":0.76700000000000002,"15min":0.58399999999999996},"max":{"1min":1.47,"5min":1.51,"15min":1.764},"last":1.05}]},{"osd":6,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.119,"15min":1.137},"min":{"1min":0.92600000000000005,"5min":0.76900000000000002,"15min":0.72799999999999998},"max":{"1min":1.407,"5min":1.4610000000000001,"15min":1.6970000000000001},"last":1.111},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":1.0269999999999999,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.3140000000000001,"5min":1.5800000000000001,"15min":1.8620000000000001},"last":1.2290000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.246,"15min":1.2290000000000001},"min":{"1min":1.0289999999999999,"5min":0.86799999999999999,"15min":0.78700000000000003},"max":{"1min":1.5640000000000001,"5min":1.627,"15min":1.655},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.2},"min":{"1min":1.054,"5min":0.89500000000000002,"15min":0.73099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.671},"last":1.0660000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20040,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":19200,"kb_avail":94351800,"statfs":{"total":96636764160,"available":96616243200,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19660800},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.68700000000000006},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":1.133,"15min":1.7370000000000001},"last":1.093},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.214,"15min":1.6080000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.79500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":1.1950000000000001,"15min":1.8069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.82799999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.3120000000000001,"15min":1.651},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.4850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.58099999999999996},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.8280000000000001},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.751,"5min":0.751,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":1.585,"15min":1.7849999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0629999999999999,"15min":1.097},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69499999999999995},"max":{"1min":1.337,"5min":1.4590000000000001,"15min":2.601},"last":0.91900000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.024,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.77400000000000002,"15min":0.71099999999999997},"max":{"1min":1.3029999999999999,"5min":1.403,"15min":2.7320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.107,"5min":1.1299999999999999,"15min":1.1559999999999999},"min":{"1min":0.84999999999999998,"5min":0.76700000000000002,"15min":0.747},"max":{"1min":1.375,"5min":1.496,"15min":2.6760000000000002},"last":1.5149999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1579999999999999,"15min":1.1910000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6560000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1559999999999999,"15min":1.1779999999999999},"min":{"1min":0.871,"5min":0.84099999999999997,"15min":0.71699999999999997},"max":{"1min":1.417,"5min":1.552,"15min":2.6230000000000002},"last":1.377}]},{"osd":7,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.218,"15min":1.2490000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.76600000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80000000000000004},"max":{"1min":1.3999999999999999,"5min":1.6200000000000001,"15min":2.7749999999999999},"last":1.282}]}]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35140,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":34304,"kb_avail":94336700,"statfs":{"total":96636764160,"available":96600780800,"internally_reserved":0,"allocated":847872,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35127296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.503,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.3560000000000001,"15min":3.798},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":1.151,"15min":2.3330000000000002},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.27500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":6.157},"last":1.179},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.879,"15min":0.97499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":5.649},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1919999999999999,"5min":1.323,"15min":6.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.294,"15min":1.9370000000000001},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.173,"5min":1.3340000000000001,"15min":1.863},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.133,"15min":1.1579999999999999},"min":{"1min":0.749,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.381,"5min":1.579,"15min":4.0499999999999998},"last":1.294}]},{"osd":5,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.65100000000000002},"max":{"1min":1.476,"5min":1.524,"15min":2.7040000000000002},"last":0.999},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.0960000000000001},"min":{"1min":0.80900000000000005,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.365,"5min":1.53,"15min":2.2509999999999999},"last":1.365}]},{"osd":6,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.234,"15min":1.2470000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.587,"5min":1.6919999999999999,"15min":3.9260000000000002},"last":1.22},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1910000000000001,"15min":1.1890000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.76400000000000001},"max":{"1min":1.556,"5min":1.5900000000000001,"15min":2.79},"last":1.127}]},{"osd":7,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.218},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.64800000000000002},"max":{"1min":1.532,"5min":1.7210000000000001,"15min":2.843},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.429,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.429}]}]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21588,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":20160,"kb_avail":94350252,"statfs":{"total":96636764160,"available":96614658048,"internally_reserved":0,"allocated":1454080,"data_stored":1017072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20643840},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65600000000000003,"15min":0.67300000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.002,"5min":1.002,"15min":1.901},"last":1.002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79300000000000004,"15min":0.78600000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2789999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.2150000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.335},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.84999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.216,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88800000000000001,"15min":0.86799999999999999},"min":{"1min":0.66300000000000003,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.089,"5min":1.282,"15min":1.3400000000000001},"last":0.77600000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.995,"15min":1.026},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.59199999999999997},"max":{"1min":1.2549999999999999,"5min":1.4590000000000001,"15min":2.2519999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0580000000000001},"min":{"1min":0.754,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.276,"5min":1.5589999999999999,"15min":1.952},"last":0.89900000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1020000000000001,"15min":1.0960000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.71899999999999997},"max":{"1min":1.5009999999999999,"5min":1.78,"15min":1.986},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.137,"15min":1.147},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.5509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.163,"15min":1.163},"min":{"1min":0.90800000000000003,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.3979999999999999,"5min":1.4750000000000001,"15min":1.7729999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.224,"15min":1.2110000000000001},"min":{"1min":0.93300000000000005,"5min":0.90800000000000003,"15min":0.61199999999999999},"max":{"1min":1.73,"5min":1.8300000000000001,"15min":2.1850000000000001},"last":1.373}]},{"osd":7,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.224,"15min":1.232},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.222,"15min":1.234},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.66400000000000003},"max":{"1min":1.554,"5min":1.554,"15min":2.3540000000000001},"last":1.4490000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:27:02.644 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:27:02.643+0000 7feabada8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:27:05.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:27:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 Processor -- start 2023-09-27T23:27:05.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 -- start start 2023-09-27T23:27:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e3406dce0 0x7f8e3406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8e340ce720 0x7f8e3415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3406d640 con 0x7f8e3406c9f0 2023-09-27T23:27:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e340cb500 con 0x7f8e3406dce0 2023-09-27T23:27:05.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e38a4a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e340cee70 con 0x7f8e340ce720 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e3406dce0 0x7f8e3406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e3406dce0 0x7f8e3406cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52708/0 (socket says 172.21.15.64:52708) 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 -- 172.21.15.64:0/459386254 learned_addr learned my addr 172.21.15.64:0/459386254 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e33fff700 1 --2- 172.21.15.64:0/459386254 >> v2:172.21.15.123:3300/0 conn(0x7f8e340ce720 0x7f8e3415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 -- 172.21.15.64:0/459386254 >> v2:172.21.15.123:3300/0 conn(0x7f8e340ce720 msgr2=0x7f8e3415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 --2- 172.21.15.64:0/459386254 >> v2:172.21.15.123:3300/0 conn(0x7f8e340ce720 0x7f8e3415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 -- 172.21.15.64:0/459386254 >> v1:172.21.15.123:6789/0 conn(0x7f8e3406c9f0 legacy=0x7f8e340cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 -- 172.21.15.64:0/459386254 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340ca950 con 0x7f8e3406dce0 2023-09-27T23:27:05.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e32ffd700 1 --2- 172.21.15.64:0/459386254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e3406dce0 0x7f8e3406cdc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8e3406dc10 tx=0x7f8e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee0ab7d04b484d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e327fc700 1 -- 172.21.15.64:0/459386254 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800b860 con 0x7f8e3406dce0 2023-09-27T23:27:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e327fc700 1 -- 172.21.15.64:0/459386254 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800b9c0 con 0x7f8e3406dce0 2023-09-27T23:27:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.327+0000 7f8e327fc700 1 -- 172.21.15.64:0/459386254 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800bd60 con 0x7f8e3406dce0 2023-09-27T23:27:05.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 -- 172.21.15.64:0/459386254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e3406dce0 msgr2=0x7f8e3406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/459386254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e3406dce0 0x7f8e3406cdc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8e3406dc10 tx=0x7f8e28009250 comp rx=0 tx=0).stop 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 -- 172.21.15.64:0/459386254 shutdown_connections 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/459386254 >> v2:172.21.15.123:3300/0 conn(0x7f8e340ce720 0x7f8e3415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/459386254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e3406dce0 0x7f8e3406cdc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 -- 172.21.15.64:0/459386254 >> 172.21.15.64:0/459386254 conn(0x7f8e34067bb0 msgr2=0x7f8e340cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 -- 172.21.15.64:0/459386254 shutdown_connections 2023-09-27T23:27:05.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.331+0000 7f8e38a4a700 1 -- 172.21.15.64:0/459386254 wait complete. 2023-09-27T23:27:05.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 Processor -- start 2023-09-27T23:27:05.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 -- start start 2023-09-27T23:27:05.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e3406c9f0 0x7f8e340c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3406dce0 0x7f8e34087e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 0x7f8e34089440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340872c0 con 0x7f8e340ce720 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340c6e70 con 0x7f8e3406dce0 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e38a4a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340c70c0 con 0x7f8e3406c9f0 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 0x7f8e34089440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e337fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e3406c9f0 0x7f8e340c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 0x7f8e34089440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52724/0 (socket says 172.21.15.64:52724) 2023-09-27T23:27:05.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 -- 172.21.15.64:0/4100805040 learned_addr learned my addr 172.21.15.64:0/4100805040 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e32ffd700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3406dce0 0x7f8e34087e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 -- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e3406c9f0 msgr2=0x7f8e340c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e3406c9f0 0x7f8e340c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 -- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3406dce0 msgr2=0x7f8e34087e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3406dce0 0x7f8e34087e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 -- 172.21.15.64:0/4100805040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280094f0 con 0x7f8e340ce720 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e33fff700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 0x7f8e34089440 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c007f20 tx=0x7f8e2c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00eb90 con 0x7f8e340ce720 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.335+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00ed10 con 0x7f8e340ce720 2023-09-27T23:27:05.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e340c73a0 con 0x7f8e340ce720 2023-09-27T23:27:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c014670 con 0x7f8e340ce720 2023-09-27T23:27:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e340c7c40 con 0x7f8e340ce720 2023-09-27T23:27:05.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3404be40 con 0x7f8e340ce720 2023-09-27T23:27:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8e2c0149e0 con 0x7f8e340ce720 2023-09-27T23:27:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e1b7fe700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8e20060a70 0x7f8e20062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8e2c07d780 con 0x7f8e340ce720 2023-09-27T23:27:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e337fe700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8e20060a70 0x7f8e20062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.339+0000 7f8e337fe700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8e20060a70 0x7f8e20062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8e24003a80 tx=0x7f8e24007ec0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:05.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.351+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8e2c04d790 con 0x7f8e340ce720 2023-09-27T23:27:05.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.507+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e340cabd0 con 0x7f8e20060a70 2023-09-27T23:27:05.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.527+0000 7f8e1b7fe700 1 -- 172.21.15.64:0/4100805040 <== mgr.5402 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307360 (secure 0 0 0) 0x7f8e340cabd0 con 0x7f8e20060a70 2023-09-27T23:27:05.533 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:05.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8e20060a70 msgr2=0x7f8e20062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8e20060a70 0x7f8e20062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8e24003a80 tx=0x7f8e24007ec0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 msgr2=0x7f8e34089440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 0x7f8e34089440 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c007f20 tx=0x7f8e2c009860 comp rx=0 tx=0).stop 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 shutdown_connections 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8e20060a70 0x7f8e20062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340ce720 0x7f8e34089440 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3406dce0 0x7f8e34087e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 --2- 172.21.15.64:0/4100805040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e3406c9f0 0x7f8e340c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 >> 172.21.15.64:0/4100805040 conn(0x7f8e34067bb0 msgr2=0x7f8e340c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 shutdown_connections 2023-09-27T23:27:05.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.531+0000 7f8e38a4a700 1 -- 172.21.15.64:0/4100805040 wait complete. 2023-09-27T23:27:05.537 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:27:05.550 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-09-27T23:27:03.766230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":194504,"kb_used_data":7816,"kb_used_omap":0,"kb_used_meta":186624,"kb_avail":754780216,"statfs":{"total":773094113280,"available":772894941184,"internally_reserved":0,"allocated":8003584,"data_stored":4582260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":191102976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.360468"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.814138+0000","last_change":"2023-09-27T23:26:56.811633+0000","last_active":"2023-09-27T23:26:56.814138+0000","last_peered":"2023-09-27T23:26:56.814138+0000","last_clean":"2023-09-27T23:26:56.814138+0000","last_became_active":"2023-09-27T23:26:56.811088+0000","last_became_peered":"2023-09-27T23:26:56.811088+0000","last_unstale":"2023-09-27T23:26:56.814138+0000","last_undegraded":"2023-09-27T23:26:56.814138+0000","last_fullsized":"2023-09-27T23:26:56.814138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:46.867145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822270+0000","last_change":"2023-09-27T23:26:56.818433+0000","last_active":"2023-09-27T23:26:56.822270+0000","last_peered":"2023-09-27T23:26:56.822270+0000","last_clean":"2023-09-27T23:26:56.822270+0000","last_became_active":"2023-09-27T23:26:56.818166+0000","last_became_peered":"2023-09-27T23:26:56.818166+0000","last_unstale":"2023-09-27T23:26:56.822270+0000","last_undegraded":"2023-09-27T23:26:56.822270+0000","last_fullsized":"2023-09-27T23:26:56.822270+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:43.997454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.829130+0000","last_change":"2023-09-27T23:26:56.821988+0000","last_active":"2023-09-27T23:26:56.829130+0000","last_peered":"2023-09-27T23:26:56.829130+0000","last_clean":"2023-09-27T23:26:56.829130+0000","last_became_active":"2023-09-27T23:26:56.821689+0000","last_became_peered":"2023-09-27T23:26:56.821689+0000","last_unstale":"2023-09-27T23:26:56.829130+0000","last_undegraded":"2023-09-27T23:26:56.829130+0000","last_fullsized":"2023-09-27T23:26:56.829130+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:44.053616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810217+0000","last_change":"2023-09-27T23:26:56.804550+0000","last_active":"2023-09-27T23:26:56.810217+0000","last_peered":"2023-09-27T23:26:56.810217+0000","last_clean":"2023-09-27T23:26:56.810217+0000","last_became_active":"2023-09-27T23:26:56.802913+0000","last_became_peered":"2023-09-27T23:26:56.802913+0000","last_unstale":"2023-09-27T23:26:56.810217+0000","last_undegraded":"2023-09-27T23:26:56.810217+0000","last_fullsized":"2023-09-27T23:26:56.810217+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:20:40.278734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800866+0000","last_change":"2023-09-27T23:26:56.796931+0000","last_active":"2023-09-27T23:26:56.800866+0000","last_peered":"2023-09-27T23:26:56.800866+0000","last_clean":"2023-09-27T23:26:56.800866+0000","last_became_active":"2023-09-27T23:26:56.796639+0000","last_became_peered":"2023-09-27T23:26:56.796639+0000","last_unstale":"2023-09-27T23:26:56.800866+0000","last_undegraded":"2023-09-27T23:26:56.800866+0000","last_fullsized":"2023-09-27T23:26:56.800866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:13:35.626367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827692+0000","last_change":"2023-09-27T23:26:56.809034+0000","last_active":"2023-09-27T23:26:56.827692+0000","last_peered":"2023-09-27T23:26:56.827692+0000","last_clean":"2023-09-27T23:26:56.827692+0000","last_became_active":"2023-09-27T23:26:56.808019+0000","last_became_peered":"2023-09-27T23:26:56.808019+0000","last_unstale":"2023-09-27T23:26:56.827692+0000","last_undegraded":"2023-09-27T23:26:56.827692+0000","last_fullsized":"2023-09-27T23:26:56.827692+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:06.599201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.820477+0000","last_change":"2023-09-27T23:26:56.810828+0000","last_active":"2023-09-27T23:26:56.820477+0000","last_peered":"2023-09-27T23:26:56.820477+0000","last_clean":"2023-09-27T23:26:56.820477+0000","last_became_active":"2023-09-27T23:26:56.810510+0000","last_became_peered":"2023-09-27T23:26:56.810510+0000","last_unstale":"2023-09-27T23:26:56.820477+0000","last_undegraded":"2023-09-27T23:26:56.820477+0000","last_fullsized":"2023-09-27T23:26:56.820477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:37:25.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822053+0000","last_change":"2023-09-27T23:26:56.810760+0000","last_active":"2023-09-27T23:26:56.822053+0000","last_peered":"2023-09-27T23:26:56.822053+0000","last_clean":"2023-09-27T23:26:56.822053+0000","last_became_active":"2023-09-27T23:26:56.810479+0000","last_became_peered":"2023-09-27T23:26:56.810479+0000","last_unstale":"2023-09-27T23:26:56.822053+0000","last_undegraded":"2023-09-27T23:26:56.822053+0000","last_fullsized":"2023-09-27T23:26:56.822053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:15:14.077484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.832378+0000","last_change":"2023-09-27T23:26:56.830036+0000","last_active":"2023-09-27T23:26:56.832378+0000","last_peered":"2023-09-27T23:26:56.832378+0000","last_clean":"2023-09-27T23:26:56.832378+0000","last_became_active":"2023-09-27T23:26:56.829721+0000","last_became_peered":"2023-09-27T23:26:56.829721+0000","last_unstale":"2023-09-27T23:26:56.832378+0000","last_undegraded":"2023-09-27T23:26:56.832378+0000","last_fullsized":"2023-09-27T23:26:56.832378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:06.599201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824033+0000","last_change":"2023-09-27T23:26:56.812859+0000","last_active":"2023-09-27T23:26:56.824033+0000","last_peered":"2023-09-27T23:26:56.824033+0000","last_clean":"2023-09-27T23:26:56.824033+0000","last_became_active":"2023-09-27T23:26:56.811956+0000","last_became_peered":"2023-09-27T23:26:56.811956+0000","last_unstale":"2023-09-27T23:26:56.824033+0000","last_undegraded":"2023-09-27T23:26:56.824033+0000","last_fullsized":"2023-09-27T23:26:56.824033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:29.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816795+0000","last_change":"2023-09-27T23:26:56.805621+0000","last_active":"2023-09-27T23:26:56.816795+0000","last_peered":"2023-09-27T23:26:56.816795+0000","last_clean":"2023-09-27T23:26:56.816795+0000","last_became_active":"2023-09-27T23:26:56.805331+0000","last_became_peered":"2023-09-27T23:26:56.805331+0000","last_unstale":"2023-09-27T23:26:56.816795+0000","last_undegraded":"2023-09-27T23:26:56.816795+0000","last_fullsized":"2023-09-27T23:26:56.816795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:05.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816125+0000","last_change":"2023-09-27T23:26:56.814235+0000","last_active":"2023-09-27T23:26:56.816125+0000","last_peered":"2023-09-27T23:26:56.816125+0000","last_clean":"2023-09-27T23:26:56.816125+0000","last_became_active":"2023-09-27T23:26:56.813878+0000","last_became_peered":"2023-09-27T23:26:56.813878+0000","last_unstale":"2023-09-27T23:26:56.816125+0000","last_undegraded":"2023-09-27T23:26:56.816125+0000","last_fullsized":"2023-09-27T23:26:56.816125+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:38.417239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831279+0000","last_change":"2023-09-27T23:26:56.829620+0000","last_active":"2023-09-27T23:26:56.831279+0000","last_peered":"2023-09-27T23:26:56.831279+0000","last_clean":"2023-09-27T23:26:56.831279+0000","last_became_active":"2023-09-27T23:26:56.829263+0000","last_became_peered":"2023-09-27T23:26:56.829263+0000","last_unstale":"2023-09-27T23:26:56.831279+0000","last_undegraded":"2023-09-27T23:26:56.831279+0000","last_fullsized":"2023-09-27T23:26:56.831279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:12.064989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.820204+0000","last_change":"2023-09-27T23:26:56.809406+0000","last_active":"2023-09-27T23:26:56.820204+0000","last_peered":"2023-09-27T23:26:56.820204+0000","last_clean":"2023-09-27T23:26:56.820204+0000","last_became_active":"2023-09-27T23:26:56.809041+0000","last_became_peered":"2023-09-27T23:26:56.809041+0000","last_unstale":"2023-09-27T23:26:56.820204+0000","last_undegraded":"2023-09-27T23:26:56.820204+0000","last_fullsized":"2023-09-27T23:26:56.820204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:15:14.077484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822441+0000","last_change":"2023-09-27T23:26:56.819439+0000","last_active":"2023-09-27T23:26:56.822441+0000","last_peered":"2023-09-27T23:26:56.822441+0000","last_clean":"2023-09-27T23:26:56.822441+0000","last_became_active":"2023-09-27T23:26:56.819086+0000","last_became_peered":"2023-09-27T23:26:56.819086+0000","last_unstale":"2023-09-27T23:26:56.822441+0000","last_undegraded":"2023-09-27T23:26:56.822441+0000","last_fullsized":"2023-09-27T23:26:56.822441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:44:25.686309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831468+0000","last_change":"2023-09-27T23:26:56.822019+0000","last_active":"2023-09-27T23:26:56.831468+0000","last_peered":"2023-09-27T23:26:56.831468+0000","last_clean":"2023-09-27T23:26:56.831468+0000","last_became_active":"2023-09-27T23:26:56.821498+0000","last_became_peered":"2023-09-27T23:26:56.821498+0000","last_unstale":"2023-09-27T23:26:56.831468+0000","last_undegraded":"2023-09-27T23:26:56.831468+0000","last_fullsized":"2023-09-27T23:26:56.831468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:38.823773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823344+0000","last_change":"2023-09-27T23:26:56.809825+0000","last_active":"2023-09-27T23:26:56.823344+0000","last_peered":"2023-09-27T23:26:56.823344+0000","last_clean":"2023-09-27T23:26:56.823344+0000","last_became_active":"2023-09-27T23:26:56.809496+0000","last_became_peered":"2023-09-27T23:26:56.809496+0000","last_unstale":"2023-09-27T23:26:56.823344+0000","last_undegraded":"2023-09-27T23:26:56.823344+0000","last_fullsized":"2023-09-27T23:26:56.823344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:26:04.894120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830665+0000","last_change":"2023-09-27T23:26:56.826954+0000","last_active":"2023-09-27T23:26:56.830665+0000","last_peered":"2023-09-27T23:26:56.830665+0000","last_clean":"2023-09-27T23:26:56.830665+0000","last_became_active":"2023-09-27T23:26:56.825620+0000","last_became_peered":"2023-09-27T23:26:56.825620+0000","last_unstale":"2023-09-27T23:26:56.830665+0000","last_undegraded":"2023-09-27T23:26:56.830665+0000","last_fullsized":"2023-09-27T23:26:56.830665+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:33:30.240296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810612+0000","last_change":"2023-09-27T23:26:56.805480+0000","last_active":"2023-09-27T23:26:56.810612+0000","last_peered":"2023-09-27T23:26:56.810612+0000","last_clean":"2023-09-27T23:26:56.810612+0000","last_became_active":"2023-09-27T23:26:56.805168+0000","last_became_peered":"2023-09-27T23:26:56.805168+0000","last_unstale":"2023-09-27T23:26:56.810612+0000","last_undegraded":"2023-09-27T23:26:56.810612+0000","last_fullsized":"2023-09-27T23:26:56.810612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:41.801806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827369+0000","last_change":"2023-09-27T23:26:56.824083+0000","last_active":"2023-09-27T23:26:56.827369+0000","last_peered":"2023-09-27T23:26:56.827369+0000","last_clean":"2023-09-27T23:26:56.827369+0000","last_became_active":"2023-09-27T23:26:56.823794+0000","last_became_peered":"2023-09-27T23:26:56.823794+0000","last_unstale":"2023-09-27T23:26:56.827369+0000","last_undegraded":"2023-09-27T23:26:56.827369+0000","last_fullsized":"2023-09-27T23:26:56.827369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:32.765279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830531+0000","last_change":"2023-09-27T23:26:56.824788+0000","last_active":"2023-09-27T23:26:56.830531+0000","last_peered":"2023-09-27T23:26:56.830531+0000","last_clean":"2023-09-27T23:26:56.830531+0000","last_became_active":"2023-09-27T23:26:56.822995+0000","last_became_peered":"2023-09-27T23:26:56.822995+0000","last_unstale":"2023-09-27T23:26:56.830531+0000","last_undegraded":"2023-09-27T23:26:56.830531+0000","last_fullsized":"2023-09-27T23:26:56.830531+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:12:30.183845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822214+0000","last_change":"2023-09-27T23:26:56.815124+0000","last_active":"2023-09-27T23:26:56.822214+0000","last_peered":"2023-09-27T23:26:56.822214+0000","last_clean":"2023-09-27T23:26:56.822214+0000","last_became_active":"2023-09-27T23:26:56.814844+0000","last_became_peered":"2023-09-27T23:26:56.814844+0000","last_unstale":"2023-09-27T23:26:56.822214+0000","last_undegraded":"2023-09-27T23:26:56.822214+0000","last_fullsized":"2023-09-27T23:26:56.822214+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:07:07.711075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830575+0000","last_change":"2023-09-27T23:26:56.825713+0000","last_active":"2023-09-27T23:26:56.830575+0000","last_peered":"2023-09-27T23:26:56.830575+0000","last_clean":"2023-09-27T23:26:56.830575+0000","last_became_active":"2023-09-27T23:26:56.825227+0000","last_became_peered":"2023-09-27T23:26:56.825227+0000","last_unstale":"2023-09-27T23:26:56.830575+0000","last_undegraded":"2023-09-27T23:26:56.830575+0000","last_fullsized":"2023-09-27T23:26:56.830575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:39.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.815977+0000","last_change":"2023-09-27T23:26:56.811719+0000","last_active":"2023-09-27T23:26:56.815977+0000","last_peered":"2023-09-27T23:26:56.815977+0000","last_clean":"2023-09-27T23:26:56.815977+0000","last_became_active":"2023-09-27T23:26:56.811372+0000","last_became_peered":"2023-09-27T23:26:56.811372+0000","last_unstale":"2023-09-27T23:26:56.815977+0000","last_undegraded":"2023-09-27T23:26:56.815977+0000","last_fullsized":"2023-09-27T23:26:56.815977+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:48:43.034321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818742+0000","last_change":"2023-09-27T23:26:56.814879+0000","last_active":"2023-09-27T23:26:56.818742+0000","last_peered":"2023-09-27T23:26:56.818742+0000","last_clean":"2023-09-27T23:26:56.818742+0000","last_became_active":"2023-09-27T23:26:56.814574+0000","last_became_peered":"2023-09-27T23:26:56.814574+0000","last_unstale":"2023-09-27T23:26:56.818742+0000","last_undegraded":"2023-09-27T23:26:56.818742+0000","last_fullsized":"2023-09-27T23:26:56.818742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:38.070621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819381+0000","last_change":"2023-09-27T23:26:56.817033+0000","last_active":"2023-09-27T23:26:56.819381+0000","last_peered":"2023-09-27T23:26:56.819381+0000","last_clean":"2023-09-27T23:26:56.819381+0000","last_became_active":"2023-09-27T23:26:56.816540+0000","last_became_peered":"2023-09-27T23:26:56.816540+0000","last_unstale":"2023-09-27T23:26:56.819381+0000","last_undegraded":"2023-09-27T23:26:56.819381+0000","last_fullsized":"2023-09-27T23:26:56.819381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:12:30.183845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831552+0000","last_change":"2023-09-27T23:26:56.825743+0000","last_active":"2023-09-27T23:26:56.831552+0000","last_peered":"2023-09-27T23:26:56.831552+0000","last_clean":"2023-09-27T23:26:56.831552+0000","last_became_active":"2023-09-27T23:26:56.823231+0000","last_became_peered":"2023-09-27T23:26:56.823231+0000","last_unstale":"2023-09-27T23:26:56.831552+0000","last_undegraded":"2023-09-27T23:26:56.831552+0000","last_fullsized":"2023-09-27T23:26:56.831552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:34:23.130491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.802160+0000","last_change":"2023-09-27T23:26:56.800806+0000","last_active":"2023-09-27T23:26:56.802160+0000","last_peered":"2023-09-27T23:26:56.802160+0000","last_clean":"2023-09-27T23:26:56.802160+0000","last_became_active":"2023-09-27T23:26:56.800324+0000","last_became_peered":"2023-09-27T23:26:56.800324+0000","last_unstale":"2023-09-27T23:26:56.802160+0000","last_undegraded":"2023-09-27T23:26:56.802160+0000","last_fullsized":"2023-09-27T23:26:56.802160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:58.847869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.808016+0000","last_change":"2023-09-27T23:26:56.805693+0000","last_active":"2023-09-27T23:26:56.808016+0000","last_peered":"2023-09-27T23:26:56.808016+0000","last_clean":"2023-09-27T23:26:56.808016+0000","last_became_active":"2023-09-27T23:26:56.805419+0000","last_became_peered":"2023-09-27T23:26:56.805419+0000","last_unstale":"2023-09-27T23:26:56.808016+0000","last_undegraded":"2023-09-27T23:26:56.808016+0000","last_fullsized":"2023-09-27T23:26:56.808016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:06.599201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830495+0000","last_change":"2023-09-27T23:26:56.825150+0000","last_active":"2023-09-27T23:26:56.830495+0000","last_peered":"2023-09-27T23:26:56.830495+0000","last_clean":"2023-09-27T23:26:56.830495+0000","last_became_active":"2023-09-27T23:26:56.822922+0000","last_became_peered":"2023-09-27T23:26:56.822922+0000","last_unstale":"2023-09-27T23:26:56.830495+0000","last_undegraded":"2023-09-27T23:26:56.830495+0000","last_fullsized":"2023-09-27T23:26:56.830495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:20:40.278734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.819184+0000","last_change":"2023-09-27T23:26:56.817217+0000","last_active":"2023-09-27T23:26:56.819184+0000","last_peered":"2023-09-27T23:26:56.819184+0000","last_clean":"2023-09-27T23:26:56.819184+0000","last_became_active":"2023-09-27T23:26:56.816937+0000","last_became_peered":"2023-09-27T23:26:56.816937+0000","last_unstale":"2023-09-27T23:26:56.819184+0000","last_undegraded":"2023-09-27T23:26:56.819184+0000","last_fullsized":"2023-09-27T23:26:56.819184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:17:55.307710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831445+0000","last_change":"2023-09-27T23:26:56.821769+0000","last_active":"2023-09-27T23:26:56.831445+0000","last_peered":"2023-09-27T23:26:56.831445+0000","last_clean":"2023-09-27T23:26:56.831445+0000","last_became_active":"2023-09-27T23:26:56.818198+0000","last_became_peered":"2023-09-27T23:26:56.818198+0000","last_unstale":"2023-09-27T23:26:56.831445+0000","last_undegraded":"2023-09-27T23:26:56.831445+0000","last_fullsized":"2023-09-27T23:26:56.831445+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:44:25.686309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.832315+0000","last_change":"2023-09-27T23:26:56.828745+0000","last_active":"2023-09-27T23:26:56.832315+0000","last_peered":"2023-09-27T23:26:56.832315+0000","last_clean":"2023-09-27T23:26:56.832315+0000","last_became_active":"2023-09-27T23:26:56.828288+0000","last_became_peered":"2023-09-27T23:26:56.828288+0000","last_unstale":"2023-09-27T23:26:56.832315+0000","last_undegraded":"2023-09-27T23:26:56.832315+0000","last_fullsized":"2023-09-27T23:26:56.832315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:37:25.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"70'4","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828290+0000","last_change":"2023-09-27T23:26:56.436638+0000","last_active":"2023-09-27T23:26:56.828290+0000","last_peered":"2023-09-27T23:26:56.828290+0000","last_clean":"2023-09-27T23:26:56.828290+0000","last_became_active":"2023-09-27T23:26:56.436325+0000","last_became_peered":"2023-09-27T23:26:56.436325+0000","last_unstale":"2023-09-27T23:26:56.828290+0000","last_undegraded":"2023-09-27T23:26:56.828290+0000","last_fullsized":"2023-09-27T23:26:56.828290+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:20:39.274221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824218+0000","last_change":"2023-09-27T23:26:56.814262+0000","last_active":"2023-09-27T23:26:56.824218+0000","last_peered":"2023-09-27T23:26:56.824218+0000","last_clean":"2023-09-27T23:26:56.824218+0000","last_became_active":"2023-09-27T23:26:56.813928+0000","last_became_peered":"2023-09-27T23:26:56.813928+0000","last_unstale":"2023-09-27T23:26:56.824218+0000","last_undegraded":"2023-09-27T23:26:56.824218+0000","last_fullsized":"2023-09-27T23:26:56.824218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:49.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823715+0000","last_change":"2023-09-27T23:26:56.429097+0000","last_active":"2023-09-27T23:26:56.823715+0000","last_peered":"2023-09-27T23:26:56.823715+0000","last_clean":"2023-09-27T23:26:56.823715+0000","last_became_active":"2023-09-27T23:26:56.428639+0000","last_became_peered":"2023-09-27T23:26:56.428639+0000","last_unstale":"2023-09-27T23:26:56.823715+0000","last_undegraded":"2023-09-27T23:26:56.823715+0000","last_fullsized":"2023-09-27T23:26:56.823715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:10:29.314492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831569+0000","last_change":"2023-09-27T23:26:56.822307+0000","last_active":"2023-09-27T23:26:56.831569+0000","last_peered":"2023-09-27T23:26:56.831569+0000","last_clean":"2023-09-27T23:26:56.831569+0000","last_became_active":"2023-09-27T23:26:56.821304+0000","last_became_peered":"2023-09-27T23:26:56.821304+0000","last_unstale":"2023-09-27T23:26:56.831569+0000","last_undegraded":"2023-09-27T23:26:56.831569+0000","last_fullsized":"2023-09-27T23:26:56.831569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:56.689414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828509+0000","last_change":"2023-09-27T23:26:56.436484+0000","last_active":"2023-09-27T23:26:56.828509+0000","last_peered":"2023-09-27T23:26:56.828509+0000","last_clean":"2023-09-27T23:26:56.828509+0000","last_became_active":"2023-09-27T23:26:56.436220+0000","last_became_peered":"2023-09-27T23:26:56.436220+0000","last_unstale":"2023-09-27T23:26:56.828509+0000","last_undegraded":"2023-09-27T23:26:56.828509+0000","last_fullsized":"2023-09-27T23:26:56.828509+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:38.217423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828010+0000","last_change":"2023-09-27T23:26:56.816908+0000","last_active":"2023-09-27T23:26:56.828010+0000","last_peered":"2023-09-27T23:26:56.828010+0000","last_clean":"2023-09-27T23:26:56.828010+0000","last_became_active":"2023-09-27T23:26:56.813414+0000","last_became_peered":"2023-09-27T23:26:56.813414+0000","last_unstale":"2023-09-27T23:26:56.828010+0000","last_undegraded":"2023-09-27T23:26:56.828010+0000","last_fullsized":"2023-09-27T23:26:56.828010+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:41:15.113008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823492+0000","last_change":"2023-09-27T23:26:56.425814+0000","last_active":"2023-09-27T23:26:56.823492+0000","last_peered":"2023-09-27T23:26:56.823492+0000","last_clean":"2023-09-27T23:26:56.823492+0000","last_became_active":"2023-09-27T23:26:56.423551+0000","last_became_peered":"2023-09-27T23:26:56.423551+0000","last_unstale":"2023-09-27T23:26:56.823492+0000","last_undegraded":"2023-09-27T23:26:56.823492+0000","last_fullsized":"2023-09-27T23:26:56.823492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:40.797293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.825718+0000","last_change":"2023-09-27T23:26:56.821293+0000","last_active":"2023-09-27T23:26:56.825718+0000","last_peered":"2023-09-27T23:26:56.825718+0000","last_clean":"2023-09-27T23:26:56.825718+0000","last_became_active":"2023-09-27T23:26:56.821019+0000","last_became_peered":"2023-09-27T23:26:56.821019+0000","last_unstale":"2023-09-27T23:26:56.825718+0000","last_undegraded":"2023-09-27T23:26:56.825718+0000","last_fullsized":"2023-09-27T23:26:56.825718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:10:10.206062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.808720+0000","last_change":"2023-09-27T23:26:56.426151+0000","last_active":"2023-09-27T23:26:56.808720+0000","last_peered":"2023-09-27T23:26:56.808720+0000","last_clean":"2023-09-27T23:26:56.808720+0000","last_became_active":"2023-09-27T23:26:56.425674+0000","last_became_peered":"2023-09-27T23:26:56.425674+0000","last_unstale":"2023-09-27T23:26:56.808720+0000","last_undegraded":"2023-09-27T23:26:56.808720+0000","last_fullsized":"2023-09-27T23:26:56.808720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:19.710475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831735+0000","last_change":"2023-09-27T23:26:56.828424+0000","last_active":"2023-09-27T23:26:56.831735+0000","last_peered":"2023-09-27T23:26:56.831735+0000","last_clean":"2023-09-27T23:26:56.831735+0000","last_became_active":"2023-09-27T23:26:56.828044+0000","last_became_peered":"2023-09-27T23:26:56.828044+0000","last_unstale":"2023-09-27T23:26:56.831735+0000","last_undegraded":"2023-09-27T23:26:56.831735+0000","last_fullsized":"2023-09-27T23:26:56.831735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:00:51.597357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827750+0000","last_change":"2023-09-27T23:26:56.436705+0000","last_active":"2023-09-27T23:26:56.827750+0000","last_peered":"2023-09-27T23:26:56.827750+0000","last_clean":"2023-09-27T23:26:56.827750+0000","last_became_active":"2023-09-27T23:26:56.436367+0000","last_became_peered":"2023-09-27T23:26:56.436367+0000","last_unstale":"2023-09-27T23:26:56.827750+0000","last_undegraded":"2023-09-27T23:26:56.827750+0000","last_fullsized":"2023-09-27T23:26:56.827750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:10:29.314492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824529+0000","last_change":"2023-09-27T23:26:56.823238+0000","last_active":"2023-09-27T23:26:56.824529+0000","last_peered":"2023-09-27T23:26:56.824529+0000","last_clean":"2023-09-27T23:26:56.824529+0000","last_became_active":"2023-09-27T23:26:56.822535+0000","last_became_peered":"2023-09-27T23:26:56.822535+0000","last_unstale":"2023-09-27T23:26:56.824529+0000","last_undegraded":"2023-09-27T23:26:56.824529+0000","last_fullsized":"2023-09-27T23:26:56.824529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:46.173544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796531+0000","last_change":"2023-09-27T23:26:56.426213+0000","last_active":"2023-09-27T23:26:56.796531+0000","last_peered":"2023-09-27T23:26:56.796531+0000","last_clean":"2023-09-27T23:26:56.796531+0000","last_became_active":"2023-09-27T23:26:56.425900+0000","last_became_peered":"2023-09-27T23:26:56.425900+0000","last_unstale":"2023-09-27T23:26:56.796531+0000","last_undegraded":"2023-09-27T23:26:56.796531+0000","last_fullsized":"2023-09-27T23:26:56.796531+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:38.377307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823517+0000","last_change":"2023-09-27T23:26:56.812947+0000","last_active":"2023-09-27T23:26:56.823517+0000","last_peered":"2023-09-27T23:26:56.823517+0000","last_clean":"2023-09-27T23:26:56.823517+0000","last_became_active":"2023-09-27T23:26:56.810262+0000","last_became_peered":"2023-09-27T23:26:56.810262+0000","last_unstale":"2023-09-27T23:26:56.823517+0000","last_undegraded":"2023-09-27T23:26:56.823517+0000","last_fullsized":"2023-09-27T23:26:56.823517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:34:23.130491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.803567+0000","last_change":"2023-09-27T23:26:56.431639+0000","last_active":"2023-09-27T23:26:56.803567+0000","last_peered":"2023-09-27T23:26:56.803567+0000","last_clean":"2023-09-27T23:26:56.803567+0000","last_became_active":"2023-09-27T23:26:56.431313+0000","last_became_peered":"2023-09-27T23:26:56.431313+0000","last_unstale":"2023-09-27T23:26:56.803567+0000","last_undegraded":"2023-09-27T23:26:56.803567+0000","last_fullsized":"2023-09-27T23:26:56.803567+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:06:14.449464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822470+0000","last_change":"2023-09-27T23:26:56.819969+0000","last_active":"2023-09-27T23:26:56.822470+0000","last_peered":"2023-09-27T23:26:56.822470+0000","last_clean":"2023-09-27T23:26:56.822470+0000","last_became_active":"2023-09-27T23:26:56.819674+0000","last_became_peered":"2023-09-27T23:26:56.819674+0000","last_unstale":"2023-09-27T23:26:56.822470+0000","last_undegraded":"2023-09-27T23:26:56.822470+0000","last_fullsized":"2023-09-27T23:26:56.822470+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:10:10.206062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.806649+0000","last_change":"2023-09-27T23:26:56.433705+0000","last_active":"2023-09-27T23:26:56.806649+0000","last_peered":"2023-09-27T23:26:56.806649+0000","last_clean":"2023-09-27T23:26:56.806649+0000","last_became_active":"2023-09-27T23:26:56.433345+0000","last_became_peered":"2023-09-27T23:26:56.433345+0000","last_unstale":"2023-09-27T23:26:56.806649+0000","last_undegraded":"2023-09-27T23:26:56.806649+0000","last_fullsized":"2023-09-27T23:26:56.806649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:40.797293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830278+0000","last_change":"2023-09-27T23:26:56.824751+0000","last_active":"2023-09-27T23:26:56.830278+0000","last_peered":"2023-09-27T23:26:56.830278+0000","last_clean":"2023-09-27T23:26:56.830278+0000","last_became_active":"2023-09-27T23:26:56.822551+0000","last_became_peered":"2023-09-27T23:26:56.822551+0000","last_unstale":"2023-09-27T23:26:56.830278+0000","last_undegraded":"2023-09-27T23:26:56.830278+0000","last_fullsized":"2023-09-27T23:26:56.830278+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:17:04.288794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.809813+0000","last_change":"2023-09-27T23:26:56.428912+0000","last_active":"2023-09-27T23:26:56.809813+0000","last_peered":"2023-09-27T23:26:56.809813+0000","last_clean":"2023-09-27T23:26:56.809813+0000","last_became_active":"2023-09-27T23:26:56.428319+0000","last_became_peered":"2023-09-27T23:26:56.428319+0000","last_unstale":"2023-09-27T23:26:56.809813+0000","last_undegraded":"2023-09-27T23:26:56.809813+0000","last_fullsized":"2023-09-27T23:26:56.809813+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:26.414570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824186+0000","last_change":"2023-09-27T23:26:56.816043+0000","last_active":"2023-09-27T23:26:56.824186+0000","last_peered":"2023-09-27T23:26:56.824186+0000","last_clean":"2023-09-27T23:26:56.824186+0000","last_became_active":"2023-09-27T23:26:56.815712+0000","last_became_peered":"2023-09-27T23:26:56.815712+0000","last_unstale":"2023-09-27T23:26:56.824186+0000","last_undegraded":"2023-09-27T23:26:56.824186+0000","last_fullsized":"2023-09-27T23:26:56.824186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:54.503767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"70'3","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.829079+0000","last_change":"2023-09-27T23:26:56.430635+0000","last_active":"2023-09-27T23:26:56.829079+0000","last_peered":"2023-09-27T23:26:56.829079+0000","last_clean":"2023-09-27T23:26:56.829079+0000","last_became_active":"2023-09-27T23:26:56.430216+0000","last_became_peered":"2023-09-27T23:26:56.430216+0000","last_unstale":"2023-09-27T23:26:56.829079+0000","last_undegraded":"2023-09-27T23:26:56.829079+0000","last_fullsized":"2023-09-27T23:26:56.829079+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:45.862631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830558+0000","last_change":"2023-09-27T23:26:56.825066+0000","last_active":"2023-09-27T23:26:56.830558+0000","last_peered":"2023-09-27T23:26:56.830558+0000","last_clean":"2023-09-27T23:26:56.830558+0000","last_became_active":"2023-09-27T23:26:56.824694+0000","last_became_peered":"2023-09-27T23:26:56.824694+0000","last_unstale":"2023-09-27T23:26:56.830558+0000","last_undegraded":"2023-09-27T23:26:56.830558+0000","last_fullsized":"2023-09-27T23:26:56.830558+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:57:09.552899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"70'6","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.833685+0000","last_change":"2023-09-27T23:26:56.438412+0000","last_active":"2023-09-27T23:26:56.833685+0000","last_peered":"2023-09-27T23:26:56.833685+0000","last_clean":"2023-09-27T23:26:56.833685+0000","last_became_active":"2023-09-27T23:26:56.438097+0000","last_became_peered":"2023-09-27T23:26:56.438097+0000","last_unstale":"2023-09-27T23:26:56.833685+0000","last_undegraded":"2023-09-27T23:26:56.833685+0000","last_fullsized":"2023-09-27T23:26:56.833685+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:59:09.616338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.821231+0000","last_change":"2023-09-27T23:26:56.818999+0000","last_active":"2023-09-27T23:26:56.821231+0000","last_peered":"2023-09-27T23:26:56.821231+0000","last_clean":"2023-09-27T23:26:56.821231+0000","last_became_active":"2023-09-27T23:26:56.818507+0000","last_became_peered":"2023-09-27T23:26:56.818507+0000","last_unstale":"2023-09-27T23:26:56.821231+0000","last_undegraded":"2023-09-27T23:26:56.821231+0000","last_fullsized":"2023-09-27T23:26:56.821231+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:39.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823136+0000","last_change":"2023-09-27T23:26:56.429881+0000","last_active":"2023-09-27T23:26:56.823136+0000","last_peered":"2023-09-27T23:26:56.823136+0000","last_clean":"2023-09-27T23:26:56.823136+0000","last_became_active":"2023-09-27T23:26:56.429578+0000","last_became_peered":"2023-09-27T23:26:56.429578+0000","last_unstale":"2023-09-27T23:26:56.823136+0000","last_undegraded":"2023-09-27T23:26:56.823136+0000","last_fullsized":"2023-09-27T23:26:56.823136+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:38.217423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828858+0000","last_change":"2023-09-27T23:26:56.818767+0000","last_active":"2023-09-27T23:26:56.828858+0000","last_peered":"2023-09-27T23:26:56.828858+0000","last_clean":"2023-09-27T23:26:56.828858+0000","last_became_active":"2023-09-27T23:26:56.818428+0000","last_became_peered":"2023-09-27T23:26:56.818428+0000","last_unstale":"2023-09-27T23:26:56.828858+0000","last_undegraded":"2023-09-27T23:26:56.828858+0000","last_fullsized":"2023-09-27T23:26:56.828858+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:27.419083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798785+0000","last_change":"2023-09-27T23:26:56.424428+0000","last_active":"2023-09-27T23:26:56.798785+0000","last_peered":"2023-09-27T23:26:56.798785+0000","last_clean":"2023-09-27T23:26:56.798785+0000","last_became_active":"2023-09-27T23:26:56.424118+0000","last_became_peered":"2023-09-27T23:26:56.424118+0000","last_unstale":"2023-09-27T23:26:56.798785+0000","last_undegraded":"2023-09-27T23:26:56.798785+0000","last_fullsized":"2023-09-27T23:26:56.798785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:18:10.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.832373+0000","last_change":"2023-09-27T23:26:56.829945+0000","last_active":"2023-09-27T23:26:56.832373+0000","last_peered":"2023-09-27T23:26:56.832373+0000","last_clean":"2023-09-27T23:26:56.832373+0000","last_became_active":"2023-09-27T23:26:56.829652+0000","last_became_peered":"2023-09-27T23:26:56.829652+0000","last_unstale":"2023-09-27T23:26:56.832373+0000","last_undegraded":"2023-09-27T23:26:56.832373+0000","last_fullsized":"2023-09-27T23:26:56.832373+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:56.689414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796611+0000","last_change":"2023-09-27T23:26:56.425629+0000","last_active":"2023-09-27T23:26:56.796611+0000","last_peered":"2023-09-27T23:26:56.796611+0000","last_clean":"2023-09-27T23:26:56.796611+0000","last_became_active":"2023-09-27T23:26:56.423579+0000","last_became_peered":"2023-09-27T23:26:56.423579+0000","last_unstale":"2023-09-27T23:26:56.796611+0000","last_undegraded":"2023-09-27T23:26:56.796611+0000","last_fullsized":"2023-09-27T23:26:56.796611+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:37.066108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830694+0000","last_change":"2023-09-27T23:26:56.827492+0000","last_active":"2023-09-27T23:26:56.830694+0000","last_peered":"2023-09-27T23:26:56.830694+0000","last_clean":"2023-09-27T23:26:56.830694+0000","last_became_active":"2023-09-27T23:26:56.826988+0000","last_became_peered":"2023-09-27T23:26:56.826988+0000","last_unstale":"2023-09-27T23:26:56.830694+0000","last_undegraded":"2023-09-27T23:26:56.830694+0000","last_fullsized":"2023-09-27T23:26:56.830694+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:39.221936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"70'8","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828947+0000","last_change":"2023-09-27T23:26:56.429262+0000","last_active":"2023-09-27T23:26:56.828947+0000","last_peered":"2023-09-27T23:26:56.828947+0000","last_clean":"2023-09-27T23:26:56.828947+0000","last_became_active":"2023-09-27T23:26:56.428964+0000","last_became_peered":"2023-09-27T23:26:56.428964+0000","last_unstale":"2023-09-27T23:26:56.828947+0000","last_undegraded":"2023-09-27T23:26:56.828947+0000","last_fullsized":"2023-09-27T23:26:56.828947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:34:37.066108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.810177+0000","last_change":"2023-09-27T23:26:56.803168+0000","last_active":"2023-09-27T23:26:56.810177+0000","last_peered":"2023-09-27T23:26:56.810177+0000","last_clean":"2023-09-27T23:26:56.810177+0000","last_became_active":"2023-09-27T23:26:56.802813+0000","last_became_peered":"2023-09-27T23:26:56.802813+0000","last_unstale":"2023-09-27T23:26:56.810177+0000","last_undegraded":"2023-09-27T23:26:56.810177+0000","last_fullsized":"2023-09-27T23:26:56.810177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:59.999313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817510+0000","last_change":"2023-09-27T23:26:56.433280+0000","last_active":"2023-09-27T23:26:56.817510+0000","last_peered":"2023-09-27T23:26:56.817510+0000","last_clean":"2023-09-27T23:26:56.817510+0000","last_became_active":"2023-09-27T23:26:56.432946+0000","last_became_peered":"2023-09-27T23:26:56.432946+0000","last_unstale":"2023-09-27T23:26:56.817510+0000","last_undegraded":"2023-09-27T23:26:56.817510+0000","last_fullsized":"2023-09-27T23:26:56.817510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:05:40.954906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816094+0000","last_change":"2023-09-27T23:26:56.813408+0000","last_active":"2023-09-27T23:26:56.816094+0000","last_peered":"2023-09-27T23:26:56.816094+0000","last_clean":"2023-09-27T23:26:56.816094+0000","last_became_active":"2023-09-27T23:26:56.813107+0000","last_became_peered":"2023-09-27T23:26:56.813107+0000","last_unstale":"2023-09-27T23:26:56.816094+0000","last_undegraded":"2023-09-27T23:26:56.816094+0000","last_fullsized":"2023-09-27T23:26:56.816094+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:44.053616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.812199+0000","last_change":"2023-09-27T23:26:56.431022+0000","last_active":"2023-09-27T23:26:56.812199+0000","last_peered":"2023-09-27T23:26:56.812199+0000","last_clean":"2023-09-27T23:26:56.812199+0000","last_became_active":"2023-09-27T23:26:56.430709+0000","last_became_peered":"2023-09-27T23:26:56.430709+0000","last_unstale":"2023-09-27T23:26:56.812199+0000","last_undegraded":"2023-09-27T23:26:56.812199+0000","last_fullsized":"2023-09-27T23:26:56.812199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:38.982539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824360+0000","last_change":"2023-09-27T23:26:56.815852+0000","last_active":"2023-09-27T23:26:56.824360+0000","last_peered":"2023-09-27T23:26:56.824360+0000","last_clean":"2023-09-27T23:26:56.824360+0000","last_became_active":"2023-09-27T23:26:56.815565+0000","last_became_peered":"2023-09-27T23:26:56.815565+0000","last_unstale":"2023-09-27T23:26:56.824360+0000","last_undegraded":"2023-09-27T23:26:56.824360+0000","last_fullsized":"2023-09-27T23:26:56.824360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:21:59.389594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798790+0000","last_change":"2023-09-27T23:26:56.425304+0000","last_active":"2023-09-27T23:26:56.798790+0000","last_peered":"2023-09-27T23:26:56.798790+0000","last_clean":"2023-09-27T23:26:56.798790+0000","last_became_active":"2023-09-27T23:26:56.425031+0000","last_became_peered":"2023-09-27T23:26:56.425031+0000","last_unstale":"2023-09-27T23:26:56.798790+0000","last_undegraded":"2023-09-27T23:26:56.798790+0000","last_fullsized":"2023-09-27T23:26:56.798790+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:45.293801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828029+0000","last_change":"2023-09-27T23:26:56.817529+0000","last_active":"2023-09-27T23:26:56.828029+0000","last_peered":"2023-09-27T23:26:56.828029+0000","last_clean":"2023-09-27T23:26:56.828029+0000","last_became_active":"2023-09-27T23:26:56.817162+0000","last_became_peered":"2023-09-27T23:26:56.817162+0000","last_unstale":"2023-09-27T23:26:56.828029+0000","last_undegraded":"2023-09-27T23:26:56.828029+0000","last_fullsized":"2023-09-27T23:26:56.828029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:07.457954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799350+0000","last_change":"2023-09-27T23:26:56.425530+0000","last_active":"2023-09-27T23:26:56.799350+0000","last_peered":"2023-09-27T23:26:56.799350+0000","last_clean":"2023-09-27T23:26:56.799350+0000","last_became_active":"2023-09-27T23:26:56.425247+0000","last_became_peered":"2023-09-27T23:26:56.425247+0000","last_unstale":"2023-09-27T23:26:56.799350+0000","last_undegraded":"2023-09-27T23:26:56.799350+0000","last_fullsized":"2023-09-27T23:26:56.799350+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:08:59.880116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822329+0000","last_change":"2023-09-27T23:26:56.819129+0000","last_active":"2023-09-27T23:26:56.822329+0000","last_peered":"2023-09-27T23:26:56.822329+0000","last_clean":"2023-09-27T23:26:56.822329+0000","last_became_active":"2023-09-27T23:26:56.818825+0000","last_became_peered":"2023-09-27T23:26:56.818825+0000","last_unstale":"2023-09-27T23:26:56.822329+0000","last_undegraded":"2023-09-27T23:26:56.822329+0000","last_fullsized":"2023-09-27T23:26:56.822329+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:35:01.738648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799681+0000","last_change":"2023-09-27T23:26:56.431567+0000","last_active":"2023-09-27T23:26:56.799681+0000","last_peered":"2023-09-27T23:26:56.799681+0000","last_clean":"2023-09-27T23:26:56.799681+0000","last_became_active":"2023-09-27T23:26:56.431206+0000","last_became_peered":"2023-09-27T23:26:56.431206+0000","last_unstale":"2023-09-27T23:26:56.799681+0000","last_undegraded":"2023-09-27T23:26:56.799681+0000","last_fullsized":"2023-09-27T23:26:56.799681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:52:27.126388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830622+0000","last_change":"2023-09-27T23:26:56.825789+0000","last_active":"2023-09-27T23:26:56.830622+0000","last_peered":"2023-09-27T23:26:56.830622+0000","last_clean":"2023-09-27T23:26:56.830622+0000","last_became_active":"2023-09-27T23:26:56.825516+0000","last_became_peered":"2023-09-27T23:26:56.825516+0000","last_unstale":"2023-09-27T23:26:56.830622+0000","last_undegraded":"2023-09-27T23:26:56.830622+0000","last_fullsized":"2023-09-27T23:26:56.830622+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:10:30.319005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.804419+0000","last_change":"2023-09-27T23:26:56.436059+0000","last_active":"2023-09-27T23:26:56.804419+0000","last_peered":"2023-09-27T23:26:56.804419+0000","last_clean":"2023-09-27T23:26:56.804419+0000","last_became_active":"2023-09-27T23:26:56.435772+0000","last_became_peered":"2023-09-27T23:26:56.435772+0000","last_unstale":"2023-09-27T23:26:56.804419+0000","last_undegraded":"2023-09-27T23:26:56.804419+0000","last_fullsized":"2023-09-27T23:26:56.804419+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:52.872997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824213+0000","last_change":"2023-09-27T23:26:56.811986+0000","last_active":"2023-09-27T23:26:56.824213+0000","last_peered":"2023-09-27T23:26:56.824213+0000","last_clean":"2023-09-27T23:26:56.824213+0000","last_became_active":"2023-09-27T23:26:56.811704+0000","last_became_peered":"2023-09-27T23:26:56.811704+0000","last_unstale":"2023-09-27T23:26:56.824213+0000","last_undegraded":"2023-09-27T23:26:56.824213+0000","last_fullsized":"2023-09-27T23:26:56.824213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:10:10.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.797203+0000","last_change":"2023-09-27T23:26:56.439391+0000","last_active":"2023-09-27T23:26:56.797203+0000","last_peered":"2023-09-27T23:26:56.797203+0000","last_clean":"2023-09-27T23:26:56.797203+0000","last_became_active":"2023-09-27T23:26:56.439021+0000","last_became_peered":"2023-09-27T23:26:56.439021+0000","last_unstale":"2023-09-27T23:26:56.797203+0000","last_undegraded":"2023-09-27T23:26:56.797203+0000","last_fullsized":"2023-09-27T23:26:56.797203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:59.110216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823537+0000","last_change":"2023-09-27T23:26:56.813169+0000","last_active":"2023-09-27T23:26:56.823537+0000","last_peered":"2023-09-27T23:26:56.823537+0000","last_clean":"2023-09-27T23:26:56.823537+0000","last_became_active":"2023-09-27T23:26:56.810577+0000","last_became_peered":"2023-09-27T23:26:56.810577+0000","last_unstale":"2023-09-27T23:26:56.823537+0000","last_undegraded":"2023-09-27T23:26:56.823537+0000","last_fullsized":"2023-09-27T23:26:56.823537+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:56.689414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.804070+0000","last_change":"2023-09-27T23:26:56.431077+0000","last_active":"2023-09-27T23:26:56.804070+0000","last_peered":"2023-09-27T23:26:56.804070+0000","last_clean":"2023-09-27T23:26:56.804070+0000","last_became_active":"2023-09-27T23:26:56.429844+0000","last_became_peered":"2023-09-27T23:26:56.429844+0000","last_unstale":"2023-09-27T23:26:56.804070+0000","last_undegraded":"2023-09-27T23:26:56.804070+0000","last_fullsized":"2023-09-27T23:26:56.804070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:38.982539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.822074+0000","last_change":"2023-09-27T23:26:56.813632+0000","last_active":"2023-09-27T23:26:56.822074+0000","last_peered":"2023-09-27T23:26:56.822074+0000","last_clean":"2023-09-27T23:26:56.822074+0000","last_became_active":"2023-09-27T23:26:56.813352+0000","last_became_peered":"2023-09-27T23:26:56.813352+0000","last_unstale":"2023-09-27T23:26:56.822074+0000","last_undegraded":"2023-09-27T23:26:56.822074+0000","last_fullsized":"2023-09-27T23:26:56.822074+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:29:26.826938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798184+0000","last_change":"2023-09-27T23:26:56.429406+0000","last_active":"2023-09-27T23:26:56.798184+0000","last_peered":"2023-09-27T23:26:56.798184+0000","last_clean":"2023-09-27T23:26:56.798184+0000","last_became_active":"2023-09-27T23:26:56.429125+0000","last_became_peered":"2023-09-27T23:26:56.429125+0000","last_unstale":"2023-09-27T23:26:56.798184+0000","last_undegraded":"2023-09-27T23:26:56.798184+0000","last_fullsized":"2023-09-27T23:26:56.798184+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:06:40.797293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":289,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.818096+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:26:56.818096+0000","last_peered":"2023-09-27T23:26:56.818096+0000","last_clean":"2023-09-27T23:26:56.818096+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:26:56.818096+0000","last_undegraded":"2023-09-27T23:26:56.818096+0000","last_fullsized":"2023-09-27T23:26:56.818096+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":139,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798953+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:26:56.798953+0000","last_peered":"2023-09-27T23:26:56.798953+0000","last_clean":"2023-09-27T23:26:56.798953+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:26:56.798953+0000","last_undegraded":"2023-09-27T23:26:56.798953+0000","last_fullsized":"2023-09-27T23:26:56.798953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.831510+0000","last_change":"2023-09-27T23:26:56.822811+0000","last_active":"2023-09-27T23:26:56.831510+0000","last_peered":"2023-09-27T23:26:56.831510+0000","last_clean":"2023-09-27T23:26:56.831510+0000","last_became_active":"2023-09-27T23:26:56.822521+0000","last_became_peered":"2023-09-27T23:26:56.822521+0000","last_unstale":"2023-09-27T23:26:56.831510+0000","last_undegraded":"2023-09-27T23:26:56.831510+0000","last_fullsized":"2023-09-27T23:26:56.831510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:26:04.894120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.798523+0000","last_change":"2023-09-27T23:26:56.428852+0000","last_active":"2023-09-27T23:26:56.798523+0000","last_peered":"2023-09-27T23:26:56.798523+0000","last_clean":"2023-09-27T23:26:56.798523+0000","last_became_active":"2023-09-27T23:26:56.428486+0000","last_became_peered":"2023-09-27T23:26:56.428486+0000","last_unstale":"2023-09-27T23:26:56.798523+0000","last_undegraded":"2023-09-27T23:26:56.798523+0000","last_fullsized":"2023-09-27T23:26:56.798523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:23:54.572803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.817957+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:26:56.817957+0000","last_peered":"2023-09-27T23:26:56.817957+0000","last_clean":"2023-09-27T23:26:56.817957+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:26:56.817957+0000","last_undegraded":"2023-09-27T23:26:56.817957+0000","last_fullsized":"2023-09-27T23:26:56.817957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.816256+0000","last_change":"2023-09-27T23:26:56.813832+0000","last_active":"2023-09-27T23:26:56.816256+0000","last_peered":"2023-09-27T23:26:56.816256+0000","last_clean":"2023-09-27T23:26:56.816256+0000","last_became_active":"2023-09-27T23:26:56.813282+0000","last_became_peered":"2023-09-27T23:26:56.813282+0000","last_unstale":"2023-09-27T23:26:56.816256+0000","last_undegraded":"2023-09-27T23:26:56.816256+0000","last_fullsized":"2023-09-27T23:26:56.816256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:00:39.221936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823181+0000","last_change":"2023-09-27T23:26:56.428421+0000","last_active":"2023-09-27T23:26:56.823181+0000","last_peered":"2023-09-27T23:26:56.823181+0000","last_clean":"2023-09-27T23:26:56.823181+0000","last_became_active":"2023-09-27T23:26:56.428110+0000","last_became_peered":"2023-09-27T23:26:56.428110+0000","last_unstale":"2023-09-27T23:26:56.823181+0000","last_undegraded":"2023-09-27T23:26:56.823181+0000","last_fullsized":"2023-09-27T23:26:56.823181+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:17:03.284281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.787944+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:26:56.787944+0000","last_peered":"2023-09-27T23:26:56.787944+0000","last_clean":"2023-09-27T23:26:56.787944+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:26:56.787944+0000","last_undegraded":"2023-09-27T23:26:56.787944+0000","last_fullsized":"2023-09-27T23:26:56.787944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827987+0000","last_change":"2023-09-27T23:26:56.818239+0000","last_active":"2023-09-27T23:26:56.827987+0000","last_peered":"2023-09-27T23:26:56.827987+0000","last_clean":"2023-09-27T23:26:56.827987+0000","last_became_active":"2023-09-27T23:26:56.817954+0000","last_became_peered":"2023-09-27T23:26:56.817954+0000","last_unstale":"2023-09-27T23:26:56.827987+0000","last_undegraded":"2023-09-27T23:26:56.827987+0000","last_fullsized":"2023-09-27T23:26:56.827987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:57:30.548112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796489+0000","last_change":"2023-09-27T23:26:56.434807+0000","last_active":"2023-09-27T23:26:56.796489+0000","last_peered":"2023-09-27T23:26:56.796489+0000","last_clean":"2023-09-27T23:26:56.796489+0000","last_became_active":"2023-09-27T23:26:56.434527+0000","last_became_peered":"2023-09-27T23:26:56.434527+0000","last_unstale":"2023-09-27T23:26:56.796489+0000","last_undegraded":"2023-09-27T23:26:56.796489+0000","last_fullsized":"2023-09-27T23:26:56.796489+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:18:10.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.796677+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:26:56.796677+0000","last_peered":"2023-09-27T23:26:56.796677+0000","last_clean":"2023-09-27T23:26:56.796677+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:26:56.796677+0000","last_undegraded":"2023-09-27T23:26:56.796677+0000","last_fullsized":"2023-09-27T23:26:56.796677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.824711+0000","last_change":"2023-09-27T23:26:56.819491+0000","last_active":"2023-09-27T23:26:56.824711+0000","last_peered":"2023-09-27T23:26:56.824711+0000","last_clean":"2023-09-27T23:26:56.824711+0000","last_became_active":"2023-09-27T23:26:56.819213+0000","last_became_peered":"2023-09-27T23:26:56.819213+0000","last_unstale":"2023-09-27T23:26:56.824711+0000","last_undegraded":"2023-09-27T23:26:56.824711+0000","last_fullsized":"2023-09-27T23:26:56.824711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:27.419083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.827945+0000","last_change":"2023-09-27T23:26:56.434974+0000","last_active":"2023-09-27T23:26:56.827945+0000","last_peered":"2023-09-27T23:26:56.827945+0000","last_clean":"2023-09-27T23:26:56.827945+0000","last_became_active":"2023-09-27T23:26:56.434674+0000","last_became_peered":"2023-09-27T23:26:56.434674+0000","last_unstale":"2023-09-27T23:26:56.827945+0000","last_undegraded":"2023-09-27T23:26:56.827945+0000","last_fullsized":"2023-09-27T23:26:56.827945+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:43.049103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799914+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:26:56.799914+0000","last_peered":"2023-09-27T23:26:56.799914+0000","last_clean":"2023-09-27T23:26:56.799914+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:26:56.799914+0000","last_undegraded":"2023-09-27T23:26:56.799914+0000","last_fullsized":"2023-09-27T23:26:56.799914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.828995+0000","last_change":"2023-09-27T23:26:56.824329+0000","last_active":"2023-09-27T23:26:56.828995+0000","last_peered":"2023-09-27T23:26:56.828995+0000","last_clean":"2023-09-27T23:26:56.828995+0000","last_became_active":"2023-09-27T23:26:56.822836+0000","last_became_peered":"2023-09-27T23:26:56.822836+0000","last_unstale":"2023-09-27T23:26:56.828995+0000","last_undegraded":"2023-09-27T23:26:56.828995+0000","last_fullsized":"2023-09-27T23:26:56.828995+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:38.417239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.823413+0000","last_change":"2023-09-27T23:26:56.430433+0000","last_active":"2023-09-27T23:26:56.823413+0000","last_peered":"2023-09-27T23:26:56.823413+0000","last_clean":"2023-09-27T23:26:56.823413+0000","last_became_active":"2023-09-27T23:26:56.430173+0000","last_became_peered":"2023-09-27T23:26:56.430173+0000","last_unstale":"2023-09-27T23:26:56.823413+0000","last_undegraded":"2023-09-27T23:26:56.823413+0000","last_fullsized":"2023-09-27T23:26:56.823413+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:26.780386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.813850+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:26:56.813850+0000","last_peered":"2023-09-27T23:26:56.813850+0000","last_clean":"2023-09-27T23:26:56.813850+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:26:56.813850+0000","last_undegraded":"2023-09-27T23:26:56.813850+0000","last_fullsized":"2023-09-27T23:26:56.813850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.830579+0000","last_change":"2023-09-27T23:26:56.825211+0000","last_active":"2023-09-27T23:26:56.830579+0000","last_peered":"2023-09-27T23:26:56.830579+0000","last_clean":"2023-09-27T23:26:56.830579+0000","last_became_active":"2023-09-27T23:26:56.824718+0000","last_became_peered":"2023-09-27T23:26:56.824718+0000","last_unstale":"2023-09-27T23:26:56.830579+0000","last_undegraded":"2023-09-27T23:26:56.830579+0000","last_fullsized":"2023-09-27T23:26:56.830579+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:27.784899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.820537+0000","last_change":"2023-09-27T23:26:56.436836+0000","last_active":"2023-09-27T23:26:56.820537+0000","last_peered":"2023-09-27T23:26:56.820537+0000","last_clean":"2023-09-27T23:26:56.820537+0000","last_became_active":"2023-09-27T23:26:56.436287+0000","last_became_peered":"2023-09-27T23:26:56.436287+0000","last_unstale":"2023-09-27T23:26:56.820537+0000","last_undegraded":"2023-09-27T23:26:56.820537+0000","last_fullsized":"2023-09-27T23:26:56.820537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:41:26.414570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.799800+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:26:56.799800+0000","last_peered":"2023-09-27T23:26:56.799800+0000","last_clean":"2023-09-27T23:26:56.799800+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:26:56.799800+0000","last_undegraded":"2023-09-27T23:26:56.799800+0000","last_fullsized":"2023-09-27T23:26:56.799800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.821988+0000","last_change":"2023-09-27T23:26:56.816231+0000","last_active":"2023-09-27T23:26:56.821988+0000","last_peered":"2023-09-27T23:26:56.821988+0000","last_clean":"2023-09-27T23:26:56.821988+0000","last_became_active":"2023-09-27T23:26:56.815931+0000","last_became_peered":"2023-09-27T23:26:56.815931+0000","last_unstale":"2023-09-27T23:26:56.821988+0000","last_undegraded":"2023-09-27T23:26:56.821988+0000","last_fullsized":"2023-09-27T23:26:56.821988+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:56.392452+0000","last_clean_scrub_stamp":"2023-09-27T23:26:56.392452+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:38.823773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.829082+0000","last_change":"2023-09-27T23:26:56.424937+0000","last_active":"2023-09-27T23:26:56.829082+0000","last_peered":"2023-09-27T23:26:56.829082+0000","last_clean":"2023-09-27T23:26:56.829082+0000","last_became_active":"2023-09-27T23:26:56.423527+0000","last_became_peered":"2023-09-27T23:26:56.423527+0000","last_unstale":"2023-09-27T23:26:56.829082+0000","last_undegraded":"2023-09-27T23:26:56.829082+0000","last_fullsized":"2023-09-27T23:26:56.829082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:26:55.387939+0000","last_clean_scrub_stamp":"2023-09-27T23:26:55.387939+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:45.293801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-09-27T23:26:56.800752+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:26:56.800752+0000","last_peered":"2023-09-27T23:26:56.800752+0000","last_clean":"2023-09-27T23:26:56.800752+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:26:56.800752+0000","last_undegraded":"2023-09-27T23:26:56.800752+0000","last_fullsized":"2023-09-27T23:26:56.800752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509603,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24652,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":23232,"kb_avail":94347188,"statfs":{"total":96636764160,"available":96611520512,"internally_reserved":0,"allocated":1445888,"data_stored":1015142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23789568},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93999999999999995,"15min":0.92700000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2190000000000001,"5min":1.363,"15min":1.7949999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.92200000000000004,"15min":0.88900000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89100000000000001,"15min":0.90000000000000002},"min":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93300000000000005,"15min":0.92100000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":1.048},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.99299999999999999,"15min":0.96999999999999997},"min":{"1min":0.68000000000000005,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94699999999999995,"15min":0.94999999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2410000000000001,"5min":1.29,"15min":1.421},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.0800000000000001,"15min":1.048},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.622},"max":{"1min":1.546,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.94899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.90700000000000003,"15min":0.91200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.093},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.99399999999999999,"15min":0.99099999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.179}]},{"osd":5,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0389999999999999,"15min":1.052},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.4139999999999999,"5min":1.51,"15min":1.5880000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.105,"15min":1.1000000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.2390000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:26:55 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1339999999999999,"15min":1.129},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.1659999999999999,"15min":1.1399999999999999},"min":{"1min":0.878,"5min":0.85199999999999998,"15min":0.73899999999999999},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.288}]}]},{"osd":6,"up_from":15,"seq":64424509602,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20416,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":19584,"kb_avail":94351424,"statfs":{"total":96636764160,"available":96615858176,"internally_reserved":0,"allocated":843776,"data_stored":424614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20054016},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.68899999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.2070000000000001,"5min":1.327,"15min":1.327},"last":0.997},{"interface":"front","average":{"1min":1.079,"5min":1.026,"15min":1.0289999999999999},"min":{"1min":0.81999999999999995,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.649},"last":1.264}]},{"osd":1,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.54700000000000004},"max":{"1min":1.2889999999999999,"5min":1.323,"15min":1.323},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88,"15min":0.91000000000000003},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1739999999999999,"5min":1.2509999999999999,"15min":1.3939999999999999},"last":1.159}]},{"osd":2,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.88},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.56799999999999995},"max":{"1min":1.0820000000000001,"5min":1.353,"15min":1.397},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60499999999999998},"max":{"1min":1.2589999999999999,"5min":1.367,"15min":1.4259999999999999},"last":1.083}]},{"osd":3,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.58999999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.734},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97999999999999998,"15min":0.96799999999999997},"min":{"1min":0.84599999999999997,"5min":0.71099999999999997,"15min":0.61899999999999999},"max":{"1min":1.2290000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.1160000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93899999999999995,"15min":0.93100000000000005},"min":{"1min":0.64900000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":1.292,"5min":1.335,"15min":1.544},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.0640000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.61899999999999999},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.04}]},{"osd":5,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.24,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.204,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":1.0269999999999999,"5min":0.83999999999999997,"15min":0.65900000000000003},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.1830000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.167,"15min":1.165},"min":{"1min":0.92800000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3660000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2370000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30408,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":29568,"kb_avail":94341432,"statfs":{"total":96636764160,"available":96605626368,"internally_reserved":0,"allocated":851968,"data_stored":425238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30277632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.151,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.878,"15min":0.89600000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.2929999999999999,"15min":1.359},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.50700000000000001},"max":{"1min":1.2969999999999999,"5min":1.3109999999999999,"15min":1.526},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.88500000000000001,"15min":0.876},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.234,"15min":1.4179999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.93000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3859999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"max":{"1min":1.27,"5min":1.4730000000000001,"15min":1.5740000000000001},"last":0.628}]},{"osd":3,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91900000000000004,"15min":0.97199999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.214,"5min":1.238,"15min":1.5640000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.024,"15min":1.014},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3819999999999999,"15min":1.954},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1599999999999999,"5min":1.2829999999999999,"15min":1.5349999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.032,"15min":1.046},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.304,"5min":1.3939999999999999,"15min":1.704},"last":1.286}]},{"osd":6,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.02,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.323,"5min":1.5580000000000001,"15min":1.6319999999999999},"last":1},{"interface":"front","average":{"1min":1.046,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.3779999999999999,"15min":1.837},"last":1.1519999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:50 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1479999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3,"5min":1.496,"15min":1.7230000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.204,"15min":1.1950000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.4319999999999999,"5min":1.633,"15min":2.0529999999999999},"last":1.083}]}]},{"osd":4,"up_from":15,"seq":64424509603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20812,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":94351028,"statfs":{"total":96636764160,"available":96615452672,"internally_reserved":0,"allocated":856064,"data_stored":426150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.91100000000000003,"15min":0.88700000000000001},"min":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.53700000000000003},"max":{"1min":1.0529999999999999,"5min":1.27,"15min":1.3080000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.3320000000000001,"5min":1.45,"15min":1.45},"last":1.028}]},{"osd":1,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.181,"5min":1.2190000000000001,"15min":1.3919999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.95999999999999996,"15min":0.92400000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.139,"5min":1.2050000000000001,"15min":1.329},"last":0.84299999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91600000000000004,"15min":0.90900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93400000000000005,"15min":0.97099999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.153,"5min":1.3660000000000001,"15min":1.522},"last":1.25}]},{"osd":3,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.92300000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0449999999999999,"5min":1.26,"15min":1.373},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.02,"15min":1.0129999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2989999999999999,"5min":1.5129999999999999,"15min":1.575},"last":0.96399999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.92400000000000004,"15min":0.92000000000000004},"min":{"1min":0.67400000000000004,"5min":0.60199999999999998,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.409,"15min":1.627},"last":1.355},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99199999999999999,"15min":1.016},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4019999999999999,"5min":1.5489999999999999,"15min":2.2730000000000001},"last":1.2090000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.054,"15min":1.0489999999999999},"min":{"1min":0.752,"5min":0.63900000000000001,"15min":0.60099999999999998},"max":{"1min":1.369,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.153,"15min":1.1579999999999999},"min":{"1min":0.83399999999999996,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.415,"5min":1.478,"15min":1.8620000000000001},"last":1.5169999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1000000000000001,"15min":1.093},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.64000000000000001},"max":{"1min":1.3280000000000001,"5min":1.627,"15min":1.726},"last":1.282},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1659999999999999,"15min":1.1719999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4319999999999999,"5min":1.859,"15min":2.532},"last":1.323}]}]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21448,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94350392,"statfs":{"total":96636764160,"available":96614801408,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.30199999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.746},"min":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.35299999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.337},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.51300000000000001,"15min":0.39800000000000002},"max":{"1min":0.95299999999999996,"5min":1.224,"15min":1.2869999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.72199999999999998,"5min":0.61699999999999999,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.3180000000000001,"15min":1.3939999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.88300000000000001,"15min":0.85699999999999998},"min":{"1min":0.78200000000000003,"5min":0.65700000000000003,"15min":0.36699999999999999},"max":{"1min":1.1479999999999999,"5min":1.288,"15min":1.353},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91600000000000004},"min":{"1min":0.65500000000000003,"5min":0.61099999999999999,"15min":0.45700000000000002},"max":{"1min":1.256,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.76000000000000001,"5min":0.67700000000000005,"15min":0.66100000000000003},"max":{"1min":1.175,"5min":1.466,"15min":1.5349999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.628},"max":{"1min":1.2030000000000001,"5min":1.3680000000000001,"15min":1.4950000000000001},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.91300000000000003,"5min":0.83099999999999996,"15min":0.626},"max":{"1min":1.3120000000000001,"5min":1.468,"15min":1.571},"last":1.087},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1479999999999999},"min":{"1min":0.96399999999999997,"5min":0.76700000000000002,"15min":0.58399999999999996},"max":{"1min":1.47,"5min":1.51,"15min":1.764},"last":1.05}]},{"osd":6,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.119,"15min":1.137},"min":{"1min":0.92600000000000005,"5min":0.76900000000000002,"15min":0.72799999999999998},"max":{"1min":1.407,"5min":1.4610000000000001,"15min":1.6970000000000001},"last":1.111},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":1.0269999999999999,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.3140000000000001,"5min":1.5800000000000001,"15min":1.8620000000000001},"last":1.2290000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:26:03 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.246,"15min":1.2290000000000001},"min":{"1min":1.0289999999999999,"5min":0.86799999999999999,"15min":0.78700000000000003},"max":{"1min":1.5640000000000001,"5min":1.627,"15min":1.655},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.2},"min":{"1min":1.054,"5min":0.89500000000000002,"15min":0.73099999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.671},"last":1.0660000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509603,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20040,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":19200,"kb_avail":94351800,"statfs":{"total":96636764160,"available":96616243200,"internally_reserved":0,"allocated":851968,"data_stored":424670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19660800},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.68700000000000006},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.39700000000000002},"max":{"1min":0.85799999999999998,"5min":1.133,"15min":1.7370000000000001},"last":1.093},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.214,"15min":1.6080000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.79500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":0.97299999999999998,"5min":1.1950000000000001,"15min":1.8069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.82799999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.3120000000000001,"15min":1.651},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.4850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.58099999999999996},"max":{"1min":1.2350000000000001,"5min":1.3580000000000001,"15min":1.8280000000000001},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0109999999999999,"15min":1.034},"min":{"1min":0.751,"5min":0.751,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":1.585,"15min":1.7849999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0629999999999999,"15min":1.097},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69499999999999995},"max":{"1min":1.337,"5min":1.4590000000000001,"15min":2.601},"last":0.91900000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.024,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.77400000000000002,"15min":0.71099999999999997},"max":{"1min":1.3029999999999999,"5min":1.403,"15min":2.7320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.107,"5min":1.1299999999999999,"15min":1.1559999999999999},"min":{"1min":0.84999999999999998,"5min":0.76700000000000002,"15min":0.747},"max":{"1min":1.375,"5min":1.496,"15min":2.6760000000000002},"last":1.5149999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1579999999999999,"15min":1.1910000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6560000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1559999999999999,"15min":1.1779999999999999},"min":{"1min":0.871,"5min":0.84099999999999997,"15min":0.71699999999999997},"max":{"1min":1.417,"5min":1.552,"15min":2.6230000000000002},"last":1.377}]},{"osd":7,"last update":"Wed Sep 27 23:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.218,"15min":1.2490000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.76600000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80000000000000004},"max":{"1min":1.3999999999999999,"5min":1.6200000000000001,"15min":2.7749999999999999},"last":1.282}]}]},{"osd":1,"up_from":14,"seq":60129542309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35140,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":34304,"kb_avail":94336700,"statfs":{"total":96636764160,"available":96600780800,"internally_reserved":0,"allocated":847872,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35127296},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.503,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.3560000000000001,"15min":3.798},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80000000000000004,"15min":0.82799999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":1.151,"15min":2.3330000000000002},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.27500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":6.157},"last":1.179},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.879,"15min":0.97499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.246,"5min":1.2609999999999999,"15min":5.649},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1919999999999999,"5min":1.323,"15min":6.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.294,"15min":1.9370000000000001},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.173,"5min":1.3340000000000001,"15min":1.863},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.133,"15min":1.1579999999999999},"min":{"1min":0.749,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.381,"5min":1.579,"15min":4.0499999999999998},"last":1.294}]},{"osd":5,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.65100000000000002},"max":{"1min":1.476,"5min":1.524,"15min":2.7040000000000002},"last":0.999},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.0960000000000001},"min":{"1min":0.80900000000000005,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.365,"5min":1.53,"15min":2.2509999999999999},"last":1.365}]},{"osd":6,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.234,"15min":1.2470000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.587,"5min":1.6919999999999999,"15min":3.9260000000000002},"last":1.22},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1910000000000001,"15min":1.1890000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.76400000000000001},"max":{"1min":1.556,"5min":1.5900000000000001,"15min":2.79},"last":1.127}]},{"osd":7,"last update":"Wed Sep 27 23:26:59 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.218},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.64800000000000002},"max":{"1min":1.532,"5min":1.7210000000000001,"15min":2.843},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.429,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.429}]}]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21588,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":20160,"kb_avail":94350252,"statfs":{"total":96636764160,"available":96614658048,"internally_reserved":0,"allocated":1454080,"data_stored":1017072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20643840},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65600000000000003,"15min":0.67300000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.002,"5min":1.002,"15min":1.901},"last":1.002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79300000000000004,"15min":0.78600000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2789999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.2150000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.335},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.84999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.216,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88800000000000001,"15min":0.86799999999999999},"min":{"1min":0.66300000000000003,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.089,"5min":1.282,"15min":1.3400000000000001},"last":0.77600000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.995,"15min":1.026},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.59199999999999997},"max":{"1min":1.2549999999999999,"5min":1.4590000000000001,"15min":2.2519999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0580000000000001},"min":{"1min":0.754,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.276,"5min":1.5589999999999999,"15min":1.952},"last":0.89900000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1020000000000001,"15min":1.0960000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.71899999999999997},"max":{"1min":1.5009999999999999,"5min":1.78,"15min":1.986},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.137,"15min":1.147},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.5509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.163,"15min":1.163},"min":{"1min":0.90800000000000003,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.3979999999999999,"5min":1.4750000000000001,"15min":1.7729999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.224,"15min":1.2110000000000001},"min":{"1min":0.93300000000000005,"5min":0.90800000000000003,"15min":0.61199999999999999},"max":{"1min":1.73,"5min":1.8300000000000001,"15min":2.1850000000000001},"last":1.373}]},{"osd":7,"last update":"Wed Sep 27 23:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.224,"15min":1.232},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.222,"15min":1.234},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.66400000000000003},"max":{"1min":1.554,"5min":1.554,"15min":2.3540000000000001},"last":1.4490000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:27:05.557 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:27:05.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:27:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.679+0000 7f0495c56700 1 Processor -- start 2023-09-27T23:27:05.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.679+0000 7f0495c56700 1 -- start start 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.679+0000 7f0495c56700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f04900ce2d0 0x7f04900ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.679+0000 7f0495c56700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04900cbbb0 con 0x7f04900cc040 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.679+0000 7f0495c56700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049006dc90 con 0x7f04900caa10 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.679+0000 7f0495c56700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04900cafa0 con 0x7f04900ce2d0 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495455700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04900cc040 0x7f049006a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48698/0 (socket says 172.21.15.64:48698) 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495455700 1 -- 172.21.15.64:0/3120046116 learned_addr learned my addr 172.21.15.64:0/3120046116 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:05.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0494c54700 1 --2- 172.21.15.64:0/3120046116 >> v2:172.21.15.123:3300/0 conn(0x7f04900ce2d0 0x7f04900ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079006137 0 0) 0x7f04900cbbb0 con 0x7f04900cc040 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f047c003570 con 0x7f04900cc040 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791901865 0 0) 0x7f049006dc90 con 0x7f04900caa10 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04900cbbb0 con 0x7f04900caa10 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2142354347 0 0) 0x7f047c003570 con 0x7f04900cc040 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 >> v2:172.21.15.123:3300/0 conn(0x7f04900ce2d0 msgr2=0x7f04900ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 --2- 172.21.15.64:0/3120046116 >> v2:172.21.15.123:3300/0 conn(0x7f04900ce2d0 0x7f04900ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 >> v1:172.21.15.123:6789/0 conn(0x7f04900caa10 legacy=0x7f0490158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04900cb280 con 0x7f04900cc040 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f04880017f0 con 0x7f04900cc040 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 >> v1:172.21.15.64:6789/0 conn(0x7f04900cc040 legacy=0x7f049006a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c003bf0 0x7f047c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c004810 0x7f047c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c0051c0 0x7f047c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04900cafa0 con 0x7f047c0051c0 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c00c0b0 con 0x7f047c003bf0 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c00c270 con 0x7f047c004810 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495455700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c0051c0 0x7f047c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c004810 0x7f047c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0494c54700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c003bf0 0x7f047c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 -- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c003bf0 msgr2=0x7f047c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c003bf0 0x7f047c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 -- 172.21.15.64:0/3120046116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c0051c0 msgr2=0x7f047c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:05.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c0051c0 0x7f047c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 -- 172.21.15.64:0/3120046116 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c003570 con 0x7f047c004810 2023-09-27T23:27:05.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495455700 1 -- 172.21.15.64:0/3120046116 reap_dead start 2023-09-27T23:27:05.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048ffff700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c004810 0x7f047c004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0480001a10 tx=0x7f0480007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a18d5b9a0be44 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:05.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000c5d0 con 0x7f047c004810 2023-09-27T23:27:05.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048000c730 con 0x7f047c004810 2023-09-27T23:27:05.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f048f7fe700 1 -- 172.21.15.64:0/3120046116 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000cad0 con 0x7f047c004810 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495c56700 1 -- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c004810 msgr2=0x7f047c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495c56700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c004810 0x7f047c004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0480001a10 tx=0x7f0480007d30 comp rx=0 tx=0).stop 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495c56700 1 -- 172.21.15.64:0/3120046116 shutdown_connections 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495c56700 1 --2- 172.21.15.64:0/3120046116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c004810 0x7f047c004c80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.683+0000 7f0495c56700 1 -- 172.21.15.64:0/3120046116 >> 172.21.15.64:0/3120046116 conn(0x7f0490067bb0 msgr2=0x7f04900cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 -- 172.21.15.64:0/3120046116 shutdown_connections 2023-09-27T23:27:05.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 -- 172.21.15.64:0/3120046116 wait complete. 2023-09-27T23:27:05.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 Processor -- start 2023-09-27T23:27:05.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 -- start start 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 0x7f049014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f049014f8d0 0x7f04900b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049015ab60 0x7f049015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04900c9b50 con 0x7f047c004810 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049015b510 con 0x7f049015ab60 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0495c56700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049016c8e0 con 0x7f049014f8d0 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0494c54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 0x7f049014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0494c54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 0x7f049014f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52748/0 (socket says 172.21.15.64:52748) 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.687+0000 7f0494c54700 1 -- 172.21.15.64:0/927462706 learned_addr learned my addr 172.21.15.64:0/927462706 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0495455700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049015ab60 0x7f049015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f048ffff700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f049014f8d0 0x7f04900b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0494c54700 1 -- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f049014f8d0 msgr2=0x7f04900b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0494c54700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f049014f8d0 0x7f04900b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0494c54700 1 -- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049015ab60 msgr2=0x7f049015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0494c54700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049015ab60 0x7f049015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0494c54700 1 -- 172.21.15.64:0/927462706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049016cbc0 con 0x7f047c004810 2023-09-27T23:27:05.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f048ffff700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f049014f8d0 0x7f04900b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:05.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0494c54700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 0x7f049014f040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0478011d70 tx=0x7f04780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:05.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800fac0 con 0x7f047c004810 2023-09-27T23:27:05.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047800fc20 con 0x7f047c004810 2023-09-27T23:27:05.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049016cea0 con 0x7f047c004810 2023-09-27T23:27:05.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478014440 con 0x7f047c004810 2023-09-27T23:27:05.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049016d460 con 0x7f047c004810 2023-09-27T23:27:05.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.691+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049004be40 con 0x7f047c004810 2023-09-27T23:27:05.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.695+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0478014870 con 0x7f047c004810 2023-09-27T23:27:05.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.695+0000 7f048cff9700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0470060aa0 0x7f0470062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.695+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0478018030 con 0x7f047c004810 2023-09-27T23:27:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.695+0000 7f048ffff700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0470060aa0 0x7f0470062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:05.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.695+0000 7f048ffff700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0470060aa0 0x7f0470062f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0490156c20 tx=0x7f0480015000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:05.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.699+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f047804dad0 con 0x7f047c004810 2023-09-27T23:27:05.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.919+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f04900cb560 con 0x7f047c004810 2023-09-27T23:27:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.927+0000 7f048cff9700 1 -- 172.21.15.64:0/927462706 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0478022020 con 0x7f047c004810 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0470060aa0 msgr2=0x7f0470062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0470060aa0 0x7f0470062f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0490156c20 tx=0x7f0480015000 comp rx=0 tx=0).stop 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 msgr2=0x7f049014f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 0x7f049014f040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0478011d70 tx=0x7f04780091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 shutdown_connections 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0470060aa0 0x7f0470062f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049015ab60 0x7f049015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f049014f8d0 0x7f04900b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 --2- 172.21.15.64:0/927462706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f047c004810 0x7f049014f040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 >> 172.21.15.64:0/927462706 conn(0x7f0490067bb0 msgr2=0x7f049015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 shutdown_connections 2023-09-27T23:27:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:05.931+0000 7f0495c56700 1 -- 172.21.15.64:0/927462706 wait complete. 2023-09-27T23:27:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:27:05.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:27:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.055+0000 7f3f56003700 1 Processor -- start 2023-09-27T23:27:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.055+0000 7f3f56003700 1 -- start start 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f56003700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f50155350 0x7f3f50159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f56003700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f500ab960 0x7f3f5006b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f56003700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f50155720 0x7f3f5006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f56003700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5015a010 con 0x7f3f50155720 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f56003700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f500b0080 con 0x7f3f500ab960 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f56003700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5006e740 con 0x7f3f50155350 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f50155720 0x7f3f5006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f50155720 0x7f3f5006b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52752/0 (socket says 172.21.15.64:52752) 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 -- 172.21.15.64:0/2042827215 learned_addr learned my addr 172.21.15.64:0/2042827215 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4f7fe700 1 --2- 172.21.15.64:0/2042827215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f50155350 0x7f3f50159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4effd700 1 --2- 172.21.15.64:0/2042827215 >> v2:172.21.15.123:3300/0 conn(0x7f3f500ab960 0x7f3f5006b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 -- 172.21.15.64:0/2042827215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f50155350 msgr2=0x7f3f50159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 --2- 172.21.15.64:0/2042827215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f50155350 0x7f3f50159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 -- 172.21.15.64:0/2042827215 >> v2:172.21.15.123:3300/0 conn(0x7f3f500ab960 msgr2=0x7f3f5006b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 --2- 172.21.15.64:0/2042827215 >> v2:172.21.15.123:3300/0 conn(0x7f3f500ab960 0x7f3f5006b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 -- 172.21.15.64:0/2042827215 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f500af470 con 0x7f3f50155720 2023-09-27T23:27:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4effd700 1 --2- 172.21.15.64:0/2042827215 >> v2:172.21.15.123:3300/0 conn(0x7f3f500ab960 0x7f3f5006b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:06.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4ffff700 1 --2- 172.21.15.64:0/2042827215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155720 0x7f3f5006b6e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3f5006abb0 tx=0x7f3f4800a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b123a664728c98e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4e7fc700 1 -- 172.21.15.64:0/2042827215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800f070 con 0x7f3f50155720 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4e7fc700 1 -- 172.21.15.64:0/2042827215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800a2e0 con 0x7f3f50155720 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.059+0000 7f3f4e7fc700 1 -- 172.21.15.64:0/2042827215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800cd50 con 0x7f3f50155720 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 -- 172.21.15.64:0/2042827215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155720 msgr2=0x7f3f5006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 --2- 172.21.15.64:0/2042827215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155720 0x7f3f5006b6e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3f5006abb0 tx=0x7f3f4800a950 comp rx=0 tx=0).stop 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 -- 172.21.15.64:0/2042827215 shutdown_connections 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 --2- 172.21.15.64:0/2042827215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155720 0x7f3f5006b6e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 --2- 172.21.15.64:0/2042827215 >> v2:172.21.15.123:3300/0 conn(0x7f3f500ab960 0x7f3f5006b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 --2- 172.21.15.64:0/2042827215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f50155350 0x7f3f50159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 -- 172.21.15.64:0/2042827215 >> 172.21.15.64:0/2042827215 conn(0x7f3f50067410 msgr2=0x7f3f500677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 -- 172.21.15.64:0/2042827215 shutdown_connections 2023-09-27T23:27:06.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 -- 172.21.15.64:0/2042827215 wait complete. 2023-09-27T23:27:06.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 Processor -- start 2023-09-27T23:27:06.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.063+0000 7f3f56003700 1 -- start start 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f500ab960 0x7f3f50140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 0x7f3f50165920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f50155720 0x7f3f50152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500a5700 con 0x7f3f50155350 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f501406a0 con 0x7f3f50155720 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50166e60 con 0x7f3f500ab960 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 0x7f3f50165920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 0x7f3f50165920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52756/0 (socket says 172.21.15.64:52756) 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 -- 172.21.15.64:0/1675919831 learned_addr learned my addr 172.21.15.64:0/1675919831 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4f7fe700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f500ab960 0x7f3f50140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4ffff700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f50155720 0x7f3f50152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 -- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f500ab960 msgr2=0x7f3f50140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f500ab960 0x7f3f50140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 -- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f50155720 msgr2=0x7f3f50152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f50155720 0x7f3f50152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 -- 172.21.15.64:0/1675919831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4800abf0 con 0x7f3f50155350 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4ffff700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f50155720 0x7f3f50152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:06.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f4effd700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 0x7f3f50165920 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400e8d0 tx=0x7f3f44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44013410 con 0x7f3f50155350 2023-09-27T23:27:06.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44013570 con 0x7f3f50155350 2023-09-27T23:27:06.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f50165e60 con 0x7f3f50155350 2023-09-27T23:27:06.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44013910 con 0x7f3f50155350 2023-09-27T23:27:06.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.067+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f50166530 con 0x7f3f50155350 2023-09-27T23:27:06.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.071+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f50071120 con 0x7f3f50155350 2023-09-27T23:27:06.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.071+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3f44013d00 con 0x7f3f50155350 2023-09-27T23:27:06.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.071+0000 7f3f377fe700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f3c060aa0 0x7f3f3c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.071+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3f4407cef0 con 0x7f3f50155350 2023-09-27T23:27:06.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.071+0000 7f3f4f7fe700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f3c060aa0 0x7f3f3c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.071+0000 7f3f4f7fe700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f3c060aa0 0x7f3f3c062f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f500c8de0 tx=0x7f3f400091a0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.079+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3f4404ce10 con 0x7f3f50155350 2023-09-27T23:27:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f50166fa0 con 0x7f3f50155350 2023-09-27T23:27:06.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f377fe700 1 -- 172.21.15.64:0/1675919831 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f3f44020090 con 0x7f3f50155350 2023-09-27T23:27:06.277 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:06.277 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:26:56.779553+0000","modified":"2023-09-27T23:26:59.790212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2178143597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2178143597},{"type":"v1","addr":"172.21.15.64:6835","nonce":2178143597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-09-27T23:27:06.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f3c060aa0 msgr2=0x7f3f3c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f3c060aa0 0x7f3f3c062f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f500c8de0 tx=0x7f3f400091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 msgr2=0x7f3f50165920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 0x7f3f50165920 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400e8d0 tx=0x7f3f44008040 comp rx=0 tx=0).stop 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 shutdown_connections 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3f3c060aa0 0x7f3f3c062f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f50155720 0x7f3f50152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f50155350 0x7f3f50165920 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 --2- 172.21.15.64:0/1675919831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f500ab960 0x7f3f50140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 >> 172.21.15.64:0/1675919831 conn(0x7f3f50067410 msgr2=0x7f3f501583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 shutdown_connections 2023-09-27T23:27:06.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.275+0000 7f3f56003700 1 -- 172.21.15.64:0/1675919831 wait complete. 2023-09-27T23:27:06.281 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-09-27T23:27:06.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:26:56.779553+0000', 'modified': '2023-09-27T23:26:59.790212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6096}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6096': {'gid': 6096, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/2178143597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 2178143597}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 2178143597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:27:06.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:27:06.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-27T23:27:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 Processor -- start 2023-09-27T23:27:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 -- start start 2023-09-27T23:27:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80ab910 0x7f8cc8159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8cc80abce0 0x7f8cc80c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80c1910 con 0x7f8cc80670e0 2023-09-27T23:27:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc8159cf0 con 0x7f8cc80abce0 2023-09-27T23:27:06.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccf196700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8159fd0 con 0x7f8cc80ab910 2023-09-27T23:27:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8cc80abce0 0x7f8cc80c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8cc80abce0 0x7f8cc80c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52758/0 (socket says 172.21.15.64:52758) 2023-09-27T23:27:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce194700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80ab910 0x7f8cc8159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccd993700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8cc80670e0 0x7f8cc80cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:56816/0 (socket says 172.21.15.64:56816) 2023-09-27T23:27:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 -- 172.21.15.64:0/3302173297 learned_addr learned my addr 172.21.15.64:0/3302173297 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 -- 172.21.15.64:0/3302173297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80ab910 msgr2=0x7f8cc8159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 --2- 172.21.15.64:0/3302173297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80ab910 0x7f8cc8159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 -- 172.21.15.64:0/3302173297 >> v1:172.21.15.123:6789/0 conn(0x7f8cc80670e0 legacy=0x7f8cc80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 -- 172.21.15.64:0/3302173297 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc815a2b0 con 0x7f8cc80abce0 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8cce995700 1 --2- 172.21.15.64:0/3302173297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc80c2f80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00099b0 tx=0x7f8cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a71a62a2d1b857 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccd192700 1 -- 172.21.15.64:0/3302173297 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0009ea0 con 0x7f8cc80abce0 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccd192700 1 -- 172.21.15.64:0/3302173297 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000b860 con 0x7f8cc80abce0 2023-09-27T23:27:06.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.815+0000 7f8ccd192700 1 -- 172.21.15.64:0/3302173297 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000bc00 con 0x7f8cc80abce0 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- 172.21.15.64:0/3302173297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 msgr2=0x7f8cc80c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3302173297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc80c2f80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00099b0 tx=0x7f8cc0009250 comp rx=0 tx=0).stop 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- 172.21.15.64:0/3302173297 shutdown_connections 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3302173297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc80c2f80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3302173297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80ab910 0x7f8cc8159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- 172.21.15.64:0/3302173297 >> 172.21.15.64:0/3302173297 conn(0x7f8cc806c080 msgr2=0x7f8cc80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- 172.21.15.64:0/3302173297 shutdown_connections 2023-09-27T23:27:06.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- 172.21.15.64:0/3302173297 wait complete. 2023-09-27T23:27:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 Processor -- start 2023-09-27T23:27:06.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- start start 2023-09-27T23:27:06.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80670e0 0x7f8cc80af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cc80ab910 0x7f8cc813b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc81659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80cadf0 con 0x7f8cc80abce0 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80b26b0 con 0x7f8cc80ab910 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccf196700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80b20f0 con 0x7f8cc80670e0 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8cce995700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc81659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8cce995700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc81659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52770/0 (socket says 172.21.15.64:52770) 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8cce995700 1 -- 172.21.15.64:0/3514972651 learned_addr learned my addr 172.21.15.64:0/3514972651 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8cce194700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80670e0 0x7f8cc80af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.819+0000 7f8ccd993700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cc80ab910 0x7f8cc813b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce995700 1 -- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80670e0 msgr2=0x7f8cc80af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce995700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80670e0 0x7f8cc80af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce995700 1 -- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cc80ab910 msgr2=0x7f8cc813b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce995700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cc80ab910 0x7f8cc813b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce995700 1 -- 172.21.15.64:0/3514972651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00094f0 con 0x7f8cc80abce0 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce194700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80670e0 0x7f8cc80af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8ccd993700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cc80ab910 0x7f8cc813b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cce995700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc81659f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ca600 tx=0x7f8cc0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cc67fc700 1 -- 172.21.15.64:0/3514972651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0025440 con 0x7f8cc80abce0 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cc67fc700 1 -- 172.21.15.64:0/3514972651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00255c0 con 0x7f8cc80abce0 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8cc67fc700 1 -- 172.21.15.64:0/3514972651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0025980 con 0x7f8cc80abce0 2023-09-27T23:27:06.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc80b23d0 con 0x7f8cc80abce0 2023-09-27T23:27:06.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.823+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc813b9a0 con 0x7f8cc80abce0 2023-09-27T23:27:06.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cc67fc700 1 -- 172.21.15.64:0/3514972651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8cc0011420 con 0x7f8cc80abce0 2023-09-27T23:27:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cc67fc700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ca8060a70 0x7f8ca8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cc67fc700 1 -- 172.21.15.64:0/3514972651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8cc007f3f0 con 0x7f8cc80abce0 2023-09-27T23:27:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce194700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ca8060a70 0x7f8ca8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 Processor -- start 2023-09-27T23:27:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 -- start start 2023-09-27T23:27:06.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce194700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ca8060a70 0x7f8ca8062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00113e0 tx=0x7f8cb00091a0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xc55450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3710 0xc55990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc55ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xb7ef30 con 0xa1e790 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xb7aed0 con 0x9d3ae0 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xb410b0 con 0x9d3710 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc55ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc55ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52776/0 (socket says 172.21.15.64:52776) 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce194700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xc55450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8ccd993700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3710 0xc55990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 -- 172.21.15.64:0/428897816 learned_addr learned my addr 172.21.15.64:0/428897816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 -- 172.21.15.64:0/428897816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3710 msgr2=0xc55990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3710 0xc55990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 -- 172.21.15.64:0/428897816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 msgr2=0xc55450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xc55450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.827+0000 7f8cce995700 1 -- 172.21.15.64:0/428897816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xa1f9e0 con 0xa1e790 2023-09-27T23:27:06.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cce995700 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc55ed0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00530f0 tx=0x7f8cc0006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeba4d9dd7a98e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cbf7fe700 1 -- 172.21.15.64:0/428897816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0058050 con 0xa1e790 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cbf7fe700 1 -- 172.21.15.64:0/428897816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00563f0 con 0xa1e790 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cbf7fe700 1 -- 172.21.15.64:0/428897816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0053920 con 0xa1e790 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 -- 172.21.15.64:0/428897816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 msgr2=0xc55ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc55ed0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00530f0 tx=0x7f8cc0006c40 comp rx=0 tx=0).stop 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 -- 172.21.15.64:0/428897816 shutdown_connections 2023-09-27T23:27:06.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc55ed0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3710 0xc55990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/428897816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xc55450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 -- 172.21.15.64:0/428897816 >> 172.21.15.64:0/428897816 conn(0x9d3eb0 msgr2=0xc0d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 -- 172.21.15.64:0/428897816 shutdown_connections 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 -- 172.21.15.64:0/428897816 wait complete. 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 -- start start 2023-09-27T23:27:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.831+0000 7f8cd26a1740 1 Processor -- start 2023-09-27T23:27:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3710 0xc35c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc36170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xb40d50 con 0xa1e790 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xb19970 con 0x9d3710 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xb81290 con 0x9d3ae0 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 10 client.0 _ll_register_callbacks cb 0xbc4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52788/0 (socket says 172.21.15.64:52788) 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 -- 172.21.15.64:0/4157990991 learned_addr learned my addr 172.21.15.64:0/4157990991 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8ccd993700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc36170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce194700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3710 0xc35c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 -- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 msgr2=0xc36170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc36170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 -- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3710 msgr2=0xc35c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3710 0xc35c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 -- 172.21.15.64:0/4157990991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc813b9a0 con 0xa1e790 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce995700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc366b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8cc007d960 tx=0x7f8cc0065000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738c8e5fd37e0685 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 1 -- 172.21.15.64:0/4157990991 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc005f070 con 0xa1e790 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 1 -- 172.21.15.64:0/4157990991 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8cc004f870 con 0xa1e790 2023-09-27T23:27:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xb19cc0 con 0xa1e790 2023-09-27T23:27:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 1 -- 172.21.15.64:0/4157990991 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc004fa30 con 0xa1e790 2023-09-27T23:27:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 10 client.6258 fetch_fsmap learned FSMap version 21 2023-09-27T23:27:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0xb49370 con 0xa1e790 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 1 -- 172.21.15.64:0/4157990991 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc006c020 con 0xa1e790 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 1 -- 172.21.15.64:0/4157990991 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8cc002d390 con 0xa1e790 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 10 client.6258 fetch_fsmap finished waiting for FSMap version 21 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 10 client.6258 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 20 client.6258 populate_metadata read hostname 'smithi064' 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc61cb0 0xc64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 4 client.6258 mds_command: new command op to 6096 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 --> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xb1d3e0 con 0xc61cb0 2023-09-27T23:27:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce194700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc61cb0 0xc64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cbcff9700 10 client.6258 ms_handle_connect on v2:172.21.15.64:6834/2178143597 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.835+0000 7f8cce194700 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc61cb0 0xc64090 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1469ca74096767c server_cookie=2409f87705075faa in_seq=0 out_seq=0 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cbcff9700 1 -- 172.21.15.64:0/4157990991 <== mds.0 v2:172.21.15.64:6834/2178143597 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0xb1d3e0 con 0xc61cb0 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cbcff9700 10 client.6258 handle_command_reply: tid=0 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 client.6258 shutdown 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 msgr2=0xc366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc366b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8cc007d960 tx=0x7f8cc0065000 comp rx=0 tx=0).stop 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 shutdown_connections 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc61cb0 0xc64090 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xa1e790 0xc366b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc36170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/4157990991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3710 0xc35c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cbcff9700 0 client.6258 ms_handle_reset on v2:172.21.15.64:6834/2178143597 2023-09-27T23:27:06.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 >> 172.21.15.64:0/4157990991 conn(0x9d3eb0 msgr2=0xc09230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 shutdown_connections 2023-09-27T23:27:06.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 1 -- 172.21.15.64:0/4157990991 wait complete. 2023-09-27T23:27:06.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.839+0000 7f8cd26a1740 20 client.6258 trim_cache size 0 max 16384 2023-09-27T23:27:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 Processor -- start 2023-09-27T23:27:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 -- start start 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xc53830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xa1e790 0xc0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xb4f610 con 0x9d3710 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xb7fef0 con 0xa1e790 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xb4ef40 con 0x9d3ae0 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xc53830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xc53830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52798/0 (socket says 172.21.15.64:52798) 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 -- 172.21.15.64:0/797439363 learned_addr learned my addr 172.21.15.64:0/797439363 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8ccd993700 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce995700 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xa1e790 0xc0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 -- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 msgr2=0xc53d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc53d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 -- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xa1e790 msgr2=0xc0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xa1e790 0xc0b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce194700 1 -- 172.21.15.64:0/797439363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8166290 con 0x9d3710 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.859+0000 7f8cce995700 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xa1e790 0xc0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xc53830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0xb3f0b0 tx=0x7f8cb0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf3ce5ed717b691 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cbffff700 1 -- 172.21.15.64:0/797439363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0026040 con 0x9d3710 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cbffff700 1 -- 172.21.15.64:0/797439363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00231a0 con 0x9d3710 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cbffff700 1 -- 172.21.15.64:0/797439363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002c6a0 con 0x9d3710 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- 172.21.15.64:0/797439363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 msgr2=0xc53830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xc53830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0xb3f0b0 tx=0x7f8cb0016000 comp rx=0 tx=0).stop 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- 172.21.15.64:0/797439363 shutdown_connections 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xa1e790 0xc0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x9d3ae0 0xc53d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/797439363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xc53830 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- 172.21.15.64:0/797439363 >> 172.21.15.64:0/797439363 conn(0x9d3eb0 msgr2=0xc08770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- 172.21.15.64:0/797439363 shutdown_connections 2023-09-27T23:27:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- 172.21.15.64:0/797439363 wait complete. 2023-09-27T23:27:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- start start 2023-09-27T23:27:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 Processor -- start 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xcae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xcaebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xa1e790 0xc0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xa50650 con 0x9d3710 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xb50550 con 0x9d3ae0 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xb7e9d0 con 0xa1e790 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cd26a1740 10 client.0 _ll_register_callbacks cb 0xa970e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xcae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xcae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52812/0 (socket says 172.21.15.64:52812) 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 -- 172.21.15.64:0/1119654611 learned_addr learned my addr 172.21.15.64:0/1119654611 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce995700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xa1e790 0xc0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8ccd993700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xcaebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 -- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xa1e790 msgr2=0xc0a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xa1e790 0xc0a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 -- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 msgr2=0xcaebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xcaebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.863+0000 7f8cce194700 1 -- 172.21.15.64:0/1119654611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb0014630 con 0x9d3710 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cce194700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xcae680 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0016040 tx=0x7f8cb0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abb82a5ce0f2378 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 1 -- 172.21.15.64:0/1119654611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0026070 con 0x9d3710 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 1 -- 172.21.15.64:0/1119654611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8cb001a040 con 0x9d3710 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cc00563f0 con 0x9d3710 2023-09-27T23:27:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 1 -- 172.21.15.64:0/1119654611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cb0039070 con 0x9d3710 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 10 client.6264 fetch_fsmap learned FSMap version 21 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0xb2fd80 con 0x9d3710 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 1 -- 172.21.15.64:0/1119654611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00563f0 con 0x9d3710 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 1 -- 172.21.15.64:0/1119654611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8cb0032020 con 0x9d3710 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 10 client.6264 fetch_fsmap finished waiting for FSMap version 21 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 10 client.6264 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 20 client.6264 populate_metadata read hostname 'smithi064' 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc0abc0 0xc0af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 4 client.6264 mds_command: new command op to 6096 tid=0[{"prefix": "status"}] 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 --> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] -- command(tid 0: {"prefix": "status"}) v1 -- 0xb4f410 con 0xc0abc0 2023-09-27T23:27:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8ccd993700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc0abc0 0xc0af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 10 client.6264 ms_handle_connect on v2:172.21.15.64:6834/2178143597 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8ccd993700 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc0abc0 0xc0af90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5150443d14204aa3 server_cookie=f8676ef63bb207b1 in_seq=0 out_seq=0 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 1 -- 172.21.15.64:0/1119654611 <== mds.0 v2:172.21.15.64:6834/2178143597 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0xb4f410 con 0xc0abc0 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 10 client.6264 handle_command_reply: tid=0 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 client.6264 shutdown 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 msgr2=0xcae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xcae680 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0016040 tx=0x7f8cb0031000 comp rx=0 tx=0).stop 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 shutdown_connections 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:6834/2178143597,v1:172.21.15.64:6835/2178143597] conn(0xc0abc0 0xc0af90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xa1e790 0xc0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cbcff9700 0 client.6264 ms_handle_reset on v2:172.21.15.64:6834/2178143597 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x9d3ae0 0xcaebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 --2- 172.21.15.64:0/1119654611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x9d3710 0xcae680 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 >> 172.21.15.64:0/1119654611 conn(0x9d3eb0 msgr2=0xc54140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 shutdown_connections 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 1 -- 172.21.15.64:0/1119654611 wait complete. 2023-09-27T23:27:06.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.867+0000 7f8cd26a1740 20 client.6264 trim_cache size 0 max 16384 2023-09-27T23:27:06.877 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:27:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:27:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:27:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6096, 2023-09-27T23:27:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 10.086777827000001, 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 21, 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 71, 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 71, 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 14.637919562 2023-09-27T23:27:06.878 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:27:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ca8060a70 msgr2=0x7f8ca8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ca8060a70 0x7f8ca8062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00113e0 tx=0x7f8cb00091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 msgr2=0x7f8cc81659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc81659f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ca600 tx=0x7f8cc0017000 comp rx=0 tx=0).stop 2023-09-27T23:27:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 shutdown_connections 2023-09-27T23:27:06.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ca8060a70 0x7f8ca8062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc80abce0 0x7f8cc81659f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cc80ab910 0x7f8cc813b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 --2- 172.21.15.64:0/3514972651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8cc80670e0 0x7f8cc80af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 >> 172.21.15.64:0/3514972651 conn(0x7f8cc806c080 msgr2=0x7f8cc8154640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 shutdown_connections 2023-09-27T23:27:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.871+0000 7f8ccf196700 1 -- 172.21.15.64:0/3514972651 wait complete. 2023-09-27T23:27:06.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:27:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 Processor -- start 2023-09-27T23:27:06.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 -- start start 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c140700 0x7fd89c140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd89c0c8570 0x7fd89c0ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c0cbb70 con 0x7fd89c0caac0 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c0caf60 con 0x7fd89c0c8570 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.995+0000 7fd8a40dd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0cb240 con 0x7fd89c140700 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd89c0c8570 0x7fd89c0ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd89c0c8570 0x7fd89c0ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52816/0 (socket says 172.21.15.64:52816) 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 -- 172.21.15.64:0/2950539580 learned_addr learned my addr 172.21.15.64:0/2950539580 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a1e79700 1 --2- 172.21.15.64:0/2950539580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c140700 0x7fd89c140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 -- 172.21.15.64:0/2950539580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c140700 msgr2=0x7fd89c140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 --2- 172.21.15.64:0/2950539580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c140700 0x7fd89c140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 -- 172.21.15.64:0/2950539580 >> v1:172.21.15.123:6789/0 conn(0x7fd89c0caac0 legacy=0x7fd89c0cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 -- 172.21.15.64:0/2950539580 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0cb520 con 0x7fd89c0c8570 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a267a700 1 --2- 172.21.15.64:0/2950539580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0c8570 0x7fd89c0ace70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd894009f20 tx=0x7fd894009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35ed5dd00dd41c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a0e77700 1 -- 172.21.15.64:0/2950539580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400e070 con 0x7fd89c0c8570 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a0e77700 1 -- 172.21.15.64:0/2950539580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89400ba80 con 0x7fd89c0c8570 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a0e77700 1 -- 172.21.15.64:0/2950539580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940092e0 con 0x7fd89c0c8570 2023-09-27T23:27:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2950539580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0c8570 msgr2=0x7fd89c0ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2950539580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0c8570 0x7fd89c0ace70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd894009f20 tx=0x7fd894009840 comp rx=0 tx=0).stop 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2950539580 shutdown_connections 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2950539580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0c8570 0x7fd89c0ace70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2950539580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c140700 0x7fd89c140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2950539580 >> 172.21.15.64:0/2950539580 conn(0x7fd89c0cd9c0 msgr2=0x7fd89c0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2950539580 shutdown_connections 2023-09-27T23:27:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:06.999+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2950539580 wait complete. 2023-09-27T23:27:07.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 Processor -- start 2023-09-27T23:27:07.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 -- start start 2023-09-27T23:27:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c0c8570 0x7fd89c167650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 0x7fd89c0c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd89c140700 0x7fd89c0c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0c5800 con 0x7fd89c0caac0 2023-09-27T23:27:07.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0c4b80 con 0x7fd89c140700 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0c4e60 con 0x7fd89c0c8570 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 0x7fd89c0c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1e79700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c0c8570 0x7fd89c167650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a267a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd89c140700 0x7fd89c0c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 0x7fd89c0c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52830/0 (socket says 172.21.15.64:52830) 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 -- 172.21.15.64:0/2539380949 learned_addr learned my addr 172.21.15.64:0/2539380949 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 -- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c0c8570 msgr2=0x7fd89c167650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c0c8570 0x7fd89c167650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 -- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd89c140700 msgr2=0x7fd89c0c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:07.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd89c140700 0x7fd89c0c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 -- 172.21.15.64:0/2539380949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd894009ae0 con 0x7fd89c0caac0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a1678700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 0x7fd89c0c3fa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00c7c0 tx=0x7fd88c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c011590 con 0x7fd89c0caac0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c0116f0 con 0x7fd89c0caac0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c0c5140 con 0x7fd89c0caac0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c011ab0 con 0x7fd89c0caac0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.003+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c167b90 con 0x7fd89c0caac0 2023-09-27T23:27:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.007+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c071120 con 0x7fd89c0caac0 2023-09-27T23:27:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.007+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd88c01b3b0 con 0x7fd89c0caac0 2023-09-27T23:27:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.007+0000 7fd891ffb700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd880060a70 0x7fd880062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.007+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd88c07c5e0 con 0x7fd89c0caac0 2023-09-27T23:27:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.007+0000 7fd8a1e79700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd880060a70 0x7fd880062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.011+0000 7fd8a1e79700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd880060a70 0x7fd880062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd89c15b5f0 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.015+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd88c04c610 con 0x7fd89c0caac0 2023-09-27T23:27:07.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.215+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd89c0cb800 con 0x7fd89c0caac0 2023-09-27T23:27:07.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.215+0000 7fd891ffb700 1 -- 172.21.15.64:0/2539380949 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd88c017070 con 0x7fd89c0caac0 2023-09-27T23:27:07.218 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd880060a70 msgr2=0x7fd880062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd880060a70 0x7fd880062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd89c15b5f0 tx=0x7fd898008040 comp rx=0 tx=0).stop 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 msgr2=0x7fd89c0c3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 0x7fd89c0c3fa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00c7c0 tx=0x7fd88c007f50 comp rx=0 tx=0).stop 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 shutdown_connections 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd880060a70 0x7fd880062f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd89c140700 0x7fd89c0c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd89c0caac0 0x7fd89c0c3fa0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 --2- 172.21.15.64:0/2539380949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd89c0c8570 0x7fd89c167650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 >> 172.21.15.64:0/2539380949 conn(0x7fd89c0cd9c0 msgr2=0x7fd89c0ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 shutdown_connections 2023-09-27T23:27:07.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:07.219+0000 7fd8a40dd700 1 -- 172.21.15.64:0/2539380949 wait complete. 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:27:07.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:27:07.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:27:07.238 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:27:07.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:27:07.238 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.245 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41757sec preferred_lft 41757sec 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:27:07.246 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:27:07.247 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:27:07.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:27:07.248 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:27:07.262 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:27:07.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:27:07.264 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:27:07.316 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:27:07.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:27:07.318 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:27:07.318 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:27:07.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:27:07.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:27:07.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:27:07.318 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:27:07.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:27:07.357 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:27:07.364 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:27:07.364 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:27:07.364 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:27:07.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:27:07.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:27:07.423 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:27:07.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:27:07.424 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:27:07.450 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:27:07.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:27:07.450 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:27:07.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:27:07.473 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:27:07.473 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:27:07.489 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:27:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:27:07.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:07.490 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:27:07.537 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:27:07.537 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:27:07.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:27:07.556 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:27:07.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:27:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:27:07.566 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:27:07.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:07.567 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:27:07.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:27:07.627+0000 7f4b60236380 -1 init, newargv = 0x55b525faeba0 newargc=15 2023-09-27T23:27:07.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[47485]: starting ceph client 2023-09-27T23:27:07.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[47485]: starting fuse 2023-09-27T23:27:08.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:27:08.692 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:27:08.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:27:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:27:08.752 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:27:08.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:08.752 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:27:08.759 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:27:08.759 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:27:08.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:27:08.760 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:27:08.761 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:27:08.840 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.47485.asok 2023-09-27T23:27:09.026 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47485.asok 2023-09-27T23:27:09.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:09.027 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47485.asok status 2023-09-27T23:27:09.125 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "47485", 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:27:09.126 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5708, 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5708 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-27T23:27:09.127 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2572843136 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2572843136 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5708 172.21.15.64:0/2572843136", 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "172.21.15.64:0/2572843136", 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 21, 2023-09-27T23:27:09.128 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 71, 2023-09-27T23:27:09.129 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:27:09.129 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:27:09.129 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:27:09.129 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:27:09.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:09.135 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:27:09.135 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:27:09.140 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:27:09.140 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:27:09.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:09.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:27:09.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:27:09.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 Processor -- start 2023-09-27T23:27:09.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 -- start start 2023-09-27T23:27:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f90880c6900 0x7f90880c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f908815cc20 0x7f908815d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908806b470 con 0x7f908806b900 2023-09-27T23:27:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908806e110 con 0x7f908815cc20 2023-09-27T23:27:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9088161d10 con 0x7f90880c6900 2023-09-27T23:27:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f908815cc20 0x7f908815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f908815cc20 0x7f908815d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52846/0 (socket says 172.21.15.64:52846) 2023-09-27T23:27:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908d221700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f90880c6900 0x7f90880c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 -- 172.21.15.64:0/2026159444 learned_addr learned my addr 172.21.15.64:0/2026159444 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 -- 172.21.15.64:0/2026159444 >> v2:172.21.15.123:3300/0 conn(0x7f90880c6900 msgr2=0x7f90880c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 --2- 172.21.15.64:0/2026159444 >> v2:172.21.15.123:3300/0 conn(0x7f90880c6900 0x7f90880c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 -- 172.21.15.64:0/2026159444 >> v1:172.21.15.123:6789/0 conn(0x7f908806b900 legacy=0x7f908806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 -- 172.21.15.64:0/2026159444 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908806bd70 con 0x7f908815cc20 2023-09-27T23:27:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908da22700 1 --2- 172.21.15.64:0/2026159444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908815d080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f908806c570 tx=0x7f9078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e99cc698c10bc47 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:09.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f907ffff700 1 -- 172.21.15.64:0/2026159444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078009e70 con 0x7f908815cc20 2023-09-27T23:27:09.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f907ffff700 1 -- 172.21.15.64:0/2026159444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907800b860 con 0x7f908815cc20 2023-09-27T23:27:09.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f907ffff700 1 -- 172.21.15.64:0/2026159444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907800bc00 con 0x7f908815cc20 2023-09-27T23:27:09.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 -- 172.21.15.64:0/2026159444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 msgr2=0x7f908815d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.263+0000 7f908e223700 1 --2- 172.21.15.64:0/2026159444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908815d080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f908806c570 tx=0x7f9078009250 comp rx=0 tx=0).stop 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- 172.21.15.64:0/2026159444 shutdown_connections 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 --2- 172.21.15.64:0/2026159444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908815d080 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 --2- 172.21.15.64:0/2026159444 >> v2:172.21.15.123:3300/0 conn(0x7f90880c6900 0x7f90880c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- 172.21.15.64:0/2026159444 >> 172.21.15.64:0/2026159444 conn(0x7f9088067bb0 msgr2=0x7f90880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- 172.21.15.64:0/2026159444 shutdown_connections 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- 172.21.15.64:0/2026159444 wait complete. 2023-09-27T23:27:09.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 Processor -- start 2023-09-27T23:27:09.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- start start 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f908806b900 0x7f90880b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90880c6900 0x7f908816d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908816dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908815db40 con 0x7f908815cc20 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880b0ad0 con 0x7f908806b900 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908e223700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880b0db0 con 0x7f90880c6900 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908da22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908816dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908da22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908816dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52848/0 (socket says 172.21.15.64:52848) 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.267+0000 7f908da22700 1 -- 172.21.15.64:0/4131017396 learned_addr learned my addr 172.21.15.64:0/4131017396 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908ca20700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90880c6900 0x7f908816d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908d221700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f908806b900 0x7f90880b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908da22700 1 -- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90880c6900 msgr2=0x7f908816d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908da22700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90880c6900 0x7f908816d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908da22700 1 -- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f908806b900 msgr2=0x7f90880b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908da22700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f908806b900 0x7f90880b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908da22700 1 -- 172.21.15.64:0/4131017396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780094f0 con 0x7f908815cc20 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908ca20700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90880c6900 0x7f908816d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908d221700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f908806b900 0x7f90880b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908da22700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908816dba0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9078009290 tx=0x7f9078017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907800bba0 con 0x7f908815cc20 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9078025420 con 0x7f908815cc20 2023-09-27T23:27:09.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780257e0 con 0x7f908815cc20 2023-09-27T23:27:09.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90880b1090 con 0x7f908815cc20 2023-09-27T23:27:09.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90880b07b0 con 0x7f908815cc20 2023-09-27T23:27:09.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908804be40 con 0x7f908815cc20 2023-09-27T23:27:09.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.271+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9078023070 con 0x7f908815cc20 2023-09-27T23:27:09.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.275+0000 7f907d7fa700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9074060a70 0x7f9074062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.275+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f907807e810 con 0x7f908815cc20 2023-09-27T23:27:09.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.275+0000 7f908d221700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9074060a70 0x7f9074062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.275+0000 7f908d221700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9074060a70 0x7f9074062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9084009d80 tx=0x7f9084008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:09.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.279+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f907804e840 con 0x7f908815cc20 2023-09-27T23:27:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.463+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f908806a580 con 0x7f9074060a70 2023-09-27T23:27:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.467+0000 7f907d7fa700 1 -- 172.21.15.64:0/4131017396 <== mgr.5402 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f908806a580 con 0x7f9074060a70 2023-09-27T23:27:09.471 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:27:09.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:27:09.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:27:09.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:27:09.471 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:27:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9074060a70 msgr2=0x7f9074062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9074060a70 0x7f9074062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9084009d80 tx=0x7f9084008040 comp rx=0 tx=0).stop 2023-09-27T23:27:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 msgr2=0x7f908816dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908816dba0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9078009290 tx=0x7f9078017000 comp rx=0 tx=0).stop 2023-09-27T23:27:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 shutdown_connections 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9074060a70 0x7f9074062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f908815cc20 0x7f908816dba0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90880c6900 0x7f908816d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 --2- 172.21.15.64:0/4131017396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f908806b900 0x7f90880b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 >> 172.21.15.64:0/4131017396 conn(0x7f9088067bb0 msgr2=0x7f9088159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 shutdown_connections 2023-09-27T23:27:09.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.471+0000 7f908e223700 1 -- 172.21.15.64:0/4131017396 wait complete. 2023-09-27T23:27:09.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:27:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 Processor -- start 2023-09-27T23:27:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- start start 2023-09-27T23:27:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f807806de00 0x7f80780c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f807806c7c0 0x7f807815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807806d970 con 0x7f80780c69a0 2023-09-27T23:27:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807806cd60 con 0x7f807806de00 2023-09-27T23:27:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807806d040 con 0x7f807806c7c0 2023-09-27T23:27:09.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f807806de00 0x7f80780c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f807806de00 0x7f80780c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52858/0 (socket says 172.21.15.64:52858) 2023-09-27T23:27:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 -- 172.21.15.64:0/1213080354 learned_addr learned my addr 172.21.15.64:0/1213080354 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:09.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807e4ea700 1 --2- 172.21.15.64:0/1213080354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f807806c7c0 0x7f807815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 -- 172.21.15.64:0/1213080354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f807806c7c0 msgr2=0x7f807815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 --2- 172.21.15.64:0/1213080354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f807806c7c0 0x7f807815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 -- 172.21.15.64:0/1213080354 >> v1:172.21.15.123:6789/0 conn(0x7f80780c69a0 legacy=0x7f807806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807cce7700 1 -- 172.21.15.64:0/1213080354 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119531515 0 0) 0x7f807806d970 con 0x7f80780c69a0 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 -- 172.21.15.64:0/1213080354 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807806d320 con 0x7f807806de00 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807d4e8700 1 --2- 172.21.15.64:0/1213080354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806de00 0x7f80780c6d70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f80680099b0 tx=0x7f8068009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df3419422e25ffd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807cce7700 1 -- 172.21.15.64:0/1213080354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806800b860 con 0x7f807806de00 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807cce7700 1 -- 172.21.15.64:0/1213080354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806800b9c0 con 0x7f807806de00 2023-09-27T23:27:09.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807cce7700 1 -- 172.21.15.64:0/1213080354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806800bd60 con 0x7f807806de00 2023-09-27T23:27:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- 172.21.15.64:0/1213080354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806de00 msgr2=0x7f80780c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 --2- 172.21.15.64:0/1213080354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806de00 0x7f80780c6d70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f80680099b0 tx=0x7f8068009250 comp rx=0 tx=0).stop 2023-09-27T23:27:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- 172.21.15.64:0/1213080354 shutdown_connections 2023-09-27T23:27:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 --2- 172.21.15.64:0/1213080354 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f807806c7c0 0x7f807815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 --2- 172.21.15.64:0/1213080354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806de00 0x7f80780c6d70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.595+0000 7f807eceb700 1 -- 172.21.15.64:0/1213080354 >> 172.21.15.64:0/1213080354 conn(0x7f8078067b50 msgr2=0x7f80780cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.599+0000 7f807eceb700 1 -- 172.21.15.64:0/1213080354 shutdown_connections 2023-09-27T23:27:09.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.599+0000 7f807eceb700 1 -- 172.21.15.64:0/1213080354 wait complete. 2023-09-27T23:27:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.599+0000 7f807eceb700 1 Processor -- start 2023-09-27T23:27:09.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.599+0000 7f807eceb700 1 -- start start 2023-09-27T23:27:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 0x7f8078170430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f807806de00 0x7f80780c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80780c69a0 0x7f80780c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807815f1a0 con 0x7f807806c7c0 2023-09-27T23:27:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078170970 con 0x7f807806de00 2023-09-27T23:27:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807815e590 con 0x7f80780c69a0 2023-09-27T23:27:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 0x7f8078170430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 0x7f8078170430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52866/0 (socket says 172.21.15.64:52866) 2023-09-27T23:27:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 -- 172.21.15.64:0/855890637 learned_addr learned my addr 172.21.15.64:0/855890637 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807d4e8700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f807806de00 0x7f80780c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807e4ea700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80780c69a0 0x7f80780c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 -- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80780c69a0 msgr2=0x7f80780c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80780c69a0 0x7f80780c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 -- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f807806de00 msgr2=0x7f80780c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f807806de00 0x7f80780c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 -- 172.21.15.64:0/855890637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680094f0 con 0x7f807806c7c0 2023-09-27T23:27:09.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807dce9700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 0x7f8078170430 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f807400c530 tx=0x7f807400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074011590 con 0x7f807806c7c0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074011710 con 0x7f807806c7c0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807815e870 con 0x7f807806c7c0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074011ad0 con 0x7f807806c7c0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.603+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807815ee30 con 0x7f807806c7c0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.607+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807804be40 con 0x7f807806c7c0 2023-09-27T23:27:09.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.607+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f807401b3b0 con 0x7f807806c7c0 2023-09-27T23:27:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.607+0000 7f806dffb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f805c060a70 0x7f805c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.607+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f807407c530 con 0x7f807806c7c0 2023-09-27T23:27:09.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.607+0000 7f807d4e8700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f805c060a70 0x7f805c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:09.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.607+0000 7f807d4e8700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f805c060a70 0x7f805c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f80680099b0 tx=0x7f8068017000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:09.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.611+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f807404c560 con 0x7f807806c7c0 2023-09-27T23:27:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:09.815+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f807806d600 con 0x7f807806c7c0 2023-09-27T23:27:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.135+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 17) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f807404c930 con 0x7f807806c7c0 2023-09-27T23:27:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.135+0000 7f806dffb700 1 -- 172.21.15.64:0/855890637 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v17) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8074046ef0 con 0x7f807806c7c0 2023-09-27T23:27:10.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f805c060a70 msgr2=0x7f805c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f805c060a70 0x7f805c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f80680099b0 tx=0x7f8068017000 comp rx=0 tx=0).stop 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 msgr2=0x7f8078170430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 0x7f8078170430 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f807400c530 tx=0x7f807400c840 comp rx=0 tx=0).stop 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 shutdown_connections 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f805c060a70 0x7f805c062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80780c69a0 0x7f80780c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f807806de00 0x7f80780c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 --2- 172.21.15.64:0/855890637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807806c7c0 0x7f8078170430 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 >> 172.21.15.64:0/855890637 conn(0x7f8078067b50 msgr2=0x7f807815bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:10.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 shutdown_connections 2023-09-27T23:27:10.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.139+0000 7f807eceb700 1 -- 172.21.15.64:0/855890637 wait complete. 2023-09-27T23:27:10.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:27:10.246 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:10.245+0000 7f680181e000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:27:10.255 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:10.251+0000 7f2c79b27000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:27:10.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e33f76700 1 Processor -- start 2023-09-27T23:27:10.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e33f76700 1 -- start start 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e33f76700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e2c0b08d0 0x7f5e2c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e33f76700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c15a3b0 con 0x7f5e2c0ab870 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e33f76700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0b0e40 con 0x7f5e2c159fb0 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e33f76700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c0b1120 con 0x7f5e2c0b08d0 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.267+0000 7f5e31511700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e2c0b08d0 0x7f5e2c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e31d12700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5e2c0ab870 0x7f5e2c0afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48702/0 (socket says 172.21.15.64:48702) 2023-09-27T23:27:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e31d12700 1 -- 172.21.15.64:0/3645905887 learned_addr learned my addr 172.21.15.64:0/3645905887 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e31511700 1 -- 172.21.15.64:0/3645905887 >> v1:172.21.15.123:6789/0 conn(0x7f5e2c159fb0 legacy=0x7f5e2c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e31511700 1 -- 172.21.15.64:0/3645905887 >> v1:172.21.15.64:6789/0 conn(0x7f5e2c0ab870 legacy=0x7f5e2c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e30d10700 1 -- 172.21.15.64:0/3645905887 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018972795 0 0) 0x7f5e2c15a3b0 con 0x7f5e2c0ab870 2023-09-27T23:27:10.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e31511700 1 -- 172.21.15.64:0/3645905887 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0b1400 con 0x7f5e2c0b08d0 2023-09-27T23:27:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e31511700 1 --2- 172.21.15.64:0/3645905887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c159a70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0098e0 tx=0x7f5e1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b6a5cf31a9c22 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e30d10700 1 -- 172.21.15.64:0/3645905887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c009e80 con 0x7f5e2c0b08d0 2023-09-27T23:27:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e30d10700 1 -- 172.21.15.64:0/3645905887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00b860 con 0x7f5e2c0b08d0 2023-09-27T23:27:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e30d10700 1 -- 172.21.15.64:0/3645905887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00bbe0 con 0x7f5e2c0b08d0 2023-09-27T23:27:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 -- 172.21.15.64:0/3645905887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0b08d0 msgr2=0x7f5e2c159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 --2- 172.21.15.64:0/3645905887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c159a70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0098e0 tx=0x7f5e1c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 -- 172.21.15.64:0/3645905887 shutdown_connections 2023-09-27T23:27:10.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 --2- 172.21.15.64:0/3645905887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c159a70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 -- 172.21.15.64:0/3645905887 >> 172.21.15.64:0/3645905887 conn(0x7f5e2c0cc580 msgr2=0x7f5e2c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:10.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 -- 172.21.15.64:0/3645905887 shutdown_connections 2023-09-27T23:27:10.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.271+0000 7f5e33f76700 1 -- 172.21.15.64:0/3645905887 wait complete. 2023-09-27T23:27:10.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 Processor -- start 2023-09-27T23:27:10.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 -- start start 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 0x7f5e2c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c0a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e2c159fb0 0x7f5e2c0a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cb6b0 con 0x7f5e2c0ab870 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c150620 con 0x7f5e2c0b08d0 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0caaa0 con 0x7f5e2c159fb0 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 0x7f5e2c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 0x7f5e2c0cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52888/0 (socket says 172.21.15.64:52888) 2023-09-27T23:27:10.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 -- 172.21.15.64:0/2946647227 learned_addr learned my addr 172.21.15.64:0/2946647227 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31511700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c0a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e32513700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e2c159fb0 0x7f5e2c0a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 -- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e2c159fb0 msgr2=0x7f5e2c0a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e2c159fb0 0x7f5e2c0a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 -- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2c0b08d0 msgr2=0x7f5e2c0a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c0a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 -- 172.21.15.64:0/2946647227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c009420 con 0x7f5e2c0ab870 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e32513700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e2c159fb0 0x7f5e2c0a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e31d12700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 0x7f5e2c0cbb40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c560 tx=0x7f5e28000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28005b30 con 0x7f5e2c0ab870 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28005c90 con 0x7f5e2c0ab870 2023-09-27T23:27:10.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0cad80 con 0x7f5e2c0ab870 2023-09-27T23:27:10.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28007320 con 0x7f5e2c0ab870 2023-09-27T23:27:10.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.275+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0cb340 con 0x7f5e2c0ab870 2023-09-27T23:27:10.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.279+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c071120 con 0x7f5e2c0ab870 2023-09-27T23:27:10.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.279+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f5e2801b3b0 con 0x7f5e2c0ab870 2023-09-27T23:27:10.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.279+0000 7f5e21ffb700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e10060a90 0x7f5e10062f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.279+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5e2807c640 con 0x7f5e2c0ab870 2023-09-27T23:27:10.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.279+0000 7f5e31511700 1 -- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e10060a90 msgr2=0x7f5e10062f40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:27:10.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.279+0000 7f5e31511700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e10060a90 0x7f5e10062f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:27:10.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.287+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5e2804c670 con 0x7f5e2c0ab870 2023-09-27T23:27:10.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.447+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f5e2c0cc2f0 con 0x7f5e2c0ab870 2023-09-27T23:27:10.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.455+0000 7f5e21ffb700 1 -- 172.21.15.64:0/2946647227 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e2804fc90 con 0x7f5e2c0ab870 2023-09-27T23:27:10.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e10060a90 msgr2=0x7f5e10062f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:27:10.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e10060a90 0x7f5e10062f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 msgr2=0x7f5e2c0cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 0x7f5e2c0cbb40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c560 tx=0x7f5e28000fe0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 shutdown_connections 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e10060a90 0x7f5e10062f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e2c159fb0 0x7f5e2c0a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2c0b08d0 0x7f5e2c0a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 --2- 172.21.15.64:0/2946647227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e2c0ab870 0x7f5e2c0cbb40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 >> 172.21.15.64:0/2946647227 conn(0x7f5e2c0cc580 msgr2=0x7f5e2c155120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 shutdown_connections 2023-09-27T23:27:10.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.459+0000 7f5e33f76700 1 -- 172.21.15.64:0/2946647227 wait complete. 2023-09-27T23:27:10.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:27:10.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.583+0000 7f930d0e7700 1 Processor -- start 2023-09-27T23:27:10.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.583+0000 7f930d0e7700 1 -- start start 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9308155a90 0x7f93080ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f930806db70 0x7f93080ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f93080adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93080c68a0 con 0x7f930806db70 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93080c6e60 con 0x7f9308155a90 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93080ae4d0 con 0x7f9308155e60 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f930806db70 0x7f93080ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f930806db70 0x7f93080ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52898/0 (socket says 172.21.15.64:52898) 2023-09-27T23:27:10.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 -- 172.21.15.64:0/658946986 learned_addr learned my addr 172.21.15.64:0/658946986 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930c8e6700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f93080adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f9307fff700 1 --2- 172.21.15.64:0/658946986 >> v2:172.21.15.123:3300/0 conn(0x7f9308155a90 0x7f93080ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 -- 172.21.15.64:0/658946986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 msgr2=0x7f93080adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f93080adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 -- 172.21.15.64:0/658946986 >> v2:172.21.15.123:3300/0 conn(0x7f9308155a90 msgr2=0x7f93080ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 --2- 172.21.15.64:0/658946986 >> v2:172.21.15.123:3300/0 conn(0x7f9308155a90 0x7f93080ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 -- 172.21.15.64:0/658946986 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080ae7b0 con 0x7f930806db70 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f9307fff700 1 --2- 172.21.15.64:0/658946986 >> v2:172.21.15.123:3300/0 conn(0x7f9308155a90 0x7f93080ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930c8e6700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f93080adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f93077fe700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f93080ad760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f930806a8d0 tx=0x7f92fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01fcede90132b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f9306ffd700 1 -- 172.21.15.64:0/658946986 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc00b860 con 0x7f930806db70 2023-09-27T23:27:10.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f9306ffd700 1 -- 172.21.15.64:0/658946986 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc00b9c0 con 0x7f930806db70 2023-09-27T23:27:10.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f9306ffd700 1 -- 172.21.15.64:0/658946986 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc00bd60 con 0x7f930806db70 2023-09-27T23:27:10.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- 172.21.15.64:0/658946986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 msgr2=0x7f93080ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f93080ad760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f930806a8d0 tx=0x7f92fc009250 comp rx=0 tx=0).stop 2023-09-27T23:27:10.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- 172.21.15.64:0/658946986 shutdown_connections 2023-09-27T23:27:10.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f93080adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- 172.21.15.64:0/658946986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f93080ad760 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 --2- 172.21.15.64:0/658946986 >> v2:172.21.15.123:3300/0 conn(0x7f9308155a90 0x7f93080ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- 172.21.15.64:0/658946986 >> 172.21.15.64:0/658946986 conn(0x7f93080c8cd0 msgr2=0x7f93080c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- 172.21.15.64:0/658946986 shutdown_connections 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.587+0000 7f930d0e7700 1 -- 172.21.15.64:0/658946986 wait complete. 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 Processor -- start 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- start start 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f9308151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9308155a90 0x7f930816ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f930816cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930816d8c0 con 0x7f930806db70 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080ab2f0 con 0x7f9308155a90 2023-09-27T23:27:10.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93080ab010 con 0x7f9308155e60 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f9308151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f9308151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52910/0 (socket says 172.21.15.64:52910) 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 -- 172.21.15.64:0/2335391780 learned_addr learned my addr 172.21.15.64:0/2335391780 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f93077fe700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9308155a90 0x7f930816ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930c8e6700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f930816cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 -- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 msgr2=0x7f930816cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f930816cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 -- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9308155a90 msgr2=0x7f930816ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9308155a90 0x7f930816ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 -- 172.21.15.64:0/2335391780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0094f0 con 0x7f930806db70 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930c8e6700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f930816cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:27:10.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f9307fff700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f9308151080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e900 tx=0x7f92f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:10.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8013410 con 0x7f930806db70 2023-09-27T23:27:10.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f8013570 con 0x7f930806db70 2023-09-27T23:27:10.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9308159d50 con 0x7f930806db70 2023-09-27T23:27:10.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8013910 con 0x7f930806db70 2023-09-27T23:27:10.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930815a310 con 0x7f930806db70 2023-09-27T23:27:10.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.591+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e80052b0 con 0x7f930806db70 2023-09-27T23:27:10.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.595+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f92f8013d40 con 0x7f930806db70 2023-09-27T23:27:10.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.595+0000 7f92effff700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f92f4060ac0 0x7f92f4062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.595+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f92f807cef0 con 0x7f930806db70 2023-09-27T23:27:10.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.595+0000 7f93077fe700 1 -- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f92f4060ac0 msgr2=0x7f92f4062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:27:10.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.595+0000 7f93077fe700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f92f4060ac0 0x7f92f4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:27:10.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.603+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f92f804ce10 con 0x7f930806db70 2023-09-27T23:27:10.672 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:10.669+0000 7f680181e000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:27:10.710 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:10.707+0000 7f2c79b27000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:27:10.720 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:10.717+0000 7f680181e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:27:10.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.759+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f92e8005830 con 0x7f930806db70 2023-09-27T23:27:10.763 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:10.759+0000 7f2c79b27000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:27:10.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.763+0000 7f92effff700 1 -- 172.21.15.64:0/2335391780 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f92f8050430 con 0x7f930806db70 2023-09-27T23:27:10.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f92f4060ac0 msgr2=0x7f92f4062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:27:10.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f92f4060ac0 0x7f92f4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 msgr2=0x7f9308151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f9308151080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e900 tx=0x7f92f8008040 comp rx=0 tx=0).stop 2023-09-27T23:27:10.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 shutdown_connections 2023-09-27T23:27:10.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f92f4060ac0 0x7f92f4062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9308155e60 0x7f930816cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9308155a90 0x7f930816ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 --2- 172.21.15.64:0/2335391780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806db70 0x7f9308151080 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 >> 172.21.15.64:0/2335391780 conn(0x7f93080c8cd0 msgr2=0x7f93080cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 shutdown_connections 2023-09-27T23:27:10.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.767+0000 7f930d0e7700 1 -- 172.21.15.64:0/2335391780 wait complete. 2023-09-27T23:27:10.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:10.780 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:27:10.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-09-27T23:27:10.826 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:10.825+0000 7f680181e000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:27:10.877 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:10.875+0000 7f2c79b27000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:27:10.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd359e700 1 Processor -- start 2023-09-27T23:27:10.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd359e700 1 -- start start 2023-09-27T23:27:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5bd406b350 0x7f5bd4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd40c85a0 con 0x7f5bd4158b00 2023-09-27T23:27:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd406d9b0 con 0x7f5bd40abe40 2023-09-27T23:27:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40c67d0 con 0x7f5bd406b350 2023-09-27T23:27:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd259c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5bd4158b00 0x7f5bd406d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48712/0 (socket says 172.21.15.64:48712) 2023-09-27T23:27:10.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd259c700 1 -- 172.21.15.64:0/887787735 learned_addr learned my addr 172.21.15.64:0/887787735 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:10.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd2d9d700 1 --2- 172.21.15.64:0/887787735 >> v2:172.21.15.123:3300/0 conn(0x7f5bd406b350 0x7f5bd4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd1d9b700 1 -- 172.21.15.64:0/887787735 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688453800 0 0) 0x7f5bd40c85a0 con 0x7f5bd4158b00 2023-09-27T23:27:10.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd1d9b700 1 -- 172.21.15.64:0/887787735 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc0003570 con 0x7f5bd4158b00 2023-09-27T23:27:10.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd2d9d700 1 -- 172.21.15.64:0/887787735 >> v1:172.21.15.123:6789/0 conn(0x7f5bd40abe40 legacy=0x7f5bd406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd2d9d700 1 -- 172.21.15.64:0/887787735 >> v1:172.21.15.64:6789/0 conn(0x7f5bd4158b00 legacy=0x7f5bd406d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd2d9d700 1 -- 172.21.15.64:0/887787735 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40c6ab0 con 0x7f5bd406b350 2023-09-27T23:27:10.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd2d9d700 1 --2- 172.21.15.64:0/887787735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd4158ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009e60 tx=0x7f5bcc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c029a9e75dffa75f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:10.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd1d9b700 1 -- 172.21.15.64:0/887787735 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00e070 con 0x7f5bd406b350 2023-09-27T23:27:10.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd1d9b700 1 -- 172.21.15.64:0/887787735 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc00bb60 con 0x7f5bd406b350 2023-09-27T23:27:10.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.903+0000 7f5bd1d9b700 1 -- 172.21.15.64:0/887787735 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc012430 con 0x7f5bd406b350 2023-09-27T23:27:10.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- 172.21.15.64:0/887787735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 msgr2=0x7f5bd4158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 --2- 172.21.15.64:0/887787735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd4158ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009e60 tx=0x7f5bcc009780 comp rx=0 tx=0).stop 2023-09-27T23:27:10.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- 172.21.15.64:0/887787735 shutdown_connections 2023-09-27T23:27:10.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 --2- 172.21.15.64:0/887787735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd4158ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- 172.21.15.64:0/887787735 >> 172.21.15.64:0/887787735 conn(0x7f5bd40c8c00 msgr2=0x7f5bd40c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:10.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- 172.21.15.64:0/887787735 shutdown_connections 2023-09-27T23:27:10.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- 172.21.15.64:0/887787735 wait complete. 2023-09-27T23:27:10.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 Processor -- start 2023-09-27T23:27:10.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- start start 2023-09-27T23:27:10.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd41515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5bd40abe40 0x7f5bd40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd4158b00 0x7f5bd416c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4152050 con 0x7f5bd4158b00 2023-09-27T23:27:10.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b1760 con 0x7f5bd406b350 2023-09-27T23:27:10.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b1a40 con 0x7f5bd40abe40 2023-09-27T23:27:10.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd41515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd4158b00 0x7f5bd416c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd41515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:46354/0 (socket says 172.21.15.64:46354) 2023-09-27T23:27:10.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bd259c700 1 -- 172.21.15.64:0/2172913816 learned_addr learned my addr 172.21.15.64:0/2172913816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:10.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.907+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5bd40abe40 0x7f5bd40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:10.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd259c700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5bd40abe40 msgr2=0x7f5bd40af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd259c700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5bd40abe40 0x7f5bd40af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd259c700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd4158b00 msgr2=0x7f5bd416c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd259c700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd4158b00 0x7f5bd416c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd259c700 1 -- 172.21.15.64:0/2172913816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc009a20 con 0x7f5bd406b350 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd2d9d700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd4158b00 0x7f5bd416c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd259c700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd41515a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c590 tx=0x7f5bc4007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4005b30 con 0x7f5bd406b350 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4005cb0 con 0x7f5bd406b350 2023-09-27T23:27:10.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd40b1d20 con 0x7f5bd406b350 2023-09-27T23:27:10.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4007320 con 0x7f5bd406b350 2023-09-27T23:27:10.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd40b0030 con 0x7f5bd406b350 2023-09-27T23:27:10.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd404be40 con 0x7f5bd406b350 2023-09-27T23:27:10.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.911+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f5bc401b3b0 con 0x7f5bd406b350 2023-09-27T23:27:10.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.915+0000 7f5bcaffd700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:10.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.915+0000 7f5bc9d9b700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 msgr2=0x7f5bb4062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:27:10.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.915+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5bc407c640 con 0x7f5bd406b350 2023-09-27T23:27:10.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.915+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:27:10.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:10.923+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5bc404c5d0 con 0x7f5bd406b350 2023-09-27T23:27:11.022 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.021+0000 7f680181e000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:27:11.070 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.069+0000 7f680181e000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:27:11.084 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.083+0000 7f2c79b27000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:27:11.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:11.107+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bd40c6d90 con 0x7f5bb4060ac0 2023-09-27T23:27:11.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:11.119+0000 7f5bc9d9b700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 msgr2=0x7f5bb4062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:27:11.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:11.119+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:27:11.135 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.131+0000 7f2c79b27000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:27:11.222 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.221+0000 7f680181e000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:27:11.278 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.277+0000 7f680181e000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:27:11.291 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.287+0000 7f2c79b27000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:27:11.337 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.337+0000 7f680181e000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:27:11.351 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.347+0000 7f2c79b27000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:27:11.412 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.411+0000 7f2c79b27000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:27:11.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:11.523+0000 7f5bc9d9b700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 msgr2=0x7f5bb4062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:27:11.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:11.523+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:27:11.634 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.633+0000 7f680181e000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:27:11.720 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.719+0000 7f2c79b27000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:27:11.744 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:11.745+0000 7f680181e000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:27:11.835 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:11.831+0000 7f2c79b27000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:27:12.025 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.025+0000 7f680181e000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:27:12.125 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.123+0000 7f2c79b27000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:27:12.186 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.185+0000 7f680181e000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:27:12.250 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.249+0000 7f680181e000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:27:12.293 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.291+0000 7f2c79b27000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:27:12.302 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.301+0000 7f680181e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:27:12.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:12.327+0000 7f5bc9d9b700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 msgr2=0x7f5bb4062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:27:12.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:12.327+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:27:12.358 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.357+0000 7f680181e000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:27:12.361 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.359+0000 7f2c79b27000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:27:12.414 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.411+0000 7f2c79b27000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:27:12.472 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.471+0000 7f2c79b27000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:27:12.531 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.529+0000 7f680181e000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:27:12.585 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.585+0000 7f680181e000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:27:12.646 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.643+0000 7f2c79b27000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:27:12.648 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.645+0000 7f680181e000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:27:12.704 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.703+0000 7f2c79b27000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:27:12.769 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.767+0000 7f2c79b27000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:27:12.797 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.797+0000 7f680181e000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:27:12.859 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:12.857+0000 7f680181e000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:27:12.927 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.923+0000 7f2c79b27000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:27:12.990 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:12.987+0000 7f2c79b27000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:27:13.447 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.445+0000 7f680181e000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:27:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:13.483+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 18) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f5bc404c9a0 con 0x7f5bd406b350 2023-09-27T23:27:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:13.483+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 msgr2=0x7f5bb4062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:27:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:13.483+0000 7f5bcaffd700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:13.615 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:27:13.611+0000 7f2c79b27000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:27:13.649 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c51cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.649 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c51cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.649 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c51cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.649 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c51cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.649 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c51cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.650 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c19c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.650 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c19c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.650 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c19c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.650 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c19c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:13.650 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:27:13.649+0000 7f67c19c8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:27:14.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.491+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mon.1 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 19) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f5bc404f780 con 0x7f5bd406b350 2023-09-27T23:27:14.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.491+0000 7f5bcaffd700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.491+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bd40c6d90 con 0x7f5bb4060ac0 2023-09-27T23:27:14.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.491+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.495+0000 7f5bc9d9b700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5bd415e3d0 tx=0x7f5bbc007ec0 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:14.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.599+0000 7f5bcaffd700 1 -- 172.21.15.64:0/2172913816 <== mgr.6280 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5bd40c6d90 con 0x7f5bb4060ac0 2023-09-27T23:27:14.603 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path snap_test_dir1 2023-09-27T23:27:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 msgr2=0x7f5bb4062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5bd415e3d0 tx=0x7f5bbc007ec0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 msgr2=0x7f5bd41515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd41515a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c590 tx=0x7f5bc4007e00 comp rx=0 tx=0).stop 2023-09-27T23:27:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 shutdown_connections 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5bb4060ac0 0x7f5bb4062ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd4158b00 0x7f5bd416c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5bd40abe40 0x7f5bd40af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 --2- 172.21.15.64:0/2172913816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bd406b350 0x7f5bd41515a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 >> 172.21.15.64:0/2172913816 conn(0x7f5bd40c8c00 msgr2=0x7f5bd415e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 shutdown_connections 2023-09-27T23:27:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.603+0000 7f5bd359e700 1 -- 172.21.15.64:0/2172913816 wait complete. 2023-09-27T23:27:14.623 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-09-27T23:27:14.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-09-27T23:27:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 Processor -- start 2023-09-27T23:27:14.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 -- start start 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc158da0 0x7f74bc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74bc0caca0 0x7f74bc06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc15d720 con 0x7f74bc06df10 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0ca290 con 0x7f74bc0caca0 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.743+0000 7f74c208c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0ca4a0 con 0x7f74bc158da0 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c0889700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f74bc06df10 0x7f74bc06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48646/0 (socket says 172.21.15.64:48646) 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c0889700 1 -- 172.21.15.64:0/1236291915 learned_addr learned my addr 172.21.15.64:0/1236291915 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:14.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c188b700 1 --2- 172.21.15.64:0/1236291915 >> v2:172.21.15.123:3300/0 conn(0x7f74bc0caca0 0x7f74bc06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c108a700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc158da0 0x7f74bc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74affff700 1 -- 172.21.15.64:0/1236291915 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922431145 0 0) 0x7f74bc15d720 con 0x7f74bc06df10 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c188b700 1 -- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc158da0 msgr2=0x7f74bc06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c188b700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc158da0 0x7f74bc06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c188b700 1 -- 172.21.15.64:0/1236291915 >> v1:172.21.15.64:6789/0 conn(0x7f74bc06df10 legacy=0x7f74bc06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c188b700 1 -- 172.21.15.64:0/1236291915 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0ca780 con 0x7f74bc0caca0 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c108a700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc158da0 0x7f74bc06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c188b700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc0caca0 0x7f74bc06ebb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009e60 tx=0x7f74b4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d3e711e07efbe61 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74affff700 1 -- 172.21.15.64:0/1236291915 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400e070 con 0x7f74bc0caca0 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74affff700 1 -- 172.21.15.64:0/1236291915 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400bb60 con 0x7f74bc0caca0 2023-09-27T23:27:14.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74affff700 1 -- 172.21.15.64:0/1236291915 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4012430 con 0x7f74bc0caca0 2023-09-27T23:27:14.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 -- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc0caca0 msgr2=0x7f74bc06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:14.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc0caca0 0x7f74bc06ebb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009e60 tx=0x7f74b4009780 comp rx=0 tx=0).stop 2023-09-27T23:27:14.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 -- 172.21.15.64:0/1236291915 shutdown_connections 2023-09-27T23:27:14.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc0caca0 0x7f74bc06ebb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 --2- 172.21.15.64:0/1236291915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc158da0 0x7f74bc06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 -- 172.21.15.64:0/1236291915 >> 172.21.15.64:0/1236291915 conn(0x7f74bc0cc760 msgr2=0x7f74bc0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:14.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 -- 172.21.15.64:0/1236291915 shutdown_connections 2023-09-27T23:27:14.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.747+0000 7f74c208c700 1 -- 172.21.15.64:0/1236291915 wait complete. 2023-09-27T23:27:14.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 Processor -- start 2023-09-27T23:27:14.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 -- start start 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 0x7f74bc14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74bc0caca0 0x7f74bc0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc158da0 0x7f74bc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1695f0 con 0x7f74bc0caca0 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1698d0 con 0x7f74bc158da0 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c208c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc169bb0 con 0x7f74bc06df10 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c0889700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74bc0caca0 0x7f74bc0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 0x7f74bc14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c188b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc158da0 0x7f74bc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 0x7f74bc14bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:43568/0 (socket says 172.21.15.64:43568) 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c188b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc158da0 0x7f74bc06cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:38598/0 (socket says 172.21.15.64:38598) 2023-09-27T23:27:14.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 -- 172.21.15.64:0/916137576 learned_addr learned my addr 172.21.15.64:0/916137576 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:14.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 -- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc158da0 msgr2=0x7f74bc06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc158da0 0x7f74bc06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 -- 172.21.15.64:0/916137576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74bc0caca0 msgr2=0x7f74bc0c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74bc0caca0 0x7f74bc0c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.751+0000 7f74c108a700 1 -- 172.21.15.64:0/916137576 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0009420 con 0x7f74bc06df10 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74c108a700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 0x7f74bc14bcd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f74b000ec80 tx=0x7f74b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000c850 con 0x7f74bc06df10 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000c9b0 con 0x7f74bc06df10 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4009a20 con 0x7f74bc06df10 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000cd30 con 0x7f74bc06df10 2023-09-27T23:27:14.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc14c210 con 0x7f74bc06df10 2023-09-27T23:27:14.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc04be40 con 0x7f74bc06df10 2023-09-27T23:27:14.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 19) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f74b0005a00 con 0x7f74bc06df10 2023-09-27T23:27:14.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.755+0000 7f74ad7fa700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f74a00607f0 0x7f74a0062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:14.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.759+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f74b007d690 con 0x7f74bc06df10 2023-09-27T23:27:14.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.759+0000 7f74c0889700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f74a00607f0 0x7f74a0062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:14.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.759+0000 7f74c0889700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f74a00607f0 0x7f74a0062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0c4b40 tx=0x7f74b8007d30 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:14.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.763+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f74b0051430 con 0x7f74bc06df10 2023-09-27T23:27:14.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:14.951+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f74bc16c710 con 0x7f74a00607f0 2023-09-27T23:27:15.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.063+0000 7f74ad7fa700 1 -- 172.21.15.64:0/916137576 <== mgr.6280 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f74bc16c710 con 0x7f74a00607f0 2023-09-27T23:27:15.065 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path snap_test_dir1 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f74a00607f0 msgr2=0x7f74a0062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f74a00607f0 0x7f74a0062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0c4b40 tx=0x7f74b8007d30 comp rx=0 tx=0).stop 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 msgr2=0x7f74bc14bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 0x7f74bc14bcd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f74b000ec80 tx=0x7f74b00091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 shutdown_connections 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f74a00607f0 0x7f74a0062ca0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74bc158da0 0x7f74bc06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74bc0caca0 0x7f74bc0c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 --2- 172.21.15.64:0/916137576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74bc06df10 0x7f74bc14bcd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 >> 172.21.15.64:0/916137576 conn(0x7f74bc0cc760 msgr2=0x7f74bc156100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:15.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 shutdown_connections 2023-09-27T23:27:15.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.067+0000 7f74c208c700 1 -- 172.21.15.64:0/916137576 wait complete. 2023-09-27T23:27:15.086 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-09-27T23:27:15.086 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-09-27-23_28_00 in ~45s... 2023-09-27T23:27:15.086 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-09-27-23_29_00 in ~105s... 2023-09-27T23:27:15.086 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-09-27T23:27:15.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-09-27T23:27:15.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.207+0000 7f6283c02700 1 Processor -- start 2023-09-27T23:27:15.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.207+0000 7f6283c02700 1 -- start start 2023-09-27T23:27:15.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c0cb130 0x7f627c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:15.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f627c06a6a0 0x7f627c06aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:15.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c0b0ac0 con 0x7f627c15c220 2023-09-27T23:27:15.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c06b030 con 0x7f627c06a6a0 2023-09-27T23:27:15.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c06b210 con 0x7f627c0cb130 2023-09-27T23:27:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628219f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f627c15c220 0x7f627c06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48662/0 (socket says 172.21.15.64:48662) 2023-09-27T23:27:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628219f700 1 -- 172.21.15.64:0/2530384485 learned_addr learned my addr 172.21.15.64:0/2530384485 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:27:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628199e700 1 --2- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c0cb130 0x7f627c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628119d700 1 --2- 172.21.15.64:0/2530384485 >> v2:172.21.15.123:3300/0 conn(0x7f627c06a6a0 0x7f627c06aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628099c700 1 -- 172.21.15.64:0/2530384485 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808064136 0 0) 0x7f627c0b0ac0 con 0x7f627c15c220 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628099c700 1 -- 172.21.15.64:0/2530384485 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6268003570 con 0x7f627c15c220 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628119d700 1 -- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c0cb130 msgr2=0x7f627c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628119d700 1 --2- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c0cb130 0x7f627c0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628119d700 1 -- 172.21.15.64:0/2530384485 >> v1:172.21.15.64:6789/0 conn(0x7f627c15c220 legacy=0x7f627c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628119d700 1 -- 172.21.15.64:0/2530384485 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c0ca810 con 0x7f627c06a6a0 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628119d700 1 --2- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c06aaf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f62700098e0 tx=0x7f62700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9061790f1d220afa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628099c700 1 -- 172.21.15.64:0/2530384485 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270011630 con 0x7f627c06a6a0 2023-09-27T23:27:15.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628099c700 1 -- 172.21.15.64:0/2530384485 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6270011790 con 0x7f627c06a6a0 2023-09-27T23:27:15.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f628099c700 1 -- 172.21.15.64:0/2530384485 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270011b10 con 0x7f627c06a6a0 2023-09-27T23:27:15.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 msgr2=0x7f627c06aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 --2- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c06aaf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f62700098e0 tx=0x7f62700091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- 172.21.15.64:0/2530384485 shutdown_connections 2023-09-27T23:27:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 --2- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c06aaf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 --2- 172.21.15.64:0/2530384485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c0cb130 0x7f627c0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- 172.21.15.64:0/2530384485 >> 172.21.15.64:0/2530384485 conn(0x7f627c067bb0 msgr2=0x7f627c0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- 172.21.15.64:0/2530384485 shutdown_connections 2023-09-27T23:27:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.211+0000 7f6283c02700 1 -- 172.21.15.64:0/2530384485 wait complete. 2023-09-27T23:27:15.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.215+0000 7f6283c02700 1 Processor -- start 2023-09-27T23:27:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.215+0000 7f6283c02700 1 -- start start 2023-09-27T23:27:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c1472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 0x7f627c15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c15c220 0x7f627c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c149da0 con 0x7f627c0cb130 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c14a080 con 0x7f627c06a6a0 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c14a360 con 0x7f627c15c220 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 0x7f627c15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 0x7f627c15b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59196/0 (socket says 172.21.15.64:59196) 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 -- 172.21.15.64:0/93022408 learned_addr learned my addr 172.21.15.64:0/93022408 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628199e700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c1472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:15.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628219f700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c15c220 0x7f627c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 -- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c15c220 msgr2=0x7f627c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c15c220 0x7f627c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 -- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 msgr2=0x7f627c1472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c1472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 -- 172.21.15.64:0/93022408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270009420 con 0x7f627c0cb130 2023-09-27T23:27:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f628119d700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 0x7f627c15b170 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f627c06f300 tx=0x7f6270017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270011900 con 0x7f627c0cb130 2023-09-27T23:27:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6270011a80 con 0x7f627c0cb130 2023-09-27T23:27:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c14a640 con 0x7f627c0cb130 2023-09-27T23:27:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627001a680 con 0x7f627c0cb130 2023-09-27T23:27:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.219+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c147800 con 0x7f627c0cb130 2023-09-27T23:27:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.223+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c14ab40 con 0x7f627c0cb130 2023-09-27T23:27:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.223+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f627001aab0 con 0x7f627c0cb130 2023-09-27T23:27:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.223+0000 7f626dffb700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62600607f0 0x7f6260062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:27:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.223+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f627007ea10 con 0x7f627c0cb130 2023-09-27T23:27:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.223+0000 7f628199e700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62600607f0 0x7f6260062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:27:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.223+0000 7f628199e700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62600607f0 0x7f6260062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6278009db0 tx=0x7f62780091a0 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:27:15.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.235+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f627004ebd0 con 0x7f627c0cb130 2023-09-27T23:27:15.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.411+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f627c147ae0 con 0x7f62600607f0 2023-09-27T23:27:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.427+0000 7f626dffb700 1 -- 172.21.15.64:0/93022408 <== mgr.6280 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f627c147ae0 con 0x7f62600607f0 2023-09-27T23:27:15.430 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:27:15.431 INFO:teuthology.orchestra.run.smithi064.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-09-27T23:27:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62600607f0 msgr2=0x7f6260062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62600607f0 0x7f6260062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6278009db0 tx=0x7f62780091a0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 msgr2=0x7f627c15b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 0x7f627c15b170 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f627c06f300 tx=0x7f6270017000 comp rx=0 tx=0).stop 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 shutdown_connections 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62600607f0 0x7f6260062ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f627c15c220 0x7f627c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f627c0cb130 0x7f627c15b170 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 --2- 172.21.15.64:0/93022408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c06a6a0 0x7f627c1472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:27:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 >> 172.21.15.64:0/93022408 conn(0x7f627c067bb0 msgr2=0x7f627c0b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:27:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 shutdown_connections 2023-09-27T23:27:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:27:15.431+0000 7f6283c02700 1 -- 172.21.15.64:0/93022408 wait complete. 2023-09-27T23:27:15.447 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-09-27T23:27:15.448 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-09-27T23:27:15.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:15.448 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:15.466 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:15.466 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:15.466 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:16.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:16.467 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:16.485 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:16.485 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:16.485 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:17.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:17.487 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:17.504 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:17.505 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:17.505 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:18.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:18.506 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:18.523 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:18.523 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:18.523 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:19.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:27:19.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:27:19.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:19.524 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:19.542 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:19.542 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:19.542 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:20.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:20.543 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:20.560 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:20.561 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:20.561 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:21.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:21.562 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:21.579 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:21.579 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:21.579 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:22.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:22.581 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:22.600 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:22.600 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:22.600 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:23.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:23.601 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:23.621 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:23.621 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:23.621 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:24.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:24.622 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:24.642 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:24.642 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:24.642 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:25.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:25.644 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:25.663 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:25.663 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:25.663 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:26.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:26.664 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:26.683 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:26.683 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:26.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:27.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:27.685 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:27.704 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:27.704 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:27.704 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:28.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:28.705 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:28.724 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:28.724 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:28.725 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:29.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:29.726 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:29.746 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:29.746 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:29.746 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:30.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:30.747 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:30.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:30.765 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:30.765 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:31.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:31.766 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:31.783 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:31.783 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:31.783 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:32.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:32.785 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:32.802 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:32.803 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:32.803 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:33.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:33.804 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:33.824 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:33.824 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:33.824 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:34.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:34.825 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:34.842 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:34.842 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:34.842 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:35.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:35.843 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:35.860 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:35.860 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:35.860 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:36.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:36.861 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:36.879 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:36.880 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:36.880 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:37.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:37.880 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:37.899 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:37.899 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:37.899 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:38.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:38.899 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:38.917 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:38.917 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:38.917 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:39.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:39.919 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:39.936 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:39.936 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:39.936 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:40.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:40.937 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:40.956 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:40.956 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:40.956 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:41.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:41.958 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:41.975 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:41.975 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:41.975 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:42.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:42.976 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:42.994 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:42.994 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:42.994 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:43.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:43.996 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:44.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:44.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:44.016 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:45.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:45.018 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:45.036 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:45.036 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:45.036 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:46.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:46.038 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:46.055 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:46.055 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:46.055 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:47.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:47.056 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:47.074 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:47.074 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:47.074 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:48.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:48.075 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:48.094 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:48.094 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:48.094 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:49.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:49.095 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:49.114 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:49.115 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:49.115 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:49.335 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:27:49.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:27:50.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:50.116 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:50.133 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:50.133 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:50.133 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:51.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:51.134 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:51.154 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:51.154 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:51.154 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:52.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:52.155 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:52.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:52.174 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:52.174 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:53.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:53.175 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:53.194 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:53.194 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:53.194 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:54.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:54.196 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:54.217 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:54.217 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:54.217 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:55.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:55.219 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:55.239 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:55.239 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:55.239 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:56.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:56.240 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:56.258 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:56.258 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:56.259 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:57.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:57.259 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:57.277 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:57.277 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:57.277 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:58.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:58.278 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:58.296 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:58.296 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:58.296 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:27:59.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:27:59.297 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:27:59.316 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:27:59.316 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:27:59.316 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:00.085 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:28:00.084+0000 7f67b41ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:28:00.085 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:28:00.084+0000 7f67b41ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:28:00.086 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:28:00.084+0000 7f67b41ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:28:00.086 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:28:00.084+0000 7f67b41ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:28:00.086 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:28:00.084+0000 7f67b41ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:28:00.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:00.317 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:00.334 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:00.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:00.335 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:00.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:00.335 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-09-27-23_28_00_UTC'] 2023-09-27T23:28:00.335 DEBUG:tasks.cephfs.test_snap_schedules:exec=1695857235.0865593, now = 1695857280.335386, timo = 45 2023-09-27T23:28:01.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:01.336 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:01.354 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:01.354 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:01.355 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:01.355 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:02.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:02.356 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:02.375 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:02.376 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:02.376 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:02.376 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:03.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:03.377 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:03.396 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:03.398 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:03.398 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:03.398 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:04.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:04.400 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:04.418 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:04.420 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:04.420 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:04.420 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:05.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:05.421 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:05.438 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:05.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:05.439 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:05.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:06.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:06.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:06.459 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:06.460 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:06.460 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:06.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:07.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:07.461 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:07.478 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:07.479 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:07.479 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:07.479 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:08.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:08.480 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:08.497 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:08.498 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:08.498 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:08.498 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:09.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:09.499 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:09.515 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:09.516 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:09.516 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:09.516 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:10.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:10.517 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:10.534 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:10.535 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:10.535 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:10.535 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:11.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:11.536 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:11.553 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:11.554 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:11.554 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:11.554 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:12.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:12.556 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:12.572 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:12.573 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:12.573 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:12.573 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:13.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:13.574 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:13.591 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:13.592 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:13.592 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:13.592 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:14.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:14.594 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:14.611 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:14.612 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:14.612 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:14.612 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:15.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:15.614 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:15.631 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:15.632 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:15.632 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:15.632 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:16.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:16.633 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:16.649 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:16.651 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:16.651 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:16.651 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:17.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:17.652 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:17.669 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:17.670 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:17.670 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:17.670 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:18.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:18.671 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:18.688 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:18.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:18.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:18.689 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:19.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:28:19.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:28:19.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:19.690 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:19.707 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:19.707 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:19.707 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:19.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:20.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:20.709 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:20.725 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:20.726 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:20.726 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:20.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:21.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:21.727 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:21.743 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:21.744 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:21.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:21.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:22.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:22.746 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:22.762 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:22.763 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:22.763 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:22.763 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:23.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:23.764 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:23.781 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:23.782 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:23.782 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:23.782 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:24.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:24.783 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:24.800 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:24.800 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:24.801 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:24.801 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:25.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:25.802 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:25.819 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:25.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:25.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:25.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:26.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:26.822 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:26.841 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:26.842 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:26.843 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:26.843 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:27.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:27.844 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:27.861 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:27.862 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:27.862 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:27.862 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:28.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:28.863 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:28.882 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:28.882 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:28.882 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:28.882 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:29.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:29.884 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:29.901 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:29.901 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:29.901 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:30.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:30.903 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:30.920 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:30.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:30.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:30.922 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:31.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:31.923 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:31.940 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:31.941 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:31.941 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:31.941 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:32.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:32.942 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:32.959 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:32.960 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:32.960 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:32.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:33.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:33.961 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:33.978 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:33.979 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:33.980 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:33.980 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:34.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:34.981 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:34.998 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:34.998 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:34.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:34.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:35.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:36.000 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:36.017 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:36.018 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:36.018 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:36.018 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:37.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:37.019 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:37.036 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:37.037 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:37.037 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:37.037 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:38.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:38.038 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:38.055 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:38.056 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:38.056 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:38.056 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:39.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:39.057 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:39.074 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:39.075 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:39.075 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:39.075 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:40.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:40.076 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:40.093 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:40.094 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:40.095 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:40.095 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:41.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:41.096 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:41.114 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:41.115 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:41.115 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:41.115 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:42.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:42.117 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:42.134 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:42.135 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:42.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:42.135 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:43.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:43.136 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:43.152 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:43.154 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:43.154 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:43.154 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:44.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:44.155 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:44.172 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:44.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:44.173 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:44.173 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:45.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:45.174 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:45.188 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:45.189 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:45.189 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:45.189 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:46.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:46.191 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:46.208 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:46.209 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:46.209 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:46.209 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:47.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:47.210 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:47.227 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:47.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:47.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:48.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:48.228 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:48.245 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:48.246 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:48.246 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:48.246 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:49.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:49.247 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:49.264 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:49.265 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:49.265 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:49.265 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:49.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:28:49.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:28:50.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:50.266 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:50.282 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:50.283 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:50.283 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:50.283 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:51.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:51.284 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:51.303 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:51.303 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:51.303 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:51.303 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:52.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:52.305 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:52.321 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:52.322 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:52.322 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:52.323 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:53.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:53.324 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:53.342 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:53.343 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:53.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:53.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:54.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:54.344 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:54.361 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:54.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:54.362 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:54.362 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:55.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:55.363 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:55.379 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:55.380 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:55.380 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:55.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:56.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:56.381 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:56.398 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:56.399 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:56.399 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:56.399 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:57.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:57.401 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:57.418 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:57.419 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:57.420 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:57.420 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:58.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:58.421 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:58.439 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:58.440 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:58.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:58.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:28:59.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:28:59.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:28:59.458 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:28:59.459 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC'} 2023-09-27T23:28:59.459 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:28:59.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:29:00.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:00.460 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:29:00.478 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:29:00.478 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_29_00_UTC 2023-09-27T23:29:00.479 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_28_00_UTC', 'scheduled-2023-09-27-23_29_00_UTC'} 2023-09-27T23:29:00.479 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-09-27-23_29_00_UTC'} 2023-09-27T23:29:00.480 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:29:00.480 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-09-27-23_29_00_UTC'] 2023-09-27T23:29:00.480 DEBUG:tasks.cephfs.test_snap_schedules:exec=1695857235.0865593, now = 1695857340.4802136, timo = 105 2023-09-27T23:29:01.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-09-27T23:29:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.601+0000 7fb0b1969700 1 Processor -- start 2023-09-27T23:29:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.601+0000 7fb0b1969700 1 -- start start 2023-09-27T23:29:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1969700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac06df10 0x7fb0ac06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:01.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1969700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac06b420 con 0x7fb0ac06ed80 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1969700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac15d720 con 0x7fb0ac158da0 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1969700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0ca290 con 0x7fb0ac06df10 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0abfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb0ac06ed80 0x7fb0ac06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42978/0 (socket says 172.21.15.64:42978) 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0abfff700 1 -- 172.21.15.64:0/2864278358 learned_addr learned my addr 172.21.15.64:0/2864278358 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1168700 1 --2- 172.21.15.64:0/2864278358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac06df10 0x7fb0ac06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235230724 0 0) 0x7fb0ac06b420 con 0x7fb0ac06ed80 2023-09-27T23:29:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb098003570 con 0x7fb0ac06ed80 2023-09-27T23:29:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329214958 0 0) 0x7fb0ac15d720 con 0x7fb0ac158da0 2023-09-27T23:29:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ac06b420 con 0x7fb0ac158da0 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 505695884 0 0) 0x7fb098003570 con 0x7fb0ac06ed80 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1168700 1 -- 172.21.15.64:0/2864278358 >> v1:172.21.15.123:6789/0 conn(0x7fb0ac158da0 legacy=0x7fb0ac06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1168700 1 -- 172.21.15.64:0/2864278358 >> v1:172.21.15.64:6789/0 conn(0x7fb0ac06ed80 legacy=0x7fb0ac06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1168700 1 -- 172.21.15.64:0/2864278358 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0ca4a0 con 0x7fb0ac06df10 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0b1168700 1 --2- 172.21.15.64:0/2864278358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac06df10 0x7fb0ac06aee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009e90 tx=0x7fb0a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276600bf15de9fec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400e070 con 0x7fb0ac06df10 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a400b870 con 0x7fb0ac06df10 2023-09-27T23:29:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.605+0000 7fb0ab7fe700 1 -- 172.21.15.64:0/2864278358 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c001470 con 0x7fb0ac06df10 2023-09-27T23:29:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- 172.21.15.64:0/2864278358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac06df10 msgr2=0x7fb0ac06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 --2- 172.21.15.64:0/2864278358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac06df10 0x7fb0ac06aee0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009e90 tx=0x7fb0a4009780 comp rx=0 tx=0).stop 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- 172.21.15.64:0/2864278358 shutdown_connections 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 --2- 172.21.15.64:0/2864278358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac06df10 0x7fb0ac06aee0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- 172.21.15.64:0/2864278358 >> 172.21.15.64:0/2864278358 conn(0x7fb0ac0cc760 msgr2=0x7fb0ac0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- 172.21.15.64:0/2864278358 shutdown_connections 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- 172.21.15.64:0/2864278358 wait complete. 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 Processor -- start 2023-09-27T23:29:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- start start 2023-09-27T23:29:01.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ac06df10 0x7fb0ac0c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:01.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 0x7fb0ac151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac158da0 0x7fb0ac06c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac152610 con 0x7fb0ac06ed80 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0c3330 con 0x7fb0ac06df10 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1969700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0c7240 con 0x7fb0ac158da0 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 0x7fb0ac151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 0x7fb0ac151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41044/0 (socket says 172.21.15.64:41044) 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 -- 172.21.15.64:0/89421411 learned_addr learned my addr 172.21.15.64:0/89421411 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b1168700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac158da0 0x7fb0ac06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b0967700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ac06df10 0x7fb0ac0c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 -- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac158da0 msgr2=0x7fb0ac06c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac158da0 0x7fb0ac06c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 -- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ac06df10 msgr2=0x7fb0ac0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ac06df10 0x7fb0ac0c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0abfff700 1 -- 172.21.15.64:0/89421411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4009a20 con 0x7fb0ac06ed80 2023-09-27T23:29:01.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.609+0000 7fb0b0967700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ac06df10 0x7fb0ac0c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0abfff700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 0x7fb0ac151b00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00cba0 tx=0x7fb09c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c007e90 con 0x7fb0ac06ed80 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c007360 con 0x7fb0ac06ed80 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac152040 con 0x7fb0ac06ed80 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c004830 con 0x7fb0ac06ed80 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac155ca0 con 0x7fb0ac06ed80 2023-09-27T23:29:01.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac04be40 con 0x7fb0ac06ed80 2023-09-27T23:29:01.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) v1 ==== 80390+0+0 (secure 0 0 0) 0x7fb09c004ba0 con 0x7fb0ac06ed80 2023-09-27T23:29:01.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0a8ff9700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060ac0 0x7fb08c062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:01.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fb09c07caa0 con 0x7fb0ac06ed80 2023-09-27T23:29:01.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.613+0000 7fb0b0967700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060ac0 0x7fb08c062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:01.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.617+0000 7fb0b0967700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060ac0 0x7fb08c062f70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0b1e30 tx=0x7fb094009040 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:01.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.621+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb09c04c9f0 con 0x7fb0ac06ed80 2023-09-27T23:29:01.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.801+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0ac06f360 con 0x7fb08c060ac0 2023-09-27T23:29:01.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.937+0000 7fb0a8ff9700 1 -- 172.21.15.64:0/89421411 <== mgr.6280 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb0ac06f360 con 0x7fb08c060ac0 2023-09-27T23:29:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path snap_test_dir1 2023-09-27T23:29:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060ac0 msgr2=0x7fb08c062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060ac0 0x7fb08c062f70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0b1e30 tx=0x7fb094009040 comp rx=0 tx=0).stop 2023-09-27T23:29:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 msgr2=0x7fb0ac151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 0x7fb0ac151b00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00cba0 tx=0x7fb09c001b50 comp rx=0 tx=0).stop 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 shutdown_connections 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb08c060ac0 0x7fb08c062f70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0ac158da0 0x7fb0ac06c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0ac06ed80 0x7fb0ac151b00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 --2- 172.21.15.64:0/89421411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ac06df10 0x7fb0ac0c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 >> 172.21.15.64:0/89421411 conn(0x7fb0ac0cc760 msgr2=0x7fb0ac0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 shutdown_connections 2023-09-27T23:29:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:01.941+0000 7fb0b1969700 1 -- 172.21.15.64:0/89421411 wait complete. 2023-09-27T23:29:01.964 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-09-27T23:29:01.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:01.965 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:29:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:29:01.980 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_29_00_UTC 2023-09-27T23:29:01.981 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-09-27-23_28_00_UTC 2023-09-27T23:29:01.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:01.981 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-09-27-23_28_00_UTC) 2023-09-27T23:29:02.007 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-09-27-23_29_00_UTC 2023-09-27T23:29:02.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:02.008 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-09-27-23_29_00_UTC) 2023-09-27T23:29:02.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:02.030 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-09-27T23:29:02.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:29:02.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd223439700 1 Processor -- start 2023-09-27T23:29:02.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd223439700 1 -- start start 2023-09-27T23:29:02.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd223439700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd21c06dcf0 0x7fd21c159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd223439700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c06c850 con 0x7fd21c06aeb0 2023-09-27T23:29:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd223439700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c0cb650 con 0x7fd21c158c90 2023-09-27T23:29:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd223439700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21c06b470 con 0x7fd21c06dcf0 2023-09-27T23:29:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd221c36700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd21c06aeb0 0x7fd21c159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:42992/0 (socket says 172.21.15.64:42992) 2023-09-27T23:29:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.205+0000 7fd221c36700 1 -- 172.21.15.64:0/2437225370 learned_addr learned my addr 172.21.15.64:0/2437225370 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:02.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd222c38700 1 --2- 172.21.15.64:0/2437225370 >> v2:172.21.15.123:3300/0 conn(0x7fd21c06dcf0 0x7fd21c159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd221435700 1 -- 172.21.15.64:0/2437225370 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814794043 0 0) 0x7fd21c06c850 con 0x7fd21c06aeb0 2023-09-27T23:29:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd221435700 1 -- 172.21.15.64:0/2437225370 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd208003570 con 0x7fd21c06aeb0 2023-09-27T23:29:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd222c38700 1 -- 172.21.15.64:0/2437225370 >> v1:172.21.15.123:6789/0 conn(0x7fd21c158c90 legacy=0x7fd21c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd222c38700 1 -- 172.21.15.64:0/2437225370 >> v1:172.21.15.64:6789/0 conn(0x7fd21c06aeb0 legacy=0x7fd21c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd221435700 1 -- 172.21.15.64:0/2437225370 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862244475 0 0) 0x7fd208003570 con 0x7fd21c06aeb0 2023-09-27T23:29:02.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd222c38700 1 -- 172.21.15.64:0/2437225370 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c06b750 con 0x7fd21c06dcf0 2023-09-27T23:29:02.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd222c38700 1 --2- 172.21.15.64:0/2437225370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c159770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd214009e60 tx=0x7fd214009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afbc19180d394a63 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd221435700 1 -- 172.21.15.64:0/2437225370 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400e070 con 0x7fd21c06dcf0 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd221435700 1 -- 172.21.15.64:0/2437225370 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21400b850 con 0x7fd21c06dcf0 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd221435700 1 -- 172.21.15.64:0/2437225370 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180013f0 con 0x7fd21c06dcf0 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 -- 172.21.15.64:0/2437225370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 msgr2=0x7fd21c159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 --2- 172.21.15.64:0/2437225370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c159770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd214009e60 tx=0x7fd214009780 comp rx=0 tx=0).stop 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 -- 172.21.15.64:0/2437225370 shutdown_connections 2023-09-27T23:29:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 --2- 172.21.15.64:0/2437225370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c159770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 -- 172.21.15.64:0/2437225370 >> 172.21.15.64:0/2437225370 conn(0x7fd21c067bb0 msgr2=0x7fd21c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 -- 172.21.15.64:0/2437225370 shutdown_connections 2023-09-27T23:29:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.209+0000 7fd223439700 1 -- 172.21.15.64:0/2437225370 wait complete. 2023-09-27T23:29:02.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 Processor -- start 2023-09-27T23:29:02.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 -- start start 2023-09-27T23:29:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 0x7fd21c1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd21c158c90 0x7fd21c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c153340 con 0x7fd21c06aeb0 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0c8c60 con 0x7fd21c06dcf0 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0c8f40 con 0x7fd21c158c90 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 0x7fd21c1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 0x7fd21c1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41052/0 (socket says 172.21.15.64:41052) 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 -- 172.21.15.64:0/1490231472 learned_addr learned my addr 172.21.15.64:0/1490231472 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222c38700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd21c158c90 0x7fd21c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd221c36700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 -- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd21c158c90 msgr2=0x7fd21c0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd21c158c90 0x7fd21c0ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 -- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 msgr2=0x7fd21c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 -- 172.21.15.64:0/1490231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214009a20 con 0x7fd21c06aeb0 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd221c36700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222c38700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd21c158c90 0x7fd21c0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd222437700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 0x7fd21c1526d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c5c0 tx=0x7fd21000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210012630 con 0x7fd21c06aeb0 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2100127b0 con 0x7fd21c06aeb0 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210012b70 con 0x7fd21c06aeb0 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c0c9220 con 0x7fd21c06aeb0 2023-09-27T23:29:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.213+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c0c97e0 con 0x7fd21c06aeb0 2023-09-27T23:29:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.217+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c04be40 con 0x7fd21c06aeb0 2023-09-27T23:29:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.217+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) v1 ==== 80390+0+0 (secure 0 0 0) 0x7fd21001b400 con 0x7fd21c06aeb0 2023-09-27T23:29:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.217+0000 7fd20e7fc700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd200060ac0 0x7fd200062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.217+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fd21007cf00 con 0x7fd21c06aeb0 2023-09-27T23:29:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.217+0000 7fd221c36700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd200060ac0 0x7fd200062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:02.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.217+0000 7fd221c36700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd200060ac0 0x7fd200062f70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd218001ed0 tx=0x7fd218009040 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:02.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.225+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd21004cdd0 con 0x7fd21c06aeb0 2023-09-27T23:29:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:02.429+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd21c06ba30 con 0x7fd21c06aeb0 2023-09-27T23:29:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.157+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fd21004d1a0 con 0x7fd21c06aeb0 2023-09-27T23:29:03.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.161+0000 7fd20e7fc700 1 -- 172.21.15.64:0/1490231472 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v23) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd21004cdd0 con 0x7fd21c06aeb0 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd200060ac0 msgr2=0x7fd200062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd200060ac0 0x7fd200062f70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd218001ed0 tx=0x7fd218009040 comp rx=0 tx=0).stop 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 msgr2=0x7fd21c1526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 0x7fd21c1526d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c5c0 tx=0x7fd21000c980 comp rx=0 tx=0).stop 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 shutdown_connections 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd200060ac0 0x7fd200062f70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd21c158c90 0x7fd21c0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd21c06dcf0 0x7fd21c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 --2- 172.21.15.64:0/1490231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd21c06aeb0 0x7fd21c1526d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 >> 172.21.15.64:0/1490231472 conn(0x7fd21c067bb0 msgr2=0x7fd21c0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 shutdown_connections 2023-09-27T23:29:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.169+0000 7fd223439700 1 -- 172.21.15.64:0/1490231472 wait complete. 2023-09-27T23:29:03.188 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:29:03.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:03.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:03.231 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:29:03.231 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:29:03.231 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:29:03.231 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:03.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[47485]: fuse finished with error 0 and tester_r 0 2023-09-27T23:29:03.244 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:29:03.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:03.245 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:29:03.283 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:03.277+0000 7f7ab9d52000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:29:03.284 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:03.279+0000 7fc50e835000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:29:03.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:29:03.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:03.367 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:29:03.374 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:29:03.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:29:03.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:29:03.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:03.375 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:29:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:29:03.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:29:03.414 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:29:03.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:03.415 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:29:03.421 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:29:03.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:29:03.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:03.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 Processor -- start 2023-09-27T23:29:03.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 -- start start 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fce1006ce90 0x7fce1015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce1015ce00 0x7fce100678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1006e1c0 con 0x7fce1006cac0 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1015d280 con 0x7fce1015ce00 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16a25700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce100cc840 con 0x7fce1006ce90 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16224700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce1015ce00 0x7fce100678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce15a23700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fce1006cac0 0x7fce100cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43010/0 (socket says 172.21.15.64:43010) 2023-09-27T23:29:03.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce15a23700 1 -- 172.21.15.64:0/3582525263 learned_addr learned my addr 172.21.15.64:0/3582525263 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce15222700 1 --2- 172.21.15.64:0/3582525263 >> v2:172.21.15.123:3300/0 conn(0x7fce1006ce90 0x7fce1015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16224700 1 -- 172.21.15.64:0/3582525263 >> v2:172.21.15.123:3300/0 conn(0x7fce1006ce90 msgr2=0x7fce1015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:03.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16224700 1 --2- 172.21.15.64:0/3582525263 >> v2:172.21.15.123:3300/0 conn(0x7fce1006ce90 0x7fce1015c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16224700 1 -- 172.21.15.64:0/3582525263 >> v1:172.21.15.64:6789/0 conn(0x7fce1006cac0 legacy=0x7fce100cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce14a21700 1 -- 172.21.15.64:0/3582525263 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385035075 0 0) 0x7fce1006e1c0 con 0x7fce1006cac0 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.529+0000 7fce16224700 1 -- 172.21.15.64:0/3582525263 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1006d2e0 con 0x7fce1015ce00 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16224700 1 --2- 172.21.15.64:0/3582525263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1015ce00 0x7fce100678e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fce1006b300 tx=0x7fce08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaa3b5ffb9aab26 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce14a21700 1 -- 172.21.15.64:0/3582525263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0800e070 con 0x7fce1015ce00 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce14a21700 1 -- 172.21.15.64:0/3582525263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0800bb60 con 0x7fce1015ce00 2023-09-27T23:29:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce14a21700 1 -- 172.21.15.64:0/3582525263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08012430 con 0x7fce1015ce00 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 -- 172.21.15.64:0/3582525263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1015ce00 msgr2=0x7fce100678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 --2- 172.21.15.64:0/3582525263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1015ce00 0x7fce100678e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fce1006b300 tx=0x7fce08009780 comp rx=0 tx=0).stop 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 -- 172.21.15.64:0/3582525263 shutdown_connections 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 --2- 172.21.15.64:0/3582525263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1015ce00 0x7fce100678e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 --2- 172.21.15.64:0/3582525263 >> v2:172.21.15.123:3300/0 conn(0x7fce1006ce90 0x7fce1015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 -- 172.21.15.64:0/3582525263 >> 172.21.15.64:0/3582525263 conn(0x7fce100cdbd0 msgr2=0x7fce100cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 -- 172.21.15.64:0/3582525263 shutdown_connections 2023-09-27T23:29:03.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 -- 172.21.15.64:0/3582525263 wait complete. 2023-09-27T23:29:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 Processor -- start 2023-09-27T23:29:03.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.533+0000 7fce16a25700 1 -- start start 2023-09-27T23:29:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 0x7fce100c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce1006ce90 0x7fce100b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce1015ce00 0x7fce100b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100b1940 con 0x7fce1006cac0 2023-09-27T23:29:03.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100b0970 con 0x7fce1015ce00 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1016c570 con 0x7fce1006ce90 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 0x7fce100c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 0x7fce100c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41094/0 (socket says 172.21.15.64:41094) 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 -- 172.21.15.64:0/2335021721 learned_addr learned my addr 172.21.15.64:0/2335021721 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15222700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce1006ce90 0x7fce100b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16224700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce1015ce00 0x7fce100b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 -- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce1006ce90 msgr2=0x7fce100b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce1006ce90 0x7fce100b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 -- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce1015ce00 msgr2=0x7fce100b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce1015ce00 0x7fce100b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 -- 172.21.15.64:0/2335021721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08009a20 con 0x7fce1006cac0 2023-09-27T23:29:03.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15222700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce1006ce90 0x7fce100b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16224700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce1015ce00 0x7fce100b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce15a23700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 0x7fce100c99d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c5b0 tx=0x7fce0c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c005b30 con 0x7fce1006cac0 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c005c90 con 0x7fce1006cac0 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1016c850 con 0x7fce1006cac0 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c007300 con 0x7fce1006cac0 2023-09-27T23:29:03.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.537+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1016ce10 con 0x7fce1006cac0 2023-09-27T23:29:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.541+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1004be40 con 0x7fce1006cac0 2023-09-27T23:29:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.541+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fce0c01b3b0 con 0x7fce1006cac0 2023-09-27T23:29:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.541+0000 7fce05ffb700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 0x7fcdf4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.541+0000 7fce15222700 1 -- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 msgr2=0x7fcdf4062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.541+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6810+0+0 (secure 0 0 0) 0x7fce0c07c860 con 0x7fce1006cac0 2023-09-27T23:29:03.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.541+0000 7fce15222700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 0x7fcdf4062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.545+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fce0c04c6e0 con 0x7fce1006cac0 2023-09-27T23:29:03.715 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:03.711+0000 7fc50e835000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:29:03.730 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:03.725+0000 7f7ab9d52000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:29:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.741+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce100671b0 con 0x7fce1006cac0 2023-09-27T23:29:03.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce15222700 1 -- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 msgr2=0x7fcdf4062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:03.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce15222700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 0x7fcdf4062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:03.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce05ffb700 1 -- 172.21.15.64:0/2335021721 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fce0c04fd00 con 0x7fce1006cac0 2023-09-27T23:29:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:03.749 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:26:56.779553+0000","modified":"2023-09-27T23:26:59.790212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2178143597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2178143597},{"type":"v1","addr":"172.21.15.64:6835","nonce":2178143597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 msgr2=0x7fcdf4062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 0x7fcdf4062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 msgr2=0x7fce100c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 0x7fce100c99d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c5b0 tx=0x7fce0c001070 comp rx=0 tx=0).stop 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 shutdown_connections 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcdf4060a70 0x7fcdf4062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce1015ce00 0x7fce100b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce1006ce90 0x7fce100b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 --2- 172.21.15.64:0/2335021721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1006cac0 0x7fce100c99d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 >> 172.21.15.64:0/2335021721 conn(0x7fce100cdbd0 msgr2=0x7fce100cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:03.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 shutdown_connections 2023-09-27T23:29:03.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.745+0000 7fce16a25700 1 -- 172.21.15.64:0/2335021721 wait complete. 2023-09-27T23:29:03.752 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-09-27T23:29:03.762 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:03.763 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:03.759+0000 7fc50e835000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:29:03.781 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:03.777+0000 7f7ab9d52000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:29:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.861+0000 7fa88bfff700 1 Processor -- start 2023-09-27T23:29:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.861+0000 7fa88bfff700 1 -- start start 2023-09-27T23:29:03.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa88c15cfd0 0x7fa88c163c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c06e1c0 con 0x7fa88c06cac0 2023-09-27T23:29:03.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0cc840 con 0x7fa88c06ce90 2023-09-27T23:29:03.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c06d2e0 con 0x7fa88c15cfd0 2023-09-27T23:29:03.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88affd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa88c06cac0 0x7fa88c0cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43024/0 (socket says 172.21.15.64:43024) 2023-09-27T23:29:03.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88affd700 1 -- 172.21.15.64:0/928539431 learned_addr learned my addr 172.21.15.64:0/928539431 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:03.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88b7fe700 1 --2- 172.21.15.64:0/928539431 >> v2:172.21.15.123:3300/0 conn(0x7fa88c15cfd0 0x7fa88c163c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa889ffb700 1 -- 172.21.15.64:0/928539431 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242742081 0 0) 0x7fa88c06e1c0 con 0x7fa88c06cac0 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa889ffb700 1 -- 172.21.15.64:0/928539431 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86c003570 con 0x7fa88c06cac0 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88b7fe700 1 -- 172.21.15.64:0/928539431 >> v1:172.21.15.123:6789/0 conn(0x7fa88c06ce90 legacy=0x7fa88c15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88b7fe700 1 -- 172.21.15.64:0/928539431 >> v1:172.21.15.64:6789/0 conn(0x7fa88c06cac0 legacy=0x7fa88c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88b7fe700 1 -- 172.21.15.64:0/928539431 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c06d5b0 con 0x7fa88c15cfd0 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88b7fe700 1 --2- 172.21.15.64:0/928539431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c163c60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0098e0 tx=0x7fa87c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38576b1305106995 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa889ffb700 1 -- 172.21.15.64:0/928539431 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00b640 con 0x7fa88c15cfd0 2023-09-27T23:29:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa889ffb700 1 -- 172.21.15.64:0/928539431 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c00b7a0 con 0x7fa88c15cfd0 2023-09-27T23:29:03.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa889ffb700 1 -- 172.21.15.64:0/928539431 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00bb20 con 0x7fa88c15cfd0 2023-09-27T23:29:03.871 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:03.867+0000 7fc50e835000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- 172.21.15.64:0/928539431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 msgr2=0x7fa88c163c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 --2- 172.21.15.64:0/928539431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c163c60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0098e0 tx=0x7fa87c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- 172.21.15.64:0/928539431 shutdown_connections 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 --2- 172.21.15.64:0/928539431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c163c60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- 172.21.15.64:0/928539431 >> 172.21.15.64:0/928539431 conn(0x7fa88c0cdbd0 msgr2=0x7fa88c0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- 172.21.15.64:0/928539431 shutdown_connections 2023-09-27T23:29:03.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.865+0000 7fa88bfff700 1 -- 172.21.15.64:0/928539431 wait complete. 2023-09-27T23:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 Processor -- start 2023-09-27T23:29:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 -- start start 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 0x7fa88c16cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c06ce90 0x7fa88c14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c14e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c16d420 con 0x7fa88c06cac0 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0b2760 con 0x7fa88c15cfd0 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0b2a40 con 0x7fa88c06ce90 2023-09-27T23:29:03.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 0x7fa88c16cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 0x7fa88c16cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41100/0 (socket says 172.21.15.64:41100) 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 -- 172.21.15.64:0/2953852175 learned_addr learned my addr 172.21.15.64:0/2953852175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88b7fe700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c14e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88a7fc700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c06ce90 0x7fa88c14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 -- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c06ce90 msgr2=0x7fa88c14c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c06ce90 0x7fa88c14c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 -- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 msgr2=0x7fa88c14e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c14e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 -- 172.21.15.64:0/2953852175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c009420 con 0x7fa88c06cac0 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.869+0000 7fa88affd700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 0x7fa88c16cee0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa88400c590 tx=0x7fa884007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884005b30 con 0x7fa88c06cac0 2023-09-27T23:29:03.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884005cb0 con 0x7fa88c06cac0 2023-09-27T23:29:03.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c0acca0 con 0x7fa88c06cac0 2023-09-27T23:29:03.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884007320 con 0x7fa88c06cac0 2023-09-27T23:29:03.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c0ad260 con 0x7fa88c06cac0 2023-09-27T23:29:03.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c04be40 con 0x7fa88c06cac0 2023-09-27T23:29:03.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fa88401b3b0 con 0x7fa88c06cac0 2023-09-27T23:29:03.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa882ffd700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa870060a70 0x7fa870062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:03.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6810+0+0 (secure 0 0 0) 0x7fa88407c8c0 con 0x7fa88c06cac0 2023-09-27T23:29:03.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa88a7fc700 1 -- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa870060a70 msgr2=0x7fa870062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:03.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.873+0000 7fa88a7fc700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa870060a70 0x7fa870062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:03.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:03.885+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa88404c740 con 0x7fa88c06cac0 2023-09-27T23:29:03.914 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:03.909+0000 7f7ab9d52000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.045+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa88c0671b0 con 0x7fa88c06cac0 2023-09-27T23:29:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa882ffd700 1 -- 172.21.15.64:0/2953852175 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+17723 (secure 0 0 0) 0x7fa884017020 con 0x7fa88c06cac0 2023-09-27T23:29:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:04.053 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:03.129921+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:26:55.337700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-27T23:26:55.749823+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:27:13.467954+0000","172.21.15.123:6832/33228":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/639937962":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/4106274724":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/2897693597":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3329695117":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3209407868":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/1309068826":"2023-09-28T23:27:13.467954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:04.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa870060a70 msgr2=0x7fa870062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa870060a70 0x7fa870062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 msgr2=0x7fa88c16cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 0x7fa88c16cee0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa88400c590 tx=0x7fa884007e00 comp rx=0 tx=0).stop 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 shutdown_connections 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa870060a70 0x7fa870062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c15cfd0 0x7fa88c14e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c06ce90 0x7fa88c14c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 --2- 172.21.15.64:0/2953852175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c06cac0 0x7fa88c16cee0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 >> 172.21.15.64:0/2953852175 conn(0x7fa88c0cdbd0 msgr2=0x7fa88c0ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 shutdown_connections 2023-09-27T23:29:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.049+0000 7fa88bfff700 1 -- 172.21.15.64:0/2953852175 wait complete. 2023-09-27T23:29:04.065 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.063+0000 7fc50e835000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:29:04.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:26:55.337700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:26:55.749823+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:29:04.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:29:04.113 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.111+0000 7fc50e835000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:04.158 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.153+0000 7f7ab9d52000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:29:04.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 Processor -- start 2023-09-27T23:29:04.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 -- start start 2023-09-27T23:29:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 0x7f64b006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 0x7f64b015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b015da20 con 0x7f64b01594b0 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b006da40 con 0x7f64b006c7c0 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64b4dae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b006cc10 con 0x7f64b006b050 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64aeffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 0x7f64b015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 0x7f64b006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64aeffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 0x7f64b015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:59724/0 (socket says 172.21.15.64:59724) 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 0x7f64b006b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:59140/0 (socket says 172.21.15.64:59140) 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64aeffd700 1 -- 172.21.15.64:0/105887805 learned_addr learned my addr 172.21.15.64:0/105887805 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64ae7fc700 1 -- 172.21.15.64:0/105887805 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265912426 0 0) 0x7f64b015da20 con 0x7f64b01594b0 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64ae7fc700 1 -- 172.21.15.64:0/105887805 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f649c003570 con 0x7f64b01594b0 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 -- 172.21.15.64:0/105887805 >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 msgr2=0x7f64b015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:04.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 --2- 172.21.15.64:0/105887805 >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 0x7f64b015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 -- 172.21.15.64:0/105887805 >> v1:172.21.15.123:6789/0 conn(0x7f64b01594b0 legacy=0x7f64b01598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 -- 172.21.15.64:0/105887805 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b006cec0 con 0x7f64b006b050 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64aeffd700 1 --2- 172.21.15.64:0/105887805 >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 0x7f64b015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64af7fe700 1 --2- 172.21.15.64:0/105887805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 0x7f64b006b420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009e60 tx=0x7f64a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff24635d5f9b5df server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64ae7fc700 1 -- 172.21.15.64:0/105887805 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000e070 con 0x7f64b006b050 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64ae7fc700 1 -- 172.21.15.64:0/105887805 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a000bab0 con 0x7f64b006b050 2023-09-27T23:29:04.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.173+0000 7f64ae7fc700 1 -- 172.21.15.64:0/105887805 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000be70 con 0x7f64b006b050 2023-09-27T23:29:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- 172.21.15.64:0/105887805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 msgr2=0x7f64b006b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 --2- 172.21.15.64:0/105887805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 0x7f64b006b420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009e60 tx=0x7f64a0009780 comp rx=0 tx=0).stop 2023-09-27T23:29:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- 172.21.15.64:0/105887805 shutdown_connections 2023-09-27T23:29:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 --2- 172.21.15.64:0/105887805 >> v2:172.21.15.123:3300/0 conn(0x7f64b006c7c0 0x7f64b015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 --2- 172.21.15.64:0/105887805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006b050 0x7f64b006b420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- 172.21.15.64:0/105887805 >> 172.21.15.64:0/105887805 conn(0x7f64b00c8cd0 msgr2=0x7f64b00c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- 172.21.15.64:0/105887805 shutdown_connections 2023-09-27T23:29:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- 172.21.15.64:0/105887805 wait complete. 2023-09-27T23:29:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 Processor -- start 2023-09-27T23:29:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- start start 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b006b050 0x7f64b00b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006c7c0 0x7f64b0156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 0x7f64b00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00b1f90 con 0x7f64b01594b0 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00b0ef0 con 0x7f64b006b050 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64b4dae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00b11d0 con 0x7f64b006c7c0 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64affff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 0x7f64b00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64af7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b006b050 0x7f64b00b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64aeffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006c7c0 0x7f64b0156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64aeffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006c7c0 0x7f64b0156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:59146/0 (socket says 172.21.15.64:59146) 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64affff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 0x7f64b00c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41106/0 (socket says 172.21.15.64:41106) 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64af7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b006b050 0x7f64b00b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:59728/0 (socket says 172.21.15.64:59728) 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.177+0000 7f64aeffd700 1 -- 172.21.15.64:0/3081800835 learned_addr learned my addr 172.21.15.64:0/3081800835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64affff700 1 -- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006c7c0 msgr2=0x7f64b0156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64affff700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006c7c0 0x7f64b0156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64affff700 1 -- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b006b050 msgr2=0x7f64b00b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64affff700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b006b050 0x7f64b00b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64affff700 1 -- 172.21.15.64:0/3081800835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980094f0 con 0x7f64b01594b0 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64affff700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 0x7f64b00c7ae0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c470 tx=0x7f64a8007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8012420 con 0x7f64b01594b0 2023-09-27T23:29:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a80125a0 con 0x7f64b01594b0 2023-09-27T23:29:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8012960 con 0x7f64b01594b0 2023-09-27T23:29:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a0009a20 con 0x7f64b01594b0 2023-09-27T23:29:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b00b1790 con 0x7f64b01594b0 2023-09-27T23:29:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b004be40 con 0x7f64b01594b0 2023-09-27T23:29:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f64a801b400 con 0x7f64b01594b0 2023-09-27T23:29:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64a77fe700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6490060a70 0x7f6490062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64af7fe700 1 -- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6490060a70 msgr2=0x7f6490062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f64a8002030 con 0x7f64b01594b0 2023-09-27T23:29:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.181+0000 7f64af7fe700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6490060a70 0x7f6490062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:04.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.189+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f64a804ca10 con 0x7f64b01594b0 2023-09-27T23:29:04.210 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.205+0000 7f7ab9d52000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:04.269 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.267+0000 7fc50e835000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:29:04.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.323+0000 7fc50e835000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:29:04.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.345+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64b006d1a0 con 0x7f64b01594b0 2023-09-27T23:29:04.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64a77fe700 1 -- 172.21.15.64:0/3081800835 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64a8050030 con 0x7f64b01594b0 2023-09-27T23:29:04.352 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6490060a70 msgr2=0x7f6490062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6490060a70 0x7f6490062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 msgr2=0x7f64b00c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 0x7f64b00c7ae0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c470 tx=0x7f64a8007cf0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 shutdown_connections 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6490060a70 0x7f6490062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64b01594b0 0x7f64b00c7ae0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64b006c7c0 0x7f64b0156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 --2- 172.21.15.64:0/3081800835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b006b050 0x7f64b00b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 >> 172.21.15.64:0/3081800835 conn(0x7f64b00c8cd0 msgr2=0x7f64b015be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 shutdown_connections 2023-09-27T23:29:04.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.349+0000 7f64b4dae700 1 -- 172.21.15.64:0/3081800835 wait complete. 2023-09-27T23:29:04.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:29:04.368 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.365+0000 7f7ab9d52000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:29:04.393 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.391+0000 7fc50e835000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:29:04.427 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.421+0000 7f7ab9d52000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:29:04.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 Processor -- start 2023-09-27T23:29:04.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 -- start start 2023-09-27T23:29:04.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 0x7f1ae40ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ae406d390 0x7f1ae406d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae40ae110 con 0x7f1ae40ace10 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae4157110 con 0x7f1ae406d390 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeb2e2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae406e0c0 con 0x7f1ae40c8bf0 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aea2e0700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1ae40ace10 0x7f1ae40c84e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43026/0 (socket says 172.21.15.64:43026) 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aea2e0700 1 -- 172.21.15.64:0/591519936 learned_addr learned my addr 172.21.15.64:0/591519936 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae9adf700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 0x7f1ae40ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aeaae1700 1 --2- 172.21.15.64:0/591519936 >> v2:172.21.15.123:3300/0 conn(0x7f1ae406d390 0x7f1ae406d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656297406 0 0) 0x7f1ae40ae110 con 0x7f1ae40ace10 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad0003570 con 0x7f1ae40ace10 2023-09-27T23:29:04.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1725535227 0 0) 0x7f1ad0003570 con 0x7f1ae40ace10 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 msgr2=0x7f1ae40ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 0x7f1ae40ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 >> v2:172.21.15.123:3300/0 conn(0x7f1ae406d390 msgr2=0x7f1ae406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 --2- 172.21.15.64:0/591519936 >> v2:172.21.15.123:3300/0 conn(0x7f1ae406d390 0x7f1ae406d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40ad7e0 con 0x7f1ae40ace10 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f1ae0001470 con 0x7f1ae40ace10 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 >> v1:172.21.15.64:6789/0 conn(0x7f1ae40ace10 legacy=0x7f1ae40c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ad0003a60 0x7f1ad0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 0x7f1ad0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ad0004d50 0x7f1ad00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae4157110 con 0x7f1ad0003a60 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae406e0c0 con 0x7f1ad0004d50 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad000ba60 con 0x7f1ae40c8bf0 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1aea2e0700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ad0003a60 0x7f1ad0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.473+0000 7f1ae9adf700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 0x7f1ad0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeaae1700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ad0004d50 0x7f1ad00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aea2e0700 1 -- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 msgr2=0x7f1ad0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aea2e0700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c8bf0 0x7f1ad0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aea2e0700 1 -- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ad0004d50 msgr2=0x7f1ad00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aea2e0700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ad0004d50 0x7f1ad00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aea2e0700 1 -- 172.21.15.64:0/591519936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0003570 con 0x7f1ad0003a60 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeaae1700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ad0004d50 0x7f1ad00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aea2e0700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ad0003a60 0x7f1ad0003e70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c3c0 tx=0x7f1ae0001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d126d5fe54be7e18 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae000cc30 con 0x7f1ad0003a60 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae000cdb0 con 0x7f1ad0003a60 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1ae92de700 1 -- 172.21.15.64:0/591519936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0005b70 con 0x7f1ad0003a60 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/591519936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ad0003a60 msgr2=0x7f1ad0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/591519936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ad0003a60 0x7f1ad0003e70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c3c0 tx=0x7f1ae0001e90 comp rx=0 tx=0).stop 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeaae1700 1 -- 172.21.15.64:0/591519936 reap_dead start 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/591519936 shutdown_connections 2023-09-27T23:29:04.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/591519936 >> 172.21.15.64:0/591519936 conn(0x7f1ae40cdc00 msgr2=0x7f1ae40ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/591519936 shutdown_connections 2023-09-27T23:29:04.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/591519936 wait complete. 2023-09-27T23:29:04.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 Processor -- start 2023-09-27T23:29:04.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.477+0000 7f1aeb2e2700 1 -- start start 2023-09-27T23:29:04.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae40c6f00 0x7f1ae416a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 0x7f1ae415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c72d0 0x7f1ae40b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40b0d50 con 0x7f1ae415a9d0 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416b130 con 0x7f1ae40c6f00 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416b410 con 0x7f1ae40c72d0 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 0x7f1ae415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 0x7f1ae415ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41110/0 (socket says 172.21.15.64:41110) 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 -- 172.21.15.64:0/531697465 learned_addr learned my addr 172.21.15.64:0/531697465 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aea2e0700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae40c6f00 0x7f1ae416a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeaae1700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c72d0 0x7f1ae40b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 -- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c72d0 msgr2=0x7f1ae40b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c72d0 0x7f1ae40b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 -- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae40c6f00 msgr2=0x7f1ae416a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae40c6f00 0x7f1ae416a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 -- 172.21.15.64:0/531697465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae416b6f0 con 0x7f1ae415a9d0 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ae9adf700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 0x7f1ae415ae90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4011d40 tx=0x7f1ad40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400f990 con 0x7f1ae415a9d0 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad400faf0 con 0x7f1ae415a9d0 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400feb0 con 0x7f1ae415a9d0 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae40b09e0 con 0x7f1ae415a9d0 2023-09-27T23:29:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae415b3d0 con 0x7f1ae415a9d0 2023-09-27T23:29:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae40b0b70 con 0x7f1ae415a9d0 2023-09-27T23:29:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.481+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f1ad4014650 con 0x7f1ae415a9d0 2023-09-27T23:29:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.485+0000 7f1ada7fc700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ac8060a70 0x7f1ac8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.485+0000 7f1aea2e0700 1 -- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ac8060a70 msgr2=0x7f1ac8062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.485+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f1ad4018030 con 0x7f1ae415a9d0 2023-09-27T23:29:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.485+0000 7f1aea2e0700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ac8060a70 0x7f1ac8062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:04.491 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.485+0000 7f7ab9d52000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:29:04.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.489+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1ad404df10 con 0x7f1ae415a9d0 2023-09-27T23:29:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ae404be40 con 0x7f1ae415a9d0 2023-09-27T23:29:04.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1ada7fc700 1 -- 172.21.15.64:0/531697465 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1ad4051530 con 0x7f1ae415a9d0 2023-09-27T23:29:04.661 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ac8060a70 msgr2=0x7f1ac8062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ac8060a70 0x7f1ac8062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 msgr2=0x7f1ae415ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 0x7f1ae415ae90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4011d40 tx=0x7f1ad40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 shutdown_connections 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ac8060a70 0x7f1ac8062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40c72d0 0x7f1ae40b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ae415a9d0 0x7f1ae415ae90 secure :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4011d40 tx=0x7f1ad40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 --2- 172.21.15.64:0/531697465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae40c6f00 0x7f1ae416a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 >> 172.21.15.64:0/531697465 conn(0x7f1ae40cdc00 msgr2=0x7f1ae40ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 shutdown_connections 2023-09-27T23:29:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.657+0000 7f1aeb2e2700 1 -- 172.21.15.64:0/531697465 wait complete. 2023-09-27T23:29:04.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:29:04.724 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.719+0000 7fc50e835000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:29:04.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 Processor -- start 2023-09-27T23:29:04.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 -- start start 2023-09-27T23:29:04.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 0x7f26b00ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26b00ca2c0 0x7f26b00ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b00cbdf0 con 0x7f26b00cb970 2023-09-27T23:29:04.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b00cc0d0 con 0x7f26b00ca2c0 2023-09-27T23:29:04.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26b5fed700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00cc3b0 con 0x7f26b0158e20 2023-09-27T23:29:04.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 0x7f26b00ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26affff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26b00ca2c0 0x7f26b00ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 0x7f26b00ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:39270/0 (socket says 172.21.15.64:39270) 2023-09-27T23:29:04.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 -- 172.21.15.64:0/1259217663 learned_addr learned my addr 172.21.15.64:0/1259217663 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:04.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26aeffd700 1 -- 172.21.15.64:0/1259217663 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579585013 0 0) 0x7f26b00cbdf0 con 0x7f26b00cb970 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 -- 172.21.15.64:0/1259217663 >> v2:172.21.15.123:3300/0 conn(0x7f26b00ca2c0 msgr2=0x7f26b00ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 --2- 172.21.15.64:0/1259217663 >> v2:172.21.15.123:3300/0 conn(0x7f26b00ca2c0 0x7f26b00ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 -- 172.21.15.64:0/1259217663 >> v1:172.21.15.123:6789/0 conn(0x7f26b00cb970 legacy=0x7f26b006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 -- 172.21.15.64:0/1259217663 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00cc690 con 0x7f26b0158e20 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26af7fe700 1 --2- 172.21.15.64:0/1259217663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 0x7f26b00ce5c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f26a00099b0 tx=0x7f26a0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26bd8900afe586ae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26aeffd700 1 -- 172.21.15.64:0/1259217663 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0009ea0 con 0x7f26b0158e20 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26aeffd700 1 -- 172.21.15.64:0/1259217663 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a000b860 con 0x7f26b0158e20 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.777+0000 7f26aeffd700 1 -- 172.21.15.64:0/1259217663 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000bc00 con 0x7f26b0158e20 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- 172.21.15.64:0/1259217663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 msgr2=0x7f26b00ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 --2- 172.21.15.64:0/1259217663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 0x7f26b00ce5c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f26a00099b0 tx=0x7f26a0009250 comp rx=0 tx=0).stop 2023-09-27T23:29:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- 172.21.15.64:0/1259217663 shutdown_connections 2023-09-27T23:29:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 --2- 172.21.15.64:0/1259217663 >> v2:172.21.15.123:3300/0 conn(0x7f26b00ca2c0 0x7f26b00ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 --2- 172.21.15.64:0/1259217663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b0158e20 0x7f26b00ce5c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- 172.21.15.64:0/1259217663 >> 172.21.15.64:0/1259217663 conn(0x7f26b0067bb0 msgr2=0x7f26b00d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- 172.21.15.64:0/1259217663 shutdown_connections 2023-09-27T23:29:04.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- 172.21.15.64:0/1259217663 wait complete. 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 Processor -- start 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- start start 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26b00ca2c0 0x7f26b0169410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 0x7f26b00c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b00b4480 0x7f26b014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00c7c20 con 0x7f26b0158e20 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00c7e50 con 0x7f26b00ca2c0 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26b5fed700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00c8130 con 0x7f26b00b4480 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 0x7f26b00c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 0x7f26b00c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50052/0 (socket says 172.21.15.64:50052) 2023-09-27T23:29:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 -- 172.21.15.64:0/3121541705 learned_addr learned my addr 172.21.15.64:0/3121541705 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26af7fe700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26b00ca2c0 0x7f26b0169410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26affff700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b00b4480 0x7f26b014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 -- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b00b4480 msgr2=0x7f26b014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b00b4480 0x7f26b014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 -- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26b00ca2c0 msgr2=0x7f26b0169410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26b00ca2c0 0x7f26b0169410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.781+0000 7f26a6ffd700 1 -- 172.21.15.64:0/3121541705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00094f0 con 0x7f26b0158e20 2023-09-27T23:29:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a6ffd700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 0x7f26b00c76e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f269800c7e0 tx=0x7f2698007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011590 con 0x7f26b0158e20 2023-09-27T23:29:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26980116f0 con 0x7f26b0158e20 2023-09-27T23:29:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b00c8410 con 0x7f26b0158e20 2023-09-27T23:29:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011ab0 con 0x7f26b0158e20 2023-09-27T23:29:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b00c8cb0 con 0x7f26b0158e20 2023-09-27T23:29:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b00c8ed0 con 0x7f26b0158e20 2023-09-27T23:29:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f269801c460 con 0x7f26b0158e20 2023-09-27T23:29:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a7fff700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2690060a70 0x7f2690062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f269807cfa0 con 0x7f26b0158e20 2023-09-27T23:29:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26af7fe700 1 -- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2690060a70 msgr2=0x7f2690062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.785+0000 7f26af7fe700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2690060a70 0x7f2690062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.797+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f26980acbb0 con 0x7f26b0158e20 2023-09-27T23:29:04.832 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.829+0000 7f7ab9d52000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:29:04.847 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:04.843+0000 7fc50e835000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:29:04.956 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:04.953+0000 7f7ab9d52000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:29:04.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.953+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26b0071120 con 0x7f26b0158e20 2023-09-27T23:29:04.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26a7fff700 1 -- 172.21.15.64:0/3121541705 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f269804ce30 con 0x7f26b0158e20 2023-09-27T23:29:04.960 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2690060a70 msgr2=0x7f2690062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2690060a70 0x7f2690062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 msgr2=0x7f26b00c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 0x7f26b00c76e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f269800c7e0 tx=0x7f2698007de0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 shutdown_connections 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2690060a70 0x7f2690062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26b00b4480 0x7f26b014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26b0158e20 0x7f26b00c76e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 --2- 172.21.15.64:0/3121541705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26b00ca2c0 0x7f26b0169410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 >> 172.21.15.64:0/3121541705 conn(0x7f26b0067bb0 msgr2=0x7f26b01562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 shutdown_connections 2023-09-27T23:29:04.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:04.957+0000 7f26b5fed700 1 -- 172.21.15.64:0/3121541705 wait complete. 2023-09-27T23:29:04.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:29:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 Processor -- start 2023-09-27T23:29:05.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 -- start start 2023-09-27T23:29:05.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7dc158c90 0x7ff7dc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7dc159770 0x7ff7dc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc06c850 con 0x7ff7dc06aeb0 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc06b470 con 0x7ff7dc159770 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7db59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc06b750 con 0x7ff7dc158c90 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7dc159770 0x7ff7dc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7dc159770 0x7ff7dc06cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50058/0 (socket says 172.21.15.64:50058) 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 -- 172.21.15.64:0/1580444956 learned_addr learned my addr 172.21.15.64:0/1580444956 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7da59c700 1 --2- 172.21.15.64:0/1580444956 >> v2:172.21.15.123:3300/0 conn(0x7ff7dc158c90 0x7ff7dc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 -- 172.21.15.64:0/1580444956 >> v2:172.21.15.123:3300/0 conn(0x7ff7dc158c90 msgr2=0x7ff7dc0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 --2- 172.21.15.64:0/1580444956 >> v2:172.21.15.123:3300/0 conn(0x7ff7dc158c90 0x7ff7dc0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 -- 172.21.15.64:0/1580444956 >> v1:172.21.15.123:6789/0 conn(0x7ff7dc06aeb0 legacy=0x7ff7dc159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.077+0000 7ff7dad9d700 1 -- 172.21.15.64:0/1580444956 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc06ba30 con 0x7ff7dc159770 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7dad9d700 1 --2- 172.21.15.64:0/1580444956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc159770 0x7ff7dc06cce0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0698f0 tx=0x7ff7d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66799ee5db791032 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7d959a700 1 -- 172.21.15.64:0/1580444956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400e070 con 0x7ff7dc159770 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7d959a700 1 -- 172.21.15.64:0/1580444956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d400ba20 con 0x7ff7dc159770 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7d959a700 1 -- 172.21.15.64:0/1580444956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40092e0 con 0x7ff7dc159770 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 -- 172.21.15.64:0/1580444956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc159770 msgr2=0x7ff7dc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 --2- 172.21.15.64:0/1580444956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc159770 0x7ff7dc06cce0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0698f0 tx=0x7ff7d4009840 comp rx=0 tx=0).stop 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 -- 172.21.15.64:0/1580444956 shutdown_connections 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 --2- 172.21.15.64:0/1580444956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc159770 0x7ff7dc06cce0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 --2- 172.21.15.64:0/1580444956 >> v2:172.21.15.123:3300/0 conn(0x7ff7dc158c90 0x7ff7dc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 -- 172.21.15.64:0/1580444956 >> 172.21.15.64:0/1580444956 conn(0x7ff7dc067bb0 msgr2=0x7ff7dc0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 -- 172.21.15.64:0/1580444956 shutdown_connections 2023-09-27T23:29:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 -- 172.21.15.64:0/1580444956 wait complete. 2023-09-27T23:29:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 Processor -- start 2023-09-27T23:29:05.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.081+0000 7ff7db59e700 1 -- start start 2023-09-27T23:29:05.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7dc06aeb0 0x7ff7dc152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 0x7ff7dc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7dc159770 0x7ff7dc0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0c9c00 con 0x7ff7dc158c90 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0c51b0 con 0x7ff7dc06aeb0 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0c5490 con 0x7ff7dc159770 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 0x7ff7dc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 0x7ff7dc0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50060/0 (socket says 172.21.15.64:50060) 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 -- 172.21.15.64:0/788182170 learned_addr learned my addr 172.21.15.64:0/788182170 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7dad9d700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7dc159770 0x7ff7dc0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7da59c700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7dc06aeb0 0x7ff7dc152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 -- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7dc159770 msgr2=0x7ff7dc0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7dc159770 0x7ff7dc0c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 -- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7dc06aeb0 msgr2=0x7ff7dc152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7dc06aeb0 0x7ff7dc152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 -- 172.21.15.64:0/788182170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009ae0 con 0x7ff7dc158c90 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7d9d9b700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 0x7ff7dc0c9180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c590 tx=0x7ff7d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0011590 con 0x7ff7dc158c90 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0011710 con 0x7ff7dc158c90 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc0c5770 con 0x7ff7dc158c90 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0011ad0 con 0x7ff7dc158c90 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc0c5d30 con 0x7ff7dc158c90 2023-09-27T23:29:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.085+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc04be40 con 0x7ff7dc158c90 2023-09-27T23:29:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.089+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7ff7d001b3b0 con 0x7ff7dc158c90 2023-09-27T23:29:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.089+0000 7ff7ca7fc700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff7bc060a70 0x7ff7bc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.089+0000 7ff7da59c700 1 -- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff7bc060a70 msgr2=0x7ff7bc062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.089+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7ff7d007c740 con 0x7ff7dc158c90 2023-09-27T23:29:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.089+0000 7ff7da59c700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff7bc060a70 0x7ff7bc062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.093+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff7d004c5d0 con 0x7ff7dc158c90 2023-09-27T23:29:05.155 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.151+0000 7fc50e835000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:29:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7dc06bd10 con 0x7ff7dc158c90 2023-09-27T23:29:05.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7ca7fc700 1 -- 172.21.15.64:0/788182170 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7d00170a0 con 0x7ff7dc158c90 2023-09-27T23:29:05.252 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff7bc060a70 msgr2=0x7ff7bc062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff7bc060a70 0x7ff7bc062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 msgr2=0x7ff7dc0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 0x7ff7dc0c9180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c590 tx=0x7ff7d000c8a0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 shutdown_connections 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff7bc060a70 0x7ff7bc062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7dc159770 0x7ff7dc0c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7dc158c90 0x7ff7dc0c9180 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 --2- 172.21.15.64:0/788182170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7dc06aeb0 0x7ff7dc152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 >> 172.21.15.64:0/788182170 conn(0x7ff7dc067bb0 msgr2=0x7ff7dc0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 shutdown_connections 2023-09-27T23:29:05.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.249+0000 7ff7db59e700 1 -- 172.21.15.64:0/788182170 wait complete. 2023-09-27T23:29:05.265 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.261+0000 7f7ab9d52000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:29:05.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:05.316 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.311+0000 7fc50e835000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 Processor -- start 2023-09-27T23:29:05.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 -- start start 2023-09-27T23:29:05.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303406ad70 0x7f303406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f303414f4d0 0x7f303414f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340afad0 con 0x7f303406e1d0 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303406b750 con 0x7f303414f4d0 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303ac25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340afdb0 con 0x7f303406ad70 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f303414f4d0 0x7f303414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f303414f4d0 0x7f303414f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50072/0 (socket says 172.21.15.64:50072) 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 -- 172.21.15.64:0/1854160542 learned_addr learned my addr 172.21.15.64:0/1854160542 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f3039422700 1 --2- 172.21.15.64:0/1854160542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303406ad70 0x7f303406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f3038c21700 1 -- 172.21.15.64:0/1854160542 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883377357 0 0) 0x7f30340afad0 con 0x7f303406e1d0 2023-09-27T23:29:05.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 -- 172.21.15.64:0/1854160542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303406ad70 msgr2=0x7f303406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 --2- 172.21.15.64:0/1854160542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303406ad70 0x7f303406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 -- 172.21.15.64:0/1854160542 >> v1:172.21.15.123:6789/0 conn(0x7f303406e1d0 legacy=0x7f3034159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.361+0000 7f303a424700 1 -- 172.21.15.64:0/1854160542 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340b0090 con 0x7f303414f4d0 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303a424700 1 --2- 172.21.15.64:0/1854160542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303414f4d0 0x7f303414f930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f302c009f20 tx=0x7f302c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c38e3a684f1c94b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f3038c21700 1 -- 172.21.15.64:0/1854160542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00e070 con 0x7f303414f4d0 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f3038c21700 1 -- 172.21.15.64:0/1854160542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00b9d0 con 0x7f303414f4d0 2023-09-27T23:29:05.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f3038c21700 1 -- 172.21.15.64:0/1854160542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0092e0 con 0x7f303414f4d0 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 -- 172.21.15.64:0/1854160542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303414f4d0 msgr2=0x7f303414f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 --2- 172.21.15.64:0/1854160542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303414f4d0 0x7f303414f930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f302c009f20 tx=0x7f302c009840 comp rx=0 tx=0).stop 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 -- 172.21.15.64:0/1854160542 shutdown_connections 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 --2- 172.21.15.64:0/1854160542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303414f4d0 0x7f303414f930 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 --2- 172.21.15.64:0/1854160542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303406ad70 0x7f303406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 -- 172.21.15.64:0/1854160542 >> 172.21.15.64:0/1854160542 conn(0x7f3034067bb0 msgr2=0x7f30340add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 -- 172.21.15.64:0/1854160542 shutdown_connections 2023-09-27T23:29:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 -- 172.21.15.64:0/1854160542 wait complete. 2023-09-27T23:29:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 Processor -- start 2023-09-27T23:29:05.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.365+0000 7f303ac25700 1 -- start start 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 0x7f3034148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f303406e1d0 0x7f30341480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303414f4d0 0x7f30340c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303414bcc0 con 0x7f303406ad70 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303414b9e0 con 0x7f303406e1d0 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303414bff0 con 0x7f303414f4d0 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 0x7f3034148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 0x7f3034148d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50078/0 (socket says 172.21.15.64:50078) 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 -- 172.21.15.64:0/2622612344 learned_addr learned my addr 172.21.15.64:0/2622612344 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303a424700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303414f4d0 0x7f30340c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039422700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f303406e1d0 0x7f30341480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 -- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303414f4d0 msgr2=0x7f30340c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303414f4d0 0x7f30340c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 -- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f303406e1d0 msgr2=0x7f30341480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f303406e1d0 0x7f30341480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 -- 172.21.15.64:0/2622612344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c009ae0 con 0x7f303406ad70 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3039c23700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 0x7f3034148d10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f303000c7e0 tx=0x7f3030007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030012700 con 0x7f303406ad70 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030012880 con 0x7f303406ad70 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303414c2d0 con 0x7f303406ad70 2023-09-27T23:29:05.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030012c40 con 0x7f303406ad70 2023-09-27T23:29:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341485f0 con 0x7f303406ad70 2023-09-27T23:29:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303414c7d0 con 0x7f303406ad70 2023-09-27T23:29:05.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.369+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f303001b400 con 0x7f303406ad70 2023-09-27T23:29:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.373+0000 7f3029ffb700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 0x7f3018062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.373+0000 7f3039422700 1 -- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 msgr2=0x7f3018062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.373+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f303007cea0 con 0x7f303406ad70 2023-09-27T23:29:05.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.373+0000 7f3039422700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 0x7f3018062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:05.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.377+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f303004cd30 con 0x7f303406ad70 2023-09-27T23:29:05.381 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.379+0000 7fc50e835000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:29:05.434 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.431+0000 7fc50e835000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:29:05.435 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.429+0000 7f7ab9d52000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:05.488 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.483+0000 7fc50e835000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:29:05.503 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.497+0000 7f7ab9d52000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:29:05.557 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.553+0000 7f7ab9d52000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:29:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.573+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f303404be40 con 0x7f303406ad70 2023-09-27T23:29:05.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.573+0000 7f3029ffb700 1 -- 172.21.15.64:0/2622612344 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f3030050350 con 0x7f303406ad70 2023-09-27T23:29:05.579 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:05.579 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:26:56.779553+0000","modified":"2023-09-27T23:26:59.790212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2178143597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2178143597},{"type":"v1","addr":"172.21.15.64:6835","nonce":2178143597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-09-27T23:29:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f3039422700 1 -- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 msgr2=0x7f3018062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f3039422700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 0x7f3018062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 msgr2=0x7f3018062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 0x7f3018062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 msgr2=0x7f3034148d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 0x7f3034148d10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f303000c7e0 tx=0x7f3030007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 shutdown_connections 2023-09-27T23:29:05.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3018060a70 0x7f3018062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f303414f4d0 0x7f30340c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f303406e1d0 0x7f30341480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 --2- 172.21.15.64:0/2622612344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f303406ad70 0x7f3034148d10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 >> 172.21.15.64:0/2622612344 conn(0x7f3034067bb0 msgr2=0x7f303406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 shutdown_connections 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.577+0000 7f303ac25700 1 -- 172.21.15.64:0/2622612344 wait complete. 2023-09-27T23:29:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-09-27T23:29:05.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:05.616 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.613+0000 7f7ab9d52000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:29:05.661 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.659+0000 7fc50e835000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.689+0000 7fb77fa2c700 1 Processor -- start 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.689+0000 7fb77fa2c700 1 -- start start 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb77815d340 0x7fb77815d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb77806b8b0 0x7fb77806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77806cf90 con 0x7fb7780b0230 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780b0f60 con 0x7fb77815d340 2023-09-27T23:29:05.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780b0c80 con 0x7fb77806b8b0 2023-09-27T23:29:05.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb77815d340 0x7fb77815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb77815d340 0x7fb77815d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50090/0 (socket says 172.21.15.64:50090) 2023-09-27T23:29:05.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 -- 172.21.15.64:0/973810511 learned_addr learned my addr 172.21.15.64:0/973810511 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77dfc9700 1 --2- 172.21.15.64:0/973810511 >> v2:172.21.15.123:3300/0 conn(0x7fb77806b8b0 0x7fb77806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 -- 172.21.15.64:0/973810511 >> v2:172.21.15.123:3300/0 conn(0x7fb77806b8b0 msgr2=0x7fb77806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 --2- 172.21.15.64:0/973810511 >> v2:172.21.15.123:3300/0 conn(0x7fb77806b8b0 0x7fb77806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 -- 172.21.15.64:0/973810511 >> v1:172.21.15.64:6789/0 conn(0x7fb7780b0230 legacy=0x7fb77806b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb76ffff700 1 -- 172.21.15.64:0/973810511 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183901 0 0) 0x7fb77806cf90 con 0x7fb7780b0230 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 -- 172.21.15.64:0/973810511 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77806c940 con 0x7fb77815d340 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77d7c8700 1 --2- 172.21.15.64:0/973810511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb77815d340 0x7fb77815d710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb7780cc660 tx=0x7fb774009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3261bf22954a516 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb76ffff700 1 -- 172.21.15.64:0/973810511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774009e70 con 0x7fb77815d340 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb76ffff700 1 -- 172.21.15.64:0/973810511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77400b860 con 0x7fb77815d340 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb76ffff700 1 -- 172.21.15.64:0/973810511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400bc00 con 0x7fb77815d340 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- 172.21.15.64:0/973810511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb77815d340 msgr2=0x7fb77815d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/973810511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb77815d340 0x7fb77815d710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb7780cc660 tx=0x7fb774009250 comp rx=0 tx=0).stop 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- 172.21.15.64:0/973810511 shutdown_connections 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/973810511 >> v2:172.21.15.123:3300/0 conn(0x7fb77806b8b0 0x7fb77806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/973810511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb77815d340 0x7fb77815d710 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- 172.21.15.64:0/973810511 >> 172.21.15.64:0/973810511 conn(0x7fb778067410 msgr2=0x7fb7780677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- 172.21.15.64:0/973810511 shutdown_connections 2023-09-27T23:29:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.693+0000 7fb77fa2c700 1 -- 172.21.15.64:0/973810511 wait complete. 2023-09-27T23:29:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 Processor -- start 2023-09-27T23:29:05.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 -- start start 2023-09-27T23:29:05.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77806b8b0 0x7fb778155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 0x7fb7780cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77815d340 0x7fb778170b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7781560b0 con 0x7fb7780b0230 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780ad910 con 0x7fb77806b8b0 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780adb60 con 0x7fb77815d340 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 0x7fb7780cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 0x7fb7780cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50106/0 (socket says 172.21.15.64:50106) 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 -- 172.21.15.64:0/783856720 learned_addr learned my addr 172.21.15.64:0/783856720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77d7c8700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77806b8b0 0x7fb778155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77dfc9700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77815d340 0x7fb778170b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 -- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77815d340 msgr2=0x7fb778170b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77815d340 0x7fb778170b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 -- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77806b8b0 msgr2=0x7fb778155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77806b8b0 0x7fb778155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 -- 172.21.15.64:0/783856720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740094f0 con 0x7fb7780b0230 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77cfc7700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 0x7fb7780cb1c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c7e0 tx=0x7fb768007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680105c0 con 0x7fb7780b0230 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768010740 con 0x7fb7780b0230 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768010b00 con 0x7fb7780b0230 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7780ade40 con 0x7fb7780b0230 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.697+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780ae400 con 0x7fb7780b0230 2023-09-27T23:29:05.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.701+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fb76801b3b0 con 0x7fb7780b0230 2023-09-27T23:29:05.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.701+0000 7fb76d7fa700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb75c060a70 0x7fb75c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.701+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7fb76807c870 con 0x7fb7780b0230 2023-09-27T23:29:05.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.701+0000 7fb77d7c8700 1 -- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb75c060a70 msgr2=0x7fb75c062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:05.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.701+0000 7fb77d7c8700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb75c060a70 0x7fb75c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:05.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.705+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778071120 con 0x7fb7780b0230 2023-09-27T23:29:05.715 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.711+0000 7fc50e835000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:29:05.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.709+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb768017080 con 0x7fb7780b0230 2023-09-27T23:29:05.778 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.775+0000 7fc50e835000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:29:05.798 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.793+0000 7f7ab9d52000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:05.854 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.849+0000 7f7ab9d52000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:29:05.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.869+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb77806cc20 con 0x7fb7780b0230 2023-09-27T23:29:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.873+0000 7fb76d7fa700 1 -- 172.21.15.64:0/783856720 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+17700 (secure 0 0 0) 0x7fb768050190 con 0x7fb7780b0230 2023-09-27T23:29:05.876 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:05.876 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":74,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:04.133048+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:26:55.337700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-27T23:26:55.749823+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:27:13.467954+0000","172.21.15.123:6832/33228":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/639937962":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/4106274724":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/2897693597":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3329695117":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3209407868":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/1309068826":"2023-09-28T23:27:13.467954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.873+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb75c060a70 msgr2=0x7fb75c062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.873+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb75c060a70 0x7fb75c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.873+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 msgr2=0x7fb7780cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.873+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 0x7fb7780cb1c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c7e0 tx=0x7fb768007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 shutdown_connections 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb75c060a70 0x7fb75c062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77815d340 0x7fb778170b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7780b0230 0x7fb7780cb1c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 --2- 172.21.15.64:0/783856720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77806b8b0 0x7fb778155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 >> 172.21.15.64:0/783856720 conn(0x7fb778067410 msgr2=0x7fb77815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 shutdown_connections 2023-09-27T23:29:05.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.877+0000 7fb77fa2c700 1 -- 172.21.15.64:0/783856720 wait complete. 2023-09-27T23:29:05.897 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:29:05.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:05.922 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:05.917+0000 7f7ab9d52000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:29:05.925 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.923+0000 7fc50e835000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:29:05.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:05.983+0000 7fc50e835000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:29:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 Processor -- start 2023-09-27T23:29:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 -- start start 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5360158df0 0x7f536006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f536006d590 0x7f53601591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360159700 0x7f536015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536006da10 con 0x7f5360158df0 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5360159f90 con 0x7f536006d590 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.985+0000 7f535f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536015a270 con 0x7f5360159700 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5360158df0 0x7f536006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5360158df0 0x7f536006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50120/0 (socket says 172.21.15.64:50120) 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 -- 172.21.15.64:0/3805170645 learned_addr learned my addr 172.21.15.64:0/3805170645 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535ed9d700 1 --2- 172.21.15.64:0/3805170645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360159700 0x7f536015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535dd9b700 1 --2- 172.21.15.64:0/3805170645 >> v2:172.21.15.123:3300/0 conn(0x7f536006d590 0x7f53601591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 -- 172.21.15.64:0/3805170645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360159700 msgr2=0x7f536015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 --2- 172.21.15.64:0/3805170645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360159700 0x7f536015ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 -- 172.21.15.64:0/3805170645 >> v2:172.21.15.123:3300/0 conn(0x7f536006d590 msgr2=0x7f53601591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 --2- 172.21.15.64:0/3805170645 >> v2:172.21.15.123:3300/0 conn(0x7f536006d590 0x7f53601591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 -- 172.21.15.64:0/3805170645 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53600c7930 con 0x7f5360158df0 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535dd9b700 1 --2- 172.21.15.64:0/3805170645 >> v2:172.21.15.123:3300/0 conn(0x7f536006d590 0x7f53601591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535e59c700 1 --2- 172.21.15.64:0/3805170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360158df0 0x7f536006d050 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f536006b3e0 tx=0x7f5350009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8f1effd43dac93 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535d59a700 1 -- 172.21.15.64:0/3805170645 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350009e70 con 0x7f5360158df0 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535d59a700 1 -- 172.21.15.64:0/3805170645 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535000b860 con 0x7f5360158df0 2023-09-27T23:29:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535d59a700 1 -- 172.21.15.64:0/3805170645 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535000bc20 con 0x7f5360158df0 2023-09-27T23:29:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 -- 172.21.15.64:0/3805170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360158df0 msgr2=0x7f536006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 --2- 172.21.15.64:0/3805170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360158df0 0x7f536006d050 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f536006b3e0 tx=0x7f5350009250 comp rx=0 tx=0).stop 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 -- 172.21.15.64:0/3805170645 shutdown_connections 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 --2- 172.21.15.64:0/3805170645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360159700 0x7f536015ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 --2- 172.21.15.64:0/3805170645 >> v2:172.21.15.123:3300/0 conn(0x7f536006d590 0x7f53601591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 --2- 172.21.15.64:0/3805170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5360158df0 0x7f536006d050 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 -- 172.21.15.64:0/3805170645 >> 172.21.15.64:0/3805170645 conn(0x7f536006c080 msgr2=0x7f53600ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 -- 172.21.15.64:0/3805170645 shutdown_connections 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 -- 172.21.15.64:0/3805170645 wait complete. 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 Processor -- start 2023-09-27T23:29:05.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.989+0000 7f535f59e700 1 -- start start 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 0x7f5360169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360158df0 0x7f53600b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5360159700 0x7f53600b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360169e50 con 0x7f536006d590 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536015bab0 con 0x7f5360159700 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536015bd90 con 0x7f5360158df0 2023-09-27T23:29:05.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 0x7f5360169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 0x7f5360169910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50134/0 (socket says 172.21.15.64:50134) 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 -- 172.21.15.64:0/335853959 learned_addr learned my addr 172.21.15.64:0/335853959 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535dd9b700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360158df0 0x7f53600b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535ed9d700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5360159700 0x7f53600b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 -- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360158df0 msgr2=0x7f53600b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360158df0 0x7f53600b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 -- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5360159700 msgr2=0x7f53600b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5360159700 0x7f53600b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 -- 172.21.15.64:0/335853959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53500094f0 con 0x7f536006d590 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535dd9b700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360158df0 0x7f53600b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535e59c700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 0x7f5360169910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f53500094c0 tx=0x7f5350017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:05.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350000bc0 con 0x7f536006d590 2023-09-27T23:29:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535000b9d0 con 0x7f536006d590 2023-09-27T23:29:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536015c070 con 0x7f536006d590 2023-09-27T23:29:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350025420 con 0x7f536006d590 2023-09-27T23:29:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53601563a0 con 0x7f536006d590 2023-09-27T23:29:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.993+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536004be40 con 0x7f536006d590 2023-09-27T23:29:06.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.997+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f5350023070 con 0x7f536006d590 2023-09-27T23:29:06.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.997+0000 7f534e7fc700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5340060a40 0x7f5340062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.997+0000 7f535dd9b700 1 -- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5340060a40 msgr2=0x7f5340062ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:06.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.997+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f535007e420 con 0x7f536006d590 2023-09-27T23:29:06.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:05.997+0000 7f535dd9b700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5340060a40 0x7f5340062ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:06.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.005+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f535004e360 con 0x7f536006d590 2023-09-27T23:29:06.089 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:06.085+0000 7f7ab9d52000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:29:06.153 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:06.149+0000 7f7ab9d52000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:29:06.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.193+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53600b0150 con 0x7f536006d590 2023-09-27T23:29:06.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.193+0000 7f534e7fc700 1 -- 172.21.15.64:0/335853959 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f5350009300 con 0x7f536006d590 2023-09-27T23:29:06.200 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:06.200 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:26:56.779553+0000","modified":"2023-09-27T23:26:59.790212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2178143597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2178143597},{"type":"v1","addr":"172.21.15.64:6835","nonce":2178143597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-09-27T23:29:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5340060a40 msgr2=0x7f5340062ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5340060a40 0x7f5340062ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 msgr2=0x7f5360169910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 0x7f5360169910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f53500094c0 tx=0x7f5350017000 comp rx=0 tx=0).stop 2023-09-27T23:29:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 shutdown_connections 2023-09-27T23:29:06.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5340060a40 0x7f5340062ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5360159700 0x7f53600b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5360158df0 0x7f53600b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 --2- 172.21.15.64:0/335853959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536006d590 0x7f5360169910 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 >> 172.21.15.64:0/335853959 conn(0x7f536006c080 msgr2=0x7f53601585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 shutdown_connections 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.197+0000 7f535f59e700 1 -- 172.21.15.64:0/335853959 wait complete. 2023-09-27T23:29:06.203 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-09-27T23:29:06.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:06.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.361+0000 7f51c3b3a700 1 Processor -- start 2023-09-27T23:29:06.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c3b3a700 1 -- start start 2023-09-27T23:29:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c3b3a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51bc159c10 0x7f51bc159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c3b3a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc0c22e0 con 0x7f51bc13fb40 2023-09-27T23:29:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c3b3a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc0cc8e0 con 0x7f51bc15a690 2023-09-27T23:29:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c3b3a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0cc6d0 con 0x7f51bc159c10 2023-09-27T23:29:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c20d7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f51bc13fb40 0x7f51bc0c2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53596/0 (socket says 172.21.15.64:53596) 2023-09-27T23:29:06.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c20d7700 1 -- 172.21.15.64:0/1208461557 learned_addr learned my addr 172.21.15.64:0/1208461557 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c18d6700 1 --2- 172.21.15.64:0/1208461557 >> v2:172.21.15.123:3300/0 conn(0x7f51bc159c10 0x7f51bc159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c08d4700 1 -- 172.21.15.64:0/1208461557 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098943249 0 0) 0x7f51bc0c22e0 con 0x7f51bc13fb40 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c08d4700 1 -- 172.21.15.64:0/1208461557 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a8003570 con 0x7f51bc13fb40 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c18d6700 1 -- 172.21.15.64:0/1208461557 >> v1:172.21.15.123:6789/0 conn(0x7f51bc15a690 legacy=0x7f51bc13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c18d6700 1 -- 172.21.15.64:0/1208461557 >> v1:172.21.15.64:6789/0 conn(0x7f51bc13fb40 legacy=0x7f51bc0c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c18d6700 1 -- 172.21.15.64:0/1208461557 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0ccbc0 con 0x7f51bc159c10 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c18d6700 1 --2- 172.21.15.64:0/1208461557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc159fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51b80098e0 tx=0x7f51b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb8f2982f9b404f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c08d4700 1 -- 172.21.15.64:0/1208461557 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800b640 con 0x7f51bc159c10 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c08d4700 1 -- 172.21.15.64:0/1208461557 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800b7a0 con 0x7f51bc159c10 2023-09-27T23:29:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.365+0000 7f51c08d4700 1 -- 172.21.15.64:0/1208461557 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800bb40 con 0x7f51bc159c10 2023-09-27T23:29:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 -- 172.21.15.64:0/1208461557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 msgr2=0x7f51bc159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/1208461557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc159fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51b80098e0 tx=0x7f51b80091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 -- 172.21.15.64:0/1208461557 shutdown_connections 2023-09-27T23:29:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/1208461557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc159fe0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 -- 172.21.15.64:0/1208461557 >> 172.21.15.64:0/1208461557 conn(0x7f51bc0cd9c0 msgr2=0x7f51bc0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 -- 172.21.15.64:0/1208461557 shutdown_connections 2023-09-27T23:29:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 -- 172.21.15.64:0/1208461557 wait complete. 2023-09-27T23:29:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 Processor -- start 2023-09-27T23:29:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.369+0000 7f51c3b3a700 1 -- start start 2023-09-27T23:29:06.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 0x7f51bc16d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51bc15a690 0x7f51bc157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc159110 con 0x7f51bc13fb40 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc158490 con 0x7f51bc159c10 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc158770 con 0x7f51bc15a690 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 0x7f51bc16d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 0x7f51bc16d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50136/0 (socket says 172.21.15.64:50136) 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 -- 172.21.15.64:0/3491992811 learned_addr learned my addr 172.21.15.64:0/3491992811 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c10d5700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c20d7700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51bc15a690 0x7f51bc157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 -- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51bc15a690 msgr2=0x7f51bc157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51bc15a690 0x7f51bc157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 -- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 msgr2=0x7f51bc157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 -- 172.21.15.64:0/3491992811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8009420 con 0x7f51bc13fb40 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c18d6700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 0x7f51bc16d810 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f51b80093f0 tx=0x7f51b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8000bc0 con 0x7f51bc13fb40 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800b910 con 0x7f51bc13fb40 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc158a50 con 0x7f51bc13fb40 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8000d20 con 0x7f51bc13fb40 2023-09-27T23:29:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc16d2f0 con 0x7f51bc13fb40 2023-09-27T23:29:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc071120 con 0x7f51bc13fb40 2023-09-27T23:29:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.373+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f51b8023070 con 0x7f51bc13fb40 2023-09-27T23:29:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.377+0000 7f51b1ffb700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 0x7f51a0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.377+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f51b807e5c0 con 0x7f51bc13fb40 2023-09-27T23:29:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.377+0000 7f51c10d5700 1 -- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 msgr2=0x7f51a0062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.377+0000 7f51c10d5700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 0x7f51a0062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:06.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.381+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f51b804e450 con 0x7f51bc13fb40 2023-09-27T23:29:06.581 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.579+0000 7fc50e835000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:29:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.581+0000 7f51c10d5700 1 -- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 msgr2=0x7f51a0062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.581+0000 7f51c10d5700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 0x7f51a0062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.581+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51bc0c1c90 con 0x7f51bc13fb40 2023-09-27T23:29:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51b1ffb700 1 -- 172.21.15.64:0/3491992811 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f51b8051a70 con 0x7f51bc13fb40 2023-09-27T23:29:06.589 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:06.589 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:26:56.779553+0000","modified":"2023-09-27T23:26:59.790212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2178143597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2178143597},{"type":"v1","addr":"172.21.15.64:6835","nonce":2178143597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-09-27T23:29:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 msgr2=0x7f51a0062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 0x7f51a0062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 msgr2=0x7f51bc16d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 0x7f51bc16d810 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f51b80093f0 tx=0x7f51b8017000 comp rx=0 tx=0).stop 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 shutdown_connections 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f51a0060a70 0x7f51a0062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51bc15a690 0x7f51bc157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc159c10 0x7f51bc157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 --2- 172.21.15.64:0/3491992811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51bc13fb40 0x7f51bc16d810 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 >> 172.21.15.64:0/3491992811 conn(0x7f51bc0cd9c0 msgr2=0x7f51bc0c3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 shutdown_connections 2023-09-27T23:29:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.585+0000 7f51c3b3a700 1 -- 172.21.15.64:0/3491992811 wait complete. 2023-09-27T23:29:06.593 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-09-27T23:29:06.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:06.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4dc27a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4dc27a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4dc27a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4dc27a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4dc27a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.690 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4d8a73700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.691 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4d8a73700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.691 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4d8a73700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.691 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4d8a73700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.691 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:06.683+0000 7fc4d8a73700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:06.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 Processor -- start 2023-09-27T23:29:06.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 -- start start 2023-09-27T23:29:06.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480670e0 0x7fe748159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe7480abce0 0x7fe7480c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480c1910 con 0x7fe7480ab910 2023-09-27T23:29:06.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe748159cf0 con 0x7fe7480abce0 2023-09-27T23:29:06.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e842700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe748159fd0 con 0x7fe7480670e0 2023-09-27T23:29:06.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe7480abce0 0x7fe7480c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe7480abce0 0x7fe7480c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50142/0 (socket says 172.21.15.64:50142) 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 -- 172.21.15.64:0/1534849980 learned_addr learned my addr 172.21.15.64:0/1534849980 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74d03f700 1 --2- 172.21.15.64:0/1534849980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480670e0 0x7fe748159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74c83e700 1 -- 172.21.15.64:0/1534849980 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086737703 0 0) 0x7fe7480c1910 con 0x7fe7480ab910 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 -- 172.21.15.64:0/1534849980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480670e0 msgr2=0x7fe748159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 --2- 172.21.15.64:0/1534849980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480670e0 0x7fe748159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 -- 172.21.15.64:0/1534849980 >> v1:172.21.15.123:6789/0 conn(0x7fe7480ab910 legacy=0x7fe7480cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.701+0000 7fe74e041700 1 -- 172.21.15.64:0/1534849980 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74815a2b0 con 0x7fe7480abce0 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 --2- 172.21.15.64:0/1534849980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe7480c2f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe740009f20 tx=0x7fe740009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdad2ffa73d806b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74c83e700 1 -- 172.21.15.64:0/1534849980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000e070 con 0x7fe7480abce0 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74c83e700 1 -- 172.21.15.64:0/1534849980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74000b9d0 con 0x7fe7480abce0 2023-09-27T23:29:06.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74c83e700 1 -- 172.21.15.64:0/1534849980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400092e0 con 0x7fe7480abce0 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- 172.21.15.64:0/1534849980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 msgr2=0x7fe7480c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 --2- 172.21.15.64:0/1534849980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe7480c2f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe740009f20 tx=0x7fe740009840 comp rx=0 tx=0).stop 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- 172.21.15.64:0/1534849980 shutdown_connections 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 --2- 172.21.15.64:0/1534849980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe7480c2f80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 --2- 172.21.15.64:0/1534849980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480670e0 0x7fe748159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- 172.21.15.64:0/1534849980 >> 172.21.15.64:0/1534849980 conn(0x7fe74806c080 msgr2=0x7fe7480ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- 172.21.15.64:0/1534849980 shutdown_connections 2023-09-27T23:29:06.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- 172.21.15.64:0/1534849980 wait complete. 2023-09-27T23:29:06.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 Processor -- start 2023-09-27T23:29:06.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- start start 2023-09-27T23:29:06.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7480670e0 0x7fe7481573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480ab910 0x7fe748164c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe74814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74814f920 con 0x7fe7480abce0 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748157980 con 0x7fe7480670e0 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e842700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe748157c60 con 0x7fe7480ab910 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe74814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe74814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50154/0 (socket says 172.21.15.64:50154) 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 -- 172.21.15.64:0/1633696244 learned_addr learned my addr 172.21.15.64:0/1633696244 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74d03f700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480ab910 0x7fe748164c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74d840700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7480670e0 0x7fe7481573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 -- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480ab910 msgr2=0x7fe748164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480ab910 0x7fe748164c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 -- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7480670e0 msgr2=0x7fe7481573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7480670e0 0x7fe7481573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74e041700 1 -- 172.21.15.64:0/1633696244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe740009ae0 con 0x7fe7480abce0 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.705+0000 7fe74d840700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7480670e0 0x7fe7481573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe74e041700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe74814f3e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ca600 tx=0x7fe740018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740007bb0 con 0x7fe7480abce0 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740007d30 con 0x7fe7480abce0 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe748157f40 con 0x7fe7480abce0 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740007ea0 con 0x7fe7480abce0 2023-09-27T23:29:06.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748158500 con 0x7fe7480abce0 2023-09-27T23:29:06.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) v1 ==== 79970+0+0 (secure 0 0 0) 0x7fe740003040 con 0x7fe7480abce0 2023-09-27T23:29:06.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74804be40 con 0x7fe7480abce0 2023-09-27T23:29:06.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.709+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7008+0+0 (secure 0 0 0) 0x7fe74007ee60 con 0x7fe7480abce0 2023-09-27T23:29:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.717+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe74004f0f0 con 0x7fe7480abce0 2023-09-27T23:29:06.813 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:06.809+0000 7f7ab9d52000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:29:06.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.877+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe748068cc0 con 0x7fe7480abce0 2023-09-27T23:29:06.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.877+0000 7fe73dffb700 1 -- 172.21.15.64:0/1633696244 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18009 (secure 0 0 0) 0x7fe740011e60 con 0x7fe7480abce0 2023-09-27T23:29:06.884 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:06.884 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:06.603621+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:26:55.337700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-27T23:26:55.749823+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/464836836":"2023-09-28T23:29:06.603497+0000","172.21.15.123:0/3730026965":"2023-09-28T23:29:06.603497+0000","172.21.15.123:0/1469942206":"2023-09-28T23:29:06.603497+0000","172.21.15.123:0/956290478":"2023-09-28T23:29:06.603497+0000","172.21.15.123:0/371101776":"2023-09-28T23:29:06.603497+0000","172.21.15.123:6833/33228":"2023-09-28T23:27:13.467954+0000","172.21.15.123:6832/33228":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/639937962":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/4106274724":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3444808867":"2023-09-28T23:29:06.603497+0000","172.21.15.123:0/2897693597":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3329695117":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/3209407868":"2023-09-28T23:27:13.467954+0000","172.21.15.123:0/1309068826":"2023-09-28T23:27:13.467954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 msgr2=0x7fe74814f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe74814f3e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ca600 tx=0x7fe740018000 comp rx=0 tx=0).stop 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 shutdown_connections 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7480abce0 0x7fe74814f3e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7480ab910 0x7fe748164c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 --2- 172.21.15.64:0/1633696244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7480670e0 0x7fe7481573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 >> 172.21.15.64:0/1633696244 conn(0x7fe74806c080 msgr2=0x7fe748067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:06.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 shutdown_connections 2023-09-27T23:29:06.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:06.881+0000 7fe74e842700 1 -- 172.21.15.64:0/1633696244 wait complete. 2023-09-27T23:29:06.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:29:07.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.001+0000 7f99234c8700 1 Processor -- start 2023-09-27T23:29:07.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- start start 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f991c06a580 0x7f991c06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f991c06d210 0x7f991c06d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c06c7c0 con 0x7f991c0b1410 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c06af40 con 0x7f991c06d210 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c06cd80 con 0x7f991c06a580 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f991c06d210 0x7f991c06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f991c06d210 0x7f991c06d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50160/0 (socket says 172.21.15.64:50160) 2023-09-27T23:29:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 -- 172.21.15.64:0/796967647 learned_addr learned my addr 172.21.15.64:0/796967647 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99224c6700 1 --1- 172.21.15.64:0/796967647 >> v1:172.21.15.64:6789/0 conn(0x7f991c0b1410 0x7f991c0ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53598/0 (socket says 172.21.15.64:53598) 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9921cc5700 1 --2- 172.21.15.64:0/796967647 >> v2:172.21.15.123:3300/0 conn(0x7f991c06a580 0x7f991c06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 -- 172.21.15.64:0/796967647 >> v2:172.21.15.123:3300/0 conn(0x7f991c06a580 msgr2=0x7f991c06aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 --2- 172.21.15.64:0/796967647 >> v2:172.21.15.123:3300/0 conn(0x7f991c06a580 0x7f991c06aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 -- 172.21.15.64:0/796967647 >> v1:172.21.15.64:6789/0 conn(0x7f991c0b1410 legacy=0x7f991c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 -- 172.21.15.64:0/796967647 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c06b8f0 con 0x7f991c06d210 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f9922cc7700 1 --2- 172.21.15.64:0/796967647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c06d620 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f991c068040 tx=0x7f9914009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426afbb071b91b99 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99214c4700 1 -- 172.21.15.64:0/796967647 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400e070 con 0x7f991c06d210 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99214c4700 1 -- 172.21.15.64:0/796967647 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991400ba50 con 0x7f991c06d210 2023-09-27T23:29:07.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99214c4700 1 -- 172.21.15.64:0/796967647 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99140092e0 con 0x7f991c06d210 2023-09-27T23:29:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- 172.21.15.64:0/796967647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 msgr2=0x7f991c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 --2- 172.21.15.64:0/796967647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c06d620 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f991c068040 tx=0x7f9914009840 comp rx=0 tx=0).stop 2023-09-27T23:29:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- 172.21.15.64:0/796967647 shutdown_connections 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 --2- 172.21.15.64:0/796967647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c06d620 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 --2- 172.21.15.64:0/796967647 >> v2:172.21.15.123:3300/0 conn(0x7f991c06a580 0x7f991c06aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- 172.21.15.64:0/796967647 >> 172.21.15.64:0/796967647 conn(0x7f991c067bb0 msgr2=0x7f991c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- 172.21.15.64:0/796967647 shutdown_connections 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.005+0000 7f99234c8700 1 -- 172.21.15.64:0/796967647 wait complete. 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 Processor -- start 2023-09-27T23:29:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 -- start start 2023-09-27T23:29:07.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f991c06a580 0x7f991c14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:07.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c0b1410 0x7f991c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c14e0b0 con 0x7f991c06d210 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c14e350 con 0x7f991c0b1410 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c14e630 con 0x7f991c06a580 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50172/0 (socket says 172.21.15.64:50172) 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 -- 172.21.15.64:0/1911270353 learned_addr learned my addr 172.21.15.64:0/1911270353 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9922cc7700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c0b1410 0x7f991c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99224c6700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f991c06a580 0x7f991c14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 -- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f991c06a580 msgr2=0x7f991c14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f991c06a580 0x7f991c14f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 -- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c0b1410 msgr2=0x7f991c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c0b1410 0x7f991c0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 -- 172.21.15.64:0/1911270353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9914009ae0 con 0x7f991c06d210 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f9921cc5700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c0c9df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f990c00d890 tx=0x7f990c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0125f0 con 0x7f991c06d210 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c012750 con 0x7f991c06d210 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c012af0 con 0x7f991c06d210 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c14e910 con 0x7f991c06d210 2023-09-27T23:29:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.009+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c14f650 con 0x7f991c06d210 2023-09-27T23:29:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.013+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c04be40 con 0x7f991c06d210 2023-09-27T23:29:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.013+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f990c01e460 con 0x7f991c06d210 2023-09-27T23:29:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.013+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7008+0+0 (secure 0 0 0) 0x7f990c07d2c0 con 0x7f991c06d210 2023-09-27T23:29:07.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.021+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f990c04d550 con 0x7f991c06d210 2023-09-27T23:29:07.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.225+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f991c0ab220 con 0x7f991c06d210 2023-09-27T23:29:07.244 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:29:07.241+0000 7feab75a1700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-09-27T23:29:07.245 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:29:07.241+0000 7feab75a1700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-09-27T23:29:07.245 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:29:07.241+0000 7feab75a1700 -1 MDSIOContextBase: failed with -108, restarting... 2023-09-27T23:29:07.262 INFO:tasks.ceph.mds.a.smithi064.stderr: -4> 2023-09-27T23:29:07.241+0000 7feab75a1700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-09-27T23:29:07.263 INFO:tasks.ceph.mds.a.smithi064.stderr: -3> 2023-09-27T23:29:07.241+0000 7feab75a1700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-09-27T23:29:07.263 INFO:tasks.ceph.mds.a.smithi064.stderr: -1> 2023-09-27T23:29:07.241+0000 7feab75a1700 -1 MDSIOContextBase: failed with -108, restarting... 2023-09-27T23:29:07.284 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:29:07.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.617+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 25) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f990c04d000 con 0x7f991c06d210 2023-09-27T23:29:07.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.621+0000 7f99127fc700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9900066da0 0x7f9900069180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.625+0000 7f99224c6700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9900066da0 0x7f9900069180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:07.629+0000 7f99224c6700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9900066da0 0x7f9900069180 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9918004fd0 tx=0x7f9918007eb0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:08.161 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:29:08.157+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:29:08.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.169+0000 7f99127fc700 1 -- 172.21.15.64:0/1911270353 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f990c06b070 con 0x7f991c06d210 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9900066da0 msgr2=0x7f9900069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9900066da0 0x7f9900069180 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9918004fd0 tx=0x7f9918007eb0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 msgr2=0x7f991c0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c0c9df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f990c00d890 tx=0x7f990c008040 comp rx=0 tx=0).stop 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 shutdown_connections 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9900066da0 0x7f9900069180 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f991c0b1410 0x7f991c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c06d210 0x7f991c0c9df0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 --2- 172.21.15.64:0/1911270353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f991c06a580 0x7f991c14f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 >> 172.21.15.64:0/1911270353 conn(0x7f991c067bb0 msgr2=0x7f991c0adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:08.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 shutdown_connections 2023-09-27T23:29:08.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.173+0000 7f99234c8700 1 -- 172.21.15.64:0/1911270353 wait complete. 2023-09-27T23:29:08.181 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:29:08.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:29:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 Processor -- start 2023-09-27T23:29:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 -- start start 2023-09-27T23:29:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd480caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd4806aeb0 0x7fdd48159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4806c850 con 0x7fdd481595a0 2023-09-27T23:29:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4806b470 con 0x7fdd4806aeb0 2023-09-27T23:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4d63c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4806b750 con 0x7fdd48158c90 2023-09-27T23:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd3ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd4806aeb0 0x7fdd48159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4ce3b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdd481595a0 0x7fdd4815c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:57910/0 (socket says 172.21.15.64:57910) 2023-09-27T23:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd3ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdd4806aeb0 0x7fdd48159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50188/0 (socket says 172.21.15.64:50188) 2023-09-27T23:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd47fff700 1 --2- 172.21.15.64:0/2780331199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd480caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd4ce3b700 1 -- 172.21.15.64:0/2780331199 learned_addr learned my addr 172.21.15.64:0/2780331199 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd477fe700 1 -- 172.21.15.64:0/2780331199 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385333946 0 0) 0x7fdd4806c850 con 0x7fdd481595a0 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd477fe700 1 -- 172.21.15.64:0/2780331199 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd34003570 con 0x7fdd481595a0 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd47fff700 1 -- 172.21.15.64:0/2780331199 >> v2:172.21.15.64:3300/0 conn(0x7fdd4806aeb0 msgr2=0x7fdd48159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd47fff700 1 --2- 172.21.15.64:0/2780331199 >> v2:172.21.15.64:3300/0 conn(0x7fdd4806aeb0 0x7fdd48159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd47fff700 1 -- 172.21.15.64:0/2780331199 >> v1:172.21.15.123:6789/0 conn(0x7fdd481595a0 legacy=0x7fdd4815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd47fff700 1 -- 172.21.15.64:0/2780331199 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4806ba30 con 0x7fdd48158c90 2023-09-27T23:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd47fff700 1 --2- 172.21.15.64:0/2780331199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd480caf40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098e0 tx=0x7fdd380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6b66d562ab57fe server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd477fe700 1 -- 172.21.15.64:0/2780331199 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38011630 con 0x7fdd48158c90 2023-09-27T23:29:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd477fe700 1 -- 172.21.15.64:0/2780331199 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd38011790 con 0x7fdd48158c90 2023-09-27T23:29:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.313+0000 7fdd477fe700 1 -- 172.21.15.64:0/2780331199 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38011b30 con 0x7fdd48158c90 2023-09-27T23:29:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 -- 172.21.15.64:0/2780331199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 msgr2=0x7fdd480caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/2780331199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd480caf40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098e0 tx=0x7fdd380091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 -- 172.21.15.64:0/2780331199 shutdown_connections 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/2780331199 >> v2:172.21.15.64:3300/0 conn(0x7fdd4806aeb0 0x7fdd48159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/2780331199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd480caf40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 -- 172.21.15.64:0/2780331199 >> 172.21.15.64:0/2780331199 conn(0x7fdd48067bb0 msgr2=0x7fdd480cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 -- 172.21.15.64:0/2780331199 shutdown_connections 2023-09-27T23:29:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 -- 172.21.15.64:0/2780331199 wait complete. 2023-09-27T23:29:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 Processor -- start 2023-09-27T23:29:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.317+0000 7fdd4d63c700 1 -- start start 2023-09-27T23:29:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 0x7fdd480c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd481694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd481595a0 0x7fdd48169a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48155180 con 0x7fdd4806aeb0 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480c9d70 con 0x7fdd481595a0 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480c9a90 con 0x7fdd48158c90 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 0x7fdd480c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 0x7fdd480c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50196/0 (socket says 172.21.15.64:50196) 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 -- 172.21.15.64:0/348927611 learned_addr learned my addr 172.21.15.64:0/348927611 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd3ffff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd481694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4ce3b700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd481595a0 0x7fdd48169a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 -- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 msgr2=0x7fdd481694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd481694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 -- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd481595a0 msgr2=0x7fdd48169a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd481595a0 0x7fdd48169a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 -- 172.21.15.64:0/348927611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38009420 con 0x7fdd4806aeb0 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4ce3b700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd481595a0 0x7fdd48169a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd3ffff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd481694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd47fff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 0x7fdd480c90d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009ed0 tx=0x7fdd38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38011900 con 0x7fdd4806aeb0 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd38011a60 con 0x7fdd4806aeb0 2023-09-27T23:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd48154b30 con 0x7fdd4806aeb0 2023-09-27T23:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38026680 con 0x7fdd4806aeb0 2023-09-27T23:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.321+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4816a600 con 0x7fdd4806aeb0 2023-09-27T23:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.325+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4804be40 con 0x7fdd4806aeb0 2023-09-27T23:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.325+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fdd38023030 con 0x7fdd4806aeb0 2023-09-27T23:29:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.325+0000 7fdd44ff9700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdd280607a0 0x7fdd28062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.325+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7094+0+0 (secure 0 0 0) 0x7fdd3807e6b0 con 0x7fdd4806aeb0 2023-09-27T23:29:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.325+0000 7fdd3ffff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdd280607a0 0x7fdd28062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.325+0000 7fdd3ffff700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdd280607a0 0x7fdd28062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd48167040 tx=0x7fdd30008040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:08.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.333+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdd3804e850 con 0x7fdd4806aeb0 2023-09-27T23:29:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:08.529+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd4806bd10 con 0x7fdd4806aeb0 2023-09-27T23:29:09.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.173+0000 7fdd44ff9700 1 -- 172.21.15.64:0/348927611 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd38051e70 con 0x7fdd4806aeb0 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdd280607a0 msgr2=0x7fdd28062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdd280607a0 0x7fdd28062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd48167040 tx=0x7fdd30008040 comp rx=0 tx=0).stop 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 msgr2=0x7fdd480c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 0x7fdd480c90d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009ed0 tx=0x7fdd38017000 comp rx=0 tx=0).stop 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 shutdown_connections 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdd280607a0 0x7fdd28062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd481595a0 0x7fdd48169a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd48158c90 0x7fdd481694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 --2- 172.21.15.64:0/348927611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4806aeb0 0x7fdd480c90d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 >> 172.21.15.64:0/348927611 conn(0x7fdd48067bb0 msgr2=0x7fdd480cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 shutdown_connections 2023-09-27T23:29:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.176+0000 7fdd4d63c700 1 -- 172.21.15.64:0/348927611 wait complete. 2023-09-27T23:29:09.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:29:09.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 Processor -- start 2023-09-27T23:29:09.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 -- start start 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9140158da0 0x7f914006e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91400caca0 0x7f914006ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914015d720 con 0x7f914006df10 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400ca290 con 0x7f91400caca0 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f91453ab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400ca4a0 con 0x7f9140158da0 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91400caca0 0x7f914006ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f913ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9140158da0 0x7f914006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9137fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f914006df10 0x7f914006a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:57926/0 (socket says 172.21.15.64:57926) 2023-09-27T23:29:09.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9137fff700 1 -- 172.21.15.64:0/4178218246 learned_addr learned my addr 172.21.15.64:0/4178218246 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:09.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91400caca0 0x7f914006ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50208/0 (socket says 172.21.15.64:50208) 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f913f7fe700 1 -- 172.21.15.64:0/4178218246 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488693040 0 0) 0x7f914015d720 con 0x7f914006df10 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f913f7fe700 1 -- 172.21.15.64:0/4178218246 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f912c003570 con 0x7f914006df10 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 -- 172.21.15.64:0/4178218246 >> v2:172.21.15.123:3300/0 conn(0x7f9140158da0 msgr2=0x7f914006e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 --2- 172.21.15.64:0/4178218246 >> v2:172.21.15.123:3300/0 conn(0x7f9140158da0 0x7f914006e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 -- 172.21.15.64:0/4178218246 >> v1:172.21.15.123:6789/0 conn(0x7f914006df10 legacy=0x7f914006a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 -- 172.21.15.64:0/4178218246 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400ca780 con 0x7f91400caca0 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f9144baa700 1 --2- 172.21.15.64:0/4178218246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f914006ebb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f914006a310 tx=0x7f9138009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b95b559953da4b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f913f7fe700 1 -- 172.21.15.64:0/4178218246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800e070 con 0x7f91400caca0 2023-09-27T23:29:09.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.324+0000 7f913f7fe700 1 -- 172.21.15.64:0/4178218246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800bab0 con 0x7f91400caca0 2023-09-27T23:29:09.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f913f7fe700 1 -- 172.21.15.64:0/4178218246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800be70 con 0x7f91400caca0 2023-09-27T23:29:09.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 -- 172.21.15.64:0/4178218246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 msgr2=0x7f914006ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:09.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 --2- 172.21.15.64:0/4178218246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f914006ebb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f914006a310 tx=0x7f9138009780 comp rx=0 tx=0).stop 2023-09-27T23:29:09.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 -- 172.21.15.64:0/4178218246 shutdown_connections 2023-09-27T23:29:09.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 --2- 172.21.15.64:0/4178218246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f914006ebb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 --2- 172.21.15.64:0/4178218246 >> v2:172.21.15.123:3300/0 conn(0x7f9140158da0 0x7f914006e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 -- 172.21.15.64:0/4178218246 >> 172.21.15.64:0/4178218246 conn(0x7f91400cc760 msgr2=0x7f91400ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:09.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 -- 172.21.15.64:0/4178218246 shutdown_connections 2023-09-27T23:29:09.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 -- 172.21.15.64:0/4178218246 wait complete. 2023-09-27T23:29:09.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 Processor -- start 2023-09-27T23:29:09.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.328+0000 7f91453ab700 1 -- start start 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f914006df10 0x7f9140153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f91400b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9140158da0 0x7f91401567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400c3c50 con 0x7f91400caca0 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b33a0 con 0x7f914006df10 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b3680 con 0x7f9140158da0 2023-09-27T23:29:09.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f91400b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f91400b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50210/0 (socket says 172.21.15.64:50210) 2023-09-27T23:29:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 -- 172.21.15.64:0/3609403212 learned_addr learned my addr 172.21.15.64:0/3609403212 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f913ffff700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f914006df10 0x7f9140153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:09.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9144baa700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9140158da0 0x7f91401567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 -- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9140158da0 msgr2=0x7f91401567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9140158da0 0x7f91401567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 -- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f914006df10 msgr2=0x7f9140153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f914006df10 0x7f9140153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 -- 172.21.15.64:0/3609403212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009a20 con 0x7f91400caca0 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f9137fff700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f91400b2aa0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f912800c7c0 tx=0x7f9128007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128005cd0 con 0x7f91400caca0 2023-09-27T23:29:09.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9128005e30 con 0x7f91400caca0 2023-09-27T23:29:09.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400b3960 con 0x7f91400caca0 2023-09-27T23:29:09.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128007320 con 0x7f91400caca0 2023-09-27T23:29:09.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.332+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9140153650 con 0x7f91400caca0 2023-09-27T23:29:09.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.336+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91400c3dc0 con 0x7f91400caca0 2023-09-27T23:29:09.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.336+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f912801b3b0 con 0x7f91400caca0 2023-09-27T23:29:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.336+0000 7f913cff9700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91200607a0 0x7f9120062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.336+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7094+0+0 (secure 0 0 0) 0x7f912807c680 con 0x7f91400caca0 2023-09-27T23:29:09.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.336+0000 7f913ffff700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91200607a0 0x7f9120062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:09.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.336+0000 7f913ffff700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91200607a0 0x7f9120062c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9130009d10 tx=0x7f91300091a0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:09.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.344+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f91400c3dc0 con 0x7f91400caca0 2023-09-27T23:29:09.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.500+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f91400c3d90 con 0x7f91400caca0 2023-09-27T23:29:09.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:09.632+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f912804caa0 con 0x7f91400caca0 2023-09-27T23:29:10.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.176+0000 7f913cff9700 1 -- 172.21.15.64:0/3609403212 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7f912804c6d0 con 0x7f91400caca0 2023-09-27T23:29:10.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91200607a0 msgr2=0x7f9120062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91200607a0 0x7f9120062c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9130009d10 tx=0x7f91300091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 msgr2=0x7f91400b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f91400b2aa0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f912800c7c0 tx=0x7f9128007e00 comp rx=0 tx=0).stop 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 shutdown_connections 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91200607a0 0x7f9120062c50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9140158da0 0x7f91401567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91400caca0 0x7f91400b2aa0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 --2- 172.21.15.64:0/3609403212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f914006df10 0x7f9140153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 >> 172.21.15.64:0/3609403212 conn(0x7f91400cc760 msgr2=0x7f91400af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:10.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 shutdown_connections 2023-09-27T23:29:10.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.188+0000 7f91453ab700 1 -- 172.21.15.64:0/3609403212 wait complete. 2023-09-27T23:29:10.195 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:29:10.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:29:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.312+0000 7f1186e43700 1 Processor -- start 2023-09-27T23:29:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.312+0000 7f1186e43700 1 -- start start 2023-09-27T23:29:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1180156cb0 0x7f11800ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800c8a00 con 0x7f11801568e0 2023-09-27T23:29:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800c8ce0 con 0x7f11800aed40 2023-09-27T23:29:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11800afa50 con 0x7f1180156cb0 2023-09-27T23:29:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185e41700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f11801568e0 0x7f11800c7f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53612/0 (socket says 172.21.15.64:53612) 2023-09-27T23:29:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185e41700 1 -- 172.21.15.64:0/1064304271 learned_addr learned my addr 172.21.15.64:0/1064304271 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185640700 1 --2- 172.21.15.64:0/1064304271 >> v2:172.21.15.123:3300/0 conn(0x7f1180156cb0 0x7f11800ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1184e3f700 1 -- 172.21.15.64:0/1064304271 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301160286 0 0) 0x7f11800c8a00 con 0x7f11801568e0 2023-09-27T23:29:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1184e3f700 1 -- 172.21.15.64:0/1064304271 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f116c003570 con 0x7f11801568e0 2023-09-27T23:29:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185640700 1 -- 172.21.15.64:0/1064304271 >> v1:172.21.15.123:6789/0 conn(0x7f11800aed40 legacy=0x7f11800afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185640700 1 -- 172.21.15.64:0/1064304271 >> v1:172.21.15.64:6789/0 conn(0x7f11801568e0 legacy=0x7f11800c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1184e3f700 1 -- 172.21.15.64:0/1064304271 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193592798 0 0) 0x7f11800c8ce0 con 0x7f11800aed40 2023-09-27T23:29:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185640700 1 -- 172.21.15.64:0/1064304271 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800c8fc0 con 0x7f1180156cb0 2023-09-27T23:29:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1185640700 1 --2- 172.21.15.64:0/1064304271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1180156cb0 0x7f11800ae800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098e0 tx=0x7f117c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43b31125338a2fbc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1184e3f700 1 -- 172.21.15.64:0/1064304271 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00b640 con 0x7f1180156cb0 2023-09-27T23:29:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1184e3f700 1 -- 172.21.15.64:0/1064304271 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b7a0 con 0x7f1180156cb0 2023-09-27T23:29:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1184e3f700 1 -- 172.21.15.64:0/1064304271 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00bb20 con 0x7f1180156cb0 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- 172.21.15.64:0/1064304271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1180156cb0 msgr2=0x7f11800ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 --2- 172.21.15.64:0/1064304271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1180156cb0 0x7f11800ae800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098e0 tx=0x7f117c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- 172.21.15.64:0/1064304271 shutdown_connections 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 --2- 172.21.15.64:0/1064304271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1180156cb0 0x7f11800ae800 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- 172.21.15.64:0/1064304271 >> 172.21.15.64:0/1064304271 conn(0x7f118006b5b0 msgr2=0x7f118006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- 172.21.15.64:0/1064304271 shutdown_connections 2023-09-27T23:29:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.316+0000 7f1186e43700 1 -- 172.21.15.64:0/1064304271 wait complete. 2023-09-27T23:29:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 Processor -- start 2023-09-27T23:29:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 -- start start 2023-09-27T23:29:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 0x7f11800cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11801568e0 0x7f118014c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1180156cb0 0x7f11800cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118014cb00 con 0x7f11800aed40 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11800cad50 con 0x7f11801568e0 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1186e43700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118014cf50 con 0x7f1180156cb0 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1185e41700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 0x7f11800cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1185e41700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 0x7f11800cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50212/0 (socket says 172.21.15.64:50212) 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1185e41700 1 -- 172.21.15.64:0/147903706 learned_addr learned my addr 172.21.15.64:0/147903706 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.320+0000 7f1185640700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11801568e0 0x7f118014c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1186642700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1180156cb0 0x7f11800cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1185e41700 1 -- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1180156cb0 msgr2=0x7f11800cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1185e41700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1180156cb0 0x7f11800cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1185e41700 1 -- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11801568e0 msgr2=0x7f118014c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1185e41700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11801568e0 0x7f118014c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:10.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1185e41700 1 -- 172.21.15.64:0/147903706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009420 con 0x7f11800aed40 2023-09-27T23:29:10.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1185e41700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 0x7f11800cb1e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f11700011e0 tx=0x7f1170007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170005b30 con 0x7f11800aed40 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1170005c90 con 0x7f11800aed40 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170007300 con 0x7f11800aed40 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11800cc390 con 0x7f11800aed40 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11800ca9e0 con 0x7f11800aed40 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11800ca560 con 0x7f11800aed40 2023-09-27T23:29:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.324+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1170007670 con 0x7f11800aed40 2023-09-27T23:29:10.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.328+0000 7f1175ffb700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1164060a70 0x7f1164062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:10.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.328+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6656+0+0 (secure 0 0 0) 0x7f117007c610 con 0x7f11800aed40 2023-09-27T23:29:10.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.328+0000 7f1185640700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1164060a70 0x7f1164062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:10.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.328+0000 7f1185640700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1164060a70 0x7f1164062f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f117c0093f0 tx=0x7f117c017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:10.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.336+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f11800ca560 con 0x7f11800aed40 2023-09-27T23:29:10.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:10.500+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11800ca560 con 0x7f11800aed40 2023-09-27T23:29:11.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.120+0000 7f1175ffb700 1 -- 172.21.15.64:0/147903706 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f11800ca560 con 0x7f11800aed40 2023-09-27T23:29:11.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1164060a70 msgr2=0x7f1164062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1164060a70 0x7f1164062f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f117c0093f0 tx=0x7f117c017000 comp rx=0 tx=0).stop 2023-09-27T23:29:11.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 msgr2=0x7f11800cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 0x7f11800cb1e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f11700011e0 tx=0x7f1170007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 shutdown_connections 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1164060a70 0x7f1164062f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1180156cb0 0x7f11800cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11801568e0 0x7f118014c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 --2- 172.21.15.64:0/147903706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11800aed40 0x7f11800cb1e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 >> 172.21.15.64:0/147903706 conn(0x7f118006b5b0 msgr2=0x7f118014eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 shutdown_connections 2023-09-27T23:29:11.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.132+0000 7f1186e43700 1 -- 172.21.15.64:0/147903706 wait complete. 2023-09-27T23:29:11.139 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:29:11.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:29:11.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.252+0000 7fb86274d700 1 Processor -- start 2023-09-27T23:29:11.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.252+0000 7fb86274d700 1 -- start start 2023-09-27T23:29:11.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.252+0000 7fb86274d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155a90 0x7fb85c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.252+0000 7fb86274d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb85c06db70 0x7fb85c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.252+0000 7fb86274d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb85c155e60 0x7fb85c0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c0c68a0 con 0x7fb85c155e60 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c0c6e60 con 0x7fb85c06db70 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0ae4d0 con 0x7fb85c155a90 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb85c155e60 0x7fb85c0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb85c155e60 0x7fb85c0adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50222/0 (socket says 172.21.15.64:50222) 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 -- 172.21.15.64:0/3817896456 learned_addr learned my addr 172.21.15.64:0/3817896456 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86174b700 1 --2- 172.21.15.64:0/3817896456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155a90 0x7fb85c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb860f4a700 1 --2- 172.21.15.64:0/3817896456 >> v2:172.21.15.123:3300/0 conn(0x7fb85c06db70 0x7fb85c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 -- 172.21.15.64:0/3817896456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155a90 msgr2=0x7fb85c0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 --2- 172.21.15.64:0/3817896456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155a90 0x7fb85c0ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 -- 172.21.15.64:0/3817896456 >> v2:172.21.15.123:3300/0 conn(0x7fb85c06db70 msgr2=0x7fb85c0ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 --2- 172.21.15.64:0/3817896456 >> v2:172.21.15.123:3300/0 conn(0x7fb85c06db70 0x7fb85c0ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 -- 172.21.15.64:0/3817896456 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0ae7b0 con 0x7fb85c155e60 2023-09-27T23:29:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb860f4a700 1 --2- 172.21.15.64:0/3817896456 >> v2:172.21.15.123:3300/0 conn(0x7fb85c06db70 0x7fb85c0ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb861f4c700 1 --2- 172.21.15.64:0/3817896456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c155e60 0x7fb85c0adca0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb85c068040 tx=0x7fb854009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a3ef8c04fa3718 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb853fff700 1 -- 172.21.15.64:0/3817896456 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400e070 con 0x7fb85c155e60 2023-09-27T23:29:11.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb853fff700 1 -- 172.21.15.64:0/3817896456 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85400ba20 con 0x7fb85c155e60 2023-09-27T23:29:11.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb853fff700 1 -- 172.21.15.64:0/3817896456 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8540092e0 con 0x7fb85c155e60 2023-09-27T23:29:11.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- 172.21.15.64:0/3817896456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c155e60 msgr2=0x7fb85c0adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 --2- 172.21.15.64:0/3817896456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c155e60 0x7fb85c0adca0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb85c068040 tx=0x7fb854009840 comp rx=0 tx=0).stop 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- 172.21.15.64:0/3817896456 shutdown_connections 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 --2- 172.21.15.64:0/3817896456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c155e60 0x7fb85c0adca0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 --2- 172.21.15.64:0/3817896456 >> v2:172.21.15.123:3300/0 conn(0x7fb85c06db70 0x7fb85c0ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 --2- 172.21.15.64:0/3817896456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155a90 0x7fb85c0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- 172.21.15.64:0/3817896456 >> 172.21.15.64:0/3817896456 conn(0x7fb85c0c8cd0 msgr2=0x7fb85c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- 172.21.15.64:0/3817896456 shutdown_connections 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.256+0000 7fb86274d700 1 -- 172.21.15.64:0/3817896456 wait complete. 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 Processor -- start 2023-09-27T23:29:11.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 -- start start 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 0x7fb85c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c155a90 0x7fb85c0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155e60 0x7fb85c0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c169360 con 0x7fb85c06db70 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0c4260 con 0x7fb85c155a90 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86274d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c169040 con 0x7fb85c155e60 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 0x7fb85c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 0x7fb85c154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50238/0 (socket says 172.21.15.64:50238) 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 -- 172.21.15.64:0/1022931359 learned_addr learned my addr 172.21.15.64:0/1022931359 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb860f4a700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c155a90 0x7fb85c0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb861f4c700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155e60 0x7fb85c0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 -- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155e60 msgr2=0x7fb85c0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155e60 0x7fb85c0c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 -- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c155a90 msgr2=0x7fb85c0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c155a90 0x7fb85c0c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.260+0000 7fb86174b700 1 -- 172.21.15.64:0/1022931359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854009ae0 con 0x7fb85c06db70 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb86174b700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 0x7fb85c154510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb85800ec80 tx=0x7fb8580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800c850 con 0x7fb85c06db70 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85800c9d0 con 0x7fb85c06db70 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c0c4d30 con 0x7fb85c06db70 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800cd90 con 0x7fb85c06db70 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c0c4770 con 0x7fb85c06db70 2023-09-27T23:29:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c04be40 con 0x7fb85c06db70 2023-09-27T23:29:11.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb858013680 con 0x7fb85c06db70 2023-09-27T23:29:11.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb8517fa700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb840060a70 0x7fb840062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7fb85807d410 con 0x7fb85c06db70 2023-09-27T23:29:11.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.264+0000 7fb860f4a700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb840060a70 0x7fb840062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.268+0000 7fb860f4a700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb840060a70 0x7fb840062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0c8c50 tx=0x7fb84c008040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.272+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb85804d4c0 con 0x7fb85c06db70 2023-09-27T23:29:11.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.432+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb85c0c6af0 con 0x7fb85c06db70 2023-09-27T23:29:11.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.440+0000 7fb8517fa700 1 -- 172.21.15.64:0/1022931359 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v19) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb858050ae0 con 0x7fb85c06db70 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb840060a70 msgr2=0x7fb840062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb840060a70 0x7fb840062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0c8c50 tx=0x7fb84c008040 comp rx=0 tx=0).stop 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 msgr2=0x7fb85c154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 0x7fb85c154510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb85800ec80 tx=0x7fb8580091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 shutdown_connections 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb840060a70 0x7fb840062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb85c155e60 0x7fb85c0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c155a90 0x7fb85c0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 --2- 172.21.15.64:0/1022931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb85c06db70 0x7fb85c154510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 >> 172.21.15.64:0/1022931359 conn(0x7fb85c0c8cd0 msgr2=0x7fb85c0c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:11.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 shutdown_connections 2023-09-27T23:29:11.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.444+0000 7fb86274d700 1 -- 172.21.15.64:0/1022931359 wait complete. 2023-09-27T23:29:11.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:29:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.572+0000 7fde6759e700 1 Processor -- start 2023-09-27T23:29:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.572+0000 7fde6759e700 1 -- start start 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde680ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde680adc60 con 0x7fde6813f6a0 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde680ad050 con 0x7fde6813f2d0 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6806d7d0 con 0x7fde680c7e90 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde65d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fde6813f6a0 0x7fde680c7780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53620/0 (socket says 172.21.15.64:53620) 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde65d9b700 1 -- 172.21.15.64:0/3619142946 learned_addr learned my addr 172.21.15.64:0/3619142946 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde66d9d700 1 --2- 172.21.15.64:0/3619142946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde680ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204278730 0 0) 0x7fde680adc60 con 0x7fde6813f6a0 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde54003570 con 0x7fde6813f6a0 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813310761 0 0) 0x7fde680ad050 con 0x7fde6813f2d0 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde680adc60 con 0x7fde6813f2d0 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde66d9d700 1 -- 172.21.15.64:0/3619142946 >> v1:172.21.15.123:6789/0 conn(0x7fde6813f2d0 legacy=0x7fde6815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde66d9d700 1 -- 172.21.15.64:0/3619142946 >> v1:172.21.15.64:6789/0 conn(0x7fde6813f6a0 legacy=0x7fde680c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde66d9d700 1 -- 172.21.15.64:0/3619142946 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680ad330 con 0x7fde680c7e90 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde66d9d700 1 --2- 172.21.15.64:0/3619142946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde680ac9b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde60009e90 tx=0x7fde60009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db59975c407874d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000e070 con 0x7fde680c7e90 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000bb80 con 0x7fde680c7e90 2023-09-27T23:29:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6559a700 1 -- 172.21.15.64:0/3619142946 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60012430 con 0x7fde680c7e90 2023-09-27T23:29:11.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- 172.21.15.64:0/3619142946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 msgr2=0x7fde680ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 --2- 172.21.15.64:0/3619142946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde680ac9b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde60009e90 tx=0x7fde60009780 comp rx=0 tx=0).stop 2023-09-27T23:29:11.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- 172.21.15.64:0/3619142946 shutdown_connections 2023-09-27T23:29:11.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 --2- 172.21.15.64:0/3619142946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde680ac9b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- 172.21.15.64:0/3619142946 >> 172.21.15.64:0/3619142946 conn(0x7fde68067b50 msgr2=0x7fde680cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- 172.21.15.64:0/3619142946 shutdown_connections 2023-09-27T23:29:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.576+0000 7fde6759e700 1 -- 172.21.15.64:0/3619142946 wait complete. 2023-09-27T23:29:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 Processor -- start 2023-09-27T23:29:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 -- start start 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde6813a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 0x7fde680a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6813f6a0 0x7fde680a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680a8750 con 0x7fde6813f2d0 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680a88e0 con 0x7fde6813f6a0 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde680c4f80 con 0x7fde680c7e90 2023-09-27T23:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde65d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 0x7fde680a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde65d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 0x7fde680a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50252/0 (socket says 172.21.15.64:50252) 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde65d9b700 1 -- 172.21.15.64:0/3730971869 learned_addr learned my addr 172.21.15.64:0/3730971869 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde66d9d700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6813f6a0 0x7fde680a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.580+0000 7fde6659c700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde6813a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde65d9b700 1 -- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 msgr2=0x7fde6813a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde65d9b700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde6813a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde65d9b700 1 -- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6813f6a0 msgr2=0x7fde680a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde65d9b700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6813f6a0 0x7fde680a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde65d9b700 1 -- 172.21.15.64:0/3730971869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60009a20 con 0x7fde6813f2d0 2023-09-27T23:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde65d9b700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 0x7fde680a44b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde5000c790 tx=0x7fde50007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50005b30 con 0x7fde6813f2d0 2023-09-27T23:29:11.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde50005c90 con 0x7fde6813f2d0 2023-09-27T23:29:11.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde680a5000 con 0x7fde6813f2d0 2023-09-27T23:29:11.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50007320 con 0x7fde6813f2d0 2023-09-27T23:29:11.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde680a5640 con 0x7fde6813f2d0 2023-09-27T23:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6804be40 con 0x7fde6813f2d0 2023-09-27T23:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.584+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fde5001b3b0 con 0x7fde6813f2d0 2023-09-27T23:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.588+0000 7fde5e7fc700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde48060aa0 0x7fde48062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.588+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7fde5007c620 con 0x7fde6813f2d0 2023-09-27T23:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.588+0000 7fde6659c700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde48060aa0 0x7fde48062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.588+0000 7fde6659c700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde48060aa0 0x7fde48062f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde58007fd0 tx=0x7fde58007d50 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.592+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fde5004c630 con 0x7fde6813f2d0 2023-09-27T23:29:11.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.756+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fde680aca10 con 0x7fde6813f2d0 2023-09-27T23:29:11.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde5e7fc700 1 -- 172.21.15.64:0/3730971869 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7fde5004fc50 con 0x7fde6813f2d0 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde48060aa0 msgr2=0x7fde48062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde48060aa0 0x7fde48062f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde58007fd0 tx=0x7fde58007d50 comp rx=0 tx=0).stop 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 msgr2=0x7fde680a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 0x7fde680a44b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde5000c790 tx=0x7fde50007e20 comp rx=0 tx=0).stop 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 shutdown_connections 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde48060aa0 0x7fde48062f50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6813f6a0 0x7fde680a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6813f2d0 0x7fde680a44b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 --2- 172.21.15.64:0/3730971869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde680c7e90 0x7fde6813a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 >> 172.21.15.64:0/3730971869 conn(0x7fde68067b50 msgr2=0x7fde680c9580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:11.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 shutdown_connections 2023-09-27T23:29:11.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.768+0000 7fde6759e700 1 -- 172.21.15.64:0/3730971869 wait complete. 2023-09-27T23:29:11.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:29:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 Processor -- start 2023-09-27T23:29:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 -- start start 2023-09-27T23:29:11.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa02006db70 0x7fa0200ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 0x7fa0200ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0200c68a0 con 0x7fa020155a90 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200c6e60 con 0x7fa02006db70 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0261c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200ae4d0 con 0x7fa020155e60 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0251c2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa020155a90 0x7fa0200ad220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53636/0 (socket says 172.21.15.64:53636) 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0251c2700 1 -- 172.21.15.64:0/115026584 learned_addr learned my addr 172.21.15.64:0/115026584 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989802761 0 0) 0x7fa0200c68a0 con 0x7fa020155a90 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 --2- 172.21.15.64:0/115026584 >> v2:172.21.15.123:3300/0 conn(0x7fa02006db70 0x7fa0200ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa008003570 con 0x7fa020155a90 2023-09-27T23:29:11.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0259c3700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 0x7fa0200ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226552560 0 0) 0x7fa008003570 con 0x7fa020155a90 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 msgr2=0x7fa0200ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 0x7fa0200ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 >> v2:172.21.15.123:3300/0 conn(0x7fa02006db70 msgr2=0x7fa0200ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 --2- 172.21.15.64:0/115026584 >> v2:172.21.15.123:3300/0 conn(0x7fa02006db70 0x7fa0200ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200ae7b0 con 0x7fa020155a90 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fa0100017f0 con 0x7fa020155a90 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 >> v1:172.21.15.64:6789/0 conn(0x7fa020155a90 legacy=0x7fa0200ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 0x7fa008003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa008003fe0 0x7fa008004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008004d50 0x7fa0080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200c6e60 con 0x7fa008003fe0 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200ae4d0 con 0x7fa008004d50 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00800ba90 con 0x7fa020155e60 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa008003fe0 0x7fa008004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0251c2700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 0x7fa008003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0259c3700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008004d50 0x7fa0080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 -- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 msgr2=0x7fa008003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa020155e60 0x7fa008003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 -- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008004d50 msgr2=0x7fa0080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008004d50 0x7fa0080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 -- 172.21.15.64:0/115026584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008003570 con 0x7fa008003fe0 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0259c3700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa008004d50 0x7fa0080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa0249c1700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa008003fe0 0x7fa008004810 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa014009c00 tx=0x7fa0140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13312591b2a3fcd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400e6b0 con 0x7fa008003fe0 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01400e810 con 0x7fa008003fe0 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.896+0000 7fa01ffff700 1 -- 172.21.15.64:0/115026584 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400ebd0 con 0x7fa008003fe0 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- 172.21.15.64:0/115026584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa008003fe0 msgr2=0x7fa008004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 --2- 172.21.15.64:0/115026584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa008003fe0 0x7fa008004810 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa014009c00 tx=0x7fa0140091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0259c3700 1 -- 172.21.15.64:0/115026584 reap_dead start 2023-09-27T23:29:11.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- 172.21.15.64:0/115026584 shutdown_connections 2023-09-27T23:29:11.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- 172.21.15.64:0/115026584 >> 172.21.15.64:0/115026584 conn(0x7fa0200c8cd0 msgr2=0x7fa0200c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:11.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- 172.21.15.64:0/115026584 shutdown_connections 2023-09-27T23:29:11.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- 172.21.15.64:0/115026584 wait complete. 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 Processor -- start 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- start start 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008003fe0 0x7fa020154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 0x7fa0200c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02015af20 0x7fa02015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200c5f00 con 0x7fa0200c5550 2023-09-27T23:29:11.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020167470 con 0x7fa02015af20 2023-09-27T23:29:11.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0261c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa020167750 con 0x7fa008003fe0 2023-09-27T23:29:11.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0249c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 0x7fa0200c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0259c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02015af20 0x7fa02015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0249c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 0x7fa0200c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50282/0 (socket says 172.21.15.64:50282) 2023-09-27T23:29:11.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0249c1700 1 -- 172.21.15.64:0/1091481826 learned_addr learned my addr 172.21.15.64:0/1091481826 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:11.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.900+0000 7fa0251c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008003fe0 0x7fa020154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0249c1700 1 -- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008003fe0 msgr2=0x7fa020154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0249c1700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008003fe0 0x7fa020154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0249c1700 1 -- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02015af20 msgr2=0x7fa02015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:11.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0249c1700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02015af20 0x7fa02015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:11.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0249c1700 1 -- 172.21.15.64:0/1091481826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020167a30 con 0x7fa0200c5550 2023-09-27T23:29:11.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0259c3700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02015af20 0x7fa02015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:11.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0249c1700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 0x7fa0200c59c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa008004730 tx=0x7fa01401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014022070 con 0x7fa0200c5550 2023-09-27T23:29:11.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01400e980 con 0x7fa0200c5550 2023-09-27T23:29:11.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014027470 con 0x7fa0200c5550 2023-09-27T23:29:11.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa020167d10 con 0x7fa0200c5550 2023-09-27T23:29:11.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201670c0 con 0x7fa0200c5550 2023-09-27T23:29:11.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02004be40 con 0x7fa0200c5550 2023-09-27T23:29:11.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa01401c450 con 0x7fa0200c5550 2023-09-27T23:29:11.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa01d7fa700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa00c060aa0 0x7fa00c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:11.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7fa01407ebe0 con 0x7fa0200c5550 2023-09-27T23:29:11.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0251c2700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa00c060aa0 0x7fa00c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:11.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.904+0000 7fa0251c2700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa00c060aa0 0x7fa00c062f50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0100014a0 tx=0x7fa010006040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:11.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:11.912+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa01404ec90 con 0x7fa0200c5550 2023-09-27T23:29:12.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.076+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0200c5f00 con 0x7fa0200c5550 2023-09-27T23:29:12.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.084+0000 7fa01d7fa700 1 -- 172.21.15.64:0/1091481826 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa01402c0a0 con 0x7fa0200c5550 2023-09-27T23:29:12.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa00c060aa0 msgr2=0x7fa00c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa00c060aa0 0x7fa00c062f50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0100014a0 tx=0x7fa010006040 comp rx=0 tx=0).stop 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 msgr2=0x7fa0200c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 0x7fa0200c59c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa008004730 tx=0x7fa01401a000 comp rx=0 tx=0).stop 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 shutdown_connections 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa00c060aa0 0x7fa00c062f50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02015af20 0x7fa02015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0200c5550 0x7fa0200c59c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 --2- 172.21.15.64:0/1091481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa008003fe0 0x7fa020154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 >> 172.21.15.64:0/1091481826 conn(0x7fa0200c8cd0 msgr2=0x7fa0200c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 shutdown_connections 2023-09-27T23:29:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.088+0000 7fa0261c4700 1 -- 172.21.15.64:0/1091481826 wait complete. 2023-09-27T23:29:12.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-09-27T23:29:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.212+0000 7f81988c7700 1 Processor -- start 2023-09-27T23:29:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.212+0000 7f81988c7700 1 -- start start 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f8194159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f81940abce0 0x7f81940c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81940c1910 con 0x7f81940ab910 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8194159cf0 con 0x7f81940abce0 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8194159fd0 con 0x7f81940670e0 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8192d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f81940ab910 0x7f81940cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53650/0 (socket says 172.21.15.64:53650) 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8192d9d700 1 -- 172.21.15.64:0/3038724666 learned_addr learned my addr 172.21.15.64:0/3038724666 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819359e700 1 --2- 172.21.15.64:0/3038724666 >> v2:172.21.15.123:3300/0 conn(0x7f81940abce0 0x7f81940c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819259c700 1 --2- 172.21.15.64:0/3038724666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f8194159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8191d9b700 1 -- 172.21.15.64:0/3038724666 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578253148 0 0) 0x7f81940c1910 con 0x7f81940ab910 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8191d9b700 1 -- 172.21.15.64:0/3038724666 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8180003570 con 0x7f81940ab910 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819259c700 1 -- 172.21.15.64:0/3038724666 >> v2:172.21.15.123:3300/0 conn(0x7f81940abce0 msgr2=0x7f81940c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819259c700 1 --2- 172.21.15.64:0/3038724666 >> v2:172.21.15.123:3300/0 conn(0x7f81940abce0 0x7f81940c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819259c700 1 -- 172.21.15.64:0/3038724666 >> v1:172.21.15.64:6789/0 conn(0x7f81940ab910 legacy=0x7f81940cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819259c700 1 -- 172.21.15.64:0/3038724666 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819415a2b0 con 0x7f81940670e0 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819359e700 1 --2- 172.21.15.64:0/3038724666 >> v2:172.21.15.123:3300/0 conn(0x7f81940abce0 0x7f81940c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f819259c700 1 --2- 172.21.15.64:0/3038724666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f8194159710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f817c0098e0 tx=0x7f817c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86cf234ef71dc46 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8191d9b700 1 -- 172.21.15.64:0/3038724666 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00f6b0 con 0x7f81940670e0 2023-09-27T23:29:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8191d9b700 1 -- 172.21.15.64:0/3038724666 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00f810 con 0x7f81940670e0 2023-09-27T23:29:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f8191d9b700 1 -- 172.21.15.64:0/3038724666 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00fbb0 con 0x7f81940670e0 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- 172.21.15.64:0/3038724666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 msgr2=0x7f8194159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 --2- 172.21.15.64:0/3038724666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f8194159710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f817c0098e0 tx=0x7f817c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- 172.21.15.64:0/3038724666 shutdown_connections 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 --2- 172.21.15.64:0/3038724666 >> v2:172.21.15.123:3300/0 conn(0x7f81940abce0 0x7f81940c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 --2- 172.21.15.64:0/3038724666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f8194159710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- 172.21.15.64:0/3038724666 >> 172.21.15.64:0/3038724666 conn(0x7f819406c080 msgr2=0x7f81940ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- 172.21.15.64:0/3038724666 shutdown_connections 2023-09-27T23:29:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.216+0000 7f81988c7700 1 -- 172.21.15.64:0/3038724666 wait complete. 2023-09-27T23:29:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 Processor -- start 2023-09-27T23:29:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 -- start start 2023-09-27T23:29:12.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f81940c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:12.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 0x7f8194163f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81940abce0 0x7f81940af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194150d70 con 0x7f81940ab910 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81940c42e0 con 0x7f81940abce0 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f81988c7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8194150160 con 0x7f81940670e0 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 0x7f8194163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 0x7f8194163f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50294/0 (socket says 172.21.15.64:50294) 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 -- 172.21.15.64:0/2106348379 learned_addr learned my addr 172.21.15.64:0/2106348379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819359e700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81940abce0 0x7f81940af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f8192d9d700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f81940c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 -- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 msgr2=0x7f81940c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f81940c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 -- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81940abce0 msgr2=0x7f81940af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81940abce0 0x7f81940af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 -- 172.21.15.64:0/2106348379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009a20 con 0x7f81940ab910 2023-09-27T23:29:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819359e700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81940abce0 0x7f81940af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.220+0000 7f819259c700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 0x7f8194163f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f819415a110 tx=0x7f817c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c01f070 con 0x7f81940ab910 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00fb90 con 0x7f81940ab910 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c009420 con 0x7f81940ab910 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0106c0 con 0x7f81940ab910 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8194150720 con 0x7f81940ab910 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819404be40 con 0x7f81940ab910 2023-09-27T23:29:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f817c023410 con 0x7f81940ab910 2023-09-27T23:29:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f818affd700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8174060a70 0x7f8174062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.224+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7f817c014070 con 0x7f81940ab910 2023-09-27T23:29:12.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.228+0000 7f8192d9d700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8174060a70 0x7f8174062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.228+0000 7f8192d9d700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8174060a70 0x7f8174062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81840058d0 tx=0x7f8184009040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:12.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.232+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f817c050510 con 0x7f81940ab910 2023-09-27T23:29:12.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:12.456+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f8194068cc0 con 0x7f81940ab910 2023-09-27T23:29:13.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.204+0000 7f818affd700 1 -- 172.21.15.64:0/2106348379 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v557) v1 ==== 154+0+0 (secure 0 0 0) 0x7f817c053b30 con 0x7f81940ab910 2023-09-27T23:29:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8174060a70 msgr2=0x7f8174062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8174060a70 0x7f8174062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81840058d0 tx=0x7f8184009040 comp rx=0 tx=0).stop 2023-09-27T23:29:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 msgr2=0x7f8194163f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 0x7f8194163f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f819415a110 tx=0x7f817c019000 comp rx=0 tx=0).stop 2023-09-27T23:29:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 shutdown_connections 2023-09-27T23:29:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8174060a70 0x7f8174062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81940abce0 0x7f81940af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81940ab910 0x7f8194163f30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 --2- 172.21.15.64:0/2106348379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81940670e0 0x7f81940c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 >> 172.21.15.64:0/2106348379 conn(0x7f819406c080 msgr2=0x7f8194152d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 shutdown_connections 2023-09-27T23:29:13.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.208+0000 7f81988c7700 1 -- 172.21.15.64:0/2106348379 wait complete. 2023-09-27T23:29:13.231 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-09-27T23:29:13.231 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-09-27T23:29:13.232 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:29:13.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-09-27T23:29:13.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 Processor -- start 2023-09-27T23:29:13.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 -- start start 2023-09-27T23:29:13.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24ec1590b0 0x7f24ec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec06a540 con 0x7f24ec158ce0 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec06b3a0 con 0x7f24ec1590b0 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec15d990 con 0x7f24ec06df20 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24ec1590b0 0x7f24ec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24ec1590b0 0x7f24ec06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50296/0 (socket says 172.21.15.64:50296) 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 -- 172.21.15.64:0/768417080 learned_addr learned my addr 172.21.15.64:0/768417080 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f208b700 1 --2- 172.21.15.64:0/768417080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:13.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 -- 172.21.15.64:0/768417080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 msgr2=0x7f24ec0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 --2- 172.21.15.64:0/768417080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 -- 172.21.15.64:0/768417080 >> v1:172.21.15.64:6789/0 conn(0x7f24ec158ce0 legacy=0x7f24ec06b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 -- 172.21.15.64:0/768417080 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec06a790 con 0x7f24ec1590b0 2023-09-27T23:29:13.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f1089700 1 --2- 172.21.15.64:0/768417080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec06d9e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009f20 tx=0x7f24e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7455b61c7cf8f950 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f0888700 1 -- 172.21.15.64:0/768417080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000e070 con 0x7f24ec1590b0 2023-09-27T23:29:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f0888700 1 -- 172.21.15.64:0/768417080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000b9d0 con 0x7f24ec1590b0 2023-09-27T23:29:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f0888700 1 -- 172.21.15.64:0/768417080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00092e0 con 0x7f24ec1590b0 2023-09-27T23:29:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 -- 172.21.15.64:0/768417080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 msgr2=0x7f24ec06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.344+0000 7f24f288c700 1 --2- 172.21.15.64:0/768417080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec06d9e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009f20 tx=0x7f24e0009840 comp rx=0 tx=0).stop 2023-09-27T23:29:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- 172.21.15.64:0/768417080 shutdown_connections 2023-09-27T23:29:13.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 --2- 172.21.15.64:0/768417080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 --2- 172.21.15.64:0/768417080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec06d9e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- 172.21.15.64:0/768417080 >> 172.21.15.64:0/768417080 conn(0x7f24ec067b70 msgr2=0x7f24ec0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:13.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- 172.21.15.64:0/768417080 shutdown_connections 2023-09-27T23:29:13.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- 172.21.15.64:0/768417080 wait complete. 2023-09-27T23:29:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 Processor -- start 2023-09-27T23:29:13.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- start start 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ec158ce0 0x7f24ec147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0c7df0 con 0x7f24ec1590b0 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0b5190 con 0x7f24ec158ce0 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f288c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0c71e0 con 0x7f24ec06df20 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50312/0 (socket says 172.21.15.64:50312) 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 -- 172.21.15.64:0/1150063425 learned_addr learned my addr 172.21.15.64:0/1150063425 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f1089700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ec158ce0 0x7f24ec147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f188a700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 -- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 msgr2=0x7f24ec0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 -- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ec158ce0 msgr2=0x7f24ec147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ec158ce0 0x7f24ec147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.348+0000 7f24f208b700 1 -- 172.21.15.64:0/1150063425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0009ae0 con 0x7f24ec1590b0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24f208b700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec0cb620 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f24e400ea50 tx=0x7f24e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e400c7f0 con 0x7f24ec1590b0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e400c970 con 0x7f24ec1590b0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec0c74c0 con 0x7f24ec1590b0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e400cd30 con 0x7f24ec1590b0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec0c7a80 con 0x7f24ec1590b0 2023-09-27T23:29:13.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.352+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec04be40 con 0x7f24ec1590b0 2023-09-27T23:29:13.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.356+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f24e4013680 con 0x7f24ec1590b0 2023-09-27T23:29:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.356+0000 7f24e9ffb700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24d8060a70 0x7f24d8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.356+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7f24e407db10 con 0x7f24ec1590b0 2023-09-27T23:29:13.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.356+0000 7f24f188a700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24d8060a70 0x7f24d8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:13.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.356+0000 7f24f188a700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24d8060a70 0x7f24d8062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003880 tx=0x7f24dc009040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:13.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.360+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f24e404dba0 con 0x7f24ec1590b0 2023-09-27T23:29:13.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:13.580+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f24ec06aa70 con 0x7f24ec1590b0 2023-09-27T23:29:14.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.208+0000 7f24e9ffb700 1 -- 172.21.15.64:0/1150063425 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v558) v1 ==== 159+0+0 (secure 0 0 0) 0x7f24e40170e0 con 0x7f24ec1590b0 2023-09-27T23:29:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24d8060a70 msgr2=0x7f24d8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24d8060a70 0x7f24d8062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003880 tx=0x7f24dc009040 comp rx=0 tx=0).stop 2023-09-27T23:29:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 msgr2=0x7f24ec0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec0cb620 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f24e400ea50 tx=0x7f24e40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 shutdown_connections 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24d8060a70 0x7f24d8062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ec1590b0 0x7f24ec0cb620 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ec158ce0 0x7f24ec147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 --2- 172.21.15.64:0/1150063425 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ec06df20 0x7f24ec0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 >> 172.21.15.64:0/1150063425 conn(0x7f24ec067b70 msgr2=0x7f24ec157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 shutdown_connections 2023-09-27T23:29:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.212+0000 7f24f288c700 1 -- 172.21.15.64:0/1150063425 wait complete. 2023-09-27T23:29:14.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:29:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 Processor -- start 2023-09-27T23:29:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 -- start start 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac158e20 0x7f24ac0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24ac0cb970 0x7f24ac0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24ac0ca2c0 0x7f24ac06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0cbdf0 con 0x7f24ac0ca2c0 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0cc0d0 con 0x7f24ac0cb970 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24b093a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0cc3b0 con 0x7f24ac158e20 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24aad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24ac0ca2c0 0x7f24ac06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24aad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24ac0ca2c0 0x7f24ac06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50326/0 (socket says 172.21.15.64:50326) 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24aad9d700 1 -- 172.21.15.64:0/1160268351 learned_addr learned my addr 172.21.15.64:0/1160268351 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24aa59c700 1 --2- 172.21.15.64:0/1160268351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac158e20 0x7f24ac0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.352+0000 7f24a9d9b700 1 --2- 172.21.15.64:0/1160268351 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0cb970 0x7f24ac0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24aad9d700 1 -- 172.21.15.64:0/1160268351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac158e20 msgr2=0x7f24ac0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24aad9d700 1 --2- 172.21.15.64:0/1160268351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac158e20 0x7f24ac0ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24aad9d700 1 -- 172.21.15.64:0/1160268351 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0cb970 msgr2=0x7f24ac0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24aad9d700 1 --2- 172.21.15.64:0/1160268351 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0cb970 0x7f24ac0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24aad9d700 1 -- 172.21.15.64:0/1160268351 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0cc690 con 0x7f24ac0ca2c0 2023-09-27T23:29:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24a9d9b700 1 --2- 172.21.15.64:0/1160268351 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0cb970 0x7f24ac0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24aad9d700 1 --2- 172.21.15.64:0/1160268351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac0ca2c0 0x7f24ac06c7c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f24ac067fb0 tx=0x7f24a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624bed61c46d2fc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24a959a700 1 -- 172.21.15.64:0/1160268351 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a400e070 con 0x7f24ac0ca2c0 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24a959a700 1 -- 172.21.15.64:0/1160268351 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400ba50 con 0x7f24ac0ca2c0 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24a959a700 1 -- 172.21.15.64:0/1160268351 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40092e0 con 0x7f24ac0ca2c0 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 -- 172.21.15.64:0/1160268351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac0ca2c0 msgr2=0x7f24ac06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 --2- 172.21.15.64:0/1160268351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac0ca2c0 0x7f24ac06c7c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f24ac067fb0 tx=0x7f24a4009840 comp rx=0 tx=0).stop 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 -- 172.21.15.64:0/1160268351 shutdown_connections 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 --2- 172.21.15.64:0/1160268351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac0ca2c0 0x7f24ac06c7c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 --2- 172.21.15.64:0/1160268351 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0cb970 0x7f24ac0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 --2- 172.21.15.64:0/1160268351 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac158e20 0x7f24ac0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 -- 172.21.15.64:0/1160268351 >> 172.21.15.64:0/1160268351 conn(0x7f24ac067bb0 msgr2=0x7f24ac0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 -- 172.21.15.64:0/1160268351 shutdown_connections 2023-09-27T23:29:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 -- 172.21.15.64:0/1160268351 wait complete. 2023-09-27T23:29:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 Processor -- start 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.356+0000 7f24b093a700 1 -- start start 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0ca2c0 0x7f24ac16d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 0x7f24ac0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0c6770 0x7f24ac14a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac1693e0 con 0x7f24ac158e20 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac16a200 con 0x7f24ac0c6770 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac16d8e0 con 0x7f24ac0ca2c0 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 0x7f24ac0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 0x7f24ac0c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50328/0 (socket says 172.21.15.64:50328) 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 -- 172.21.15.64:0/1409657607 learned_addr learned my addr 172.21.15.64:0/1409657607 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24aa59c700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0ca2c0 0x7f24ac16d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24aad9d700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0c6770 0x7f24ac14a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 -- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0ca2c0 msgr2=0x7f24ac16d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0ca2c0 0x7f24ac16d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 -- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0c6770 msgr2=0x7f24ac14a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0c6770 0x7f24ac14a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 -- 172.21.15.64:0/1409657607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4009ae0 con 0x7f24ac158e20 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24aa59c700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0ca2c0 0x7f24ac16d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24a9d9b700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 0x7f24ac0c6230 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e900 tx=0x7f249c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c00eea0 con 0x7f24ac158e20 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c00c580 con 0x7f24ac158e20 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c00c920 con 0x7f24ac158e20 2023-09-27T23:29:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac1695f0 con 0x7f24ac158e20 2023-09-27T23:29:14.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac169bb0 con 0x7f24ac158e20 2023-09-27T23:29:14.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.360+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac071120 con 0x7f24ac158e20 2023-09-27T23:29:14.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.364+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f249c00ccd0 con 0x7f24ac158e20 2023-09-27T23:29:14.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.364+0000 7f249a7fc700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f248c060a70 0x7f248c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.364+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7f249c07cf50 con 0x7f24ac158e20 2023-09-27T23:29:14.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.364+0000 7f24aa59c700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f248c060a70 0x7f248c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.364+0000 7f24aa59c700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f248c060a70 0x7f248c062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24a0006fd0 tx=0x7f24a0008040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:14.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.368+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f249c050980 con 0x7f24ac158e20 2023-09-27T23:29:14.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.528+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f24ac169f50 con 0x7f24ac158e20 2023-09-27T23:29:14.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f249a7fc700 1 -- 172.21.15.64:0/1409657607 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f249c050510 con 0x7f24ac158e20 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f248c060a70 msgr2=0x7f248c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f248c060a70 0x7f248c062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24a0006fd0 tx=0x7f24a0008040 comp rx=0 tx=0).stop 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 msgr2=0x7f24ac0c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 0x7f24ac0c6230 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e900 tx=0x7f249c008040 comp rx=0 tx=0).stop 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 shutdown_connections 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f248c060a70 0x7f248c062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0c6770 0x7f24ac14a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24ac158e20 0x7f24ac0c6230 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 --2- 172.21.15.64:0/1409657607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0ca2c0 0x7f24ac16d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 >> 172.21.15.64:0/1409657607 conn(0x7f24ac067bb0 msgr2=0x7f24ac06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 shutdown_connections 2023-09-27T23:29:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.540+0000 7f24b093a700 1 -- 172.21.15.64:0/1409657607 wait complete. 2023-09-27T23:29:14.560 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:29:14.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:14.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:14.600 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:29:14.600 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:29:14.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:14.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d60968700 1 Processor -- start 2023-09-27T23:29:14.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d60968700 1 -- start start 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d60968700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c0cc610 0x7f7d5c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d60968700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0c6630 con 0x7f7d5c067c90 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d60968700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0c6910 con 0x7f7d5c1502b0 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d60968700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0c6bf0 con 0x7f7d5c0cc610 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d5a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7d5c067c90 0x7f7d5c14fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51020/0 (socket says 172.21.15.64:51020) 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.704+0000 7f7d5a59c700 1 -- 172.21.15.64:0/851696923 learned_addr learned my addr 172.21.15.64:0/851696923 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:14.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5ad9d700 1 --2- 172.21.15.64:0/851696923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c0cc610 0x7f7d5c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5959a700 1 -- 172.21.15.64:0/851696923 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459390089 0 0) 0x7f7d5c0c6910 con 0x7f7d5c1502b0 2023-09-27T23:29:14.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5959a700 1 -- 172.21.15.64:0/851696923 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d48003570 con 0x7f7d5c1502b0 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5959a700 1 -- 172.21.15.64:0/851696923 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656712545 0 0) 0x7f7d5c0c6630 con 0x7f7d5c067c90 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5ad9d700 1 -- 172.21.15.64:0/851696923 >> v1:172.21.15.123:6789/0 conn(0x7f7d5c1502b0 legacy=0x7f7d5c0c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5ad9d700 1 -- 172.21.15.64:0/851696923 >> v1:172.21.15.64:6789/0 conn(0x7f7d5c067c90 legacy=0x7f7d5c14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5ad9d700 1 -- 172.21.15.64:0/851696923 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0c6ed0 con 0x7f7d5c0cc610 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5ad9d700 1 --2- 172.21.15.64:0/851696923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c0cc610 0x7f7d5c0c6010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009e90 tx=0x7f7d54009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af949d7bcdaf8e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5959a700 1 -- 172.21.15.64:0/851696923 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400e070 con 0x7f7d5c0cc610 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5959a700 1 -- 172.21.15.64:0/851696923 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5400bc30 con 0x7f7d5c0cc610 2023-09-27T23:29:14.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d5959a700 1 -- 172.21.15.64:0/851696923 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54012430 con 0x7f7d5c0cc610 2023-09-27T23:29:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 -- 172.21.15.64:0/851696923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c0cc610 msgr2=0x7f7d5c0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 --2- 172.21.15.64:0/851696923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c0cc610 0x7f7d5c0c6010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009e90 tx=0x7f7d54009780 comp rx=0 tx=0).stop 2023-09-27T23:29:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 -- 172.21.15.64:0/851696923 shutdown_connections 2023-09-27T23:29:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 --2- 172.21.15.64:0/851696923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c0cc610 0x7f7d5c0c6010 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 -- 172.21.15.64:0/851696923 >> 172.21.15.64:0/851696923 conn(0x7f7d5c0cd6d0 msgr2=0x7f7d5c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:14.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 -- 172.21.15.64:0/851696923 shutdown_connections 2023-09-27T23:29:14.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.708+0000 7f7d60968700 1 -- 172.21.15.64:0/851696923 wait complete. 2023-09-27T23:29:14.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 Processor -- start 2023-09-27T23:29:14.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 -- start start 2023-09-27T23:29:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 0x7f7d5c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d5c0cc610 0x7f7d5c0b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c1502b0 0x7f7d5c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0c86c0 con 0x7f7d5c067c90 2023-09-27T23:29:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c151590 con 0x7f7d5c0cc610 2023-09-27T23:29:14.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c151870 con 0x7f7d5c1502b0 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 0x7f7d5c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 0x7f7d5c14df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39704/0 (socket says 172.21.15.64:39704) 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 -- 172.21.15.64:0/1673973200 learned_addr learned my addr 172.21.15.64:0/1673973200 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d59d9b700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d5c0cc610 0x7f7d5c0b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5ad9d700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c1502b0 0x7f7d5c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 -- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c1502b0 msgr2=0x7f7d5c0b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c1502b0 0x7f7d5c0b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 -- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d5c0cc610 msgr2=0x7f7d5c0b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d5c0cc610 0x7f7d5c0b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 -- 172.21.15.64:0/1673973200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54009a20 con 0x7f7d5c067c90 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5ad9d700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c1502b0 0x7f7d5c0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:14.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d5a59c700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 0x7f7d5c14df70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c7e0 tx=0x7f7d4c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c005b30 con 0x7f7d5c067c90 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c005c90 con 0x7f7d5c067c90 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c151b50 con 0x7f7d5c067c90 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c007300 con 0x7f7d5c067c90 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.712+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c14e4b0 con 0x7f7d5c067c90 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.716+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c071120 con 0x7f7d5c067c90 2023-09-27T23:29:14.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.716+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7d4c0076b0 con 0x7f7d5c067c90 2023-09-27T23:29:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.716+0000 7f7d527fc700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d3c060aa0 0x7f7d3c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.716+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6242+0+0 (secure 0 0 0) 0x7f7d4c07c5a0 con 0x7f7d5c067c90 2023-09-27T23:29:14.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.716+0000 7f7d59d9b700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d3c060aa0 0x7f7d3c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:14.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.716+0000 7f7d59d9b700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d3c060aa0 0x7f7d3c062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c16c820 tx=0x7f7d44007d30 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:14.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.728+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7d4c04c650 con 0x7f7d5c067c90 2023-09-27T23:29:14.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.928+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5c0642a0 con 0x7f7d5c067c90 2023-09-27T23:29:14.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d527fc700 1 -- 172.21.15.64:0/1673973200 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f7d4c017de0 con 0x7f7d5c067c90 2023-09-27T23:29:14.937 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:14.937 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3077038660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3077038660},{"type":"v1","addr":"172.21.15.123:6836","nonce":3077038660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1946713250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1946713250},{"type":"v1","addr":"172.21.15.123:6837","nonce":1946713250}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/164832075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":164832075},{"type":"v1","addr":"172.21.15.64:6835","nonce":164832075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":6093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/50936478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":50936478},{"type":"v1","addr":"172.21.15.64:6833","nonce":50936478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2023-09-27T23:29:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d3c060aa0 msgr2=0x7f7d3c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d3c060aa0 0x7f7d3c062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c16c820 tx=0x7f7d44007d30 comp rx=0 tx=0).stop 2023-09-27T23:29:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 msgr2=0x7f7d5c14df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 0x7f7d5c14df70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c7e0 tx=0x7f7d4c007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 shutdown_connections 2023-09-27T23:29:14.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7d3c060aa0 0x7f7d3c062f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d5c1502b0 0x7f7d5c0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d5c0cc610 0x7f7d5c0b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 --2- 172.21.15.64:0/1673973200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5c067c90 0x7f7d5c14df70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:14.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 >> 172.21.15.64:0/1673973200 conn(0x7f7d5c0cd6d0 msgr2=0x7f7d5c14ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:14.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 shutdown_connections 2023-09-27T23:29:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:14.932+0000 7f7d60968700 1 -- 172.21.15.64:0/1673973200 wait complete. 2023-09-27T23:29:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 23 2023-09-27T23:29:14.952 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:29:14.952 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:29:14.952 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:29:14.952 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:14.952 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:29:14.953 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:29:14.953 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:29:14.953 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:14.953 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:29:14.953 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:29:14.953 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:29:14.953 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:14.954 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:29:14.954 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:29:14.954 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:29:14.954 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:14.967 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:29:14.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:29:14.968 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:29:14.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:29:14.970 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:29:14.971 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:14.971 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:29:14.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:29:14.973 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:29:14.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:29:14.973 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:29:14.974 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:15.017 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:29:15.018 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:15.019 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:29:15.019 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:15.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:29:15.038 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:29:15.046 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:29:15.097 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:29:15.098 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:29:15.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 Processor -- start 2023-09-27T23:29:15.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- start start 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f91280b37a0 0x7f9128161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b3ca0 con 0x7f9128156ad0 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b25f0 con 0x7f9128156ea0 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912815b870 con 0x7f91280b37a0 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9126ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9128156ad0 0x7f91280cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51046/0 (socket says 172.21.15.64:51046) 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9126ffd700 1 -- 172.21.15.64:0/1533235562 learned_addr learned my addr 172.21.15.64:0/1533235562 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:15.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f91277fe700 1 --2- 172.21.15.64:0/1533235562 >> v2:172.21.15.123:3300/0 conn(0x7f91280b37a0 0x7f9128161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785057065 0 0) 0x7f91280b3ca0 con 0x7f9128156ad0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003570 con 0x7f9128156ad0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539445273 0 0) 0x7f91280b25f0 con 0x7f9128156ea0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91280b3ca0 con 0x7f9128156ea0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f91277fe700 1 -- 172.21.15.64:0/1533235562 >> v1:172.21.15.123:6789/0 conn(0x7f9128156ea0 legacy=0x7f91280cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f91277fe700 1 -- 172.21.15.64:0/1533235562 >> v1:172.21.15.64:6789/0 conn(0x7f9128156ad0 legacy=0x7f91280cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f91277fe700 1 -- 172.21.15.64:0/1533235562 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912806d300 con 0x7f91280b37a0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f91277fe700 1 --2- 172.21.15.64:0/1533235562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f9128161c00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9120009e60 tx=0x7f9120009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b0d996107a0094 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000e070 con 0x7f91280b37a0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912000bab0 con 0x7f91280b37a0 2023-09-27T23:29:15.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f9125ffb700 1 -- 172.21.15.64:0/1533235562 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000be70 con 0x7f91280b37a0 2023-09-27T23:29:15.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- 172.21.15.64:0/1533235562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 msgr2=0x7f9128161c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:15.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 --2- 172.21.15.64:0/1533235562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f9128161c00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9120009e60 tx=0x7f9120009780 comp rx=0 tx=0).stop 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- 172.21.15.64:0/1533235562 shutdown_connections 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 --2- 172.21.15.64:0/1533235562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f9128161c00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- 172.21.15.64:0/1533235562 >> 172.21.15.64:0/1533235562 conn(0x7f91280cdda0 msgr2=0x7f91280ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- 172.21.15.64:0/1533235562 shutdown_connections 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.120+0000 7f912d343700 1 -- 172.21.15.64:0/1533235562 wait complete. 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 Processor -- start 2023-09-27T23:29:15.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- start start 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f91281523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9128156ad0 0x7f9128157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 0x7f91281402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280ae600 con 0x7f9128156ea0 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128152c50 con 0x7f91280b37a0 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9128152f30 con 0x7f9128156ad0 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 0x7f91281402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:15.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 0x7f91281402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39732/0 (socket says 172.21.15.64:39732) 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 -- 172.21.15.64:0/1104309755 learned_addr learned my addr 172.21.15.64:0/1104309755 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91267fc700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9128156ad0 0x7f9128157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f9126ffd700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f91281523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 -- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9128156ad0 msgr2=0x7f9128157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9128156ad0 0x7f9128157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 -- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 msgr2=0x7f91281523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f91281523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 -- 172.21.15.64:0/1104309755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120009a20 con 0x7f9128156ea0 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f91277fe700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 0x7f91281402e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f912806f820 tx=0x7f9120019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120012db0 con 0x7f9128156ea0 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120026480 con 0x7f9128156ea0 2023-09-27T23:29:15.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9128153210 con 0x7f9128156ea0 2023-09-27T23:29:15.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120026840 con 0x7f9128156ea0 2023-09-27T23:29:15.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91280ae900 con 0x7f9128156ea0 2023-09-27T23:29:15.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.124+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91280aea90 con 0x7f9128156ea0 2023-09-27T23:29:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.128+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f912001c400 con 0x7f9128156ea0 2023-09-27T23:29:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.128+0000 7f910effd700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9114060a70 0x7f9114062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.128+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6214+0+0 (secure 0 0 0) 0x7f912007f0a0 con 0x7f9128156ea0 2023-09-27T23:29:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.128+0000 7f9126ffd700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9114060a70 0x7f9114062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:15.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.128+0000 7f9126ffd700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9114060a70 0x7f9114062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9118007fd0 tx=0x7f9118009040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:15.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.132+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f91280aea90 con 0x7f9128156ea0 2023-09-27T23:29:15.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:15.308+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f91280aea90 con 0x7f9128156ea0 2023-09-27T23:29:16.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f910effd700 1 -- 172.21.15.64:0/1104309755 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f91280aea90 con 0x7f9128156ea0 2023-09-27T23:29:16.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9114060a70 msgr2=0x7f9114062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9114060a70 0x7f9114062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9118007fd0 tx=0x7f9118009040 comp rx=0 tx=0).stop 2023-09-27T23:29:16.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 msgr2=0x7f91281402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 0x7f91281402e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f912806f820 tx=0x7f9120019000 comp rx=0 tx=0).stop 2023-09-27T23:29:16.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 shutdown_connections 2023-09-27T23:29:16.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9114060a70 0x7f9114062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9128156ea0 0x7f91281402e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9128156ad0 0x7f9128157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 --2- 172.21.15.64:0/1104309755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91280b37a0 0x7f91281523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 >> 172.21.15.64:0/1104309755 conn(0x7f91280cdda0 msgr2=0x7f91281508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.224+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 shutdown_connections 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.228+0000 7f912d343700 1 -- 172.21.15.64:0/1104309755 wait complete. 2023-09-27T23:29:16.233 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:29:16.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:29:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 Processor -- start 2023-09-27T23:29:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 -- start start 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c06b4f0 0x7f5a0c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5a0c0a8850 0x7f5a0c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c06d470 con 0x7f5a0c158cb0 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c06c860 con 0x7f5a0c0a8850 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a112bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c06cb40 con 0x7f5a0c06b4f0 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.404+0000 7f5a0bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5a0c158cb0 0x7f5a0c06ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51058/0 (socket says 172.21.15.64:51058) 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0bfff700 1 -- 172.21.15.64:0/15316424 learned_addr learned my addr 172.21.15.64:0/15316424 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a10abe700 1 --2- 172.21.15.64:0/15316424 >> v2:172.21.15.64:3300/0 conn(0x7f5a0c0a8850 0x7f5a0c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0b7fe700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c06b4f0 0x7f5a0c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0affd700 1 -- 172.21.15.64:0/15316424 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707815358 0 0) 0x7f5a0c06d470 con 0x7f5a0c158cb0 2023-09-27T23:29:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0affd700 1 -- 172.21.15.64:0/15316424 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f8003570 con 0x7f5a0c158cb0 2023-09-27T23:29:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a10abe700 1 -- 172.21.15.64:0/15316424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c06b4f0 msgr2=0x7f5a0c0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a10abe700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c06b4f0 0x7f5a0c0b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a10abe700 1 -- 172.21.15.64:0/15316424 >> v1:172.21.15.64:6789/0 conn(0x7f5a0c158cb0 legacy=0x7f5a0c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a10abe700 1 -- 172.21.15.64:0/15316424 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c06ce20 con 0x7f5a0c0a8850 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0b7fe700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c06b4f0 0x7f5a0c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a10abe700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c0a8850 0x7f5a0c06d900 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009910 tx=0x7f59fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122e9e169c5f8a4e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0affd700 1 -- 172.21.15.64:0/15316424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc011630 con 0x7f5a0c0a8850 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0affd700 1 -- 172.21.15.64:0/15316424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc011790 con 0x7f5a0c0a8850 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a0affd700 1 -- 172.21.15.64:0/15316424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc011b30 con 0x7f5a0c0a8850 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 -- 172.21.15.64:0/15316424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c0a8850 msgr2=0x7f5a0c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c0a8850 0x7f5a0c06d900 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009910 tx=0x7f59fc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 -- 172.21.15.64:0/15316424 shutdown_connections 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c0a8850 0x7f5a0c06d900 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 --2- 172.21.15.64:0/15316424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c06b4f0 0x7f5a0c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 -- 172.21.15.64:0/15316424 >> 172.21.15.64:0/15316424 conn(0x7f5a0c0684d0 msgr2=0x7f5a0c0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 -- 172.21.15.64:0/15316424 shutdown_connections 2023-09-27T23:29:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.408+0000 7f5a112bf700 1 -- 172.21.15.64:0/15316424 wait complete. 2023-09-27T23:29:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 Processor -- start 2023-09-27T23:29:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 -- start start 2023-09-27T23:29:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a0c06b4f0 0x7f5a0c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c0a8850 0x7f5a0c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 0x7f5a0c154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cb9d0 con 0x7f5a0c158cb0 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c16cb50 con 0x7f5a0c06b4f0 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a112bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cb6f0 con 0x7f5a0c0a8850 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 0x7f5a0c154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 0x7f5a0c154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39750/0 (socket says 172.21.15.64:39750) 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 -- 172.21.15.64:0/3631282845 learned_addr learned my addr 172.21.15.64:0/3631282845 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a0b7fe700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c0a8850 0x7f5a0c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a0bfff700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a0c06b4f0 0x7f5a0c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 -- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c0a8850 msgr2=0x7f5a0c0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c0a8850 0x7f5a0c0caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 -- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a0c06b4f0 msgr2=0x7f5a0c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a0c06b4f0 0x7f5a0c15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.412+0000 7f5a10abe700 1 -- 172.21.15.64:0/3631282845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4009a20 con 0x7f5a0c158cb0 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a10abe700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 0x7f5a0c154f30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f59fc013000 tx=0x7f59fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0253f0 con 0x7f5a0c158cb0 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc025570 con 0x7f5a0c158cb0 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc025930 con 0x7f5a0c158cb0 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc009420 con 0x7f5a0c158cb0 2023-09-27T23:29:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c16c7e0 con 0x7f5a0c158cb0 2023-09-27T23:29:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c16cce0 con 0x7f5a0c158cb0 2023-09-27T23:29:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.416+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f59fc025ca0 con 0x7f5a0c158cb0 2023-09-27T23:29:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.420+0000 7f5a03fff700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59ec060aa0 0x7f59ec062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.420+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f59fc07ef50 con 0x7f5a0c158cb0 2023-09-27T23:29:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.420+0000 7f5a0bfff700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59ec060aa0 0x7f59ec062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.420+0000 7f5a0bfff700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59ec060aa0 0x7f59ec062f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5a04007fd0 tx=0x7f5a04009040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:16.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.424+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f59fc052d40 con 0x7f5a0c158cb0 2023-09-27T23:29:16.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.640+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5a0c04be40 con 0x7f5a0c158cb0 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a03fff700 1 -- 172.21.15.64:0/3631282845 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f59fc033020 con 0x7f5a0c158cb0 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:29:16.649 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.650 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.651 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:29:16.651 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:29:16.651 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.651 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.651 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:16.651 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:29:16.652 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.654 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.654 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:29:16.654 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:29:16.654 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.654 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.655 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:16.655 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:16.655 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.655 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.656 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.657 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:29:16.657 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:29:16.657 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.657 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.658 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.658 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.658 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.658 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.658 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:29:16.659 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:29:16.659 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.659 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.659 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.659 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.660 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:29:16.661 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:29:16.662 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.663 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.664 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:29:16.666 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.667 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:29:16.668 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:29:16.669 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.671 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:29:16.672 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:29:16.673 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:16.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:16.674 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:29:16.674 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59ec060aa0 msgr2=0x7f59ec062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59ec060aa0 0x7f59ec062f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5a04007fd0 tx=0x7f5a04009040 comp rx=0 tx=0).stop 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 msgr2=0x7f5a0c154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 0x7f5a0c154f30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f59fc013000 tx=0x7f59fc017000 comp rx=0 tx=0).stop 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 shutdown_connections 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f59ec060aa0 0x7f59ec062f50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c158cb0 0x7f5a0c154f30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a0c0a8850 0x7f5a0c0caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 --2- 172.21.15.64:0/3631282845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a0c06b4f0 0x7f5a0c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 >> 172.21.15.64:0/3631282845 conn(0x7f5a0c0684d0 msgr2=0x7f5a0c0aac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:16.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 shutdown_connections 2023-09-27T23:29:16.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.644+0000 7f5a112bf700 1 -- 172.21.15.64:0/3631282845 wait complete. 2023-09-27T23:29:16.678 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:16.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e130e700 1 Processor -- start 2023-09-27T23:29:16.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e130e700 1 -- start start 2023-09-27T23:29:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e130e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23dc159590 0x7f23dc158200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e130e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0c63a0 con 0x7f23dc159160 2023-09-27T23:29:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e130e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0accb0 con 0x7f23dc0c5fd0 2023-09-27T23:29:16.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e130e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc06a7b0 con 0x7f23dc159590 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e0b0d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23dc159590 0x7f23dc158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e0b0d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23dc159590 0x7f23dc158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39766/0 (socket says 172.21.15.64:39766) 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e0b0d700 1 -- 172.21.15.64:0/3867548862 learned_addr learned my addr 172.21.15.64:0/3867548862 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e0b0d700 1 -- 172.21.15.64:0/3867548862 >> v1:172.21.15.123:6789/0 conn(0x7f23dc0c5fd0 legacy=0x7f23dc0cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23daffd700 1 -- 172.21.15.64:0/3867548862 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724916448 0 0) 0x7f23dc0c63a0 con 0x7f23dc159160 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e0b0d700 1 -- 172.21.15.64:0/3867548862 >> v1:172.21.15.64:6789/0 conn(0x7f23dc159160 legacy=0x7f23dc0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.780+0000 7f23e0b0d700 1 -- 172.21.15.64:0/3867548862 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0acf90 con 0x7f23dc159590 2023-09-27T23:29:16.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e0b0d700 1 --2- 172.21.15.64:0/3867548862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159590 0x7f23dc158200 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009f20 tx=0x7f23d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7982a8c6fda862 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:16.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23daffd700 1 -- 172.21.15.64:0/3867548862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d400e070 con 0x7f23dc159590 2023-09-27T23:29:16.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23daffd700 1 -- 172.21.15.64:0/3867548862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d400ba80 con 0x7f23dc159590 2023-09-27T23:29:16.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23daffd700 1 -- 172.21.15.64:0/3867548862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d40092e0 con 0x7f23dc159590 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 -- 172.21.15.64:0/3867548862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159590 msgr2=0x7f23dc158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 --2- 172.21.15.64:0/3867548862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159590 0x7f23dc158200 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009f20 tx=0x7f23d4009840 comp rx=0 tx=0).stop 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 -- 172.21.15.64:0/3867548862 shutdown_connections 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 --2- 172.21.15.64:0/3867548862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159590 0x7f23dc158200 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 -- 172.21.15.64:0/3867548862 >> 172.21.15.64:0/3867548862 conn(0x7f23dc067bb0 msgr2=0x7f23dc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 -- 172.21.15.64:0/3867548862 shutdown_connections 2023-09-27T23:29:16.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.784+0000 7f23e130e700 1 -- 172.21.15.64:0/3867548862 wait complete. 2023-09-27T23:29:16.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 Processor -- start 2023-09-27T23:29:16.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 -- start start 2023-09-27T23:29:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc0c5fd0 0x7f23dc0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 0x7f23dc14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23dc159590 0x7f23dc0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0c5870 con 0x7f23dc159160 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0c4c60 con 0x7f23dc0c5fd0 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e130e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0c4f40 con 0x7f23dc159590 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 0x7f23dc14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc0c5fd0 0x7f23dc0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 0x7f23dc14b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39778/0 (socket says 172.21.15.64:39778) 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 -- 172.21.15.64:0/3671613223 learned_addr learned my addr 172.21.15.64:0/3671613223 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23e0b0d700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23dc159590 0x7f23dc0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 -- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23dc159590 msgr2=0x7f23dc0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23dc159590 0x7f23dc0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 -- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc0c5fd0 msgr2=0x7f23dc0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc0c5fd0 0x7f23dc0c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.788+0000 7f23db7fe700 1 -- 172.21.15.64:0/3671613223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4009ae0 con 0x7f23dc159160 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23db7fe700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 0x7f23dc14b160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00c7e0 tx=0x7f23cc007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc011590 con 0x7f23dc159160 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc011710 con 0x7f23dc159160 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc0c5220 con 0x7f23dc159160 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc011ad0 con 0x7f23dc159160 2023-09-27T23:29:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc0c5d00 con 0x7f23dc159160 2023-09-27T23:29:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f23cc01b3b0 con 0x7f23dc159160 2023-09-27T23:29:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc0c59e0 con 0x7f23dc159160 2023-09-27T23:29:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23c3fff700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f23c8060a70 0x7f23c8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f23cc07c310 con 0x7f23dc159160 2023-09-27T23:29:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.792+0000 7f23dbfff700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f23c8060a70 0x7f23c8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.796+0000 7f23dbfff700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f23c8060a70 0x7f23c8062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23d0003a80 tx=0x7f23d0007ec0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:16.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.800+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f23cc04c5f0 con 0x7f23dc159160 2023-09-27T23:29:16.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.964+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23dc0c56a0 con 0x7f23dc159160 2023-09-27T23:29:16.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23c3fff700 1 -- 172.21.15.64:0/3671613223 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f23cc017090 con 0x7f23dc159160 2023-09-27T23:29:16.973 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:16.973 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":83,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:16.212193+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f23c8060a70 msgr2=0x7f23c8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f23c8060a70 0x7f23c8062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23d0003a80 tx=0x7f23d0007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 msgr2=0x7f23dc14b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 0x7f23dc14b160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00c7e0 tx=0x7f23cc007da0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 shutdown_connections 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f23c8060a70 0x7f23c8062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23dc159590 0x7f23dc0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23dc159160 0x7f23dc14b160 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 --2- 172.21.15.64:0/3671613223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc0c5fd0 0x7f23dc0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 >> 172.21.15.64:0/3671613223 conn(0x7f23dc067bb0 msgr2=0x7f23dc14bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 shutdown_connections 2023-09-27T23:29:16.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:16.968+0000 7f23e130e700 1 -- 172.21.15.64:0/3671613223 wait complete. 2023-09-27T23:29:16.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:29:16.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:29:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.096+0000 7f2db1a7c700 1 Processor -- start 2023-09-27T23:29:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.096+0000 7f2db1a7c700 1 -- start start 2023-09-27T23:29:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2dac159790 0x7f2dac0684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac067790 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac068030 con 0x7f2dac0b0cb0 2023-09-27T23:29:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dac0b1200 con 0x7f2dac159790 2023-09-27T23:29:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dabfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2dac0b1ed0 0x7f2dac159b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51076/0 (socket says 172.21.15.64:51076) 2023-09-27T23:29:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dabfff700 1 -- 172.21.15.64:0/2181332771 learned_addr learned my addr 172.21.15.64:0/2181332771 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:17.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dab7fe700 1 --2- 172.21.15.64:0/2181332771 >> v2:172.21.15.123:3300/0 conn(0x7f2dac159790 0x7f2dac0684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426626412 0 0) 0x7f2dac067790 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d94003570 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054361737 0 0) 0x7f2dac068030 con 0x7f2dac0b0cb0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dac067790 con 0x7f2dac0b0cb0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741938656 0 0) 0x7f2d94003570 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dab7fe700 1 -- 172.21.15.64:0/2181332771 >> v1:172.21.15.123:6789/0 conn(0x7f2dac0b0cb0 legacy=0x7f2dac0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dab7fe700 1 -- 172.21.15.64:0/2181332771 >> v1:172.21.15.64:6789/0 conn(0x7f2dac0b1ed0 legacy=0x7f2dac159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dab7fe700 1 -- 172.21.15.64:0/2181332771 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac0679e0 con 0x7f2dac159790 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2dab7fe700 1 --2- 172.21.15.64:0/2181332771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac159790 0x7f2dac0684c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2da00098e0 tx=0x7f2da00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=508a5bff15136dec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40017f0 con 0x7f2dac159790 2023-09-27T23:29:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0011420 con 0x7f2dac159790 2023-09-27T23:29:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2daa7fc700 1 -- 172.21.15.64:0/2181332771 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da00117c0 con 0x7f2dac159790 2023-09-27T23:29:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- 172.21.15.64:0/2181332771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac159790 msgr2=0x7f2dac0684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/2181332771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac159790 0x7f2dac0684c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2da00098e0 tx=0x7f2da00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- 172.21.15.64:0/2181332771 shutdown_connections 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/2181332771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac159790 0x7f2dac0684c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- 172.21.15.64:0/2181332771 >> 172.21.15.64:0/2181332771 conn(0x7f2dac0cdef0 msgr2=0x7f2dac0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- 172.21.15.64:0/2181332771 shutdown_connections 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.100+0000 7f2db1a7c700 1 -- 172.21.15.64:0/2181332771 wait complete. 2023-09-27T23:29:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 Processor -- start 2023-09-27T23:29:17.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 -- start start 2023-09-27T23:29:17.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac0b0cb0 0x7f2dac148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 0x7f2dac0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2dac159790 0x7f2dac0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0c7830 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0c80d0 con 0x7f2dac0b0cb0 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2db1a7c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0c9450 con 0x7f2dac159790 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 0x7f2dac0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 0x7f2dac0c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39782/0 (socket says 172.21.15.64:39782) 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 -- 172.21.15.64:0/1481734538 learned_addr learned my addr 172.21.15.64:0/1481734538 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2dab7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac0b0cb0 0x7f2dac148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2dabfff700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2dac159790 0x7f2dac0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 -- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2dac159790 msgr2=0x7f2dac0c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2dac159790 0x7f2dac0c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 -- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac0b0cb0 msgr2=0x7f2dac148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac0b0cb0 0x7f2dac148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 -- 172.21.15.64:0/1481734538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0009420 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.104+0000 7f2daaffd700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 0x7f2dac0c8a90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00c7f0 tx=0x7f2d9c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c010510 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c010690 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac0c7a80 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c010a50 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac0c8560 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dac0c8260 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2d9c01b3b0 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2d937fe700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2d98060a70 0x7f2d98062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2d9c07c630 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.108+0000 7f2dab7fe700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2d98060a70 0x7f2d98062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.112+0000 7f2dab7fe700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2d98060a70 0x7f2d98062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2da00093f0 tx=0x7f2da0016000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.116+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2d9c04c9b0 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.280+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2dac071120 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.280+0000 7f2d937fe700 1 -- 172.21.15.64:0/1481734538 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d9c017020 con 0x7f2dac0b1ed0 2023-09-27T23:29:17.286 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:29:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.280+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2d98060a70 msgr2=0x7f2d98062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.280+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2d98060a70 0x7f2d98062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2da00093f0 tx=0x7f2da0016000 comp rx=0 tx=0).stop 2023-09-27T23:29:17.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.280+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 msgr2=0x7f2dac0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.280+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 0x7f2dac0c8a90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00c7f0 tx=0x7f2d9c007de0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 shutdown_connections 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2d98060a70 0x7f2d98062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2dac159790 0x7f2dac0c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dac0b1ed0 0x7f2dac0c8a90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 --2- 172.21.15.64:0/1481734538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dac0b0cb0 0x7f2dac148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 >> 172.21.15.64:0/1481734538 conn(0x7f2dac0cdef0 msgr2=0x7f2dac155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 shutdown_connections 2023-09-27T23:29:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.284+0000 7f2db1a7c700 1 -- 172.21.15.64:0/1481734538 wait complete. 2023-09-27T23:29:17.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:29:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.404+0000 7f1127fff700 1 Processor -- start 2023-09-27T23:29:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.404+0000 7f1127fff700 1 -- start start 2023-09-27T23:29:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280ce2d0 0x7f11280ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11280cc040 0x7f1128159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11280cbbb0 con 0x7f11280caa10 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112806dc90 con 0x7f11280cc040 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280cafa0 con 0x7f11280ce2d0 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11280cc040 0x7f1128159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11280cc040 0x7f1128159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39794/0 (socket says 172.21.15.64:39794) 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 -- 172.21.15.64:0/1308776773 learned_addr learned my addr 172.21.15.64:0/1308776773 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1126ffd700 1 --2- 172.21.15.64:0/1308776773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280ce2d0 0x7f11280ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 -- 172.21.15.64:0/1308776773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280ce2d0 msgr2=0x7f11280ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 --2- 172.21.15.64:0/1308776773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280ce2d0 0x7f11280ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 -- 172.21.15.64:0/1308776773 >> v1:172.21.15.123:6789/0 conn(0x7f11280caa10 legacy=0x7f1128158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 -- 172.21.15.64:0/1308776773 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280cb280 con 0x7f11280cc040 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11277fe700 1 --2- 172.21.15.64:0/1308776773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280cc040 0x7f1128159010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1120009f20 tx=0x7f1120009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d6889cf9ef9f9c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11267fc700 1 -- 172.21.15.64:0/1308776773 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000e070 con 0x7f11280cc040 2023-09-27T23:29:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11267fc700 1 -- 172.21.15.64:0/1308776773 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112000ba80 con 0x7f11280cc040 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f11267fc700 1 -- 172.21.15.64:0/1308776773 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11200092e0 con 0x7f11280cc040 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- 172.21.15.64:0/1308776773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280cc040 msgr2=0x7f1128159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 --2- 172.21.15.64:0/1308776773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280cc040 0x7f1128159010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1120009f20 tx=0x7f1120009840 comp rx=0 tx=0).stop 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- 172.21.15.64:0/1308776773 shutdown_connections 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 --2- 172.21.15.64:0/1308776773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280cc040 0x7f1128159010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 --2- 172.21.15.64:0/1308776773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280ce2d0 0x7f11280ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- 172.21.15.64:0/1308776773 >> 172.21.15.64:0/1308776773 conn(0x7f1128067bb0 msgr2=0x7f11280cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- 172.21.15.64:0/1308776773 shutdown_connections 2023-09-27T23:29:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.408+0000 7f1127fff700 1 -- 172.21.15.64:0/1308776773 wait complete. 2023-09-27T23:29:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 Processor -- start 2023-09-27T23:29:17.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 -- start start 2023-09-27T23:29:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11280caa10 0x7f11280c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280cc040 0x7f11281596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 0x7f11280c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280c9910 con 0x7f11280ce2d0 2023-09-27T23:29:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b3750 con 0x7f11280caa10 2023-09-27T23:29:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1127fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b2e20 con 0x7f11280cc040 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 0x7f11280c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 0x7f11280c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39810/0 (socket says 172.21.15.64:39810) 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 -- 172.21.15.64:0/3624925863 learned_addr learned my addr 172.21.15.64:0/3624925863 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f1126ffd700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11280caa10 0x7f11280c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f111ffff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280cc040 0x7f11281596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 -- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280cc040 msgr2=0x7f11281596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280cc040 0x7f11281596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 -- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11280caa10 msgr2=0x7f11280c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11280caa10 0x7f11280c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 -- 172.21.15.64:0/3624925863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120009ae0 con 0x7f11280ce2d0 2023-09-27T23:29:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f11277fe700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 0x7f11280c7040 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f112000b6c0 tx=0x7f1120018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.412+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11200032f0 con 0x7f11280ce2d0 2023-09-27T23:29:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120008d10 con 0x7f11280ce2d0 2023-09-27T23:29:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11280b3100 con 0x7f11280ce2d0 2023-09-27T23:29:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120026690 con 0x7f11280ce2d0 2023-09-27T23:29:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11280c94e0 con 0x7f11280ce2d0 2023-09-27T23:29:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11280c96d0 con 0x7f11280ce2d0 2023-09-27T23:29:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1120003040 con 0x7f11280ce2d0 2023-09-27T23:29:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f111effd700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1108060a70 0x7f1108062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f112007eef0 con 0x7f11280ce2d0 2023-09-27T23:29:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f1126ffd700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1108060a70 0x7f1108062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.416+0000 7f1126ffd700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1108060a70 0x7f1108062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1118006fd0 tx=0x7f1118008040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.420+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f11280c96d0 con 0x7f11280ce2d0 2023-09-27T23:29:17.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.576+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f11280ca050 con 0x7f11280ce2d0 2023-09-27T23:29:17.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.576+0000 7f111effd700 1 -- 172.21.15.64:0/3624925863 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7f11280ca050 con 0x7f11280ce2d0 2023-09-27T23:29:17.583 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:29:17.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1108060a70 msgr2=0x7f1108062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1108060a70 0x7f1108062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1118006fd0 tx=0x7f1118008040 comp rx=0 tx=0).stop 2023-09-27T23:29:17.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 msgr2=0x7f11280c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 0x7f11280c7040 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f112000b6c0 tx=0x7f1120018000 comp rx=0 tx=0).stop 2023-09-27T23:29:17.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 shutdown_connections 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1108060a70 0x7f1108062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11280ce2d0 0x7f11280c7040 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11280cc040 0x7f11281596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 --2- 172.21.15.64:0/3624925863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11280caa10 0x7f11280c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 >> 172.21.15.64:0/3624925863 conn(0x7f1128067bb0 msgr2=0x7f11280c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 shutdown_connections 2023-09-27T23:29:17.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.580+0000 7f1127fff700 1 -- 172.21.15.64:0/3624925863 wait complete. 2023-09-27T23:29:17.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:29:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.704+0000 7f4899302700 1 Processor -- start 2023-09-27T23:29:17.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.704+0000 7f4899302700 1 -- start start 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48940c6900 0x7f48940c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489406b470 con 0x7f489406b900 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489406e110 con 0x7f489415cc20 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489415d790 con 0x7f48940c6900 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f48937fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f489406b900 0x7f489406da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51084/0 (socket says 172.21.15.64:51084) 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f48937fe700 1 -- 172.21.15.64:0/3411810418 learned_addr learned my addr 172.21.15.64:0/3411810418 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:17.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4893fff700 1 --2- 172.21.15.64:0/3411810418 >> v2:172.21.15.64:3300/0 conn(0x7f48940c6900 0x7f48940c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4892ffd700 1 -- 172.21.15.64:0/3411810418 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720827971 0 0) 0x7f489406b470 con 0x7f489406b900 2023-09-27T23:29:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4892ffd700 1 -- 172.21.15.64:0/3411810418 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4880003570 con 0x7f489406b900 2023-09-27T23:29:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4893fff700 1 -- 172.21.15.64:0/3411810418 >> v1:172.21.15.123:6789/0 conn(0x7f489415cc20 legacy=0x7f489415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4893fff700 1 -- 172.21.15.64:0/3411810418 >> v1:172.21.15.64:6789/0 conn(0x7f489406b900 legacy=0x7f489406da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4893fff700 1 -- 172.21.15.64:0/3411810418 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894161d10 con 0x7f48940c6900 2023-09-27T23:29:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4893fff700 1 --2- 172.21.15.64:0/3411810418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48940c6900 0x7f48940c6cd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415c751834a66a94 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4892ffd700 1 -- 172.21.15.64:0/3411810418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400b640 con 0x7f48940c6900 2023-09-27T23:29:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4892ffd700 1 -- 172.21.15.64:0/3411810418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b7a0 con 0x7f48940c6900 2023-09-27T23:29:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4892ffd700 1 -- 172.21.15.64:0/3411810418 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bb20 con 0x7f48940c6900 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- 172.21.15.64:0/3411810418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48940c6900 msgr2=0x7f48940c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 --2- 172.21.15.64:0/3411810418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48940c6900 0x7f48940c6cd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- 172.21.15.64:0/3411810418 shutdown_connections 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 --2- 172.21.15.64:0/3411810418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48940c6900 0x7f48940c6cd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- 172.21.15.64:0/3411810418 >> 172.21.15.64:0/3411810418 conn(0x7f4894067bb0 msgr2=0x7f48940cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- 172.21.15.64:0/3411810418 shutdown_connections 2023-09-27T23:29:17.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.708+0000 7f4899302700 1 -- 172.21.15.64:0/3411810418 wait complete. 2023-09-27T23:29:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.712+0000 7f4899302700 1 Processor -- start 2023-09-27T23:29:17.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.712+0000 7f4899302700 1 -- start start 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489406b900 0x7f48940b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48940c6900 0x7f489414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 0x7f48940c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b0e60 con 0x7f489415cc20 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894159570 con 0x7f48940c6900 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b0250 con 0x7f489406b900 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 0x7f48940c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 0x7f48940c0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39840/0 (socket says 172.21.15.64:39840) 2023-09-27T23:29:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 -- 172.21.15.64:0/1920806077 learned_addr learned my addr 172.21.15.64:0/1920806077 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4893fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489406b900 0x7f48940b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f48937fe700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48940c6900 0x7f489414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 -- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489406b900 msgr2=0x7f48940b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489406b900 0x7f48940b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 -- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48940c6900 msgr2=0x7f489414a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48940c6900 0x7f489414a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 -- 172.21.15.64:0/1920806077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884009420 con 0x7f489415cc20 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4898b01700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 0x7f48940c0860 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c870 tx=0x7f488c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c012630 con 0x7f489415cc20 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c0127b0 con 0x7f489415cc20 2023-09-27T23:29:17.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48940b0530 con 0x7f489415cc20 2023-09-27T23:29:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c012b70 con 0x7f489415cc20 2023-09-27T23:29:17.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.716+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48940b0af0 con 0x7f489415cc20 2023-09-27T23:29:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.720+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f488c003030 con 0x7f489415cc20 2023-09-27T23:29:17.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.720+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48940b1010 con 0x7f489415cc20 2023-09-27T23:29:17.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.720+0000 7f488bfff700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4874060a70 0x7f4874062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:17.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.720+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f488c07ccc0 con 0x7f489415cc20 2023-09-27T23:29:17.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.720+0000 7f4893fff700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4874060a70 0x7f4874062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.720+0000 7f4893fff700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4874060a70 0x7f4874062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48840093f0 tx=0x7f4884017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:17.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.728+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f48940b1010 con 0x7f489415cc20 2023-09-27T23:29:17.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.892+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f48940b1010 con 0x7f489415cc20 2023-09-27T23:29:17.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.892+0000 7f488bfff700 1 -- 172.21.15.64:0/1920806077 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+126 (secure 0 0 0) 0x7f48940b1010 con 0x7f489415cc20 2023-09-27T23:29:17.898 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:17.899 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:29:17.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4874060a70 msgr2=0x7f4874062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4874060a70 0x7f4874062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48840093f0 tx=0x7f4884017000 comp rx=0 tx=0).stop 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 msgr2=0x7f48940c0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 0x7f48940c0860 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c870 tx=0x7f488c001670 comp rx=0 tx=0).stop 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 shutdown_connections 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4874060a70 0x7f4874062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489415cc20 0x7f48940c0860 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48940c6900 0x7f489414a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 --2- 172.21.15.64:0/1920806077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489406b900 0x7f48940b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 >> 172.21.15.64:0/1920806077 conn(0x7f4894067bb0 msgr2=0x7f489415a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 shutdown_connections 2023-09-27T23:29:17.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:17.896+0000 7f4899302700 1 -- 172.21.15.64:0/1920806077 wait complete. 2023-09-27T23:29:17.912 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:29:17.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:29:18.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.020+0000 7fb080be8700 1 Processor -- start 2023-09-27T23:29:18.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.020+0000 7fb080be8700 1 -- start start 2023-09-27T23:29:18.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb07c06af50 0x7fb07c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb07c06d6b0 0x7fb07c06db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c0cc140 con 0x7fb07c06d6b0 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c0cb3f0 con 0x7fb07c06af50 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c06d220 con 0x7fb07c0aedc0 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb07c06d6b0 0x7fb07c06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb07c06d6b0 0x7fb07c06db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39844/0 (socket says 172.21.15.64:39844) 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 -- 172.21.15.64:0/2361972064 learned_addr learned my addr 172.21.15.64:0/2361972064 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07b7fe700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07affd700 1 --2- 172.21.15.64:0/2361972064 >> v2:172.21.15.123:3300/0 conn(0x7fb07c06af50 0x7fb07c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 -- 172.21.15.64:0/2361972064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 msgr2=0x7fb07c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:18.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 -- 172.21.15.64:0/2361972064 >> v2:172.21.15.123:3300/0 conn(0x7fb07c06af50 msgr2=0x7fb07c0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 --2- 172.21.15.64:0/2361972064 >> v2:172.21.15.123:3300/0 conn(0x7fb07c06af50 0x7fb07c0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 -- 172.21.15.64:0/2361972064 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c0cb190 con 0x7fb07c06d6b0 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07b7fe700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07bfff700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06d6b0 0x7fb07c06db30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb07c068210 tx=0x7fb074009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e2472685e852c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07a7fc700 1 -- 172.21.15.64:0/2361972064 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07400e070 con 0x7fb07c06d6b0 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07a7fc700 1 -- 172.21.15.64:0/2361972064 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07400ba50 con 0x7fb07c06d6b0 2023-09-27T23:29:18.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb07a7fc700 1 -- 172.21.15.64:0/2361972064 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740092e0 con 0x7fb07c06d6b0 2023-09-27T23:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- 172.21.15.64:0/2361972064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06d6b0 msgr2=0x7fb07c06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06d6b0 0x7fb07c06db30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb07c068210 tx=0x7fb074009840 comp rx=0 tx=0).stop 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- 172.21.15.64:0/2361972064 shutdown_connections 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06d6b0 0x7fb07c06db30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- 172.21.15.64:0/2361972064 >> v2:172.21.15.123:3300/0 conn(0x7fb07c06af50 0x7fb07c0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 --2- 172.21.15.64:0/2361972064 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- 172.21.15.64:0/2361972064 >> 172.21.15.64:0/2361972064 conn(0x7fb07c0cc760 msgr2=0x7fb07c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- 172.21.15.64:0/2361972064 shutdown_connections 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.024+0000 7fb080be8700 1 -- 172.21.15.64:0/2361972064 wait complete. 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 Processor -- start 2023-09-27T23:29:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 -- start start 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 0x7fb07c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c06d6b0 0x7fb07c0c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0c5e80 con 0x7fb07c06af50 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0c3fa0 con 0x7fb07c06d6b0 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0c6910 con 0x7fb07c0aedc0 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 0x7fb07c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 0x7fb07c151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39858/0 (socket says 172.21.15.64:39858) 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 -- 172.21.15.64:0/215409393 learned_addr learned my addr 172.21.15.64:0/215409393 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07bfff700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07affd700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c06d6b0 0x7fb07c0c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 -- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 msgr2=0x7fb07c0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 -- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c06d6b0 msgr2=0x7fb07c0c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c06d6b0 0x7fb07c0c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 -- 172.21.15.64:0/215409393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009ae0 con 0x7fb07c06af50 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb07b7fe700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 0x7fb07c151b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00e900 tx=0x7fb06c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c013410 con 0x7fb07c06af50 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c152040 con 0x7fb07c06af50 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c013570 con 0x7fb07c06af50 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.028+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c0c6350 con 0x7fb07c06af50 2023-09-27T23:29:18.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.032+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0138f0 con 0x7fb07c06af50 2023-09-27T23:29:18.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.032+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c04be40 con 0x7fb07c06af50 2023-09-27T23:29:18.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.032+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb06c013a50 con 0x7fb07c06af50 2023-09-27T23:29:18.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.032+0000 7fb0737fe700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb05c060a70 0x7fb05c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.032+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fb06c07c9c0 con 0x7fb07c06af50 2023-09-27T23:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.036+0000 7fb07affd700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb05c060a70 0x7fb05c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.036+0000 7fb07affd700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb05c060a70 0x7fb05c062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb07c15b1f0 tx=0x7fb064008040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:18.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.040+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb06c04cb90 con 0x7fb07c06af50 2023-09-27T23:29:18.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.204+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb07c06f3d0 con 0x7fb07c06af50 2023-09-27T23:29:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.304+0000 7fb0737fe700 1 -- 172.21.15.64:0/215409393 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb06c0200b0 con 0x7fb07c06af50 2023-09-27T23:29:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.312+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb05c060a70 msgr2=0x7fb05c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.312+0000 7fb080be8700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb05c060a70 0x7fb05c062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb07c15b1f0 tx=0x7fb064008040 comp rx=0 tx=0).stop 2023-09-27T23:29:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.312+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 msgr2=0x7fb07c151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.312+0000 7fb080be8700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 0x7fb07c151b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00e900 tx=0x7fb06c008040 comp rx=0 tx=0).stop 2023-09-27T23:29:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 shutdown_connections 2023-09-27T23:29:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb05c060a70 0x7fb05c062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb07c0aedc0 0x7fb07c0c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c06d6b0 0x7fb07c0c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 --2- 172.21.15.64:0/215409393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb07c06af50 0x7fb07c151b00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 >> 172.21.15.64:0/215409393 conn(0x7fb07c0cc760 msgr2=0x7fb07c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 shutdown_connections 2023-09-27T23:29:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.316+0000 7fb080be8700 1 -- 172.21.15.64:0/215409393 wait complete. 2023-09-27T23:29:18.324 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:29:18.335 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:29:18.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa95d700 1 Processor -- start 2023-09-27T23:29:18.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa95d700 1 -- start start 2023-09-27T23:29:18.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa95d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40cc040 0x7ff7a4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa95d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a40cbbb0 con 0x7ff7a40ce2d0 2023-09-27T23:29:18.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa95d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a406dc90 con 0x7ff7a40caa10 2023-09-27T23:29:18.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa95d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40cafa0 con 0x7ff7a40cc040 2023-09-27T23:29:18.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7a995b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff7a40ce2d0 0x7ff7a4158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51086/0 (socket says 172.21.15.64:51086) 2023-09-27T23:29:18.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7a995b700 1 -- 172.21.15.64:0/1172952991 learned_addr learned my addr 172.21.15.64:0/1172952991 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:18.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.444+0000 7ff7aa15c700 1 --2- 172.21.15.64:0/1172952991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40cc040 0x7ff7a4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521589457 0 0) 0x7ff7a40cbbb0 con 0x7ff7a40ce2d0 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff78c003570 con 0x7ff7a40ce2d0 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478265730 0 0) 0x7ff7a406dc90 con 0x7ff7a40caa10 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a40cbbb0 con 0x7ff7a40caa10 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa15c700 1 -- 172.21.15.64:0/1172952991 >> v1:172.21.15.123:6789/0 conn(0x7ff7a40caa10 legacy=0x7ff7a406a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778085868 0 0) 0x7ff78c003570 con 0x7ff7a40ce2d0 2023-09-27T23:29:18.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa15c700 1 -- 172.21.15.64:0/1172952991 >> v1:172.21.15.64:6789/0 conn(0x7ff7a40ce2d0 legacy=0x7ff7a4158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa15c700 1 -- 172.21.15.64:0/1172952991 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40cb280 con 0x7ff7a40cc040 2023-09-27T23:29:18.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa15c700 1 --2- 172.21.15.64:0/1172952991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40cc040 0x7ff7a4159010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009910 tx=0x7ff79c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b413e8023d9eb62 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7940017f0 con 0x7ff7a40cc040 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c011420 con 0x7ff7a40cc040 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7a8959700 1 -- 172.21.15.64:0/1172952991 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0117c0 con 0x7ff7a40cc040 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 -- 172.21.15.64:0/1172952991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40cc040 msgr2=0x7ff7a4159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/1172952991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40cc040 0x7ff7a4159010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009910 tx=0x7ff79c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 -- 172.21.15.64:0/1172952991 shutdown_connections 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/1172952991 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40cc040 0x7ff7a4159010 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 -- 172.21.15.64:0/1172952991 >> 172.21.15.64:0/1172952991 conn(0x7ff7a4067bb0 msgr2=0x7ff7a40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 -- 172.21.15.64:0/1172952991 shutdown_connections 2023-09-27T23:29:18.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.448+0000 7ff7aa95d700 1 -- 172.21.15.64:0/1172952991 wait complete. 2023-09-27T23:29:18.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 Processor -- start 2023-09-27T23:29:18.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 -- start start 2023-09-27T23:29:18.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40caa10 0x7ff7a40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 0x7ff7a40b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40ce2d0 0x7ff7a4144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40b3b50 con 0x7ff7a40cc040 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40b3e30 con 0x7ff7a40ce2d0 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa95d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40b4110 con 0x7ff7a40caa10 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7a915a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 0x7ff7a40b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7a915a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 0x7ff7a40b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39874/0 (socket says 172.21.15.64:39874) 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7a915a700 1 -- 172.21.15.64:0/2899324286 learned_addr learned my addr 172.21.15.64:0/2899324286 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7a995b700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40caa10 0x7ff7a40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.452+0000 7ff7aa15c700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40ce2d0 0x7ff7a4144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a915a700 1 -- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40caa10 msgr2=0x7ff7a40c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a915a700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40caa10 0x7ff7a40c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a915a700 1 -- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40ce2d0 msgr2=0x7ff7a4144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a915a700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40ce2d0 0x7ff7a4144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a915a700 1 -- 172.21.15.64:0/2899324286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c009420 con 0x7ff7a40cc040 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a915a700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 0x7ff7a40b4b10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c790 tx=0x7ff798007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798010510 con 0x7ff7a40cc040 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798010670 con 0x7ff7a40cc040 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a40b43f0 con 0x7ff7a40cc040 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798010a10 con 0x7ff7a40cc040 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a40b3700 con 0x7ff7a40cc040 2023-09-27T23:29:18.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.456+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a404be40 con 0x7ff7a40cc040 2023-09-27T23:29:18.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.460+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff79801b3b0 con 0x7ff7a40cc040 2023-09-27T23:29:18.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.460+0000 7ff7a1ffb700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff790060aa0 0x7ff790062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:18.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.460+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5896+0+0 (secure 0 0 0) 0x7ff79807cba0 con 0x7ff7a40cc040 2023-09-27T23:29:18.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.460+0000 7ff7a995b700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff790060aa0 0x7ff790062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:18.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.460+0000 7ff7a995b700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff790060aa0 0x7ff790062f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff794001560 tx=0x7ff794000f40 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:18.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.468+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff79804cdb0 con 0x7ff7a40cc040 2023-09-27T23:29:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:18.632+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff7a40cb560 con 0x7ff7a40cc040 2023-09-27T23:29:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.320+0000 7ff7a1ffb700 1 -- 172.21.15.64:0/2899324286 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff798017070 con 0x7ff7a40cc040 2023-09-27T23:29:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff790060aa0 msgr2=0x7ff790062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff790060aa0 0x7ff790062f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff794001560 tx=0x7ff794000f40 comp rx=0 tx=0).stop 2023-09-27T23:29:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 msgr2=0x7ff7a40b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 0x7ff7a40b4b10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c790 tx=0x7ff798007e10 comp rx=0 tx=0).stop 2023-09-27T23:29:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 shutdown_connections 2023-09-27T23:29:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff790060aa0 0x7ff790062f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40ce2d0 0x7ff7a4144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.332+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7a40cc040 0x7ff7a40b4b10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.336+0000 7ff7aa95d700 1 --2- 172.21.15.64:0/2899324286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7a40caa10 0x7ff7a40c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.336+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 >> 172.21.15.64:0/2899324286 conn(0x7ff7a4067bb0 msgr2=0x7ff7a40c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.344+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 shutdown_connections 2023-09-27T23:29:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.344+0000 7ff7aa95d700 1 -- 172.21.15.64:0/2899324286 wait complete. 2023-09-27T23:29:19.352 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:29:19.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:29:19.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.468+0000 7fd25cc26700 1 Processor -- start 2023-09-27T23:29:19.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.468+0000 7fd25cc26700 1 -- start start 2023-09-27T23:29:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2580c7170 0x7fd2580ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580ad0c0 con 0x7fd258150200 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2580af530 con 0x7fd2580c7170 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c7610 con 0x7fd2580acc60 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd256ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2580c7170 0x7fd2580ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd2577fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd258150200 0x7fd2580ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51096/0 (socket says 172.21.15.64:51096) 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd2577fe700 1 -- 172.21.15.64:0/1640695727 learned_addr learned my addr 172.21.15.64:0/1640695727 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd257fff700 1 --2- 172.21.15.64:0/1640695727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd256ffd700 1 -- 172.21.15.64:0/1640695727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 msgr2=0x7fd2580cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd256ffd700 1 --2- 172.21.15.64:0/1640695727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd256ffd700 1 -- 172.21.15.64:0/1640695727 >> v1:172.21.15.64:6789/0 conn(0x7fd258150200 legacy=0x7fd2580ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd256ffd700 1 -- 172.21.15.64:0/1640695727 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580af250 con 0x7fd2580c7170 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd256ffd700 1 --2- 172.21.15.64:0/1640695727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2580ac720 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098e0 tx=0x7fd2400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7481989600de46 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd2567fc700 1 -- 172.21.15.64:0/1640695727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000b640 con 0x7fd2580c7170 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd2567fc700 1 -- 172.21.15.64:0/1640695727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24000b7a0 con 0x7fd2580c7170 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd2567fc700 1 -- 172.21.15.64:0/1640695727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000bb40 con 0x7fd2580c7170 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- 172.21.15.64:0/1640695727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 msgr2=0x7fd2580ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1640695727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2580ac720 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098e0 tx=0x7fd2400091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- 172.21.15.64:0/1640695727 shutdown_connections 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1640695727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1640695727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2580ac720 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- 172.21.15.64:0/1640695727 >> 172.21.15.64:0/1640695727 conn(0x7fd2580ccb30 msgr2=0x7fd2580ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- 172.21.15.64:0/1640695727 shutdown_connections 2023-09-27T23:29:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.472+0000 7fd25cc26700 1 -- 172.21.15.64:0/1640695727 wait complete. 2023-09-27T23:29:19.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.476+0000 7fd25cc26700 1 Processor -- start 2023-09-27T23:29:19.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.476+0000 7fd25cc26700 1 -- start start 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2581460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd258150200 0x7fd2580af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd258147f50 con 0x7fd2580c7170 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580afe10 con 0x7fd258150200 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2581467e0 con 0x7fd2580acc60 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2581460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2581460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39900/0 (socket says 172.21.15.64:39900) 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 -- 172.21.15.64:0/1386893618 learned_addr learned my addr 172.21.15.64:0/1386893618 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd2577fe700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd257fff700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd258150200 0x7fd2580af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 -- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 msgr2=0x7fd2580c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 -- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd258150200 msgr2=0x7fd2580af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd258150200 0x7fd2580af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 -- 172.21.15.64:0/1386893618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240009420 con 0x7fd2580c7170 2023-09-27T23:29:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd256ffd700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2581460c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd25806b3a0 tx=0x7fd240017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24001bc40 con 0x7fd2580c7170 2023-09-27T23:29:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24001bdc0 con 0x7fd2580c7170 2023-09-27T23:29:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2580b0040 con 0x7fd2580c7170 2023-09-27T23:29:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240011670 con 0x7fd2580c7170 2023-09-27T23:29:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2580b0600 con 0x7fd2580c7170 2023-09-27T23:29:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.480+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25814fe00 con 0x7fd2580c7170 2023-09-27T23:29:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.484+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd240011a20 con 0x7fd2580c7170 2023-09-27T23:29:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.484+0000 7fd24f7fe700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd238060aa0 0x7fd238062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.484+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fd24007e5a0 con 0x7fd2580c7170 2023-09-27T23:29:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.484+0000 7fd2577fe700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd238060aa0 0x7fd238062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.484+0000 7fd2577fe700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd238060aa0 0x7fd238062f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd248007130 tx=0x7fd248007eb0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:19.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.492+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd25814fe00 con 0x7fd2580c7170 2023-09-27T23:29:19.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:29:19.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:29:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.684+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd25814fe00 con 0x7fd2580c7170 2023-09-27T23:29:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.696+0000 7fd24f7fe700 1 -- 172.21.15.64:0/1386893618 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v25) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd24004e630 con 0x7fd2580c7170 2023-09-27T23:29:19.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd238060aa0 msgr2=0x7fd238062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd238060aa0 0x7fd238062f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd248007130 tx=0x7fd248007eb0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 msgr2=0x7fd2581460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2581460c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd25806b3a0 tx=0x7fd240017000 comp rx=0 tx=0).stop 2023-09-27T23:29:19.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 shutdown_connections 2023-09-27T23:29:19.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd238060aa0 0x7fd238062f50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd258150200 0x7fd2580af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2580c7170 0x7fd2581460c0 secure :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd25806b3a0 tx=0x7fd240017000 comp rx=0 tx=0).stop 2023-09-27T23:29:19.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 --2- 172.21.15.64:0/1386893618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2580acc60 0x7fd2580c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.704+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 >> 172.21.15.64:0/1386893618 conn(0x7fd2580ccb30 msgr2=0x7fd25814bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:19.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.712+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 shutdown_connections 2023-09-27T23:29:19.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.712+0000 7fd25cc26700 1 -- 172.21.15.64:0/1386893618 wait complete. 2023-09-27T23:29:19.722 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 10 and data pool 11 2023-09-27T23:29:19.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 Processor -- start 2023-09-27T23:29:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 -- start start 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b40cad10 0x7fa5b40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa5b406b750 0x7fa5b406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b406b2c0 con 0x7fa5b406df20 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b406a6b0 con 0x7fa5b406b750 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b8ed3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b406a990 con 0x7fa5b40cad10 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa5b406df20 0x7fa5b415bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51110/0 (socket says 172.21.15.64:51110) 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b3fff700 1 -- 172.21.15.64:0/1654414825 learned_addr learned my addr 172.21.15.64:0/1654414825 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:19.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1654414825 >> v2:172.21.15.64:3300/0 conn(0x7fa5b406b750 0x7fa5b406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b37fe700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b40cad10 0x7fa5b40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b2ffd700 1 -- 172.21.15.64:0/1654414825 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936995646 0 0) 0x7fa5b406b2c0 con 0x7fa5b406df20 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b2ffd700 1 -- 172.21.15.64:0/1654414825 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a0003570 con 0x7fa5b406df20 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5aaffd700 1 -- 172.21.15.64:0/1654414825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b40cad10 msgr2=0x7fa5b40cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b40cad10 0x7fa5b40cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5aaffd700 1 -- 172.21.15.64:0/1654414825 >> v1:172.21.15.64:6789/0 conn(0x7fa5b406df20 legacy=0x7fa5b415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b2ffd700 1 -- 172.21.15.64:0/1654414825 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781265973 0 0) 0x7fa5a0003570 con 0x7fa5b406df20 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5aaffd700 1 -- 172.21.15.64:0/1654414825 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b406ac70 con 0x7fa5b406b750 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b37fe700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b40cad10 0x7fa5b40cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406b750 0x7fa5b406d9e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0098e0 tx=0x7fa59c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfedb7073f62e2cb server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b2ffd700 1 -- 172.21.15.64:0/1654414825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c011630 con 0x7fa5b406b750 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b2ffd700 1 -- 172.21.15.64:0/1654414825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c011790 con 0x7fa5b406b750 2023-09-27T23:29:19.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.832+0000 7fa5b2ffd700 1 -- 172.21.15.64:0/1654414825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c011b30 con 0x7fa5b406b750 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1654414825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406b750 msgr2=0x7fa5b406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406b750 0x7fa5b406d9e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0098e0 tx=0x7fa59c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1654414825 shutdown_connections 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406b750 0x7fa5b406d9e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1654414825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b40cad10 0x7fa5b40cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1654414825 >> 172.21.15.64:0/1654414825 conn(0x7fa5b4067bb0 msgr2=0x7fa5b40cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1654414825 shutdown_connections 2023-09-27T23:29:19.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1654414825 wait complete. 2023-09-27T23:29:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 Processor -- start 2023-09-27T23:29:19.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.836+0000 7fa5b8ed3700 1 -- start start 2023-09-27T23:29:19.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 0x7fa5b40b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b40cad10 0x7fa5b4169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b414f880 0x7fa5b4152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40b1980 con 0x7fa5b406df20 2023-09-27T23:29:19.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4169680 con 0x7fa5b40cad10 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4158560 con 0x7fa5b414f880 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 0x7fa5b40b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 0x7fa5b40b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39920/0 (socket says 172.21.15.64:39920) 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 -- 172.21.15.64:0/1286615996 learned_addr learned my addr 172.21.15.64:0/1286615996 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b40cad10 0x7fa5b4169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b3fff700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b414f880 0x7fa5b4152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 -- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b414f880 msgr2=0x7fa5b4152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b414f880 0x7fa5b4152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 -- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b40cad10 msgr2=0x7fa5b4169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b40cad10 0x7fa5b4169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:19.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 -- 172.21.15.64:0/1286615996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4009420 con 0x7fa5b406df20 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b40cad10 0x7fa5b4169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b37fe700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 0x7fa5b40b1bc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ec80 tx=0x7fa5a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400c9d0 con 0x7fa5b406df20 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a400cb30 con 0x7fa5b406df20 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c009420 con 0x7fa5b406df20 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400ceb0 con 0x7fa5b406df20 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.840+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b4169880 con 0x7fa5b406df20 2023-09-27T23:29:19.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.844+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b404be40 con 0x7fa5b406df20 2023-09-27T23:29:19.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.844+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa5a4013680 con 0x7fa5b406df20 2023-09-27T23:29:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.844+0000 7fa5abfff700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa594060aa0 0x7fa594062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.844+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa5a407d510 con 0x7fa5b406df20 2023-09-27T23:29:19.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.844+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa594060aa0 0x7fa594062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:19.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.844+0000 7fa5aaffd700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa594060aa0 0x7fa594062f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4070b00 tx=0x7fa59c017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:19.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:19.848+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa5a404d4a0 con 0x7fa5b406df20 2023-09-27T23:29:20.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.012+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5b406af50 con 0x7fa5b406df20 2023-09-27T23:29:20.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5abfff700 1 -- 172.21.15.64:0/1286615996 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+17087 (secure 0 0 0) 0x7fa5a4050ac0 con 0x7fa5b406df20 2023-09-27T23:29:20.021 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:20.022 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":86,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:19.692162+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:18.249013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:18.681348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa594060aa0 msgr2=0x7fa594062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa594060aa0 0x7fa594062f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4070b00 tx=0x7fa59c017000 comp rx=0 tx=0).stop 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 msgr2=0x7fa5b40b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 0x7fa5b40b1bc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ec80 tx=0x7fa5a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 shutdown_connections 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa594060aa0 0x7fa594062f50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5b414f880 0x7fa5b4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5b40cad10 0x7fa5b4169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 --2- 172.21.15.64:0/1286615996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5b406df20 0x7fa5b40b1bc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 >> 172.21.15.64:0/1286615996 conn(0x7fa5b4067bb0 msgr2=0x7fa5b40cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:20.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.016+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 shutdown_connections 2023-09-27T23:29:20.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.020+0000 7fa5b8ed3700 1 -- 172.21.15.64:0/1286615996 wait complete. 2023-09-27T23:29:20.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:20.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 Processor -- start 2023-09-27T23:29:20.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 -- start start 2023-09-27T23:29:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f445c0b1410 0x7f445c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c06c7c0 con 0x7f445c06a580 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c06cd80 con 0x7f445c0b1410 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f4460903700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c06b8f0 con 0x7f445c06aa00 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f445c0b1410 0x7f445c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445affd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f445c06a580 0x7f445c15a860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:35540/0 (socket says 172.21.15.64:35540) 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445affd700 1 -- 172.21.15.64:0/1569786178 learned_addr learned my addr 172.21.15.64:0/1569786178 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f445c0b1410 0x7f445c0ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39922/0 (socket says 172.21.15.64:39922) 2023-09-27T23:29:20.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445bfff700 1 --2- 172.21.15.64:0/1569786178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 -- 172.21.15.64:0/1569786178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 msgr2=0x7f445c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 --2- 172.21.15.64:0/1569786178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445a7fc700 1 -- 172.21.15.64:0/1569786178 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963870148 0 0) 0x7f445c06c7c0 con 0x7f445c06a580 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 -- 172.21.15.64:0/1569786178 >> v1:172.21.15.123:6789/0 conn(0x7f445c06a580 legacy=0x7f445c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 -- 172.21.15.64:0/1569786178 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0ac1e0 con 0x7f445c0b1410 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.140+0000 7f445b7fe700 1 --2- 172.21.15.64:0/1569786178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0ab440 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f444c0098e0 tx=0x7f444c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acc655056efe519 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445a7fc700 1 -- 172.21.15.64:0/1569786178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00b640 con 0x7f445c0b1410 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445a7fc700 1 -- 172.21.15.64:0/1569786178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c00b7a0 con 0x7f445c0b1410 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445a7fc700 1 -- 172.21.15.64:0/1569786178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00bb40 con 0x7f445c0b1410 2023-09-27T23:29:20.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- 172.21.15.64:0/1569786178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 msgr2=0x7f445c0ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 --2- 172.21.15.64:0/1569786178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0ab440 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f444c0098e0 tx=0x7f444c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- 172.21.15.64:0/1569786178 shutdown_connections 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 --2- 172.21.15.64:0/1569786178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 --2- 172.21.15.64:0/1569786178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0ab440 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- 172.21.15.64:0/1569786178 >> 172.21.15.64:0/1569786178 conn(0x7f445c067bb0 msgr2=0x7f445c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- 172.21.15.64:0/1569786178 shutdown_connections 2023-09-27T23:29:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- 172.21.15.64:0/1569786178 wait complete. 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 Processor -- start 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- start start 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c06a580 0x7f445c0c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c16d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c4740 con 0x7f445c0b1410 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c1930 con 0x7f445c06a580 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f4460903700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c21d0 con 0x7f445c06aa00 2023-09-27T23:29:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39928/0 (socket says 172.21.15.64:39928) 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 -- 172.21.15.64:0/1107591594 learned_addr learned my addr 172.21.15.64:0/1107591594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445affd700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c16d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445b7fe700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c06a580 0x7f445c0c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 -- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 msgr2=0x7f445c16d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c16d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 -- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c06a580 msgr2=0x7f445c0c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c06a580 0x7f445c0c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 -- 172.21.15.64:0/1107591594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009420 con 0x7f445c0b1410 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f445bfff700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0b0470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4454009ec0 tx=0x7f445400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400cc20 con 0x7f445c0b1410 2023-09-27T23:29:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.144+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445400f040 con 0x7f445c0b1410 2023-09-27T23:29:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c0c6ec0 con 0x7f445c0b1410 2023-09-27T23:29:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454014480 con 0x7f445c0b1410 2023-09-27T23:29:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c0c1e60 con 0x7f445c0b1410 2023-09-27T23:29:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c0c2360 con 0x7f445c0b1410 2023-09-27T23:29:20.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4454014830 con 0x7f445c0b1410 2023-09-27T23:29:20.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f44537fe700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4440060aa0 0x7f4440062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f445407e680 con 0x7f445c0b1410 2023-09-27T23:29:20.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.148+0000 7f445b7fe700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4440060aa0 0x7f4440062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.152+0000 7f445b7fe700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4440060aa0 0x7f4440062f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f444c0091e0 tx=0x7f444c017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.156+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f44540520b0 con 0x7f445c0b1410 2023-09-27T23:29:20.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.320+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f445c04be40 con 0x7f445c0b1410 2023-09-27T23:29:20.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f44537fe700 1 -- 172.21.15.64:0/1107591594 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f4454018090 con 0x7f445c0b1410 2023-09-27T23:29:20.328 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:20.328 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":86,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:19.692162+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:18.249013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:18.681348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4440060aa0 msgr2=0x7f4440062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4440060aa0 0x7f4440062f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f444c0091e0 tx=0x7f444c017000 comp rx=0 tx=0).stop 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 msgr2=0x7f445c0b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0b0470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4454009ec0 tx=0x7f445400b040 comp rx=0 tx=0).stop 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 shutdown_connections 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4440060aa0 0x7f4440062f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f445c0b1410 0x7f445c0b0470 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f445c06aa00 0x7f445c16d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 --2- 172.21.15.64:0/1107591594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c06a580 0x7f445c0c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 >> 172.21.15.64:0/1107591594 conn(0x7f445c067bb0 msgr2=0x7f445c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:20.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 shutdown_connections 2023-09-27T23:29:20.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.324+0000 7f4460903700 1 -- 172.21.15.64:0/1107591594 wait complete. 2023-09-27T23:29:20.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:29:20.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 Processor -- start 2023-09-27T23:29:20.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 -- start start 2023-09-27T23:29:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbe413f7e0 0x7fdbe41564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdbe40cba00 0x7fdbe40cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe40cc3a0 con 0x7fdbe413fbb0 2023-09-27T23:29:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4157950 con 0x7fdbe413f7e0 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe9e58700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4156af0 con 0x7fdbe40cba00 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbe413f7e0 0x7fdbe41564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbe413f7e0 0x7fdbe41564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39938/0 (socket says 172.21.15.64:39938) 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 -- 172.21.15.64:0/2076274720 learned_addr learned my addr 172.21.15.64:0/2076274720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe3fff700 1 --2- 172.21.15.64:0/2076274720 >> v2:172.21.15.123:3300/0 conn(0x7fdbe40cba00 0x7fdbe40cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 -- 172.21.15.64:0/2076274720 >> v2:172.21.15.123:3300/0 conn(0x7fdbe40cba00 msgr2=0x7fdbe40cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 --2- 172.21.15.64:0/2076274720 >> v2:172.21.15.123:3300/0 conn(0x7fdbe40cba00 0x7fdbe40cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 -- 172.21.15.64:0/2076274720 >> v1:172.21.15.123:6789/0 conn(0x7fdbe413fbb0 legacy=0x7fdbe40af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.452+0000 7fdbe37fe700 1 -- 172.21.15.64:0/2076274720 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4156dd0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe37fe700 1 --2- 172.21.15.64:0/2076274720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe41564a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40c9840 tx=0x7fdbd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81224095db2c5c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe27fc700 1 -- 172.21.15.64:0/2076274720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400e070 con 0x7fdbe413f7e0 2023-09-27T23:29:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe27fc700 1 -- 172.21.15.64:0/2076274720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd400bae0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe27fc700 1 -- 172.21.15.64:0/2076274720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40092e0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2076274720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 msgr2=0x7fdbe41564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2076274720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe41564a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40c9840 tx=0x7fdbd4009840 comp rx=0 tx=0).stop 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2076274720 shutdown_connections 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2076274720 >> v2:172.21.15.123:3300/0 conn(0x7fdbe40cba00 0x7fdbe40cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2076274720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe41564a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2076274720 >> 172.21.15.64:0/2076274720 conn(0x7fdbe40cd8f0 msgr2=0x7fdbe40c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2076274720 shutdown_connections 2023-09-27T23:29:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.456+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2076274720 wait complete. 2023-09-27T23:29:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 Processor -- start 2023-09-27T23:29:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 -- start start 2023-09-27T23:29:20.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbe40cba00 0x7fdbe4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe413aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbe413fbb0 0x7fdbe40c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe413afe0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40c8250 con 0x7fdbe40cba00 2023-09-27T23:29:20.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40c8530 con 0x7fdbe413fbb0 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe413aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbe40cba00 0x7fdbe4069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbe40cba00 0x7fdbe4069430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:56950/0 (socket says 172.21.15.64:56950) 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe413aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39952/0 (socket says 172.21.15.64:39952) 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe37fe700 1 -- 172.21.15.64:0/2657232847 learned_addr learned my addr 172.21.15.64:0/2657232847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe3fff700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbe413fbb0 0x7fdbe40c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 -- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbe413fbb0 msgr2=0x7fdbe40c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbe413fbb0 0x7fdbe40c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 -- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbe40cba00 msgr2=0x7fdbe4069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbe40cba00 0x7fdbe4069430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 -- 172.21.15.64:0/2657232847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4009ae0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe2ffd700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe413aaa0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c490 tx=0x7fdbcc007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00ce40 con 0x7fdbe413f7e0 2023-09-27T23:29:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc011590 con 0x7fdbe413f7e0 2023-09-27T23:29:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc011950 con 0x7fdbe413f7e0 2023-09-27T23:29:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.460+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe40c8810 con 0x7fdbe413f7e0 2023-09-27T23:29:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.464+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe40c7f20 con 0x7fdbe413f7e0 2023-09-27T23:29:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.464+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe4071120 con 0x7fdbe413f7e0 2023-09-27T23:29:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.464+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fdbcc011cc0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.464+0000 7fdbdb7fe700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdbc4060aa0 0x7fdbc4062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.464+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdbcc07c280 con 0x7fdbe413f7e0 2023-09-27T23:29:20.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.468+0000 7fdbe37fe700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdbc4060aa0 0x7fdbc4062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.468+0000 7fdbe37fe700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdbc4060aa0 0x7fdbc4062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4003440 tx=0x7fdbd4018000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.476+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdbcc04c290 con 0x7fdbe413f7e0 2023-09-27T23:29:20.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.664+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdbe41570b0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.708+0000 7fdbdb7fe700 1 -- 172.21.15.64:0/2657232847 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdbcc04f8b0 con 0x7fdbe413f7e0 2023-09-27T23:29:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdbc4060aa0 msgr2=0x7fdbc4062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdbc4060aa0 0x7fdbc4062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4003440 tx=0x7fdbd4018000 comp rx=0 tx=0).stop 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 msgr2=0x7fdbe413aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe413aaa0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c490 tx=0x7fdbcc007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 shutdown_connections 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdbc4060aa0 0x7fdbc4062f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbe413fbb0 0x7fdbe40c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe413f7e0 0x7fdbe413aaa0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 --2- 172.21.15.64:0/2657232847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbe40cba00 0x7fdbe4069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 >> 172.21.15.64:0/2657232847 conn(0x7fdbe40cd8f0 msgr2=0x7fdbe413dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 shutdown_connections 2023-09-27T23:29:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.712+0000 7fdbe9e58700 1 -- 172.21.15.64:0/2657232847 wait complete. 2023-09-27T23:29:20.732 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:29:20.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:29:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 Processor -- start 2023-09-27T23:29:20.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 -- start start 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6444154780 0x7f644415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64440adab0 0x7f64440adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644415ae30 con 0x7f644406baa0 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64440abf70 con 0x7f64440adab0 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a8ab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444154b50 con 0x7f6444154780 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a0aa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64440adab0 0x7f64440adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a0aa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64440adab0 0x7f64440adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39958/0 (socket says 172.21.15.64:39958) 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f64490a8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f644406baa0 0x7f6444157b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51116/0 (socket says 172.21.15.64:51116) 2023-09-27T23:29:20.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.840+0000 7f644a0aa700 1 -- 172.21.15.64:0/1147712565 learned_addr learned my addr 172.21.15.64:0/1147712565 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:20.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f64498a9700 1 --2- 172.21.15.64:0/1147712565 >> v2:172.21.15.123:3300/0 conn(0x7f6444154780 0x7f644415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a0aa700 1 -- 172.21.15.64:0/1147712565 >> v2:172.21.15.123:3300/0 conn(0x7f6444154780 msgr2=0x7f644415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a0aa700 1 --2- 172.21.15.64:0/1147712565 >> v2:172.21.15.123:3300/0 conn(0x7f6444154780 0x7f644415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a0aa700 1 -- 172.21.15.64:0/1147712565 >> v1:172.21.15.64:6789/0 conn(0x7f644406baa0 legacy=0x7f6444157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a0aa700 1 -- 172.21.15.64:0/1147712565 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440ac210 con 0x7f64440adab0 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a0aa700 1 --2- 172.21.15.64:0/1147712565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f64440adeb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f644406a3f0 tx=0x7f643c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4db092cfd06086c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f64488a7700 1 -- 172.21.15.64:0/1147712565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00e070 con 0x7f64440adab0 2023-09-27T23:29:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f64488a7700 1 -- 172.21.15.64:0/1147712565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00ba50 con 0x7f64440adab0 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f64488a7700 1 -- 172.21.15.64:0/1147712565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c0092e0 con 0x7f64440adab0 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 -- 172.21.15.64:0/1147712565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 msgr2=0x7f64440adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1147712565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f64440adeb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f644406a3f0 tx=0x7f643c009840 comp rx=0 tx=0).stop 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 -- 172.21.15.64:0/1147712565 shutdown_connections 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1147712565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f64440adeb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1147712565 >> v2:172.21.15.123:3300/0 conn(0x7f6444154780 0x7f644415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 -- 172.21.15.64:0/1147712565 >> 172.21.15.64:0/1147712565 conn(0x7f6444067b50 msgr2=0x7f64440cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 -- 172.21.15.64:0/1147712565 shutdown_connections 2023-09-27T23:29:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 -- 172.21.15.64:0/1147712565 wait complete. 2023-09-27T23:29:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 Processor -- start 2023-09-27T23:29:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.844+0000 7f644a8ab700 1 -- start start 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f644406baa0 0x7f64440c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f6444153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6444154780 0x7f6444153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64440b0060 con 0x7f64440adab0 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64440c5540 con 0x7f6444154780 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6444167ff0 con 0x7f644406baa0 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f6444153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f6444153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39970/0 (socket says 172.21.15.64:39970) 2023-09-27T23:29:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 -- 172.21.15.64:0/1473359337 learned_addr learned my addr 172.21.15.64:0/1473359337 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64498a9700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f644406baa0 0x7f64440c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a0aa700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6444154780 0x7f6444153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 -- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f644406baa0 msgr2=0x7f64440c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f644406baa0 0x7f64440c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 -- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6444154780 msgr2=0x7f6444153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6444154780 0x7f6444153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:20.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 -- 172.21.15.64:0/1473359337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c009ae0 con 0x7f64440adab0 2023-09-27T23:29:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f64490a8700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f6444153410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6434001eb0 tx=0x7f6434007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434005a70 con 0x7f64440adab0 2023-09-27T23:29:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434005bd0 con 0x7f64440adab0 2023-09-27T23:29:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64440af730 con 0x7f64440adab0 2023-09-27T23:29:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340070d0 con 0x7f64440adab0 2023-09-27T23:29:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.848+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64440afcf0 con 0x7f64440adab0 2023-09-27T23:29:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.852+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644404be40 con 0x7f64440adab0 2023-09-27T23:29:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.852+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6434007480 con 0x7f64440adab0 2023-09-27T23:29:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.852+0000 7f6439ffb700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060aa0 0x7f6428062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.852+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f643407c4d0 con 0x7f64440adab0 2023-09-27T23:29:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.852+0000 7f64498a9700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060aa0 0x7f6428062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.852+0000 7f64498a9700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060aa0 0x7f6428062f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6440006fd0 tx=0x7f6440008040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:20.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:20.860+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f643404c500 con 0x7f64440adab0 2023-09-27T23:29:21.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.064+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f64440cc1a0 con 0x7f64440adab0 2023-09-27T23:29:21.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.712+0000 7f6439ffb700 1 -- 172.21.15.64:0/1473359337 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7f643404fb20 con 0x7f64440adab0 2023-09-27T23:29:21.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060aa0 msgr2=0x7f6428062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:21.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060aa0 0x7f6428062f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6440006fd0 tx=0x7f6440008040 comp rx=0 tx=0).stop 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 msgr2=0x7f6444153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f6444153410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6434001eb0 tx=0x7f6434007f40 comp rx=0 tx=0).stop 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 shutdown_connections 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060aa0 0x7f6428062f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6444154780 0x7f6444153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64440adab0 0x7f6444153410 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 --2- 172.21.15.64:0/1473359337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f644406baa0 0x7f64440c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 >> 172.21.15.64:0/1473359337 conn(0x7f6444067b50 msgr2=0x7f64440c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 shutdown_connections 2023-09-27T23:29:21.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.716+0000 7f644a8ab700 1 -- 172.21.15.64:0/1473359337 wait complete. 2023-09-27T23:29:21.738 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:29:21.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:21.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 Processor -- start 2023-09-27T23:29:21.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- start start 2023-09-27T23:29:21.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47e006de20 0x7f47e00c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:21.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006c7c0 0x7f47e00c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:21.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e006d990 con 0x7f47e0155960 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e006cd80 con 0x7f47e006de20 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e006d060 con 0x7f47e006c7c0 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47dffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006c7c0 0x7f47e00c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47e006de20 0x7f47e00c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47e006de20 0x7f47e00c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:56978/0 (socket says 172.21.15.64:56978) 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 -- 172.21.15.64:0/1860171650 learned_addr learned my addr 172.21.15.64:0/1860171650 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:21.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47df7fe700 1 --1- 172.21.15.64:0/1860171650 >> v1:172.21.15.123:6789/0 conn(0x7f47e0155960 0x7f47e015a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:35572/0 (socket says 172.21.15.64:35572) 2023-09-27T23:29:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 -- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006c7c0 msgr2=0x7f47e00c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 --2- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006c7c0 0x7f47e00c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 -- 172.21.15.64:0/1860171650 >> v1:172.21.15.123:6789/0 conn(0x7f47e0155960 legacy=0x7f47e015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:21.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 -- 172.21.15.64:0/1860171650 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e006d340 con 0x7f47e006de20 2023-09-27T23:29:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47deffd700 1 --2- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006de20 0x7f47e00c77f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f47c80099b0 tx=0x7f47c8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a4b4f3d238aaae server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47de7fc700 1 -- 172.21.15.64:0/1860171650 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8009ea0 con 0x7f47e006de20 2023-09-27T23:29:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47de7fc700 1 -- 172.21.15.64:0/1860171650 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800b860 con 0x7f47e006de20 2023-09-27T23:29:21.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47de7fc700 1 -- 172.21.15.64:0/1860171650 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800bc00 con 0x7f47e006de20 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006de20 msgr2=0x7f47e00c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006de20 0x7f47e00c77f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f47c80099b0 tx=0x7f47c8009250 comp rx=0 tx=0).stop 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1860171650 shutdown_connections 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006c7c0 0x7f47e00c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1860171650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006de20 0x7f47e00c77f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.852+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1860171650 >> 172.21.15.64:0/1860171650 conn(0x7f47e0067bb0 msgr2=0x7f47e00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1860171650 shutdown_connections 2023-09-27T23:29:21.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1860171650 wait complete. 2023-09-27T23:29:21.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 Processor -- start 2023-09-27T23:29:21.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 -- start start 2023-09-27T23:29:21.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006c7c0 0x7f47e013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:21.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006de20 0x7f47e00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:21.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.856+0000 7f47e4e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 0x7f47e00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:21.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47e4e5f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e013f980 con 0x7f47e0155960 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47e4e5f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0167cc0 con 0x7f47e006c7c0 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47e4e5f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0166e40 con 0x7f47e006de20 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 0x7f47e00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 0x7f47e00caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39982/0 (socket says 172.21.15.64:39982) 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 -- 172.21.15.64:0/1437741630 learned_addr learned my addr 172.21.15.64:0/1437741630 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47deffd700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006de20 0x7f47e00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47df7fe700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006c7c0 0x7f47e013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 -- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006de20 msgr2=0x7f47e00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006de20 0x7f47e00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 -- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006c7c0 msgr2=0x7f47e013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006c7c0 0x7f47e013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 -- 172.21.15.64:0/1437741630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80094f0 con 0x7f47e0155960 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47deffd700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006de20 0x7f47e00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47dffff700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 0x7f47e00caa30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009ec0 tx=0x7f47d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d800cb10 con 0x7f47e0155960 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d800f040 con 0x7f47e0155960 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e01670b0 con 0x7f47e0155960 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d800cee0 con 0x7f47e0155960 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e0167670 con 0x7f47e0155960 2023-09-27T23:29:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.860+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e0167e70 con 0x7f47e0155960 2023-09-27T23:29:21.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.864+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f47d8014690 con 0x7f47e0155960 2023-09-27T23:29:21.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.864+0000 7f47d77fe700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c0060a70 0x7f47c0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:21.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.864+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f47d807db90 con 0x7f47e0155960 2023-09-27T23:29:21.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.864+0000 7f47df7fe700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c0060a70 0x7f47c0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:21.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.864+0000 7f47df7fe700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c0060a70 0x7f47c0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47d0007130 tx=0x7f47d0000bc0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:21.872+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f47e0167e70 con 0x7f47e0155960 2023-09-27T23:29:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.068+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47e0167e70 con 0x7f47e0155960 2023-09-27T23:29:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.068+0000 7f47d77fe700 1 -- 172.21.15.64:0/1437741630 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f47e0167e70 con 0x7f47e0155960 2023-09-27T23:29:22.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:22.075 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:19.692498+0000","modified":"2023-09-27T23:29:21.709311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6610":{"gid":6610,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2730764960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2730764960},{"type":"v1","addr":"172.21.15.123:6837","nonce":2730764960}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c0060a70 msgr2=0x7f47c0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c0060a70 0x7f47c0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47d0007130 tx=0x7f47d0000bc0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 msgr2=0x7f47e00caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 0x7f47e00caa30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009ec0 tx=0x7f47d800b040 comp rx=0 tx=0).stop 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 shutdown_connections 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f47c0060a70 0x7f47c0062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47e0155960 0x7f47e00caa30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47e006de20 0x7f47e00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 --2- 172.21.15.64:0/1437741630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e006c7c0 0x7f47e013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 >> 172.21.15.64:0/1437741630 conn(0x7f47e0067bb0 msgr2=0x7f47e014f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 shutdown_connections 2023-09-27T23:29:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.072+0000 7f47e4e5f700 1 -- 172.21.15.64:0/1437741630 wait complete. 2023-09-27T23:29:22.078 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-27T23:29:22.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:22.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 Processor -- start 2023-09-27T23:29:22.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- start start 2023-09-27T23:29:22.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0ab910 0x7f28dc159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28dc0670e0 0x7f28dc0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc0c1910 con 0x7f28dc0cd430 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc0ac220 con 0x7f28dc0670e0 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc159cf0 con 0x7f28dc0ab910 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28dc0670e0 0x7f28dc0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28dc0670e0 0x7f28dc0abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39994/0 (socket says 172.21.15.64:39994) 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 -- 172.21.15.64:0/2448914758 learned_addr learned my addr 172.21.15.64:0/2448914758 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28dbfff700 1 --2- 172.21.15.64:0/2448914758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0ab910 0x7f28dc159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 -- 172.21.15.64:0/2448914758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0ab910 msgr2=0x7f28dc159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:22.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 --2- 172.21.15.64:0/2448914758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0ab910 0x7f28dc159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 -- 172.21.15.64:0/2448914758 >> v1:172.21.15.123:6789/0 conn(0x7f28dc0cd430 legacy=0x7f28dc0c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:22.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 -- 172.21.15.64:0/2448914758 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc159fd0 con 0x7f28dc0670e0 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28db7fe700 1 --2- 172.21.15.64:0/2448914758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc0abce0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00bc50 tx=0x7f28cc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaebff26ac05a90 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28daffd700 1 -- 172.21.15.64:0/2448914758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00be60 con 0x7f28dc0670e0 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28daffd700 1 -- 172.21.15.64:0/2448914758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc00ddc0 con 0x7f28dc0670e0 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28daffd700 1 -- 172.21.15.64:0/2448914758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0106a0 con 0x7f28dc0670e0 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- 172.21.15.64:0/2448914758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 msgr2=0x7f28dc0abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 --2- 172.21.15.64:0/2448914758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc0abce0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00bc50 tx=0x7f28cc00b4f0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- 172.21.15.64:0/2448914758 shutdown_connections 2023-09-27T23:29:22.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 --2- 172.21.15.64:0/2448914758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc0abce0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 --2- 172.21.15.64:0/2448914758 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0ab910 0x7f28dc159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- 172.21.15.64:0/2448914758 >> 172.21.15.64:0/2448914758 conn(0x7f28dc06c080 msgr2=0x7f28dc0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- 172.21.15.64:0/2448914758 shutdown_connections 2023-09-27T23:29:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.192+0000 7f28e17b6700 1 -- 172.21.15.64:0/2448914758 wait complete. 2023-09-27T23:29:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 Processor -- start 2023-09-27T23:29:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 -- start start 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc13e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc0ab910 0x7f28dc0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0cd430 0x7f28dc168680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc14dcb0 con 0x7f28dc0670e0 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc169220 con 0x7f28dc0ab910 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e17b6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0cb4f0 con 0x7f28dc0cd430 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc13e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc13e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40010/0 (socket says 172.21.15.64:40010) 2023-09-27T23:29:22.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28dbfff700 1 -- 172.21.15.64:0/3178159535 learned_addr learned my addr 172.21.15.64:0/3178159535 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28db7fe700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc0ab910 0x7f28dc0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.196+0000 7f28e0fb5700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0cd430 0x7f28dc168680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28dbfff700 1 -- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0cd430 msgr2=0x7f28dc168680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28dbfff700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0cd430 0x7f28dc168680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28dbfff700 1 -- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc0ab910 msgr2=0x7f28dc0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28dbfff700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc0ab910 0x7f28dc0cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28dbfff700 1 -- 172.21.15.64:0/3178159535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc00b790 con 0x7f28dc0670e0 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28dbfff700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc13e490 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f28d000d860 tx=0x7f28d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00125f0 con 0x7f28dc0670e0 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d0012750 con 0x7f28dc0670e0 2023-09-27T23:29:22.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc0cb7d0 con 0x7f28dc0670e0 2023-09-27T23:29:22.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0012b10 con 0x7f28dc0670e0 2023-09-27T23:29:22.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc13e9d0 con 0x7f28dc0670e0 2023-09-27T23:29:22.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc1589e0 con 0x7f28dc0670e0 2023-09-27T23:29:22.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.200+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f28d001e460 con 0x7f28dc0670e0 2023-09-27T23:29:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.204+0000 7f28c3fff700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28c8060a70 0x7f28c8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.204+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f28d007d680 con 0x7f28dc0670e0 2023-09-27T23:29:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.204+0000 7f28db7fe700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28c8060a70 0x7f28c8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.204+0000 7f28db7fe700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28c8060a70 0x7f28c8062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00bc50 tx=0x7f28cc01a000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.212+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f28dc1589e0 con 0x7f28dc0670e0 2023-09-27T23:29:22.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.368+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28dc0ca380 con 0x7f28dc0670e0 2023-09-27T23:29:22.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28c3fff700 1 -- 172.21.15.64:0/3178159535 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f28dc0ca380 con 0x7f28dc0670e0 2023-09-27T23:29:22.377 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:22.377 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":87,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:20.695370+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:18.249013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:18.681348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28c8060a70 msgr2=0x7f28c8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28c8060a70 0x7f28c8062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00bc50 tx=0x7f28cc01a000 comp rx=0 tx=0).stop 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 msgr2=0x7f28dc13e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc13e490 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f28d000d860 tx=0x7f28d0008040 comp rx=0 tx=0).stop 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 shutdown_connections 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f28c8060a70 0x7f28c8062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28dc0cd430 0x7f28dc168680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc0ab910 0x7f28dc0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 --2- 172.21.15.64:0/3178159535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28dc0670e0 0x7f28dc13e490 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 >> 172.21.15.64:0/3178159535 conn(0x7f28dc06c080 msgr2=0x7f28dc0c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 shutdown_connections 2023-09-27T23:29:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.372+0000 7f28e17b6700 1 -- 172.21.15.64:0/3178159535 wait complete. 2023-09-27T23:29:22.391 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:29:22.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:22.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.492+0000 7f1cbe108700 1 Processor -- start 2023-09-27T23:29:22.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.492+0000 7f1cbe108700 1 -- start start 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cb80c6cd0 0x7f1cb806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb815cc20 0x7f1cb815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb806b470 con 0x7f1cb80c6900 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb806be40 con 0x7f1cb80c6cd0 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb806e110 con 0x7f1cb815cc20 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cb80c6cd0 0x7f1cb806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cb80c6cd0 0x7f1cb806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40024/0 (socket says 172.21.15.64:40024) 2023-09-27T23:29:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 -- 172.21.15.64:0/2753826971 learned_addr learned my addr 172.21.15.64:0/2753826971 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbd907700 1 --2- 172.21.15.64:0/2753826971 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb815cc20 0x7f1cb815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 -- 172.21.15.64:0/2753826971 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb815cc20 msgr2=0x7f1cb815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 --2- 172.21.15.64:0/2753826971 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb815cc20 0x7f1cb815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 -- 172.21.15.64:0/2753826971 >> v1:172.21.15.123:6789/0 conn(0x7f1cb80c6900 legacy=0x7f1cb806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 -- 172.21.15.64:0/2753826971 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8161d10 con 0x7f1cb80c6cd0 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbc905700 1 --2- 172.21.15.64:0/2753826971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6cd0 0x7f1cb806b900 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099b0 tx=0x7f1cb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80508199c9995208 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cabfff700 1 -- 172.21.15.64:0/2753826971 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4009ea0 con 0x7f1cb80c6cd0 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cabfff700 1 -- 172.21.15.64:0/2753826971 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb400b860 con 0x7f1cb80c6cd0 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cabfff700 1 -- 172.21.15.64:0/2753826971 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400bc00 con 0x7f1cb80c6cd0 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- 172.21.15.64:0/2753826971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6cd0 msgr2=0x7f1cb806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 --2- 172.21.15.64:0/2753826971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6cd0 0x7f1cb806b900 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099b0 tx=0x7f1cb4009250 comp rx=0 tx=0).stop 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- 172.21.15.64:0/2753826971 shutdown_connections 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 --2- 172.21.15.64:0/2753826971 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb815cc20 0x7f1cb815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 --2- 172.21.15.64:0/2753826971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6cd0 0x7f1cb806b900 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- 172.21.15.64:0/2753826971 >> 172.21.15.64:0/2753826971 conn(0x7f1cb8067bb0 msgr2=0x7f1cb80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- 172.21.15.64:0/2753826971 shutdown_connections 2023-09-27T23:29:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.496+0000 7f1cbe108700 1 -- 172.21.15.64:0/2753826971 wait complete. 2023-09-27T23:29:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 Processor -- start 2023-09-27T23:29:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 -- start start 2023-09-27T23:29:22.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 0x7f1cb81478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb80c6cd0 0x7f1cb80ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cb815cc20 0x7f1cb80caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb815edd0 con 0x7f1cb80c6900 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb81595b0 con 0x7f1cb815cc20 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbe108700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb815e1c0 con 0x7f1cb80c6cd0 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 0x7f1cb81478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbc905700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb80c6cd0 0x7f1cb80ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 0x7f1cb81478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40030/0 (socket says 172.21.15.64:40030) 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 -- 172.21.15.64:0/864555805 learned_addr learned my addr 172.21.15.64:0/864555805 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd907700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cb815cc20 0x7f1cb80caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 -- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb80c6cd0 msgr2=0x7f1cb80ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb80c6cd0 0x7f1cb80ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 -- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cb815cc20 msgr2=0x7f1cb80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cb815cc20 0x7f1cb80caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd106700 1 -- 172.21.15.64:0/864555805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb40094f0 con 0x7f1cb80c6900 2023-09-27T23:29:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.500+0000 7f1cbd907700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cb815cc20 0x7f1cb80caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1cbd106700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 0x7f1cb81478b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c590 tx=0x7f1cac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011590 con 0x7f1cb80c6900 2023-09-27T23:29:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac011710 con 0x7f1cb80c6900 2023-09-27T23:29:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb815e4a0 con 0x7f1cb80c6900 2023-09-27T23:29:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011ad0 con 0x7f1cb80c6900 2023-09-27T23:29:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb815ea60 con 0x7f1cb80c6900 2023-09-27T23:29:22.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb804be40 con 0x7f1cb80c6900 2023-09-27T23:29:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1cac01b3b0 con 0x7f1cb80c6900 2023-09-27T23:29:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1ca97fa700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1c9c060a70 0x7f1c9c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.504+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1cac07c630 con 0x7f1cb80c6900 2023-09-27T23:29:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.508+0000 7f1cbc905700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1c9c060a70 0x7f1c9c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.508+0000 7f1cbc905700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1c9c060a70 0x7f1c9c062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099b0 tx=0x7f1cb4017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.516+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1cac04c660 con 0x7f1cb80c6900 2023-09-27T23:29:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.676+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cb815ef10 con 0x7f1c9c060a70 2023-09-27T23:29:22.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.696+0000 7f1ca97fa700 1 -- 172.21.15.64:0/864555805 <== mgr.5816 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295373 (secure 0 0 0) 0x7f1cb400b9d0 con 0x7f1c9c060a70 2023-09-27T23:29:22.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:22.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1c9c060a70 msgr2=0x7f1c9c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1c9c060a70 0x7f1c9c062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40099b0 tx=0x7f1cb4017000 comp rx=0 tx=0).stop 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 msgr2=0x7f1cb81478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 0x7f1cb81478b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c590 tx=0x7f1cac00c8a0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 shutdown_connections 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1c9c060a70 0x7f1c9c062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cb815cc20 0x7f1cb80caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cb80c6cd0 0x7f1cb80ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 --2- 172.21.15.64:0/864555805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cb80c6900 0x7f1cb81478b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 >> 172.21.15.64:0/864555805 conn(0x7f1cb8067bb0 msgr2=0x7f1cb8160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:22.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 shutdown_connections 2023-09-27T23:29:22.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.700+0000 7f1cbe108700 1 -- 172.21.15.64:0/864555805 wait complete. 2023-09-27T23:29:22.708 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:22.720 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:22.633866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":248,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":199284,"kb_used_data":9012,"kb_used_omap":0,"kb_used_meta":190208,"kb_avail":754775436,"statfs":{"total":773094113280,"available":772890046464,"internally_reserved":0,"allocated":9228288,"data_stored":5264169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":194772992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.323067"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746666+0000","last_change":"2023-09-27T23:29:19.735623+0000","last_active":"2023-09-27T23:29:19.746666+0000","last_peered":"2023-09-27T23:29:19.746666+0000","last_clean":"2023-09-27T23:29:19.746666+0000","last_became_active":"2023-09-27T23:29:19.735239+0000","last_became_peered":"2023-09-27T23:29:19.735239+0000","last_unstale":"2023-09-27T23:29:19.746666+0000","last_undegraded":"2023-09-27T23:29:19.746666+0000","last_fullsized":"2023-09-27T23:29:19.746666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:50:25.868487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742588+0000","last_change":"2023-09-27T23:29:19.731017+0000","last_active":"2023-09-27T23:29:19.742588+0000","last_peered":"2023-09-27T23:29:19.742588+0000","last_clean":"2023-09-27T23:29:19.742588+0000","last_became_active":"2023-09-27T23:29:19.729277+0000","last_became_peered":"2023-09-27T23:29:19.729277+0000","last_unstale":"2023-09-27T23:29:19.742588+0000","last_undegraded":"2023-09-27T23:29:19.742588+0000","last_fullsized":"2023-09-27T23:29:19.742588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:43.879878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742363+0000","last_change":"2023-09-27T23:29:19.729793+0000","last_active":"2023-09-27T23:29:19.742363+0000","last_peered":"2023-09-27T23:29:19.742363+0000","last_clean":"2023-09-27T23:29:19.742363+0000","last_became_active":"2023-09-27T23:29:19.729183+0000","last_became_peered":"2023-09-27T23:29:19.729183+0000","last_unstale":"2023-09-27T23:29:19.742363+0000","last_undegraded":"2023-09-27T23:29:19.742363+0000","last_fullsized":"2023-09-27T23:29:19.742363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:09.106318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742724+0000","last_change":"2023-09-27T23:29:19.731251+0000","last_active":"2023-09-27T23:29:19.742724+0000","last_peered":"2023-09-27T23:29:19.742724+0000","last_clean":"2023-09-27T23:29:19.742724+0000","last_became_active":"2023-09-27T23:29:19.729904+0000","last_became_peered":"2023-09-27T23:29:19.729904+0000","last_unstale":"2023-09-27T23:29:19.742724+0000","last_undegraded":"2023-09-27T23:29:19.742724+0000","last_fullsized":"2023-09-27T23:29:19.742724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:41.029315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742755+0000","last_change":"2023-09-27T23:29:19.731464+0000","last_active":"2023-09-27T23:29:19.742755+0000","last_peered":"2023-09-27T23:29:19.742755+0000","last_clean":"2023-09-27T23:29:19.742755+0000","last_became_active":"2023-09-27T23:29:19.729102+0000","last_became_peered":"2023-09-27T23:29:19.729102+0000","last_unstale":"2023-09-27T23:29:19.742755+0000","last_undegraded":"2023-09-27T23:29:19.742755+0000","last_fullsized":"2023-09-27T23:29:19.742755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:30:18.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746307+0000","last_change":"2023-09-27T23:29:19.731873+0000","last_active":"2023-09-27T23:29:19.746307+0000","last_peered":"2023-09-27T23:29:19.746307+0000","last_clean":"2023-09-27T23:29:19.746307+0000","last_became_active":"2023-09-27T23:29:19.730797+0000","last_became_peered":"2023-09-27T23:29:19.730797+0000","last_unstale":"2023-09-27T23:29:19.746307+0000","last_undegraded":"2023-09-27T23:29:19.746307+0000","last_fullsized":"2023-09-27T23:29:19.746307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:24:01.984490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742203+0000","last_change":"2023-09-27T23:29:19.355491+0000","last_active":"2023-09-27T23:29:19.742203+0000","last_peered":"2023-09-27T23:29:19.742203+0000","last_clean":"2023-09-27T23:29:19.742203+0000","last_became_active":"2023-09-27T23:29:19.355192+0000","last_became_peered":"2023-09-27T23:29:19.355192+0000","last_unstale":"2023-09-27T23:29:19.742203+0000","last_undegraded":"2023-09-27T23:29:19.742203+0000","last_fullsized":"2023-09-27T23:29:19.742203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:54:30.443743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742614+0000","last_change":"2023-09-27T23:29:19.356245+0000","last_active":"2023-09-27T23:29:19.742614+0000","last_peered":"2023-09-27T23:29:19.742614+0000","last_clean":"2023-09-27T23:29:19.742614+0000","last_became_active":"2023-09-27T23:29:19.355967+0000","last_became_peered":"2023-09-27T23:29:19.355967+0000","last_unstale":"2023-09-27T23:29:19.742614+0000","last_undegraded":"2023-09-27T23:29:19.742614+0000","last_fullsized":"2023-09-27T23:29:19.742614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:19.665630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.719234+0000","last_change":"2023-09-27T23:29:19.352763+0000","last_active":"2023-09-27T23:29:19.719234+0000","last_peered":"2023-09-27T23:29:19.719234+0000","last_clean":"2023-09-27T23:29:19.719234+0000","last_became_active":"2023-09-27T23:29:19.352450+0000","last_became_peered":"2023-09-27T23:29:19.352450+0000","last_unstale":"2023-09-27T23:29:19.719234+0000","last_undegraded":"2023-09-27T23:29:19.719234+0000","last_fullsized":"2023-09-27T23:29:19.719234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:09.789659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746329+0000","last_change":"2023-09-27T23:29:19.734575+0000","last_active":"2023-09-27T23:29:19.746329+0000","last_peered":"2023-09-27T23:29:19.746329+0000","last_clean":"2023-09-27T23:29:19.746329+0000","last_became_active":"2023-09-27T23:29:19.730734+0000","last_became_peered":"2023-09-27T23:29:19.730734+0000","last_unstale":"2023-09-27T23:29:19.746329+0000","last_undegraded":"2023-09-27T23:29:19.746329+0000","last_fullsized":"2023-09-27T23:29:19.746329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:47.061803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716915+0000","last_change":"2023-09-27T23:29:19.351569+0000","last_active":"2023-09-27T23:29:19.716915+0000","last_peered":"2023-09-27T23:29:19.716915+0000","last_clean":"2023-09-27T23:29:19.716915+0000","last_became_active":"2023-09-27T23:29:19.351295+0000","last_became_peered":"2023-09-27T23:29:19.351295+0000","last_unstale":"2023-09-27T23:29:19.716915+0000","last_undegraded":"2023-09-27T23:29:19.716915+0000","last_fullsized":"2023-09-27T23:29:19.716915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:38.540430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742267+0000","last_change":"2023-09-27T23:29:19.359846+0000","last_active":"2023-09-27T23:29:19.742267+0000","last_peered":"2023-09-27T23:29:19.742267+0000","last_clean":"2023-09-27T23:29:19.742267+0000","last_became_active":"2023-09-27T23:29:19.359535+0000","last_became_peered":"2023-09-27T23:29:19.359535+0000","last_unstale":"2023-09-27T23:29:19.742267+0000","last_undegraded":"2023-09-27T23:29:19.742267+0000","last_fullsized":"2023-09-27T23:29:19.742267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:37.201350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742073+0000","last_change":"2023-09-27T23:29:19.729613+0000","last_active":"2023-09-27T23:29:19.742073+0000","last_peered":"2023-09-27T23:29:19.742073+0000","last_clean":"2023-09-27T23:29:19.742073+0000","last_became_active":"2023-09-27T23:29:19.729107+0000","last_became_peered":"2023-09-27T23:29:19.729107+0000","last_unstale":"2023-09-27T23:29:19.742073+0000","last_undegraded":"2023-09-27T23:29:19.742073+0000","last_fullsized":"2023-09-27T23:29:19.742073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:39:43.514982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742014+0000","last_change":"2023-09-27T23:29:19.729669+0000","last_active":"2023-09-27T23:29:19.742014+0000","last_peered":"2023-09-27T23:29:19.742014+0000","last_clean":"2023-09-27T23:29:19.742014+0000","last_became_active":"2023-09-27T23:29:19.729046+0000","last_became_peered":"2023-09-27T23:29:19.729046+0000","last_unstale":"2023-09-27T23:29:19.742014+0000","last_undegraded":"2023-09-27T23:29:19.742014+0000","last_fullsized":"2023-09-27T23:29:19.742014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:01.763686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716949+0000","last_change":"2023-09-27T23:29:19.355551+0000","last_active":"2023-09-27T23:29:19.716949+0000","last_peered":"2023-09-27T23:29:19.716949+0000","last_clean":"2023-09-27T23:29:19.716949+0000","last_became_active":"2023-09-27T23:29:19.355244+0000","last_became_peered":"2023-09-27T23:29:19.355244+0000","last_unstale":"2023-09-27T23:29:19.716949+0000","last_undegraded":"2023-09-27T23:29:19.716949+0000","last_fullsized":"2023-09-27T23:29:19.716949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:09.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742096+0000","last_change":"2023-09-27T23:29:19.355604+0000","last_active":"2023-09-27T23:29:19.742096+0000","last_peered":"2023-09-27T23:29:19.742096+0000","last_clean":"2023-09-27T23:29:19.742096+0000","last_became_active":"2023-09-27T23:29:19.355207+0000","last_became_peered":"2023-09-27T23:29:19.355207+0000","last_unstale":"2023-09-27T23:29:19.742096+0000","last_undegraded":"2023-09-27T23:29:19.742096+0000","last_fullsized":"2023-09-27T23:29:19.742096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:33.460035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'161","reported_seq":376,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.360221+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:15.360221+0000","last_peered":"2023-09-27T23:29:15.360221+0000","last_clean":"2023-09-27T23:29:15.360221+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:15.360221+0000","last_undegraded":"2023-09-27T23:29:15.360221+0000","last_fullsized":"2023-09-27T23:29:15.360221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":161,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358364+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:15.358364+0000","last_peered":"2023-09-27T23:29:15.358364+0000","last_clean":"2023-09-27T23:29:15.358364+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:15.358364+0000","last_undegraded":"2023-09-27T23:29:15.358364+0000","last_fullsized":"2023-09-27T23:29:15.358364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359761+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:15.359761+0000","last_peered":"2023-09-27T23:29:15.359761+0000","last_clean":"2023-09-27T23:29:15.359761+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:15.359761+0000","last_undegraded":"2023-09-27T23:29:15.359761+0000","last_fullsized":"2023-09-27T23:29:15.359761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.119882+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:15.119882+0000","last_peered":"2023-09-27T23:29:15.119882+0000","last_clean":"2023-09-27T23:29:15.119882+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:15.119882+0000","last_undegraded":"2023-09-27T23:29:15.119882+0000","last_fullsized":"2023-09-27T23:29:15.119882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.717030+0000","last_change":"2023-09-27T23:29:19.351439+0000","last_active":"2023-09-27T23:29:19.717030+0000","last_peered":"2023-09-27T23:29:19.717030+0000","last_clean":"2023-09-27T23:29:19.717030+0000","last_became_active":"2023-09-27T23:29:19.350793+0000","last_became_peered":"2023-09-27T23:29:19.350793+0000","last_unstale":"2023-09-27T23:29:19.717030+0000","last_undegraded":"2023-09-27T23:29:19.717030+0000","last_fullsized":"2023-09-27T23:29:19.717030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:44:07.068721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358360+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:15.358360+0000","last_peered":"2023-09-27T23:29:15.358360+0000","last_clean":"2023-09-27T23:29:15.358360+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:15.358360+0000","last_undegraded":"2023-09-27T23:29:15.358360+0000","last_fullsized":"2023-09-27T23:29:15.358360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355205+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:15.355205+0000","last_peered":"2023-09-27T23:29:15.355205+0000","last_clean":"2023-09-27T23:29:15.355205+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:15.355205+0000","last_undegraded":"2023-09-27T23:29:15.355205+0000","last_fullsized":"2023-09-27T23:29:15.355205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742337+0000","last_change":"2023-09-27T23:29:19.351252+0000","last_active":"2023-09-27T23:29:19.742337+0000","last_peered":"2023-09-27T23:29:19.742337+0000","last_clean":"2023-09-27T23:29:19.742337+0000","last_became_active":"2023-09-27T23:29:19.350970+0000","last_became_peered":"2023-09-27T23:29:19.350970+0000","last_unstale":"2023-09-27T23:29:19.742337+0000","last_undegraded":"2023-09-27T23:29:19.742337+0000","last_fullsized":"2023-09-27T23:29:19.742337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:02.449177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359957+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:15.359957+0000","last_peered":"2023-09-27T23:29:15.359957+0000","last_clean":"2023-09-27T23:29:15.359957+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:15.359957+0000","last_undegraded":"2023-09-27T23:29:15.359957+0000","last_fullsized":"2023-09-27T23:29:15.359957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355216+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:15.355216+0000","last_peered":"2023-09-27T23:29:15.355216+0000","last_clean":"2023-09-27T23:29:15.355216+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:15.355216+0000","last_undegraded":"2023-09-27T23:29:15.355216+0000","last_fullsized":"2023-09-27T23:29:15.355216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742021+0000","last_change":"2023-09-27T23:29:19.359807+0000","last_active":"2023-09-27T23:29:19.742021+0000","last_peered":"2023-09-27T23:29:19.742021+0000","last_clean":"2023-09-27T23:29:19.742021+0000","last_became_active":"2023-09-27T23:29:19.359497+0000","last_became_peered":"2023-09-27T23:29:19.359497+0000","last_unstale":"2023-09-27T23:29:19.742021+0000","last_undegraded":"2023-09-27T23:29:19.742021+0000","last_fullsized":"2023-09-27T23:29:19.742021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:12:13.225733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.714700+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:19.714700+0000","last_peered":"2023-09-27T23:29:19.714700+0000","last_clean":"2023-09-27T23:29:19.714700+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:19.714700+0000","last_undegraded":"2023-09-27T23:29:19.714700+0000","last_fullsized":"2023-09-27T23:29:19.714700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25500,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":23936,"kb_avail":94346340,"statfs":{"total":96636764160,"available":96610652160,"internally_reserved":0,"allocated":1593344,"data_stored":1098094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24510464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.749},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.878},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.052}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.016},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.121}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0820000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21264,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350576,"statfs":{"total":96636764160,"available":96614989824,"internally_reserved":0,"allocated":1056768,"data_stored":535629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":0.84099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.874}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.753}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":1.244},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.284},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.232}]}]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31192,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":30208,"kb_avail":94340648,"statfs":{"total":96636764160,"available":96604823552,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30932992},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":1.034},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.2330000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":0.747},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.113},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.081}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":0.94999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.196}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.262}]}]},{"osd":4,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21272,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20288,"kb_avail":94350568,"statfs":{"total":96636764160,"available":96614981632,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20774912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.091},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.89700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":1.3169999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":0.98399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.135}]}]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21848,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":94349992,"statfs":{"total":96636764160,"available":96614391808,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.86599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.238},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.357}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":0.999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.321}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.0620000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20312,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":19328,"kb_avail":94351528,"statfs":{"total":96636764160,"available":96615964672,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19791872},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":1.018}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":1.272},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.214},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.048}]}]},{"osd":1,"up_from":14,"seq":60129542336,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":34560,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35389440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.621},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":0.496}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":0.95299999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22352,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349488,"statfs":{"total":96636764160,"available":96613875712,"internally_reserved":0,"allocated":1581056,"data_stored":1091816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.62},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.435},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.022},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":0.80200000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.083}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.127},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:22.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:22.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7fad01d1d700 1 Processor -- start 2023-09-27T23:29:22.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7fad01d1d700 1 -- start start 2023-09-27T23:29:22.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7fad01d1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc15b0d0 0x7facfc0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7fad01d1d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facfc0cd1c0 con 0x7facfc0c9420 2023-09-27T23:29:22.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7fad01d1d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facfc0cdd60 con 0x7facfc15ad00 2023-09-27T23:29:22.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7fad01d1d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc15b520 con 0x7facfc15b0d0 2023-09-27T23:29:22.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfbfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7facfc0c9420 0x7facfc156a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51128/0 (socket says 172.21.15.64:51128) 2023-09-27T23:29:22.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfbfff700 1 -- 172.21.15.64:0/2886541969 learned_addr learned my addr 172.21.15.64:0/2886541969 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:22.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfaffd700 1 --2- 172.21.15.64:0/2886541969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc15b0d0 0x7facfc0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530492626 0 0) 0x7facfc0cd1c0 con 0x7facfc0c9420 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facdc003570 con 0x7facfc0c9420 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020504502 0 0) 0x7facfc0cdd60 con 0x7facfc15ad00 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facfc0cd1c0 con 0x7facfc15ad00 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2925484210 0 0) 0x7facdc003570 con 0x7facfc0c9420 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfaffd700 1 -- 172.21.15.64:0/2886541969 >> v1:172.21.15.123:6789/0 conn(0x7facfc15ad00 legacy=0x7facfc0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfaffd700 1 -- 172.21.15.64:0/2886541969 >> v1:172.21.15.64:6789/0 conn(0x7facfc0c9420 legacy=0x7facfc156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfaffd700 1 -- 172.21.15.64:0/2886541969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0cc5b0 con 0x7facfc15b0d0 2023-09-27T23:29:22.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.832+0000 7facfaffd700 1 --2- 172.21.15.64:0/2886541969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc15b0d0 0x7facfc0c8ee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facec009910 tx=0x7facec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c121f9c7b6a6a0c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40017f0 con 0x7facfc15b0d0 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec011420 con 0x7facfc15b0d0 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7facfa7fc700 1 -- 172.21.15.64:0/2886541969 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec0117a0 con 0x7facfc15b0d0 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 -- 172.21.15.64:0/2886541969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc15b0d0 msgr2=0x7facfc0c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 --2- 172.21.15.64:0/2886541969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc15b0d0 0x7facfc0c8ee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facec009910 tx=0x7facec0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 -- 172.21.15.64:0/2886541969 shutdown_connections 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 --2- 172.21.15.64:0/2886541969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc15b0d0 0x7facfc0c8ee0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 -- 172.21.15.64:0/2886541969 >> 172.21.15.64:0/2886541969 conn(0x7facfc0ce000 msgr2=0x7facfc0ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 -- 172.21.15.64:0/2886541969 shutdown_connections 2023-09-27T23:29:22.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.836+0000 7fad01d1d700 1 -- 172.21.15.64:0/2886541969 wait complete. 2023-09-27T23:29:22.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 Processor -- start 2023-09-27T23:29:22.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 -- start start 2023-09-27T23:29:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc0c9420 0x7facfc0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facfc15ad00 0x7facfc149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 0x7facfc1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0b17f0 con 0x7facfc15b0d0 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0b1ad0 con 0x7facfc15ad00 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0b1db0 con 0x7facfc0c9420 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 0x7facfc1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 0x7facfc1497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40040/0 (socket says 172.21.15.64:40040) 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 -- 172.21.15.64:0/486390797 learned_addr learned my addr 172.21.15.64:0/486390797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfb7fe700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc0c9420 0x7facfc0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfaffd700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facfc15ad00 0x7facfc149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 -- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc0c9420 msgr2=0x7facfc0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc0c9420 0x7facfc0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 -- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facfc15ad00 msgr2=0x7facfc149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facfc15ad00 0x7facfc149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 -- 172.21.15.64:0/486390797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec009420 con 0x7facfc15b0d0 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facfbfff700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 0x7facfc1497a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7facf400cb70 tx=0x7facf4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4004dc0 con 0x7facfc15b0d0 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf400a5d0 con 0x7facfc15b0d0 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc0b2090 con 0x7facfc15b0d0 2023-09-27T23:29:22.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400a990 con 0x7facfc15b0d0 2023-09-27T23:29:22.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.840+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc0c7ec0 con 0x7facfc15b0d0 2023-09-27T23:29:22.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.844+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc0b2590 con 0x7facfc15b0d0 2023-09-27T23:29:22.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.844+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7facf400ad00 con 0x7facfc15b0d0 2023-09-27T23:29:22.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.844+0000 7facf37fe700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face0060a70 0x7face0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:22.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.844+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7facf407ca70 con 0x7facfc15b0d0 2023-09-27T23:29:22.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.844+0000 7facfb7fe700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face0060a70 0x7face0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:22.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.844+0000 7facfb7fe700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face0060a70 0x7face0062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7face4007fd0 tx=0x7face4007d50 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:22.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:22.852+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7facf40505b0 con 0x7facfc15b0d0 2023-09-27T23:29:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.012+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facfc071120 con 0x7face0060a70 2023-09-27T23:29:23.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.028+0000 7facf37fe700 1 -- 172.21.15.64:0/486390797 <== mgr.5816 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295373 (secure 0 0 0) 0x7facfc071120 con 0x7face0060a70 2023-09-27T23:29:23.040 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face0060a70 msgr2=0x7face0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face0060a70 0x7face0062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7face4007fd0 tx=0x7face4007d50 comp rx=0 tx=0).stop 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 msgr2=0x7facfc1497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 0x7facfc1497a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7facf400cb70 tx=0x7facf4007c70 comp rx=0 tx=0).stop 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 shutdown_connections 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face0060a70 0x7face0062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15b0d0 0x7facfc1497a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facfc15ad00 0x7facfc149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 --2- 172.21.15.64:0/486390797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facfc0c9420 0x7facfc0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 >> 172.21.15.64:0/486390797 conn(0x7facfc0ce000 msgr2=0x7facfc0ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 shutdown_connections 2023-09-27T23:29:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.036+0000 7fad01d1d700 1 -- 172.21.15.64:0/486390797 wait complete. 2023-09-27T23:29:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:23.055 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:22.633866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":248,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":199284,"kb_used_data":9012,"kb_used_omap":0,"kb_used_meta":190208,"kb_avail":754775436,"statfs":{"total":773094113280,"available":772890046464,"internally_reserved":0,"allocated":9228288,"data_stored":5264169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":194772992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.323067"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746666+0000","last_change":"2023-09-27T23:29:19.735623+0000","last_active":"2023-09-27T23:29:19.746666+0000","last_peered":"2023-09-27T23:29:19.746666+0000","last_clean":"2023-09-27T23:29:19.746666+0000","last_became_active":"2023-09-27T23:29:19.735239+0000","last_became_peered":"2023-09-27T23:29:19.735239+0000","last_unstale":"2023-09-27T23:29:19.746666+0000","last_undegraded":"2023-09-27T23:29:19.746666+0000","last_fullsized":"2023-09-27T23:29:19.746666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:50:25.868487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742588+0000","last_change":"2023-09-27T23:29:19.731017+0000","last_active":"2023-09-27T23:29:19.742588+0000","last_peered":"2023-09-27T23:29:19.742588+0000","last_clean":"2023-09-27T23:29:19.742588+0000","last_became_active":"2023-09-27T23:29:19.729277+0000","last_became_peered":"2023-09-27T23:29:19.729277+0000","last_unstale":"2023-09-27T23:29:19.742588+0000","last_undegraded":"2023-09-27T23:29:19.742588+0000","last_fullsized":"2023-09-27T23:29:19.742588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:43.879878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742363+0000","last_change":"2023-09-27T23:29:19.729793+0000","last_active":"2023-09-27T23:29:19.742363+0000","last_peered":"2023-09-27T23:29:19.742363+0000","last_clean":"2023-09-27T23:29:19.742363+0000","last_became_active":"2023-09-27T23:29:19.729183+0000","last_became_peered":"2023-09-27T23:29:19.729183+0000","last_unstale":"2023-09-27T23:29:19.742363+0000","last_undegraded":"2023-09-27T23:29:19.742363+0000","last_fullsized":"2023-09-27T23:29:19.742363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:09.106318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742724+0000","last_change":"2023-09-27T23:29:19.731251+0000","last_active":"2023-09-27T23:29:19.742724+0000","last_peered":"2023-09-27T23:29:19.742724+0000","last_clean":"2023-09-27T23:29:19.742724+0000","last_became_active":"2023-09-27T23:29:19.729904+0000","last_became_peered":"2023-09-27T23:29:19.729904+0000","last_unstale":"2023-09-27T23:29:19.742724+0000","last_undegraded":"2023-09-27T23:29:19.742724+0000","last_fullsized":"2023-09-27T23:29:19.742724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:41.029315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742755+0000","last_change":"2023-09-27T23:29:19.731464+0000","last_active":"2023-09-27T23:29:19.742755+0000","last_peered":"2023-09-27T23:29:19.742755+0000","last_clean":"2023-09-27T23:29:19.742755+0000","last_became_active":"2023-09-27T23:29:19.729102+0000","last_became_peered":"2023-09-27T23:29:19.729102+0000","last_unstale":"2023-09-27T23:29:19.742755+0000","last_undegraded":"2023-09-27T23:29:19.742755+0000","last_fullsized":"2023-09-27T23:29:19.742755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:30:18.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746307+0000","last_change":"2023-09-27T23:29:19.731873+0000","last_active":"2023-09-27T23:29:19.746307+0000","last_peered":"2023-09-27T23:29:19.746307+0000","last_clean":"2023-09-27T23:29:19.746307+0000","last_became_active":"2023-09-27T23:29:19.730797+0000","last_became_peered":"2023-09-27T23:29:19.730797+0000","last_unstale":"2023-09-27T23:29:19.746307+0000","last_undegraded":"2023-09-27T23:29:19.746307+0000","last_fullsized":"2023-09-27T23:29:19.746307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:24:01.984490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742203+0000","last_change":"2023-09-27T23:29:19.355491+0000","last_active":"2023-09-27T23:29:19.742203+0000","last_peered":"2023-09-27T23:29:19.742203+0000","last_clean":"2023-09-27T23:29:19.742203+0000","last_became_active":"2023-09-27T23:29:19.355192+0000","last_became_peered":"2023-09-27T23:29:19.355192+0000","last_unstale":"2023-09-27T23:29:19.742203+0000","last_undegraded":"2023-09-27T23:29:19.742203+0000","last_fullsized":"2023-09-27T23:29:19.742203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:54:30.443743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742614+0000","last_change":"2023-09-27T23:29:19.356245+0000","last_active":"2023-09-27T23:29:19.742614+0000","last_peered":"2023-09-27T23:29:19.742614+0000","last_clean":"2023-09-27T23:29:19.742614+0000","last_became_active":"2023-09-27T23:29:19.355967+0000","last_became_peered":"2023-09-27T23:29:19.355967+0000","last_unstale":"2023-09-27T23:29:19.742614+0000","last_undegraded":"2023-09-27T23:29:19.742614+0000","last_fullsized":"2023-09-27T23:29:19.742614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:19.665630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.719234+0000","last_change":"2023-09-27T23:29:19.352763+0000","last_active":"2023-09-27T23:29:19.719234+0000","last_peered":"2023-09-27T23:29:19.719234+0000","last_clean":"2023-09-27T23:29:19.719234+0000","last_became_active":"2023-09-27T23:29:19.352450+0000","last_became_peered":"2023-09-27T23:29:19.352450+0000","last_unstale":"2023-09-27T23:29:19.719234+0000","last_undegraded":"2023-09-27T23:29:19.719234+0000","last_fullsized":"2023-09-27T23:29:19.719234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:09.789659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746329+0000","last_change":"2023-09-27T23:29:19.734575+0000","last_active":"2023-09-27T23:29:19.746329+0000","last_peered":"2023-09-27T23:29:19.746329+0000","last_clean":"2023-09-27T23:29:19.746329+0000","last_became_active":"2023-09-27T23:29:19.730734+0000","last_became_peered":"2023-09-27T23:29:19.730734+0000","last_unstale":"2023-09-27T23:29:19.746329+0000","last_undegraded":"2023-09-27T23:29:19.746329+0000","last_fullsized":"2023-09-27T23:29:19.746329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:47.061803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716915+0000","last_change":"2023-09-27T23:29:19.351569+0000","last_active":"2023-09-27T23:29:19.716915+0000","last_peered":"2023-09-27T23:29:19.716915+0000","last_clean":"2023-09-27T23:29:19.716915+0000","last_became_active":"2023-09-27T23:29:19.351295+0000","last_became_peered":"2023-09-27T23:29:19.351295+0000","last_unstale":"2023-09-27T23:29:19.716915+0000","last_undegraded":"2023-09-27T23:29:19.716915+0000","last_fullsized":"2023-09-27T23:29:19.716915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:38.540430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742267+0000","last_change":"2023-09-27T23:29:19.359846+0000","last_active":"2023-09-27T23:29:19.742267+0000","last_peered":"2023-09-27T23:29:19.742267+0000","last_clean":"2023-09-27T23:29:19.742267+0000","last_became_active":"2023-09-27T23:29:19.359535+0000","last_became_peered":"2023-09-27T23:29:19.359535+0000","last_unstale":"2023-09-27T23:29:19.742267+0000","last_undegraded":"2023-09-27T23:29:19.742267+0000","last_fullsized":"2023-09-27T23:29:19.742267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:37.201350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742073+0000","last_change":"2023-09-27T23:29:19.729613+0000","last_active":"2023-09-27T23:29:19.742073+0000","last_peered":"2023-09-27T23:29:19.742073+0000","last_clean":"2023-09-27T23:29:19.742073+0000","last_became_active":"2023-09-27T23:29:19.729107+0000","last_became_peered":"2023-09-27T23:29:19.729107+0000","last_unstale":"2023-09-27T23:29:19.742073+0000","last_undegraded":"2023-09-27T23:29:19.742073+0000","last_fullsized":"2023-09-27T23:29:19.742073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:39:43.514982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742014+0000","last_change":"2023-09-27T23:29:19.729669+0000","last_active":"2023-09-27T23:29:19.742014+0000","last_peered":"2023-09-27T23:29:19.742014+0000","last_clean":"2023-09-27T23:29:19.742014+0000","last_became_active":"2023-09-27T23:29:19.729046+0000","last_became_peered":"2023-09-27T23:29:19.729046+0000","last_unstale":"2023-09-27T23:29:19.742014+0000","last_undegraded":"2023-09-27T23:29:19.742014+0000","last_fullsized":"2023-09-27T23:29:19.742014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:01.763686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716949+0000","last_change":"2023-09-27T23:29:19.355551+0000","last_active":"2023-09-27T23:29:19.716949+0000","last_peered":"2023-09-27T23:29:19.716949+0000","last_clean":"2023-09-27T23:29:19.716949+0000","last_became_active":"2023-09-27T23:29:19.355244+0000","last_became_peered":"2023-09-27T23:29:19.355244+0000","last_unstale":"2023-09-27T23:29:19.716949+0000","last_undegraded":"2023-09-27T23:29:19.716949+0000","last_fullsized":"2023-09-27T23:29:19.716949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:09.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742096+0000","last_change":"2023-09-27T23:29:19.355604+0000","last_active":"2023-09-27T23:29:19.742096+0000","last_peered":"2023-09-27T23:29:19.742096+0000","last_clean":"2023-09-27T23:29:19.742096+0000","last_became_active":"2023-09-27T23:29:19.355207+0000","last_became_peered":"2023-09-27T23:29:19.355207+0000","last_unstale":"2023-09-27T23:29:19.742096+0000","last_undegraded":"2023-09-27T23:29:19.742096+0000","last_fullsized":"2023-09-27T23:29:19.742096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:33.460035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'161","reported_seq":376,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.360221+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:15.360221+0000","last_peered":"2023-09-27T23:29:15.360221+0000","last_clean":"2023-09-27T23:29:15.360221+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:15.360221+0000","last_undegraded":"2023-09-27T23:29:15.360221+0000","last_fullsized":"2023-09-27T23:29:15.360221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":161,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358364+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:15.358364+0000","last_peered":"2023-09-27T23:29:15.358364+0000","last_clean":"2023-09-27T23:29:15.358364+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:15.358364+0000","last_undegraded":"2023-09-27T23:29:15.358364+0000","last_fullsized":"2023-09-27T23:29:15.358364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359761+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:15.359761+0000","last_peered":"2023-09-27T23:29:15.359761+0000","last_clean":"2023-09-27T23:29:15.359761+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:15.359761+0000","last_undegraded":"2023-09-27T23:29:15.359761+0000","last_fullsized":"2023-09-27T23:29:15.359761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.119882+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:15.119882+0000","last_peered":"2023-09-27T23:29:15.119882+0000","last_clean":"2023-09-27T23:29:15.119882+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:15.119882+0000","last_undegraded":"2023-09-27T23:29:15.119882+0000","last_fullsized":"2023-09-27T23:29:15.119882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.717030+0000","last_change":"2023-09-27T23:29:19.351439+0000","last_active":"2023-09-27T23:29:19.717030+0000","last_peered":"2023-09-27T23:29:19.717030+0000","last_clean":"2023-09-27T23:29:19.717030+0000","last_became_active":"2023-09-27T23:29:19.350793+0000","last_became_peered":"2023-09-27T23:29:19.350793+0000","last_unstale":"2023-09-27T23:29:19.717030+0000","last_undegraded":"2023-09-27T23:29:19.717030+0000","last_fullsized":"2023-09-27T23:29:19.717030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:44:07.068721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358360+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:15.358360+0000","last_peered":"2023-09-27T23:29:15.358360+0000","last_clean":"2023-09-27T23:29:15.358360+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:15.358360+0000","last_undegraded":"2023-09-27T23:29:15.358360+0000","last_fullsized":"2023-09-27T23:29:15.358360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355205+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:15.355205+0000","last_peered":"2023-09-27T23:29:15.355205+0000","last_clean":"2023-09-27T23:29:15.355205+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:15.355205+0000","last_undegraded":"2023-09-27T23:29:15.355205+0000","last_fullsized":"2023-09-27T23:29:15.355205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742337+0000","last_change":"2023-09-27T23:29:19.351252+0000","last_active":"2023-09-27T23:29:19.742337+0000","last_peered":"2023-09-27T23:29:19.742337+0000","last_clean":"2023-09-27T23:29:19.742337+0000","last_became_active":"2023-09-27T23:29:19.350970+0000","last_became_peered":"2023-09-27T23:29:19.350970+0000","last_unstale":"2023-09-27T23:29:19.742337+0000","last_undegraded":"2023-09-27T23:29:19.742337+0000","last_fullsized":"2023-09-27T23:29:19.742337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:02.449177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359957+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:15.359957+0000","last_peered":"2023-09-27T23:29:15.359957+0000","last_clean":"2023-09-27T23:29:15.359957+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:15.359957+0000","last_undegraded":"2023-09-27T23:29:15.359957+0000","last_fullsized":"2023-09-27T23:29:15.359957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355216+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:15.355216+0000","last_peered":"2023-09-27T23:29:15.355216+0000","last_clean":"2023-09-27T23:29:15.355216+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:15.355216+0000","last_undegraded":"2023-09-27T23:29:15.355216+0000","last_fullsized":"2023-09-27T23:29:15.355216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742021+0000","last_change":"2023-09-27T23:29:19.359807+0000","last_active":"2023-09-27T23:29:19.742021+0000","last_peered":"2023-09-27T23:29:19.742021+0000","last_clean":"2023-09-27T23:29:19.742021+0000","last_became_active":"2023-09-27T23:29:19.359497+0000","last_became_peered":"2023-09-27T23:29:19.359497+0000","last_unstale":"2023-09-27T23:29:19.742021+0000","last_undegraded":"2023-09-27T23:29:19.742021+0000","last_fullsized":"2023-09-27T23:29:19.742021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:12:13.225733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.714700+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:19.714700+0000","last_peered":"2023-09-27T23:29:19.714700+0000","last_clean":"2023-09-27T23:29:19.714700+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:19.714700+0000","last_undegraded":"2023-09-27T23:29:19.714700+0000","last_fullsized":"2023-09-27T23:29:19.714700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25500,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":23936,"kb_avail":94346340,"statfs":{"total":96636764160,"available":96610652160,"internally_reserved":0,"allocated":1593344,"data_stored":1098094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24510464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.749},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.878},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.052}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.016},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.121}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0820000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21264,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350576,"statfs":{"total":96636764160,"available":96614989824,"internally_reserved":0,"allocated":1056768,"data_stored":535629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":0.84099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.874}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.753}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":1.244},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.284},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.232}]}]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31192,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":30208,"kb_avail":94340648,"statfs":{"total":96636764160,"available":96604823552,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30932992},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":1.034},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.2330000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":0.747},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.113},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.081}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":0.94999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.196}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.262}]}]},{"osd":4,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21272,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20288,"kb_avail":94350568,"statfs":{"total":96636764160,"available":96614981632,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20774912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.091},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.89700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":1.3169999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":0.98399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.135}]}]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21848,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":94349992,"statfs":{"total":96636764160,"available":96614391808,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.86599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.238},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.357}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":0.999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.321}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.0620000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20312,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":19328,"kb_avail":94351528,"statfs":{"total":96636764160,"available":96615964672,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19791872},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":1.018}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":1.272},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.214},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.048}]}]},{"osd":1,"up_from":14,"seq":60129542336,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":34560,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35389440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.621},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":0.496}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":0.95299999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22352,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349488,"statfs":{"total":96636764160,"available":96613875712,"internally_reserved":0,"allocated":1581056,"data_stored":1091816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.62},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.435},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.022},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":0.80200000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.083}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.127},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:23.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.160+0000 7fc6462ee700 1 Processor -- start 2023-09-27T23:29:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.160+0000 7fc6462ee700 1 -- start start 2023-09-27T23:29:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc6400b1410 0x7fc6400ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006aa00 0x7fc64015af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64006c7c0 con 0x7fc64006a580 2023-09-27T23:29:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64006cd80 con 0x7fc6400b1410 2023-09-27T23:29:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64006b8f0 con 0x7fc64006aa00 2023-09-27T23:29:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc6400b1410 0x7fc6400ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc6400b1410 0x7fc6400ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40042/0 (socket says 172.21.15.64:40042) 2023-09-27T23:29:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 -- 172.21.15.64:0/2211380874 learned_addr learned my addr 172.21.15.64:0/2211380874 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc645aed700 1 --2- 172.21.15.64:0/2211380874 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006aa00 0x7fc64015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc637fff700 1 -- 172.21.15.64:0/2211380874 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660364653 0 0) 0x7fc64006c7c0 con 0x7fc64006a580 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 -- 172.21.15.64:0/2211380874 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006aa00 msgr2=0x7fc64015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 --2- 172.21.15.64:0/2211380874 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006aa00 0x7fc64015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 -- 172.21.15.64:0/2211380874 >> v1:172.21.15.123:6789/0 conn(0x7fc64006a580 legacy=0x7fc64015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 -- 172.21.15.64:0/2211380874 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6400ac1e0 con 0x7fc6400b1410 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6452ec700 1 --2- 172.21.15.64:0/2211380874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6400b1410 0x7fc6400ab440 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc6300099b0 tx=0x7fc630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ca882cb03b1e7c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc637fff700 1 -- 172.21.15.64:0/2211380874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630009ea0 con 0x7fc6400b1410 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc637fff700 1 -- 172.21.15.64:0/2211380874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63000b860 con 0x7fc6400b1410 2023-09-27T23:29:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc637fff700 1 -- 172.21.15.64:0/2211380874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63000bc00 con 0x7fc6400b1410 2023-09-27T23:29:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- 172.21.15.64:0/2211380874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6400b1410 msgr2=0x7fc6400ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 --2- 172.21.15.64:0/2211380874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6400b1410 0x7fc6400ab440 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc6300099b0 tx=0x7fc630009250 comp rx=0 tx=0).stop 2023-09-27T23:29:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- 172.21.15.64:0/2211380874 shutdown_connections 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 --2- 172.21.15.64:0/2211380874 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006aa00 0x7fc64015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 --2- 172.21.15.64:0/2211380874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6400b1410 0x7fc6400ab440 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- 172.21.15.64:0/2211380874 >> 172.21.15.64:0/2211380874 conn(0x7fc640067bb0 msgr2=0x7fc6400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- 172.21.15.64:0/2211380874 shutdown_connections 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.164+0000 7fc6462ee700 1 -- 172.21.15.64:0/2211380874 wait complete. 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 Processor -- start 2023-09-27T23:29:23.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 -- start start 2023-09-27T23:29:23.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006a580 0x7fc64014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64006aa00 0x7fc64016af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6400b1410 0x7fc64014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc640155280 con 0x7fc64006aa00 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc640154df0 con 0x7fc6400b1410 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.168+0000 7fc6462ee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64014ebb0 con 0x7fc64006a580 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc645aed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6400b1410 0x7fc64014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64006aa00 0x7fc64016af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6452ec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006a580 0x7fc64014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6452ec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006a580 0x7fc64014e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49122/0 (socket says 172.21.15.64:49122) 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6452ec700 1 -- 172.21.15.64:0/989183251 learned_addr learned my addr 172.21.15.64:0/989183251 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:23.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 -- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006a580 msgr2=0x7fc64014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006a580 0x7fc64014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 -- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6400b1410 msgr2=0x7fc64014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6400b1410 0x7fc64014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 -- 172.21.15.64:0/989183251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6300094f0 con 0x7fc64006aa00 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc644aeb700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64006aa00 0x7fc64016af00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c510 tx=0x7fc63c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00ce80 con 0x7fc64006aa00 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c011590 con 0x7fc64006aa00 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc640154200 con 0x7fc64006aa00 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c011950 con 0x7fc64006aa00 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc640154820 con 0x7fc64006aa00 2023-09-27T23:29:23.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.172+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc640154f80 con 0x7fc64006aa00 2023-09-27T23:29:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.176+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc63c011d00 con 0x7fc64006aa00 2023-09-27T23:29:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.176+0000 7fc6357fa700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc624060a70 0x7fc624062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.176+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc63c07c420 con 0x7fc64006aa00 2023-09-27T23:29:23.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.176+0000 7fc6452ec700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc624060a70 0x7fc624062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.176+0000 7fc6452ec700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc624060a70 0x7fc624062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc630003080 tx=0x7fc630017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.184+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc63c04c450 con 0x7fc64006aa00 2023-09-27T23:29:23.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.348+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc64004be40 con 0x7fc624060a70 2023-09-27T23:29:23.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.364+0000 7fc6357fa700 1 -- 172.21.15.64:0/989183251 <== mgr.5816 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295373 (secure 0 0 0) 0x7fc64004be40 con 0x7fc624060a70 2023-09-27T23:29:23.375 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:23.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.368+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc624060a70 msgr2=0x7fc624062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.368+0000 7fc6462ee700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc624060a70 0x7fc624062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc630003080 tx=0x7fc630017000 comp rx=0 tx=0).stop 2023-09-27T23:29:23.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.368+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64006aa00 msgr2=0x7fc64016af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.368+0000 7fc6462ee700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64006aa00 0x7fc64016af00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c510 tx=0x7fc63c007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 shutdown_connections 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc624060a70 0x7fc624062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6400b1410 0x7fc64014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc64006aa00 0x7fc64016af00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 --2- 172.21.15.64:0/989183251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc64006a580 0x7fc64014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 >> 172.21.15.64:0/989183251 conn(0x7fc640067bb0 msgr2=0x7fc6400ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 shutdown_connections 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.372+0000 7fc6462ee700 1 -- 172.21.15.64:0/989183251 wait complete. 2023-09-27T23:29:23.378 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:23.389 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:22.633866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":248,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":199284,"kb_used_data":9012,"kb_used_omap":0,"kb_used_meta":190208,"kb_avail":754775436,"statfs":{"total":773094113280,"available":772890046464,"internally_reserved":0,"allocated":9228288,"data_stored":5264169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":194772992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.323067"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746666+0000","last_change":"2023-09-27T23:29:19.735623+0000","last_active":"2023-09-27T23:29:19.746666+0000","last_peered":"2023-09-27T23:29:19.746666+0000","last_clean":"2023-09-27T23:29:19.746666+0000","last_became_active":"2023-09-27T23:29:19.735239+0000","last_became_peered":"2023-09-27T23:29:19.735239+0000","last_unstale":"2023-09-27T23:29:19.746666+0000","last_undegraded":"2023-09-27T23:29:19.746666+0000","last_fullsized":"2023-09-27T23:29:19.746666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:50:25.868487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742588+0000","last_change":"2023-09-27T23:29:19.731017+0000","last_active":"2023-09-27T23:29:19.742588+0000","last_peered":"2023-09-27T23:29:19.742588+0000","last_clean":"2023-09-27T23:29:19.742588+0000","last_became_active":"2023-09-27T23:29:19.729277+0000","last_became_peered":"2023-09-27T23:29:19.729277+0000","last_unstale":"2023-09-27T23:29:19.742588+0000","last_undegraded":"2023-09-27T23:29:19.742588+0000","last_fullsized":"2023-09-27T23:29:19.742588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:43.879878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742363+0000","last_change":"2023-09-27T23:29:19.729793+0000","last_active":"2023-09-27T23:29:19.742363+0000","last_peered":"2023-09-27T23:29:19.742363+0000","last_clean":"2023-09-27T23:29:19.742363+0000","last_became_active":"2023-09-27T23:29:19.729183+0000","last_became_peered":"2023-09-27T23:29:19.729183+0000","last_unstale":"2023-09-27T23:29:19.742363+0000","last_undegraded":"2023-09-27T23:29:19.742363+0000","last_fullsized":"2023-09-27T23:29:19.742363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:09.106318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742724+0000","last_change":"2023-09-27T23:29:19.731251+0000","last_active":"2023-09-27T23:29:19.742724+0000","last_peered":"2023-09-27T23:29:19.742724+0000","last_clean":"2023-09-27T23:29:19.742724+0000","last_became_active":"2023-09-27T23:29:19.729904+0000","last_became_peered":"2023-09-27T23:29:19.729904+0000","last_unstale":"2023-09-27T23:29:19.742724+0000","last_undegraded":"2023-09-27T23:29:19.742724+0000","last_fullsized":"2023-09-27T23:29:19.742724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:41.029315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742755+0000","last_change":"2023-09-27T23:29:19.731464+0000","last_active":"2023-09-27T23:29:19.742755+0000","last_peered":"2023-09-27T23:29:19.742755+0000","last_clean":"2023-09-27T23:29:19.742755+0000","last_became_active":"2023-09-27T23:29:19.729102+0000","last_became_peered":"2023-09-27T23:29:19.729102+0000","last_unstale":"2023-09-27T23:29:19.742755+0000","last_undegraded":"2023-09-27T23:29:19.742755+0000","last_fullsized":"2023-09-27T23:29:19.742755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:30:18.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746307+0000","last_change":"2023-09-27T23:29:19.731873+0000","last_active":"2023-09-27T23:29:19.746307+0000","last_peered":"2023-09-27T23:29:19.746307+0000","last_clean":"2023-09-27T23:29:19.746307+0000","last_became_active":"2023-09-27T23:29:19.730797+0000","last_became_peered":"2023-09-27T23:29:19.730797+0000","last_unstale":"2023-09-27T23:29:19.746307+0000","last_undegraded":"2023-09-27T23:29:19.746307+0000","last_fullsized":"2023-09-27T23:29:19.746307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:24:01.984490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742203+0000","last_change":"2023-09-27T23:29:19.355491+0000","last_active":"2023-09-27T23:29:19.742203+0000","last_peered":"2023-09-27T23:29:19.742203+0000","last_clean":"2023-09-27T23:29:19.742203+0000","last_became_active":"2023-09-27T23:29:19.355192+0000","last_became_peered":"2023-09-27T23:29:19.355192+0000","last_unstale":"2023-09-27T23:29:19.742203+0000","last_undegraded":"2023-09-27T23:29:19.742203+0000","last_fullsized":"2023-09-27T23:29:19.742203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:54:30.443743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742614+0000","last_change":"2023-09-27T23:29:19.356245+0000","last_active":"2023-09-27T23:29:19.742614+0000","last_peered":"2023-09-27T23:29:19.742614+0000","last_clean":"2023-09-27T23:29:19.742614+0000","last_became_active":"2023-09-27T23:29:19.355967+0000","last_became_peered":"2023-09-27T23:29:19.355967+0000","last_unstale":"2023-09-27T23:29:19.742614+0000","last_undegraded":"2023-09-27T23:29:19.742614+0000","last_fullsized":"2023-09-27T23:29:19.742614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:19.665630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.719234+0000","last_change":"2023-09-27T23:29:19.352763+0000","last_active":"2023-09-27T23:29:19.719234+0000","last_peered":"2023-09-27T23:29:19.719234+0000","last_clean":"2023-09-27T23:29:19.719234+0000","last_became_active":"2023-09-27T23:29:19.352450+0000","last_became_peered":"2023-09-27T23:29:19.352450+0000","last_unstale":"2023-09-27T23:29:19.719234+0000","last_undegraded":"2023-09-27T23:29:19.719234+0000","last_fullsized":"2023-09-27T23:29:19.719234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:09.789659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746329+0000","last_change":"2023-09-27T23:29:19.734575+0000","last_active":"2023-09-27T23:29:19.746329+0000","last_peered":"2023-09-27T23:29:19.746329+0000","last_clean":"2023-09-27T23:29:19.746329+0000","last_became_active":"2023-09-27T23:29:19.730734+0000","last_became_peered":"2023-09-27T23:29:19.730734+0000","last_unstale":"2023-09-27T23:29:19.746329+0000","last_undegraded":"2023-09-27T23:29:19.746329+0000","last_fullsized":"2023-09-27T23:29:19.746329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:47.061803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716915+0000","last_change":"2023-09-27T23:29:19.351569+0000","last_active":"2023-09-27T23:29:19.716915+0000","last_peered":"2023-09-27T23:29:19.716915+0000","last_clean":"2023-09-27T23:29:19.716915+0000","last_became_active":"2023-09-27T23:29:19.351295+0000","last_became_peered":"2023-09-27T23:29:19.351295+0000","last_unstale":"2023-09-27T23:29:19.716915+0000","last_undegraded":"2023-09-27T23:29:19.716915+0000","last_fullsized":"2023-09-27T23:29:19.716915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:38.540430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742267+0000","last_change":"2023-09-27T23:29:19.359846+0000","last_active":"2023-09-27T23:29:19.742267+0000","last_peered":"2023-09-27T23:29:19.742267+0000","last_clean":"2023-09-27T23:29:19.742267+0000","last_became_active":"2023-09-27T23:29:19.359535+0000","last_became_peered":"2023-09-27T23:29:19.359535+0000","last_unstale":"2023-09-27T23:29:19.742267+0000","last_undegraded":"2023-09-27T23:29:19.742267+0000","last_fullsized":"2023-09-27T23:29:19.742267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:37.201350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742073+0000","last_change":"2023-09-27T23:29:19.729613+0000","last_active":"2023-09-27T23:29:19.742073+0000","last_peered":"2023-09-27T23:29:19.742073+0000","last_clean":"2023-09-27T23:29:19.742073+0000","last_became_active":"2023-09-27T23:29:19.729107+0000","last_became_peered":"2023-09-27T23:29:19.729107+0000","last_unstale":"2023-09-27T23:29:19.742073+0000","last_undegraded":"2023-09-27T23:29:19.742073+0000","last_fullsized":"2023-09-27T23:29:19.742073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:39:43.514982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742014+0000","last_change":"2023-09-27T23:29:19.729669+0000","last_active":"2023-09-27T23:29:19.742014+0000","last_peered":"2023-09-27T23:29:19.742014+0000","last_clean":"2023-09-27T23:29:19.742014+0000","last_became_active":"2023-09-27T23:29:19.729046+0000","last_became_peered":"2023-09-27T23:29:19.729046+0000","last_unstale":"2023-09-27T23:29:19.742014+0000","last_undegraded":"2023-09-27T23:29:19.742014+0000","last_fullsized":"2023-09-27T23:29:19.742014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:01.763686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716949+0000","last_change":"2023-09-27T23:29:19.355551+0000","last_active":"2023-09-27T23:29:19.716949+0000","last_peered":"2023-09-27T23:29:19.716949+0000","last_clean":"2023-09-27T23:29:19.716949+0000","last_became_active":"2023-09-27T23:29:19.355244+0000","last_became_peered":"2023-09-27T23:29:19.355244+0000","last_unstale":"2023-09-27T23:29:19.716949+0000","last_undegraded":"2023-09-27T23:29:19.716949+0000","last_fullsized":"2023-09-27T23:29:19.716949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:09.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742096+0000","last_change":"2023-09-27T23:29:19.355604+0000","last_active":"2023-09-27T23:29:19.742096+0000","last_peered":"2023-09-27T23:29:19.742096+0000","last_clean":"2023-09-27T23:29:19.742096+0000","last_became_active":"2023-09-27T23:29:19.355207+0000","last_became_peered":"2023-09-27T23:29:19.355207+0000","last_unstale":"2023-09-27T23:29:19.742096+0000","last_undegraded":"2023-09-27T23:29:19.742096+0000","last_fullsized":"2023-09-27T23:29:19.742096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:33.460035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'161","reported_seq":376,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.360221+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:15.360221+0000","last_peered":"2023-09-27T23:29:15.360221+0000","last_clean":"2023-09-27T23:29:15.360221+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:15.360221+0000","last_undegraded":"2023-09-27T23:29:15.360221+0000","last_fullsized":"2023-09-27T23:29:15.360221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":161,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358364+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:15.358364+0000","last_peered":"2023-09-27T23:29:15.358364+0000","last_clean":"2023-09-27T23:29:15.358364+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:15.358364+0000","last_undegraded":"2023-09-27T23:29:15.358364+0000","last_fullsized":"2023-09-27T23:29:15.358364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359761+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:15.359761+0000","last_peered":"2023-09-27T23:29:15.359761+0000","last_clean":"2023-09-27T23:29:15.359761+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:15.359761+0000","last_undegraded":"2023-09-27T23:29:15.359761+0000","last_fullsized":"2023-09-27T23:29:15.359761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.119882+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:15.119882+0000","last_peered":"2023-09-27T23:29:15.119882+0000","last_clean":"2023-09-27T23:29:15.119882+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:15.119882+0000","last_undegraded":"2023-09-27T23:29:15.119882+0000","last_fullsized":"2023-09-27T23:29:15.119882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.717030+0000","last_change":"2023-09-27T23:29:19.351439+0000","last_active":"2023-09-27T23:29:19.717030+0000","last_peered":"2023-09-27T23:29:19.717030+0000","last_clean":"2023-09-27T23:29:19.717030+0000","last_became_active":"2023-09-27T23:29:19.350793+0000","last_became_peered":"2023-09-27T23:29:19.350793+0000","last_unstale":"2023-09-27T23:29:19.717030+0000","last_undegraded":"2023-09-27T23:29:19.717030+0000","last_fullsized":"2023-09-27T23:29:19.717030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:44:07.068721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358360+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:15.358360+0000","last_peered":"2023-09-27T23:29:15.358360+0000","last_clean":"2023-09-27T23:29:15.358360+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:15.358360+0000","last_undegraded":"2023-09-27T23:29:15.358360+0000","last_fullsized":"2023-09-27T23:29:15.358360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355205+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:15.355205+0000","last_peered":"2023-09-27T23:29:15.355205+0000","last_clean":"2023-09-27T23:29:15.355205+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:15.355205+0000","last_undegraded":"2023-09-27T23:29:15.355205+0000","last_fullsized":"2023-09-27T23:29:15.355205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742337+0000","last_change":"2023-09-27T23:29:19.351252+0000","last_active":"2023-09-27T23:29:19.742337+0000","last_peered":"2023-09-27T23:29:19.742337+0000","last_clean":"2023-09-27T23:29:19.742337+0000","last_became_active":"2023-09-27T23:29:19.350970+0000","last_became_peered":"2023-09-27T23:29:19.350970+0000","last_unstale":"2023-09-27T23:29:19.742337+0000","last_undegraded":"2023-09-27T23:29:19.742337+0000","last_fullsized":"2023-09-27T23:29:19.742337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:02.449177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359957+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:15.359957+0000","last_peered":"2023-09-27T23:29:15.359957+0000","last_clean":"2023-09-27T23:29:15.359957+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:15.359957+0000","last_undegraded":"2023-09-27T23:29:15.359957+0000","last_fullsized":"2023-09-27T23:29:15.359957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355216+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:15.355216+0000","last_peered":"2023-09-27T23:29:15.355216+0000","last_clean":"2023-09-27T23:29:15.355216+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:15.355216+0000","last_undegraded":"2023-09-27T23:29:15.355216+0000","last_fullsized":"2023-09-27T23:29:15.355216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742021+0000","last_change":"2023-09-27T23:29:19.359807+0000","last_active":"2023-09-27T23:29:19.742021+0000","last_peered":"2023-09-27T23:29:19.742021+0000","last_clean":"2023-09-27T23:29:19.742021+0000","last_became_active":"2023-09-27T23:29:19.359497+0000","last_became_peered":"2023-09-27T23:29:19.359497+0000","last_unstale":"2023-09-27T23:29:19.742021+0000","last_undegraded":"2023-09-27T23:29:19.742021+0000","last_fullsized":"2023-09-27T23:29:19.742021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:12:13.225733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.714700+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:19.714700+0000","last_peered":"2023-09-27T23:29:19.714700+0000","last_clean":"2023-09-27T23:29:19.714700+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:19.714700+0000","last_undegraded":"2023-09-27T23:29:19.714700+0000","last_fullsized":"2023-09-27T23:29:19.714700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25500,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":23936,"kb_avail":94346340,"statfs":{"total":96636764160,"available":96610652160,"internally_reserved":0,"allocated":1593344,"data_stored":1098094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24510464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.749},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.878},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.052}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.016},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.121}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0820000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21264,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350576,"statfs":{"total":96636764160,"available":96614989824,"internally_reserved":0,"allocated":1056768,"data_stored":535629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":0.84099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.874}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.753}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":1.244},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.284},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.232}]}]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31192,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":30208,"kb_avail":94340648,"statfs":{"total":96636764160,"available":96604823552,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30932992},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":1.034},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.2330000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":0.747},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.113},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.081}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":0.94999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.196}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.262}]}]},{"osd":4,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21272,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20288,"kb_avail":94350568,"statfs":{"total":96636764160,"available":96614981632,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20774912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.091},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.89700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":1.3169999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":0.98399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.135}]}]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21848,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":94349992,"statfs":{"total":96636764160,"available":96614391808,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.86599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.238},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.357}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":0.999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.321}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.0620000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20312,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":19328,"kb_avail":94351528,"statfs":{"total":96636764160,"available":96615964672,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19791872},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":1.018}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":1.272},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.214},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.048}]}]},{"osd":1,"up_from":14,"seq":60129542336,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":34560,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35389440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.621},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":0.496}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":0.95299999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22352,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349488,"statfs":{"total":96636764160,"available":96613875712,"internally_reserved":0,"allocated":1581056,"data_stored":1091816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.62},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.435},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.022},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":0.80200000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.083}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.127},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:23.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-09-27T23:29:23.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-09-27T23:29:23.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-09-27T23:29:23.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-09-27T23:29:23.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-09-27T23:29:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-09-27T23:29:23.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-09-27T23:29:23.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-09-27T23:29:23.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-09-27T23:29:23.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-09-27T23:29:23.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-09-27T23:29:23.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:18.299206+0000', 'last_change': '2023-09-27T23:29:18.299206+0000', 'last_active': '2023-09-27T23:29:18.299206+0000', 'last_peered': '2023-09-27T23:29:18.299206+0000', 'last_clean': '2023-09-27T23:29:18.299206+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:18.299206+0000', 'last_undegraded': '2023-09-27T23:29:18.299206+0000', 'last_fullsized': '2023-09-27T23:29:18.299206+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:18.299206+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-09-27T23:29:23.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:19.306227+0000', 'last_change': '2023-09-27T23:29:19.306227+0000', 'last_active': '2023-09-27T23:29:19.306227+0000', 'last_peered': '2023-09-27T23:29:19.306227+0000', 'last_clean': '2023-09-27T23:29:19.306227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:19.306227+0000', 'last_undegraded': '2023-09-27T23:29:19.306227+0000', 'last_fullsized': '2023-09-27T23:29:19.306227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:19.306227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:23.412 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:29:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.512+0000 7f48774c0700 1 Processor -- start 2023-09-27T23:29:23.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.512+0000 7f48774c0700 1 -- start start 2023-09-27T23:29:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.512+0000 7f48774c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f487006e1d0 0x7f4870159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f487006b210 0x7f487014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487015a260 con 0x7f487006ad70 2023-09-27T23:29:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48700afad0 con 0x7f487006e1d0 2023-09-27T23:29:23.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48700afdb0 con 0x7f487006b210 2023-09-27T23:29:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f487006e1d0 0x7f4870159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f4875cbd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f487006ad70 0x7f487014f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51144/0 (socket says 172.21.15.64:51144) 2023-09-27T23:29:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f487006e1d0 0x7f4870159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40062/0 (socket says 172.21.15.64:40062) 2023-09-27T23:29:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f4875cbd700 1 -- 172.21.15.64:0/4069689521 learned_addr learned my addr 172.21.15.64:0/4069689521 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:23.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f4876cbf700 1 --2- 172.21.15.64:0/4069689521 >> v2:172.21.15.123:3300/0 conn(0x7f487006b210 0x7f487014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 -- 172.21.15.64:0/4069689521 >> v2:172.21.15.123:3300/0 conn(0x7f487006b210 msgr2=0x7f487014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 --2- 172.21.15.64:0/4069689521 >> v2:172.21.15.123:3300/0 conn(0x7f487006b210 0x7f487014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48754bc700 1 -- 172.21.15.64:0/4069689521 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847280462 0 0) 0x7f487015a260 con 0x7f487006ad70 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 -- 172.21.15.64:0/4069689521 >> v1:172.21.15.64:6789/0 conn(0x7f487006ad70 legacy=0x7f487014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 -- 172.21.15.64:0/4069689521 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48700b0090 con 0x7f487006e1d0 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48764be700 1 --2- 172.21.15.64:0/4069689521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006e1d0 0x7f4870159d20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f487006a7f0 tx=0x7f486c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cf64d14c8dc7f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48754bc700 1 -- 172.21.15.64:0/4069689521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c011630 con 0x7f487006e1d0 2023-09-27T23:29:23.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48754bc700 1 -- 172.21.15.64:0/4069689521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c011790 con 0x7f487006e1d0 2023-09-27T23:29:23.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48754bc700 1 -- 172.21.15.64:0/4069689521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c011b30 con 0x7f487006e1d0 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- 172.21.15.64:0/4069689521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006e1d0 msgr2=0x7f4870159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 --2- 172.21.15.64:0/4069689521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006e1d0 0x7f4870159d20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f487006a7f0 tx=0x7f486c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- 172.21.15.64:0/4069689521 shutdown_connections 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 --2- 172.21.15.64:0/4069689521 >> v2:172.21.15.123:3300/0 conn(0x7f487006b210 0x7f487014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 --2- 172.21.15.64:0/4069689521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006e1d0 0x7f4870159d20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- 172.21.15.64:0/4069689521 >> 172.21.15.64:0/4069689521 conn(0x7f4870067bb0 msgr2=0x7f48700add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- 172.21.15.64:0/4069689521 shutdown_connections 2023-09-27T23:29:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.516+0000 7f48774c0700 1 -- 172.21.15.64:0/4069689521 wait complete. 2023-09-27T23:29:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 Processor -- start 2023-09-27T23:29:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 -- start start 2023-09-27T23:29:23.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487006ad70 0x7f4870141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 0x7f4870153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f487006e1d0 0x7f48700ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4870155340 con 0x7f487006b210 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48701421f0 con 0x7f487006ad70 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48774c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4870154730 con 0x7f487006e1d0 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 0x7f4870153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 0x7f4870153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40072/0 (socket says 172.21.15.64:40072) 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 -- 172.21.15.64:0/890938597 learned_addr learned my addr 172.21.15.64:0/890938597 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4876cbf700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f487006e1d0 0x7f48700ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48764be700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487006ad70 0x7f4870141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 -- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f487006e1d0 msgr2=0x7f48700ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f487006e1d0 0x7f48700ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 -- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487006ad70 msgr2=0x7f4870141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487006ad70 0x7f4870141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 -- 172.21.15.64:0/890938597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009420 con 0x7f487006b210 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f4875cbd700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 0x7f4870153e30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f486000c7e0 tx=0x7f4860007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.520+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860005b30 con 0x7f487006b210 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4870154a10 con 0x7f487006b210 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4860005c90 con 0x7f487006b210 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860007300 con 0x7f487006b210 2023-09-27T23:29:23.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4870154fd0 con 0x7f487006b210 2023-09-27T23:29:23.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487004be40 con 0x7f487006b210 2023-09-27T23:29:23.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f486001b3b0 con 0x7f487006b210 2023-09-27T23:29:23.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48667fc700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4854060a70 0x7f4854062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.524+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f486007c670 con 0x7f487006b210 2023-09-27T23:29:23.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.528+0000 7f48764be700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4854060a70 0x7f4854062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.528+0000 7f48764be700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4854060a70 0x7f4854062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f486c0093f0 tx=0x7f486c017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.536+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f486004c6a0 con 0x7f487006b210 2023-09-27T23:29:23.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.752+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f487006bb80 con 0x7f487006b210 2023-09-27T23:29:23.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.752+0000 7f48667fc700 1 -- 172.21.15.64:0/890938597 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1089 (secure 0 0 0) 0x7f4860001d60 con 0x7f487006b210 2023-09-27T23:29:23.759 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:23.759 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":962,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":77},{"state_name":"active+clean","count":28}],"num_pgs":105,"num_pools":4,"num_objects":7,"data_bytes":590460,"bytes_used":204066816,"bytes_avail":772890046464,"bytes_total":773094113280,"unknown_pgs_ratio":0.73333334922790527,"write_bytes_sec":473,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":27,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":6610}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-09-27T23:28:15.519432+0000","services":{}},"progress_events":{}} 2023-09-27T23:29:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4854060a70 msgr2=0x7f4854062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4854060a70 0x7f4854062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f486c0093f0 tx=0x7f486c017000 comp rx=0 tx=0).stop 2023-09-27T23:29:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 msgr2=0x7f4870153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 0x7f4870153e30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f486000c7e0 tx=0x7f4860007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 shutdown_connections 2023-09-27T23:29:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4854060a70 0x7f4854062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f487006e1d0 0x7f48700ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487006b210 0x7f4870153e30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 --2- 172.21.15.64:0/890938597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487006ad70 0x7f4870141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 >> 172.21.15.64:0/890938597 conn(0x7f4870067bb0 msgr2=0x7f487015b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 shutdown_connections 2023-09-27T23:29:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.756+0000 7f48774c0700 1 -- 172.21.15.64:0/890938597 wait complete. 2023-09-27T23:29:23.773 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:29:23.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.876+0000 7f008cf25700 1 Processor -- start 2023-09-27T23:29:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.876+0000 7f008cf25700 1 -- start start 2023-09-27T23:29:23.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0088155350 0x7f0088159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00880ab960 0x7f008806b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00880b0080 con 0x7f0088155720 2023-09-27T23:29:23.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008815a010 con 0x7f00880ab960 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008806e740 con 0x7f0088155350 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0086d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0088155720 0x7f008806d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51150/0 (socket says 172.21.15.64:51150) 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0086d9d700 1 -- 172.21.15.64:0/3001509803 learned_addr learned my addr 172.21.15.64:0/3001509803 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f007ffff700 1 --2- 172.21.15.64:0/3001509803 >> v2:172.21.15.64:3300/0 conn(0x7f00880ab960 0x7f008806b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008659c700 1 --2- 172.21.15.64:0/3001509803 >> v2:172.21.15.123:3300/0 conn(0x7f0088155350 0x7f0088159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0085d9b700 1 -- 172.21.15.64:0/3001509803 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176420385 0 0) 0x7f00880b0080 con 0x7f0088155720 2023-09-27T23:29:23.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0085d9b700 1 -- 172.21.15.64:0/3001509803 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0074003570 con 0x7f0088155720 2023-09-27T23:29:23.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f007ffff700 1 -- 172.21.15.64:0/3001509803 >> v2:172.21.15.123:3300/0 conn(0x7f0088155350 msgr2=0x7f0088159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f007ffff700 1 --2- 172.21.15.64:0/3001509803 >> v2:172.21.15.123:3300/0 conn(0x7f0088155350 0x7f0088159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f007ffff700 1 -- 172.21.15.64:0/3001509803 >> v1:172.21.15.64:6789/0 conn(0x7f0088155720 legacy=0x7f008806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f007ffff700 1 -- 172.21.15.64:0/3001509803 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880af470 con 0x7f00880ab960 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f007ffff700 1 --2- 172.21.15.64:0/3001509803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008806b1a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f008806afa0 tx=0x7f007000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5bc670131671fb server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0085d9b700 1 -- 172.21.15.64:0/3001509803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000aea0 con 0x7f00880ab960 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0085d9b700 1 -- 172.21.15.64:0/3001509803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007000c970 con 0x7f00880ab960 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f0085d9b700 1 -- 172.21.15.64:0/3001509803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000cd10 con 0x7f00880ab960 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- 172.21.15.64:0/3001509803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 msgr2=0x7f008806b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 --2- 172.21.15.64:0/3001509803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008806b1a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f008806afa0 tx=0x7f007000a330 comp rx=0 tx=0).stop 2023-09-27T23:29:23.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- 172.21.15.64:0/3001509803 shutdown_connections 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 --2- 172.21.15.64:0/3001509803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008806b1a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 --2- 172.21.15.64:0/3001509803 >> v2:172.21.15.123:3300/0 conn(0x7f0088155350 0x7f0088159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- 172.21.15.64:0/3001509803 >> 172.21.15.64:0/3001509803 conn(0x7f0088067410 msgr2=0x7f00880677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- 172.21.15.64:0/3001509803 shutdown_connections 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.880+0000 7f008cf25700 1 -- 172.21.15.64:0/3001509803 wait complete. 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 Processor -- start 2023-09-27T23:29:23.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 -- start start 2023-09-27T23:29:23.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008814f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0088155350 0x7f00880a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088155720 0x7f0088156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880a5ce0 con 0x7f00880ab960 2023-09-27T23:29:23.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0088166fb0 con 0x7f0088155720 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008cf25700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00880aaf60 con 0x7f0088155350 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008814f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f007ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0088155350 0x7f00880a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008814f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40086/0 (socket says 172.21.15.64:40086) 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 -- 172.21.15.64:0/3126107339 learned_addr learned my addr 172.21.15.64:0/3126107339 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f0086d9d700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088155720 0x7f0088156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 -- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0088155350 msgr2=0x7f00880a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0088155350 0x7f00880a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 -- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088155720 msgr2=0x7f0088156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:23.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088155720 0x7f0088156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.884+0000 7f008659c700 1 -- 172.21.15.64:0/3126107339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007000a5d0 con 0x7f00880ab960 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f008659c700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008814f1c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f007800ecb0 tx=0x7f00780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800c850 con 0x7f00880ab960 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007800c9d0 con 0x7f00880ab960 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00880a5440 con 0x7f00880ab960 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800cd90 con 0x7f00880ab960 2023-09-27T23:29:23.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00880a5a00 con 0x7f00880ab960 2023-09-27T23:29:23.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0088071120 con 0x7f00880ab960 2023-09-27T23:29:23.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0078013680 con 0x7f00880ab960 2023-09-27T23:29:23.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007e7fc700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0068060a70 0x7f0068062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:23.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f007807d350 con 0x7f00880ab960 2023-09-27T23:29:23.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.888+0000 7f007ffff700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0068060a70 0x7f0068062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:23.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.892+0000 7f007ffff700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0068060a70 0x7f0068062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f007000a370 tx=0x7f007001a000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:23.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:23.896+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f007804d380 con 0x7f00880ab960 2023-09-27T23:29:24.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.056+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00880af750 con 0x7f0068060a70 2023-09-27T23:29:24.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.072+0000 7f007e7fc700 1 -- 172.21.15.64:0/3126107339 <== mgr.5816 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295373 (secure 0 0 0) 0x7f00880af750 con 0x7f0068060a70 2023-09-27T23:29:24.082 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0068060a70 msgr2=0x7f0068062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0068060a70 0x7f0068062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f007000a370 tx=0x7f007001a000 comp rx=0 tx=0).stop 2023-09-27T23:29:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 msgr2=0x7f008814f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008814f1c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f007800ecb0 tx=0x7f00780091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 shutdown_connections 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0068060a70 0x7f0068062f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088155720 0x7f0088156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0088155350 0x7f00880a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 --2- 172.21.15.64:0/3126107339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00880ab960 0x7f008814f1c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 >> 172.21.15.64:0/3126107339 conn(0x7f0088067410 msgr2=0x7f00881523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 shutdown_connections 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:24.076+0000 7f008cf25700 1 -- 172.21.15.64:0/3126107339 wait complete. 2023-09-27T23:29:24.085 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:24.097 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:22.633866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":248,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":199284,"kb_used_data":9012,"kb_used_omap":0,"kb_used_meta":190208,"kb_avail":754775436,"statfs":{"total":773094113280,"available":772890046464,"internally_reserved":0,"allocated":9228288,"data_stored":5264169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":194772992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.323067"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746666+0000","last_change":"2023-09-27T23:29:19.735623+0000","last_active":"2023-09-27T23:29:19.746666+0000","last_peered":"2023-09-27T23:29:19.746666+0000","last_clean":"2023-09-27T23:29:19.746666+0000","last_became_active":"2023-09-27T23:29:19.735239+0000","last_became_peered":"2023-09-27T23:29:19.735239+0000","last_unstale":"2023-09-27T23:29:19.746666+0000","last_undegraded":"2023-09-27T23:29:19.746666+0000","last_fullsized":"2023-09-27T23:29:19.746666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:50:25.868487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742588+0000","last_change":"2023-09-27T23:29:19.731017+0000","last_active":"2023-09-27T23:29:19.742588+0000","last_peered":"2023-09-27T23:29:19.742588+0000","last_clean":"2023-09-27T23:29:19.742588+0000","last_became_active":"2023-09-27T23:29:19.729277+0000","last_became_peered":"2023-09-27T23:29:19.729277+0000","last_unstale":"2023-09-27T23:29:19.742588+0000","last_undegraded":"2023-09-27T23:29:19.742588+0000","last_fullsized":"2023-09-27T23:29:19.742588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:43.879878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742363+0000","last_change":"2023-09-27T23:29:19.729793+0000","last_active":"2023-09-27T23:29:19.742363+0000","last_peered":"2023-09-27T23:29:19.742363+0000","last_clean":"2023-09-27T23:29:19.742363+0000","last_became_active":"2023-09-27T23:29:19.729183+0000","last_became_peered":"2023-09-27T23:29:19.729183+0000","last_unstale":"2023-09-27T23:29:19.742363+0000","last_undegraded":"2023-09-27T23:29:19.742363+0000","last_fullsized":"2023-09-27T23:29:19.742363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:09.106318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742724+0000","last_change":"2023-09-27T23:29:19.731251+0000","last_active":"2023-09-27T23:29:19.742724+0000","last_peered":"2023-09-27T23:29:19.742724+0000","last_clean":"2023-09-27T23:29:19.742724+0000","last_became_active":"2023-09-27T23:29:19.729904+0000","last_became_peered":"2023-09-27T23:29:19.729904+0000","last_unstale":"2023-09-27T23:29:19.742724+0000","last_undegraded":"2023-09-27T23:29:19.742724+0000","last_fullsized":"2023-09-27T23:29:19.742724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:41.029315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742755+0000","last_change":"2023-09-27T23:29:19.731464+0000","last_active":"2023-09-27T23:29:19.742755+0000","last_peered":"2023-09-27T23:29:19.742755+0000","last_clean":"2023-09-27T23:29:19.742755+0000","last_became_active":"2023-09-27T23:29:19.729102+0000","last_became_peered":"2023-09-27T23:29:19.729102+0000","last_unstale":"2023-09-27T23:29:19.742755+0000","last_undegraded":"2023-09-27T23:29:19.742755+0000","last_fullsized":"2023-09-27T23:29:19.742755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:30:18.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746307+0000","last_change":"2023-09-27T23:29:19.731873+0000","last_active":"2023-09-27T23:29:19.746307+0000","last_peered":"2023-09-27T23:29:19.746307+0000","last_clean":"2023-09-27T23:29:19.746307+0000","last_became_active":"2023-09-27T23:29:19.730797+0000","last_became_peered":"2023-09-27T23:29:19.730797+0000","last_unstale":"2023-09-27T23:29:19.746307+0000","last_undegraded":"2023-09-27T23:29:19.746307+0000","last_fullsized":"2023-09-27T23:29:19.746307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:24:01.984490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742203+0000","last_change":"2023-09-27T23:29:19.355491+0000","last_active":"2023-09-27T23:29:19.742203+0000","last_peered":"2023-09-27T23:29:19.742203+0000","last_clean":"2023-09-27T23:29:19.742203+0000","last_became_active":"2023-09-27T23:29:19.355192+0000","last_became_peered":"2023-09-27T23:29:19.355192+0000","last_unstale":"2023-09-27T23:29:19.742203+0000","last_undegraded":"2023-09-27T23:29:19.742203+0000","last_fullsized":"2023-09-27T23:29:19.742203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:54:30.443743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742614+0000","last_change":"2023-09-27T23:29:19.356245+0000","last_active":"2023-09-27T23:29:19.742614+0000","last_peered":"2023-09-27T23:29:19.742614+0000","last_clean":"2023-09-27T23:29:19.742614+0000","last_became_active":"2023-09-27T23:29:19.355967+0000","last_became_peered":"2023-09-27T23:29:19.355967+0000","last_unstale":"2023-09-27T23:29:19.742614+0000","last_undegraded":"2023-09-27T23:29:19.742614+0000","last_fullsized":"2023-09-27T23:29:19.742614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:19.665630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.719234+0000","last_change":"2023-09-27T23:29:19.352763+0000","last_active":"2023-09-27T23:29:19.719234+0000","last_peered":"2023-09-27T23:29:19.719234+0000","last_clean":"2023-09-27T23:29:19.719234+0000","last_became_active":"2023-09-27T23:29:19.352450+0000","last_became_peered":"2023-09-27T23:29:19.352450+0000","last_unstale":"2023-09-27T23:29:19.719234+0000","last_undegraded":"2023-09-27T23:29:19.719234+0000","last_fullsized":"2023-09-27T23:29:19.719234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:09.789659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746329+0000","last_change":"2023-09-27T23:29:19.734575+0000","last_active":"2023-09-27T23:29:19.746329+0000","last_peered":"2023-09-27T23:29:19.746329+0000","last_clean":"2023-09-27T23:29:19.746329+0000","last_became_active":"2023-09-27T23:29:19.730734+0000","last_became_peered":"2023-09-27T23:29:19.730734+0000","last_unstale":"2023-09-27T23:29:19.746329+0000","last_undegraded":"2023-09-27T23:29:19.746329+0000","last_fullsized":"2023-09-27T23:29:19.746329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:47.061803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716915+0000","last_change":"2023-09-27T23:29:19.351569+0000","last_active":"2023-09-27T23:29:19.716915+0000","last_peered":"2023-09-27T23:29:19.716915+0000","last_clean":"2023-09-27T23:29:19.716915+0000","last_became_active":"2023-09-27T23:29:19.351295+0000","last_became_peered":"2023-09-27T23:29:19.351295+0000","last_unstale":"2023-09-27T23:29:19.716915+0000","last_undegraded":"2023-09-27T23:29:19.716915+0000","last_fullsized":"2023-09-27T23:29:19.716915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:38.540430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742267+0000","last_change":"2023-09-27T23:29:19.359846+0000","last_active":"2023-09-27T23:29:19.742267+0000","last_peered":"2023-09-27T23:29:19.742267+0000","last_clean":"2023-09-27T23:29:19.742267+0000","last_became_active":"2023-09-27T23:29:19.359535+0000","last_became_peered":"2023-09-27T23:29:19.359535+0000","last_unstale":"2023-09-27T23:29:19.742267+0000","last_undegraded":"2023-09-27T23:29:19.742267+0000","last_fullsized":"2023-09-27T23:29:19.742267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:37.201350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742073+0000","last_change":"2023-09-27T23:29:19.729613+0000","last_active":"2023-09-27T23:29:19.742073+0000","last_peered":"2023-09-27T23:29:19.742073+0000","last_clean":"2023-09-27T23:29:19.742073+0000","last_became_active":"2023-09-27T23:29:19.729107+0000","last_became_peered":"2023-09-27T23:29:19.729107+0000","last_unstale":"2023-09-27T23:29:19.742073+0000","last_undegraded":"2023-09-27T23:29:19.742073+0000","last_fullsized":"2023-09-27T23:29:19.742073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:39:43.514982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742014+0000","last_change":"2023-09-27T23:29:19.729669+0000","last_active":"2023-09-27T23:29:19.742014+0000","last_peered":"2023-09-27T23:29:19.742014+0000","last_clean":"2023-09-27T23:29:19.742014+0000","last_became_active":"2023-09-27T23:29:19.729046+0000","last_became_peered":"2023-09-27T23:29:19.729046+0000","last_unstale":"2023-09-27T23:29:19.742014+0000","last_undegraded":"2023-09-27T23:29:19.742014+0000","last_fullsized":"2023-09-27T23:29:19.742014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:01.763686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716949+0000","last_change":"2023-09-27T23:29:19.355551+0000","last_active":"2023-09-27T23:29:19.716949+0000","last_peered":"2023-09-27T23:29:19.716949+0000","last_clean":"2023-09-27T23:29:19.716949+0000","last_became_active":"2023-09-27T23:29:19.355244+0000","last_became_peered":"2023-09-27T23:29:19.355244+0000","last_unstale":"2023-09-27T23:29:19.716949+0000","last_undegraded":"2023-09-27T23:29:19.716949+0000","last_fullsized":"2023-09-27T23:29:19.716949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:09.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742096+0000","last_change":"2023-09-27T23:29:19.355604+0000","last_active":"2023-09-27T23:29:19.742096+0000","last_peered":"2023-09-27T23:29:19.742096+0000","last_clean":"2023-09-27T23:29:19.742096+0000","last_became_active":"2023-09-27T23:29:19.355207+0000","last_became_peered":"2023-09-27T23:29:19.355207+0000","last_unstale":"2023-09-27T23:29:19.742096+0000","last_undegraded":"2023-09-27T23:29:19.742096+0000","last_fullsized":"2023-09-27T23:29:19.742096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:33.460035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'161","reported_seq":376,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.360221+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:15.360221+0000","last_peered":"2023-09-27T23:29:15.360221+0000","last_clean":"2023-09-27T23:29:15.360221+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:15.360221+0000","last_undegraded":"2023-09-27T23:29:15.360221+0000","last_fullsized":"2023-09-27T23:29:15.360221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":161,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358364+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:15.358364+0000","last_peered":"2023-09-27T23:29:15.358364+0000","last_clean":"2023-09-27T23:29:15.358364+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:15.358364+0000","last_undegraded":"2023-09-27T23:29:15.358364+0000","last_fullsized":"2023-09-27T23:29:15.358364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359761+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:15.359761+0000","last_peered":"2023-09-27T23:29:15.359761+0000","last_clean":"2023-09-27T23:29:15.359761+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:15.359761+0000","last_undegraded":"2023-09-27T23:29:15.359761+0000","last_fullsized":"2023-09-27T23:29:15.359761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.119882+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:15.119882+0000","last_peered":"2023-09-27T23:29:15.119882+0000","last_clean":"2023-09-27T23:29:15.119882+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:15.119882+0000","last_undegraded":"2023-09-27T23:29:15.119882+0000","last_fullsized":"2023-09-27T23:29:15.119882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.717030+0000","last_change":"2023-09-27T23:29:19.351439+0000","last_active":"2023-09-27T23:29:19.717030+0000","last_peered":"2023-09-27T23:29:19.717030+0000","last_clean":"2023-09-27T23:29:19.717030+0000","last_became_active":"2023-09-27T23:29:19.350793+0000","last_became_peered":"2023-09-27T23:29:19.350793+0000","last_unstale":"2023-09-27T23:29:19.717030+0000","last_undegraded":"2023-09-27T23:29:19.717030+0000","last_fullsized":"2023-09-27T23:29:19.717030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:44:07.068721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.358360+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:15.358360+0000","last_peered":"2023-09-27T23:29:15.358360+0000","last_clean":"2023-09-27T23:29:15.358360+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:15.358360+0000","last_undegraded":"2023-09-27T23:29:15.358360+0000","last_fullsized":"2023-09-27T23:29:15.358360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355205+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:15.355205+0000","last_peered":"2023-09-27T23:29:15.355205+0000","last_clean":"2023-09-27T23:29:15.355205+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:15.355205+0000","last_undegraded":"2023-09-27T23:29:15.355205+0000","last_fullsized":"2023-09-27T23:29:15.355205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742337+0000","last_change":"2023-09-27T23:29:19.351252+0000","last_active":"2023-09-27T23:29:19.742337+0000","last_peered":"2023-09-27T23:29:19.742337+0000","last_clean":"2023-09-27T23:29:19.742337+0000","last_became_active":"2023-09-27T23:29:19.350970+0000","last_became_peered":"2023-09-27T23:29:19.350970+0000","last_unstale":"2023-09-27T23:29:19.742337+0000","last_undegraded":"2023-09-27T23:29:19.742337+0000","last_fullsized":"2023-09-27T23:29:19.742337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:02.449177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.359957+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:15.359957+0000","last_peered":"2023-09-27T23:29:15.359957+0000","last_clean":"2023-09-27T23:29:15.359957+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:15.359957+0000","last_undegraded":"2023-09-27T23:29:15.359957+0000","last_fullsized":"2023-09-27T23:29:15.359957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:18.299206+0000","last_change":"2023-09-27T23:29:18.299206+0000","last_active":"2023-09-27T23:29:18.299206+0000","last_peered":"2023-09-27T23:29:18.299206+0000","last_clean":"2023-09-27T23:29:18.299206+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:18.299206+0000","last_undegraded":"2023-09-27T23:29:18.299206+0000","last_fullsized":"2023-09-27T23:29:18.299206+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-27T23:29:15.355216+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:15.355216+0000","last_peered":"2023-09-27T23:29:15.355216+0000","last_clean":"2023-09-27T23:29:15.355216+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:15.355216+0000","last_undegraded":"2023-09-27T23:29:15.355216+0000","last_fullsized":"2023-09-27T23:29:15.355216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:19.306227+0000","last_change":"2023-09-27T23:29:19.306227+0000","last_active":"2023-09-27T23:29:19.306227+0000","last_peered":"2023-09-27T23:29:19.306227+0000","last_clean":"2023-09-27T23:29:19.306227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:19.306227+0000","last_undegraded":"2023-09-27T23:29:19.306227+0000","last_fullsized":"2023-09-27T23:29:19.306227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742021+0000","last_change":"2023-09-27T23:29:19.359807+0000","last_active":"2023-09-27T23:29:19.742021+0000","last_peered":"2023-09-27T23:29:19.742021+0000","last_clean":"2023-09-27T23:29:19.742021+0000","last_became_active":"2023-09-27T23:29:19.359497+0000","last_became_peered":"2023-09-27T23:29:19.359497+0000","last_unstale":"2023-09-27T23:29:19.742021+0000","last_undegraded":"2023-09-27T23:29:19.742021+0000","last_fullsized":"2023-09-27T23:29:19.742021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:12:13.225733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.714700+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:19.714700+0000","last_peered":"2023-09-27T23:29:19.714700+0000","last_clean":"2023-09-27T23:29:19.714700+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:19.714700+0000","last_undegraded":"2023-09-27T23:29:19.714700+0000","last_fullsized":"2023-09-27T23:29:19.714700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25500,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":23936,"kb_avail":94346340,"statfs":{"total":96636764160,"available":96610652160,"internally_reserved":0,"allocated":1593344,"data_stored":1098094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24510464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.749},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.878},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.052}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.016},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.121}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0820000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21264,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350576,"statfs":{"total":96636764160,"available":96614989824,"internally_reserved":0,"allocated":1056768,"data_stored":535629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":0.84099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.874}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.753}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":1.244},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.284},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.232}]}]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31192,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":30208,"kb_avail":94340648,"statfs":{"total":96636764160,"available":96604823552,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30932992},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":1.034},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.2330000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":0.747},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.113},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.081}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":0.94999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.196}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.262}]}]},{"osd":4,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21272,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20288,"kb_avail":94350568,"statfs":{"total":96636764160,"available":96614981632,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20774912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.091},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.89700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":1.3169999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":0.98399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.135}]}]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21848,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":94349992,"statfs":{"total":96636764160,"available":96614391808,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.86599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.238},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.357}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":0.999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.321}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.0620000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20312,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":19328,"kb_avail":94351528,"statfs":{"total":96636764160,"available":96615964672,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19791872},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":1.018}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":1.272},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.214},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.048}]}]},{"osd":1,"up_from":14,"seq":60129542336,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":34560,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":999424,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35389440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.621},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":0.496}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.91000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":0.95299999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22352,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349488,"statfs":{"total":96636764160,"available":96613875712,"internally_reserved":0,"allocated":1581056,"data_stored":1091816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.62},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.435},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.022},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":0.80200000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.083}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.127},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:27.105 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:27.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 Processor -- start 2023-09-27T23:29:27.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 -- start start 2023-09-27T23:29:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1050159200 0x7f10500c8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10501595d0 0x7f105006b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105015a3e0 con 0x7f10500aba30 2023-09-27T23:29:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10500c8a90 con 0x7f1050159200 2023-09-27T23:29:27.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f105829c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500b0bf0 con 0x7f10501595d0 2023-09-27T23:29:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1056839700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f10500aba30 0x7f105015cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54806/0 (socket says 172.21.15.64:54806) 2023-09-27T23:29:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1056839700 1 -- 172.21.15.64:0/3943139602 learned_addr learned my addr 172.21.15.64:0/3943139602 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055837700 1 --2- 172.21.15.64:0/3943139602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10501595d0 0x7f105006b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1056038700 1 --2- 172.21.15.64:0/3943139602 >> v2:172.21.15.123:3300/0 conn(0x7f1050159200 0x7f10500c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055036700 1 -- 172.21.15.64:0/3943139602 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965413733 0 0) 0x7f105015a3e0 con 0x7f10500aba30 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055036700 1 -- 172.21.15.64:0/3943139602 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1038003570 con 0x7f10500aba30 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055837700 1 -- 172.21.15.64:0/3943139602 >> v2:172.21.15.123:3300/0 conn(0x7f1050159200 msgr2=0x7f10500c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055837700 1 --2- 172.21.15.64:0/3943139602 >> v2:172.21.15.123:3300/0 conn(0x7f1050159200 0x7f10500c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055837700 1 -- 172.21.15.64:0/3943139602 >> v1:172.21.15.64:6789/0 conn(0x7f10500aba30 legacy=0x7f105015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.216+0000 7f1055837700 1 -- 172.21.15.64:0/3943139602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050159ab0 con 0x7f10501595d0 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f1055837700 1 --2- 172.21.15.64:0/3943139602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10501595d0 0x7f105006b1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f104c0098e0 tx=0x7f104c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d172d2a84d08fbf server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:27.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f1055036700 1 -- 172.21.15.64:0/3943139602 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c011630 con 0x7f10501595d0 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f1055036700 1 -- 172.21.15.64:0/3943139602 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c011790 con 0x7f10501595d0 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f1055036700 1 -- 172.21.15.64:0/3943139602 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c011b30 con 0x7f10501595d0 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 -- 172.21.15.64:0/3943139602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10501595d0 msgr2=0x7f105006b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 --2- 172.21.15.64:0/3943139602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10501595d0 0x7f105006b1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f104c0098e0 tx=0x7f104c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 -- 172.21.15.64:0/3943139602 shutdown_connections 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 --2- 172.21.15.64:0/3943139602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10501595d0 0x7f105006b1a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 --2- 172.21.15.64:0/3943139602 >> v2:172.21.15.123:3300/0 conn(0x7f1050159200 0x7f10500c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 -- 172.21.15.64:0/3943139602 >> 172.21.15.64:0/3943139602 conn(0x7f1050067410 msgr2=0x7f10500677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:27.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 -- 172.21.15.64:0/3943139602 shutdown_connections 2023-09-27T23:29:27.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 -- 172.21.15.64:0/3943139602 wait complete. 2023-09-27T23:29:27.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 Processor -- start 2023-09-27T23:29:27.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.220+0000 7f105829c700 1 -- start start 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10500aba30 0x7f105013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1050159200 0x7f10500aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 0x7f1050154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500aa520 con 0x7f10501595d0 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500ad060 con 0x7f10500aba30 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500acd80 con 0x7f1050159200 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 0x7f1050154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 0x7f1050154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53646/0 (socket says 172.21.15.64:53646) 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 -- 172.21.15.64:0/348727987 learned_addr learned my addr 172.21.15.64:0/348727987 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1055837700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1050159200 0x7f10500aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056038700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10500aba30 0x7f105013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 -- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1050159200 msgr2=0x7f10500aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1050159200 0x7f10500aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 -- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10500aba30 msgr2=0x7f105013d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10500aba30 0x7f105013d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 -- 172.21.15.64:0/348727987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009420 con 0x7f10501595d0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f1056839700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 0x7f1050154250 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f104800cca0 tx=0x7f1048001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800a8d0 con 0x7f10501595d0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104800aa30 con 0x7f10501595d0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10500a9ed0 con 0x7f10501595d0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800adb0 con 0x7f10501595d0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105013d8a0 con 0x7f10501595d0 2023-09-27T23:29:27.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.224+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10500aa690 con 0x7f10501595d0 2023-09-27T23:29:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.228+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1048007300 con 0x7f10501595d0 2023-09-27T23:29:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.228+0000 7f10467fc700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f103c060a70 0x7f103c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.228+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f104807d580 con 0x7f10501595d0 2023-09-27T23:29:27.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.228+0000 7f1056038700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f103c060a70 0x7f103c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.228+0000 7f1056038700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f103c060a70 0x7f103c062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1040009db0 tx=0x7f10400091a0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:27.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.236+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f104804d500 con 0x7f10501595d0 2023-09-27T23:29:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.400+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1050071120 con 0x7f103c060a70 2023-09-27T23:29:27.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.420+0000 7f10467fc700 1 -- 172.21.15.64:0/348727987 <== mgr.5816 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307402 (secure 0 0 0) 0x7f1050071120 con 0x7f103c060a70 2023-09-27T23:29:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f103c060a70 msgr2=0x7f103c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f103c060a70 0x7f103c062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1040009db0 tx=0x7f10400091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 msgr2=0x7f1050154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 0x7f1050154250 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f104800cca0 tx=0x7f1048001ae0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 shutdown_connections 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f103c060a70 0x7f103c062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10501595d0 0x7f1050154250 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1050159200 0x7f10500aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 --2- 172.21.15.64:0/348727987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10500aba30 0x7f105013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 >> 172.21.15.64:0/348727987 conn(0x7f1050067410 msgr2=0x7f10500c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 shutdown_connections 2023-09-27T23:29:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.424+0000 7f105829c700 1 -- 172.21.15.64:0/348727987 wait complete. 2023-09-27T23:29:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:27.445 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:29:26.635387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":200712,"kb_used_data":9416,"kb_used_omap":0,"kb_used_meta":191232,"kb_avail":754774008,"statfs":{"total":773094113280,"available":772888584192,"internally_reserved":0,"allocated":9641984,"data_stored":5469644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":195821568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.310771"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.745424+0000","last_change":"2023-09-27T23:29:19.737178+0000","last_active":"2023-09-27T23:29:19.745424+0000","last_peered":"2023-09-27T23:29:19.745424+0000","last_clean":"2023-09-27T23:29:19.745424+0000","last_became_active":"2023-09-27T23:29:19.736720+0000","last_became_peered":"2023-09-27T23:29:19.736720+0000","last_unstale":"2023-09-27T23:29:19.745424+0000","last_undegraded":"2023-09-27T23:29:19.745424+0000","last_fullsized":"2023-09-27T23:29:19.745424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:09.106318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746666+0000","last_change":"2023-09-27T23:29:19.735623+0000","last_active":"2023-09-27T23:29:19.746666+0000","last_peered":"2023-09-27T23:29:19.746666+0000","last_clean":"2023-09-27T23:29:19.746666+0000","last_became_active":"2023-09-27T23:29:19.735239+0000","last_became_peered":"2023-09-27T23:29:19.735239+0000","last_unstale":"2023-09-27T23:29:19.746666+0000","last_undegraded":"2023-09-27T23:29:19.746666+0000","last_fullsized":"2023-09-27T23:29:19.746666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:50:25.868487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.726920+0000","last_change":"2023-09-27T23:29:19.718319+0000","last_active":"2023-09-27T23:29:19.726920+0000","last_peered":"2023-09-27T23:29:19.726920+0000","last_clean":"2023-09-27T23:29:19.726920+0000","last_became_active":"2023-09-27T23:29:19.717322+0000","last_became_peered":"2023-09-27T23:29:19.717322+0000","last_unstale":"2023-09-27T23:29:19.726920+0000","last_undegraded":"2023-09-27T23:29:19.726920+0000","last_fullsized":"2023-09-27T23:29:19.726920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:25:16.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.739513+0000","last_change":"2023-09-27T23:29:19.733296+0000","last_active":"2023-09-27T23:29:19.739513+0000","last_peered":"2023-09-27T23:29:19.739513+0000","last_clean":"2023-09-27T23:29:19.739513+0000","last_became_active":"2023-09-27T23:29:19.733079+0000","last_became_peered":"2023-09-27T23:29:19.733079+0000","last_unstale":"2023-09-27T23:29:19.739513+0000","last_undegraded":"2023-09-27T23:29:19.739513+0000","last_fullsized":"2023-09-27T23:29:19.739513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:31:47.729705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.734289+0000","last_change":"2023-09-27T23:29:19.725644+0000","last_active":"2023-09-27T23:29:19.734289+0000","last_peered":"2023-09-27T23:29:19.734289+0000","last_clean":"2023-09-27T23:29:19.734289+0000","last_became_active":"2023-09-27T23:29:19.725299+0000","last_became_peered":"2023-09-27T23:29:19.725299+0000","last_unstale":"2023-09-27T23:29:19.734289+0000","last_undegraded":"2023-09-27T23:29:19.734289+0000","last_fullsized":"2023-09-27T23:29:19.734289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:19:12.842090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742588+0000","last_change":"2023-09-27T23:29:19.731017+0000","last_active":"2023-09-27T23:29:19.742588+0000","last_peered":"2023-09-27T23:29:19.742588+0000","last_clean":"2023-09-27T23:29:19.742588+0000","last_became_active":"2023-09-27T23:29:19.729277+0000","last_became_peered":"2023-09-27T23:29:19.729277+0000","last_unstale":"2023-09-27T23:29:19.742588+0000","last_undegraded":"2023-09-27T23:29:19.742588+0000","last_fullsized":"2023-09-27T23:29:19.742588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:43.879878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716780+0000","last_change":"2023-09-27T23:29:19.712994+0000","last_active":"2023-09-27T23:29:19.716780+0000","last_peered":"2023-09-27T23:29:19.716780+0000","last_clean":"2023-09-27T23:29:19.716780+0000","last_became_active":"2023-09-27T23:29:19.712691+0000","last_became_peered":"2023-09-27T23:29:19.712691+0000","last_unstale":"2023-09-27T23:29:19.716780+0000","last_undegraded":"2023-09-27T23:29:19.716780+0000","last_fullsized":"2023-09-27T23:29:19.716780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:47.121344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742363+0000","last_change":"2023-09-27T23:29:19.729793+0000","last_active":"2023-09-27T23:29:19.742363+0000","last_peered":"2023-09-27T23:29:19.742363+0000","last_clean":"2023-09-27T23:29:19.742363+0000","last_became_active":"2023-09-27T23:29:19.729183+0000","last_became_peered":"2023-09-27T23:29:19.729183+0000","last_unstale":"2023-09-27T23:29:19.742363+0000","last_undegraded":"2023-09-27T23:29:19.742363+0000","last_fullsized":"2023-09-27T23:29:19.742363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:09.106318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.727746+0000","last_change":"2023-09-27T23:29:19.719883+0000","last_active":"2023-09-27T23:29:19.727746+0000","last_peered":"2023-09-27T23:29:19.727746+0000","last_clean":"2023-09-27T23:29:19.727746+0000","last_became_active":"2023-09-27T23:29:19.719224+0000","last_became_peered":"2023-09-27T23:29:19.719224+0000","last_unstale":"2023-09-27T23:29:19.727746+0000","last_undegraded":"2023-09-27T23:29:19.727746+0000","last_fullsized":"2023-09-27T23:29:19.727746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:54:31.450763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.732112+0000","last_change":"2023-09-27T23:29:19.724805+0000","last_active":"2023-09-27T23:29:19.732112+0000","last_peered":"2023-09-27T23:29:19.732112+0000","last_clean":"2023-09-27T23:29:19.732112+0000","last_became_active":"2023-09-27T23:29:19.724322+0000","last_became_peered":"2023-09-27T23:29:19.724322+0000","last_unstale":"2023-09-27T23:29:19.732112+0000","last_undegraded":"2023-09-27T23:29:19.732112+0000","last_fullsized":"2023-09-27T23:29:19.732112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:30:34.103109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.745554+0000","last_change":"2023-09-27T23:29:19.737784+0000","last_active":"2023-09-27T23:29:19.745554+0000","last_peered":"2023-09-27T23:29:19.745554+0000","last_clean":"2023-09-27T23:29:19.745554+0000","last_became_active":"2023-09-27T23:29:19.736489+0000","last_became_peered":"2023-09-27T23:29:19.736489+0000","last_unstale":"2023-09-27T23:29:19.745554+0000","last_undegraded":"2023-09-27T23:29:19.745554+0000","last_fullsized":"2023-09-27T23:29:19.745554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:35.928907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.728199+0000","last_change":"2023-09-27T23:29:19.721389+0000","last_active":"2023-09-27T23:29:19.728199+0000","last_peered":"2023-09-27T23:29:19.728199+0000","last_clean":"2023-09-27T23:29:19.728199+0000","last_became_active":"2023-09-27T23:29:19.716231+0000","last_became_peered":"2023-09-27T23:29:19.716231+0000","last_unstale":"2023-09-27T23:29:19.728199+0000","last_undegraded":"2023-09-27T23:29:19.728199+0000","last_fullsized":"2023-09-27T23:29:19.728199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:05:00.505579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.733948+0000","last_change":"2023-09-27T23:29:19.728964+0000","last_active":"2023-09-27T23:29:19.733948+0000","last_peered":"2023-09-27T23:29:19.733948+0000","last_clean":"2023-09-27T23:29:19.733948+0000","last_became_active":"2023-09-27T23:29:19.728680+0000","last_became_peered":"2023-09-27T23:29:19.728680+0000","last_unstale":"2023-09-27T23:29:19.733948+0000","last_undegraded":"2023-09-27T23:29:19.733948+0000","last_fullsized":"2023-09-27T23:29:19.733948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:14:27.072020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.734030+0000","last_change":"2023-09-27T23:29:19.721832+0000","last_active":"2023-09-27T23:29:19.734030+0000","last_peered":"2023-09-27T23:29:19.734030+0000","last_clean":"2023-09-27T23:29:19.734030+0000","last_became_active":"2023-09-27T23:29:19.718937+0000","last_became_peered":"2023-09-27T23:29:19.718937+0000","last_unstale":"2023-09-27T23:29:19.734030+0000","last_undegraded":"2023-09-27T23:29:19.734030+0000","last_fullsized":"2023-09-27T23:29:19.734030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:10.181312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.721094+0000","last_change":"2023-09-27T23:29:19.717129+0000","last_active":"2023-09-27T23:29:19.721094+0000","last_peered":"2023-09-27T23:29:19.721094+0000","last_clean":"2023-09-27T23:29:19.721094+0000","last_became_active":"2023-09-27T23:29:19.716907+0000","last_became_peered":"2023-09-27T23:29:19.716907+0000","last_unstale":"2023-09-27T23:29:19.721094+0000","last_undegraded":"2023-09-27T23:29:19.721094+0000","last_fullsized":"2023-09-27T23:29:19.721094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:56:01.277420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742233+0000","last_change":"2023-09-27T23:29:19.733956+0000","last_active":"2023-09-27T23:29:19.742233+0000","last_peered":"2023-09-27T23:29:19.742233+0000","last_clean":"2023-09-27T23:29:19.742233+0000","last_became_active":"2023-09-27T23:29:19.733669+0000","last_became_peered":"2023-09-27T23:29:19.733669+0000","last_unstale":"2023-09-27T23:29:19.742233+0000","last_undegraded":"2023-09-27T23:29:19.742233+0000","last_fullsized":"2023-09-27T23:29:19.742233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:50:17.569308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.734026+0000","last_change":"2023-09-27T23:29:19.720923+0000","last_active":"2023-09-27T23:29:19.734026+0000","last_peered":"2023-09-27T23:29:19.734026+0000","last_clean":"2023-09-27T23:29:19.734026+0000","last_became_active":"2023-09-27T23:29:19.719308+0000","last_became_peered":"2023-09-27T23:29:19.719308+0000","last_unstale":"2023-09-27T23:29:19.734026+0000","last_undegraded":"2023-09-27T23:29:19.734026+0000","last_fullsized":"2023-09-27T23:29:19.734026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:20.672650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.739151+0000","last_change":"2023-09-27T23:29:19.730296+0000","last_active":"2023-09-27T23:29:19.739151+0000","last_peered":"2023-09-27T23:29:19.739151+0000","last_clean":"2023-09-27T23:29:19.739151+0000","last_became_active":"2023-09-27T23:29:19.729822+0000","last_became_peered":"2023-09-27T23:29:19.729822+0000","last_unstale":"2023-09-27T23:29:19.739151+0000","last_undegraded":"2023-09-27T23:29:19.739151+0000","last_fullsized":"2023-09-27T23:29:19.739151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:11:13.494055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742724+0000","last_change":"2023-09-27T23:29:19.731251+0000","last_active":"2023-09-27T23:29:19.742724+0000","last_peered":"2023-09-27T23:29:19.742724+0000","last_clean":"2023-09-27T23:29:19.742724+0000","last_became_active":"2023-09-27T23:29:19.729904+0000","last_became_peered":"2023-09-27T23:29:19.729904+0000","last_unstale":"2023-09-27T23:29:19.742724+0000","last_undegraded":"2023-09-27T23:29:19.742724+0000","last_fullsized":"2023-09-27T23:29:19.742724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:41.029315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.725900+0000","last_change":"2023-09-27T23:29:19.719089+0000","last_active":"2023-09-27T23:29:19.725900+0000","last_peered":"2023-09-27T23:29:19.725900+0000","last_clean":"2023-09-27T23:29:19.725900+0000","last_became_active":"2023-09-27T23:29:19.718808+0000","last_became_peered":"2023-09-27T23:29:19.718808+0000","last_unstale":"2023-09-27T23:29:19.725900+0000","last_undegraded":"2023-09-27T23:29:19.725900+0000","last_fullsized":"2023-09-27T23:29:19.725900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:38.311950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.741765+0000","last_change":"2023-09-27T23:29:19.727027+0000","last_active":"2023-09-27T23:29:19.741765+0000","last_peered":"2023-09-27T23:29:19.741765+0000","last_clean":"2023-09-27T23:29:19.741765+0000","last_became_active":"2023-09-27T23:29:19.726485+0000","last_became_peered":"2023-09-27T23:29:19.726485+0000","last_unstale":"2023-09-27T23:29:19.741765+0000","last_undegraded":"2023-09-27T23:29:19.741765+0000","last_fullsized":"2023-09-27T23:29:19.741765+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:25:16.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.725839+0000","last_change":"2023-09-27T23:29:19.720715+0000","last_active":"2023-09-27T23:29:19.725839+0000","last_peered":"2023-09-27T23:29:19.725839+0000","last_clean":"2023-09-27T23:29:19.725839+0000","last_became_active":"2023-09-27T23:29:19.720407+0000","last_became_peered":"2023-09-27T23:29:19.720407+0000","last_unstale":"2023-09-27T23:29:19.725839+0000","last_undegraded":"2023-09-27T23:29:19.725839+0000","last_fullsized":"2023-09-27T23:29:19.725839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:11.465442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.737372+0000","last_change":"2023-09-27T23:29:19.724702+0000","last_active":"2023-09-27T23:29:19.737372+0000","last_peered":"2023-09-27T23:29:19.737372+0000","last_clean":"2023-09-27T23:29:19.737372+0000","last_became_active":"2023-09-27T23:29:19.724427+0000","last_became_peered":"2023-09-27T23:29:19.724427+0000","last_unstale":"2023-09-27T23:29:19.737372+0000","last_undegraded":"2023-09-27T23:29:19.737372+0000","last_fullsized":"2023-09-27T23:29:19.737372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:50:25.868487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746931+0000","last_change":"2023-09-27T23:29:19.743879+0000","last_active":"2023-09-27T23:29:19.746931+0000","last_peered":"2023-09-27T23:29:19.746931+0000","last_clean":"2023-09-27T23:29:19.746931+0000","last_became_active":"2023-09-27T23:29:19.743588+0000","last_became_peered":"2023-09-27T23:29:19.743588+0000","last_unstale":"2023-09-27T23:29:19.746931+0000","last_undegraded":"2023-09-27T23:29:19.746931+0000","last_fullsized":"2023-09-27T23:29:19.746931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:31.904400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.738950+0000","last_change":"2023-09-27T23:29:19.725589+0000","last_active":"2023-09-27T23:29:19.738950+0000","last_peered":"2023-09-27T23:29:19.738950+0000","last_clean":"2023-09-27T23:29:19.738950+0000","last_became_active":"2023-09-27T23:29:19.725314+0000","last_became_peered":"2023-09-27T23:29:19.725314+0000","last_unstale":"2023-09-27T23:29:19.738950+0000","last_undegraded":"2023-09-27T23:29:19.738950+0000","last_fullsized":"2023-09-27T23:29:19.738950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:08:49.813732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.734467+0000","last_change":"2023-09-27T23:29:19.732491+0000","last_active":"2023-09-27T23:29:19.734467+0000","last_peered":"2023-09-27T23:29:19.734467+0000","last_clean":"2023-09-27T23:29:19.734467+0000","last_became_active":"2023-09-27T23:29:19.732145+0000","last_became_peered":"2023-09-27T23:29:19.732145+0000","last_unstale":"2023-09-27T23:29:19.734467+0000","last_undegraded":"2023-09-27T23:29:19.734467+0000","last_fullsized":"2023-09-27T23:29:19.734467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:39:43.514982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.730704+0000","last_change":"2023-09-27T23:29:19.721363+0000","last_active":"2023-09-27T23:29:19.730704+0000","last_peered":"2023-09-27T23:29:19.730704+0000","last_clean":"2023-09-27T23:29:19.730704+0000","last_became_active":"2023-09-27T23:29:19.721080+0000","last_became_peered":"2023-09-27T23:29:19.721080+0000","last_unstale":"2023-09-27T23:29:19.730704+0000","last_undegraded":"2023-09-27T23:29:19.730704+0000","last_fullsized":"2023-09-27T23:29:19.730704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:50:17.569308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.725884+0000","last_change":"2023-09-27T23:29:19.720920+0000","last_active":"2023-09-27T23:29:19.725884+0000","last_peered":"2023-09-27T23:29:19.725884+0000","last_clean":"2023-09-27T23:29:19.725884+0000","last_became_active":"2023-09-27T23:29:19.719256+0000","last_became_peered":"2023-09-27T23:29:19.719256+0000","last_unstale":"2023-09-27T23:29:19.725884+0000","last_undegraded":"2023-09-27T23:29:19.725884+0000","last_fullsized":"2023-09-27T23:29:19.725884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:58:25.604977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.726600+0000","last_change":"2023-09-27T23:29:19.717888+0000","last_active":"2023-09-27T23:29:19.726600+0000","last_peered":"2023-09-27T23:29:19.726600+0000","last_clean":"2023-09-27T23:29:19.726600+0000","last_became_active":"2023-09-27T23:29:19.717619+0000","last_became_peered":"2023-09-27T23:29:19.717619+0000","last_unstale":"2023-09-27T23:29:19.726600+0000","last_undegraded":"2023-09-27T23:29:19.726600+0000","last_fullsized":"2023-09-27T23:29:19.726600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:11.465442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.734458+0000","last_change":"2023-09-27T23:29:19.732241+0000","last_active":"2023-09-27T23:29:19.734458+0000","last_peered":"2023-09-27T23:29:19.734458+0000","last_clean":"2023-09-27T23:29:19.734458+0000","last_became_active":"2023-09-27T23:29:19.731860+0000","last_became_peered":"2023-09-27T23:29:19.731860+0000","last_unstale":"2023-09-27T23:29:19.734458+0000","last_undegraded":"2023-09-27T23:29:19.734458+0000","last_fullsized":"2023-09-27T23:29:19.734458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:30:18.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.723303+0000","last_change":"2023-09-27T23:29:19.720800+0000","last_active":"2023-09-27T23:29:19.723303+0000","last_peered":"2023-09-27T23:29:19.723303+0000","last_clean":"2023-09-27T23:29:19.723303+0000","last_became_active":"2023-09-27T23:29:19.720455+0000","last_became_peered":"2023-09-27T23:29:19.720455+0000","last_unstale":"2023-09-27T23:29:19.723303+0000","last_undegraded":"2023-09-27T23:29:19.723303+0000","last_fullsized":"2023-09-27T23:29:19.723303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:14:14.614096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742755+0000","last_change":"2023-09-27T23:29:19.731464+0000","last_active":"2023-09-27T23:29:19.742755+0000","last_peered":"2023-09-27T23:29:19.742755+0000","last_clean":"2023-09-27T23:29:19.742755+0000","last_became_active":"2023-09-27T23:29:19.729102+0000","last_became_peered":"2023-09-27T23:29:19.729102+0000","last_unstale":"2023-09-27T23:29:19.742755+0000","last_undegraded":"2023-09-27T23:29:19.742755+0000","last_fullsized":"2023-09-27T23:29:19.742755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:30:18.082342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746562+0000","last_change":"2023-09-27T23:29:19.742828+0000","last_active":"2023-09-27T23:29:19.746562+0000","last_peered":"2023-09-27T23:29:19.746562+0000","last_clean":"2023-09-27T23:29:19.746562+0000","last_became_active":"2023-09-27T23:29:19.742543+0000","last_became_peered":"2023-09-27T23:29:19.742543+0000","last_unstale":"2023-09-27T23:29:19.746562+0000","last_undegraded":"2023-09-27T23:29:19.746562+0000","last_fullsized":"2023-09-27T23:29:19.746562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:47.159413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"86'4","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742107+0000","last_change":"2023-09-27T23:29:19.344994+0000","last_active":"2023-09-27T23:29:19.742107+0000","last_peered":"2023-09-27T23:29:19.742107+0000","last_clean":"2023-09-27T23:29:19.742107+0000","last_became_active":"2023-09-27T23:29:19.344677+0000","last_became_peered":"2023-09-27T23:29:19.344677+0000","last_unstale":"2023-09-27T23:29:19.742107+0000","last_undegraded":"2023-09-27T23:29:19.742107+0000","last_fullsized":"2023-09-27T23:29:19.742107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:37.201350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.727721+0000","last_change":"2023-09-27T23:29:19.719519+0000","last_active":"2023-09-27T23:29:19.727721+0000","last_peered":"2023-09-27T23:29:19.727721+0000","last_clean":"2023-09-27T23:29:19.727721+0000","last_became_active":"2023-09-27T23:29:19.718608+0000","last_became_peered":"2023-09-27T23:29:19.718608+0000","last_unstale":"2023-09-27T23:29:19.727721+0000","last_undegraded":"2023-09-27T23:29:19.727721+0000","last_fullsized":"2023-09-27T23:29:19.727721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:10.796679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.736826+0000","last_change":"2023-09-27T23:29:19.349716+0000","last_active":"2023-09-27T23:29:19.736826+0000","last_peered":"2023-09-27T23:29:19.736826+0000","last_clean":"2023-09-27T23:29:19.736826+0000","last_became_active":"2023-09-27T23:29:19.349440+0000","last_became_peered":"2023-09-27T23:29:19.349440+0000","last_unstale":"2023-09-27T23:29:19.736826+0000","last_undegraded":"2023-09-27T23:29:19.736826+0000","last_fullsized":"2023-09-27T23:29:19.736826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:34.921886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746307+0000","last_change":"2023-09-27T23:29:19.731873+0000","last_active":"2023-09-27T23:29:19.746307+0000","last_peered":"2023-09-27T23:29:19.746307+0000","last_clean":"2023-09-27T23:29:19.746307+0000","last_became_active":"2023-09-27T23:29:19.730797+0000","last_became_peered":"2023-09-27T23:29:19.730797+0000","last_unstale":"2023-09-27T23:29:19.746307+0000","last_undegraded":"2023-09-27T23:29:19.746307+0000","last_fullsized":"2023-09-27T23:29:19.746307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:24:01.984490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746998+0000","last_change":"2023-09-27T23:29:19.346657+0000","last_active":"2023-09-27T23:29:19.746998+0000","last_peered":"2023-09-27T23:29:19.746998+0000","last_clean":"2023-09-27T23:29:19.746998+0000","last_became_active":"2023-09-27T23:29:19.346350+0000","last_became_peered":"2023-09-27T23:29:19.346350+0000","last_unstale":"2023-09-27T23:29:19.746998+0000","last_undegraded":"2023-09-27T23:29:19.746998+0000","last_fullsized":"2023-09-27T23:29:19.746998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:57:24.635028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.741917+0000","last_change":"2023-09-27T23:29:19.730052+0000","last_active":"2023-09-27T23:29:19.741917+0000","last_peered":"2023-09-27T23:29:19.741917+0000","last_clean":"2023-09-27T23:29:19.741917+0000","last_became_active":"2023-09-27T23:29:19.728472+0000","last_became_peered":"2023-09-27T23:29:19.728472+0000","last_unstale":"2023-09-27T23:29:19.741917+0000","last_undegraded":"2023-09-27T23:29:19.741917+0000","last_fullsized":"2023-09-27T23:29:19.741917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:24:01.984490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742203+0000","last_change":"2023-09-27T23:29:19.355491+0000","last_active":"2023-09-27T23:29:19.742203+0000","last_peered":"2023-09-27T23:29:19.742203+0000","last_clean":"2023-09-27T23:29:19.742203+0000","last_became_active":"2023-09-27T23:29:19.355192+0000","last_became_peered":"2023-09-27T23:29:19.355192+0000","last_unstale":"2023-09-27T23:29:19.742203+0000","last_undegraded":"2023-09-27T23:29:19.742203+0000","last_fullsized":"2023-09-27T23:29:19.742203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:54:30.443743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.728234+0000","last_change":"2023-09-27T23:29:19.725838+0000","last_active":"2023-09-27T23:29:19.728234+0000","last_peered":"2023-09-27T23:29:19.728234+0000","last_clean":"2023-09-27T23:29:19.728234+0000","last_became_active":"2023-09-27T23:29:19.725274+0000","last_became_peered":"2023-09-27T23:29:19.725274+0000","last_unstale":"2023-09-27T23:29:19.728234+0000","last_undegraded":"2023-09-27T23:29:19.728234+0000","last_fullsized":"2023-09-27T23:29:19.728234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:01.763686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.740454+0000","last_change":"2023-09-27T23:29:19.358041+0000","last_active":"2023-09-27T23:29:19.740454+0000","last_peered":"2023-09-27T23:29:19.740454+0000","last_clean":"2023-09-27T23:29:19.740454+0000","last_became_active":"2023-09-27T23:29:19.357670+0000","last_became_peered":"2023-09-27T23:29:19.357670+0000","last_unstale":"2023-09-27T23:29:19.740454+0000","last_undegraded":"2023-09-27T23:29:19.740454+0000","last_fullsized":"2023-09-27T23:29:19.740454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:36.666809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.741812+0000","last_change":"2023-09-27T23:29:19.733155+0000","last_active":"2023-09-27T23:29:19.741812+0000","last_peered":"2023-09-27T23:29:19.741812+0000","last_clean":"2023-09-27T23:29:19.741812+0000","last_became_active":"2023-09-27T23:29:19.732841+0000","last_became_peered":"2023-09-27T23:29:19.732841+0000","last_unstale":"2023-09-27T23:29:19.741812+0000","last_undegraded":"2023-09-27T23:29:19.741812+0000","last_fullsized":"2023-09-27T23:29:19.741812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:04.903182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742614+0000","last_change":"2023-09-27T23:29:19.356245+0000","last_active":"2023-09-27T23:29:19.742614+0000","last_peered":"2023-09-27T23:29:19.742614+0000","last_clean":"2023-09-27T23:29:19.742614+0000","last_became_active":"2023-09-27T23:29:19.355967+0000","last_became_peered":"2023-09-27T23:29:19.355967+0000","last_unstale":"2023-09-27T23:29:19.742614+0000","last_undegraded":"2023-09-27T23:29:19.742614+0000","last_fullsized":"2023-09-27T23:29:19.742614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:44:19.665630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.729061+0000","last_change":"2023-09-27T23:29:19.720205+0000","last_active":"2023-09-27T23:29:19.729061+0000","last_peered":"2023-09-27T23:29:19.729061+0000","last_clean":"2023-09-27T23:29:19.729061+0000","last_became_active":"2023-09-27T23:29:19.719900+0000","last_became_peered":"2023-09-27T23:29:19.719900+0000","last_unstale":"2023-09-27T23:29:19.729061+0000","last_undegraded":"2023-09-27T23:29:19.729061+0000","last_fullsized":"2023-09-27T23:29:19.729061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:10:34.634192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.699294+0000","last_change":"2023-09-27T23:29:19.345891+0000","last_active":"2023-09-27T23:29:19.699294+0000","last_peered":"2023-09-27T23:29:19.699294+0000","last_clean":"2023-09-27T23:29:19.699294+0000","last_became_active":"2023-09-27T23:29:19.345582+0000","last_became_peered":"2023-09-27T23:29:19.345582+0000","last_unstale":"2023-09-27T23:29:19.699294+0000","last_undegraded":"2023-09-27T23:29:19.699294+0000","last_fullsized":"2023-09-27T23:29:19.699294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:12:02.052538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.726715+0000","last_change":"2023-09-27T23:29:19.723918+0000","last_active":"2023-09-27T23:29:19.726715+0000","last_peered":"2023-09-27T23:29:19.726715+0000","last_clean":"2023-09-27T23:29:19.726715+0000","last_became_active":"2023-09-27T23:29:19.723648+0000","last_became_peered":"2023-09-27T23:29:19.723648+0000","last_unstale":"2023-09-27T23:29:19.726715+0000","last_undegraded":"2023-09-27T23:29:19.726715+0000","last_fullsized":"2023-09-27T23:29:19.726715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:27:03.107016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.711877+0000","last_change":"2023-09-27T23:29:19.343124+0000","last_active":"2023-09-27T23:29:19.711877+0000","last_peered":"2023-09-27T23:29:19.711877+0000","last_clean":"2023-09-27T23:29:19.711877+0000","last_became_active":"2023-09-27T23:29:19.342770+0000","last_became_peered":"2023-09-27T23:29:19.342770+0000","last_unstale":"2023-09-27T23:29:19.711877+0000","last_undegraded":"2023-09-27T23:29:19.711877+0000","last_fullsized":"2023-09-27T23:29:19.711877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:20:36.791681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.723177+0000","last_change":"2023-09-27T23:29:19.720172+0000","last_active":"2023-09-27T23:29:19.723177+0000","last_peered":"2023-09-27T23:29:19.723177+0000","last_clean":"2023-09-27T23:29:19.723177+0000","last_became_active":"2023-09-27T23:29:19.719812+0000","last_became_peered":"2023-09-27T23:29:19.719812+0000","last_unstale":"2023-09-27T23:29:19.723177+0000","last_undegraded":"2023-09-27T23:29:19.723177+0000","last_fullsized":"2023-09-27T23:29:19.723177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:02:38.852874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.714906+0000","last_change":"2023-09-27T23:29:19.359819+0000","last_active":"2023-09-27T23:29:19.714906+0000","last_peered":"2023-09-27T23:29:19.714906+0000","last_clean":"2023-09-27T23:29:19.714906+0000","last_became_active":"2023-09-27T23:29:19.359510+0000","last_became_peered":"2023-09-27T23:29:19.359510+0000","last_unstale":"2023-09-27T23:29:19.714906+0000","last_undegraded":"2023-09-27T23:29:19.714906+0000","last_fullsized":"2023-09-27T23:29:19.714906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:34:25.799666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746897+0000","last_change":"2023-09-27T23:29:19.742859+0000","last_active":"2023-09-27T23:29:19.746897+0000","last_peered":"2023-09-27T23:29:19.746897+0000","last_clean":"2023-09-27T23:29:19.746897+0000","last_became_active":"2023-09-27T23:29:19.742597+0000","last_became_peered":"2023-09-27T23:29:19.742597+0000","last_unstale":"2023-09-27T23:29:19.746897+0000","last_undegraded":"2023-09-27T23:29:19.746897+0000","last_fullsized":"2023-09-27T23:29:19.746897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:14:53.256289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.719234+0000","last_change":"2023-09-27T23:29:19.352763+0000","last_active":"2023-09-27T23:29:19.719234+0000","last_peered":"2023-09-27T23:29:19.719234+0000","last_clean":"2023-09-27T23:29:19.719234+0000","last_became_active":"2023-09-27T23:29:19.352450+0000","last_became_peered":"2023-09-27T23:29:19.352450+0000","last_unstale":"2023-09-27T23:29:19.719234+0000","last_undegraded":"2023-09-27T23:29:19.719234+0000","last_fullsized":"2023-09-27T23:29:19.719234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:09.789659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746329+0000","last_change":"2023-09-27T23:29:19.734575+0000","last_active":"2023-09-27T23:29:19.746329+0000","last_peered":"2023-09-27T23:29:19.746329+0000","last_clean":"2023-09-27T23:29:19.746329+0000","last_became_active":"2023-09-27T23:29:19.730734+0000","last_became_peered":"2023-09-27T23:29:19.730734+0000","last_unstale":"2023-09-27T23:29:19.746329+0000","last_undegraded":"2023-09-27T23:29:19.746329+0000","last_fullsized":"2023-09-27T23:29:19.746329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:47.061803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"86'3","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.733756+0000","last_change":"2023-09-27T23:29:19.347964+0000","last_active":"2023-09-27T23:29:19.733756+0000","last_peered":"2023-09-27T23:29:19.733756+0000","last_clean":"2023-09-27T23:29:19.733756+0000","last_became_active":"2023-09-27T23:29:19.347464+0000","last_became_peered":"2023-09-27T23:29:19.347464+0000","last_unstale":"2023-09-27T23:29:19.733756+0000","last_undegraded":"2023-09-27T23:29:19.733756+0000","last_fullsized":"2023-09-27T23:29:19.733756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:25.771652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.717988+0000","last_change":"2023-09-27T23:29:19.715184+0000","last_active":"2023-09-27T23:29:19.717988+0000","last_peered":"2023-09-27T23:29:19.717988+0000","last_clean":"2023-09-27T23:29:19.717988+0000","last_became_active":"2023-09-27T23:29:19.714547+0000","last_became_peered":"2023-09-27T23:29:19.714547+0000","last_unstale":"2023-09-27T23:29:19.717988+0000","last_undegraded":"2023-09-27T23:29:19.717988+0000","last_fullsized":"2023-09-27T23:29:19.717988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:38.208371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"86'6","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.743308+0000","last_change":"2023-09-27T23:29:19.347120+0000","last_active":"2023-09-27T23:29:19.743308+0000","last_peered":"2023-09-27T23:29:19.743308+0000","last_clean":"2023-09-27T23:29:19.743308+0000","last_became_active":"2023-09-27T23:29:19.345010+0000","last_became_peered":"2023-09-27T23:29:19.345010+0000","last_unstale":"2023-09-27T23:29:19.743308+0000","last_undegraded":"2023-09-27T23:29:19.743308+0000","last_fullsized":"2023-09-27T23:29:19.743308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:39:29.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.745435+0000","last_change":"2023-09-27T23:29:19.737153+0000","last_active":"2023-09-27T23:29:19.745435+0000","last_peered":"2023-09-27T23:29:19.745435+0000","last_clean":"2023-09-27T23:29:19.745435+0000","last_became_active":"2023-09-27T23:29:19.736716+0000","last_became_peered":"2023-09-27T23:29:19.736716+0000","last_unstale":"2023-09-27T23:29:19.745435+0000","last_undegraded":"2023-09-27T23:29:19.745435+0000","last_fullsized":"2023-09-27T23:29:19.745435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:41.029315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.740951+0000","last_change":"2023-09-27T23:29:19.338818+0000","last_active":"2023-09-27T23:29:19.740951+0000","last_peered":"2023-09-27T23:29:19.740951+0000","last_clean":"2023-09-27T23:29:19.740951+0000","last_became_active":"2023-09-27T23:29:19.338413+0000","last_became_peered":"2023-09-27T23:29:19.338413+0000","last_unstale":"2023-09-27T23:29:19.740951+0000","last_undegraded":"2023-09-27T23:29:19.740951+0000","last_fullsized":"2023-09-27T23:29:19.740951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:54.532266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.728205+0000","last_change":"2023-09-27T23:29:19.724677+0000","last_active":"2023-09-27T23:29:19.728205+0000","last_peered":"2023-09-27T23:29:19.728205+0000","last_clean":"2023-09-27T23:29:19.728205+0000","last_became_active":"2023-09-27T23:29:19.724400+0000","last_became_peered":"2023-09-27T23:29:19.724400+0000","last_unstale":"2023-09-27T23:29:19.728205+0000","last_undegraded":"2023-09-27T23:29:19.728205+0000","last_fullsized":"2023-09-27T23:29:19.728205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:58:24.087157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716915+0000","last_change":"2023-09-27T23:29:19.351569+0000","last_active":"2023-09-27T23:29:19.716915+0000","last_peered":"2023-09-27T23:29:19.716915+0000","last_clean":"2023-09-27T23:29:19.716915+0000","last_became_active":"2023-09-27T23:29:19.351295+0000","last_became_peered":"2023-09-27T23:29:19.351295+0000","last_unstale":"2023-09-27T23:29:19.716915+0000","last_undegraded":"2023-09-27T23:29:19.716915+0000","last_fullsized":"2023-09-27T23:29:19.716915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:38.540430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.734006+0000","last_change":"2023-09-27T23:29:19.730779+0000","last_active":"2023-09-27T23:29:19.734006+0000","last_peered":"2023-09-27T23:29:19.734006+0000","last_clean":"2023-09-27T23:29:19.734006+0000","last_became_active":"2023-09-27T23:29:19.730465+0000","last_became_peered":"2023-09-27T23:29:19.730465+0000","last_unstale":"2023-09-27T23:29:19.734006+0000","last_undegraded":"2023-09-27T23:29:19.734006+0000","last_fullsized":"2023-09-27T23:29:19.734006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:29:20.824047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.710329+0000","last_change":"2023-09-27T23:29:19.348135+0000","last_active":"2023-09-27T23:29:19.710329+0000","last_peered":"2023-09-27T23:29:19.710329+0000","last_clean":"2023-09-27T23:29:19.710329+0000","last_became_active":"2023-09-27T23:29:19.347799+0000","last_became_peered":"2023-09-27T23:29:19.347799+0000","last_unstale":"2023-09-27T23:29:19.710329+0000","last_undegraded":"2023-09-27T23:29:19.710329+0000","last_fullsized":"2023-09-27T23:29:19.710329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:19:09.789659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.727881+0000","last_change":"2023-09-27T23:29:19.719838+0000","last_active":"2023-09-27T23:29:19.727881+0000","last_peered":"2023-09-27T23:29:19.727881+0000","last_clean":"2023-09-27T23:29:19.727881+0000","last_became_active":"2023-09-27T23:29:19.718790+0000","last_became_peered":"2023-09-27T23:29:19.718790+0000","last_unstale":"2023-09-27T23:29:19.727881+0000","last_undegraded":"2023-09-27T23:29:19.727881+0000","last_fullsized":"2023-09-27T23:29:19.727881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:12:14.232753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"86'8","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746482+0000","last_change":"2023-09-27T23:29:19.348148+0000","last_active":"2023-09-27T23:29:19.746482+0000","last_peered":"2023-09-27T23:29:19.746482+0000","last_clean":"2023-09-27T23:29:19.746482+0000","last_became_active":"2023-09-27T23:29:19.347813+0000","last_became_peered":"2023-09-27T23:29:19.347813+0000","last_unstale":"2023-09-27T23:29:19.746482+0000","last_undegraded":"2023-09-27T23:29:19.746482+0000","last_fullsized":"2023-09-27T23:29:19.746482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:48:26.412338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.721293+0000","last_change":"2023-09-27T23:29:19.716992+0000","last_active":"2023-09-27T23:29:19.721293+0000","last_peered":"2023-09-27T23:29:19.721293+0000","last_clean":"2023-09-27T23:29:19.721293+0000","last_became_active":"2023-09-27T23:29:19.716722+0000","last_became_peered":"2023-09-27T23:29:19.716722+0000","last_unstale":"2023-09-27T23:29:19.721293+0000","last_undegraded":"2023-09-27T23:29:19.721293+0000","last_fullsized":"2023-09-27T23:29:19.721293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:43.004101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742267+0000","last_change":"2023-09-27T23:29:19.359846+0000","last_active":"2023-09-27T23:29:19.742267+0000","last_peered":"2023-09-27T23:29:19.742267+0000","last_clean":"2023-09-27T23:29:19.742267+0000","last_became_active":"2023-09-27T23:29:19.359535+0000","last_became_peered":"2023-09-27T23:29:19.359535+0000","last_unstale":"2023-09-27T23:29:19.742267+0000","last_undegraded":"2023-09-27T23:29:19.742267+0000","last_fullsized":"2023-09-27T23:29:19.742267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:37.201350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.724008+0000","last_change":"2023-09-27T23:29:19.720802+0000","last_active":"2023-09-27T23:29:19.724008+0000","last_peered":"2023-09-27T23:29:19.724008+0000","last_clean":"2023-09-27T23:29:19.724008+0000","last_became_active":"2023-09-27T23:29:19.720468+0000","last_became_peered":"2023-09-27T23:29:19.720468+0000","last_unstale":"2023-09-27T23:29:19.724008+0000","last_undegraded":"2023-09-27T23:29:19.724008+0000","last_fullsized":"2023-09-27T23:29:19.724008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:16:43.271231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.711330+0000","last_change":"2023-09-27T23:29:19.348853+0000","last_active":"2023-09-27T23:29:19.711330+0000","last_peered":"2023-09-27T23:29:19.711330+0000","last_clean":"2023-09-27T23:29:19.711330+0000","last_became_active":"2023-09-27T23:29:19.347543+0000","last_became_peered":"2023-09-27T23:29:19.347543+0000","last_unstale":"2023-09-27T23:29:19.711330+0000","last_undegraded":"2023-09-27T23:29:19.711330+0000","last_fullsized":"2023-09-27T23:29:19.711330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:38.540430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.746982+0000","last_change":"2023-09-27T23:29:19.743966+0000","last_active":"2023-09-27T23:29:19.746982+0000","last_peered":"2023-09-27T23:29:19.746982+0000","last_clean":"2023-09-27T23:29:19.746982+0000","last_became_active":"2023-09-27T23:29:19.743568+0000","last_became_peered":"2023-09-27T23:29:19.743568+0000","last_unstale":"2023-09-27T23:29:19.746982+0000","last_undegraded":"2023-09-27T23:29:19.746982+0000","last_fullsized":"2023-09-27T23:29:19.746982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:19:25.440263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.710403+0000","last_change":"2023-09-27T23:29:19.344721+0000","last_active":"2023-09-27T23:29:19.710403+0000","last_peered":"2023-09-27T23:29:19.710403+0000","last_clean":"2023-09-27T23:29:19.710403+0000","last_became_active":"2023-09-27T23:29:19.344389+0000","last_became_peered":"2023-09-27T23:29:19.344389+0000","last_unstale":"2023-09-27T23:29:19.710403+0000","last_undegraded":"2023-09-27T23:29:19.710403+0000","last_fullsized":"2023-09-27T23:29:19.710403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:25:03.533509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.737869+0000","last_change":"2023-09-27T23:29:19.725569+0000","last_active":"2023-09-27T23:29:19.737869+0000","last_peered":"2023-09-27T23:29:19.737869+0000","last_clean":"2023-09-27T23:29:19.737869+0000","last_became_active":"2023-09-27T23:29:19.725309+0000","last_became_peered":"2023-09-27T23:29:19.725309+0000","last_unstale":"2023-09-27T23:29:19.737869+0000","last_undegraded":"2023-09-27T23:29:19.737869+0000","last_fullsized":"2023-09-27T23:29:19.737869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:43.879878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.711560+0000","last_change":"2023-09-27T23:29:19.350214+0000","last_active":"2023-09-27T23:29:19.711560+0000","last_peered":"2023-09-27T23:29:19.711560+0000","last_clean":"2023-09-27T23:29:19.711560+0000","last_became_active":"2023-09-27T23:29:19.349397+0000","last_became_peered":"2023-09-27T23:29:19.349397+0000","last_unstale":"2023-09-27T23:29:19.711560+0000","last_undegraded":"2023-09-27T23:29:19.711560+0000","last_fullsized":"2023-09-27T23:29:19.711560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:19:24.433243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.727655+0000","last_change":"2023-09-27T23:29:19.718944+0000","last_active":"2023-09-27T23:29:19.727655+0000","last_peered":"2023-09-27T23:29:19.727655+0000","last_clean":"2023-09-27T23:29:19.727655+0000","last_became_active":"2023-09-27T23:29:19.718541+0000","last_became_peered":"2023-09-27T23:29:19.718541+0000","last_unstale":"2023-09-27T23:29:19.727655+0000","last_undegraded":"2023-09-27T23:29:19.727655+0000","last_fullsized":"2023-09-27T23:29:19.727655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:39.547450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.708295+0000","last_change":"2023-09-27T23:29:19.347741+0000","last_active":"2023-09-27T23:29:19.708295+0000","last_peered":"2023-09-27T23:29:19.708295+0000","last_clean":"2023-09-27T23:29:19.708295+0000","last_became_active":"2023-09-27T23:29:19.347376+0000","last_became_peered":"2023-09-27T23:29:19.347376+0000","last_unstale":"2023-09-27T23:29:19.708295+0000","last_undegraded":"2023-09-27T23:29:19.708295+0000","last_fullsized":"2023-09-27T23:29:19.708295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:59:05.935866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742073+0000","last_change":"2023-09-27T23:29:19.729613+0000","last_active":"2023-09-27T23:29:19.742073+0000","last_peered":"2023-09-27T23:29:19.742073+0000","last_clean":"2023-09-27T23:29:19.742073+0000","last_became_active":"2023-09-27T23:29:19.729107+0000","last_became_peered":"2023-09-27T23:29:19.729107+0000","last_unstale":"2023-09-27T23:29:19.742073+0000","last_undegraded":"2023-09-27T23:29:19.742073+0000","last_fullsized":"2023-09-27T23:29:19.742073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:39:43.514982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.699644+0000","last_change":"2023-09-27T23:29:19.349372+0000","last_active":"2023-09-27T23:29:19.699644+0000","last_peered":"2023-09-27T23:29:19.699644+0000","last_clean":"2023-09-27T23:29:19.699644+0000","last_became_active":"2023-09-27T23:29:19.349060+0000","last_became_peered":"2023-09-27T23:29:19.349060+0000","last_unstale":"2023-09-27T23:29:19.699644+0000","last_undegraded":"2023-09-27T23:29:19.699644+0000","last_fullsized":"2023-09-27T23:29:19.699644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:45:12.369072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.726404+0000","last_change":"2023-09-27T23:29:19.721338+0000","last_active":"2023-09-27T23:29:19.726404+0000","last_peered":"2023-09-27T23:29:19.726404+0000","last_clean":"2023-09-27T23:29:19.726404+0000","last_became_active":"2023-09-27T23:29:19.721032+0000","last_became_peered":"2023-09-27T23:29:19.721032+0000","last_unstale":"2023-09-27T23:29:19.726404+0000","last_undegraded":"2023-09-27T23:29:19.726404+0000","last_fullsized":"2023-09-27T23:29:19.726404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:55.439186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.713149+0000","last_change":"2023-09-27T23:29:19.348382+0000","last_active":"2023-09-27T23:29:19.713149+0000","last_peered":"2023-09-27T23:29:19.713149+0000","last_clean":"2023-09-27T23:29:19.713149+0000","last_became_active":"2023-09-27T23:29:19.347818+0000","last_became_peered":"2023-09-27T23:29:19.347818+0000","last_unstale":"2023-09-27T23:29:19.713149+0000","last_undegraded":"2023-09-27T23:29:19.713149+0000","last_fullsized":"2023-09-27T23:29:19.713149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:12.490512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742014+0000","last_change":"2023-09-27T23:29:19.729669+0000","last_active":"2023-09-27T23:29:19.742014+0000","last_peered":"2023-09-27T23:29:19.742014+0000","last_clean":"2023-09-27T23:29:19.742014+0000","last_became_active":"2023-09-27T23:29:19.729046+0000","last_became_peered":"2023-09-27T23:29:19.729046+0000","last_unstale":"2023-09-27T23:29:19.742014+0000","last_undegraded":"2023-09-27T23:29:19.742014+0000","last_fullsized":"2023-09-27T23:29:19.742014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:01.763686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.716949+0000","last_change":"2023-09-27T23:29:19.355551+0000","last_active":"2023-09-27T23:29:19.716949+0000","last_peered":"2023-09-27T23:29:19.716949+0000","last_clean":"2023-09-27T23:29:19.716949+0000","last_became_active":"2023-09-27T23:29:19.355244+0000","last_became_peered":"2023-09-27T23:29:19.355244+0000","last_unstale":"2023-09-27T23:29:19.716949+0000","last_undegraded":"2023-09-27T23:29:19.716949+0000","last_fullsized":"2023-09-27T23:29:19.716949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:09.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.736433+0000","last_change":"2023-09-27T23:29:19.729977+0000","last_active":"2023-09-27T23:29:19.736433+0000","last_peered":"2023-09-27T23:29:19.736433+0000","last_clean":"2023-09-27T23:29:19.736433+0000","last_became_active":"2023-09-27T23:29:19.729726+0000","last_became_peered":"2023-09-27T23:29:19.729726+0000","last_unstale":"2023-09-27T23:29:19.736433+0000","last_undegraded":"2023-09-27T23:29:19.736433+0000","last_fullsized":"2023-09-27T23:29:19.736433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:08:49.813732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742096+0000","last_change":"2023-09-27T23:29:19.355604+0000","last_active":"2023-09-27T23:29:19.742096+0000","last_peered":"2023-09-27T23:29:19.742096+0000","last_clean":"2023-09-27T23:29:19.742096+0000","last_became_active":"2023-09-27T23:29:19.355207+0000","last_became_peered":"2023-09-27T23:29:19.355207+0000","last_unstale":"2023-09-27T23:29:19.742096+0000","last_undegraded":"2023-09-27T23:29:19.742096+0000","last_fullsized":"2023-09-27T23:29:19.742096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:33.460035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'161","reported_seq":380,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.713046+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:19.713046+0000","last_peered":"2023-09-27T23:29:19.713046+0000","last_clean":"2023-09-27T23:29:19.713046+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:19.713046+0000","last_undegraded":"2023-09-27T23:29:19.713046+0000","last_fullsized":"2023-09-27T23:29:19.713046+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":161,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.711241+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:19.711241+0000","last_peered":"2023-09-27T23:29:19.711241+0000","last_clean":"2023-09-27T23:29:19.711241+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:19.711241+0000","last_undegraded":"2023-09-27T23:29:19.711241+0000","last_fullsized":"2023-09-27T23:29:19.711241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.733648+0000","last_change":"2023-09-27T23:29:19.728248+0000","last_active":"2023-09-27T23:29:19.733648+0000","last_peered":"2023-09-27T23:29:19.733648+0000","last_clean":"2023-09-27T23:29:19.733648+0000","last_became_active":"2023-09-27T23:29:19.727964+0000","last_became_peered":"2023-09-27T23:29:19.727964+0000","last_unstale":"2023-09-27T23:29:19.733648+0000","last_undegraded":"2023-09-27T23:29:19.733648+0000","last_fullsized":"2023-09-27T23:29:19.733648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:04.903182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.711378+0000","last_change":"2023-09-27T23:29:19.350925+0000","last_active":"2023-09-27T23:29:19.711378+0000","last_peered":"2023-09-27T23:29:19.711378+0000","last_clean":"2023-09-27T23:29:19.711378+0000","last_became_active":"2023-09-27T23:29:19.350612+0000","last_became_peered":"2023-09-27T23:29:19.350612+0000","last_unstale":"2023-09-27T23:29:19.711378+0000","last_undegraded":"2023-09-27T23:29:19.711378+0000","last_fullsized":"2023-09-27T23:29:19.711378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:12.490512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.712951+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:19.712951+0000","last_peered":"2023-09-27T23:29:19.712951+0000","last_clean":"2023-09-27T23:29:19.712951+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:19.712951+0000","last_undegraded":"2023-09-27T23:29:19.712951+0000","last_fullsized":"2023-09-27T23:29:19.712951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.736499+0000","last_change":"2023-09-27T23:29:19.730888+0000","last_active":"2023-09-27T23:29:19.736499+0000","last_peered":"2023-09-27T23:29:19.736499+0000","last_clean":"2023-09-27T23:29:19.736499+0000","last_became_active":"2023-09-27T23:29:19.730429+0000","last_became_peered":"2023-09-27T23:29:19.730429+0000","last_unstale":"2023-09-27T23:29:19.736499+0000","last_undegraded":"2023-09-27T23:29:19.736499+0000","last_fullsized":"2023-09-27T23:29:19.736499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:53:09.529458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.710759+0000","last_change":"2023-09-27T23:29:19.347517+0000","last_active":"2023-09-27T23:29:19.710759+0000","last_peered":"2023-09-27T23:29:19.710759+0000","last_clean":"2023-09-27T23:29:19.710759+0000","last_became_active":"2023-09-27T23:29:19.347173+0000","last_became_peered":"2023-09-27T23:29:19.347173+0000","last_unstale":"2023-09-27T23:29:19.710759+0000","last_undegraded":"2023-09-27T23:29:19.710759+0000","last_fullsized":"2023-09-27T23:29:19.710759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:19:11.835069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.709566+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:19.709566+0000","last_peered":"2023-09-27T23:29:19.709566+0000","last_clean":"2023-09-27T23:29:19.709566+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:19.709566+0000","last_undegraded":"2023-09-27T23:29:19.709566+0000","last_fullsized":"2023-09-27T23:29:19.709566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.722570+0000","last_change":"2023-09-27T23:29:19.714450+0000","last_active":"2023-09-27T23:29:19.722570+0000","last_peered":"2023-09-27T23:29:19.722570+0000","last_clean":"2023-09-27T23:29:19.722570+0000","last_became_active":"2023-09-27T23:29:19.714165+0000","last_became_peered":"2023-09-27T23:29:19.714165+0000","last_unstale":"2023-09-27T23:29:19.722570+0000","last_undegraded":"2023-09-27T23:29:19.722570+0000","last_fullsized":"2023-09-27T23:29:19.722570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:30:34.103109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.717030+0000","last_change":"2023-09-27T23:29:19.351439+0000","last_active":"2023-09-27T23:29:19.717030+0000","last_peered":"2023-09-27T23:29:19.717030+0000","last_clean":"2023-09-27T23:29:19.717030+0000","last_became_active":"2023-09-27T23:29:19.350793+0000","last_became_peered":"2023-09-27T23:29:19.350793+0000","last_unstale":"2023-09-27T23:29:19.717030+0000","last_undegraded":"2023-09-27T23:29:19.717030+0000","last_fullsized":"2023-09-27T23:29:19.717030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:44:07.068721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.710834+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:19.710834+0000","last_peered":"2023-09-27T23:29:19.710834+0000","last_clean":"2023-09-27T23:29:19.710834+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:19.710834+0000","last_undegraded":"2023-09-27T23:29:19.710834+0000","last_fullsized":"2023-09-27T23:29:19.710834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.733997+0000","last_change":"2023-09-27T23:29:19.720526+0000","last_active":"2023-09-27T23:29:19.733997+0000","last_peered":"2023-09-27T23:29:19.733997+0000","last_clean":"2023-09-27T23:29:19.733997+0000","last_became_active":"2023-09-27T23:29:19.720245+0000","last_became_peered":"2023-09-27T23:29:19.720245+0000","last_unstale":"2023-09-27T23:29:19.733997+0000","last_undegraded":"2023-09-27T23:29:19.733997+0000","last_fullsized":"2023-09-27T23:29:19.733997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:55.900355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.713884+0000","last_change":"2023-09-27T23:29:19.351226+0000","last_active":"2023-09-27T23:29:19.713884+0000","last_peered":"2023-09-27T23:29:19.713884+0000","last_clean":"2023-09-27T23:29:19.713884+0000","last_became_active":"2023-09-27T23:29:19.350896+0000","last_became_peered":"2023-09-27T23:29:19.350896+0000","last_unstale":"2023-09-27T23:29:19.713884+0000","last_undegraded":"2023-09-27T23:29:19.713884+0000","last_fullsized":"2023-09-27T23:29:19.713884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.893334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.701508+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:19.701508+0000","last_peered":"2023-09-27T23:29:19.701508+0000","last_clean":"2023-09-27T23:29:19.701508+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:19.701508+0000","last_undegraded":"2023-09-27T23:29:19.701508+0000","last_fullsized":"2023-09-27T23:29:19.701508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.722844+0000","last_change":"2023-09-27T23:29:19.714863+0000","last_active":"2023-09-27T23:29:19.722844+0000","last_peered":"2023-09-27T23:29:19.722844+0000","last_clean":"2023-09-27T23:29:19.722844+0000","last_became_active":"2023-09-27T23:29:19.714319+0000","last_became_peered":"2023-09-27T23:29:19.714319+0000","last_unstale":"2023-09-27T23:29:19.722844+0000","last_undegraded":"2023-09-27T23:29:19.722844+0000","last_fullsized":"2023-09-27T23:29:19.722844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:08:20.357870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742337+0000","last_change":"2023-09-27T23:29:19.351252+0000","last_active":"2023-09-27T23:29:19.742337+0000","last_peered":"2023-09-27T23:29:19.742337+0000","last_clean":"2023-09-27T23:29:19.742337+0000","last_became_active":"2023-09-27T23:29:19.350970+0000","last_became_peered":"2023-09-27T23:29:19.350970+0000","last_unstale":"2023-09-27T23:29:19.742337+0000","last_undegraded":"2023-09-27T23:29:19.742337+0000","last_fullsized":"2023-09-27T23:29:19.742337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:02.449177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.718927+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:19.718927+0000","last_peered":"2023-09-27T23:29:19.718927+0000","last_clean":"2023-09-27T23:29:19.718927+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:19.718927+0000","last_undegraded":"2023-09-27T23:29:19.718927+0000","last_fullsized":"2023-09-27T23:29:19.718927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742086+0000","last_change":"2023-09-27T23:29:19.728910+0000","last_active":"2023-09-27T23:29:19.742086+0000","last_peered":"2023-09-27T23:29:19.742086+0000","last_clean":"2023-09-27T23:29:19.742086+0000","last_became_active":"2023-09-27T23:29:19.728598+0000","last_became_peered":"2023-09-27T23:29:19.728598+0000","last_unstale":"2023-09-27T23:29:19.742086+0000","last_undegraded":"2023-09-27T23:29:19.742086+0000","last_fullsized":"2023-09-27T23:29:19.742086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:10:47.061803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"86'2","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.747050+0000","last_change":"2023-09-27T23:29:19.353539+0000","last_active":"2023-09-27T23:29:19.747050+0000","last_peered":"2023-09-27T23:29:19.747050+0000","last_clean":"2023-09-27T23:29:19.747050+0000","last_became_active":"2023-09-27T23:29:19.353230+0000","last_became_peered":"2023-09-27T23:29:19.353230+0000","last_unstale":"2023-09-27T23:29:19.747050+0000","last_undegraded":"2023-09-27T23:29:19.747050+0000","last_fullsized":"2023-09-27T23:29:19.747050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:31:21.338865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.699635+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:19.699635+0000","last_peered":"2023-09-27T23:29:19.699635+0000","last_clean":"2023-09-27T23:29:19.699635+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:19.699635+0000","last_undegraded":"2023-09-27T23:29:19.699635+0000","last_fullsized":"2023-09-27T23:29:19.699635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.741739+0000","last_change":"2023-09-27T23:29:19.731948+0000","last_active":"2023-09-27T23:29:19.741739+0000","last_peered":"2023-09-27T23:29:19.741739+0000","last_clean":"2023-09-27T23:29:19.741739+0000","last_became_active":"2023-09-27T23:29:19.726803+0000","last_became_peered":"2023-09-27T23:29:19.726803+0000","last_unstale":"2023-09-27T23:29:19.741739+0000","last_undegraded":"2023-09-27T23:29:19.741739+0000","last_fullsized":"2023-09-27T23:29:19.741739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:19.306227+0000","last_clean_scrub_stamp":"2023-09-27T23:29:19.306227+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:53:09.529458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.742021+0000","last_change":"2023-09-27T23:29:19.359807+0000","last_active":"2023-09-27T23:29:19.742021+0000","last_peered":"2023-09-27T23:29:19.742021+0000","last_clean":"2023-09-27T23:29:19.742021+0000","last_became_active":"2023-09-27T23:29:19.359497+0000","last_became_peered":"2023-09-27T23:29:19.359497+0000","last_unstale":"2023-09-27T23:29:19.742021+0000","last_undegraded":"2023-09-27T23:29:19.742021+0000","last_fullsized":"2023-09-27T23:29:19.742021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:18.299206+0000","last_clean_scrub_stamp":"2023-09-27T23:29:18.299206+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:12:13.225733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-27T23:29:19.714700+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:19.714700+0000","last_peered":"2023-09-27T23:29:19.714700+0000","last_clean":"2023-09-27T23:29:19.714700+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:19.714700+0000","last_undegraded":"2023-09-27T23:29:19.714700+0000","last_fullsized":"2023-09-27T23:29:19.714700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509631,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25684,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":24064,"kb_avail":94346156,"statfs":{"total":96636764160,"available":96610463744,"internally_reserved":0,"allocated":1650688,"data_stored":1126493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24641536},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":1.5489999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.337}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":1.282}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.6000000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.151},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.2450000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":1.79},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.7190000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.839},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.4139999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21264,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":20224,"kb_avail":94350576,"statfs":{"total":96636764160,"available":96614989824,"internally_reserved":0,"allocated":1056768,"data_stored":535629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20709376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":0.84099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.874}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.753}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":1.244},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.0600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.284},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.232}]}]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":30336,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":1052672,"data_stored":535627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31064064},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":1.095},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.266}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":1.1679999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.131},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.0660000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":1.222},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":1.0189999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.323}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":1.202},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.296}]}]},{"osd":4,"up_from":15,"seq":64424509631,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21516,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":20480,"kb_avail":94350324,"statfs":{"total":96636764160,"available":96614731776,"internally_reserved":0,"allocated":1052672,"data_stored":535607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20971520},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":1.046},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.90200000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":0.96699999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.998},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":0.94099999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.093},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":1.115}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.2729999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.2410000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509630,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22032,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":20992,"kb_avail":94349808,"statfs":{"total":96636764160,"available":96614203392,"internally_reserved":0,"allocated":1056768,"data_stored":535649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21495808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.71699999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":0.86199999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.113}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":0.997}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.02}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.1719999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509631,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20564,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":19520,"kb_avail":94351276,"statfs":{"total":96636764160,"available":96615706624,"internally_reserved":0,"allocated":1060864,"data_stored":537073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19988480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.57199999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.69899999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":1.123},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":1.345}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":1.2689999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":1.288},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542337,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35740,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":34688,"kb_avail":94336100,"statfs":{"total":96636764160,"available":96600166400,"internally_reserved":0,"allocated":1069056,"data_stored":537661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35520512},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.755}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":1.091}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":1.1599999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":1.016},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":1.2070000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":1.121}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":1.2889999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.2450000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22540,"kb_used_data":1604,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":94349300,"statfs":{"total":96636764160,"available":96613683200,"internally_reserved":0,"allocated":1642496,"data_stored":1125905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.67900000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.621},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.99099999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.84899999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":1.1200000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.222},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.462}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.419}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":1.5660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:27.642 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:29:27.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:29:27.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.776+0000 7f19ded83700 1 Processor -- start 2023-09-27T23:29:27.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.776+0000 7f19ded83700 1 -- start start 2023-09-27T23:29:27.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19d806b2c0 0x7f19d806b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d806c7c0 con 0x7f19d806d680 2023-09-27T23:29:27.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d806cd80 con 0x7f19d806d210 2023-09-27T23:29:27.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d80b0940 con 0x7f19d806b2c0 2023-09-27T23:29:27.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dcb1f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19d806b2c0 0x7f19d806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dd320700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f19d806d680 0x7f19d8158d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54808/0 (socket says 172.21.15.64:54808) 2023-09-27T23:29:27.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dd320700 1 -- 172.21.15.64:0/340467617 learned_addr learned my addr 172.21.15.64:0/340467617 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:27.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dcb1f700 1 -- 172.21.15.64:0/340467617 >> v1:172.21.15.123:6789/0 conn(0x7f19d806d210 legacy=0x7f19d80afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dcb1f700 1 -- 172.21.15.64:0/340467617 >> v1:172.21.15.64:6789/0 conn(0x7f19d806d680 legacy=0x7f19d8158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19d77fe700 1 -- 172.21.15.64:0/340467617 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604733920 0 0) 0x7f19d806cd80 con 0x7f19d806d210 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dcb1f700 1 -- 172.21.15.64:0/340467617 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80b0c20 con 0x7f19d806b2c0 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19dcb1f700 1 --2- 172.21.15.64:0/340467617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806b2c0 0x7f19d806b690 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0098e0 tx=0x7f19cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f182cf5dbff0d50a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19d77fe700 1 -- 172.21.15.64:0/340467617 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc009e80 con 0x7f19d806b2c0 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19d77fe700 1 -- 172.21.15.64:0/340467617 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00b760 con 0x7f19d806b2c0 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19d77fe700 1 -- 172.21.15.64:0/340467617 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00bae0 con 0x7f19d806b2c0 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 -- 172.21.15.64:0/340467617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806b2c0 msgr2=0x7f19d806b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 --2- 172.21.15.64:0/340467617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806b2c0 0x7f19d806b690 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0098e0 tx=0x7f19cc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.780+0000 7f19ded83700 1 -- 172.21.15.64:0/340467617 shutdown_connections 2023-09-27T23:29:27.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.784+0000 7f19ded83700 1 --2- 172.21.15.64:0/340467617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806b2c0 0x7f19d806b690 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.784+0000 7f19ded83700 1 -- 172.21.15.64:0/340467617 >> 172.21.15.64:0/340467617 conn(0x7f19d80cc580 msgr2=0x7f19d80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:27.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.784+0000 7f19ded83700 1 -- 172.21.15.64:0/340467617 shutdown_connections 2023-09-27T23:29:27.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.784+0000 7f19ded83700 1 -- 172.21.15.64:0/340467617 wait complete. 2023-09-27T23:29:27.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.784+0000 7f19ded83700 1 Processor -- start 2023-09-27T23:29:27.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.784+0000 7f19ded83700 1 -- start start 2023-09-27T23:29:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d806b2c0 0x7f19d80ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19d806d210 0x7f19d80ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 0x7f19d80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80ad980 con 0x7f19d806d680 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80adc60 con 0x7f19d806b2c0 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80adf40 con 0x7f19d806d210 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 0x7f19d80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 0x7f19d80a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53662/0 (socket says 172.21.15.64:53662) 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 -- 172.21.15.64:0/1946969320 learned_addr learned my addr 172.21.15.64:0/1946969320 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19d7fff700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19d806d210 0x7f19d80ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dcb1f700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d806b2c0 0x7f19d80ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 -- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19d806d210 msgr2=0x7f19d80ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19d806d210 0x7f19d80ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 -- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d806b2c0 msgr2=0x7f19d80ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d806b2c0 0x7f19d80ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 -- 172.21.15.64:0/1946969320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009420 con 0x7f19d806d680 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19d7fff700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19d806d210 0x7f19d80ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dcb1f700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d806b2c0 0x7f19d80ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19dd320700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 0x7f19d80a9280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f19d0002230 tx=0x7f19d0001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:27.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d000ab30 con 0x7f19d806d680 2023-09-27T23:29:27.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d000ac90 con 0x7f19d806d680 2023-09-27T23:29:27.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d80ae220 con 0x7f19d806d680 2023-09-27T23:29:27.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0007340 con 0x7f19d806d680 2023-09-27T23:29:27.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.788+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81586d0 con 0x7f19d806d680 2023-09-27T23:29:27.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.792+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d80ae720 con 0x7f19d806d680 2023-09-27T23:29:27.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.792+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f19d00076f0 con 0x7f19d806d680 2023-09-27T23:29:27.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.792+0000 7f19d4ff9700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19bc060a70 0x7f19bc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:27.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.792+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f19d007d6b0 con 0x7f19d806d680 2023-09-27T23:29:27.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.792+0000 7f19dcb1f700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19bc060a70 0x7f19bc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:27.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.792+0000 7f19dcb1f700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19bc060a70 0x7f19bc062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0030c0 tx=0x7f19cc016000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:27.800+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f19d004d760 con 0x7f19d806d680 2023-09-27T23:29:28.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.020+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f19d8071120 con 0x7f19d806d680 2023-09-27T23:29:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.028+0000 7f19d4ff9700 1 -- 172.21.15.64:0/1946969320 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f19d0050d80 con 0x7f19d806d680 2023-09-27T23:29:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19bc060a70 msgr2=0x7f19bc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19bc060a70 0x7f19bc062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0030c0 tx=0x7f19cc016000 comp rx=0 tx=0).stop 2023-09-27T23:29:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 msgr2=0x7f19d80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 0x7f19d80a9280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f19d0002230 tx=0x7f19d0001b90 comp rx=0 tx=0).stop 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 shutdown_connections 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19bc060a70 0x7f19bc062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d806d680 0x7f19d80a9280 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19d806d210 0x7f19d80ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 --2- 172.21.15.64:0/1946969320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d806b2c0 0x7f19d80ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.032+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 >> 172.21.15.64:0/1946969320 conn(0x7f19d80cc580 msgr2=0x7f19d806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.036+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 shutdown_connections 2023-09-27T23:29:28.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.036+0000 7f19ded83700 1 -- 172.21.15.64:0/1946969320 wait complete. 2023-09-27T23:29:28.042 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:29:28.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 Processor -- start 2023-09-27T23:29:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 -- start start 2023-09-27T23:29:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d70158ee0 0x7f1d70067af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d7006d760 0x7f1d700b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700688d0 con 0x7f1d701592b0 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d700b1510 con 0x7f1d70158ee0 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d7859b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7006a180 con 0x7f1d7006d760 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d76337700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d70158ee0 0x7f1d70067af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d75b36700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d701592b0 0x7f1d7006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54814/0 (socket says 172.21.15.64:54814) 2023-09-27T23:29:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.160+0000 7f1d75b36700 1 -- 172.21.15.64:0/3603258975 learned_addr learned my addr 172.21.15.64:0/3603258975 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d76b38700 1 --2- 172.21.15.64:0/3603258975 >> v2:172.21.15.123:3300/0 conn(0x7f1d7006d760 0x7f1d700b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d75335700 1 -- 172.21.15.64:0/3603258975 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489573993 0 0) 0x7f1d700688d0 con 0x7f1d701592b0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d75335700 1 -- 172.21.15.64:0/3603258975 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c003570 con 0x7f1d701592b0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d76337700 1 -- 172.21.15.64:0/3603258975 >> v2:172.21.15.123:3300/0 conn(0x7f1d7006d760 msgr2=0x7f1d700b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d76337700 1 --2- 172.21.15.64:0/3603258975 >> v2:172.21.15.123:3300/0 conn(0x7f1d7006d760 0x7f1d700b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d76337700 1 -- 172.21.15.64:0/3603258975 >> v1:172.21.15.64:6789/0 conn(0x7f1d701592b0 legacy=0x7f1d7006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d76337700 1 -- 172.21.15.64:0/3603258975 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700685f0 con 0x7f1d70158ee0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d76337700 1 --2- 172.21.15.64:0/3603258975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d70158ee0 0x7f1d70067af0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d7006b720 tx=0x7f1d600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b642043df743bc10 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d75335700 1 -- 172.21.15.64:0/3603258975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60011630 con 0x7f1d70158ee0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d75335700 1 -- 172.21.15.64:0/3603258975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60011790 con 0x7f1d70158ee0 2023-09-27T23:29:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d75335700 1 -- 172.21.15.64:0/3603258975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60011b30 con 0x7f1d70158ee0 2023-09-27T23:29:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 -- 172.21.15.64:0/3603258975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d70158ee0 msgr2=0x7f1d70067af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 --2- 172.21.15.64:0/3603258975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d70158ee0 0x7f1d70067af0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d7006b720 tx=0x7f1d600091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 -- 172.21.15.64:0/3603258975 shutdown_connections 2023-09-27T23:29:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 --2- 172.21.15.64:0/3603258975 >> v2:172.21.15.123:3300/0 conn(0x7f1d7006d760 0x7f1d700b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 --2- 172.21.15.64:0/3603258975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d70158ee0 0x7f1d70067af0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 -- 172.21.15.64:0/3603258975 >> 172.21.15.64:0/3603258975 conn(0x7f1d700cd990 msgr2=0x7f1d700ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 -- 172.21.15.64:0/3603258975 shutdown_connections 2023-09-27T23:29:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.164+0000 7f1d7859b700 1 -- 172.21.15.64:0/3603258975 wait complete. 2023-09-27T23:29:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 Processor -- start 2023-09-27T23:29:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 -- start start 2023-09-27T23:29:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 0x7f1d7015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d70158ee0 0x7f1d70146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d701592b0 0x7f1d701572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700a4440 con 0x7f1d7006d760 2023-09-27T23:29:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700a5260 con 0x7f1d70158ee0 2023-09-27T23:29:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.168+0000 7f1d7859b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7015bf10 con 0x7f1d701592b0 2023-09-27T23:29:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 0x7f1d7015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 0x7f1d7015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53674/0 (socket says 172.21.15.64:53674) 2023-09-27T23:29:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 -- 172.21.15.64:0/1004556184 learned_addr learned my addr 172.21.15.64:0/1004556184 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d75b36700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d70158ee0 0x7f1d70146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76b38700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d701592b0 0x7f1d701572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 -- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d701592b0 msgr2=0x7f1d701572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d701592b0 0x7f1d701572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 -- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d70158ee0 msgr2=0x7f1d70146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d70158ee0 0x7f1d70146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 -- 172.21.15.64:0/1004556184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009420 con 0x7f1d7006d760 2023-09-27T23:29:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d75b36700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d70158ee0 0x7f1d70146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d76337700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 0x7f1d7015b9d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1d60013000 tx=0x7f1d60017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60011b10 con 0x7f1d7006d760 2023-09-27T23:29:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60000c90 con 0x7f1d7006d760 2023-09-27T23:29:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d700a4650 con 0x7f1d7006d760 2023-09-27T23:29:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6001a680 con 0x7f1d7006d760 2023-09-27T23:29:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d700a4c10 con 0x7f1d7006d760 2023-09-27T23:29:28.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.172+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d70071120 con 0x7f1d7006d760 2023-09-27T23:29:28.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.176+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1d6001a9f0 con 0x7f1d7006d760 2023-09-27T23:29:28.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.176+0000 7f1d667fc700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d54060aa0 0x7f1d54062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.176+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1d6007ec90 con 0x7f1d7006d760 2023-09-27T23:29:28.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.176+0000 7f1d75b36700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d54060aa0 0x7f1d54062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.176+0000 7f1d75b36700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d54060aa0 0x7f1d54062f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d70153130 tx=0x7f1d6c009040 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.180+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1d6004ec20 con 0x7f1d7006d760 2023-09-27T23:29:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.376+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d700b0ec0 con 0x7f1d7006d760 2023-09-27T23:29:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.376+0000 7f1d667fc700 1 -- 172.21.15.64:0/1004556184 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1d60015020 con 0x7f1d7006d760 2023-09-27T23:29:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:28.384 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:19.692498+0000","modified":"2023-09-27T23:29:21.709311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6610":{"gid":6610,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2730764960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2730764960},{"type":"v1","addr":"172.21.15.123:6837","nonce":2730764960}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-09-27T23:29:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d54060aa0 msgr2=0x7f1d54062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d54060aa0 0x7f1d54062f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d70153130 tx=0x7f1d6c009040 comp rx=0 tx=0).stop 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 msgr2=0x7f1d7015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 0x7f1d7015b9d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1d60013000 tx=0x7f1d60017000 comp rx=0 tx=0).stop 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 shutdown_connections 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d54060aa0 0x7f1d54062f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d701592b0 0x7f1d701572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d70158ee0 0x7f1d70146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 --2- 172.21.15.64:0/1004556184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d7006d760 0x7f1d7015b9d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 >> 172.21.15.64:0/1004556184 conn(0x7f1d700cd990 msgr2=0x7f1d70155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 shutdown_connections 2023-09-27T23:29:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.380+0000 7f1d7859b700 1 -- 172.21.15.64:0/1004556184 wait complete. 2023-09-27T23:29:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-27T23:29:28.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:29:19.692498+0000', 'modified': '2023-09-27T23:29:21.709311+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6610}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6610': {'gid': 6610, 'name': 'b', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6837/2730764960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2730764960}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2730764960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:29:28.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:29:28.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-27T23:29:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde09746700 1 Processor -- start 2023-09-27T23:29:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde09746700 1 -- start start 2023-09-27T23:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde09746700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde04158910 0x7fde040c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde09746700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde040accb0 con 0x7fde04159160 2023-09-27T23:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde09746700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde040acf90 con 0x7fde04159530 2023-09-27T23:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde09746700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde04158d70 con 0x7fde04158910 2023-09-27T23:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde03fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fde04159160 0x7fde040adcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54824/0 (socket says 172.21.15.64:54824) 2023-09-27T23:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde03fff700 1 -- 172.21.15.64:0/1180231500 learned_addr learned my addr 172.21.15.64:0/1180231500 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:28.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde08f45700 1 --2- 172.21.15.64:0/1180231500 >> v2:172.21.15.123:3300/0 conn(0x7fde04158910 0x7fde040c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957651854 0 0) 0x7fde040accb0 con 0x7fde04159160 2023-09-27T23:29:28.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddf0003570 con 0x7fde04159160 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725533896 0 0) 0x7fde040acf90 con 0x7fde04159530 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde040accb0 con 0x7fde04159530 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde08f45700 1 -- 172.21.15.64:0/1180231500 >> v1:172.21.15.123:6789/0 conn(0x7fde04159530 legacy=0x7fde04158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2872608689 0 0) 0x7fddf0003570 con 0x7fde04159160 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde08f45700 1 -- 172.21.15.64:0/1180231500 >> v1:172.21.15.64:6789/0 conn(0x7fde04159160 legacy=0x7fde040adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde08f45700 1 -- 172.21.15.64:0/1180231500 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde040ad270 con 0x7fde04158910 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.916+0000 7fde08f45700 1 --2- 172.21.15.64:0/1180231500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04158910 0x7fde040c5fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0098e0 tx=0x7fddfc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=820f7acc045233ec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4001470 con 0x7fde04158910 2023-09-27T23:29:28.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc011420 con 0x7fde04158910 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde02ffd700 1 -- 172.21.15.64:0/1180231500 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0117c0 con 0x7fde04158910 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 -- 172.21.15.64:0/1180231500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04158910 msgr2=0x7fde040c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 --2- 172.21.15.64:0/1180231500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04158910 0x7fde040c5fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0098e0 tx=0x7fddfc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 -- 172.21.15.64:0/1180231500 shutdown_connections 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 --2- 172.21.15.64:0/1180231500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04158910 0x7fde040c5fd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 -- 172.21.15.64:0/1180231500 >> 172.21.15.64:0/1180231500 conn(0x7fde04067bb0 msgr2=0x7fde040cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 -- 172.21.15.64:0/1180231500 shutdown_connections 2023-09-27T23:29:28.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 -- 172.21.15.64:0/1180231500 wait complete. 2023-09-27T23:29:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 Processor -- start 2023-09-27T23:29:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.920+0000 7fde09746700 1 -- start start 2023-09-27T23:29:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 0x7fde040af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04159160 0x7fde040a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde04159530 0x7fde040b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde040aff70 con 0x7fde04158910 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde040b0250 con 0x7fde04159160 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde040b0530 con 0x7fde04159530 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 0x7fde040af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 0x7fde040af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53678/0 (socket says 172.21.15.64:53678) 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 -- 172.21.15.64:0/2639371996 learned_addr learned my addr 172.21.15.64:0/2639371996 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde08f45700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde04159530 0x7fde040b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde037fe700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04159160 0x7fde040a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 -- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde04159530 msgr2=0x7fde040b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde04159530 0x7fde040b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 -- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04159160 msgr2=0x7fde040a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04159160 0x7fde040a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 -- 172.21.15.64:0/2639371996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc009420 con 0x7fde04158910 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde03fff700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 0x7fde040af8e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fddf400cb70 tx=0x7fddf4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fddfbfff700 1 -- 172.21.15.64:0/2639371996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf400ce80 con 0x7fde04158910 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fddfbfff700 1 -- 172.21.15.64:0/2639371996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf4007360 con 0x7fde04158910 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde040b0810 con 0x7fde04158910 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fddfbfff700 1 -- 172.21.15.64:0/2639371996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4007700 con 0x7fde04158910 2023-09-27T23:29:28.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.924+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde04157830 con 0x7fde04158910 2023-09-27T23:29:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fddfbfff700 1 -- 172.21.15.64:0/2639371996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fddf4004a40 con 0x7fde04158910 2023-09-27T23:29:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fddfbfff700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdde4060a70 0x7fdde4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fddfbfff700 1 -- 172.21.15.64:0/2639371996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fddf407c920 con 0x7fde04158910 2023-09-27T23:29:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fde037fe700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdde4060a70 0x7fdde4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fde037fe700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdde4060a70 0x7fdde4062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde040aa470 tx=0x7fddec007f90 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fde0cc51740 1 Processor -- start 2023-09-27T23:29:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.928+0000 7fde0cc51740 1 -- start start 2023-09-27T23:29:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004ae0 0x1281e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x1282390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x12828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x11aff30 con 0x1004710 2023-09-27T23:29:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x11abed0 con 0x1004ae0 2023-09-27T23:29:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x11720b0 con 0x104f790 2023-09-27T23:29:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x1282390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x1282390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53690/0 (socket says 172.21.15.64:53690) 2023-09-27T23:29:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde08f45700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x12828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde03fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004ae0 0x1281e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 -- 172.21.15.64:0/2978450943 learned_addr learned my addr 172.21.15.64:0/2978450943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:28.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 -- 172.21.15.64:0/2978450943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 msgr2=0x12828d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x12828d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 -- 172.21.15.64:0/2978450943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004ae0 msgr2=0x1281e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004ae0 0x1281e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 -- 172.21.15.64:0/2978450943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10509e0 con 0x1004710 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde037fe700 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x1282390 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fddec00fcb0 tx=0x7fddec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2b05c5a8676f90 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fddf97fa700 1 -- 172.21.15.64:0/2978450943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec002c60 con 0x1004710 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fddf97fa700 1 -- 172.21.15.64:0/2978450943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec002dc0 con 0x1004710 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fddf97fa700 1 -- 172.21.15.64:0/2978450943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec003260 con 0x1004710 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- 172.21.15.64:0/2978450943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 msgr2=0x1282390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x1282390 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fddec00fcb0 tx=0x7fddec015000 comp rx=0 tx=0).stop 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- 172.21.15.64:0/2978450943 shutdown_connections 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x12828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x1282390 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2978450943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004ae0 0x1281e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- 172.21.15.64:0/2978450943 >> 172.21.15.64:0/2978450943 conn(0x1004eb0 msgr2=0x123e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- 172.21.15.64:0/2978450943 shutdown_connections 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.932+0000 7fde0cc51740 1 -- 172.21.15.64:0/2978450943 wait complete. 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 -- start start 2023-09-27T23:29:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 Processor -- start 2023-09-27T23:29:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x123e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x1267c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x104f790 0x1268140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1171d50 con 0x1004710 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x114a970 con 0x104f790 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x10e8430 con 0x1004ae0 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 10 client.0 _ll_register_callbacks cb 0x11f5b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x123e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x123e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53702/0 (socket says 172.21.15.64:53702) 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 -- 172.21.15.64:0/4115448201 learned_addr learned my addr 172.21.15.64:0/4115448201 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde037fe700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x1267c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde08f45700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x104f790 0x1268140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 -- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 msgr2=0x1267c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x1267c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 -- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x104f790 msgr2=0x1268140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x104f790 0x1268140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 -- 172.21.15.64:0/4115448201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde040accb0 con 0x1004710 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde03fff700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x123e740 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fddf400a340 tx=0x7fddf4053040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336e41e04995a473 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fdde27fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:29:28.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fdde27fc700 1 -- 172.21.15.64:0/4115448201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4076020 con 0x1004710 2023-09-27T23:29:28.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fdde27fc700 1 -- 172.21.15.64:0/4115448201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fddf404ea10 con 0x1004710 2023-09-27T23:29:28.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.936+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x114acc0 con 0x1004710 2023-09-27T23:29:28.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 1 -- 172.21.15.64:0/4115448201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf4050aa0 con 0x1004710 2023-09-27T23:29:28.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 10 client.6786 fetch_fsmap learned FSMap version 27 2023-09-27T23:29:28.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x117a370 con 0x1004710 2023-09-27T23:29:28.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 1 -- 172.21.15.64:0/4115448201 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x114acc0 con 0x1004710 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 1 -- 172.21.15.64:0/4115448201 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fddf404fae0 con 0x1004710 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 10 client.6786 fetch_fsmap finished waiting for FSMap version 27 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 10 client.6786 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 20 client.6786 populate_metadata read hostname 'smithi064' 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x128e6b0 0x1290a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 4 client.6786 mds_command: new command op to 6610 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:29:28.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 --> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x114e3e0 con 0x128e6b0 2023-09-27T23:29:28.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde037fe700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x128e6b0 0x1290a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 10 client.6786 ms_handle_connect on v2:172.21.15.123:6836/2730764960 2023-09-27T23:29:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde037fe700 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x128e6b0 0x1290a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b829830e7a1c118 server_cookie=f7323475a55f5bcf in_seq=0 out_seq=0 2023-09-27T23:29:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 1 -- 172.21.15.64:0/4115448201 <== mds.0 v2:172.21.15.123:6836/2730764960 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x114e3e0 con 0x128e6b0 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 10 client.6786 handle_command_reply: tid=0 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 client.6786 shutdown 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 msgr2=0x123e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x123e740 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fddf400a340 tx=0x7fddf4053040 comp rx=0 tx=0).stop 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 shutdown_connections 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x128e6b0 0x1290a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x104f790 0x1268140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fde0cc51740 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x1267c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.940+0000 7fdde27fc700 0 client.6786 ms_handle_reset on v2:172.21.15.123:6836/2730764960 2023-09-27T23:29:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.944+0000 7fde0cc51740 1 --2- 172.21.15.64:0/4115448201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004710 0x123e740 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.944+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 >> 172.21.15.64:0/4115448201 conn(0x1004eb0 msgr2=0x1236ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.944+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 shutdown_connections 2023-09-27T23:29:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.944+0000 7fde0cc51740 1 -- 172.21.15.64:0/4115448201 wait complete. 2023-09-27T23:29:28.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.944+0000 7fde0cc51740 20 client.6786 trim_cache size 0 max 16384 2023-09-27T23:29:28.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 Processor -- start 2023-09-27T23:29:28.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 -- start start 2023-09-27T23:29:28.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1237760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 0x1237ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x1290a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b0490 con 0x1004ae0 2023-09-27T23:29:28.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x114cb80 con 0x1004710 2023-09-27T23:29:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.960+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x11b2400 con 0x104f790 2023-09-27T23:29:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 0x1237ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 0x1237ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53714/0 (socket says 172.21.15.64:53714) 2023-09-27T23:29:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 -- 172.21.15.64:0/1201506437 learned_addr learned my addr 172.21.15.64:0/1201506437 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde08f45700 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x1290a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde03fff700 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1237760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 -- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 msgr2=0x1290a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x1290a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 -- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 msgr2=0x1237760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1237760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 -- 172.21.15.64:0/1201506437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde040b18b0 con 0x1004ae0 2023-09-27T23:29:28.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde03fff700 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1237760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde037fe700 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 0x1237ca0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fddec023040 tx=0x7fddec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51305ce537856312 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fddf9ffb700 1 -- 172.21.15.64:0/1201506437 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec031070 con 0x1004ae0 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fddf9ffb700 1 -- 172.21.15.64:0/1201506437 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec003d30 con 0x1004ae0 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fddf9ffb700 1 -- 172.21.15.64:0/1201506437 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec003650 con 0x1004ae0 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- 172.21.15.64:0/1201506437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 msgr2=0x1237ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 0x1237ca0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fddec023040 tx=0x7fddec019000 comp rx=0 tx=0).stop 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- 172.21.15.64:0/1201506437 shutdown_connections 2023-09-27T23:29:28.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x104f790 0x1290a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1004ae0 0x1237ca0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- 172.21.15.64:0/1201506437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1237760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- 172.21.15.64:0/1201506437 >> 172.21.15.64:0/1201506437 conn(0x1004eb0 msgr2=0x1267e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- 172.21.15.64:0/1201506437 shutdown_connections 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- 172.21.15.64:0/1201506437 wait complete. 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- start start 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 Processor -- start 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1262b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x12630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 0x1263610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x115d550 con 0x104f790 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1155510 con 0x1004710 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x11ab6e0 con 0x1004ae0 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.964+0000 7fde0cc51740 10 client.0 _ll_register_callbacks cb 0x10c80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 0x1263610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 0x1263610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53718/0 (socket says 172.21.15.64:53718) 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 -- 172.21.15.64:0/2290862127 learned_addr learned my addr 172.21.15.64:0/2290862127 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde037fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x12630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde03fff700 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1262b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 -- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 msgr2=0x12630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x12630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 -- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 msgr2=0x1262b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:28.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1262b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 -- 172.21.15.64:0/2290862127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x10509e0 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde08f45700 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 0x1263610 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fddfc002000 tx=0x7fddfc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672e8e4226429729 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fdde27fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fdde27fc700 1 -- 172.21.15.64:0/2290862127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc01ca90 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fdde27fc700 1 -- 172.21.15.64:0/2290862127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fddfc01cbf0 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddec002dc0 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fdde27fc700 1 -- 172.21.15.64:0/2290862127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddfc0293f0 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 10 client.6792 fetch_fsmap learned FSMap version 27 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x1160d80 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fdde27fc700 1 -- 172.21.15.64:0/2290862127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec002dc0 con 0x104f790 2023-09-27T23:29:28.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fdde27fc700 1 -- 172.21.15.64:0/2290862127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fddfc028070 con 0x104f790 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 10 client.6792 fetch_fsmap finished waiting for FSMap version 27 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 10 client.6792 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 20 client.6792 populate_metadata read hostname 'smithi064' 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x12e1530 0x12e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 4 client.6792 mds_command: new command op to 6610 tid=0[{"prefix": "status"}] 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 --> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1180410 con 0x12e1530 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.968+0000 7fde03fff700 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x12e1530 0x12e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fdde27fc700 10 client.6792 ms_handle_connect on v2:172.21.15.123:6836/2730764960 2023-09-27T23:29:28.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde03fff700 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x12e1530 0x12e3910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c95ea7f9a5a97c server_cookie=53e255a0913f3e89 in_seq=0 out_seq=0 2023-09-27T23:29:28.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fdde27fc700 1 -- 172.21.15.64:0/2290862127 <== mds.0 v2:172.21.15.123:6836/2730764960 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1180410 con 0x12e1530 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fdde27fc700 10 client.6792 handle_command_reply: tid=0 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 client.6792 shutdown 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 msgr2=0x1263610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 0x1263610 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fddfc002000 tx=0x7fddfc016000 comp rx=0 tx=0).stop 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 shutdown_connections 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:6836/2730764960,v1:172.21.15.123:6837/2730764960] conn(0x12e1530 0x12e3910 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x104f790 0x1263610 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1004ae0 0x12630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 --2- 172.21.15.64:0/2290862127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1004710 0x1262b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fdde27fc700 0 client.6792 ms_handle_reset on v2:172.21.15.123:6836/2730764960 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 >> 172.21.15.64:0/2290862127 conn(0x1004eb0 msgr2=0x1266c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 shutdown_connections 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 1 -- 172.21.15.64:0/2290862127 wait complete. 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde0cc51740 20 client.6792 trim_cache size 0 max 16384 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:29:28.983 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6610, 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.2784970050000002, 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 27, 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 87, 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 87, 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 13.875535080000001 2023-09-27T23:29:28.984 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:29:28.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdde4060a70 msgr2=0x7fdde4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdde4060a70 0x7fdde4062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde040aa470 tx=0x7fddec007f90 comp rx=0 tx=0).stop 2023-09-27T23:29:28.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 msgr2=0x7fde040af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:28.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 0x7fde040af8e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fddf400cb70 tx=0x7fddf4001b50 comp rx=0 tx=0).stop 2023-09-27T23:29:28.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 shutdown_connections 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdde4060a70 0x7fdde4062f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde04159530 0x7fde040b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde04159160 0x7fde040a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 --2- 172.21.15.64:0/2639371996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde04158910 0x7fde040af8e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 >> 172.21.15.64:0/2639371996 conn(0x7fde04067bb0 msgr2=0x7fde040cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 shutdown_connections 2023-09-27T23:29:28.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:28.972+0000 7fde09746700 1 -- 172.21.15.64:0/2639371996 wait complete. 2023-09-27T23:29:28.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:29:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.100+0000 7fc82e15e700 1 Processor -- start 2023-09-27T23:29:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.100+0000 7fc82e15e700 1 -- start start 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc82815a210 0x7fc82806b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8280c97c0 0x7fc82806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8280cb030 con 0x7fc82815a5e0 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8281599a0 con 0x7fc82815a210 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8280cb290 con 0x7fc8280c97c0 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc82815a210 0x7fc82806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc82815a210 0x7fc82806b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53722/0 (socket says 172.21.15.64:53722) 2023-09-27T23:29:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 -- 172.21.15.64:0/2550869315 learned_addr learned my addr 172.21.15.64:0/2550869315 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:29.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc827fff700 1 --2- 172.21.15.64:0/2550869315 >> v2:172.21.15.123:3300/0 conn(0x7fc8280c97c0 0x7fc82806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 -- 172.21.15.64:0/2550869315 >> v2:172.21.15.123:3300/0 conn(0x7fc8280c97c0 msgr2=0x7fc82806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 --2- 172.21.15.64:0/2550869315 >> v2:172.21.15.123:3300/0 conn(0x7fc8280c97c0 0x7fc82806b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 -- 172.21.15.64:0/2550869315 >> v1:172.21.15.123:6789/0 conn(0x7fc82815a5e0 legacy=0x7fc8280c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 -- 172.21.15.64:0/2550869315 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8280cb570 con 0x7fc82815a210 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8267fc700 1 -- 172.21.15.64:0/2550869315 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539566420 0 0) 0x7fc8280cb030 con 0x7fc82815a5e0 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc827fff700 1 --2- 172.21.15.64:0/2550869315 >> v2:172.21.15.123:3300/0 conn(0x7fc8280c97c0 0x7fc82806b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:29.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8277fe700 1 --2- 172.21.15.64:0/2550869315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82806b280 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc828154fc0 tx=0x7fc818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636ea9475e9df1c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8267fc700 1 -- 172.21.15.64:0/2550869315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818009e70 con 0x7fc82815a210 2023-09-27T23:29:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8267fc700 1 -- 172.21.15.64:0/2550869315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81800b860 con 0x7fc82815a210 2023-09-27T23:29:29.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc8267fc700 1 -- 172.21.15.64:0/2550869315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800bc00 con 0x7fc82815a210 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- 172.21.15.64:0/2550869315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 msgr2=0x7fc82806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 --2- 172.21.15.64:0/2550869315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82806b280 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc828154fc0 tx=0x7fc818009250 comp rx=0 tx=0).stop 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- 172.21.15.64:0/2550869315 shutdown_connections 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 --2- 172.21.15.64:0/2550869315 >> v2:172.21.15.123:3300/0 conn(0x7fc8280c97c0 0x7fc82806b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 --2- 172.21.15.64:0/2550869315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82806b280 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- 172.21.15.64:0/2550869315 >> 172.21.15.64:0/2550869315 conn(0x7fc828067410 msgr2=0x7fc8280677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- 172.21.15.64:0/2550869315 shutdown_connections 2023-09-27T23:29:29.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.104+0000 7fc82e15e700 1 -- 172.21.15.64:0/2550869315 wait complete. 2023-09-27T23:29:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 Processor -- start 2023-09-27T23:29:29.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 -- start start 2023-09-27T23:29:29.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8280c97c0 0x7fc82813f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:29.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82816a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:29.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82815a5e0 0x7fc82816a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:29.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b0ea0 con 0x7fc82815a210 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828152570 con 0x7fc82815a5e0 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc82e15e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc828151c40 con 0x7fc8280c97c0 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82816a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc8277fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8280c97c0 0x7fc82813f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82816a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53734/0 (socket says 172.21.15.64:53734) 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 -- 172.21.15.64:0/748959514 learned_addr learned my addr 172.21.15.64:0/748959514 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc827fff700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82815a5e0 0x7fc82816a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 -- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8280c97c0 msgr2=0x7fc82813f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8280c97c0 0x7fc82813f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 -- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82815a5e0 msgr2=0x7fc82816a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82815a5e0 0x7fc82816a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.108+0000 7fc826ffd700 1 -- 172.21.15.64:0/748959514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820009a20 con 0x7fc82815a210 2023-09-27T23:29:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc826ffd700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82816a370 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c7e0 tx=0x7fc81c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c011590 con 0x7fc82815a210 2023-09-27T23:29:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0116f0 con 0x7fc82815a210 2023-09-27T23:29:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc82e15e700 1 -- 172.21.15.64:0/748959514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8180094f0 con 0x7fc82815a210 2023-09-27T23:29:29.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c011ab0 con 0x7fc82815a210 2023-09-27T23:29:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc82e15e700 1 -- 172.21.15.64:0/748959514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc828152200 con 0x7fc82815a210 2023-09-27T23:29:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8000052b0 con 0x7fc82815a210 2023-09-27T23:29:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc81c01c460 con 0x7fc82815a210 2023-09-27T23:29:29.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80f7fe700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc814060a70 0x7fc814062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc81c07cdf0 con 0x7fc82815a210 2023-09-27T23:29:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.112+0000 7fc8277fe700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc814060a70 0x7fc814062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:29.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.116+0000 7fc8277fe700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc814060a70 0x7fc814062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc828154fc0 tx=0x7fc818017000 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:29.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.124+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc81c04cd80 con 0x7fc82815a210 2023-09-27T23:29:29.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.324+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc800005830 con 0x7fc82815a210 2023-09-27T23:29:29.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.324+0000 7fc80f7fe700 1 -- 172.21.15.64:0/748959514 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc81c0503a0 con 0x7fc82815a210 2023-09-27T23:29:29.330 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:29:29.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc814060a70 msgr2=0x7fc814062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:29.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc814060a70 0x7fc814062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc828154fc0 tx=0x7fc818017000 comp rx=0 tx=0).stop 2023-09-27T23:29:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 msgr2=0x7fc82816a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82816a370 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c7e0 tx=0x7fc81c007de0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 shutdown_connections 2023-09-27T23:29:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc814060a70 0x7fc814062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82815a5e0 0x7fc82816a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc82815a210 0x7fc82816a370 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 --2- 172.21.15.64:0/748959514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8280c97c0 0x7fc82813f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 >> 172.21.15.64:0/748959514 conn(0x7fc828067410 msgr2=0x7fc8280c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 shutdown_connections 2023-09-27T23:29:29.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:29.328+0000 7fc80d7fa700 1 -- 172.21.15.64:0/748959514 wait complete. 2023-09-27T23:29:29.344 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:29:29.344 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:29:29.344 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:29:29.344 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:29:29.344 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:29.345 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:29:29.345 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:29:29.345 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:29:29.345 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:29:29.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:29.346 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:29:29.353 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:29:29.354 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41615sec preferred_lft 41615sec 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:29:29.355 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:29:29.356 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.357 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:29:29.357 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:29:29.357 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:29:29.357 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:29:29.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:29.357 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:29:29.414 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:29:29.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:29.417 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:29:29.425 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:29:29.425 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:29:29.425 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:29:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:29:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:29:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:29:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:29:29.426 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:29:29.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:29.427 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:29:29.427 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:29:29.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:29:29.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:29:29.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:29:29.428 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:29:29.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:29.468 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:29:29.475 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:29:29.475 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:29:29.475 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:29:29.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:29.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:29:29.532 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:29:29.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:29:29.532 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:29.536 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:29:29.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:29:29.537 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:29.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:29:29.557 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:29:29.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:29:29.614 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:29:29.615 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:29:29.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:29.615 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:29:29.622 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:29:29.622 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:29:29.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:29:29.637 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:29:29.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:29:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:29:29.693 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:29:29.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:29.693 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:29:29.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:29:29.704+0000 7f0cf9a68380 -1 init, newargv = 0x562f38d3daf0 newargc=15 2023-09-27T23:29:29.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[49524]: starting ceph client 2023-09-27T23:29:29.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[49524]: starting fuse 2023-09-27T23:29:30.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:29:30.819 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:29:30.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:29:30.829 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:29:30.830 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:29:30.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:30.831 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:29:30.878 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:29:30.879 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:29:30.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:29:30.879 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:29:30.879 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:29:30.879 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:29:30.879 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:29:30.880 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:29:30.881 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:29:30.970 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.49524.asok 2023-09-27T23:29:31.155 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49524.asok 2023-09-27T23:29:31.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:31.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49524.asok status 2023-09-27T23:29:31.246 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:29:31.246 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:29:31.246 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "49524", 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:29:31.247 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:29:31.248 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 6804, 2023-09-27T23:29:31.248 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:29:31.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:29:31.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 6804 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 963557435 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:31.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 963557435 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.6804 192.168.0.1:0/963557435", 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/963557435", 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 27, 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 87, 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:29:31.251 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:29:31.252 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:29:31.252 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:29:31.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:31.256 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:29:31.256 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:31.264 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:29:31.264 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:31.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:31.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:31.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:29:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 Processor -- start 2023-09-27T23:29:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- start start 2023-09-27T23:29:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f406a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79f406d210 0x7f79f406d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f406c7c0 con 0x7f79f40b1410 2023-09-27T23:29:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f406cd80 con 0x7f79f406d210 2023-09-27T23:29:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f406af10 con 0x7f79f406a580 2023-09-27T23:29:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79f406d210 0x7f79f406d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f79f406d210 0x7f79f406d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53736/0 (socket says 172.21.15.64:53736) 2023-09-27T23:29:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 -- 172.21.15.64:0/2935889075 learned_addr learned my addr 172.21.15.64:0/2935889075 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79f99c7700 1 --2- 172.21.15.64:0/2935889075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f406a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 -- 172.21.15.64:0/2935889075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 msgr2=0x7f79f406a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 --2- 172.21.15.64:0/2935889075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f406a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 -- 172.21.15.64:0/2935889075 >> v1:172.21.15.123:6789/0 conn(0x7f79f40b1410 legacy=0x7f79f40ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 -- 172.21.15.64:0/2935889075 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f406b8f0 con 0x7f79f406d210 2023-09-27T23:29:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fa9c9700 1 --2- 172.21.15.64:0/2935889075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f406d650 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009f20 tx=0x7f79ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f415a249a056fd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79f91c6700 1 -- 172.21.15.64:0/2935889075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00e070 con 0x7f79f406d210 2023-09-27T23:29:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79f91c6700 1 -- 172.21.15.64:0/2935889075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00b9d0 con 0x7f79f406d210 2023-09-27T23:29:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79f91c6700 1 -- 172.21.15.64:0/2935889075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0092e0 con 0x7f79f406d210 2023-09-27T23:29:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- 172.21.15.64:0/2935889075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 msgr2=0x7f79f406d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/2935889075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f406d650 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009f20 tx=0x7f79ec009840 comp rx=0 tx=0).stop 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- 172.21.15.64:0/2935889075 shutdown_connections 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/2935889075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f406d650 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/2935889075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- 172.21.15.64:0/2935889075 >> 172.21.15.64:0/2935889075 conn(0x7f79f4067bb0 msgr2=0x7f79f40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- 172.21.15.64:0/2935889075 shutdown_connections 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.384+0000 7f79fb1ca700 1 -- 172.21.15.64:0/2935889075 wait complete. 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 Processor -- start 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 -- start start 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f41690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f40b1410 0x7f79f40c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f416caa0 con 0x7f79f406d210 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f416c750 con 0x7f79f40b1410 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fb1ca700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41699a0 con 0x7f79f406a580 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f41690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f41690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53738/0 (socket says 172.21.15.64:53738) 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fa1c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 -- 172.21.15.64:0/284744591 learned_addr learned my addr 172.21.15.64:0/284744591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fa9c9700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f40b1410 0x7f79f40c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 -- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 msgr2=0x7f79f4148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f4148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 -- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f40b1410 msgr2=0x7f79f40c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f40b1410 0x7f79f40c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 -- 172.21.15.64:0/284744591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec009ae0 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79fa9c9700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f40b1410 0x7f79f40c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79f99c7700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f41690f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f79f000ec80 tx=0x7f79f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000c850 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f000c9b0 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.388+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000cd70 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f4169c80 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f4169630 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f404be40 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f79f0013680 con 0x7f79f406d210 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79ea7fc700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79d8060a70 0x7f79d8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f79f007d410 con 0x7f79f406d210 2023-09-27T23:29:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.392+0000 7f79fa1c8700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79d8060a70 0x7f79d8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.396+0000 7f79fa1c8700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79d8060a70 0x7f79d8062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79e40039e0 tx=0x7f79e4007ec0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:31.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.400+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f79f004d3a0 con 0x7f79f406d210 2023-09-27T23:29:31.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.588+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f79f40aaaa0 con 0x7f79d8060a70 2023-09-27T23:29:31.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.592+0000 7f79ea7fc700 1 -- 172.21.15.64:0/284744591 <== mgr.5816 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f79f40aaaa0 con 0x7f79d8060a70 2023-09-27T23:29:31.597 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:29:31.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:31.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:29:31.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:31.597 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:29:31.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79d8060a70 msgr2=0x7f79d8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79d8060a70 0x7f79d8062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79e40039e0 tx=0x7f79e4007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 msgr2=0x7f79f41690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f41690f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f79f000ec80 tx=0x7f79f00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 shutdown_connections 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79d8060a70 0x7f79d8062f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f40b1410 0x7f79f40c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f406d210 0x7f79f41690f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 --2- 172.21.15.64:0/284744591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79f406a580 0x7f79f4148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 >> 172.21.15.64:0/284744591 conn(0x7f79f4067bb0 msgr2=0x7f79f41561e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 shutdown_connections 2023-09-27T23:29:31.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.596+0000 7f79fb1ca700 1 -- 172.21.15.64:0/284744591 wait complete. 2023-09-27T23:29:31.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:29:31.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 Processor -- start 2023-09-27T23:29:31.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 -- start start 2023-09-27T23:29:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f630c0c8bf0 0x7f630c0ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 0x7f630c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c0ae110 con 0x7f630c0ace10 2023-09-27T23:29:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c157110 con 0x7f630c0c8bf0 2023-09-27T23:29:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c06e0c0 con 0x7f630c06d390 2023-09-27T23:29:31.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f630b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f630c0c8bf0 0x7f630c0ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 0x7f630c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f630bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f630c0ace10 0x7f630c0c84e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59400/0 (socket says 172.21.15.64:59400) 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f630bfff700 1 -- 172.21.15.64:0/3404118948 learned_addr learned my addr 172.21.15.64:0/3404118948 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 0x7f630c06d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:50054/0 (socket says 172.21.15.64:50054) 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 -- 172.21.15.64:0/3404118948 >> v2:172.21.15.123:3300/0 conn(0x7f630c0c8bf0 msgr2=0x7f630c0ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 --2- 172.21.15.64:0/3404118948 >> v2:172.21.15.123:3300/0 conn(0x7f630c0c8bf0 0x7f630c0ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 -- 172.21.15.64:0/3404118948 >> v1:172.21.15.123:6789/0 conn(0x7f630c0ace10 legacy=0x7f630c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 -- 172.21.15.64:0/3404118948 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0ad7e0 con 0x7f630c06d390 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6310c11700 1 --2- 172.21.15.64:0/3404118948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 0x7f630c06d830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6304009e60 tx=0x7f6304009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65a756be03eab66 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f630affd700 1 -- 172.21.15.64:0/3404118948 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400e070 con 0x7f630c06d390 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f630affd700 1 -- 172.21.15.64:0/3404118948 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400bb60 con 0x7f630c06d390 2023-09-27T23:29:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f630affd700 1 -- 172.21.15.64:0/3404118948 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304012430 con 0x7f630c06d390 2023-09-27T23:29:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 -- 172.21.15.64:0/3404118948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 msgr2=0x7f630c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.728+0000 7f6311412700 1 --2- 172.21.15.64:0/3404118948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 0x7f630c06d830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6304009e60 tx=0x7f6304009780 comp rx=0 tx=0).stop 2023-09-27T23:29:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- 172.21.15.64:0/3404118948 shutdown_connections 2023-09-27T23:29:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 --2- 172.21.15.64:0/3404118948 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c06d390 0x7f630c06d830 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 --2- 172.21.15.64:0/3404118948 >> v2:172.21.15.123:3300/0 conn(0x7f630c0c8bf0 0x7f630c0ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- 172.21.15.64:0/3404118948 >> 172.21.15.64:0/3404118948 conn(0x7f630c0cdc00 msgr2=0x7f630c0ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- 172.21.15.64:0/3404118948 shutdown_connections 2023-09-27T23:29:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- 172.21.15.64:0/3404118948 wait complete. 2023-09-27T23:29:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 Processor -- start 2023-09-27T23:29:31.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- start start 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630c06d390 0x7f630c0c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c0ace10 0x7f630c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 0x7f630c14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0c5f50 con 0x7f630c0c8bf0 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0c5340 con 0x7f630c06d390 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6311412700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0c5620 con 0x7f630c0ace10 2023-09-27T23:29:31.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6310c11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 0x7f630c14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6310c11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 0x7f630c14f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53754/0 (socket says 172.21.15.64:53754) 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f6310c11700 1 -- 172.21.15.64:0/2586687664 learned_addr learned my addr 172.21.15.64:0/2586687664 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f630b7fe700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c0ace10 0x7f630c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.732+0000 7f630bfff700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630c06d390 0x7f630c0c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6310c11700 1 -- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c0ace10 msgr2=0x7f630c1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6310c11700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c0ace10 0x7f630c1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6310c11700 1 -- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630c06d390 msgr2=0x7f630c0c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6310c11700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630c06d390 0x7f630c0c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6310c11700 1 -- 172.21.15.64:0/2586687664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304009a20 con 0x7f630c0c8bf0 2023-09-27T23:29:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6310c11700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 0x7f630c14f720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f630c06e200 tx=0x7f6304019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304012ea0 con 0x7f630c0c8bf0 2023-09-27T23:29:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6304027470 con 0x7f630c0c8bf0 2023-09-27T23:29:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c0c5900 con 0x7f630c0c8bf0 2023-09-27T23:29:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304027850 con 0x7f630c0c8bf0 2023-09-27T23:29:31.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c14fc60 con 0x7f630c0c8bf0 2023-09-27T23:29:31.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c0c6100 con 0x7f630c0c8bf0 2023-09-27T23:29:31.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6304024070 con 0x7f630c0c8bf0 2023-09-27T23:29:31.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.736+0000 7f62f3fff700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62f8060a70 0x7f62f8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:31.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.740+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f630407fed0 con 0x7f630c0c8bf0 2023-09-27T23:29:31.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.740+0000 7f630bfff700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62f8060a70 0x7f62f8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:31.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.740+0000 7f630bfff700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62f8060a70 0x7f62f8062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007130 tx=0x7f62fc007eb0 comp rx=0 tx=0).ready entity=mgr.5816 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.744+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f630404fee0 con 0x7f630c0c8bf0 2023-09-27T23:29:31.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:31.948+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f630c04be40 con 0x7f630c0c8bf0 2023-09-27T23:29:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.160+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 27) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f6304003040 con 0x7f630c0c8bf0 2023-09-27T23:29:32.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.164+0000 7f62f3fff700 1 -- 172.21.15.64:0/2586687664 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v27) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6304053970 con 0x7f630c0c8bf0 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62f8060a70 msgr2=0x7f62f8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62f8060a70 0x7f62f8062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007130 tx=0x7f62fc007eb0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 msgr2=0x7f630c14f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 0x7f630c14f720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f630c06e200 tx=0x7f6304019000 comp rx=0 tx=0).stop 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 shutdown_connections 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62f8060a70 0x7f62f8062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f630c0c8bf0 0x7f630c14f720 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f630c0ace10 0x7f630c1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 --2- 172.21.15.64:0/2586687664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630c06d390 0x7f630c0c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 >> 172.21.15.64:0/2586687664 conn(0x7f630c0cdc00 msgr2=0x7f630c0b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:32.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 shutdown_connections 2023-09-27T23:29:32.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.168+0000 7f6311412700 1 -- 172.21.15.64:0/2586687664 wait complete. 2023-09-27T23:29:32.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:29:32.289 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:32.286+0000 7f5e1e318000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:29:32.293 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:32.288+0000 7fe86a1fc000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:29:32.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.292+0000 7f3fc759e700 1 Processor -- start 2023-09-27T23:29:32.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.292+0000 7f3fc759e700 1 -- start start 2023-09-27T23:29:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3fc8150200 0x7f3fc80c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ae770 0x7f3fc80aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fc80ac720 0x7f3fc80acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80af530 con 0x7f3fc80ac720 2023-09-27T23:29:32.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80ad0c0 con 0x7f3fc8150200 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80af250 con 0x7f3fc80ae770 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fc80ac720 0x7f3fc80acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fc80ac720 0x7f3fc80acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53768/0 (socket says 172.21.15.64:53768) 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 -- 172.21.15.64:0/3126156651 learned_addr learned my addr 172.21.15.64:0/3126156651 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc5d9b700 1 --2- 172.21.15.64:0/3126156651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ae770 0x7f3fc80aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc659c700 1 --2- 172.21.15.64:0/3126156651 >> v2:172.21.15.123:3300/0 conn(0x7f3fc8150200 0x7f3fc80c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 -- 172.21.15.64:0/3126156651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ae770 msgr2=0x7f3fc80aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 --2- 172.21.15.64:0/3126156651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ae770 0x7f3fc80aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 -- 172.21.15.64:0/3126156651 >> v2:172.21.15.123:3300/0 conn(0x7f3fc8150200 msgr2=0x7f3fc80c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 --2- 172.21.15.64:0/3126156651 >> v2:172.21.15.123:3300/0 conn(0x7f3fc8150200 0x7f3fc80c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 -- 172.21.15.64:0/3126156651 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80ca8a0 con 0x7f3fc80ac720 2023-09-27T23:29:32.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc6d9d700 1 --2- 172.21.15.64:0/3126156651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc80ac720 0x7f3fc80acb80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3fc806b4e0 tx=0x7f3fc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304bee0bd0c3a84f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:32.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc559a700 1 -- 172.21.15.64:0/3126156651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00092e0 con 0x7f3fc80ac720 2023-09-27T23:29:32.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc559a700 1 -- 172.21.15.64:0/3126156651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc000ba50 con 0x7f3fc80ac720 2023-09-27T23:29:32.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc559a700 1 -- 172.21.15.64:0/3126156651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000be10 con 0x7f3fc80ac720 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- 172.21.15.64:0/3126156651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc80ac720 msgr2=0x7f3fc80acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- 172.21.15.64:0/3126156651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc80ac720 0x7f3fc80acb80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3fc806b4e0 tx=0x7f3fc0009840 comp rx=0 tx=0).stop 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- 172.21.15.64:0/3126156651 shutdown_connections 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- 172.21.15.64:0/3126156651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc80ac720 0x7f3fc80acb80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- 172.21.15.64:0/3126156651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ae770 0x7f3fc80aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 --2- 172.21.15.64:0/3126156651 >> v2:172.21.15.123:3300/0 conn(0x7f3fc8150200 0x7f3fc80c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- 172.21.15.64:0/3126156651 >> 172.21.15.64:0/3126156651 conn(0x7f3fc80ccb30 msgr2=0x7f3fc80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- 172.21.15.64:0/3126156651 shutdown_connections 2023-09-27T23:29:32.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.296+0000 7f3fc759e700 1 -- 172.21.15.64:0/3126156651 wait complete. 2023-09-27T23:29:32.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 Processor -- start 2023-09-27T23:29:32.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 -- start start 2023-09-27T23:29:32.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 0x7f3fc80c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fc80ae770 0x7f3fc8069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc8150200 0x7f3fc8069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8068a70 con 0x7f3fc8150200 2023-09-27T23:29:32.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8067e60 con 0x7f3fc80ae770 2023-09-27T23:29:32.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8068140 con 0x7f3fc80ac720 2023-09-27T23:29:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 0x7f3fc80c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fc80ae770 0x7f3fc8069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 0x7f3fc80c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:50098/0 (socket says 172.21.15.64:50098) 2023-09-27T23:29:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc659c700 1 -- 172.21.15.64:0/2527419054 learned_addr learned my addr 172.21.15.64:0/2527419054 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:32.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc8150200 0x7f3fc8069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 -- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 msgr2=0x7f3fc80c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 0x7f3fc80c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 -- 172.21.15.64:0/2527419054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc8150200 msgr2=0x7f3fc8069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc8150200 0x7f3fc8069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 -- 172.21.15.64:0/2527419054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009ae0 con 0x7f3fc80ae770 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc659c700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 0x7f3fc80c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc5d9b700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fc80ae770 0x7f3fc8069550 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d890 tx=0x7f3fbc008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0125f0 con 0x7f3fc80ae770 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc012770 con 0x7f3fc80ae770 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8068420 con 0x7f3fc80ae770 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc012b30 con 0x7f3fc80ae770 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.300+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc8067af0 con 0x7f3fc80ae770 2023-09-27T23:29:32.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.304+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc814ab30 con 0x7f3fc80ae770 2023-09-27T23:29:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.304+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 27) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f3fbc013480 con 0x7f3fc80ae770 2023-09-27T23:29:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.304+0000 7f3fb67fc700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fa8060ac0 0x7f3fa8062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.304+0000 7f3fc659c700 1 -- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fa8060ac0 msgr2=0x7f3fa8062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.304+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3fbc07dec0 con 0x7f3fc80ae770 2023-09-27T23:29:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.304+0000 7f3fc659c700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fa8060ac0 0x7f3fa8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:32.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.312+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3fbc051a00 con 0x7f3fc80ae770 2023-09-27T23:29:32.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.480+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3fc804be40 con 0x7f3fc80ae770 2023-09-27T23:29:32.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.488+0000 7f3fb67fc700 1 -- 172.21.15.64:0/2527419054 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3fbc051590 con 0x7f3fc80ae770 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fa8060ac0 msgr2=0x7f3fa8062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fa8060ac0 0x7f3fa8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fc80ae770 msgr2=0x7f3fc8069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fc80ae770 0x7f3fc8069550 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d890 tx=0x7f3fbc008040 comp rx=0 tx=0).stop 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 shutdown_connections 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fa8060ac0 0x7f3fa8062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fc8150200 0x7f3fc8069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fc80ae770 0x7f3fc8069550 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 --2- 172.21.15.64:0/2527419054 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fc80ac720 0x7f3fc80c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 >> 172.21.15.64:0/2527419054 conn(0x7f3fc80ccb30 msgr2=0x7f3fc814d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 shutdown_connections 2023-09-27T23:29:32.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.492+0000 7f3fc759e700 1 -- 172.21.15.64:0/2527419054 wait complete. 2023-09-27T23:29:32.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:29:32.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 Processor -- start 2023-09-27T23:29:32.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 -- start start 2023-09-27T23:29:32.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4694142680 0x7f46940c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f469406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46940cc7e0 con 0x7f4694142a50 2023-09-27T23:29:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46940cbbf0 con 0x7f4694142680 2023-09-27T23:29:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.616+0000 7f4698b63700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940cbeb0 con 0x7f46940c8f40 2023-09-27T23:29:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f46937fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4694142680 0x7f46940c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f469406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4693fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4694142a50 0x7f46940ca7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59410/0 (socket says 172.21.15.64:59410) 2023-09-27T23:29:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f469406d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:50104/0 (socket says 172.21.15.64:50104) 2023-09-27T23:29:32.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4693fff700 1 -- 172.21.15.64:0/3726728543 learned_addr learned my addr 172.21.15.64:0/3726728543 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:32.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f46927fc700 1 -- 172.21.15.64:0/3726728543 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086318239 0 0) 0x7f46940cc7e0 con 0x7f4694142a50 2023-09-27T23:29:32.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f46927fc700 1 -- 172.21.15.64:0/3726728543 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f467c003570 con 0x7f4694142a50 2023-09-27T23:29:32.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 -- 172.21.15.64:0/3726728543 >> v2:172.21.15.123:3300/0 conn(0x7f4694142680 msgr2=0x7f46940c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:32.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 --2- 172.21.15.64:0/3726728543 >> v2:172.21.15.123:3300/0 conn(0x7f4694142680 0x7f46940c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 -- 172.21.15.64:0/3726728543 >> v1:172.21.15.123:6789/0 conn(0x7f4694142a50 legacy=0x7f46940ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 -- 172.21.15.64:0/3726728543 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940cc190 con 0x7f46940c8f40 2023-09-27T23:29:32.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4692ffd700 1 --2- 172.21.15.64:0/3726728543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f469406d050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f46880098e0 tx=0x7f46880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=893a3bf47785ca7b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:32.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f46927fc700 1 -- 172.21.15.64:0/3726728543 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688011630 con 0x7f46940c8f40 2023-09-27T23:29:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f46927fc700 1 -- 172.21.15.64:0/3726728543 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688011790 con 0x7f46940c8f40 2023-09-27T23:29:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f46927fc700 1 -- 172.21.15.64:0/3726728543 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688011b30 con 0x7f46940c8f40 2023-09-27T23:29:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 -- 172.21.15.64:0/3726728543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 msgr2=0x7f469406d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 --2- 172.21.15.64:0/3726728543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f469406d050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f46880098e0 tx=0x7f46880091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 -- 172.21.15.64:0/3726728543 shutdown_connections 2023-09-27T23:29:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 --2- 172.21.15.64:0/3726728543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f469406d050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 --2- 172.21.15.64:0/3726728543 >> v2:172.21.15.123:3300/0 conn(0x7f4694142680 0x7f46940c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 -- 172.21.15.64:0/3726728543 >> 172.21.15.64:0/3726728543 conn(0x7f46940ccb30 msgr2=0x7f46940ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:32.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 -- 172.21.15.64:0/3726728543 shutdown_connections 2023-09-27T23:29:32.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 -- 172.21.15.64:0/3726728543 wait complete. 2023-09-27T23:29:32.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 Processor -- start 2023-09-27T23:29:32.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.620+0000 7f4698b63700 1 -- start start 2023-09-27T23:29:32.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f46940ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4694142680 0x7f46940672f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 0x7f4694067830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940a96c0 con 0x7f4694142a50 2023-09-27T23:29:32.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469416dc90 con 0x7f4694142680 2023-09-27T23:29:32.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940b05e0 con 0x7f46940c8f40 2023-09-27T23:29:32.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 0x7f4694067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 0x7f4694067830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53784/0 (socket says 172.21.15.64:53784) 2023-09-27T23:29:32.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 -- 172.21.15.64:0/2432802431 learned_addr learned my addr 172.21.15.64:0/2432802431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:32.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f46937fe700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f46940ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4692ffd700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4694142680 0x7f46940672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:32.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 -- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 msgr2=0x7f46940ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f46940ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 -- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4694142680 msgr2=0x7f46940672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4694142680 0x7f46940672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 -- 172.21.15.64:0/2432802431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688009420 con 0x7f4694142a50 2023-09-27T23:29:32.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4693fff700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 0x7f4694067830 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f468c001460 tx=0x7f468c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:32.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00a870 con 0x7f4694142a50 2023-09-27T23:29:32.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c00a9d0 con 0x7f4694142a50 2023-09-27T23:29:32.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46940ab710 con 0x7f4694142a50 2023-09-27T23:29:32.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00ad50 con 0x7f4694142a50 2023-09-27T23:29:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46940aff90 con 0x7f4694142a50 2023-09-27T23:29:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469404be40 con 0x7f4694142a50 2023-09-27T23:29:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.624+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 27) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f468c007300 con 0x7f4694142a50 2023-09-27T23:29:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.628+0000 7f467b7fe700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4680060ac0 0x7f4680062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.628+0000 7f46937fe700 1 -- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4680060ac0 msgr2=0x7f4680062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:32.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.628+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f468c07d410 con 0x7f4694142a50 2023-09-27T23:29:32.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.628+0000 7f46937fe700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4680060ac0 0x7f4680062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:32.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.632+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f468c04d4c0 con 0x7f4694142a50 2023-09-27T23:29:32.715 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:32.710+0000 7f5e1e318000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:29:32.748 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:32.744+0000 7fe86a1fc000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:29:32.763 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:32.758+0000 7f5e1e318000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:29:32.800 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:32.796+0000 7fe86a1fc000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:29:32.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.796+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f46940cc470 con 0x7f4694142a50 2023-09-27T23:29:32.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.804+0000 7f467b7fe700 1 -- 172.21.15.64:0/2432802431 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f468c050ae0 con 0x7f4694142a50 2023-09-27T23:29:32.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4680060ac0 msgr2=0x7f4680062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:32.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4680060ac0 0x7f4680062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 msgr2=0x7f4694067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:32.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 0x7f4694067830 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f468c001460 tx=0x7f468c001a40 comp rx=0 tx=0).stop 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 shutdown_connections 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4680060ac0 0x7f4680062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4694142a50 0x7f4694067830 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4694142680 0x7f46940672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 --2- 172.21.15.64:0/2432802431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46940c8f40 0x7f46940ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 >> 172.21.15.64:0/2432802431 conn(0x7f46940ccb30 msgr2=0x7f46940c1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 shutdown_connections 2023-09-27T23:29:32.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.808+0000 7f4698b63700 1 -- 172.21.15.64:0/2432802431 wait complete. 2023-09-27T23:29:32.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:32.829 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:29:32.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-09-27T23:29:32.870 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:32.866+0000 7f5e1e318000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:32.917 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:32.912+0000 7fe86a1fc000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:33.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 Processor -- start 2023-09-27T23:29:33.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 -- start start 2023-09-27T23:29:33.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240c0d30 0x7f5e24068050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e240aeb50 0x7f5e240ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e240ad6b0 con 0x7f5e240c1100 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e24068730 con 0x7f5e240aeb50 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e24068a10 con 0x7f5e240c0d30 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e22d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e240aeb50 0x7f5e240ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240c0d30 0x7f5e24068050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e22d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e240aeb50 0x7f5e240ac5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:34364/0 (socket says 172.21.15.64:34364) 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e22d9d700 1 -- 172.21.15.64:0/2438446967 learned_addr learned my addr 172.21.15.64:0/2438446967 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e2259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240c0d30 0x7f5e24068050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:50112/0 (socket says 172.21.15.64:50112) 2023-09-27T23:29:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:32.996+0000 7f5e21d9b700 1 --1- 172.21.15.64:0/2438446967 >> v1:172.21.15.123:6789/0 conn(0x7f5e240c1100 0x7f5e240c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:59412/0 (socket says 172.21.15.64:59412) 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 -- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240c0d30 msgr2=0x7f5e24068050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 --2- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240c0d30 0x7f5e24068050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 -- 172.21.15.64:0/2438446967 >> v1:172.21.15.123:6789/0 conn(0x7f5e240c1100 legacy=0x7f5e240c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 -- 172.21.15.64:0/2438446967 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24068cf0 con 0x7f5e240aeb50 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 --2- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240aeb50 0x7f5e240ac5b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009f20 tx=0x7f5e1c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f0a505de048a79 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2159a700 1 -- 172.21.15.64:0/2438446967 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00e070 con 0x7f5e240aeb50 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2159a700 1 -- 172.21.15.64:0/2438446967 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00b9d0 con 0x7f5e240aeb50 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2159a700 1 -- 172.21.15.64:0/2438446967 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0092e0 con 0x7f5e240aeb50 2023-09-27T23:29:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240aeb50 msgr2=0x7f5e240ac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 --2- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240aeb50 0x7f5e240ac5b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009f20 tx=0x7f5e1c009840 comp rx=0 tx=0).stop 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- 172.21.15.64:0/2438446967 shutdown_connections 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 --2- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240aeb50 0x7f5e240ac5b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 --2- 172.21.15.64:0/2438446967 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240c0d30 0x7f5e24068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- 172.21.15.64:0/2438446967 >> 172.21.15.64:0/2438446967 conn(0x7f5e240ce5e0 msgr2=0x7f5e240ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- 172.21.15.64:0/2438446967 shutdown_connections 2023-09-27T23:29:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- 172.21.15.64:0/2438446967 wait complete. 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 Processor -- start 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- start start 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240aeb50 0x7f5e240c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240c0d30 0x7f5e240afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e240c1100 0x7f5e24155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240c29f0 con 0x7f5e240c1100 2023-09-27T23:29:33.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240c2cd0 con 0x7f5e240c0d30 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240c2fb0 con 0x7f5e240aeb50 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e240c1100 0x7f5e24155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e240c1100 0x7f5e24155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53792/0 (socket says 172.21.15.64:53792) 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e2259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240aeb50 0x7f5e240c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e22d9d700 1 -- 172.21.15.64:0/1348197723 learned_addr learned my addr 172.21.15.64:0/1348197723 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.000+0000 7f5e21d9b700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240c0d30 0x7f5e240afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e22d9d700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240aeb50 msgr2=0x7f5e240c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e22d9d700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e240aeb50 0x7f5e240c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e22d9d700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240c0d30 msgr2=0x7f5e240afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e22d9d700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e240c0d30 0x7f5e240afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e22d9d700 1 -- 172.21.15.64:0/1348197723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c009ae0 con 0x7f5e240c1100 2023-09-27T23:29:33.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e22d9d700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e240c1100 0x7f5e24155090 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00b610 tx=0x7f5e1c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:33.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c007b50 con 0x7f5e240c1100 2023-09-27T23:29:33.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c007cd0 con 0x7f5e240c1100 2023-09-27T23:29:33.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e2359e700 1 -- 172.21.15.64:0/1348197723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e240c3290 con 0x7f5e240c1100 2023-09-27T23:29:33.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c01d670 con 0x7f5e240c1100 2023-09-27T23:29:33.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e2359e700 1 -- 172.21.15.64:0/1348197723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e240c2650 con 0x7f5e240c1100 2023-09-27T23:29:33.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e2359e700 1 -- 172.21.15.64:0/1348197723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e240c3790 con 0x7f5e240c1100 2023-09-27T23:29:33.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 27) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f5e1c003050 con 0x7f5e240c1100 2023-09-27T23:29:33.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e127fc700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:33.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e2259c700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 msgr2=0x7f5e04062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:33.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5e1c07ef40 con 0x7f5e240c1100 2023-09-27T23:29:33.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.004+0000 7f5e2259c700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.012+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5e1c04ee60 con 0x7f5e240c1100 2023-09-27T23:29:33.064 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.058+0000 7f5e1e318000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:29:33.112 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.110+0000 7f5e1e318000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:33.126 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.120+0000 7fe86a1fc000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:29:33.178 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.172+0000 7fe86a1fc000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:33.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.192+0000 7f5e2359e700 1 -- 172.21.15.64:0/1348197723 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e2404be40 con 0x7f5e04060ac0 2023-09-27T23:29:33.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.208+0000 7f5e2259c700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 msgr2=0x7f5e04062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:33.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.208+0000 7f5e2259c700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:33.262 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.258+0000 7f5e1e318000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:29:33.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.322+0000 7f5e1e318000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:29:33.335 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.328+0000 7fe86a1fc000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:29:33.389 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.386+0000 7f5e1e318000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:29:33.393 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.388+0000 7fe86a1fc000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:29:33.455 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.448+0000 7fe86a1fc000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:29:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.612+0000 7f5e2259c700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 msgr2=0x7f5e04062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:33.612+0000 7f5e2259c700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:29:33.709 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.706+0000 7f5e1e318000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:29:33.760 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.756+0000 7fe86a1fc000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:29:33.820 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:33.814+0000 7f5e1e318000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:29:33.875 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:33.868+0000 7fe86a1fc000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:29:34.098 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.094+0000 7f5e1e318000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:29:34.168 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.164+0000 7fe86a1fc000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:29:34.259 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.254+0000 7f5e1e318000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:34.323 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.318+0000 7f5e1e318000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:29:34.337 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.332+0000 7fe86a1fc000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:34.376 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.370+0000 7f5e1e318000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:29:34.405 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.400+0000 7fe86a1fc000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:29:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:34.416+0000 7f5e2259c700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 msgr2=0x7f5e04062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:34.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:34.416+0000 7f5e2259c700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:29:34.429 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.426+0000 7f5e1e318000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:29:34.459 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.452+0000 7fe86a1fc000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:29:34.516 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.512+0000 7fe86a1fc000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:29:34.603 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.598+0000 7f5e1e318000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:34.656 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.654+0000 7f5e1e318000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:29:34.694 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.688+0000 7fe86a1fc000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:34.717 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.714+0000 7f5e1e318000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:29:34.750 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.744+0000 7fe86a1fc000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:29:34.816 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.808+0000 7fe86a1fc000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:29:34.866 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.862+0000 7f5e1e318000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:29:34.926 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:34.922+0000 7f5e1e318000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:29:34.973 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:34.968+0000 7fe86a1fc000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:29:35.036 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:35.032+0000 7fe86a1fc000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:29:35.514 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.510+0000 7f5e1e318000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:29:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:35.544+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 28) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f5e1c07d1b0 con 0x7f5e240c1100 2023-09-27T23:29:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:35.544+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 msgr2=0x7f5e04062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:35.544+0000 7f5e127fc700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:35.663 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:35.656+0000 7fe86a1fc000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:29:35.698 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5de1cc9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.698 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5de1cc9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.698 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5de1cc9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.698 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5de1cc9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.698 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5de1cc9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.699 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5ddecc3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.699 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5ddecc3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.699 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5ddecc3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.699 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5ddecc3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:35.700 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:35.694+0000 7f5ddecc3700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.548+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 29) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f5e1c025400 con 0x7f5e240c1100 2023-09-27T23:29:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.548+0000 7f5e127fc700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:36.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.548+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e2404be40 con 0x7f5e04060ac0 2023-09-27T23:29:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.556+0000 7f5e2259c700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.560+0000 7f5e2259c700 1 --2- 172.21.15.64:0/1348197723 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e04060ac0 0x7f5e04062ef0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e140097c0 tx=0x7f5e140091a0 comp rx=0 tx=0).ready entity=mgr.6137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:36.575 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:36.570+0000 7f5de8cd7700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-09-27T23:29:36.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.568+0000 7f5e127fc700 1 -- 172.21.15.64:0/1348197723 <== mgr.6137 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f5e2404be40 con 0x7f5e04060ac0 2023-09-27T23:29:36.578 INFO:teuthology.orchestra.run.smithi064.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-09-27T23:29:36.582 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-09-27T23:29:36.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:29:36.583 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-09-27T23:29:36.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:29:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe9268700 1 Processor -- start 2023-09-27T23:29:36.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe9268700 1 -- start start 2023-09-27T23:29:36.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe9268700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefe40cb970 0x7fefe406acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe9268700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe40cbdf0 con 0x7fefe40cecd0 2023-09-27T23:29:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe9268700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe40cc0d0 con 0x7fefe4158e20 2023-09-27T23:29:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe9268700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe40cc3b0 con 0x7fefe40cb970 2023-09-27T23:29:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fefe40cecd0 0x7fefe406b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52582/0 (socket says 172.21.15.64:52582) 2023-09-27T23:29:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe37fe700 1 -- 172.21.15.64:0/1312577116 learned_addr learned my addr 172.21.15.64:0/1312577116 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.712+0000 7fefe27fc700 1 --2- 172.21.15.64:0/1312577116 >> v2:172.21.15.123:3300/0 conn(0x7fefe40cb970 0x7fefe406acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205637784 0 0) 0x7fefe40cbdf0 con 0x7fefe40cecd0 2023-09-27T23:29:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd0003570 con 0x7fefe40cecd0 2023-09-27T23:29:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437990506 0 0) 0x7fefe40cc0d0 con 0x7fefe4158e20 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe40cbdf0 con 0x7fefe4158e20 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe27fc700 1 -- 172.21.15.64:0/1312577116 >> v1:172.21.15.123:6789/0 conn(0x7fefe4158e20 legacy=0x7fefe40ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe27fc700 1 -- 172.21.15.64:0/1312577116 >> v1:172.21.15.64:6789/0 conn(0x7fefe40cecd0 legacy=0x7fefe406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe27fc700 1 -- 172.21.15.64:0/1312577116 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40cc690 con 0x7fefe40cb970 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe27fc700 1 --2- 172.21.15.64:0/1312577116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cb970 0x7fefe406acc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009910 tx=0x7fefd40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5e90d82b7508e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011630 con 0x7fefe40cb970 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4011790 con 0x7fefe40cb970 2023-09-27T23:29:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe1ffb700 1 -- 172.21.15.64:0/1312577116 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011b10 con 0x7fefe40cb970 2023-09-27T23:29:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe9268700 1 -- 172.21.15.64:0/1312577116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cb970 msgr2=0x7fefe406acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe9268700 1 --2- 172.21.15.64:0/1312577116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cb970 0x7fefe406acc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009910 tx=0x7fefd40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe9268700 1 -- 172.21.15.64:0/1312577116 shutdown_connections 2023-09-27T23:29:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe9268700 1 --2- 172.21.15.64:0/1312577116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cb970 0x7fefe406acc0 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009910 tx=0x7fefd40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.716+0000 7fefe9268700 1 -- 172.21.15.64:0/1312577116 >> 172.21.15.64:0/1312577116 conn(0x7fefe4067bb0 msgr2=0x7fefe40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.720+0000 7fefe9268700 1 -- 172.21.15.64:0/1312577116 shutdown_connections 2023-09-27T23:29:36.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.720+0000 7fefe9268700 1 -- 172.21.15.64:0/1312577116 wait complete. 2023-09-27T23:29:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.720+0000 7fefe9268700 1 Processor -- start 2023-09-27T23:29:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.720+0000 7fefe9268700 1 -- start start 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 0x7fefe414ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefe4158e20 0x7fefe40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cd0f0 0x7fefe40c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe414e2f0 con 0x7fefe40cecd0 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40b2740 con 0x7fefe40cd0f0 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe414f780 con 0x7fefe4158e20 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 0x7fefe414ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 0x7fefe414ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32778/0 (socket says 172.21.15.64:32778) 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 -- 172.21.15.64:0/755420855 learned_addr learned my addr 172.21.15.64:0/755420855 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe37fe700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cd0f0 0x7fefe40c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe27fc700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefe4158e20 0x7fefe40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 -- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefe4158e20 msgr2=0x7fefe40ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefe4158e20 0x7fefe40ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 -- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cd0f0 msgr2=0x7fefe40c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cd0f0 0x7fefe40c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 -- 172.21.15.64:0/755420855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009420 con 0x7fefe40cecd0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe2ffd700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 0x7fefe414ddb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c7e0 tx=0x7fefcc007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011590 con 0x7fefe40cecd0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc011710 con 0x7fefe40cecd0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe414ebd0 con 0x7fefe40cecd0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011ad0 con 0x7fefe40cecd0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe414f190 con 0x7fefe40cecd0 2023-09-27T23:29:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe4071120 con 0x7fefe40cecd0 2023-09-27T23:29:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.724+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 29) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fefcc01b3b0 con 0x7fefe40cecd0 2023-09-27T23:29:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.728+0000 7fefdaffd700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fefc4060820 0x7fefc4062cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.728+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fefcc07c2e0 con 0x7fefe40cecd0 2023-09-27T23:29:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.728+0000 7fefe27fc700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fefc4060820 0x7fefc4062cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:36.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.728+0000 7fefe27fc700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fefc4060820 0x7fefc4062cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fefe406a570 tx=0x7fefd4016000 comp rx=0 tx=0).ready entity=mgr.6137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.736+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fefcc04c570 con 0x7fefe40cecd0 2023-09-27T23:29:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:36.928+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fefe406a7a0 con 0x7fefe40cecd0 2023-09-27T23:29:37.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.608+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fefcc04c940 con 0x7fefe40cecd0 2023-09-27T23:29:37.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.608+0000 7fefdaffd700 1 -- 172.21.15.64:0/755420855 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v30) v1 ==== 93+0+0 (secure 0 0 0) 0x7fefcc050000 con 0x7fefe40cecd0 2023-09-27T23:29:37.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fefc4060820 msgr2=0x7fefc4062cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:37.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fefc4060820 0x7fefc4062cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fefe406a570 tx=0x7fefd4016000 comp rx=0 tx=0).stop 2023-09-27T23:29:37.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 msgr2=0x7fefe414ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:37.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 0x7fefe414ddb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c7e0 tx=0x7fefcc007da0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 shutdown_connections 2023-09-27T23:29:37.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fefc4060820 0x7fefc4062cd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefe40cd0f0 0x7fefe40c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefe4158e20 0x7fefe40ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 --2- 172.21.15.64:0/755420855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe40cecd0 0x7fefe414ddb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 >> 172.21.15.64:0/755420855 conn(0x7fefe4067bb0 msgr2=0x7fefe414c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:37.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 shutdown_connections 2023-09-27T23:29:37.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.612+0000 7fefe9268700 1 -- 172.21.15.64:0/755420855 wait complete. 2023-09-27T23:29:37.630 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:29:37.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:37.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:37.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:29:37.657 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:29:37.657 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:29:37.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:29:37.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[49524]: fuse finished with error 0 and tester_r 0 2023-09-27T23:29:37.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:29:37.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:37.700 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:29:37.727 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:37.720+0000 7fdbce914000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:29:37.736 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:37.734+0000 7f9c30798000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:29:37.821 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:29:37.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:37.821 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:29:37.827 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:29:37.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:29:37.828 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:29:37.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:37.829 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:29:37.869 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:29:37.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:29:37.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:29:37.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:29:37.870 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:29:37.877 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:29:37.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:29:37.878 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:37.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4c31b700 1 Processor -- start 2023-09-27T23:29:37.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4c31b700 1 -- start start 2023-09-27T23:29:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4c31b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9e440cc580 0x7f9e440afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4c31b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440cd500 con 0x7f9e440b3600 2023-09-27T23:29:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4c31b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440b39d0 con 0x7f9e440b03b0 2023-09-27T23:29:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4c31b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440c8f30 con 0x7f9e440cc580 2023-09-27T23:29:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4a0b7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9e440b3600 0x7f9e440ae3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52590/0 (socket says 172.21.15.64:52590) 2023-09-27T23:29:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4a0b7700 1 -- 172.21.15.64:0/3867963577 learned_addr learned my addr 172.21.15.64:0/3867963577 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:37.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859859999 0 0) 0x7f9e440cd500 con 0x7f9e440b3600 2023-09-27T23:29:37.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e30003570 con 0x7f9e440b3600 2023-09-27T23:29:37.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e498b6700 1 --2- 172.21.15.64:0/3867963577 >> v2:172.21.15.123:3300/0 conn(0x7f9e440cc580 0x7f9e440afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:37.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4292675168 0 0) 0x7f9e30003570 con 0x7f9e440b3600 2023-09-27T23:29:37.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 >> v2:172.21.15.123:3300/0 conn(0x7f9e440cc580 msgr2=0x7f9e440afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:37.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 --2- 172.21.15.64:0/3867963577 >> v2:172.21.15.123:3300/0 conn(0x7f9e440cc580 0x7f9e440afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 >> v1:172.21.15.123:6789/0 conn(0x7f9e440b03b0 legacy=0x7f9e440c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:37.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440cc9d0 con 0x7f9e440b3600 2023-09-27T23:29:37.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f9e40001470 con 0x7f9e440b3600 2023-09-27T23:29:37.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 >> v1:172.21.15.64:6789/0 conn(0x7f9e440b3600 legacy=0x7f9e440ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30003a60 0x7f9e30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e30004810 0x7f9e30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300053c0 0x7f9e3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440c8f30 con 0x7f9e300053c0 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000c2d0 con 0x7f9e30003a60 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000c490 con 0x7f9e30004810 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300053c0 0x7f9e3000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e4a0b7700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30003a60 0x7f9e30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.980+0000 7f9e498b6700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e30004810 0x7f9e30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e30004810 msgr2=0x7f9e30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e30004810 0x7f9e30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30003a60 msgr2=0x7f9e30003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e30003a60 0x7f9e30003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/3867963577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30003570 con 0x7f9e300053c0 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/3867963577 reap_dead start 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300053c0 0x7f9e3000ba60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c001fc0 tx=0x7f9e3c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66e5f2d369e93f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c001950 con 0x7f9e300053c0 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c00c8b0 con 0x7f9e300053c0 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e490b5700 1 -- 172.21.15.64:0/3867963577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00cc90 con 0x7f9e300053c0 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 -- 172.21.15.64:0/3867963577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300053c0 msgr2=0x7f9e3000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300053c0 0x7f9e3000ba60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c001fc0 tx=0x7f9e3c001640 comp rx=0 tx=0).stop 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 -- 172.21.15.64:0/3867963577 shutdown_connections 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/3867963577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300053c0 0x7f9e3000ba60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 -- 172.21.15.64:0/3867963577 >> 172.21.15.64:0/3867963577 conn(0x7f9e4406c7c0 msgr2=0x7f9e4406cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 -- 172.21.15.64:0/3867963577 shutdown_connections 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 -- 172.21.15.64:0/3867963577 wait complete. 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 Processor -- start 2023-09-27T23:29:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.984+0000 7f9e4c31b700 1 -- start start 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e300053c0 0x7f9e4406a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e440693a0 0x7f9e44067af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 0x7f9e440c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44068f10 con 0x7f9e4415ad70 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44068030 con 0x7f9e440693a0 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44068300 con 0x7f9e300053c0 2023-09-27T23:29:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 0x7f9e440c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 0x7f9e440c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32816/0 (socket says 172.21.15.64:32816) 2023-09-27T23:29:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/1177896852 learned_addr learned my addr 172.21.15.64:0/1177896852 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a0b7700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e300053c0 0x7f9e4406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e498b6700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e440693a0 0x7f9e44067af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e300053c0 msgr2=0x7f9e4406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e300053c0 0x7f9e4406a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e440693a0 msgr2=0x7f9e44067af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e440693a0 0x7f9e44067af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 -- 172.21.15.64:0/1177896852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440685e0 con 0x7f9e4415ad70 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4a8b8700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 0x7f9e440c75c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9e440c5020 tx=0x7f9e3c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c001950 con 0x7f9e4415ad70 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c001250 con 0x7f9e4415ad70 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440688c0 con 0x7f9e4415ad70 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c01e870 con 0x7f9e4415ad70 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e44069810 con 0x7f9e4415ad70 2023-09-27T23:29:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.988+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f9e3c01ec60 con 0x7f9e4415ad70 2023-09-27T23:29:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.992+0000 7f9e3a7fc700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 0x7f9e28062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.992+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44069080 con 0x7f9e4415ad70 2023-09-27T23:29:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.992+0000 7f9e4a0b7700 1 -- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 msgr2=0x7f9e28062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.992+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f9e3c07ecc0 con 0x7f9e4415ad70 2023-09-27T23:29:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.992+0000 7f9e4a0b7700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 0x7f9e28062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:37.996+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9e3c0524b0 con 0x7f9e4415ad70 2023-09-27T23:29:38.154 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.148+0000 7fdbce914000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:29:38.173 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.170+0000 7f9c30798000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:29:38.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.192+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e44071120 con 0x7f9e4415ad70 2023-09-27T23:29:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.192+0000 7f9e3a7fc700 1 -- 172.21.15.64:0/1177896852 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f9e3c017020 con 0x7f9e4415ad70 2023-09-27T23:29:38.200 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:38.200 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:19.692498+0000","modified":"2023-09-27T23:29:21.709311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6610":{"gid":6610,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2730764960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2730764960},{"type":"v1","addr":"172.21.15.123:6837","nonce":2730764960}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-09-27T23:29:38.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4a0b7700 1 -- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 msgr2=0x7f9e28062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:38.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4a0b7700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 0x7f9e28062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 msgr2=0x7f9e28062fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 0x7f9e28062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 msgr2=0x7f9e440c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 0x7f9e440c75c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9e440c5020 tx=0x7f9e3c015000 comp rx=0 tx=0).stop 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 shutdown_connections 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e28060b30 0x7f9e28062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4415ad70 0x7f9e440c75c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e440693a0 0x7f9e44067af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 --2- 172.21.15.64:0/1177896852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e300053c0 0x7f9e4406a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 >> 172.21.15.64:0/1177896852 conn(0x7f9e4406c7c0 msgr2=0x7f9e4415d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 shutdown_connections 2023-09-27T23:29:38.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.196+0000 7f9e4c31b700 1 -- 172.21.15.64:0/1177896852 wait complete. 2023-09-27T23:29:38.206 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-27T23:29:38.206 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.200+0000 7fdbce914000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:29:38.213 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:38.220 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.218+0000 7f9c30798000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:29:38.325 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.320+0000 7fdbce914000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:38.326 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.322+0000 7f9c30798000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:38.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 Processor -- start 2023-09-27T23:29:38.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 -- start start 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a6406d760 0x7f4a641591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a64159700 0x7f4a640c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a64159f90 con 0x7f4a64158df0 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6415a270 con 0x7f4a64159700 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a6a2f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640c7930 con 0x7f4a6406d760 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.320+0000 7f4a69af7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a64159700 0x7f4a640c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a692f6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4a64158df0 0x7f4a6406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52598/0 (socket says 172.21.15.64:52598) 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a692f6700 1 -- 172.21.15.64:0/1374696853 learned_addr learned my addr 172.21.15.64:0/1374696853 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a68af5700 1 --2- 172.21.15.64:0/1374696853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a6406d760 0x7f4a641591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a5bfff700 1 -- 172.21.15.64:0/1374696853 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273788170 0 0) 0x7f4a64159f90 con 0x7f4a64158df0 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a5bfff700 1 -- 172.21.15.64:0/1374696853 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a50003570 con 0x7f4a64158df0 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a69af7700 1 -- 172.21.15.64:0/1374696853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a6406d760 msgr2=0x7f4a641591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a69af7700 1 --2- 172.21.15.64:0/1374696853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a6406d760 0x7f4a641591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a69af7700 1 -- 172.21.15.64:0/1374696853 >> v1:172.21.15.64:6789/0 conn(0x7f4a64158df0 legacy=0x7f4a6406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a69af7700 1 -- 172.21.15.64:0/1374696853 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a640c7c10 con 0x7f4a64159700 2023-09-27T23:29:38.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a69af7700 1 --2- 172.21.15.64:0/1374696853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a64159700 0x7f4a640c8130 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009e60 tx=0x7f4a5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337a77e18cbfc3c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a5bfff700 1 -- 172.21.15.64:0/1374696853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00e070 con 0x7f4a64159700 2023-09-27T23:29:38.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a5bfff700 1 -- 172.21.15.64:0/1374696853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c00bab0 con 0x7f4a64159700 2023-09-27T23:29:38.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a5bfff700 1 -- 172.21.15.64:0/1374696853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00be30 con 0x7f4a64159700 2023-09-27T23:29:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/1374696853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a64159700 msgr2=0x7f4a640c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/1374696853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a64159700 0x7f4a640c8130 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009e60 tx=0x7f4a5c009780 comp rx=0 tx=0).stop 2023-09-27T23:29:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/1374696853 shutdown_connections 2023-09-27T23:29:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/1374696853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a64159700 0x7f4a640c8130 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/1374696853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a6406d760 0x7f4a641591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/1374696853 >> 172.21.15.64:0/1374696853 conn(0x7f4a6406c080 msgr2=0x7f4a640ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/1374696853 shutdown_connections 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/1374696853 wait complete. 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 Processor -- start 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.324+0000 7f4a6a2f8700 1 -- start start 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 0x7f4a640af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a64158df0 0x7f4a6415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a64159700 0x7f4a64158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6415d3f0 con 0x7f4a6406d760 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640afe60 con 0x7f4a64158df0 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640b0140 con 0x7f4a64159700 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 0x7f4a640af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 0x7f4a640af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32840/0 (socket says 172.21.15.64:32840) 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 -- 172.21.15.64:0/3125468032 learned_addr learned my addr 172.21.15.64:0/3125468032 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a69af7700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a64159700 0x7f4a64158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a68af5700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a64158df0 0x7f4a6415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 -- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a64159700 msgr2=0x7f4a64158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a64159700 0x7f4a64158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 -- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a64158df0 msgr2=0x7f4a6415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a64158df0 0x7f4a6415ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 -- 172.21.15.64:0/3125468032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c009a20 con 0x7f4a6406d760 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a692f6700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 0x7f4a640af880 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4a600011e0 tx=0x7f4a60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60005b30 con 0x7f4a6406d760 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a60005c90 con 0x7f4a6406d760 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60007320 con 0x7f4a6406d760 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a640b0420 con 0x7f4a6406d760 2023-09-27T23:29:38.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a64158940 con 0x7f4a6406d760 2023-09-27T23:29:38.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6404be40 con 0x7f4a6406d760 2023-09-27T23:29:38.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.328+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f4a60007690 con 0x7f4a6406d760 2023-09-27T23:29:38.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.332+0000 7f4a597fa700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a48060b00 0x7f4a48062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.332+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f4a6007c980 con 0x7f4a6406d760 2023-09-27T23:29:38.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.332+0000 7f4a68af5700 1 -- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a48060b00 msgr2=0x7f4a48062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:38.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.332+0000 7f4a68af5700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a48060b00 0x7f4a48062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:38.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.336+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4a6004c640 con 0x7f4a6406d760 2023-09-27T23:29:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.496+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a6415b5b0 con 0x7f4a6406d760 2023-09-27T23:29:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a597fa700 1 -- 172.21.15.64:0/3125468032 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+17572 (secure 0 0 0) 0x7f4a60017090 con 0x7f4a6406d760 2023-09-27T23:29:38.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:38.507 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:35.535069+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:18.249013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:18.681348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3072261341":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/761037533":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/4251293646":"2023-09-28T23:29:35.534987+0000","172.21.15.123:6832/33228":"2023-09-28T23:29:35.534987+0000","172.21.15.123:6833/33228":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/2638398233":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/1622918165":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/2133044744":"2023-09-28T23:29:35.534987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a48060b00 msgr2=0x7f4a48062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a48060b00 0x7f4a48062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 msgr2=0x7f4a640af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 0x7f4a640af880 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4a600011e0 tx=0x7f4a60007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 shutdown_connections 2023-09-27T23:29:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4a48060b00 0x7f4a48062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a64159700 0x7f4a64158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a64158df0 0x7f4a6415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 --2- 172.21.15.64:0/3125468032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a6406d760 0x7f4a640af880 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 >> 172.21.15.64:0/3125468032 conn(0x7f4a6406c080 msgr2=0x7f4a6406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 shutdown_connections 2023-09-27T23:29:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.500+0000 7f4a6a2f8700 1 -- 172.21.15.64:0/3125468032 wait complete. 2023-09-27T23:29:38.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:29:18.249013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:29:18.681348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:29:38.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:29:38.522 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.518+0000 7f9c30798000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:29:38.534 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.528+0000 7fdbce914000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:29:38.571 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.566+0000 7f9c30798000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:38.585 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.580+0000 7fdbce914000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:38.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.620+0000 7f4949175700 1 Processor -- start 2023-09-27T23:29:38.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.620+0000 7f4949175700 1 -- start start 2023-09-27T23:29:38.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 0x7f49440b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440ac390 con 0x7f494406ae20 2023-09-27T23:29:38.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440ac670 con 0x7f49440ab6d0 2023-09-27T23:29:38.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440ac950 con 0x7f494406dbb0 2023-09-27T23:29:38.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4943fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f494406ae20 0x7f494406b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52606/0 (socket says 172.21.15.64:52606) 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4943fff700 1 -- 172.21.15.64:0/4198294900 learned_addr learned my addr 172.21.15.64:0/4198294900 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4948974700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 0x7f49440b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721940849 0 0) 0x7f49440ac390 con 0x7f494406ae20 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f492c003570 con 0x7f494406ae20 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3032082814 0 0) 0x7f492c003570 con 0x7f494406ae20 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 msgr2=0x7f49440b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:38.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 0x7f49440b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 >> v1:172.21.15.123:6789/0 conn(0x7f49440ab6d0 legacy=0x7f49440abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440acc30 con 0x7f494406ae20 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f4934001470 con 0x7f494406ae20 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 >> v1:172.21.15.64:6789/0 conn(0x7f494406ae20 legacy=0x7f494406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f492c003a90 0x7f492c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f492c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 0x7f492c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440ac950 con 0x7f492c003a90 2023-09-27T23:29:38.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4943fff700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f492c003a90 0x7f492c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00bb10 con 0x7f492c004810 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00bcd0 con 0x7f494406dbb0 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4948974700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 0x7f492c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f492c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 -- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 msgr2=0x7f492c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494406dbb0 0x7f492c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 -- 172.21.15.64:0/4198294900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f492c003a90 msgr2=0x7f492c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f492c003a90 0x7f492c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 -- 172.21.15.64:0/4198294900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c003570 con 0x7f492c004810 2023-09-27T23:29:38.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f49437fe700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f492c004c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f49380019a0 tx=0x7f4938007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08e22a904d5e644 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493800c5d0 con 0x7f492c004810 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493800c750 con 0x7f492c004810 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4942ffd700 1 -- 172.21.15.64:0/4198294900 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493800cb10 con 0x7f492c004810 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 msgr2=0x7f492c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 --2- 172.21.15.64:0/4198294900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f492c004c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f49380019a0 tx=0x7f4938007d10 comp rx=0 tx=0).stop 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4948974700 1 -- 172.21.15.64:0/4198294900 reap_dead start 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- 172.21.15.64:0/4198294900 shutdown_connections 2023-09-27T23:29:38.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- 172.21.15.64:0/4198294900 >> 172.21.15.64:0/4198294900 conn(0x7f49440c8cd0 msgr2=0x7f49440c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:38.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- 172.21.15.64:0/4198294900 shutdown_connections 2023-09-27T23:29:38.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.624+0000 7f4949175700 1 -- 172.21.15.64:0/4198294900 wait complete. 2023-09-27T23:29:38.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 Processor -- start 2023-09-27T23:29:38.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 -- start start 2023-09-27T23:29:38.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f49440c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 0x7f494416a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494416a900 0x7f49441550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494416ad80 con 0x7f49440c4430 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c5d60 con 0x7f492c004810 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c5150 con 0x7f494416a900 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 0x7f494416a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 0x7f494416a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32864/0 (socket says 172.21.15.64:32864) 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 -- 172.21.15.64:0/1158400417 learned_addr learned my addr 172.21.15.64:0/1158400417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4943fff700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f49440c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4948974700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494416a900 0x7f49441550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 -- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494416a900 msgr2=0x7f49441550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494416a900 0x7f49441550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 -- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 msgr2=0x7f49440c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f49440c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 -- 172.21.15.64:0/1158400417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440c5430 con 0x7f49440c4430 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f49437fe700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 0x7f494416a3c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f493800c3c0 tx=0x7f4938015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493800c5d0 con 0x7f49440c4430 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493800ce80 con 0x7f49440c4430 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440c5710 con 0x7f49440c4430 2023-09-27T23:29:38.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493800caf0 con 0x7f49440c4430 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.628+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494415af10 con 0x7f49440c4430 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.632+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49440c5ef0 con 0x7f49440c4430 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.632+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f4938019070 con 0x7f49440c4430 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.632+0000 7f492bfff700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4930060b00 0x7f4930062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.632+0000 7f4943fff700 1 -- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4930060b00 msgr2=0x7f4930062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.632+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f493807ea10 con 0x7f49440c4430 2023-09-27T23:29:38.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.632+0000 7f4943fff700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4930060b00 0x7f4930062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:38.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.636+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f493804e830 con 0x7f49440c4430 2023-09-27T23:29:38.725 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.722+0000 7f9c30798000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:29:38.769 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.764+0000 7fdbce914000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:29:38.781 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.778+0000 7f9c30798000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:29:38.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.804+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f494404be40 con 0x7f49440c4430 2023-09-27T23:29:38.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.804+0000 7f492bfff700 1 -- 172.21.15.64:0/1158400417 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49380220a0 con 0x7f49440c4430 2023-09-27T23:29:38.810 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:29:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4930060b00 msgr2=0x7f4930062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4930060b00 0x7f4930062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 msgr2=0x7f494416a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 0x7f494416a3c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f493800c3c0 tx=0x7f4938015000 comp rx=0 tx=0).stop 2023-09-27T23:29:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 shutdown_connections 2023-09-27T23:29:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4930060b00 0x7f4930062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f494416a900 0x7f49441550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49440c4430 0x7f494416a3c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 --2- 172.21.15.64:0/1158400417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492c004810 0x7f49440c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 >> 172.21.15.64:0/1158400417 conn(0x7f49440c8cd0 msgr2=0x7f49440b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:38.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 shutdown_connections 2023-09-27T23:29:38.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.808+0000 7f4949175700 1 -- 172.21.15.64:0/1158400417 wait complete. 2023-09-27T23:29:38.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:29:38.830 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.824+0000 7fdbce914000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:29:38.839 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:38.834+0000 7f9c30798000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:29:38.894 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:38.888+0000 7fdbce914000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:29:38.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.924+0000 7fde36557700 1 Processor -- start 2023-09-27T23:29:38.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.924+0000 7fde36557700 1 -- start start 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde30158fd0 0x7fde3006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3006d750 con 0x7fde300cc610 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300cbaf0 con 0x7fde300cc1e0 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006e120 con 0x7fde30158fd0 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35d56700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fde300cc610 0x7fde300c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52620/0 (socket says 172.21.15.64:52620) 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35d56700 1 -- 172.21.15.64:0/2640136820 learned_addr learned my addr 172.21.15.64:0/2640136820 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:38.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35555700 1 --2- 172.21.15.64:0/2640136820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde30158fd0 0x7fde3006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663607878 0 0) 0x7fde3006d750 con 0x7fde300cc610 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde1c003570 con 0x7fde300cc610 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558295667 0 0) 0x7fde300cbaf0 con 0x7fde300cc1e0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3006d750 con 0x7fde300cc1e0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35555700 1 -- 172.21.15.64:0/2640136820 >> v1:172.21.15.123:6789/0 conn(0x7fde300cc1e0 legacy=0x7fde300c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35555700 1 -- 172.21.15.64:0/2640136820 >> v1:172.21.15.64:6789/0 conn(0x7fde300cc610 legacy=0x7fde300c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35555700 1 -- 172.21.15.64:0/2640136820 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3006cb40 con 0x7fde30158fd0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde35555700 1 --2- 172.21.15.64:0/2640136820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde30158fd0 0x7fde3006dbe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009910 tx=0x7fde2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f67b5cd86040df3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c011630 con 0x7fde30158fd0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c011790 con 0x7fde30158fd0 2023-09-27T23:29:38.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde27fff700 1 -- 172.21.15.64:0/2640136820 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c011b10 con 0x7fde30158fd0 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- 172.21.15.64:0/2640136820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde30158fd0 msgr2=0x7fde3006dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 --2- 172.21.15.64:0/2640136820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde30158fd0 0x7fde3006dbe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009910 tx=0x7fde2c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- 172.21.15.64:0/2640136820 shutdown_connections 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 --2- 172.21.15.64:0/2640136820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde30158fd0 0x7fde3006dbe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- 172.21.15.64:0/2640136820 >> 172.21.15.64:0/2640136820 conn(0x7fde300cdc00 msgr2=0x7fde300ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- 172.21.15.64:0/2640136820 shutdown_connections 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.928+0000 7fde36557700 1 -- 172.21.15.64:0/2640136820 wait complete. 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 Processor -- start 2023-09-27T23:29:38.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 -- start start 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde300cc1e0 0x7fde300b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde300cc610 0x7fde301508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 0x7fde3015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300b10d0 con 0x7fde30158fd0 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3015cde0 con 0x7fde300cc610 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3015d0c0 con 0x7fde300cc1e0 2023-09-27T23:29:38.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 0x7fde3015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 0x7fde3015c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32868/0 (socket says 172.21.15.64:32868) 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 -- 172.21.15.64:0/4206957460 learned_addr learned my addr 172.21.15.64:0/4206957460 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde34d54700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde300cc610 0x7fde301508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35555700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde300cc1e0 0x7fde300b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 -- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde300cc1e0 msgr2=0x7fde300b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde300cc1e0 0x7fde300b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 -- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde300cc610 msgr2=0x7fde301508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde300cc610 0x7fde301508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:38.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 -- 172.21.15.64:0/4206957460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009420 con 0x7fde30158fd0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde35d56700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 0x7fde3015c490 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fde28001460 tx=0x7fde28001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800a8b0 con 0x7fde30158fd0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2800aa10 con 0x7fde30158fd0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3015d3a0 con 0x7fde30158fd0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800adb0 con 0x7fde30158fd0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.932+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3015d960 con 0x7fde30158fd0 2023-09-27T23:29:38.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.936+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3004be40 con 0x7fde30158fd0 2023-09-27T23:29:38.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.936+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fde28007300 con 0x7fde30158fd0 2023-09-27T23:29:38.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.936+0000 7fde257fa700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde14060b00 0x7fde14062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:38.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.936+0000 7fde35555700 1 -- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde14060b00 msgr2=0x7fde14062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:38.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.936+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fde2807d7f0 con 0x7fde30158fd0 2023-09-27T23:29:38.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.936+0000 7fde35555700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde14060b00 0x7fde14062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:38.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:38.940+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fde2804d530 con 0x7fde30158fd0 2023-09-27T23:29:39.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fde3006ce20 con 0x7fde30158fd0 2023-09-27T23:29:39.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde257fa700 1 -- 172.21.15.64:0/4206957460 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7fde28050b50 con 0x7fde30158fd0 2023-09-27T23:29:39.106 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:29:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde14060b00 msgr2=0x7fde14062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde14060b00 0x7fde14062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 msgr2=0x7fde3015c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 0x7fde3015c490 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fde28001460 tx=0x7fde28001ac0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 shutdown_connections 2023-09-27T23:29:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.100+0000 7fde36557700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde14060b00 0x7fde14062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.104+0000 7fde36557700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde30158fd0 0x7fde3015c490 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.104+0000 7fde36557700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde300cc610 0x7fde301508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.104+0000 7fde36557700 1 --2- 172.21.15.64:0/4206957460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde300cc1e0 0x7fde300b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.104+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 >> 172.21.15.64:0/4206957460 conn(0x7fde300cdc00 msgr2=0x7fde300c40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.104+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 shutdown_connections 2023-09-27T23:29:39.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.104+0000 7fde36557700 1 -- 172.21.15.64:0/4206957460 wait complete. 2023-09-27T23:29:39.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:29:39.136 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.134+0000 7f9c30798000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:29:39.226 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.220+0000 7fdbce914000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:29:39.247 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.242+0000 7f9c30798000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:29:39.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 Processor -- start 2023-09-27T23:29:39.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 -- start start 2023-09-27T23:29:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81ec06b360 0x7f81ec0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec06b7d0 con 0x7f81ec155520 2023-09-27T23:29:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0ad020 con 0x7f81ec06b360 2023-09-27T23:29:39.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81eb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f81ec155520 0x7f81ec0c7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52636/0 (socket says 172.21.15.64:52636) 2023-09-27T23:29:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81ebfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81ec06b360 0x7f81ec0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81eb7fe700 1 -- 172.21.15.64:0/2166475451 learned_addr learned my addr 172.21.15.64:0/2166475451 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81f0b5c700 1 -- 172.21.15.64:0/2166475451 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0c81e0 con 0x7f81ec0ae6c0 2023-09-27T23:29:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.276+0000 7f81eaffd700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ea7fc700 1 -- 172.21.15.64:0/2166475451 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025790622 0 0) 0x7f81ec06b7d0 con 0x7f81ec155520 2023-09-27T23:29:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ea7fc700 1 -- 172.21.15.64:0/2166475451 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d4003570 con 0x7f81ec155520 2023-09-27T23:29:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ebfff700 1 -- 172.21.15.64:0/2166475451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 msgr2=0x7f81ec06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ebfff700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec06ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ebfff700 1 -- 172.21.15.64:0/2166475451 >> v1:172.21.15.64:6789/0 conn(0x7f81ec155520 legacy=0x7f81ec0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ea7fc700 1 -- 172.21.15.64:0/2166475451 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3960640103 0 0) 0x7f81d4003570 con 0x7f81ec155520 2023-09-27T23:29:39.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ebfff700 1 -- 172.21.15.64:0/2166475451 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0aca10 con 0x7f81ec06b360 2023-09-27T23:29:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81eaffd700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ebfff700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0ad4b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009e60 tx=0x7f81e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5a7386fb6a501 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ea7fc700 1 -- 172.21.15.64:0/2166475451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400e070 con 0x7f81ec06b360 2023-09-27T23:29:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ea7fc700 1 -- 172.21.15.64:0/2166475451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e400b8e0 con 0x7f81ec06b360 2023-09-27T23:29:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81ea7fc700 1 -- 172.21.15.64:0/2166475451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc001470 con 0x7f81ec06b360 2023-09-27T23:29:39.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 -- 172.21.15.64:0/2166475451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 msgr2=0x7f81ec0ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0ad4b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009e60 tx=0x7f81e4009780 comp rx=0 tx=0).stop 2023-09-27T23:29:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 -- 172.21.15.64:0/2166475451 shutdown_connections 2023-09-27T23:29:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0ad4b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/2166475451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 -- 172.21.15.64:0/2166475451 >> 172.21.15.64:0/2166475451 conn(0x7f81ec0c8cd0 msgr2=0x7f81ec0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:39.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 -- 172.21.15.64:0/2166475451 shutdown_connections 2023-09-27T23:29:39.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 -- 172.21.15.64:0/2166475451 wait complete. 2023-09-27T23:29:39.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.280+0000 7f81f0b5c700 1 Processor -- start 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 -- start start 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81ec155520 0x7f81ec1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec169200 con 0x7f81ec06b360 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec1685f0 con 0x7f81ec155520 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec1688d0 con 0x7f81ec0ae6c0 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32888/0 (socket says 172.21.15.64:32888) 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 -- 172.21.15.64:0/3965124739 learned_addr learned my addr 172.21.15.64:0/3965124739 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:39.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eaffd700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81ebfff700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81ec155520 0x7f81ec1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 -- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 msgr2=0x7f81ec1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 -- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81ec155520 msgr2=0x7f81ec1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81ec155520 0x7f81ec1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 -- 172.21.15.64:0/3965124739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e000b6d0 con 0x7f81ec06b360 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eaffd700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81ebfff700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81ec155520 0x7f81ec1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81eb7fe700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0b0fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00cb90 tx=0x7f81dc001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc001d40 con 0x7f81ec06b360 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc007360 con 0x7f81ec06b360 2023-09-27T23:29:39.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc004830 con 0x7f81ec06b360 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e4009a20 con 0x7f81ec06b360 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.284+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec168e90 con 0x7f81ec06b360 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.288+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec04be40 con 0x7f81ec06b360 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.288+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f81dc004ba0 con 0x7f81ec06b360 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.288+0000 7f81d37fe700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81d8060b00 0x7f81d8062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.288+0000 7f81eaffd700 1 -- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81d8060b00 msgr2=0x7f81d8062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.288+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f81dc07cc60 con 0x7f81ec06b360 2023-09-27T23:29:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.288+0000 7f81eaffd700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81d8060b00 0x7f81d8062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:39.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.292+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f81dc04ca40 con 0x7f81ec06b360 2023-09-27T23:29:39.341 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.336+0000 7fdbce914000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:29:39.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81ec0c7c20 con 0x7f81ec06b360 2023-09-27T23:29:39.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81d37fe700 1 -- 172.21.15.64:0/3965124739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81dc018020 con 0x7f81ec06b360 2023-09-27T23:29:39.465 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:29:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81d8060b00 msgr2=0x7f81d8062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81d8060b00 0x7f81d8062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 msgr2=0x7f81ec0b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0b0fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00cb90 tx=0x7f81dc001ab0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 shutdown_connections 2023-09-27T23:29:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81d8060b00 0x7f81d8062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81ec155520 0x7f81ec1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81ec0ae6c0 0x7f81ec1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 --2- 172.21.15.64:0/3965124739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec06b360 0x7f81ec0b0fb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 >> 172.21.15.64:0/3965124739 conn(0x7f81ec0c8cd0 msgr2=0x7f81ec0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:39.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 shutdown_connections 2023-09-27T23:29:39.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.460+0000 7f81f0b5c700 1 -- 172.21.15.64:0/3965124739 wait complete. 2023-09-27T23:29:39.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:29:39.528 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.522+0000 7f9c30798000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:29:39.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 Processor -- start 2023-09-27T23:29:39.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 -- start start 2023-09-27T23:29:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85c81547a0 0x7f85c80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85c806d050 0x7f85c806d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80c7820 con 0x7f85c8157fc0 2023-09-27T23:29:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c806d9d0 con 0x7f85c81547a0 2023-09-27T23:29:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.576+0000 7f85cc823700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c8158b90 con 0x7f85c806d050 2023-09-27T23:29:39.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85c81547a0 0x7f85c80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f85c81547a0 0x7f85c80ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32902/0 (socket says 172.21.15.64:32902) 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 -- 172.21.15.64:0/735799693 learned_addr learned my addr 172.21.15.64:0/735799693 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c7fff700 1 --2- 172.21.15.64:0/735799693 >> v2:172.21.15.123:3300/0 conn(0x7f85c806d050 0x7f85c806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 -- 172.21.15.64:0/735799693 >> v2:172.21.15.123:3300/0 conn(0x7f85c806d050 msgr2=0x7f85c806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 --2- 172.21.15.64:0/735799693 >> v2:172.21.15.123:3300/0 conn(0x7f85c806d050 0x7f85c806d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 -- 172.21.15.64:0/735799693 >> v1:172.21.15.123:6789/0 conn(0x7f85c8157fc0 legacy=0x7f85c8158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 -- 172.21.15.64:0/735799693 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80c7210 con 0x7f85c81547a0 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c77fe700 1 --2- 172.21.15.64:0/735799693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c81547a0 0x7f85c80ad730 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f85c80b2b10 tx=0x7f85b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaa1b67a13542ca server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c67fc700 1 -- 172.21.15.64:0/735799693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800b820 con 0x7f85c81547a0 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c67fc700 1 -- 172.21.15.64:0/735799693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800b980 con 0x7f85c81547a0 2023-09-27T23:29:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85c67fc700 1 -- 172.21.15.64:0/735799693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800bd20 con 0x7f85c81547a0 2023-09-27T23:29:39.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 -- 172.21.15.64:0/735799693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c81547a0 msgr2=0x7f85c80ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 --2- 172.21.15.64:0/735799693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c81547a0 0x7f85c80ad730 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f85c80b2b10 tx=0x7f85b8009250 comp rx=0 tx=0).stop 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 -- 172.21.15.64:0/735799693 shutdown_connections 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 --2- 172.21.15.64:0/735799693 >> v2:172.21.15.123:3300/0 conn(0x7f85c806d050 0x7f85c806d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 --2- 172.21.15.64:0/735799693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c81547a0 0x7f85c80ad730 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 -- 172.21.15.64:0/735799693 >> 172.21.15.64:0/735799693 conn(0x7f85c80ccb30 msgr2=0x7f85c80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 -- 172.21.15.64:0/735799693 shutdown_connections 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 -- 172.21.15.64:0/735799693 wait complete. 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 Processor -- start 2023-09-27T23:29:39.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.580+0000 7f85cc823700 1 -- start start 2023-09-27T23:29:39.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 0x7f85c80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c81547a0 0x7f85c80b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c8157fc0 0x7f85c8149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8151aa0 con 0x7f85c806d050 2023-09-27T23:29:39.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8150a90 con 0x7f85c81547a0 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8150d30 con 0x7f85c8157fc0 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 0x7f85c80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 0x7f85c80b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32920/0 (socket says 172.21.15.64:32920) 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 -- 172.21.15.64:0/3841017152 learned_addr learned my addr 172.21.15.64:0/3841017152 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c7fff700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c8157fc0 0x7f85c8149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c6ffd700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c81547a0 0x7f85c80b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 -- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c8157fc0 msgr2=0x7f85c8149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c8157fc0 0x7f85c8149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 -- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c81547a0 msgr2=0x7f85c80b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c81547a0 0x7f85c80b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 -- 172.21.15.64:0/3841017152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80094f0 con 0x7f85c806d050 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85c77fe700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 0x7f85c80b1c40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f85b80030c0 tx=0x7f85b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8011d00 con 0x7f85c806d050 2023-09-27T23:29:39.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8020070 con 0x7f85c806d050 2023-09-27T23:29:39.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b801b6a0 con 0x7f85c806d050 2023-09-27T23:29:39.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c8067f80 con 0x7f85c806d050 2023-09-27T23:29:39.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c81517c0 con 0x7f85c806d050 2023-09-27T23:29:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c804be40 con 0x7f85c806d050 2023-09-27T23:29:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.584+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f85b8022030 con 0x7f85c806d050 2023-09-27T23:29:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.588+0000 7f85af7fe700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85b4060b00 0x7f85b4062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.588+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f85b807f2c0 con 0x7f85c806d050 2023-09-27T23:29:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.588+0000 7f85c6ffd700 1 -- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85b4060b00 msgr2=0x7f85b4062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:39.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.588+0000 7f85c6ffd700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85b4060b00 0x7f85b4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:39.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.592+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f85b804f020 con 0x7f85c806d050 2023-09-27T23:29:39.641 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.632+0000 7fdbce914000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:29:39.690 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.686+0000 7f9c30798000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.748+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85c804be40 con 0x7f85c806d050 2023-09-27T23:29:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.748+0000 7f85af7fe700 1 -- 172.21.15.64:0/3841017152 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85b8011e60 con 0x7f85c806d050 2023-09-27T23:29:39.754 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:29:39.754 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.750+0000 7f9c30798000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:29:39.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.748+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85b4060b00 msgr2=0x7f85b4062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:39.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.748+0000 7f85cc823700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85b4060b00 0x7f85b4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 msgr2=0x7f85c80b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 0x7f85c80b1c40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f85b80030c0 tx=0x7f85b8016000 comp rx=0 tx=0).stop 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 shutdown_connections 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85b4060b00 0x7f85b4062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c8157fc0 0x7f85c8149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c81547a0 0x7f85c80b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 --2- 172.21.15.64:0/3841017152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c806d050 0x7f85c80b1c40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 >> 172.21.15.64:0/3841017152 conn(0x7f85c80ccb30 msgr2=0x7f85c80c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:39.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 shutdown_connections 2023-09-27T23:29:39.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.752+0000 7f85cc823700 1 -- 172.21.15.64:0/3841017152 wait complete. 2023-09-27T23:29:39.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:39.807 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.802+0000 7f9c30798000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:29:39.811 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.804+0000 7fdbce914000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:29:39.860 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:39.854+0000 7f9c30798000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:29:39.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.860+0000 7fa776d2a700 1 Processor -- start 2023-09-27T23:29:39.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.860+0000 7fa776d2a700 1 -- start start 2023-09-27T23:29:39.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700b2310 0x7fa77015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa7700c8e80 0x7fa770067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7700670e0 con 0x7fa7700b26e0 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77015a700 con 0x7fa7700c8e80 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700676a0 con 0x7fa7700b2310 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775527700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa7700b26e0 0x7fa7700c8770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52648/0 (socket says 172.21.15.64:52648) 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775527700 1 -- 172.21.15.64:0/2790387951 learned_addr learned my addr 172.21.15.64:0/2790387951 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775d28700 1 --2- 172.21.15.64:0/2790387951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700b2310 0x7fa77015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776529700 1 --2- 172.21.15.64:0/2790387951 >> v2:172.21.15.123:3300/0 conn(0x7fa7700c8e80 0x7fa770067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa774d26700 1 -- 172.21.15.64:0/2790387951 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900143933 0 0) 0x7fa7700670e0 con 0x7fa7700b26e0 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa774d26700 1 -- 172.21.15.64:0/2790387951 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c003570 con 0x7fa7700b26e0 2023-09-27T23:29:39.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775d28700 1 -- 172.21.15.64:0/2790387951 >> v2:172.21.15.123:3300/0 conn(0x7fa7700c8e80 msgr2=0x7fa770067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775d28700 1 --2- 172.21.15.64:0/2790387951 >> v2:172.21.15.123:3300/0 conn(0x7fa7700c8e80 0x7fa770067b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775d28700 1 -- 172.21.15.64:0/2790387951 >> v1:172.21.15.64:6789/0 conn(0x7fa7700b26e0 legacy=0x7fa7700c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775d28700 1 -- 172.21.15.64:0/2790387951 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700c3510 con 0x7fa7700b2310 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa775d28700 1 --2- 172.21.15.64:0/2790387951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700b2310 0x7fa77015a1c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098e0 tx=0x7fa76c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95cf48b060592b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa774d26700 1 -- 172.21.15.64:0/2790387951 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c011630 con 0x7fa7700b2310 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa774d26700 1 -- 172.21.15.64:0/2790387951 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c011790 con 0x7fa7700b2310 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa774d26700 1 -- 172.21.15.64:0/2790387951 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c011b30 con 0x7fa7700b2310 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- 172.21.15.64:0/2790387951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700b2310 msgr2=0x7fa77015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 --2- 172.21.15.64:0/2790387951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700b2310 0x7fa77015a1c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098e0 tx=0x7fa76c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- 172.21.15.64:0/2790387951 shutdown_connections 2023-09-27T23:29:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 --2- 172.21.15.64:0/2790387951 >> v2:172.21.15.123:3300/0 conn(0x7fa7700c8e80 0x7fa770067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 --2- 172.21.15.64:0/2790387951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700b2310 0x7fa77015a1c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- 172.21.15.64:0/2790387951 >> 172.21.15.64:0/2790387951 conn(0x7fa77006c080 msgr2=0x7fa7700ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- 172.21.15.64:0/2790387951 shutdown_connections 2023-09-27T23:29:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- 172.21.15.64:0/2790387951 wait complete. 2023-09-27T23:29:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 Processor -- start 2023-09-27T23:29:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.864+0000 7fa776d2a700 1 -- start start 2023-09-27T23:29:39.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 0x7fa770145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7700b26e0 0x7fa7700b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700c8e80 0x7fa7700b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c4180 con 0x7fa7700b2310 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c4360 con 0x7fa7700b26e0 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c4640 con 0x7fa7700c8e80 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 0x7fa770145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 0x7fa770145670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32922/0 (socket says 172.21.15.64:32922) 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 -- 172.21.15.64:0/3216238210 learned_addr learned my addr 172.21.15.64:0/3216238210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775527700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7700b26e0 0x7fa7700b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776529700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700c8e80 0x7fa7700b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 -- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700c8e80 msgr2=0x7fa7700b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700c8e80 0x7fa7700b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 -- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7700b26e0 msgr2=0x7fa7700b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7700b26e0 0x7fa7700b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 -- 172.21.15.64:0/3216238210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c009420 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775d28700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 0x7fa770145670 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa7700677e0 tx=0x7fa76c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c011900 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c011a60 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c025640 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700c4920 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700c4ee0 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77006a560 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fa76c025a30 con 0x7fa7700b2310 2023-09-27T23:29:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa765ffb700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 0x7fa754062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa76c07ebe0 con 0x7fa7700b2310 2023-09-27T23:29:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775527700 1 -- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 msgr2=0x7fa754062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.868+0000 7fa775527700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 0x7fa754062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:39.880 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.876+0000 7fdbce914000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:29:39.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:39.876+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa77006a560 con 0x7fa7700b2310 2023-09-27T23:29:39.934 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.928+0000 7fdbce914000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:29:39.990 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:39.984+0000 7fdbce914000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:29:40.032 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:40.030+0000 7f9c30798000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:40.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.068+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa770159500 con 0x7fa7700b2310 2023-09-27T23:29:40.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa765ffb700 1 -- 172.21.15.64:0/3216238210 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fa770159500 con 0x7fa7700b2310 2023-09-27T23:29:40.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa775527700 1 -- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 msgr2=0x7fa754062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:40.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa775527700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 0x7fa754062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:40.077 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:40.077 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:19.692498+0000","modified":"2023-09-27T23:29:21.709311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6610":{"gid":6610,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2730764960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2730764960},{"type":"v1","addr":"172.21.15.123:6837","nonce":2730764960}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-09-27T23:29:40.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 msgr2=0x7fa754062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:40.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 0x7fa754062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 msgr2=0x7fa770145670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 0x7fa770145670 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa7700677e0 tx=0x7fa76c017000 comp rx=0 tx=0).stop 2023-09-27T23:29:40.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 shutdown_connections 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa754060b00 0x7fa754062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7700c8e80 0x7fa7700b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7700b26e0 0x7fa7700b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 --2- 172.21.15.64:0/3216238210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7700b2310 0x7fa770145670 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 >> 172.21.15.64:0/3216238210 conn(0x7fa77006c080 msgr2=0x7fa77006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 shutdown_connections 2023-09-27T23:29:40.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.072+0000 7fa776d2a700 1 -- 172.21.15.64:0/3216238210 wait complete. 2023-09-27T23:29:40.081 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-27T23:29:40.087 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:40.082+0000 7f9c30798000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:29:40.090 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:40.150 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:40.146+0000 7f9c30798000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:29:40.171 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:40.164+0000 7fdbce914000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:29:40.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.184+0000 7f0a67fff700 1 Processor -- start 2023-09-27T23:29:40.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- start start 2023-09-27T23:29:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a680670e0 0x7f0a68159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a680c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a680c1910 con 0x7f0a680ab910 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a68159cf0 con 0x7f0a680670e0 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a68159fd0 con 0x7f0a680abce0 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a680670e0 0x7f0a68159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a680670e0 0x7f0a68159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32928/0 (socket says 172.21.15.64:32928) 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 -- 172.21.15.64:0/2680224750 learned_addr learned my addr 172.21.15.64:0/2680224750 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a677fe700 1 --2- 172.21.15.64:0/2680224750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a680c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 -- 172.21.15.64:0/2680224750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 msgr2=0x7f0a680c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 --2- 172.21.15.64:0/2680224750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a680c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 -- 172.21.15.64:0/2680224750 >> v1:172.21.15.123:6789/0 conn(0x7f0a680ab910 legacy=0x7f0a680cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 -- 172.21.15.64:0/2680224750 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6815a2b0 con 0x7f0a680670e0 2023-09-27T23:29:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a667fc700 1 --2- 172.21.15.64:0/2680224750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68159710 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0099b0 tx=0x7f0a5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c44b2e43368bd54 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a65ffb700 1 -- 172.21.15.64:0/2680224750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00b860 con 0x7f0a680670e0 2023-09-27T23:29:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a65ffb700 1 -- 172.21.15.64:0/2680224750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c00b9c0 con 0x7f0a680670e0 2023-09-27T23:29:40.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a65ffb700 1 -- 172.21.15.64:0/2680224750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00bd60 con 0x7f0a680670e0 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- 172.21.15.64:0/2680224750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 msgr2=0x7f0a68159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 --2- 172.21.15.64:0/2680224750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68159710 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0099b0 tx=0x7f0a5c009250 comp rx=0 tx=0).stop 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- 172.21.15.64:0/2680224750 shutdown_connections 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 --2- 172.21.15.64:0/2680224750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a680c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 --2- 172.21.15.64:0/2680224750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68159710 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- 172.21.15.64:0/2680224750 >> 172.21.15.64:0/2680224750 conn(0x7f0a6806c080 msgr2=0x7f0a680ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:40.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- 172.21.15.64:0/2680224750 shutdown_connections 2023-09-27T23:29:40.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.188+0000 7f0a67fff700 1 -- 172.21.15.64:0/2680224750 wait complete. 2023-09-27T23:29:40.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 Processor -- start 2023-09-27T23:29:40.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 -- start start 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a680ab910 0x7f0a680b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a68165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681528d0 con 0x7f0a680670e0 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a68166a90 con 0x7f0a680ab910 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a67fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a68166d70 con 0x7f0a680abce0 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32930/0 (socket says 172.21.15.64:32930) 2023-09-27T23:29:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 -- 172.21.15.64:0/3247073479 learned_addr learned my addr 172.21.15.64:0/3247073479 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a677fe700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a68165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a667fc700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a680ab910 0x7f0a680b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 -- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 msgr2=0x7f0a68165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a68165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 -- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a680ab910 msgr2=0x7f0a680b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a680ab910 0x7f0a680b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 -- 172.21.15.64:0/3247073479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0094f0 con 0x7f0a680670e0 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a66ffd700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68147490 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c590 tx=0x7f0a58007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.192+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58011590 con 0x7f0a680670e0 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a58011730 con 0x7f0a680670e0 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a680af920 con 0x7f0a680670e0 2023-09-27T23:29:40.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58011af0 con 0x7f0a680670e0 2023-09-27T23:29:40.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a680afee0 con 0x7f0a680670e0 2023-09-27T23:29:40.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680b0100 con 0x7f0a680670e0 2023-09-27T23:29:40.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f0a58012420 con 0x7f0a680670e0 2023-09-27T23:29:40.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a4effd700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0a54060b00 0x7f0a54062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a667fc700 1 -- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0a54060b00 msgr2=0x7f0a54062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:40.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f0a5807c630 con 0x7f0a680670e0 2023-09-27T23:29:40.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.196+0000 7f0a667fc700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0a54060b00 0x7f0a54062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:40.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.204+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0a58049bb0 con 0x7f0a680670e0 2023-09-27T23:29:40.227 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:40.220+0000 7fdbce914000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:29:40.292 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:40.288+0000 7fdbce914000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:29:40.300 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:40.298+0000 7f9c30798000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:29:40.359 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:40.354+0000 7f9c30798000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:29:40.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.360+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a6804be40 con 0x7f0a680670e0 2023-09-27T23:29:40.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a4effd700 1 -- 172.21.15.64:0/3247073479 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+17572 (secure 0 0 0) 0x7f0a58017090 con 0x7f0a680670e0 2023-09-27T23:29:40.369 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:40.369 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:35.535069+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:18.249013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:18.681348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3072261341":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/761037533":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/4251293646":"2023-09-28T23:29:35.534987+0000","172.21.15.123:6832/33228":"2023-09-28T23:29:35.534987+0000","172.21.15.123:6833/33228":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/2638398233":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/1622918165":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/2133044744":"2023-09-28T23:29:35.534987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0a54060b00 msgr2=0x7f0a54062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0a54060b00 0x7f0a54062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 msgr2=0x7f0a68147490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68147490 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c590 tx=0x7f0a58007f30 comp rx=0 tx=0).stop 2023-09-27T23:29:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 shutdown_connections 2023-09-27T23:29:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0a54060b00 0x7f0a54062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a680abce0 0x7f0a68165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a680ab910 0x7f0a680b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 --2- 172.21.15.64:0/3247073479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a680670e0 0x7f0a68147490 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 >> 172.21.15.64:0/3247073479 conn(0x7f0a6806c080 msgr2=0x7f0a68157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 shutdown_connections 2023-09-27T23:29:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.364+0000 7f0a67fff700 1 -- 172.21.15.64:0/3247073479 wait complete. 2023-09-27T23:29:40.382 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:29:40.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:40.470 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:40.464+0000 7fdbce914000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:29:40.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.480+0000 7f1d0ffff700 1 Processor -- start 2023-09-27T23:29:40.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.480+0000 7f1d0ffff700 1 -- start start 2023-09-27T23:29:40.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d100670e0 0x7f1d10159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d100c1910 con 0x7f1d100ab910 2023-09-27T23:29:40.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d10159cf0 con 0x7f1d100abce0 2023-09-27T23:29:40.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d10159fd0 con 0x7f1d100670e0 2023-09-27T23:29:40.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d100ab910 0x7f1d100cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52660/0 (socket says 172.21.15.64:52660) 2023-09-27T23:29:40.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0effd700 1 -- 172.21.15.64:0/2987486400 learned_addr learned my addr 172.21.15.64:0/2987486400 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0e7fc700 1 --2- 172.21.15.64:0/2987486400 >> v2:172.21.15.123:3300/0 conn(0x7f1d100670e0 0x7f1d10159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0dffb700 1 -- 172.21.15.64:0/2987486400 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982062970 0 0) 0x7f1d100c1910 con 0x7f1d100ab910 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0dffb700 1 -- 172.21.15.64:0/2987486400 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cfc003570 con 0x7f1d100ab910 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0e7fc700 1 -- 172.21.15.64:0/2987486400 >> v1:172.21.15.123:6789/0 conn(0x7f1d100abce0 legacy=0x7f1d100c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0dffb700 1 -- 172.21.15.64:0/2987486400 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757812450 0 0) 0x7f1cfc003570 con 0x7f1d100ab910 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0e7fc700 1 -- 172.21.15.64:0/2987486400 >> v1:172.21.15.64:6789/0 conn(0x7f1d100ab910 legacy=0x7f1d100cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0e7fc700 1 -- 172.21.15.64:0/2987486400 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1015a2b0 con 0x7f1d100670e0 2023-09-27T23:29:40.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0e7fc700 1 --2- 172.21.15.64:0/2987486400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100670e0 0x7f1d10159710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009e60 tx=0x7f1cf8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99001bc8c8424b96 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0dffb700 1 -- 172.21.15.64:0/2987486400 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800e070 con 0x7f1d100670e0 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0dffb700 1 -- 172.21.15.64:0/2987486400 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf800b850 con 0x7f1d100670e0 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0dffb700 1 -- 172.21.15.64:0/2987486400 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00001470 con 0x7f1d100670e0 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- 172.21.15.64:0/2987486400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100670e0 msgr2=0x7f1d10159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/2987486400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100670e0 0x7f1d10159710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009e60 tx=0x7f1cf8009780 comp rx=0 tx=0).stop 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- 172.21.15.64:0/2987486400 shutdown_connections 2023-09-27T23:29:40.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/2987486400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100670e0 0x7f1d10159710 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- 172.21.15.64:0/2987486400 >> 172.21.15.64:0/2987486400 conn(0x7f1d1006c080 msgr2=0x7f1d100ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:40.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- 172.21.15.64:0/2987486400 shutdown_connections 2023-09-27T23:29:40.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- 172.21.15.64:0/2987486400 wait complete. 2023-09-27T23:29:40.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 Processor -- start 2023-09-27T23:29:40.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.484+0000 7f1d0ffff700 1 -- start start 2023-09-27T23:29:40.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d100670e0 0x7f1d10149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100ab910 0x7f1d100cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 0x7f1d1006d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100aab50 con 0x7f1d100abce0 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100cc310 con 0x7f1d100ab910 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100a9ab0 con 0x7f1d100670e0 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 0x7f1d1006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100ab910 0x7f1d100cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 0x7f1d1006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32934/0 (socket says 172.21.15.64:32934) 2023-09-27T23:29:40.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 -- 172.21.15.64:0/1151142853 learned_addr learned my addr 172.21.15.64:0/1151142853 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0effd700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d100670e0 0x7f1d10149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 -- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d100670e0 msgr2=0x7f1d10149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d100670e0 0x7f1d10149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 -- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100ab910 msgr2=0x7f1d100cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100ab910 0x7f1d100cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 -- 172.21.15.64:0/1151142853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8009a20 con 0x7f1d100abce0 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0e7fc700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100ab910 0x7f1d100cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0f7fe700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 0x7f1d1006d050 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c840 tx=0x7f1d08007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:40.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08011590 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d080116f0 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d100a9d90 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08011ab0 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d100aa350 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.488+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1004be40 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.492+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f1d0801b3b0 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.492+0000 7f1d06ffd700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 0x7f1cf0062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.492+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f1d0807c9b0 con 0x7f1d100abce0 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.492+0000 7f1d0effd700 1 -- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 msgr2=0x7f1cf0062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:40.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.492+0000 7f1d0effd700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 0x7f1cf0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:40.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.500+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1d0804c710 con 0x7f1d100abce0 2023-09-27T23:29:40.538 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:40.532+0000 7fdbce914000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:29:40.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.692+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d10068cc0 con 0x7f1d100abce0 2023-09-27T23:29:40.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.692+0000 7f1d06ffd700 1 -- 172.21.15.64:0/1151142853 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1d08017020 con 0x7f1d100abce0 2023-09-27T23:29:40.697 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:40.697 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:19.692498+0000","modified":"2023-09-27T23:29:21.709311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6610":{"gid":6610,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2730764960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2730764960},{"type":"v1","addr":"172.21.15.123:6837","nonce":2730764960}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-09-27T23:29:40.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0effd700 1 -- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 msgr2=0x7f1cf0062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:40.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0effd700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 0x7f1cf0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:29:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 msgr2=0x7f1cf0062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 0x7f1cf0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 msgr2=0x7f1d1006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 0x7f1d1006d050 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c840 tx=0x7f1d08007e60 comp rx=0 tx=0).stop 2023-09-27T23:29:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 shutdown_connections 2023-09-27T23:29:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf0060b00 0x7f1cf0062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d100abce0 0x7f1d1006d050 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d100ab910 0x7f1d100cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 --2- 172.21.15.64:0/1151142853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d100670e0 0x7f1d10149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 >> 172.21.15.64:0/1151142853 conn(0x7f1d1006c080 msgr2=0x7f1d100c3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 shutdown_connections 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.696+0000 7f1d0ffff700 1 -- 172.21.15.64:0/1151142853 wait complete. 2023-09-27T23:29:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-27T23:29:40.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.808+0000 7ff8b3f5e700 1 Processor -- start 2023-09-27T23:29:40.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- start start 2023-09-27T23:29:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff8ac1592b0 0x7ff8ac06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8ac06d590 0x7ff8ac0b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac0688d0 con 0x7ff8ac158ee0 2023-09-27T23:29:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac0b1510 con 0x7ff8ac06d590 2023-09-27T23:29:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac06d9b0 con 0x7ff8ac1592b0 2023-09-27T23:29:40.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b24fb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8ac06d590 0x7ff8ac0b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b1cfa700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff8ac158ee0 0x7ff8ac067af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52674/0 (socket says 172.21.15.64:52674) 2023-09-27T23:29:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b1cfa700 1 -- 172.21.15.64:0/2127173922 learned_addr learned my addr 172.21.15.64:0/2127173922 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:40.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b14f9700 1 --2- 172.21.15.64:0/2127173922 >> v2:172.21.15.123:3300/0 conn(0x7ff8ac1592b0 0x7ff8ac06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b24fb700 1 -- 172.21.15.64:0/2127173922 >> v2:172.21.15.123:3300/0 conn(0x7ff8ac1592b0 msgr2=0x7ff8ac06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b24fb700 1 --2- 172.21.15.64:0/2127173922 >> v2:172.21.15.123:3300/0 conn(0x7ff8ac1592b0 0x7ff8ac06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b24fb700 1 -- 172.21.15.64:0/2127173922 >> v1:172.21.15.64:6789/0 conn(0x7ff8ac158ee0 legacy=0x7ff8ac067af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b24fb700 1 -- 172.21.15.64:0/2127173922 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac06a180 con 0x7ff8ac06d590 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b24fb700 1 --2- 172.21.15.64:0/2127173922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac06d590 0x7ff8ac0b19a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0ac380 tx=0x7ff8a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13da692e80f60c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b0cf8700 1 -- 172.21.15.64:0/2127173922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400e070 con 0x7ff8ac06d590 2023-09-27T23:29:40.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b0cf8700 1 -- 172.21.15.64:0/2127173922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a400bb60 con 0x7ff8ac06d590 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b0cf8700 1 -- 172.21.15.64:0/2127173922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4012430 con 0x7ff8ac06d590 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/2127173922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac06d590 msgr2=0x7ff8ac0b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/2127173922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac06d590 0x7ff8ac0b19a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0ac380 tx=0x7ff8a4009780 comp rx=0 tx=0).stop 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/2127173922 shutdown_connections 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/2127173922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac06d590 0x7ff8ac0b19a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/2127173922 >> v2:172.21.15.123:3300/0 conn(0x7ff8ac1592b0 0x7ff8ac06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/2127173922 >> 172.21.15.64:0/2127173922 conn(0x7ff8ac0cd990 msgr2=0x7ff8ac0ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/2127173922 shutdown_connections 2023-09-27T23:29:40.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.812+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/2127173922 wait complete. 2023-09-27T23:29:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 Processor -- start 2023-09-27T23:29:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 -- start start 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8ac06d590 0x7ff8ac0a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac158ee0 0x7ff8ac154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 0x7ff8ac15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a5f00 con 0x7ff8ac1592b0 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a5660 con 0x7ff8ac158ee0 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a5940 con 0x7ff8ac06d590 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 0x7ff8ac15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 0x7ff8ac15b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32958/0 (socket says 172.21.15.64:32958) 2023-09-27T23:29:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b1cfa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8ac06d590 0x7ff8ac0a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 -- 172.21.15.64:0/877129439 learned_addr learned my addr 172.21.15.64:0/877129439 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b14f9700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac158ee0 0x7ff8ac154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 -- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8ac06d590 msgr2=0x7ff8ac0a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8ac06d590 0x7ff8ac0a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 -- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac158ee0 msgr2=0x7ff8ac154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac158ee0 0x7ff8ac154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 -- 172.21.15.64:0/877129439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a4009a20 con 0x7ff8ac1592b0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b24fb700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 0x7ff8ac15b190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40097c0 tx=0x7ff8a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4027470 con 0x7ff8ac1592b0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a40275f0 con 0x7ff8ac1592b0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac0a5c20 con 0x7ff8ac1592b0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40279b0 con 0x7ff8ac1592b0 2023-09-27T23:29:40.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.816+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac0a5340 con 0x7ff8ac1592b0 2023-09-27T23:29:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.820+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac06d380 con 0x7ff8ac1592b0 2023-09-27T23:29:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.820+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 30) v1 ==== 80418+0+0 (secure 0 0 0) 0x7ff8a4024070 con 0x7ff8ac1592b0 2023-09-27T23:29:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.820+0000 7ff8a1ffb700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff890060b00 0x7ff890062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.820+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6722+0+0 (secure 0 0 0) 0x7ff8a407fd30 con 0x7ff8ac1592b0 2023-09-27T23:29:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.820+0000 7ff8b1cfa700 1 -- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff890060b00 msgr2=0x7ff890062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:29:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.820+0000 7ff8b1cfa700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff890060b00 0x7ff890062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:29:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.828+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff8ac06d380 con 0x7ff8ac1592b0 2023-09-27T23:29:40.950 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:40.946+0000 7f9c30798000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:29:40.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.980+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 31) v1 ==== 79970+0+0 (secure 0 0 0) 0x7ff8a4003040 con 0x7ff8ac1592b0 2023-09-27T23:29:40.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.980+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff890060b00 msgr2=0x7ff890062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:29:40.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:40.980+0000 7ff8a1ffb700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff890060b00 0x7ff890062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.028+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8ac06d380 con 0x7ff8ac1592b0 2023-09-27T23:29:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.028+0000 7ff8a1ffb700 1 -- 172.21.15.64:0/877129439 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4605 (secure 0 0 0) 0x7ff8ac06d380 con 0x7ff8ac1592b0 2023-09-27T23:29:41.036 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:41.036 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:19.692498+0000","modified":"2023-09-27T23:29:21.709311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6610":{"gid":6610,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2730764960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2730764960},{"type":"v1","addr":"172.21.15.123:6837","nonce":2730764960}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-09-27T23:29:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 msgr2=0x7ff8ac15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 0x7ff8ac15b190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40097c0 tx=0x7ff8a4019000 comp rx=0 tx=0).stop 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 shutdown_connections 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff890060b00 0x7ff890062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8ac1592b0 0x7ff8ac15b190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac158ee0 0x7ff8ac154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 --2- 172.21.15.64:0/877129439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8ac06d590 0x7ff8ac0a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 >> 172.21.15.64:0/877129439 conn(0x7ff8ac0cd990 msgr2=0x7ff8ac0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 shutdown_connections 2023-09-27T23:29:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.032+0000 7ff8b3f5e700 1 -- 172.21.15.64:0/877129439 wait complete. 2023-09-27T23:29:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-27T23:29:41.050 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:41.050 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfd9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.051 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfd9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.051 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfd9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.051 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfd9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.051 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfd9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.051 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfa1d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.052 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfa1d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.052 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfa1d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.052 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfa1d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.052 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:29:41.046+0000 7f9bfa1d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:29:41.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6de5df8700 1 Processor -- start 2023-09-27T23:29:41.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6de5df8700 1 -- start start 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6de5df8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 0x7f6de0159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6de5df8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de00c1910 con 0x7f6de00670e0 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6de5df8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de0159cf0 con 0x7f6de00ab910 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6de5df8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0159fd0 con 0x7f6de00c3690 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6ddffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6de00670e0 0x7f6de00c2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52682/0 (socket says 172.21.15.64:52682) 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.144+0000 7f6ddffff700 1 -- 172.21.15.64:0/3425534083 learned_addr learned my addr 172.21.15.64:0/3425534083 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:41.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de55f7700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 0x7f6de0159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318629858 0 0) 0x7f6de00c1910 con 0x7f6de00670e0 2023-09-27T23:29:41.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd0003570 con 0x7f6de00670e0 2023-09-27T23:29:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212926445 0 0) 0x7f6dd0003570 con 0x7f6de00670e0 2023-09-27T23:29:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 msgr2=0x7f6de0159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 0x7f6de0159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 >> v1:172.21.15.123:6789/0 conn(0x7f6de00ab910 legacy=0x7f6de00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de015a2b0 con 0x7f6de00670e0 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f6dcc001470 con 0x7f6de00670e0 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 >> v1:172.21.15.64:6789/0 conn(0x7f6de00670e0 legacy=0x7f6de00c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 0x7f6dd00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0159fd0 con 0x7f6dd0003a90 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000dd70 con 0x7f6dd0004810 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000df30 con 0x7f6de00c3690 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddffff700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de55f7700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 0x7f6dd00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 -- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 msgr2=0x7f6dd00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00c3690 0x7f6dd00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 -- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0004810 msgr2=0x7f6dd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 -- 172.21.15.64:0/3425534083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0003570 con 0x7f6dd0003a90 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddffff700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de4df6700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003ea0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40018b0 tx=0x7f6dd4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2f913b0912e138 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4012420 con 0x7f6dd0003a90 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd4012580 con 0x7f6dd0003a90 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6ddf7fe700 1 -- 172.21.15.64:0/3425534083 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4012920 con 0x7f6dd0003a90 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de5df8700 1 -- 172.21.15.64:0/3425534083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0003a90 msgr2=0x7f6dd0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de5df8700 1 --2- 172.21.15.64:0/3425534083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd0003a90 0x7f6dd0003ea0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40018b0 tx=0x7f6dd4007d30 comp rx=0 tx=0).stop 2023-09-27T23:29:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de55f7700 1 -- 172.21.15.64:0/3425534083 reap_dead start 2023-09-27T23:29:41.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de5df8700 1 -- 172.21.15.64:0/3425534083 shutdown_connections 2023-09-27T23:29:41.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de5df8700 1 -- 172.21.15.64:0/3425534083 >> 172.21.15.64:0/3425534083 conn(0x7f6de006c080 msgr2=0x7f6de00ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:41.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de5df8700 1 -- 172.21.15.64:0/3425534083 shutdown_connections 2023-09-27T23:29:41.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.148+0000 7f6de5df8700 1 -- 172.21.15.64:0/3425534083 wait complete. 2023-09-27T23:29:41.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 Processor -- start 2023-09-27T23:29:41.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- start start 2023-09-27T23:29:41.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0003a90 0x7f6de016bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 0x7f6de00c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00a8cc0 0x7f6de00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0169a80 con 0x7f6de00a8850 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0168e90 con 0x7f6dd0003a90 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0169150 con 0x7f6de00a8cc0 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 0x7f6de00c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 0x7f6de00c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32976/0 (socket says 172.21.15.64:32976) 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 -- 172.21.15.64:0/2547013185 learned_addr learned my addr 172.21.15.64:0/2547013185 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de4df6700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0003a90 0x7f6de016bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de55f7700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00a8cc0 0x7f6de00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 -- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00a8cc0 msgr2=0x7f6de00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00a8cc0 0x7f6de00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 -- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0003a90 msgr2=0x7f6de016bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0003a90 0x7f6de016bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 -- 172.21.15.64:0/2547013185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0169430 con 0x7f6de00a8850 2023-09-27T23:29:41.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de55f7700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00a8cc0 0x7f6de00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de4df6700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0003a90 0x7f6de016bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddffff700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 0x7f6de00c9870 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc001ed0 tx=0x7f6de0157140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00c7b0 con 0x7f6de00a8850 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de0169710 con 0x7f6de00a8850 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00c910 con 0x7f6de00a8850 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00ccb0 con 0x7f6de00a8850 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de00ab260 con 0x7f6de00a8850 2023-09-27T23:29:41.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.152+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de00ab480 con 0x7f6de00a8850 2023-09-27T23:29:41.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.156+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f6dcc013630 con 0x7f6de00a8850 2023-09-27T23:29:41.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.156+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f6dcc07cdf0 con 0x7f6de00a8850 2023-09-27T23:29:41.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.160+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6de00ab480 con 0x7f6de00a8850 2023-09-27T23:29:41.211 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:29:41.204+0000 7fdbce914000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:29:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.320+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de00ab480 con 0x7f6de00a8850 2023-09-27T23:29:41.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.320+0000 7f6ddcff9700 1 -- 172.21.15.64:0/2547013185 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18012 (secure 0 0 0) 0x7f6de00ab480 con 0x7f6de00a8850 2023-09-27T23:29:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:41.328 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":89,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:40.971378+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:18.249013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:18.681348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2693941965":"2023-09-28T23:29:40.971254+0000","172.21.15.123:0/920434615":"2023-09-28T23:29:40.971254+0000","172.21.15.123:0/1798562803":"2023-09-28T23:29:40.971254+0000","172.21.15.123:0/1332431270":"2023-09-28T23:29:40.971254+0000","172.21.15.123:0/1240082340":"2023-09-28T23:29:40.971254+0000","172.21.15.123:0/2747459857":"2023-09-28T23:29:40.971254+0000","172.21.15.123:0/3072261341":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/761037533":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/4251293646":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/1772876840":"2023-09-28T23:29:40.971254+0000","172.21.15.123:6832/33228":"2023-09-28T23:29:35.534987+0000","172.21.15.123:6833/33228":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/2638398233":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/1622918165":"2023-09-28T23:29:35.534987+0000","172.21.15.123:0/2133044744":"2023-09-28T23:29:35.534987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:41.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 msgr2=0x7f6de00c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 0x7f6de00c9870 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc001ed0 tx=0x7f6de0157140 comp rx=0 tx=0).stop 2023-09-27T23:29:41.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 shutdown_connections 2023-09-27T23:29:41.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6de00a8cc0 0x7f6de00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6de00a8850 0x7f6de00c9870 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 --2- 172.21.15.64:0/2547013185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd0003a90 0x7f6de016bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 >> 172.21.15.64:0/2547013185 conn(0x7f6de006c080 msgr2=0x7f6de01578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 shutdown_connections 2023-09-27T23:29:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.324+0000 7f6de5df8700 1 -- 172.21.15.64:0/2547013185 wait complete. 2023-09-27T23:29:41.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:29:41.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 Processor -- start 2023-09-27T23:29:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 -- start start 2023-09-27T23:29:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde1415d050 0x7fde1406a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde14160e60 0x7fde140af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1406ba50 con 0x7fde140afd20 2023-09-27T23:29:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1406bee0 con 0x7fde1415d050 2023-09-27T23:29:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde140af600 con 0x7fde14160e60 2023-09-27T23:29:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde1415d050 0x7fde1406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde1415d050 0x7fde1406a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32978/0 (socket says 172.21.15.64:32978) 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 -- 172.21.15.64:0/2891321256 learned_addr learned my addr 172.21.15.64:0/2891321256 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde13fff700 1 --2- 172.21.15.64:0/2891321256 >> v2:172.21.15.123:3300/0 conn(0x7fde14160e60 0x7fde140af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 -- 172.21.15.64:0/2891321256 >> v2:172.21.15.123:3300/0 conn(0x7fde14160e60 msgr2=0x7fde140af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 --2- 172.21.15.64:0/2891321256 >> v2:172.21.15.123:3300/0 conn(0x7fde14160e60 0x7fde140af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 -- 172.21.15.64:0/2891321256 >> v1:172.21.15.64:6789/0 conn(0x7fde140afd20 legacy=0x7fde1406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 -- 172.21.15.64:0/2891321256 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde140b0140 con 0x7fde1415d050 2023-09-27T23:29:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde13fff700 1 --2- 172.21.15.64:0/2891321256 >> v2:172.21.15.123:3300/0 conn(0x7fde14160e60 0x7fde140af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde18bef700 1 --2- 172.21.15.64:0/2891321256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde1415d050 0x7fde1406a740 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fde1406a150 tx=0x7fddfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f0d1dbab00b279 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde137fe700 1 -- 172.21.15.64:0/2891321256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b860 con 0x7fde1415d050 2023-09-27T23:29:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde137fe700 1 -- 172.21.15.64:0/2891321256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc00b9c0 con 0x7fde1415d050 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde137fe700 1 -- 172.21.15.64:0/2891321256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00bd60 con 0x7fde1415d050 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 -- 172.21.15.64:0/2891321256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde1415d050 msgr2=0x7fde1406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.440+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2891321256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde1415d050 0x7fde1406a740 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fde1406a150 tx=0x7fddfc009250 comp rx=0 tx=0).stop 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- 172.21.15.64:0/2891321256 shutdown_connections 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2891321256 >> v2:172.21.15.123:3300/0 conn(0x7fde14160e60 0x7fde140af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2891321256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde1415d050 0x7fde1406a740 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- 172.21.15.64:0/2891321256 >> 172.21.15.64:0/2891321256 conn(0x7fde14067b60 msgr2=0x7fde140ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- 172.21.15.64:0/2891321256 shutdown_connections 2023-09-27T23:29:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- 172.21.15.64:0/2891321256 wait complete. 2023-09-27T23:29:41.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 Processor -- start 2023-09-27T23:29:41.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- start start 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde140afd20 0x7fde1416e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde1415d050 0x7fde14163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 0x7fde1416b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde14163dd0 con 0x7fde14160e60 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1416ee20 con 0x7fde140afd20 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde1ae53700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1416f100 con 0x7fde1415d050 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde193f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 0x7fde1416b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde193f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 0x7fde1416b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32980/0 (socket says 172.21.15.64:32980) 2023-09-27T23:29:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.444+0000 7fde193f0700 1 -- 172.21.15.64:0/2793516834 learned_addr learned my addr 172.21.15.64:0/2793516834 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde13fff700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde1415d050 0x7fde14163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde18bef700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde140afd20 0x7fde1416e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde193f0700 1 -- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde1415d050 msgr2=0x7fde14163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde193f0700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde1415d050 0x7fde14163890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde193f0700 1 -- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde140afd20 msgr2=0x7fde1416e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde193f0700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde140afd20 0x7fde1416e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde193f0700 1 -- 172.21.15.64:0/2793516834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0094f0 con 0x7fde14160e60 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde193f0700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 0x7fde1416b1d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00c840 tx=0x7fde0c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c012630 con 0x7fde14160e60 2023-09-27T23:29:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c0127b0 con 0x7fde14160e60 2023-09-27T23:29:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1416f3e0 con 0x7fde14160e60 2023-09-27T23:29:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c012b70 con 0x7fde14160e60 2023-09-27T23:29:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1416f9a0 con 0x7fde14160e60 2023-09-27T23:29:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1416fbc0 con 0x7fde14160e60 2023-09-27T23:29:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.448+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 31) v1 ==== 79970+0+0 (secure 0 0 0) 0x7fde0c01a3a0 con 0x7fde14160e60 2023-09-27T23:29:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.452+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7023+0+0 (secure 0 0 0) 0x7fde0c07ce80 con 0x7fde14160e60 2023-09-27T23:29:41.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.464+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fde0c04d100 con 0x7fde14160e60 2023-09-27T23:29:41.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.660+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fde14071120 con 0x7fde14160e60 2023-09-27T23:29:41.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.988+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 32) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fde0c04cbb0 con 0x7fde14160e60 2023-09-27T23:29:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.988+0000 7fde10ff9700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddf4066da0 0x7fddf4069180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:41.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.988+0000 7fde18bef700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddf4066da0 0x7fddf4069180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:41.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:41.992+0000 7fde18bef700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddf4066da0 0x7fddf4069180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde1406a150 tx=0x7fddfc017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:42.616 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:29:42.612+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:29:42.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.628+0000 7fde10ff9700 1 -- 172.21.15.64:0/2793516834 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fde0c069ec0 con 0x7fde14160e60 2023-09-27T23:29:42.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddf4066da0 msgr2=0x7fddf4069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:42.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddf4066da0 0x7fddf4069180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde1406a150 tx=0x7fddfc017000 comp rx=0 tx=0).stop 2023-09-27T23:29:42.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 msgr2=0x7fde1416b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:42.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 0x7fde1416b1d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00c840 tx=0x7fde0c001670 comp rx=0 tx=0).stop 2023-09-27T23:29:42.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 shutdown_connections 2023-09-27T23:29:42.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddf4066da0 0x7fddf4069180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde14160e60 0x7fde1416b1d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde1415d050 0x7fde14163890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 --2- 172.21.15.64:0/2793516834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde140afd20 0x7fde1416e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 >> 172.21.15.64:0/2793516834 conn(0x7fde14067b60 msgr2=0x7fde1415a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:42.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 shutdown_connections 2023-09-27T23:29:42.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.632+0000 7fde1ae53700 1 -- 172.21.15.64:0/2793516834 wait complete. 2023-09-27T23:29:42.639 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:29:42.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:29:42.661 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:29:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 Processor -- start 2023-09-27T23:29:42.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 -- start start 2023-09-27T23:29:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05a80cb970 0x7f05a80ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f05a80ca2c0 0x7f05a806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80cbdf0 con 0x7f05a80cb970 2023-09-27T23:29:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80cc0d0 con 0x7f05a80ca2c0 2023-09-27T23:29:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.764+0000 7f05ae488700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80cc3b0 con 0x7f05a8158e20 2023-09-27T23:29:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05a80cb970 0x7f05a80ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05a80cb970 0x7f05a80ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32990/0 (socket says 172.21.15.64:32990) 2023-09-27T23:29:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 -- 172.21.15.64:0/3357373791 learned_addr learned my addr 172.21.15.64:0/3357373791 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a7fff700 1 --2- 172.21.15.64:0/3357373791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05aca25700 1 --2- 172.21.15.64:0/3357373791 >> v2:172.21.15.123:3300/0 conn(0x7f05a80ca2c0 0x7f05a806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 -- 172.21.15.64:0/3357373791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 msgr2=0x7f05a80ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 --2- 172.21.15.64:0/3357373791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 -- 172.21.15.64:0/3357373791 >> v2:172.21.15.123:3300/0 conn(0x7f05a80ca2c0 msgr2=0x7f05a806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 --2- 172.21.15.64:0/3357373791 >> v2:172.21.15.123:3300/0 conn(0x7f05a80ca2c0 0x7f05a806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 -- 172.21.15.64:0/3357373791 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80cc690 con 0x7f05a80cb970 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a77fe700 1 --2- 172.21.15.64:0/3357373791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80cb970 0x7f05a80ceb00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cfd10 tx=0x7f0598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4373b746b63d59aa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a6ffd700 1 -- 172.21.15.64:0/3357373791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800b640 con 0x7f05a80cb970 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a6ffd700 1 -- 172.21.15.64:0/3357373791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800b7a0 con 0x7f05a80cb970 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05a6ffd700 1 -- 172.21.15.64:0/3357373791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800bb60 con 0x7f05a80cb970 2023-09-27T23:29:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 -- 172.21.15.64:0/3357373791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80cb970 msgr2=0x7f05a80ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 --2- 172.21.15.64:0/3357373791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80cb970 0x7f05a80ceb00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cfd10 tx=0x7f0598009250 comp rx=0 tx=0).stop 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 -- 172.21.15.64:0/3357373791 shutdown_connections 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 --2- 172.21.15.64:0/3357373791 >> v2:172.21.15.123:3300/0 conn(0x7f05a80ca2c0 0x7f05a806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 --2- 172.21.15.64:0/3357373791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80cb970 0x7f05a80ceb00 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 --2- 172.21.15.64:0/3357373791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 -- 172.21.15.64:0/3357373791 >> 172.21.15.64:0/3357373791 conn(0x7f05a8067bb0 msgr2=0x7f05a80d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 -- 172.21.15.64:0/3357373791 shutdown_connections 2023-09-27T23:29:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.768+0000 7f05ae488700 1 -- 172.21.15.64:0/3357373791 wait complete. 2023-09-27T23:29:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 Processor -- start 2023-09-27T23:29:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 -- start start 2023-09-27T23:29:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 0x7f05a816c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80b4ec0 0x7f05a80a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8154740 con 0x7f05a80ca2c0 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80aaa90 con 0x7f05a80b4ec0 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a816d190 con 0x7f05a8158e20 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 0x7f05a816c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 0x7f05a816c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32998/0 (socket says 172.21.15.64:32998) 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 -- 172.21.15.64:0/3888876392 learned_addr learned my addr 172.21.15.64:0/3888876392 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05aca25700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80b4ec0 0x7f05a80a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a77fe700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 -- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 msgr2=0x7f05a80aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 -- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80b4ec0 msgr2=0x7f05a80a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80b4ec0 0x7f05a80a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 -- 172.21.15.64:0/3888876392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980094f0 con 0x7f05a80ca2c0 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05a7fff700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 0x7f05a816c310 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f059c00e930 tx=0x7f059c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c013680 con 0x7f05a80ca2c0 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c0137e0 con 0x7f05a80ca2c0 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a816c8f0 con 0x7f05a80ca2c0 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c013b80 con 0x7f05a80ca2c0 2023-09-27T23:29:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.772+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a816ceb0 con 0x7f05a80ca2c0 2023-09-27T23:29:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.776+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a8071120 con 0x7f05a80ca2c0 2023-09-27T23:29:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.776+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 32) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f059c0123d0 con 0x7f05a80ca2c0 2023-09-27T23:29:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.776+0000 7f058ffff700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f05940607a0 0x7f0594062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.776+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7109+0+0 (secure 0 0 0) 0x7f059c07d010 con 0x7f05a80ca2c0 2023-09-27T23:29:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.776+0000 7f05a77fe700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f05940607a0 0x7f0594062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:42.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.780+0000 7f05a77fe700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f05940607a0 0x7f0594062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cfd10 tx=0x7f0598017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:42.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.784+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f059c04d120 con 0x7f05a80ca2c0 2023-09-27T23:29:42.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:42.984+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05a8064270 con 0x7f05a80ca2c0 2023-09-27T23:29:43.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.628+0000 7f058ffff700 1 -- 172.21.15.64:0/3888876392 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f059c050740 con 0x7f05a80ca2c0 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f05940607a0 msgr2=0x7f0594062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f05940607a0 0x7f0594062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cfd10 tx=0x7f0598017000 comp rx=0 tx=0).stop 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 msgr2=0x7f05a816c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 0x7f05a816c310 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f059c00e930 tx=0x7f059c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 shutdown_connections 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f05940607a0 0x7f0594062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80b4ec0 0x7f05a80a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05a8158e20 0x7f05a80aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 --2- 172.21.15.64:0/3888876392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80ca2c0 0x7f05a816c310 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 >> 172.21.15.64:0/3888876392 conn(0x7f05a8067bb0 msgr2=0x7f05a806abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:43.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 shutdown_connections 2023-09-27T23:29:43.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.636+0000 7f05ae488700 1 -- 172.21.15.64:0/3888876392 wait complete. 2023-09-27T23:29:43.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:29:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.776+0000 7f9df759e700 1 Processor -- start 2023-09-27T23:29:43.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.776+0000 7f9df759e700 1 -- start start 2023-09-27T23:29:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9df8158eb0 0x7f9df806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80c2a90 con 0x7f9df80c2610 2023-09-27T23:29:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80c2cd0 con 0x7f9df806d280 2023-09-27T23:29:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df80c2fb0 con 0x7f9df8158eb0 2023-09-27T23:29:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df5d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9df8158eb0 0x7f9df806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df6d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9df80c2610 0x7f9df80c7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52698/0 (socket says 172.21.15.64:52698) 2023-09-27T23:29:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df6d9d700 1 -- 172.21.15.64:0/4152652449 learned_addr learned my addr 172.21.15.64:0/4152652449 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758725142 0 0) 0x7f9df0001490 con 0x7f9df80c2610 2023-09-27T23:29:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de4003570 con 0x7f9df80c2610 2023-09-27T23:29:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283327303 0 0) 0x7f9df80c2cd0 con 0x7f9df806d280 2023-09-27T23:29:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df0001490 con 0x7f9df806d280 2023-09-27T23:29:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df5d9b700 1 -- 172.21.15.64:0/4152652449 >> v1:172.21.15.123:6789/0 conn(0x7f9df806d280 legacy=0x7f9df81587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df5d9b700 1 -- 172.21.15.64:0/4152652449 >> v1:172.21.15.64:6789/0 conn(0x7f9df80c2610 legacy=0x7f9df80c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908762547 0 0) 0x7f9de4003570 con 0x7f9df80c2610 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df5d9b700 1 -- 172.21.15.64:0/4152652449 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80c3290 con 0x7f9df8158eb0 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df5d9b700 1 --2- 172.21.15.64:0/4152652449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df8158eb0 0x7f9df806d650 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9de00098e0 tx=0x7f9de00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca8cca689c1f44 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0011630 con 0x7f9df8158eb0 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0011790 con 0x7f9df8158eb0 2023-09-27T23:29:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.780+0000 7f9df559a700 1 -- 172.21.15.64:0/4152652449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0011b30 con 0x7f9df8158eb0 2023-09-27T23:29:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 -- 172.21.15.64:0/4152652449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df8158eb0 msgr2=0x7f9df806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 --2- 172.21.15.64:0/4152652449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df8158eb0 0x7f9df806d650 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9de00098e0 tx=0x7f9de00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 -- 172.21.15.64:0/4152652449 shutdown_connections 2023-09-27T23:29:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 --2- 172.21.15.64:0/4152652449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df8158eb0 0x7f9df806d650 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 -- 172.21.15.64:0/4152652449 >> 172.21.15.64:0/4152652449 conn(0x7f9df8069e80 msgr2=0x7f9df806b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 -- 172.21.15.64:0/4152652449 shutdown_connections 2023-09-27T23:29:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 -- 172.21.15.64:0/4152652449 wait complete. 2023-09-27T23:29:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 Processor -- start 2023-09-27T23:29:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.784+0000 7f9df759e700 1 -- start start 2023-09-27T23:29:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df806d280 0x7f9df814d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 0x7f9df80c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9df8158eb0 0x7f9df80c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80af810 con 0x7f9df80c2610 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80aa310 con 0x7f9df806d280 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80a9700 con 0x7f9df8158eb0 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 0x7f9df80c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 0x7f9df80c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33018/0 (socket says 172.21.15.64:33018) 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 -- 172.21.15.64:0/1703818017 learned_addr learned my addr 172.21.15.64:0/1703818017 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df659c700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df806d280 0x7f9df814d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df6d9d700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9df8158eb0 0x7f9df80c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 -- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9df8158eb0 msgr2=0x7f9df80c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9df8158eb0 0x7f9df80c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 -- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df806d280 msgr2=0x7f9df814d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df806d280 0x7f9df814d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 -- 172.21.15.64:0/1703818017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0009420 con 0x7f9df80c2610 2023-09-27T23:29:43.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df5d9b700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 0x7f9df80c4c70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9df806b150 tx=0x7f9de0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:43.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de001f070 con 0x7f9df80c2610 2023-09-27T23:29:43.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0000bc0 con 0x7f9df80c2610 2023-09-27T23:29:43.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df80a99e0 con 0x7f9df80c2610 2023-09-27T23:29:43.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0025420 con 0x7f9df80c2610 2023-09-27T23:29:43.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.788+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df80a9fa0 con 0x7f9df80c2610 2023-09-27T23:29:43.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.792+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9de0023070 con 0x7f9df80c2610 2023-09-27T23:29:43.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.792+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df80aa480 con 0x7f9df80c2610 2023-09-27T23:29:43.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.792+0000 7f9dee7fc700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9dd8060a70 0x7f9dd8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:43.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.792+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7109+0+0 (secure 0 0 0) 0x7f9de007e6b0 con 0x7f9df80c2610 2023-09-27T23:29:43.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.792+0000 7f9df659c700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9dd8060a70 0x7f9dd8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:43.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.796+0000 7f9df659c700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9dd8060a70 0x7f9dd8062f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9de8007fd0 tx=0x7f9de8007d50 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.800+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9de004e400 con 0x7f9df80c2610 2023-09-27T23:29:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:43.968+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9df804be40 con 0x7f9df80c2610 2023-09-27T23:29:44.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.644+0000 7f9dee7fc700 1 -- 172.21.15.64:0/1703818017 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9de0051a20 con 0x7f9df80c2610 2023-09-27T23:29:44.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9dd8060a70 msgr2=0x7f9dd8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:44.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9dd8060a70 0x7f9dd8062f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9de8007fd0 tx=0x7f9de8007d50 comp rx=0 tx=0).stop 2023-09-27T23:29:44.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 msgr2=0x7f9df80c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:44.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 0x7f9df80c4c70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9df806b150 tx=0x7f9de0017000 comp rx=0 tx=0).stop 2023-09-27T23:29:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 shutdown_connections 2023-09-27T23:29:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9dd8060a70 0x7f9dd8062f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9df8158eb0 0x7f9df80c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9df80c2610 0x7f9df80c4c70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 --2- 172.21.15.64:0/1703818017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df806d280 0x7f9df814d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 >> 172.21.15.64:0/1703818017 conn(0x7f9df8069e80 msgr2=0x7f9df80c7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 shutdown_connections 2023-09-27T23:29:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.652+0000 7f9df759e700 1 -- 172.21.15.64:0/1703818017 wait complete. 2023-09-27T23:29:44.660 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:29:44.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:29:44.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.772+0000 7fab2898b700 1 Processor -- start 2023-09-27T23:29:44.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.772+0000 7fab2898b700 1 -- start start 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fab240acd20 0x7fab240cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab240ad0f0 0x7fab240cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240adf30 con 0x7fab240ae3c0 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240cd6f0 con 0x7fab240ad0f0 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240cd9d0 con 0x7fab240acd20 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab240ad0f0 0x7fab240cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab240ad0f0 0x7fab240cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40556/0 (socket says 172.21.15.64:40556) 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 -- 172.21.15.64:0/658352637 learned_addr learned my addr 172.21.15.64:0/658352637 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:44.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2259c700 1 --2- 172.21.15.64:0/658352637 >> v2:172.21.15.123:3300/0 conn(0x7fab240acd20 0x7fab240cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:44.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 -- 172.21.15.64:0/658352637 >> v2:172.21.15.123:3300/0 conn(0x7fab240acd20 msgr2=0x7fab240cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:44.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 --2- 172.21.15.64:0/658352637 >> v2:172.21.15.123:3300/0 conn(0x7fab240acd20 0x7fab240cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 -- 172.21.15.64:0/658352637 >> v1:172.21.15.123:6789/0 conn(0x7fab240ae3c0 legacy=0x7fab240ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:44.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 -- 172.21.15.64:0/658352637 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240adbc0 con 0x7fab240ad0f0 2023-09-27T23:29:44.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab21d9b700 1 --2- 172.21.15.64:0/658352637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240cceb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fab240c4d40 tx=0x7fab18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f55d3d27a355abc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:44.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2159a700 1 -- 172.21.15.64:0/658352637 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18009e70 con 0x7fab240ad0f0 2023-09-27T23:29:44.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2159a700 1 -- 172.21.15.64:0/658352637 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1800b860 con 0x7fab240ad0f0 2023-09-27T23:29:44.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2159a700 1 -- 172.21.15.64:0/658352637 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800bc00 con 0x7fab240ad0f0 2023-09-27T23:29:44.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- 172.21.15.64:0/658352637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 msgr2=0x7fab240cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:44.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 --2- 172.21.15.64:0/658352637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240cceb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fab240c4d40 tx=0x7fab18009250 comp rx=0 tx=0).stop 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- 172.21.15.64:0/658352637 shutdown_connections 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 --2- 172.21.15.64:0/658352637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240cceb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 --2- 172.21.15.64:0/658352637 >> v2:172.21.15.123:3300/0 conn(0x7fab240acd20 0x7fab240cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- 172.21.15.64:0/658352637 >> 172.21.15.64:0/658352637 conn(0x7fab240cdce0 msgr2=0x7fab240ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- 172.21.15.64:0/658352637 shutdown_connections 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- 172.21.15.64:0/658352637 wait complete. 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 Processor -- start 2023-09-27T23:29:44.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.776+0000 7fab2898b700 1 -- start start 2023-09-27T23:29:44.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab240acd20 0x7fab240c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:44.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:44.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab240ae3c0 0x7fab24167630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:44.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24167b70 con 0x7fab240ad0f0 2023-09-27T23:29:44.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24153460 con 0x7fab240acd20 2023-09-27T23:29:44.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab24166b70 con 0x7fab240ae3c0 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40564/0 (socket says 172.21.15.64:40564) 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 -- 172.21.15.64:0/4052568168 learned_addr learned my addr 172.21.15.64:0/4052568168 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2259c700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab240acd20 0x7fab240c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab22d9d700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab240ae3c0 0x7fab24167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 -- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab240ae3c0 msgr2=0x7fab24167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab240ae3c0 0x7fab24167630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 -- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab240acd20 msgr2=0x7fab240c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab240acd20 0x7fab240c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 -- 172.21.15.64:0/4052568168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180094f0 con 0x7fab240ad0f0 2023-09-27T23:29:44.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab21d9b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240c59d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:44.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800bbe0 con 0x7fab240ad0f0 2023-09-27T23:29:44.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab18011420 con 0x7fab240ad0f0 2023-09-27T23:29:44.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240c6040 con 0x7fab240ad0f0 2023-09-27T23:29:44.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab180117e0 con 0x7fab240ad0f0 2023-09-27T23:29:44.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab24166e30 con 0x7fab240ad0f0 2023-09-27T23:29:44.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab24071120 con 0x7fab240ad0f0 2023-09-27T23:29:44.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fab18011bd0 con 0x7fab240ad0f0 2023-09-27T23:29:44.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab127fc700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fab04060a70 0x7fab04062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:44.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6671+0+0 (secure 0 0 0) 0x7fab1807ea50 con 0x7fab240ad0f0 2023-09-27T23:29:44.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.780+0000 7fab2259c700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fab04060a70 0x7fab04062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:44.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.784+0000 7fab2259c700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fab04060a70 0x7fab04062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab24154d50 tx=0x7fab140091a0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:44.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.788+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fab1804e960 con 0x7fab240ad0f0 2023-09-27T23:29:44.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:44.944+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fab240cd3f0 con 0x7fab240ad0f0 2023-09-27T23:29:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.660+0000 7fab127fc700 1 -- 172.21.15.64:0/4052568168 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7fab18051f80 con 0x7fab240ad0f0 2023-09-27T23:29:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fab04060a70 msgr2=0x7fab04062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fab04060a70 0x7fab04062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab24154d50 tx=0x7fab140091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 msgr2=0x7fab240c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240c59d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18017000 comp rx=0 tx=0).stop 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 shutdown_connections 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fab04060a70 0x7fab04062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab240ae3c0 0x7fab24167630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab240ad0f0 0x7fab240c59d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 --2- 172.21.15.64:0/4052568168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab240acd20 0x7fab240c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 >> 172.21.15.64:0/4052568168 conn(0x7fab240cdce0 msgr2=0x7fab240c32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 shutdown_connections 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.668+0000 7fab2898b700 1 -- 172.21.15.64:0/4052568168 wait complete. 2023-09-27T23:29:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:29:45.688 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:29:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.788+0000 7f51530a3700 1 Processor -- start 2023-09-27T23:29:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.788+0000 7f51530a3700 1 -- start start 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f514c13f2d0 0x7f514c0ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f514c0c7780 0x7f514c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c0adc60 con 0x7f514c13f6a0 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c06d7d0 con 0x7f514c0c7780 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c0c8140 con 0x7f514c13f2d0 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f514c0c7780 0x7f514c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f514c0c7780 0x7f514c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40566/0 (socket says 172.21.15.64:40566) 2023-09-27T23:29:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 -- 172.21.15.64:0/389802537 learned_addr learned my addr 172.21.15.64:0/389802537 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51520a1700 1 --2- 172.21.15.64:0/389802537 >> v2:172.21.15.123:3300/0 conn(0x7f514c13f2d0 0x7f514c0ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 -- 172.21.15.64:0/389802537 >> v2:172.21.15.123:3300/0 conn(0x7f514c13f2d0 msgr2=0x7f514c0ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 --2- 172.21.15.64:0/389802537 >> v2:172.21.15.123:3300/0 conn(0x7f514c13f2d0 0x7f514c0ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 -- 172.21.15.64:0/389802537 >> v1:172.21.15.123:6789/0 conn(0x7f514c13f6a0 legacy=0x7f514c15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 -- 172.21.15.64:0/389802537 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0ad050 con 0x7f514c0c7780 2023-09-27T23:29:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51528a2700 1 --2- 172.21.15.64:0/389802537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c0c7780 0x7f514c0c7c00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f514c069bf0 tx=0x7f5144009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc181a47546d9102 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f515109f700 1 -- 172.21.15.64:0/389802537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400e070 con 0x7f514c0c7780 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f515109f700 1 -- 172.21.15.64:0/389802537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514400ba00 con 0x7f514c0c7780 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f515109f700 1 -- 172.21.15.64:0/389802537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440092e0 con 0x7f514c0c7780 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- 172.21.15.64:0/389802537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c0c7780 msgr2=0x7f514c0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 --2- 172.21.15.64:0/389802537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c0c7780 0x7f514c0c7c00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f514c069bf0 tx=0x7f5144009840 comp rx=0 tx=0).stop 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- 172.21.15.64:0/389802537 shutdown_connections 2023-09-27T23:29:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 --2- 172.21.15.64:0/389802537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c0c7780 0x7f514c0c7c00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 --2- 172.21.15.64:0/389802537 >> v2:172.21.15.123:3300/0 conn(0x7f514c13f2d0 0x7f514c0ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- 172.21.15.64:0/389802537 >> 172.21.15.64:0/389802537 conn(0x7f514c067b50 msgr2=0x7f514c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- 172.21.15.64:0/389802537 shutdown_connections 2023-09-27T23:29:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.792+0000 7f51530a3700 1 -- 172.21.15.64:0/389802537 wait complete. 2023-09-27T23:29:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 Processor -- start 2023-09-27T23:29:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 -- start start 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 0x7f514c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c13f2d0 0x7f514c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f514c13f6a0 0x7f514c06aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c157690 con 0x7f514c13f2d0 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c155a80 con 0x7f514c0c7780 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51530a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c156a80 con 0x7f514c13f6a0 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51518a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c13f2d0 0x7f514c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 0x7f514c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 0x7f514c155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:50012/0 (socket says 172.21.15.64:50012) 2023-09-27T23:29:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 -- 172.21.15.64:0/2209104467 learned_addr learned my addr 172.21.15.64:0/2209104467 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51528a2700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f514c13f6a0 0x7f514c06aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 -- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f514c13f6a0 msgr2=0x7f514c06aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f514c13f6a0 0x7f514c06aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 -- 172.21.15.64:0/2209104467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c13f2d0 msgr2=0x7f514c1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c13f2d0 0x7f514c1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.796+0000 7f51520a1700 1 -- 172.21.15.64:0/2209104467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144009ae0 con 0x7f514c0c7780 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f51520a1700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 0x7f514c155540 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f514000d890 tx=0x7f5140008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400125f0 con 0x7f514c0c7780 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5140012750 con 0x7f514c0c7780 2023-09-27T23:29:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c156d60 con 0x7f514c0c7780 2023-09-27T23:29:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140012af0 con 0x7f514c0c7780 2023-09-27T23:29:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c157320 con 0x7f514c0c7780 2023-09-27T23:29:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c04be40 con 0x7f514c0c7780 2023-09-27T23:29:45.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f514001e460 con 0x7f514c0c7780 2023-09-27T23:29:45.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.800+0000 7f513e7fc700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5130060a70 0x7f5130062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.804+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f514007d760 con 0x7f514c0c7780 2023-09-27T23:29:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.804+0000 7f51518a0700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5130060a70 0x7f5130062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:45.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.804+0000 7f51518a0700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5130060a70 0x7f5130062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f514c06b610 tx=0x7f5148007ea0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:45.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.812+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f51400511f0 con 0x7f514c0c7780 2023-09-27T23:29:45.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.984+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f514c06ba60 con 0x7f514c0c7780 2023-09-27T23:29:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f513e7fc700 1 -- 172.21.15.64:0/2209104467 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5140050d80 con 0x7f514c0c7780 2023-09-27T23:29:46.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5130060a70 msgr2=0x7f5130062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5130060a70 0x7f5130062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f514c06b610 tx=0x7f5148007ea0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 msgr2=0x7f514c155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 0x7f514c155540 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f514000d890 tx=0x7f5140008040 comp rx=0 tx=0).stop 2023-09-27T23:29:46.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 shutdown_connections 2023-09-27T23:29:46.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5130060a70 0x7f5130062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f514c13f6a0 0x7f514c06aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f514c13f2d0 0x7f514c1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 --2- 172.21.15.64:0/2209104467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f514c0c7780 0x7f514c155540 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:45.996+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 >> 172.21.15.64:0/2209104467 conn(0x7f514c067b50 msgr2=0x7f514c0c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:46.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.000+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 shutdown_connections 2023-09-27T23:29:46.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.000+0000 7f51530a3700 1 -- 172.21.15.64:0/2209104467 wait complete. 2023-09-27T23:29:46.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.124+0000 7f98bdf2f700 1 Processor -- start 2023-09-27T23:29:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.124+0000 7f98bdf2f700 1 -- start start 2023-09-27T23:29:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98bdf2f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98b80ae1c0 0x7f98b80ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98bdf2f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98b815cc00 0x7f98b815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98bdf2f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b815d560 con 0x7f98b80addf0 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98bdf2f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80ae610 con 0x7f98b80ae1c0 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98bdf2f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80ac6b0 con 0x7f98b815cc00 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98b80ae1c0 0x7f98b80ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98b80ae1c0 0x7f98b80ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40586/0 (socket says 172.21.15.64:40586) 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 -- 172.21.15.64:0/113803375 learned_addr learned my addr 172.21.15.64:0/113803375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98bd72e700 1 --2- 172.21.15.64:0/113803375 >> v2:172.21.15.123:3300/0 conn(0x7f98b815cc00 0x7f98b815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 -- 172.21.15.64:0/113803375 >> v2:172.21.15.123:3300/0 conn(0x7f98b815cc00 msgr2=0x7f98b815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 --2- 172.21.15.64:0/113803375 >> v2:172.21.15.123:3300/0 conn(0x7f98b815cc00 0x7f98b815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 -- 172.21.15.64:0/113803375 >> v1:172.21.15.123:6789/0 conn(0x7f98b80addf0 legacy=0x7f98b806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:46.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 -- 172.21.15.64:0/113803375 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80c68a0 con 0x7f98b80ae1c0 2023-09-27T23:29:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.128+0000 7f98b7fff700 1 --2- 172.21.15.64:0/113803375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80ae1c0 0x7f98b80ac170 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f98b80c32a0 tx=0x7f98a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b540de7616997738 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98b77fe700 1 -- 172.21.15.64:0/113803375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000b860 con 0x7f98b80ae1c0 2023-09-27T23:29:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98b77fe700 1 -- 172.21.15.64:0/113803375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a000b9c0 con 0x7f98b80ae1c0 2023-09-27T23:29:46.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98b77fe700 1 -- 172.21.15.64:0/113803375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000bd60 con 0x7f98b80ae1c0 2023-09-27T23:29:46.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 -- 172.21.15.64:0/113803375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80ae1c0 msgr2=0x7f98b80ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/113803375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80ae1c0 0x7f98b80ac170 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f98b80c32a0 tx=0x7f98a0009250 comp rx=0 tx=0).stop 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 -- 172.21.15.64:0/113803375 shutdown_connections 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/113803375 >> v2:172.21.15.123:3300/0 conn(0x7f98b815cc00 0x7f98b815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/113803375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80ae1c0 0x7f98b80ac170 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 -- 172.21.15.64:0/113803375 >> 172.21.15.64:0/113803375 conn(0x7f98b80c8cd0 msgr2=0x7f98b80c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 -- 172.21.15.64:0/113803375 shutdown_connections 2023-09-27T23:29:46.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.132+0000 7f98bdf2f700 1 -- 172.21.15.64:0/113803375 wait complete. 2023-09-27T23:29:46.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 Processor -- start 2023-09-27T23:29:46.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 -- start start 2023-09-27T23:29:46.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 0x7f98b80c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b80ae1c0 0x7f98b80af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b815cc00 0x7f98b814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c5650 con 0x7f98b80addf0 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80af8d0 con 0x7f98b80ae1c0 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bdf2f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c4a40 con 0x7f98b815cc00 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 0x7f98b80c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 0x7f98b80c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40598/0 (socket says 172.21.15.64:40598) 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 -- 172.21.15.64:0/2791997738 learned_addr learned my addr 172.21.15.64:0/2791997738 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bd72e700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b815cc00 0x7f98b814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98b7fff700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b80ae1c0 0x7f98b80af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 -- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b815cc00 msgr2=0x7f98b814b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b815cc00 0x7f98b814b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 -- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b80ae1c0 msgr2=0x7f98b80af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b80ae1c0 0x7f98b80af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.136+0000 7f98bcf2d700 1 -- 172.21.15.64:0/2791997738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00094f0 con 0x7f98b80addf0 2023-09-27T23:29:46.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98bcf2d700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 0x7f98b80c43d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c510 tx=0x7f98b000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011590 con 0x7f98b80addf0 2023-09-27T23:29:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0011710 con 0x7f98b80addf0 2023-09-27T23:29:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b80c4d20 con 0x7f98b80addf0 2023-09-27T23:29:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011ad0 con 0x7f98b80addf0 2023-09-27T23:29:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b80c52e0 con 0x7f98b80addf0 2023-09-27T23:29:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b804be40 con 0x7f98b80addf0 2023-09-27T23:29:46.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.140+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f98b001b3b0 con 0x7f98b80addf0 2023-09-27T23:29:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.144+0000 7f98b4ff9700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9898060aa0 0x7f9898062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.144+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f98b007c400 con 0x7f98b80addf0 2023-09-27T23:29:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.144+0000 7f98b7fff700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9898060aa0 0x7f9898062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.144+0000 7f98b7fff700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9898060aa0 0x7f9898062f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98b80c32a0 tx=0x7f98a0017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.148+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f98b004c4a0 con 0x7f98b80addf0 2023-09-27T23:29:46.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.312+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f98b80c6b80 con 0x7f98b80addf0 2023-09-27T23:29:46.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98b4ff9700 1 -- 172.21.15.64:0/2791997738 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7f98b004fac0 con 0x7f98b80addf0 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9898060aa0 msgr2=0x7f9898062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9898060aa0 0x7f9898062f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98b80c32a0 tx=0x7f98a0017000 comp rx=0 tx=0).stop 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 msgr2=0x7f98b80c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 0x7f98b80c43d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c510 tx=0x7f98b000c820 comp rx=0 tx=0).stop 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 shutdown_connections 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9898060aa0 0x7f9898062f50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98b815cc00 0x7f98b814b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b80ae1c0 0x7f98b80af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 --2- 172.21.15.64:0/2791997738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b80addf0 0x7f98b80c43d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 >> 172.21.15.64:0/2791997738 conn(0x7f98b80c8cd0 msgr2=0x7f98b8157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:46.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 shutdown_connections 2023-09-27T23:29:46.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.320+0000 7f98bdf2f700 1 -- 172.21.15.64:0/2791997738 wait complete. 2023-09-27T23:29:46.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:29:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.448+0000 7f497359e700 1 Processor -- start 2023-09-27T23:29:46.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.448+0000 7f497359e700 1 -- start start 2023-09-27T23:29:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f497406aeb0 0x7f4974159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49741595a0 0x7f4974159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497406c850 con 0x7f4974158c90 2023-09-27T23:29:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740cb650 con 0x7f49741595a0 2023-09-27T23:29:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497406b470 con 0x7f497406aeb0 2023-09-27T23:29:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49741595a0 0x7f4974159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49741595a0 0x7f4974159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40614/0 (socket says 172.21.15.64:40614) 2023-09-27T23:29:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 -- 172.21.15.64:0/1864915520 learned_addr learned my addr 172.21.15.64:0/1864915520 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:46.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4971d9b700 1 --2- 172.21.15.64:0/1864915520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f497406aeb0 0x7f4974159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 -- 172.21.15.64:0/1864915520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f497406aeb0 msgr2=0x7f4974159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 --2- 172.21.15.64:0/1864915520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f497406aeb0 0x7f4974159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 -- 172.21.15.64:0/1864915520 >> v1:172.21.15.123:6789/0 conn(0x7f4974158c90 legacy=0x7f49740caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497159a700 1 -- 172.21.15.64:0/1864915520 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76069823 0 0) 0x7f497406c850 con 0x7f4974158c90 2023-09-27T23:29:46.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 -- 172.21.15.64:0/1864915520 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497406b750 con 0x7f49741595a0 2023-09-27T23:29:46.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f4972d9d700 1 --2- 172.21.15.64:0/1864915520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49741595a0 0x7f4974159a20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f496c009f20 tx=0x7f496c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36ed63035c0f46 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497159a700 1 -- 172.21.15.64:0/1864915520 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00e070 con 0x7f49741595a0 2023-09-27T23:29:46.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497159a700 1 -- 172.21.15.64:0/1864915520 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00b9d0 con 0x7f49741595a0 2023-09-27T23:29:46.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.452+0000 7f497159a700 1 -- 172.21.15.64:0/1864915520 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c0092e0 con 0x7f49741595a0 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- 172.21.15.64:0/1864915520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49741595a0 msgr2=0x7f4974159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 --2- 172.21.15.64:0/1864915520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49741595a0 0x7f4974159a20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f496c009f20 tx=0x7f496c009840 comp rx=0 tx=0).stop 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- 172.21.15.64:0/1864915520 shutdown_connections 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 --2- 172.21.15.64:0/1864915520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49741595a0 0x7f4974159a20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 --2- 172.21.15.64:0/1864915520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f497406aeb0 0x7f4974159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- 172.21.15.64:0/1864915520 >> 172.21.15.64:0/1864915520 conn(0x7f4974067bb0 msgr2=0x7f49740cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- 172.21.15.64:0/1864915520 shutdown_connections 2023-09-27T23:29:46.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- 172.21.15.64:0/1864915520 wait complete. 2023-09-27T23:29:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 Processor -- start 2023-09-27T23:29:46.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- start start 2023-09-27T23:29:46.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 0x7f49741525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4974158c90 0x7f497415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49741595a0 0x7f49740b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974157580 con 0x7f497406aeb0 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974157860 con 0x7f4974158c90 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4974157b40 con 0x7f49741595a0 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 0x7f49741525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 0x7f49741525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40618/0 (socket says 172.21.15.64:40618) 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 -- 172.21.15.64:0/3008357433 learned_addr learned my addr 172.21.15.64:0/3008357433 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f4972d9d700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49741595a0 0x7f49740b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f4971d9b700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4974158c90 0x7f497415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 -- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49741595a0 msgr2=0x7f49740b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49741595a0 0x7f49740b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 -- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4974158c90 msgr2=0x7f497415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4974158c90 0x7f497415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 -- 172.21.15.64:0/3008357433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009ae0 con 0x7f497406aeb0 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f4971d9b700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4974158c90 0x7f497415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.456+0000 7f497259c700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 0x7f49741525e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f496400c7e0 tx=0x7f4964007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964011590 con 0x7f497406aeb0 2023-09-27T23:29:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964011710 con 0x7f497406aeb0 2023-09-27T23:29:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4974157e20 con 0x7f497406aeb0 2023-09-27T23:29:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964011ad0 con 0x7f497406aeb0 2023-09-27T23:29:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497415cb50 con 0x7f497406aeb0 2023-09-27T23:29:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497404be40 con 0x7f497406aeb0 2023-09-27T23:29:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f496401b3b0 con 0x7f497406aeb0 2023-09-27T23:29:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f496a7fc700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4954060a70 0x7f4954062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f496407c6b0 con 0x7f497406aeb0 2023-09-27T23:29:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.460+0000 7f4971d9b700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4954060a70 0x7f4954062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.464+0000 7f4971d9b700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4954060a70 0x7f4954062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f49740b26d0 tx=0x7f495c008040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.468+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f496404c750 con 0x7f497406aeb0 2023-09-27T23:29:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.632+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f49741576c0 con 0x7f497406aeb0 2023-09-27T23:29:46.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f496a7fc700 1 -- 172.21.15.64:0/3008357433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49741576c0 con 0x7f497406aeb0 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4954060a70 msgr2=0x7f4954062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4954060a70 0x7f4954062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f49740b26d0 tx=0x7f495c008040 comp rx=0 tx=0).stop 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 msgr2=0x7f49741525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 0x7f49741525e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f496400c7e0 tx=0x7f4964007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 shutdown_connections 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4954060a70 0x7f4954062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49741595a0 0x7f49740b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4974158c90 0x7f497415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 --2- 172.21.15.64:0/3008357433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f497406aeb0 0x7f49741525e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 >> 172.21.15.64:0/3008357433 conn(0x7f4974067bb0 msgr2=0x7f49740cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:46.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 shutdown_connections 2023-09-27T23:29:46.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.644+0000 7f497359e700 1 -- 172.21.15.64:0/3008357433 wait complete. 2023-09-27T23:29:46.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-09-27T23:29:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.760+0000 7fdeb4879700 1 Processor -- start 2023-09-27T23:29:46.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- start start 2023-09-27T23:29:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdeb00acdd0 0x7fdeb00c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 0x7fdeb015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb00ccb00 con 0x7fdeb015ac00 2023-09-27T23:29:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb015b040 con 0x7fdeb00acdd0 2023-09-27T23:29:46.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00cbef0 con 0x7fdeb00c8360 2023-09-27T23:29:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 0x7fdeb015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeaed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdeb00acdd0 0x7fdeb00c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 0x7fdeb015a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:40014/0 (socket says 172.21.15.64:40014) 2023-09-27T23:29:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 -- 172.21.15.64:0/1279116687 learned_addr learned my addr 172.21.15.64:0/1279116687 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 -- 172.21.15.64:0/1279116687 >> v2:172.21.15.123:3300/0 conn(0x7fdeb00acdd0 msgr2=0x7fdeb00c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 --2- 172.21.15.64:0/1279116687 >> v2:172.21.15.123:3300/0 conn(0x7fdeb00acdd0 0x7fdeb00c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 -- 172.21.15.64:0/1279116687 >> v1:172.21.15.123:6789/0 conn(0x7fdeb015ac00 legacy=0x7fdeb00ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 -- 172.21.15.64:0/1279116687 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00cc1d0 con 0x7fdeb00c8360 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeaed9d700 1 --2- 172.21.15.64:0/1279116687 >> v2:172.21.15.123:3300/0 conn(0x7fdeb00acdd0 0x7fdeb00c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeae59c700 1 --2- 172.21.15.64:0/1279116687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 0x7fdeb015a6c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009f20 tx=0x7fdea4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db279bf69e48ff8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeadd9b700 1 -- 172.21.15.64:0/1279116687 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400e070 con 0x7fdeb00c8360 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeadd9b700 1 -- 172.21.15.64:0/1279116687 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400b9d0 con 0x7fdeb00c8360 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeadd9b700 1 -- 172.21.15.64:0/1279116687 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40092e0 con 0x7fdeb00c8360 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- 172.21.15.64:0/1279116687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 msgr2=0x7fdeb015a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 --2- 172.21.15.64:0/1279116687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 0x7fdeb015a6c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009f20 tx=0x7fdea4009840 comp rx=0 tx=0).stop 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- 172.21.15.64:0/1279116687 shutdown_connections 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 --2- 172.21.15.64:0/1279116687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00c8360 0x7fdeb015a6c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 --2- 172.21.15.64:0/1279116687 >> v2:172.21.15.123:3300/0 conn(0x7fdeb00acdd0 0x7fdeb00c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- 172.21.15.64:0/1279116687 >> 172.21.15.64:0/1279116687 conn(0x7fdeb00cdbd0 msgr2=0x7fdeb00cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:46.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- 172.21.15.64:0/1279116687 shutdown_connections 2023-09-27T23:29:46.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.764+0000 7fdeb4879700 1 -- 172.21.15.64:0/1279116687 wait complete. 2023-09-27T23:29:46.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 Processor -- start 2023-09-27T23:29:46.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 -- start start 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00acdd0 0x7fdeb016d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb00c8360 0x7fdeb01546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 0x7fdeb00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb016d820 con 0x7fdeb015ac00 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb0068020 con 0x7fdeb00c8360 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeb4879700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb0154bf0 con 0x7fdeb00acdd0 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeaf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 0x7fdeb00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeaed9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00acdd0 0x7fdeb016d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeaf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 0x7fdeb00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40624/0 (socket says 172.21.15.64:40624) 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeaf59e700 1 -- 172.21.15.64:0/2397539254 learned_addr learned my addr 172.21.15.64:0/2397539254 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeae59c700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb00c8360 0x7fdeb01546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeae59c700 1 -- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00acdd0 msgr2=0x7fdeb016d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeae59c700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00acdd0 0x7fdeb016d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeae59c700 1 -- 172.21.15.64:0/2397539254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 msgr2=0x7fdeb00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeae59c700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 0x7fdeb00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeae59c700 1 -- 172.21.15.64:0/2397539254 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea0009420 con 0x7fdeb00c8360 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeaf59e700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 0x7fdeb00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.768+0000 7fdeaed9d700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00acdd0 0x7fdeb016d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fdeae59c700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb00c8360 0x7fdeb01546b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdea4015000 tx=0x7fdea4018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40088f0 con 0x7fdeb00c8360 2023-09-27T23:29:46.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4008a90 con 0x7fdeb00c8360 2023-09-27T23:29:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea4009ae0 con 0x7fdeb00c8360 2023-09-27T23:29:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4008e50 con 0x7fdeb00c8360 2023-09-27T23:29:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb0067780 con 0x7fdeb00c8360 2023-09-27T23:29:46.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb00b2220 con 0x7fdeb00c8360 2023-09-27T23:29:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fdea4003040 con 0x7fdeb00c8360 2023-09-27T23:29:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fde96ffd700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde9c060aa0 0x7fde9c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:46.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fdea407f0c0 con 0x7fdeb00c8360 2023-09-27T23:29:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.772+0000 7fdeaed9d700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde9c060aa0 0x7fde9c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:46.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.776+0000 7fdeaed9d700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde9c060aa0 0x7fde9c062f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdea0011580 tx=0x7fdea00091a0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:46.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:46.784+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdea404f0d0 con 0x7fdeb00c8360 2023-09-27T23:29:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.004+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fdeb00b2220 con 0x7fdeb00c8360 2023-09-27T23:29:47.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.668+0000 7fde96ffd700 1 -- 172.21.15.64:0/2397539254 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v587) v1 ==== 156+0+0 (secure 0 0 0) 0x7fdeb00b2220 con 0x7fdeb00c8360 2023-09-27T23:29:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde9c060aa0 msgr2=0x7fde9c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde9c060aa0 0x7fde9c062f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdea0011580 tx=0x7fdea00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb00c8360 msgr2=0x7fdeb01546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb00c8360 0x7fdeb01546b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdea4015000 tx=0x7fdea4018000 comp rx=0 tx=0).stop 2023-09-27T23:29:47.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 shutdown_connections 2023-09-27T23:29:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde9c060aa0 0x7fde9c062f50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeb015ac00 0x7fdeb00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb00c8360 0x7fdeb01546b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 --2- 172.21.15.64:0/2397539254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdeb00acdd0 0x7fdeb016d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 >> 172.21.15.64:0/2397539254 conn(0x7fdeb00cdbd0 msgr2=0x7fdeb0155850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:47.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 shutdown_connections 2023-09-27T23:29:47.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.672+0000 7fdeb4879700 1 -- 172.21.15.64:0/2397539254 wait complete. 2023-09-27T23:29:47.696 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-09-27T23:29:47.696 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-09-27T23:29:47.696 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:29:47.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-09-27T23:29:47.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.800+0000 7fe39b59e700 1 Processor -- start 2023-09-27T23:29:47.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- start start 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe39c1613d0 0x7fe39c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39c0ca490 con 0x7fe39c068b00 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39c0c8ff0 con 0x7fe39c0ca010 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39c0c92d0 con 0x7fe39c1613d0 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe39c068b00 0x7fe39c160cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38442/0 (socket says 172.21.15.64:38442) 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39a59c700 1 -- 172.21.15.64:0/1231782299 learned_addr learned my addr 172.21.15.64:0/1231782299 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:47.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe399d9b700 1 --2- 172.21.15.64:0/1231782299 >> v2:172.21.15.123:3300/0 conn(0x7fe39c1613d0 0x7fe39c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:47.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39959a700 1 -- 172.21.15.64:0/1231782299 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082897986 0 0) 0x7fe39c0ca490 con 0x7fe39c068b00 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39959a700 1 -- 172.21.15.64:0/1231782299 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe37c003570 con 0x7fe39c068b00 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe399d9b700 1 -- 172.21.15.64:0/1231782299 >> v1:172.21.15.123:6789/0 conn(0x7fe39c0ca010 legacy=0x7fe39c06cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe399d9b700 1 -- 172.21.15.64:0/1231782299 >> v1:172.21.15.64:6789/0 conn(0x7fe39c068b00 legacy=0x7fe39c160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe399d9b700 1 -- 172.21.15.64:0/1231782299 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0c95b0 con 0x7fe39c1613d0 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe399d9b700 1 --2- 172.21.15.64:0/1231782299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0cc190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe3840098e0 tx=0x7fe3840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df026661b1193392 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39959a700 1 -- 172.21.15.64:0/1231782299 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400b640 con 0x7fe39c1613d0 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39959a700 1 -- 172.21.15.64:0/1231782299 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38400b7a0 con 0x7fe39c1613d0 2023-09-27T23:29:47.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39959a700 1 -- 172.21.15.64:0/1231782299 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400bb40 con 0x7fe39c1613d0 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- 172.21.15.64:0/1231782299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 msgr2=0x7fe39c0cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 --2- 172.21.15.64:0/1231782299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0cc190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe3840098e0 tx=0x7fe3840091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- 172.21.15.64:0/1231782299 shutdown_connections 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 --2- 172.21.15.64:0/1231782299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0cc190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- 172.21.15.64:0/1231782299 >> 172.21.15.64:0/1231782299 conn(0x7fe39c0cdbd0 msgr2=0x7fe39c0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- 172.21.15.64:0/1231782299 shutdown_connections 2023-09-27T23:29:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.804+0000 7fe39b59e700 1 -- 172.21.15.64:0/1231782299 wait complete. 2023-09-27T23:29:47.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 Processor -- start 2023-09-27T23:29:47.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 -- start start 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 0x7fe39c0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe39c0ca010 0x7fe39c0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0c5740 con 0x7fe39c068b00 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0b3690 con 0x7fe39c1613d0 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0b2a80 con 0x7fe39c0ca010 2023-09-27T23:29:47.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 0x7fe39c0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 0x7fe39c0c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40636/0 (socket says 172.21.15.64:40636) 2023-09-27T23:29:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.808+0000 7fe39a59c700 1 -- 172.21.15.64:0/2382817494 learned_addr learned my addr 172.21.15.64:0/2382817494 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe399d9b700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe39c0ca010 0x7fe39c0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:47.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39ad9d700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39a59c700 1 -- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe39c0ca010 msgr2=0x7fe39c0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39a59c700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe39c0ca010 0x7fe39c0b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39a59c700 1 -- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 msgr2=0x7fe39c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39a59c700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39a59c700 1 -- 172.21.15.64:0/2382817494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384009420 con 0x7fe39c068b00 2023-09-27T23:29:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39a59c700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 0x7fe39c0c6000 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c7c0 tx=0x7fe38c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c005cd0 con 0x7fe39c068b00 2023-09-27T23:29:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38c005e30 con 0x7fe39c068b00 2023-09-27T23:29:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c0b2d60 con 0x7fe39c068b00 2023-09-27T23:29:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c007320 con 0x7fe39c068b00 2023-09-27T23:29:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c0b3320 con 0x7fe39c068b00 2023-09-27T23:29:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c04be40 con 0x7fe39c068b00 2023-09-27T23:29:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.812+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe38c01b3b0 con 0x7fe39c068b00 2023-09-27T23:29:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.816+0000 7fe3927fc700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe380060a70 0x7fe380062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.816+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fe38c07c750 con 0x7fe39c068b00 2023-09-27T23:29:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.816+0000 7fe399d9b700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe380060a70 0x7fe380062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.816+0000 7fe399d9b700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe380060a70 0x7fe380062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe39c1617d0 tx=0x7fe384017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:47.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:47.824+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe38c04c7f0 con 0x7fe39c068b00 2023-09-27T23:29:48.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.044+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fe39c0671b0 con 0x7fe39c068b00 2023-09-27T23:29:48.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.672+0000 7fe3927fc700 1 -- 172.21.15.64:0/2382817494 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v588) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe38c07ae60 con 0x7fe39c068b00 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe380060a70 msgr2=0x7fe380062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe380060a70 0x7fe380062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe39c1617d0 tx=0x7fe384017000 comp rx=0 tx=0).stop 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 msgr2=0x7fe39c0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 0x7fe39c0c6000 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c7c0 tx=0x7fe38c007e00 comp rx=0 tx=0).stop 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 shutdown_connections 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe380060a70 0x7fe380062f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe39c1613d0 0x7fe39c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe39c0ca010 0x7fe39c0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 --2- 172.21.15.64:0/2382817494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe39c068b00 0x7fe39c0c6000 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 >> 172.21.15.64:0/2382817494 conn(0x7fe39c0cdbd0 msgr2=0x7fe39c0cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:48.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 shutdown_connections 2023-09-27T23:29:48.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.676+0000 7fe39b59e700 1 -- 172.21.15.64:0/2382817494 wait complete. 2023-09-27T23:29:48.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:29:48.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.812+0000 7f5cd5bf1700 1 Processor -- start 2023-09-27T23:29:48.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.812+0000 7f5cd5bf1700 1 -- start start 2023-09-27T23:29:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd5bf1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5cd00c7170 0x7f5cd00ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd5bf1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cd00cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd5bf1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd00ad0c0 con 0x7f5cd0150200 2023-09-27T23:29:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd5bf1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd00af530 con 0x7f5cd00c7170 2023-09-27T23:29:48.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd5bf1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00c7610 con 0x7f5cd00acc60 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd4bef700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5cd0150200 0x7f5cd00ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38456/0 (socket says 172.21.15.64:38456) 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd4bef700 1 -- 172.21.15.64:0/2546051947 learned_addr learned my addr 172.21.15.64:0/2546051947 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc7fff700 1 --2- 172.21.15.64:0/2546051947 >> v2:172.21.15.123:3300/0 conn(0x7f5cd00c7170 0x7f5cd00ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cd00cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019816522 0 0) 0x7f5ccc001110 con 0x7f5cd0150200 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb8003570 con 0x7f5cd0150200 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 956720288 0 0) 0x7f5cb8003570 con 0x7f5cd0150200 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 msgr2=0x7f5cd00cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cd00cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 >> v2:172.21.15.123:3300/0 conn(0x7f5cd00c7170 msgr2=0x7f5cd00ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:48.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 --2- 172.21.15.64:0/2546051947 >> v2:172.21.15.123:3300/0 conn(0x7f5cd00c7170 0x7f5cd00ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd00af250 con 0x7f5cd0150200 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f5ccc001600 con 0x7f5cd0150200 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 >> v1:172.21.15.64:6789/0 conn(0x7f5cd0150200 legacy=0x7f5cd00ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cb8003a60 0x7f5cb8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb8004810 0x7f5cb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00af530 con 0x7f5cb8003a60 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00c7610 con 0x7f5cb8004810 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb800ba60 con 0x7f5cd00acc60 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc7fff700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb8004810 0x7f5cb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd4bef700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cb8003a60 0x7f5cb8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 -- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb8004810 msgr2=0x7f5cb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb8004810 0x7f5cb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 -- 172.21.15.64:0/2546051947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cb8003a60 msgr2=0x7f5cb8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cb8003a60 0x7f5cb8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 -- 172.21.15.64:0/2546051947 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8003570 con 0x7f5cd00acc60 2023-09-27T23:29:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cd53f0700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cb80051c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8010570 tx=0x7f5cc800d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ef51b9f601309d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.816+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8010cf0 con 0x7f5cd00acc60 2023-09-27T23:29:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8011040 con 0x7f5cd00acc60 2023-09-27T23:29:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cc77fe700 1 -- 172.21.15.64:0/2546051947 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800b300 con 0x7f5cd00acc60 2023-09-27T23:29:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 msgr2=0x7f5cb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/2546051947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd00acc60 0x7f5cb80051c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8010570 tx=0x7f5cc800d040 comp rx=0 tx=0).stop 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd53f0700 1 -- 172.21.15.64:0/2546051947 reap_dead start 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/2546051947 shutdown_connections 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/2546051947 >> 172.21.15.64:0/2546051947 conn(0x7f5cd00ccb30 msgr2=0x7f5cd00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/2546051947 shutdown_connections 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/2546051947 wait complete. 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 Processor -- start 2023-09-27T23:29:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.820+0000 7f5cd5bf1700 1 -- start start 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd00acc60 0x7f5cd00a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 0x7f5cd014b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd0068b10 0x7f5cd0068f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00c3be0 con 0x7f5cd00b0b00 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00694c0 con 0x7f5cd00acc60 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00a9150 con 0x7f5cd0068b10 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 0x7f5cd014b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 0x7f5cd014b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40654/0 (socket says 172.21.15.64:40654) 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 -- 172.21.15.64:0/3111681066 learned_addr learned my addr 172.21.15.64:0/3111681066 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd53f0700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd0068b10 0x7f5cd0068f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd4bef700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd00acc60 0x7f5cd00a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 -- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd0068b10 msgr2=0x7f5cd0068f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd0068b10 0x7f5cd0068f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 -- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd00acc60 msgr2=0x7f5cd00a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd00acc60 0x7f5cd00a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 -- 172.21.15.64:0/3111681066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd00a9430 con 0x7f5cd00b0b00 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc7fff700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 0x7f5cd014b6a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0006ac0 tx=0x7f5cc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0011ec0 con 0x7f5cd00b0b00 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc000f940 con 0x7f5cd00b0b00 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd00a9710 con 0x7f5cd00b0b00 2023-09-27T23:29:48.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000fcc0 con 0x7f5cd00b0b00 2023-09-27T23:29:48.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd00a9cd0 con 0x7f5cd00b0b00 2023-09-27T23:29:48.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.824+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd004be40 con 0x7f5cd00b0b00 2023-09-27T23:29:48.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.828+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5cc0014650 con 0x7f5cd00b0b00 2023-09-27T23:29:48.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.828+0000 7f5cc4ff9700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5cbc060a70 0x7f5cbc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.828+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f5cc0018030 con 0x7f5cd00b0b00 2023-09-27T23:29:48.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.828+0000 7f5cd4bef700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5cbc060a70 0x7f5cbc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:48.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.828+0000 7f5cd4bef700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5cbc060a70 0x7f5cbc062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007fd0 tx=0x7f5ccc007a80 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:48.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.836+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5cc00514a0 con 0x7f5cd00b0b00 2023-09-27T23:29:49.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:48.996+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5cd006cbf0 con 0x7f5cd00b0b00 2023-09-27T23:29:49.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.004+0000 7f5cc4ff9700 1 -- 172.21.15.64:0/3111681066 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5cc00220c0 con 0x7f5cd00b0b00 2023-09-27T23:29:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5cbc060a70 msgr2=0x7f5cbc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5cbc060a70 0x7f5cbc062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007fd0 tx=0x7f5ccc007a80 comp rx=0 tx=0).stop 2023-09-27T23:29:49.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 msgr2=0x7f5cd014b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 0x7f5cd014b6a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0006ac0 tx=0x7f5cc00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 shutdown_connections 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5cbc060a70 0x7f5cbc062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cd0068b10 0x7f5cd0068f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd00b0b00 0x7f5cd014b6a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 --2- 172.21.15.64:0/3111681066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd00acc60 0x7f5cd00a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 >> 172.21.15.64:0/3111681066 conn(0x7f5cd00ccb30 msgr2=0x7f5cd014c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 shutdown_connections 2023-09-27T23:29:49.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.008+0000 7f5cd5bf1700 1 -- 172.21.15.64:0/3111681066 wait complete. 2023-09-27T23:29:49.027 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:29:49.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:49.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:29:49.066 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:29:49.067 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:29:49.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa92d12b700 1 Processor -- start 2023-09-27T23:29:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa92d12b700 1 -- start start 2023-09-27T23:29:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa92d12b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9280c6cd0 0x7fa92806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa92d12b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92806b470 con 0x7fa9280c6900 2023-09-27T23:29:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa92d12b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92806e110 con 0x7fa92815cc20 2023-09-27T23:29:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa92d12b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92815d7b0 con 0x7fa9280c6cd0 2023-09-27T23:29:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa927fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa9280c6900 0x7fa92806da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38458/0 (socket says 172.21.15.64:38458) 2023-09-27T23:29:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa927fff700 1 -- 172.21.15.64:0/2347300955 learned_addr learned my addr 172.21.15.64:0/2347300955 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:49.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa9277fe700 1 --2- 172.21.15.64:0/2347300955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9280c6cd0 0x7fa92806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542362250 0 0) 0x7fa92806b470 con 0x7fa9280c6900 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa910003570 con 0x7fa9280c6900 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801084530 0 0) 0x7fa92806e110 con 0x7fa92815cc20 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92806b470 con 0x7fa92815cc20 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa9277fe700 1 -- 172.21.15.64:0/2347300955 >> v1:172.21.15.123:6789/0 conn(0x7fa92815cc20 legacy=0x7fa92815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa9277fe700 1 -- 172.21.15.64:0/2347300955 >> v1:172.21.15.64:6789/0 conn(0x7fa9280c6900 legacy=0x7fa92806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 510907651 0 0) 0x7fa910003570 con 0x7fa9280c6900 2023-09-27T23:29:49.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa9277fe700 1 -- 172.21.15.64:0/2347300955 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928161d10 con 0x7fa9280c6cd0 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.176+0000 7fa9277fe700 1 --2- 172.21.15.64:0/2347300955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9280c6cd0 0x7fa92806b900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa918009940 tx=0x7fa9180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3705b1a3d1b585a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c001470 con 0x7fa9280c6cd0 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918011420 con 0x7fa9280c6cd0 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa926ffd700 1 -- 172.21.15.64:0/2347300955 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9180117a0 con 0x7fa9280c6cd0 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- 172.21.15.64:0/2347300955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9280c6cd0 msgr2=0x7fa92806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 --2- 172.21.15.64:0/2347300955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9280c6cd0 0x7fa92806b900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa918009940 tx=0x7fa9180091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- 172.21.15.64:0/2347300955 shutdown_connections 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 --2- 172.21.15.64:0/2347300955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9280c6cd0 0x7fa92806b900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- 172.21.15.64:0/2347300955 >> 172.21.15.64:0/2347300955 conn(0x7fa928067bb0 msgr2=0x7fa9280cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- 172.21.15.64:0/2347300955 shutdown_connections 2023-09-27T23:29:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- 172.21.15.64:0/2347300955 wait complete. 2023-09-27T23:29:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 Processor -- start 2023-09-27T23:29:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- start start 2023-09-27T23:29:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9280c6900 0x7fa9281479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 0x7fa9280aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92815cc20 0x7fa928170ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280b04c0 con 0x7fa9280c6cd0 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280af8b0 con 0x7fa9280c6900 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92d12b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280afb90 con 0x7fa92815cc20 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 0x7fa9280aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 0x7fa9280aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40668/0 (socket says 172.21.15.64:40668) 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 -- 172.21.15.64:0/3893408642 learned_addr learned my addr 172.21.15.64:0/3893408642 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa92c92a700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92815cc20 0x7fa928170ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa927fff700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9280c6900 0x7fa9281479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 -- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92815cc20 msgr2=0x7fa928170ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92815cc20 0x7fa928170ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 -- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9280c6900 msgr2=0x7fa9281479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9280c6900 0x7fa9281479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 -- 172.21.15.64:0/3893408642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918009420 con 0x7fa9280c6cd0 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa927fff700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9280c6900 0x7fa9281479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa9277fe700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 0x7fa9280aedf0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa918009860 tx=0x7fa918016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91801bae0 con 0x7fa9280c6cd0 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.180+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918020070 con 0x7fa9280c6cd0 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9280afe70 con 0x7fa9280c6cd0 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91801c450 con 0x7fa9280c6cd0 2023-09-27T23:29:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9280c0aa0 con 0x7fa9280c6cd0 2023-09-27T23:29:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9280b0630 con 0x7fa9280c6cd0 2023-09-27T23:29:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa91801c840 con 0x7fa9280c6cd0 2023-09-27T23:29:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa90ffff700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa914060a70 0x7fa914062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fa918022070 con 0x7fa9280c6cd0 2023-09-27T23:29:49.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa927fff700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa914060a70 0x7fa914062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.184+0000 7fa927fff700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa914060a70 0x7fa914062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa91c000ff0 tx=0x7fa91c000bc0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:49.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.192+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa91804ebc0 con 0x7fa9280c6cd0 2023-09-27T23:29:49.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.388+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92804be40 con 0x7fa9280c6cd0 2023-09-27T23:29:49.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa90ffff700 1 -- 172.21.15.64:0/3893408642 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fa91802b090 con 0x7fa9280c6cd0 2023-09-27T23:29:49.397 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:49.397 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2127165715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2127165715},{"type":"v1","addr":"172.21.15.64:6833","nonce":2127165715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1640204914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1640204914},{"type":"v1","addr":"172.21.15.64:6835","nonce":1640204914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":6607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1845318623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1845318623},{"type":"v1","addr":"172.21.15.123:6835","nonce":1845318623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":7000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2917356207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2917356207},{"type":"v1","addr":"172.21.15.123:6837","nonce":2917356207}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[]} 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa914060a70 msgr2=0x7fa914062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa914060a70 0x7fa914062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa91c000ff0 tx=0x7fa91c000bc0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 msgr2=0x7fa9280aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 0x7fa9280aedf0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa918009860 tx=0x7fa918016000 comp rx=0 tx=0).stop 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 shutdown_connections 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa914060a70 0x7fa914062f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92815cc20 0x7fa928170ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9280c6cd0 0x7fa9280aedf0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 --2- 172.21.15.64:0/3893408642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9280c6900 0x7fa9281479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 >> 172.21.15.64:0/3893408642 conn(0x7fa928067bb0 msgr2=0x7fa92815bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:49.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 shutdown_connections 2023-09-27T23:29:49.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.392+0000 7fa92d12b700 1 -- 172.21.15.64:0/3893408642 wait complete. 2023-09-27T23:29:49.401 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 29 2023-09-27T23:29:49.412 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:29:49.412 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:29:49.412 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:29:49.412 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:49.413 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:29:49.413 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:29:49.413 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:29:49.413 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:49.413 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:29:49.413 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:29:49.413 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:29:49.414 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:49.414 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:29:49.414 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:29:49.414 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:29:49.414 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:29:49.422 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:29:49.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:29:49.426 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:29:49.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:29:49.431 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:29:49.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:29:49.432 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:29:49.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:29:49.469 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:29:49.469 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:49.470 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:29:49.470 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:49.474 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:29:49.474 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:29:49.474 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:49.475 DEBUG:teuthology.parallel:result is None 2023-09-27T23:29:49.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:29:49.538 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:29:49.553 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:29:49.556 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:29:49.559 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:29:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.572+0000 7fc7f3fff700 1 Processor -- start 2023-09-27T23:29:49.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.572+0000 7fc7f3fff700 1 -- start start 2023-09-27T23:29:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.572+0000 7fc7f3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f4156770 0x7fc7f4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f40ca8c0 con 0x7fc7f406b480 2023-09-27T23:29:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f40cb700 con 0x7fc7f40c8b50 2023-09-27T23:29:49.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f406bd20 con 0x7fc7f4156770 2023-09-27T23:29:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc7f406b480 0x7fc7f40af120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38494/0 (socket says 172.21.15.64:38494) 2023-09-27T23:29:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f37fe700 1 -- 172.21.15.64:0/2785611899 learned_addr learned my addr 172.21.15.64:0/2785611899 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f2ffd700 1 --2- 172.21.15.64:0/2785611899 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f4156770 0x7fc7f4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f1ffb700 1 -- 172.21.15.64:0/2785611899 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228676191 0 0) 0x7fc7f40ca8c0 con 0x7fc7f406b480 2023-09-27T23:29:49.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f1ffb700 1 -- 172.21.15.64:0/2785611899 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7dc003570 con 0x7fc7f406b480 2023-09-27T23:29:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2785611899 >> v1:172.21.15.123:6789/0 conn(0x7fc7f40c8b50 legacy=0x7fc7f406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2785611899 >> v1:172.21.15.64:6789/0 conn(0x7fc7f406b480 legacy=0x7fc7f40af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f1ffb700 1 -- 172.21.15.64:0/2785611899 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4115700605 0 0) 0x7fc7dc003570 con 0x7fc7f406b480 2023-09-27T23:29:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2785611899 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40caaf0 con 0x7fc7f4156770 2023-09-27T23:29:49.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.576+0000 7fc7f2ffd700 1 --2- 172.21.15.64:0/2785611899 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f4156770 0x7fc7f4156b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009940 tx=0x7fc7e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683d84e31cca45c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:49.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f1ffb700 1 -- 172.21.15.64:0/2785611899 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0017f0 con 0x7fc7f4156770 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f1ffb700 1 -- 172.21.15.64:0/2785611899 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4011420 con 0x7fc7f4156770 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f1ffb700 1 -- 172.21.15.64:0/2785611899 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40117c0 con 0x7fc7f4156770 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2785611899 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f4156770 msgr2=0x7fc7f4156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 --2- 172.21.15.64:0/2785611899 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f4156770 0x7fc7f4156b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009940 tx=0x7fc7e40091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2785611899 shutdown_connections 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 --2- 172.21.15.64:0/2785611899 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f4156770 0x7fc7f4156b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2785611899 >> 172.21.15.64:0/2785611899 conn(0x7fc7f4067bb0 msgr2=0x7fc7f40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2785611899 shutdown_connections 2023-09-27T23:29:49.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.584+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2785611899 wait complete. 2023-09-27T23:29:49.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:29:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 Processor -- start 2023-09-27T23:29:49.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 -- start start 2023-09-27T23:29:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 0x7fc7f4150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f40c8b50 0x7fc7f414c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f4156770 0x7fc7f414cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f414d910 con 0x7fc7f406b480 2023-09-27T23:29:49.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4151950 con 0x7fc7f4156770 2023-09-27T23:29:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4150940 con 0x7fc7f40c8b50 2023-09-27T23:29:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 0x7fc7f4150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 0x7fc7f4150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40678/0 (socket says 172.21.15.64:40678) 2023-09-27T23:29:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f4156770 0x7fc7f414cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2405468426 learned_addr learned my addr 172.21.15.64:0/2405468426 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:49.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f27fc700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f40c8b50 0x7fc7f414c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f40c8b50 msgr2=0x7fc7f414c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f40c8b50 0x7fc7f414c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f4156770 msgr2=0x7fc7f414cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f4156770 0x7fc7f414cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f2ffd700 1 -- 172.21.15.64:0/2405468426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e4009420 con 0x7fc7f406b480 2023-09-27T23:29:49.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f37fe700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f4156770 0x7fc7f414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.588+0000 7fc7f27fc700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f40c8b50 0x7fc7f414c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7f2ffd700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 0x7fc7f4150400 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4069fb0 tx=0x7fc7e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e401cb30 con 0x7fc7f406b480 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e401cc90 con 0x7fc7f406b480 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e401a680 con 0x7fc7f406b480 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2405468426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f4151620 con 0x7fc7f406b480 2023-09-27T23:29:49.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7f3fff700 1 -- 172.21.15.64:0/2405468426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f414d350 con 0x7fc7f406b480 2023-09-27T23:29:49.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.592+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc7e401aa30 con 0x7fc7f406b480 2023-09-27T23:29:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.596+0000 7fc7daffd700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7e0060a70 0x7fc7e0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.596+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fc7e407ea30 con 0x7fc7f406b480 2023-09-27T23:29:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.596+0000 7fc7f27fc700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7e0060a70 0x7fc7e0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.596+0000 7fc7f27fc700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7e0060a70 0x7fc7e0062f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4156fb0 tx=0x7fc7e8009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.596+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f4151480 con 0x7fc7f406b480 2023-09-27T23:29:49.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.604+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc7f4151480 con 0x7fc7f406b480 2023-09-27T23:29:49.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:49.776+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc7f4151480 con 0x7fc7f406b480 2023-09-27T23:29:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7daffd700 1 -- 172.21.15.64:0/2405468426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc7f4151480 con 0x7fc7f406b480 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7e0060a70 msgr2=0x7fc7e0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7e0060a70 0x7fc7e0062f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4156fb0 tx=0x7fc7e8009040 comp rx=0 tx=0).stop 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 msgr2=0x7fc7f4150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 0x7fc7f4150400 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4069fb0 tx=0x7fc7e4016000 comp rx=0 tx=0).stop 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 shutdown_connections 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7e0060a70 0x7fc7e0062f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f4156770 0x7fc7f414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f40c8b50 0x7fc7f414c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 --2- 172.21.15.64:0/2405468426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f406b480 0x7fc7f4150400 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.696+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 >> 172.21.15.64:0/2405468426 conn(0x7fc7f4067bb0 msgr2=0x7fc7f4158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:50.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.700+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 shutdown_connections 2023-09-27T23:29:50.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.700+0000 7fc7d8ff9700 1 -- 172.21.15.64:0/2405468426 wait complete. 2023-09-27T23:29:50.705 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:29:50.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:29:50.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 Processor -- start 2023-09-27T23:29:50.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 -- start start 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f0c140ae0 0x7f3f0c0680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f0c068630 0x7f3f0c0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c06e150 con 0x7f3f0c140710 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c068a70 con 0x7f3f0c140ae0 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f13de9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c06d540 con 0x7f3f0c068630 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f0c140ae0 0x7f3f0c0680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f0c140ae0 0x7f3f0c0680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40680/0 (socket says 172.21.15.64:40680) 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 -- 172.21.15.64:0/4023020675 learned_addr learned my addr 172.21.15.64:0/4023020675 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f12386700 1 --2- 172.21.15.64:0/4023020675 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c068630 0x7f3f0c0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 -- 172.21.15.64:0/4023020675 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c068630 msgr2=0x7f3f0c0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:50.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 --2- 172.21.15.64:0/4023020675 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c068630 0x7f3f0c0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 -- 172.21.15.64:0/4023020675 >> v1:172.21.15.64:6789/0 conn(0x7f3f0c140710 legacy=0x7f3f0c0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 -- 172.21.15.64:0/4023020675 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c06d820 con 0x7f3f0c140ae0 2023-09-27T23:29:50.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f11384700 1 --2- 172.21.15.64:0/4023020675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140ae0 0x7f3f0c0680f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06bfc0 tx=0x7f3f08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c26402915ee589 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f10b83700 1 -- 172.21.15.64:0/4023020675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800b860 con 0x7f3f0c140ae0 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f10b83700 1 -- 172.21.15.64:0/4023020675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0800b9c0 con 0x7f3f0c140ae0 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.824+0000 7f3f10b83700 1 -- 172.21.15.64:0/4023020675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800bd60 con 0x7f3f0c140ae0 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- 172.21.15.64:0/4023020675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140ae0 msgr2=0x7f3f0c0680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 --2- 172.21.15.64:0/4023020675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140ae0 0x7f3f0c0680f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06bfc0 tx=0x7f3f08009250 comp rx=0 tx=0).stop 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- 172.21.15.64:0/4023020675 shutdown_connections 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 --2- 172.21.15.64:0/4023020675 >> v2:172.21.15.123:3300/0 conn(0x7f3f0c068630 0x7f3f0c0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 --2- 172.21.15.64:0/4023020675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140ae0 0x7f3f0c0680f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- 172.21.15.64:0/4023020675 >> 172.21.15.64:0/4023020675 conn(0x7f3f0c0c8850 msgr2=0x7f3f0c0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- 172.21.15.64:0/4023020675 shutdown_connections 2023-09-27T23:29:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- 172.21.15.64:0/4023020675 wait complete. 2023-09-27T23:29:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 Processor -- start 2023-09-27T23:29:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- start start 2023-09-27T23:29:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c068630 0x7f3f0c13ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 0x7f3f0c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c140ae0 0x7f3f0c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15ac00 con 0x7f3f0c140710 2023-09-27T23:29:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15b1c0 con 0x7f3f0c068630 2023-09-27T23:29:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f13de9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1595b0 con 0x7f3f0c140ae0 2023-09-27T23:29:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f11384700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 0x7f3f0c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f11384700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 0x7f3f0c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40684/0 (socket says 172.21.15.64:40684) 2023-09-27T23:29:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.828+0000 7f3f11384700 1 -- 172.21.15.64:0/137093404 learned_addr learned my addr 172.21.15.64:0/137093404 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11b85700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c068630 0x7f3f0c13ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f12386700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c140ae0 0x7f3f0c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11384700 1 -- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c140ae0 msgr2=0x7f3f0c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11384700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c140ae0 0x7f3f0c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11384700 1 -- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c068630 msgr2=0x7f3f0c13ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11384700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c068630 0x7f3f0c13ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11384700 1 -- 172.21.15.64:0/137093404 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f080094f0 con 0x7f3f0c140710 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f11384700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 0x7f3f0c15a2e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06a280 tx=0x7f3f08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08024440 con 0x7f3f0c140710 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f080245a0 con 0x7f3f0c140710 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c159890 con 0x7f3f0c140710 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08024960 con 0x7f3f0c140710 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c13e310 con 0x7f3f0c140710 2023-09-27T23:29:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.832+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c071120 con 0x7f3f0c140710 2023-09-27T23:29:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.836+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3f08024d50 con 0x7f3f0c140710 2023-09-27T23:29:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.836+0000 7f3efdffb700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef0060a70 0x7f3ef0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.836+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3f0807e910 con 0x7f3f0c140710 2023-09-27T23:29:50.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.836+0000 7f3f11b85700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef0060a70 0x7f3ef0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:50.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.836+0000 7f3f11b85700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef0060a70 0x7f3ef0062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3f000039e0 tx=0x7f3f00007ec0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:50.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:50.844+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3f0804ed10 con 0x7f3f0c140710 2023-09-27T23:29:51.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.056+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f0c0642a0 con 0x7f3f0c140710 2023-09-27T23:29:51.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.056+0000 7f3efdffb700 1 -- 172.21.15.64:0/137093404 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3f08052330 con 0x7f3f0c140710 2023-09-27T23:29:51.063 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:51.063 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:29:51.063 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:51.064 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:29:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.066 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.066 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:51.066 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:51.066 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:51.068 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.068 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.068 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:29:51.068 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.070 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.070 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:29:51.070 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:29:51.070 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.070 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.070 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:29:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.072 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.072 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.072 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.072 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.074 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.074 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:29:51.074 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:29:51.074 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.074 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:29:51.075 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.076 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.078 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:29:51.079 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.080 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:29:51.081 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:29:51.083 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:29:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:29:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:29:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:29:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:29:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:29:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:29:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:29:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:29:51.089 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef0060a70 msgr2=0x7f3ef0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef0060a70 0x7f3ef0062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3f000039e0 tx=0x7f3f00007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 msgr2=0x7f3f0c15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 0x7f3f0c15a2e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06a280 tx=0x7f3f08017000 comp rx=0 tx=0).stop 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 shutdown_connections 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3ef0060a70 0x7f3ef0062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f0c140ae0 0x7f3f0c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0c140710 0x7f3f0c15a2e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 --2- 172.21.15.64:0/137093404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f0c068630 0x7f3f0c13ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 >> 172.21.15.64:0/137093404 conn(0x7f3f0c0c8850 msgr2=0x7f3f0c0cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 shutdown_connections 2023-09-27T23:29:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.060+0000 7f3f13de9700 1 -- 172.21.15.64:0/137093404 wait complete. 2023-09-27T23:29:51.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:51.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 Processor -- start 2023-09-27T23:29:51.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- start start 2023-09-27T23:29:51.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64c4158e70 0x7f64c40afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64c406a540 0x7f64c406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c40cc470 0x7f64c40cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c40ca5a0 con 0x7f64c4158e70 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c406dac0 con 0x7f64c406a540 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40b0260 con 0x7f64c40cc470 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64c4158e70 0x7f64c40afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64c4158e70 0x7f64c40afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40694/0 (socket says 172.21.15.64:40694) 2023-09-27T23:29:51.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 -- 172.21.15.64:0/3678045643 learned_addr learned my addr 172.21.15.64:0/3678045643 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64ca4df700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c40cc470 0x7f64c40cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c94dd700 1 --2- 172.21.15.64:0/3678045643 >> v2:172.21.15.123:3300/0 conn(0x7f64c406a540 0x7f64c406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 -- 172.21.15.64:0/3678045643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c40cc470 msgr2=0x7f64c40cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c40cc470 0x7f64c40cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 -- 172.21.15.64:0/3678045643 >> v2:172.21.15.123:3300/0 conn(0x7f64c406a540 msgr2=0x7f64c406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 --2- 172.21.15.64:0/3678045643 >> v2:172.21.15.123:3300/0 conn(0x7f64c406a540 0x7f64c406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 -- 172.21.15.64:0/3678045643 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c406d220 con 0x7f64c4158e70 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64ca4df700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c40cc470 0x7f64c40cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c94dd700 1 --2- 172.21.15.64:0/3678045643 >> v2:172.21.15.123:3300/0 conn(0x7f64c406a540 0x7f64c406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c9cde700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c4158e70 0x7f64c40afd20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cfd20 tx=0x7f64c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba74b49d9e46879 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c8cdc700 1 -- 172.21.15.64:0/3678045643 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000b860 con 0x7f64c4158e70 2023-09-27T23:29:51.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c8cdc700 1 -- 172.21.15.64:0/3678045643 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c000b9c0 con 0x7f64c4158e70 2023-09-27T23:29:51.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64c8cdc700 1 -- 172.21.15.64:0/3678045643 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000bd80 con 0x7f64c4158e70 2023-09-27T23:29:51.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- 172.21.15.64:0/3678045643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c4158e70 msgr2=0x7f64c40afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c4158e70 0x7f64c40afd20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cfd20 tx=0x7f64c0009250 comp rx=0 tx=0).stop 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- 172.21.15.64:0/3678045643 shutdown_connections 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c40cc470 0x7f64c40cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- 172.21.15.64:0/3678045643 >> v2:172.21.15.123:3300/0 conn(0x7f64c406a540 0x7f64c406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 --2- 172.21.15.64:0/3678045643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c4158e70 0x7f64c40afd20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.208+0000 7f64cace0700 1 -- 172.21.15.64:0/3678045643 >> 172.21.15.64:0/3678045643 conn(0x7f64c40670e0 msgr2=0x7f64c40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.212+0000 7f64cace0700 1 -- 172.21.15.64:0/3678045643 shutdown_connections 2023-09-27T23:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.212+0000 7f64cace0700 1 -- 172.21.15.64:0/3678045643 wait complete. 2023-09-27T23:29:51.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.212+0000 7f64cace0700 1 Processor -- start 2023-09-27T23:29:51.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.212+0000 7f64cace0700 1 -- start start 2023-09-27T23:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c406a540 0x7f64c415b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 0x7f64c4169820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c4158e70 0x7f64c41510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4159ef0 con 0x7f64c40cc470 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4169d60 con 0x7f64c406a540 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41592e0 con 0x7f64c4158e70 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 0x7f64c4169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 0x7f64c4169820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40704/0 (socket says 172.21.15.64:40704) 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 -- 172.21.15.64:0/2227730824 learned_addr learned my addr 172.21.15.64:0/2227730824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c9cde700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c406a540 0x7f64c415b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64ca4df700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c4158e70 0x7f64c41510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 -- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c4158e70 msgr2=0x7f64c41510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c4158e70 0x7f64c41510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 -- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c406a540 msgr2=0x7f64c415b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c406a540 0x7f64c415b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 -- 172.21.15.64:0/2227730824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00094f0 con 0x7f64c40cc470 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64ca4df700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c4158e70 0x7f64c41510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64c94dd700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 0x7f64c4169820 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c880 tx=0x7f64b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4011600 con 0x7f64c40cc470 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4011780 con 0x7f64c40cc470 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c41595c0 con 0x7f64c40cc470 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4011b40 con 0x7f64c40cc470 2023-09-27T23:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.216+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c4159b80 con 0x7f64c40cc470 2023-09-27T23:29:51.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.220+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f64b4018350 con 0x7f64c40cc470 2023-09-27T23:29:51.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.220+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c404be40 con 0x7f64c40cc470 2023-09-27T23:29:51.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.220+0000 7f64b9ffb700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64a8060a70 0x7f64a8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.220+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f64b407cdd0 con 0x7f64c40cc470 2023-09-27T23:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.220+0000 7f64c9cde700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64a8060a70 0x7f64a8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.220+0000 7f64c9cde700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64a8060a70 0x7f64a8062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cfd20 tx=0x7f64c0017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.228+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f64b404d040 con 0x7f64c40cc470 2023-09-27T23:29:51.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.392+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64c415a030 con 0x7f64c40cc470 2023-09-27T23:29:51.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.392+0000 7f64b9ffb700 1 -- 172.21.15.64:0/2227730824 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+13856 (secure 0 0 0) 0x7f64b4050660 con 0x7f64c40cc470 2023-09-27T23:29:51.400 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:51.400 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":97,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:50.683792+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:51.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64a8060a70 msgr2=0x7f64a8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64a8060a70 0x7f64a8062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cfd20 tx=0x7f64c0017000 comp rx=0 tx=0).stop 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 msgr2=0x7f64c4169820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 0x7f64c4169820 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c880 tx=0x7f64b4008040 comp rx=0 tx=0).stop 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 shutdown_connections 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64a8060a70 0x7f64a8062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64c4158e70 0x7f64c41510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64c40cc470 0x7f64c4169820 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 --2- 172.21.15.64:0/2227730824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c406a540 0x7f64c415b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 >> 172.21.15.64:0/2227730824 conn(0x7f64c40670e0 msgr2=0x7f64c406ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 shutdown_connections 2023-09-27T23:29:51.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.396+0000 7f64cace0700 1 -- 172.21.15.64:0/2227730824 wait complete. 2023-09-27T23:29:51.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:29:51.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:29:51.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.524+0000 7fe1fcbe6700 1 Processor -- start 2023-09-27T23:29:51.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1fcbe6700 1 -- start start 2023-09-27T23:29:51.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1fcbe6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806dbb0 0x7fe1f80b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1fcbe6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f80ac390 con 0x7fe1f80ab6d0 2023-09-27T23:29:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1fcbe6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f80ac670 con 0x7fe1f806ae20 2023-09-27T23:29:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1fcbe6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80ac950 con 0x7fe1f806dbb0 2023-09-27T23:29:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1eeffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe1f80ab6d0 0x7fe1f80abb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38502/0 (socket says 172.21.15.64:38502) 2023-09-27T23:29:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1eeffd700 1 -- 172.21.15.64:0/2336219563 learned_addr learned my addr 172.21.15.64:0/2336219563 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:51.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f7fff700 1 --2- 172.21.15.64:0/2336219563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806dbb0 0x7fe1f80b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806260500 0 0) 0x7fe1f80ac390 con 0x7fe1f80ab6d0 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e4003570 con 0x7fe1f80ab6d0 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114050139 0 0) 0x7fe1f80ac670 con 0x7fe1f806ae20 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f80ac390 con 0x7fe1f806ae20 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f7fff700 1 -- 172.21.15.64:0/2336219563 >> v1:172.21.15.123:6789/0 conn(0x7fe1f806ae20 legacy=0x7fe1f806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f7fff700 1 -- 172.21.15.64:0/2336219563 >> v1:172.21.15.64:6789/0 conn(0x7fe1f80ab6d0 legacy=0x7fe1f80abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f7fff700 1 -- 172.21.15.64:0/2336219563 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f80acc30 con 0x7fe1f806dbb0 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f7fff700 1 --2- 172.21.15.64:0/2336219563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806dbb0 0x7fe1f80b1440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009e90 tx=0x7fe1f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351593a8e2cce39a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000e070 con 0x7fe1f806dbb0 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f000bb80 con 0x7fe1f806dbb0 2023-09-27T23:29:51.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.528+0000 7fe1f6ffd700 1 -- 172.21.15.64:0/2336219563 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0012430 con 0x7fe1f806dbb0 2023-09-27T23:29:51.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/2336219563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806dbb0 msgr2=0x7fe1f80b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/2336219563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806dbb0 0x7fe1f80b1440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009e90 tx=0x7fe1f0009780 comp rx=0 tx=0).stop 2023-09-27T23:29:51.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/2336219563 shutdown_connections 2023-09-27T23:29:51.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/2336219563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806dbb0 0x7fe1f80b1440 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/2336219563 >> 172.21.15.64:0/2336219563 conn(0x7fe1f80c8cd0 msgr2=0x7fe1f80c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:51.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/2336219563 shutdown_connections 2023-09-27T23:29:51.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/2336219563 wait complete. 2023-09-27T23:29:51.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 Processor -- start 2023-09-27T23:29:51.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.532+0000 7fe1fcbe6700 1 -- start start 2023-09-27T23:29:51.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806ae20 0x7fe1f80a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 0x7fe1f80afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f80ab6d0 0x7fe1f80b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80c2ff0 con 0x7fe1f806dbb0 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80a50a0 con 0x7fe1f80ab6d0 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80a4dc0 con 0x7fe1f806ae20 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 0x7fe1f80afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 0x7fe1f80afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40712/0 (socket says 172.21.15.64:40712) 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 -- 172.21.15.64:0/611764182 learned_addr learned my addr 172.21.15.64:0/611764182 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1f77fe700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806ae20 0x7fe1f80a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1f7fff700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f80ab6d0 0x7fe1f80b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 -- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806ae20 msgr2=0x7fe1f80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806ae20 0x7fe1f80a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 -- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f80ab6d0 msgr2=0x7fe1f80b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f80ab6d0 0x7fe1f80b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 -- 172.21.15.64:0/611764182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009a20 con 0x7fe1f806dbb0 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1eeffd700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 0x7fe1f80afbe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c790 tx=0x7fe1e0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0005b30 con 0x7fe1f806dbb0 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0005c90 con 0x7fe1f806dbb0 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f816a1c0 con 0x7fe1f806dbb0 2023-09-27T23:29:51.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007300 con 0x7fe1f806dbb0 2023-09-27T23:29:51.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.536+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f816a780 con 0x7fe1f806dbb0 2023-09-27T23:29:51.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.540+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f804be40 con 0x7fe1f806dbb0 2023-09-27T23:29:51.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.540+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe1e001b3b0 con 0x7fe1f806dbb0 2023-09-27T23:29:51.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.540+0000 7fe1effff700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1d8060a70 0x7fe1d8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.540+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fe1e007c430 con 0x7fe1f806dbb0 2023-09-27T23:29:51.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.540+0000 7fe1f77fe700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1d8060a70 0x7fe1d8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.540+0000 7fe1f77fe700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1d8060a70 0x7fe1d8062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8007fd0 tx=0x7fe1e8007d50 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.548+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe1e004c710 con 0x7fe1f806dbb0 2023-09-27T23:29:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.708+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1f80b1980 con 0x7fe1f806dbb0 2023-09-27T23:29:51.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.708+0000 7fe1effff700 1 -- 172.21.15.64:0/611764182 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1e00170a0 con 0x7fe1f806dbb0 2023-09-27T23:29:51.714 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1d8060a70 msgr2=0x7fe1d8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1d8060a70 0x7fe1d8062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8007fd0 tx=0x7fe1e8007d50 comp rx=0 tx=0).stop 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 msgr2=0x7fe1f80afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 0x7fe1f80afbe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c790 tx=0x7fe1e0007ea0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 shutdown_connections 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1d8060a70 0x7fe1d8062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1f80ab6d0 0x7fe1f80b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f806dbb0 0x7fe1f80afbe0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 --2- 172.21.15.64:0/611764182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1f806ae20 0x7fe1f80a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 >> 172.21.15.64:0/611764182 conn(0x7fe1f80c8cd0 msgr2=0x7fe1f80c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:51.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 shutdown_connections 2023-09-27T23:29:51.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.712+0000 7fe1fcbe6700 1 -- 172.21.15.64:0/611764182 wait complete. 2023-09-27T23:29:51.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:29:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 Processor -- start 2023-09-27T23:29:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 -- start start 2023-09-27T23:29:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0158c90 0x7f86a00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86a0159770 0x7f86a006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a006c850 con 0x7f86a006aeb0 2023-09-27T23:29:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a006b470 con 0x7f86a0159770 2023-09-27T23:29:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f86a4d45700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a006b750 con 0x7f86a0158c90 2023-09-27T23:29:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f869ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86a0159770 0x7f86a006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f869f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0158c90 0x7f86a00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f869effd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f86a006aeb0 0x7f86a0159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:36296/0 (socket says 172.21.15.64:36296) 2023-09-27T23:29:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.840+0000 7f869effd700 1 -- 172.21.15.64:0/1893535838 learned_addr learned my addr 172.21.15.64:0/1893535838 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:51.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869e7fc700 1 -- 172.21.15.64:0/1893535838 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351111391 0 0) 0x7f86a006c850 con 0x7f86a006aeb0 2023-09-27T23:29:51.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869e7fc700 1 -- 172.21.15.64:0/1893535838 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8680003570 con 0x7f86a006aeb0 2023-09-27T23:29:51.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869ffff700 1 -- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0158c90 msgr2=0x7f86a00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:51.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869ffff700 1 --2- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0158c90 0x7f86a00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869ffff700 1 -- 172.21.15.64:0/1893535838 >> v1:172.21.15.123:6789/0 conn(0x7f86a006aeb0 legacy=0x7f86a0159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869ffff700 1 -- 172.21.15.64:0/1893535838 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a006ba30 con 0x7f86a0159770 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869ffff700 1 --2- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0159770 0x7f86a006cce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8698009e60 tx=0x7f8698009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76f2e4eead99fc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869e7fc700 1 -- 172.21.15.64:0/1893535838 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800e070 con 0x7f86a0159770 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869e7fc700 1 -- 172.21.15.64:0/1893535838 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869800bab0 con 0x7f86a0159770 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f869e7fc700 1 -- 172.21.15.64:0/1893535838 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800be70 con 0x7f86a0159770 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 -- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0159770 msgr2=0x7f86a006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 --2- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0159770 0x7f86a006cce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8698009e60 tx=0x7f8698009780 comp rx=0 tx=0).stop 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 -- 172.21.15.64:0/1893535838 shutdown_connections 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 --2- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0159770 0x7f86a006cce0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 --2- 172.21.15.64:0/1893535838 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0158c90 0x7f86a00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 -- 172.21.15.64:0/1893535838 >> 172.21.15.64:0/1893535838 conn(0x7f86a0067bb0 msgr2=0x7f86a00cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 -- 172.21.15.64:0/1893535838 shutdown_connections 2023-09-27T23:29:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 -- 172.21.15.64:0/1893535838 wait complete. 2023-09-27T23:29:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 Processor -- start 2023-09-27T23:29:51.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.844+0000 7f86a4d45700 1 -- start start 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 0x7f86a0146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0158c90 0x7f86a0155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0159770 0x7f86a00b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00b3b00 con 0x7f86a006aeb0 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0157c40 con 0x7f86a0158c90 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0157f20 con 0x7f86a0159770 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 0x7f86a0146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 0x7f86a0146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40726/0 (socket says 172.21.15.64:40726) 2023-09-27T23:29:51.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 -- 172.21.15.64:0/963407546 learned_addr learned my addr 172.21.15.64:0/963407546 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869ffff700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0159770 0x7f86a00b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869effd700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0158c90 0x7f86a0155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 -- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0159770 msgr2=0x7f86a00b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0159770 0x7f86a00b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 -- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0158c90 msgr2=0x7f86a0155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0158c90 0x7f86a0155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 -- 172.21.15.64:0/963407546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698009a20 con 0x7f86a006aeb0 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f869f7fe700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 0x7f86a0146710 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f869000ec80 tx=0x7f86900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000c8e0 con 0x7f86a006aeb0 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a0158200 con 0x7f86a006aeb0 2023-09-27T23:29:51.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869000ca40 con 0x7f86a006aeb0 2023-09-27T23:29:51.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000cde0 con 0x7f86a006aeb0 2023-09-27T23:29:51.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.848+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a0146c50 con 0x7f86a006aeb0 2023-09-27T23:29:51.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.852+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a004be40 con 0x7f86a006aeb0 2023-09-27T23:29:51.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.852+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8690013680 con 0x7f86a006aeb0 2023-09-27T23:29:51.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.852+0000 7f86977fe700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8684060a70 0x7f8684062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.852+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f869007d920 con 0x7f86a006aeb0 2023-09-27T23:29:51.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.852+0000 7f869effd700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8684060a70 0x7f8684062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:51.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.856+0000 7f869effd700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8684060a70 0x7f8684062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86a00b5920 tx=0x7f8688009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:51.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:51.856+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f869004dc00 con 0x7f86a006aeb0 2023-09-27T23:29:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.016+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f86a006bd10 con 0x7f86a006aeb0 2023-09-27T23:29:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.016+0000 7f86977fe700 1 -- 172.21.15.64:0/963407546 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f86900170a0 con 0x7f86a006aeb0 2023-09-27T23:29:52.023 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:29:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8684060a70 msgr2=0x7f8684062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8684060a70 0x7f8684062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86a00b5920 tx=0x7f8688009040 comp rx=0 tx=0).stop 2023-09-27T23:29:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 msgr2=0x7f86a0146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 0x7f86a0146710 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f869000ec80 tx=0x7f86900091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 shutdown_connections 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8684060a70 0x7f8684062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86a0159770 0x7f86a00b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a0158c90 0x7f86a0155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 --2- 172.21.15.64:0/963407546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86a006aeb0 0x7f86a0146710 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 >> 172.21.15.64:0/963407546 conn(0x7f86a0067bb0 msgr2=0x7f86a00caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 shutdown_connections 2023-09-27T23:29:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.020+0000 7f86a4d45700 1 -- 172.21.15.64:0/963407546 wait complete. 2023-09-27T23:29:52.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:29:52.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.144+0000 7fc87aeb8700 1 Processor -- start 2023-09-27T23:29:52.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- start start 2023-09-27T23:29:52.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8740cb920 0x7fc87415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc87415cee0 0x7fc87406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874068dc0 con 0x7fc874158d40 2023-09-27T23:29:52.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740cc910 con 0x7fc8740cb920 2023-09-27T23:29:52.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740682a0 con 0x7fc87415cee0 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87a6b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc87415cee0 0x7fc87406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8740cb920 0x7fc87415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87a6b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc87415cee0 0x7fc87406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:40196/0 (socket says 172.21.15.64:40196) 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc8740cb920 0x7fc87415c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:50218/0 (socket says 172.21.15.64:50218) 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87a6b7700 1 -- 172.21.15.64:0/2944167428 learned_addr learned my addr 172.21.15.64:0/2944167428 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc879eb6700 1 --1- 172.21.15.64:0/2944167428 >> v1:172.21.15.123:6789/0 conn(0x7fc874158d40 0x7fc8740cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:36304/0 (socket says 172.21.15.64:36304) 2023-09-27T23:29:52.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 -- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc87415cee0 msgr2=0x7fc87406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 --2- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc87415cee0 0x7fc87406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 -- 172.21.15.64:0/2944167428 >> v1:172.21.15.123:6789/0 conn(0x7fc874158d40 legacy=0x7fc8740cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 -- 172.21.15.64:0/2944167428 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874068490 con 0x7fc8740cb920 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc8796b5700 1 --2- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8740cb920 0x7fc87415c9a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc8640098e0 tx=0x7fc8640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7148284f2b7dd047 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc878eb4700 1 -- 172.21.15.64:0/2944167428 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400b640 con 0x7fc8740cb920 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc878eb4700 1 -- 172.21.15.64:0/2944167428 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400b7a0 con 0x7fc8740cb920 2023-09-27T23:29:52.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc878eb4700 1 -- 172.21.15.64:0/2944167428 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400bb40 con 0x7fc8740cb920 2023-09-27T23:29:52.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8740cb920 msgr2=0x7fc87415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8740cb920 0x7fc87415c9a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc8640098e0 tx=0x7fc8640091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- 172.21.15.64:0/2944167428 shutdown_connections 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc87415cee0 0x7fc87406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/2944167428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8740cb920 0x7fc87415c9a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- 172.21.15.64:0/2944167428 >> 172.21.15.64:0/2944167428 conn(0x7fc8740cdbc0 msgr2=0x7fc8740cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- 172.21.15.64:0/2944167428 shutdown_connections 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.148+0000 7fc87aeb8700 1 -- 172.21.15.64:0/2944167428 wait complete. 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 Processor -- start 2023-09-27T23:29:52.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 -- start start 2023-09-27T23:29:52.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 0x7fc874152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc874158d40 0x7fc8740adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc87415cee0 0x7fc8740c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874159110 con 0x7fc8740cb920 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741599b0 con 0x7fc87415cee0 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87aeb8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740c5f80 con 0x7fc874158d40 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 0x7fc874152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc87a6b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc87415cee0 0x7fc8740c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 0x7fc874152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40734/0 (socket says 172.21.15.64:40734) 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 -- 172.21.15.64:0/3213559736 learned_addr learned my addr 172.21.15.64:0/3213559736 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:52.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc8796b5700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc874158d40 0x7fc8740adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 -- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc874158d40 msgr2=0x7fc8740adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc874158d40 0x7fc8740adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 -- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc87415cee0 msgr2=0x7fc8740c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc87415cee0 0x7fc8740c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 -- 172.21.15.64:0/3213559736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009420 con 0x7fc8740cb920 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.152+0000 7fc879eb6700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 0x7fc874152470 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c7f0 tx=0x7fc870007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870005a70 con 0x7fc8740cb920 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870005bf0 con 0x7fc8740cb920 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8700070d0 con 0x7fc8740cb920 2023-09-27T23:29:52.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874159360 con 0x7fc8740cb920 2023-09-27T23:29:52.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741529b0 con 0x7fc8740cb920 2023-09-27T23:29:52.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87415ca70 con 0x7fc8740cb920 2023-09-27T23:29:52.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc870007480 con 0x7fc8740cb920 2023-09-27T23:29:52.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc869ffb700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc858060a70 0x7fc858062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc87007c2a0 con 0x7fc8740cb920 2023-09-27T23:29:52.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.156+0000 7fc8796b5700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc858060a70 0x7fc858062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.160+0000 7fc8796b5700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc858060a70 0x7fc858062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8640093f0 tx=0x7fc864017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.168+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc87415ca70 con 0x7fc8740cb920 2023-09-27T23:29:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.328+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc87415ca70 con 0x7fc8740cb920 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.328+0000 7fc869ffb700 1 -- 172.21.15.64:0/3213559736 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc870017090 con 0x7fc8740cb920 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:29:52.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:29:52.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:29:52.336 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:29:52.336 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:29:52.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:29:52.336 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:29:52.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc858060a70 msgr2=0x7fc858062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc858060a70 0x7fc858062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8640093f0 tx=0x7fc864017000 comp rx=0 tx=0).stop 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 msgr2=0x7fc874152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 0x7fc874152470 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c7f0 tx=0x7fc870007f70 comp rx=0 tx=0).stop 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 shutdown_connections 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc858060a70 0x7fc858062f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc87415cee0 0x7fc8740c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc874158d40 0x7fc8740adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 --2- 172.21.15.64:0/3213559736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8740cb920 0x7fc874152470 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 >> 172.21.15.64:0/3213559736 conn(0x7fc8740cdbc0 msgr2=0x7fc8740b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:52.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 shutdown_connections 2023-09-27T23:29:52.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.332+0000 7fc87aeb8700 1 -- 172.21.15.64:0/3213559736 wait complete. 2023-09-27T23:29:52.349 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:29:52.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:29:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.464+0000 7f159b59e700 1 Processor -- start 2023-09-27T23:29:52.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.464+0000 7f159b59e700 1 -- start start 2023-09-27T23:29:52.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c0687f0 0x7f159c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c0ad3b0 con 0x7f159c0b02b0 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c069120 con 0x7f159c0687f0 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c068c50 con 0x7f159c158a10 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c0687f0 0x7f159c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f1599d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f159c0b02b0 0x7f159c0680e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38516/0 (socket says 172.21.15.64:38516) 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f1599d9b700 1 -- 172.21.15.64:0/3447175373 learned_addr learned my addr 172.21.15.64:0/3447175373 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159a59c700 1 --2- 172.21.15.64:0/3447175373 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159959a700 1 -- 172.21.15.64:0/3447175373 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960885437 0 0) 0x7f159c0ad3b0 con 0x7f159c0b02b0 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159959a700 1 -- 172.21.15.64:0/3447175373 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1588003570 con 0x7f159c0b02b0 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159ad9d700 1 -- 172.21.15.64:0/3447175373 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 msgr2=0x7f159c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159ad9d700 1 --2- 172.21.15.64:0/3447175373 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159ad9d700 1 -- 172.21.15.64:0/3447175373 >> v1:172.21.15.64:6789/0 conn(0x7f159c0b02b0 legacy=0x7f159c0680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159ad9d700 1 -- 172.21.15.64:0/3447175373 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c068e40 con 0x7f159c0687f0 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159ad9d700 1 --2- 172.21.15.64:0/3447175373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0687f0 0x7f159c06d7d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1594009e60 tx=0x7f1594009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da14f8c9456276ef server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159959a700 1 -- 172.21.15.64:0/3447175373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159400e070 con 0x7f159c0687f0 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159959a700 1 -- 172.21.15.64:0/3447175373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159400bab0 con 0x7f159c0687f0 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159959a700 1 -- 172.21.15.64:0/3447175373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159400be70 con 0x7f159c0687f0 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- 172.21.15.64:0/3447175373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0687f0 msgr2=0x7f159c06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 --2- 172.21.15.64:0/3447175373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0687f0 0x7f159c06d7d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1594009e60 tx=0x7f1594009780 comp rx=0 tx=0).stop 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- 172.21.15.64:0/3447175373 shutdown_connections 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 --2- 172.21.15.64:0/3447175373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0687f0 0x7f159c06d7d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 --2- 172.21.15.64:0/3447175373 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- 172.21.15.64:0/3447175373 >> 172.21.15.64:0/3447175373 conn(0x7f159c0ce5e0 msgr2=0x7f159c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:52.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- 172.21.15.64:0/3447175373 shutdown_connections 2023-09-27T23:29:52.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.468+0000 7f159b59e700 1 -- 172.21.15.64:0/3447175373 wait complete. 2023-09-27T23:29:52.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 Processor -- start 2023-09-27T23:29:52.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 -- start start 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c0687f0 0x7f159c16b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 0x7f159c0c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c16bb80 con 0x7f159c0cd200 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0cd680 con 0x7f159c0687f0 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c168700 con 0x7f159c158a10 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 0x7f159c0c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 0x7f159c0c1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40746/0 (socket says 172.21.15.64:40746) 2023-09-27T23:29:52.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.472+0000 7f159ad9d700 1 -- 172.21.15.64:0/2290914738 learned_addr learned my addr 172.21.15.64:0/2290914738 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:52.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159a59c700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c0687f0 0x7f159c16b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f1599d9b700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159ad9d700 1 -- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 msgr2=0x7f159c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159ad9d700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159ad9d700 1 -- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c0687f0 msgr2=0x7f159c16b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159ad9d700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c0687f0 0x7f159c16b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159ad9d700 1 -- 172.21.15.64:0/2290914738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594009a20 con 0x7f159c0cd200 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159ad9d700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 0x7f159c0c1730 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f159c06a440 tx=0x7f1594019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594012d40 con 0x7f159c0cd200 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594027470 con 0x7f159c0cd200 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c169560 con 0x7f159c0cd200 2023-09-27T23:29:52.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594012ea0 con 0x7f159c0cd200 2023-09-27T23:29:52.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c168c30 con 0x7f159c0cd200 2023-09-27T23:29:52.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.476+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c1696f0 con 0x7f159c0cd200 2023-09-27T23:29:52.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.480+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1594024070 con 0x7f159c0cd200 2023-09-27T23:29:52.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.480+0000 7f15927fc700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f157c060a70 0x7f157c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.480+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f159407f3e0 con 0x7f159c0cd200 2023-09-27T23:29:52.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.480+0000 7f159a59c700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f157c060a70 0x7f157c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.480+0000 7f159a59c700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f157c060a70 0x7f157c062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f158c009db0 tx=0x7f158c0091a0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.488+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f159404f760 con 0x7f159c0cd200 2023-09-27T23:29:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.648+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f159c04be40 con 0x7f159c0cd200 2023-09-27T23:29:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.716+0000 7f15927fc700 1 -- 172.21.15.64:0/2290914738 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1594052d80 con 0x7f159c0cd200 2023-09-27T23:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f157c060a70 msgr2=0x7f157c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f157c060a70 0x7f157c062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f158c009db0 tx=0x7f158c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 msgr2=0x7f159c0c1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 0x7f159c0c1730 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f159c06a440 tx=0x7f1594019000 comp rx=0 tx=0).stop 2023-09-27T23:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 shutdown_connections 2023-09-27T23:29:52.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f157c060a70 0x7f157c062f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c0cd200 0x7f159c0c1730 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c158a10 0x7f159c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 --2- 172.21.15.64:0/2290914738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c0687f0 0x7f159c16b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 >> 172.21.15.64:0/2290914738 conn(0x7f159c0ce5e0 msgr2=0x7f159c0ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 shutdown_connections 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.720+0000 7f159b59e700 1 -- 172.21.15.64:0/2290914738 wait complete. 2023-09-27T23:29:52.728 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:29:52.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:29:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 Processor -- start 2023-09-27T23:29:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 -- start start 2023-09-27T23:29:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29e00ace10 0x7f29e00ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29e00c84e0 0x7f29e00c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e00c8e80 con 0x7f29e006d390 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e0157110 con 0x7f29e00ace10 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e5902700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e00ae110 con 0x7f29e00c84e0 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29e00ace10 0x7f29e00ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29e00ace10 0x7f29e00ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40756/0 (socket says 172.21.15.64:40756) 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 -- 172.21.15.64:0/630380065 learned_addr learned my addr 172.21.15.64:0/630380065 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29d7fff700 1 --2- 172.21.15.64:0/630380065 >> v2:172.21.15.123:3300/0 conn(0x7f29e00c84e0 0x7f29e00c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 -- 172.21.15.64:0/630380065 >> v2:172.21.15.123:3300/0 conn(0x7f29e00c84e0 msgr2=0x7f29e00c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 --2- 172.21.15.64:0/630380065 >> v2:172.21.15.123:3300/0 conn(0x7f29e00c84e0 0x7f29e00c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 -- 172.21.15.64:0/630380065 >> v1:172.21.15.123:6789/0 conn(0x7f29e006d390 legacy=0x7f29e006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 -- 172.21.15.64:0/630380065 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e006e0c0 con 0x7f29e00ace10 2023-09-27T23:29:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.848+0000 7f29e4900700 1 --2- 172.21.15.64:0/630380065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e00ad1e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f29e00ce8a0 tx=0x7f29dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15adb7f0a957d68e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29d77fe700 1 -- 172.21.15.64:0/630380065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc009e70 con 0x7f29e00ace10 2023-09-27T23:29:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29d77fe700 1 -- 172.21.15.64:0/630380065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc00b860 con 0x7f29e00ace10 2023-09-27T23:29:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29d77fe700 1 -- 172.21.15.64:0/630380065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00bc00 con 0x7f29e00ace10 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 -- 172.21.15.64:0/630380065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 msgr2=0x7f29e00ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 --2- 172.21.15.64:0/630380065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e00ad1e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f29e00ce8a0 tx=0x7f29dc009250 comp rx=0 tx=0).stop 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 -- 172.21.15.64:0/630380065 shutdown_connections 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 --2- 172.21.15.64:0/630380065 >> v2:172.21.15.123:3300/0 conn(0x7f29e00c84e0 0x7f29e00c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 --2- 172.21.15.64:0/630380065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e00ad1e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 -- 172.21.15.64:0/630380065 >> 172.21.15.64:0/630380065 conn(0x7f29e00cdc00 msgr2=0x7f29e00ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 -- 172.21.15.64:0/630380065 shutdown_connections 2023-09-27T23:29:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 -- 172.21.15.64:0/630380065 wait complete. 2023-09-27T23:29:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 Processor -- start 2023-09-27T23:29:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.852+0000 7f29e5902700 1 -- start start 2023-09-27T23:29:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e006d390 0x7f29e00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e00c84e0 0x7f29e00b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00b2a30 con 0x7f29e00ace10 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00b3400 con 0x7f29e00c84e0 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00b1e20 con 0x7f29e006d390 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e014c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40764/0 (socket says 172.21.15.64:40764) 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 -- 172.21.15.64:0/2881116415 learned_addr learned my addr 172.21.15.64:0/2881116415 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5101700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e00c84e0 0x7f29e00b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e4900700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e006d390 0x7f29e00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 -- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e006d390 msgr2=0x7f29e00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e006d390 0x7f29e00b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 -- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e00c84e0 msgr2=0x7f29e00b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e00c84e0 0x7f29e00b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:52.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 -- 172.21.15.64:0/2881116415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc0094f0 con 0x7f29e00ace10 2023-09-27T23:29:52.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e4900700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e006d390 0x7f29e00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:52.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d7fff700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e014c530 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ec80 tx=0x7f29d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000c850 con 0x7f29e00ace10 2023-09-27T23:29:52.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d000c9b0 con 0x7f29e00ace10 2023-09-27T23:29:52.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e00b2100 con 0x7f29e00ace10 2023-09-27T23:29:52.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000cd50 con 0x7f29e00ace10 2023-09-27T23:29:52.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.856+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e00b26c0 con 0x7f29e00ace10 2023-09-27T23:29:52.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.860+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e00b2be0 con 0x7f29e00ace10 2023-09-27T23:29:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.860+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f29d0013680 con 0x7f29e00ace10 2023-09-27T23:29:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.860+0000 7f29d4ff9700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29cc060aa0 0x7f29cc062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.860+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f29d007d870 con 0x7f29e00ace10 2023-09-27T23:29:52.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.860+0000 7f29e4900700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29cc060aa0 0x7f29cc062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:52.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.860+0000 7f29e4900700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29cc060aa0 0x7f29cc062f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29e00ce8a0 tx=0x7f29dc017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:52.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:52.864+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f29d004da80 con 0x7f29e00ace10 2023-09-27T23:29:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.024+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f29e004be40 con 0x7f29e00ace10 2023-09-27T23:29:53.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.732+0000 7f29d4ff9700 1 -- 172.21.15.64:0/2881116415 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) v1 ==== 170+0+0 (secure 0 0 0) 0x7f29d00170d0 con 0x7f29e00ace10 2023-09-27T23:29:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29cc060aa0 msgr2=0x7f29cc062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29cc060aa0 0x7f29cc062f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29e00ce8a0 tx=0x7f29dc017000 comp rx=0 tx=0).stop 2023-09-27T23:29:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 msgr2=0x7f29e014c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e014c530 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ec80 tx=0x7f29d00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 shutdown_connections 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29cc060aa0 0x7f29cc062f50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e00c84e0 0x7f29e00b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e00ace10 0x7f29e014c530 secure :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ec80 tx=0x7f29d00091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 --2- 172.21.15.64:0/2881116415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e006d390 0x7f29e00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 >> 172.21.15.64:0/2881116415 conn(0x7f29e00cdc00 msgr2=0x7f29e015ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 shutdown_connections 2023-09-27T23:29:53.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.756+0000 7f29e5902700 1 -- 172.21.15.64:0/2881116415 wait complete. 2023-09-27T23:29:53.763 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:29:53.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:29:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.880+0000 7fe836b19700 1 Processor -- start 2023-09-27T23:29:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.880+0000 7fe836b19700 1 -- start start 2023-09-27T23:29:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe830159770 0x7fe83006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83006c850 con 0x7fe83006aeb0 2023-09-27T23:29:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83006b470 con 0x7fe830159770 2023-09-27T23:29:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83006b750 con 0x7fe830158c90 2023-09-27T23:29:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe830159770 0x7fe83006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe830159770 0x7fe83006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40772/0 (socket says 172.21.15.64:40772) 2023-09-27T23:29:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/251414102 learned_addr learned my addr 172.21.15.64:0/251414102 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe835b17700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/251414102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 msgr2=0x7fe8300caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/251414102 >> v1:172.21.15.123:6789/0 conn(0x7fe83006aeb0 legacy=0x7fe830159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:53.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/251414102 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83006ba30 con 0x7fe830159770 2023-09-27T23:29:53.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe835b17700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:53.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe83006cce0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe828009f20 tx=0x7fe828009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706a195dc9a82aba server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:53.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe834b15700 1 -- 172.21.15.64:0/251414102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800e070 con 0x7fe830159770 2023-09-27T23:29:53.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe834b15700 1 -- 172.21.15.64:0/251414102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82800ba80 con 0x7fe830159770 2023-09-27T23:29:53.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe834b15700 1 -- 172.21.15.64:0/251414102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8280092e0 con 0x7fe830159770 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/251414102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 msgr2=0x7fe83006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe83006cce0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe828009f20 tx=0x7fe828009840 comp rx=0 tx=0).stop 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/251414102 shutdown_connections 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe83006cce0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- 172.21.15.64:0/251414102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/251414102 >> 172.21.15.64:0/251414102 conn(0x7fe830067bb0 msgr2=0x7fe8300cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/251414102 shutdown_connections 2023-09-27T23:29:53.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/251414102 wait complete. 2023-09-27T23:29:53.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 Processor -- start 2023-09-27T23:29:53.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- start start 2023-09-27T23:29:53.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83006aeb0 0x7fe8301571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:53.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe8300c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300c8110 con 0x7fe830159770 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830157710 con 0x7fe83006aeb0 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe830157940 con 0x7fe830158c90 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe8300c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe8300c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40782/0 (socket says 172.21.15.64:40782) 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/26742654 learned_addr learned my addr 172.21.15.64:0/26742654 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe835b17700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83006aeb0 0x7fe8301571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe835316700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 msgr2=0x7fe8300c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83006aeb0 msgr2=0x7fe8301571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83006aeb0 0x7fe8301571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 -- 172.21.15.64:0/26742654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe828009ae0 con 0x7fe830159770 2023-09-27T23:29:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe835b17700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83006aeb0 0x7fe8301571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe835316700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836318700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe8300c6620 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe82800b6c0 tx=0x7fe828018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8280032f0 con 0x7fe830159770 2023-09-27T23:29:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82801c410 con 0x7fe830159770 2023-09-27T23:29:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe830157c20 con 0x7fe830159770 2023-09-27T23:29:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82801c7b0 con 0x7fe830159770 2023-09-27T23:29:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.888+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8301581e0 con 0x7fe830159770 2023-09-27T23:29:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.892+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8300c8280 con 0x7fe830159770 2023-09-27T23:29:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.892+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe828003040 con 0x7fe830159770 2023-09-27T23:29:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.892+0000 7fe821ffb700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe814060a70 0x7fe814062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.892+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fe82807eab0 con 0x7fe830159770 2023-09-27T23:29:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.892+0000 7fe835b17700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe814060a70 0x7fe814062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.896+0000 7fe835b17700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe814060a70 0x7fe814062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe82c006f00 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:53.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:53.900+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe82804ea30 con 0x7fe830159770 2023-09-27T23:29:54.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.099+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe83004be40 con 0x7fe830159770 2023-09-27T23:29:54.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.111+0000 7fe821ffb700 1 -- 172.21.15.64:0/26742654 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v31) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe828052050 con 0x7fe830159770 2023-09-27T23:29:54.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.131+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe814060a70 msgr2=0x7fe814062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe814060a70 0x7fe814062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe82c006f00 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2023-09-27T23:29:54.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 msgr2=0x7fe8300c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe8300c6620 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe82800b6c0 tx=0x7fe828018000 comp rx=0 tx=0).stop 2023-09-27T23:29:54.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 shutdown_connections 2023-09-27T23:29:54.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe814060a70 0x7fe814062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe830159770 0x7fe8300c6620 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe830158c90 0x7fe8300c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 --2- 172.21.15.64:0/26742654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe83006aeb0 0x7fe8301571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 >> 172.21.15.64:0/26742654 conn(0x7fe830067bb0 msgr2=0x7fe8300caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:54.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 shutdown_connections 2023-09-27T23:29:54.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.135+0000 7fe836b19700 1 -- 172.21.15.64:0/26742654 wait complete. 2023-09-27T23:29:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 12 and data pool 13 2023-09-27T23:29:54.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:54.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.259+0000 7fe63359e700 1 Processor -- start 2023-09-27T23:29:54.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- start start 2023-09-27T23:29:54.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe634158e70 0x7fe6340afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe63406a540 0x7fe63406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6340ca5a0 con 0x7fe6340cc470 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63406dac0 con 0x7fe63406a540 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6340b0260 con 0x7fe634158e70 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe63406a540 0x7fe63406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe63406a540 0x7fe63406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40784/0 (socket says 172.21.15.64:40784) 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe632d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe6340cc470 0x7fe6340cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38532/0 (socket says 172.21.15.64:38532) 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 -- 172.21.15.64:0/710674779 learned_addr learned my addr 172.21.15.64:0/710674779 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63259c700 1 --2- 172.21.15.64:0/710674779 >> v2:172.21.15.123:3300/0 conn(0x7fe634158e70 0x7fe6340afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 -- 172.21.15.64:0/710674779 >> v2:172.21.15.123:3300/0 conn(0x7fe634158e70 msgr2=0x7fe6340afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 --2- 172.21.15.64:0/710674779 >> v2:172.21.15.123:3300/0 conn(0x7fe634158e70 0x7fe6340afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 -- 172.21.15.64:0/710674779 >> v1:172.21.15.64:6789/0 conn(0x7fe6340cc470 legacy=0x7fe6340cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe631d9b700 1 -- 172.21.15.64:0/710674779 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991101382 0 0) 0x7fe6340ca5a0 con 0x7fe6340cc470 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 -- 172.21.15.64:0/710674779 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63406d220 con 0x7fe63406a540 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63259c700 1 --2- 172.21.15.64:0/710674779 >> v2:172.21.15.123:3300/0 conn(0x7fe634158e70 0x7fe6340afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe62bfff700 1 --2- 172.21.15.64:0/710674779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe63406a540 0x7fe63406c9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe63406a210 tx=0x7fe61c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c65eb9c7d08e82c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe631d9b700 1 -- 172.21.15.64:0/710674779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c009e70 con 0x7fe63406a540 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe631d9b700 1 -- 172.21.15.64:0/710674779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c00b860 con 0x7fe63406a540 2023-09-27T23:29:54.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe631d9b700 1 -- 172.21.15.64:0/710674779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00bc20 con 0x7fe63406a540 2023-09-27T23:29:54.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- 172.21.15.64:0/710674779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe63406a540 msgr2=0x7fe63406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 --2- 172.21.15.64:0/710674779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe63406a540 0x7fe63406c9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe63406a210 tx=0x7fe61c009250 comp rx=0 tx=0).stop 2023-09-27T23:29:54.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- 172.21.15.64:0/710674779 shutdown_connections 2023-09-27T23:29:54.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 --2- 172.21.15.64:0/710674779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe63406a540 0x7fe63406c9d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 --2- 172.21.15.64:0/710674779 >> v2:172.21.15.123:3300/0 conn(0x7fe634158e70 0x7fe6340afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.263+0000 7fe63359e700 1 -- 172.21.15.64:0/710674779 >> 172.21.15.64:0/710674779 conn(0x7fe6340670e0 msgr2=0x7fe6340674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:54.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.267+0000 7fe63359e700 1 -- 172.21.15.64:0/710674779 shutdown_connections 2023-09-27T23:29:54.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.267+0000 7fe63359e700 1 -- 172.21.15.64:0/710674779 wait complete. 2023-09-27T23:29:54.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.267+0000 7fe63359e700 1 Processor -- start 2023-09-27T23:29:54.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.267+0000 7fe63359e700 1 -- start start 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe63406a540 0x7fe63415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 0x7fe63415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe634158e70 0x7fe6340c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c6800 con 0x7fe6340cc470 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c9830 con 0x7fe63406a540 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c6520 con 0x7fe634158e70 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 0x7fe63415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 0x7fe63415de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40790/0 (socket says 172.21.15.64:40790) 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 -- 172.21.15.64:0/1779291363 learned_addr learned my addr 172.21.15.64:0/1779291363 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:54.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe63406a540 0x7fe63415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe632d9d700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe634158e70 0x7fe6340c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 -- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe634158e70 msgr2=0x7fe6340c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe634158e70 0x7fe6340c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 -- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe63406a540 msgr2=0x7fe63415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe63406a540 0x7fe63415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 -- 172.21.15.64:0/1779291363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009420 con 0x7fe6340cc470 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63259c700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe63406a540 0x7fe63415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62bfff700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 0x7fe63415de40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe61c006fd0 tx=0x7fe61c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c01f070 con 0x7fe6340cc470 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c00b9d0 con 0x7fe6340cc470 2023-09-27T23:29:54.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c0094f0 con 0x7fe6340cc470 2023-09-27T23:29:54.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c011420 con 0x7fe6340cc470 2023-09-27T23:29:54.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6340c94c0 con 0x7fe6340cc470 2023-09-27T23:29:54.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.271+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6340c9650 con 0x7fe6340cc470 2023-09-27T23:29:54.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.275+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe61c0117f0 con 0x7fe6340cc470 2023-09-27T23:29:54.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.275+0000 7fe62a7fc700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe614060a70 0x7fe614062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.275+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe61c07e470 con 0x7fe6340cc470 2023-09-27T23:29:54.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.275+0000 7fe63259c700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe614060a70 0x7fe614062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.275+0000 7fe63259c700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe614060a70 0x7fe614062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe63416c550 tx=0x7fe6240091a0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.283+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe61c04e480 con 0x7fe6340cc470 2023-09-27T23:29:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.447+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6340c9040 con 0x7fe6340cc470 2023-09-27T23:29:54.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.447+0000 7fe62a7fc700 1 -- 172.21.15.64:0/1779291363 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17090 (secure 0 0 0) 0x7fe6340c9040 con 0x7fe6340cc470 2023-09-27T23:29:54.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:54.452 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":100,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:54.106595+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:52.694282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:53.074953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe614060a70 msgr2=0x7fe614062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe614060a70 0x7fe614062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe63416c550 tx=0x7fe6240091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 msgr2=0x7fe63415de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 0x7fe63415de40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe61c006fd0 tx=0x7fe61c017000 comp rx=0 tx=0).stop 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 shutdown_connections 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe614060a70 0x7fe614062f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe634158e70 0x7fe6340c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340cc470 0x7fe63415de40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 --2- 172.21.15.64:0/1779291363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe63406a540 0x7fe63415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 >> 172.21.15.64:0/1779291363 conn(0x7fe6340670e0 msgr2=0x7fe634158290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 shutdown_connections 2023-09-27T23:29:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.451+0000 7fe63359e700 1 -- 172.21.15.64:0/1779291363 wait complete. 2023-09-27T23:29:54.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:54.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.575+0000 7f2c4ed02700 1 Processor -- start 2023-09-27T23:29:54.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.575+0000 7f2c4ed02700 1 -- start start 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ed02700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c480c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ed02700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c480b0670 0x7f2c480b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ed02700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4806cf00 con 0x7f2c480b13b0 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ed02700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c480ae7e0 con 0x7f2c480b0670 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ed02700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480cbc50 con 0x7f2c48155990 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4dd00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c480c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c480b0670 0x7f2c480b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c480b0670 0x7f2c480b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:50272/0 (socket says 172.21.15.64:50272) 2023-09-27T23:29:54.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4dd00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c480c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:40262/0 (socket says 172.21.15.64:40262) 2023-09-27T23:29:54.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4e501700 1 --1- 172.21.15.64:0/2336768748 >> v1:172.21.15.123:6789/0 conn(0x7f2c480b13b0 0x7f2c4806d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:36334/0 (socket says 172.21.15.64:36334) 2023-09-27T23:29:54.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 -- 172.21.15.64:0/2336768748 learned_addr learned my addr 172.21.15.64:0/2336768748 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 -- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 msgr2=0x7f2c480c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 --2- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c480c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 -- 172.21.15.64:0/2336768748 >> v1:172.21.15.123:6789/0 conn(0x7f2c480b13b0 legacy=0x7f2c4806d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 -- 172.21.15.64:0/2336768748 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c480b01d0 con 0x7f2c480b0670 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4d4ff700 1 --2- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b0670 0x7f2c480b0e70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c380099b0 tx=0x7f2c38009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36751443a77571c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ccfe700 1 -- 172.21.15.64:0/2336768748 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38009ea0 con 0x7f2c480b0670 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ccfe700 1 -- 172.21.15.64:0/2336768748 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3800b860 con 0x7f2c480b0670 2023-09-27T23:29:54.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.579+0000 7f2c4ccfe700 1 -- 172.21.15.64:0/2336768748 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800bc00 con 0x7f2c480b0670 2023-09-27T23:29:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 -- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b0670 msgr2=0x7f2c480b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b0670 0x7f2c480b0e70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c380099b0 tx=0x7f2c38009250 comp rx=0 tx=0).stop 2023-09-27T23:29:54.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 -- 172.21.15.64:0/2336768748 shutdown_connections 2023-09-27T23:29:54.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b0670 0x7f2c480b0e70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/2336768748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c480c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 -- 172.21.15.64:0/2336768748 >> 172.21.15.64:0/2336768748 conn(0x7f2c48067bb0 msgr2=0x7f2c480add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:54.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 -- 172.21.15.64:0/2336768748 shutdown_connections 2023-09-27T23:29:54.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 -- 172.21.15.64:0/2336768748 wait complete. 2023-09-27T23:29:54.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 Processor -- start 2023-09-27T23:29:54.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.583+0000 7f2c4ed02700 1 -- start start 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 0x7f2c4814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b13b0 0x7f2c480c2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c48157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48157500 con 0x7f2c480b0670 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48150340 con 0x7f2c480b13b0 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481568f0 con 0x7f2c48155990 2023-09-27T23:29:54.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 0x7f2c4814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 0x7f2c4814fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40804/0 (socket says 172.21.15.64:40804) 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 -- 172.21.15.64:0/3137491742 learned_addr learned my addr 172.21.15.64:0/3137491742 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4d4ff700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b13b0 0x7f2c480c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4e501700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c48157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 -- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 msgr2=0x7f2c48157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c48157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 -- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b13b0 msgr2=0x7f2c480c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b13b0 0x7f2c480c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 -- 172.21.15.64:0/3137491742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380094f0 con 0x7f2c480b0670 2023-09-27T23:29:54.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4e501700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c48157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:54.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4dd00700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 0x7f2c4814fe00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e930 tx=0x7f2c440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44011790 con 0x7f2c480b0670 2023-09-27T23:29:54.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c440118f0 con 0x7f2c480b0670 2023-09-27T23:29:54.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c48156bd0 con 0x7f2c480b0670 2023-09-27T23:29:54.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44011c90 con 0x7f2c480b0670 2023-09-27T23:29:54.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c48157190 con 0x7f2c480b0670 2023-09-27T23:29:54.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.587+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4804be40 con 0x7f2c480b0670 2023-09-27T23:29:54.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.591+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2c44019440 con 0x7f2c480b0670 2023-09-27T23:29:54.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.591+0000 7f2c3dffb700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c2c060aa0 0x7f2c2c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.591+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2c4407f060 con 0x7f2c480b0670 2023-09-27T23:29:54.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.591+0000 7f2c4d4ff700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c2c060aa0 0x7f2c2c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.591+0000 7f2c4d4ff700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c2c060aa0 0x7f2c2c062f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c380099b0 tx=0x7f2c38017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.595+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2c440aebb0 con 0x7f2c480b0670 2023-09-27T23:29:54.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.759+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4806a290 con 0x7f2c480b0670 2023-09-27T23:29:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.759+0000 7f2c3dffb700 1 -- 172.21.15.64:0/3137491742 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17090 (secure 0 0 0) 0x7f2c4404eff0 con 0x7f2c480b0670 2023-09-27T23:29:54.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:54.763 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":100,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:54.106595+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:52.694282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:53.074953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c2c060aa0 msgr2=0x7f2c2c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c2c060aa0 0x7f2c2c062f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c380099b0 tx=0x7f2c38017000 comp rx=0 tx=0).stop 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 msgr2=0x7f2c4814fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 0x7f2c4814fe00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e930 tx=0x7f2c440091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 shutdown_connections 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c2c060aa0 0x7f2c2c062f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c48155990 0x7f2c48157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c480b13b0 0x7f2c480c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 --2- 172.21.15.64:0/3137491742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c480b0670 0x7f2c4814fe00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 >> 172.21.15.64:0/3137491742 conn(0x7f2c48067bb0 msgr2=0x7f2c48155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 shutdown_connections 2023-09-27T23:29:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.763+0000 7f2c4ed02700 1 -- 172.21.15.64:0/3137491742 wait complete. 2023-09-27T23:29:54.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:29:54.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 Processor -- start 2023-09-27T23:29:54.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 -- start start 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff510150200 0x7ff5100c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ae770 0x7ff5100aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100af250 con 0x7ff5100ac720 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5100af530 con 0x7ff510150200 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff514f44700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100ca8a0 con 0x7ff5100ae770 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff510150200 0x7ff5100c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff510150200 0x7ff5100c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54856/0 (socket says 172.21.15.64:54856) 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 -- 172.21.15.64:0/2104217037 learned_addr learned my addr 172.21.15.64:0/2104217037 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff506ffd700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ae770 0x7ff5100aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 -- 172.21.15.64:0/2104217037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ae770 msgr2=0x7ff5100aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ae770 0x7ff5100aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 -- 172.21.15.64:0/2104217037 >> v1:172.21.15.64:6789/0 conn(0x7ff5100ac720 legacy=0x7ff5100cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 -- 172.21.15.64:0/2104217037 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100cab80 con 0x7ff510150200 2023-09-27T23:29:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff506ffd700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ae770 0x7ff5100aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:29:54.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50f7fe700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff510150200 0x7ff5100c7170 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff50000bc50 tx=0x7ff50000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0452464f59f4842 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.891+0000 7ff50effd700 1 -- 172.21.15.64:0/2104217037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000be60 con 0x7ff510150200 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff50effd700 1 -- 172.21.15.64:0/2104217037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50000de00 con 0x7ff510150200 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff50effd700 1 -- 172.21.15.64:0/2104217037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000196b0 con 0x7ff510150200 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 -- 172.21.15.64:0/2104217037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff510150200 msgr2=0x7ff5100c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff510150200 0x7ff5100c7170 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff50000bc50 tx=0x7ff50000b4f0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 -- 172.21.15.64:0/2104217037 shutdown_connections 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ae770 0x7ff5100aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 --2- 172.21.15.64:0/2104217037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff510150200 0x7ff5100c7170 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 -- 172.21.15.64:0/2104217037 >> 172.21.15.64:0/2104217037 conn(0x7ff5100ccb30 msgr2=0x7ff5100ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 -- 172.21.15.64:0/2104217037 shutdown_connections 2023-09-27T23:29:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.895+0000 7ff514f44700 1 -- 172.21.15.64:0/2104217037 wait complete. 2023-09-27T23:29:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 Processor -- start 2023-09-27T23:29:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 -- start start 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ac720 0x7ff5100c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 0x7ff51006d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff510150200 0x7ff51006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101581e0 con 0x7ff5100ae770 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c6d90 con 0x7ff510150200 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff514f44700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c7070 con 0x7ff5100ac720 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 0x7ff51006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 0x7ff51006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54868/0 (socket says 172.21.15.64:54868) 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 -- 172.21.15.64:0/4126230703 learned_addr learned my addr 172.21.15.64:0/4126230703 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff50f7fe700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ac720 0x7ff5100c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff50ffff700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff510150200 0x7ff51006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 -- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ac720 msgr2=0x7ff5100c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ac720 0x7ff5100c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 -- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff510150200 msgr2=0x7ff51006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff510150200 0x7ff51006d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:54.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 -- 172.21.15.64:0/4126230703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50000b790 con 0x7ff5100ae770 2023-09-27T23:29:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff506ffd700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 0x7ff51006d050 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c830 tx=0x7ff4f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8011600 con 0x7ff5100ae770 2023-09-27T23:29:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.899+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f8011760 con 0x7ff5100ae770 2023-09-27T23:29:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100c7350 con 0x7ff5100ae770 2023-09-27T23:29:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8011b20 con 0x7ff5100ae770 2023-09-27T23:29:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100c4820 con 0x7ff5100ae770 2023-09-27T23:29:54.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff510158390 con 0x7ff5100ae770 2023-09-27T23:29:54.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff4f8018360 con 0x7ff5100ae770 2023-09-27T23:29:54.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff507fff700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4f0060a70 0x7ff4f0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:54.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff4f807d150 con 0x7ff5100ae770 2023-09-27T23:29:54.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.903+0000 7ff50f7fe700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4f0060a70 0x7ff4f0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:54.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.907+0000 7ff50f7fe700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4f0060a70 0x7ff4f0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff50000bc50 tx=0x7ff50001b000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:54.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:54.911+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff510158390 con 0x7ff5100ae770 2023-09-27T23:29:55.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.107+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff51004be40 con 0x7ff5100ae770 2023-09-27T23:29:55.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff507fff700 1 -- 172.21.15.64:0/4126230703 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff4f804d070 con 0x7ff5100ae770 2023-09-27T23:29:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4f0060a70 msgr2=0x7ff4f0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4f0060a70 0x7ff4f0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff50000bc50 tx=0x7ff50001b000 comp rx=0 tx=0).stop 2023-09-27T23:29:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 msgr2=0x7ff51006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 0x7ff51006d050 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c830 tx=0x7ff4f8008040 comp rx=0 tx=0).stop 2023-09-27T23:29:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 shutdown_connections 2023-09-27T23:29:55.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4f0060a70 0x7ff4f0062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff510150200 0x7ff51006d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5100ae770 0x7ff51006d050 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 --2- 172.21.15.64:0/4126230703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5100ac720 0x7ff5100c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 >> 172.21.15.64:0/4126230703 conn(0x7ff5100ccb30 msgr2=0x7ff5100b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 shutdown_connections 2023-09-27T23:29:55.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.127+0000 7ff514f44700 1 -- 172.21.15.64:0/4126230703 wait complete. 2023-09-27T23:29:55.145 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:29:55.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:29:55.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189289f700 1 Processor -- start 2023-09-27T23:29:55.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189289f700 1 -- start start 2023-09-27T23:29:55.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189289f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f188c0ce990 0x7f188c15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:55.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189289f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0cbd90 con 0x7f188c0ce5c0 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189289f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0cb180 con 0x7f188c159110 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189289f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c15da10 con 0x7f188c0ce990 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.255+0000 7f189209e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f188c0ce990 0x7f188c15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189209e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f188c0ce990 0x7f188c15d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54872/0 (socket says 172.21.15.64:54872) 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189209e700 1 -- 172.21.15.64:0/42061238 learned_addr learned my addr 172.21.15.64:0/42061238 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189189d700 1 --1- 172.21.15.64:0/42061238 >> v1:172.21.15.64:6789/0 conn(0x7f188c0ce5c0 0x7f188c158a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38590/0 (socket says 172.21.15.64:38590) 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189209e700 1 -- 172.21.15.64:0/42061238 >> v1:172.21.15.123:6789/0 conn(0x7f188c159110 legacy=0x7f188c06a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189209e700 1 -- 172.21.15.64:0/42061238 >> v1:172.21.15.64:6789/0 conn(0x7f188c0ce5c0 legacy=0x7f188c158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189209e700 1 -- 172.21.15.64:0/42061238 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0cb460 con 0x7f188c0ce990 2023-09-27T23:29:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189209e700 1 --2- 172.21.15.64:0/42061238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce990 0x7f188c15d4d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1884009f50 tx=0x7f1884009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f52af8d1dbdf240 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:55.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189089b700 1 -- 172.21.15.64:0/42061238 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400e070 con 0x7f188c0ce990 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189089b700 1 -- 172.21.15.64:0/42061238 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188400ba70 con 0x7f188c0ce990 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189089b700 1 -- 172.21.15.64:0/42061238 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18840092e0 con 0x7f188c0ce990 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 -- 172.21.15.64:0/42061238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce990 msgr2=0x7f188c15d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 --2- 172.21.15.64:0/42061238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce990 0x7f188c15d4d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1884009f50 tx=0x7f1884009840 comp rx=0 tx=0).stop 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 -- 172.21.15.64:0/42061238 shutdown_connections 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 --2- 172.21.15.64:0/42061238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce990 0x7f188c15d4d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 -- 172.21.15.64:0/42061238 >> 172.21.15.64:0/42061238 conn(0x7f188c067bb0 msgr2=0x7f188c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 -- 172.21.15.64:0/42061238 shutdown_connections 2023-09-27T23:29:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.259+0000 7f189289f700 1 -- 172.21.15.64:0/42061238 wait complete. 2023-09-27T23:29:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 Processor -- start 2023-09-27T23:29:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 -- start start 2023-09-27T23:29:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 0x7f188c157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f188c0ce990 0x7f188c1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f188c159110 0x7f188c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c157610 con 0x7f188c0ce5c0 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c6fb0 con 0x7f188c159110 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189289f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c157330 con 0x7f188c0ce990 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 0x7f188c157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 0x7f188c157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54880/0 (socket says 172.21.15.64:54880) 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 -- 172.21.15.64:0/1101412934 learned_addr learned my addr 172.21.15.64:0/1101412934 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189209e700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f188c159110 0x7f188c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189109c700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f188c0ce990 0x7f188c1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 -- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f188c0ce990 msgr2=0x7f188c1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f188c0ce990 0x7f188c1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 -- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f188c159110 msgr2=0x7f188c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f188c159110 0x7f188c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 -- 172.21.15.64:0/1101412934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884009ae0 con 0x7f188c0ce5c0 2023-09-27T23:29:55.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.263+0000 7f189189d700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 0x7f188c157e50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f188800c7c0 tx=0x7f1888007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888005a70 con 0x7f188c0ce5c0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1888005bd0 con 0x7f188c0ce5c0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c0c7250 con 0x7f188c0ce5c0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880070d0 con 0x7f188c0ce5c0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c158390 con 0x7f188c0ce5c0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c04be40 con 0x7f188c0ce5c0 2023-09-27T23:29:55.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1888007440 con 0x7f188c0ce5c0 2023-09-27T23:29:55.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f1881ffb700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1870060a70 0x7f1870062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:55.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.267+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f188807c590 con 0x7f188c0ce5c0 2023-09-27T23:29:55.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.271+0000 7f189109c700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1870060a70 0x7f1870062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:55.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.271+0000 7f189109c700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1870060a70 0x7f1870062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f188c156ca0 tx=0x7f187c007ec0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:55.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.275+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f188804c520 con 0x7f188c0ce5c0 2023-09-27T23:29:55.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:55.475+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f188c0cb740 con 0x7f188c0ce5c0 2023-09-27T23:29:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.131+0000 7f1881ffb700 1 -- 172.21.15.64:0/1101412934 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f188804fb40 con 0x7f188c0ce5c0 2023-09-27T23:29:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1870060a70 msgr2=0x7f1870062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1870060a70 0x7f1870062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f188c156ca0 tx=0x7f187c007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 msgr2=0x7f188c157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 0x7f188c157e50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f188800c7c0 tx=0x7f1888007f70 comp rx=0 tx=0).stop 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 shutdown_connections 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1870060a70 0x7f1870062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f188c159110 0x7f188c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f188c0ce990 0x7f188c1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 --2- 172.21.15.64:0/1101412934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f188c0ce5c0 0x7f188c157e50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 >> 172.21.15.64:0/1101412934 conn(0x7f188c067bb0 msgr2=0x7f188c152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 shutdown_connections 2023-09-27T23:29:56.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.135+0000 7f189289f700 1 -- 172.21.15.64:0/1101412934 wait complete. 2023-09-27T23:29:56.156 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:29:56.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:29:56.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 Processor -- start 2023-09-27T23:29:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 -- start start 2023-09-27T23:29:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc2dc0c6cd0 0x7fc2dc06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2dc15cc20 0x7fc2dc15d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc06b470 con 0x7fc2dc0c6900 2023-09-27T23:29:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc06be40 con 0x7fc2dc15cc20 2023-09-27T23:29:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e2713700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc06e110 con 0x7fc2dc0c6cd0 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1f12700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2dc15cc20 0x7fc2dc15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1711700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc2dc0c6900 0x7fc2dc06da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38592/0 (socket says 172.21.15.64:38592) 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1f12700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2dc15cc20 0x7fc2dc15d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54896/0 (socket says 172.21.15.64:54896) 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1711700 1 -- 172.21.15.64:0/2672387431 learned_addr learned my addr 172.21.15.64:0/2672387431 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e0f10700 1 --2- 172.21.15.64:0/2672387431 >> v2:172.21.15.123:3300/0 conn(0x7fc2dc0c6cd0 0x7fc2dc06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1f12700 1 -- 172.21.15.64:0/2672387431 >> v2:172.21.15.123:3300/0 conn(0x7fc2dc0c6cd0 msgr2=0x7fc2dc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1f12700 1 --2- 172.21.15.64:0/2672387431 >> v2:172.21.15.123:3300/0 conn(0x7fc2dc0c6cd0 0x7fc2dc06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1f12700 1 -- 172.21.15.64:0/2672387431 >> v1:172.21.15.64:6789/0 conn(0x7fc2dc0c6900 legacy=0x7fc2dc06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2d3fff700 1 -- 172.21.15.64:0/2672387431 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291072646 0 0) 0x7fc2dc06b470 con 0x7fc2dc0c6900 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.263+0000 7fc2e1f12700 1 -- 172.21.15.64:0/2672387431 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc161d10 con 0x7fc2dc15cc20 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e1f12700 1 --2- 172.21.15.64:0/2672387431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc15cc20 0x7fc2dc15d080 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc06a030 tx=0x7fc2d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c42c8fdb8608ab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2d3fff700 1 -- 172.21.15.64:0/2672387431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400e070 con 0x7fc2dc15cc20 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2d3fff700 1 -- 172.21.15.64:0/2672387431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d400bb60 con 0x7fc2dc15cc20 2023-09-27T23:29:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2d3fff700 1 -- 172.21.15.64:0/2672387431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4012430 con 0x7fc2dc15cc20 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 -- 172.21.15.64:0/2672387431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc15cc20 msgr2=0x7fc2dc15d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 --2- 172.21.15.64:0/2672387431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc15cc20 0x7fc2dc15d080 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc06a030 tx=0x7fc2d4009780 comp rx=0 tx=0).stop 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 -- 172.21.15.64:0/2672387431 shutdown_connections 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 --2- 172.21.15.64:0/2672387431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc15cc20 0x7fc2dc15d080 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 --2- 172.21.15.64:0/2672387431 >> v2:172.21.15.123:3300/0 conn(0x7fc2dc0c6cd0 0x7fc2dc06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 -- 172.21.15.64:0/2672387431 >> 172.21.15.64:0/2672387431 conn(0x7fc2dc067bb0 msgr2=0x7fc2dc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 -- 172.21.15.64:0/2672387431 shutdown_connections 2023-09-27T23:29:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.267+0000 7fc2e2713700 1 -- 172.21.15.64:0/2672387431 wait complete. 2023-09-27T23:29:56.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 Processor -- start 2023-09-27T23:29:56.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 -- start start 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 0x7fc2dc15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc0c6cd0 0x7fc2dc152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2dc15cc20 0x7fc2dc159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc16d5b0 con 0x7fc2dc0c6900 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0aee80 con 0x7fc2dc0c6cd0 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0af160 con 0x7fc2dc15cc20 2023-09-27T23:29:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 0x7fc2dc15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 0x7fc2dc15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54912/0 (socket says 172.21.15.64:54912) 2023-09-27T23:29:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 -- 172.21.15.64:0/3262142739 learned_addr learned my addr 172.21.15.64:0/3262142739 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e0f10700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc0c6cd0 0x7fc2dc152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1f12700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2dc15cc20 0x7fc2dc159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 -- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2dc15cc20 msgr2=0x7fc2dc159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2dc15cc20 0x7fc2dc159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 -- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc0c6cd0 msgr2=0x7fc2dc152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc0c6cd0 0x7fc2dc152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 -- 172.21.15.64:0/3262142739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4009a20 con 0x7fc2dc0c6900 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1f12700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2dc15cc20 0x7fc2dc159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e1711700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 0x7fc2dc15b1b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c7e0 tx=0x7fc2cc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc005b30 con 0x7fc2dc0c6900 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc005c90 con 0x7fc2dc0c6900 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0af440 con 0x7fc2dc0c6900 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc007300 con 0x7fc2dc0c6900 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.271+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc15b6f0 con 0x7fc2dc0c6900 2023-09-27T23:29:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.275+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc04be40 con 0x7fc2dc0c6900 2023-09-27T23:29:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.275+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc2cc0076b0 con 0x7fc2dc0c6900 2023-09-27T23:29:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.275+0000 7fc2d17fa700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2c0060a70 0x7fc2c0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.275+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc2cc07c6b0 con 0x7fc2dc0c6900 2023-09-27T23:29:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.275+0000 7fc2e0f10700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2c0060a70 0x7fc2c0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.275+0000 7fc2e0f10700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2c0060a70 0x7fc2c0062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0adcc0 tx=0x7fc2d80091a0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.287+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc2cc04c6e0 con 0x7fc2dc0c6900 2023-09-27T23:29:56.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.491+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2dc06a580 con 0x7fc2dc0c6900 2023-09-27T23:29:56.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.491+0000 7fc2d17fa700 1 -- 172.21.15.64:0/3262142739 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fc2cc04fd00 con 0x7fc2dc0c6900 2023-09-27T23:29:56.495 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:56.495 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:54.107357+0000","modified":"2023-09-27T23:29:56.124127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1623763192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1623763192},{"type":"v1","addr":"172.21.15.64:6835","nonce":1623763192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-09-27T23:29:56.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2c0060a70 msgr2=0x7fc2c0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2c0060a70 0x7fc2c0062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0adcc0 tx=0x7fc2d80091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 msgr2=0x7fc2dc15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 0x7fc2dc15b1b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c7e0 tx=0x7fc2cc007df0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 shutdown_connections 2023-09-27T23:29:56.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc2c0060a70 0x7fc2c0062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2dc15cc20 0x7fc2dc159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2dc0c6cd0 0x7fc2dc152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 --2- 172.21.15.64:0/3262142739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2dc0c6900 0x7fc2dc15b1b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 >> 172.21.15.64:0/3262142739 conn(0x7fc2dc067bb0 msgr2=0x7fc2dc0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 shutdown_connections 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.495+0000 7fc2e2713700 1 -- 172.21.15.64:0/3262142739 wait complete. 2023-09-27T23:29:56.499 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2023-09-27T23:29:56.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:29:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcf09b1700 1 Processor -- start 2023-09-27T23:29:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcf09b1700 1 -- start start 2023-09-27T23:29:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcf09b1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcec06a580 0x7fdcec06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcf09b1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec06c7c0 con 0x7fdcec0b1410 2023-09-27T23:29:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcf09b1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec06cd80 con 0x7fdcec06d210 2023-09-27T23:29:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcf09b1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec06af40 con 0x7fdcec06a580 2023-09-27T23:29:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdceb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdcec0b1410 0x7fdcec0ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38604/0 (socket says 172.21.15.64:38604) 2023-09-27T23:29:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdceb7fe700 1 -- 172.21.15.64:0/448425540 learned_addr learned my addr 172.21.15.64:0/448425540 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdceaffd700 1 --2- 172.21.15.64:0/448425540 >> v2:172.21.15.123:3300/0 conn(0x7fdcec06a580 0x7fdcec06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.615+0000 7fdcea7fc700 1 -- 172.21.15.64:0/448425540 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145530649 0 0) 0x7fdcec06c7c0 con 0x7fdcec0b1410 2023-09-27T23:29:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcea7fc700 1 -- 172.21.15.64:0/448425540 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd8003570 con 0x7fdcec0b1410 2023-09-27T23:29:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdceaffd700 1 -- 172.21.15.64:0/448425540 >> v1:172.21.15.123:6789/0 conn(0x7fdcec06d210 legacy=0x7fdcec06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdceaffd700 1 -- 172.21.15.64:0/448425540 >> v1:172.21.15.64:6789/0 conn(0x7fdcec0b1410 legacy=0x7fdcec0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcea7fc700 1 -- 172.21.15.64:0/448425540 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597995459 0 0) 0x7fdcd8003570 con 0x7fdcec0b1410 2023-09-27T23:29:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdceaffd700 1 -- 172.21.15.64:0/448425540 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec06b8f0 con 0x7fdcec06a580 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdceaffd700 1 --2- 172.21.15.64:0/448425540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06a580 0x7fdcec06aa00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009e60 tx=0x7fdcd4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ee37615f364f89 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcea7fc700 1 -- 172.21.15.64:0/448425540 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400e070 con 0x7fdcec06a580 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcea7fc700 1 -- 172.21.15.64:0/448425540 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd400bb60 con 0x7fdcec06a580 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcea7fc700 1 -- 172.21.15.64:0/448425540 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400bee0 con 0x7fdcec06a580 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 -- 172.21.15.64:0/448425540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06a580 msgr2=0x7fdcec06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/448425540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06a580 0x7fdcec06aa00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009e60 tx=0x7fdcd4009780 comp rx=0 tx=0).stop 2023-09-27T23:29:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 -- 172.21.15.64:0/448425540 shutdown_connections 2023-09-27T23:29:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/448425540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06a580 0x7fdcec06aa00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 -- 172.21.15.64:0/448425540 >> 172.21.15.64:0/448425540 conn(0x7fdcec067bb0 msgr2=0x7fdcec0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 -- 172.21.15.64:0/448425540 shutdown_connections 2023-09-27T23:29:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 -- 172.21.15.64:0/448425540 wait complete. 2023-09-27T23:29:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 Processor -- start 2023-09-27T23:29:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.619+0000 7fdcf09b1700 1 -- start start 2023-09-27T23:29:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec06a580 0x7fdcec158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06d210 0x7fdcec167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 0x7fdcec0c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0c45f0 con 0x7fdcec0b1410 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec158db0 con 0x7fdcec06d210 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0c5150 con 0x7fdcec06a580 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 0x7fdcec0c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 0x7fdcec0c2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54922/0 (socket says 172.21.15.64:54922) 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 -- 172.21.15.64:0/688318379 learned_addr learned my addr 172.21.15.64:0/688318379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdceaffd700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06d210 0x7fdcec167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdceb7fe700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec06a580 0x7fdcec158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 -- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec06a580 msgr2=0x7fdcec158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec06a580 0x7fdcec158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 -- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06d210 msgr2=0x7fdcec167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06d210 0x7fdcec167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 -- 172.21.15.64:0/688318379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009a20 con 0x7fdcec0b1410 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcebfff700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 0x7fdcec0c2440 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c560 tx=0x7fdce4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4011590 con 0x7fdcec0b1410 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce40116f0 con 0x7fdcec0b1410 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec1681e0 con 0x7fdcec0b1410 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4011ab0 con 0x7fdcec0b1410 2023-09-27T23:29:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.623+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec0c4b00 con 0x7fdcec0b1410 2023-09-27T23:29:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.627+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fdce401b3b0 con 0x7fdcec0b1410 2023-09-27T23:29:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.627+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec15c7c0 con 0x7fdcec0b1410 2023-09-27T23:29:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.627+0000 7fdce37fe700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdccc060aa0 0x7fdccc062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.627+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdce407c4a0 con 0x7fdcec0b1410 2023-09-27T23:29:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.627+0000 7fdceb7fe700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdccc060aa0 0x7fdccc062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.627+0000 7fdceb7fe700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdccc060aa0 0x7fdccc062f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007fd0 tx=0x7fdcdc007d10 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.631+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdcec15c7c0 con 0x7fdcec0b1410 2023-09-27T23:29:56.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.791+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcec15c7c0 con 0x7fdcec0b1410 2023-09-27T23:29:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.791+0000 7fdce37fe700 1 -- 172.21.15.64:0/688318379 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+17077 (secure 0 0 0) 0x7fdcec15c7c0 con 0x7fdcec0b1410 2023-09-27T23:29:56.796 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:56.796 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":101,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:29:55.111006+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:52.694282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:53.074953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdccc060aa0 msgr2=0x7fdccc062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdccc060aa0 0x7fdccc062f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007fd0 tx=0x7fdcdc007d10 comp rx=0 tx=0).stop 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 msgr2=0x7fdcec0c2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 0x7fdcec0c2440 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c560 tx=0x7fdce4007ef0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 shutdown_connections 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdccc060aa0 0x7fdccc062f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcec0b1410 0x7fdcec0c2440 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec06d210 0x7fdcec167ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 --2- 172.21.15.64:0/688318379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec06a580 0x7fdcec158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 >> 172.21.15.64:0/688318379 conn(0x7fdcec067bb0 msgr2=0x7fdcec0aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:56.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 shutdown_connections 2023-09-27T23:29:56.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.795+0000 7fdcf09b1700 1 -- 172.21.15.64:0/688318379 wait complete. 2023-09-27T23:29:56.810 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:29:56.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c6e108700 1 Processor -- start 2023-09-27T23:29:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c6e108700 1 -- start start 2023-09-27T23:29:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c6e108700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c680ae520 0x7f4c681556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c6e108700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680c1b90 con 0x7f4c6806c7c0 2023-09-27T23:29:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c6e108700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6806a9d0 con 0x7f4c681552f0 2023-09-27T23:29:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c6e108700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680c8710 con 0x7f4c680ae520 2023-09-27T23:29:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c680ae520 0x7f4c681556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c680ae520 0x7f4c681556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54936/0 (socket says 172.21.15.64:54936) 2023-09-27T23:29:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 -- 172.21.15.64:0/1556699552 learned_addr learned my addr 172.21.15.64:0/1556699552 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c67fff700 1 --1- 172.21.15.64:0/1556699552 >> v1:172.21.15.64:6789/0 conn(0x7f4c6806c7c0 0x7f4c6806cbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38618/0 (socket says 172.21.15.64:38618) 2023-09-27T23:29:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 -- 172.21.15.64:0/1556699552 >> v1:172.21.15.123:6789/0 conn(0x7f4c681552f0 legacy=0x7f4c680c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 -- 172.21.15.64:0/1556699552 >> v1:172.21.15.64:6789/0 conn(0x7f4c6806c7c0 legacy=0x7f4c6806cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 -- 172.21.15.64:0/1556699552 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6806acb0 con 0x7f4c680ae520 2023-09-27T23:29:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c66ffd700 1 --2- 172.21.15.64:0/1556699552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c680ae520 0x7f4c681556c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d1a0 tx=0x7f4c5c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8538e1229da80fab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c667fc700 1 -- 172.21.15.64:0/1556699552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c010070 con 0x7f4c680ae520 2023-09-27T23:29:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c667fc700 1 -- 172.21.15.64:0/1556699552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c00b0d0 con 0x7f4c680ae520 2023-09-27T23:29:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.915+0000 7f4c667fc700 1 -- 172.21.15.64:0/1556699552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00b490 con 0x7f4c680ae520 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- 172.21.15.64:0/1556699552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c680ae520 msgr2=0x7f4c681556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 --2- 172.21.15.64:0/1556699552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c680ae520 0x7f4c681556c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d1a0 tx=0x7f4c5c00ba50 comp rx=0 tx=0).stop 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- 172.21.15.64:0/1556699552 shutdown_connections 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 --2- 172.21.15.64:0/1556699552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c680ae520 0x7f4c681556c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- 172.21.15.64:0/1556699552 >> 172.21.15.64:0/1556699552 conn(0x7f4c680c8850 msgr2=0x7f4c680c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- 172.21.15.64:0/1556699552 shutdown_connections 2023-09-27T23:29:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- 172.21.15.64:0/1556699552 wait complete. 2023-09-27T23:29:56.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 Processor -- start 2023-09-27T23:29:56.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- start start 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4c6806c7c0 0x7f4c68169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c680ae520 0x7f4c68166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 0x7f4c68158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68156320 con 0x7f4c681552f0 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681565f0 con 0x7f4c680ae520 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c6e108700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681568d0 con 0x7f4c6806c7c0 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 0x7f4c68158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c66ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c680ae520 0x7f4c68166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 0x7f4c68158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54938/0 (socket says 172.21.15.64:54938) 2023-09-27T23:29:56.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c67fff700 1 -- 172.21.15.64:0/3724310013 learned_addr learned my addr 172.21.15.64:0/3724310013 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:56.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.919+0000 7f4c677fe700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4c6806c7c0 0x7f4c68169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c67fff700 1 -- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4c6806c7c0 msgr2=0x7f4c68169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c67fff700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4c6806c7c0 0x7f4c68169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c67fff700 1 -- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c680ae520 msgr2=0x7f4c68166a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c67fff700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c680ae520 0x7f4c68166a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c67fff700 1 -- 172.21.15.64:0/3724310013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c00bcf0 con 0x7f4c681552f0 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c66ffd700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c680ae520 0x7f4c68166a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c677fe700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4c6806c7c0 0x7f4c68169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:56.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c67fff700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 0x7f4c68158240 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c4e0 tx=0x7f4c60007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000ce90 con 0x7f4c681552f0 2023-09-27T23:29:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c60005a70 con 0x7f4c681552f0 2023-09-27T23:29:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c68156bb0 con 0x7f4c681552f0 2023-09-27T23:29:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60005e10 con 0x7f4c681552f0 2023-09-27T23:29:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680adc10 con 0x7f4c681552f0 2023-09-27T23:29:56.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.923+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68071120 con 0x7f4c681552f0 2023-09-27T23:29:56.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.927+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4c600072e0 con 0x7f4c681552f0 2023-09-27T23:29:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.927+0000 7f4c4f7fe700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4c54060a70 0x7f4c54062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.927+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4c6007c4d0 con 0x7f4c681552f0 2023-09-27T23:29:56.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.927+0000 7f4c677fe700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4c54060a70 0x7f4c54062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:56.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.927+0000 7f4c677fe700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4c54060a70 0x7f4c54062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c68157f60 tx=0x7f4c58009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:56.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:56.935+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4c6004c460 con 0x7f4c681552f0 2023-09-27T23:29:57.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.091+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c680642a0 con 0x7f4c54060a70 2023-09-27T23:29:57.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.111+0000 7f4c4f7fe700 1 -- 172.21.15.64:0/3724310013 <== mgr.6209 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294571 (secure 0 0 0) 0x7f4c680642a0 con 0x7f4c54060a70 2023-09-27T23:29:57.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4c54060a70 msgr2=0x7f4c54062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4c54060a70 0x7f4c54062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c68157f60 tx=0x7f4c58009040 comp rx=0 tx=0).stop 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 msgr2=0x7f4c68158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 0x7f4c68158240 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c4e0 tx=0x7f4c60007f50 comp rx=0 tx=0).stop 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 shutdown_connections 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4c54060a70 0x7f4c54062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c681552f0 0x7f4c68158240 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c680ae520 0x7f4c68166a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 --2- 172.21.15.64:0/3724310013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4c6806c7c0 0x7f4c68169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 >> 172.21.15.64:0/3724310013 conn(0x7f4c680c8850 msgr2=0x7f4c680c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 shutdown_connections 2023-09-27T23:29:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.115+0000 7f4c6e108700 1 -- 172.21.15.64:0/3724310013 wait complete. 2023-09-27T23:29:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:57.134 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:56.999892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":40,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":207372,"kb_used_data":10380,"kb_used_omap":0,"kb_used_meta":196928,"kb_avail":754767348,"statfs":{"total":773094113280,"available":772881764352,"internally_reserved":0,"allocated":10629120,"data_stored":6016433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":201654272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.279463"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.148704+0000","last_change":"2023-09-27T23:29:54.145945+0000","last_active":"2023-09-27T23:29:54.148704+0000","last_peered":"2023-09-27T23:29:54.148704+0000","last_clean":"2023-09-27T23:29:54.148704+0000","last_became_active":"2023-09-27T23:29:54.145660+0000","last_became_peered":"2023-09-27T23:29:54.145660+0000","last_unstale":"2023-09-27T23:29:54.148704+0000","last_undegraded":"2023-09-27T23:29:54.148704+0000","last_fullsized":"2023-09-27T23:29:54.148704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:47.159474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145554+0000","last_change":"2023-09-27T23:29:54.139474+0000","last_active":"2023-09-27T23:29:54.145554+0000","last_peered":"2023-09-27T23:29:54.145554+0000","last_clean":"2023-09-27T23:29:54.145554+0000","last_became_active":"2023-09-27T23:29:54.139204+0000","last_became_peered":"2023-09-27T23:29:54.139204+0000","last_unstale":"2023-09-27T23:29:54.145554+0000","last_undegraded":"2023-09-27T23:29:54.145554+0000","last_fullsized":"2023-09-27T23:29:54.145554+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:09:54.976064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149543+0000","last_change":"2023-09-27T23:29:54.147371+0000","last_active":"2023-09-27T23:29:54.149543+0000","last_peered":"2023-09-27T23:29:54.149543+0000","last_clean":"2023-09-27T23:29:54.149543+0000","last_became_active":"2023-09-27T23:29:54.146786+0000","last_became_peered":"2023-09-27T23:29:54.146786+0000","last_unstale":"2023-09-27T23:29:54.149543+0000","last_undegraded":"2023-09-27T23:29:54.149543+0000","last_fullsized":"2023-09-27T23:29:54.149543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:34:51.507706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153647+0000","last_change":"2023-09-27T23:29:54.151837+0000","last_active":"2023-09-27T23:29:54.153647+0000","last_peered":"2023-09-27T23:29:54.153647+0000","last_clean":"2023-09-27T23:29:54.153647+0000","last_became_active":"2023-09-27T23:29:54.151587+0000","last_became_peered":"2023-09-27T23:29:54.151587+0000","last_unstale":"2023-09-27T23:29:54.153647+0000","last_undegraded":"2023-09-27T23:29:54.153647+0000","last_fullsized":"2023-09-27T23:29:54.153647+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:28.711624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146058+0000","last_change":"2023-09-27T23:29:54.137918+0000","last_active":"2023-09-27T23:29:54.146058+0000","last_peered":"2023-09-27T23:29:54.146058+0000","last_clean":"2023-09-27T23:29:54.146058+0000","last_became_active":"2023-09-27T23:29:54.137403+0000","last_became_peered":"2023-09-27T23:29:54.137403+0000","last_unstale":"2023-09-27T23:29:54.146058+0000","last_undegraded":"2023-09-27T23:29:54.146058+0000","last_fullsized":"2023-09-27T23:29:54.146058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:19:59.273479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145734+0000","last_change":"2023-09-27T23:29:54.141094+0000","last_active":"2023-09-27T23:29:54.145734+0000","last_peered":"2023-09-27T23:29:54.145734+0000","last_clean":"2023-09-27T23:29:54.145734+0000","last_became_active":"2023-09-27T23:29:54.140830+0000","last_became_peered":"2023-09-27T23:29:54.140830+0000","last_unstale":"2023-09-27T23:29:54.145734+0000","last_undegraded":"2023-09-27T23:29:54.145734+0000","last_fullsized":"2023-09-27T23:29:54.145734+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:48.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.147079+0000","last_change":"2023-09-27T23:29:54.144027+0000","last_active":"2023-09-27T23:29:54.147079+0000","last_peered":"2023-09-27T23:29:54.147079+0000","last_clean":"2023-09-27T23:29:54.147079+0000","last_became_active":"2023-09-27T23:29:54.143684+0000","last_became_peered":"2023-09-27T23:29:54.143684+0000","last_unstale":"2023-09-27T23:29:54.147079+0000","last_undegraded":"2023-09-27T23:29:54.147079+0000","last_fullsized":"2023-09-27T23:29:54.147079+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:22:41.890804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"89'183","reported_seq":466,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610106+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:49.610106+0000","last_peered":"2023-09-27T23:29:49.610106+0000","last_clean":"2023-09-27T23:29:49.610106+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:49.610106+0000","last_undegraded":"2023-09-27T23:29:49.610106+0000","last_fullsized":"2023-09-27T23:29:49.610106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":183,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863081+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:49.863081+0000","last_peered":"2023-09-27T23:29:49.863081+0000","last_clean":"2023-09-27T23:29:49.863081+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:49.863081+0000","last_undegraded":"2023-09-27T23:29:49.863081+0000","last_fullsized":"2023-09-27T23:29:49.863081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610203+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:49.610203+0000","last_peered":"2023-09-27T23:29:49.610203+0000","last_clean":"2023-09-27T23:29:49.610203+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:49.610203+0000","last_undegraded":"2023-09-27T23:29:49.610203+0000","last_fullsized":"2023-09-27T23:29:49.610203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117910+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:54.117910+0000","last_peered":"2023-09-27T23:29:54.117910+0000","last_clean":"2023-09-27T23:29:54.117910+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:54.117910+0000","last_undegraded":"2023-09-27T23:29:54.117910+0000","last_fullsized":"2023-09-27T23:29:54.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863080+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:49.863080+0000","last_peered":"2023-09-27T23:29:49.863080+0000","last_clean":"2023-09-27T23:29:49.863080+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:49.863080+0000","last_undegraded":"2023-09-27T23:29:49.863080+0000","last_fullsized":"2023-09-27T23:29:49.863080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.118980+0000","last_change":"2023-09-27T23:29:53.753232+0000","last_active":"2023-09-27T23:29:54.118980+0000","last_peered":"2023-09-27T23:29:54.118980+0000","last_clean":"2023-09-27T23:29:54.118980+0000","last_became_active":"2023-09-27T23:29:53.752871+0000","last_became_peered":"2023-09-27T23:29:53.752871+0000","last_unstale":"2023-09-27T23:29:54.118980+0000","last_undegraded":"2023-09-27T23:29:54.118980+0000","last_fullsized":"2023-09-27T23:29:54.118980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:12:40.768556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575761+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:49.575761+0000","last_peered":"2023-09-27T23:29:49.575761+0000","last_clean":"2023-09-27T23:29:49.575761+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:49.575761+0000","last_undegraded":"2023-09-27T23:29:49.575761+0000","last_fullsized":"2023-09-27T23:29:49.575761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149523+0000","last_change":"2023-09-27T23:29:54.147151+0000","last_active":"2023-09-27T23:29:54.149523+0000","last_peered":"2023-09-27T23:29:54.149523+0000","last_clean":"2023-09-27T23:29:54.149523+0000","last_became_active":"2023-09-27T23:29:54.146502+0000","last_became_peered":"2023-09-27T23:29:54.146502+0000","last_unstale":"2023-09-27T23:29:54.149523+0000","last_undegraded":"2023-09-27T23:29:54.149523+0000","last_fullsized":"2023-09-27T23:29:54.149523+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:22.885061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610219+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:49.610219+0000","last_peered":"2023-09-27T23:29:49.610219+0000","last_clean":"2023-09-27T23:29:49.610219+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:49.610219+0000","last_undegraded":"2023-09-27T23:29:49.610219+0000","last_fullsized":"2023-09-27T23:29:49.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575709+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:49.575709+0000","last_peered":"2023-09-27T23:29:49.575709+0000","last_clean":"2023-09-27T23:29:49.575709+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:49.575709+0000","last_undegraded":"2023-09-27T23:29:49.575709+0000","last_fullsized":"2023-09-27T23:29:49.575709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138284+0000","last_change":"2023-09-27T23:29:54.131847+0000","last_active":"2023-09-27T23:29:54.138284+0000","last_peered":"2023-09-27T23:29:54.138284+0000","last_clean":"2023-09-27T23:29:54.138284+0000","last_became_active":"2023-09-27T23:29:54.131553+0000","last_became_peered":"2023-09-27T23:29:54.131553+0000","last_unstale":"2023-09-27T23:29:54.138284+0000","last_undegraded":"2023-09-27T23:29:54.138284+0000","last_fullsized":"2023-09-27T23:29:54.138284+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:07.706937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117722+0000","last_change":"2023-09-27T23:29:53.754772+0000","last_active":"2023-09-27T23:29:54.117722+0000","last_peered":"2023-09-27T23:29:54.117722+0000","last_clean":"2023-09-27T23:29:54.117722+0000","last_became_active":"2023-09-27T23:29:53.754460+0000","last_became_peered":"2023-09-27T23:29:53.754460+0000","last_unstale":"2023-09-27T23:29:54.117722+0000","last_undegraded":"2023-09-27T23:29:54.117722+0000","last_fullsized":"2023-09-27T23:29:54.117722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:18.033695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.612127+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:49.612127+0000","last_peered":"2023-09-27T23:29:49.612127+0000","last_clean":"2023-09-27T23:29:49.612127+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:49.612127+0000","last_undegraded":"2023-09-27T23:29:49.612127+0000","last_fullsized":"2023-09-27T23:29:49.612127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25984,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":94345856,"statfs":{"total":96636764160,"available":96610156544,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":1.093}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.02}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.98199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.93100000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.149},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21564,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350276,"statfs":{"total":96636764160,"available":96614682624,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.073},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":1.2090000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.014},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":0.878},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.167}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.115}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.754},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.268},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2390000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":34624,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35454976},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.1899999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":1.379},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":1.22}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":1.3180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.071},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.3520000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.2849999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21832,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":20672,"kb_avail":94350008,"statfs":{"total":96636764160,"available":96614408192,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21168128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.78300000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":1.4850000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.609},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":0.89700000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.282}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.7929999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22280,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349560,"statfs":{"total":96636764160,"available":96613949440,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.63700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.605}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.3779999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.556}]}]},{"osd":2,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21116,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94350724,"statfs":{"total":96636764160,"available":96615141376,"internally_reserved":0,"allocated":1232896,"data_stored":632438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.78700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":1.091},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.97999999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.004}]}]},{"osd":1,"up_from":14,"seq":60129542342,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36028,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.752},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":1.0029999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":1.044},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":1.105},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":1.204},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":1.256}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.2310000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22784,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94349056,"statfs":{"total":96636764160,"available":96613433344,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.82599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.96899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.89600000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.056},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.3129999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":1.252}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:57.140 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:57.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb9900b2700 1 Processor -- start 2023-09-27T23:29:57.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb9900b2700 1 -- start start 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb9900b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb988155ed0 0x7fb98806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb9900b2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880ca3d0 con 0x7fb9880c87b0 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb9900b2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98806e4c0 con 0x7fb988155b00 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb9900b2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98806e7a0 con 0x7fb988155ed0 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98e64f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb988155ed0 0x7fb98806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98d64d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb9880c87b0 0x7fb9880ca8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38634/0 (socket says 172.21.15.64:38634) 2023-09-27T23:29:57.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98d64d700 1 -- 172.21.15.64:0/1385373257 learned_addr learned my addr 172.21.15.64:0/1385373257 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98e64f700 1 -- 172.21.15.64:0/1385373257 >> v1:172.21.15.123:6789/0 conn(0x7fb988155b00 legacy=0x7fb9880c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98e64f700 1 -- 172.21.15.64:0/1385373257 >> v1:172.21.15.64:6789/0 conn(0x7fb9880c87b0 legacy=0x7fb9880ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98ce4c700 1 -- 172.21.15.64:0/1385373257 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482254807 0 0) 0x7fb9880ca3d0 con 0x7fb9880c87b0 2023-09-27T23:29:57.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.243+0000 7fb98e64f700 1 -- 172.21.15.64:0/1385373257 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98806e0e0 con 0x7fb988155ed0 2023-09-27T23:29:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb98e64f700 1 --2- 172.21.15.64:0/1385373257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb98806c7c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb980009e60 tx=0x7fb980009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882ad37546985609 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb98ce4c700 1 -- 172.21.15.64:0/1385373257 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000e070 con 0x7fb988155ed0 2023-09-27T23:29:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb98ce4c700 1 -- 172.21.15.64:0/1385373257 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98000bc30 con 0x7fb988155ed0 2023-09-27T23:29:57.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb98ce4c700 1 -- 172.21.15.64:0/1385373257 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980012430 con 0x7fb988155ed0 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 -- 172.21.15.64:0/1385373257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 msgr2=0x7fb98806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 --2- 172.21.15.64:0/1385373257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb98806c7c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb980009e60 tx=0x7fb980009780 comp rx=0 tx=0).stop 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 -- 172.21.15.64:0/1385373257 shutdown_connections 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 --2- 172.21.15.64:0/1385373257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb98806c7c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 -- 172.21.15.64:0/1385373257 >> 172.21.15.64:0/1385373257 conn(0x7fb9880c90f0 msgr2=0x7fb9880c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 -- 172.21.15.64:0/1385373257 shutdown_connections 2023-09-27T23:29:57.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.247+0000 7fb9900b2700 1 -- 172.21.15.64:0/1385373257 wait complete. 2023-09-27T23:29:57.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 Processor -- start 2023-09-27T23:29:57.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 -- start start 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9880c87b0 0x7fb988156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb988155b00 0x7fb9880c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb9880c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880c5e80 con 0x7fb988155ed0 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988158990 con 0x7fb9880c87b0 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb9900b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880c5270 con 0x7fb988155b00 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb9880c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb9880c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54964/0 (socket says 172.21.15.64:54964) 2023-09-27T23:29:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 -- 172.21.15.64:0/2408798035 learned_addr learned my addr 172.21.15.64:0/2408798035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98de4e700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9880c87b0 0x7fb988156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98d64d700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb988155b00 0x7fb9880c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 -- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb988155b00 msgr2=0x7fb9880c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb988155b00 0x7fb9880c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 -- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9880c87b0 msgr2=0x7fb988156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9880c87b0 0x7fb988156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 -- 172.21.15.64:0/2408798035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb980009a20 con 0x7fb988155ed0 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98d64d700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb988155b00 0x7fb9880c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb98e64f700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb9880c4c40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb988067500 tx=0x7fb980019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.251+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980012e20 con 0x7fb988155ed0 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98001b630 con 0x7fb988155ed0 2023-09-27T23:29:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9880c5550 con 0x7fb988155ed0 2023-09-27T23:29:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98001b9d0 con 0x7fb988155ed0 2023-09-27T23:29:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9880c5b10 con 0x7fb988155ed0 2023-09-27T23:29:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9880c6030 con 0x7fb988155ed0 2023-09-27T23:29:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb98001c400 con 0x7fb988155ed0 2023-09-27T23:29:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb979ffb700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb96c060a70 0x7fb96c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb98007f260 con 0x7fb988155ed0 2023-09-27T23:29:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb98de4e700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb96c060a70 0x7fb96c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.255+0000 7fb98de4e700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb96c060a70 0x7fb96c062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb984003a80 tx=0x7fb984007ec0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.263+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb98004f0e0 con 0x7fb988155ed0 2023-09-27T23:29:57.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.423+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb988071120 con 0x7fb96c060a70 2023-09-27T23:29:57.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.443+0000 7fb979ffb700 1 -- 172.21.15.64:0/2408798035 <== mgr.6209 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294571 (secure 0 0 0) 0x7fb988071120 con 0x7fb96c060a70 2023-09-27T23:29:57.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb96c060a70 msgr2=0x7fb96c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb96c060a70 0x7fb96c062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb984003a80 tx=0x7fb984007ec0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 msgr2=0x7fb9880c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb9880c4c40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb988067500 tx=0x7fb980019000 comp rx=0 tx=0).stop 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 shutdown_connections 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb96c060a70 0x7fb96c062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb988155ed0 0x7fb9880c4c40 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb988155b00 0x7fb9880c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 --2- 172.21.15.64:0/2408798035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9880c87b0 0x7fb988156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 >> 172.21.15.64:0/2408798035 conn(0x7fb9880c90f0 msgr2=0x7fb9880c1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 shutdown_connections 2023-09-27T23:29:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.451+0000 7fb9900b2700 1 -- 172.21.15.64:0/2408798035 wait complete. 2023-09-27T23:29:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:57.469 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:56.999892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":40,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":207372,"kb_used_data":10380,"kb_used_omap":0,"kb_used_meta":196928,"kb_avail":754767348,"statfs":{"total":773094113280,"available":772881764352,"internally_reserved":0,"allocated":10629120,"data_stored":6016433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":201654272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.279463"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.148704+0000","last_change":"2023-09-27T23:29:54.145945+0000","last_active":"2023-09-27T23:29:54.148704+0000","last_peered":"2023-09-27T23:29:54.148704+0000","last_clean":"2023-09-27T23:29:54.148704+0000","last_became_active":"2023-09-27T23:29:54.145660+0000","last_became_peered":"2023-09-27T23:29:54.145660+0000","last_unstale":"2023-09-27T23:29:54.148704+0000","last_undegraded":"2023-09-27T23:29:54.148704+0000","last_fullsized":"2023-09-27T23:29:54.148704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:47.159474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145554+0000","last_change":"2023-09-27T23:29:54.139474+0000","last_active":"2023-09-27T23:29:54.145554+0000","last_peered":"2023-09-27T23:29:54.145554+0000","last_clean":"2023-09-27T23:29:54.145554+0000","last_became_active":"2023-09-27T23:29:54.139204+0000","last_became_peered":"2023-09-27T23:29:54.139204+0000","last_unstale":"2023-09-27T23:29:54.145554+0000","last_undegraded":"2023-09-27T23:29:54.145554+0000","last_fullsized":"2023-09-27T23:29:54.145554+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:09:54.976064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149543+0000","last_change":"2023-09-27T23:29:54.147371+0000","last_active":"2023-09-27T23:29:54.149543+0000","last_peered":"2023-09-27T23:29:54.149543+0000","last_clean":"2023-09-27T23:29:54.149543+0000","last_became_active":"2023-09-27T23:29:54.146786+0000","last_became_peered":"2023-09-27T23:29:54.146786+0000","last_unstale":"2023-09-27T23:29:54.149543+0000","last_undegraded":"2023-09-27T23:29:54.149543+0000","last_fullsized":"2023-09-27T23:29:54.149543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:34:51.507706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153647+0000","last_change":"2023-09-27T23:29:54.151837+0000","last_active":"2023-09-27T23:29:54.153647+0000","last_peered":"2023-09-27T23:29:54.153647+0000","last_clean":"2023-09-27T23:29:54.153647+0000","last_became_active":"2023-09-27T23:29:54.151587+0000","last_became_peered":"2023-09-27T23:29:54.151587+0000","last_unstale":"2023-09-27T23:29:54.153647+0000","last_undegraded":"2023-09-27T23:29:54.153647+0000","last_fullsized":"2023-09-27T23:29:54.153647+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:28.711624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146058+0000","last_change":"2023-09-27T23:29:54.137918+0000","last_active":"2023-09-27T23:29:54.146058+0000","last_peered":"2023-09-27T23:29:54.146058+0000","last_clean":"2023-09-27T23:29:54.146058+0000","last_became_active":"2023-09-27T23:29:54.137403+0000","last_became_peered":"2023-09-27T23:29:54.137403+0000","last_unstale":"2023-09-27T23:29:54.146058+0000","last_undegraded":"2023-09-27T23:29:54.146058+0000","last_fullsized":"2023-09-27T23:29:54.146058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:19:59.273479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145734+0000","last_change":"2023-09-27T23:29:54.141094+0000","last_active":"2023-09-27T23:29:54.145734+0000","last_peered":"2023-09-27T23:29:54.145734+0000","last_clean":"2023-09-27T23:29:54.145734+0000","last_became_active":"2023-09-27T23:29:54.140830+0000","last_became_peered":"2023-09-27T23:29:54.140830+0000","last_unstale":"2023-09-27T23:29:54.145734+0000","last_undegraded":"2023-09-27T23:29:54.145734+0000","last_fullsized":"2023-09-27T23:29:54.145734+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:48.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.147079+0000","last_change":"2023-09-27T23:29:54.144027+0000","last_active":"2023-09-27T23:29:54.147079+0000","last_peered":"2023-09-27T23:29:54.147079+0000","last_clean":"2023-09-27T23:29:54.147079+0000","last_became_active":"2023-09-27T23:29:54.143684+0000","last_became_peered":"2023-09-27T23:29:54.143684+0000","last_unstale":"2023-09-27T23:29:54.147079+0000","last_undegraded":"2023-09-27T23:29:54.147079+0000","last_fullsized":"2023-09-27T23:29:54.147079+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:22:41.890804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"89'183","reported_seq":466,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610106+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:49.610106+0000","last_peered":"2023-09-27T23:29:49.610106+0000","last_clean":"2023-09-27T23:29:49.610106+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:49.610106+0000","last_undegraded":"2023-09-27T23:29:49.610106+0000","last_fullsized":"2023-09-27T23:29:49.610106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":183,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863081+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:49.863081+0000","last_peered":"2023-09-27T23:29:49.863081+0000","last_clean":"2023-09-27T23:29:49.863081+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:49.863081+0000","last_undegraded":"2023-09-27T23:29:49.863081+0000","last_fullsized":"2023-09-27T23:29:49.863081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610203+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:49.610203+0000","last_peered":"2023-09-27T23:29:49.610203+0000","last_clean":"2023-09-27T23:29:49.610203+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:49.610203+0000","last_undegraded":"2023-09-27T23:29:49.610203+0000","last_fullsized":"2023-09-27T23:29:49.610203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117910+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:54.117910+0000","last_peered":"2023-09-27T23:29:54.117910+0000","last_clean":"2023-09-27T23:29:54.117910+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:54.117910+0000","last_undegraded":"2023-09-27T23:29:54.117910+0000","last_fullsized":"2023-09-27T23:29:54.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863080+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:49.863080+0000","last_peered":"2023-09-27T23:29:49.863080+0000","last_clean":"2023-09-27T23:29:49.863080+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:49.863080+0000","last_undegraded":"2023-09-27T23:29:49.863080+0000","last_fullsized":"2023-09-27T23:29:49.863080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.118980+0000","last_change":"2023-09-27T23:29:53.753232+0000","last_active":"2023-09-27T23:29:54.118980+0000","last_peered":"2023-09-27T23:29:54.118980+0000","last_clean":"2023-09-27T23:29:54.118980+0000","last_became_active":"2023-09-27T23:29:53.752871+0000","last_became_peered":"2023-09-27T23:29:53.752871+0000","last_unstale":"2023-09-27T23:29:54.118980+0000","last_undegraded":"2023-09-27T23:29:54.118980+0000","last_fullsized":"2023-09-27T23:29:54.118980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:12:40.768556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575761+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:49.575761+0000","last_peered":"2023-09-27T23:29:49.575761+0000","last_clean":"2023-09-27T23:29:49.575761+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:49.575761+0000","last_undegraded":"2023-09-27T23:29:49.575761+0000","last_fullsized":"2023-09-27T23:29:49.575761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149523+0000","last_change":"2023-09-27T23:29:54.147151+0000","last_active":"2023-09-27T23:29:54.149523+0000","last_peered":"2023-09-27T23:29:54.149523+0000","last_clean":"2023-09-27T23:29:54.149523+0000","last_became_active":"2023-09-27T23:29:54.146502+0000","last_became_peered":"2023-09-27T23:29:54.146502+0000","last_unstale":"2023-09-27T23:29:54.149523+0000","last_undegraded":"2023-09-27T23:29:54.149523+0000","last_fullsized":"2023-09-27T23:29:54.149523+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:22.885061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610219+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:49.610219+0000","last_peered":"2023-09-27T23:29:49.610219+0000","last_clean":"2023-09-27T23:29:49.610219+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:49.610219+0000","last_undegraded":"2023-09-27T23:29:49.610219+0000","last_fullsized":"2023-09-27T23:29:49.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575709+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:49.575709+0000","last_peered":"2023-09-27T23:29:49.575709+0000","last_clean":"2023-09-27T23:29:49.575709+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:49.575709+0000","last_undegraded":"2023-09-27T23:29:49.575709+0000","last_fullsized":"2023-09-27T23:29:49.575709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138284+0000","last_change":"2023-09-27T23:29:54.131847+0000","last_active":"2023-09-27T23:29:54.138284+0000","last_peered":"2023-09-27T23:29:54.138284+0000","last_clean":"2023-09-27T23:29:54.138284+0000","last_became_active":"2023-09-27T23:29:54.131553+0000","last_became_peered":"2023-09-27T23:29:54.131553+0000","last_unstale":"2023-09-27T23:29:54.138284+0000","last_undegraded":"2023-09-27T23:29:54.138284+0000","last_fullsized":"2023-09-27T23:29:54.138284+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:07.706937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117722+0000","last_change":"2023-09-27T23:29:53.754772+0000","last_active":"2023-09-27T23:29:54.117722+0000","last_peered":"2023-09-27T23:29:54.117722+0000","last_clean":"2023-09-27T23:29:54.117722+0000","last_became_active":"2023-09-27T23:29:53.754460+0000","last_became_peered":"2023-09-27T23:29:53.754460+0000","last_unstale":"2023-09-27T23:29:54.117722+0000","last_undegraded":"2023-09-27T23:29:54.117722+0000","last_fullsized":"2023-09-27T23:29:54.117722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:18.033695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.612127+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:49.612127+0000","last_peered":"2023-09-27T23:29:49.612127+0000","last_clean":"2023-09-27T23:29:49.612127+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:49.612127+0000","last_undegraded":"2023-09-27T23:29:49.612127+0000","last_fullsized":"2023-09-27T23:29:49.612127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25984,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":94345856,"statfs":{"total":96636764160,"available":96610156544,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":1.093}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.02}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.98199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.93100000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.149},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21564,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350276,"statfs":{"total":96636764160,"available":96614682624,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.073},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":1.2090000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.014},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":0.878},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.167}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.115}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.754},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.268},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2390000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":34624,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35454976},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.1899999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":1.379},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":1.22}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":1.3180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.071},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.3520000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.2849999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21832,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":20672,"kb_avail":94350008,"statfs":{"total":96636764160,"available":96614408192,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21168128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.78300000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":1.4850000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.609},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":0.89700000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.282}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.7929999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22280,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349560,"statfs":{"total":96636764160,"available":96613949440,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.63700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.605}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.3779999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.556}]}]},{"osd":2,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21116,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94350724,"statfs":{"total":96636764160,"available":96615141376,"internally_reserved":0,"allocated":1232896,"data_stored":632438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.78700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":1.091},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.97999999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.004}]}]},{"osd":1,"up_from":14,"seq":60129542342,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36028,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.752},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":1.0029999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":1.044},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":1.105},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":1.204},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":1.256}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.2310000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22784,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94349056,"statfs":{"total":96636764160,"available":96613433344,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.82599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.96899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.89600000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.056},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.3129999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":1.252}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:57.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.623+0000 7f5aa5568700 1 Processor -- start 2023-09-27T23:29:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.623+0000 7f5aa5568700 1 -- start start 2023-09-27T23:29:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5aa5568700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5aa00b0810 0x7f5aa00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5aa5568700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa00af530 con 0x7f5aa00af9c0 2023-09-27T23:29:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5aa5568700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa0159660 con 0x7f5aa006be10 2023-09-27T23:29:57.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5aa5568700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa0159470 con 0x7f5aa00b0810 2023-09-27T23:29:57.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5aa4d67700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5aa00af9c0 0x7f5aa0154b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38646/0 (socket says 172.21.15.64:38646) 2023-09-27T23:29:57.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5aa4d67700 1 -- 172.21.15.64:0/2215833641 learned_addr learned my addr 172.21.15.64:0/2215833641 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:57.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9ffff700 1 --2- 172.21.15.64:0/2215833641 >> v2:172.21.15.123:3300/0 conn(0x7f5aa00b0810 0x7f5aa00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/2215833641 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114368273 0 0) 0x7f5aa00af530 con 0x7f5aa00af9c0 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/2215833641 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a8c003570 con 0x7f5aa00af9c0 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9ffff700 1 -- 172.21.15.64:0/2215833641 >> v1:172.21.15.123:6789/0 conn(0x7f5aa006be10 legacy=0x7f5aa0158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9ffff700 1 -- 172.21.15.64:0/2215833641 >> v1:172.21.15.64:6789/0 conn(0x7f5aa00af9c0 legacy=0x7f5aa0154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9ffff700 1 -- 172.21.15.64:0/2215833641 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0159940 con 0x7f5aa00b0810 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9ffff700 1 --2- 172.21.15.64:0/2215833641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa00b0be0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5a900098e0 tx=0x7f5a900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44287baceafe51df server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/2215833641 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000b6d0 con 0x7f5aa00b0810 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/2215833641 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000b830 con 0x7f5aa00b0810 2023-09-27T23:29:57.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.627+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/2215833641 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000bbd0 con 0x7f5aa00b0810 2023-09-27T23:29:57.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 -- 172.21.15.64:0/2215833641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 msgr2=0x7f5aa00b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 --2- 172.21.15.64:0/2215833641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa00b0be0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5a900098e0 tx=0x7f5a900091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 -- 172.21.15.64:0/2215833641 shutdown_connections 2023-09-27T23:29:57.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 --2- 172.21.15.64:0/2215833641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa00b0be0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 -- 172.21.15.64:0/2215833641 >> 172.21.15.64:0/2215833641 conn(0x7f5aa0067bb0 msgr2=0x7f5aa00add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 -- 172.21.15.64:0/2215833641 shutdown_connections 2023-09-27T23:29:57.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 -- 172.21.15.64:0/2215833641 wait complete. 2023-09-27T23:29:57.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 Processor -- start 2023-09-27T23:29:57.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.631+0000 7f5aa5568700 1 -- start start 2023-09-27T23:29:57.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5aa006be10 0x7f5aa0149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 0x7f5aa01670f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa014f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa014e3e0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa014e6c0 con 0x7f5aa00b0810 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa014e9a0 con 0x7f5aa006be10 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 0x7f5aa01670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 0x7f5aa01670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54978/0 (socket says 172.21.15.64:54978) 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 -- 172.21.15.64:0/1417176171 learned_addr learned my addr 172.21.15.64:0/1417176171 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a9ffff700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5aa006be10 0x7f5aa0149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa4d67700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa014f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 -- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5aa006be10 msgr2=0x7f5aa0149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5aa006be10 0x7f5aa0149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 -- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 msgr2=0x7f5aa014f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa014f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 -- 172.21.15.64:0/1417176171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90009420 con 0x7f5aa00af9c0 2023-09-27T23:29:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a97fff700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 0x7f5aa01670f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c7c0 tx=0x7f5a88007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88011590 con 0x7f5aa00af9c0 2023-09-27T23:29:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a880116f0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa014ec80 con 0x7f5aa00af9c0 2023-09-27T23:29:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88011ab0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.635+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa014a2e0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.639+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa004be40 con 0x7f5aa00af9c0 2023-09-27T23:29:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.639+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5a8801b3b0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.639+0000 7f5a9cff9700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a80060a70 0x7f5a80062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.639+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5a8807c4c0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.639+0000 7f5a9ffff700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a80060a70 0x7f5a80062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.639+0000 7f5a9ffff700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a80060a70 0x7f5a80062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a900093f0 tx=0x7f5a90017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.647+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5a8804c4f0 con 0x7f5aa00af9c0 2023-09-27T23:29:57.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.803+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5aa0159c20 con 0x7f5a80060a70 2023-09-27T23:29:57.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.819+0000 7f5a9cff9700 1 -- 172.21.15.64:0/1417176171 <== mgr.6209 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294571 (secure 0 0 0) 0x7f5aa0159c20 con 0x7f5a80060a70 2023-09-27T23:29:57.826 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a80060a70 msgr2=0x7f5a80062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a80060a70 0x7f5a80062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a900093f0 tx=0x7f5a90017000 comp rx=0 tx=0).stop 2023-09-27T23:29:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 msgr2=0x7f5aa01670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 0x7f5aa01670f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c7c0 tx=0x7f5a88007f50 comp rx=0 tx=0).stop 2023-09-27T23:29:57.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 shutdown_connections 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a80060a70 0x7f5a80062f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa00b0810 0x7f5aa014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa00af9c0 0x7f5aa01670f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 --2- 172.21.15.64:0/1417176171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5aa006be10 0x7f5aa0149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 >> 172.21.15.64:0/1417176171 conn(0x7f5aa0067bb0 msgr2=0x7f5aa00cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.823+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 shutdown_connections 2023-09-27T23:29:57.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.827+0000 7f5aa5568700 1 -- 172.21.15.64:0/1417176171 wait complete. 2023-09-27T23:29:57.831 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:57.841 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:56.999892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":40,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":207372,"kb_used_data":10380,"kb_used_omap":0,"kb_used_meta":196928,"kb_avail":754767348,"statfs":{"total":773094113280,"available":772881764352,"internally_reserved":0,"allocated":10629120,"data_stored":6016433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":201654272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.279463"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.148704+0000","last_change":"2023-09-27T23:29:54.145945+0000","last_active":"2023-09-27T23:29:54.148704+0000","last_peered":"2023-09-27T23:29:54.148704+0000","last_clean":"2023-09-27T23:29:54.148704+0000","last_became_active":"2023-09-27T23:29:54.145660+0000","last_became_peered":"2023-09-27T23:29:54.145660+0000","last_unstale":"2023-09-27T23:29:54.148704+0000","last_undegraded":"2023-09-27T23:29:54.148704+0000","last_fullsized":"2023-09-27T23:29:54.148704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:47.159474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145554+0000","last_change":"2023-09-27T23:29:54.139474+0000","last_active":"2023-09-27T23:29:54.145554+0000","last_peered":"2023-09-27T23:29:54.145554+0000","last_clean":"2023-09-27T23:29:54.145554+0000","last_became_active":"2023-09-27T23:29:54.139204+0000","last_became_peered":"2023-09-27T23:29:54.139204+0000","last_unstale":"2023-09-27T23:29:54.145554+0000","last_undegraded":"2023-09-27T23:29:54.145554+0000","last_fullsized":"2023-09-27T23:29:54.145554+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:09:54.976064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149543+0000","last_change":"2023-09-27T23:29:54.147371+0000","last_active":"2023-09-27T23:29:54.149543+0000","last_peered":"2023-09-27T23:29:54.149543+0000","last_clean":"2023-09-27T23:29:54.149543+0000","last_became_active":"2023-09-27T23:29:54.146786+0000","last_became_peered":"2023-09-27T23:29:54.146786+0000","last_unstale":"2023-09-27T23:29:54.149543+0000","last_undegraded":"2023-09-27T23:29:54.149543+0000","last_fullsized":"2023-09-27T23:29:54.149543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:34:51.507706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153647+0000","last_change":"2023-09-27T23:29:54.151837+0000","last_active":"2023-09-27T23:29:54.153647+0000","last_peered":"2023-09-27T23:29:54.153647+0000","last_clean":"2023-09-27T23:29:54.153647+0000","last_became_active":"2023-09-27T23:29:54.151587+0000","last_became_peered":"2023-09-27T23:29:54.151587+0000","last_unstale":"2023-09-27T23:29:54.153647+0000","last_undegraded":"2023-09-27T23:29:54.153647+0000","last_fullsized":"2023-09-27T23:29:54.153647+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:28.711624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146058+0000","last_change":"2023-09-27T23:29:54.137918+0000","last_active":"2023-09-27T23:29:54.146058+0000","last_peered":"2023-09-27T23:29:54.146058+0000","last_clean":"2023-09-27T23:29:54.146058+0000","last_became_active":"2023-09-27T23:29:54.137403+0000","last_became_peered":"2023-09-27T23:29:54.137403+0000","last_unstale":"2023-09-27T23:29:54.146058+0000","last_undegraded":"2023-09-27T23:29:54.146058+0000","last_fullsized":"2023-09-27T23:29:54.146058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:19:59.273479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145734+0000","last_change":"2023-09-27T23:29:54.141094+0000","last_active":"2023-09-27T23:29:54.145734+0000","last_peered":"2023-09-27T23:29:54.145734+0000","last_clean":"2023-09-27T23:29:54.145734+0000","last_became_active":"2023-09-27T23:29:54.140830+0000","last_became_peered":"2023-09-27T23:29:54.140830+0000","last_unstale":"2023-09-27T23:29:54.145734+0000","last_undegraded":"2023-09-27T23:29:54.145734+0000","last_fullsized":"2023-09-27T23:29:54.145734+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:48.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.147079+0000","last_change":"2023-09-27T23:29:54.144027+0000","last_active":"2023-09-27T23:29:54.147079+0000","last_peered":"2023-09-27T23:29:54.147079+0000","last_clean":"2023-09-27T23:29:54.147079+0000","last_became_active":"2023-09-27T23:29:54.143684+0000","last_became_peered":"2023-09-27T23:29:54.143684+0000","last_unstale":"2023-09-27T23:29:54.147079+0000","last_undegraded":"2023-09-27T23:29:54.147079+0000","last_fullsized":"2023-09-27T23:29:54.147079+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:22:41.890804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"89'183","reported_seq":466,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610106+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:49.610106+0000","last_peered":"2023-09-27T23:29:49.610106+0000","last_clean":"2023-09-27T23:29:49.610106+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:49.610106+0000","last_undegraded":"2023-09-27T23:29:49.610106+0000","last_fullsized":"2023-09-27T23:29:49.610106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":183,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863081+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:49.863081+0000","last_peered":"2023-09-27T23:29:49.863081+0000","last_clean":"2023-09-27T23:29:49.863081+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:49.863081+0000","last_undegraded":"2023-09-27T23:29:49.863081+0000","last_fullsized":"2023-09-27T23:29:49.863081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610203+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:49.610203+0000","last_peered":"2023-09-27T23:29:49.610203+0000","last_clean":"2023-09-27T23:29:49.610203+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:49.610203+0000","last_undegraded":"2023-09-27T23:29:49.610203+0000","last_fullsized":"2023-09-27T23:29:49.610203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117910+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:54.117910+0000","last_peered":"2023-09-27T23:29:54.117910+0000","last_clean":"2023-09-27T23:29:54.117910+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:54.117910+0000","last_undegraded":"2023-09-27T23:29:54.117910+0000","last_fullsized":"2023-09-27T23:29:54.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863080+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:49.863080+0000","last_peered":"2023-09-27T23:29:49.863080+0000","last_clean":"2023-09-27T23:29:49.863080+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:49.863080+0000","last_undegraded":"2023-09-27T23:29:49.863080+0000","last_fullsized":"2023-09-27T23:29:49.863080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.118980+0000","last_change":"2023-09-27T23:29:53.753232+0000","last_active":"2023-09-27T23:29:54.118980+0000","last_peered":"2023-09-27T23:29:54.118980+0000","last_clean":"2023-09-27T23:29:54.118980+0000","last_became_active":"2023-09-27T23:29:53.752871+0000","last_became_peered":"2023-09-27T23:29:53.752871+0000","last_unstale":"2023-09-27T23:29:54.118980+0000","last_undegraded":"2023-09-27T23:29:54.118980+0000","last_fullsized":"2023-09-27T23:29:54.118980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:12:40.768556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575761+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:49.575761+0000","last_peered":"2023-09-27T23:29:49.575761+0000","last_clean":"2023-09-27T23:29:49.575761+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:49.575761+0000","last_undegraded":"2023-09-27T23:29:49.575761+0000","last_fullsized":"2023-09-27T23:29:49.575761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149523+0000","last_change":"2023-09-27T23:29:54.147151+0000","last_active":"2023-09-27T23:29:54.149523+0000","last_peered":"2023-09-27T23:29:54.149523+0000","last_clean":"2023-09-27T23:29:54.149523+0000","last_became_active":"2023-09-27T23:29:54.146502+0000","last_became_peered":"2023-09-27T23:29:54.146502+0000","last_unstale":"2023-09-27T23:29:54.149523+0000","last_undegraded":"2023-09-27T23:29:54.149523+0000","last_fullsized":"2023-09-27T23:29:54.149523+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:22.885061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610219+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:49.610219+0000","last_peered":"2023-09-27T23:29:49.610219+0000","last_clean":"2023-09-27T23:29:49.610219+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:49.610219+0000","last_undegraded":"2023-09-27T23:29:49.610219+0000","last_fullsized":"2023-09-27T23:29:49.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575709+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:49.575709+0000","last_peered":"2023-09-27T23:29:49.575709+0000","last_clean":"2023-09-27T23:29:49.575709+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:49.575709+0000","last_undegraded":"2023-09-27T23:29:49.575709+0000","last_fullsized":"2023-09-27T23:29:49.575709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138284+0000","last_change":"2023-09-27T23:29:54.131847+0000","last_active":"2023-09-27T23:29:54.138284+0000","last_peered":"2023-09-27T23:29:54.138284+0000","last_clean":"2023-09-27T23:29:54.138284+0000","last_became_active":"2023-09-27T23:29:54.131553+0000","last_became_peered":"2023-09-27T23:29:54.131553+0000","last_unstale":"2023-09-27T23:29:54.138284+0000","last_undegraded":"2023-09-27T23:29:54.138284+0000","last_fullsized":"2023-09-27T23:29:54.138284+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:07.706937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117722+0000","last_change":"2023-09-27T23:29:53.754772+0000","last_active":"2023-09-27T23:29:54.117722+0000","last_peered":"2023-09-27T23:29:54.117722+0000","last_clean":"2023-09-27T23:29:54.117722+0000","last_became_active":"2023-09-27T23:29:53.754460+0000","last_became_peered":"2023-09-27T23:29:53.754460+0000","last_unstale":"2023-09-27T23:29:54.117722+0000","last_undegraded":"2023-09-27T23:29:54.117722+0000","last_fullsized":"2023-09-27T23:29:54.117722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:18.033695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.612127+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:49.612127+0000","last_peered":"2023-09-27T23:29:49.612127+0000","last_clean":"2023-09-27T23:29:49.612127+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:49.612127+0000","last_undegraded":"2023-09-27T23:29:49.612127+0000","last_fullsized":"2023-09-27T23:29:49.612127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25984,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":94345856,"statfs":{"total":96636764160,"available":96610156544,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":1.093}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.02}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.98199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.93100000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.149},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21564,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350276,"statfs":{"total":96636764160,"available":96614682624,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.073},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":1.2090000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.014},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":0.878},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.167}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.115}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.754},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.268},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2390000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":34624,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35454976},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.1899999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":1.379},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":1.22}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":1.3180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.071},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.3520000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.2849999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21832,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":20672,"kb_avail":94350008,"statfs":{"total":96636764160,"available":96614408192,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21168128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.78300000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":1.4850000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.609},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":0.89700000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.282}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.7929999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22280,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349560,"statfs":{"total":96636764160,"available":96613949440,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.63700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.605}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.3779999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.556}]}]},{"osd":2,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21116,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94350724,"statfs":{"total":96636764160,"available":96615141376,"internally_reserved":0,"allocated":1232896,"data_stored":632438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.78700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":1.091},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.97999999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.004}]}]},{"osd":1,"up_from":14,"seq":60129542342,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36028,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.752},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":1.0029999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":1.044},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":1.105},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":1.204},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":1.256}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.2310000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22784,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94349056,"statfs":{"total":96636764160,"available":96613433344,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.82599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.96899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.89600000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.056},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.3129999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":1.252}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:29:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-09-27T23:29:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-09-27T23:29:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-09-27T23:29:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-09-27T23:29:57.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-09-27T23:29:57.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-09-27T23:29:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-09-27T23:29:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-09-27T23:29:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-09-27T23:29:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-09-27T23:29:57.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-09-27T23:29:57.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-09-27T23:29:57.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-09-27T23:29:57.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-09-27T23:29:57.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-09-27T23:29:57.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-09-27T23:29:57.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-09-27T23:29:57.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-09-27T23:29:57.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-09-27T23:29:57.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-09-27T23:29:57.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-09-27T23:29:57.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-09-27T23:29:57.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-09-27T23:29:57.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-09-27T23:29:57.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-09-27T23:29:57.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-09-27T23:29:57.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-09-27T23:29:57.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-09-27T23:29:57.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-09-27T23:29:57.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-09-27T23:29:57.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-09-27T23:29:57.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-09-27T23:29:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-09-27T23:29:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-09-27T23:29:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-09-27T23:29:57.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-09-27T23:29:57.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-09-27T23:29:57.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-09-27T23:29:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-09-27T23:29:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-09-27T23:29:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-09-27T23:29:57.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-09-27T23:29:57.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-09-27T23:29:57.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-09-27T23:29:57.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-09-27T23:29:57.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-09-27T23:29:57.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-09-27T23:29:57.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-09-27T23:29:57.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-09-27T23:29:57.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-09-27T23:29:57.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-09-27T23:29:57.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-09-27T23:29:57.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-09-27T23:29:57.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-09-27T23:29:57.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-09-27T23:29:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-09-27T23:29:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-09-27T23:29:57.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-09-27T23:29:57.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:53.716937+0000', 'last_change': '2023-09-27T23:29:53.716937+0000', 'last_active': '2023-09-27T23:29:53.716937+0000', 'last_peered': '2023-09-27T23:29:53.716937+0000', 'last_clean': '2023-09-27T23:29:53.716937+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:53.716937+0000', 'last_undegraded': '2023-09-27T23:29:53.716937+0000', 'last_fullsized': '2023-09-27T23:29:53.716937+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:53.716937+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-09-27T23:29:57.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:29:52.705384+0000', 'last_change': '2023-09-27T23:29:52.705384+0000', 'last_active': '2023-09-27T23:29:52.705384+0000', 'last_peered': '2023-09-27T23:29:52.705384+0000', 'last_clean': '2023-09-27T23:29:52.705384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:29:52.705384+0000', 'last_undegraded': '2023-09-27T23:29:52.705384+0000', 'last_fullsized': '2023-09-27T23:29:52.705384+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'last_clean_scrub_stamp': '2023-09-27T23:29:52.705384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:29:57.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:29:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 Processor -- start 2023-09-27T23:29:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 -- start start 2023-09-27T23:29:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f306806de20 0x7f30680c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f306806c7c0 0x7f30680c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306806d990 con 0x7f3068155960 2023-09-27T23:29:57.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306806cd80 con 0x7f306806de20 2023-09-27T23:29:57.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f306cbc1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306806d060 con 0x7f306806c7c0 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.979+0000 7f3066ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f306806de20 0x7f30680c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f306806de20 0x7f30680c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54994/0 (socket says 172.21.15.64:54994) 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 -- 172.21.15.64:0/2200616501 learned_addr learned my addr 172.21.15.64:0/2200616501 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3067fff700 1 --2- 172.21.15.64:0/2200616501 >> v2:172.21.15.123:3300/0 conn(0x7f306806c7c0 0x7f30680c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 -- 172.21.15.64:0/2200616501 >> v2:172.21.15.123:3300/0 conn(0x7f306806c7c0 msgr2=0x7f30680c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 --2- 172.21.15.64:0/2200616501 >> v2:172.21.15.123:3300/0 conn(0x7f306806c7c0 0x7f30680c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 -- 172.21.15.64:0/2200616501 >> v1:172.21.15.123:6789/0 conn(0x7f3068155960 legacy=0x7f306815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 -- 172.21.15.64:0/2200616501 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306806d340 con 0x7f306806de20 2023-09-27T23:29:57.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f3066ffd700 1 --2- 172.21.15.64:0/2200616501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f30680c77f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f306806a380 tx=0x7f305000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae7c636616d8c81 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f30667fc700 1 -- 172.21.15.64:0/2200616501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305000dba0 con 0x7f306806de20 2023-09-27T23:29:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f30667fc700 1 -- 172.21.15.64:0/2200616501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305000dd00 con 0x7f306806de20 2023-09-27T23:29:57.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f30667fc700 1 -- 172.21.15.64:0/2200616501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30500196d0 con 0x7f306806de20 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 -- 172.21.15.64:0/2200616501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 msgr2=0x7f30680c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2200616501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f30680c77f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f306806a380 tx=0x7f305000b4f0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 -- 172.21.15.64:0/2200616501 shutdown_connections 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2200616501 >> v2:172.21.15.123:3300/0 conn(0x7f306806c7c0 0x7f30680c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2200616501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f30680c77f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 -- 172.21.15.64:0/2200616501 >> 172.21.15.64:0/2200616501 conn(0x7f3068067bb0 msgr2=0x7f30680cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 -- 172.21.15.64:0/2200616501 shutdown_connections 2023-09-27T23:29:57.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.983+0000 7f306cbc1700 1 -- 172.21.15.64:0/2200616501 wait complete. 2023-09-27T23:29:57.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 Processor -- start 2023-09-27T23:29:57.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 -- start start 2023-09-27T23:29:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f306806c7c0 0x7f306815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f3068150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3068155960 0x7f30681505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068151a60 con 0x7f306806de20 2023-09-27T23:29:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068151d40 con 0x7f3068155960 2023-09-27T23:29:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3068152020 con 0x7f306806c7c0 2023-09-27T23:29:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f3068150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3067fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3068155960 0x7f30681505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f3068150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55006/0 (socket says 172.21.15.64:55006) 2023-09-27T23:29:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 -- 172.21.15.64:0/2000878869 learned_addr learned my addr 172.21.15.64:0/2000878869 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:57.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f30677fe700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f306806c7c0 0x7f306815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 -- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f306806c7c0 msgr2=0x7f306815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f306806c7c0 0x7f306815eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 -- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3068155960 msgr2=0x7f30681505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3068155960 0x7f30681505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 -- 172.21.15.64:0/2000878869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305000b790 con 0x7f306806de20 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3067fff700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3068155960 0x7f30681505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f3066ffd700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f3068150090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f305000b760 tx=0x7f305001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305000de70 con 0x7f306806de20 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050010470 con 0x7f306806de20 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050010830 con 0x7f306806de20 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068152300 con 0x7f306806de20 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.987+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306815f530 con 0x7f306806de20 2023-09-27T23:29:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.991+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306804be40 con 0x7f306806de20 2023-09-27T23:29:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.991+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3050022030 con 0x7f306806de20 2023-09-27T23:29:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.991+0000 7f305f7fe700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3048060a70 0x7f3048062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:57.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.991+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3050014070 con 0x7f306806de20 2023-09-27T23:29:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.991+0000 7f30677fe700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3048060a70 0x7f3048062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:57.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:57.991+0000 7f30677fe700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3048060a70 0x7f3048062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30680cb270 tx=0x7f3058009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:58.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.003+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3050050da0 con 0x7f306806de20 2023-09-27T23:29:58.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.231+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3068151ba0 con 0x7f306806de20 2023-09-27T23:29:58.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.231+0000 7f305f7fe700 1 -- 172.21.15.64:0/2000878869 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3221 (secure 0 0 0) 0x7f3068151ba0 con 0x7f306806de20 2023-09-27T23:29:58.235 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:58.235 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":997,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":101,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":85},{"state_name":"active+clean","count":20}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":212348928,"bytes_avail":772881764352,"bytes_total":773094113280,"unknown_pgs_ratio":0.8095238208770752},"fsmap":{"epoch":33,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":7107}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-09-27T23:29:42.995756+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3048060a70 msgr2=0x7f3048062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3048060a70 0x7f3048062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30680cb270 tx=0x7f3058009040 comp rx=0 tx=0).stop 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 msgr2=0x7f3068150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f3068150090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f305000b760 tx=0x7f305001b000 comp rx=0 tx=0).stop 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 shutdown_connections 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3048060a70 0x7f3048062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3068155960 0x7f30681505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f306806de20 0x7f3068150090 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 --2- 172.21.15.64:0/2000878869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f306806c7c0 0x7f306815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 >> 172.21.15.64:0/2000878869 conn(0x7f3068067bb0 msgr2=0x7f30680ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 shutdown_connections 2023-09-27T23:29:58.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.235+0000 7f306cbc1700 1 -- 172.21.15.64:0/2000878869 wait complete. 2023-09-27T23:29:58.249 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:29:58.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:29:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 Processor -- start 2023-09-27T23:29:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- start start 2023-09-27T23:29:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe3006a580 0x7efe3006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe3006d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3006c7c0 con 0x7efe300b1410 2023-09-27T23:29:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3006af40 con 0x7efe3006a580 2023-09-27T23:29:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3006cd80 con 0x7efe3006d210 2023-09-27T23:29:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe3006a580 0x7efe3006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe36053700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efe300b1410 0x7efe300ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38650/0 (socket says 172.21.15.64:38650) 2023-09-27T23:29:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe3006a580 0x7efe3006aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55020/0 (socket says 172.21.15.64:55020) 2023-09-27T23:29:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe36053700 1 -- 172.21.15.64:0/2932037135 learned_addr learned my addr 172.21.15.64:0/2932037135 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:29:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe36854700 1 --2- 172.21.15.64:0/2932037135 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe3006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:58.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 -- 172.21.15.64:0/2932037135 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 msgr2=0x7efe3006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 --2- 172.21.15.64:0/2932037135 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe3006d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 -- 172.21.15.64:0/2932037135 >> v1:172.21.15.64:6789/0 conn(0x7efe300b1410 legacy=0x7efe300ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 -- 172.21.15.64:0/2932037135 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3006b8f0 con 0x7efe3006a580 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35051700 1 -- 172.21.15.64:0/2932037135 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031145227 0 0) 0x7efe3006c7c0 con 0x7efe300b1410 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35852700 1 --2- 172.21.15.64:0/2932037135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe3006aa00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efe200098e0 tx=0x7efe200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b305a573a34eb2b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35051700 1 -- 172.21.15.64:0/2932037135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2000b640 con 0x7efe3006a580 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35051700 1 -- 172.21.15.64:0/2932037135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2000b7a0 con 0x7efe3006a580 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe35051700 1 -- 172.21.15.64:0/2932037135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2000bb40 con 0x7efe3006a580 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- 172.21.15.64:0/2932037135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 msgr2=0x7efe3006aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 --2- 172.21.15.64:0/2932037135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe3006aa00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efe200098e0 tx=0x7efe200091a0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- 172.21.15.64:0/2932037135 shutdown_connections 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 --2- 172.21.15.64:0/2932037135 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe3006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 --2- 172.21.15.64:0/2932037135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe3006aa00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- 172.21.15.64:0/2932037135 >> 172.21.15.64:0/2932037135 conn(0x7efe30067bb0 msgr2=0x7efe300cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- 172.21.15.64:0/2932037135 shutdown_connections 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.359+0000 7efe37055700 1 -- 172.21.15.64:0/2932037135 wait complete. 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 Processor -- start 2023-09-27T23:29:58.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 -- start start 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe30154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe300adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe300b1410 0x7efe300ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe301697a0 con 0x7efe3006a580 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3016a5a0 con 0x7efe300b1410 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe37055700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe30169990 con 0x7efe3006d210 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe30154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe30154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55034/0 (socket says 172.21.15.64:55034) 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 -- 172.21.15.64:0/2125562105 learned_addr learned my addr 172.21.15.64:0/2125562105 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:29:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe35852700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe300adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36854700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe300b1410 0x7efe300ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 -- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 msgr2=0x7efe300adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe300adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 -- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe300b1410 msgr2=0x7efe300ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe300b1410 0x7efe300ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 -- 172.21.15.64:0/2125562105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20009420 con 0x7efe3006a580 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe36053700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe30154500 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c5b0 tx=0x7efe2c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.363+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c005b30 con 0x7efe3006a580 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c005c90 con 0x7efe3006a580 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe30169c70 con 0x7efe3006a580 2023-09-27T23:29:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c007300 con 0x7efe3006a580 2023-09-27T23:29:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3016a230 con 0x7efe3006a580 2023-09-27T23:29:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3004be40 con 0x7efe3006a580 2023-09-27T23:29:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7efe2c01b3b0 con 0x7efe3006a580 2023-09-27T23:29:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe267fc700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efe14060a70 0x7efe14062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:29:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.367+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efe2c07c540 con 0x7efe3006a580 2023-09-27T23:29:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.371+0000 7efe35852700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efe14060a70 0x7efe14062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:29:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.371+0000 7efe35852700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efe14060a70 0x7efe14062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe200091e0 tx=0x7efe20017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:29:58.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.375+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efe2c04c570 con 0x7efe3006a580 2023-09-27T23:29:58.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.527+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe3016a6e0 con 0x7efe14060a70 2023-09-27T23:29:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.547+0000 7efe267fc700 1 -- 172.21.15.64:0/2125562105 <== mgr.6209 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294571 (secure 0 0 0) 0x7efe20006c50 con 0x7efe14060a70 2023-09-27T23:29:58.552 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:29:58.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efe14060a70 msgr2=0x7efe14062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efe14060a70 0x7efe14062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe200091e0 tx=0x7efe20017000 comp rx=0 tx=0).stop 2023-09-27T23:29:58.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 msgr2=0x7efe30154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe30154500 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c5b0 tx=0x7efe2c001070 comp rx=0 tx=0).stop 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 shutdown_connections 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efe14060a70 0x7efe14062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe300b1410 0x7efe300ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe3006d210 0x7efe300adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 --2- 172.21.15.64:0/2125562105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3006a580 0x7efe30154500 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 >> 172.21.15.64:0/2125562105 conn(0x7efe30067bb0 msgr2=0x7efe300aa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 shutdown_connections 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:29:58.551+0000 7efe37055700 1 -- 172.21.15.64:0/2125562105 wait complete. 2023-09-27T23:29:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:29:58.568 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:29:56.999892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":40,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":207372,"kb_used_data":10380,"kb_used_omap":0,"kb_used_meta":196928,"kb_avail":754767348,"statfs":{"total":773094113280,"available":772881764352,"internally_reserved":0,"allocated":10629120,"data_stored":6016433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":201654272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.279463"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.148704+0000","last_change":"2023-09-27T23:29:54.145945+0000","last_active":"2023-09-27T23:29:54.148704+0000","last_peered":"2023-09-27T23:29:54.148704+0000","last_clean":"2023-09-27T23:29:54.148704+0000","last_became_active":"2023-09-27T23:29:54.145660+0000","last_became_peered":"2023-09-27T23:29:54.145660+0000","last_unstale":"2023-09-27T23:29:54.148704+0000","last_undegraded":"2023-09-27T23:29:54.148704+0000","last_fullsized":"2023-09-27T23:29:54.148704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:47.159474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145554+0000","last_change":"2023-09-27T23:29:54.139474+0000","last_active":"2023-09-27T23:29:54.145554+0000","last_peered":"2023-09-27T23:29:54.145554+0000","last_clean":"2023-09-27T23:29:54.145554+0000","last_became_active":"2023-09-27T23:29:54.139204+0000","last_became_peered":"2023-09-27T23:29:54.139204+0000","last_unstale":"2023-09-27T23:29:54.145554+0000","last_undegraded":"2023-09-27T23:29:54.145554+0000","last_fullsized":"2023-09-27T23:29:54.145554+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:09:54.976064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149543+0000","last_change":"2023-09-27T23:29:54.147371+0000","last_active":"2023-09-27T23:29:54.149543+0000","last_peered":"2023-09-27T23:29:54.149543+0000","last_clean":"2023-09-27T23:29:54.149543+0000","last_became_active":"2023-09-27T23:29:54.146786+0000","last_became_peered":"2023-09-27T23:29:54.146786+0000","last_unstale":"2023-09-27T23:29:54.149543+0000","last_undegraded":"2023-09-27T23:29:54.149543+0000","last_fullsized":"2023-09-27T23:29:54.149543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:34:51.507706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153647+0000","last_change":"2023-09-27T23:29:54.151837+0000","last_active":"2023-09-27T23:29:54.153647+0000","last_peered":"2023-09-27T23:29:54.153647+0000","last_clean":"2023-09-27T23:29:54.153647+0000","last_became_active":"2023-09-27T23:29:54.151587+0000","last_became_peered":"2023-09-27T23:29:54.151587+0000","last_unstale":"2023-09-27T23:29:54.153647+0000","last_undegraded":"2023-09-27T23:29:54.153647+0000","last_fullsized":"2023-09-27T23:29:54.153647+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:28.711624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146058+0000","last_change":"2023-09-27T23:29:54.137918+0000","last_active":"2023-09-27T23:29:54.146058+0000","last_peered":"2023-09-27T23:29:54.146058+0000","last_clean":"2023-09-27T23:29:54.146058+0000","last_became_active":"2023-09-27T23:29:54.137403+0000","last_became_peered":"2023-09-27T23:29:54.137403+0000","last_unstale":"2023-09-27T23:29:54.146058+0000","last_undegraded":"2023-09-27T23:29:54.146058+0000","last_fullsized":"2023-09-27T23:29:54.146058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:19:59.273479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145734+0000","last_change":"2023-09-27T23:29:54.141094+0000","last_active":"2023-09-27T23:29:54.145734+0000","last_peered":"2023-09-27T23:29:54.145734+0000","last_clean":"2023-09-27T23:29:54.145734+0000","last_became_active":"2023-09-27T23:29:54.140830+0000","last_became_peered":"2023-09-27T23:29:54.140830+0000","last_unstale":"2023-09-27T23:29:54.145734+0000","last_undegraded":"2023-09-27T23:29:54.145734+0000","last_fullsized":"2023-09-27T23:29:54.145734+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:48.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.147079+0000","last_change":"2023-09-27T23:29:54.144027+0000","last_active":"2023-09-27T23:29:54.147079+0000","last_peered":"2023-09-27T23:29:54.147079+0000","last_clean":"2023-09-27T23:29:54.147079+0000","last_became_active":"2023-09-27T23:29:54.143684+0000","last_became_peered":"2023-09-27T23:29:54.143684+0000","last_unstale":"2023-09-27T23:29:54.147079+0000","last_undegraded":"2023-09-27T23:29:54.147079+0000","last_fullsized":"2023-09-27T23:29:54.147079+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:22:41.890804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"89'183","reported_seq":466,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610106+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:49.610106+0000","last_peered":"2023-09-27T23:29:49.610106+0000","last_clean":"2023-09-27T23:29:49.610106+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:49.610106+0000","last_undegraded":"2023-09-27T23:29:49.610106+0000","last_fullsized":"2023-09-27T23:29:49.610106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":183,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863081+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:49.863081+0000","last_peered":"2023-09-27T23:29:49.863081+0000","last_clean":"2023-09-27T23:29:49.863081+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:49.863081+0000","last_undegraded":"2023-09-27T23:29:49.863081+0000","last_fullsized":"2023-09-27T23:29:49.863081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610203+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:49.610203+0000","last_peered":"2023-09-27T23:29:49.610203+0000","last_clean":"2023-09-27T23:29:49.610203+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:49.610203+0000","last_undegraded":"2023-09-27T23:29:49.610203+0000","last_fullsized":"2023-09-27T23:29:49.610203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117910+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:54.117910+0000","last_peered":"2023-09-27T23:29:54.117910+0000","last_clean":"2023-09-27T23:29:54.117910+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:54.117910+0000","last_undegraded":"2023-09-27T23:29:54.117910+0000","last_fullsized":"2023-09-27T23:29:54.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.863080+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:49.863080+0000","last_peered":"2023-09-27T23:29:49.863080+0000","last_clean":"2023-09-27T23:29:49.863080+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:49.863080+0000","last_undegraded":"2023-09-27T23:29:49.863080+0000","last_fullsized":"2023-09-27T23:29:49.863080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.118980+0000","last_change":"2023-09-27T23:29:53.753232+0000","last_active":"2023-09-27T23:29:54.118980+0000","last_peered":"2023-09-27T23:29:54.118980+0000","last_clean":"2023-09-27T23:29:54.118980+0000","last_became_active":"2023-09-27T23:29:53.752871+0000","last_became_peered":"2023-09-27T23:29:53.752871+0000","last_unstale":"2023-09-27T23:29:54.118980+0000","last_undegraded":"2023-09-27T23:29:54.118980+0000","last_fullsized":"2023-09-27T23:29:54.118980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:12:40.768556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575761+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:49.575761+0000","last_peered":"2023-09-27T23:29:49.575761+0000","last_clean":"2023-09-27T23:29:49.575761+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:49.575761+0000","last_undegraded":"2023-09-27T23:29:49.575761+0000","last_fullsized":"2023-09-27T23:29:49.575761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149523+0000","last_change":"2023-09-27T23:29:54.147151+0000","last_active":"2023-09-27T23:29:54.149523+0000","last_peered":"2023-09-27T23:29:54.149523+0000","last_clean":"2023-09-27T23:29:54.149523+0000","last_became_active":"2023-09-27T23:29:54.146502+0000","last_became_peered":"2023-09-27T23:29:54.146502+0000","last_unstale":"2023-09-27T23:29:54.149523+0000","last_undegraded":"2023-09-27T23:29:54.149523+0000","last_fullsized":"2023-09-27T23:29:54.149523+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:22.885061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.610219+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:49.610219+0000","last_peered":"2023-09-27T23:29:49.610219+0000","last_clean":"2023-09-27T23:29:49.610219+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:49.610219+0000","last_undegraded":"2023-09-27T23:29:49.610219+0000","last_fullsized":"2023-09-27T23:29:49.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:53.716937+0000","last_change":"2023-09-27T23:29:53.716937+0000","last_active":"2023-09-27T23:29:53.716937+0000","last_peered":"2023-09-27T23:29:53.716937+0000","last_clean":"2023-09-27T23:29:53.716937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:53.716937+0000","last_undegraded":"2023-09-27T23:29:53.716937+0000","last_fullsized":"2023-09-27T23:29:53.716937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:29:52.705384+0000","last_change":"2023-09-27T23:29:52.705384+0000","last_active":"2023-09-27T23:29:52.705384+0000","last_peered":"2023-09-27T23:29:52.705384+0000","last_clean":"2023-09-27T23:29:52.705384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:29:52.705384+0000","last_undegraded":"2023-09-27T23:29:52.705384+0000","last_fullsized":"2023-09-27T23:29:52.705384+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.575709+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:49.575709+0000","last_peered":"2023-09-27T23:29:49.575709+0000","last_clean":"2023-09-27T23:29:49.575709+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:49.575709+0000","last_undegraded":"2023-09-27T23:29:49.575709+0000","last_fullsized":"2023-09-27T23:29:49.575709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138284+0000","last_change":"2023-09-27T23:29:54.131847+0000","last_active":"2023-09-27T23:29:54.138284+0000","last_peered":"2023-09-27T23:29:54.138284+0000","last_clean":"2023-09-27T23:29:54.138284+0000","last_became_active":"2023-09-27T23:29:54.131553+0000","last_became_peered":"2023-09-27T23:29:54.131553+0000","last_unstale":"2023-09-27T23:29:54.138284+0000","last_undegraded":"2023-09-27T23:29:54.138284+0000","last_fullsized":"2023-09-27T23:29:54.138284+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:07.706937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117722+0000","last_change":"2023-09-27T23:29:53.754772+0000","last_active":"2023-09-27T23:29:54.117722+0000","last_peered":"2023-09-27T23:29:54.117722+0000","last_clean":"2023-09-27T23:29:54.117722+0000","last_became_active":"2023-09-27T23:29:53.754460+0000","last_became_peered":"2023-09-27T23:29:53.754460+0000","last_unstale":"2023-09-27T23:29:54.117722+0000","last_undegraded":"2023-09-27T23:29:54.117722+0000","last_fullsized":"2023-09-27T23:29:54.117722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:18.033695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-09-27T23:29:49.612127+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:49.612127+0000","last_peered":"2023-09-27T23:29:49.612127+0000","last_clean":"2023-09-27T23:29:49.612127+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:49.612127+0000","last_undegraded":"2023-09-27T23:29:49.612127+0000","last_fullsized":"2023-09-27T23:29:49.612127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25984,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":94345856,"statfs":{"total":96636764160,"available":96610156544,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":1.093}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.02}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":0.86599999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.98199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.93100000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.149},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.0509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21564,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":20416,"kb_avail":94350276,"statfs":{"total":96636764160,"available":96614682624,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20905984},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.073},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":1.2090000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.014},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":0.878},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.167}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":1.115}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.754},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.268},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2390000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35784,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":34624,"kb_avail":94336056,"statfs":{"total":96636764160,"available":96600121344,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35454976},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89200000000000002,"15min":0.874},"min":{"1min":0.67400000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.179,"5min":1.2090000000000001,"15min":1.2909999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91700000000000004,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.1970000000000001,"5min":1.3440000000000001,"15min":1.359},"last":1.1899999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.526},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.76700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.4179999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.91500000000000004},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1100000000000001,"5min":1.391,"15min":1.391},"last":1.379},{"interface":"front","average":{"1min":1.081,"5min":0.97399999999999998,"15min":0.94899999999999995},"min":{"1min":0.86199999999999999,"5min":0.628,"15min":0.61499999999999999},"max":{"1min":1.2210000000000001,"5min":1.369,"15min":1.5740000000000001},"last":1.22}]},{"osd":3,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.73999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.2270000000000001,"15min":1.5640000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.016,"15min":1.0149999999999999},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2410000000000001,"5min":1.3720000000000001,"15min":1.954},"last":1.149}]},{"osd":4,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.624,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.153,"5min":1.341,"15min":1.5349999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.0249999999999999},"min":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2769999999999999,"5min":1.399,"15min":1.704},"last":1.3180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.71699999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3240000000000001,"5min":1.429,"15min":1.6319999999999999},"last":1.071},{"interface":"front","average":{"1min":1.093,"5min":1.103,"15min":1.101},"min":{"1min":0.874,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.298,"5min":1.4039999999999999,"15min":1.837},"last":1.3520000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.91200000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.5189999999999999,"15min":1.7230000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":1.0549999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.4730000000000001,"5min":1.4890000000000001,"15min":2.0529999999999999},"last":1.2849999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":21832,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":20672,"kb_avail":94350008,"statfs":{"total":96636764160,"available":96614408192,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21168128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.78300000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":1.4850000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.609},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":0.89700000000000002}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.282}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.7929999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22280,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349560,"statfs":{"total":96636764160,"available":96613949440,"internally_reserved":0,"allocated":1179648,"data_stored":604073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.63700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":1.605}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":1.3779999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":1.556}]}]},{"osd":2,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21116,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94350724,"statfs":{"total":96636764160,"available":96615141376,"internally_reserved":0,"allocated":1232896,"data_stored":632438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.78700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":1.091},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.97999999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.004}]}]},{"osd":1,"up_from":14,"seq":60129542342,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36028,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":1167360,"data_stored":597760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.752},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":1.0029999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":1.044},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":1.105},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":1.204},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":1.256}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.2310000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22784,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94349056,"statfs":{"total":96636764160,"available":96613433344,"internally_reserved":0,"allocated":1761280,"data_stored":1188128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.82599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.96899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":0.89600000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.056},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":1.3129999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":1.252}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:00.023 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:30:00.019+0000 7fb6fd07c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:30:01.576 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:30:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7482b3d700 1 Processor -- start 2023-09-27T23:30:01.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7482b3d700 1 -- start start 2023-09-27T23:30:01.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7482b3d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f747c06c7c0 0x7f747c0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:01.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7482b3d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c0ca7a0 con 0x7f747c156a70 2023-09-27T23:30:01.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7482b3d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c06b020 con 0x7f747c0b1500 2023-09-27T23:30:01.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7482b3d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c0b1980 con 0x7f747c06c7c0 2023-09-27T23:30:01.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7481b3b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f747c156a70 0x7f747c06a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38654/0 (socket says 172.21.15.64:38654) 2023-09-27T23:30:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7481b3b700 1 -- 172.21.15.64:0/2244457177 learned_addr learned my addr 172.21.15.64:0/2244457177 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f748133a700 1 --2- 172.21.15.64:0/2244457177 >> v2:172.21.15.64:3300/0 conn(0x7f747c06c7c0 0x7f747c0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7480b39700 1 -- 172.21.15.64:0/2244457177 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891003618 0 0) 0x7f747c0ca7a0 con 0x7f747c156a70 2023-09-27T23:30:01.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.691+0000 7f7480b39700 1 -- 172.21.15.64:0/2244457177 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7468003570 con 0x7f747c156a70 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f748133a700 1 -- 172.21.15.64:0/2244457177 >> v1:172.21.15.123:6789/0 conn(0x7f747c0b1500 legacy=0x7f747c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7480b39700 1 -- 172.21.15.64:0/2244457177 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125878241 0 0) 0x7f747c06b020 con 0x7f747c0b1500 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f748133a700 1 -- 172.21.15.64:0/2244457177 >> v1:172.21.15.64:6789/0 conn(0x7f747c156a70 legacy=0x7f747c06a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f748133a700 1 -- 172.21.15.64:0/2244457177 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0ca470 con 0x7f747c06c7c0 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f748133a700 1 --2- 172.21.15.64:0/2244457177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c06c7c0 0x7f747c0b0fc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f74700098e0 tx=0x7f74700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34be43f69bc78227 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7480b39700 1 -- 172.21.15.64:0/2244457177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000b640 con 0x7f747c06c7c0 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7480b39700 1 -- 172.21.15.64:0/2244457177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747000b7a0 con 0x7f747c06c7c0 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7480b39700 1 -- 172.21.15.64:0/2244457177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000bb40 con 0x7f747c06c7c0 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 -- 172.21.15.64:0/2244457177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c06c7c0 msgr2=0x7f747c0b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 --2- 172.21.15.64:0/2244457177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c06c7c0 0x7f747c0b0fc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f74700098e0 tx=0x7f74700091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 -- 172.21.15.64:0/2244457177 shutdown_connections 2023-09-27T23:30:01.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 --2- 172.21.15.64:0/2244457177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c06c7c0 0x7f747c0b0fc0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 -- 172.21.15.64:0/2244457177 >> 172.21.15.64:0/2244457177 conn(0x7f747c067bb0 msgr2=0x7f747c0ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:01.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 -- 172.21.15.64:0/2244457177 shutdown_connections 2023-09-27T23:30:01.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 -- 172.21.15.64:0/2244457177 wait complete. 2023-09-27T23:30:01.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 Processor -- start 2023-09-27T23:30:01.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.695+0000 7f7482b3d700 1 -- start start 2023-09-27T23:30:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f747c06c7c0 0x7f747c0a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f747c0b1500 0x7f747c0a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:01.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 0x7f747c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0a4440 con 0x7f747c156a70 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0a46e0 con 0x7f747c0b1500 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0a49c0 con 0x7f747c06c7c0 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 0x7f747c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7481b3b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f747c06c7c0 0x7f747c0a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 0x7f747c15b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55044/0 (socket says 172.21.15.64:55044) 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 -- 172.21.15.64:0/1865796240 learned_addr learned my addr 172.21.15.64:0/1865796240 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748133a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f747c0b1500 0x7f747c0a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 -- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f747c06c7c0 msgr2=0x7f747c0a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f747c06c7c0 0x7f747c0a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 -- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f747c0b1500 msgr2=0x7f747c0a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f747c0b1500 0x7f747c0a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 -- 172.21.15.64:0/1865796240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470009420 con 0x7f747c156a70 2023-09-27T23:30:01.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f748233c700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 0x7f747c15b7e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f747800c5d0 tx=0x7f7478007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478011590 con 0x7f747c156a70 2023-09-27T23:30:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74780116f0 con 0x7f747c156a70 2023-09-27T23:30:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c0a4ca0 con 0x7f747c156a70 2023-09-27T23:30:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478011ab0 con 0x7f747c156a70 2023-09-27T23:30:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c0a6610 con 0x7f747c156a70 2023-09-27T23:30:01.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.699+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c04be40 con 0x7f747c156a70 2023-09-27T23:30:01.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.703+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f747801b3b0 con 0x7f747c156a70 2023-09-27T23:30:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.703+0000 7f746dffb700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7460060a70 0x7f7460062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.703+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f747807c620 con 0x7f747c156a70 2023-09-27T23:30:01.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.703+0000 7f7481b3b700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7460060a70 0x7f7460062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:01.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.703+0000 7f7481b3b700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7460060a70 0x7f7460062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7474000f80 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:01.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.711+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f747804c650 con 0x7f747c156a70 2023-09-27T23:30:01.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.875+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f747c0a6860 con 0x7f7460060a70 2023-09-27T23:30:01.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.895+0000 7f746dffb700 1 -- 172.21.15.64:0/1865796240 <== mgr.6209 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307776 (secure 0 0 0) 0x7f747400f920 con 0x7f7460060a70 2023-09-27T23:30:01.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7460060a70 msgr2=0x7f7460062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7460060a70 0x7f7460062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7474000f80 tx=0x7f7474009040 comp rx=0 tx=0).stop 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 msgr2=0x7f747c15b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 0x7f747c15b7e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f747800c5d0 tx=0x7f7478007d70 comp rx=0 tx=0).stop 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 shutdown_connections 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7460060a70 0x7f7460062f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f747c156a70 0x7f747c15b7e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f747c0b1500 0x7f747c0a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 --2- 172.21.15.64:0/1865796240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f747c06c7c0 0x7f747c0a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:01.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 >> 172.21.15.64:0/1865796240 conn(0x7f747c067bb0 msgr2=0x7f747c0ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:01.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 shutdown_connections 2023-09-27T23:30:01.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:01.899+0000 7f7482b3d700 1 -- 172.21.15.64:0/1865796240 wait complete. 2023-09-27T23:30:01.904 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:30:01.915 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:30:01.001541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":208904,"kb_used_data":10824,"kb_used_omap":0,"kb_used_meta":198016,"kb_avail":754765816,"statfs":{"total":773094113280,"available":772880195584,"internally_reserved":0,"allocated":11083776,"data_stored":6240420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":202768384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.268250"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.156481+0000","last_change":"2023-09-27T23:29:54.149646+0000","last_active":"2023-09-27T23:29:54.156481+0000","last_peered":"2023-09-27T23:29:54.156481+0000","last_clean":"2023-09-27T23:29:54.156481+0000","last_became_active":"2023-09-27T23:29:54.149377+0000","last_became_peered":"2023-09-27T23:29:54.149377+0000","last_unstale":"2023-09-27T23:29:54.156481+0000","last_undegraded":"2023-09-27T23:29:54.156481+0000","last_fullsized":"2023-09-27T23:29:54.156481+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:58:19.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155946+0000","last_change":"2023-09-27T23:29:54.150094+0000","last_active":"2023-09-27T23:29:54.155946+0000","last_peered":"2023-09-27T23:29:54.155946+0000","last_clean":"2023-09-27T23:29:54.155946+0000","last_became_active":"2023-09-27T23:29:54.149800+0000","last_became_peered":"2023-09-27T23:29:54.149800+0000","last_unstale":"2023-09-27T23:29:54.155946+0000","last_undegraded":"2023-09-27T23:29:54.155946+0000","last_fullsized":"2023-09-27T23:29:54.155946+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:46:49.118154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.154230+0000","last_change":"2023-09-27T23:29:54.149182+0000","last_active":"2023-09-27T23:29:54.154230+0000","last_peered":"2023-09-27T23:29:54.154230+0000","last_clean":"2023-09-27T23:29:54.154230+0000","last_became_active":"2023-09-27T23:29:54.148885+0000","last_became_peered":"2023-09-27T23:29:54.148885+0000","last_unstale":"2023-09-27T23:29:54.154230+0000","last_undegraded":"2023-09-27T23:29:54.154230+0000","last_fullsized":"2023-09-27T23:29:54.154230+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:56:46.830551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.148704+0000","last_change":"2023-09-27T23:29:54.145945+0000","last_active":"2023-09-27T23:29:54.148704+0000","last_peered":"2023-09-27T23:29:54.148704+0000","last_clean":"2023-09-27T23:29:54.148704+0000","last_became_active":"2023-09-27T23:29:54.145660+0000","last_became_peered":"2023-09-27T23:29:54.145660+0000","last_unstale":"2023-09-27T23:29:54.148704+0000","last_undegraded":"2023-09-27T23:29:54.148704+0000","last_fullsized":"2023-09-27T23:29:54.148704+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:47.159474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.154782+0000","last_change":"2023-09-27T23:29:54.147946+0000","last_active":"2023-09-27T23:29:54.154782+0000","last_peered":"2023-09-27T23:29:54.154782+0000","last_clean":"2023-09-27T23:29:54.154782+0000","last_became_active":"2023-09-27T23:29:54.147652+0000","last_became_peered":"2023-09-27T23:29:54.147652+0000","last_unstale":"2023-09-27T23:29:54.154782+0000","last_undegraded":"2023-09-27T23:29:54.154782+0000","last_fullsized":"2023-09-27T23:29:54.154782+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:20:47.395911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.140718+0000","last_change":"2023-09-27T23:29:54.133589+0000","last_active":"2023-09-27T23:29:54.140718+0000","last_peered":"2023-09-27T23:29:54.140718+0000","last_clean":"2023-09-27T23:29:54.140718+0000","last_became_active":"2023-09-27T23:29:54.133208+0000","last_became_peered":"2023-09-27T23:29:54.133208+0000","last_unstale":"2023-09-27T23:29:54.140718+0000","last_undegraded":"2023-09-27T23:29:54.140718+0000","last_fullsized":"2023-09-27T23:29:54.140718+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:23.746629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145554+0000","last_change":"2023-09-27T23:29:54.139474+0000","last_active":"2023-09-27T23:29:54.145554+0000","last_peered":"2023-09-27T23:29:54.145554+0000","last_clean":"2023-09-27T23:29:54.145554+0000","last_became_active":"2023-09-27T23:29:54.139204+0000","last_became_peered":"2023-09-27T23:29:54.139204+0000","last_unstale":"2023-09-27T23:29:54.145554+0000","last_undegraded":"2023-09-27T23:29:54.145554+0000","last_fullsized":"2023-09-27T23:29:54.145554+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:09:54.976064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.150338+0000","last_change":"2023-09-27T23:29:54.147347+0000","last_active":"2023-09-27T23:29:54.150338+0000","last_peered":"2023-09-27T23:29:54.150338+0000","last_clean":"2023-09-27T23:29:54.150338+0000","last_became_active":"2023-09-27T23:29:54.147004+0000","last_became_peered":"2023-09-27T23:29:54.147004+0000","last_unstale":"2023-09-27T23:29:54.150338+0000","last_undegraded":"2023-09-27T23:29:54.150338+0000","last_fullsized":"2023-09-27T23:29:54.150338+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:41:01.768737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146559+0000","last_change":"2023-09-27T23:29:54.141416+0000","last_active":"2023-09-27T23:29:54.146559+0000","last_peered":"2023-09-27T23:29:54.146559+0000","last_clean":"2023-09-27T23:29:54.146559+0000","last_became_active":"2023-09-27T23:29:54.141141+0000","last_became_peered":"2023-09-27T23:29:54.141141+0000","last_unstale":"2023-09-27T23:29:54.146559+0000","last_undegraded":"2023-09-27T23:29:54.146559+0000","last_fullsized":"2023-09-27T23:29:54.146559+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:20.350095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153810+0000","last_change":"2023-09-27T23:29:54.148080+0000","last_active":"2023-09-27T23:29:54.153810+0000","last_peered":"2023-09-27T23:29:54.153810+0000","last_clean":"2023-09-27T23:29:54.153810+0000","last_became_active":"2023-09-27T23:29:54.146099+0000","last_became_peered":"2023-09-27T23:29:54.146099+0000","last_unstale":"2023-09-27T23:29:54.153810+0000","last_undegraded":"2023-09-27T23:29:54.153810+0000","last_fullsized":"2023-09-27T23:29:54.153810+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:44:23.414880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153381+0000","last_change":"2023-09-27T23:29:54.135482+0000","last_active":"2023-09-27T23:29:54.153381+0000","last_peered":"2023-09-27T23:29:54.153381+0000","last_clean":"2023-09-27T23:29:54.153381+0000","last_became_active":"2023-09-27T23:29:54.135216+0000","last_became_peered":"2023-09-27T23:29:54.135216+0000","last_unstale":"2023-09-27T23:29:54.153381+0000","last_undegraded":"2023-09-27T23:29:54.153381+0000","last_fullsized":"2023-09-27T23:29:54.153381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:59:58.366862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153905+0000","last_change":"2023-09-27T23:29:54.148767+0000","last_active":"2023-09-27T23:29:54.153905+0000","last_peered":"2023-09-27T23:29:54.153905+0000","last_clean":"2023-09-27T23:29:54.153905+0000","last_became_active":"2023-09-27T23:29:54.148443+0000","last_became_peered":"2023-09-27T23:29:54.148443+0000","last_unstale":"2023-09-27T23:29:54.153905+0000","last_undegraded":"2023-09-27T23:29:54.153905+0000","last_fullsized":"2023-09-27T23:29:54.153905+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:35.380158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.143726+0000","last_change":"2023-09-27T23:29:54.136988+0000","last_active":"2023-09-27T23:29:54.143726+0000","last_peered":"2023-09-27T23:29:54.143726+0000","last_clean":"2023-09-27T23:29:54.143726+0000","last_became_active":"2023-09-27T23:29:54.136662+0000","last_became_peered":"2023-09-27T23:29:54.136662+0000","last_unstale":"2023-09-27T23:29:54.143726+0000","last_undegraded":"2023-09-27T23:29:54.143726+0000","last_fullsized":"2023-09-27T23:29:54.143726+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:51:27.097027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149543+0000","last_change":"2023-09-27T23:29:54.147371+0000","last_active":"2023-09-27T23:29:54.149543+0000","last_peered":"2023-09-27T23:29:54.149543+0000","last_clean":"2023-09-27T23:29:54.149543+0000","last_became_active":"2023-09-27T23:29:54.146786+0000","last_became_peered":"2023-09-27T23:29:54.146786+0000","last_unstale":"2023-09-27T23:29:54.149543+0000","last_undegraded":"2023-09-27T23:29:54.149543+0000","last_fullsized":"2023-09-27T23:29:54.149543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:34:51.507706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.143838+0000","last_change":"2023-09-27T23:29:54.139672+0000","last_active":"2023-09-27T23:29:54.143838+0000","last_peered":"2023-09-27T23:29:54.143838+0000","last_clean":"2023-09-27T23:29:54.143838+0000","last_became_active":"2023-09-27T23:29:54.139409+0000","last_became_peered":"2023-09-27T23:29:54.139409+0000","last_unstale":"2023-09-27T23:29:54.143838+0000","last_undegraded":"2023-09-27T23:29:54.143838+0000","last_fullsized":"2023-09-27T23:29:54.143838+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:13:29.771348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153977+0000","last_change":"2023-09-27T23:29:54.149487+0000","last_active":"2023-09-27T23:29:54.153977+0000","last_peered":"2023-09-27T23:29:54.153977+0000","last_clean":"2023-09-27T23:29:54.153977+0000","last_became_active":"2023-09-27T23:29:54.149178+0000","last_became_peered":"2023-09-27T23:29:54.149178+0000","last_unstale":"2023-09-27T23:29:54.153977+0000","last_undegraded":"2023-09-27T23:29:54.153977+0000","last_fullsized":"2023-09-27T23:29:54.153977+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:04:37.887946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146618+0000","last_change":"2023-09-27T23:29:54.140540+0000","last_active":"2023-09-27T23:29:54.146618+0000","last_peered":"2023-09-27T23:29:54.146618+0000","last_clean":"2023-09-27T23:29:54.146618+0000","last_became_active":"2023-09-27T23:29:54.140254+0000","last_became_peered":"2023-09-27T23:29:54.140254+0000","last_unstale":"2023-09-27T23:29:54.146618+0000","last_undegraded":"2023-09-27T23:29:54.146618+0000","last_fullsized":"2023-09-27T23:29:54.146618+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:17:08.506090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153647+0000","last_change":"2023-09-27T23:29:54.151837+0000","last_active":"2023-09-27T23:29:54.153647+0000","last_peered":"2023-09-27T23:29:54.153647+0000","last_clean":"2023-09-27T23:29:54.153647+0000","last_became_active":"2023-09-27T23:29:54.151587+0000","last_became_peered":"2023-09-27T23:29:54.151587+0000","last_unstale":"2023-09-27T23:29:54.153647+0000","last_undegraded":"2023-09-27T23:29:54.153647+0000","last_fullsized":"2023-09-27T23:29:54.153647+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:28.711624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146060+0000","last_change":"2023-09-27T23:29:54.137566+0000","last_active":"2023-09-27T23:29:54.146060+0000","last_peered":"2023-09-27T23:29:54.146060+0000","last_clean":"2023-09-27T23:29:54.146060+0000","last_became_active":"2023-09-27T23:29:54.137296+0000","last_became_peered":"2023-09-27T23:29:54.137296+0000","last_unstale":"2023-09-27T23:29:54.146060+0000","last_undegraded":"2023-09-27T23:29:54.146060+0000","last_fullsized":"2023-09-27T23:29:54.146060+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:58:19.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146058+0000","last_change":"2023-09-27T23:29:54.137918+0000","last_active":"2023-09-27T23:29:54.146058+0000","last_peered":"2023-09-27T23:29:54.146058+0000","last_clean":"2023-09-27T23:29:54.146058+0000","last_became_active":"2023-09-27T23:29:54.137403+0000","last_became_peered":"2023-09-27T23:29:54.137403+0000","last_unstale":"2023-09-27T23:29:54.146058+0000","last_undegraded":"2023-09-27T23:29:54.146058+0000","last_fullsized":"2023-09-27T23:29:54.146058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:19:59.273479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145734+0000","last_change":"2023-09-27T23:29:54.141094+0000","last_active":"2023-09-27T23:29:54.145734+0000","last_peered":"2023-09-27T23:29:54.145734+0000","last_clean":"2023-09-27T23:29:54.145734+0000","last_became_active":"2023-09-27T23:29:54.140830+0000","last_became_peered":"2023-09-27T23:29:54.140830+0000","last_unstale":"2023-09-27T23:29:54.145734+0000","last_undegraded":"2023-09-27T23:29:54.145734+0000","last_fullsized":"2023-09-27T23:29:54.145734+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:48.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153505+0000","last_change":"2023-09-27T23:29:54.140582+0000","last_active":"2023-09-27T23:29:54.153505+0000","last_peered":"2023-09-27T23:29:54.153505+0000","last_clean":"2023-09-27T23:29:54.153505+0000","last_became_active":"2023-09-27T23:29:54.140028+0000","last_became_peered":"2023-09-27T23:29:54.140028+0000","last_unstale":"2023-09-27T23:29:54.153505+0000","last_undegraded":"2023-09-27T23:29:54.153505+0000","last_fullsized":"2023-09-27T23:29:54.153505+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:33:32.376150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.154015+0000","last_change":"2023-09-27T23:29:54.148890+0000","last_active":"2023-09-27T23:29:54.154015+0000","last_peered":"2023-09-27T23:29:54.154015+0000","last_clean":"2023-09-27T23:29:54.154015+0000","last_became_active":"2023-09-27T23:29:54.148546+0000","last_became_peered":"2023-09-27T23:29:54.148546+0000","last_unstale":"2023-09-27T23:29:54.154015+0000","last_undegraded":"2023-09-27T23:29:54.154015+0000","last_fullsized":"2023-09-27T23:29:54.154015+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:39:10.374845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.147079+0000","last_change":"2023-09-27T23:29:54.144027+0000","last_active":"2023-09-27T23:29:54.147079+0000","last_peered":"2023-09-27T23:29:54.147079+0000","last_clean":"2023-09-27T23:29:54.147079+0000","last_became_active":"2023-09-27T23:29:54.143684+0000","last_became_peered":"2023-09-27T23:29:54.143684+0000","last_unstale":"2023-09-27T23:29:54.147079+0000","last_undegraded":"2023-09-27T23:29:54.147079+0000","last_fullsized":"2023-09-27T23:29:54.147079+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:22:41.890804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155992+0000","last_change":"2023-09-27T23:29:54.150388+0000","last_active":"2023-09-27T23:29:54.155992+0000","last_peered":"2023-09-27T23:29:54.155992+0000","last_clean":"2023-09-27T23:29:54.155992+0000","last_became_active":"2023-09-27T23:29:54.149770+0000","last_became_peered":"2023-09-27T23:29:54.149770+0000","last_unstale":"2023-09-27T23:29:54.155992+0000","last_undegraded":"2023-09-27T23:29:54.155992+0000","last_fullsized":"2023-09-27T23:29:54.155992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:56:05.113048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.151106+0000","last_change":"2023-09-27T23:29:54.148716+0000","last_active":"2023-09-27T23:29:54.151106+0000","last_peered":"2023-09-27T23:29:54.151106+0000","last_clean":"2023-09-27T23:29:54.151106+0000","last_became_active":"2023-09-27T23:29:54.148390+0000","last_became_peered":"2023-09-27T23:29:54.148390+0000","last_unstale":"2023-09-27T23:29:54.151106+0000","last_undegraded":"2023-09-27T23:29:54.151106+0000","last_fullsized":"2023-09-27T23:29:54.151106+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:31:09.048038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146655+0000","last_change":"2023-09-27T23:29:54.140860+0000","last_active":"2023-09-27T23:29:54.146655+0000","last_peered":"2023-09-27T23:29:54.146655+0000","last_clean":"2023-09-27T23:29:54.146655+0000","last_became_active":"2023-09-27T23:29:54.139853+0000","last_became_peered":"2023-09-27T23:29:54.139853+0000","last_unstale":"2023-09-27T23:29:54.146655+0000","last_undegraded":"2023-09-27T23:29:54.146655+0000","last_fullsized":"2023-09-27T23:29:54.146655+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:29:43.811616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153871+0000","last_change":"2023-09-27T23:29:54.148237+0000","last_active":"2023-09-27T23:29:54.153871+0000","last_peered":"2023-09-27T23:29:54.153871+0000","last_clean":"2023-09-27T23:29:54.153871+0000","last_became_active":"2023-09-27T23:29:54.146095+0000","last_became_peered":"2023-09-27T23:29:54.146095+0000","last_unstale":"2023-09-27T23:29:54.153871+0000","last_undegraded":"2023-09-27T23:29:54.153871+0000","last_fullsized":"2023-09-27T23:29:54.153871+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:00:47.206311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155627+0000","last_change":"2023-09-27T23:29:54.145257+0000","last_active":"2023-09-27T23:29:54.155627+0000","last_peered":"2023-09-27T23:29:54.155627+0000","last_clean":"2023-09-27T23:29:54.155627+0000","last_became_active":"2023-09-27T23:29:54.144050+0000","last_became_peered":"2023-09-27T23:29:54.144050+0000","last_unstale":"2023-09-27T23:29:54.155627+0000","last_undegraded":"2023-09-27T23:29:54.155627+0000","last_fullsized":"2023-09-27T23:29:54.155627+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:33:41.929037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155604+0000","last_change":"2023-09-27T23:29:54.144970+0000","last_active":"2023-09-27T23:29:54.155604+0000","last_peered":"2023-09-27T23:29:54.155604+0000","last_clean":"2023-09-27T23:29:54.155604+0000","last_became_active":"2023-09-27T23:29:54.144562+0000","last_became_peered":"2023-09-27T23:29:54.144562+0000","last_unstale":"2023-09-27T23:29:54.155604+0000","last_undegraded":"2023-09-27T23:29:54.155604+0000","last_fullsized":"2023-09-27T23:29:54.155604+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:25:10.674135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146742+0000","last_change":"2023-09-27T23:29:54.141580+0000","last_active":"2023-09-27T23:29:54.146742+0000","last_peered":"2023-09-27T23:29:54.146742+0000","last_clean":"2023-09-27T23:29:54.146742+0000","last_became_active":"2023-09-27T23:29:54.140936+0000","last_became_peered":"2023-09-27T23:29:54.140936+0000","last_unstale":"2023-09-27T23:29:54.146742+0000","last_undegraded":"2023-09-27T23:29:54.146742+0000","last_fullsized":"2023-09-27T23:29:54.146742+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:33:26.316674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.157139+0000","last_change":"2023-09-27T23:29:54.155924+0000","last_active":"2023-09-27T23:29:54.157139+0000","last_peered":"2023-09-27T23:29:54.157139+0000","last_clean":"2023-09-27T23:29:54.157139+0000","last_became_active":"2023-09-27T23:29:54.155574+0000","last_became_peered":"2023-09-27T23:29:54.155574+0000","last_unstale":"2023-09-27T23:29:54.157139+0000","last_undegraded":"2023-09-27T23:29:54.157139+0000","last_fullsized":"2023-09-27T23:29:54.157139+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:54:17.858339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.156385+0000","last_change":"2023-09-27T23:29:54.152947+0000","last_active":"2023-09-27T23:29:54.156385+0000","last_peered":"2023-09-27T23:29:54.156385+0000","last_clean":"2023-09-27T23:29:54.156385+0000","last_became_active":"2023-09-27T23:29:54.152681+0000","last_became_peered":"2023-09-27T23:29:54.152681+0000","last_unstale":"2023-09-27T23:29:54.156385+0000","last_undegraded":"2023-09-27T23:29:54.156385+0000","last_fullsized":"2023-09-27T23:29:54.156385+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:33:40.398773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.152989+0000","last_change":"2023-09-27T23:29:53.759866+0000","last_active":"2023-09-27T23:29:54.152989+0000","last_peered":"2023-09-27T23:29:54.152989+0000","last_clean":"2023-09-27T23:29:54.152989+0000","last_became_active":"2023-09-27T23:29:53.759565+0000","last_became_peered":"2023-09-27T23:29:53.759565+0000","last_unstale":"2023-09-27T23:29:54.152989+0000","last_undegraded":"2023-09-27T23:29:54.152989+0000","last_fullsized":"2023-09-27T23:29:54.152989+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:51:26.085474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.156822+0000","last_change":"2023-09-27T23:29:54.152791+0000","last_active":"2023-09-27T23:29:54.156822+0000","last_peered":"2023-09-27T23:29:54.156822+0000","last_clean":"2023-09-27T23:29:54.156822+0000","last_became_active":"2023-09-27T23:29:54.152532+0000","last_became_peered":"2023-09-27T23:29:54.152532+0000","last_unstale":"2023-09-27T23:29:54.156822+0000","last_undegraded":"2023-09-27T23:29:54.156822+0000","last_fullsized":"2023-09-27T23:29:54.156822+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:25:57.129780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.150837+0000","last_change":"2023-09-27T23:29:53.752670+0000","last_active":"2023-09-27T23:29:54.150837+0000","last_peered":"2023-09-27T23:29:54.150837+0000","last_clean":"2023-09-27T23:29:54.150837+0000","last_became_active":"2023-09-27T23:29:53.752115+0000","last_became_peered":"2023-09-27T23:29:53.752115+0000","last_unstale":"2023-09-27T23:29:54.150837+0000","last_undegraded":"2023-09-27T23:29:54.150837+0000","last_fullsized":"2023-09-27T23:29:54.150837+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:58:18.183120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.157159+0000","last_change":"2023-09-27T23:29:54.155928+0000","last_active":"2023-09-27T23:29:54.157159+0000","last_peered":"2023-09-27T23:29:54.157159+0000","last_clean":"2023-09-27T23:29:54.157159+0000","last_became_active":"2023-09-27T23:29:54.155578+0000","last_became_peered":"2023-09-27T23:29:54.155578+0000","last_unstale":"2023-09-27T23:29:54.157159+0000","last_undegraded":"2023-09-27T23:29:54.157159+0000","last_fullsized":"2023-09-27T23:29:54.157159+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:49:20.067550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145985+0000","last_change":"2023-09-27T23:29:53.749655+0000","last_active":"2023-09-27T23:29:54.145985+0000","last_peered":"2023-09-27T23:29:54.145985+0000","last_clean":"2023-09-27T23:29:54.145985+0000","last_became_active":"2023-09-27T23:29:53.749328+0000","last_became_peered":"2023-09-27T23:29:53.749328+0000","last_unstale":"2023-09-27T23:29:54.145985+0000","last_undegraded":"2023-09-27T23:29:54.145985+0000","last_fullsized":"2023-09-27T23:29:54.145985+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:25:56.118227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.144135+0000","last_change":"2023-09-27T23:29:54.137110+0000","last_active":"2023-09-27T23:29:54.144135+0000","last_peered":"2023-09-27T23:29:54.144135+0000","last_clean":"2023-09-27T23:29:54.144135+0000","last_became_active":"2023-09-27T23:29:54.136785+0000","last_became_peered":"2023-09-27T23:29:54.136785+0000","last_unstale":"2023-09-27T23:29:54.144135+0000","last_undegraded":"2023-09-27T23:29:54.144135+0000","last_fullsized":"2023-09-27T23:29:54.144135+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:29:59.112120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145900+0000","last_change":"2023-09-27T23:29:53.756169+0000","last_active":"2023-09-27T23:29:54.145900+0000","last_peered":"2023-09-27T23:29:54.145900+0000","last_clean":"2023-09-27T23:29:54.145900+0000","last_became_active":"2023-09-27T23:29:53.755849+0000","last_became_peered":"2023-09-27T23:29:53.755849+0000","last_unstale":"2023-09-27T23:29:54.145900+0000","last_undegraded":"2023-09-27T23:29:54.145900+0000","last_fullsized":"2023-09-27T23:29:54.145900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:00:46.194758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155556+0000","last_change":"2023-09-27T23:29:54.144248+0000","last_active":"2023-09-27T23:29:54.155556+0000","last_peered":"2023-09-27T23:29:54.155556+0000","last_clean":"2023-09-27T23:29:54.155556+0000","last_became_active":"2023-09-27T23:29:54.143925+0000","last_became_peered":"2023-09-27T23:29:54.143925+0000","last_unstale":"2023-09-27T23:29:54.155556+0000","last_undegraded":"2023-09-27T23:29:54.155556+0000","last_fullsized":"2023-09-27T23:29:54.155556+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:56:55.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155354+0000","last_change":"2023-09-27T23:29:53.761394+0000","last_active":"2023-09-27T23:29:54.155354+0000","last_peered":"2023-09-27T23:29:54.155354+0000","last_clean":"2023-09-27T23:29:54.155354+0000","last_became_active":"2023-09-27T23:29:53.761071+0000","last_became_peered":"2023-09-27T23:29:53.761071+0000","last_unstale":"2023-09-27T23:29:54.155354+0000","last_undegraded":"2023-09-27T23:29:54.155354+0000","last_fullsized":"2023-09-27T23:29:54.155354+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:24.104302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146695+0000","last_change":"2023-09-27T23:29:54.141211+0000","last_active":"2023-09-27T23:29:54.146695+0000","last_peered":"2023-09-27T23:29:54.146695+0000","last_clean":"2023-09-27T23:29:54.146695+0000","last_became_active":"2023-09-27T23:29:54.140914+0000","last_became_peered":"2023-09-27T23:29:54.140914+0000","last_unstale":"2023-09-27T23:29:54.146695+0000","last_undegraded":"2023-09-27T23:29:54.146695+0000","last_fullsized":"2023-09-27T23:29:54.146695+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:48:30.428466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.150849+0000","last_change":"2023-09-27T23:29:53.754295+0000","last_active":"2023-09-27T23:29:54.150849+0000","last_peered":"2023-09-27T23:29:54.150849+0000","last_clean":"2023-09-27T23:29:54.150849+0000","last_became_active":"2023-09-27T23:29:53.753979+0000","last_became_peered":"2023-09-27T23:29:53.753979+0000","last_unstale":"2023-09-27T23:29:54.150849+0000","last_undegraded":"2023-09-27T23:29:54.150849+0000","last_fullsized":"2023-09-27T23:29:54.150849+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:18:29.954953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.147827+0000","last_change":"2023-09-27T23:29:54.145170+0000","last_active":"2023-09-27T23:29:54.147827+0000","last_peered":"2023-09-27T23:29:54.147827+0000","last_clean":"2023-09-27T23:29:54.147827+0000","last_became_active":"2023-09-27T23:29:54.144850+0000","last_became_peered":"2023-09-27T23:29:54.144850+0000","last_unstale":"2023-09-27T23:29:54.147827+0000","last_undegraded":"2023-09-27T23:29:54.147827+0000","last_fullsized":"2023-09-27T23:29:54.147827+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:20.350095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.127384+0000","last_change":"2023-09-27T23:29:53.757455+0000","last_active":"2023-09-27T23:29:54.127384+0000","last_peered":"2023-09-27T23:29:54.127384+0000","last_clean":"2023-09-27T23:29:54.127384+0000","last_became_active":"2023-09-27T23:29:53.757165+0000","last_became_peered":"2023-09-27T23:29:53.757165+0000","last_unstale":"2023-09-27T23:29:54.127384+0000","last_undegraded":"2023-09-27T23:29:54.127384+0000","last_fullsized":"2023-09-27T23:29:54.127384+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:49:19.055997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.143820+0000","last_change":"2023-09-27T23:29:54.139573+0000","last_active":"2023-09-27T23:29:54.143820+0000","last_peered":"2023-09-27T23:29:54.143820+0000","last_clean":"2023-09-27T23:29:54.143820+0000","last_became_active":"2023-09-27T23:29:54.139280+0000","last_became_peered":"2023-09-27T23:29:54.139280+0000","last_unstale":"2023-09-27T23:29:54.143820+0000","last_undegraded":"2023-09-27T23:29:54.143820+0000","last_fullsized":"2023-09-27T23:29:54.143820+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:18:30.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.123763+0000","last_change":"2023-09-27T23:29:53.752341+0000","last_active":"2023-09-27T23:29:54.123763+0000","last_peered":"2023-09-27T23:29:54.123763+0000","last_clean":"2023-09-27T23:29:54.123763+0000","last_became_active":"2023-09-27T23:29:53.751988+0000","last_became_peered":"2023-09-27T23:29:53.751988+0000","last_unstale":"2023-09-27T23:29:54.123763+0000","last_undegraded":"2023-09-27T23:29:54.123763+0000","last_fullsized":"2023-09-27T23:29:54.123763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:58:30.658465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.152867+0000","last_change":"2023-09-27T23:29:54.145087+0000","last_active":"2023-09-27T23:29:54.152867+0000","last_peered":"2023-09-27T23:29:54.152867+0000","last_clean":"2023-09-27T23:29:54.152867+0000","last_became_active":"2023-09-27T23:29:54.144604+0000","last_became_peered":"2023-09-27T23:29:54.144604+0000","last_unstale":"2023-09-27T23:29:54.152867+0000","last_undegraded":"2023-09-27T23:29:54.152867+0000","last_fullsized":"2023-09-27T23:29:54.152867+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:25.115855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138773+0000","last_change":"2023-09-27T23:29:53.760006+0000","last_active":"2023-09-27T23:29:54.138773+0000","last_peered":"2023-09-27T23:29:54.138773+0000","last_clean":"2023-09-27T23:29:54.138773+0000","last_became_active":"2023-09-27T23:29:53.759692+0000","last_became_peered":"2023-09-27T23:29:53.759692+0000","last_unstale":"2023-09-27T23:29:54.138773+0000","last_undegraded":"2023-09-27T23:29:54.138773+0000","last_fullsized":"2023-09-27T23:29:54.138773+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:41:00.757184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.136996+0000","last_change":"2023-09-27T23:29:54.130794+0000","last_active":"2023-09-27T23:29:54.136996+0000","last_peered":"2023-09-27T23:29:54.136996+0000","last_clean":"2023-09-27T23:29:54.136996+0000","last_became_active":"2023-09-27T23:29:54.130481+0000","last_became_peered":"2023-09-27T23:29:54.130481+0000","last_unstale":"2023-09-27T23:29:54.136996+0000","last_undegraded":"2023-09-27T23:29:54.136996+0000","last_fullsized":"2023-09-27T23:29:54.136996+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:14.308578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.139131+0000","last_change":"2023-09-27T23:29:53.756366+0000","last_active":"2023-09-27T23:29:54.139131+0000","last_peered":"2023-09-27T23:29:54.139131+0000","last_clean":"2023-09-27T23:29:54.139131+0000","last_became_active":"2023-09-27T23:29:53.756080+0000","last_became_peered":"2023-09-27T23:29:53.756080+0000","last_unstale":"2023-09-27T23:29:54.139131+0000","last_undegraded":"2023-09-27T23:29:54.139131+0000","last_fullsized":"2023-09-27T23:29:54.139131+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:43:48.485177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.154087+0000","last_change":"2023-09-27T23:29:54.151357+0000","last_active":"2023-09-27T23:29:54.154087+0000","last_peered":"2023-09-27T23:29:54.154087+0000","last_clean":"2023-09-27T23:29:54.154087+0000","last_became_active":"2023-09-27T23:29:54.150976+0000","last_became_peered":"2023-09-27T23:29:54.150976+0000","last_unstale":"2023-09-27T23:29:54.154087+0000","last_undegraded":"2023-09-27T23:29:54.154087+0000","last_fullsized":"2023-09-27T23:29:54.154087+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:13:29.771348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155802+0000","last_change":"2023-09-27T23:29:53.760601+0000","last_active":"2023-09-27T23:29:54.155802+0000","last_peered":"2023-09-27T23:29:54.155802+0000","last_clean":"2023-09-27T23:29:54.155802+0000","last_became_active":"2023-09-27T23:29:53.760318+0000","last_became_peered":"2023-09-27T23:29:53.760318+0000","last_unstale":"2023-09-27T23:29:54.155802+0000","last_undegraded":"2023-09-27T23:29:54.155802+0000","last_fullsized":"2023-09-27T23:29:54.155802+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:10.798249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146674+0000","last_change":"2023-09-27T23:29:54.141069+0000","last_active":"2023-09-27T23:29:54.146674+0000","last_peered":"2023-09-27T23:29:54.146674+0000","last_clean":"2023-09-27T23:29:54.146674+0000","last_became_active":"2023-09-27T23:29:54.140223+0000","last_became_peered":"2023-09-27T23:29:54.140223+0000","last_unstale":"2023-09-27T23:29:54.146674+0000","last_undegraded":"2023-09-27T23:29:54.146674+0000","last_fullsized":"2023-09-27T23:29:54.146674+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:07.979895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.157964+0000","last_change":"2023-09-27T23:29:53.762507+0000","last_active":"2023-09-27T23:29:54.157964+0000","last_peered":"2023-09-27T23:29:54.157964+0000","last_clean":"2023-09-27T23:29:54.157964+0000","last_became_active":"2023-09-27T23:29:53.762159+0000","last_became_peered":"2023-09-27T23:29:53.762159+0000","last_unstale":"2023-09-27T23:29:54.157964+0000","last_undegraded":"2023-09-27T23:29:54.157964+0000","last_fullsized":"2023-09-27T23:29:54.157964+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:21.873508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.144126+0000","last_change":"2023-09-27T23:29:54.136962+0000","last_active":"2023-09-27T23:29:54.144126+0000","last_peered":"2023-09-27T23:29:54.144126+0000","last_clean":"2023-09-27T23:29:54.144126+0000","last_became_active":"2023-09-27T23:29:54.136534+0000","last_became_peered":"2023-09-27T23:29:54.136534+0000","last_unstale":"2023-09-27T23:29:54.144126+0000","last_undegraded":"2023-09-27T23:29:54.144126+0000","last_fullsized":"2023-09-27T23:29:54.144126+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:18:30.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153253+0000","last_change":"2023-09-27T23:29:53.751684+0000","last_active":"2023-09-27T23:29:54.153253+0000","last_peered":"2023-09-27T23:29:54.153253+0000","last_clean":"2023-09-27T23:29:54.153253+0000","last_became_active":"2023-09-27T23:29:53.751392+0000","last_became_peered":"2023-09-27T23:29:53.751392+0000","last_unstale":"2023-09-27T23:29:54.153253+0000","last_undegraded":"2023-09-27T23:29:54.153253+0000","last_fullsized":"2023-09-27T23:29:54.153253+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:15:38.526053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145221+0000","last_change":"2023-09-27T23:29:54.143257+0000","last_active":"2023-09-27T23:29:54.145221+0000","last_peered":"2023-09-27T23:29:54.145221+0000","last_clean":"2023-09-27T23:29:54.145221+0000","last_became_active":"2023-09-27T23:29:54.142912+0000","last_became_peered":"2023-09-27T23:29:54.142912+0000","last_unstale":"2023-09-27T23:29:54.145221+0000","last_undegraded":"2023-09-27T23:29:54.145221+0000","last_fullsized":"2023-09-27T23:29:54.145221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:21:33.823036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.121030+0000","last_change":"2023-09-27T23:29:53.752602+0000","last_active":"2023-09-27T23:29:54.121030+0000","last_peered":"2023-09-27T23:29:54.121030+0000","last_clean":"2023-09-27T23:29:54.121030+0000","last_became_active":"2023-09-27T23:29:53.752213+0000","last_became_peered":"2023-09-27T23:29:53.752213+0000","last_unstale":"2023-09-27T23:29:54.121030+0000","last_undegraded":"2023-09-27T23:29:54.121030+0000","last_fullsized":"2023-09-27T23:29:54.121030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:54:16.846786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155567+0000","last_change":"2023-09-27T23:29:54.144387+0000","last_active":"2023-09-27T23:29:54.155567+0000","last_peered":"2023-09-27T23:29:54.155567+0000","last_clean":"2023-09-27T23:29:54.155567+0000","last_became_active":"2023-09-27T23:29:54.144075+0000","last_became_peered":"2023-09-27T23:29:54.144075+0000","last_unstale":"2023-09-27T23:29:54.155567+0000","last_undegraded":"2023-09-27T23:29:54.155567+0000","last_fullsized":"2023-09-27T23:29:54.155567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.634916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.124688+0000","last_change":"2023-09-27T23:29:53.763637+0000","last_active":"2023-09-27T23:29:54.124688+0000","last_peered":"2023-09-27T23:29:54.124688+0000","last_clean":"2023-09-27T23:29:54.124688+0000","last_became_active":"2023-09-27T23:29:53.763337+0000","last_became_peered":"2023-09-27T23:29:53.763337+0000","last_unstale":"2023-09-27T23:29:54.124688+0000","last_undegraded":"2023-09-27T23:29:54.124688+0000","last_fullsized":"2023-09-27T23:29:54.124688+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:46.147921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.155915+0000","last_change":"2023-09-27T23:29:54.150045+0000","last_active":"2023-09-27T23:29:54.155915+0000","last_peered":"2023-09-27T23:29:54.155915+0000","last_clean":"2023-09-27T23:29:54.155915+0000","last_became_active":"2023-09-27T23:29:54.148068+0000","last_became_peered":"2023-09-27T23:29:54.148068+0000","last_unstale":"2023-09-27T23:29:54.155915+0000","last_undegraded":"2023-09-27T23:29:54.155915+0000","last_fullsized":"2023-09-27T23:29:54.155915+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:20:25.592904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.150844+0000","last_change":"2023-09-27T23:29:53.757161+0000","last_active":"2023-09-27T23:29:54.150844+0000","last_peered":"2023-09-27T23:29:54.150844+0000","last_clean":"2023-09-27T23:29:54.150844+0000","last_became_active":"2023-09-27T23:29:53.756699+0000","last_became_peered":"2023-09-27T23:29:53.756699+0000","last_unstale":"2023-09-27T23:29:54.150844+0000","last_undegraded":"2023-09-27T23:29:54.150844+0000","last_fullsized":"2023-09-27T23:29:54.150844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:42:05.066017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.136800+0000","last_change":"2023-09-27T23:29:54.133639+0000","last_active":"2023-09-27T23:29:54.136800+0000","last_peered":"2023-09-27T23:29:54.136800+0000","last_clean":"2023-09-27T23:29:54.136800+0000","last_became_active":"2023-09-27T23:29:54.133348+0000","last_became_peered":"2023-09-27T23:29:54.133348+0000","last_unstale":"2023-09-27T23:29:54.136800+0000","last_undegraded":"2023-09-27T23:29:54.136800+0000","last_fullsized":"2023-09-27T23:29:54.136800+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:36:42.184816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.144111+0000","last_change":"2023-09-27T23:29:53.758552+0000","last_active":"2023-09-27T23:29:54.144111+0000","last_peered":"2023-09-27T23:29:54.144111+0000","last_clean":"2023-09-27T23:29:54.144111+0000","last_became_active":"2023-09-27T23:29:53.758245+0000","last_became_peered":"2023-09-27T23:29:53.758245+0000","last_unstale":"2023-09-27T23:29:54.144111+0000","last_undegraded":"2023-09-27T23:29:54.144111+0000","last_fullsized":"2023-09-27T23:29:54.144111+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:35:44.784156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.156346+0000","last_change":"2023-09-27T23:29:54.153086+0000","last_active":"2023-09-27T23:29:54.156346+0000","last_peered":"2023-09-27T23:29:54.156346+0000","last_clean":"2023-09-27T23:29:54.156346+0000","last_became_active":"2023-09-27T23:29:54.152436+0000","last_became_peered":"2023-09-27T23:29:54.152436+0000","last_unstale":"2023-09-27T23:29:54.156346+0000","last_undegraded":"2023-09-27T23:29:54.156346+0000","last_fullsized":"2023-09-27T23:29:54.156346+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:19.445063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.121937+0000","last_change":"2023-09-27T23:29:53.754076+0000","last_active":"2023-09-27T23:29:54.121937+0000","last_peered":"2023-09-27T23:29:54.121937+0000","last_clean":"2023-09-27T23:29:54.121937+0000","last_became_active":"2023-09-27T23:29:53.753710+0000","last_became_peered":"2023-09-27T23:29:53.753710+0000","last_unstale":"2023-09-27T23:29:54.121937+0000","last_undegraded":"2023-09-27T23:29:54.121937+0000","last_fullsized":"2023-09-27T23:29:54.121937+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:39:26.040742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.152333+0000","last_change":"2023-09-27T23:29:54.143748+0000","last_active":"2023-09-27T23:29:54.152333+0000","last_peered":"2023-09-27T23:29:54.152333+0000","last_clean":"2023-09-27T23:29:54.152333+0000","last_became_active":"2023-09-27T23:29:54.143352+0000","last_became_peered":"2023-09-27T23:29:54.143352+0000","last_unstale":"2023-09-27T23:29:54.152333+0000","last_undegraded":"2023-09-27T23:29:54.152333+0000","last_fullsized":"2023-09-27T23:29:54.152333+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:52.090157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.140485+0000","last_change":"2023-09-27T23:29:53.756792+0000","last_active":"2023-09-27T23:29:54.140485+0000","last_peered":"2023-09-27T23:29:54.140485+0000","last_clean":"2023-09-27T23:29:54.140485+0000","last_became_active":"2023-09-27T23:29:53.756480+0000","last_became_peered":"2023-09-27T23:29:53.756480+0000","last_unstale":"2023-09-27T23:29:54.140485+0000","last_undegraded":"2023-09-27T23:29:54.140485+0000","last_fullsized":"2023-09-27T23:29:54.140485+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:22.735076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146631+0000","last_change":"2023-09-27T23:29:54.140669+0000","last_active":"2023-09-27T23:29:54.146631+0000","last_peered":"2023-09-27T23:29:54.146631+0000","last_clean":"2023-09-27T23:29:54.146631+0000","last_became_active":"2023-09-27T23:29:54.139486+0000","last_became_peered":"2023-09-27T23:29:54.139486+0000","last_unstale":"2023-09-27T23:29:54.146631+0000","last_undegraded":"2023-09-27T23:29:54.146631+0000","last_fullsized":"2023-09-27T23:29:54.146631+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:02:32.827320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138961+0000","last_change":"2023-09-27T23:29:53.752694+0000","last_active":"2023-09-27T23:29:54.138961+0000","last_peered":"2023-09-27T23:29:54.138961+0000","last_clean":"2023-09-27T23:29:54.138961+0000","last_became_active":"2023-09-27T23:29:53.752424+0000","last_became_peered":"2023-09-27T23:29:53.752424+0000","last_unstale":"2023-09-27T23:29:54.138961+0000","last_undegraded":"2023-09-27T23:29:54.138961+0000","last_fullsized":"2023-09-27T23:29:54.138961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:19.338542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146443+0000","last_change":"2023-09-27T23:29:54.139650+0000","last_active":"2023-09-27T23:29:54.146443+0000","last_peered":"2023-09-27T23:29:54.146443+0000","last_clean":"2023-09-27T23:29:54.146443+0000","last_became_active":"2023-09-27T23:29:54.139363+0000","last_became_peered":"2023-09-27T23:29:54.139363+0000","last_unstale":"2023-09-27T23:29:54.146443+0000","last_undegraded":"2023-09-27T23:29:54.146443+0000","last_fullsized":"2023-09-27T23:29:54.146443+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:04.293592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.129768+0000","last_change":"2023-09-27T23:29:53.757054+0000","last_active":"2023-09-27T23:29:54.129768+0000","last_peered":"2023-09-27T23:29:54.129768+0000","last_clean":"2023-09-27T23:29:54.129768+0000","last_became_active":"2023-09-27T23:29:53.756721+0000","last_became_peered":"2023-09-27T23:29:53.756721+0000","last_unstale":"2023-09-27T23:29:54.129768+0000","last_undegraded":"2023-09-27T23:29:54.129768+0000","last_fullsized":"2023-09-27T23:29:54.129768+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:58.548462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.150331+0000","last_change":"2023-09-27T23:29:54.140437+0000","last_active":"2023-09-27T23:29:54.150331+0000","last_peered":"2023-09-27T23:29:54.150331+0000","last_clean":"2023-09-27T23:29:54.150331+0000","last_became_active":"2023-09-27T23:29:54.140050+0000","last_became_peered":"2023-09-27T23:29:54.140050+0000","last_unstale":"2023-09-27T23:29:54.150331+0000","last_undegraded":"2023-09-27T23:29:54.150331+0000","last_fullsized":"2023-09-27T23:29:54.150331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:25.115855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.129100+0000","last_change":"2023-09-27T23:29:53.752553+0000","last_active":"2023-09-27T23:29:54.129100+0000","last_peered":"2023-09-27T23:29:54.129100+0000","last_clean":"2023-09-27T23:29:54.129100+0000","last_became_active":"2023-09-27T23:29:53.752226+0000","last_became_peered":"2023-09-27T23:29:53.752226+0000","last_unstale":"2023-09-27T23:29:54.129100+0000","last_undegraded":"2023-09-27T23:29:54.129100+0000","last_fullsized":"2023-09-27T23:29:54.129100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:16:52.337711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.151111+0000","last_change":"2023-09-27T23:29:54.148731+0000","last_active":"2023-09-27T23:29:54.151111+0000","last_peered":"2023-09-27T23:29:54.151111+0000","last_clean":"2023-09-27T23:29:54.151111+0000","last_became_active":"2023-09-27T23:29:54.148347+0000","last_became_peered":"2023-09-27T23:29:54.148347+0000","last_unstale":"2023-09-27T23:29:54.151111+0000","last_undegraded":"2023-09-27T23:29:54.151111+0000","last_fullsized":"2023-09-27T23:29:54.151111+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:23.746629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138875+0000","last_change":"2023-09-27T23:29:53.754388+0000","last_active":"2023-09-27T23:29:54.138875+0000","last_peered":"2023-09-27T23:29:54.138875+0000","last_clean":"2023-09-27T23:29:54.138875+0000","last_became_active":"2023-09-27T23:29:53.753463+0000","last_became_peered":"2023-09-27T23:29:53.753463+0000","last_unstale":"2023-09-27T23:29:54.138875+0000","last_undegraded":"2023-09-27T23:29:54.138875+0000","last_fullsized":"2023-09-27T23:29:54.138875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:03.282039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153804+0000","last_change":"2023-09-27T23:29:54.148058+0000","last_active":"2023-09-27T23:29:54.153804+0000","last_peered":"2023-09-27T23:29:54.153804+0000","last_clean":"2023-09-27T23:29:54.153804+0000","last_became_active":"2023-09-27T23:29:54.146065+0000","last_became_peered":"2023-09-27T23:29:54.146065+0000","last_unstale":"2023-09-27T23:29:54.153804+0000","last_undegraded":"2023-09-27T23:29:54.153804+0000","last_fullsized":"2023-09-27T23:29:54.153804+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:16:53.349264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.128940+0000","last_change":"2023-09-27T23:29:53.753553+0000","last_active":"2023-09-27T23:29:54.128940+0000","last_peered":"2023-09-27T23:29:54.128940+0000","last_clean":"2023-09-27T23:29:54.128940+0000","last_became_active":"2023-09-27T23:29:53.753217+0000","last_became_peered":"2023-09-27T23:29:53.753217+0000","last_unstale":"2023-09-27T23:29:54.128940+0000","last_undegraded":"2023-09-27T23:29:54.128940+0000","last_fullsized":"2023-09-27T23:29:54.128940+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:13:28.759795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.153202+0000","last_change":"2023-09-27T23:29:54.132406+0000","last_active":"2023-09-27T23:29:54.153202+0000","last_peered":"2023-09-27T23:29:54.153202+0000","last_clean":"2023-09-27T23:29:54.153202+0000","last_became_active":"2023-09-27T23:29:54.132159+0000","last_became_peered":"2023-09-27T23:29:54.132159+0000","last_unstale":"2023-09-27T23:29:54.153202+0000","last_undegraded":"2023-09-27T23:29:54.153202+0000","last_fullsized":"2023-09-27T23:29:54.153202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:10:42.075356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.148328+0000","last_change":"2023-09-27T23:29:53.767697+0000","last_active":"2023-09-27T23:29:54.148328+0000","last_peered":"2023-09-27T23:29:54.148328+0000","last_clean":"2023-09-27T23:29:54.148328+0000","last_became_active":"2023-09-27T23:29:53.767416+0000","last_became_peered":"2023-09-27T23:29:53.767416+0000","last_unstale":"2023-09-27T23:29:54.148328+0000","last_undegraded":"2023-09-27T23:29:54.148328+0000","last_fullsized":"2023-09-27T23:29:54.148328+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:47.665741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"89'183","reported_seq":471,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.127320+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:29:54.127320+0000","last_peered":"2023-09-27T23:29:54.127320+0000","last_clean":"2023-09-27T23:29:54.127320+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:29:54.127320+0000","last_undegraded":"2023-09-27T23:29:54.127320+0000","last_fullsized":"2023-09-27T23:29:54.127320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":183,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":87,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.124678+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:29:54.124678+0000","last_peered":"2023-09-27T23:29:54.124678+0000","last_clean":"2023-09-27T23:29:54.124678+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:29:54.124678+0000","last_undegraded":"2023-09-27T23:29:54.124678+0000","last_fullsized":"2023-09-27T23:29:54.124678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.156505+0000","last_change":"2023-09-27T23:29:54.150256+0000","last_active":"2023-09-27T23:29:54.156505+0000","last_peered":"2023-09-27T23:29:54.156505+0000","last_clean":"2023-09-27T23:29:54.156505+0000","last_became_active":"2023-09-27T23:29:54.149966+0000","last_became_peered":"2023-09-27T23:29:54.149966+0000","last_unstale":"2023-09-27T23:29:54.156505+0000","last_undegraded":"2023-09-27T23:29:54.156505+0000","last_fullsized":"2023-09-27T23:29:54.156505+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:18:30.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.145138+0000","last_change":"2023-09-27T23:29:53.755820+0000","last_active":"2023-09-27T23:29:54.145138+0000","last_peered":"2023-09-27T23:29:54.145138+0000","last_clean":"2023-09-27T23:29:54.145138+0000","last_became_active":"2023-09-27T23:29:53.755517+0000","last_became_peered":"2023-09-27T23:29:53.755517+0000","last_unstale":"2023-09-27T23:29:54.145138+0000","last_undegraded":"2023-09-27T23:29:54.145138+0000","last_fullsized":"2023-09-27T23:29:54.145138+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:51.204327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.127256+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:29:54.127256+0000","last_peered":"2023-09-27T23:29:54.127256+0000","last_clean":"2023-09-27T23:29:54.127256+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:29:54.127256+0000","last_undegraded":"2023-09-27T23:29:54.127256+0000","last_fullsized":"2023-09-27T23:29:54.127256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.144227+0000","last_change":"2023-09-27T23:29:54.138058+0000","last_active":"2023-09-27T23:29:54.144227+0000","last_peered":"2023-09-27T23:29:54.144227+0000","last_clean":"2023-09-27T23:29:54.144227+0000","last_became_active":"2023-09-27T23:29:54.137661+0000","last_became_peered":"2023-09-27T23:29:54.137661+0000","last_unstale":"2023-09-27T23:29:54.144227+0000","last_undegraded":"2023-09-27T23:29:54.144227+0000","last_fullsized":"2023-09-27T23:29:54.144227+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:52:50.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.152940+0000","last_change":"2023-09-27T23:29:53.762832+0000","last_active":"2023-09-27T23:29:54.152940+0000","last_peered":"2023-09-27T23:29:54.152940+0000","last_clean":"2023-09-27T23:29:54.152940+0000","last_became_active":"2023-09-27T23:29:53.762544+0000","last_became_peered":"2023-09-27T23:29:53.762544+0000","last_unstale":"2023-09-27T23:29:54.152940+0000","last_undegraded":"2023-09-27T23:29:54.152940+0000","last_fullsized":"2023-09-27T23:29:54.152940+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:24:05.456240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117910+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:29:54.117910+0000","last_peered":"2023-09-27T23:29:54.117910+0000","last_clean":"2023-09-27T23:29:54.117910+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:29:54.117910+0000","last_undegraded":"2023-09-27T23:29:54.117910+0000","last_fullsized":"2023-09-27T23:29:54.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.150952+0000","last_change":"2023-09-27T23:29:54.147316+0000","last_active":"2023-09-27T23:29:54.150952+0000","last_peered":"2023-09-27T23:29:54.150952+0000","last_clean":"2023-09-27T23:29:54.150952+0000","last_became_active":"2023-09-27T23:29:54.147025+0000","last_became_peered":"2023-09-27T23:29:54.147025+0000","last_unstale":"2023-09-27T23:29:54.150952+0000","last_undegraded":"2023-09-27T23:29:54.150952+0000","last_fullsized":"2023-09-27T23:29:54.150952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:13.746544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.130961+0000","last_change":"2023-09-27T23:29:53.758409+0000","last_active":"2023-09-27T23:29:54.130961+0000","last_peered":"2023-09-27T23:29:54.130961+0000","last_clean":"2023-09-27T23:29:54.130961+0000","last_became_active":"2023-09-27T23:29:53.757173+0000","last_became_peered":"2023-09-27T23:29:53.757173+0000","last_unstale":"2023-09-27T23:29:54.130961+0000","last_undegraded":"2023-09-27T23:29:54.130961+0000","last_fullsized":"2023-09-27T23:29:54.130961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:15:02.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":87,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.124045+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:29:54.124045+0000","last_peered":"2023-09-27T23:29:54.124045+0000","last_clean":"2023-09-27T23:29:54.124045+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:29:54.124045+0000","last_undegraded":"2023-09-27T23:29:54.124045+0000","last_fullsized":"2023-09-27T23:29:54.124045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.146492+0000","last_change":"2023-09-27T23:29:54.140209+0000","last_active":"2023-09-27T23:29:54.146492+0000","last_peered":"2023-09-27T23:29:54.146492+0000","last_clean":"2023-09-27T23:29:54.146492+0000","last_became_active":"2023-09-27T23:29:54.139933+0000","last_became_peered":"2023-09-27T23:29:54.139933+0000","last_unstale":"2023-09-27T23:29:54.146492+0000","last_undegraded":"2023-09-27T23:29:54.146492+0000","last_fullsized":"2023-09-27T23:29:54.146492+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:49:25.483548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.118980+0000","last_change":"2023-09-27T23:29:53.753232+0000","last_active":"2023-09-27T23:29:54.118980+0000","last_peered":"2023-09-27T23:29:54.118980+0000","last_clean":"2023-09-27T23:29:54.118980+0000","last_became_active":"2023-09-27T23:29:53.752871+0000","last_became_peered":"2023-09-27T23:29:53.752871+0000","last_unstale":"2023-09-27T23:29:54.118980+0000","last_undegraded":"2023-09-27T23:29:54.118980+0000","last_fullsized":"2023-09-27T23:29:54.118980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:12:40.768556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.128126+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:29:54.128126+0000","last_peered":"2023-09-27T23:29:54.128126+0000","last_clean":"2023-09-27T23:29:54.128126+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:29:54.128126+0000","last_undegraded":"2023-09-27T23:29:54.128126+0000","last_fullsized":"2023-09-27T23:29:54.128126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.149523+0000","last_change":"2023-09-27T23:29:54.147151+0000","last_active":"2023-09-27T23:29:54.149523+0000","last_peered":"2023-09-27T23:29:54.149523+0000","last_clean":"2023-09-27T23:29:54.149523+0000","last_became_active":"2023-09-27T23:29:54.146502+0000","last_became_peered":"2023-09-27T23:29:54.146502+0000","last_unstale":"2023-09-27T23:29:54.149523+0000","last_undegraded":"2023-09-27T23:29:54.149523+0000","last_fullsized":"2023-09-27T23:29:54.149523+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:22.885061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138308+0000","last_change":"2023-09-27T23:29:53.751738+0000","last_active":"2023-09-27T23:29:54.138308+0000","last_peered":"2023-09-27T23:29:54.138308+0000","last_clean":"2023-09-27T23:29:54.138308+0000","last_became_active":"2023-09-27T23:29:53.751417+0000","last_became_peered":"2023-09-27T23:29:53.751417+0000","last_unstale":"2023-09-27T23:29:54.138308+0000","last_undegraded":"2023-09-27T23:29:54.138308+0000","last_fullsized":"2023-09-27T23:29:54.138308+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:53:34.368605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.129466+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:29:54.129466+0000","last_peered":"2023-09-27T23:29:54.129466+0000","last_clean":"2023-09-27T23:29:54.129466+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:29:54.129466+0000","last_undegraded":"2023-09-27T23:29:54.129466+0000","last_fullsized":"2023-09-27T23:29:54.129466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.143159+0000","last_change":"2023-09-27T23:29:54.138881+0000","last_active":"2023-09-27T23:29:54.143159+0000","last_peered":"2023-09-27T23:29:54.143159+0000","last_clean":"2023-09-27T23:29:54.143159+0000","last_became_active":"2023-09-27T23:29:54.138505+0000","last_became_peered":"2023-09-27T23:29:54.138505+0000","last_unstale":"2023-09-27T23:29:54.143159+0000","last_undegraded":"2023-09-27T23:29:54.143159+0000","last_fullsized":"2023-09-27T23:29:54.143159+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:11.809802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.120521+0000","last_change":"2023-09-27T23:29:53.746502+0000","last_active":"2023-09-27T23:29:54.120521+0000","last_peered":"2023-09-27T23:29:54.120521+0000","last_clean":"2023-09-27T23:29:54.120521+0000","last_became_active":"2023-09-27T23:29:53.746217+0000","last_became_peered":"2023-09-27T23:29:53.746217+0000","last_unstale":"2023-09-27T23:29:54.120521+0000","last_undegraded":"2023-09-27T23:29:54.120521+0000","last_fullsized":"2023-09-27T23:29:54.120521+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:51.078604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.122016+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:29:54.122016+0000","last_peered":"2023-09-27T23:29:54.122016+0000","last_clean":"2023-09-27T23:29:54.122016+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:29:54.122016+0000","last_undegraded":"2023-09-27T23:29:54.122016+0000","last_fullsized":"2023-09-27T23:29:54.122016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.138284+0000","last_change":"2023-09-27T23:29:54.131847+0000","last_active":"2023-09-27T23:29:54.138284+0000","last_peered":"2023-09-27T23:29:54.138284+0000","last_clean":"2023-09-27T23:29:54.138284+0000","last_became_active":"2023-09-27T23:29:54.131553+0000","last_became_peered":"2023-09-27T23:29:54.131553+0000","last_unstale":"2023-09-27T23:29:54.138284+0000","last_undegraded":"2023-09-27T23:29:54.138284+0000","last_fullsized":"2023-09-27T23:29:54.138284+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:53.716937+0000","last_clean_scrub_stamp":"2023-09-27T23:29:53.716937+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:07.706937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.117722+0000","last_change":"2023-09-27T23:29:53.754772+0000","last_active":"2023-09-27T23:29:54.117722+0000","last_peered":"2023-09-27T23:29:54.117722+0000","last_clean":"2023-09-27T23:29:54.117722+0000","last_became_active":"2023-09-27T23:29:53.754460+0000","last_became_peered":"2023-09-27T23:29:53.754460+0000","last_unstale":"2023-09-27T23:29:54.117722+0000","last_undegraded":"2023-09-27T23:29:54.117722+0000","last_fullsized":"2023-09-27T23:29:54.117722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:29:52.705384+0000","last_clean_scrub_stamp":"2023-09-27T23:29:52.705384+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:18.033695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-09-27T23:29:54.144433+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:29:54.144433+0000","last_peered":"2023-09-27T23:29:54.144433+0000","last_clean":"2023-09-27T23:29:54.144433+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:29:54.144433+0000","last_undegraded":"2023-09-27T23:29:54.144433+0000","last_fullsized":"2023-09-27T23:29:54.144433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509637,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26176,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":24384,"kb_avail":94345664,"statfs":{"total":96636764160,"available":96609959936,"internally_reserved":0,"allocated":1826816,"data_stored":1222322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24969216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1779999999999999,"5min":1.3100000000000001,"15min":1.7949999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.90900000000000003},"min":{"1min":0.60799999999999998,"5min":0.50600000000000001,"15min":0.47199999999999998},"max":{"1min":1.278,"5min":1.712,"15min":1.712},"last":0.95499999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.69999999999999996,"5min":0.57199999999999995,"15min":0.41399999999999998},"max":{"1min":1.242,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.75600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2390000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.03}]},{"osd":2,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.67700000000000005,"5min":0.307,"15min":0.307},"max":{"1min":1.1080000000000001,"5min":2.681,"15min":2.681},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.623,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":1.718,"15min":1.718},"last":1.006}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.94399999999999995},"min":{"1min":0.72499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":1.29,"15min":1.421},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.052,"15min":1.0469999999999999},"min":{"1min":0.71799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93100000000000005,"15min":0.90000000000000002},"min":{"1min":0.67600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.278,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.996,"15min":0.98299999999999998},"min":{"1min":0.67000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.179,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.0649999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.032,"15min":1.032},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.2470000000000001,"5min":1.51,"15min":1.5880000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1140000000000001,"15min":1.097},"min":{"1min":0.75900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.1319999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1579999999999999,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.407,"5min":1.806,"15min":1.806},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.143,"15min":1.1279999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.292,"5min":2.548,"15min":2.548},"last":1.212}]}]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21828,"kb_used_data":1212,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94350012,"statfs":{"total":96636764160,"available":96614412288,"internally_reserved":0,"allocated":1241088,"data_stored":632550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91800000000000004,"15min":0.94299999999999995},"min":{"1min":0.58199999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.129,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.038},{"interface":"front","average":{"1min":1.093,"5min":1.028,"15min":1.026},"min":{"1min":0.89200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.321,"5min":1.5149999999999999,"15min":1.649},"last":1.079}]},{"osd":1,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.88200000000000001},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.163,"5min":1.323,"15min":1.323},"last":1.177},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.877,"15min":0.89900000000000002},"min":{"1min":0.61099999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.872},"min":{"1min":0.52500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.353,"15min":1.397},"last":1.119},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.236,"5min":1.294,"15min":1.4259999999999999},"last":0.91800000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93799999999999994,"15min":0.92800000000000005},"min":{"1min":0.70899999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1870000000000001,"5min":1.421,"15min":1.734},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.74399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.278,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2749999999999999,"5min":1.323,"15min":1.544},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.73899999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.306,"5min":1.6040000000000001,"15min":1.6060000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98499999999999999,"15min":0.97799999999999998},"min":{"1min":0.67400000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.286,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.157,"15min":1.161},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.357,"5min":1.4410000000000001,"15min":1.5700000000000001},"last":1.24}]},{"osd":7,"last update":"Wed Sep 27 23:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1499999999999999,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.393,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":0.879},{"interface":"front","average":{"1min":1.135,"5min":1.1439999999999999,"15min":1.151},"min":{"1min":0.98899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3540000000000001,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.208}]}]},{"osd":5,"up_from":15,"seq":64424509636,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35976,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":34752,"kb_avail":94335864,"statfs":{"total":96636764160,"available":96599924736,"internally_reserved":0,"allocated":1245184,"data_stored":633442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35586048},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1519999999999999,"5min":1.1890000000000001,"15min":1.2909999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.27,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.91600000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.2969999999999999,"15min":1.526},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.877,"15min":0.89000000000000001},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":1.331},"last":0.82999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.52900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97999999999999998,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.276,"5min":1.369,"15min":1.5740000000000001},"last":0.874}]},{"osd":3,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2290000000000001,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.0189999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.88700000000000001},"min":{"1min":0.45200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.222,"5min":1.341,"15min":1.341},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.70599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.1020000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3799999999999999,"5min":1.429,"15min":1.5580000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.139,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3520000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.123,"15min":1.123},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.6679999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2050000000000001,"15min":1.1890000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.633},"last":1.1830000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509637,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22008,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":20800,"kb_avail":94349832,"statfs":{"total":96636764160,"available":96614227968,"internally_reserved":0,"allocated":1228800,"data_stored":631884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21299200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.90100000000000002},"min":{"1min":0.68600000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1970000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.45,"15min":1.45},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.873,"15min":0.88600000000000001},"min":{"1min":0.68000000000000005,"5min":0.627,"15min":0.52900000000000003},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.3919999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.58199999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.329},"last":0.93500000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.627,"5min":0.497,"15min":0.497},"max":{"1min":1.321,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.032,"5min":0.96599999999999997,"15min":0.96499999999999997},"min":{"1min":0.77100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.522},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.256,"15min":1.373},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.038,"15min":1.018},"min":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2430000000000001,"5min":1.5129999999999999,"15min":1.575},"last":1.141}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57599999999999996},"max":{"1min":1.1990000000000001,"5min":1.355,"15min":1.627},"last":1.113},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.016,"15min":1.006},"min":{"1min":0.65900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.284,"5min":1.478,"15min":2.2730000000000001},"last":1.0029999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3460000000000001,"5min":1.4239999999999999,"15min":1.7909999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.167,"15min":1.1519999999999999},"min":{"1min":1.0409999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3600000000000001,"5min":1.5169999999999999,"15min":1.8620000000000001},"last":1.179}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.091},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.222,"5min":1.627,"15min":1.726},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.159},"min":{"1min":0.86499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":1.859,"15min":2.532},"last":1.194}]}]},{"osd":3,"up_from":15,"seq":64424509636,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22468,"kb_used_data":1212,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":94349372,"statfs":{"total":96636764160,"available":96613756928,"internally_reserved":0,"allocated":1241088,"data_stored":633420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.2410000000000001,"5min":1.3240000000000001,"15min":1.3600000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.76100000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.387,"5min":1.393,"15min":1.393},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0740000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.59299999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.749}]},{"osd":2,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.64400000000000002,"5min":0.43099999999999999,"15min":0.36699999999999999},"max":{"1min":1.1910000000000001,"5min":1.323,"15min":1.353},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92300000000000004,"15min":0.92600000000000005},"min":{"1min":0.64900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.5720000000000001},"last":1.01}]},{"osd":4,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.97599999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.69199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.407,"5min":1.522,"15min":1.522},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.82899999999999996,"5min":0.66000000000000003,"15min":0.626},"max":{"1min":1.4379999999999999,"5min":1.4810000000000001,"15min":1.571},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1559999999999999},"min":{"1min":0.874,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.399,"5min":1.714,"15min":1.764},"last":0.80700000000000005}]},{"osd":6,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.117,"15min":1.1240000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6970000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.242,"5min":1.204,"15min":1.218},"min":{"1min":0.80300000000000005,"5min":0.73699999999999999,"15min":0.66300000000000003},"max":{"1min":1.4990000000000001,"5min":1.591,"15min":1.8620000000000001},"last":1.032}]},{"osd":7,"last update":"Wed Sep 27 23:29:09 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2290000000000001,"15min":1.236},"min":{"1min":1.004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4330000000000001,"5min":1.655,"15min":1.655},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.202},"min":{"1min":0.746,"5min":0.746,"15min":0.73099999999999998},"max":{"1min":1.629,"5min":1.629,"15min":1.671},"last":0.94799999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21116,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":19904,"kb_avail":94350724,"statfs":{"total":96636764160,"available":96615141376,"internally_reserved":0,"allocated":1232896,"data_stored":632438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20381696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.54000000000000004,"5min":0.47099999999999997,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.7370000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.48799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1870000000000001,"5min":1.903,"15min":1.903},"last":0.78700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.38900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.8069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.228,"5min":1.391,"15min":1.651},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.115,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94699999999999995,"15min":0.93799999999999994},"min":{"1min":0.72399999999999998,"5min":0.58599999999999997,"15min":0.58099999999999996},"max":{"1min":1.3120000000000001,"5min":1.444,"15min":1.8280000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.038,"15min":1.038},"min":{"1min":0.83999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.7849999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4279999999999999,"5min":1.7390000000000001,"15min":2.601},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.02,"15min":1.0369999999999999},"min":{"1min":0.73299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2789999999999999,"5min":2.0459999999999998,"15min":2.7320000000000002},"last":1.091},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.141},"min":{"1min":0.84999999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.421,"5min":1.5169999999999999,"15min":2.6760000000000002},"last":0.97999999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.173},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.595,"5min":1.9450000000000001,"15min":2.6560000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.173,"15min":1.1679999999999999},"min":{"1min":0.91600000000000004,"5min":0.871,"15min":0.71699999999999997},"max":{"1min":1.3029999999999999,"5min":2.0129999999999999,"15min":2.6230000000000002},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Sep 27 23:29:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.222,"15min":1.2290000000000001},"min":{"1min":1.002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6910000000000001,"5min":1.9510000000000001,"15min":2.7490000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.226},"min":{"1min":1.0780000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.375,"5min":2.073,"15min":2.7749999999999999},"last":1.004}]}]},{"osd":1,"up_from":14,"seq":60129542343,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":1212,"kb_used_omap":0,"kb_used_meta":35072,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":1241088,"data_stored":632078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35913728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.1020000000000001,"5min":1.3560000000000001,"15min":3.798},"last":0.504},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.82399999999999995},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.40000000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":2.3330000000000002},"last":0.92500000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.27500000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":6.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.87,"15min":0.91500000000000004},"min":{"1min":0.56000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.26,"5min":1.26,"15min":5.649},"last":0.67000000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.349,"5min":1.349,"15min":6.1379999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.53300000000000003,"5min":0.43099999999999999,"15min":0.35899999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.9370000000000001},"last":1.03}]},{"osd":4,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.752,"5min":0.63100000000000001,"15min":0.48799999999999999},"max":{"1min":1.2569999999999999,"5min":1.464,"15min":1.863},"last":1.204},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.119,"15min":1.1379999999999999},"min":{"1min":0.83699999999999997,"5min":0.749,"15min":0.60099999999999998},"max":{"1min":1.4330000000000001,"5min":1.5369999999999999,"15min":4.0499999999999998},"last":0.97599999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0920000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"max":{"1min":1.393,"5min":1.546,"15min":2.7040000000000002},"last":1.425},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.097},"min":{"1min":0.86599999999999999,"5min":0.78400000000000003,"15min":0.55000000000000004},"max":{"1min":1.2889999999999999,"5min":1.4710000000000001,"15min":2.2509999999999999},"last":1.3320000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2130000000000001,"15min":1.2310000000000001},"min":{"1min":0.86699999999999999,"5min":0.61599999999999999,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.6879999999999999,"15min":3.9260000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.1870000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76400000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":2.79},"last":1.4850000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.64800000000000002},"max":{"1min":1.877,"5min":1.877,"15min":2.843},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.95499999999999996,"5min":0.78700000000000003,"15min":0.52700000000000002},"max":{"1min":1.381,"5min":1.6339999999999999,"15min":2.5670000000000002},"last":1.288}]}]},{"osd":0,"up_from":15,"seq":64424509638,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23040,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":94348800,"statfs":{"total":96636764160,"available":96613171200,"internally_reserved":0,"allocated":1826816,"data_stored":1222286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.51700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.901},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.56200000000000006,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.373},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.335},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.2250000000000001,"15min":1.2769999999999999},"last":0.626},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.73699999999999999,"5min":0.65100000000000002,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.282,"15min":1.3400000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.98699999999999999,"15min":1.0149999999999999},"min":{"1min":0.751,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.242,"5min":1.3580000000000001,"15min":2.2519999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.028,"5min":1.0229999999999999,"15min":1.046},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.68300000000000005},"max":{"1min":1.397,"5min":1.5589999999999999,"15min":1.952},"last":1.0049999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0900000000000001,"15min":1.095},"min":{"1min":0.89300000000000002,"5min":0.75600000000000001,"15min":0.71899999999999997},"max":{"1min":1.427,"5min":1.5209999999999999,"15min":1.986},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.107,"15min":1.1339999999999999},"min":{"1min":0.84599999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.452,"5min":1.5509999999999999,"15min":2.0710000000000002},"last":0.86799999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.149,"15min":1.1659999999999999},"min":{"1min":0.94599999999999995,"5min":0.82399999999999995,"15min":0.69799999999999995},"max":{"1min":1.5069999999999999,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.96099999999999997,"5min":0.67700000000000005,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.73,"15min":2.1850000000000001},"last":1.254}]},{"osd":7,"last update":"Wed Sep 27 23:29:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2290000000000001,"15min":1.2390000000000001},"min":{"1min":1.026,"5min":0.72899999999999998,"15min":0.71999999999999997},"max":{"1min":1.589,"5min":1.8109999999999999,"15min":2.3130000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.66400000000000003},"max":{"1min":1.798,"5min":1.798,"15min":2.3540000000000001},"last":1.0720000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:01.922 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:30:01.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:30:02.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.039+0000 7f85c65e0700 1 Processor -- start 2023-09-27T23:30:02.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.039+0000 7f85c65e0700 1 -- start start 2023-09-27T23:30:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85c00ce2d0 0x7f85c00ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c00cbbb0 con 0x7f85c00cc040 2023-09-27T23:30:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c015d960 con 0x7f85c00ce2d0 2023-09-27T23:30:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c006dc90 con 0x7f85c00caa10 2023-09-27T23:30:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c5ddf700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f85c00cc040 0x7f85c0158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38660/0 (socket says 172.21.15.64:38660) 2023-09-27T23:30:02.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c5ddf700 1 -- 172.21.15.64:0/3790163283 learned_addr learned my addr 172.21.15.64:0/3790163283 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c4ddd700 1 --2- 172.21.15.64:0/3790163283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c55de700 1 --2- 172.21.15.64:0/3790163283 >> v2:172.21.15.123:3300/0 conn(0x7f85c00ce2d0 0x7f85c00ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85b7fff700 1 -- 172.21.15.64:0/3790163283 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257262184 0 0) 0x7f85c00cbbb0 con 0x7f85c00cc040 2023-09-27T23:30:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85b7fff700 1 -- 172.21.15.64:0/3790163283 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ac003570 con 0x7f85c00cc040 2023-09-27T23:30:02.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c4ddd700 1 -- 172.21.15.64:0/3790163283 >> v2:172.21.15.123:3300/0 conn(0x7f85c00ce2d0 msgr2=0x7f85c00ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c4ddd700 1 --2- 172.21.15.64:0/3790163283 >> v2:172.21.15.123:3300/0 conn(0x7f85c00ce2d0 0x7f85c00ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c4ddd700 1 -- 172.21.15.64:0/3790163283 >> v1:172.21.15.64:6789/0 conn(0x7f85c00cc040 legacy=0x7f85c0158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c4ddd700 1 -- 172.21.15.64:0/3790163283 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00cafa0 con 0x7f85c00caa10 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c4ddd700 1 --2- 172.21.15.64:0/3790163283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c015d420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85b00098e0 tx=0x7f85b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1636c6dc770c5b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85b7fff700 1 -- 172.21.15.64:0/3790163283 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0011630 con 0x7f85c00caa10 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85b7fff700 1 -- 172.21.15.64:0/3790163283 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b0011790 con 0x7f85c00caa10 2023-09-27T23:30:02.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85b7fff700 1 -- 172.21.15.64:0/3790163283 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0011b10 con 0x7f85c00caa10 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- 172.21.15.64:0/3790163283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 msgr2=0x7f85c015d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3790163283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c015d420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85b00098e0 tx=0x7f85b00091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- 172.21.15.64:0/3790163283 shutdown_connections 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3790163283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c015d420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3790163283 >> v2:172.21.15.123:3300/0 conn(0x7f85c00ce2d0 0x7f85c00ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- 172.21.15.64:0/3790163283 >> 172.21.15.64:0/3790163283 conn(0x7f85c0067bb0 msgr2=0x7f85c00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- 172.21.15.64:0/3790163283 shutdown_connections 2023-09-27T23:30:02.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.043+0000 7f85c65e0700 1 -- 172.21.15.64:0/3790163283 wait complete. 2023-09-27T23:30:02.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 Processor -- start 2023-09-27T23:30:02.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 -- start start 2023-09-27T23:30:02.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c00cc040 0x7f85c0168ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 0x7f85c0169420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0151980 con 0x7f85c00ce2d0 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c014f860 con 0x7f85c00cc040 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c65e0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00c8bd0 con 0x7f85c00caa10 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c5ddf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 0x7f85c0169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c5ddf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 0x7f85c0169420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55054/0 (socket says 172.21.15.64:55054) 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c5ddf700 1 -- 172.21.15.64:0/3389067055 learned_addr learned my addr 172.21.15.64:0/3389067055 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c55de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.047+0000 7f85c4ddd700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c00cc040 0x7f85c0168ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c5ddf700 1 -- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 msgr2=0x7f85c014f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c5ddf700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c014f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c5ddf700 1 -- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c00cc040 msgr2=0x7f85c0168ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c5ddf700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c00cc040 0x7f85c0168ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c5ddf700 1 -- 172.21.15.64:0/3389067055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0009420 con 0x7f85c00ce2d0 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c5ddf700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 0x7f85c0169420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001460 tx=0x7f85b8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800a890 con 0x7f85c00ce2d0 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800a9f0 con 0x7f85c00ce2d0 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800ad70 con 0x7f85c00ce2d0 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c00c8eb0 con 0x7f85c00ce2d0 2023-09-27T23:30:02.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c00c9470 con 0x7f85c00ce2d0 2023-09-27T23:30:02.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.051+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c0151af0 con 0x7f85c00ce2d0 2023-09-27T23:30:02.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.055+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f85b8007300 con 0x7f85c00ce2d0 2023-09-27T23:30:02.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.055+0000 7f85b57fa700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85a4060a70 0x7f85a4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.055+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f85b807d590 con 0x7f85c00ce2d0 2023-09-27T23:30:02.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.055+0000 7f85c55de700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85a4060a70 0x7f85a4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.055+0000 7f85c55de700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85a4060a70 0x7f85a4062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85c015ba00 tx=0x7f85bc0091a0 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:02.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.059+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f85b804d530 con 0x7f85c00ce2d0 2023-09-27T23:30:02.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.279+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f85c004be40 con 0x7f85c00ce2d0 2023-09-27T23:30:02.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85b57fa700 1 -- 172.21.15.64:0/3389067055 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f85b8050b50 con 0x7f85c00ce2d0 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85a4060a70 msgr2=0x7f85a4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85a4060a70 0x7f85a4062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85c015ba00 tx=0x7f85bc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 msgr2=0x7f85c0169420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 0x7f85c0169420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001460 tx=0x7f85b8001ae0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 shutdown_connections 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f85a4060a70 0x7f85a4062f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85c00ce2d0 0x7f85c0169420 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85c00cc040 0x7f85c0168ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 --2- 172.21.15.64:0/3389067055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85c00caa10 0x7f85c014f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 >> 172.21.15.64:0/3389067055 conn(0x7f85c0067bb0 msgr2=0x7f85c0155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 shutdown_connections 2023-09-27T23:30:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.291+0000 7f85c65e0700 1 -- 172.21.15.64:0/3389067055 wait complete. 2023-09-27T23:30:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:30:02.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 Processor -- start 2023-09-27T23:30:02.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 -- start start 2023-09-27T23:30:02.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c2c158f70 0x7f2c2c067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c2c159340 0x7f2c2c0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c068c70 con 0x7f2c2c159340 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c068060 con 0x7f2c2c158f70 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c068340 con 0x7f2c2c069100 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c2c159340 0x7f2c2c0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c2c159340 0x7f2c2c0cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55062/0 (socket says 172.21.15.64:55062) 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 -- 172.21.15.64:0/1322293134 learned_addr learned my addr 172.21.15.64:0/1322293134 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c30f6a700 1 --2- 172.21.15.64:0/1322293134 >> v2:172.21.15.123:3300/0 conn(0x7f2c2c158f70 0x7f2c2c067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c2bfff700 1 --2- 172.21.15.64:0/1322293134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 -- 172.21.15.64:0/1322293134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 msgr2=0x7f2c2c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 --2- 172.21.15.64:0/1322293134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c0cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 -- 172.21.15.64:0/1322293134 >> v2:172.21.15.123:3300/0 conn(0x7f2c2c158f70 msgr2=0x7f2c2c067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 --2- 172.21.15.64:0/1322293134 >> v2:172.21.15.123:3300/0 conn(0x7f2c2c158f70 0x7f2c2c067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 -- 172.21.15.64:0/1322293134 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c068620 con 0x7f2c2c159340 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c3176b700 1 --2- 172.21.15.64:0/1322293134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c159340 0x7f2c2c0cb3e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0cdec0 tx=0x7f2c1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171bb67cdc65107d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c2b7fe700 1 -- 172.21.15.64:0/1322293134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c009e70 con 0x7f2c2c159340 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c2b7fe700 1 -- 172.21.15.64:0/1322293134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c00b860 con 0x7f2c2c159340 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c2b7fe700 1 -- 172.21.15.64:0/1322293134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00bc00 con 0x7f2c2c159340 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 -- 172.21.15.64:0/1322293134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c159340 msgr2=0x7f2c2c0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.411+0000 7f2c331ce700 1 --2- 172.21.15.64:0/1322293134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c159340 0x7f2c2c0cb3e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0cdec0 tx=0x7f2c1c009250 comp rx=0 tx=0).stop 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- 172.21.15.64:0/1322293134 shutdown_connections 2023-09-27T23:30:02.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 --2- 172.21.15.64:0/1322293134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c159340 0x7f2c2c0cb3e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 --2- 172.21.15.64:0/1322293134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 --2- 172.21.15.64:0/1322293134 >> v2:172.21.15.123:3300/0 conn(0x7f2c2c158f70 0x7f2c2c067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- 172.21.15.64:0/1322293134 >> 172.21.15.64:0/1322293134 conn(0x7f2c2c0ccc40 msgr2=0x7f2c2c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- 172.21.15.64:0/1322293134 shutdown_connections 2023-09-27T23:30:02.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- 172.21.15.64:0/1322293134 wait complete. 2023-09-27T23:30:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 Processor -- start 2023-09-27T23:30:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- start start 2023-09-27T23:30:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 0x7f2c2c0b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c2c159340 0x7f2c2c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c16a420 con 0x7f2c2c158f70 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c8cb0 con 0x7f2c2c159340 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c331ce700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c8380 con 0x7f2c2c069100 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 0x7f2c2c0b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 0x7f2c2c0b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55078/0 (socket says 172.21.15.64:55078) 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 -- 172.21.15.64:0/40197283 learned_addr learned my addr 172.21.15.64:0/40197283 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c30f6a700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c3176b700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c2c159340 0x7f2c2c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 -- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 msgr2=0x7f2c2c1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 -- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c2c159340 msgr2=0x7f2c2c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c2c159340 0x7f2c2c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 -- 172.21.15.64:0/40197283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c0094f0 con 0x7f2c2c158f70 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c3176b700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c2c159340 0x7f2c2c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.415+0000 7f2c2bfff700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 0x7f2c2c0b38c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400d8c0 tx=0x7f2c14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140125f0 con 0x7f2c2c158f70 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c14012750 con 0x7f2c2c158f70 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c0c8660 con 0x7f2c2c158f70 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14012af0 con 0x7f2c2c158f70 2023-09-27T23:30:02.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c1565f0 con 0x7f2c2c158f70 2023-09-27T23:30:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c0c8e40 con 0x7f2c2c158f70 2023-09-27T23:30:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2c1401e460 con 0x7f2c2c158f70 2023-09-27T23:30:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c28ff9700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c0c060a70 0x7f2c0c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2c1407d700 con 0x7f2c2c158f70 2023-09-27T23:30:02.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.419+0000 7f2c30f6a700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c0c060a70 0x7f2c0c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:02.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.423+0000 7f2c30f6a700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c0c060a70 0x7f2c0c062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400f570 tx=0x7f2c2400b040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:02.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.427+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2c1404d730 con 0x7f2c2c158f70 2023-09-27T23:30:02.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.623+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c2c071120 con 0x7f2c2c158f70 2023-09-27T23:30:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.623+0000 7f2c28ff9700 1 -- 172.21.15.64:0/40197283 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f2c14050d50 con 0x7f2c2c158f70 2023-09-27T23:30:02.626 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:02.626 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:54.107357+0000","modified":"2023-09-27T23:29:56.124127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1623763192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1623763192},{"type":"v1","addr":"172.21.15.64:6835","nonce":1623763192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-09-27T23:30:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c0c060a70 msgr2=0x7f2c0c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c0c060a70 0x7f2c0c062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400f570 tx=0x7f2c2400b040 comp rx=0 tx=0).stop 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 msgr2=0x7f2c2c0b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 0x7f2c2c0b38c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400d8c0 tx=0x7f2c14008040 comp rx=0 tx=0).stop 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 shutdown_connections 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c0c060a70 0x7f2c0c062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c2c159340 0x7f2c2c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2c158f70 0x7f2c2c0b38c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 --2- 172.21.15.64:0/40197283 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c2c069100 0x7f2c2c1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 >> 172.21.15.64:0/40197283 conn(0x7f2c2c0ccc40 msgr2=0x7f2c2c0ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 shutdown_connections 2023-09-27T23:30:02.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:02.627+0000 7f2c331ce700 1 -- 172.21.15.64:0/40197283 wait complete. 2023-09-27T23:30:02.630 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2023-09-27T23:30:03.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:29:54.107357+0000', 'modified': '2023-09-27T23:29:56.124127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7107}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7107': {'gid': 7107, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/1623763192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 1623763192}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 1623763192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:30:03.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:30:03.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-27T23:30:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 Processor -- start 2023-09-27T23:30:03.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 -- start start 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cb810 0x7ff85c0072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff85c0cbbe0 0x7ff85c054490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c055430 con 0x7ff85c008c30 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0506b0 con 0x7ff85c0cbbe0 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff868f8b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c054b90 con 0x7ff85c0cb810 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff863fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff85c008c30 0x7ff85c0090b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38666/0 (socket says 172.21.15.64:38666) 2023-09-27T23:30:03.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.163+0000 7ff863fff700 1 -- 172.21.15.64:0/3378329254 learned_addr learned my addr 172.21.15.64:0/3378329254 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8637fe700 1 --2- 172.21.15.64:0/3378329254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cb810 0x7ff85c0072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff862ffd700 1 --2- 172.21.15.64:0/3378329254 >> v2:172.21.15.123:3300/0 conn(0x7ff85c0cbbe0 0x7ff85c054490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8627fc700 1 -- 172.21.15.64:0/3378329254 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974749458 0 0) 0x7ff85c055430 con 0x7ff85c008c30 2023-09-27T23:30:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8627fc700 1 -- 172.21.15.64:0/3378329254 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff854003570 con 0x7ff85c008c30 2023-09-27T23:30:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8637fe700 1 -- 172.21.15.64:0/3378329254 >> v2:172.21.15.123:3300/0 conn(0x7ff85c0cbbe0 msgr2=0x7ff85c054490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8637fe700 1 --2- 172.21.15.64:0/3378329254 >> v2:172.21.15.123:3300/0 conn(0x7ff85c0cbbe0 0x7ff85c054490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8637fe700 1 -- 172.21.15.64:0/3378329254 >> v1:172.21.15.64:6789/0 conn(0x7ff85c008c30 legacy=0x7ff85c0090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8637fe700 1 -- 172.21.15.64:0/3378329254 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c054e70 con 0x7ff85c0cb810 2023-09-27T23:30:03.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8637fe700 1 --2- 172.21.15.64:0/3378329254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cb810 0x7ff85c0072d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff858009900 tx=0x7ff858009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fda617595010771 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8627fc700 1 -- 172.21.15.64:0/3378329254 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858005c10 con 0x7ff85c0cb810 2023-09-27T23:30:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8627fc700 1 -- 172.21.15.64:0/3378329254 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff858005d70 con 0x7ff85c0cb810 2023-09-27T23:30:03.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff8627fc700 1 -- 172.21.15.64:0/3378329254 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858006240 con 0x7ff85c0cb810 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff868f8b700 1 -- 172.21.15.64:0/3378329254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cb810 msgr2=0x7ff85c0072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.167+0000 7ff868f8b700 1 --2- 172.21.15.64:0/3378329254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cb810 0x7ff85c0072d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff858009900 tx=0x7ff858009120 comp rx=0 tx=0).stop 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- 172.21.15.64:0/3378329254 shutdown_connections 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 --2- 172.21.15.64:0/3378329254 >> v2:172.21.15.123:3300/0 conn(0x7ff85c0cbbe0 0x7ff85c054490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 --2- 172.21.15.64:0/3378329254 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cb810 0x7ff85c0072d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- 172.21.15.64:0/3378329254 >> 172.21.15.64:0/3378329254 conn(0x7ff85c052920 msgr2=0x7ff85c052cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- 172.21.15.64:0/3378329254 shutdown_connections 2023-09-27T23:30:03.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- 172.21.15.64:0/3378329254 wait complete. 2023-09-27T23:30:03.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 Processor -- start 2023-09-27T23:30:03.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- start start 2023-09-27T23:30:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff85c008c30 0x7ff85c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff85c0cb810 0x7ff85c0e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cbbe0 0x7ff85c0e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0e5eb0 con 0x7ff85c008c30 2023-09-27T23:30:03.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0e28d0 con 0x7ff85c0cb810 2023-09-27T23:30:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff868f8b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0e3730 con 0x7ff85c0cbbe0 2023-09-27T23:30:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff85c008c30 0x7ff85c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff85c008c30 0x7ff85c0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55080/0 (socket says 172.21.15.64:55080) 2023-09-27T23:30:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff8637fe700 1 -- 172.21.15.64:0/2132287788 learned_addr learned my addr 172.21.15.64:0/2132287788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff862ffd700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff85c0cb810 0x7ff85c0e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff863fff700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cbbe0 0x7ff85c0e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff862ffd700 1 -- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cbbe0 msgr2=0x7ff85c0e6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff862ffd700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cbbe0 0x7ff85c0e6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff862ffd700 1 -- 172.21.15.64:0/2132287788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff85c008c30 msgr2=0x7ff85c0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff862ffd700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff85c008c30 0x7ff85c0caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.171+0000 7ff862ffd700 1 -- 172.21.15.64:0/2132287788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580093c0 con 0x7ff85c0cb810 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff862ffd700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff85c0cb810 0x7ff85c0e3bc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff85000e960 tx=0x7ff8500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff84f7fe700 1 -- 172.21.15.64:0/2132287788 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850013680 con 0x7ff85c0cb810 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff84f7fe700 1 -- 172.21.15.64:0/2132287788 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8500137e0 con 0x7ff85c0cb810 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c0e2b20 con 0x7ff85c0cb810 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff84f7fe700 1 -- 172.21.15.64:0/2132287788 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850013ba0 con 0x7ff85c0cb810 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c0e30e0 con 0x7ff85c0cb810 2023-09-27T23:30:03.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff84f7fe700 1 -- 172.21.15.64:0/2132287788 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff8500123d0 con 0x7ff85c0cb810 2023-09-27T23:30:03.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff84f7fe700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff848060aa0 0x7ff848062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff84f7fe700 1 -- 172.21.15.64:0/2132287788 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff85007d090 con 0x7ff85c0cb810 2023-09-27T23:30:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.175+0000 7ff8637fe700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff848060aa0 0x7ff848062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff8637fe700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff848060aa0 0x7ff848062f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff858006d00 tx=0x7ff858016000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 Processor -- start 2023-09-27T23:30:03.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 -- start start 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x23ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x2170710 0x23ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x21bb790 0x23ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x231bf30 con 0x21bb790 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2317ed0 con 0x2170710 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.179+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x22de0b0 con 0x2170ae0 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x23ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x2170710 0x23ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x23ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:45184/0 (socket says 172.21.15.64:45184) 2023-09-27T23:30:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff863fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x21bb790 0x23ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff8637fe700 1 -- 172.21.15.64:0/4101834186 learned_addr learned my addr 172.21.15.64:0/4101834186 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 -- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 msgr2=0x23ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x23ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 -- 172.21.15.64:0/4101834186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x21bb790 msgr2=0x23ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x21bb790 0x23ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 -- 172.21.15.64:0/4101834186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21bc9e0 con 0x2170710 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff8637fe700 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x23ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff863fff700 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x21bb790 0x23ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff862ffd700 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x2170710 0x23ee390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff85007b220 tx=0x7ff85004d2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb2eb521a260c19 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff84cff9700 1 -- 172.21.15.64:0/4101834186 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850050c10 con 0x2170710 2023-09-27T23:30:03.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff84cff9700 1 -- 172.21.15.64:0/4101834186 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850051e90 con 0x2170710 2023-09-27T23:30:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff84cff9700 1 -- 172.21.15.64:0/4101834186 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850057e20 con 0x2170710 2023-09-27T23:30:03.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 -- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x2170710 msgr2=0x23ee390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x2170710 0x23ee390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff85007b220 tx=0x7ff85004d2b0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 -- 172.21.15.64:0/4101834186 shutdown_connections 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x21bb790 0x23ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x2170710 0x23ee390 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 --2- 172.21.15.64:0/4101834186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x23ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 -- 172.21.15.64:0/4101834186 >> 172.21.15.64:0/4101834186 conn(0x2170eb0 msgr2=0x23aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 -- 172.21.15.64:0/4101834186 shutdown_connections 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 -- 172.21.15.64:0/4101834186 wait complete. 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 -- start start 2023-09-27T23:30:03.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.183+0000 7ff86c496740 1 Processor -- start 2023-09-27T23:30:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x23aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x23d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x22ddd50 con 0x2170ae0 2023-09-27T23:30:03.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x22b6970 con 0x21bb790 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x22e0920 con 0x2170710 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 10 client.0 _ll_register_callbacks cb 0x2361b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x23d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff863fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x23d3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55094/0 (socket says 172.21.15.64:55094) 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x23aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 -- 172.21.15.64:0/840682530 learned_addr learned my addr 172.21.15.64:0/840682530 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 -- 172.21.15.64:0/840682530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 msgr2=0x23aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x23aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 -- 172.21.15.64:0/840682530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 msgr2=0x23d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 -- 172.21.15.64:0/840682530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c054e70 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff862ffd700 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x23d3c00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff85007b2d0 tx=0x7ff85007d020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab87c2f48e043550 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 1 -- 172.21.15.64:0/840682530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850070070 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 1 -- 172.21.15.64:0/840682530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff8500560c0 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22b6cc0 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 1 -- 172.21.15.64:0/840682530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8500512f0 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 10 client.7278 fetch_fsmap learned FSMap version 33 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x7ff854003570 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 1 -- 172.21.15.64:0/840682530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85005c080 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 1 -- 172.21.15.64:0/840682530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff850073030 con 0x2170ae0 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 10 client.7278 fetch_fsmap finished waiting for FSMap version 33 2023-09-27T23:30:03.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 10 client.7278 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 20 client.7278 populate_metadata read hostname 'smithi064' 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23fa6b0 0x23fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 4 client.7278 mds_command: new command op to 7107 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 --> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x22ba3e0 con 0x23fa6b0 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff8637fe700 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23fa6b0 0x23fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff8637fe700 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23fa6b0 0x23fca90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d9735a44e30b02 server_cookie=5ebe58f44a8c3bf5 in_seq=0 out_seq=0 2023-09-27T23:30:03.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.187+0000 7ff841ffb700 10 client.7278 ms_handle_connect on v2:172.21.15.64:6834/1623763192 2023-09-27T23:30:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff841ffb700 1 -- 172.21.15.64:0/840682530 <== mds.0 v2:172.21.15.64:6834/1623763192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x22ba3e0 con 0x23fa6b0 2023-09-27T23:30:03.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff841ffb700 10 client.7278 handle_command_reply: tid=0 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 client.7278 shutdown 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 msgr2=0x23d3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x23d3c00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff85007b2d0 tx=0x7ff85007d020 comp rx=0 tx=0).stop 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 shutdown_connections 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x23d3c00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23fa6b0 0x23fca90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 --2- 172.21.15.64:0/840682530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x23aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff841ffb700 0 client.7278 ms_handle_reset on v2:172.21.15.64:6834/1623763192 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 >> 172.21.15.64:0/840682530 conn(0x2170eb0 msgr2=0x23a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 shutdown_connections 2023-09-27T23:30:03.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 1 -- 172.21.15.64:0/840682530 wait complete. 2023-09-27T23:30:03.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.191+0000 7ff86c496740 20 client.7278 trim_cache size 0 max 16384 2023-09-27T23:30:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 Processor -- start 2023-09-27T23:30:03.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 -- start start 2023-09-27T23:30:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 0x24436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x2443c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x2444150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ed650 con 0x2170710 2023-09-27T23:30:03.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x231e290 con 0x21bb790 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x22c3ec0 con 0x2170ae0 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 0x24436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 0x24436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55110/0 (socket says 172.21.15.64:55110) 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 -- 172.21.15.64:0/1830020390 learned_addr learned my addr 172.21.15.64:0/1830020390 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff863fff700 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x2444150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff862ffd700 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x2443c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 -- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 msgr2=0x2443c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x2443c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 -- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 msgr2=0x2444150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x2444150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff8637fe700 1 -- 172.21.15.64:0/1830020390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0e72f0 con 0x2170710 2023-09-27T23:30:03.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.211+0000 7ff863fff700 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x2444150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff8637fe700 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 0x24436d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff858009340 tx=0x7ff858013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c570f0a4ddcd1810 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff84d7fa700 1 -- 172.21.15.64:0/1830020390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858031070 con 0x2170710 2023-09-27T23:30:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff84d7fa700 1 -- 172.21.15.64:0/1830020390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff858032040 con 0x2170710 2023-09-27T23:30:03.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff84d7fa700 1 -- 172.21.15.64:0/1830020390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858023070 con 0x2170710 2023-09-27T23:30:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- 172.21.15.64:0/1830020390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 msgr2=0x24436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 0x24436d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff858009340 tx=0x7ff858013000 comp rx=0 tx=0).stop 2023-09-27T23:30:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- 172.21.15.64:0/1830020390 shutdown_connections 2023-09-27T23:30:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x2444150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170ae0 0x2443c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- 172.21.15.64:0/1830020390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170710 0x24436d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- 172.21.15.64:0/1830020390 >> 172.21.15.64:0/1830020390 conn(0x2170eb0 msgr2=0x23d2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- 172.21.15.64:0/1830020390 shutdown_connections 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- 172.21.15.64:0/1830020390 wait complete. 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- start start 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 Processor -- start 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x2447ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x2448010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2313b50 con 0x2170ae0 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x22ebf40 con 0x21bb790 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x22c6c90 con 0x2170710 2023-09-27T23:30:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff86c496740 10 client.0 _ll_register_callbacks cb 0x22340e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff862ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x2448010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff8637fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x2447ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff862ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x2448010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55112/0 (socket says 172.21.15.64:55112) 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff863fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.215+0000 7ff862ffd700 1 -- 172.21.15.64:0/3503412889 learned_addr learned my addr 172.21.15.64:0/3503412889 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff862ffd700 1 -- 172.21.15.64:0/3503412889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 msgr2=0x2447ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff862ffd700 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x2447ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff862ffd700 1 -- 172.21.15.64:0/3503412889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 msgr2=0x23ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff862ffd700 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff862ffd700 1 -- 172.21.15.64:0/3503412889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff858006650 con 0x2170ae0 2023-09-27T23:30:03.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff862ffd700 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x2448010 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff850050b40 tx=0x7ff850077040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4379d154d7560bbc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:30:03.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 1 -- 172.21.15.64:0/3503412889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850070060 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 1 -- 172.21.15.64:0/3503412889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff8500560c0 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff850051e90 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 1 -- 172.21.15.64:0/3503412889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff850050c10 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 10 client.7284 fetch_fsmap learned FSMap version 33 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x7ff85c0e30e0 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 1 -- 172.21.15.64:0/3503412889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85005c020 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 1 -- 172.21.15.64:0/3503412889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff85007b300 con 0x2170ae0 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 10 client.7284 fetch_fsmap finished waiting for FSMap version 33 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 10 client.7284 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 20 client.7284 populate_metadata read hostname 'smithi064' 2023-09-27T23:30:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23cf040 0x23cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 4 client.7284 mds_command: new command op to 7107 tid=0[{"prefix": "status"}] 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 --> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] -- command(tid 0: {"prefix": "status"}) v1 -- 0x22ec410 con 0x23cf040 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff8637fe700 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23cf040 0x23cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 10 client.7284 ms_handle_connect on v2:172.21.15.64:6834/1623763192 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff8637fe700 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23cf040 0x23cf410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2d6ad12ccd886c server_cookie=82eb3dbb3898dae3 in_seq=0 out_seq=0 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 1 -- 172.21.15.64:0/3503412889 <== mds.0 v2:172.21.15.64:6834/1623763192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x22ec410 con 0x23cf040 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 10 client.7284 handle_command_reply: tid=0 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 client.7284 shutdown 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 msgr2=0x2448010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x2448010 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff850050b40 tx=0x7ff850077040 comp rx=0 tx=0).stop 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 shutdown_connections 2023-09-27T23:30:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x21bb790 0x23ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2170ae0 0x2448010 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.64:6834/1623763192,v1:172.21.15.64:6835/1623763192] conn(0x23cf040 0x23cf410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 --2- 172.21.15.64:0/3503412889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x2170710 0x2447ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff841ffb700 0 client.7284 ms_handle_reset on v2:172.21.15.64:6834/1623763192 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 >> 172.21.15.64:0/3503412889 conn(0x2170eb0 msgr2=0x23a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 shutdown_connections 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 1 -- 172.21.15.64:0/3503412889 wait complete. 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.219+0000 7ff86c496740 20 client.7284 trim_cache size 0 max 16384 2023-09-27T23:30:03.227 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7107, 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.1109079689999994, 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 33, 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 101, 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 101, 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 13.661183578999999 2023-09-27T23:30:03.228 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:30:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff848060aa0 msgr2=0x7ff848062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff848060aa0 0x7ff848062f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff858006d00 tx=0x7ff858016000 comp rx=0 tx=0).stop 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff85c0cb810 msgr2=0x7ff85c0e3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff85c0cb810 0x7ff85c0e3bc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff85000e960 tx=0x7ff8500091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 shutdown_connections 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff848060aa0 0x7ff848062f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff85c0cbbe0 0x7ff85c0e6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff85c0cb810 0x7ff85c0e3bc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 --2- 172.21.15.64:0/2132287788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff85c008c30 0x7ff85c0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 >> 172.21.15.64:0/2132287788 conn(0x7ff85c052920 msgr2=0x7ff85c038ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 shutdown_connections 2023-09-27T23:30:03.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.223+0000 7ff868f8b700 1 -- 172.21.15.64:0/2132287788 wait complete. 2023-09-27T23:30:03.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:30:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 Processor -- start 2023-09-27T23:30:03.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 -- start start 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6b01591f0 0x7fb6b006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b00ce810 0x7fb6b00ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b006b730 con 0x7fb6b0158e20 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b006ab20 con 0x7fb6b01591f0 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b589b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b006ae00 con 0x7fb6b00ce810 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b4899700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb6b0158e20 0x7fb6b00ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38680/0 (socket says 172.21.15.64:38680) 2023-09-27T23:30:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b4899700 1 -- 172.21.15.64:0/3025262739 learned_addr learned my addr 172.21.15.64:0/3025262739 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6affff700 1 --2- 172.21.15.64:0/3025262739 >> v2:172.21.15.123:3300/0 conn(0x7fb6b01591f0 0x7fb6b006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6b509a700 1 --2- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b00ce810 0x7fb6b00ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.351+0000 7fb6af7fe700 1 -- 172.21.15.64:0/3025262739 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484339428 0 0) 0x7fb6b006b730 con 0x7fb6b0158e20 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6af7fe700 1 -- 172.21.15.64:0/3025262739 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb69c003570 con 0x7fb6b0158e20 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6affff700 1 -- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b00ce810 msgr2=0x7fb6b00ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6affff700 1 --2- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b00ce810 0x7fb6b00ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6affff700 1 -- 172.21.15.64:0/3025262739 >> v1:172.21.15.64:6789/0 conn(0x7fb6b0158e20 legacy=0x7fb6b00ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6affff700 1 -- 172.21.15.64:0/3025262739 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b006b0e0 con 0x7fb6b01591f0 2023-09-27T23:30:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6affff700 1 --2- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b01591f0 0x7fb6b006a540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00098e0 tx=0x7fb6a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6efb20ce978bd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6af7fe700 1 -- 172.21.15.64:0/3025262739 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011630 con 0x7fb6b01591f0 2023-09-27T23:30:03.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6af7fe700 1 -- 172.21.15.64:0/3025262739 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0011790 con 0x7fb6b01591f0 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6af7fe700 1 -- 172.21.15.64:0/3025262739 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011b30 con 0x7fb6b01591f0 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 -- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b01591f0 msgr2=0x7fb6b006a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 --2- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b01591f0 0x7fb6b006a540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00098e0 tx=0x7fb6a00091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 -- 172.21.15.64:0/3025262739 shutdown_connections 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 --2- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b00ce810 0x7fb6b00ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 --2- 172.21.15.64:0/3025262739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b01591f0 0x7fb6b006a540 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 -- 172.21.15.64:0/3025262739 >> 172.21.15.64:0/3025262739 conn(0x7fb6b0067bb0 msgr2=0x7fb6b00cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 -- 172.21.15.64:0/3025262739 shutdown_connections 2023-09-27T23:30:03.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 -- 172.21.15.64:0/3025262739 wait complete. 2023-09-27T23:30:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 Processor -- start 2023-09-27T23:30:03.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.355+0000 7fb6b589b700 1 -- start start 2023-09-27T23:30:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b00ce810 0x7fb6b00b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b0158e20 0x7fb6b0148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 0x7fb6b00b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00b3310 con 0x7fb6b01591f0 2023-09-27T23:30:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0156170 con 0x7fb6b00ce810 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00b5100 con 0x7fb6b0158e20 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 0x7fb6b00b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 0x7fb6b00b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55128/0 (socket says 172.21.15.64:55128) 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 -- 172.21.15.64:0/676597301 learned_addr learned my addr 172.21.15.64:0/676597301 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6affff700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b0158e20 0x7fb6b0148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b4899700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b00ce810 0x7fb6b00b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 -- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b0158e20 msgr2=0x7fb6b0148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b0158e20 0x7fb6b0148720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 -- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b00ce810 msgr2=0x7fb6b00b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b00ce810 0x7fb6b00b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 -- 172.21.15.64:0/676597301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009420 con 0x7fb6b01591f0 2023-09-27T23:30:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b509a700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 0x7fb6b00b3d60 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009e90 tx=0x7fb6a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800cb70 con 0x7fb6b01591f0 2023-09-27T23:30:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a800f040 con 0x7fb6b01591f0 2023-09-27T23:30:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b01558d0 con 0x7fb6b01591f0 2023-09-27T23:30:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8014480 con 0x7fb6b01591f0 2023-09-27T23:30:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b00b3840 con 0x7fb6b01591f0 2023-09-27T23:30:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.359+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b0155cd0 con 0x7fb6b01591f0 2023-09-27T23:30:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.363+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb6a80147f0 con 0x7fb6b01591f0 2023-09-27T23:30:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.363+0000 7fb6acff9700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb690060aa0 0x7fb690062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.363+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb6a807df70 con 0x7fb6b01591f0 2023-09-27T23:30:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.363+0000 7fb6b4899700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb690060aa0 0x7fb690062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:03.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.363+0000 7fb6b4899700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb690060aa0 0x7fb690062f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb698007fd0 tx=0x7fb698009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:03.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.371+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb6a804ddf0 con 0x7fb6b01591f0 2023-09-27T23:30:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.575+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6b00b36a0 con 0x7fb6b01591f0 2023-09-27T23:30:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.575+0000 7fb6acff9700 1 -- 172.21.15.64:0/676597301 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb6b00b36a0 con 0x7fb6b01591f0 2023-09-27T23:30:03.579 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb690060aa0 msgr2=0x7fb690062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb690060aa0 0x7fb690062f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb698007fd0 tx=0x7fb698009040 comp rx=0 tx=0).stop 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 msgr2=0x7fb6b00b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 0x7fb6b00b3d60 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009e90 tx=0x7fb6a800b040 comp rx=0 tx=0).stop 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 shutdown_connections 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb690060aa0 0x7fb690062f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b01591f0 0x7fb6b00b3d60 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b0158e20 0x7fb6b0148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 --2- 172.21.15.64:0/676597301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b00ce810 0x7fb6b00b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 >> 172.21.15.64:0/676597301 conn(0x7fb6b0067bb0 msgr2=0x7fb6b014a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 shutdown_connections 2023-09-27T23:30:03.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:03.579+0000 7fb6b589b700 1 -- 172.21.15.64:0/676597301 wait complete. 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:30:03.593 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:30:03.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:03.594 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:30:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41581sec preferred_lft 41581sec 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:30:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:30:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.602 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:30:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:30:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:30:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:03.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:03.602 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:30:03.615 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:30:03.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:03.618 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:30:03.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:03.626 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:30:03.626 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:30:03.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:30:03.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:30:03.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:30:03.626 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:30:03.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:03.663 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:30:03.669 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:30:03.669 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:30:03.670 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:30:03.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:03.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:30:03.686 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:30:03.686 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:30:03.686 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:03.693 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:30:03.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:30:03.693 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:03.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:30:03.714 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:30:03.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:30:03.769 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:30:03.770 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:30:03.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:03.771 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:30:03.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:30:03.776 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:30:03.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:30:03.791 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:30:03.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:30:03.803 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:30:03.804 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:30:03.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:03.804 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:30:03.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:30:03.855+0000 7f9878c93380 -1 init, newargv = 0x55fc54623df0 newargc=15 2023-09-27T23:30:03.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[51251]: starting ceph client 2023-09-27T23:30:03.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[51251]: starting fuse 2023-09-27T23:30:04.955 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:30:04.972 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:30:04.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:30:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:30:04.989 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:30:04.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:04.990 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:30:05.038 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:30:05.038 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:30:05.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:30:05.039 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:05.040 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:30:05.041 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:30:05.126 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.51251.asok 2023-09-27T23:30:05.308 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51251.asok 2023-09-27T23:30:05.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:05.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51251.asok status 2023-09-27T23:30:05.405 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:30:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:30:05.405 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "51251", 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7296, 2023-09-27T23:30:05.406 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 7296 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2239625170 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:30:05.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2239625170 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.7296 192.168.0.1:0/2239625170", 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2239625170", 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 33, 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 101, 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:30:05.408 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:30:05.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:05.415 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:30:05.415 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:05.421 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:30:05.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:05.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:05.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:05.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:30:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 Processor -- start 2023-09-27T23:30:05.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 -- start start 2023-09-27T23:30:05.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e3006b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1e3006baa0 0x7f1e300b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3006b0d0 con 0x7f1e30158cc0 2023-09-27T23:30:05.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3006beb0 con 0x7f1e3006baa0 2023-09-27T23:30:05.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e366c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300b4790 con 0x7f1e3006dc00 2023-09-27T23:30:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e2f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e3006b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e2f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e3006b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:39134/0 (socket says 172.21.15.64:39134) 2023-09-27T23:30:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e2f7fe700 1 -- 172.21.15.64:0/4283829099 learned_addr learned my addr 172.21.15.64:0/4283829099 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.587+0000 7f1e34c66700 1 --2- 172.21.15.64:0/4283829099 >> v2:172.21.15.123:3300/0 conn(0x7f1e3006baa0 0x7f1e300b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2effd700 1 -- 172.21.15.64:0/4283829099 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448246539 0 0) 0x7f1e3006b0d0 con 0x7f1e30158cc0 2023-09-27T23:30:05.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2effd700 1 -- 172.21.15.64:0/4283829099 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e18003570 con 0x7f1e30158cc0 2023-09-27T23:30:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2f7fe700 1 -- 172.21.15.64:0/4283829099 >> v2:172.21.15.123:3300/0 conn(0x7f1e3006baa0 msgr2=0x7f1e300b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2f7fe700 1 --2- 172.21.15.64:0/4283829099 >> v2:172.21.15.123:3300/0 conn(0x7f1e3006baa0 0x7f1e300b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2f7fe700 1 -- 172.21.15.64:0/4283829099 >> v1:172.21.15.123:6789/0 conn(0x7f1e30158cc0 legacy=0x7f1e3006d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2f7fe700 1 -- 172.21.15.64:0/4283829099 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3006a7e0 con 0x7f1e3006dc00 2023-09-27T23:30:05.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2f7fe700 1 --2- 172.21.15.64:0/4283829099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e3006b560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e240099b0 tx=0x7f1e24009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae2de7b7d703ead server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:05.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2effd700 1 -- 172.21.15.64:0/4283829099 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400b680 con 0x7f1e3006dc00 2023-09-27T23:30:05.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2effd700 1 -- 172.21.15.64:0/4283829099 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2400b7e0 con 0x7f1e3006dc00 2023-09-27T23:30:05.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e2effd700 1 -- 172.21.15.64:0/4283829099 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400bb60 con 0x7f1e3006dc00 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 -- 172.21.15.64:0/4283829099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 msgr2=0x7f1e3006b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 --2- 172.21.15.64:0/4283829099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e3006b560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e240099b0 tx=0x7f1e24009250 comp rx=0 tx=0).stop 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 -- 172.21.15.64:0/4283829099 shutdown_connections 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 --2- 172.21.15.64:0/4283829099 >> v2:172.21.15.123:3300/0 conn(0x7f1e3006baa0 0x7f1e300b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 --2- 172.21.15.64:0/4283829099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e3006b560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 -- 172.21.15.64:0/4283829099 >> 172.21.15.64:0/4283829099 conn(0x7f1e30067bb0 msgr2=0x7f1e300cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 -- 172.21.15.64:0/4283829099 shutdown_connections 2023-09-27T23:30:05.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.591+0000 7f1e366c9700 1 -- 172.21.15.64:0/4283829099 wait complete. 2023-09-27T23:30:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 Processor -- start 2023-09-27T23:30:05.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 -- start start 2023-09-27T23:30:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 0x7f1e300b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e300a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e30158cc0 0x7f1e300a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3015f8f0 con 0x7f1e3006baa0 2023-09-27T23:30:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30152ac0 con 0x7f1e30158cc0 2023-09-27T23:30:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3015f610 con 0x7f1e3006dc00 2023-09-27T23:30:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 0x7f1e300b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 0x7f1e300b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36162/0 (socket says 172.21.15.64:36162) 2023-09-27T23:30:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 -- 172.21.15.64:0/2720207555 learned_addr learned my addr 172.21.15.64:0/2720207555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2f7fe700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e300a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e34c66700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e30158cc0 0x7f1e300a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 -- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 msgr2=0x7f1e300a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e300a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 -- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e30158cc0 msgr2=0x7f1e300a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e30158cc0 0x7f1e300a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 -- 172.21.15.64:0/2720207555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240094f0 con 0x7f1e3006baa0 2023-09-27T23:30:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e34c66700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e30158cc0 0x7f1e300a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e2ffff700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 0x7f1e300b2240 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000c7f0 tx=0x7f1e20007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20010510 con 0x7f1e3006baa0 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20010670 con 0x7f1e3006baa0 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20010a30 con 0x7f1e3006baa0 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e30152470 con 0x7f1e3006baa0 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.595+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3015e890 con 0x7f1e3006baa0 2023-09-27T23:30:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.599+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e30071120 con 0x7f1e3006baa0 2023-09-27T23:30:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.599+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1e2001b3b0 con 0x7f1e3006baa0 2023-09-27T23:30:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.599+0000 7f1e17fff700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1e1c060a70 0x7f1e1c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.599+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1e2007cb40 con 0x7f1e3006baa0 2023-09-27T23:30:05.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.599+0000 7f1e2f7fe700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1e1c060a70 0x7f1e1c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.599+0000 7f1e2f7fe700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1e1c060a70 0x7f1e1c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1e240099b0 tx=0x7f1e24017000 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.603+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1e2004cad0 con 0x7f1e3006baa0 2023-09-27T23:30:05.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.791+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e30152c00 con 0x7f1e1c060a70 2023-09-27T23:30:05.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.795+0000 7f1e17fff700 1 -- 172.21.15.64:0/2720207555 <== mgr.6209 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1e24000c50 con 0x7f1e1c060a70 2023-09-27T23:30:05.798 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:30:05.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:05.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:30:05.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:05.798 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:30:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.795+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1e1c060a70 msgr2=0x7f1e1c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.795+0000 7f1e366c9700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1e1c060a70 0x7f1e1c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1e240099b0 tx=0x7f1e24017000 comp rx=0 tx=0).stop 2023-09-27T23:30:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 msgr2=0x7f1e300b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 0x7f1e300b2240 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000c7f0 tx=0x7f1e20007dc0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 shutdown_connections 2023-09-27T23:30:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1e1c060a70 0x7f1e1c062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e30158cc0 0x7f1e300a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e3006dc00 0x7f1e300a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 --2- 172.21.15.64:0/2720207555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3006baa0 0x7f1e300b2240 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 >> 172.21.15.64:0/2720207555 conn(0x7f1e30067bb0 msgr2=0x7f1e3015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 shutdown_connections 2023-09-27T23:30:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.799+0000 7f1e366c9700 1 -- 172.21.15.64:0/2720207555 wait complete. 2023-09-27T23:30:05.812 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:30:05.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.919+0000 7fb83f217700 1 Processor -- start 2023-09-27T23:30:05.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.919+0000 7fb83f217700 1 -- start start 2023-09-27T23:30:05.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83f217700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8380b02e0 0x7fb838159d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83f217700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83806c7c0 con 0x7fb83806b690 2023-09-27T23:30:05.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83f217700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83806cd80 con 0x7fb83806b2c0 2023-09-27T23:30:05.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83f217700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8380b0720 con 0x7fb8380b02e0 2023-09-27T23:30:05.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb83806b690 0x7fb8380afbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56646/0 (socket says 172.21.15.64:56646) 2023-09-27T23:30:05.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 -- 172.21.15.64:0/2604226761 learned_addr learned my addr 172.21.15.64:0/2604226761 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:05.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83d7b4700 1 --2- 172.21.15.64:0/2604226761 >> v2:172.21.15.123:3300/0 conn(0x7fb8380b02e0 0x7fb838159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391973002 0 0) 0x7fb83806c7c0 con 0x7fb83806b690 2023-09-27T23:30:05.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb820003570 con 0x7fb83806b690 2023-09-27T23:30:05.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31891657 0 0) 0x7fb83806cd80 con 0x7fb83806b2c0 2023-09-27T23:30:05.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb83806c7c0 con 0x7fb83806b2c0 2023-09-27T23:30:05.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287167532 0 0) 0x7fb820003570 con 0x7fb83806b690 2023-09-27T23:30:05.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 >> v2:172.21.15.123:3300/0 conn(0x7fb8380b02e0 msgr2=0x7fb838159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 --2- 172.21.15.64:0/2604226761 >> v2:172.21.15.123:3300/0 conn(0x7fb8380b02e0 0x7fb838159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 >> v1:172.21.15.123:6789/0 conn(0x7fb83806b2c0 legacy=0x7fb83806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380b0940 con 0x7fb83806b690 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83d7b4700 1 --2- 172.21.15.64:0/2604226761 >> v2:172.21.15.123:3300/0 conn(0x7fb8380b02e0 0x7fb838159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fb828001470 con 0x7fb83806b690 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 >> v1:172.21.15.64:6789/0 conn(0x7fb83806b690 legacy=0x7fb8380afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb820003bf0 0x7fb820003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb820004810 0x7fb820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8200051c0 0x7fb82000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380b0720 con 0x7fb820004810 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000c160 con 0x7fb8200051c0 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000c320 con 0x7fb820003bf0 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb820004810 0x7fb820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83d7b4700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8200051c0 0x7fb82000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83cfb3700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb820003bf0 0x7fb820003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 -- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb820003bf0 msgr2=0x7fb820003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb820003bf0 0x7fb820003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 -- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8200051c0 msgr2=0x7fb82000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8200051c0 0x7fb82000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb83d7b4700 1 -- 172.21.15.64:0/2604226761 reap_dead start 2023-09-27T23:30:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 -- 172.21.15.64:0/2604226761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820003570 con 0x7fb820004810 2023-09-27T23:30:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb837fff700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb820004810 0x7fb820004c80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c2a0 tx=0x7fb828001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f308c3babcb5107c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800cb30 con 0x7fb820004810 2023-09-27T23:30:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82800ccb0 con 0x7fb820004810 2023-09-27T23:30:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.923+0000 7fb8377fe700 1 -- 172.21.15.64:0/2604226761 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828005b70 con 0x7fb820004810 2023-09-27T23:30:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 -- 172.21.15.64:0/2604226761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb820004810 msgr2=0x7fb820004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb820004810 0x7fb820004c80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c2a0 tx=0x7fb828001e10 comp rx=0 tx=0).stop 2023-09-27T23:30:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 -- 172.21.15.64:0/2604226761 shutdown_connections 2023-09-27T23:30:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 --2- 172.21.15.64:0/2604226761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb820004810 0x7fb820004c80 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 -- 172.21.15.64:0/2604226761 >> 172.21.15.64:0/2604226761 conn(0x7fb8380cc580 msgr2=0x7fb8380cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 -- 172.21.15.64:0/2604226761 shutdown_connections 2023-09-27T23:30:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 -- 172.21.15.64:0/2604226761 wait complete. 2023-09-27T23:30:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 Processor -- start 2023-09-27T23:30:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.927+0000 7fb83f217700 1 -- start start 2023-09-27T23:30:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb820004810 0x7fb838143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb83816a290 0x7fb8380ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 0x7fb8380cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380cb3b0 con 0x7fb8380caa10 2023-09-27T23:30:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380ad6f0 con 0x7fb820004810 2023-09-27T23:30:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380acae0 con 0x7fb83816a290 2023-09-27T23:30:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 0x7fb8380cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 0x7fb8380cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36180/0 (socket says 172.21.15.64:36180) 2023-09-27T23:30:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 -- 172.21.15.64:0/2702134805 learned_addr learned my addr 172.21.15.64:0/2702134805 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83cfb3700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb820004810 0x7fb838143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb837fff700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb83816a290 0x7fb8380ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 -- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb83816a290 msgr2=0x7fb8380ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb83816a290 0x7fb8380ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 -- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb820004810 msgr2=0x7fb838143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb820004810 0x7fb838143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 -- 172.21.15.64:0/2702134805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380acdc0 con 0x7fb8380caa10 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83cfb3700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb820004810 0x7fb838143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb837fff700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb83816a290 0x7fb8380ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:05.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83d7b4700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 0x7fb8380cae70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c080 tx=0x7fb83000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000e040 con 0x7fb8380caa10 2023-09-27T23:30:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83000f040 con 0x7fb8380caa10 2023-09-27T23:30:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830015430 con 0x7fb8380caa10 2023-09-27T23:30:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8380ad0a0 con 0x7fb8380caa10 2023-09-27T23:30:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.931+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8381437b0 con 0x7fb8380caa10 2023-09-27T23:30:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.935+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb838071120 con 0x7fb8380caa10 2023-09-27T23:30:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.935+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 33) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb830015820 con 0x7fb8380caa10 2023-09-27T23:30:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.935+0000 7fb834ff9700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb824060a70 0x7fb824062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.935+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb83007dd20 con 0x7fb8380caa10 2023-09-27T23:30:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.935+0000 7fb83cfb3700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb824060a70 0x7fb824062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.935+0000 7fb83cfb3700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb824060a70 0x7fb824062f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb82c000c00 tx=0x7fb82c009040 comp rx=0 tx=0).ready entity=mgr.6209 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:05.939+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb830082070 con 0x7fb8380caa10 2023-09-27T23:30:06.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:06.143+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb8380ad830 con 0x7fb8380caa10 2023-09-27T23:30:07.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.155+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 34) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fb83004dff0 con 0x7fb8380caa10 2023-09-27T23:30:07.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.159+0000 7fb834ff9700 1 -- 172.21.15.64:0/2702134805 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v34) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb83004dc20 con 0x7fb8380caa10 2023-09-27T23:30:07.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.159+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb824060a70 msgr2=0x7fb824062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.159+0000 7fb83f217700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb824060a70 0x7fb824062f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb82c000c00 tx=0x7fb82c009040 comp rx=0 tx=0).stop 2023-09-27T23:30:07.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.159+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 msgr2=0x7fb8380cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.159+0000 7fb83f217700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 0x7fb8380cae70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c080 tx=0x7fb83000b040 comp rx=0 tx=0).stop 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 shutdown_connections 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb824060a70 0x7fb824062f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8380caa10 0x7fb8380cae70 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb83816a290 0x7fb8380ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 --2- 172.21.15.64:0/2702134805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb820004810 0x7fb838143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 >> 172.21.15.64:0/2702134805 conn(0x7fb8380cc580 msgr2=0x7fb83815df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:07.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 shutdown_connections 2023-09-27T23:30:07.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.163+0000 7fb83f217700 1 -- 172.21.15.64:0/2702134805 wait complete. 2023-09-27T23:30:07.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:30:07.275 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:07.270+0000 7f05f9d58000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:30:07.280 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:07.279+0000 7f1f630a1000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:30:07.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.287+0000 7fe9e03aa700 1 Processor -- start 2023-09-27T23:30:07.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.287+0000 7fe9e03aa700 1 -- start start 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9d806b2c0 0x7fe9d806b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9d806d210 0x7fe9d806d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d806c7c0 con 0x7fe9d80afbd0 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d80b0760 con 0x7fe9d806b2c0 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d806cd80 con 0x7fe9d806d210 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9d806b2c0 0x7fe9d806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9d806b2c0 0x7fe9d806b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36204/0 (socket says 172.21.15.64:36204) 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 -- 172.21.15.64:0/207679186 learned_addr learned my addr 172.21.15.64:0/207679186 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:07.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9dd945700 1 --2- 172.21.15.64:0/207679186 >> v2:172.21.15.123:3300/0 conn(0x7fe9d806d210 0x7fe9d806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 -- 172.21.15.64:0/207679186 >> v2:172.21.15.123:3300/0 conn(0x7fe9d806d210 msgr2=0x7fe9d806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:07.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 --2- 172.21.15.64:0/207679186 >> v2:172.21.15.123:3300/0 conn(0x7fe9d806d210 0x7fe9d806d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 -- 172.21.15.64:0/207679186 >> v1:172.21.15.123:6789/0 conn(0x7fe9d80afbd0 legacy=0x7fe9d80b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 -- 172.21.15.64:0/207679186 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80b0940 con 0x7fe9d806b2c0 2023-09-27T23:30:07.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9de146700 1 --2- 172.21.15.64:0/207679186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806b2c0 0x7fe9d806b690 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80b25d0 tx=0x7fe9d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab495bfe0d54904 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:07.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9dd144700 1 -- 172.21.15.64:0/207679186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4009e70 con 0x7fe9d806b2c0 2023-09-27T23:30:07.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9dd144700 1 -- 172.21.15.64:0/207679186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d400b860 con 0x7fe9d806b2c0 2023-09-27T23:30:07.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9dd144700 1 -- 172.21.15.64:0/207679186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400bc00 con 0x7fe9d806b2c0 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- 172.21.15.64:0/207679186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806b2c0 msgr2=0x7fe9d806b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/207679186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806b2c0 0x7fe9d806b690 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80b25d0 tx=0x7fe9d4009250 comp rx=0 tx=0).stop 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- 172.21.15.64:0/207679186 shutdown_connections 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/207679186 >> v2:172.21.15.123:3300/0 conn(0x7fe9d806d210 0x7fe9d806d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/207679186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806b2c0 0x7fe9d806b690 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- 172.21.15.64:0/207679186 >> 172.21.15.64:0/207679186 conn(0x7fe9d80cc580 msgr2=0x7fe9d80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- 172.21.15.64:0/207679186 shutdown_connections 2023-09-27T23:30:07.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.291+0000 7fe9e03aa700 1 -- 172.21.15.64:0/207679186 wait complete. 2023-09-27T23:30:07.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 Processor -- start 2023-09-27T23:30:07.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 -- start start 2023-09-27T23:30:07.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9d806b2c0 0x7fe9d8144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 0x7fe9d80aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9d80afbd0 0x7fe9d80ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80c1670 con 0x7fe9d806d210 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d815ba70 con 0x7fe9d80afbd0 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9e03aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d815b180 con 0x7fe9d806b2c0 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 0x7fe9d80aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 0x7fe9d80aac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36210/0 (socket says 172.21.15.64:36210) 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 -- 172.21.15.64:0/1443662481 learned_addr learned my addr 172.21.15.64:0/1443662481 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9de146700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9d806b2c0 0x7fe9d8144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9de947700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9d80afbd0 0x7fe9d80ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 -- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9d806b2c0 msgr2=0x7fe9d8144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9d806b2c0 0x7fe9d8144310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 -- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9d80afbd0 msgr2=0x7fe9d80ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9d80afbd0 0x7fe9d80ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9dd945700 1 -- 172.21.15.64:0/1443662481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40094f0 con 0x7fe9d806d210 2023-09-27T23:30:07.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9de947700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9d80afbd0 0x7fe9d80ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:07.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.295+0000 7fe9de146700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9d806b2c0 0x7fe9d8144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:07.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.299+0000 7fe9dd945700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 0x7fe9d80aac90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800ec80 tx=0x7fe9c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:07.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.303+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800c9d0 con 0x7fe9d806d210 2023-09-27T23:30:07.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.303+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c800cb30 con 0x7fe9d806d210 2023-09-27T23:30:07.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.303+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d815b440 con 0x7fe9d806d210 2023-09-27T23:30:07.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.303+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8013470 con 0x7fe9d806d210 2023-09-27T23:30:07.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.303+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d815b700 con 0x7fe9d806d210 2023-09-27T23:30:07.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.307+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d815bbe0 con 0x7fe9d806d210 2023-09-27T23:30:07.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.307+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 34) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fe9c8013820 con 0x7fe9d806d210 2023-09-27T23:30:07.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.307+0000 7fe9ce7fc700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9bc060ac0 0x7fe9bc062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.307+0000 7fe9de146700 1 -- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9bc060ac0 msgr2=0x7fe9bc062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:07.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.307+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe9c807d4d0 con 0x7fe9d806d210 2023-09-27T23:30:07.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.307+0000 7fe9de146700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9bc060ac0 0x7fe9bc062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:07.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.315+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe9c804d500 con 0x7fe9d806d210 2023-09-27T23:30:07.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.479+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe9d8071120 con 0x7fe9d806d210 2023-09-27T23:30:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9ce7fc700 1 -- 172.21.15.64:0/1443662481 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe9c8050b20 con 0x7fe9d806d210 2023-09-27T23:30:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9bc060ac0 msgr2=0x7fe9bc062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9bc060ac0 0x7fe9bc062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 msgr2=0x7fe9d80aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 0x7fe9d80aac90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800ec80 tx=0x7fe9c80091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 shutdown_connections 2023-09-27T23:30:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9bc060ac0 0x7fe9bc062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9d80afbd0 0x7fe9d80ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9d806d210 0x7fe9d80aac90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 --2- 172.21.15.64:0/1443662481 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9d806b2c0 0x7fe9d8144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 >> 172.21.15.64:0/1443662481 conn(0x7fe9d80cc580 msgr2=0x7fe9d80b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 shutdown_connections 2023-09-27T23:30:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.491+0000 7fe9e03aa700 1 -- 172.21.15.64:0/1443662481 wait complete. 2023-09-27T23:30:07.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:30:07.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 Processor -- start 2023-09-27T23:30:07.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 -- start start 2023-09-27T23:30:07.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97f80c7170 0x7f97f80ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97f80acc60 0x7f97f80cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80c7610 con 0x7f97f8150200 2023-09-27T23:30:07.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80af530 con 0x7f97f80c7170 2023-09-27T23:30:07.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97fd010700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80ad080 con 0x7f97f80acc60 2023-09-27T23:30:07.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97f77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97f80c7170 0x7f97f80ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97f77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97f80c7170 0x7f97f80ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36222/0 (socket says 172.21.15.64:36222) 2023-09-27T23:30:07.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.615+0000 7f97f77fe700 1 -- 172.21.15.64:0/3901207564 learned_addr learned my addr 172.21.15.64:0/3901207564 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:07.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fc80f700 1 --2- 172.21.15.64:0/3901207564 >> v2:172.21.15.123:3300/0 conn(0x7f97f80acc60 0x7f97f80cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f77fe700 1 -- 172.21.15.64:0/3901207564 >> v2:172.21.15.123:3300/0 conn(0x7f97f80acc60 msgr2=0x7f97f80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:07.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f77fe700 1 --2- 172.21.15.64:0/3901207564 >> v2:172.21.15.123:3300/0 conn(0x7f97f80acc60 0x7f97f80cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f77fe700 1 -- 172.21.15.64:0/3901207564 >> v1:172.21.15.123:6789/0 conn(0x7f97f8150200 legacy=0x7f97f80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:07.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f77fe700 1 -- 172.21.15.64:0/3901207564 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80af250 con 0x7f97f80c7170 2023-09-27T23:30:07.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f77fe700 1 --2- 172.21.15.64:0/3901207564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f80ac720 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f97f806b3d0 tx=0x7f97ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e958375a17d6cf81 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:07.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f6ffd700 1 -- 172.21.15.64:0/3901207564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00b7f0 con 0x7f97f80c7170 2023-09-27T23:30:07.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f6ffd700 1 -- 172.21.15.64:0/3901207564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec00b950 con 0x7f97f80c7170 2023-09-27T23:30:07.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97f6ffd700 1 -- 172.21.15.64:0/3901207564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00bcf0 con 0x7f97f80c7170 2023-09-27T23:30:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 -- 172.21.15.64:0/3901207564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 msgr2=0x7f97f80ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 --2- 172.21.15.64:0/3901207564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f80ac720 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f97f806b3d0 tx=0x7f97ec009250 comp rx=0 tx=0).stop 2023-09-27T23:30:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 -- 172.21.15.64:0/3901207564 shutdown_connections 2023-09-27T23:30:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 --2- 172.21.15.64:0/3901207564 >> v2:172.21.15.123:3300/0 conn(0x7f97f80acc60 0x7f97f80cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 --2- 172.21.15.64:0/3901207564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f80ac720 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 -- 172.21.15.64:0/3901207564 >> 172.21.15.64:0/3901207564 conn(0x7f97f80ccb30 msgr2=0x7f97f80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 -- 172.21.15.64:0/3901207564 shutdown_connections 2023-09-27T23:30:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 -- 172.21.15.64:0/3901207564 wait complete. 2023-09-27T23:30:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 Processor -- start 2023-09-27T23:30:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.619+0000 7f97fd010700 1 -- start start 2023-09-27T23:30:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f80acc60 0x7f97f8069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f8141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97f8150200 0x7f97f80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8069090 con 0x7f97f80c7170 2023-09-27T23:30:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80681a0 con 0x7f97f80acc60 2023-09-27T23:30:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8068480 con 0x7f97f8150200 2023-09-27T23:30:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f8141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f8141b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36230/0 (socket says 172.21.15.64:36230) 2023-09-27T23:30:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 -- 172.21.15.64:0/646560365 learned_addr learned my addr 172.21.15.64:0/646560365 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f7fff700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f80acc60 0x7f97f8069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fc80f700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97f8150200 0x7f97f80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 -- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97f8150200 msgr2=0x7f97f80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97f8150200 0x7f97f80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 -- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f80acc60 msgr2=0x7f97f8069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f80acc60 0x7f97f8069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 -- 172.21.15.64:0/646560365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0094f0 con 0x7f97f80c7170 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fc80f700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97f8150200 0x7f97f80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97f77fe700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f8141b30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009290 tx=0x7f97ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec006bc0 con 0x7f97f80c7170 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f8068760 con 0x7f97f80c7170 2023-09-27T23:30:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec00bc60 con 0x7f97f80c7170 2023-09-27T23:30:07.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0238c0 con 0x7f97f80c7170 2023-09-27T23:30:07.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f8068d20 con 0x7f97f80c7170 2023-09-27T23:30:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f8069200 con 0x7f97f80c7170 2023-09-27T23:30:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.623+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 34) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f97ec01f040 con 0x7f97f80c7170 2023-09-27T23:30:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.627+0000 7f97dffff700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f97e4060ac0 0x7f97e4062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.627+0000 7f97f7fff700 1 -- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f97e4060ac0 msgr2=0x7f97e4062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.627+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f97ec082410 con 0x7f97f80c7170 2023-09-27T23:30:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.627+0000 7f97f7fff700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f97e4060ac0 0x7f97e4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:07.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.631+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f97ec052440 con 0x7f97f80c7170 2023-09-27T23:30:07.697 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:07.694+0000 7f05f9d58000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:30:07.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:07.742+0000 7f05f9d58000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:30:07.752 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:07.751+0000 7f1f630a1000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:30:07.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.799+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f97f804be40 con 0x7f97f80c7170 2023-09-27T23:30:07.806 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:07.803+0000 7f1f630a1000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:30:07.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.807+0000 7f97dffff700 1 -- 172.21.15.64:0/646560365 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f97ec055a60 con 0x7f97f80c7170 2023-09-27T23:30:07.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f97e4060ac0 msgr2=0x7f97e4062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:07.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f97e4060ac0 0x7f97e4062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 msgr2=0x7f97f8141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f8141b30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009290 tx=0x7f97ec018000 comp rx=0 tx=0).stop 2023-09-27T23:30:07.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 shutdown_connections 2023-09-27T23:30:07.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f97e4060ac0 0x7f97e4062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97f8150200 0x7f97f80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f80c7170 0x7f97f8141b30 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 --2- 172.21.15.64:0/646560365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f80acc60 0x7f97f8069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 >> 172.21.15.64:0/646560365 conn(0x7f97f80ccb30 msgr2=0x7f97f80c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:07.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 shutdown_connections 2023-09-27T23:30:07.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.811+0000 7f97fd010700 1 -- 172.21.15.64:0/646560365 wait complete. 2023-09-27T23:30:07.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-09-27T23:30:07.851 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:07.846+0000 7f05f9d58000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:07.920 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:07.919+0000 7f1f630a1000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:07.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.931+0000 7f6441ff1700 1 Processor -- start 2023-09-27T23:30:07.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.931+0000 7f6441ff1700 1 -- start start 2023-09-27T23:30:07.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f6441ff1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 0x7f643c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f6441ff1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c0c7e70 con 0x7f643c156b90 2023-09-27T23:30:07.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f6441ff1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c0c72a0 con 0x7f643c156f60 2023-09-27T23:30:07.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f6441ff1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c8c50 con 0x7f643c0c8300 2023-09-27T23:30:07.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f643c156b90 0x7f643c0ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56656/0 (socket says 172.21.15.64:56656) 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 -- 172.21.15.64:0/689593206 learned_addr learned my addr 172.21.15.64:0/689593206 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643bfff700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 0x7f643c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438844655 0 0) 0x7f643c0c7e70 con 0x7f643c156b90 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6424003570 con 0x7f643c156b90 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148010240 0 0) 0x7f643c0c72a0 con 0x7f643c156f60 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c0c7e70 con 0x7f643c156f60 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1283919489 0 0) 0x7f6424003570 con 0x7f643c156b90 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 msgr2=0x7f643c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 0x7f643c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 >> v1:172.21.15.123:6789/0 conn(0x7f643c156f60 legacy=0x7f643c0bfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c0c7540 con 0x7f643c156b90 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f642c001470 con 0x7f643c156b90 2023-09-27T23:30:07.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 >> v1:172.21.15.64:6789/0 conn(0x7f643c156b90 legacy=0x7f643c0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f6424004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 0x7f6424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6424004d50 0x7f64240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c8c50 con 0x7f6424003c20 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642400bbf0 con 0x7f6424004d50 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642400bdb0 con 0x7f643c0c8300 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f6424004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643affd700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 0x7f6424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643bfff700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6424004d50 0x7f64240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 -- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 msgr2=0x7f6424004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c0c8300 0x7f6424004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 -- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6424004d50 msgr2=0x7f64240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:07.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6424004d50 0x7f64240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 -- 172.21.15.64:0/689593206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6424003570 con 0x7f6424003c20 2023-09-27T23:30:07.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643b7fe700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f6424004010 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f642c007fd0 tx=0x7f643c0ae1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c973ca627c0c471a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:07.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00cae0 con 0x7f6424003c20 2023-09-27T23:30:07.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c00cc60 con 0x7f6424003c20 2023-09-27T23:30:07.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.935+0000 7f643a7fc700 1 -- 172.21.15.64:0/689593206 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c005920 con 0x7f6424003c20 2023-09-27T23:30:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 -- 172.21.15.64:0/689593206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 msgr2=0x7f6424004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 --2- 172.21.15.64:0/689593206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f6424004010 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f642c007fd0 tx=0x7f643c0ae1d0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f643bfff700 1 -- 172.21.15.64:0/689593206 reap_dead start 2023-09-27T23:30:07.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 -- 172.21.15.64:0/689593206 shutdown_connections 2023-09-27T23:30:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 -- 172.21.15.64:0/689593206 >> 172.21.15.64:0/689593206 conn(0x7f643c0c8db0 msgr2=0x7f643c0c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 -- 172.21.15.64:0/689593206 shutdown_connections 2023-09-27T23:30:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 -- 172.21.15.64:0/689593206 wait complete. 2023-09-27T23:30:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 Processor -- start 2023-09-27T23:30:07.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.939+0000 7f6441ff1700 1 -- start start 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f643c166d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f643c152de0 0x7f643c144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c1672b0 0x7f643c16a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c146170 con 0x7f6424003c20 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c145560 con 0x7f643c152de0 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c145840 con 0x7f643c1672b0 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f643c166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f643c152de0 0x7f643c144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f643c166d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36248/0 (socket says 172.21.15.64:36248) 2023-09-27T23:30:07.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 -- 172.21.15.64:0/577793557 learned_addr learned my addr 172.21.15.64:0/577793557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643bfff700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c1672b0 0x7f643c16a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c1672b0 msgr2=0x7f643c16a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f643c1672b0 0x7f643c16a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f643c152de0 msgr2=0x7f643c144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f643c152de0 0x7f643c144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 -- 172.21.15.64:0/577793557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c145b20 con 0x7f6424003c20 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f643b7fe700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424003c20 0x7f643c166d70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f64240057d0 tx=0x7f642c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c024070 con 0x7f6424003c20 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c004830 con 0x7f6424003c20 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 -- 172.21.15.64:0/577793557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c145e00 con 0x7f6424003c20 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c004bb0 con 0x7f6424003c20 2023-09-27T23:30:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 -- 172.21.15.64:0/577793557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c144f90 con 0x7f6424003c20 2023-09-27T23:30:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.943+0000 7f6441ff1700 1 -- 172.21.15.64:0/577793557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c145f90 con 0x7f6424003c20 2023-09-27T23:30:07.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.947+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 34) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f642c007070 con 0x7f6424003c20 2023-09-27T23:30:07.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.947+0000 7f64237fe700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:07.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.947+0000 7f643affd700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 msgr2=0x7f6428062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:07.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.947+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f642c07e4f0 con 0x7f6424003c20 2023-09-27T23:30:07.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.947+0000 7f643affd700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:07.955+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f642c04e520 con 0x7f6424003c20 2023-09-27T23:30:08.046 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.042+0000 7f05f9d58000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:30:08.094 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.090+0000 7f05f9d58000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:08.139 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.135+0000 7f1f630a1000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:30:08.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:08.135+0000 7f6441ff1700 1 -- 172.21.15.64:0/577793557 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f643c1462b0 con 0x7f6428060ac0 2023-09-27T23:30:08.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:08.151+0000 7f643affd700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 msgr2=0x7f6428062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:08.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:08.151+0000 7f643affd700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:30:08.193 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.191+0000 7f1f630a1000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:08.246 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.242+0000 7f05f9d58000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:30:08.303 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.298+0000 7f05f9d58000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:30:08.362 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.358+0000 7f05f9d58000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:30:08.363 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.359+0000 7f1f630a1000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:30:08.426 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.423+0000 7f1f630a1000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:30:08.492 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.487+0000 7f1f630a1000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:30:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:08.555+0000 7f643affd700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 msgr2=0x7f6428062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:08.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:08.555+0000 7f643affd700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:30:08.659 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.654+0000 7f05f9d58000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:30:08.770 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:08.766+0000 7f05f9d58000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:30:08.827 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.823+0000 7f1f630a1000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:30:08.951 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:08.947+0000 7f1f630a1000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:30:09.050 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.046+0000 7f05f9d58000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:30:09.212 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.210+0000 7f05f9d58000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:09.269 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.267+0000 7f1f630a1000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:30:09.276 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.274+0000 7f05f9d58000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:30:09.330 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.326+0000 7f05f9d58000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:30:09.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:09.359+0000 7f643affd700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 msgr2=0x7f6428062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:09.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:09.359+0000 7f643affd700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:30:09.382 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.378+0000 7f05f9d58000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:30:09.442 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.439+0000 7f1f630a1000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:09.510 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.507+0000 7f1f630a1000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:30:09.556 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.554+0000 7f05f9d58000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:09.563 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.559+0000 7f1f630a1000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:30:09.611 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.606+0000 7f05f9d58000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:30:09.621 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.619+0000 7f1f630a1000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:30:09.673 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.670+0000 7f05f9d58000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:30:09.801 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.799+0000 7f1f630a1000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:09.825 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.822+0000 7f05f9d58000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:30:09.856 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.855+0000 7f1f630a1000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:30:09.886 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:09.882+0000 7f05f9d58000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:30:09.921 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:09.919+0000 7f1f630a1000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:30:10.077 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:10.075+0000 7f1f630a1000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:30:10.141 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:10.139+0000 7f1f630a1000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:30:10.478 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.474+0000 7f05f9d58000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:30:10.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:10.515+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 35) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f642c04e8f0 con 0x7f6424003c20 2023-09-27T23:30:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:10.515+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 msgr2=0x7f6428062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:10.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:10.515+0000 7f64237fe700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:10.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05ba703700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05ba703700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05ba703700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05ba703700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05ba703700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.686 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05bd709700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.686 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05bd709700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.686 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05bd709700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.686 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05bd709700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.686 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:10.682+0000 7f05bd709700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:10.774 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:10.771+0000 7f1f630a1000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:30:11.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.519+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 36) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f642c058d30 con 0x7f6424003c20 2023-09-27T23:30:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.519+0000 7f64237fe700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.519+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6428069510 con 0x7f6428060ac0 2023-09-27T23:30:11.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.527+0000 7f643affd700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.527+0000 7f643affd700 1 --2- 172.21.15.64:0/577793557 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6428060ac0 0x7f6428062ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f643c1531d0 tx=0x7f6430009040 comp rx=0 tx=0).ready entity=mgr.7291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:11.540 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:11.538+0000 7f05c4717700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-09-27T23:30:11.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.539+0000 7f64237fe700 1 -- 172.21.15.64:0/577793557 <== mgr.7291 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f6428069510 con 0x7f6428060ac0 2023-09-27T23:30:11.544 INFO:teuthology.orchestra.run.smithi064.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-09-27T23:30:11.547 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-09-27T23:30:11.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:30:11.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 Processor -- start 2023-09-27T23:30:11.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 -- start start 2023-09-27T23:30:11.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69940ce2d0 0x7f69940ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f69940cc040 0x7f6994159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69940cbbb0 con 0x7f69940caa10 2023-09-27T23:30:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699406dc90 con 0x7f69940ce2d0 2023-09-27T23:30:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6999330700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69940cafa0 con 0x7f69940cc040 2023-09-27T23:30:11.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69940ce2d0 0x7f69940ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69940ce2d0 0x7f69940ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36254/0 (socket says 172.21.15.64:36254) 2023-09-27T23:30:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 -- 172.21.15.64:0/1862666406 learned_addr learned my addr 172.21.15.64:0/1862666406 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:11.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6998b2f700 1 --2- 172.21.15.64:0/1862666406 >> v2:172.21.15.123:3300/0 conn(0x7f69940cc040 0x7f6994159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 -- 172.21.15.64:0/1862666406 >> v2:172.21.15.123:3300/0 conn(0x7f69940cc040 msgr2=0x7f6994159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:11.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 --2- 172.21.15.64:0/1862666406 >> v2:172.21.15.123:3300/0 conn(0x7f69940cc040 0x7f6994159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:11.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 -- 172.21.15.64:0/1862666406 >> v1:172.21.15.123:6789/0 conn(0x7f69940caa10 legacy=0x7f6994158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:11.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 -- 172.21.15.64:0/1862666406 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940cb280 con 0x7f69940ce2d0 2023-09-27T23:30:11.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6993fff700 1 --2- 172.21.15.64:0/1862666406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f69940ce6a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f69940ca350 tx=0x7f6988009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddea2ae8f9a91678 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:11.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6992ffd700 1 -- 172.21.15.64:0/1862666406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988009e70 con 0x7f69940ce2d0 2023-09-27T23:30:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6992ffd700 1 -- 172.21.15.64:0/1862666406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698800b860 con 0x7f69940ce2d0 2023-09-27T23:30:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.659+0000 7f6992ffd700 1 -- 172.21.15.64:0/1862666406 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698800bc00 con 0x7f69940ce2d0 2023-09-27T23:30:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- 172.21.15.64:0/1862666406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 msgr2=0x7f69940ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 --2- 172.21.15.64:0/1862666406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f69940ce6a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f69940ca350 tx=0x7f6988009250 comp rx=0 tx=0).stop 2023-09-27T23:30:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- 172.21.15.64:0/1862666406 shutdown_connections 2023-09-27T23:30:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 --2- 172.21.15.64:0/1862666406 >> v2:172.21.15.123:3300/0 conn(0x7f69940cc040 0x7f6994159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:11.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 --2- 172.21.15.64:0/1862666406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f69940ce6a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:11.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- 172.21.15.64:0/1862666406 >> 172.21.15.64:0/1862666406 conn(0x7f6994067bb0 msgr2=0x7f69940cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:11.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- 172.21.15.64:0/1862666406 shutdown_connections 2023-09-27T23:30:11.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- 172.21.15.64:0/1862666406 wait complete. 2023-09-27T23:30:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 Processor -- start 2023-09-27T23:30:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- start start 2023-09-27T23:30:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69940caa10 0x7f6994149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69940cc040 0x7f69940b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f6994159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994149e50 con 0x7f69940ce2d0 2023-09-27T23:30:11.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699415bfa0 con 0x7f69940cc040 2023-09-27T23:30:11.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6999330700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6994158770 con 0x7f69940caa10 2023-09-27T23:30:11.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f6994159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f6994159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36264/0 (socket says 172.21.15.64:36264) 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 -- 172.21.15.64:0/2869647890 learned_addr learned my addr 172.21.15.64:0/2869647890 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6993fff700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69940caa10 0x7f6994149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f69937fe700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69940cc040 0x7f69940b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 -- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69940caa10 msgr2=0x7f6994149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69940caa10 0x7f6994149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:11.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 -- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69940cc040 msgr2=0x7f69940b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:11.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69940cc040 0x7f69940b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:11.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.663+0000 7f6998b2f700 1 -- 172.21.15.64:0/2869647890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880094f0 con 0x7f69940ce2d0 2023-09-27T23:30:11.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f6998b2f700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f6994159250 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f698c009e90 tx=0x7f698c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:11.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c00cc20 con 0x7f69940ce2d0 2023-09-27T23:30:11.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c00f040 con 0x7f69940ce2d0 2023-09-27T23:30:11.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6994158a50 con 0x7f69940ce2d0 2023-09-27T23:30:11.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c014480 con 0x7f69940ce2d0 2023-09-27T23:30:11.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6994154d00 con 0x7f69940ce2d0 2023-09-27T23:30:11.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6994155eb0 con 0x7f69940ce2d0 2023-09-27T23:30:11.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.667+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 36) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f698c014830 con 0x7f69940ce2d0 2023-09-27T23:30:11.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.671+0000 7f697bfff700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69800607f0 0x7f6980062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:11.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.671+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f698c07d930 con 0x7f69940ce2d0 2023-09-27T23:30:11.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.671+0000 7f6993fff700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69800607f0 0x7f6980062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:11.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.671+0000 7f6993fff700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69800607f0 0x7f6980062ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69940ca350 tx=0x7f6988017000 comp rx=0 tx=0).ready entity=mgr.7291 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:11.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.679+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6994155eb0 con 0x7f69940ce2d0 2023-09-27T23:30:11.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:11.887+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f6994155eb0 con 0x7f69940ce2d0 2023-09-27T23:30:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.575+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f698c04dec0 con 0x7f69940ce2d0 2023-09-27T23:30:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.575+0000 7f697bfff700 1 -- 172.21.15.64:0/2869647890 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7f698c04fb50 con 0x7f69940ce2d0 2023-09-27T23:30:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69800607f0 msgr2=0x7f6980062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69800607f0 0x7f6980062ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69940ca350 tx=0x7f6988017000 comp rx=0 tx=0).stop 2023-09-27T23:30:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 msgr2=0x7f6994159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f6994159250 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f698c009e90 tx=0x7f698c00b040 comp rx=0 tx=0).stop 2023-09-27T23:30:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 shutdown_connections 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f69800607f0 0x7f6980062ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69940ce2d0 0x7f6994159250 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69940cc040 0x7f69940b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 --2- 172.21.15.64:0/2869647890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69940caa10 0x7f6994149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 >> 172.21.15.64:0/2869647890 conn(0x7f6994067bb0 msgr2=0x7f69941557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 shutdown_connections 2023-09-27T23:30:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.579+0000 7f6999330700 1 -- 172.21.15.64:0/2869647890 wait complete. 2023-09-27T23:30:12.599 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:30:12.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:30:12.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:30:12.636 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:30:12.637 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:30:12.637 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:30:12.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:12.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[51251]: fuse finished with error 0 and tester_r 0 2023-09-27T23:30:12.679 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:30:12.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:12.680 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:30:12.692 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:12.690+0000 7f82809ff000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:30:12.703 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:12.699+0000 7f87c0c8c000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:30:12.803 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:30:12.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:12.803 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:30:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:30:12.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:30:12.811 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:30:12.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:12.811 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:30:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:30:12.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:30:12.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:30:12.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:12.851 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:30:12.857 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:30:12.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:30:12.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:12.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 Processor -- start 2023-09-27T23:30:12.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- start start 2023-09-27T23:30:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d40067c90 0x7f0d40068060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d4014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d400c6630 con 0x7f0d400c5900 2023-09-27T23:30:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d400c6440 con 0x7f0d40067c90 2023-09-27T23:30:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400c6910 con 0x7f0d4014fba0 2023-09-27T23:30:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d40067c90 0x7f0d40068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d40067c90 0x7f0d40068060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36298/0 (socket says 172.21.15.64:36298) 2023-09-27T23:30:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 -- 172.21.15.64:0/2011090965 learned_addr learned my addr 172.21.15.64:0/2011090965 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3dd9b700 1 --2- 172.21.15.64:0/2011090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d4014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 -- 172.21.15.64:0/2011090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 msgr2=0x7f0d4014ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 --2- 172.21.15.64:0/2011090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d4014ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 -- 172.21.15.64:0/2011090965 >> v1:172.21.15.64:6789/0 conn(0x7f0d400c5900 legacy=0x7f0d400c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 -- 172.21.15.64:0/2011090965 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400c6bf0 con 0x7f0d40067c90 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3e59c700 1 --2- 172.21.15.64:0/2011090965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d40068060 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0d300099b0 tx=0x7f0d30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c879a496327fdba server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3d59a700 1 -- 172.21.15.64:0/2011090965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000b860 con 0x7f0d40067c90 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3d59a700 1 -- 172.21.15.64:0/2011090965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3000b9c0 con 0x7f0d40067c90 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d3d59a700 1 -- 172.21.15.64:0/2011090965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000bd60 con 0x7f0d40067c90 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- 172.21.15.64:0/2011090965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 msgr2=0x7f0d40068060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 --2- 172.21.15.64:0/2011090965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d40068060 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0d300099b0 tx=0x7f0d30009250 comp rx=0 tx=0).stop 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- 172.21.15.64:0/2011090965 shutdown_connections 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 --2- 172.21.15.64:0/2011090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d4014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 --2- 172.21.15.64:0/2011090965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d40068060 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- 172.21.15.64:0/2011090965 >> 172.21.15.64:0/2011090965 conn(0x7f0d400cd6d0 msgr2=0x7f0d4006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- 172.21.15.64:0/2011090965 shutdown_connections 2023-09-27T23:30:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.963+0000 7f0d44a45700 1 -- 172.21.15.64:0/2011090965 wait complete. 2023-09-27T23:30:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 Processor -- start 2023-09-27T23:30:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 -- start start 2023-09-27T23:30:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d4015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d400c5900 0x7f0d4014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d400c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4015b910 con 0x7f0d40067c90 2023-09-27T23:30:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4015af40 con 0x7f0d400c5900 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400ca3f0 con 0x7f0d4014fba0 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d4015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d4015b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36310/0 (socket says 172.21.15.64:36310) 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 -- 172.21.15.64:0/607969886 learned_addr learned my addr 172.21.15.64:0/607969886 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3dd9b700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d400c5900 0x7f0d4014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3ed9d700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d400c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 -- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 msgr2=0x7f0d400c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d400c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 -- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d400c5900 msgr2=0x7f0d4014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d400c5900 0x7f0d4014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 -- 172.21.15.64:0/607969886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300094f0 con 0x7f0d40067c90 2023-09-27T23:30:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3ed9d700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d400c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3dd9b700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d400c5900 0x7f0d4014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d3e59c700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d4015b3d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009fd0 tx=0x7f0d30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30024440 con 0x7f0d40067c90 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d300245a0 con 0x7f0d40067c90 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d400ca110 con 0x7f0d40067c90 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30024940 con 0x7f0d40067c90 2023-09-27T23:30:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.967+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4015abd0 con 0x7f0d40067c90 2023-09-27T23:30:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.971+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d40071120 con 0x7f0d40067c90 2023-09-27T23:30:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.971+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f0d3001a420 con 0x7f0d40067c90 2023-09-27T23:30:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.971+0000 7f0d2e7fc700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 0x7f0d20062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.971+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f0d3007f290 con 0x7f0d40067c90 2023-09-27T23:30:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.971+0000 7f0d3dd9b700 1 -- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 msgr2=0x7f0d20062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.971+0000 7f0d3dd9b700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 0x7f0d20062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:12.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:12.979+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0d3004eff0 con 0x7f0d40067c90 2023-09-27T23:30:13.107 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.102+0000 7f82809ff000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:30:13.148 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.147+0000 7f87c0c8c000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:30:13.155 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.150+0000 7f82809ff000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:30:13.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.171+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d400642a0 con 0x7f0d40067c90 2023-09-27T23:30:13.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d3dd9b700 1 -- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 msgr2=0x7f0d20062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:13.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d3dd9b700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 0x7f0d20062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:30:13.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/607969886 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f0d30052610 con 0x7f0d40067c90 2023-09-27T23:30:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:13.177 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:54.107357+0000","modified":"2023-09-27T23:29:56.124127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1623763192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1623763192},{"type":"v1","addr":"172.21.15.64:6835","nonce":1623763192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-09-27T23:30:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 msgr2=0x7f0d20062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 0x7f0d20062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 msgr2=0x7f0d4015b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d4015b3d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009fd0 tx=0x7f0d30017000 comp rx=0 tx=0).stop 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 shutdown_connections 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d20060b00 0x7f0d20062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d4014fba0 0x7f0d400c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d400c5900 0x7f0d4014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 --2- 172.21.15.64:0/607969886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d40067c90 0x7f0d4015b3d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 >> 172.21.15.64:0/607969886 conn(0x7f0d400cd6d0 msgr2=0x7f0d400b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 shutdown_connections 2023-09-27T23:30:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.175+0000 7f0d44a45700 1 -- 172.21.15.64:0/607969886 wait complete. 2023-09-27T23:30:13.181 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2023-09-27T23:30:13.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:13.203 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.199+0000 7f87c0c8c000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:30:13.260 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.258+0000 7f82809ff000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:13.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7eff06036700 1 Processor -- start 2023-09-27T23:30:13.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7eff06036700 1 -- start start 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7eff06036700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff000c97c0 0x7eff00161e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7eff06036700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000cb030 con 0x7eff0006b990 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7eff06036700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000cb290 con 0x7eff0015a210 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7eff06036700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff001599a0 con 0x7eff000c97c0 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefeffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7eff0006b990 0x7eff000c90b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56674/0 (socket says 172.21.15.64:56674) 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefeffd700 1 -- 172.21.15.64:0/3037986732 learned_addr learned my addr 172.21.15.64:0/3037986732 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:13.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefffff700 1 --2- 172.21.15.64:0/3037986732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff000c97c0 0x7eff00161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598611976 0 0) 0x7eff000cb030 con 0x7eff0006b990 2023-09-27T23:30:13.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee8003570 con 0x7eff0006b990 2023-09-27T23:30:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653224494 0 0) 0x7eff000cb290 con 0x7eff0015a210 2023-09-27T23:30:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff000cb030 con 0x7eff0015a210 2023-09-27T23:30:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefffff700 1 -- 172.21.15.64:0/3037986732 >> v1:172.21.15.123:6789/0 conn(0x7eff0015a210 legacy=0x7eff0006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefffff700 1 -- 172.21.15.64:0/3037986732 >> v1:172.21.15.64:6789/0 conn(0x7eff0006b990 legacy=0x7eff000c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542341540 0 0) 0x7efee8003570 con 0x7eff0006b990 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.287+0000 7efefffff700 1 -- 172.21.15.64:0/3037986732 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000cb570 con 0x7eff000c97c0 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7efefffff700 1 --2- 172.21.15.64:0/3037986732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff000c97c0 0x7eff00161e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efef8009e90 tx=0x7efef8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7b58b2660ebdc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef800e070 con 0x7eff000c97c0 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef800b870 con 0x7eff000c97c0 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7efefe7fc700 1 -- 172.21.15.64:0/3037986732 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4001490 con 0x7eff000c97c0 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 -- 172.21.15.64:0/3037986732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff000c97c0 msgr2=0x7eff00161e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 --2- 172.21.15.64:0/3037986732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff000c97c0 0x7eff00161e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efef8009e90 tx=0x7efef8009780 comp rx=0 tx=0).stop 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 -- 172.21.15.64:0/3037986732 shutdown_connections 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 --2- 172.21.15.64:0/3037986732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff000c97c0 0x7eff00161e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 -- 172.21.15.64:0/3037986732 >> 172.21.15.64:0/3037986732 conn(0x7eff00067410 msgr2=0x7eff000677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:13.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 -- 172.21.15.64:0/3037986732 shutdown_connections 2023-09-27T23:30:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 -- 172.21.15.64:0/3037986732 wait complete. 2023-09-27T23:30:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 Processor -- start 2023-09-27T23:30:13.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.291+0000 7eff06036700 1 -- start start 2023-09-27T23:30:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff0006b990 0x7eff00141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 0x7eff000ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff0015a210 0x7eff000c1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000ae7a0 con 0x7eff000c97c0 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000adb90 con 0x7eff0006b990 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff000ade70 con 0x7eff0015a210 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 0x7eff000ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 0x7eff000ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36322/0 (socket says 172.21.15.64:36322) 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 -- 172.21.15.64:0/2507957696 learned_addr learned my addr 172.21.15.64:0/2507957696 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefffff700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff0015a210 0x7eff000c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efeff7fe700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff0006b990 0x7eff00141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 -- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff0015a210 msgr2=0x7eff000c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff0015a210 0x7eff000c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 -- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff0006b990 msgr2=0x7eff00141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff0006b990 0x7eff00141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 -- 172.21.15.64:0/2507957696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8009a20 con 0x7eff000c97c0 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efefeffd700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 0x7eff000ad180 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efef400cb60 tx=0x7efef4001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4007e90 con 0x7eff000c97c0 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400ce70 con 0x7eff000c97c0 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff000ae150 con 0x7eff000c97c0 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4007590 con 0x7eff000c97c0 2023-09-27T23:30:13.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff001415b0 con 0x7eff000c97c0 2023-09-27T23:30:13.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff000ae910 con 0x7eff000c97c0 2023-09-27T23:30:13.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.295+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7efef4004a40 con 0x7eff000c97c0 2023-09-27T23:30:13.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.299+0000 7efee77fe700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efeec060b00 0x7efeec062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.299+0000 7efeff7fe700 1 -- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efeec060b00 msgr2=0x7efeec062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:13.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.299+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7efef407ca30 con 0x7eff000c97c0 2023-09-27T23:30:13.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.299+0000 7efeff7fe700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efeec060b00 0x7efeec062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:13.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.303+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efef404abb0 con 0x7eff000c97c0 2023-09-27T23:30:13.315 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.311+0000 7f87c0c8c000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:13.456 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.454+0000 7f82809ff000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:30:13.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.471+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff00071120 con 0x7eff000c97c0 2023-09-27T23:30:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.471+0000 7efee77fe700 1 -- 172.21.15.64:0/2507957696 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17575 (secure 0 0 0) 0x7efef4018090 con 0x7eff000c97c0 2023-09-27T23:30:13.476 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:13.476 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":102,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:10.499965+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:52.694282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:53.074953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/1017331161":"2023-09-28T23:30:10.499869+0000","172.21.15.123:6833/33228":"2023-09-28T23:30:10.499869+0000","172.21.15.123:6832/33228":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/4275324614":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/2296684733":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/1309790114":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/1755447900":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/739794733":"2023-09-28T23:30:10.499869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:13.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efeec060b00 msgr2=0x7efeec062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efeec060b00 0x7efeec062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 msgr2=0x7eff000ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 0x7eff000ad180 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efef400cb60 tx=0x7efef4001af0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 shutdown_connections 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efeec060b00 0x7efeec062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff0015a210 0x7eff000c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000c97c0 0x7eff000ad180 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 --2- 172.21.15.64:0/2507957696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff0006b990 0x7eff00141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 >> 172.21.15.64:0/2507957696 conn(0x7eff00067410 msgr2=0x7eff000c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:13.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 shutdown_connections 2023-09-27T23:30:13.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.475+0000 7eff06036700 1 -- 172.21.15.64:0/2507957696 wait complete. 2023-09-27T23:30:13.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:29:52.694282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:29:53.074953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:30:13.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:30:13.504 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.502+0000 7f82809ff000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:13.522 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.519+0000 7f87c0c8c000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:30:13.580 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.579+0000 7f87c0c8c000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:13.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0516e700 1 Processor -- start 2023-09-27T23:30:13.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0516e700 1 -- start start 2023-09-27T23:30:13.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0516e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e00157fc0 0x7f4e00158480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0516e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000c7820 con 0x7f4e0006d050 2023-09-27T23:30:13.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0516e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000c7210 con 0x7f4e001547a0 2023-09-27T23:30:13.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0516e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000ade40 con 0x7f4e00157fc0 2023-09-27T23:30:13.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0496d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e0006d050 0x7f4e000c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56684/0 (socket says 172.21.15.64:56684) 2023-09-27T23:30:13.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4e0496d700 1 -- 172.21.15.64:0/3539361620 learned_addr learned my addr 172.21.15.64:0/3539361620 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:13.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dff7fe700 1 --2- 172.21.15.64:0/3539361620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e00157fc0 0x7f4e00158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225479377 0 0) 0x7f4e000c7820 con 0x7f4e0006d050 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dec003570 con 0x7f4e0006d050 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564340826 0 0) 0x7f4e000c7210 con 0x7f4e001547a0 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e000c7820 con 0x7f4e001547a0 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dff7fe700 1 -- 172.21.15.64:0/3539361620 >> v1:172.21.15.123:6789/0 conn(0x7f4e001547a0 legacy=0x7f4e000ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dff7fe700 1 -- 172.21.15.64:0/3539361620 >> v1:172.21.15.64:6789/0 conn(0x7f4e0006d050 legacy=0x7f4e000c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.583+0000 7f4dff7fe700 1 -- 172.21.15.64:0/3539361620 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e000ae030 con 0x7f4e00157fc0 2023-09-27T23:30:13.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4dff7fe700 1 --2- 172.21.15.64:0/3539361620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e00157fc0 0x7f4e00158480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009910 tx=0x7f4de80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cded79c60bf8e8f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8011630 con 0x7f4e00157fc0 2023-09-27T23:30:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8011790 con 0x7f4e00157fc0 2023-09-27T23:30:13.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4dfeffd700 1 -- 172.21.15.64:0/3539361620 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8011b10 con 0x7f4e00157fc0 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 -- 172.21.15.64:0/3539361620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e00157fc0 msgr2=0x7f4e00158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 --2- 172.21.15.64:0/3539361620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e00157fc0 0x7f4e00158480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009910 tx=0x7f4de80091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 -- 172.21.15.64:0/3539361620 shutdown_connections 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 --2- 172.21.15.64:0/3539361620 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e00157fc0 0x7f4e00158480 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 -- 172.21.15.64:0/3539361620 >> 172.21.15.64:0/3539361620 conn(0x7f4e000ccb30 msgr2=0x7f4e000ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 -- 172.21.15.64:0/3539361620 shutdown_connections 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 -- 172.21.15.64:0/3539361620 wait complete. 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 Processor -- start 2023-09-27T23:30:13.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.587+0000 7f4e0516e700 1 -- start start 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0006d050 0x7f4e00148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e001547a0 0x7f4e00167da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 0x7f4e00069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00156c60 con 0x7f4e00157fc0 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00149610 con 0x7f4e001547a0 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001682e0 con 0x7f4e0006d050 2023-09-27T23:30:13.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 0x7f4e00069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 0x7f4e00069230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36336/0 (socket says 172.21.15.64:36336) 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 -- 172.21.15.64:0/4151464315 learned_addr learned my addr 172.21.15.64:0/4151464315 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4dff7fe700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e001547a0 0x7f4e00167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4dfffff700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0006d050 0x7f4e00148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 -- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0006d050 msgr2=0x7f4e00148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0006d050 0x7f4e00148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 -- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e001547a0 msgr2=0x7f4e00167da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e001547a0 0x7f4e00167da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 -- 172.21.15.64:0/4151464315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8009420 con 0x7f4e00157fc0 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4dfffff700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0006d050 0x7f4e00148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4dff7fe700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e001547a0 0x7f4e00167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0496d700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 0x7f4e00069230 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4df800cc60 tx=0x7f4df8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:13.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800ce70 con 0x7f4e00157fc0 2023-09-27T23:30:13.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df800ab70 con 0x7f4e00157fc0 2023-09-27T23:30:13.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00149000 con 0x7f4e00157fc0 2023-09-27T23:30:13.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df80070f0 con 0x7f4e00157fc0 2023-09-27T23:30:13.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e001492a0 con 0x7f4e00157fc0 2023-09-27T23:30:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e00148e60 con 0x7f4e00157fc0 2023-09-27T23:30:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.591+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f4df8007460 con 0x7f4e00157fc0 2023-09-27T23:30:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.595+0000 7f4df7fff700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4de0060b00 0x7f4de0062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.595+0000 7f4dfffff700 1 -- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4de0060b00 msgr2=0x7f4de0062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.595+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f4df807d940 con 0x7f4e00157fc0 2023-09-27T23:30:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.595+0000 7f4dfffff700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4de0060b00 0x7f4de0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:13.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.599+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4e00148e60 con 0x7f4e00157fc0 2023-09-27T23:30:13.656 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.650+0000 7f82809ff000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:30:13.712 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.706+0000 7f82809ff000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:30:13.749 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.747+0000 7f87c0c8c000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:30:13.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.763+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e00148e60 con 0x7f4e00157fc0 2023-09-27T23:30:13.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4df7fff700 1 -- 172.21.15.64:0/4151464315 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e00148e60 con 0x7f4e00157fc0 2023-09-27T23:30:13.768 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:30:13.770 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:13.766+0000 7f82809ff000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4de0060b00 msgr2=0x7f4de0062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4de0060b00 0x7f4de0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 msgr2=0x7f4e00069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 0x7f4e00069230 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4df800cc60 tx=0x7f4df8001ae0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 shutdown_connections 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4de0060b00 0x7f4de0062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e00157fc0 0x7f4e00069230 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e001547a0 0x7f4e00167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 --2- 172.21.15.64:0/4151464315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e0006d050 0x7f4e00148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 >> 172.21.15.64:0/4151464315 conn(0x7f4e000ccb30 msgr2=0x7f4e000c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 shutdown_connections 2023-09-27T23:30:13.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.767+0000 7f4e0516e700 1 -- 172.21.15.64:0/4151464315 wait complete. 2023-09-27T23:30:13.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:30:13.827 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.823+0000 7f87c0c8c000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:30:13.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 Processor -- start 2023-09-27T23:30:13.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- start start 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ff8158b00 0x7f3ff8158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ff806bb50 0x7f3ff815d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff806b6c0 con 0x7f3ff806a4f0 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff815d880 con 0x7f3ff806bb50 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff806aab0 con 0x7f3ff8158b00 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ff806bb50 0x7f3ff815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ff806bb50 0x7f3ff815d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36350/0 (socket says 172.21.15.64:36350) 2023-09-27T23:30:13.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/819779428 learned_addr learned my addr 172.21.15.64:0/819779428 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:13.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffe4f6700 1 --2- 172.21.15.64:0/819779428 >> v2:172.21.15.123:3300/0 conn(0x7f3ff8158b00 0x7f3ff8158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/819779428 >> v2:172.21.15.123:3300/0 conn(0x7f3ff8158b00 msgr2=0x7f3ff8158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 --2- 172.21.15.64:0/819779428 >> v2:172.21.15.123:3300/0 conn(0x7f3ff8158b00 0x7f3ff8158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/819779428 >> v1:172.21.15.123:6789/0 conn(0x7f3ff806a4f0 legacy=0x7f3ff80cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/819779428 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff806ad90 con 0x7f3ff806bb50 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffdcf5700 1 --2- 172.21.15.64:0/819779428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff815d340 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3ff806a2e0 tx=0x7f3fe8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1b43235d3df8dd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffd4f4700 1 -- 172.21.15.64:0/819779428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8009e70 con 0x7f3ff806bb50 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffd4f4700 1 -- 172.21.15.64:0/819779428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe800b860 con 0x7f3ff806bb50 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3ffd4f4700 1 -- 172.21.15.64:0/819779428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800bc00 con 0x7f3ff806bb50 2023-09-27T23:30:13.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- 172.21.15.64:0/819779428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 msgr2=0x7f3ff815d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/819779428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff815d340 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3ff806a2e0 tx=0x7f3fe8009250 comp rx=0 tx=0).stop 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- 172.21.15.64:0/819779428 shutdown_connections 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/819779428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff815d340 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/819779428 >> v2:172.21.15.123:3300/0 conn(0x7f3ff8158b00 0x7f3ff8158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- 172.21.15.64:0/819779428 >> 172.21.15.64:0/819779428 conn(0x7f3ff8067bb0 msgr2=0x7f3ff80cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- 172.21.15.64:0/819779428 shutdown_connections 2023-09-27T23:30:13.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.895+0000 7f3fff4f8700 1 -- 172.21.15.64:0/819779428 wait complete. 2023-09-27T23:30:13.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 Processor -- start 2023-09-27T23:30:13.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 -- start start 2023-09-27T23:30:13.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806a4f0 0x7f3ff815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ff8158b00 0x7f3ff815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80b5400 con 0x7f3ff806bb50 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80b47f0 con 0x7f3ff806a4f0 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80b4ad0 con 0x7f3ff8158b00 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff80b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36358/0 (socket says 172.21.15.64:36358) 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/3245151666 learned_addr learned my addr 172.21.15.64:0/3245151666 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffe4f6700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806a4f0 0x7f3ff815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffecf7700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ff8158b00 0x7f3ff815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ff8158b00 msgr2=0x7f3ff815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ff8158b00 0x7f3ff815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806a4f0 msgr2=0x7f3ff815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806a4f0 0x7f3ff815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 -- 172.21.15.64:0/3245151666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80094f0 con 0x7f3ff806bb50 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3ffdcf5700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff80b3d30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8069150 tx=0x7f3fe8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800bbe0 con 0x7f3ff806bb50 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8000c90 con 0x7f3ff806bb50 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8011670 con 0x7f3ff806bb50 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff80b4db0 con 0x7f3ff806bb50 2023-09-27T23:30:13.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.899+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff8157ed0 con 0x7f3ff806bb50 2023-09-27T23:30:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff81691b0 con 0x7f3ff806bb50 2023-09-27T23:30:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f3fe8011a60 con 0x7f3ff806bb50 2023-09-27T23:30:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f3fee7fc700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fdc060b00 0x7f3fdc062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f3ffe4f6700 1 -- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fdc060b00 msgr2=0x7f3fdc062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f3fe807ece0 con 0x7f3ff806bb50 2023-09-27T23:30:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f3ffe4f6700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fdc060b00 0x7f3fdc062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:13.907 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:13.903+0000 7f87c0c8c000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:30:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:13.911+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3fe804e9a0 con 0x7f3ff806bb50 2023-09-27T23:30:14.066 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.062+0000 7f82809ff000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:30:14.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.067+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ff804be40 con 0x7f3ff806bb50 2023-09-27T23:30:14.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fee7fc700 1 -- 172.21.15.64:0/3245151666 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3fe80150a0 con 0x7f3ff806bb50 2023-09-27T23:30:14.072 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:30:14.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fdc060b00 msgr2=0x7f3fdc062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fdc060b00 0x7f3fdc062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 msgr2=0x7f3ff80b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff80b3d30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8069150 tx=0x7f3fe8017000 comp rx=0 tx=0).stop 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 shutdown_connections 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fdc060b00 0x7f3fdc062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ff8158b00 0x7f3ff815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff806bb50 0x7f3ff80b3d30 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 --2- 172.21.15.64:0/3245151666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806a4f0 0x7f3ff815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 >> 172.21.15.64:0/3245151666 conn(0x7f3ff8067bb0 msgr2=0x7f3ff80afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 shutdown_connections 2023-09-27T23:30:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.071+0000 7f3fff4f8700 1 -- 172.21.15.64:0/3245151666 wait complete. 2023-09-27T23:30:14.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:30:14.177 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.174+0000 7f82809ff000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:30:14.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.191+0000 7fef6bfff700 1 Processor -- start 2023-09-27T23:30:14.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.191+0000 7fef6bfff700 1 -- start start 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef6c0c7170 0x7fef6c0ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef6c0acc60 0x7fef6c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c0c7610 con 0x7fef6c150200 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c0af530 con 0x7fef6c0c7170 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c0ad080 con 0x7fef6c0acc60 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef6c0c7170 0x7fef6c0ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef6c0c7170 0x7fef6c0ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36366/0 (socket says 172.21.15.64:36366) 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 -- 172.21.15.64:0/3512975417 learned_addr learned my addr 172.21.15.64:0/3512975417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6b7fe700 1 --2- 172.21.15.64:0/3512975417 >> v2:172.21.15.123:3300/0 conn(0x7fef6c0acc60 0x7fef6c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 -- 172.21.15.64:0/3512975417 >> v2:172.21.15.123:3300/0 conn(0x7fef6c0acc60 msgr2=0x7fef6c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 --2- 172.21.15.64:0/3512975417 >> v2:172.21.15.123:3300/0 conn(0x7fef6c0acc60 0x7fef6c0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 -- 172.21.15.64:0/3512975417 >> v1:172.21.15.123:6789/0 conn(0x7fef6c150200 legacy=0x7fef6c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 -- 172.21.15.64:0/3512975417 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0af250 con 0x7fef6c0c7170 2023-09-27T23:30:14.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6a7fc700 1 --2- 172.21.15.64:0/3512975417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c0c7170 0x7fef6c0ac720 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fef6c06f280 tx=0x7fef6000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cc322498b2f52f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:14.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef69ffb700 1 -- 172.21.15.64:0/3512975417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000dc50 con 0x7fef6c0c7170 2023-09-27T23:30:14.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef69ffb700 1 -- 172.21.15.64:0/3512975417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000ddb0 con 0x7fef6c0c7170 2023-09-27T23:30:14.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef69ffb700 1 -- 172.21.15.64:0/3512975417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef600196d0 con 0x7fef6c0c7170 2023-09-27T23:30:14.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- 172.21.15.64:0/3512975417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c0c7170 msgr2=0x7fef6c0ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 --2- 172.21.15.64:0/3512975417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c0c7170 0x7fef6c0ac720 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fef6c06f280 tx=0x7fef6000b4f0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- 172.21.15.64:0/3512975417 shutdown_connections 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 --2- 172.21.15.64:0/3512975417 >> v2:172.21.15.123:3300/0 conn(0x7fef6c0acc60 0x7fef6c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 --2- 172.21.15.64:0/3512975417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c0c7170 0x7fef6c0ac720 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- 172.21.15.64:0/3512975417 >> 172.21.15.64:0/3512975417 conn(0x7fef6c0ccb30 msgr2=0x7fef6c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- 172.21.15.64:0/3512975417 shutdown_connections 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.195+0000 7fef6bfff700 1 -- 172.21.15.64:0/3512975417 wait complete. 2023-09-27T23:30:14.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 Processor -- start 2023-09-27T23:30:14.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 -- start start 2023-09-27T23:30:14.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef6c0acc60 0x7fef6c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6c0c7170 0x7fef6c1496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 0x7fef6c13c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0af8d0 con 0x7fef6c150200 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0b0450 con 0x7fef6c0c7170 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0b1e00 con 0x7fef6c0acc60 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 0x7fef6c13c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 0x7fef6c13c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36376/0 (socket says 172.21.15.64:36376) 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 -- 172.21.15.64:0/1602449266 learned_addr learned my addr 172.21.15.64:0/1602449266 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6a7fc700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6c0c7170 0x7fef6c1496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6affd700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef6c0acc60 0x7fef6c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 -- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef6c0acc60 msgr2=0x7fef6c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef6c0acc60 0x7fef6c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 -- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6c0c7170 msgr2=0x7fef6c1496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6c0c7170 0x7fef6c1496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 -- 172.21.15.64:0/1602449266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6000b790 con 0x7fef6c150200 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6b7fe700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 0x7fef6c13c350 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fef64007f50 tx=0x7fef64009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400eb70 con 0x7fef6c150200 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6400ecf0 con 0x7fef6c150200 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64014670 con 0x7fef6c150200 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c0afb20 con 0x7fef6c150200 2023-09-27T23:30:14.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.199+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c0b00e0 con 0x7fef6c150200 2023-09-27T23:30:14.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.203+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0b05e0 con 0x7fef6c150200 2023-09-27T23:30:14.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.203+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fef640149e0 con 0x7fef6c150200 2023-09-27T23:30:14.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.203+0000 7fef52ffd700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef58060b00 0x7fef58062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.203+0000 7fef6affd700 1 -- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef58060b00 msgr2=0x7fef58062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:14.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.203+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fef6407da40 con 0x7fef6c150200 2023-09-27T23:30:14.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.203+0000 7fef6affd700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef58060b00 0x7fef58062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:14.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.211+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fef6407fe60 con 0x7fef6c150200 2023-09-27T23:30:14.239 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:14.235+0000 7f87c0c8c000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:30:14.358 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:14.355+0000 7f87c0c8c000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:30:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.359+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef6c04be40 con 0x7fef6c150200 2023-09-27T23:30:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.359+0000 7fef52ffd700 1 -- 172.21.15.64:0/1602449266 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef6404d7a0 con 0x7fef6c150200 2023-09-27T23:30:14.362 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef58060b00 msgr2=0x7fef58062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef58060b00 0x7fef58062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 msgr2=0x7fef6c13c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 0x7fef6c13c350 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fef64007f50 tx=0x7fef64009860 comp rx=0 tx=0).stop 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 shutdown_connections 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef58060b00 0x7fef58062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c150200 0x7fef6c13c350 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6c0c7170 0x7fef6c1496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 --2- 172.21.15.64:0/1602449266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef6c0acc60 0x7fef6c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 >> 172.21.15.64:0/1602449266 conn(0x7fef6c0ccb30 msgr2=0x7fef6c0c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 shutdown_connections 2023-09-27T23:30:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.363+0000 7fef6bfff700 1 -- 172.21.15.64:0/1602449266 wait complete. 2023-09-27T23:30:14.375 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:30:14.458 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.454+0000 7f82809ff000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:30:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 Processor -- start 2023-09-27T23:30:14.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 -- start start 2023-09-27T23:30:14.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0cc06ae20 0x7fa0cc06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0cc0b1440 0x7fa0cc0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc06b730 con 0x7fa0cc0ab6d0 2023-09-27T23:30:14.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc0ac390 con 0x7fa0cc06ae20 2023-09-27T23:30:14.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.483+0000 7fa0cb59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc0ac670 con 0x7fa0cc0b1440 2023-09-27T23:30:14.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0c9d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa0cc0ab6d0 0x7fa0cc0abb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56688/0 (socket says 172.21.15.64:56688) 2023-09-27T23:30:14.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0c9d9b700 1 -- 172.21.15.64:0/986463248 learned_addr learned my addr 172.21.15.64:0/986463248 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:14.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0ca59c700 1 --2- 172.21.15.64:0/986463248 >> v2:172.21.15.64:3300/0 conn(0x7fa0cc06ae20 0x7fa0cc06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cad9d700 1 --2- 172.21.15.64:0/986463248 >> v2:172.21.15.123:3300/0 conn(0x7fa0cc0b1440 0x7fa0cc0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0ca59c700 1 -- 172.21.15.64:0/986463248 >> v2:172.21.15.123:3300/0 conn(0x7fa0cc0b1440 msgr2=0x7fa0cc0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:14.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0ca59c700 1 --2- 172.21.15.64:0/986463248 >> v2:172.21.15.123:3300/0 conn(0x7fa0cc0b1440 0x7fa0cc0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0ca59c700 1 -- 172.21.15.64:0/986463248 >> v1:172.21.15.64:6789/0 conn(0x7fa0cc0ab6d0 legacy=0x7fa0cc0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0ca59c700 1 -- 172.21.15.64:0/986463248 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0ac950 con 0x7fa0cc06ae20 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0ca59c700 1 --2- 172.21.15.64:0/986463248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc06b1f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0aee90 tx=0x7fa0bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fb52ed4c16ef55 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0c959a700 1 -- 172.21.15.64:0/986463248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011630 con 0x7fa0cc06ae20 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0c959a700 1 -- 172.21.15.64:0/986463248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc011790 con 0x7fa0cc06ae20 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0c959a700 1 -- 172.21.15.64:0/986463248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011b30 con 0x7fa0cc06ae20 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 -- 172.21.15.64:0/986463248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 msgr2=0x7fa0cc06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/986463248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc06b1f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0aee90 tx=0x7fa0bc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 -- 172.21.15.64:0/986463248 shutdown_connections 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/986463248 >> v2:172.21.15.123:3300/0 conn(0x7fa0cc0b1440 0x7fa0cc0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/986463248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc06b1f0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 -- 172.21.15.64:0/986463248 >> 172.21.15.64:0/986463248 conn(0x7fa0cc0c8cd0 msgr2=0x7fa0cc0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 -- 172.21.15.64:0/986463248 shutdown_connections 2023-09-27T23:30:14.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.487+0000 7fa0cb59e700 1 -- 172.21.15.64:0/986463248 wait complete. 2023-09-27T23:30:14.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 Processor -- start 2023-09-27T23:30:14.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 -- start start 2023-09-27T23:30:14.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0cc0ab6d0 0x7fa0cc167020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0cc0b1440 0x7fa0cc0a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc155ff0 con 0x7fa0cc06ae20 2023-09-27T23:30:14.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc15a190 con 0x7fa0cc0ab6d0 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc159580 con 0x7fa0cc0b1440 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36404/0 (socket says 172.21.15.64:36404) 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 -- 172.21.15.64:0/3303154269 learned_addr learned my addr 172.21.15.64:0/3303154269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0c9d9b700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0cc0ab6d0 0x7fa0cc167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cad9d700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0cc0b1440 0x7fa0cc0a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 -- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0cc0b1440 msgr2=0x7fa0cc0a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0cc0b1440 0x7fa0cc0a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 -- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0cc0ab6d0 msgr2=0x7fa0cc167020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0cc0ab6d0 0x7fa0cc167020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 -- 172.21.15.64:0/3303154269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009420 con 0x7fa0cc06ae20 2023-09-27T23:30:14.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0c9d9b700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0cc0ab6d0 0x7fa0cc167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cad9d700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0cc0b1440 0x7fa0cc0a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ca59c700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc155ab0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0680c0 tx=0x7fa0bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011b10 con 0x7fa0cc06ae20 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0253f0 con 0x7fa0cc06ae20 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc159860 con 0x7fa0cc06ae20 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc025790 con 0x7fa0cc06ae20 2023-09-27T23:30:14.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.491+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc159e20 con 0x7fa0cc06ae20 2023-09-27T23:30:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.495+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fa0bc025b80 con 0x7fa0cc06ae20 2023-09-27T23:30:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.495+0000 7fa0ba7fc700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0ac060b30 0x7fa0ac062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.495+0000 7fa0c9d9b700 1 -- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0ac060b30 msgr2=0x7fa0ac062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.495+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa0bc07ec30 con 0x7fa0cc06ae20 2023-09-27T23:30:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.495+0000 7fa0c9d9b700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0ac060b30 0x7fa0ac062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:14.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.495+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc04be40 con 0x7fa0cc06ae20 2023-09-27T23:30:14.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.499+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa0bc015020 con 0x7fa0cc06ae20 2023-09-27T23:30:14.621 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.618+0000 7f82809ff000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:14.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0cc0acc30 con 0x7fa0cc06ae20 2023-09-27T23:30:14.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0ba7fc700 1 -- 172.21.15.64:0/3303154269 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0bc07e030 con 0x7fa0cc06ae20 2023-09-27T23:30:14.665 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0ac060b30 msgr2=0x7fa0ac062fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0ac060b30 0x7fa0ac062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 msgr2=0x7fa0cc155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc155ab0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0680c0 tx=0x7fa0bc017000 comp rx=0 tx=0).stop 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 shutdown_connections 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0ac060b30 0x7fa0ac062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0cc0b1440 0x7fa0cc0a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0cc0ab6d0 0x7fa0cc167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 --2- 172.21.15.64:0/3303154269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc06ae20 0x7fa0cc155ab0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 >> 172.21.15.64:0/3303154269 conn(0x7fa0cc0c8cd0 msgr2=0x7fa0cc0abb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 shutdown_connections 2023-09-27T23:30:14.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.663+0000 7fa0cb59e700 1 -- 172.21.15.64:0/3303154269 wait complete. 2023-09-27T23:30:14.673 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:14.671+0000 7f87c0c8c000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:30:14.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:14.688 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.682+0000 7f82809ff000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:30:14.738 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.734+0000 7f82809ff000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:30:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 Processor -- start 2023-09-27T23:30:14.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 -- start start 2023-09-27T23:30:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c0b1410 0x7fb72c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb72c06a580 0x7fb72c06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c06c7c0 con 0x7fb72c15a860 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c06cd80 con 0x7fb72c06a580 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c06af10 con 0x7fb72c0b1410 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb72c06a580 0x7fb72c06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb72c06a580 0x7fb72c06a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41908/0 (socket says 172.21.15.64:41908) 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 -- 172.21.15.64:0/4245438713 learned_addr learned my addr 172.21.15.64:0/4245438713 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb72a59c700 1 --2- 172.21.15.64:0/4245438713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c0b1410 0x7fb72c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 -- 172.21.15.64:0/4245438713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c0b1410 msgr2=0x7fb72c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:14.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 --2- 172.21.15.64:0/4245438713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c0b1410 0x7fb72c0ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 -- 172.21.15.64:0/4245438713 >> v1:172.21.15.123:6789/0 conn(0x7fb72c15a860 legacy=0x7fb72c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.775+0000 7fb721d9b700 1 -- 172.21.15.64:0/4245438713 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c06b8f0 con 0x7fb72c06a580 2023-09-27T23:30:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb721d9b700 1 --2- 172.21.15.64:0/4245438713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c06a580 0x7fb72c06a9d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb7140099b0 tx=0x7fb714009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01ef2409eecc02c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb729d9b700 1 -- 172.21.15.64:0/4245438713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400b860 con 0x7fb72c06a580 2023-09-27T23:30:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb729d9b700 1 -- 172.21.15.64:0/4245438713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71400b9c0 con 0x7fb72c06a580 2023-09-27T23:30:14.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb729d9b700 1 -- 172.21.15.64:0/4245438713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400bd60 con 0x7fb72c06a580 2023-09-27T23:30:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- 172.21.15.64:0/4245438713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c06a580 msgr2=0x7fb72c06a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 --2- 172.21.15.64:0/4245438713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c06a580 0x7fb72c06a9d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb7140099b0 tx=0x7fb714009250 comp rx=0 tx=0).stop 2023-09-27T23:30:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- 172.21.15.64:0/4245438713 shutdown_connections 2023-09-27T23:30:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 --2- 172.21.15.64:0/4245438713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c06a580 0x7fb72c06a9d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 --2- 172.21.15.64:0/4245438713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c0b1410 0x7fb72c0ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- 172.21.15.64:0/4245438713 >> 172.21.15.64:0/4245438713 conn(0x7fb72c067bb0 msgr2=0x7fb72c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- 172.21.15.64:0/4245438713 shutdown_connections 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- 172.21.15.64:0/4245438713 wait complete. 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 Processor -- start 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- start start 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c06a580 0x7fb72c0c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 0x7fb72c168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c15a860 0x7fb72c0c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c4ea0 con 0x7fb72c0b1410 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c0a20 con 0x7fb72c15a860 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb72b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c4600 con 0x7fb72c06a580 2023-09-27T23:30:14.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb721d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 0x7fb72c168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb721d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 0x7fb72c168220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41924/0 (socket says 172.21.15.64:41924) 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.779+0000 7fb721d9b700 1 -- 172.21.15.64:0/999755106 learned_addr learned my addr 172.21.15.64:0/999755106 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72a59c700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c06a580 0x7fb72c0c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72ad9d700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c15a860 0x7fb72c0c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb721d9b700 1 -- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c06a580 msgr2=0x7fb72c0c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb721d9b700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c06a580 0x7fb72c0c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb721d9b700 1 -- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c15a860 msgr2=0x7fb72c0c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb721d9b700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c15a860 0x7fb72c0c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb721d9b700 1 -- 172.21.15.64:0/999755106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140094f0 con 0x7fb72c0b1410 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72ad9d700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c15a860 0x7fb72c0c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:14.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb721d9b700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 0x7fb72c168220 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb714006fd0 tx=0x7fb714017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714024440 con 0x7fb72c0b1410 2023-09-27T23:30:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7140245c0 con 0x7fb72c0b1410 2023-09-27T23:30:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c0c48e0 con 0x7fb72c0b1410 2023-09-27T23:30:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714024980 con 0x7fb72c0b1410 2023-09-27T23:30:14.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c0aaa30 con 0x7fb72c0b1410 2023-09-27T23:30:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c04be40 con 0x7fb72c0b1410 2023-09-27T23:30:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fb71401a420 con 0x7fb72c0b1410 2023-09-27T23:30:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb722ffd700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb70c060b00 0x7fb70c062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72a59c700 1 -- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb70c060b00 msgr2=0x7fb70c062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fb71407f1f0 con 0x7fb72c0b1410 2023-09-27T23:30:14.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.783+0000 7fb72a59c700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb70c060b00 0x7fb70c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:14.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.791+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb71404ef50 con 0x7fb72c0b1410 2023-09-27T23:30:14.794 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.789+0000 7f82809ff000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:30:14.847 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:14.843+0000 7f87c0c8c000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:14.912 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:14.907+0000 7f87c0c8c000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:30:14.965 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:14.961+0000 7f82809ff000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:14.975 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:14.971+0000 7f87c0c8c000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:30:14.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb72c0c0b60 con 0x7fb72c0b1410 2023-09-27T23:30:14.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb722ffd700 1 -- 172.21.15.64:0/999755106 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb714052570 con 0x7fb72c0b1410 2023-09-27T23:30:14.985 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:14.985 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:54.107357+0000","modified":"2023-09-27T23:29:56.124127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1623763192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1623763192},{"type":"v1","addr":"172.21.15.64:6835","nonce":1623763192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-09-27T23:30:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb70c060b00 msgr2=0x7fb70c062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb70c060b00 0x7fb70c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 msgr2=0x7fb72c168220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 0x7fb72c168220 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb714006fd0 tx=0x7fb714017000 comp rx=0 tx=0).stop 2023-09-27T23:30:14.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 shutdown_connections 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb70c060b00 0x7fb70c062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c15a860 0x7fb72c0c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb72c0b1410 0x7fb72c168220 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 --2- 172.21.15.64:0/999755106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb72c06a580 0x7fb72c0c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 >> 172.21.15.64:0/999755106 conn(0x7fb72c067bb0 msgr2=0x7fb72c155750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 shutdown_connections 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:14.983+0000 7fb72b59e700 1 -- 172.21.15.64:0/999755106 wait complete. 2023-09-27T23:30:14.989 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2023-09-27T23:30:14.999 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:15.018 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.013+0000 7f82809ff000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:30:15.031 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:15.027+0000 7f87c0c8c000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:30:15.082 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.081+0000 7f82809ff000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:30:15.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 Processor -- start 2023-09-27T23:30:15.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 -- start start 2023-09-27T23:30:15.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5614158df0 0x7f561406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f561406d590 0x7f56141591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561406d9e0 con 0x7f5614159700 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5614159f90 con 0x7f5614158df0 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561b4f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561415a270 con 0x7f561406d590 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f561a4f3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5614158df0 0x7f561406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.103+0000 7f5619cf2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f561406d590 0x7f56141591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561acf4700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5614159700 0x7f561415ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:35644/0 (socket says 172.21.15.64:35644) 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561acf4700 1 -- 172.21.15.64:0/1712791149 learned_addr learned my addr 172.21.15.64:0/1712791149 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561a4f3700 1 -- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f561406d590 msgr2=0x7f56141591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:15.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561a4f3700 1 --2- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f561406d590 0x7f56141591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f56194f1700 1 -- 172.21.15.64:0/1712791149 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879006615 0 0) 0x7f561406d9e0 con 0x7f5614159700 2023-09-27T23:30:15.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561a4f3700 1 -- 172.21.15.64:0/1712791149 >> v1:172.21.15.123:6789/0 conn(0x7f5614159700 legacy=0x7f561415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561a4f3700 1 -- 172.21.15.64:0/1712791149 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140c7930 con 0x7f5614158df0 2023-09-27T23:30:15.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f5619cf2700 1 --2- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f561406d590 0x7f56141591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:15.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561a4f3700 1 --2- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5614158df0 0x7f561406d050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56100098e0 tx=0x7f56100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=939202555f68d021 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:15.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f56194f1700 1 -- 172.21.15.64:0/1712791149 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000b640 con 0x7f5614158df0 2023-09-27T23:30:15.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f56194f1700 1 -- 172.21.15.64:0/1712791149 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561000b7a0 con 0x7f5614158df0 2023-09-27T23:30:15.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f56194f1700 1 -- 172.21.15.64:0/1712791149 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000bb40 con 0x7f5614158df0 2023-09-27T23:30:15.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 -- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5614158df0 msgr2=0x7f561406d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 --2- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5614158df0 0x7f561406d050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56100098e0 tx=0x7f56100091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 -- 172.21.15.64:0/1712791149 shutdown_connections 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 --2- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f561406d590 0x7f56141591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 --2- 172.21.15.64:0/1712791149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5614158df0 0x7f561406d050 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 -- 172.21.15.64:0/1712791149 >> 172.21.15.64:0/1712791149 conn(0x7f561406c080 msgr2=0x7f56140ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 -- 172.21.15.64:0/1712791149 shutdown_connections 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.107+0000 7f561b4f5700 1 -- 172.21.15.64:0/1712791149 wait complete. 2023-09-27T23:30:15.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 Processor -- start 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 -- start start 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561406d590 0x7f561415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 0x7f5614157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5614159700 0x7f561415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56140c4b60 con 0x7f5614158df0 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561415c1a0 con 0x7f561406d590 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561415c460 con 0x7f5614159700 2023-09-27T23:30:15.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 0x7f5614157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 0x7f5614157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41926/0 (socket says 172.21.15.64:41926) 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 -- 172.21.15.64:0/3562388320 learned_addr learned my addr 172.21.15.64:0/3562388320 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561a4f3700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561406d590 0x7f561415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561acf4700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5614159700 0x7f561415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 -- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5614159700 msgr2=0x7f561415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5614159700 0x7f561415bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 -- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561406d590 msgr2=0x7f561415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561406d590 0x7f561415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 -- 172.21.15.64:0/3562388320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604009420 con 0x7f5614158df0 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f5619cf2700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 0x7f5614157930 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f560400ec80 tx=0x7f56040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400c850 con 0x7f5614158df0 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5610009420 con 0x7f5614158df0 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400c9b0 con 0x7f5614158df0 2023-09-27T23:30:15.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400cd30 con 0x7f5614158df0 2023-09-27T23:30:15.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.111+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561415ca20 con 0x7f5614158df0 2023-09-27T23:30:15.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.115+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561404be40 con 0x7f5614158df0 2023-09-27T23:30:15.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.115+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f5604013680 con 0x7f5614158df0 2023-09-27T23:30:15.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.115+0000 7f560a7fc700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f55f8060b00 0x7f55f8062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.115+0000 7f561a4f3700 1 -- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f55f8060b00 msgr2=0x7f55f8062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:15.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.115+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f560407da70 con 0x7f5614158df0 2023-09-27T23:30:15.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.115+0000 7f561a4f3700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f55f8060b00 0x7f55f8062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:15.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.123+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f560404d7d0 con 0x7f5614158df0 2023-09-27T23:30:15.227 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:15.223+0000 7f87c0c8c000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:15.229 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.229+0000 7f82809ff000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:30:15.287 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:15.283+0000 7f87c0c8c000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:30:15.289 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.289+0000 7f82809ff000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:30:15.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.287+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f561406b190 con 0x7f5614158df0 2023-09-27T23:30:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.287+0000 7f560a7fc700 1 -- 172.21.15.64:0/3562388320 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17575 (secure 0 0 0) 0x7f5604050df0 con 0x7f5614158df0 2023-09-27T23:30:15.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:15.292 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":102,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:10.499965+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:52.694282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:53.074953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/1017331161":"2023-09-28T23:30:10.499869+0000","172.21.15.123:6833/33228":"2023-09-28T23:30:10.499869+0000","172.21.15.123:6832/33228":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/4275324614":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/2296684733":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/1309790114":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/1755447900":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/739794733":"2023-09-28T23:30:10.499869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f55f8060b00 msgr2=0x7f55f8062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f55f8060b00 0x7f55f8062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 msgr2=0x7f5614157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 0x7f5614157930 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f560400ec80 tx=0x7f56040091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 shutdown_connections 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f55f8060b00 0x7f55f8062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5614159700 0x7f561415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5614158df0 0x7f5614157930 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 --2- 172.21.15.64:0/3562388320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561406d590 0x7f561415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 >> 172.21.15.64:0/3562388320 conn(0x7f561406c080 msgr2=0x7f56140670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 shutdown_connections 2023-09-27T23:30:15.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.291+0000 7f561b4f5700 1 -- 172.21.15.64:0/3562388320 wait complete. 2023-09-27T23:30:15.307 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:30:15.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:15.356 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:15.351+0000 7f87c0c8c000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:30:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b83f5700 1 Processor -- start 2023-09-27T23:30:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b83f5700 1 -- start start 2023-09-27T23:30:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b83f5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84b01595d0 0x7f84b00c8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b83f5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b015a3e0 con 0x7f84b0159200 2023-09-27T23:30:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b83f5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b00c8a90 con 0x7f84b006b8b0 2023-09-27T23:30:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b83f5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b0159ab0 con 0x7f84b01595d0 2023-09-27T23:30:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b6191700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f84b0159200 0x7f84b006b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40658/0 (socket says 172.21.15.64:40658) 2023-09-27T23:30:15.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b6191700 1 -- 172.21.15.64:0/4021516329 learned_addr learned my addr 172.21.15.64:0/4021516329 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b6992700 1 --2- 172.21.15.64:0/4021516329 >> v2:172.21.15.123:3300/0 conn(0x7f84b01595d0 0x7f84b00c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007123998 0 0) 0x7f84b015a3e0 con 0x7f84b0159200 2023-09-27T23:30:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f849c003570 con 0x7f84b0159200 2023-09-27T23:30:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329056187 0 0) 0x7f84b00c8a90 con 0x7f84b006b8b0 2023-09-27T23:30:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b015a3e0 con 0x7f84b006b8b0 2023-09-27T23:30:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520316454 0 0) 0x7f849c003570 con 0x7f84b0159200 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 >> v2:172.21.15.123:3300/0 conn(0x7f84b01595d0 msgr2=0x7f84b00c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 --2- 172.21.15.64:0/4021516329 >> v2:172.21.15.123:3300/0 conn(0x7f84b01595d0 0x7f84b00c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 >> v1:172.21.15.123:6789/0 conn(0x7f84b006b8b0 legacy=0x7f84b00afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0159d90 con 0x7f84b0159200 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f84ac001470 con 0x7f84b0159200 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 >> v1:172.21.15.64:6789/0 conn(0x7f84b0159200 legacy=0x7f84b006b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c003c40 0x7f849c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f849c004810 0x7f849c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c005210 0x7f849c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0159ab0 con 0x7f849c004810 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c00c1b0 con 0x7f849c005210 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c00c370 con 0x7f849c003c40 2023-09-27T23:30:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f849c004810 0x7f849c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b6191700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c003c40 0x7f849c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b6992700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c005210 0x7f849c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 -- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c003c40 msgr2=0x7f849c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f849c003c40 0x7f849c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 -- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c005210 msgr2=0x7f849c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c005210 0x7f849c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 -- 172.21.15.64:0/4021516329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c003570 con 0x7f849c004810 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b6992700 1 -- 172.21.15.64:0/4021516329 reap_dead start 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.455+0000 7f84b5990700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f849c004810 0x7f849c004c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f84b006bcb0 tx=0x7f84a0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8833105d8024eac7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:15.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000c590 con 0x7f849c004810 2023-09-27T23:30:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a000c6f0 con 0x7f849c004810 2023-09-27T23:30:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b518f700 1 -- 172.21.15.64:0/4021516329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000c880 con 0x7f849c004810 2023-09-27T23:30:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- 172.21.15.64:0/4021516329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f849c004810 msgr2=0x7f849c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f849c004810 0x7f849c004c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f84b006bcb0 tx=0x7f84a0007d30 comp rx=0 tx=0).stop 2023-09-27T23:30:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- 172.21.15.64:0/4021516329 shutdown_connections 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 --2- 172.21.15.64:0/4021516329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f849c004810 0x7f849c004c80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- 172.21.15.64:0/4021516329 >> 172.21.15.64:0/4021516329 conn(0x7f84b0067410 msgr2=0x7f84b00677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- 172.21.15.64:0/4021516329 shutdown_connections 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- 172.21.15.64:0/4021516329 wait complete. 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 Processor -- start 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- start start 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c004810 0x7f84b00ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b00ae500 0x7f84b0169620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 0x7f84b016c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b00ae070 con 0x7f84b01584a0 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0169de0 con 0x7f849c004810 2023-09-27T23:30:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b83f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0169b60 con 0x7f84b00ae500 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 0x7f84b016c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 0x7f84b016c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41950/0 (socket says 172.21.15.64:41950) 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 -- 172.21.15.64:0/2619535527 learned_addr learned my addr 172.21.15.64:0/2619535527 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b5990700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b00ae500 0x7f84b0169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6191700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c004810 0x7f84b00ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 -- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b00ae500 msgr2=0x7f84b0169620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b00ae500 0x7f84b0169620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 -- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c004810 msgr2=0x7f84b00ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c004810 0x7f84b00ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 -- 172.21.15.64:0/2619535527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b00ad740 con 0x7f84b01584a0 2023-09-27T23:30:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.459+0000 7f84b6992700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 0x7f84b016c1b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f84a80094d0 tx=0x7f84a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800e040 con 0x7f84b01584a0 2023-09-27T23:30:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a800f040 con 0x7f84b01584a0 2023-09-27T23:30:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b00ada20 con 0x7f84b01584a0 2023-09-27T23:30:15.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800fda0 con 0x7f84b01584a0 2023-09-27T23:30:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b0158920 con 0x7f84b01584a0 2023-09-27T23:30:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f84a8015640 con 0x7f84b01584a0 2023-09-27T23:30:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b0158b70 con 0x7f84b01584a0 2023-09-27T23:30:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84a67fc700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 0x7f84940630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84b6191700 1 -- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 msgr2=0x7f84940630b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f84a807db80 con 0x7f84b01584a0 2023-09-27T23:30:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.467+0000 7f84b6191700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 0x7f84940630b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.475+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f84a804d850 con 0x7f84b01584a0 2023-09-27T23:30:15.525 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:15.523+0000 7f87c0c8c000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:30:15.589 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:15.587+0000 7f87c0c8c000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:30:15.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.667+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84b0071120 con 0x7f84b01584a0 2023-09-27T23:30:15.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.667+0000 7f84a67fc700 1 -- 172.21.15.64:0/2619535527 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f84a8050e70 con 0x7f84b01584a0 2023-09-27T23:30:15.671 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:15.672 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:54.107357+0000","modified":"2023-09-27T23:29:56.124127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1623763192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1623763192},{"type":"v1","addr":"172.21.15.64:6835","nonce":1623763192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-09-27T23:30:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b6191700 1 -- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 msgr2=0x7f84940630b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b6191700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 0x7f84940630b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:30:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 msgr2=0x7f84940630b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 0x7f84940630b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 msgr2=0x7f84b016c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 0x7f84b016c1b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f84a80094d0 tx=0x7f84a800b040 comp rx=0 tx=0).stop 2023-09-27T23:30:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 shutdown_connections 2023-09-27T23:30:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8494060c00 0x7f84940630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84b01584a0 0x7f84b016c1b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b00ae500 0x7f84b0169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 --2- 172.21.15.64:0/2619535527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f849c004810 0x7f84b00ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 >> 172.21.15.64:0/2619535527 conn(0x7f84b0067410 msgr2=0x7f84b015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 shutdown_connections 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.671+0000 7f84b83f5700 1 -- 172.21.15.64:0/2619535527 wait complete. 2023-09-27T23:30:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2023-09-27T23:30:15.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 Processor -- start 2023-09-27T23:30:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- start start 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb374159ba0 0x7fb3740670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3740ca420 0x7fb374159f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3740cb8a0 con 0x7fb37415a4b0 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb374067620 con 0x7fb374159ba0 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3740cac90 con 0x7fb3740ca420 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb374159ba0 0x7fb3740670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb374159ba0 0x7fb3740670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41954/0 (socket says 172.21.15.64:41954) 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 -- 172.21.15.64:0/3856048352 learned_addr learned my addr 172.21.15.64:0/3856048352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb3737fe700 1 --2- 172.21.15.64:0/3856048352 >> v2:172.21.15.123:3300/0 conn(0x7fb3740ca420 0x7fb374159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 -- 172.21.15.64:0/3856048352 >> v2:172.21.15.123:3300/0 conn(0x7fb3740ca420 msgr2=0x7fb374159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 --2- 172.21.15.64:0/3856048352 >> v2:172.21.15.123:3300/0 conn(0x7fb3740ca420 0x7fb374159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 -- 172.21.15.64:0/3856048352 >> v1:172.21.15.123:6789/0 conn(0x7fb37415a4b0 legacy=0x7fb3741558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 -- 172.21.15.64:0/3856048352 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3740caf70 con 0x7fb374159ba0 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb3737fe700 1 --2- 172.21.15.64:0/3856048352 >> v2:172.21.15.123:3300/0 conn(0x7fb3740ca420 0x7fb374159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb373fff700 1 --2- 172.21.15.64:0/3856048352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb374159ba0 0x7fb3740670e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f20 tx=0x7fb36c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b859c468b9774ba4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb372ffd700 1 -- 172.21.15.64:0/3856048352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00e070 con 0x7fb374159ba0 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb372ffd700 1 -- 172.21.15.64:0/3856048352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c00ba00 con 0x7fb374159ba0 2023-09-27T23:30:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb372ffd700 1 -- 172.21.15.64:0/3856048352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0092e0 con 0x7fb374159ba0 2023-09-27T23:30:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- 172.21.15.64:0/3856048352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb374159ba0 msgr2=0x7fb3740670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/3856048352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb374159ba0 0x7fb3740670e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f20 tx=0x7fb36c009840 comp rx=0 tx=0).stop 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- 172.21.15.64:0/3856048352 shutdown_connections 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/3856048352 >> v2:172.21.15.123:3300/0 conn(0x7fb3740ca420 0x7fb374159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/3856048352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb374159ba0 0x7fb3740670e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- 172.21.15.64:0/3856048352 >> 172.21.15.64:0/3856048352 conn(0x7fb3740cd990 msgr2=0x7fb3740ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- 172.21.15.64:0/3856048352 shutdown_connections 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.787+0000 7fb37a8d5700 1 -- 172.21.15.64:0/3856048352 wait complete. 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 Processor -- start 2023-09-27T23:30:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 -- start start 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3740ca420 0x7fb374147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb374159ba0 0x7fb374142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 0x7fb3741462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb374145ae0 con 0x7fb37415a4b0 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb374148430 con 0x7fb3740ca420 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb374146e50 con 0x7fb374159ba0 2023-09-27T23:30:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 0x7fb3741462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 0x7fb3741462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41962/0 (socket says 172.21.15.64:41962) 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 -- 172.21.15.64:0/837874268 learned_addr learned my addr 172.21.15.64:0/837874268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb373fff700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3740ca420 0x7fb374147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb3737fe700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb374159ba0 0x7fb374142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 -- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb374159ba0 msgr2=0x7fb374142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb374159ba0 0x7fb374142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 -- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3740ca420 msgr2=0x7fb374147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3740ca420 0x7fb374147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 -- 172.21.15.64:0/837874268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c009ae0 con 0x7fb37415a4b0 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb378e72700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 0x7fb3741462e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb36400c5b0 tx=0x7fb364007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364011590 con 0x7fb37415a4b0 2023-09-27T23:30:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364011710 con 0x7fb37415a4b0 2023-09-27T23:30:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb374147130 con 0x7fb37415a4b0 2023-09-27T23:30:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.791+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364011ad0 con 0x7fb37415a4b0 2023-09-27T23:30:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3741476f0 con 0x7fb37415a4b0 2023-09-27T23:30:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb374071120 con 0x7fb37415a4b0 2023-09-27T23:30:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 37) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fb36401c460 con 0x7fb37415a4b0 2023-09-27T23:30:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb36bfff700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb354060b00 0x7fb354062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb373fff700 1 -- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb354060b00 msgr2=0x7fb354062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fb36407cf80 con 0x7fb37415a4b0 2023-09-27T23:30:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.795+0000 7fb373fff700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb354060b00 0x7fb354062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:15.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.803+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb3640acbb0 con 0x7fb37415a4b0 2023-09-27T23:30:15.879 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.877+0000 7f82809ff000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:30:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.915+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 38) v1 ==== 79970+0+0 (secure 0 0 0) 0x7fb36404d0b0 con 0x7fb37415a4b0 2023-09-27T23:30:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.915+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb354060b00 msgr2=0x7fb354062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:15.915+0000 7fb36bfff700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb354060b00 0x7fb354062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:15.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824ac3d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824ac3d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824ac3d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824ac3d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824ac3d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824dc43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824dc43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824dc43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.988 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824dc43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:15.988 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:15.985+0000 7f824dc43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:16.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.011+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb37414ba30 con 0x7fb37415a4b0 2023-09-27T23:30:16.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.011+0000 7fb36bfff700 1 -- 172.21.15.64:0/837874268 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb364068540 con 0x7fb37415a4b0 2023-09-27T23:30:16.014 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:16.015 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:29:54.107357+0000","modified":"2023-09-27T23:29:56.124127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1623763192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1623763192},{"type":"v1","addr":"172.21.15.64:6835","nonce":1623763192}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 msgr2=0x7fb3741462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 0x7fb3741462e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb36400c5b0 tx=0x7fb364007f60 comp rx=0 tx=0).stop 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 shutdown_connections 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb354060b00 0x7fb354062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb37415a4b0 0x7fb3741462e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb374159ba0 0x7fb374142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 --2- 172.21.15.64:0/837874268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3740ca420 0x7fb374147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 >> 172.21.15.64:0/837874268 conn(0x7fb3740cd990 msgr2=0x7fb3740ad810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:16.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 shutdown_connections 2023-09-27T23:30:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.015+0000 7fb37a8d5700 1 -- 172.21.15.64:0/837874268 wait complete. 2023-09-27T23:30:16.018 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2023-09-27T23:30:16.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:16.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 Processor -- start 2023-09-27T23:30:16.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 -- start start 2023-09-27T23:30:16.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff5006e1d0 0x7eff50159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5006b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff5014f4d0 0x7eff5014f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5006b750 con 0x7eff5006e1d0 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5015a260 con 0x7eff5014f4d0 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff57113700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff500afad0 con 0x7eff5006ad70 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff56111700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff5006e1d0 0x7eff50159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff56111700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff5006e1d0 0x7eff50159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41968/0 (socket says 172.21.15.64:41968) 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.127+0000 7eff56111700 1 -- 172.21.15.64:0/4252803895 learned_addr learned my addr 172.21.15.64:0/4252803895 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56912700 1 --2- 172.21.15.64:0/4252803895 >> v2:172.21.15.123:3300/0 conn(0x7eff5014f4d0 0x7eff5014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff55910700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56111700 1 -- 172.21.15.64:0/4252803895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 msgr2=0x7eff5006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56111700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56111700 1 -- 172.21.15.64:0/4252803895 >> v2:172.21.15.123:3300/0 conn(0x7eff5014f4d0 msgr2=0x7eff5014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56111700 1 --2- 172.21.15.64:0/4252803895 >> v2:172.21.15.123:3300/0 conn(0x7eff5014f4d0 0x7eff5014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56111700 1 -- 172.21.15.64:0/4252803895 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff500afdb0 con 0x7eff5006e1d0 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56912700 1 --2- 172.21.15.64:0/4252803895 >> v2:172.21.15.123:3300/0 conn(0x7eff5014f4d0 0x7eff5014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff55910700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff56111700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff50159d20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7eff5006ebc0 tx=0x7eff4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ab33460cba490b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff5510f700 1 -- 172.21.15.64:0/4252803895 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00b860 con 0x7eff5006e1d0 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff5510f700 1 -- 172.21.15.64:0/4252803895 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c00b9c0 con 0x7eff5006e1d0 2023-09-27T23:30:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff5510f700 1 -- 172.21.15.64:0/4252803895 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00bd60 con 0x7eff5006e1d0 2023-09-27T23:30:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 -- 172.21.15.64:0/4252803895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 msgr2=0x7eff50159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff50159d20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7eff5006ebc0 tx=0x7eff4c009250 comp rx=0 tx=0).stop 2023-09-27T23:30:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 -- 172.21.15.64:0/4252803895 shutdown_connections 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 --2- 172.21.15.64:0/4252803895 >> v2:172.21.15.123:3300/0 conn(0x7eff5014f4d0 0x7eff5014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 --2- 172.21.15.64:0/4252803895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff50159d20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 -- 172.21.15.64:0/4252803895 >> 172.21.15.64:0/4252803895 conn(0x7eff50067bb0 msgr2=0x7eff500add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 -- 172.21.15.64:0/4252803895 shutdown_connections 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 -- 172.21.15.64:0/4252803895 wait complete. 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 Processor -- start 2023-09-27T23:30:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.131+0000 7eff57113700 1 -- start start 2023-09-27T23:30:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff500ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5014f4d0 0x7eff500ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff500ae590 con 0x7eff5006e1d0 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5013a700 con 0x7eff5014f4d0 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5013a9e0 con 0x7eff5006ad70 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff500ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff500ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41980/0 (socket says 172.21.15.64:41980) 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 -- 172.21.15.64:0/2200161441 learned_addr learned my addr 172.21.15.64:0/2200161441 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff56111700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff56912700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5014f4d0 0x7eff500ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 -- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 msgr2=0x7eff5014d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5014d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 -- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5014f4d0 msgr2=0x7eff500ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5014f4d0 0x7eff500ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 -- 172.21.15.64:0/2200161441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0094f0 con 0x7eff5006e1d0 2023-09-27T23:30:16.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff56912700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5014f4d0 0x7eff500ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:16.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff56111700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5014d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff55910700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff500ac390 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c850 tx=0x7eff40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011600 con 0x7eff5006e1d0 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40011760 con 0x7eff5006e1d0 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5013acc0 con 0x7eff5006e1d0 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011b00 con 0x7eff5006e1d0 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5013b280 con 0x7eff5006e1d0 2023-09-27T23:30:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.135+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5004be40 con 0x7eff5006e1d0 2023-09-27T23:30:16.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.139+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 79970+0+0 (secure 0 0 0) 0x7eff400073b0 con 0x7eff5006e1d0 2023-09-27T23:30:16.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.139+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7023+0+0 (secure 0 0 0) 0x7eff4007ccd0 con 0x7eff5006e1d0 2023-09-27T23:30:16.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.143+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff4004cd80 con 0x7eff5006e1d0 2023-09-27T23:30:16.252 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:16.247+0000 7f87c0c8c000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:30:16.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.311+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff5006bb80 con 0x7eff5006e1d0 2023-09-27T23:30:16.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.311+0000 7eff467fc700 1 -- 172.21.15.64:0/2200161441 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+18014 (secure 0 0 0) 0x7eff40050570 con 0x7eff5006e1d0 2023-09-27T23:30:16.316 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:16.316 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":103,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:15.903474+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:29:52.694282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-09-27T23:29:53.074953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/1713303267":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/1435592409":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/82474964":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/2161199679":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/3071670819":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/1017331161":"2023-09-28T23:30:10.499869+0000","172.21.15.123:6833/33228":"2023-09-28T23:30:10.499869+0000","172.21.15.123:6832/33228":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/4275324614":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/2483483875":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/2296684733":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/4009411845":"2023-09-28T23:30:15.903329+0000","172.21.15.123:0/1309790114":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/1755447900":"2023-09-28T23:30:10.499869+0000","172.21.15.123:0/739794733":"2023-09-28T23:30:10.499869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:16.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 msgr2=0x7eff500ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff500ac390 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c850 tx=0x7eff40008040 comp rx=0 tx=0).stop 2023-09-27T23:30:16.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 shutdown_connections 2023-09-27T23:30:16.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff5014f4d0 0x7eff500ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff5006e1d0 0x7eff500ac390 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 --2- 172.21.15.64:0/2200161441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff5006ad70 0x7eff5014d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 >> 172.21.15.64:0/2200161441 conn(0x7eff50067bb0 msgr2=0x7eff5015b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:16.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 shutdown_connections 2023-09-27T23:30:16.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.315+0000 7eff57113700 1 -- 172.21.15.64:0/2200161441 wait complete. 2023-09-27T23:30:16.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:30:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 Processor -- start 2023-09-27T23:30:16.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 -- start start 2023-09-27T23:30:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 0x7f448c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f448c06a580 0x7f448c06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c06c7c0 con 0x7f448c15a860 2023-09-27T23:30:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c06cd80 con 0x7f448c06a580 2023-09-27T23:30:16.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491971700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06af10 con 0x7f448c0b1410 2023-09-27T23:30:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491170700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f448c15a860 0x7f448c15aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40672/0 (socket says 172.21.15.64:40672) 2023-09-27T23:30:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f4491170700 1 -- 172.21.15.64:0/2669892843 learned_addr learned my addr 172.21.15.64:0/2669892843 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448bfff700 1 --2- 172.21.15.64:0/2669892843 >> v2:172.21.15.123:3300/0 conn(0x7f448c06a580 0x7f448c06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f449096f700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 0x7f448c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780504026 0 0) 0x7f448c06c7c0 con 0x7f448c15a860 2023-09-27T23:30:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4478003570 con 0x7f448c15a860 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1828591887 0 0) 0x7f4478003570 con 0x7f448c15a860 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 msgr2=0x7f448c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 0x7f448c0ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 >> v2:172.21.15.123:3300/0 conn(0x7f448c06a580 msgr2=0x7f448c06a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 --2- 172.21.15.64:0/2669892843 >> v2:172.21.15.123:3300/0 conn(0x7f448c06a580 0x7f448c06a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c06b8f0 con 0x7f448c15a860 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f44840017f0 con 0x7f448c15a860 2023-09-27T23:30:16.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 >> v1:172.21.15.64:6789/0 conn(0x7f448c15a860 legacy=0x7f448c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4478003a60 0x7f4478003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4478004810 0x7f4478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.435+0000 7f448b7fe700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 0x7f44780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06cd80 con 0x7f4478003a60 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06af10 con 0x7f4478004810 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447800ba60 con 0x7f448c0b1410 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448bfff700 1 --2- 172.21.15.64:0/2669892843 >> v2:172.21.15.123:3300/0 conn(0x7f448c06a580 0x7f448c06a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4478003a60 0x7f4478003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491170700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 0x7f44780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448bfff700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4478004810 0x7f4478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 -- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 msgr2=0x7f44780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c0b1410 0x7f44780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 -- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4478004810 msgr2=0x7f4478004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4478004810 0x7f4478004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 -- 172.21.15.64:0/2669892843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478003570 con 0x7f4478003a60 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4478003a60 0x7f4478003e70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f447c009c30 tx=0x7f447c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dc3ccf30c87afa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00e710 con 0x7f4478003a60 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c00e870 con 0x7f4478003a60 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f448b7fe700 1 -- 172.21.15.64:0/2669892843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00ec30 con 0x7f4478003a60 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- 172.21.15.64:0/2669892843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4478003a60 msgr2=0x7f4478003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 --2- 172.21.15.64:0/2669892843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4478003a60 0x7f4478003e70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f447c009c30 tx=0x7f447c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491170700 1 -- 172.21.15.64:0/2669892843 reap_dead start 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- 172.21.15.64:0/2669892843 shutdown_connections 2023-09-27T23:30:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- 172.21.15.64:0/2669892843 >> 172.21.15.64:0/2669892843 conn(0x7f448c067bb0 msgr2=0x7f448c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- 172.21.15.64:0/2669892843 shutdown_connections 2023-09-27T23:30:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- 172.21.15.64:0/2669892843 wait complete. 2023-09-27T23:30:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 Processor -- start 2023-09-27T23:30:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- start start 2023-09-27T23:30:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 0x7f448c0c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c14cc80 0x7f448c0c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448c0c5fd0 0x7f448c0c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c78d0 con 0x7f448c14c8b0 2023-09-27T23:30:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c7d60 con 0x7f448c0c5fd0 2023-09-27T23:30:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f4491971700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c6cc0 con 0x7f448c14cc80 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.439+0000 7f449096f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 0x7f448c0c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 0x7f448c0c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42000/0 (socket says 172.21.15.64:42000) 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 -- 172.21.15.64:0/2850991412 learned_addr learned my addr 172.21.15.64:0/2850991412 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f448bfff700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c14cc80 0x7f448c0c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4491170700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448c0c5fd0 0x7f448c0c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 -- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c14cc80 msgr2=0x7f448c0c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c14cc80 0x7f448c0c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 -- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448c0c5fd0 msgr2=0x7f448c0c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448c0c5fd0 0x7f448c0c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 -- 172.21.15.64:0/2850991412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c0c6fa0 con 0x7f448c14c8b0 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f449096f700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 0x7f448c0c4c70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f44780046d0 tx=0x7f447c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c023070 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c00e9e0 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c0c7280 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00e710 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c0c6980 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 38) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f447c013680 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.443+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c0ad450 con 0x7f448c14c8b0 2023-09-27T23:30:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.447+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f447c07e390 con 0x7f448c14c8b0 2023-09-27T23:30:16.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.455+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f448c0ad450 con 0x7f448c14c8b0 2023-09-27T23:30:16.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.647+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f448c0ad450 con 0x7f448c14c8b0 2023-09-27T23:30:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.919+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 39) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f447c04ea20 con 0x7f448c14c8b0 2023-09-27T23:30:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.919+0000 7f4488ff9700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f446c066d70 0x7f446c069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:16.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.923+0000 7f448bfff700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f446c066d70 0x7f446c069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:16.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:16.927+0000 7f448bfff700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f446c066d70 0x7f446c069150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f448c0ab0c0 tx=0x7f44740091a0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:17.590 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:30:17.587+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:30:17.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.607+0000 7f4488ff9700 1 -- 172.21.15.64:0/2850991412 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f447c080380 con 0x7f448c14c8b0 2023-09-27T23:30:17.611 INFO:tasks.ceph.mds.a.smithi064.stderr: -1432> 2023-09-27T23:30:00.019+0000 7fb6fd07c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f446c066d70 msgr2=0x7f446c069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f446c066d70 0x7f446c069150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f448c0ab0c0 tx=0x7f44740091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 msgr2=0x7f448c0c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 0x7f448c0c4c70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f44780046d0 tx=0x7f447c01a000 comp rx=0 tx=0).stop 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 shutdown_connections 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f446c066d70 0x7f446c069150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448c0c5fd0 0x7f448c0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f448c14cc80 0x7f448c0c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 --2- 172.21.15.64:0/2850991412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f448c14c8b0 0x7f448c0c4c70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 >> 172.21.15.64:0/2850991412 conn(0x7f448c067bb0 msgr2=0x7f448c1563b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:17.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 shutdown_connections 2023-09-27T23:30:17.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.611+0000 7f4491971700 1 -- 172.21.15.64:0/2850991412 wait complete. 2023-09-27T23:30:17.614 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:30:17.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:30:17.639 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:30:17.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.739+0000 7efc31d5a700 1 Processor -- start 2023-09-27T23:30:17.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.739+0000 7efc31d5a700 1 -- start start 2023-09-27T23:30:17.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efc2c06ae20 0x7efc2c06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc2c0ab6d0 0x7efc2c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c06b730 con 0x7efc2c0b1440 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2c0ac390 con 0x7efc2c0ab6d0 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2c0ac670 con 0x7efc2c06ae20 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc2c0ab6d0 0x7efc2c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc2c0ab6d0 0x7efc2c0abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42024/0 (socket says 172.21.15.64:42024) 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 -- 172.21.15.64:0/4257080119 learned_addr learned my addr 172.21.15.64:0/4257080119 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:17.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc30d58700 1 --2- 172.21.15.64:0/4257080119 >> v2:172.21.15.123:3300/0 conn(0x7efc2c06ae20 0x7efc2c06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 -- 172.21.15.64:0/4257080119 >> v2:172.21.15.123:3300/0 conn(0x7efc2c06ae20 msgr2=0x7efc2c06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 --2- 172.21.15.64:0/4257080119 >> v2:172.21.15.123:3300/0 conn(0x7efc2c06ae20 0x7efc2c06b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 -- 172.21.15.64:0/4257080119 >> v1:172.21.15.123:6789/0 conn(0x7efc2c0b1440 legacy=0x7efc2c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 -- 172.21.15.64:0/4257080119 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c0ac950 con 0x7efc2c0ab6d0 2023-09-27T23:30:17.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2bfff700 1 --2- 172.21.15.64:0/4257080119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0ab6d0 0x7efc2c0abb50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0c9dc0 tx=0x7efc18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29692c20c9fc10ab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2b7fe700 1 -- 172.21.15.64:0/4257080119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18009e70 con 0x7efc2c0ab6d0 2023-09-27T23:30:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2b7fe700 1 -- 172.21.15.64:0/4257080119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800b860 con 0x7efc2c0ab6d0 2023-09-27T23:30:17.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc2b7fe700 1 -- 172.21.15.64:0/4257080119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800bc00 con 0x7efc2c0ab6d0 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 -- 172.21.15.64:0/4257080119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0ab6d0 msgr2=0x7efc2c0abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 --2- 172.21.15.64:0/4257080119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0ab6d0 0x7efc2c0abb50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0c9dc0 tx=0x7efc18009250 comp rx=0 tx=0).stop 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 -- 172.21.15.64:0/4257080119 shutdown_connections 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 --2- 172.21.15.64:0/4257080119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0ab6d0 0x7efc2c0abb50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 --2- 172.21.15.64:0/4257080119 >> v2:172.21.15.123:3300/0 conn(0x7efc2c06ae20 0x7efc2c06b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.743+0000 7efc31d5a700 1 -- 172.21.15.64:0/4257080119 >> 172.21.15.64:0/4257080119 conn(0x7efc2c0c8cd0 msgr2=0x7efc2c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 -- 172.21.15.64:0/4257080119 shutdown_connections 2023-09-27T23:30:17.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 -- 172.21.15.64:0/4257080119 wait complete. 2023-09-27T23:30:17.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 Processor -- start 2023-09-27T23:30:17.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 -- start start 2023-09-27T23:30:17.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc2c06ae20 0x7efc2c0a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:17.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc2c0ab6d0 0x7efc2c168710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:17.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.747+0000 7efc31d5a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 0x7efc2c0c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:17.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31559700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 0x7efc2c0c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31d5a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0aa760 con 0x7efc2c0b1440 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31559700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 0x7efc2c0c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42032/0 (socket says 172.21.15.64:42032) 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31559700 1 -- 172.21.15.64:0/3959094170 learned_addr learned my addr 172.21.15.64:0/3959094170 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0a9490 con 0x7efc2c06ae20 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0aa2d0 con 0x7efc2c0ab6d0 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc2c06ae20 0x7efc2c0a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:17.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc2bfff700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc2c0ab6d0 0x7efc2c168710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:17.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 -- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc2c0ab6d0 msgr2=0x7efc2c168710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:17.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc2c0ab6d0 0x7efc2c168710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 -- 172.21.15.64:0/3959094170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 msgr2=0x7efc2c0c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:17.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 0x7efc2c0c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:17.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180094f0 con 0x7efc2c06ae20 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31559700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 0x7efc2c0c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc30d58700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc2c06ae20 0x7efc2c0a8f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e930 tx=0x7efc200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20013680 con 0x7efc2c06ae20 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc200137e0 con 0x7efc2c06ae20 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20013ba0 con 0x7efc2c06ae20 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c0a96c0 con 0x7efc2c06ae20 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c0a9c80 con 0x7efc2c06ae20 2023-09-27T23:30:17.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.751+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c0c7a20 con 0x7efc2c06ae20 2023-09-27T23:30:17.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.755+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 39) v1 ==== 80097+0+0 (secure 0 0 0) 0x7efc200253d0 con 0x7efc2c06ae20 2023-09-27T23:30:17.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.755+0000 7efc28ff9700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efc100607a0 0x7efc10062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:17.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.755+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7109+0+0 (secure 0 0 0) 0x7efc2007d780 con 0x7efc2c06ae20 2023-09-27T23:30:17.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.755+0000 7efc2bfff700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efc100607a0 0x7efc10062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:17.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.755+0000 7efc2bfff700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efc100607a0 0x7efc10062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0c9dc0 tx=0x7efc18017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:17.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.763+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efc2004d8f0 con 0x7efc2c06ae20 2023-09-27T23:30:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:17.967+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efc2c04be40 con 0x7efc2c06ae20 2023-09-27T23:30:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.607+0000 7efc28ff9700 1 -- 172.21.15.64:0/3959094170 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc20050f10 con 0x7efc2c06ae20 2023-09-27T23:30:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efc100607a0 msgr2=0x7efc10062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efc100607a0 0x7efc10062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0c9dc0 tx=0x7efc18017000 comp rx=0 tx=0).stop 2023-09-27T23:30:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc2c06ae20 msgr2=0x7efc2c0a8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc2c06ae20 0x7efc2c0a8f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e930 tx=0x7efc200091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 shutdown_connections 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7efc100607a0 0x7efc10062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc2c0b1440 0x7efc2c0c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc2c0ab6d0 0x7efc2c168710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 --2- 172.21.15.64:0/3959094170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc2c06ae20 0x7efc2c0a8f50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 >> 172.21.15.64:0/3959094170 conn(0x7efc2c0c8cd0 msgr2=0x7efc2c0b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 shutdown_connections 2023-09-27T23:30:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.615+0000 7efc31d5a700 1 -- 172.21.15.64:0/3959094170 wait complete. 2023-09-27T23:30:18.633 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:30:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.795+0000 7f0dcfe68700 1 Processor -- start 2023-09-27T23:30:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.795+0000 7f0dcfe68700 1 -- start start 2023-09-27T23:30:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0dc8158e50 0x7f0dc80c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0dc8159220 0x7f0dc8068380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc806d0d0 con 0x7f0dc8069310 2023-09-27T23:30:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc806c7c0 con 0x7f0dc8159220 2023-09-27T23:30:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc80c7b40 con 0x7f0dc8158e50 2023-09-27T23:30:18.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0dc8159220 0x7f0dc8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0dc8159220 0x7f0dc8068380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42038/0 (socket says 172.21.15.64:42038) 2023-09-27T23:30:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 -- 172.21.15.64:0/299715730 learned_addr learned my addr 172.21.15.64:0/299715730 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcdc04700 1 --2- 172.21.15.64:0/299715730 >> v2:172.21.15.123:3300/0 conn(0x7f0dc8158e50 0x7f0dc80c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 -- 172.21.15.64:0/299715730 >> v2:172.21.15.123:3300/0 conn(0x7f0dc8158e50 msgr2=0x7f0dc80c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 --2- 172.21.15.64:0/299715730 >> v2:172.21.15.123:3300/0 conn(0x7f0dc8158e50 0x7f0dc80c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 -- 172.21.15.64:0/299715730 >> v1:172.21.15.123:6789/0 conn(0x7f0dc8069310 legacy=0x7f0dc815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 -- 172.21.15.64:0/299715730 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc806ca80 con 0x7f0dc8159220 2023-09-27T23:30:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcd403700 1 --2- 172.21.15.64:0/299715730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8159220 0x7f0dc8068380 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80c9e20 tx=0x7f0db8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2324622e46c292a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dccc02700 1 -- 172.21.15.64:0/299715730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800b860 con 0x7f0dc8159220 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dccc02700 1 -- 172.21.15.64:0/299715730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800b9c0 con 0x7f0dc8159220 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dccc02700 1 -- 172.21.15.64:0/299715730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800bd80 con 0x7f0dc8159220 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 -- 172.21.15.64:0/299715730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8159220 msgr2=0x7f0dc8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/299715730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8159220 0x7f0dc8068380 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80c9e20 tx=0x7f0db8009250 comp rx=0 tx=0).stop 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 -- 172.21.15.64:0/299715730 shutdown_connections 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/299715730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8159220 0x7f0dc8068380 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/299715730 >> v2:172.21.15.123:3300/0 conn(0x7f0dc8158e50 0x7f0dc80c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.799+0000 7f0dcfe68700 1 -- 172.21.15.64:0/299715730 >> 172.21.15.64:0/299715730 conn(0x7f0dc80c8db0 msgr2=0x7f0dc80c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 -- 172.21.15.64:0/299715730 shutdown_connections 2023-09-27T23:30:18.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 -- 172.21.15.64:0/299715730 wait complete. 2023-09-27T23:30:18.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 Processor -- start 2023-09-27T23:30:18.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 -- start start 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8069310 0x7f0dc816d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 0x7f0dc80af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8159220 0x7f0dc80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80b1800 con 0x7f0dc8158e50 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80a9a60 con 0x7f0dc8159220 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcfe68700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80b1a50 con 0x7f0dc8069310 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcd403700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 0x7f0dc80af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:18.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcd403700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 0x7f0dc80af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42044/0 (socket says 172.21.15.64:42044) 2023-09-27T23:30:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcd403700 1 -- 172.21.15.64:0/1955714221 learned_addr learned my addr 172.21.15.64:0/1955714221 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dcdc04700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8069310 0x7f0dc816d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:18.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.803+0000 7f0dce405700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8159220 0x7f0dc80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcdc04700 1 -- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8159220 msgr2=0x7f0dc80c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcdc04700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8159220 0x7f0dc80c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcdc04700 1 -- 172.21.15.64:0/1955714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 msgr2=0x7f0dc80af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcdc04700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 0x7f0dc80af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcdc04700 1 -- 172.21.15.64:0/1955714221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80094f0 con 0x7f0dc8069310 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcd403700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 0x7f0dc80af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcdc04700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8069310 0x7f0dc816d680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400d8c0 tx=0x7f0dc4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40125f0 con 0x7f0dc8069310 2023-09-27T23:30:18.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc80b1d30 con 0x7f0dc8069310 2023-09-27T23:30:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4012750 con 0x7f0dc8069310 2023-09-27T23:30:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4012af0 con 0x7f0dc8069310 2023-09-27T23:30:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc80c6080 con 0x7f0dc8069310 2023-09-27T23:30:18.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.807+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc80b1f70 con 0x7f0dc8069310 2023-09-27T23:30:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.811+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 39) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f0dc4013480 con 0x7f0dc8069310 2023-09-27T23:30:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.811+0000 7f0dbdffb700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0dac0607d0 0x7f0dac062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.811+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7109+0+0 (secure 0 0 0) 0x7f0dc407d2f0 con 0x7f0dc8069310 2023-09-27T23:30:18.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.811+0000 7f0dcd403700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0dac0607d0 0x7f0dac062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:18.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.811+0000 7f0dcd403700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0dac0607d0 0x7f0dac062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009340 tx=0x7f0db8017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:18.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.823+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0dc80b1f70 con 0x7f0dc8069310 2023-09-27T23:30:18.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.935+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0dc404d700 con 0x7f0dc8069310 2023-09-27T23:30:18.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:18.991+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0dc80c6590 con 0x7f0dc8069310 2023-09-27T23:30:19.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:30:19.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:30:19.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.943+0000 7f0dbdffb700 1 -- 172.21.15.64:0/1955714221 <== mon.2 v2:172.21.15.123:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0dc80c6590 con 0x7f0dc8069310 2023-09-27T23:30:19.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0dac0607d0 msgr2=0x7f0dac062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:19.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0dac0607d0 0x7f0dac062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009340 tx=0x7f0db8017000 comp rx=0 tx=0).stop 2023-09-27T23:30:19.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8069310 msgr2=0x7f0dc816d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:19.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8069310 0x7f0dc816d680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400d8c0 tx=0x7f0dc4008040 comp rx=0 tx=0).stop 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 shutdown_connections 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0dac0607d0 0x7f0dac062c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8159220 0x7f0dc80c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc8158e50 0x7f0dc80af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 --2- 172.21.15.64:0/1955714221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8069310 0x7f0dc816d680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 >> 172.21.15.64:0/1955714221 conn(0x7f0dc80c8db0 msgr2=0x7f0dc80acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 shutdown_connections 2023-09-27T23:30:19.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:19.947+0000 7f0dcfe68700 1 -- 172.21.15.64:0/1955714221 wait complete. 2023-09-27T23:30:19.952 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:30:19.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:30:20.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.079+0000 7f2177781700 1 Processor -- start 2023-09-27T23:30:20.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.079+0000 7f2177781700 1 -- start start 2023-09-27T23:30:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21700cb500 0x7f217015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f217015c760 0x7f21700b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21700b0ac0 con 0x7f21700cb130 2023-09-27T23:30:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217015cbb0 con 0x7f21700cb500 2023-09-27T23:30:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217006b210 con 0x7f217015c760 2023-09-27T23:30:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21700cb500 0x7f217015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21700cb500 0x7f217015c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42060/0 (socket says 172.21.15.64:42060) 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 -- 172.21.15.64:0/2796832753 learned_addr learned my addr 172.21.15.64:0/2796832753 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2175d1e700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f217015c760 0x7f21700b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 -- 172.21.15.64:0/2796832753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f217015c760 msgr2=0x7f21700b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f217015c760 0x7f21700b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 -- 172.21.15.64:0/2796832753 >> v1:172.21.15.123:6789/0 conn(0x7f21700cb130 legacy=0x7f217006a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 -- 172.21.15.64:0/2796832753 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700ca810 con 0x7f21700cb500 2023-09-27T23:30:20.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2175d1e700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f217015c760 0x7f21700b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:20.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2174d1c700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb500 0x7f217015c220 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f21640099b0 tx=0x7f2164009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cde02835378608d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2163fff700 1 -- 172.21.15.64:0/2796832753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400b860 con 0x7f21700cb500 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2163fff700 1 -- 172.21.15.64:0/2796832753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400b9c0 con 0x7f21700cb500 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2163fff700 1 -- 172.21.15.64:0/2796832753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400bd60 con 0x7f21700cb500 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- 172.21.15.64:0/2796832753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb500 msgr2=0x7f217015c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb500 0x7f217015c220 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f21640099b0 tx=0x7f2164009250 comp rx=0 tx=0).stop 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- 172.21.15.64:0/2796832753 shutdown_connections 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f217015c760 0x7f21700b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 --2- 172.21.15.64:0/2796832753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb500 0x7f217015c220 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- 172.21.15.64:0/2796832753 >> 172.21.15.64:0/2796832753 conn(0x7f2170067bb0 msgr2=0x7f21700cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:20.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- 172.21.15.64:0/2796832753 shutdown_connections 2023-09-27T23:30:20.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.083+0000 7f2177781700 1 -- 172.21.15.64:0/2796832753 wait complete. 2023-09-27T23:30:20.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 Processor -- start 2023-09-27T23:30:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 -- start start 2023-09-27T23:30:20.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 0x7f217014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:20.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cb500 0x7f2170169360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:20.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f217015c760 0x7f21700cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217016a330 con 0x7f21700cb130 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217016cea0 con 0x7f217015c760 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2177781700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217014b1f0 con 0x7f21700cb500 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f217551d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 0x7f217014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f217551d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 0x7f217014acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42072/0 (socket says 172.21.15.64:42072) 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f217551d700 1 -- 172.21.15.64:0/3078987958 learned_addr learned my addr 172.21.15.64:0/3078987958 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2175d1e700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f217015c760 0x7f21700cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.087+0000 7f2174d1c700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cb500 0x7f2170169360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f217551d700 1 -- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cb500 msgr2=0x7f2170169360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f217551d700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cb500 0x7f2170169360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f217551d700 1 -- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f217015c760 msgr2=0x7f21700cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:20.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f217551d700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f217015c760 0x7f21700cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f217551d700 1 -- 172.21.15.64:0/3078987958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640094f0 con 0x7f21700cb130 2023-09-27T23:30:20.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f217551d700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 0x7f217014acb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c590 tx=0x7f216c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:20.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c011590 con 0x7f21700cb130 2023-09-27T23:30:20.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c011710 con 0x7f21700cb130 2023-09-27T23:30:20.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2170169d70 con 0x7f21700cb130 2023-09-27T23:30:20.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c011ad0 con 0x7f21700cb130 2023-09-27T23:30:20.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217016cb30 con 0x7f21700cb130 2023-09-27T23:30:20.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f216c01b3b0 con 0x7f21700cb130 2023-09-27T23:30:20.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217016c990 con 0x7f21700cb130 2023-09-27T23:30:20.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f21617fa700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2154060aa0 0x7f2154062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:20.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6671+0+0 (secure 0 0 0) 0x7f216c07c630 con 0x7f21700cb130 2023-09-27T23:30:20.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.091+0000 7f2174d1c700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2154060aa0 0x7f2154062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:20.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.095+0000 7f2174d1c700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2154060aa0 0x7f2154062f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21640099b0 tx=0x7f2164017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.099+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f217016c990 con 0x7f21700cb130 2023-09-27T23:30:20.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.267+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f217016c990 con 0x7f21700cb130 2023-09-27T23:30:20.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.943+0000 7f21617fa700 1 -- 172.21.15.64:0/3078987958 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7f217016c990 con 0x7f21700cb130 2023-09-27T23:30:20.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2154060aa0 msgr2=0x7f2154062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2154060aa0 0x7f2154062f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21640099b0 tx=0x7f2164017000 comp rx=0 tx=0).stop 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 msgr2=0x7f217014acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 0x7f217014acb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c590 tx=0x7f216c00c8a0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 shutdown_connections 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2154060aa0 0x7f2154062f50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f217015c760 0x7f21700cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cb500 0x7f2170169360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 --2- 172.21.15.64:0/3078987958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cb130 0x7f217014acb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 >> 172.21.15.64:0/3078987958 conn(0x7f2170067bb0 msgr2=0x7f217006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 shutdown_connections 2023-09-27T23:30:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:20.959+0000 7f2177781700 1 -- 172.21.15.64:0/3078987958 wait complete. 2023-09-27T23:30:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:30:20.976 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:30:21.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.079+0000 7f6dc3f73700 1 Processor -- start 2023-09-27T23:30:21.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.079+0000 7f6dc3f73700 1 -- start start 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc3f73700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6dbc0cb970 0x7f6dbc06acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc3f73700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6dbc0cecd0 0x7f6dbc06b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc3f73700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc0cbdf0 con 0x7f6dbc158e20 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc3f73700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0cc0d0 con 0x7f6dbc0cb970 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc3f73700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0cc3b0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6dbc0cb970 0x7f6dbc06acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6dbc0cb970 0x7f6dbc06acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42086/0 (socket says 172.21.15.64:42086) 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 -- 172.21.15.64:0/2234629902 learned_addr learned my addr 172.21.15.64:0/2234629902 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc2510700 1 --2- 172.21.15.64:0/2234629902 >> v2:172.21.15.123:3300/0 conn(0x7f6dbc0cecd0 0x7f6dbc06b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 -- 172.21.15.64:0/2234629902 >> v2:172.21.15.123:3300/0 conn(0x7f6dbc0cecd0 msgr2=0x7f6dbc06b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 --2- 172.21.15.64:0/2234629902 >> v2:172.21.15.123:3300/0 conn(0x7f6dbc0cecd0 0x7f6dbc06b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 -- 172.21.15.64:0/2234629902 >> v1:172.21.15.123:6789/0 conn(0x7f6dbc158e20 legacy=0x7f6dbc0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 -- 172.21.15.64:0/2234629902 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0cc690 con 0x7f6dbc0cb970 2023-09-27T23:30:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc150e700 1 --2- 172.21.15.64:0/2234629902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cb970 0x7f6dbc06acc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099b0 tx=0x7f6dac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1838277c3c7079fc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc0d0d700 1 -- 172.21.15.64:0/2234629902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00b820 con 0x7f6dbc0cb970 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc0d0d700 1 -- 172.21.15.64:0/2234629902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00b980 con 0x7f6dbc0cb970 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.083+0000 7f6dc0d0d700 1 -- 172.21.15.64:0/2234629902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00bd20 con 0x7f6dbc0cb970 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 -- 172.21.15.64:0/2234629902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cb970 msgr2=0x7f6dbc06acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/2234629902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cb970 0x7f6dbc06acc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099b0 tx=0x7f6dac009250 comp rx=0 tx=0).stop 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 -- 172.21.15.64:0/2234629902 shutdown_connections 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/2234629902 >> v2:172.21.15.123:3300/0 conn(0x7f6dbc0cecd0 0x7f6dbc06b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/2234629902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cb970 0x7f6dbc06acc0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 -- 172.21.15.64:0/2234629902 >> 172.21.15.64:0/2234629902 conn(0x7f6dbc067bb0 msgr2=0x7f6dbc0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:21.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 -- 172.21.15.64:0/2234629902 shutdown_connections 2023-09-27T23:30:21.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 -- 172.21.15.64:0/2234629902 wait complete. 2023-09-27T23:30:21.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 Processor -- start 2023-09-27T23:30:21.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.087+0000 7f6dc3f73700 1 -- start start 2023-09-27T23:30:21.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 0x7f6dbc14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dbc158e20 0x7f6dbc0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dbc0c9370 0x7f6dbc0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc156aa0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b41c0 con 0x7f6dbc158e20 2023-09-27T23:30:21.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b35b0 con 0x7f6dbc0c9370 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 0x7f6dbc14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 0x7f6dbc14aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42092/0 (socket says 172.21.15.64:42092) 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 -- 172.21.15.64:0/1189133844 learned_addr learned my addr 172.21.15.64:0/1189133844 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc150e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dbc158e20 0x7f6dbc0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc2510700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dbc0c9370 0x7f6dbc0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 -- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dbc0c9370 msgr2=0x7f6dbc0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dbc0c9370 0x7f6dbc0c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 -- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dbc158e20 msgr2=0x7f6dbc0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dbc158e20 0x7f6dbc0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 -- 172.21.15.64:0/1189133844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac0094f0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc2510700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dbc0c9370 0x7f6dbc0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:21.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc1d0f700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 0x7f6dbc14aca0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c590 tx=0x7f6db800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8011590 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8011710 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc0b3890 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8011ad0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc0b3e50 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.091+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc071120 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.095+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6db801b3b0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.095+0000 7f6db1ffb700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6da0060a70 0x7f6da0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.095+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f6db807c6d0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.095+0000 7f6dc150e700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6da0060a70 0x7f6da0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.095+0000 7f6dc150e700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6da0060a70 0x7f6da0062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099b0 tx=0x7f6dac017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.103+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6db804c6d0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.267+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f6dbc06a7a0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.275+0000 7f6db1ffb700 1 -- 172.21.15.64:0/1189133844 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v31) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6db80170b0 con 0x7f6dbc0cecd0 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6da0060a70 msgr2=0x7f6da0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6da0060a70 0x7f6da0062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099b0 tx=0x7f6dac017000 comp rx=0 tx=0).stop 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 msgr2=0x7f6dbc14aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 0x7f6dbc14aca0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c590 tx=0x7f6db800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 shutdown_connections 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6da0060a70 0x7f6da0062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dbc0c9370 0x7f6dbc0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dbc158e20 0x7f6dbc0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 --2- 172.21.15.64:0/1189133844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dbc0cecd0 0x7f6dbc14aca0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 >> 172.21.15.64:0/1189133844 conn(0x7f6dbc067bb0 msgr2=0x7f6dbc14caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:21.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 shutdown_connections 2023-09-27T23:30:21.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.279+0000 7f6dc3f73700 1 -- 172.21.15.64:0/1189133844 wait complete. 2023-09-27T23:30:21.296 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:30:21.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.407+0000 7f9596321700 1 Processor -- start 2023-09-27T23:30:21.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- start start 2023-09-27T23:30:21.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f959006b2c0 0x7f959006b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959006c7c0 con 0x7f959006d210 2023-09-27T23:30:21.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959006cd80 con 0x7f959006d680 2023-09-27T23:30:21.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900b0940 con 0x7f959006b2c0 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f959006d210 0x7f95900afbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40676/0 (socket says 172.21.15.64:40676) 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f959006b2c0 0x7f959006b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958f7fe700 1 -- 172.21.15.64:0/3767640472 learned_addr learned my addr 172.21.15.64:0/3767640472 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958effd700 1 -- 172.21.15.64:0/3767640472 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872819170 0 0) 0x7f959006c7c0 con 0x7f959006d210 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958effd700 1 -- 172.21.15.64:0/3767640472 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f957c003570 con 0x7f959006d210 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958ffff700 1 -- 172.21.15.64:0/3767640472 >> v1:172.21.15.123:6789/0 conn(0x7f959006d680 legacy=0x7f9590158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958ffff700 1 -- 172.21.15.64:0/3767640472 >> v1:172.21.15.64:6789/0 conn(0x7f959006d210 legacy=0x7f95900afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958ffff700 1 -- 172.21.15.64:0/3767640472 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900b0c20 con 0x7f959006b2c0 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958ffff700 1 --2- 172.21.15.64:0/3767640472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006b2c0 0x7f959006b690 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f95780098e0 tx=0x7f95780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbf6781f74ddce1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958effd700 1 -- 172.21.15.64:0/3767640472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800b640 con 0x7f959006b2c0 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958effd700 1 -- 172.21.15.64:0/3767640472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957800b7a0 con 0x7f959006b2c0 2023-09-27T23:30:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f958effd700 1 -- 172.21.15.64:0/3767640472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800bb20 con 0x7f959006b2c0 2023-09-27T23:30:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- 172.21.15.64:0/3767640472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006b2c0 msgr2=0x7f959006b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 --2- 172.21.15.64:0/3767640472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006b2c0 0x7f959006b690 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f95780098e0 tx=0x7f95780091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- 172.21.15.64:0/3767640472 shutdown_connections 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 --2- 172.21.15.64:0/3767640472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006b2c0 0x7f959006b690 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- 172.21.15.64:0/3767640472 >> 172.21.15.64:0/3767640472 conn(0x7f95900cc580 msgr2=0x7f95900cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- 172.21.15.64:0/3767640472 shutdown_connections 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.411+0000 7f9596321700 1 -- 172.21.15.64:0/3767640472 wait complete. 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 Processor -- start 2023-09-27T23:30:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 -- start start 2023-09-27T23:30:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 0x7f9590144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959006d210 0x7f95900ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006d680 0x7f95900ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590159d70 con 0x7f959006d680 2023-09-27T23:30:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590159160 con 0x7f959006d210 2023-09-27T23:30:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f9596321700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9590159440 con 0x7f959006b2c0 2023-09-27T23:30:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 0x7f9590144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f95948be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006d680 0x7f95900ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959006d210 0x7f95900ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 0x7f9590144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49454/0 (socket says 172.21.15.64:49454) 2023-09-27T23:30:21.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 -- 172.21.15.64:0/1060408321 learned_addr learned my addr 172.21.15.64:0/1060408321 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 -- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959006d210 msgr2=0x7f95900ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959006d210 0x7f95900ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 -- 172.21.15.64:0/1060408321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006d680 msgr2=0x7f95900ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006d680 0x7f95900ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f958ffff700 1 -- 172.21.15.64:0/1060408321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009420 con 0x7f959006b2c0 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.415+0000 7f95948be700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006d680 0x7f95900ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f958ffff700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 0x7f9590144430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9590064900 tx=0x7f9578017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800bb20 con 0x7f959006b2c0 2023-09-27T23:30:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9590159720 con 0x7f959006b2c0 2023-09-27T23:30:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9578000c90 con 0x7f959006b2c0 2023-09-27T23:30:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578011630 con 0x7f959006b2c0 2023-09-27T23:30:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9590144970 con 0x7f959006b2c0 2023-09-27T23:30:21.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9590071120 con 0x7f959006b2c0 2023-09-27T23:30:21.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9578011a60 con 0x7f959006b2c0 2023-09-27T23:30:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.419+0000 7f9587fff700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9570060a70 0x7f9570062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.423+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f957807ecf0 con 0x7f959006b2c0 2023-09-27T23:30:21.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.423+0000 7f958f7fe700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9570060a70 0x7f9570062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.423+0000 7f958f7fe700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9570060a70 0x7f9570062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f95900abe50 tx=0x7f9580009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.427+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f957804ed90 con 0x7f959006b2c0 2023-09-27T23:30:21.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.591+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f959006c0b0 con 0x7f959006b2c0 2023-09-27T23:30:21.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.603+0000 7f9587fff700 1 -- 172.21.15.64:0/1060408321 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7f95780523b0 con 0x7f959006b2c0 2023-09-27T23:30:21.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9570060a70 msgr2=0x7f9570062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9570060a70 0x7f9570062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f95900abe50 tx=0x7f9580009040 comp rx=0 tx=0).stop 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 msgr2=0x7f9590144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 0x7f9590144430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9590064900 tx=0x7f9578017000 comp rx=0 tx=0).stop 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 shutdown_connections 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9570060a70 0x7f9570062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959006d680 0x7f95900ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959006d210 0x7f95900ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 --2- 172.21.15.64:0/1060408321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f959006b2c0 0x7f9590144430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 >> 172.21.15.64:0/1060408321 conn(0x7f95900cc580 msgr2=0x7f959006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 shutdown_connections 2023-09-27T23:30:21.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.607+0000 7f9596321700 1 -- 172.21.15.64:0/1060408321 wait complete. 2023-09-27T23:30:21.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:30:21.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.731+0000 7fe07f56b700 1 Processor -- start 2023-09-27T23:30:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.731+0000 7fe07f56b700 1 -- start start 2023-09-27T23:30:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe078158e20 0x7fe0780ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0780cb970 0x7fe0780ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0780cbdf0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780cc0d0 con 0x7fe0780cb970 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780cc3b0 con 0x7fe078158e20 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0780cb970 0x7fe0780ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0780cb970 0x7fe0780ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42118/0 (socket says 172.21.15.64:42118) 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 -- 172.21.15.64:0/8486906 learned_addr learned my addr 172.21.15.64:0/8486906 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07d307700 1 --2- 172.21.15.64:0/8486906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe078158e20 0x7fe0780ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 -- 172.21.15.64:0/8486906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe078158e20 msgr2=0x7fe0780ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 --2- 172.21.15.64:0/8486906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe078158e20 0x7fe0780ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 -- 172.21.15.64:0/8486906 >> v1:172.21.15.123:6789/0 conn(0x7fe0780ca2c0 legacy=0x7fe07806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 -- 172.21.15.64:0/8486906 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780cc690 con 0x7fe0780cb970 2023-09-27T23:30:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07cb06700 1 --2- 172.21.15.64:0/8486906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780cb970 0x7fe0780ceb00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe0740099b0 tx=0x7fe074009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3056c7b663377f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe06bfff700 1 -- 172.21.15.64:0/8486906 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400b860 con 0x7fe0780cb970 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe06bfff700 1 -- 172.21.15.64:0/8486906 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07400b9c0 con 0x7fe0780cb970 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe06bfff700 1 -- 172.21.15.64:0/8486906 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400bd60 con 0x7fe0780cb970 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- 172.21.15.64:0/8486906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780cb970 msgr2=0x7fe0780ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 --2- 172.21.15.64:0/8486906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780cb970 0x7fe0780ceb00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe0740099b0 tx=0x7fe074009250 comp rx=0 tx=0).stop 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- 172.21.15.64:0/8486906 shutdown_connections 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 --2- 172.21.15.64:0/8486906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780cb970 0x7fe0780ceb00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 --2- 172.21.15.64:0/8486906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe078158e20 0x7fe0780ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- 172.21.15.64:0/8486906 >> 172.21.15.64:0/8486906 conn(0x7fe078067bb0 msgr2=0x7fe0780d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- 172.21.15.64:0/8486906 shutdown_connections 2023-09-27T23:30:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.735+0000 7fe07f56b700 1 -- 172.21.15.64:0/8486906 wait complete. 2023-09-27T23:30:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 Processor -- start 2023-09-27T23:30:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 -- start start 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 0x7fe07814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe078158e20 0x7fe07814c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0780c9870 0x7fe078154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07814e900 con 0x7fe0780ca2c0 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07814dcf0 con 0x7fe078158e20 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07f56b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07814dfd0 con 0x7fe0780c9870 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07d307700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 0x7fe07814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07d307700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 0x7fe07814d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42128/0 (socket says 172.21.15.64:42128) 2023-09-27T23:30:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.739+0000 7fe07d307700 1 -- 172.21.15.64:0/1116046764 learned_addr learned my addr 172.21.15.64:0/1116046764 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07cb06700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe078158e20 0x7fe07814c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07db08700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0780c9870 0x7fe078154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07d307700 1 -- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0780c9870 msgr2=0x7fe078154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07d307700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0780c9870 0x7fe078154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07d307700 1 -- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe078158e20 msgr2=0x7fe07814c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07d307700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe078158e20 0x7fe07814c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07d307700 1 -- 172.21.15.64:0/1116046764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740094f0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07d307700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 0x7fe07814d3a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00d860 tx=0x7fe06c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0125f0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c012750 con 0x7fe0780ca2c0 2023-09-27T23:30:21.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07814e2b0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c012b10 con 0x7fe0780ca2c0 2023-09-27T23:30:21.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07814d8e0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.743+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe078071120 con 0x7fe0780ca2c0 2023-09-27T23:30:21.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.747+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe06c01e460 con 0x7fe0780ca2c0 2023-09-27T23:30:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.747+0000 7fe0697fa700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe05c060a70 0x7fe05c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.747+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fe06c07d620 con 0x7fe0780ca2c0 2023-09-27T23:30:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.747+0000 7fe07cb06700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe05c060a70 0x7fe05c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.747+0000 7fe07cb06700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe05c060a70 0x7fe05c062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0740099b0 tx=0x7fe074017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:21.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.751+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe06c051150 con 0x7fe0780ca2c0 2023-09-27T23:30:21.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.915+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe07806a7a0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.923+0000 7fe0697fa700 1 -- 172.21.15.64:0/1116046764 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe06c050ce0 con 0x7fe0780ca2c0 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe05c060a70 msgr2=0x7fe05c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe05c060a70 0x7fe05c062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0740099b0 tx=0x7fe074017000 comp rx=0 tx=0).stop 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 msgr2=0x7fe07814d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 0x7fe07814d3a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00d860 tx=0x7fe06c008040 comp rx=0 tx=0).stop 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 shutdown_connections 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe05c060a70 0x7fe05c062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0780c9870 0x7fe078154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe078158e20 0x7fe07814c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 --2- 172.21.15.64:0/1116046764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0780ca2c0 0x7fe07814d3a0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 >> 172.21.15.64:0/1116046764 conn(0x7fe078067bb0 msgr2=0x7fe078156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 shutdown_connections 2023-09-27T23:30:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:21.927+0000 7fe07f56b700 1 -- 172.21.15.64:0/1116046764 wait complete. 2023-09-27T23:30:21.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-09-27T23:30:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 Processor -- start 2023-09-27T23:30:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 -- start start 2023-09-27T23:30:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c0c15b0 0x7f3b0c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b0c0aeba0 0x7f3b0c0680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c0c7190 con 0x7f3b0c15d320 2023-09-27T23:30:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c1b50 con 0x7f3b0c0aeba0 2023-09-27T23:30:22.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b12cae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c1e30 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b0c0aeba0 0x7f3b0c0680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b0c0aeba0 0x7f3b0c0680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42138/0 (socket says 172.21.15.64:42138) 2023-09-27T23:30:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 -- 172.21.15.64:0/961400855 learned_addr learned my addr 172.21.15.64:0/961400855 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b0bfff700 1 --2- 172.21.15.64:0/961400855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c0c15b0 0x7f3b0c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 -- 172.21.15.64:0/961400855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c0c15b0 msgr2=0x7f3b0c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 --2- 172.21.15.64:0/961400855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c0c15b0 0x7f3b0c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 -- 172.21.15.64:0/961400855 >> v1:172.21.15.123:6789/0 conn(0x7f3b0c15d320 legacy=0x7f3b0c06d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.051+0000 7f3b1124b700 1 -- 172.21.15.64:0/961400855 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0c6e20 con 0x7f3b0c0aeba0 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b1124b700 1 --2- 172.21.15.64:0/961400855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c0680f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009f20 tx=0x7f3b04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b9bf9e80c037ac server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b0b7fe700 1 -- 172.21.15.64:0/961400855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400e070 con 0x7f3b0c0aeba0 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b0b7fe700 1 -- 172.21.15.64:0/961400855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0400b9d0 con 0x7f3b0c0aeba0 2023-09-27T23:30:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b0b7fe700 1 -- 172.21.15.64:0/961400855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b040092e0 con 0x7f3b0c0aeba0 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 -- 172.21.15.64:0/961400855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0aeba0 msgr2=0x7f3b0c0680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 --2- 172.21.15.64:0/961400855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c0680f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009f20 tx=0x7f3b04009840 comp rx=0 tx=0).stop 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 -- 172.21.15.64:0/961400855 shutdown_connections 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 --2- 172.21.15.64:0/961400855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c0680f0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 --2- 172.21.15.64:0/961400855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c0c15b0 0x7f3b0c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 -- 172.21.15.64:0/961400855 >> 172.21.15.64:0/961400855 conn(0x7f3b0c0c8850 msgr2=0x7f3b0c0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 -- 172.21.15.64:0/961400855 shutdown_connections 2023-09-27T23:30:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.055+0000 7f3b12cae700 1 -- 172.21.15.64:0/961400855 wait complete. 2023-09-27T23:30:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 Processor -- start 2023-09-27T23:30:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 -- start start 2023-09-27T23:30:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 0x7f3b0c0c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c15d320 0x7f3b0c15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c150d40 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c15da20 con 0x7f3b0c0aeba0 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b12cae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0afaf0 con 0x7f3b0c15d320 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 0x7f3b0c0c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 0x7f3b0c0c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42142/0 (socket says 172.21.15.64:42142) 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 -- 172.21.15.64:0/1927335376 learned_addr learned my addr 172.21.15.64:0/1927335376 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b1124b700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c15d320 0x7f3b0c15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b10a4a700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 -- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c15d320 msgr2=0x7f3b0c15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c15d320 0x7f3b0c15fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 -- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b0c0aeba0 msgr2=0x7f3b0c150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 -- 172.21.15.64:0/1927335376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009ae0 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b10a4a700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.059+0000 7f3b0bfff700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 0x7f3b0c0c2e90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00ec80 tx=0x7f3afc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00c850 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00c9b0 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c160610 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00cd50 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c0af4a0 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c071120 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3afc013680 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b08ff9700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3af8060a70 0x7f3af8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f3afc07d6f0 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.063+0000 7f3b10a4a700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3af8060a70 0x7f3af8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.067+0000 7f3b10a4a700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3af8060a70 0x7f3af8062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007fd0 tx=0x7f3b00009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.071+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3afc04d790 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.295+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f3b0c0afc30 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.963+0000 7f3b08ff9700 1 -- 172.21.15.64:0/1927335376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v618) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3afc050db0 con 0x7f3b0c0c15b0 2023-09-27T23:30:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3af8060a70 msgr2=0x7f3af8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3af8060a70 0x7f3af8062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b00007fd0 tx=0x7f3b00009040 comp rx=0 tx=0).stop 2023-09-27T23:30:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 msgr2=0x7f3b0c0c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 0x7f3b0c0c2e90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00ec80 tx=0x7f3afc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 shutdown_connections 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3af8060a70 0x7f3af8062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b0c15d320 0x7f3b0c15fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b0c0c15b0 0x7f3b0c0c2e90 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 --2- 172.21.15.64:0/1927335376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b0c0aeba0 0x7f3b0c150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 >> 172.21.15.64:0/1927335376 conn(0x7f3b0c0c8850 msgr2=0x7f3b0c06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 shutdown_connections 2023-09-27T23:30:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:22.967+0000 7f3b12cae700 1 -- 172.21.15.64:0/1927335376 wait complete. 2023-09-27T23:30:22.986 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-09-27T23:30:22.989 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-09-27T23:30:22.990 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:30:22.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-09-27T23:30:23.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 Processor -- start 2023-09-27T23:30:23.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 -- start start 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2acc0c7f00 0x7f2acc0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2acc06d990 con 0x7f2acc06de20 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2acc15a570 con 0x7f2acc0c7f00 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ad0d87700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc06cd80 con 0x7f2acc155960 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2acc0c7f00 0x7f2acc0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ac3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2acc06de20 0x7f2acc0c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40690/0 (socket says 172.21.15.64:40690) 2023-09-27T23:30:23.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2ac3fff700 1 -- 172.21.15.64:0/2748826289 learned_addr learned my addr 172.21.15.64:0/2748826289 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:23.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acb7fe700 1 --2- 172.21.15.64:0/2748826289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:23.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acbfff700 1 -- 172.21.15.64:0/2748826289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 msgr2=0x7f2acc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:23.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acbfff700 1 --2- 172.21.15.64:0/2748826289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acbfff700 1 -- 172.21.15.64:0/2748826289 >> v1:172.21.15.64:6789/0 conn(0x7f2acc06de20 legacy=0x7f2acc0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acaffd700 1 -- 172.21.15.64:0/2748826289 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894545963 0 0) 0x7f2acc06d990 con 0x7f2acc06de20 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acbfff700 1 -- 172.21.15.64:0/2748826289 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc06d060 con 0x7f2acc0c7f00 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.107+0000 7f2acbfff700 1 --2- 172.21.15.64:0/2748826289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc0c7f00 0x7f2acc0aca10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009e60 tx=0x7f2ac4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b2ea93d1c1f62e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2acaffd700 1 -- 172.21.15.64:0/2748826289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400e070 con 0x7f2acc0c7f00 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2acaffd700 1 -- 172.21.15.64:0/2748826289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac400bb60 con 0x7f2acc0c7f00 2023-09-27T23:30:23.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2acaffd700 1 -- 172.21.15.64:0/2748826289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4012430 con 0x7f2acc0c7f00 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 -- 172.21.15.64:0/2748826289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc0c7f00 msgr2=0x7f2acc0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/2748826289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc0c7f00 0x7f2acc0aca10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009e60 tx=0x7f2ac4009780 comp rx=0 tx=0).stop 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 -- 172.21.15.64:0/2748826289 shutdown_connections 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/2748826289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc0c7f00 0x7f2acc0aca10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/2748826289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 -- 172.21.15.64:0/2748826289 >> 172.21.15.64:0/2748826289 conn(0x7f2acc067bb0 msgr2=0x7f2acc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 -- 172.21.15.64:0/2748826289 shutdown_connections 2023-09-27T23:30:23.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.111+0000 7f2ad0d87700 1 -- 172.21.15.64:0/2748826289 wait complete. 2023-09-27T23:30:23.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 Processor -- start 2023-09-27T23:30:23.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 -- start start 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 0x7f2acc146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2acc0c7f00 0x7f2acc144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc13b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0b11d0 con 0x7f2acc06de20 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc145c20 con 0x7f2acc0c7f00 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0b02b0 con 0x7f2acc155960 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 0x7f2acc146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 0x7f2acc146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42168/0 (socket says 172.21.15.64:42168) 2023-09-27T23:30:23.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 -- 172.21.15.64:0/3727175269 learned_addr learned my addr 172.21.15.64:0/3727175269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:23.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ac3fff700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2acc0c7f00 0x7f2acc144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:23.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acbfff700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc13b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 -- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 msgr2=0x7f2acc13b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc13b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 -- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2acc0c7f00 msgr2=0x7f2acc144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2acc0c7f00 0x7f2acc144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 -- 172.21.15.64:0/3727175269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4009a20 con 0x7f2acc06de20 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ac3fff700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2acc0c7f00 0x7f2acc144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2acb7fe700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 0x7f2acc146870 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00e980 tx=0x7f2abc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00ee70 con 0x7f2acc06de20 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc00c850 con 0x7f2acc06de20 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc0b05b0 con 0x7f2acc06de20 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00cc50 con 0x7f2acc06de20 2023-09-27T23:30:23.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.115+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc0b0df0 con 0x7f2acc06de20 2023-09-27T23:30:23.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.119+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc0b1310 con 0x7f2acc06de20 2023-09-27T23:30:23.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.119+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2abc013680 con 0x7f2acc06de20 2023-09-27T23:30:23.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.119+0000 7f2ac37fe700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2aac060a70 0x7f2aac062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:23.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.119+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f2abc07d1f0 con 0x7f2acc06de20 2023-09-27T23:30:23.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.119+0000 7f2ac3fff700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2aac060a70 0x7f2aac062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:23.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.123+0000 7f2ac3fff700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2aac060a70 0x7f2aac062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2acc154fc0 tx=0x7f2ab4009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:23.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.127+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2acc0b1310 con 0x7f2acc06de20 2023-09-27T23:30:23.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.343+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f2acc0b1310 con 0x7f2acc06de20 2023-09-27T23:30:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.971+0000 7f2ac37fe700 1 -- 172.21.15.64:0/3727175269 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v619) v1 ==== 186+0+0 (secure 0 0 0) 0x7f2acc0b1310 con 0x7f2acc06de20 2023-09-27T23:30:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2aac060a70 msgr2=0x7f2aac062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2aac060a70 0x7f2aac062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2acc154fc0 tx=0x7f2ab4009040 comp rx=0 tx=0).stop 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 msgr2=0x7f2acc146870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 0x7f2acc146870 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00e980 tx=0x7f2abc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 shutdown_connections 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2aac060a70 0x7f2aac062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2acc155960 0x7f2acc13b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2acc0c7f00 0x7f2acc144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 --2- 172.21.15.64:0/3727175269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2acc06de20 0x7f2acc146870 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 >> 172.21.15.64:0/3727175269 conn(0x7f2acc067bb0 msgr2=0x7f2acc06abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 shutdown_connections 2023-09-27T23:30:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:23.975+0000 7f2ad0d87700 1 -- 172.21.15.64:0/3727175269 wait complete. 2023-09-27T23:30:23.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:30:24.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.107+0000 7f876b59e700 1 Processor -- start 2023-09-27T23:30:24.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.107+0000 7f876b59e700 1 -- start start 2023-09-27T23:30:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f876c13f2d0 0x7f876c0ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0adc60 con 0x7f876c0c7780 2023-09-27T23:30:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0ad050 con 0x7f876c13f6a0 2023-09-27T23:30:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c0ad330 con 0x7f876c13f2d0 2023-09-27T23:30:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876a59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f876c13f2d0 0x7f876c0ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876ad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f876c0c7780 0x7f876c06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40704/0 (socket says 172.21.15.64:40704) 2023-09-27T23:30:24.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876ad9d700 1 -- 172.21.15.64:0/4192675238 learned_addr learned my addr 172.21.15.64:0/4192675238 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876959a700 1 -- 172.21.15.64:0/4192675238 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975849268 0 0) 0x7f876c0adc60 con 0x7f876c0c7780 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876959a700 1 -- 172.21.15.64:0/4192675238 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8758003570 con 0x7f876c0c7780 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876959a700 1 -- 172.21.15.64:0/4192675238 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71434695 0 0) 0x7f876c0ad050 con 0x7f876c13f6a0 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876a59c700 1 -- 172.21.15.64:0/4192675238 >> v1:172.21.15.123:6789/0 conn(0x7f876c13f6a0 legacy=0x7f876c15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876a59c700 1 -- 172.21.15.64:0/4192675238 >> v1:172.21.15.64:6789/0 conn(0x7f876c0c7780 legacy=0x7f876c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876a59c700 1 -- 172.21.15.64:0/4192675238 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0ad610 con 0x7f876c13f2d0 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876a59c700 1 --2- 172.21.15.64:0/4192675238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c0ac9b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f875c0098e0 tx=0x7f875c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802b4222af5b147c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876959a700 1 -- 172.21.15.64:0/4192675238 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00b640 con 0x7f876c13f2d0 2023-09-27T23:30:24.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876959a700 1 -- 172.21.15.64:0/4192675238 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00b7a0 con 0x7f876c13f2d0 2023-09-27T23:30:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876959a700 1 -- 172.21.15.64:0/4192675238 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00bb20 con 0x7f876c13f2d0 2023-09-27T23:30:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- 172.21.15.64:0/4192675238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 msgr2=0x7f876c0ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 --2- 172.21.15.64:0/4192675238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c0ac9b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f875c0098e0 tx=0x7f875c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- 172.21.15.64:0/4192675238 shutdown_connections 2023-09-27T23:30:24.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 --2- 172.21.15.64:0/4192675238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c0ac9b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- 172.21.15.64:0/4192675238 >> 172.21.15.64:0/4192675238 conn(0x7f876c067b50 msgr2=0x7f876c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- 172.21.15.64:0/4192675238 shutdown_connections 2023-09-27T23:30:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.111+0000 7f876b59e700 1 -- 172.21.15.64:0/4192675238 wait complete. 2023-09-27T23:30:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 Processor -- start 2023-09-27T23:30:24.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 -- start start 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f876c0c7780 0x7f876c138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c13a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876c13f6a0 0x7f876c13a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1478f0 con 0x7f876c13f2d0 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c13aeb0 con 0x7f876c13f6a0 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c13b160 con 0x7f876c0c7780 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c13a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c13a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42174/0 (socket says 172.21.15.64:42174) 2023-09-27T23:30:24.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 -- 172.21.15.64:0/50004250 learned_addr learned my addr 172.21.15.64:0/50004250 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876a59c700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f876c0c7780 0x7f876c138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876ad9d700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876c13f6a0 0x7f876c13a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 -- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f876c0c7780 msgr2=0x7f876c138f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f876c0c7780 0x7f876c138f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 -- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876c13f6a0 msgr2=0x7f876c13a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876c13f6a0 0x7f876c13a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 -- 172.21.15.64:0/50004250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009420 con 0x7f876c13f2d0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876ad9d700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876c13f6a0 0x7f876c13a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f8769d9b700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c13a430 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8754001560 tx=0x7f8754001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400a780 con 0x7f876c13f2d0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400a900 con 0x7f876c13f2d0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400acc0 con 0x7f876c13f2d0 2023-09-27T23:30:24.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c13b440 con 0x7f876c13f2d0 2023-09-27T23:30:24.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.115+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c147580 con 0x7f876c13f2d0 2023-09-27T23:30:24.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.119+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c147aa0 con 0x7f876c13f2d0 2023-09-27T23:30:24.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.119+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8754003030 con 0x7f876c13f2d0 2023-09-27T23:30:24.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.119+0000 7f87627fc700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f874c060a70 0x7f874c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.119+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f875407d270 con 0x7f876c13f2d0 2023-09-27T23:30:24.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.119+0000 7f876a59c700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f874c060a70 0x7f874c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.123+0000 7f876a59c700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f874c060a70 0x7f874c062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f875c0093f0 tx=0x7f875c017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.127+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f876c147aa0 con 0x7f876c13f2d0 2023-09-27T23:30:24.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.287+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f876c13ca90 con 0x7f876c13f2d0 2023-09-27T23:30:24.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f87627fc700 1 -- 172.21.15.64:0/50004250 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f875404d270 con 0x7f876c13f2d0 2023-09-27T23:30:24.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f874c060a70 msgr2=0x7f874c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f874c060a70 0x7f874c062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f875c0093f0 tx=0x7f875c017000 comp rx=0 tx=0).stop 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 msgr2=0x7f876c13a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c13a430 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8754001560 tx=0x7f8754001770 comp rx=0 tx=0).stop 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 shutdown_connections 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f874c060a70 0x7f874c062f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876c13f6a0 0x7f876c13a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876c13f2d0 0x7f876c13a430 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 --2- 172.21.15.64:0/50004250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f876c0c7780 0x7f876c138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 >> 172.21.15.64:0/50004250 conn(0x7f876c067b50 msgr2=0x7f876c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 shutdown_connections 2023-09-27T23:30:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.299+0000 7f876b59e700 1 -- 172.21.15.64:0/50004250 wait complete. 2023-09-27T23:30:24.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:30:24.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:30:24.331 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:30:24.352 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:30:24.352 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:30:24.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.459+0000 7f5406179700 1 Processor -- start 2023-09-27T23:30:24.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.459+0000 7f5406179700 1 -- start start 2023-09-27T23:30:24.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f540006d760 0x7f54001591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5400159700 0x7f54000c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5400159f90 con 0x7f5400158df0 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540015a270 con 0x7f5400159700 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000c7930 con 0x7f540006d760 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405978700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5400159700 0x7f54000c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405177700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5400158df0 0x7f540006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40710/0 (socket says 172.21.15.64:40710) 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405177700 1 -- 172.21.15.64:0/89090071 learned_addr learned my addr 172.21.15.64:0/89090071 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5404976700 1 --2- 172.21.15.64:0/89090071 >> v2:172.21.15.123:3300/0 conn(0x7f540006d760 0x7f54001591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405978700 1 -- 172.21.15.64:0/89090071 >> v2:172.21.15.123:3300/0 conn(0x7f540006d760 msgr2=0x7f54001591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405978700 1 --2- 172.21.15.64:0/89090071 >> v2:172.21.15.123:3300/0 conn(0x7f540006d760 0x7f54001591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405978700 1 -- 172.21.15.64:0/89090071 >> v1:172.21.15.64:6789/0 conn(0x7f5400158df0 legacy=0x7f540006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405978700 1 -- 172.21.15.64:0/89090071 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000c7c10 con 0x7f5400159700 2023-09-27T23:30:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5405978700 1 --2- 172.21.15.64:0/89090071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400159700 0x7f54000c8130 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5400069200 tx=0x7f53f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b674ceaa83a95d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f53f7fff700 1 -- 172.21.15.64:0/89090071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f800e070 con 0x7f5400159700 2023-09-27T23:30:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f53f7fff700 1 -- 172.21.15.64:0/89090071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f800bb60 con 0x7f5400159700 2023-09-27T23:30:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f53f7fff700 1 -- 172.21.15.64:0/89090071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8012430 con 0x7f5400159700 2023-09-27T23:30:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- 172.21.15.64:0/89090071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400159700 msgr2=0x7f54000c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 --2- 172.21.15.64:0/89090071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400159700 0x7f54000c8130 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5400069200 tx=0x7f53f8009780 comp rx=0 tx=0).stop 2023-09-27T23:30:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- 172.21.15.64:0/89090071 shutdown_connections 2023-09-27T23:30:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 --2- 172.21.15.64:0/89090071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400159700 0x7f54000c8130 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 --2- 172.21.15.64:0/89090071 >> v2:172.21.15.123:3300/0 conn(0x7f540006d760 0x7f54001591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- 172.21.15.64:0/89090071 >> 172.21.15.64:0/89090071 conn(0x7f540006c080 msgr2=0x7f54000ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- 172.21.15.64:0/89090071 shutdown_connections 2023-09-27T23:30:24.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.463+0000 7f5406179700 1 -- 172.21.15.64:0/89090071 wait complete. 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 Processor -- start 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 -- start start 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540006d760 0x7f54000b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 0x7f54000c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5400159700 0x7f540016c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540015b840 con 0x7f5400158df0 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540015bb20 con 0x7f540006d760 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540015be00 con 0x7f5400159700 2023-09-27T23:30:24.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 0x7f54000c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 0x7f54000c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42188/0 (socket says 172.21.15.64:42188) 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 -- 172.21.15.64:0/2151532701 learned_addr learned my addr 172.21.15.64:0/2151532701 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5405177700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540006d760 0x7f54000b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5405978700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5400159700 0x7f540016c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 -- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5400159700 msgr2=0x7f540016c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5400159700 0x7f540016c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 -- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540006d760 msgr2=0x7f54000b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540006d760 0x7f54000b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 -- 172.21.15.64:0/2151532701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8009a20 con 0x7f5400158df0 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5405978700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5400159700 0x7f540016c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5404976700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 0x7f54000c44c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e8d0 tx=0x7f53fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00ee70 con 0x7f5400158df0 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc00c8e0 con 0x7f5400158df0 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540015c0e0 con 0x7f5400158df0 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00cc80 con 0x7f5400158df0 2023-09-27T23:30:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.467+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54000c7160 con 0x7f5400158df0 2023-09-27T23:30:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.471+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f53fc013680 con 0x7f5400158df0 2023-09-27T23:30:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.471+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54000c91d0 con 0x7f5400158df0 2023-09-27T23:30:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.471+0000 7f53f57fa700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f53e4060a70 0x7f53e4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.471+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f53fc07d1f0 con 0x7f5400158df0 2023-09-27T23:30:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.471+0000 7f5405177700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f53e4060a70 0x7f53e4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.471+0000 7f5405177700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f53e4060a70 0x7f53e4062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001820 tx=0x7f53f0009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.475+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f54000c91d0 con 0x7f5400158df0 2023-09-27T23:30:24.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54000c7160 con 0x7f5400158df0 2023-09-27T23:30:24.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f53f57fa700 1 -- 172.21.15.64:0/2151532701 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f53fc017de0 con 0x7f5400158df0 2023-09-27T23:30:24.677 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:24.677 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2132184505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2132184505},{"type":"v1","addr":"172.21.15.64:6833","nonce":2132184505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/693510709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":693510709},{"type":"v1","addr":"172.21.15.123:6837","nonce":693510709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7101,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1466755506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1466755506},{"type":"v1","addr":"172.21.15.123:6835","nonce":1466755506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/4186426025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4186426025},{"type":"v1","addr":"172.21.15.64:6835","nonce":4186426025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-09-27T23:30:24.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f53e4060a70 msgr2=0x7f53e4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f53e4060a70 0x7f53e4062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001820 tx=0x7f53f0009040 comp rx=0 tx=0).stop 2023-09-27T23:30:24.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 msgr2=0x7f54000c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 0x7f54000c44c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e8d0 tx=0x7f53fc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 shutdown_connections 2023-09-27T23:30:24.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f53e4060a70 0x7f53e4062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5400159700 0x7f540016c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5400158df0 0x7f54000c44c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 --2- 172.21.15.64:0/2151532701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540006d760 0x7f54000b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.675+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 >> 172.21.15.64:0/2151532701 conn(0x7f540006c080 msgr2=0x7f540006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.679+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 shutdown_connections 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.679+0000 7f5406179700 1 -- 172.21.15.64:0/2151532701 wait complete. 2023-09-27T23:30:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2023-09-27T23:30:24.692 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:30:24.692 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:30:24.692 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:30:24.692 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:30:24.693 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:30:24.693 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:30:24.693 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:30:24.693 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:30:24.693 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:30:24.693 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:30:24.694 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:30:24.694 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:30:24.694 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:30:24.694 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:30:24.694 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:30:24.694 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:30:24.703 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:30:24.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:30:24.706 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:30:24.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:30:24.709 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:30:24.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:30:24.710 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:30:24.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:30:24.711 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:30:24.711 DEBUG:teuthology.parallel:result is None 2023-09-27T23:30:24.712 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:30:24.713 DEBUG:teuthology.parallel:result is None 2023-09-27T23:30:24.749 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:30:24.749 DEBUG:teuthology.parallel:result is None 2023-09-27T23:30:24.750 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:30:24.750 DEBUG:teuthology.parallel:result is None 2023-09-27T23:30:24.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:30:24.782 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:30:24.783 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:30:24.818 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:30:24.839 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:30:24.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.871+0000 7fe90a3fb700 1 Processor -- start 2023-09-27T23:30:24.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.871+0000 7fe90a3fb700 1 -- start start 2023-09-27T23:30:24.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe90a3fb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe904158e20 0x7fe90406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe90a3fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9040ce810 0x7fe9040ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe90a3fb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90406b730 con 0x7fe9041591f0 2023-09-27T23:30:24.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe90a3fb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90406ab20 con 0x7fe904158e20 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe90a3fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90406ae00 con 0x7fe9040ce810 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe904158e20 0x7fe90406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe904158e20 0x7fe90406a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40076/0 (socket says 172.21.15.64:40076) 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 -- 172.21.15.64:0/430148735 learned_addr learned my addr 172.21.15.64:0/430148735 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe909bfa700 1 --2- 172.21.15.64:0/430148735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9040ce810 0x7fe9040ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 -- 172.21.15.64:0/430148735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9040ce810 msgr2=0x7fe9040ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 --2- 172.21.15.64:0/430148735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9040ce810 0x7fe9040ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 -- 172.21.15.64:0/430148735 >> v1:172.21.15.123:6789/0 conn(0x7fe9041591f0 legacy=0x7fe9040ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 -- 172.21.15.64:0/430148735 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90406b0e0 con 0x7fe904158e20 2023-09-27T23:30:24.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe9093f9700 1 --2- 172.21.15.64:0/430148735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe904158e20 0x7fe90406a540 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009f20 tx=0x7fe8fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45e9b8cecf5987f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe8fbfff700 1 -- 172.21.15.64:0/430148735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00e070 con 0x7fe904158e20 2023-09-27T23:30:24.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe8fbfff700 1 -- 172.21.15.64:0/430148735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00ba80 con 0x7fe904158e20 2023-09-27T23:30:24.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.875+0000 7fe8fbfff700 1 -- 172.21.15.64:0/430148735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0092e0 con 0x7fe904158e20 2023-09-27T23:30:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 -- 172.21.15.64:0/430148735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe904158e20 msgr2=0x7fe90406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/430148735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe904158e20 0x7fe90406a540 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009f20 tx=0x7fe8fc009840 comp rx=0 tx=0).stop 2023-09-27T23:30:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 -- 172.21.15.64:0/430148735 shutdown_connections 2023-09-27T23:30:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/430148735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9040ce810 0x7fe9040ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/430148735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe904158e20 0x7fe90406a540 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 -- 172.21.15.64:0/430148735 >> 172.21.15.64:0/430148735 conn(0x7fe904067bb0 msgr2=0x7fe9040cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 -- 172.21.15.64:0/430148735 shutdown_connections 2023-09-27T23:30:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 -- 172.21.15.64:0/430148735 wait complete. 2023-09-27T23:30:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 Processor -- start 2023-09-27T23:30:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.879+0000 7fe90a3fb700 1 -- start start 2023-09-27T23:30:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 0x7fe904152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe904158e20 0x7fe90415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9041591f0 0x7fe9041564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904152f50 con 0x7fe9040ce810 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90415b450 con 0x7fe904158e20 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040c88a0 con 0x7fe9041591f0 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 0x7fe904152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 0x7fe904152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40082/0 (socket says 172.21.15.64:40082) 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 -- 172.21.15.64:0/3604748408 learned_addr learned my addr 172.21.15.64:0/3604748408 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe909bfa700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9041591f0 0x7fe9041564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe908bf8700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe904158e20 0x7fe90415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 -- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9041591f0 msgr2=0x7fe9041564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9041591f0 0x7fe9041564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 -- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe904158e20 msgr2=0x7fe90415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe904158e20 0x7fe90415aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 -- 172.21.15.64:0/3604748408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc009ae0 con 0x7fe9040ce810 2023-09-27T23:30:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe9093f9700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 0x7fe904152a10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00b6c0 tx=0x7fe8fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0088b0 con 0x7fe9040ce810 2023-09-27T23:30:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc008a10 con 0x7fe9040ce810 2023-09-27T23:30:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9040c7f70 con 0x7fe9040ce810 2023-09-27T23:30:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc008dd0 con 0x7fe9040ce810 2023-09-27T23:30:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.883+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9040c8530 con 0x7fe9040ce810 2023-09-27T23:30:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.887+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90406a8e0 con 0x7fe9040ce810 2023-09-27T23:30:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.887+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe8fc003040 con 0x7fe9040ce810 2023-09-27T23:30:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.887+0000 7fe8f97fa700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe8e8060a70 0x7fe8e8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.887+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fe8fc07ef90 con 0x7fe9040ce810 2023-09-27T23:30:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.887+0000 7fe908bf8700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe8e8060a70 0x7fe8e8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.891+0000 7fe908bf8700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe8e8060a70 0x7fe8e8062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe9040cf6c0 tx=0x7fe8f4007ec0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:24.895+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe90406a8e0 con 0x7fe9040ce810 2023-09-27T23:30:25.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.071+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe90406a8e0 con 0x7fe9040ce810 2023-09-27T23:30:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.991+0000 7fe8f97fa700 1 -- 172.21.15.64:0/3604748408 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe90406a8e0 con 0x7fe9040ce810 2023-09-27T23:30:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe8e8060a70 msgr2=0x7fe8e8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe8e8060a70 0x7fe8e8062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe9040cf6c0 tx=0x7fe8f4007ec0 comp rx=0 tx=0).stop 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 msgr2=0x7fe904152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 0x7fe904152a10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00b6c0 tx=0x7fe8fc018000 comp rx=0 tx=0).stop 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 shutdown_connections 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe8e8060a70 0x7fe8e8062f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9041591f0 0x7fe9041564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe904158e20 0x7fe90415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 --2- 172.21.15.64:0/3604748408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9040ce810 0x7fe904152a10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 >> 172.21.15.64:0/3604748408 conn(0x7fe904067bb0 msgr2=0x7fe9040b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 shutdown_connections 2023-09-27T23:30:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:25.995+0000 7fe90a3fb700 1 -- 172.21.15.64:0/3604748408 wait complete. 2023-09-27T23:30:25.999 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:30:26.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:30:26.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.127+0000 7f317f790700 1 Processor -- start 2023-09-27T23:30:26.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.127+0000 7f317f790700 1 -- start start 2023-09-27T23:30:26.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.127+0000 7f317f790700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780b0000 0x7f31780a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31780ae220 0x7f31780ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3178157d60 con 0x7f31781572c0 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31780aebe0 con 0x7f31780ae220 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3178158900 con 0x7f31780b0000 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317d52c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f31781572c0 0x7f31781581f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:51836/0 (socket says 172.21.15.64:51836) 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317d52c700 1 -- 172.21.15.64:0/4030395398 learned_addr learned my addr 172.21.15.64:0/4030395398 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317cd2b700 1 --2- 172.21.15.64:0/4030395398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780b0000 0x7f31780a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317dd2d700 1 --2- 172.21.15.64:0/4030395398 >> v2:172.21.15.123:3300/0 conn(0x7f31780ae220 0x7f31780ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f316ffff700 1 -- 172.21.15.64:0/4030395398 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357888534 0 0) 0x7f3178157d60 con 0x7f31781572c0 2023-09-27T23:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f316ffff700 1 -- 172.21.15.64:0/4030395398 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3164003570 con 0x7f31781572c0 2023-09-27T23:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317cd2b700 1 -- 172.21.15.64:0/4030395398 >> v2:172.21.15.123:3300/0 conn(0x7f31780ae220 msgr2=0x7f31780ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317cd2b700 1 --2- 172.21.15.64:0/4030395398 >> v2:172.21.15.123:3300/0 conn(0x7f31780ae220 0x7f31780ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317cd2b700 1 -- 172.21.15.64:0/4030395398 >> v1:172.21.15.123:6789/0 conn(0x7f31781572c0 legacy=0x7f31781581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317cd2b700 1 -- 172.21.15.64:0/4030395398 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31780cd160 con 0x7f31780b0000 2023-09-27T23:30:26.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317cd2b700 1 --2- 172.21.15.64:0/4030395398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780b0000 0x7f31780a6fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f31680098e0 tx=0x7f31680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2440bc659d96ab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f316ffff700 1 -- 172.21.15.64:0/4030395398 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168011630 con 0x7f31780b0000 2023-09-27T23:30:26.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f316ffff700 1 -- 172.21.15.64:0/4030395398 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3168011790 con 0x7f31780b0000 2023-09-27T23:30:26.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f316ffff700 1 -- 172.21.15.64:0/4030395398 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168011b30 con 0x7f31780b0000 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- 172.21.15.64:0/4030395398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780b0000 msgr2=0x7f31780a6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 --2- 172.21.15.64:0/4030395398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780b0000 0x7f31780a6fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f31680098e0 tx=0x7f31680091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- 172.21.15.64:0/4030395398 shutdown_connections 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 --2- 172.21.15.64:0/4030395398 >> v2:172.21.15.123:3300/0 conn(0x7f31780ae220 0x7f31780ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 --2- 172.21.15.64:0/4030395398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780b0000 0x7f31780a6fe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- 172.21.15.64:0/4030395398 >> 172.21.15.64:0/4030395398 conn(0x7f31780af4f0 msgr2=0x7f31780af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- 172.21.15.64:0/4030395398 shutdown_connections 2023-09-27T23:30:26.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.131+0000 7f317f790700 1 -- 172.21.15.64:0/4030395398 wait complete. 2023-09-27T23:30:26.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 Processor -- start 2023-09-27T23:30:26.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 -- start start 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780ae220 0x7f3178154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31780b0000 0x7f31781536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 0x7f3178097240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3178154140 con 0x7f31781572c0 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31780a9880 con 0x7f31780b0000 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317f790700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3178153be0 con 0x7f31780ae220 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 0x7f3178097240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317d52c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780ae220 0x7f3178154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317cd2b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31780b0000 0x7f31781536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 0x7f3178097240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40084/0 (socket says 172.21.15.64:40084) 2023-09-27T23:30:26.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 -- 172.21.15.64:0/4273346269 learned_addr learned my addr 172.21.15.64:0/4273346269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:26.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 -- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780ae220 msgr2=0x7f3178154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780ae220 0x7f3178154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 -- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31780b0000 msgr2=0x7f31781536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31780b0000 0x7f31781536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.135+0000 7f317dd2d700 1 -- 172.21.15.64:0/4273346269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168009420 con 0x7f31781572c0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f317dd2d700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 0x7f3178097240 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3170009ec0 tx=0x7f317000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000cb70 con 0x7f31781572c0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317000f040 con 0x7f31781572c0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3178153e60 con 0x7f31781572c0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3170014480 con 0x7f31781572c0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31780a9510 con 0x7f31781572c0 2023-09-27T23:30:26.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31780a9a10 con 0x7f31781572c0 2023-09-27T23:30:26.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.139+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3170014830 con 0x7f31781572c0 2023-09-27T23:30:26.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.143+0000 7f316d7fa700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f315c060a70 0x7f315c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.143+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f317007d780 con 0x7f31781572c0 2023-09-27T23:30:26.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.143+0000 7f317d52c700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f315c060a70 0x7f315c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.143+0000 7f317d52c700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f315c060a70 0x7f315c062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3174007fd0 tx=0x7f3174009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.151+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f317004d9f0 con 0x7f31781572c0 2023-09-27T23:30:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.375+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3178071120 con 0x7f31781572c0 2023-09-27T23:30:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.375+0000 7f316d7fa700 1 -- 172.21.15.64:0/4273346269 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f317007d090 con 0x7f31781572c0 2023-09-27T23:30:26.378 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:30:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.382 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:30:26.383 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.384 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.386 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:30:26.387 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.388 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:30:26.389 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.391 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.392 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:30:26.393 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:30:26.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:30:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:30:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:30:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:30:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f315c060a70 msgr2=0x7f315c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f315c060a70 0x7f315c062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3174007fd0 tx=0x7f3174009040 comp rx=0 tx=0).stop 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 msgr2=0x7f3178097240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 0x7f3178097240 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3170009ec0 tx=0x7f317000b040 comp rx=0 tx=0).stop 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 shutdown_connections 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f315c060a70 0x7f315c062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31781572c0 0x7f3178097240 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31780b0000 0x7f31781536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 --2- 172.21.15.64:0/4273346269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31780ae220 0x7f3178154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 >> 172.21.15.64:0/4273346269 conn(0x7f31780af4f0 msgr2=0x7f3178154e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:26.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 shutdown_connections 2023-09-27T23:30:26.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.379+0000 7f317f790700 1 -- 172.21.15.64:0/4273346269 wait complete. 2023-09-27T23:30:26.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:26.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.499+0000 7f9ced893700 1 Processor -- start 2023-09-27T23:30:26.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.499+0000 7f9ced893700 1 -- start start 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ce80ae140 0x7f9ce80a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ce80ae510 0x7f9ce80c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce806dc20 con 0x7f9ce806d180 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce806d940 con 0x7f9ce80ae510 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce80c6e20 con 0x7f9ce80ae140 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ce80ae510 0x7f9ce80c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ce77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9ce806d180 0x7f9ce80680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45682/0 (socket says 172.21.15.64:45682) 2023-09-27T23:30:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ce80ae510 0x7f9ce80c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40088/0 (socket says 172.21.15.64:40088) 2023-09-27T23:30:26.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 -- 172.21.15.64:0/1798728706 learned_addr learned my addr 172.21.15.64:0/1798728706 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ce6ffd700 1 --2- 172.21.15.64:0/1798728706 >> v2:172.21.15.123:3300/0 conn(0x7f9ce80ae140 0x7f9ce80a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 -- 172.21.15.64:0/1798728706 >> v2:172.21.15.123:3300/0 conn(0x7f9ce80ae140 msgr2=0x7f9ce80a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 --2- 172.21.15.64:0/1798728706 >> v2:172.21.15.123:3300/0 conn(0x7f9ce80ae140 0x7f9ce80a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 -- 172.21.15.64:0/1798728706 >> v1:172.21.15.64:6789/0 conn(0x7f9ce806d180 legacy=0x7f9ce80680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 -- 172.21.15.64:0/1798728706 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80c7100 con 0x7f9ce80ae510 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9cdffff700 1 --2- 172.21.15.64:0/1798728706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce80ae510 0x7f9ce80c1d60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80cbd40 tx=0x7f9cd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bd5efe985a6f97 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ce67fc700 1 -- 172.21.15.64:0/1798728706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000b860 con 0x7f9ce80ae510 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ce67fc700 1 -- 172.21.15.64:0/1798728706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd000b9c0 con 0x7f9ce80ae510 2023-09-27T23:30:26.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ce67fc700 1 -- 172.21.15.64:0/1798728706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000bd60 con 0x7f9ce80ae510 2023-09-27T23:30:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- 172.21.15.64:0/1798728706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce80ae510 msgr2=0x7f9ce80c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 --2- 172.21.15.64:0/1798728706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce80ae510 0x7f9ce80c1d60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80cbd40 tx=0x7f9cd0009250 comp rx=0 tx=0).stop 2023-09-27T23:30:26.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- 172.21.15.64:0/1798728706 shutdown_connections 2023-09-27T23:30:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 --2- 172.21.15.64:0/1798728706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce80ae510 0x7f9ce80c1d60 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 --2- 172.21.15.64:0/1798728706 >> v2:172.21.15.123:3300/0 conn(0x7f9ce80ae140 0x7f9ce80a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- 172.21.15.64:0/1798728706 >> 172.21.15.64:0/1798728706 conn(0x7f9ce80c8850 msgr2=0x7f9ce80c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- 172.21.15.64:0/1798728706 shutdown_connections 2023-09-27T23:30:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.503+0000 7f9ced893700 1 -- 172.21.15.64:0/1798728706 wait complete. 2023-09-27T23:30:26.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 Processor -- start 2023-09-27T23:30:26.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 -- start start 2023-09-27T23:30:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 0x7f9ce814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ce80ae140 0x7f9ce80ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ce80ae510 0x7f9ce81564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8159a00 con 0x7f9ce806d180 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80ab8f0 con 0x7f9ce80ae510 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ced893700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80abbd0 con 0x7f9ce80ae140 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 0x7f9ce814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 0x7f9ce814ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40104/0 (socket says 172.21.15.64:40104) 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 -- 172.21.15.64:0/1943372345 learned_addr learned my addr 172.21.15.64:0/1943372345 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9cdffff700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ce80ae140 0x7f9ce80ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce77fe700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ce80ae510 0x7f9ce81564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 -- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ce80ae140 msgr2=0x7f9ce80ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ce80ae140 0x7f9ce80ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 -- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ce80ae510 msgr2=0x7f9ce81564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.507+0000 7f9ce6ffd700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ce80ae510 0x7f9ce81564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9ce6ffd700 1 -- 172.21.15.64:0/1943372345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00094f0 con 0x7f9ce806d180 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9ce6ffd700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 0x7f9ce814ff60 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800d890 tx=0x7f9cd8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd80125f0 con 0x7f9ce806d180 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd8012750 con 0x7f9ce806d180 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce80abeb0 con 0x7f9ce806d180 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8012af0 con 0x7f9ce806d180 2023-09-27T23:30:26.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce80ac470 con 0x7f9ce806d180 2023-09-27T23:30:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce8071120 con 0x7f9ce806d180 2023-09-27T23:30:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9cd801e460 con 0x7f9ce806d180 2023-09-27T23:30:26.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9cdeffd700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9cc8060a70 0x7f9cc8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.511+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f9cd807d350 con 0x7f9ce806d180 2023-09-27T23:30:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.515+0000 7f9cdffff700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9cc8060a70 0x7f9cc8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.515+0000 7f9cdffff700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9cc8060a70 0x7f9cc8062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80ae960 tx=0x7f9cd0017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.519+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9cd8051160 con 0x7f9ce806d180 2023-09-27T23:30:26.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.679+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ce80642a0 con 0x7f9ce806d180 2023-09-27T23:30:26.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9cdeffd700 1 -- 172.21.15.64:0/1943372345 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+13857 (secure 0 0 0) 0x7f9cd8050cf0 con 0x7f9ce806d180 2023-09-27T23:30:26.685 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:26.685 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":111,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:25.974735+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:26.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9cc8060a70 msgr2=0x7f9cc8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9cc8060a70 0x7f9cc8062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80ae960 tx=0x7f9cd0017000 comp rx=0 tx=0).stop 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 msgr2=0x7f9ce814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 0x7f9ce814ff60 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800d890 tx=0x7f9cd8008040 comp rx=0 tx=0).stop 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 shutdown_connections 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9cc8060a70 0x7f9cc8062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ce80ae510 0x7f9ce81564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ce80ae140 0x7f9ce80ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 --2- 172.21.15.64:0/1943372345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ce806d180 0x7f9ce814ff60 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 >> 172.21.15.64:0/1943372345 conn(0x7f9ce80c8850 msgr2=0x7f9ce8151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 shutdown_connections 2023-09-27T23:30:26.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.683+0000 7f9ced893700 1 -- 172.21.15.64:0/1943372345 wait complete. 2023-09-27T23:30:26.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:30:26.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:30:26.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 Processor -- start 2023-09-27T23:30:26.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 -- start start 2023-09-27T23:30:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3bec158e20 0x7f3bec0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec0ca2c0 0x7f3bec0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0cbdf0 con 0x7f3bec0cb970 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0cc0d0 con 0x7f3bec158e20 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf2c16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0cc3b0 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bebfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3bec0cb970 0x7f3bec06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45696/0 (socket says 172.21.15.64:45696) 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bebfff700 1 -- 172.21.15.64:0/2104131060 learned_addr learned my addr 172.21.15.64:0/2104131060 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf11b3700 1 --2- 172.21.15.64:0/2104131060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec0ca2c0 0x7f3bec0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3bf09b2700 1 --2- 172.21.15.64:0/2104131060 >> v2:172.21.15.123:3300/0 conn(0x7f3bec158e20 0x7f3bec0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3beb7fe700 1 -- 172.21.15.64:0/2104131060 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849704287 0 0) 0x7f3bec0cbdf0 con 0x7f3bec0cb970 2023-09-27T23:30:26.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.815+0000 7f3beb7fe700 1 -- 172.21.15.64:0/2104131060 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd4003570 con 0x7f3bec0cb970 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3beb7fe700 1 -- 172.21.15.64:0/2104131060 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907458927 0 0) 0x7f3bd4003570 con 0x7f3bec0cb970 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf11b3700 1 -- 172.21.15.64:0/2104131060 >> v2:172.21.15.123:3300/0 conn(0x7f3bec158e20 msgr2=0x7f3bec0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf11b3700 1 --2- 172.21.15.64:0/2104131060 >> v2:172.21.15.123:3300/0 conn(0x7f3bec158e20 0x7f3bec0ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf11b3700 1 -- 172.21.15.64:0/2104131060 >> v1:172.21.15.64:6789/0 conn(0x7f3bec0cb970 legacy=0x7f3bec06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf11b3700 1 -- 172.21.15.64:0/2104131060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0cc690 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf11b3700 1 --2- 172.21.15.64:0/2104131060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec0ca2c0 0x7f3bec0ceb00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009e90 tx=0x7f3be4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=785691549566039b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3beb7fe700 1 -- 172.21.15.64:0/2104131060 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400e070 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3beb7fe700 1 -- 172.21.15.64:0/2104131060 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400b870 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3beb7fe700 1 -- 172.21.15.64:0/2104131060 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc001470 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 -- 172.21.15.64:0/2104131060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec0ca2c0 msgr2=0x7f3bec0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/2104131060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec0ca2c0 0x7f3bec0ceb00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009e90 tx=0x7f3be4009780 comp rx=0 tx=0).stop 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 -- 172.21.15.64:0/2104131060 shutdown_connections 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/2104131060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec0ca2c0 0x7f3bec0ceb00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/2104131060 >> v2:172.21.15.123:3300/0 conn(0x7f3bec158e20 0x7f3bec0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 -- 172.21.15.64:0/2104131060 >> 172.21.15.64:0/2104131060 conn(0x7f3bec067bb0 msgr2=0x7f3bec0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 -- 172.21.15.64:0/2104131060 shutdown_connections 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 -- 172.21.15.64:0/2104131060 wait complete. 2023-09-27T23:30:26.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 Processor -- start 2023-09-27T23:30:26.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.819+0000 7f3bf2c16700 1 -- start start 2023-09-27T23:30:26.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 0x7f3bec169100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bec158e20 0x7f3bec14a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec14ee10 0x7f3bec0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec14f660 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec14f940 con 0x7f3bec158e20 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec14fc20 con 0x7f3bec14ee10 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 0x7f3bec169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 0x7f3bec169100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40118/0 (socket says 172.21.15.64:40118) 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 -- 172.21.15.64:0/1983855720 learned_addr learned my addr 172.21.15.64:0/1983855720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bebfff700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bec158e20 0x7f3bec14a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf11b3700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec14ee10 0x7f3bec0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 -- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec14ee10 msgr2=0x7f3bec0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec14ee10 0x7f3bec0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 -- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bec158e20 msgr2=0x7f3bec14a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bec158e20 0x7f3bec14a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 -- 172.21.15.64:0/1983855720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4009a20 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf09b2700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 0x7f3bec169100 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e930 tx=0x7f3be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0013680 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be00137e0 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0013b80 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec14ff00 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.823+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0c6720 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.827+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec071120 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.827+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3be000c3e0 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.827+0000 7f3be8ff9700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bd8060a70 0x7f3bd8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:26.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.827+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3be007cef0 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.827+0000 7f3bebfff700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bd8060a70 0x7f3bd8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:26.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.827+0000 7f3bebfff700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bd8060a70 0x7f3bd8062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0d0a00 tx=0x7f3bdc001580 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:26.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.831+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3be004d270 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.991+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bec0c7170 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3be8ff9700 1 -- 172.21.15.64:0/1983855720 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3be0050890 con 0x7f3bec0ca2c0 2023-09-27T23:30:26.997 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:30:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bd8060a70 msgr2=0x7f3bd8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bd8060a70 0x7f3bd8062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0d0a00 tx=0x7f3bdc001580 comp rx=0 tx=0).stop 2023-09-27T23:30:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 msgr2=0x7f3bec169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 0x7f3bec169100 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e930 tx=0x7f3be00091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 shutdown_connections 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3bd8060a70 0x7f3bd8062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bec14ee10 0x7f3bec0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bec158e20 0x7f3bec14a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 --2- 172.21.15.64:0/1983855720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bec0ca2c0 0x7f3bec169100 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 >> 172.21.15.64:0/1983855720 conn(0x7f3bec067bb0 msgr2=0x7f3bec155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 shutdown_connections 2023-09-27T23:30:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:26.995+0000 7f3bf2c16700 1 -- 172.21.15.64:0/1983855720 wait complete. 2023-09-27T23:30:27.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:30:27.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 Processor -- start 2023-09-27T23:30:27.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 -- start start 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f444c13d2f0 0x7f444c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f444c06b060 0x7f444c06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0cc0e0 con 0x7f444c13cf20 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c06b9e0 con 0x7f444c13d2f0 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4453b75700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c0cb4d0 con 0x7f444c06b060 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4451110700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f444c13d2f0 0x7f444c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4451110700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f444c13d2f0 0x7f444c0cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40124/0 (socket says 172.21.15.64:40124) 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4451110700 1 -- 172.21.15.64:0/3133247423 learned_addr learned my addr 172.21.15.64:0/3133247423 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:27.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.119+0000 7f4452112700 1 --2- 172.21.15.64:0/3133247423 >> v2:172.21.15.123:3300/0 conn(0x7f444c06b060 0x7f444c06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4451110700 1 -- 172.21.15.64:0/3133247423 >> v2:172.21.15.123:3300/0 conn(0x7f444c06b060 msgr2=0x7f444c06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4451110700 1 --2- 172.21.15.64:0/3133247423 >> v2:172.21.15.123:3300/0 conn(0x7f444c06b060 0x7f444c06b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4451110700 1 -- 172.21.15.64:0/3133247423 >> v1:172.21.15.123:6789/0 conn(0x7f444c13cf20 legacy=0x7f444c0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4451110700 1 -- 172.21.15.64:0/3133247423 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0cb7b0 con 0x7f444c13d2f0 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4451110700 1 --2- 172.21.15.64:0/3133247423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13d2f0 0x7f444c0cadf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f444c0cc050 tx=0x7f443c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98404e4b6aa9ac0a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f445090f700 1 -- 172.21.15.64:0/3133247423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c009e70 con 0x7f444c13d2f0 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f445090f700 1 -- 172.21.15.64:0/3133247423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c00b860 con 0x7f444c13d2f0 2023-09-27T23:30:27.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f445090f700 1 -- 172.21.15.64:0/3133247423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00bc00 con 0x7f444c13d2f0 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 -- 172.21.15.64:0/3133247423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13d2f0 msgr2=0x7f444c0cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 --2- 172.21.15.64:0/3133247423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13d2f0 0x7f444c0cadf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f444c0cc050 tx=0x7f443c009250 comp rx=0 tx=0).stop 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 -- 172.21.15.64:0/3133247423 shutdown_connections 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 --2- 172.21.15.64:0/3133247423 >> v2:172.21.15.123:3300/0 conn(0x7f444c06b060 0x7f444c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 --2- 172.21.15.64:0/3133247423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13d2f0 0x7f444c0cadf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 -- 172.21.15.64:0/3133247423 >> 172.21.15.64:0/3133247423 conn(0x7f444c0cc580 msgr2=0x7f444c0cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 -- 172.21.15.64:0/3133247423 shutdown_connections 2023-09-27T23:30:27.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.123+0000 7f4453b75700 1 -- 172.21.15.64:0/3133247423 wait complete. 2023-09-27T23:30:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 Processor -- start 2023-09-27T23:30:27.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 -- start start 2023-09-27T23:30:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f444c06b060 0x7f444c0c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 0x7f444c0aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f444c13d2f0 0x7f444c155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0aae50 con 0x7f444c13cf20 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0a85a0 con 0x7f444c06b060 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0a8880 con 0x7f444c13d2f0 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 0x7f444c0aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 0x7f444c0aa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40140/0 (socket says 172.21.15.64:40140) 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 -- 172.21.15.64:0/1085356924 learned_addr learned my addr 172.21.15.64:0/1085356924 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451911700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f444c06b060 0x7f444c0c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4452112700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f444c13d2f0 0x7f444c155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 -- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f444c13d2f0 msgr2=0x7f444c155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f444c13d2f0 0x7f444c155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 -- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f444c06b060 msgr2=0x7f444c0c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f444c06b060 0x7f444c0c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 -- 172.21.15.64:0/1085356924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c0094f0 con 0x7f444c13cf20 2023-09-27T23:30:27.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4451110700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 0x7f444c0aa910 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f444c06e9d0 tx=0x7f443c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00bbe0 con 0x7f444c13cf20 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c011420 con 0x7f444c13cf20 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c0a8b60 con 0x7f444c13cf20 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0117a0 con 0x7f444c13cf20 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.127+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c155ac0 con 0x7f444c13cf20 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.131+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c155c50 con 0x7f444c13cf20 2023-09-27T23:30:27.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.131+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f443c011b90 con 0x7f444c13cf20 2023-09-27T23:30:27.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.131+0000 7f4441ffb700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4430060a70 0x7f4430062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.131+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f443c07e670 con 0x7f444c13cf20 2023-09-27T23:30:27.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.131+0000 7f4451911700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4430060a70 0x7f4430062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.131+0000 7f4451911700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4430060a70 0x7f4430062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4448003a80 tx=0x7f4448007ec0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.139+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f443c04e950 con 0x7f444c13cf20 2023-09-27T23:30:27.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f444c0a9060 con 0x7f444c13cf20 2023-09-27T23:30:27.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4441ffb700 1 -- 172.21.15.64:0/1085356924 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f443c051f70 con 0x7f444c13cf20 2023-09-27T23:30:27.306 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:30:27.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4430060a70 msgr2=0x7f4430062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4430060a70 0x7f4430062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4448003a80 tx=0x7f4448007ec0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 msgr2=0x7f444c0aa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 0x7f444c0aa910 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f444c06e9d0 tx=0x7f443c017000 comp rx=0 tx=0).stop 2023-09-27T23:30:27.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 shutdown_connections 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4430060a70 0x7f4430062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f444c13d2f0 0x7f444c155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f444c13cf20 0x7f444c0aa910 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.303+0000 7f4453b75700 1 --2- 172.21.15.64:0/1085356924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f444c06b060 0x7f444c0c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.307+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 >> 172.21.15.64:0/1085356924 conn(0x7f444c0cc580 msgr2=0x7f444c0abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.307+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 shutdown_connections 2023-09-27T23:30:27.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.307+0000 7f4453b75700 1 -- 172.21.15.64:0/1085356924 wait complete. 2023-09-27T23:30:27.320 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:30:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 Processor -- start 2023-09-27T23:30:27.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 -- start start 2023-09-27T23:30:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cd80cb920 0x7f1cd815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cd815cee0 0x7f1cd806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd8068dc0 con 0x7f1cd8158d40 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80cc910 con 0x7f1cd815cee0 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cde447700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80682a0 con 0x7f1cd80cb920 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cd815cee0 0x7f1cd806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cd815cee0 0x7f1cd806c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40152/0 (socket says 172.21.15.64:40152) 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 -- 172.21.15.64:0/487402005 learned_addr learned my addr 172.21.15.64:0/487402005 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cdcc44700 1 --2- 172.21.15.64:0/487402005 >> v2:172.21.15.123:3300/0 conn(0x7f1cd80cb920 0x7f1cd815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 -- 172.21.15.64:0/487402005 >> v2:172.21.15.123:3300/0 conn(0x7f1cd80cb920 msgr2=0x7f1cd815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:27.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 --2- 172.21.15.64:0/487402005 >> v2:172.21.15.123:3300/0 conn(0x7f1cd80cb920 0x7f1cd815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 -- 172.21.15.64:0/487402005 >> v1:172.21.15.123:6789/0 conn(0x7f1cd8158d40 legacy=0x7f1cd80cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 -- 172.21.15.64:0/487402005 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8068490 con 0x7f1cd815cee0 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1cddc46700 1 --2- 172.21.15.64:0/487402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd815cee0 0x7f1cd806c7c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80b3bc0 tx=0x7f1cd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579127759727e8e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1ccbfff700 1 -- 172.21.15.64:0/487402005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000e070 con 0x7f1cd815cee0 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1ccbfff700 1 -- 172.21.15.64:0/487402005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000ba50 con 0x7f1cd815cee0 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.427+0000 7f1ccbfff700 1 -- 172.21.15.64:0/487402005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00092e0 con 0x7f1cd815cee0 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 -- 172.21.15.64:0/487402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd815cee0 msgr2=0x7f1cd806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 --2- 172.21.15.64:0/487402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd815cee0 0x7f1cd806c7c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80b3bc0 tx=0x7f1cd0009840 comp rx=0 tx=0).stop 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 -- 172.21.15.64:0/487402005 shutdown_connections 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 --2- 172.21.15.64:0/487402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd815cee0 0x7f1cd806c7c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 --2- 172.21.15.64:0/487402005 >> v2:172.21.15.123:3300/0 conn(0x7f1cd80cb920 0x7f1cd815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 -- 172.21.15.64:0/487402005 >> 172.21.15.64:0/487402005 conn(0x7f1cd80cdbc0 msgr2=0x7f1cd80cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 -- 172.21.15.64:0/487402005 shutdown_connections 2023-09-27T23:30:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 -- 172.21.15.64:0/487402005 wait complete. 2023-09-27T23:30:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 Processor -- start 2023-09-27T23:30:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.431+0000 7f1cde447700 1 -- start start 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 0x7f1cd815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8158d40 0x7f1cd81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd815cee0 0x7f1cd816ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80c6650 con 0x7f1cd80cb920 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8154860 con 0x7f1cd8158d40 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8159870 con 0x7f1cd815cee0 2023-09-27T23:30:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 0x7f1cd815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 0x7f1cd815bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40162/0 (socket says 172.21.15.64:40162) 2023-09-27T23:30:27.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 -- 172.21.15.64:0/387051837 learned_addr learned my addr 172.21.15.64:0/387051837 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:27.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdcc44700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8158d40 0x7f1cd81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cddc46700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd815cee0 0x7f1cd816ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 -- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd815cee0 msgr2=0x7f1cd816ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd815cee0 0x7f1cd816ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 -- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8158d40 msgr2=0x7f1cd81572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8158d40 0x7f1cd81572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 -- 172.21.15.64:0/387051837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009ae0 con 0x7f1cd80cb920 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cdd445700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 0x7f1cd815bdf0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c7c0 tx=0x7f1cd4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4011590 con 0x7f1cd80cb920 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd4011710 con 0x7f1cd80cb920 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd80c6860 con 0x7f1cd80cb920 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4011ad0 con 0x7f1cd80cb920 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd80c6bf0 con 0x7f1cd80cb920 2023-09-27T23:30:27.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.435+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd80c7060 con 0x7f1cd80cb920 2023-09-27T23:30:27.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.439+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1cd401b3b0 con 0x7f1cd80cb920 2023-09-27T23:30:27.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.439+0000 7f1cc97fa700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cbc060a70 0x7f1cbc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.439+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1cd407c140 con 0x7f1cd80cb920 2023-09-27T23:30:27.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.439+0000 7f1cdcc44700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cbc060a70 0x7f1cbc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.439+0000 7f1cdcc44700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cbc060a70 0x7f1cbc062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc006fd0 tx=0x7f1ccc008040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.447+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1cd80c7060 con 0x7f1cd80cb920 2023-09-27T23:30:27.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1cd80c7060 con 0x7f1cd80cb920 2023-09-27T23:30:27.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cc97fa700 1 -- 172.21.15.64:0/387051837 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v111) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1cd80c7060 con 0x7f1cd80cb920 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:30:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:30:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:27.615 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:30:27.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cbc060a70 msgr2=0x7f1cbc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cbc060a70 0x7f1cbc062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc006fd0 tx=0x7f1ccc008040 comp rx=0 tx=0).stop 2023-09-27T23:30:27.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 msgr2=0x7f1cd815bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 0x7f1cd815bdf0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c7c0 tx=0x7f1cd4007f40 comp rx=0 tx=0).stop 2023-09-27T23:30:27.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 shutdown_connections 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.611+0000 7f1cde447700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cbc060a70 0x7f1cbc062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.615+0000 7f1cde447700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd815cee0 0x7f1cd816ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.615+0000 7f1cde447700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8158d40 0x7f1cd81572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.615+0000 7f1cde447700 1 --2- 172.21.15.64:0/387051837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cd80cb920 0x7f1cd815bdf0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.615+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 >> 172.21.15.64:0/387051837 conn(0x7f1cd80cdbc0 msgr2=0x7f1cd80ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.615+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 shutdown_connections 2023-09-27T23:30:27.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.615+0000 7f1cde447700 1 -- 172.21.15.64:0/387051837 wait complete. 2023-09-27T23:30:27.627 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:30:27.628 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:30:27.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.735+0000 7f1900edc700 1 Processor -- start 2023-09-27T23:30:27.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.735+0000 7f1900edc700 1 -- start start 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f1900edc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18fc158da0 0x7f18fc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f1900edc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f1900edc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc15d720 con 0x7f18fc0caca0 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f1900edc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18fc0ca290 con 0x7f18fc158da0 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f1900edc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0ca4a0 con 0x7f18fc06df10 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fbfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f18fc0caca0 0x7f18fc06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45704/0 (socket says 172.21.15.64:45704) 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fbfff700 1 -- 172.21.15.64:0/3243734606 learned_addr learned my addr 172.21.15.64:0/3243734606 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fb7fe700 1 --2- 172.21.15.64:0/3243734606 >> v2:172.21.15.123:3300/0 conn(0x7f18fc158da0 0x7f18fc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18f2ffd700 1 --2- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18faffd700 1 -- 172.21.15.64:0/3243734606 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408834393 0 0) 0x7f18fc15d720 con 0x7f18fc0caca0 2023-09-27T23:30:27.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18faffd700 1 -- 172.21.15.64:0/3243734606 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003570 con 0x7f18fc0caca0 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fb7fe700 1 -- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 msgr2=0x7f18fc06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fb7fe700 1 --2- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fb7fe700 1 -- 172.21.15.64:0/3243734606 >> v1:172.21.15.64:6789/0 conn(0x7f18fc0caca0 legacy=0x7f18fc06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fb7fe700 1 -- 172.21.15.64:0/3243734606 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0ca780 con 0x7f18fc158da0 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18fb7fe700 1 --2- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc158da0 0x7f18fc06e670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098e0 tx=0x7f18ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5727500053c54 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18faffd700 1 -- 172.21.15.64:0/3243734606 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011630 con 0x7f18fc158da0 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18faffd700 1 -- 172.21.15.64:0/3243734606 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec011790 con 0x7f18fc158da0 2023-09-27T23:30:27.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.739+0000 7f18faffd700 1 -- 172.21.15.64:0/3243734606 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011b30 con 0x7f18fc158da0 2023-09-27T23:30:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 -- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc158da0 msgr2=0x7f18fc06e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 --2- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc158da0 0x7f18fc06e670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098e0 tx=0x7f18ec0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 -- 172.21.15.64:0/3243734606 shutdown_connections 2023-09-27T23:30:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 --2- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc06ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 --2- 172.21.15.64:0/3243734606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc158da0 0x7f18fc06e670 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 -- 172.21.15.64:0/3243734606 >> 172.21.15.64:0/3243734606 conn(0x7f18fc0cc760 msgr2=0x7f18fc0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 -- 172.21.15.64:0/3243734606 shutdown_connections 2023-09-27T23:30:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 -- 172.21.15.64:0/3243734606 wait complete. 2023-09-27T23:30:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 Processor -- start 2023-09-27T23:30:27.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.743+0000 7f1900edc700 1 -- start start 2023-09-27T23:30:27.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc0caca0 0x7f18fc0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 0x7f18fc0c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1476b0 con 0x7f18fc158da0 2023-09-27T23:30:27.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0c3760 con 0x7f18fc0caca0 2023-09-27T23:30:27.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1575f0 con 0x7f18fc06df10 2023-09-27T23:30:27.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 0x7f18fc0c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 0x7f18fc0c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40174/0 (socket says 172.21.15.64:40174) 2023-09-27T23:30:27.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 -- 172.21.15.64:0/31235787 learned_addr learned my addr 172.21.15.64:0/31235787 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:27.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18f2ffd700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc0caca0 0x7f18fc0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fb7fe700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 -- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 msgr2=0x7f18fc156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 -- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc0caca0 msgr2=0x7f18fc0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:27.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc0caca0 0x7f18fc0c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 -- 172.21.15.64:0/31235787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec009420 con 0x7f18fc158da0 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18fbfff700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 0x7f18fc0c8da0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cca0 tx=0x7f18f4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400a8d0 con 0x7f18fc158da0 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f400aa30 con 0x7f18fc158da0 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc1578d0 con 0x7f18fc158da0 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400adb0 con 0x7f18fc158da0 2023-09-27T23:30:27.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.747+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc157e90 con 0x7f18fc158da0 2023-09-27T23:30:27.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.751+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc0c38f0 con 0x7f18fc158da0 2023-09-27T23:30:27.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.751+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f18f4007300 con 0x7f18fc158da0 2023-09-27T23:30:27.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.751+0000 7f18f3fff700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18dc060a70 0x7f18dc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:27.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.751+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f18f407d150 con 0x7f18fc158da0 2023-09-27T23:30:27.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.751+0000 7f18fb7fe700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18dc060a70 0x7f18dc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:27.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.751+0000 7f18fb7fe700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18dc060a70 0x7f18dc062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0091e0 tx=0x7f18ec017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:27.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.759+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f18f404d4c0 con 0x7f18fc158da0 2023-09-27T23:30:27.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:27.923+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f18fc04be40 con 0x7f18fc158da0 2023-09-27T23:30:28.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.003+0000 7f18f3fff700 1 -- 172.21.15.64:0/31235787 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v112) v1 ==== 138+0+0 (secure 0 0 0) 0x7f18f4050ae0 con 0x7f18fc158da0 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18dc060a70 msgr2=0x7f18dc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18dc060a70 0x7f18dc062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0091e0 tx=0x7f18ec017000 comp rx=0 tx=0).stop 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 msgr2=0x7f18fc0c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 0x7f18fc0c8da0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cca0 tx=0x7f18f4001ae0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 shutdown_connections 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18dc060a70 0x7f18dc062f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18fc158da0 0x7f18fc0c8da0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18fc0caca0 0x7f18fc0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 --2- 172.21.15.64:0/31235787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18fc06df10 0x7f18fc156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.011+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 >> 172.21.15.64:0/31235787 conn(0x7f18fc0cc760 msgr2=0x7f18fc0af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:28.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.015+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 shutdown_connections 2023-09-27T23:30:28.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.015+0000 7f1900edc700 1 -- 172.21.15.64:0/31235787 wait complete. 2023-09-27T23:30:28.019 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:30:28.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:30:28.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.135+0000 7f700e285700 1 Processor -- start 2023-09-27T23:30:28.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.135+0000 7f700e285700 1 -- start start 2023-09-27T23:30:28.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70080cecd0 0x7f700806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080cbdf0 con 0x7f70080cb970 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080cc0d0 con 0x7f7008158e20 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70080cc3b0 con 0x7f70080cecd0 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70080cecd0 0x7f700806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70080cecd0 0x7f700806b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40190/0 (socket says 172.21.15.64:40190) 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 -- 172.21.15.64:0/1264951330 learned_addr learned my addr 172.21.15.64:0/1264951330 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f70077fe700 1 --1- 172.21.15.64:0/1264951330 >> v1:172.21.15.64:6789/0 conn(0x7f70080cb970 0x7f700806acc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45720/0 (socket says 172.21.15.64:45720) 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 -- 172.21.15.64:0/1264951330 >> v1:172.21.15.123:6789/0 conn(0x7f7008158e20 legacy=0x7f70080ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 -- 172.21.15.64:0/1264951330 >> v1:172.21.15.64:6789/0 conn(0x7f70080cb970 legacy=0x7f700806acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:28.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 -- 172.21.15.64:0/1264951330 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080cc690 con 0x7f70080cecd0 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700c822700 1 --2- 172.21.15.64:0/1264951330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700806b3d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7000009f50 tx=0x7f7000009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d1f4d11efb12d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f7006ffd700 1 -- 172.21.15.64:0/1264951330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000e070 con 0x7f70080cecd0 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f7006ffd700 1 -- 172.21.15.64:0/1264951330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700000ba70 con 0x7f70080cecd0 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f7006ffd700 1 -- 172.21.15.64:0/1264951330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000092e0 con 0x7f70080cecd0 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- 172.21.15.64:0/1264951330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 msgr2=0x7f700806b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 --2- 172.21.15.64:0/1264951330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700806b3d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7000009f50 tx=0x7f7000009840 comp rx=0 tx=0).stop 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- 172.21.15.64:0/1264951330 shutdown_connections 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 --2- 172.21.15.64:0/1264951330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700806b3d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- 172.21.15.64:0/1264951330 >> 172.21.15.64:0/1264951330 conn(0x7f7008067bb0 msgr2=0x7f70080d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:28.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- 172.21.15.64:0/1264951330 shutdown_connections 2023-09-27T23:30:28.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.139+0000 7f700e285700 1 -- 172.21.15.64:0/1264951330 wait complete. 2023-09-27T23:30:28.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 Processor -- start 2023-09-27T23:30:28.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 -- start start 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7008158e20 0x7f70080b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70080b5a20 0x7f700816d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080b5ec0 con 0x7f70080cecd0 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700814eec0 con 0x7f70080b5a20 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700e285700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700814f0f0 con 0x7f7008158e20 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f7007fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:28.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f7007fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700814d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40202/0 (socket says 172.21.15.64:40202) 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f7007fff700 1 -- 172.21.15.64:0/950941442 learned_addr learned my addr 172.21.15.64:0/950941442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f700c822700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70080b5a20 0x7f700816d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.143+0000 7f70077fe700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7008158e20 0x7f70080b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f7007fff700 1 -- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7008158e20 msgr2=0x7f70080b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f7007fff700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7008158e20 0x7f70080b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f7007fff700 1 -- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70080b5a20 msgr2=0x7f700816d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f7007fff700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70080b5a20 0x7f700816d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f7007fff700 1 -- 172.21.15.64:0/950941442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000009ae0 con 0x7f70080cecd0 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f700c822700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70080b5a20 0x7f700816d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f7007fff700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700814d3a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800c5c0 tx=0x7f6ff800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011590 con 0x7f70080cecd0 2023-09-27T23:30:28.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011710 con 0x7f70080cecd0 2023-09-27T23:30:28.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700814f3d0 con 0x7f70080cecd0 2023-09-27T23:30:28.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011ad0 con 0x7f70080cecd0 2023-09-27T23:30:28.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700814f990 con 0x7f70080cecd0 2023-09-27T23:30:28.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.147+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008071120 con 0x7f70080cecd0 2023-09-27T23:30:28.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.151+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6ff8012420 con 0x7f70080cecd0 2023-09-27T23:30:28.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.151+0000 7f6ffffff700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6fe8060a70 0x7f6fe8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:28.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.151+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f6ff807c400 con 0x7f70080cecd0 2023-09-27T23:30:28.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.151+0000 7f70077fe700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6fe8060a70 0x7f6fe8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:28.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.151+0000 7f70077fe700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6fe8060a70 0x7f6fe8062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0007fd0 tx=0x7f6ff0007d70 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:28.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.159+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6ff8049bb0 con 0x7f70080cecd0 2023-09-27T23:30:28.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:28.315+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f700806a7a0 con 0x7f70080cecd0 2023-09-27T23:30:29.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.019+0000 7f6ffffff700 1 -- 172.21.15.64:0/950941442 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v113) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ff80170d0 con 0x7f70080cecd0 2023-09-27T23:30:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.035+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6fe8060a70 msgr2=0x7f6fe8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.035+0000 7f700e285700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6fe8060a70 0x7f6fe8062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0007fd0 tx=0x7f6ff0007d70 comp rx=0 tx=0).stop 2023-09-27T23:30:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.035+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 msgr2=0x7f700814d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.035+0000 7f700e285700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700814d3a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800c5c0 tx=0x7f6ff800c980 comp rx=0 tx=0).stop 2023-09-27T23:30:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 shutdown_connections 2023-09-27T23:30:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6fe8060a70 0x7f6fe8062f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70080b5a20 0x7f700816d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7008158e20 0x7f70080b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 --2- 172.21.15.64:0/950941442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70080cecd0 0x7f700814d3a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 >> 172.21.15.64:0/950941442 conn(0x7f7008067bb0 msgr2=0x7f700814cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 shutdown_connections 2023-09-27T23:30:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.039+0000 7f700e285700 1 -- 172.21.15.64:0/950941442 wait complete. 2023-09-27T23:30:29.045 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:30:29.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:30:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.163+0000 7f6bfd9f5700 1 Processor -- start 2023-09-27T23:30:29.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.163+0000 7f6bfd9f5700 1 -- start start 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6bf8150200 0x7f6bf80c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf80aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bf80ac720 0x7f6bf80acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf80af530 con 0x7f6bf8150200 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf80ad0c0 con 0x7f6bf80ac720 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80af250 con 0x7f6bf80ae770 2023-09-27T23:30:29.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6bf8150200 0x7f6bf80c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6bf8150200 0x7f6bf80c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40214/0 (socket says 172.21.15.64:40214) 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 -- 172.21.15.64:0/3903899302 learned_addr learned my addr 172.21.15.64:0/3903899302 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6beffff700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf80aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd1f4700 1 --2- 172.21.15.64:0/3903899302 >> v2:172.21.15.123:3300/0 conn(0x7f6bf80ac720 0x7f6bf80acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 -- 172.21.15.64:0/3903899302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 msgr2=0x7f6bf80aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf80aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 -- 172.21.15.64:0/3903899302 >> v2:172.21.15.123:3300/0 conn(0x7f6bf80ac720 msgr2=0x7f6bf80acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 --2- 172.21.15.64:0/3903899302 >> v2:172.21.15.123:3300/0 conn(0x7f6bf80ac720 0x7f6bf80acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 -- 172.21.15.64:0/3903899302 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80ca8a0 con 0x7f6bf8150200 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6beffff700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf80aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:29.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfc9f3700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf80c7170 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6bf806b490 tx=0x7f6bf400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495cb156a38bd85a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bef7fe700 1 -- 172.21.15.64:0/3903899302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400dc50 con 0x7f6bf8150200 2023-09-27T23:30:29.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bef7fe700 1 -- 172.21.15.64:0/3903899302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf400ddb0 con 0x7f6bf8150200 2023-09-27T23:30:29.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bef7fe700 1 -- 172.21.15.64:0/3903899302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40196d0 con 0x7f6bf8150200 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3903899302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 msgr2=0x7f6bf80c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf80c7170 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6bf806b490 tx=0x7f6bf400b4f0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3903899302 shutdown_connections 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3903899302 >> v2:172.21.15.123:3300/0 conn(0x7f6bf80ac720 0x7f6bf80acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf80aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3903899302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf80c7170 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3903899302 >> 172.21.15.64:0/3903899302 conn(0x7f6bf80ccb30 msgr2=0x7f6bf80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3903899302 shutdown_connections 2023-09-27T23:30:29.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.167+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3903899302 wait complete. 2023-09-27T23:30:29.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 Processor -- start 2023-09-27T23:30:29.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 -- start start 2023-09-27T23:30:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bf80ac720 0x7f6bf813ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf8157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf8168080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf81598a0 con 0x7f6bf8150200 2023-09-27T23:30:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf8159b80 con 0x7f6bf80ac720 2023-09-27T23:30:29.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd9f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf8159e60 con 0x7f6bf80ae770 2023-09-27T23:30:29.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf8168080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6beffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf8157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfc9f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bf80ac720 0x7f6bf813ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf8168080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40220/0 (socket says 172.21.15.64:40220) 2023-09-27T23:30:29.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 -- 172.21.15.64:0/3993960423 learned_addr learned my addr 172.21.15.64:0/3993960423 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 -- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 msgr2=0x7f6bf8157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf8157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 -- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bf80ac720 msgr2=0x7f6bf813ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bf80ac720 0x7f6bf813ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.171+0000 7f6bfd1f4700 1 -- 172.21.15.64:0/3993960423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf400b790 con 0x7f6bf8150200 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6bfd1f4700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf8168080 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0007f20 tx=0x7f6bf0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000eb90 con 0x7f6bf8150200 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf000ed10 con 0x7f6bf8150200 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf815a140 con 0x7f6bf8150200 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0014670 con 0x7f6bf8150200 2023-09-27T23:30:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf81575a0 con 0x7f6bf8150200 2023-09-27T23:30:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf815a360 con 0x7f6bf8150200 2023-09-27T23:30:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.175+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6bf00149e0 con 0x7f6bf8150200 2023-09-27T23:30:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.179+0000 7f6becff9700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6be4060a70 0x7f6be4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.179+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f6bf007e610 con 0x7f6bf8150200 2023-09-27T23:30:29.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.179+0000 7f6bfc9f3700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6be4060a70 0x7f6be4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.179+0000 7f6bfc9f3700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6be4060a70 0x7f6be4062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6bf806b490 tx=0x7f6bf401b000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.183+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6bf004e680 con 0x7f6bf8150200 2023-09-27T23:30:29.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.387+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6bf804be40 con 0x7f6bf8150200 2023-09-27T23:30:29.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.411+0000 7f6becff9700 1 -- 172.21.15.64:0/3993960423 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v37) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6bf0051ca0 con 0x7f6bf8150200 2023-09-27T23:30:29.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6be4060a70 msgr2=0x7f6be4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6be4060a70 0x7f6be4062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6bf806b490 tx=0x7f6bf401b000 comp rx=0 tx=0).stop 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 msgr2=0x7f6bf8168080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf8168080 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0007f20 tx=0x7f6bf0009860 comp rx=0 tx=0).stop 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 shutdown_connections 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6be4060a70 0x7f6be4062f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf8150200 0x7f6bf8168080 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bf80ae770 0x7f6bf8157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 --2- 172.21.15.64:0/3993960423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bf80ac720 0x7f6bf813ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 >> 172.21.15.64:0/3993960423 conn(0x7f6bf80ccb30 msgr2=0x7f6bf80c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 shutdown_connections 2023-09-27T23:30:29.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.427+0000 7f6bfd9f5700 1 -- 172.21.15.64:0/3993960423 wait complete. 2023-09-27T23:30:29.432 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 14 and data pool 15 2023-09-27T23:30:29.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:29.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.539+0000 7fa6a309e700 1 Processor -- start 2023-09-27T23:30:29.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.539+0000 7fa6a309e700 1 -- start start 2023-09-27T23:30:29.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c0b1410 0x7fa69c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69c06c7c0 con 0x7fa69c06a9d0 2023-09-27T23:30:29.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69c06cd80 con 0x7fa69c06a580 2023-09-27T23:30:29.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c15b680 con 0x7fa69c0b1410 2023-09-27T23:30:29.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a289d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa69c06a9d0 0x7fa69c15af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45726/0 (socket says 172.21.15.64:45726) 2023-09-27T23:30:29.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a289d700 1 -- 172.21.15.64:0/2777353034 learned_addr learned my addr 172.21.15.64:0/2777353034 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:29.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a209c700 1 --2- 172.21.15.64:0/2777353034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c0b1410 0x7fa69c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101457347 0 0) 0x7fa69c06c7c0 con 0x7fa69c06a9d0 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa688003570 con 0x7fa69c06a9d0 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460989359 0 0) 0x7fa69c06cd80 con 0x7fa69c06a580 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa69c06c7c0 con 0x7fa69c06a580 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 914207479 0 0) 0x7fa688003570 con 0x7fa69c06a9d0 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a209c700 1 -- 172.21.15.64:0/2777353034 >> v1:172.21.15.123:6789/0 conn(0x7fa69c06a580 legacy=0x7fa69c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a209c700 1 -- 172.21.15.64:0/2777353034 >> v1:172.21.15.64:6789/0 conn(0x7fa69c06a9d0 legacy=0x7fa69c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a209c700 1 -- 172.21.15.64:0/2777353034 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c06b8f0 con 0x7fa69c0b1410 2023-09-27T23:30:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a209c700 1 --2- 172.21.15.64:0/2777353034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c0b1410 0x7fa69c0ab440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa698009910 tx=0x7fa6980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2afb70191736f69 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940017f0 con 0x7fa69c0b1410 2023-09-27T23:30:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa698011420 con 0x7fa69c0b1410 2023-09-27T23:30:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a109a700 1 -- 172.21.15.64:0/2777353034 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6980117a0 con 0x7fa69c0b1410 2023-09-27T23:30:29.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- 172.21.15.64:0/2777353034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c0b1410 msgr2=0x7fa69c0ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 --2- 172.21.15.64:0/2777353034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c0b1410 0x7fa69c0ab440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa698009910 tx=0x7fa6980091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- 172.21.15.64:0/2777353034 shutdown_connections 2023-09-27T23:30:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 --2- 172.21.15.64:0/2777353034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c0b1410 0x7fa69c0ab440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- 172.21.15.64:0/2777353034 >> 172.21.15.64:0/2777353034 conn(0x7fa69c067bb0 msgr2=0x7fa69c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- 172.21.15.64:0/2777353034 shutdown_connections 2023-09-27T23:30:29.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.543+0000 7fa6a309e700 1 -- 172.21.15.64:0/2777353034 wait complete. 2023-09-27T23:30:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 Processor -- start 2023-09-27T23:30:29.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 -- start start 2023-09-27T23:30:29.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c06a580 0x7fa69c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69c06a9d0 0x7fa69c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 0x7fa69c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c16a510 con 0x7fa69c15a390 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c169900 con 0x7fa69c06a9d0 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a309e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c169be0 con 0x7fa69c06a580 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a289d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 0x7fa69c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a289d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 0x7fa69c0c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40232/0 (socket says 172.21.15.64:40232) 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.547+0000 7fa6a289d700 1 -- 172.21.15.64:0/3802856915 learned_addr learned my addr 172.21.15.64:0/3802856915 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:29.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a209c700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c06a580 0x7fa69c1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a189b700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69c06a9d0 0x7fa69c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a289d700 1 -- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c06a580 msgr2=0x7fa69c1537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a289d700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c06a580 0x7fa69c1537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a289d700 1 -- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69c06a9d0 msgr2=0x7fa69c159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a289d700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69c06a9d0 0x7fa69c159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a289d700 1 -- 172.21.15.64:0/3802856915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa698009420 con 0x7fa69c15a390 2023-09-27T23:30:29.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a289d700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 0x7fa69c0c9460 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa69400cca0 tx=0x7fa6940018e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694004dc0 con 0x7fa69c15a390 2023-09-27T23:30:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69400a6a0 con 0x7fa69c15a390 2023-09-27T23:30:29.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c169ec0 con 0x7fa69c15a390 2023-09-27T23:30:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69400aa60 con 0x7fa69c15a390 2023-09-27T23:30:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69c0aab60 con 0x7fa69c15a390 2023-09-27T23:30:29.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.551+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69c06c300 con 0x7fa69c15a390 2023-09-27T23:30:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.555+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa694007150 con 0x7fa69c15a390 2023-09-27T23:30:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.555+0000 7fa6927fc700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060a70 0x7fa680062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.555+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa69407ccc0 con 0x7fa69c15a390 2023-09-27T23:30:29.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.555+0000 7fa6a209c700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060a70 0x7fa680062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.555+0000 7fa6a209c700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060a70 0x7fa680062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa69c06c540 tx=0x7fa698016000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.559+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa69404cd50 con 0x7fa69c15a390 2023-09-27T23:30:29.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.719+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa69c156b60 con 0x7fa69c15a390 2023-09-27T23:30:29.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.723+0000 7fa6927fc700 1 -- 172.21.15.64:0/3802856915 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17098 (secure 0 0 0) 0x7fa69c156b60 con 0x7fa69c15a390 2023-09-27T23:30:29.726 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:29.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":114,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:29.390515+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:30:27.964715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-27T23:30:28.361565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060a70 msgr2=0x7fa680062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060a70 0x7fa680062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa69c06c540 tx=0x7fa698016000 comp rx=0 tx=0).stop 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 msgr2=0x7fa69c0c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 0x7fa69c0c9460 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa69400cca0 tx=0x7fa6940018e0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 shutdown_connections 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa680060a70 0x7fa680062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa69c15a390 0x7fa69c0c9460 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa69c06a9d0 0x7fa69c159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 --2- 172.21.15.64:0/3802856915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa69c06a580 0x7fa69c1537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 >> 172.21.15.64:0/3802856915 conn(0x7fa69c067bb0 msgr2=0x7fa69c154030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:29.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 shutdown_connections 2023-09-27T23:30:29.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.727+0000 7fa6a309e700 1 -- 172.21.15.64:0/3802856915 wait complete. 2023-09-27T23:30:29.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 Processor -- start 2023-09-27T23:30:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 -- start start 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62c406de20 0x7f62c40c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406c7c0 0x7f62c40c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c406d990 con 0x7f62c4155960 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c406cd80 con 0x7f62c406de20 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c9bd1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c406d060 con 0x7f62c406c7c0 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62c406de20 0x7f62c40c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62c406de20 0x7f62c40c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40236/0 (socket says 172.21.15.64:40236) 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c3fff700 1 -- 172.21.15.64:0/357471789 learned_addr learned my addr 172.21.15.64:0/357471789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:29.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.839+0000 7f62c93d0700 1 --2- 172.21.15.64:0/357471789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406c7c0 0x7f62c40c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c3fff700 1 -- 172.21.15.64:0/357471789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406c7c0 msgr2=0x7f62c40c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c3fff700 1 --2- 172.21.15.64:0/357471789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406c7c0 0x7f62c40c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c3fff700 1 -- 172.21.15.64:0/357471789 >> v1:172.21.15.123:6789/0 conn(0x7f62c4155960 legacy=0x7f62c415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c3fff700 1 -- 172.21.15.64:0/357471789 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c406d340 con 0x7f62c406de20 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c3fff700 1 --2- 172.21.15.64:0/357471789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406de20 0x7f62c40c77f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009f20 tx=0x7f62b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9fffcef0273ee0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c37fe700 1 -- 172.21.15.64:0/357471789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400e070 con 0x7f62c406de20 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c37fe700 1 -- 172.21.15.64:0/357471789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b400b9d0 con 0x7f62c406de20 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c37fe700 1 -- 172.21.15.64:0/357471789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40092e0 con 0x7f62c406de20 2023-09-27T23:30:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 -- 172.21.15.64:0/357471789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406de20 msgr2=0x7f62c40c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/357471789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406de20 0x7f62c40c77f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009f20 tx=0x7f62b4009840 comp rx=0 tx=0).stop 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 -- 172.21.15.64:0/357471789 shutdown_connections 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/357471789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406c7c0 0x7f62c40c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/357471789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406de20 0x7f62c40c77f0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 -- 172.21.15.64:0/357471789 >> 172.21.15.64:0/357471789 conn(0x7f62c4067bb0 msgr2=0x7f62c40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 -- 172.21.15.64:0/357471789 shutdown_connections 2023-09-27T23:30:29.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 -- 172.21.15.64:0/357471789 wait complete. 2023-09-27T23:30:29.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 Processor -- start 2023-09-27T23:30:29.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.843+0000 7f62c9bd1700 1 -- start start 2023-09-27T23:30:29.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 0x7f62c414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406de20 0x7f62c4149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c4155960 0x7f62c4169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c414aa40 con 0x7f62c406c7c0 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c414ad20 con 0x7f62c4155960 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c414b000 con 0x7f62c406de20 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 0x7f62c414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 0x7f62c414a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40246/0 (socket says 172.21.15.64:40246) 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 -- 172.21.15.64:0/3799645910 learned_addr learned my addr 172.21.15.64:0/3799645910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c93d0700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c4155960 0x7f62c4169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c3fff700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406de20 0x7f62c4149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 -- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406de20 msgr2=0x7f62c4149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406de20 0x7f62c4149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 -- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c4155960 msgr2=0x7f62c4169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c4155960 0x7f62c4169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 -- 172.21.15.64:0/3799645910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4009ae0 con 0x7f62c406c7c0 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c3fff700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406de20 0x7f62c4149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c8bcf700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 0x7f62c414a3f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c4e0 tx=0x7f62b800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8011590 con 0x7f62c406c7c0 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b8011710 con 0x7f62c406c7c0 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8011ad0 con 0x7f62c406c7c0 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c414b2e0 con 0x7f62c406c7c0 2023-09-27T23:30:29.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c41499e0 con 0x7f62c406c7c0 2023-09-27T23:30:29.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.847+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c40ae8e0 con 0x7f62c406c7c0 2023-09-27T23:30:29.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.851+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f62b801b3b0 con 0x7f62c406c7c0 2023-09-27T23:30:29.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.851+0000 7f62c0ff9700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62b0060a70 0x7f62b0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.851+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f62b807c590 con 0x7f62c406c7c0 2023-09-27T23:30:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.851+0000 7f62c3fff700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62b0060a70 0x7f62b0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:29.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.851+0000 7f62c3fff700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62b0060a70 0x7f62b0062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009880 tx=0x7f62b4018000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:29.859+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f62b804c5c0 con 0x7f62c406c7c0 2023-09-27T23:30:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.019+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62c40ac370 con 0x7f62c406c7c0 2023-09-27T23:30:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.019+0000 7f62c0ff9700 1 -- 172.21.15.64:0/3799645910 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f62c40ac370 con 0x7f62c406c7c0 2023-09-27T23:30:30.023 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:30.024 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":114,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:29.390515+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:30:27.964715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-27T23:30:28.361565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62b0060a70 msgr2=0x7f62b0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62b0060a70 0x7f62b0062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009880 tx=0x7f62b4018000 comp rx=0 tx=0).stop 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 msgr2=0x7f62c414a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 0x7f62c414a3f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c4e0 tx=0x7f62b800c7f0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 shutdown_connections 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62b0060a70 0x7f62b0062f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62c4155960 0x7f62c4169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62c406de20 0x7f62c4149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 --2- 172.21.15.64:0/3799645910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62c406c7c0 0x7f62c414a3f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 >> 172.21.15.64:0/3799645910 conn(0x7f62c4067bb0 msgr2=0x7f62c406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 shutdown_connections 2023-09-27T23:30:30.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.023+0000 7f62c9bd1700 1 -- 172.21.15.64:0/3799645910 wait complete. 2023-09-27T23:30:30.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:30:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.143+0000 7f9490c5c700 1 Processor -- start 2023-09-27T23:30:30.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.143+0000 7f9490c5c700 1 -- start start 2023-09-27T23:30:30.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f948c150200 0x7f948c0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f948c0ae770 0x7f948c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c0af250 con 0x7f948c0ac720 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0af530 con 0x7f948c150200 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0ca8a0 con 0x7f948c0ae770 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f948c150200 0x7f948c0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f948c150200 0x7f948c0c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40260/0 (socket says 172.21.15.64:40260) 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 -- 172.21.15.64:0/4233533157 learned_addr learned my addr 172.21.15.64:0/4233533157 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9483fff700 1 --2- 172.21.15.64:0/4233533157 >> v2:172.21.15.123:3300/0 conn(0x7f948c0ae770 0x7f948c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 -- 172.21.15.64:0/4233533157 >> v2:172.21.15.123:3300/0 conn(0x7f948c0ae770 msgr2=0x7f948c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 --2- 172.21.15.64:0/4233533157 >> v2:172.21.15.123:3300/0 conn(0x7f948c0ae770 0x7f948c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 -- 172.21.15.64:0/4233533157 >> v1:172.21.15.123:6789/0 conn(0x7f948c0ac720 legacy=0x7f948c0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 -- 172.21.15.64:0/4233533157 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0cab80 con 0x7f948c150200 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948b7fe700 1 --2- 172.21.15.64:0/4233533157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c150200 0x7f948c0c7170 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f948c06b3d0 tx=0x7f947c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8f3ec19a87e493 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948affd700 1 -- 172.21.15.64:0/4233533157 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00b820 con 0x7f948c150200 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948affd700 1 -- 172.21.15.64:0/4233533157 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c00b980 con 0x7f948c150200 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f948affd700 1 -- 172.21.15.64:0/4233533157 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00bd20 con 0x7f948c150200 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- 172.21.15.64:0/4233533157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c150200 msgr2=0x7f948c0c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 --2- 172.21.15.64:0/4233533157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c150200 0x7f948c0c7170 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f948c06b3d0 tx=0x7f947c009250 comp rx=0 tx=0).stop 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- 172.21.15.64:0/4233533157 shutdown_connections 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 --2- 172.21.15.64:0/4233533157 >> v2:172.21.15.123:3300/0 conn(0x7f948c0ae770 0x7f948c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 --2- 172.21.15.64:0/4233533157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c150200 0x7f948c0c7170 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- 172.21.15.64:0/4233533157 >> 172.21.15.64:0/4233533157 conn(0x7f948c0ccb30 msgr2=0x7f948c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:30.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- 172.21.15.64:0/4233533157 shutdown_connections 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.147+0000 7f9490c5c700 1 -- 172.21.15.64:0/4233533157 wait complete. 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.151+0000 7f9490c5c700 1 Processor -- start 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.151+0000 7f9490c5c700 1 -- start start 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 0x7f948c069260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f948c0ae770 0x7f948c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c150200 0x7f948c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0a4440 con 0x7f948c0ac720 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c14cad0 con 0x7f948c150200 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c14cd00 con 0x7f948c0ae770 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 0x7f948c069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 0x7f948c069260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40268/0 (socket says 172.21.15.64:40268) 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 -- 172.21.15.64:0/205986424 learned_addr learned my addr 172.21.15.64:0/205986424 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948bfff700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c150200 0x7f948c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9483fff700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f948c0ae770 0x7f948c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 -- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f948c0ae770 msgr2=0x7f948c14b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f948c0ae770 0x7f948c14b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 -- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c150200 msgr2=0x7f948c0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c150200 0x7f948c0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 -- 172.21.15.64:0/205986424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0094f0 con 0x7f948c0ac720 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f948b7fe700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 0x7f948c069260 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f947c0030c0 tx=0x7f947c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011d00 con 0x7f948c0ac720 2023-09-27T23:30:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c020070 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c14cfa0 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c01b6a0 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.155+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c14d840 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.159+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c14d390 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.159+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f947c022030 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.159+0000 7f94837fe700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f946c060aa0 0x7f946c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.159+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f947c07f260 con 0x7f948c0ac720 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.159+0000 7f9483fff700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f946c060aa0 0x7f946c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.159+0000 7f9483fff700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f946c060aa0 0x7f946c062f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f948c0c8930 tx=0x7f9474008040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.167+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f947c04f310 con 0x7f948c0ac720 2023-09-27T23:30:30.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.363+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f948c14d390 con 0x7f948c0ac720 2023-09-27T23:30:30.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f94837fe700 1 -- 172.21.15.64:0/205986424 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f947c052930 con 0x7f948c0ac720 2023-09-27T23:30:30.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f946c060aa0 msgr2=0x7f946c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f946c060aa0 0x7f946c062f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f948c0c8930 tx=0x7f9474008040 comp rx=0 tx=0).stop 2023-09-27T23:30:30.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 msgr2=0x7f948c069260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 0x7f948c069260 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f947c0030c0 tx=0x7f947c016000 comp rx=0 tx=0).stop 2023-09-27T23:30:30.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 shutdown_connections 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f946c060aa0 0x7f946c062f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f948c150200 0x7f948c0c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f948c0ae770 0x7f948c14b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 --2- 172.21.15.64:0/205986424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c0ac720 0x7f948c069260 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 >> 172.21.15.64:0/205986424 conn(0x7f948c0ccb30 msgr2=0x7f948c0c31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 shutdown_connections 2023-09-27T23:30:30.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.411+0000 7f9490c5c700 1 -- 172.21.15.64:0/205986424 wait complete. 2023-09-27T23:30:30.430 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:30:30.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:30:30.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.539+0000 7fe8b71be700 1 Processor -- start 2023-09-27T23:30:30.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.539+0000 7fe8b71be700 1 -- start start 2023-09-27T23:30:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8b00ab870 0x7fe8b00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8b015a180 0x7fe8b006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00b0e40 con 0x7fe8b00b08d0 2023-09-27T23:30:30.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00b1120 con 0x7fe8b00ab870 2023-09-27T23:30:30.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00b1400 con 0x7fe8b015a180 2023-09-27T23:30:30.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b4f5a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8b00ab870 0x7fe8b00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8affff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe8b00b08d0 0x7fe8b0159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45738/0 (socket says 172.21.15.64:45738) 2023-09-27T23:30:30.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8affff700 1 -- 172.21.15.64:0/2716654713 learned_addr learned my addr 172.21.15.64:0/2716654713 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:30.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b575b700 1 --2- 172.21.15.64:0/2716654713 >> v2:172.21.15.123:3300/0 conn(0x7fe8b015a180 0x7fe8b006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8af7fe700 1 -- 172.21.15.64:0/2716654713 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119701709 0 0) 0x7fe8b00b0e40 con 0x7fe8b00b08d0 2023-09-27T23:30:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8af7fe700 1 -- 172.21.15.64:0/2716654713 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a0003570 con 0x7fe8b00b08d0 2023-09-27T23:30:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/2716654713 >> v2:172.21.15.123:3300/0 conn(0x7fe8b015a180 msgr2=0x7fe8b006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b4f5a700 1 --2- 172.21.15.64:0/2716654713 >> v2:172.21.15.123:3300/0 conn(0x7fe8b015a180 0x7fe8b006c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/2716654713 >> v1:172.21.15.64:6789/0 conn(0x7fe8b00b08d0 legacy=0x7fe8b0159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/2716654713 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00b16e0 con 0x7fe8b00ab870 2023-09-27T23:30:30.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b4f5a700 1 --2- 172.21.15.64:0/2716654713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00afdb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe8b015aa50 tx=0x7fe8a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55070256aa3a4e51 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:30.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8af7fe700 1 -- 172.21.15.64:0/2716654713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4011630 con 0x7fe8b00ab870 2023-09-27T23:30:30.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8af7fe700 1 -- 172.21.15.64:0/2716654713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a40117b0 con 0x7fe8b00ab870 2023-09-27T23:30:30.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8af7fe700 1 -- 172.21.15.64:0/2716654713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4011b70 con 0x7fe8b00ab870 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 -- 172.21.15.64:0/2716654713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 msgr2=0x7fe8b00afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 --2- 172.21.15.64:0/2716654713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00afdb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe8b015aa50 tx=0x7fe8a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 -- 172.21.15.64:0/2716654713 shutdown_connections 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 --2- 172.21.15.64:0/2716654713 >> v2:172.21.15.123:3300/0 conn(0x7fe8b015a180 0x7fe8b006c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 --2- 172.21.15.64:0/2716654713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00afdb0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.543+0000 7fe8b71be700 1 -- 172.21.15.64:0/2716654713 >> 172.21.15.64:0/2716654713 conn(0x7fe8b00cc580 msgr2=0x7fe8b00cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.547+0000 7fe8b71be700 1 -- 172.21.15.64:0/2716654713 shutdown_connections 2023-09-27T23:30:30.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.547+0000 7fe8b71be700 1 -- 172.21.15.64:0/2716654713 wait complete. 2023-09-27T23:30:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.547+0000 7fe8b71be700 1 Processor -- start 2023-09-27T23:30:30.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.547+0000 7fe8b71be700 1 -- start start 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8b00b08d0 0x7fe8b0151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b015a180 0x7fe8b0167420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0151560 con 0x7fe8b00ab870 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0167cd0 con 0x7fe8b015a180 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0167fb0 con 0x7fe8b00b08d0 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40290/0 (socket says 172.21.15.64:40290) 2023-09-27T23:30:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/680087647 learned_addr learned my addr 172.21.15.64:0/680087647 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8affff700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8b00b08d0 0x7fe8b0151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b575b700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b015a180 0x7fe8b0167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8b00b08d0 msgr2=0x7fe8b0151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8b00b08d0 0x7fe8b0151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b015a180 msgr2=0x7fe8b0167420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b015a180 0x7fe8b0167420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 -- 172.21.15.64:0/680087647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009420 con 0x7fe8b00ab870 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b4f5a700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00adb20 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4013000 tx=0x7fe8a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:30.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a401f070 con 0x7fe8b00ab870 2023-09-27T23:30:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4000bc0 con 0x7fe8b00ab870 2023-09-27T23:30:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b0168290 con 0x7fe8b00ab870 2023-09-27T23:30:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4011b50 con 0x7fe8b00ab870 2023-09-27T23:30:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.551+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b016ad40 con 0x7fe8b00ab870 2023-09-27T23:30:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.555+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe8a4023070 con 0x7fe8b00ab870 2023-09-27T23:30:30.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.555+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0071120 con 0x7fe8b00ab870 2023-09-27T23:30:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.555+0000 7fe8acff9700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe894060a70 0x7fe894062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:30.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.555+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe8a407e320 con 0x7fe8b00ab870 2023-09-27T23:30:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.555+0000 7fe8affff700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe894060a70 0x7fe894062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:30.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.555+0000 7fe8affff700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe894060a70 0x7fe894062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0153a90 tx=0x7fe89c009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:30.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.563+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe8a404e350 con 0x7fe8b00ab870 2023-09-27T23:30:30.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:30.763+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe8b00c98f0 con 0x7fe8b00ab870 2023-09-27T23:30:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.415+0000 7fe8acff9700 1 -- 172.21.15.64:0/680087647 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8a4051970 con 0x7fe8b00ab870 2023-09-27T23:30:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe894060a70 msgr2=0x7fe894062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe894060a70 0x7fe894062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0153a90 tx=0x7fe89c009040 comp rx=0 tx=0).stop 2023-09-27T23:30:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 msgr2=0x7fe8b00adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00adb20 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4013000 tx=0x7fe8a4017000 comp rx=0 tx=0).stop 2023-09-27T23:30:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 shutdown_connections 2023-09-27T23:30:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe894060a70 0x7fe894062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8b015a180 0x7fe8b0167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8b00b08d0 0x7fe8b0151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 --2- 172.21.15.64:0/680087647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8b00ab870 0x7fe8b00adb20 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 >> 172.21.15.64:0/680087647 conn(0x7fe8b00cc580 msgr2=0x7fe8b00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 shutdown_connections 2023-09-27T23:30:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.419+0000 7fe8b71be700 1 -- 172.21.15.64:0/680087647 wait complete. 2023-09-27T23:30:31.440 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:30:31.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:31.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfffff700 1 Processor -- start 2023-09-27T23:30:31.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfffff700 1 -- start start 2023-09-27T23:30:31.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6c000b1580 0x7f6c0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c000c9c20 con 0x7f6c000c97d0 2023-09-27T23:30:31.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0006cf00 con 0x7f6c00155990 2023-09-27T23:30:31.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c000ae7e0 con 0x7f6c000b1580 2023-09-27T23:30:31.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfe7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6c000c97d0 0x7f6c000b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45740/0 (socket says 172.21.15.64:45740) 2023-09-27T23:30:31.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfe7fc700 1 -- 172.21.15.64:0/780738434 learned_addr learned my addr 172.21.15.64:0/780738434 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:31.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bff7fe700 1 --2- 172.21.15.64:0/780738434 >> v2:172.21.15.123:3300/0 conn(0x7f6c000b1580 0x7f6c0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.547+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513456489 0 0) 0x7f6c000c9c20 con 0x7f6c000c97d0 2023-09-27T23:30:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be8003570 con 0x7f6c000c97d0 2023-09-27T23:30:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532745740 0 0) 0x7f6c0006cf00 con 0x7f6c00155990 2023-09-27T23:30:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c000c9c20 con 0x7f6c00155990 2023-09-27T23:30:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bff7fe700 1 -- 172.21.15.64:0/780738434 >> v1:172.21.15.123:6789/0 conn(0x7f6c00155990 legacy=0x7f6c000b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bff7fe700 1 -- 172.21.15.64:0/780738434 >> v1:172.21.15.64:6789/0 conn(0x7f6c000c97d0 legacy=0x7f6c000b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bff7fe700 1 -- 172.21.15.64:0/780738434 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c000b01d0 con 0x7f6c000b1580 2023-09-27T23:30:31.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bff7fe700 1 --2- 172.21.15.64:0/780738434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000b1580 0x7f6c0006d390 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80098e0 tx=0x7f6bf80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8988a10cad4501 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:31.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8011630 con 0x7f6c000b1580 2023-09-27T23:30:31.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8011790 con 0x7f6c000b1580 2023-09-27T23:30:31.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfdffb700 1 -- 172.21.15.64:0/780738434 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8011b30 con 0x7f6c000b1580 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 -- 172.21.15.64:0/780738434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000b1580 msgr2=0x7f6c0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 --2- 172.21.15.64:0/780738434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000b1580 0x7f6c0006d390 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80098e0 tx=0x7f6bf80091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 -- 172.21.15.64:0/780738434 shutdown_connections 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 --2- 172.21.15.64:0/780738434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000b1580 0x7f6c0006d390 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 -- 172.21.15.64:0/780738434 >> 172.21.15.64:0/780738434 conn(0x7f6c00067bb0 msgr2=0x7f6c000add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 -- 172.21.15.64:0/780738434 shutdown_connections 2023-09-27T23:30:31.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.551+0000 7f6bfffff700 1 -- 172.21.15.64:0/780738434 wait complete. 2023-09-27T23:30:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 Processor -- start 2023-09-27T23:30:31.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 -- start start 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 0x7f6c00144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000c97d0 0x7f6c000c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c00155990 0x7f6c0006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c000c44c0 con 0x7f6c000b1580 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c001452c0 con 0x7f6c000c97d0 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c000c71c0 con 0x7f6c00155990 2023-09-27T23:30:31.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 0x7f6c00144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 0x7f6c00144d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40300/0 (socket says 172.21.15.64:40300) 2023-09-27T23:30:31.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 -- 172.21.15.64:0/417876867 learned_addr learned my addr 172.21.15.64:0/417876867 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:31.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfe7fc700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000c97d0 0x7f6c000c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bff7fe700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c00155990 0x7f6c0006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 -- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c00155990 msgr2=0x7f6c0006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c00155990 0x7f6c0006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 -- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000c97d0 msgr2=0x7f6c000c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000c97d0 0x7f6c000c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 -- 172.21.15.64:0/417876867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8009420 con 0x7f6c000b1580 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfeffd700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 0x7f6c00144d80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000c900 tx=0x7f6bf0002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0012630 con 0x7f6c000b1580 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0012790 con 0x7f6c000b1580 2023-09-27T23:30:31.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c000c6e90 con 0x7f6c000b1580 2023-09-27T23:30:31.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0012b50 con 0x7f6c000b1580 2023-09-27T23:30:31.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.555+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c000c3f00 con 0x7f6c000b1580 2023-09-27T23:30:31.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.559+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0004be40 con 0x7f6c000b1580 2023-09-27T23:30:31.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.559+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6bf001b400 con 0x7f6c000b1580 2023-09-27T23:30:31.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.559+0000 7f6be6ffd700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6bec060a70 0x7f6bec062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.559+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6bf007d3f0 con 0x7f6c000b1580 2023-09-27T23:30:31.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.559+0000 7f6bfe7fc700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6bec060a70 0x7f6bec062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.559+0000 7f6bfe7fc700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6bec060a70 0x7f6bec062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c000c4ca0 tx=0x7f6bf4009040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.567+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6bf004d420 con 0x7f6c000b1580 2023-09-27T23:30:31.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.763+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c000c8ad0 con 0x7f6c000b1580 2023-09-27T23:30:31.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.767+0000 7f6be6ffd700 1 -- 172.21.15.64:0/417876867 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6bf0050a40 con 0x7f6c000b1580 2023-09-27T23:30:31.769 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:31.769 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:30:29.391491+0000","modified":"2023-09-27T23:30:31.410091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7605":{"gid":7605,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1829030850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1829030850},{"type":"v1","addr":"172.21.15.64:6835","nonce":1829030850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6bec060a70 msgr2=0x7f6bec062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6bec060a70 0x7f6bec062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c000c4ca0 tx=0x7f6bf4009040 comp rx=0 tx=0).stop 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 msgr2=0x7f6c00144d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 0x7f6c00144d80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000c900 tx=0x7f6bf0002780 comp rx=0 tx=0).stop 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 shutdown_connections 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6bec060a70 0x7f6bec062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c00155990 0x7f6c0006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c000c97d0 0x7f6c000c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 --2- 172.21.15.64:0/417876867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c000b1580 0x7f6c00144d80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 >> 172.21.15.64:0/417876867 conn(0x7f6c00067bb0 msgr2=0x7f6c000af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 shutdown_connections 2023-09-27T23:30:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.771+0000 7f6bfffff700 1 -- 172.21.15.64:0/417876867 wait complete. 2023-09-27T23:30:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2023-09-27T23:30:31.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:30:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 Processor -- start 2023-09-27T23:30:31.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 -- start start 2023-09-27T23:30:31.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9f4c0cc470 0x7f9f4c0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c0ca5a0 con 0x7f9f4c06a540 2023-09-27T23:30:31.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4c06dac0 con 0x7f9f4c0cc470 2023-09-27T23:30:31.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.887+0000 7f9f5174d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0b0260 con 0x7f9f4c158e70 2023-09-27T23:30:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9f4c0cc470 0x7f9f4c0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f4bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9f4c0cc470 0x7f9f4c0cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:58298/0 (socket says 172.21.15.64:58298) 2023-09-27T23:30:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 -- 172.21.15.64:0/2446239329 learned_addr learned my addr 172.21.15.64:0/2446239329 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 -- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 msgr2=0x7f9f4c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:31.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 --2- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c0afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 -- 172.21.15.64:0/2446239329 >> v1:172.21.15.123:6789/0 conn(0x7f9f4c06a540 legacy=0x7f9f4c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 -- 172.21.15.64:0/2446239329 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c06d220 con 0x7f9f4c0cc470 2023-09-27T23:30:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f4b7fe700 1 -- 172.21.15.64:0/2446239329 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025291845 0 0) 0x7f9f4c0ca5a0 con 0x7f9f4c06a540 2023-09-27T23:30:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f4bfff700 1 --2- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:31.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f50f4c700 1 --2- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cc8b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009f20 tx=0x7f9f44009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f426aaa05dc772 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f4b7fe700 1 -- 172.21.15.64:0/2446239329 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400e070 con 0x7f9f4c0cc470 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f4b7fe700 1 -- 172.21.15.64:0/2446239329 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400b9d0 con 0x7f9f4c0cc470 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f4b7fe700 1 -- 172.21.15.64:0/2446239329 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440092e0 con 0x7f9f4c0cc470 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 -- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 msgr2=0x7f9f4c0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 --2- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cc8b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009f20 tx=0x7f9f44009840 comp rx=0 tx=0).stop 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 -- 172.21.15.64:0/2446239329 shutdown_connections 2023-09-27T23:30:31.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 --2- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cc8b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 --2- 172.21.15.64:0/2446239329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 -- 172.21.15.64:0/2446239329 >> 172.21.15.64:0/2446239329 conn(0x7f9f4c0670e0 msgr2=0x7f9f4c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:31.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 -- 172.21.15.64:0/2446239329 shutdown_connections 2023-09-27T23:30:31.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.891+0000 7f9f5174d700 1 -- 172.21.15.64:0/2446239329 wait complete. 2023-09-27T23:30:31.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 Processor -- start 2023-09-27T23:30:31.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 -- start start 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f4c06a540 0x7f9f4c0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c14cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c16a040 con 0x7f9f4c06a540 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c169430 con 0x7f9f4c0cc470 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f5174d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c169710 con 0x7f9f4c158e70 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f4c06a540 0x7f9f4c0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f4c06a540 0x7f9f4c0b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40316/0 (socket says 172.21.15.64:40316) 2023-09-27T23:30:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f4bfff700 1 -- 172.21.15.64:0/3220986928 learned_addr learned my addr 172.21.15.64:0/3220986928 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f50f4c700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c14cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 -- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 msgr2=0x7f9f4c14cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c14cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 -- 172.21.15.64:0/3220986928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f4c06a540 msgr2=0x7f9f4c0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f4c06a540 0x7f9f4c0b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 -- 172.21.15.64:0/3220986928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009420 con 0x7f9f4c0cc470 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.895+0000 7f9f43fff700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cefd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c510 tx=0x7f9f34007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400ce80 con 0x7f9f4c0cc470 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34011590 con 0x7f9f4c0cc470 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44009ae0 con 0x7f9f4c0cc470 2023-09-27T23:30:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34011950 con 0x7f9f4c0cc470 2023-09-27T23:30:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c169cd0 con 0x7f9f4c0cc470 2023-09-27T23:30:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c04be40 con 0x7f9f4c0cc470 2023-09-27T23:30:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9f34011d40 con 0x7f9f4c0cc470 2023-09-27T23:30:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.899+0000 7f9f48ff9700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9f2c060a70 0x7f9f2c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.903+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9f3407c4c0 con 0x7f9f4c0cc470 2023-09-27T23:30:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.903+0000 7f9f4bfff700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9f2c060a70 0x7f9f2c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.903+0000 7f9f4bfff700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9f2c060a70 0x7f9f2c062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c011310 tx=0x7f9f3c0091a0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:31.915+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9f3404c4f0 con 0x7f9f4c0cc470 2023-09-27T23:30:32.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.083+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f4c06bb10 con 0x7f9f4c0cc470 2023-09-27T23:30:32.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.083+0000 7f9f48ff9700 1 -- 172.21.15.64:0/3220986928 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+17085 (secure 0 0 0) 0x7f9f34017090 con 0x7f9f4c0cc470 2023-09-27T23:30:32.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:32.088 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":115,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:30.394971+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:30:27.964715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-27T23:30:28.361565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:30:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9f2c060a70 msgr2=0x7f9f2c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9f2c060a70 0x7f9f2c062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c011310 tx=0x7f9f3c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 msgr2=0x7f9f4c0cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cefd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c510 tx=0x7f9f34007ec0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 shutdown_connections 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9f2c060a70 0x7f9f2c062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f4c158e70 0x7f9f4c14cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f4c0cc470 0x7f9f4c0cefd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 --2- 172.21.15.64:0/3220986928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f4c06a540 0x7f9f4c0b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 >> 172.21.15.64:0/3220986928 conn(0x7f9f4c0670e0 msgr2=0x7f9f4c158330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 shutdown_connections 2023-09-27T23:30:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.087+0000 7f9f5174d700 1 -- 172.21.15.64:0/3220986928 wait complete. 2023-09-27T23:30:32.102 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:30:32.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:30:32.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 Processor -- start 2023-09-27T23:30:32.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 -- start start 2023-09-27T23:30:32.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3624155960 0x7f362415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 0x7f36240c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362406d990 con 0x7f362406c7c0 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362415a570 con 0x7f3624155960 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f362a474700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362406cd80 con 0x7f362406de20 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f3629c73700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f362406c7c0 0x7f36240c5fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45754/0 (socket says 172.21.15.64:45754) 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f3629c73700 1 -- 172.21.15.64:0/1044504993 learned_addr learned my addr 172.21.15.64:0/1044504993 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f3629472700 1 --2- 172.21.15.64:0/1044504993 >> v2:172.21.15.123:3300/0 conn(0x7f3624155960 0x7f362415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f3628c71700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 0x7f36240c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398019716 0 0) 0x7f362406d990 con 0x7f362406c7c0 2023-09-27T23:30:32.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3610003570 con 0x7f362406c7c0 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062028844 0 0) 0x7f3610003570 con 0x7f362406c7c0 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 msgr2=0x7f36240c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 0x7f36240c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 >> v2:172.21.15.123:3300/0 conn(0x7f3624155960 msgr2=0x7f362415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 --2- 172.21.15.64:0/1044504993 >> v2:172.21.15.123:3300/0 conn(0x7f3624155960 0x7f362415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362406d060 con 0x7f362406c7c0 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f361c0017f0 con 0x7f362406c7c0 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f3629472700 1 --2- 172.21.15.64:0/1044504993 >> v2:172.21.15.123:3300/0 conn(0x7f3624155960 0x7f362415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:32.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 >> v1:172.21.15.64:6789/0 conn(0x7f362406c7c0 legacy=0x7f36240c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 0x7f3610003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610003fe0 0x7f3610004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362415a570 con 0x7f3610004d50 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362406cd80 con 0x7f3610003fe0 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361000ba90 con 0x7f362406de20 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.211+0000 7f3629c73700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629472700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 0x7f3610003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3628c71700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610003fe0 0x7f3610004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 -- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 msgr2=0x7f3610003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362406de20 0x7f3610003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 -- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610003fe0 msgr2=0x7f3610004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610003fe0 0x7f3610004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 -- 172.21.15.64:0/1044504993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610003570 con 0x7f3610004d50 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36100051c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f361c00a420 tx=0x7f361c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcb95ed4a944c8e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0104a0 con 0x7f3610004d50 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c010620 con 0x7f3610004d50 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f361bfff700 1 -- 172.21.15.64:0/1044504993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0109e0 con 0x7f3610004d50 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 -- 172.21.15.64:0/1044504993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 msgr2=0x7f36100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 --2- 172.21.15.64:0/1044504993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36100051c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f361c00a420 tx=0x7f361c002210 comp rx=0 tx=0).stop 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f3629c73700 1 -- 172.21.15.64:0/1044504993 reap_dead start 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 -- 172.21.15.64:0/1044504993 shutdown_connections 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 -- 172.21.15.64:0/1044504993 >> 172.21.15.64:0/1044504993 conn(0x7f3624067bb0 msgr2=0x7f36240cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 -- 172.21.15.64:0/1044504993 shutdown_connections 2023-09-27T23:30:32.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 -- 172.21.15.64:0/1044504993 wait complete. 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 Processor -- start 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.215+0000 7f362a474700 1 -- start start 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36240c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36240c6990 0x7f362414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362414c120 0x7f3624153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c7d40 con 0x7f3610004d50 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c7130 con 0x7f36240c6990 2023-09-27T23:30:32.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c7410 con 0x7f362414c120 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36240c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36240c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40330/0 (socket says 172.21.15.64:40330) 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 -- 172.21.15.64:0/555930447 learned_addr learned my addr 172.21.15.64:0/555930447 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3628c71700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36240c6990 0x7f362414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629c73700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362414c120 0x7f3624153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 -- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362414c120 msgr2=0x7f3624153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362414c120 0x7f3624153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 -- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36240c6990 msgr2=0x7f362414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36240c6990 0x7f362414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 -- 172.21.15.64:0/555930447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240c76f0 con 0x7f3610004d50 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3628c71700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36240c6990 0x7f362414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629c73700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362414c120 0x7f3624153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:32.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f3629472700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36240c8aa0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3614011d70 tx=0x7f36140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400fcd0 con 0x7f3610004d50 2023-09-27T23:30:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400fe30 con 0x7f3610004d50 2023-09-27T23:30:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36240c79d0 con 0x7f3610004d50 2023-09-27T23:30:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361401d6d0 con 0x7f3610004d50 2023-09-27T23:30:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.219+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3624153d60 con 0x7f3610004d50 2023-09-27T23:30:32.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.223+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3624164fd0 con 0x7f3610004d50 2023-09-27T23:30:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.223+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f361401dac0 con 0x7f3610004d50 2023-09-27T23:30:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.223+0000 7f36197fa700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3608060a70 0x7f3608062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.223+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3614018070 con 0x7f3610004d50 2023-09-27T23:30:32.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.223+0000 7f3628c71700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3608060a70 0x7f3608062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.223+0000 7f3628c71700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3608060a70 0x7f3608062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3624159fe0 tx=0x7f3620008040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.231+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f361404e080 con 0x7f3610004d50 2023-09-27T23:30:32.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.387+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f362404be40 con 0x7f3608060a70 2023-09-27T23:30:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.407+0000 7f36197fa700 1 -- 172.21.15.64:0/555930447 <== mgr.6578 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296656 (secure 0 0 0) 0x7f362404be40 con 0x7f3608060a70 2023-09-27T23:30:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:32.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3608060a70 msgr2=0x7f3608062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3608060a70 0x7f3608062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3624159fe0 tx=0x7f3620008040 comp rx=0 tx=0).stop 2023-09-27T23:30:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 msgr2=0x7f36240c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36240c8aa0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3614011d70 tx=0x7f36140091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 shutdown_connections 2023-09-27T23:30:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3608060a70 0x7f3608062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f362414c120 0x7f3624153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36240c6990 0x7f362414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 --2- 172.21.15.64:0/555930447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3610004d50 0x7f36240c8aa0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.411+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 >> 172.21.15.64:0/555930447 conn(0x7f3624067bb0 msgr2=0x7f362414dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.415+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 shutdown_connections 2023-09-27T23:30:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.415+0000 7f362a474700 1 -- 172.21.15.64:0/555930447 wait complete. 2023-09-27T23:30:32.418 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:30:32.431 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:30:31.936263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591904,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":340,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":211748,"kb_used_data":11876,"kb_used_omap":0,"kb_used_meta":199808,"kb_avail":754762972,"statfs":{"total":773094113280,"available":772877283328,"internally_reserved":0,"allocated":12161024,"data_stored":6828752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":204603392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001197"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437015+0000","last_change":"2023-09-27T23:30:29.424776+0000","last_active":"2023-09-27T23:30:29.437015+0000","last_peered":"2023-09-27T23:30:29.437015+0000","last_clean":"2023-09-27T23:30:29.437015+0000","last_became_active":"2023-09-27T23:30:29.424350+0000","last_became_peered":"2023-09-27T23:30:29.424350+0000","last_unstale":"2023-09-27T23:30:29.437015+0000","last_undegraded":"2023-09-27T23:30:29.437015+0000","last_fullsized":"2023-09-27T23:30:29.437015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:07.044123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437217+0000","last_change":"2023-09-27T23:30:29.431912+0000","last_active":"2023-09-27T23:30:29.437217+0000","last_peered":"2023-09-27T23:30:29.437217+0000","last_clean":"2023-09-27T23:30:29.437217+0000","last_became_active":"2023-09-27T23:30:29.431583+0000","last_became_peered":"2023-09-27T23:30:29.431583+0000","last_unstale":"2023-09-27T23:30:29.437217+0000","last_undegraded":"2023-09-27T23:30:29.437217+0000","last_fullsized":"2023-09-27T23:30:29.437217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:00:22.489672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447931+0000","last_change":"2023-09-27T23:30:29.445370+0000","last_active":"2023-09-27T23:30:29.447931+0000","last_peered":"2023-09-27T23:30:29.447931+0000","last_clean":"2023-09-27T23:30:29.447931+0000","last_became_active":"2023-09-27T23:30:29.444846+0000","last_became_peered":"2023-09-27T23:30:29.444846+0000","last_unstale":"2023-09-27T23:30:29.447931+0000","last_undegraded":"2023-09-27T23:30:29.447931+0000","last_fullsized":"2023-09-27T23:30:29.447931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:41.665511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447849+0000","last_change":"2023-09-27T23:30:29.437968+0000","last_active":"2023-09-27T23:30:29.447849+0000","last_peered":"2023-09-27T23:30:29.447849+0000","last_clean":"2023-09-27T23:30:29.447849+0000","last_became_active":"2023-09-27T23:30:29.437657+0000","last_became_peered":"2023-09-27T23:30:29.437657+0000","last_unstale":"2023-09-27T23:30:29.447849+0000","last_undegraded":"2023-09-27T23:30:29.447849+0000","last_fullsized":"2023-09-27T23:30:29.447849+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:51.757870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447819+0000","last_change":"2023-09-27T23:30:29.434868+0000","last_active":"2023-09-27T23:30:29.447819+0000","last_peered":"2023-09-27T23:30:29.447819+0000","last_clean":"2023-09-27T23:30:29.447819+0000","last_became_active":"2023-09-27T23:30:29.434622+0000","last_became_peered":"2023-09-27T23:30:29.434622+0000","last_unstale":"2023-09-27T23:30:29.447819+0000","last_undegraded":"2023-09-27T23:30:29.447819+0000","last_fullsized":"2023-09-27T23:30:29.447819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:31:18.606971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434942+0000","last_change":"2023-09-27T23:30:29.421559+0000","last_active":"2023-09-27T23:30:29.434942+0000","last_peered":"2023-09-27T23:30:29.434942+0000","last_clean":"2023-09-27T23:30:29.434942+0000","last_became_active":"2023-09-27T23:30:29.420693+0000","last_became_peered":"2023-09-27T23:30:29.420693+0000","last_unstale":"2023-09-27T23:30:29.434942+0000","last_undegraded":"2023-09-27T23:30:29.434942+0000","last_fullsized":"2023-09-27T23:30:29.434942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:11.243612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445884+0000","last_change":"2023-09-27T23:30:29.443588+0000","last_active":"2023-09-27T23:30:29.445884+0000","last_peered":"2023-09-27T23:30:29.445884+0000","last_clean":"2023-09-27T23:30:29.445884+0000","last_became_active":"2023-09-27T23:30:29.443289+0000","last_became_peered":"2023-09-27T23:30:29.443289+0000","last_unstale":"2023-09-27T23:30:29.445884+0000","last_undegraded":"2023-09-27T23:30:29.445884+0000","last_fullsized":"2023-09-27T23:30:29.445884+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:57:06.402886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445278+0000","last_change":"2023-09-27T23:30:29.434541+0000","last_active":"2023-09-27T23:30:29.445278+0000","last_peered":"2023-09-27T23:30:29.445278+0000","last_clean":"2023-09-27T23:30:29.445278+0000","last_became_active":"2023-09-27T23:30:29.434260+0000","last_became_peered":"2023-09-27T23:30:29.434260+0000","last_unstale":"2023-09-27T23:30:29.445278+0000","last_undegraded":"2023-09-27T23:30:29.445278+0000","last_fullsized":"2023-09-27T23:30:29.445278+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:26:16.902235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436939+0000","last_change":"2023-09-27T23:30:29.032760+0000","last_active":"2023-09-27T23:30:29.436939+0000","last_peered":"2023-09-27T23:30:29.436939+0000","last_clean":"2023-09-27T23:30:29.436939+0000","last_became_active":"2023-09-27T23:30:29.032431+0000","last_became_peered":"2023-09-27T23:30:29.032431+0000","last_unstale":"2023-09-27T23:30:29.436939+0000","last_undegraded":"2023-09-27T23:30:29.436939+0000","last_fullsized":"2023-09-27T23:30:29.436939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:09:55.740875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446028+0000","last_change":"2023-09-27T23:30:29.438718+0000","last_active":"2023-09-27T23:30:29.446028+0000","last_peered":"2023-09-27T23:30:29.446028+0000","last_clean":"2023-09-27T23:30:29.446028+0000","last_became_active":"2023-09-27T23:30:29.438454+0000","last_became_peered":"2023-09-27T23:30:29.438454+0000","last_unstale":"2023-09-27T23:30:29.446028+0000","last_undegraded":"2023-09-27T23:30:29.446028+0000","last_fullsized":"2023-09-27T23:30:29.446028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:58:47.626281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401580+0000","last_change":"2023-09-27T23:30:29.035856+0000","last_active":"2023-09-27T23:30:29.401580+0000","last_peered":"2023-09-27T23:30:29.401580+0000","last_clean":"2023-09-27T23:30:29.401580+0000","last_became_active":"2023-09-27T23:30:29.035384+0000","last_became_peered":"2023-09-27T23:30:29.035384+0000","last_unstale":"2023-09-27T23:30:29.401580+0000","last_undegraded":"2023-09-27T23:30:29.401580+0000","last_fullsized":"2023-09-27T23:30:29.401580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:16:29.194838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402510+0000","last_change":"2023-09-27T23:30:29.022763+0000","last_active":"2023-09-27T23:30:29.402510+0000","last_peered":"2023-09-27T23:30:29.402510+0000","last_clean":"2023-09-27T23:30:29.402510+0000","last_became_active":"2023-09-27T23:30:29.021867+0000","last_became_peered":"2023-09-27T23:30:29.021867+0000","last_unstale":"2023-09-27T23:30:29.402510+0000","last_undegraded":"2023-09-27T23:30:29.402510+0000","last_fullsized":"2023-09-27T23:30:29.402510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:37.446360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445578+0000","last_change":"2023-09-27T23:30:29.434529+0000","last_active":"2023-09-27T23:30:29.445578+0000","last_peered":"2023-09-27T23:30:29.445578+0000","last_clean":"2023-09-27T23:30:29.445578+0000","last_became_active":"2023-09-27T23:30:29.434185+0000","last_became_peered":"2023-09-27T23:30:29.434185+0000","last_unstale":"2023-09-27T23:30:29.445578+0000","last_undegraded":"2023-09-27T23:30:29.445578+0000","last_fullsized":"2023-09-27T23:30:29.445578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:06.448424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.449495+0000","last_change":"2023-09-27T23:30:29.040154+0000","last_active":"2023-09-27T23:30:29.449495+0000","last_peered":"2023-09-27T23:30:29.449495+0000","last_clean":"2023-09-27T23:30:29.449495+0000","last_became_active":"2023-09-27T23:30:29.039855+0000","last_became_peered":"2023-09-27T23:30:29.039855+0000","last_unstale":"2023-09-27T23:30:29.449495+0000","last_undegraded":"2023-09-27T23:30:29.449495+0000","last_fullsized":"2023-09-27T23:30:29.449495+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:26:42.757699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434186+0000","last_change":"2023-09-27T23:30:29.422077+0000","last_active":"2023-09-27T23:30:29.434186+0000","last_peered":"2023-09-27T23:30:29.434186+0000","last_clean":"2023-09-27T23:30:29.434186+0000","last_became_active":"2023-09-27T23:30:29.421784+0000","last_became_peered":"2023-09-27T23:30:29.421784+0000","last_unstale":"2023-09-27T23:30:29.434186+0000","last_undegraded":"2023-09-27T23:30:29.434186+0000","last_fullsized":"2023-09-27T23:30:29.434186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:59.674679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445131+0000","last_change":"2023-09-27T23:30:29.434466+0000","last_active":"2023-09-27T23:30:29.445131+0000","last_peered":"2023-09-27T23:30:29.445131+0000","last_clean":"2023-09-27T23:30:29.445131+0000","last_became_active":"2023-09-27T23:30:29.434133+0000","last_became_peered":"2023-09-27T23:30:29.434133+0000","last_unstale":"2023-09-27T23:30:29.445131+0000","last_undegraded":"2023-09-27T23:30:29.445131+0000","last_fullsized":"2023-09-27T23:30:29.445131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:46.166597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425237+0000","last_change":"2023-09-27T23:30:29.040263+0000","last_active":"2023-09-27T23:30:29.425237+0000","last_peered":"2023-09-27T23:30:29.425237+0000","last_clean":"2023-09-27T23:30:29.425237+0000","last_became_active":"2023-09-27T23:30:29.039976+0000","last_became_peered":"2023-09-27T23:30:29.039976+0000","last_unstale":"2023-09-27T23:30:29.425237+0000","last_undegraded":"2023-09-27T23:30:29.425237+0000","last_fullsized":"2023-09-27T23:30:29.425237+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:13.841678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437983+0000","last_change":"2023-09-27T23:30:29.031398+0000","last_active":"2023-09-27T23:30:29.437983+0000","last_peered":"2023-09-27T23:30:29.437983+0000","last_clean":"2023-09-27T23:30:29.437983+0000","last_became_active":"2023-09-27T23:30:29.031067+0000","last_became_peered":"2023-09-27T23:30:29.031067+0000","last_unstale":"2023-09-27T23:30:29.437983+0000","last_undegraded":"2023-09-27T23:30:29.437983+0000","last_fullsized":"2023-09-27T23:30:29.437983+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.823350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434246+0000","last_change":"2023-09-27T23:30:29.420956+0000","last_active":"2023-09-27T23:30:29.434246+0000","last_peered":"2023-09-27T23:30:29.434246+0000","last_clean":"2023-09-27T23:30:29.434246+0000","last_became_active":"2023-09-27T23:30:29.419860+0000","last_became_peered":"2023-09-27T23:30:29.419860+0000","last_unstale":"2023-09-27T23:30:29.434246+0000","last_undegraded":"2023-09-27T23:30:29.434246+0000","last_fullsized":"2023-09-27T23:30:29.434246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:10.232721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437190+0000","last_change":"2023-09-27T23:30:29.428257+0000","last_active":"2023-09-27T23:30:29.437190+0000","last_peered":"2023-09-27T23:30:29.437190+0000","last_clean":"2023-09-27T23:30:29.437190+0000","last_became_active":"2023-09-27T23:30:29.427972+0000","last_became_peered":"2023-09-27T23:30:29.427972+0000","last_unstale":"2023-09-27T23:30:29.437190+0000","last_undegraded":"2023-09-27T23:30:29.437190+0000","last_fullsized":"2023-09-27T23:30:29.437190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:15:35.465950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446021+0000","last_change":"2023-09-27T23:30:29.435122+0000","last_active":"2023-09-27T23:30:29.446021+0000","last_peered":"2023-09-27T23:30:29.446021+0000","last_clean":"2023-09-27T23:30:29.446021+0000","last_became_active":"2023-09-27T23:30:29.434814+0000","last_became_peered":"2023-09-27T23:30:29.434814+0000","last_unstale":"2023-09-27T23:30:29.446021+0000","last_undegraded":"2023-09-27T23:30:29.446021+0000","last_fullsized":"2023-09-27T23:30:29.446021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:54.480949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"103'205","reported_seq":556,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979261+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:30:24.979261+0000","last_peered":"2023-09-27T23:30:24.979261+0000","last_clean":"2023-09-27T23:30:24.979261+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:30:24.979261+0000","last_undegraded":"2023-09-27T23:30:24.979261+0000","last_fullsized":"2023-09-27T23:30:24.979261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979599+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:30:24.979599+0000","last_peered":"2023-09-27T23:30:24.979599+0000","last_clean":"2023-09-27T23:30:24.979599+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:30:24.979599+0000","last_undegraded":"2023-09-27T23:30:24.979599+0000","last_fullsized":"2023-09-27T23:30:24.979599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974626+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:30:24.974626+0000","last_peered":"2023-09-27T23:30:24.974626+0000","last_clean":"2023-09-27T23:30:24.974626+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:30:24.974626+0000","last_undegraded":"2023-09-27T23:30:24.974626+0000","last_fullsized":"2023-09-27T23:30:24.974626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445158+0000","last_change":"2023-09-27T23:30:29.046235+0000","last_active":"2023-09-27T23:30:29.445158+0000","last_peered":"2023-09-27T23:30:29.445158+0000","last_clean":"2023-09-27T23:30:29.445158+0000","last_became_active":"2023-09-27T23:30:29.045955+0000","last_became_peered":"2023-09-27T23:30:29.045955+0000","last_unstale":"2023-09-27T23:30:29.445158+0000","last_undegraded":"2023-09-27T23:30:29.445158+0000","last_fullsized":"2023-09-27T23:30:29.445158+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:39:54.902280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402923+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:30:29.402923+0000","last_peered":"2023-09-27T23:30:29.402923+0000","last_clean":"2023-09-27T23:30:29.402923+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:30:29.402923+0000","last_undegraded":"2023-09-27T23:30:29.402923+0000","last_fullsized":"2023-09-27T23:30:29.402923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979214+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:30:24.979214+0000","last_peered":"2023-09-27T23:30:24.979214+0000","last_clean":"2023-09-27T23:30:24.979214+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:30:24.979214+0000","last_undegraded":"2023-09-27T23:30:24.979214+0000","last_fullsized":"2023-09-27T23:30:24.979214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401712+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:30:29.401712+0000","last_peered":"2023-09-27T23:30:29.401712+0000","last_clean":"2023-09-27T23:30:29.401712+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:30:29.401712+0000","last_undegraded":"2023-09-27T23:30:29.401712+0000","last_fullsized":"2023-09-27T23:30:29.401712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.861948+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:30:24.861948+0000","last_peered":"2023-09-27T23:30:24.861948+0000","last_clean":"2023-09-27T23:30:24.861948+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:30:24.861948+0000","last_undegraded":"2023-09-27T23:30:24.861948+0000","last_fullsized":"2023-09-27T23:30:24.861948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":14,"acting":14,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26492,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":24576,"kb_avail":94345348,"statfs":{"total":96636764160,"available":96609636352,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25165824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.877,"15min":0.91100000000000003},"min":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.712,"15min":1.712},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.89900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.337,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.256}]},{"osd":2,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.56599999999999995,"5min":0.307,"15min":0.307},"max":{"1min":1.3129999999999999,"5min":2.681,"15min":2.681},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.3520000000000001,"5min":1.718,"15min":1.718},"last":1.194}]},{"osd":3,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1000000000000001,"5min":1.29,"15min":1.421},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.60499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3240000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.145},{"interface":"front","average":{"1min":0.997,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.69599999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2450000000000001,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0469999999999999,"15min":1.0289999999999999},"min":{"1min":0.79600000000000004,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.292},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.099},"min":{"1min":0.90300000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7190000000000001,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.359}]},{"osd":6,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.169,"15min":1.131},"min":{"1min":0.97299999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.145,"15min":1.1279999999999999},"min":{"1min":0.97799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4139999999999999,"5min":2.548,"15min":2.548},"last":1.3280000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22060,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":20736,"kb_avail":94349780,"statfs":{"total":96636764160,"available":96614174720,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21233664},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.55500000000000005,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1279999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.022,"15min":1.0209999999999999},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2230000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.88200000000000001},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.248,"5min":1.2889999999999999,"15min":1.323},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88900000000000001,"15min":0.89800000000000002},"min":{"1min":0.60799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.153}]},{"osd":2,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85999999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.222,"15min":1.353},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.68799999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94299999999999995,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.734},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.256,"5min":1.278,"15min":1.4970000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.61299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.046},{"interface":"front","average":{"1min":1.004,"5min":1.0369999999999999,"15min":1.0309999999999999},"min":{"1min":0.65100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5980000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97699999999999998,"15min":0.97499999999999998},"min":{"1min":0.68700000000000006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.02},{"interface":"front","average":{"1min":1.163,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3740000000000001,"5min":1.4259999999999999,"15min":1.49},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.143},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6719999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.147},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.6619999999999999},"last":1.423}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36216,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335624,"statfs":{"total":96636764160,"available":96599678976,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1519999999999999,"5min":1.1890000000000001,"15min":1.2909999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.27,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.2969999999999999,"15min":1.526},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.877,"15min":0.89000000000000001},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":1.331},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.52900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97999999999999998,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.276,"5min":1.369,"15min":1.5740000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2290000000000001,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.1100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.88700000000000001},"min":{"1min":0.45200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.222,"5min":1.341,"15min":1.341},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.70599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3799999999999999,"5min":1.429,"15min":1.5580000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3520000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.04}]},{"osd":7,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.123,"15min":1.123},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.6679999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2050000000000001,"15min":1.1890000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.633},"last":1.3089999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22524,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349316,"statfs":{"total":96636764160,"available":96613699584,"internally_reserved":0,"allocated":1429504,"data_stored":729891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2230000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.1639999999999999,"5min":1.45,"15min":1.45},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.52900000000000003},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.3919999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.329},"last":1.032}]},{"osd":2,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.2430000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.331,"5min":1.3660000000000001,"15min":1.407},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.256,"15min":1.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.018},"min":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":1.1439999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.90600000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.57599999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.78400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0489999999999999,"15min":1.0329999999999999},"min":{"1min":0.77300000000000002,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.149},"min":{"1min":0.97599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.514,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.236}]},{"osd":7,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.117,"15min":1.091},"min":{"1min":0.86899999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4770000000000001,"5min":1.627,"15min":1.627},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1759999999999999,"15min":1.155},"min":{"1min":0.97199999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.2609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22764,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94349076,"statfs":{"total":96636764160,"available":96613453824,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":1.1930000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.012},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2230000000000001,"5min":1.393,"15min":1.393},"last":0.874}]},{"osd":1,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.323,"15min":1.3440000000000001},"last":0.501},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.96899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0720000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.4810000000000001,"15min":1.571},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.127,"15min":1.153},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.714,"15min":1.764},"last":1.1950000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1000000000000001,"15min":1.113},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2829999999999999,"5min":1.5329999999999999,"15min":1.587},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.2130000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.1459999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2130000000000001,"15min":1.232},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4119999999999999,"5min":1.655,"15min":1.655},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.177,"15min":1.196},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.556,"5min":1.629,"15min":1.671},"last":1.0780000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509643,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21368,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350472,"statfs":{"total":96636764160,"available":96614883328,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97899999999999998,"5min":1.093,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.903,"15min":1.903},"last":0.99299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.499,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.105,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.439,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.032,"5min":1.391,"15min":1.391},"last":0.75700000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78900000000000003,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.444,"15min":1.444},"last":0.91800000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0369999999999999,"15min":1.036},"min":{"1min":0.751,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4239999999999999,"5min":1.4319999999999999,"15min":1.585},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5620000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.238}]},{"osd":5,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.032,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4339999999999999,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.397,"5min":1.5169999999999999,"15min":1.645},"last":1.319}]},{"osd":6,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1619999999999999,"15min":1.163},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.45,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.1679999999999999,"15min":1.159},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4359999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.052}]},{"osd":7,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.69699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.663,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.347},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2170000000000001},"min":{"1min":0.79700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.502,"5min":2.073,"15min":2.073},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542350,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":35520,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1409024,"data_stored":728231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36372480},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.123,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95799999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.56699999999999995,"5min":0.48199999999999998,"15min":0.45800000000000002},"max":{"1min":0.81999999999999995,"5min":1.2490000000000001,"15min":1.321},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1599999999999999,"5min":1.696,"15min":1.696},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.97599999999999998,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.431,"5min":1.464,"15min":1.464},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1180000000000001,"15min":1.127},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4410000000000001,"5min":1.5369999999999999,"15min":1.774},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.425,"5min":1.546,"15min":1.623},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1020000000000001,"15min":1.0980000000000001},"min":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88900000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.22,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.23,"5min":1.1890000000000001,"15min":1.1899999999999999},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.93400000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.216,"15min":1.23},"min":{"1min":0.88300000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5609999999999999,"5min":1.877,"15min":1.877},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2030000000000001,"15min":1.2170000000000001},"min":{"1min":0.95299999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.55,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23420,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":21504,"kb_avail":94348420,"statfs":{"total":96636764160,"available":96612782080,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22020096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":1.3,"15min":1.3},"last":0.69299999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.45600000000000002},"max":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99099999999999999,"5min":1.228,"15min":1.232},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.83999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.117,"5min":1.216,"15min":1.2769999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.47},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.995,"5min":1.022,"15min":1.0389999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.357,"5min":1.5589999999999999,"15min":1.587},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.081,"15min":1.091},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.272,"5min":1.5009999999999999,"15min":1.78},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.113,"15min":1.137},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":4.6779999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.147,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.4370000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.202,"15min":1.2150000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.73,"15min":1.8300000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2170000000000001,"15min":1.238},"min":{"1min":0.78500000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1919999999999999,"15min":1.222},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.5660000000000001,"5min":1.798,"15min":1.798},"last":0.96699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:32.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:30:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 Processor -- start 2023-09-27T23:30:32.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 -- start start 2023-09-27T23:30:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23ac0c7f00 0x7f23ac0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac06d990 con 0x7f23ac06de20 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac15a570 con 0x7f23ac0c7f00 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23b0aa2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac06cd80 con 0x7f23ac155960 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23abfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23ac0c7f00 0x7f23ac0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23aaffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f23ac06de20 0x7f23ac0c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45770/0 (socket says 172.21.15.64:45770) 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23aaffd700 1 -- 172.21.15.64:0/486125631 learned_addr learned my addr 172.21.15.64:0/486125631 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23ab7fe700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23abfff700 1 -- 172.21.15.64:0/486125631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 msgr2=0x7f23ac15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23abfff700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23abfff700 1 -- 172.21.15.64:0/486125631 >> v1:172.21.15.64:6789/0 conn(0x7f23ac06de20 legacy=0x7f23ac0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23aa7fc700 1 -- 172.21.15.64:0/486125631 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152105573 0 0) 0x7f23ac06d990 con 0x7f23ac06de20 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23abfff700 1 -- 172.21.15.64:0/486125631 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac06d060 con 0x7f23ac0c7f00 2023-09-27T23:30:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23ab7fe700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23abfff700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac0aca10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009e60 tx=0x7f23a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc78f4177fe1f785 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23aa7fc700 1 -- 172.21.15.64:0/486125631 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400e070 con 0x7f23ac0c7f00 2023-09-27T23:30:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23aa7fc700 1 -- 172.21.15.64:0/486125631 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a400bc30 con 0x7f23ac0c7f00 2023-09-27T23:30:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.543+0000 7f23aa7fc700 1 -- 172.21.15.64:0/486125631 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4012430 con 0x7f23ac0c7f00 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- 172.21.15.64:0/486125631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 msgr2=0x7f23ac0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac0aca10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009e60 tx=0x7f23a4009780 comp rx=0 tx=0).stop 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- 172.21.15.64:0/486125631 shutdown_connections 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac0aca10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/486125631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- 172.21.15.64:0/486125631 >> 172.21.15.64:0/486125631 conn(0x7f23ac067bb0 msgr2=0x7f23ac0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- 172.21.15.64:0/486125631 shutdown_connections 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- 172.21.15.64:0/486125631 wait complete. 2023-09-27T23:30:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 Processor -- start 2023-09-27T23:30:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- start start 2023-09-27T23:30:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23ac06de20 0x7f23ac0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac152970 con 0x7f23ac0c7f00 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac151d60 con 0x7f23ac06de20 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23b0aa2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac152040 con 0x7f23ac155960 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40340/0 (socket says 172.21.15.64:40340) 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 -- 172.21.15.64:0/3280943103 learned_addr learned my addr 172.21.15.64:0/3280943103 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23abfff700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23ab7fe700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23ac06de20 0x7f23ac0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 -- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 msgr2=0x7f23ac153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 -- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23ac06de20 msgr2=0x7f23ac0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23ac06de20 0x7f23ac0b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23aaffd700 1 -- 172.21.15.64:0/3280943103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4009a20 con 0x7f23ac0c7f00 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.547+0000 7f23ab7fe700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23ac06de20 0x7f23ac0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23aaffd700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac152e00 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f239400c7e0 tx=0x7f2394007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394005b30 con 0x7f23ac0c7f00 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394005c90 con 0x7f23ac0c7f00 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac152320 con 0x7f23ac0c7f00 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394007300 con 0x7f23ac0c7f00 2023-09-27T23:30:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac0b1540 con 0x7f23ac0c7f00 2023-09-27T23:30:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac152ae0 con 0x7f23ac0c7f00 2023-09-27T23:30:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f23940076b0 con 0x7f23ac0c7f00 2023-09-27T23:30:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23a37fe700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f238c060a70 0x7f238c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f239407c6e0 con 0x7f23ac0c7f00 2023-09-27T23:30:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.551+0000 7f23ab7fe700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f238c060a70 0x7f238c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.555+0000 7f23ab7fe700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f238c060a70 0x7f238c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f239c006fd0 tx=0x7f239c008040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.559+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f239404c710 con 0x7f23ac0c7f00 2023-09-27T23:30:32.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.723+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23ac04be40 con 0x7f238c060a70 2023-09-27T23:30:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.739+0000 7f23a37fe700 1 -- 172.21.15.64:0/3280943103 <== mgr.6578 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296656 (secure 0 0 0) 0x7f23ac04be40 con 0x7f238c060a70 2023-09-27T23:30:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f238c060a70 msgr2=0x7f238c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f238c060a70 0x7f238c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f239c006fd0 tx=0x7f239c008040 comp rx=0 tx=0).stop 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 msgr2=0x7f23ac152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac152e00 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f239400c7e0 tx=0x7f2394007df0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 shutdown_connections 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f238c060a70 0x7f238c062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23ac155960 0x7f23ac153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23ac0c7f00 0x7f23ac152e00 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 --2- 172.21.15.64:0/3280943103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23ac06de20 0x7f23ac0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 >> 172.21.15.64:0/3280943103 conn(0x7f23ac067bb0 msgr2=0x7f23ac0ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 shutdown_connections 2023-09-27T23:30:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.747+0000 7f23b0aa2700 1 -- 172.21.15.64:0/3280943103 wait complete. 2023-09-27T23:30:32.751 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:30:32.765 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:30:31.936263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591904,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":340,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":211748,"kb_used_data":11876,"kb_used_omap":0,"kb_used_meta":199808,"kb_avail":754762972,"statfs":{"total":773094113280,"available":772877283328,"internally_reserved":0,"allocated":12161024,"data_stored":6828752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":204603392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001197"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437015+0000","last_change":"2023-09-27T23:30:29.424776+0000","last_active":"2023-09-27T23:30:29.437015+0000","last_peered":"2023-09-27T23:30:29.437015+0000","last_clean":"2023-09-27T23:30:29.437015+0000","last_became_active":"2023-09-27T23:30:29.424350+0000","last_became_peered":"2023-09-27T23:30:29.424350+0000","last_unstale":"2023-09-27T23:30:29.437015+0000","last_undegraded":"2023-09-27T23:30:29.437015+0000","last_fullsized":"2023-09-27T23:30:29.437015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:07.044123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437217+0000","last_change":"2023-09-27T23:30:29.431912+0000","last_active":"2023-09-27T23:30:29.437217+0000","last_peered":"2023-09-27T23:30:29.437217+0000","last_clean":"2023-09-27T23:30:29.437217+0000","last_became_active":"2023-09-27T23:30:29.431583+0000","last_became_peered":"2023-09-27T23:30:29.431583+0000","last_unstale":"2023-09-27T23:30:29.437217+0000","last_undegraded":"2023-09-27T23:30:29.437217+0000","last_fullsized":"2023-09-27T23:30:29.437217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:00:22.489672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447931+0000","last_change":"2023-09-27T23:30:29.445370+0000","last_active":"2023-09-27T23:30:29.447931+0000","last_peered":"2023-09-27T23:30:29.447931+0000","last_clean":"2023-09-27T23:30:29.447931+0000","last_became_active":"2023-09-27T23:30:29.444846+0000","last_became_peered":"2023-09-27T23:30:29.444846+0000","last_unstale":"2023-09-27T23:30:29.447931+0000","last_undegraded":"2023-09-27T23:30:29.447931+0000","last_fullsized":"2023-09-27T23:30:29.447931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:41.665511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447849+0000","last_change":"2023-09-27T23:30:29.437968+0000","last_active":"2023-09-27T23:30:29.447849+0000","last_peered":"2023-09-27T23:30:29.447849+0000","last_clean":"2023-09-27T23:30:29.447849+0000","last_became_active":"2023-09-27T23:30:29.437657+0000","last_became_peered":"2023-09-27T23:30:29.437657+0000","last_unstale":"2023-09-27T23:30:29.447849+0000","last_undegraded":"2023-09-27T23:30:29.447849+0000","last_fullsized":"2023-09-27T23:30:29.447849+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:51.757870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447819+0000","last_change":"2023-09-27T23:30:29.434868+0000","last_active":"2023-09-27T23:30:29.447819+0000","last_peered":"2023-09-27T23:30:29.447819+0000","last_clean":"2023-09-27T23:30:29.447819+0000","last_became_active":"2023-09-27T23:30:29.434622+0000","last_became_peered":"2023-09-27T23:30:29.434622+0000","last_unstale":"2023-09-27T23:30:29.447819+0000","last_undegraded":"2023-09-27T23:30:29.447819+0000","last_fullsized":"2023-09-27T23:30:29.447819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:31:18.606971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434942+0000","last_change":"2023-09-27T23:30:29.421559+0000","last_active":"2023-09-27T23:30:29.434942+0000","last_peered":"2023-09-27T23:30:29.434942+0000","last_clean":"2023-09-27T23:30:29.434942+0000","last_became_active":"2023-09-27T23:30:29.420693+0000","last_became_peered":"2023-09-27T23:30:29.420693+0000","last_unstale":"2023-09-27T23:30:29.434942+0000","last_undegraded":"2023-09-27T23:30:29.434942+0000","last_fullsized":"2023-09-27T23:30:29.434942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:11.243612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445884+0000","last_change":"2023-09-27T23:30:29.443588+0000","last_active":"2023-09-27T23:30:29.445884+0000","last_peered":"2023-09-27T23:30:29.445884+0000","last_clean":"2023-09-27T23:30:29.445884+0000","last_became_active":"2023-09-27T23:30:29.443289+0000","last_became_peered":"2023-09-27T23:30:29.443289+0000","last_unstale":"2023-09-27T23:30:29.445884+0000","last_undegraded":"2023-09-27T23:30:29.445884+0000","last_fullsized":"2023-09-27T23:30:29.445884+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:57:06.402886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445278+0000","last_change":"2023-09-27T23:30:29.434541+0000","last_active":"2023-09-27T23:30:29.445278+0000","last_peered":"2023-09-27T23:30:29.445278+0000","last_clean":"2023-09-27T23:30:29.445278+0000","last_became_active":"2023-09-27T23:30:29.434260+0000","last_became_peered":"2023-09-27T23:30:29.434260+0000","last_unstale":"2023-09-27T23:30:29.445278+0000","last_undegraded":"2023-09-27T23:30:29.445278+0000","last_fullsized":"2023-09-27T23:30:29.445278+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:26:16.902235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436939+0000","last_change":"2023-09-27T23:30:29.032760+0000","last_active":"2023-09-27T23:30:29.436939+0000","last_peered":"2023-09-27T23:30:29.436939+0000","last_clean":"2023-09-27T23:30:29.436939+0000","last_became_active":"2023-09-27T23:30:29.032431+0000","last_became_peered":"2023-09-27T23:30:29.032431+0000","last_unstale":"2023-09-27T23:30:29.436939+0000","last_undegraded":"2023-09-27T23:30:29.436939+0000","last_fullsized":"2023-09-27T23:30:29.436939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:09:55.740875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446028+0000","last_change":"2023-09-27T23:30:29.438718+0000","last_active":"2023-09-27T23:30:29.446028+0000","last_peered":"2023-09-27T23:30:29.446028+0000","last_clean":"2023-09-27T23:30:29.446028+0000","last_became_active":"2023-09-27T23:30:29.438454+0000","last_became_peered":"2023-09-27T23:30:29.438454+0000","last_unstale":"2023-09-27T23:30:29.446028+0000","last_undegraded":"2023-09-27T23:30:29.446028+0000","last_fullsized":"2023-09-27T23:30:29.446028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:58:47.626281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401580+0000","last_change":"2023-09-27T23:30:29.035856+0000","last_active":"2023-09-27T23:30:29.401580+0000","last_peered":"2023-09-27T23:30:29.401580+0000","last_clean":"2023-09-27T23:30:29.401580+0000","last_became_active":"2023-09-27T23:30:29.035384+0000","last_became_peered":"2023-09-27T23:30:29.035384+0000","last_unstale":"2023-09-27T23:30:29.401580+0000","last_undegraded":"2023-09-27T23:30:29.401580+0000","last_fullsized":"2023-09-27T23:30:29.401580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:16:29.194838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402510+0000","last_change":"2023-09-27T23:30:29.022763+0000","last_active":"2023-09-27T23:30:29.402510+0000","last_peered":"2023-09-27T23:30:29.402510+0000","last_clean":"2023-09-27T23:30:29.402510+0000","last_became_active":"2023-09-27T23:30:29.021867+0000","last_became_peered":"2023-09-27T23:30:29.021867+0000","last_unstale":"2023-09-27T23:30:29.402510+0000","last_undegraded":"2023-09-27T23:30:29.402510+0000","last_fullsized":"2023-09-27T23:30:29.402510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:37.446360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445578+0000","last_change":"2023-09-27T23:30:29.434529+0000","last_active":"2023-09-27T23:30:29.445578+0000","last_peered":"2023-09-27T23:30:29.445578+0000","last_clean":"2023-09-27T23:30:29.445578+0000","last_became_active":"2023-09-27T23:30:29.434185+0000","last_became_peered":"2023-09-27T23:30:29.434185+0000","last_unstale":"2023-09-27T23:30:29.445578+0000","last_undegraded":"2023-09-27T23:30:29.445578+0000","last_fullsized":"2023-09-27T23:30:29.445578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:06.448424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.449495+0000","last_change":"2023-09-27T23:30:29.040154+0000","last_active":"2023-09-27T23:30:29.449495+0000","last_peered":"2023-09-27T23:30:29.449495+0000","last_clean":"2023-09-27T23:30:29.449495+0000","last_became_active":"2023-09-27T23:30:29.039855+0000","last_became_peered":"2023-09-27T23:30:29.039855+0000","last_unstale":"2023-09-27T23:30:29.449495+0000","last_undegraded":"2023-09-27T23:30:29.449495+0000","last_fullsized":"2023-09-27T23:30:29.449495+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:26:42.757699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434186+0000","last_change":"2023-09-27T23:30:29.422077+0000","last_active":"2023-09-27T23:30:29.434186+0000","last_peered":"2023-09-27T23:30:29.434186+0000","last_clean":"2023-09-27T23:30:29.434186+0000","last_became_active":"2023-09-27T23:30:29.421784+0000","last_became_peered":"2023-09-27T23:30:29.421784+0000","last_unstale":"2023-09-27T23:30:29.434186+0000","last_undegraded":"2023-09-27T23:30:29.434186+0000","last_fullsized":"2023-09-27T23:30:29.434186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:59.674679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445131+0000","last_change":"2023-09-27T23:30:29.434466+0000","last_active":"2023-09-27T23:30:29.445131+0000","last_peered":"2023-09-27T23:30:29.445131+0000","last_clean":"2023-09-27T23:30:29.445131+0000","last_became_active":"2023-09-27T23:30:29.434133+0000","last_became_peered":"2023-09-27T23:30:29.434133+0000","last_unstale":"2023-09-27T23:30:29.445131+0000","last_undegraded":"2023-09-27T23:30:29.445131+0000","last_fullsized":"2023-09-27T23:30:29.445131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:46.166597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425237+0000","last_change":"2023-09-27T23:30:29.040263+0000","last_active":"2023-09-27T23:30:29.425237+0000","last_peered":"2023-09-27T23:30:29.425237+0000","last_clean":"2023-09-27T23:30:29.425237+0000","last_became_active":"2023-09-27T23:30:29.039976+0000","last_became_peered":"2023-09-27T23:30:29.039976+0000","last_unstale":"2023-09-27T23:30:29.425237+0000","last_undegraded":"2023-09-27T23:30:29.425237+0000","last_fullsized":"2023-09-27T23:30:29.425237+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:13.841678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437983+0000","last_change":"2023-09-27T23:30:29.031398+0000","last_active":"2023-09-27T23:30:29.437983+0000","last_peered":"2023-09-27T23:30:29.437983+0000","last_clean":"2023-09-27T23:30:29.437983+0000","last_became_active":"2023-09-27T23:30:29.031067+0000","last_became_peered":"2023-09-27T23:30:29.031067+0000","last_unstale":"2023-09-27T23:30:29.437983+0000","last_undegraded":"2023-09-27T23:30:29.437983+0000","last_fullsized":"2023-09-27T23:30:29.437983+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.823350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434246+0000","last_change":"2023-09-27T23:30:29.420956+0000","last_active":"2023-09-27T23:30:29.434246+0000","last_peered":"2023-09-27T23:30:29.434246+0000","last_clean":"2023-09-27T23:30:29.434246+0000","last_became_active":"2023-09-27T23:30:29.419860+0000","last_became_peered":"2023-09-27T23:30:29.419860+0000","last_unstale":"2023-09-27T23:30:29.434246+0000","last_undegraded":"2023-09-27T23:30:29.434246+0000","last_fullsized":"2023-09-27T23:30:29.434246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:10.232721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437190+0000","last_change":"2023-09-27T23:30:29.428257+0000","last_active":"2023-09-27T23:30:29.437190+0000","last_peered":"2023-09-27T23:30:29.437190+0000","last_clean":"2023-09-27T23:30:29.437190+0000","last_became_active":"2023-09-27T23:30:29.427972+0000","last_became_peered":"2023-09-27T23:30:29.427972+0000","last_unstale":"2023-09-27T23:30:29.437190+0000","last_undegraded":"2023-09-27T23:30:29.437190+0000","last_fullsized":"2023-09-27T23:30:29.437190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:15:35.465950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446021+0000","last_change":"2023-09-27T23:30:29.435122+0000","last_active":"2023-09-27T23:30:29.446021+0000","last_peered":"2023-09-27T23:30:29.446021+0000","last_clean":"2023-09-27T23:30:29.446021+0000","last_became_active":"2023-09-27T23:30:29.434814+0000","last_became_peered":"2023-09-27T23:30:29.434814+0000","last_unstale":"2023-09-27T23:30:29.446021+0000","last_undegraded":"2023-09-27T23:30:29.446021+0000","last_fullsized":"2023-09-27T23:30:29.446021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:54.480949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"103'205","reported_seq":556,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979261+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:30:24.979261+0000","last_peered":"2023-09-27T23:30:24.979261+0000","last_clean":"2023-09-27T23:30:24.979261+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:30:24.979261+0000","last_undegraded":"2023-09-27T23:30:24.979261+0000","last_fullsized":"2023-09-27T23:30:24.979261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979599+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:30:24.979599+0000","last_peered":"2023-09-27T23:30:24.979599+0000","last_clean":"2023-09-27T23:30:24.979599+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:30:24.979599+0000","last_undegraded":"2023-09-27T23:30:24.979599+0000","last_fullsized":"2023-09-27T23:30:24.979599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974626+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:30:24.974626+0000","last_peered":"2023-09-27T23:30:24.974626+0000","last_clean":"2023-09-27T23:30:24.974626+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:30:24.974626+0000","last_undegraded":"2023-09-27T23:30:24.974626+0000","last_fullsized":"2023-09-27T23:30:24.974626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445158+0000","last_change":"2023-09-27T23:30:29.046235+0000","last_active":"2023-09-27T23:30:29.445158+0000","last_peered":"2023-09-27T23:30:29.445158+0000","last_clean":"2023-09-27T23:30:29.445158+0000","last_became_active":"2023-09-27T23:30:29.045955+0000","last_became_peered":"2023-09-27T23:30:29.045955+0000","last_unstale":"2023-09-27T23:30:29.445158+0000","last_undegraded":"2023-09-27T23:30:29.445158+0000","last_fullsized":"2023-09-27T23:30:29.445158+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:39:54.902280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402923+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:30:29.402923+0000","last_peered":"2023-09-27T23:30:29.402923+0000","last_clean":"2023-09-27T23:30:29.402923+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:30:29.402923+0000","last_undegraded":"2023-09-27T23:30:29.402923+0000","last_fullsized":"2023-09-27T23:30:29.402923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979214+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:30:24.979214+0000","last_peered":"2023-09-27T23:30:24.979214+0000","last_clean":"2023-09-27T23:30:24.979214+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:30:24.979214+0000","last_undegraded":"2023-09-27T23:30:24.979214+0000","last_fullsized":"2023-09-27T23:30:24.979214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401712+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:30:29.401712+0000","last_peered":"2023-09-27T23:30:29.401712+0000","last_clean":"2023-09-27T23:30:29.401712+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:30:29.401712+0000","last_undegraded":"2023-09-27T23:30:29.401712+0000","last_fullsized":"2023-09-27T23:30:29.401712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.861948+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:30:24.861948+0000","last_peered":"2023-09-27T23:30:24.861948+0000","last_clean":"2023-09-27T23:30:24.861948+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:30:24.861948+0000","last_undegraded":"2023-09-27T23:30:24.861948+0000","last_fullsized":"2023-09-27T23:30:24.861948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":14,"acting":14,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26492,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":24576,"kb_avail":94345348,"statfs":{"total":96636764160,"available":96609636352,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25165824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.877,"15min":0.91100000000000003},"min":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.712,"15min":1.712},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.89900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.337,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.256}]},{"osd":2,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.56599999999999995,"5min":0.307,"15min":0.307},"max":{"1min":1.3129999999999999,"5min":2.681,"15min":2.681},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.3520000000000001,"5min":1.718,"15min":1.718},"last":1.194}]},{"osd":3,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1000000000000001,"5min":1.29,"15min":1.421},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.60499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3240000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.145},{"interface":"front","average":{"1min":0.997,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.69599999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2450000000000001,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0469999999999999,"15min":1.0289999999999999},"min":{"1min":0.79600000000000004,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.292},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.099},"min":{"1min":0.90300000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7190000000000001,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.359}]},{"osd":6,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.169,"15min":1.131},"min":{"1min":0.97299999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.145,"15min":1.1279999999999999},"min":{"1min":0.97799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4139999999999999,"5min":2.548,"15min":2.548},"last":1.3280000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22060,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":20736,"kb_avail":94349780,"statfs":{"total":96636764160,"available":96614174720,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21233664},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.55500000000000005,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1279999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.022,"15min":1.0209999999999999},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2230000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.88200000000000001},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.248,"5min":1.2889999999999999,"15min":1.323},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88900000000000001,"15min":0.89800000000000002},"min":{"1min":0.60799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.153}]},{"osd":2,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85999999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.222,"15min":1.353},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.68799999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94299999999999995,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.734},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.256,"5min":1.278,"15min":1.4970000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.61299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.046},{"interface":"front","average":{"1min":1.004,"5min":1.0369999999999999,"15min":1.0309999999999999},"min":{"1min":0.65100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5980000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97699999999999998,"15min":0.97499999999999998},"min":{"1min":0.68700000000000006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.02},{"interface":"front","average":{"1min":1.163,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3740000000000001,"5min":1.4259999999999999,"15min":1.49},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.143},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6719999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.147},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.6619999999999999},"last":1.423}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36216,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335624,"statfs":{"total":96636764160,"available":96599678976,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1519999999999999,"5min":1.1890000000000001,"15min":1.2909999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.27,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.2969999999999999,"15min":1.526},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.877,"15min":0.89000000000000001},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":1.331},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.52900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97999999999999998,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.276,"5min":1.369,"15min":1.5740000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2290000000000001,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.1100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.88700000000000001},"min":{"1min":0.45200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.222,"5min":1.341,"15min":1.341},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.70599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3799999999999999,"5min":1.429,"15min":1.5580000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3520000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.04}]},{"osd":7,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.123,"15min":1.123},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.6679999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2050000000000001,"15min":1.1890000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.633},"last":1.3089999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22524,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349316,"statfs":{"total":96636764160,"available":96613699584,"internally_reserved":0,"allocated":1429504,"data_stored":729891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2230000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.1639999999999999,"5min":1.45,"15min":1.45},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.52900000000000003},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.3919999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.329},"last":1.032}]},{"osd":2,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.2430000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.331,"5min":1.3660000000000001,"15min":1.407},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.256,"15min":1.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.018},"min":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":1.1439999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.90600000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.57599999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.78400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0489999999999999,"15min":1.0329999999999999},"min":{"1min":0.77300000000000002,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.149},"min":{"1min":0.97599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.514,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.236}]},{"osd":7,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.117,"15min":1.091},"min":{"1min":0.86899999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4770000000000001,"5min":1.627,"15min":1.627},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1759999999999999,"15min":1.155},"min":{"1min":0.97199999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.2609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22764,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94349076,"statfs":{"total":96636764160,"available":96613453824,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":1.1930000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.012},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2230000000000001,"5min":1.393,"15min":1.393},"last":0.874}]},{"osd":1,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.323,"15min":1.3440000000000001},"last":0.501},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.96899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0720000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.4810000000000001,"15min":1.571},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.127,"15min":1.153},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.714,"15min":1.764},"last":1.1950000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1000000000000001,"15min":1.113},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2829999999999999,"5min":1.5329999999999999,"15min":1.587},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.2130000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.1459999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2130000000000001,"15min":1.232},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4119999999999999,"5min":1.655,"15min":1.655},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.177,"15min":1.196},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.556,"5min":1.629,"15min":1.671},"last":1.0780000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509643,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21368,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350472,"statfs":{"total":96636764160,"available":96614883328,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97899999999999998,"5min":1.093,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.903,"15min":1.903},"last":0.99299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.499,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.105,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.439,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.032,"5min":1.391,"15min":1.391},"last":0.75700000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78900000000000003,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.444,"15min":1.444},"last":0.91800000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0369999999999999,"15min":1.036},"min":{"1min":0.751,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4239999999999999,"5min":1.4319999999999999,"15min":1.585},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5620000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.238}]},{"osd":5,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.032,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4339999999999999,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.397,"5min":1.5169999999999999,"15min":1.645},"last":1.319}]},{"osd":6,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1619999999999999,"15min":1.163},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.45,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.1679999999999999,"15min":1.159},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4359999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.052}]},{"osd":7,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.69699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.663,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.347},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2170000000000001},"min":{"1min":0.79700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.502,"5min":2.073,"15min":2.073},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542350,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":35520,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1409024,"data_stored":728231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36372480},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.123,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95799999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.56699999999999995,"5min":0.48199999999999998,"15min":0.45800000000000002},"max":{"1min":0.81999999999999995,"5min":1.2490000000000001,"15min":1.321},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1599999999999999,"5min":1.696,"15min":1.696},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.97599999999999998,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.431,"5min":1.464,"15min":1.464},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1180000000000001,"15min":1.127},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4410000000000001,"5min":1.5369999999999999,"15min":1.774},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.425,"5min":1.546,"15min":1.623},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1020000000000001,"15min":1.0980000000000001},"min":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88900000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.22,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.23,"5min":1.1890000000000001,"15min":1.1899999999999999},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.93400000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.216,"15min":1.23},"min":{"1min":0.88300000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5609999999999999,"5min":1.877,"15min":1.877},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2030000000000001,"15min":1.2170000000000001},"min":{"1min":0.95299999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.55,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23420,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":21504,"kb_avail":94348420,"statfs":{"total":96636764160,"available":96612782080,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22020096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":1.3,"15min":1.3},"last":0.69299999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.45600000000000002},"max":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99099999999999999,"5min":1.228,"15min":1.232},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.83999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.117,"5min":1.216,"15min":1.2769999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.47},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.995,"5min":1.022,"15min":1.0389999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.357,"5min":1.5589999999999999,"15min":1.587},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.081,"15min":1.091},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.272,"5min":1.5009999999999999,"15min":1.78},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.113,"15min":1.137},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":4.6779999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.147,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.4370000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.202,"15min":1.2150000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.73,"15min":1.8300000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2170000000000001,"15min":1.238},"min":{"1min":0.78500000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1919999999999999,"15min":1.222},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.5660000000000001,"5min":1.798,"15min":1.798},"last":0.96699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:32.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:30:32.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 Processor -- start 2023-09-27T23:30:32.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 -- start start 2023-09-27T23:30:32.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140ae0 0x7f1fa00680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fa0068630 0x7f1fa00ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa006e150 con 0x7f1fa0140710 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa0068ab0 con 0x7f1fa0068630 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1fa5ab8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006d540 con 0x7f1fa0140ae0 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1fa0140710 0x7f1fa00c70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45776/0 (socket says 172.21.15.64:45776) 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9f7fe700 1 -- 172.21.15.64:0/1656055122 learned_addr learned my addr 172.21.15.64:0/1656055122 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9ffff700 1 --2- 172.21.15.64:0/1656055122 >> v2:172.21.15.64:3300/0 conn(0x7f1fa0068630 0x7f1fa00ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9effd700 1 --2- 172.21.15.64:0/1656055122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140ae0 0x7f1fa00680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9ffff700 1 -- 172.21.15.64:0/1656055122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140ae0 msgr2=0x7f1fa00680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9ffff700 1 --2- 172.21.15.64:0/1656055122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140ae0 0x7f1fa00680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9ffff700 1 -- 172.21.15.64:0/1656055122 >> v1:172.21.15.64:6789/0 conn(0x7f1fa0140710 legacy=0x7f1fa00c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9ffff700 1 -- 172.21.15.64:0/1656055122 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa006d820 con 0x7f1fa0068630 2023-09-27T23:30:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9ffff700 1 --2- 172.21.15.64:0/1656055122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa00ac250 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009e60 tx=0x7f1f98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05b9b59dafdbec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9e7fc700 1 -- 172.21.15.64:0/1656055122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800e070 con 0x7f1fa0068630 2023-09-27T23:30:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9e7fc700 1 -- 172.21.15.64:0/1656055122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9800bb60 con 0x7f1fa0068630 2023-09-27T23:30:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.915+0000 7f1f9e7fc700 1 -- 172.21.15.64:0/1656055122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98012430 con 0x7f1fa0068630 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1656055122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 msgr2=0x7f1fa00ac250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1656055122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa00ac250 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009e60 tx=0x7f1f98009780 comp rx=0 tx=0).stop 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1656055122 shutdown_connections 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1656055122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa00ac250 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1656055122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140ae0 0x7f1fa00680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1656055122 >> 172.21.15.64:0/1656055122 conn(0x7f1fa00c8850 msgr2=0x7f1fa00c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1656055122 shutdown_connections 2023-09-27T23:30:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1656055122 wait complete. 2023-09-27T23:30:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 Processor -- start 2023-09-27T23:30:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- start start 2023-09-27T23:30:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140710 0x7f1fa00aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa0140ae0 0x7f1fa00af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00c2e90 con 0x7f1fa0068630 2023-09-27T23:30:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00a9f50 con 0x7f1fa0140ae0 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1fa5ab8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00a9340 con 0x7f1fa0140710 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa0156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40354/0 (socket says 172.21.15.64:40354) 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9f7fe700 1 -- 172.21.15.64:0/1642243235 learned_addr learned my addr 172.21.15.64:0/1642243235 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9effd700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140710 0x7f1fa00aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9ffff700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa0140ae0 0x7f1fa00af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9f7fe700 1 -- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140710 msgr2=0x7f1fa00aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9f7fe700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140710 0x7f1fa00aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.919+0000 7f1f9f7fe700 1 -- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa0140ae0 msgr2=0x7f1fa00af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f9f7fe700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa0140ae0 0x7f1fa00af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f9f7fe700 1 -- 172.21.15.64:0/1642243235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009a20 con 0x7f1fa0068630 2023-09-27T23:30:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f9f7fe700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa0156950 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400c590 tx=0x7f1f94007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94005b30 con 0x7f1fa0068630 2023-09-27T23:30:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94005cb0 con 0x7f1fa0068630 2023-09-27T23:30:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa00a9620 con 0x7f1fa0068630 2023-09-27T23:30:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94007320 con 0x7f1fa0068630 2023-09-27T23:30:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa00a9be0 con 0x7f1fa0068630 2023-09-27T23:30:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa0071120 con 0x7f1fa0068630 2023-09-27T23:30:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1f9401b3b0 con 0x7f1fa0068630 2023-09-27T23:30:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f877fe700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f8c060aa0 0x7f1f8c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.923+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1f9407c3e0 con 0x7f1fa0068630 2023-09-27T23:30:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.927+0000 7f1f9effd700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f8c060aa0 0x7f1f8c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.927+0000 7f1f9effd700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f8c060aa0 0x7f1f8c062f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00a8e20 tx=0x7f1f900091a0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:32.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:32.931+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1f9404c410 con 0x7f1fa0068630 2023-09-27T23:30:33.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.091+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fa00aa090 con 0x7f1f8c060aa0 2023-09-27T23:30:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.107+0000 7f1f877fe700 1 -- 172.21.15.64:0/1642243235 <== mgr.6578 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296656 (secure 0 0 0) 0x7f1f90011a20 con 0x7f1f8c060aa0 2023-09-27T23:30:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f8c060aa0 msgr2=0x7f1f8c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f8c060aa0 0x7f1f8c062f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00a8e20 tx=0x7f1f900091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 msgr2=0x7f1fa0156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa0156950 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400c590 tx=0x7f1f94007f90 comp rx=0 tx=0).stop 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 shutdown_connections 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f8c060aa0 0x7f1f8c062f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa0140ae0 0x7f1fa00af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa0140710 0x7f1fa00aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 --2- 172.21.15.64:0/1642243235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa0068630 0x7f1fa0156950 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.111+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 >> 172.21.15.64:0/1642243235 conn(0x7f1fa00c8850 msgr2=0x7f1fa00c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.115+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 shutdown_connections 2023-09-27T23:30:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.115+0000 7f1fa5ab8700 1 -- 172.21.15.64:0/1642243235 wait complete. 2023-09-27T23:30:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:30:33.130 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:30:31.936263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591904,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":340,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":211748,"kb_used_data":11876,"kb_used_omap":0,"kb_used_meta":199808,"kb_avail":754762972,"statfs":{"total":773094113280,"available":772877283328,"internally_reserved":0,"allocated":12161024,"data_stored":6828752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":204603392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001197"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437015+0000","last_change":"2023-09-27T23:30:29.424776+0000","last_active":"2023-09-27T23:30:29.437015+0000","last_peered":"2023-09-27T23:30:29.437015+0000","last_clean":"2023-09-27T23:30:29.437015+0000","last_became_active":"2023-09-27T23:30:29.424350+0000","last_became_peered":"2023-09-27T23:30:29.424350+0000","last_unstale":"2023-09-27T23:30:29.437015+0000","last_undegraded":"2023-09-27T23:30:29.437015+0000","last_fullsized":"2023-09-27T23:30:29.437015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:07.044123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437217+0000","last_change":"2023-09-27T23:30:29.431912+0000","last_active":"2023-09-27T23:30:29.437217+0000","last_peered":"2023-09-27T23:30:29.437217+0000","last_clean":"2023-09-27T23:30:29.437217+0000","last_became_active":"2023-09-27T23:30:29.431583+0000","last_became_peered":"2023-09-27T23:30:29.431583+0000","last_unstale":"2023-09-27T23:30:29.437217+0000","last_undegraded":"2023-09-27T23:30:29.437217+0000","last_fullsized":"2023-09-27T23:30:29.437217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:00:22.489672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447931+0000","last_change":"2023-09-27T23:30:29.445370+0000","last_active":"2023-09-27T23:30:29.447931+0000","last_peered":"2023-09-27T23:30:29.447931+0000","last_clean":"2023-09-27T23:30:29.447931+0000","last_became_active":"2023-09-27T23:30:29.444846+0000","last_became_peered":"2023-09-27T23:30:29.444846+0000","last_unstale":"2023-09-27T23:30:29.447931+0000","last_undegraded":"2023-09-27T23:30:29.447931+0000","last_fullsized":"2023-09-27T23:30:29.447931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:41.665511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447849+0000","last_change":"2023-09-27T23:30:29.437968+0000","last_active":"2023-09-27T23:30:29.447849+0000","last_peered":"2023-09-27T23:30:29.447849+0000","last_clean":"2023-09-27T23:30:29.447849+0000","last_became_active":"2023-09-27T23:30:29.437657+0000","last_became_peered":"2023-09-27T23:30:29.437657+0000","last_unstale":"2023-09-27T23:30:29.447849+0000","last_undegraded":"2023-09-27T23:30:29.447849+0000","last_fullsized":"2023-09-27T23:30:29.447849+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:51.757870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447819+0000","last_change":"2023-09-27T23:30:29.434868+0000","last_active":"2023-09-27T23:30:29.447819+0000","last_peered":"2023-09-27T23:30:29.447819+0000","last_clean":"2023-09-27T23:30:29.447819+0000","last_became_active":"2023-09-27T23:30:29.434622+0000","last_became_peered":"2023-09-27T23:30:29.434622+0000","last_unstale":"2023-09-27T23:30:29.447819+0000","last_undegraded":"2023-09-27T23:30:29.447819+0000","last_fullsized":"2023-09-27T23:30:29.447819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:31:18.606971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434942+0000","last_change":"2023-09-27T23:30:29.421559+0000","last_active":"2023-09-27T23:30:29.434942+0000","last_peered":"2023-09-27T23:30:29.434942+0000","last_clean":"2023-09-27T23:30:29.434942+0000","last_became_active":"2023-09-27T23:30:29.420693+0000","last_became_peered":"2023-09-27T23:30:29.420693+0000","last_unstale":"2023-09-27T23:30:29.434942+0000","last_undegraded":"2023-09-27T23:30:29.434942+0000","last_fullsized":"2023-09-27T23:30:29.434942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:11.243612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445884+0000","last_change":"2023-09-27T23:30:29.443588+0000","last_active":"2023-09-27T23:30:29.445884+0000","last_peered":"2023-09-27T23:30:29.445884+0000","last_clean":"2023-09-27T23:30:29.445884+0000","last_became_active":"2023-09-27T23:30:29.443289+0000","last_became_peered":"2023-09-27T23:30:29.443289+0000","last_unstale":"2023-09-27T23:30:29.445884+0000","last_undegraded":"2023-09-27T23:30:29.445884+0000","last_fullsized":"2023-09-27T23:30:29.445884+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:57:06.402886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445278+0000","last_change":"2023-09-27T23:30:29.434541+0000","last_active":"2023-09-27T23:30:29.445278+0000","last_peered":"2023-09-27T23:30:29.445278+0000","last_clean":"2023-09-27T23:30:29.445278+0000","last_became_active":"2023-09-27T23:30:29.434260+0000","last_became_peered":"2023-09-27T23:30:29.434260+0000","last_unstale":"2023-09-27T23:30:29.445278+0000","last_undegraded":"2023-09-27T23:30:29.445278+0000","last_fullsized":"2023-09-27T23:30:29.445278+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:26:16.902235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436939+0000","last_change":"2023-09-27T23:30:29.032760+0000","last_active":"2023-09-27T23:30:29.436939+0000","last_peered":"2023-09-27T23:30:29.436939+0000","last_clean":"2023-09-27T23:30:29.436939+0000","last_became_active":"2023-09-27T23:30:29.032431+0000","last_became_peered":"2023-09-27T23:30:29.032431+0000","last_unstale":"2023-09-27T23:30:29.436939+0000","last_undegraded":"2023-09-27T23:30:29.436939+0000","last_fullsized":"2023-09-27T23:30:29.436939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:09:55.740875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446028+0000","last_change":"2023-09-27T23:30:29.438718+0000","last_active":"2023-09-27T23:30:29.446028+0000","last_peered":"2023-09-27T23:30:29.446028+0000","last_clean":"2023-09-27T23:30:29.446028+0000","last_became_active":"2023-09-27T23:30:29.438454+0000","last_became_peered":"2023-09-27T23:30:29.438454+0000","last_unstale":"2023-09-27T23:30:29.446028+0000","last_undegraded":"2023-09-27T23:30:29.446028+0000","last_fullsized":"2023-09-27T23:30:29.446028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:58:47.626281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401580+0000","last_change":"2023-09-27T23:30:29.035856+0000","last_active":"2023-09-27T23:30:29.401580+0000","last_peered":"2023-09-27T23:30:29.401580+0000","last_clean":"2023-09-27T23:30:29.401580+0000","last_became_active":"2023-09-27T23:30:29.035384+0000","last_became_peered":"2023-09-27T23:30:29.035384+0000","last_unstale":"2023-09-27T23:30:29.401580+0000","last_undegraded":"2023-09-27T23:30:29.401580+0000","last_fullsized":"2023-09-27T23:30:29.401580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:16:29.194838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402510+0000","last_change":"2023-09-27T23:30:29.022763+0000","last_active":"2023-09-27T23:30:29.402510+0000","last_peered":"2023-09-27T23:30:29.402510+0000","last_clean":"2023-09-27T23:30:29.402510+0000","last_became_active":"2023-09-27T23:30:29.021867+0000","last_became_peered":"2023-09-27T23:30:29.021867+0000","last_unstale":"2023-09-27T23:30:29.402510+0000","last_undegraded":"2023-09-27T23:30:29.402510+0000","last_fullsized":"2023-09-27T23:30:29.402510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:37.446360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445578+0000","last_change":"2023-09-27T23:30:29.434529+0000","last_active":"2023-09-27T23:30:29.445578+0000","last_peered":"2023-09-27T23:30:29.445578+0000","last_clean":"2023-09-27T23:30:29.445578+0000","last_became_active":"2023-09-27T23:30:29.434185+0000","last_became_peered":"2023-09-27T23:30:29.434185+0000","last_unstale":"2023-09-27T23:30:29.445578+0000","last_undegraded":"2023-09-27T23:30:29.445578+0000","last_fullsized":"2023-09-27T23:30:29.445578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:06.448424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.449495+0000","last_change":"2023-09-27T23:30:29.040154+0000","last_active":"2023-09-27T23:30:29.449495+0000","last_peered":"2023-09-27T23:30:29.449495+0000","last_clean":"2023-09-27T23:30:29.449495+0000","last_became_active":"2023-09-27T23:30:29.039855+0000","last_became_peered":"2023-09-27T23:30:29.039855+0000","last_unstale":"2023-09-27T23:30:29.449495+0000","last_undegraded":"2023-09-27T23:30:29.449495+0000","last_fullsized":"2023-09-27T23:30:29.449495+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:26:42.757699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434186+0000","last_change":"2023-09-27T23:30:29.422077+0000","last_active":"2023-09-27T23:30:29.434186+0000","last_peered":"2023-09-27T23:30:29.434186+0000","last_clean":"2023-09-27T23:30:29.434186+0000","last_became_active":"2023-09-27T23:30:29.421784+0000","last_became_peered":"2023-09-27T23:30:29.421784+0000","last_unstale":"2023-09-27T23:30:29.434186+0000","last_undegraded":"2023-09-27T23:30:29.434186+0000","last_fullsized":"2023-09-27T23:30:29.434186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:59.674679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445131+0000","last_change":"2023-09-27T23:30:29.434466+0000","last_active":"2023-09-27T23:30:29.445131+0000","last_peered":"2023-09-27T23:30:29.445131+0000","last_clean":"2023-09-27T23:30:29.445131+0000","last_became_active":"2023-09-27T23:30:29.434133+0000","last_became_peered":"2023-09-27T23:30:29.434133+0000","last_unstale":"2023-09-27T23:30:29.445131+0000","last_undegraded":"2023-09-27T23:30:29.445131+0000","last_fullsized":"2023-09-27T23:30:29.445131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:46.166597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425237+0000","last_change":"2023-09-27T23:30:29.040263+0000","last_active":"2023-09-27T23:30:29.425237+0000","last_peered":"2023-09-27T23:30:29.425237+0000","last_clean":"2023-09-27T23:30:29.425237+0000","last_became_active":"2023-09-27T23:30:29.039976+0000","last_became_peered":"2023-09-27T23:30:29.039976+0000","last_unstale":"2023-09-27T23:30:29.425237+0000","last_undegraded":"2023-09-27T23:30:29.425237+0000","last_fullsized":"2023-09-27T23:30:29.425237+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:13.841678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437983+0000","last_change":"2023-09-27T23:30:29.031398+0000","last_active":"2023-09-27T23:30:29.437983+0000","last_peered":"2023-09-27T23:30:29.437983+0000","last_clean":"2023-09-27T23:30:29.437983+0000","last_became_active":"2023-09-27T23:30:29.031067+0000","last_became_peered":"2023-09-27T23:30:29.031067+0000","last_unstale":"2023-09-27T23:30:29.437983+0000","last_undegraded":"2023-09-27T23:30:29.437983+0000","last_fullsized":"2023-09-27T23:30:29.437983+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.823350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434246+0000","last_change":"2023-09-27T23:30:29.420956+0000","last_active":"2023-09-27T23:30:29.434246+0000","last_peered":"2023-09-27T23:30:29.434246+0000","last_clean":"2023-09-27T23:30:29.434246+0000","last_became_active":"2023-09-27T23:30:29.419860+0000","last_became_peered":"2023-09-27T23:30:29.419860+0000","last_unstale":"2023-09-27T23:30:29.434246+0000","last_undegraded":"2023-09-27T23:30:29.434246+0000","last_fullsized":"2023-09-27T23:30:29.434246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:10.232721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437190+0000","last_change":"2023-09-27T23:30:29.428257+0000","last_active":"2023-09-27T23:30:29.437190+0000","last_peered":"2023-09-27T23:30:29.437190+0000","last_clean":"2023-09-27T23:30:29.437190+0000","last_became_active":"2023-09-27T23:30:29.427972+0000","last_became_peered":"2023-09-27T23:30:29.427972+0000","last_unstale":"2023-09-27T23:30:29.437190+0000","last_undegraded":"2023-09-27T23:30:29.437190+0000","last_fullsized":"2023-09-27T23:30:29.437190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:15:35.465950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446021+0000","last_change":"2023-09-27T23:30:29.435122+0000","last_active":"2023-09-27T23:30:29.446021+0000","last_peered":"2023-09-27T23:30:29.446021+0000","last_clean":"2023-09-27T23:30:29.446021+0000","last_became_active":"2023-09-27T23:30:29.434814+0000","last_became_peered":"2023-09-27T23:30:29.434814+0000","last_unstale":"2023-09-27T23:30:29.446021+0000","last_undegraded":"2023-09-27T23:30:29.446021+0000","last_fullsized":"2023-09-27T23:30:29.446021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:54.480949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"103'205","reported_seq":556,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979261+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:30:24.979261+0000","last_peered":"2023-09-27T23:30:24.979261+0000","last_clean":"2023-09-27T23:30:24.979261+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:30:24.979261+0000","last_undegraded":"2023-09-27T23:30:24.979261+0000","last_fullsized":"2023-09-27T23:30:24.979261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979599+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:30:24.979599+0000","last_peered":"2023-09-27T23:30:24.979599+0000","last_clean":"2023-09-27T23:30:24.979599+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:30:24.979599+0000","last_undegraded":"2023-09-27T23:30:24.979599+0000","last_fullsized":"2023-09-27T23:30:24.979599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974626+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:30:24.974626+0000","last_peered":"2023-09-27T23:30:24.974626+0000","last_clean":"2023-09-27T23:30:24.974626+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:30:24.974626+0000","last_undegraded":"2023-09-27T23:30:24.974626+0000","last_fullsized":"2023-09-27T23:30:24.974626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445158+0000","last_change":"2023-09-27T23:30:29.046235+0000","last_active":"2023-09-27T23:30:29.445158+0000","last_peered":"2023-09-27T23:30:29.445158+0000","last_clean":"2023-09-27T23:30:29.445158+0000","last_became_active":"2023-09-27T23:30:29.045955+0000","last_became_peered":"2023-09-27T23:30:29.045955+0000","last_unstale":"2023-09-27T23:30:29.445158+0000","last_undegraded":"2023-09-27T23:30:29.445158+0000","last_fullsized":"2023-09-27T23:30:29.445158+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:39:54.902280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402923+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:30:29.402923+0000","last_peered":"2023-09-27T23:30:29.402923+0000","last_clean":"2023-09-27T23:30:29.402923+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:30:29.402923+0000","last_undegraded":"2023-09-27T23:30:29.402923+0000","last_fullsized":"2023-09-27T23:30:29.402923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979214+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:30:24.979214+0000","last_peered":"2023-09-27T23:30:24.979214+0000","last_clean":"2023-09-27T23:30:24.979214+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:30:24.979214+0000","last_undegraded":"2023-09-27T23:30:24.979214+0000","last_fullsized":"2023-09-27T23:30:24.979214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401712+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:30:29.401712+0000","last_peered":"2023-09-27T23:30:29.401712+0000","last_clean":"2023-09-27T23:30:29.401712+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:30:29.401712+0000","last_undegraded":"2023-09-27T23:30:29.401712+0000","last_fullsized":"2023-09-27T23:30:29.401712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.861948+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:30:24.861948+0000","last_peered":"2023-09-27T23:30:24.861948+0000","last_clean":"2023-09-27T23:30:24.861948+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:30:24.861948+0000","last_undegraded":"2023-09-27T23:30:24.861948+0000","last_fullsized":"2023-09-27T23:30:24.861948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":14,"acting":14,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26492,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":24576,"kb_avail":94345348,"statfs":{"total":96636764160,"available":96609636352,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25165824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.877,"15min":0.91100000000000003},"min":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.712,"15min":1.712},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.89900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.337,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.256}]},{"osd":2,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.56599999999999995,"5min":0.307,"15min":0.307},"max":{"1min":1.3129999999999999,"5min":2.681,"15min":2.681},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.3520000000000001,"5min":1.718,"15min":1.718},"last":1.194}]},{"osd":3,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1000000000000001,"5min":1.29,"15min":1.421},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.60499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3240000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.145},{"interface":"front","average":{"1min":0.997,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.69599999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2450000000000001,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0469999999999999,"15min":1.0289999999999999},"min":{"1min":0.79600000000000004,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.292},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.099},"min":{"1min":0.90300000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7190000000000001,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.359}]},{"osd":6,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.169,"15min":1.131},"min":{"1min":0.97299999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.145,"15min":1.1279999999999999},"min":{"1min":0.97799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4139999999999999,"5min":2.548,"15min":2.548},"last":1.3280000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22060,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":20736,"kb_avail":94349780,"statfs":{"total":96636764160,"available":96614174720,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21233664},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.55500000000000005,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1279999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.022,"15min":1.0209999999999999},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2230000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.88200000000000001},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.248,"5min":1.2889999999999999,"15min":1.323},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88900000000000001,"15min":0.89800000000000002},"min":{"1min":0.60799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.153}]},{"osd":2,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85999999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.222,"15min":1.353},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.68799999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94299999999999995,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.734},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.256,"5min":1.278,"15min":1.4970000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.61299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.046},{"interface":"front","average":{"1min":1.004,"5min":1.0369999999999999,"15min":1.0309999999999999},"min":{"1min":0.65100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5980000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97699999999999998,"15min":0.97499999999999998},"min":{"1min":0.68700000000000006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.02},{"interface":"front","average":{"1min":1.163,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3740000000000001,"5min":1.4259999999999999,"15min":1.49},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.143},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6719999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.147},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.6619999999999999},"last":1.423}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36216,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335624,"statfs":{"total":96636764160,"available":96599678976,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1519999999999999,"5min":1.1890000000000001,"15min":1.2909999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.27,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.2969999999999999,"15min":1.526},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.877,"15min":0.89000000000000001},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":1.331},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.52900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97999999999999998,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.276,"5min":1.369,"15min":1.5740000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2290000000000001,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.1100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.88700000000000001},"min":{"1min":0.45200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.222,"5min":1.341,"15min":1.341},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.70599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3799999999999999,"5min":1.429,"15min":1.5580000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3520000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.04}]},{"osd":7,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.123,"15min":1.123},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.6679999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2050000000000001,"15min":1.1890000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.633},"last":1.3089999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22524,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349316,"statfs":{"total":96636764160,"available":96613699584,"internally_reserved":0,"allocated":1429504,"data_stored":729891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2230000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.1639999999999999,"5min":1.45,"15min":1.45},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.52900000000000003},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.3919999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.329},"last":1.032}]},{"osd":2,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.2430000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.331,"5min":1.3660000000000001,"15min":1.407},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.256,"15min":1.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.018},"min":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":1.1439999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.90600000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.57599999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.78400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0489999999999999,"15min":1.0329999999999999},"min":{"1min":0.77300000000000002,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.149},"min":{"1min":0.97599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.514,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.236}]},{"osd":7,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.117,"15min":1.091},"min":{"1min":0.86899999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4770000000000001,"5min":1.627,"15min":1.627},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1759999999999999,"15min":1.155},"min":{"1min":0.97199999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.2609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22764,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94349076,"statfs":{"total":96636764160,"available":96613453824,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":1.1930000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.012},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2230000000000001,"5min":1.393,"15min":1.393},"last":0.874}]},{"osd":1,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.323,"15min":1.3440000000000001},"last":0.501},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.96899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0720000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.4810000000000001,"15min":1.571},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.127,"15min":1.153},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.714,"15min":1.764},"last":1.1950000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1000000000000001,"15min":1.113},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2829999999999999,"5min":1.5329999999999999,"15min":1.587},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.2130000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.1459999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2130000000000001,"15min":1.232},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4119999999999999,"5min":1.655,"15min":1.655},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.177,"15min":1.196},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.556,"5min":1.629,"15min":1.671},"last":1.0780000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509643,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21368,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350472,"statfs":{"total":96636764160,"available":96614883328,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97899999999999998,"5min":1.093,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.903,"15min":1.903},"last":0.99299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.499,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.105,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.439,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.032,"5min":1.391,"15min":1.391},"last":0.75700000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78900000000000003,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.444,"15min":1.444},"last":0.91800000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0369999999999999,"15min":1.036},"min":{"1min":0.751,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4239999999999999,"5min":1.4319999999999999,"15min":1.585},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5620000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.238}]},{"osd":5,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.032,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4339999999999999,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.397,"5min":1.5169999999999999,"15min":1.645},"last":1.319}]},{"osd":6,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1619999999999999,"15min":1.163},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.45,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.1679999999999999,"15min":1.159},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4359999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.052}]},{"osd":7,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.69699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.663,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.347},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2170000000000001},"min":{"1min":0.79700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.502,"5min":2.073,"15min":2.073},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542350,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":35520,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1409024,"data_stored":728231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36372480},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.123,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95799999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.56699999999999995,"5min":0.48199999999999998,"15min":0.45800000000000002},"max":{"1min":0.81999999999999995,"5min":1.2490000000000001,"15min":1.321},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1599999999999999,"5min":1.696,"15min":1.696},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.97599999999999998,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.431,"5min":1.464,"15min":1.464},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1180000000000001,"15min":1.127},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4410000000000001,"5min":1.5369999999999999,"15min":1.774},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.425,"5min":1.546,"15min":1.623},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1020000000000001,"15min":1.0980000000000001},"min":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88900000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.22,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.23,"5min":1.1890000000000001,"15min":1.1899999999999999},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.93400000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.216,"15min":1.23},"min":{"1min":0.88300000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5609999999999999,"5min":1.877,"15min":1.877},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2030000000000001,"15min":1.2170000000000001},"min":{"1min":0.95299999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.55,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23420,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":21504,"kb_avail":94348420,"statfs":{"total":96636764160,"available":96612782080,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22020096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":1.3,"15min":1.3},"last":0.69299999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.45600000000000002},"max":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99099999999999999,"5min":1.228,"15min":1.232},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.83999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.117,"5min":1.216,"15min":1.2769999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.47},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.995,"5min":1.022,"15min":1.0389999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.357,"5min":1.5589999999999999,"15min":1.587},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.081,"15min":1.091},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.272,"5min":1.5009999999999999,"15min":1.78},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.113,"15min":1.137},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":4.6779999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.147,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.4370000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.202,"15min":1.2150000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.73,"15min":1.8300000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2170000000000001,"15min":1.238},"min":{"1min":0.78500000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1919999999999999,"15min":1.222},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.5660000000000001,"5min":1.798,"15min":1.798},"last":0.96699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:33.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-09-27T23:30:33.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-09-27T23:30:33.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-09-27T23:30:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-09-27T23:30:33.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2023-09-27T23:30:33.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-09-27T23:30:33.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2023-09-27T23:30:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2023-09-27T23:30:33.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-09-27T23:30:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-09-27T23:30:33.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-09-27T23:30:33.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2023-09-27T23:30:33.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:29.002998+0000', 'last_change': '2023-09-27T23:30:29.002998+0000', 'last_active': '2023-09-27T23:30:29.002998+0000', 'last_peered': '2023-09-27T23:30:29.002998+0000', 'last_clean': '2023-09-27T23:30:29.002998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:29.002998+0000', 'last_undegraded': '2023-09-27T23:30:29.002998+0000', 'last_fullsized': '2023-09-27T23:30:29.002998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:29.002998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2023-09-27T23:30:33.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:30:27.990481+0000', 'last_change': '2023-09-27T23:30:27.990481+0000', 'last_active': '2023-09-27T23:30:27.990481+0000', 'last_peered': '2023-09-27T23:30:27.990481+0000', 'last_clean': '2023-09-27T23:30:27.990481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:30:27.990481+0000', 'last_undegraded': '2023-09-27T23:30:27.990481+0000', 'last_fullsized': '2023-09-27T23:30:27.990481+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'last_clean_scrub_stamp': '2023-09-27T23:30:27.990481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:30:33.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:30:33.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.255+0000 7f7f88dba700 1 Processor -- start 2023-09-27T23:30:33.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.255+0000 7f7f88dba700 1 -- start start 2023-09-27T23:30:33.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f84155960 0x7f7f8415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f8406de20 0x7f7f840c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7f8406c7c0 0x7f7f840c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8406d990 con 0x7f7f84155960 2023-09-27T23:30:33.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8415a570 con 0x7f7f8406c7c0 2023-09-27T23:30:33.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8406cd80 con 0x7f7f8406de20 2023-09-27T23:30:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f84155960 0x7f7f8415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f84155960 0x7f7f8415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40362/0 (socket says 172.21.15.64:40362) 2023-09-27T23:30:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 -- 172.21.15.64:0/3060006228 learned_addr learned my addr 172.21.15.64:0/3060006228 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f82ffd700 1 --2- 172.21.15.64:0/3060006228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f8406de20 0x7f7f840c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f83fff700 1 --2- 172.21.15.64:0/3060006228 >> v2:172.21.15.123:3300/0 conn(0x7f7f8406c7c0 0x7f7f840c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 -- 172.21.15.64:0/3060006228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f8406de20 msgr2=0x7f7f840c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3060006228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f8406de20 0x7f7f840c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 -- 172.21.15.64:0/3060006228 >> v2:172.21.15.123:3300/0 conn(0x7f7f8406c7c0 msgr2=0x7f7f840c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3060006228 >> v2:172.21.15.123:3300/0 conn(0x7f7f8406c7c0 0x7f7f840c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 -- 172.21.15.64:0/3060006228 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8406d060 con 0x7f7f84155960 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3060006228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f84155960 0x7f7f8415a030 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406e720 tx=0x7f7f74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb242294163b4d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f827fc700 1 -- 172.21.15.64:0/3060006228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400b640 con 0x7f7f84155960 2023-09-27T23:30:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f827fc700 1 -- 172.21.15.64:0/3060006228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400b7a0 con 0x7f7f84155960 2023-09-27T23:30:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f827fc700 1 -- 172.21.15.64:0/3060006228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400bb60 con 0x7f7f84155960 2023-09-27T23:30:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- 172.21.15.64:0/3060006228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f84155960 msgr2=0x7f7f8415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3060006228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f84155960 0x7f7f8415a030 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406e720 tx=0x7f7f74009250 comp rx=0 tx=0).stop 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- 172.21.15.64:0/3060006228 shutdown_connections 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3060006228 >> v2:172.21.15.123:3300/0 conn(0x7f7f8406c7c0 0x7f7f840c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3060006228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f8406de20 0x7f7f840c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3060006228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f84155960 0x7f7f8415a030 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- 172.21.15.64:0/3060006228 >> 172.21.15.64:0/3060006228 conn(0x7f7f84067bb0 msgr2=0x7f7f840cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- 172.21.15.64:0/3060006228 shutdown_connections 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.259+0000 7f7f88dba700 1 -- 172.21.15.64:0/3060006228 wait complete. 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 Processor -- start 2023-09-27T23:30:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 -- start start 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8406c7c0 0x7f7f841438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 0x7f7f8413ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f84155960 0x7f7f8413b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f840b19d0 con 0x7f7f8406de20 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8413c1a0 con 0x7f7f8406c7c0 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8413b870 con 0x7f7f84155960 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 0x7f7f8413ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 0x7f7f8413ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40376/0 (socket says 172.21.15.64:40376) 2023-09-27T23:30:33.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 -- 172.21.15.64:0/3205481138 learned_addr learned my addr 172.21.15.64:0/3205481138 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f83fff700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f84155960 0x7f7f8413b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8406c7c0 0x7f7f841438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 -- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f84155960 msgr2=0x7f7f8413b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f84155960 0x7f7f8413b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 -- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8406c7c0 msgr2=0x7f7f841438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8406c7c0 0x7f7f841438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 -- 172.21.15.64:0/3205481138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740094f0 con 0x7f7f8406de20 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8406c7c0 0x7f7f841438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f82ffd700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 0x7f7f8413ad20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00ec80 tx=0x7f7f6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00c9d0 con 0x7f7f8406de20 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00cb50 con 0x7f7f8406de20 2023-09-27T23:30:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8413bb50 con 0x7f7f8406de20 2023-09-27T23:30:33.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c013470 con 0x7f7f8406de20 2023-09-27T23:30:33.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.263+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f84143e20 con 0x7f7f8406de20 2023-09-27T23:30:33.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.267+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f84166fe0 con 0x7f7f8406de20 2023-09-27T23:30:33.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.267+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7f6c013820 con 0x7f7f8406de20 2023-09-27T23:30:33.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.267+0000 7f7f7b7fe700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f64060a70 0x7f7f64062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.267+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7f6c07d610 con 0x7f7f8406de20 2023-09-27T23:30:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.267+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f64060a70 0x7f7f64062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.267+0000 7f7f837fe700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f64060a70 0x7f7f64062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406e720 tx=0x7f7f74017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:33.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.275+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7f84166fe0 con 0x7f7f8406de20 2023-09-27T23:30:33.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7f84166fe0 con 0x7f7f8406de20 2023-09-27T23:30:33.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f7b7fe700 1 -- 172.21.15.64:0/3205481138 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3287 (secure 0 0 0) 0x7f7f6c017cf0 con 0x7f7f8406de20 2023-09-27T23:30:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:33.497 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1032,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":115,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":74},{"state_name":"active+clean","count":31}],"num_pgs":105,"num_pools":4,"num_objects":11,"data_bytes":591904,"bytes_used":216829952,"bytes_avail":772877283328,"bytes_total":773094113280,"unknown_pgs_ratio":0.70476192235946655,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":39,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7605}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-09-27T23:30:17.932176+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f64060a70 msgr2=0x7f7f64062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f64060a70 0x7f7f64062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406e720 tx=0x7f7f74017000 comp rx=0 tx=0).stop 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 msgr2=0x7f7f8413ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 0x7f7f8413ad20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00ec80 tx=0x7f7f6c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 shutdown_connections 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7f64060a70 0x7f7f64062f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f84155960 0x7f7f8413b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8406de20 0x7f7f8413ad20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 --2- 172.21.15.64:0/3205481138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8406c7c0 0x7f7f841438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 >> 172.21.15.64:0/3205481138 conn(0x7f7f84067bb0 msgr2=0x7f7f840c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:33.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 shutdown_connections 2023-09-27T23:30:33.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.495+0000 7f7f88dba700 1 -- 172.21.15.64:0/3205481138 wait complete. 2023-09-27T23:30:33.512 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:30:33.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:30:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec88fa6700 1 Processor -- start 2023-09-27T23:30:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec88fa6700 1 -- start start 2023-09-27T23:30:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec88fa6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec8415bd00 0x7fec8415c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec88fa6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8406b2c0 con 0x7fec840cad10 2023-09-27T23:30:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec88fa6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8406e0f0 con 0x7fec840cb0e0 2023-09-27T23:30:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec88fa6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8406a6b0 con 0x7fec8415bd00 2023-09-27T23:30:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec837fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fec840cad10 0x7fec8406b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45788/0 (socket says 172.21.15.64:45788) 2023-09-27T23:30:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec837fe700 1 -- 172.21.15.64:0/1744118160 learned_addr learned my addr 172.21.15.64:0/1744118160 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec83fff700 1 --2- 172.21.15.64:0/1744118160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec8415bd00 0x7fec8415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163146979 0 0) 0x7fec8406b2c0 con 0x7fec840cad10 2023-09-27T23:30:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec70003570 con 0x7fec840cad10 2023-09-27T23:30:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629220764 0 0) 0x7fec8406e0f0 con 0x7fec840cb0e0 2023-09-27T23:30:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec8406b2c0 con 0x7fec840cb0e0 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544915884 0 0) 0x7fec70003570 con 0x7fec840cad10 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec83fff700 1 -- 172.21.15.64:0/1744118160 >> v1:172.21.15.123:6789/0 conn(0x7fec840cb0e0 legacy=0x7fec8406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec83fff700 1 -- 172.21.15.64:0/1744118160 >> v1:172.21.15.64:6789/0 conn(0x7fec840cad10 legacy=0x7fec8406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec83fff700 1 -- 172.21.15.64:0/1744118160 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8406a990 con 0x7fec8415bd00 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.615+0000 7fec83fff700 1 --2- 172.21.15.64:0/1744118160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec8415bd00 0x7fec8415c180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009e90 tx=0x7fec7c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7399bf52175a0286 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00e070 con 0x7fec8415bd00 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c00b870 con 0x7fec8415bd00 2023-09-27T23:30:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec82ffd700 1 -- 172.21.15.64:0/1744118160 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74001470 con 0x7fec8415bd00 2023-09-27T23:30:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- 172.21.15.64:0/1744118160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec8415bd00 msgr2=0x7fec8415c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 --2- 172.21.15.64:0/1744118160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec8415bd00 0x7fec8415c180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009e90 tx=0x7fec7c009780 comp rx=0 tx=0).stop 2023-09-27T23:30:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- 172.21.15.64:0/1744118160 shutdown_connections 2023-09-27T23:30:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 --2- 172.21.15.64:0/1744118160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec8415bd00 0x7fec8415c180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- 172.21.15.64:0/1744118160 >> 172.21.15.64:0/1744118160 conn(0x7fec84067bb0 msgr2=0x7fec840cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- 172.21.15.64:0/1744118160 shutdown_connections 2023-09-27T23:30:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- 172.21.15.64:0/1744118160 wait complete. 2023-09-27T23:30:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 Processor -- start 2023-09-27T23:30:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- start start 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec840cad10 0x7fec8414f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec840cb0e0 0x7fec841512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 0x7fec840b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84168ba0 con 0x7fec8415bd00 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec841699e0 con 0x7fec840cb0e0 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec88fa6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec84168dd0 con 0x7fec840cad10 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 0x7fec840b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 0x7fec840b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40392/0 (socket says 172.21.15.64:40392) 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec83fff700 1 -- 172.21.15.64:0/3775407826 learned_addr learned my addr 172.21.15.64:0/3775407826 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec837fe700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec840cad10 0x7fec8414f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.619+0000 7fec7bfff700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec840cb0e0 0x7fec841512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec83fff700 1 -- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec840cad10 msgr2=0x7fec8414f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec83fff700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec840cad10 0x7fec8414f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec83fff700 1 -- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec840cb0e0 msgr2=0x7fec841512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec83fff700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec840cb0e0 0x7fec841512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec83fff700 1 -- 172.21.15.64:0/3775407826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c009a20 con 0x7fec8415bd00 2023-09-27T23:30:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec83fff700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 0x7fec840b0770 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009de0 tx=0x7fec7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c012b50 con 0x7fec8415bd00 2023-09-27T23:30:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c012cb0 con 0x7fec8415bd00 2023-09-27T23:30:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec841690b0 con 0x7fec8415bd00 2023-09-27T23:30:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c01e680 con 0x7fec8415bd00 2023-09-27T23:30:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec84169670 con 0x7fec8415bd00 2023-09-27T23:30:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8404be40 con 0x7fec8415bd00 2023-09-27T23:30:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.623+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fec7c01eab0 con 0x7fec8415bd00 2023-09-27T23:30:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.627+0000 7fec7b7fe700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fec64060aa0 0x7fec64062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.627+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fec7c07f280 con 0x7fec8415bd00 2023-09-27T23:30:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.627+0000 7fec837fe700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fec64060aa0 0x7fec64062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:33.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.627+0000 7fec837fe700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fec64060aa0 0x7fec64062f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec740011e0 tx=0x7fec74001620 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.631+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fec7c04f220 con 0x7fec8415bd00 2023-09-27T23:30:33.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.787+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec84169b20 con 0x7fec64060aa0 2023-09-27T23:30:33.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.803+0000 7fec7b7fe700 1 -- 172.21.15.64:0/3775407826 <== mgr.6578 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296656 (secure 0 0 0) 0x7fec84169b20 con 0x7fec64060aa0 2023-09-27T23:30:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fec64060aa0 msgr2=0x7fec64062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fec64060aa0 0x7fec64062f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec740011e0 tx=0x7fec74001620 comp rx=0 tx=0).stop 2023-09-27T23:30:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 msgr2=0x7fec840b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 0x7fec840b0770 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009de0 tx=0x7fec7c017000 comp rx=0 tx=0).stop 2023-09-27T23:30:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 shutdown_connections 2023-09-27T23:30:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fec64060aa0 0x7fec64062f50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec8415bd00 0x7fec840b0770 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec840cb0e0 0x7fec841512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 --2- 172.21.15.64:0/3775407826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec840cad10 0x7fec8414f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 >> 172.21.15.64:0/3775407826 conn(0x7fec84067bb0 msgr2=0x7fec840b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 shutdown_connections 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:33.807+0000 7fec88fa6700 1 -- 172.21.15.64:0/3775407826 wait complete. 2023-09-27T23:30:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:30:33.825 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:30:31.936263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591904,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":340,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":211748,"kb_used_data":11876,"kb_used_omap":0,"kb_used_meta":199808,"kb_avail":754762972,"statfs":{"total":773094113280,"available":772877283328,"internally_reserved":0,"allocated":12161024,"data_stored":6828752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":204603392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001197"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437015+0000","last_change":"2023-09-27T23:30:29.424776+0000","last_active":"2023-09-27T23:30:29.437015+0000","last_peered":"2023-09-27T23:30:29.437015+0000","last_clean":"2023-09-27T23:30:29.437015+0000","last_became_active":"2023-09-27T23:30:29.424350+0000","last_became_peered":"2023-09-27T23:30:29.424350+0000","last_unstale":"2023-09-27T23:30:29.437015+0000","last_undegraded":"2023-09-27T23:30:29.437015+0000","last_fullsized":"2023-09-27T23:30:29.437015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:07.044123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437217+0000","last_change":"2023-09-27T23:30:29.431912+0000","last_active":"2023-09-27T23:30:29.437217+0000","last_peered":"2023-09-27T23:30:29.437217+0000","last_clean":"2023-09-27T23:30:29.437217+0000","last_became_active":"2023-09-27T23:30:29.431583+0000","last_became_peered":"2023-09-27T23:30:29.431583+0000","last_unstale":"2023-09-27T23:30:29.437217+0000","last_undegraded":"2023-09-27T23:30:29.437217+0000","last_fullsized":"2023-09-27T23:30:29.437217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:00:22.489672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447931+0000","last_change":"2023-09-27T23:30:29.445370+0000","last_active":"2023-09-27T23:30:29.447931+0000","last_peered":"2023-09-27T23:30:29.447931+0000","last_clean":"2023-09-27T23:30:29.447931+0000","last_became_active":"2023-09-27T23:30:29.444846+0000","last_became_peered":"2023-09-27T23:30:29.444846+0000","last_unstale":"2023-09-27T23:30:29.447931+0000","last_undegraded":"2023-09-27T23:30:29.447931+0000","last_fullsized":"2023-09-27T23:30:29.447931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:41.665511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447849+0000","last_change":"2023-09-27T23:30:29.437968+0000","last_active":"2023-09-27T23:30:29.447849+0000","last_peered":"2023-09-27T23:30:29.447849+0000","last_clean":"2023-09-27T23:30:29.447849+0000","last_became_active":"2023-09-27T23:30:29.437657+0000","last_became_peered":"2023-09-27T23:30:29.437657+0000","last_unstale":"2023-09-27T23:30:29.447849+0000","last_undegraded":"2023-09-27T23:30:29.447849+0000","last_fullsized":"2023-09-27T23:30:29.447849+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:51.757870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447819+0000","last_change":"2023-09-27T23:30:29.434868+0000","last_active":"2023-09-27T23:30:29.447819+0000","last_peered":"2023-09-27T23:30:29.447819+0000","last_clean":"2023-09-27T23:30:29.447819+0000","last_became_active":"2023-09-27T23:30:29.434622+0000","last_became_peered":"2023-09-27T23:30:29.434622+0000","last_unstale":"2023-09-27T23:30:29.447819+0000","last_undegraded":"2023-09-27T23:30:29.447819+0000","last_fullsized":"2023-09-27T23:30:29.447819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:31:18.606971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434942+0000","last_change":"2023-09-27T23:30:29.421559+0000","last_active":"2023-09-27T23:30:29.434942+0000","last_peered":"2023-09-27T23:30:29.434942+0000","last_clean":"2023-09-27T23:30:29.434942+0000","last_became_active":"2023-09-27T23:30:29.420693+0000","last_became_peered":"2023-09-27T23:30:29.420693+0000","last_unstale":"2023-09-27T23:30:29.434942+0000","last_undegraded":"2023-09-27T23:30:29.434942+0000","last_fullsized":"2023-09-27T23:30:29.434942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:11.243612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445884+0000","last_change":"2023-09-27T23:30:29.443588+0000","last_active":"2023-09-27T23:30:29.445884+0000","last_peered":"2023-09-27T23:30:29.445884+0000","last_clean":"2023-09-27T23:30:29.445884+0000","last_became_active":"2023-09-27T23:30:29.443289+0000","last_became_peered":"2023-09-27T23:30:29.443289+0000","last_unstale":"2023-09-27T23:30:29.445884+0000","last_undegraded":"2023-09-27T23:30:29.445884+0000","last_fullsized":"2023-09-27T23:30:29.445884+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:57:06.402886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445278+0000","last_change":"2023-09-27T23:30:29.434541+0000","last_active":"2023-09-27T23:30:29.445278+0000","last_peered":"2023-09-27T23:30:29.445278+0000","last_clean":"2023-09-27T23:30:29.445278+0000","last_became_active":"2023-09-27T23:30:29.434260+0000","last_became_peered":"2023-09-27T23:30:29.434260+0000","last_unstale":"2023-09-27T23:30:29.445278+0000","last_undegraded":"2023-09-27T23:30:29.445278+0000","last_fullsized":"2023-09-27T23:30:29.445278+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:26:16.902235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436939+0000","last_change":"2023-09-27T23:30:29.032760+0000","last_active":"2023-09-27T23:30:29.436939+0000","last_peered":"2023-09-27T23:30:29.436939+0000","last_clean":"2023-09-27T23:30:29.436939+0000","last_became_active":"2023-09-27T23:30:29.032431+0000","last_became_peered":"2023-09-27T23:30:29.032431+0000","last_unstale":"2023-09-27T23:30:29.436939+0000","last_undegraded":"2023-09-27T23:30:29.436939+0000","last_fullsized":"2023-09-27T23:30:29.436939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:09:55.740875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446028+0000","last_change":"2023-09-27T23:30:29.438718+0000","last_active":"2023-09-27T23:30:29.446028+0000","last_peered":"2023-09-27T23:30:29.446028+0000","last_clean":"2023-09-27T23:30:29.446028+0000","last_became_active":"2023-09-27T23:30:29.438454+0000","last_became_peered":"2023-09-27T23:30:29.438454+0000","last_unstale":"2023-09-27T23:30:29.446028+0000","last_undegraded":"2023-09-27T23:30:29.446028+0000","last_fullsized":"2023-09-27T23:30:29.446028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:58:47.626281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401580+0000","last_change":"2023-09-27T23:30:29.035856+0000","last_active":"2023-09-27T23:30:29.401580+0000","last_peered":"2023-09-27T23:30:29.401580+0000","last_clean":"2023-09-27T23:30:29.401580+0000","last_became_active":"2023-09-27T23:30:29.035384+0000","last_became_peered":"2023-09-27T23:30:29.035384+0000","last_unstale":"2023-09-27T23:30:29.401580+0000","last_undegraded":"2023-09-27T23:30:29.401580+0000","last_fullsized":"2023-09-27T23:30:29.401580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:16:29.194838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402510+0000","last_change":"2023-09-27T23:30:29.022763+0000","last_active":"2023-09-27T23:30:29.402510+0000","last_peered":"2023-09-27T23:30:29.402510+0000","last_clean":"2023-09-27T23:30:29.402510+0000","last_became_active":"2023-09-27T23:30:29.021867+0000","last_became_peered":"2023-09-27T23:30:29.021867+0000","last_unstale":"2023-09-27T23:30:29.402510+0000","last_undegraded":"2023-09-27T23:30:29.402510+0000","last_fullsized":"2023-09-27T23:30:29.402510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:37.446360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445578+0000","last_change":"2023-09-27T23:30:29.434529+0000","last_active":"2023-09-27T23:30:29.445578+0000","last_peered":"2023-09-27T23:30:29.445578+0000","last_clean":"2023-09-27T23:30:29.445578+0000","last_became_active":"2023-09-27T23:30:29.434185+0000","last_became_peered":"2023-09-27T23:30:29.434185+0000","last_unstale":"2023-09-27T23:30:29.445578+0000","last_undegraded":"2023-09-27T23:30:29.445578+0000","last_fullsized":"2023-09-27T23:30:29.445578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:06.448424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.449495+0000","last_change":"2023-09-27T23:30:29.040154+0000","last_active":"2023-09-27T23:30:29.449495+0000","last_peered":"2023-09-27T23:30:29.449495+0000","last_clean":"2023-09-27T23:30:29.449495+0000","last_became_active":"2023-09-27T23:30:29.039855+0000","last_became_peered":"2023-09-27T23:30:29.039855+0000","last_unstale":"2023-09-27T23:30:29.449495+0000","last_undegraded":"2023-09-27T23:30:29.449495+0000","last_fullsized":"2023-09-27T23:30:29.449495+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:26:42.757699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434186+0000","last_change":"2023-09-27T23:30:29.422077+0000","last_active":"2023-09-27T23:30:29.434186+0000","last_peered":"2023-09-27T23:30:29.434186+0000","last_clean":"2023-09-27T23:30:29.434186+0000","last_became_active":"2023-09-27T23:30:29.421784+0000","last_became_peered":"2023-09-27T23:30:29.421784+0000","last_unstale":"2023-09-27T23:30:29.434186+0000","last_undegraded":"2023-09-27T23:30:29.434186+0000","last_fullsized":"2023-09-27T23:30:29.434186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:59.674679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445131+0000","last_change":"2023-09-27T23:30:29.434466+0000","last_active":"2023-09-27T23:30:29.445131+0000","last_peered":"2023-09-27T23:30:29.445131+0000","last_clean":"2023-09-27T23:30:29.445131+0000","last_became_active":"2023-09-27T23:30:29.434133+0000","last_became_peered":"2023-09-27T23:30:29.434133+0000","last_unstale":"2023-09-27T23:30:29.445131+0000","last_undegraded":"2023-09-27T23:30:29.445131+0000","last_fullsized":"2023-09-27T23:30:29.445131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:46.166597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425237+0000","last_change":"2023-09-27T23:30:29.040263+0000","last_active":"2023-09-27T23:30:29.425237+0000","last_peered":"2023-09-27T23:30:29.425237+0000","last_clean":"2023-09-27T23:30:29.425237+0000","last_became_active":"2023-09-27T23:30:29.039976+0000","last_became_peered":"2023-09-27T23:30:29.039976+0000","last_unstale":"2023-09-27T23:30:29.425237+0000","last_undegraded":"2023-09-27T23:30:29.425237+0000","last_fullsized":"2023-09-27T23:30:29.425237+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:13.841678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437983+0000","last_change":"2023-09-27T23:30:29.031398+0000","last_active":"2023-09-27T23:30:29.437983+0000","last_peered":"2023-09-27T23:30:29.437983+0000","last_clean":"2023-09-27T23:30:29.437983+0000","last_became_active":"2023-09-27T23:30:29.031067+0000","last_became_peered":"2023-09-27T23:30:29.031067+0000","last_unstale":"2023-09-27T23:30:29.437983+0000","last_undegraded":"2023-09-27T23:30:29.437983+0000","last_fullsized":"2023-09-27T23:30:29.437983+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.823350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434246+0000","last_change":"2023-09-27T23:30:29.420956+0000","last_active":"2023-09-27T23:30:29.434246+0000","last_peered":"2023-09-27T23:30:29.434246+0000","last_clean":"2023-09-27T23:30:29.434246+0000","last_became_active":"2023-09-27T23:30:29.419860+0000","last_became_peered":"2023-09-27T23:30:29.419860+0000","last_unstale":"2023-09-27T23:30:29.434246+0000","last_undegraded":"2023-09-27T23:30:29.434246+0000","last_fullsized":"2023-09-27T23:30:29.434246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:10.232721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437190+0000","last_change":"2023-09-27T23:30:29.428257+0000","last_active":"2023-09-27T23:30:29.437190+0000","last_peered":"2023-09-27T23:30:29.437190+0000","last_clean":"2023-09-27T23:30:29.437190+0000","last_became_active":"2023-09-27T23:30:29.427972+0000","last_became_peered":"2023-09-27T23:30:29.427972+0000","last_unstale":"2023-09-27T23:30:29.437190+0000","last_undegraded":"2023-09-27T23:30:29.437190+0000","last_fullsized":"2023-09-27T23:30:29.437190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:15:35.465950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446021+0000","last_change":"2023-09-27T23:30:29.435122+0000","last_active":"2023-09-27T23:30:29.446021+0000","last_peered":"2023-09-27T23:30:29.446021+0000","last_clean":"2023-09-27T23:30:29.446021+0000","last_became_active":"2023-09-27T23:30:29.434814+0000","last_became_peered":"2023-09-27T23:30:29.434814+0000","last_unstale":"2023-09-27T23:30:29.446021+0000","last_undegraded":"2023-09-27T23:30:29.446021+0000","last_fullsized":"2023-09-27T23:30:29.446021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:54.480949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"103'205","reported_seq":556,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979261+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:30:24.979261+0000","last_peered":"2023-09-27T23:30:24.979261+0000","last_clean":"2023-09-27T23:30:24.979261+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:30:24.979261+0000","last_undegraded":"2023-09-27T23:30:24.979261+0000","last_fullsized":"2023-09-27T23:30:24.979261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979599+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:30:24.979599+0000","last_peered":"2023-09-27T23:30:24.979599+0000","last_clean":"2023-09-27T23:30:24.979599+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:30:24.979599+0000","last_undegraded":"2023-09-27T23:30:24.979599+0000","last_fullsized":"2023-09-27T23:30:24.979599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974626+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:30:24.974626+0000","last_peered":"2023-09-27T23:30:24.974626+0000","last_clean":"2023-09-27T23:30:24.974626+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:30:24.974626+0000","last_undegraded":"2023-09-27T23:30:24.974626+0000","last_fullsized":"2023-09-27T23:30:24.974626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445158+0000","last_change":"2023-09-27T23:30:29.046235+0000","last_active":"2023-09-27T23:30:29.445158+0000","last_peered":"2023-09-27T23:30:29.445158+0000","last_clean":"2023-09-27T23:30:29.445158+0000","last_became_active":"2023-09-27T23:30:29.045955+0000","last_became_peered":"2023-09-27T23:30:29.045955+0000","last_unstale":"2023-09-27T23:30:29.445158+0000","last_undegraded":"2023-09-27T23:30:29.445158+0000","last_fullsized":"2023-09-27T23:30:29.445158+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:39:54.902280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.974339+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:30:24.974339+0000","last_peered":"2023-09-27T23:30:24.974339+0000","last_clean":"2023-09-27T23:30:24.974339+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:30:24.974339+0000","last_undegraded":"2023-09-27T23:30:24.974339+0000","last_fullsized":"2023-09-27T23:30:24.974339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402923+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:30:29.402923+0000","last_peered":"2023-09-27T23:30:29.402923+0000","last_clean":"2023-09-27T23:30:29.402923+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:30:29.402923+0000","last_undegraded":"2023-09-27T23:30:29.402923+0000","last_fullsized":"2023-09-27T23:30:29.402923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.979214+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:30:24.979214+0000","last_peered":"2023-09-27T23:30:24.979214+0000","last_clean":"2023-09-27T23:30:24.979214+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:30:24.979214+0000","last_undegraded":"2023-09-27T23:30:24.979214+0000","last_fullsized":"2023-09-27T23:30:24.979214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401712+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:30:29.401712+0000","last_peered":"2023-09-27T23:30:29.401712+0000","last_clean":"2023-09-27T23:30:29.401712+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:30:29.401712+0000","last_undegraded":"2023-09-27T23:30:29.401712+0000","last_fullsized":"2023-09-27T23:30:29.401712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:29.002998+0000","last_change":"2023-09-27T23:30:29.002998+0000","last_active":"2023-09-27T23:30:29.002998+0000","last_peered":"2023-09-27T23:30:29.002998+0000","last_clean":"2023-09-27T23:30:29.002998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:29.002998+0000","last_undegraded":"2023-09-27T23:30:29.002998+0000","last_fullsized":"2023-09-27T23:30:29.002998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:30:27.990481+0000","last_change":"2023-09-27T23:30:27.990481+0000","last_active":"2023-09-27T23:30:27.990481+0000","last_peered":"2023-09-27T23:30:27.990481+0000","last_clean":"2023-09-27T23:30:27.990481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:30:27.990481+0000","last_undegraded":"2023-09-27T23:30:27.990481+0000","last_fullsized":"2023-09-27T23:30:27.990481+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-09-27T23:30:24.861948+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:30:24.861948+0000","last_peered":"2023-09-27T23:30:24.861948+0000","last_clean":"2023-09-27T23:30:24.861948+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:30:24.861948+0000","last_undegraded":"2023-09-27T23:30:24.861948+0000","last_fullsized":"2023-09-27T23:30:24.861948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1536,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":14,"acting":14,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26492,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":24576,"kb_avail":94345348,"statfs":{"total":96636764160,"available":96609636352,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25165824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.877,"15min":0.91100000000000003},"min":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.712,"15min":1.712},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.89900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.337,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.256}]},{"osd":2,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.56599999999999995,"5min":0.307,"15min":0.307},"max":{"1min":1.3129999999999999,"5min":2.681,"15min":2.681},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.3520000000000001,"5min":1.718,"15min":1.718},"last":1.194}]},{"osd":3,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1000000000000001,"5min":1.29,"15min":1.421},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.60499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3240000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.145},{"interface":"front","average":{"1min":0.997,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.69599999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2450000000000001,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0469999999999999,"15min":1.0289999999999999},"min":{"1min":0.79600000000000004,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.292},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.099},"min":{"1min":0.90300000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7190000000000001,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.359}]},{"osd":6,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.169,"15min":1.131},"min":{"1min":0.97299999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.145,"15min":1.1279999999999999},"min":{"1min":0.97799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4139999999999999,"5min":2.548,"15min":2.548},"last":1.3280000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22060,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":20736,"kb_avail":94349780,"statfs":{"total":96636764160,"available":96614174720,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21233664},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.55500000000000005,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1279999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.022,"15min":1.0209999999999999},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2230000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.88200000000000001},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.248,"5min":1.2889999999999999,"15min":1.323},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88900000000000001,"15min":0.89800000000000002},"min":{"1min":0.60799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.153}]},{"osd":2,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85999999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.222,"15min":1.353},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.68799999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94299999999999995,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.734},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.256,"5min":1.278,"15min":1.4970000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.61299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.046},{"interface":"front","average":{"1min":1.004,"5min":1.0369999999999999,"15min":1.0309999999999999},"min":{"1min":0.65100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5980000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97699999999999998,"15min":0.97499999999999998},"min":{"1min":0.68700000000000006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.02},{"interface":"front","average":{"1min":1.163,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3740000000000001,"5min":1.4259999999999999,"15min":1.49},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.143},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6719999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.147},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.6619999999999999},"last":1.423}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36216,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":34880,"kb_avail":94335624,"statfs":{"total":96636764160,"available":96599678976,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35717120},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1519999999999999,"5min":1.1890000000000001,"15min":1.2909999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.27,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.2969999999999999,"15min":1.526},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.877,"15min":0.89000000000000001},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":1.331},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.52900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97999999999999998,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.276,"5min":1.369,"15min":1.5740000000000001},"last":1.2030000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2290000000000001,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.1100000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.88700000000000001},"min":{"1min":0.45200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.222,"5min":1.341,"15min":1.341},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.70599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.069}]},{"osd":6,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3799999999999999,"5min":1.429,"15min":1.5580000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3520000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.04}]},{"osd":7,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.123,"15min":1.123},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.6679999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2050000000000001,"15min":1.1890000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.633},"last":1.3089999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22524,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349316,"statfs":{"total":96636764160,"available":96613699584,"internally_reserved":0,"allocated":1429504,"data_stored":729891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2230000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.1639999999999999,"5min":1.45,"15min":1.45},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.52900000000000003},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.3919999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.329},"last":1.032}]},{"osd":2,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.2430000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.331,"5min":1.3660000000000001,"15min":1.407},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.256,"15min":1.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.018},"min":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":1.1439999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.90600000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.57599999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.78400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0489999999999999,"15min":1.0329999999999999},"min":{"1min":0.77300000000000002,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.149},"min":{"1min":0.97599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.514,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.236}]},{"osd":7,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.117,"15min":1.091},"min":{"1min":0.86899999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4770000000000001,"5min":1.627,"15min":1.627},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1759999999999999,"15min":1.155},"min":{"1min":0.97199999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.2609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509642,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22764,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94349076,"statfs":{"total":96636764160,"available":96613453824,"internally_reserved":0,"allocated":1347584,"data_stored":694107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":1.1930000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.012},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2230000000000001,"5min":1.393,"15min":1.393},"last":0.874}]},{"osd":1,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.323,"15min":1.3440000000000001},"last":0.501},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.96899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0720000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.4810000000000001,"15min":1.571},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.127,"15min":1.153},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.714,"15min":1.764},"last":1.1950000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1000000000000001,"15min":1.113},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2829999999999999,"5min":1.5329999999999999,"15min":1.587},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.2130000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.1459999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2130000000000001,"15min":1.232},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4119999999999999,"5min":1.655,"15min":1.655},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.177,"15min":1.196},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.556,"5min":1.629,"15min":1.671},"last":1.0780000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509643,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21368,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":20032,"kb_avail":94350472,"statfs":{"total":96636764160,"available":96614883328,"internally_reserved":0,"allocated":1359872,"data_stored":700420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20512768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97899999999999998,"5min":1.093,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.903,"15min":1.903},"last":0.99299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.499,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.105,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.439,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.032,"5min":1.391,"15min":1.391},"last":0.75700000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78900000000000003,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.444,"15min":1.444},"last":0.91800000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0369999999999999,"15min":1.036},"min":{"1min":0.751,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4239999999999999,"5min":1.4319999999999999,"15min":1.585},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5620000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.238}]},{"osd":5,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.032,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4339999999999999,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.397,"5min":1.5169999999999999,"15min":1.645},"last":1.319}]},{"osd":6,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1619999999999999,"15min":1.163},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.45,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.1679999999999999,"15min":1.159},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4359999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.052}]},{"osd":7,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.69699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.663,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.347},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2170000000000001},"min":{"1min":0.79700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.502,"5min":2.073,"15min":2.073},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542350,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":35520,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1409024,"data_stored":728231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36372480},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.123,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95799999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.56699999999999995,"5min":0.48199999999999998,"15min":0.45800000000000002},"max":{"1min":0.81999999999999995,"5min":1.2490000000000001,"15min":1.321},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1599999999999999,"5min":1.696,"15min":1.696},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.97599999999999998,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.431,"5min":1.464,"15min":1.464},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1180000000000001,"15min":1.127},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4410000000000001,"5min":1.5369999999999999,"15min":1.774},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.425,"5min":1.546,"15min":1.623},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1020000000000001,"15min":1.0980000000000001},"min":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88900000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.22,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.23,"5min":1.1890000000000001,"15min":1.1899999999999999},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.93400000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.216,"15min":1.23},"min":{"1min":0.88300000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5609999999999999,"5min":1.877,"15min":1.877},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2030000000000001,"15min":1.2170000000000001},"min":{"1min":0.95299999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.55,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23420,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":21504,"kb_avail":94348420,"statfs":{"total":96636764160,"available":96612782080,"internally_reserved":0,"allocated":1953792,"data_stored":1290788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22020096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":1.3,"15min":1.3},"last":0.69299999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.45600000000000002},"max":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99099999999999999,"5min":1.228,"15min":1.232},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.83999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.117,"5min":1.216,"15min":1.2769999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.47},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.995,"5min":1.022,"15min":1.0389999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.357,"5min":1.5589999999999999,"15min":1.587},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.081,"15min":1.091},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.272,"5min":1.5009999999999999,"15min":1.78},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.113,"15min":1.137},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":4.6779999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.147,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.4370000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.202,"15min":1.2150000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.73,"15min":1.8300000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2170000000000001,"15min":1.238},"min":{"1min":0.78500000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1919999999999999,"15min":1.222},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.5660000000000001,"5min":1.798,"15min":1.798},"last":0.96699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:35.312 INFO:tasks.ceph.mds.c.smithi064.stderr:2023-09-27T23:30:35.307+0000 7f60e121f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:30:36.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:30:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.979+0000 7fa10a39f700 1 Processor -- start 2023-09-27T23:30:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.979+0000 7fa10a39f700 1 -- start start 2023-09-27T23:30:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa104159160 0x7fa104159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa1040adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa1040ae210 0x7fa1040cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10406a7b0 con 0x7fa104159160 2023-09-27T23:30:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040ad550 con 0x7fa1040ae210 2023-09-27T23:30:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c6400 con 0x7fa1040c5fd0 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa104159160 0x7fa104159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa104159160 0x7fa104159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34568/0 (socket says 172.21.15.64:34568) 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 -- 172.21.15.64:0/1687017387 learned_addr learned my addr 172.21.15.64:0/1687017387 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa109b9e700 1 --2- 172.21.15.64:0/1687017387 >> v2:172.21.15.123:3300/0 conn(0x7fa1040ae210 0x7fa1040cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa108b9c700 1 --2- 172.21.15.64:0/1687017387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa1040adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 -- 172.21.15.64:0/1687017387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 msgr2=0x7fa1040adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 --2- 172.21.15.64:0/1687017387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa1040adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 -- 172.21.15.64:0/1687017387 >> v2:172.21.15.123:3300/0 conn(0x7fa1040ae210 msgr2=0x7fa1040cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 --2- 172.21.15.64:0/1687017387 >> v2:172.21.15.123:3300/0 conn(0x7fa1040ae210 0x7fa1040cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 -- 172.21.15.64:0/1687017387 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040accb0 con 0x7fa104159160 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10939d700 1 --2- 172.21.15.64:0/1687017387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa104159530 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa1040ab0f0 tx=0x7fa100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9e5f299a7c6d5d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa0fbfff700 1 -- 172.21.15.64:0/1687017387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000b640 con 0x7fa104159160 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa0fbfff700 1 -- 172.21.15.64:0/1687017387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10000b7a0 con 0x7fa104159160 2023-09-27T23:30:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa0fbfff700 1 -- 172.21.15.64:0/1687017387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000bb40 con 0x7fa104159160 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- 172.21.15.64:0/1687017387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 msgr2=0x7fa104159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- 172.21.15.64:0/1687017387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa104159530 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa1040ab0f0 tx=0x7fa100009250 comp rx=0 tx=0).stop 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- 172.21.15.64:0/1687017387 shutdown_connections 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- 172.21.15.64:0/1687017387 >> v2:172.21.15.123:3300/0 conn(0x7fa1040ae210 0x7fa1040cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- 172.21.15.64:0/1687017387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa1040adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 --2- 172.21.15.64:0/1687017387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa104159530 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- 172.21.15.64:0/1687017387 >> 172.21.15.64:0/1687017387 conn(0x7fa104067bb0 msgr2=0x7fa1040cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- 172.21.15.64:0/1687017387 shutdown_connections 2023-09-27T23:30:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.983+0000 7fa10a39f700 1 -- 172.21.15.64:0/1687017387 wait complete. 2023-09-27T23:30:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.987+0000 7fa10a39f700 1 Processor -- start 2023-09-27T23:30:36.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.987+0000 7fa10a39f700 1 -- start start 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.987+0000 7fa10a39f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1040ae210 0x7fa1040c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.987+0000 7fa10a39f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa10414c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa1040af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c43e0 con 0x7fa104159160 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c37d0 con 0x7fa1040ae210 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c3ab0 con 0x7fa1040c5fd0 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa1040af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10939d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1040ae210 0x7fa1040c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa1040af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34576/0 (socket says 172.21.15.64:34576) 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10939d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1040ae210 0x7fa1040c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60640/0 (socket says 172.21.15.64:60640) 2023-09-27T23:30:36.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 -- 172.21.15.64:0/3739944598 learned_addr learned my addr 172.21.15.64:0/3739944598 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa108b9c700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa10414c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 -- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 msgr2=0x7fa10414c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa10414c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 -- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1040ae210 msgr2=0x7fa1040c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1040ae210 0x7fa1040c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:36.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 -- 172.21.15.64:0/3739944598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1000094f0 con 0x7fa104159160 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa109b9e700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa1040af130 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e90 tx=0x7fa0fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00cb70 con 0x7fa104159160 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00f040 con 0x7fa104159160 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1040c3d90 con 0x7fa104159160 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc014480 con 0x7fa104159160 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1040c3410 con 0x7fa104159160 2023-09-27T23:30:36.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.991+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1040c4590 con 0x7fa104159160 2023-09-27T23:30:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.995+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa0fc014830 con 0x7fa104159160 2023-09-27T23:30:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.995+0000 7fa0f97fa700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0e8060aa0 0x7fa0e8062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:36.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.995+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa0fc07de60 con 0x7fa104159160 2023-09-27T23:30:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.995+0000 7fa10939d700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0e8060aa0 0x7fa0e8062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:36.995+0000 7fa10939d700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0e8060aa0 0x7fa0e8062f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1040ab0f0 tx=0x7fa100017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.003+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa1040c4590 con 0x7fa104159160 2023-09-27T23:30:37.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.167+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa10404be40 con 0x7fa0e8060aa0 2023-09-27T23:30:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.187+0000 7fa0f97fa700 1 -- 172.21.15.64:0/3739944598 <== mgr.6578 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308130 (secure 0 0 0) 0x7fa10404be40 con 0x7fa0e8060aa0 2023-09-27T23:30:37.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0e8060aa0 msgr2=0x7fa0e8062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0e8060aa0 0x7fa0e8062f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1040ab0f0 tx=0x7fa100017000 comp rx=0 tx=0).stop 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 msgr2=0x7fa1040af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa1040af130 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e90 tx=0x7fa0fc00b040 comp rx=0 tx=0).stop 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 shutdown_connections 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0e8060aa0 0x7fa0e8062f50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa104159160 0x7fa1040af130 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa1040c5fd0 0x7fa10414c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 --2- 172.21.15.64:0/3739944598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1040ae210 0x7fa1040c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 >> 172.21.15.64:0/3739944598 conn(0x7fa104067bb0 msgr2=0x7fa1040c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:37.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 shutdown_connections 2023-09-27T23:30:37.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.191+0000 7fa10a39f700 1 -- 172.21.15.64:0/3739944598 wait complete. 2023-09-27T23:30:37.196 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:30:37.208 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:30:35.938083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":216968,"kb_used_data":12232,"kb_used_omap":0,"kb_used_meta":204672,"kb_avail":754757752,"statfs":{"total":773094113280,"available":772871938048,"internally_reserved":0,"allocated":12525568,"data_stored":7011196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":209584128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.924307"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437015+0000","last_change":"2023-09-27T23:30:29.424776+0000","last_active":"2023-09-27T23:30:29.437015+0000","last_peered":"2023-09-27T23:30:29.437015+0000","last_clean":"2023-09-27T23:30:29.437015+0000","last_became_active":"2023-09-27T23:30:29.424350+0000","last_became_peered":"2023-09-27T23:30:29.424350+0000","last_unstale":"2023-09-27T23:30:29.437015+0000","last_undegraded":"2023-09-27T23:30:29.437015+0000","last_fullsized":"2023-09-27T23:30:29.437015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:07.044123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436859+0000","last_change":"2023-09-27T23:30:29.433433+0000","last_active":"2023-09-27T23:30:29.436859+0000","last_peered":"2023-09-27T23:30:29.436859+0000","last_clean":"2023-09-27T23:30:29.436859+0000","last_became_active":"2023-09-27T23:30:29.433122+0000","last_became_peered":"2023-09-27T23:30:29.433122+0000","last_unstale":"2023-09-27T23:30:29.436859+0000","last_undegraded":"2023-09-27T23:30:29.436859+0000","last_fullsized":"2023-09-27T23:30:29.436859+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:39:55.914797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437217+0000","last_change":"2023-09-27T23:30:29.431912+0000","last_active":"2023-09-27T23:30:29.437217+0000","last_peered":"2023-09-27T23:30:29.437217+0000","last_clean":"2023-09-27T23:30:29.437217+0000","last_became_active":"2023-09-27T23:30:29.431583+0000","last_became_peered":"2023-09-27T23:30:29.431583+0000","last_unstale":"2023-09-27T23:30:29.437217+0000","last_undegraded":"2023-09-27T23:30:29.437217+0000","last_fullsized":"2023-09-27T23:30:29.437217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:00:22.489672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425906+0000","last_change":"2023-09-27T23:30:29.416696+0000","last_active":"2023-09-27T23:30:29.425906+0000","last_peered":"2023-09-27T23:30:29.425906+0000","last_clean":"2023-09-27T23:30:29.425906+0000","last_became_active":"2023-09-27T23:30:29.416413+0000","last_became_peered":"2023-09-27T23:30:29.416413+0000","last_unstale":"2023-09-27T23:30:29.425906+0000","last_undegraded":"2023-09-27T23:30:29.425906+0000","last_fullsized":"2023-09-27T23:30:29.425906+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:04:16.351723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437077+0000","last_change":"2023-09-27T23:30:29.425761+0000","last_active":"2023-09-27T23:30:29.437077+0000","last_peered":"2023-09-27T23:30:29.437077+0000","last_clean":"2023-09-27T23:30:29.437077+0000","last_became_active":"2023-09-27T23:30:29.425483+0000","last_became_peered":"2023-09-27T23:30:29.425483+0000","last_unstale":"2023-09-27T23:30:29.437077+0000","last_undegraded":"2023-09-27T23:30:29.437077+0000","last_fullsized":"2023-09-27T23:30:29.437077+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:56:37.846630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436999+0000","last_change":"2023-09-27T23:30:29.434270+0000","last_active":"2023-09-27T23:30:29.436999+0000","last_peered":"2023-09-27T23:30:29.436999+0000","last_clean":"2023-09-27T23:30:29.436999+0000","last_became_active":"2023-09-27T23:30:29.433887+0000","last_became_peered":"2023-09-27T23:30:29.433887+0000","last_unstale":"2023-09-27T23:30:29.436999+0000","last_undegraded":"2023-09-27T23:30:29.436999+0000","last_fullsized":"2023-09-27T23:30:29.436999+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:29.029841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436977+0000","last_change":"2023-09-27T23:30:29.433983+0000","last_active":"2023-09-27T23:30:29.436977+0000","last_peered":"2023-09-27T23:30:29.436977+0000","last_clean":"2023-09-27T23:30:29.436977+0000","last_became_active":"2023-09-27T23:30:29.433682+0000","last_became_peered":"2023-09-27T23:30:29.433682+0000","last_unstale":"2023-09-27T23:30:29.436977+0000","last_undegraded":"2023-09-27T23:30:29.436977+0000","last_fullsized":"2023-09-27T23:30:29.436977+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:42:19.807031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.428083+0000","last_change":"2023-09-27T23:30:29.423908+0000","last_active":"2023-09-27T23:30:29.428083+0000","last_peered":"2023-09-27T23:30:29.428083+0000","last_clean":"2023-09-27T23:30:29.428083+0000","last_became_active":"2023-09-27T23:30:29.423501+0000","last_became_peered":"2023-09-27T23:30:29.423501+0000","last_unstale":"2023-09-27T23:30:29.428083+0000","last_undegraded":"2023-09-27T23:30:29.428083+0000","last_fullsized":"2023-09-27T23:30:29.428083+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:34:47.282763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447931+0000","last_change":"2023-09-27T23:30:29.445370+0000","last_active":"2023-09-27T23:30:29.447931+0000","last_peered":"2023-09-27T23:30:29.447931+0000","last_clean":"2023-09-27T23:30:29.447931+0000","last_became_active":"2023-09-27T23:30:29.444846+0000","last_became_peered":"2023-09-27T23:30:29.444846+0000","last_unstale":"2023-09-27T23:30:29.447931+0000","last_undegraded":"2023-09-27T23:30:29.447931+0000","last_fullsized":"2023-09-27T23:30:29.447931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:41.665511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.442862+0000","last_change":"2023-09-27T23:30:29.439736+0000","last_active":"2023-09-27T23:30:29.442862+0000","last_peered":"2023-09-27T23:30:29.442862+0000","last_clean":"2023-09-27T23:30:29.442862+0000","last_became_active":"2023-09-27T23:30:29.439393+0000","last_became_peered":"2023-09-27T23:30:29.439393+0000","last_unstale":"2023-09-27T23:30:29.442862+0000","last_undegraded":"2023-09-27T23:30:29.442862+0000","last_fullsized":"2023-09-27T23:30:29.442862+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:08:16.844452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.448802+0000","last_change":"2023-09-27T23:30:29.445735+0000","last_active":"2023-09-27T23:30:29.448802+0000","last_peered":"2023-09-27T23:30:29.448802+0000","last_clean":"2023-09-27T23:30:29.448802+0000","last_became_active":"2023-09-27T23:30:29.445398+0000","last_became_peered":"2023-09-27T23:30:29.445398+0000","last_unstale":"2023-09-27T23:30:29.448802+0000","last_undegraded":"2023-09-27T23:30:29.448802+0000","last_fullsized":"2023-09-27T23:30:29.448802+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:31.517151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447849+0000","last_change":"2023-09-27T23:30:29.437968+0000","last_active":"2023-09-27T23:30:29.447849+0000","last_peered":"2023-09-27T23:30:29.447849+0000","last_clean":"2023-09-27T23:30:29.447849+0000","last_became_active":"2023-09-27T23:30:29.437657+0000","last_became_peered":"2023-09-27T23:30:29.437657+0000","last_unstale":"2023-09-27T23:30:29.447849+0000","last_undegraded":"2023-09-27T23:30:29.447849+0000","last_fullsized":"2023-09-27T23:30:29.447849+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:51.757870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445154+0000","last_change":"2023-09-27T23:30:29.443653+0000","last_active":"2023-09-27T23:30:29.445154+0000","last_peered":"2023-09-27T23:30:29.445154+0000","last_clean":"2023-09-27T23:30:29.445154+0000","last_became_active":"2023-09-27T23:30:29.443357+0000","last_became_peered":"2023-09-27T23:30:29.443357+0000","last_unstale":"2023-09-27T23:30:29.445154+0000","last_undegraded":"2023-09-27T23:30:29.445154+0000","last_fullsized":"2023-09-27T23:30:29.445154+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:07:01.443372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.431199+0000","last_change":"2023-09-27T23:30:29.428408+0000","last_active":"2023-09-27T23:30:29.431199+0000","last_peered":"2023-09-27T23:30:29.431199+0000","last_clean":"2023-09-27T23:30:29.431199+0000","last_became_active":"2023-09-27T23:30:29.428086+0000","last_became_peered":"2023-09-27T23:30:29.428086+0000","last_unstale":"2023-09-27T23:30:29.431199+0000","last_undegraded":"2023-09-27T23:30:29.431199+0000","last_fullsized":"2023-09-27T23:30:29.431199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:00:18.813953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425739+0000","last_change":"2023-09-27T23:30:29.414299+0000","last_active":"2023-09-27T23:30:29.425739+0000","last_peered":"2023-09-27T23:30:29.425739+0000","last_clean":"2023-09-27T23:30:29.425739+0000","last_became_active":"2023-09-27T23:30:29.414020+0000","last_became_peered":"2023-09-27T23:30:29.414020+0000","last_unstale":"2023-09-27T23:30:29.425739+0000","last_undegraded":"2023-09-27T23:30:29.425739+0000","last_fullsized":"2023-09-27T23:30:29.425739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:22:16.700995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.427793+0000","last_change":"2023-09-27T23:30:29.421116+0000","last_active":"2023-09-27T23:30:29.427793+0000","last_peered":"2023-09-27T23:30:29.427793+0000","last_clean":"2023-09-27T23:30:29.427793+0000","last_became_active":"2023-09-27T23:30:29.420805+0000","last_became_peered":"2023-09-27T23:30:29.420805+0000","last_unstale":"2023-09-27T23:30:29.427793+0000","last_undegraded":"2023-09-27T23:30:29.427793+0000","last_fullsized":"2023-09-27T23:30:29.427793+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:00:08.227772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.417625+0000","last_change":"2023-09-27T23:30:29.415644+0000","last_active":"2023-09-27T23:30:29.417625+0000","last_peered":"2023-09-27T23:30:29.417625+0000","last_clean":"2023-09-27T23:30:29.417625+0000","last_became_active":"2023-09-27T23:30:29.415337+0000","last_became_peered":"2023-09-27T23:30:29.415337+0000","last_unstale":"2023-09-27T23:30:29.417625+0000","last_undegraded":"2023-09-27T23:30:29.417625+0000","last_fullsized":"2023-09-27T23:30:29.417625+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:06:13.848158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425914+0000","last_change":"2023-09-27T23:30:29.416772+0000","last_active":"2023-09-27T23:30:29.425914+0000","last_peered":"2023-09-27T23:30:29.425914+0000","last_clean":"2023-09-27T23:30:29.425914+0000","last_became_active":"2023-09-27T23:30:29.416476+0000","last_became_peered":"2023-09-27T23:30:29.416476+0000","last_unstale":"2023-09-27T23:30:29.425914+0000","last_undegraded":"2023-09-27T23:30:29.425914+0000","last_fullsized":"2023-09-27T23:30:29.425914+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:56:37.846630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436580+0000","last_change":"2023-09-27T23:30:29.434943+0000","last_active":"2023-09-27T23:30:29.436580+0000","last_peered":"2023-09-27T23:30:29.436580+0000","last_clean":"2023-09-27T23:30:29.436580+0000","last_became_active":"2023-09-27T23:30:29.434667+0000","last_became_peered":"2023-09-27T23:30:29.434667+0000","last_unstale":"2023-09-27T23:30:29.436580+0000","last_undegraded":"2023-09-27T23:30:29.436580+0000","last_fullsized":"2023-09-27T23:30:29.436580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:48.502103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447819+0000","last_change":"2023-09-27T23:30:29.434868+0000","last_active":"2023-09-27T23:30:29.447819+0000","last_peered":"2023-09-27T23:30:29.447819+0000","last_clean":"2023-09-27T23:30:29.447819+0000","last_became_active":"2023-09-27T23:30:29.434622+0000","last_became_peered":"2023-09-27T23:30:29.434622+0000","last_unstale":"2023-09-27T23:30:29.447819+0000","last_undegraded":"2023-09-27T23:30:29.447819+0000","last_fullsized":"2023-09-27T23:30:29.447819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:31:18.606971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434647+0000","last_change":"2023-09-27T23:30:29.432321+0000","last_active":"2023-09-27T23:30:29.434647+0000","last_peered":"2023-09-27T23:30:29.434647+0000","last_clean":"2023-09-27T23:30:29.434647+0000","last_became_active":"2023-09-27T23:30:29.432019+0000","last_became_peered":"2023-09-27T23:30:29.432019+0000","last_unstale":"2023-09-27T23:30:29.434647+0000","last_undegraded":"2023-09-27T23:30:29.434647+0000","last_fullsized":"2023-09-27T23:30:29.434647+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:31:44.404078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.427480+0000","last_change":"2023-09-27T23:30:29.423146+0000","last_active":"2023-09-27T23:30:29.427480+0000","last_peered":"2023-09-27T23:30:29.427480+0000","last_clean":"2023-09-27T23:30:29.427480+0000","last_became_active":"2023-09-27T23:30:29.422727+0000","last_became_peered":"2023-09-27T23:30:29.422727+0000","last_unstale":"2023-09-27T23:30:29.427480+0000","last_undegraded":"2023-09-27T23:30:29.427480+0000","last_fullsized":"2023-09-27T23:30:29.427480+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:47.331660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.427471+0000","last_change":"2023-09-27T23:30:29.423104+0000","last_active":"2023-09-27T23:30:29.427471+0000","last_peered":"2023-09-27T23:30:29.427471+0000","last_clean":"2023-09-27T23:30:29.427471+0000","last_became_active":"2023-09-27T23:30:29.422754+0000","last_became_peered":"2023-09-27T23:30:29.422754+0000","last_unstale":"2023-09-27T23:30:29.427471+0000","last_undegraded":"2023-09-27T23:30:29.427471+0000","last_fullsized":"2023-09-27T23:30:29.427471+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:11:18.794394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.442281+0000","last_change":"2023-09-27T23:30:29.438643+0000","last_active":"2023-09-27T23:30:29.442281+0000","last_peered":"2023-09-27T23:30:29.442281+0000","last_clean":"2023-09-27T23:30:29.442281+0000","last_became_active":"2023-09-27T23:30:29.438340+0000","last_became_peered":"2023-09-27T23:30:29.438340+0000","last_unstale":"2023-09-27T23:30:29.442281+0000","last_undegraded":"2023-09-27T23:30:29.442281+0000","last_fullsized":"2023-09-27T23:30:29.442281+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:33.738047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437012+0000","last_change":"2023-09-27T23:30:29.423266+0000","last_active":"2023-09-27T23:30:29.437012+0000","last_peered":"2023-09-27T23:30:29.437012+0000","last_clean":"2023-09-27T23:30:29.437012+0000","last_became_active":"2023-09-27T23:30:29.422943+0000","last_became_peered":"2023-09-27T23:30:29.422943+0000","last_unstale":"2023-09-27T23:30:29.437012+0000","last_undegraded":"2023-09-27T23:30:29.437012+0000","last_fullsized":"2023-09-27T23:30:29.437012+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:27.509407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.428109+0000","last_change":"2023-09-27T23:30:29.425111+0000","last_active":"2023-09-27T23:30:29.428109+0000","last_peered":"2023-09-27T23:30:29.428109+0000","last_clean":"2023-09-27T23:30:29.428109+0000","last_became_active":"2023-09-27T23:30:29.424844+0000","last_became_peered":"2023-09-27T23:30:29.424844+0000","last_unstale":"2023-09-27T23:30:29.428109+0000","last_undegraded":"2023-09-27T23:30:29.428109+0000","last_fullsized":"2023-09-27T23:30:29.428109+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:28.843203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.433954+0000","last_change":"2023-09-27T23:30:29.429639+0000","last_active":"2023-09-27T23:30:29.433954+0000","last_peered":"2023-09-27T23:30:29.433954+0000","last_clean":"2023-09-27T23:30:29.433954+0000","last_became_active":"2023-09-27T23:30:29.429278+0000","last_became_peered":"2023-09-27T23:30:29.429278+0000","last_unstale":"2023-09-27T23:30:29.433954+0000","last_undegraded":"2023-09-27T23:30:29.433954+0000","last_fullsized":"2023-09-27T23:30:29.433954+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:40:20.419882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437161+0000","last_change":"2023-09-27T23:30:29.426447+0000","last_active":"2023-09-27T23:30:29.437161+0000","last_peered":"2023-09-27T23:30:29.437161+0000","last_clean":"2023-09-27T23:30:29.437161+0000","last_became_active":"2023-09-27T23:30:29.426139+0000","last_became_peered":"2023-09-27T23:30:29.426139+0000","last_unstale":"2023-09-27T23:30:29.437161+0000","last_undegraded":"2023-09-27T23:30:29.437161+0000","last_fullsized":"2023-09-27T23:30:29.437161+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:26:07.958558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.418984+0000","last_change":"2023-09-27T23:30:29.415639+0000","last_active":"2023-09-27T23:30:29.418984+0000","last_peered":"2023-09-27T23:30:29.418984+0000","last_clean":"2023-09-27T23:30:29.418984+0000","last_became_active":"2023-09-27T23:30:29.415380+0000","last_became_peered":"2023-09-27T23:30:29.415380+0000","last_unstale":"2023-09-27T23:30:29.418984+0000","last_undegraded":"2023-09-27T23:30:29.418984+0000","last_fullsized":"2023-09-27T23:30:29.418984+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:53.072911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436674+0000","last_change":"2023-09-27T23:30:29.435116+0000","last_active":"2023-09-27T23:30:29.436674+0000","last_peered":"2023-09-27T23:30:29.436674+0000","last_clean":"2023-09-27T23:30:29.436674+0000","last_became_active":"2023-09-27T23:30:29.434782+0000","last_became_peered":"2023-09-27T23:30:29.434782+0000","last_unstale":"2023-09-27T23:30:29.436674+0000","last_undegraded":"2023-09-27T23:30:29.436674+0000","last_fullsized":"2023-09-27T23:30:29.436674+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:51:53.554558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.428763+0000","last_change":"2023-09-27T23:30:29.422326+0000","last_active":"2023-09-27T23:30:29.428763+0000","last_peered":"2023-09-27T23:30:29.428763+0000","last_clean":"2023-09-27T23:30:29.428763+0000","last_became_active":"2023-09-27T23:30:29.422038+0000","last_became_peered":"2023-09-27T23:30:29.422038+0000","last_unstale":"2023-09-27T23:30:29.428763+0000","last_undegraded":"2023-09-27T23:30:29.428763+0000","last_fullsized":"2023-09-27T23:30:29.428763+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:55:23.042248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434942+0000","last_change":"2023-09-27T23:30:29.421559+0000","last_active":"2023-09-27T23:30:29.434942+0000","last_peered":"2023-09-27T23:30:29.434942+0000","last_clean":"2023-09-27T23:30:29.434942+0000","last_became_active":"2023-09-27T23:30:29.420693+0000","last_became_peered":"2023-09-27T23:30:29.420693+0000","last_unstale":"2023-09-27T23:30:29.434942+0000","last_undegraded":"2023-09-27T23:30:29.434942+0000","last_fullsized":"2023-09-27T23:30:29.434942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:11.243612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.430604+0000","last_change":"2023-09-27T23:30:29.425574+0000","last_active":"2023-09-27T23:30:29.430604+0000","last_peered":"2023-09-27T23:30:29.430604+0000","last_clean":"2023-09-27T23:30:29.430604+0000","last_became_active":"2023-09-27T23:30:29.425283+0000","last_became_peered":"2023-09-27T23:30:29.425283+0000","last_unstale":"2023-09-27T23:30:29.430604+0000","last_undegraded":"2023-09-27T23:30:29.430604+0000","last_fullsized":"2023-09-27T23:30:29.430604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:38:27.173838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"114'4","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.440936+0000","last_change":"2023-09-27T23:30:29.032706+0000","last_active":"2023-09-27T23:30:29.440936+0000","last_peered":"2023-09-27T23:30:29.440936+0000","last_clean":"2023-09-27T23:30:29.440936+0000","last_became_active":"2023-09-27T23:30:29.032361+0000","last_became_peered":"2023-09-27T23:30:29.032361+0000","last_unstale":"2023-09-27T23:30:29.440936+0000","last_undegraded":"2023-09-27T23:30:29.440936+0000","last_fullsized":"2023-09-27T23:30:29.440936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:22:15.688478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.432300+0000","last_change":"2023-09-27T23:30:29.428182+0000","last_active":"2023-09-27T23:30:29.432300+0000","last_peered":"2023-09-27T23:30:29.432300+0000","last_clean":"2023-09-27T23:30:29.432300+0000","last_became_active":"2023-09-27T23:30:29.427923+0000","last_became_peered":"2023-09-27T23:30:29.427923+0000","last_unstale":"2023-09-27T23:30:29.432300+0000","last_undegraded":"2023-09-27T23:30:29.432300+0000","last_fullsized":"2023-09-27T23:30:29.432300+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:08:16.844452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.439328+0000","last_change":"2023-09-27T23:30:29.040533+0000","last_active":"2023-09-27T23:30:29.439328+0000","last_peered":"2023-09-27T23:30:29.439328+0000","last_clean":"2023-09-27T23:30:29.439328+0000","last_became_active":"2023-09-27T23:30:29.040198+0000","last_became_peered":"2023-09-27T23:30:29.040198+0000","last_unstale":"2023-09-27T23:30:29.439328+0000","last_undegraded":"2023-09-27T23:30:29.439328+0000","last_fullsized":"2023-09-27T23:30:29.439328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:48.918177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445884+0000","last_change":"2023-09-27T23:30:29.443588+0000","last_active":"2023-09-27T23:30:29.445884+0000","last_peered":"2023-09-27T23:30:29.445884+0000","last_clean":"2023-09-27T23:30:29.445884+0000","last_became_active":"2023-09-27T23:30:29.443289+0000","last_became_peered":"2023-09-27T23:30:29.443289+0000","last_unstale":"2023-09-27T23:30:29.445884+0000","last_undegraded":"2023-09-27T23:30:29.445884+0000","last_fullsized":"2023-09-27T23:30:29.445884+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:57:06.402886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436754+0000","last_change":"2023-09-27T23:30:29.045530+0000","last_active":"2023-09-27T23:30:29.436754+0000","last_peered":"2023-09-27T23:30:29.436754+0000","last_clean":"2023-09-27T23:30:29.436754+0000","last_became_active":"2023-09-27T23:30:29.045159+0000","last_became_peered":"2023-09-27T23:30:29.045159+0000","last_unstale":"2023-09-27T23:30:29.436754+0000","last_undegraded":"2023-09-27T23:30:29.436754+0000","last_fullsized":"2023-09-27T23:30:29.436754+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:46:57.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445278+0000","last_change":"2023-09-27T23:30:29.434541+0000","last_active":"2023-09-27T23:30:29.445278+0000","last_peered":"2023-09-27T23:30:29.445278+0000","last_clean":"2023-09-27T23:30:29.445278+0000","last_became_active":"2023-09-27T23:30:29.434260+0000","last_became_peered":"2023-09-27T23:30:29.434260+0000","last_unstale":"2023-09-27T23:30:29.445278+0000","last_undegraded":"2023-09-27T23:30:29.445278+0000","last_fullsized":"2023-09-27T23:30:29.445278+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:26:16.902235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.444176+0000","last_change":"2023-09-27T23:30:29.044128+0000","last_active":"2023-09-27T23:30:29.444176+0000","last_peered":"2023-09-27T23:30:29.444176+0000","last_clean":"2023-09-27T23:30:29.444176+0000","last_became_active":"2023-09-27T23:30:29.043721+0000","last_became_peered":"2023-09-27T23:30:29.043721+0000","last_unstale":"2023-09-27T23:30:29.444176+0000","last_undegraded":"2023-09-27T23:30:29.444176+0000","last_fullsized":"2023-09-27T23:30:29.444176+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:04:15.339206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.448301+0000","last_change":"2023-09-27T23:30:29.445632+0000","last_active":"2023-09-27T23:30:29.448301+0000","last_peered":"2023-09-27T23:30:29.448301+0000","last_clean":"2023-09-27T23:30:29.448301+0000","last_became_active":"2023-09-27T23:30:29.445355+0000","last_became_peered":"2023-09-27T23:30:29.445355+0000","last_unstale":"2023-09-27T23:30:29.448301+0000","last_undegraded":"2023-09-27T23:30:29.448301+0000","last_fullsized":"2023-09-27T23:30:29.448301+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:55:37.173433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436939+0000","last_change":"2023-09-27T23:30:29.032760+0000","last_active":"2023-09-27T23:30:29.436939+0000","last_peered":"2023-09-27T23:30:29.436939+0000","last_clean":"2023-09-27T23:30:29.436939+0000","last_became_active":"2023-09-27T23:30:29.032431+0000","last_became_peered":"2023-09-27T23:30:29.032431+0000","last_unstale":"2023-09-27T23:30:29.436939+0000","last_undegraded":"2023-09-27T23:30:29.436939+0000","last_fullsized":"2023-09-27T23:30:29.436939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:09:55.740875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434178+0000","last_change":"2023-09-27T23:30:29.430408+0000","last_active":"2023-09-27T23:30:29.434178+0000","last_peered":"2023-09-27T23:30:29.434178+0000","last_clean":"2023-09-27T23:30:29.434178+0000","last_became_active":"2023-09-27T23:30:29.430075+0000","last_became_peered":"2023-09-27T23:30:29.430075+0000","last_unstale":"2023-09-27T23:30:29.434178+0000","last_undegraded":"2023-09-27T23:30:29.434178+0000","last_fullsized":"2023-09-27T23:30:29.434178+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:15:04.027176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437383+0000","last_change":"2023-09-27T23:30:29.043813+0000","last_active":"2023-09-27T23:30:29.437383+0000","last_peered":"2023-09-27T23:30:29.437383+0000","last_clean":"2023-09-27T23:30:29.437383+0000","last_became_active":"2023-09-27T23:30:29.043500+0000","last_became_peered":"2023-09-27T23:30:29.043500+0000","last_unstale":"2023-09-27T23:30:29.437383+0000","last_undegraded":"2023-09-27T23:30:29.437383+0000","last_fullsized":"2023-09-27T23:30:29.437383+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:21.084187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.422272+0000","last_change":"2023-09-27T23:30:29.413897+0000","last_active":"2023-09-27T23:30:29.422272+0000","last_peered":"2023-09-27T23:30:29.422272+0000","last_clean":"2023-09-27T23:30:29.422272+0000","last_became_active":"2023-09-27T23:30:29.413600+0000","last_became_peered":"2023-09-27T23:30:29.413600+0000","last_unstale":"2023-09-27T23:30:29.422272+0000","last_undegraded":"2023-09-27T23:30:29.422272+0000","last_fullsized":"2023-09-27T23:30:29.422272+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:24.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.415901+0000","last_change":"2023-09-27T23:30:29.046037+0000","last_active":"2023-09-27T23:30:29.415901+0000","last_peered":"2023-09-27T23:30:29.415901+0000","last_clean":"2023-09-27T23:30:29.415901+0000","last_became_active":"2023-09-27T23:30:29.045696+0000","last_became_peered":"2023-09-27T23:30:29.045696+0000","last_unstale":"2023-09-27T23:30:29.415901+0000","last_undegraded":"2023-09-27T23:30:29.415901+0000","last_fullsized":"2023-09-27T23:30:29.415901+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:32.725530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446028+0000","last_change":"2023-09-27T23:30:29.438718+0000","last_active":"2023-09-27T23:30:29.446028+0000","last_peered":"2023-09-27T23:30:29.446028+0000","last_clean":"2023-09-27T23:30:29.446028+0000","last_became_active":"2023-09-27T23:30:29.438454+0000","last_became_peered":"2023-09-27T23:30:29.438454+0000","last_unstale":"2023-09-27T23:30:29.446028+0000","last_undegraded":"2023-09-27T23:30:29.446028+0000","last_fullsized":"2023-09-27T23:30:29.446028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:58:47.626281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401580+0000","last_change":"2023-09-27T23:30:29.035856+0000","last_active":"2023-09-27T23:30:29.401580+0000","last_peered":"2023-09-27T23:30:29.401580+0000","last_clean":"2023-09-27T23:30:29.401580+0000","last_became_active":"2023-09-27T23:30:29.035384+0000","last_became_peered":"2023-09-27T23:30:29.035384+0000","last_unstale":"2023-09-27T23:30:29.401580+0000","last_undegraded":"2023-09-27T23:30:29.401580+0000","last_fullsized":"2023-09-27T23:30:29.401580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:16:29.194838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.438525+0000","last_change":"2023-09-27T23:30:29.433754+0000","last_active":"2023-09-27T23:30:29.438525+0000","last_peered":"2023-09-27T23:30:29.438525+0000","last_clean":"2023-09-27T23:30:29.438525+0000","last_became_active":"2023-09-27T23:30:29.433494+0000","last_became_peered":"2023-09-27T23:30:29.433494+0000","last_unstale":"2023-09-27T23:30:29.438525+0000","last_undegraded":"2023-09-27T23:30:29.438525+0000","last_fullsized":"2023-09-27T23:30:29.438525+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:46:44.768611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402510+0000","last_change":"2023-09-27T23:30:29.022763+0000","last_active":"2023-09-27T23:30:29.402510+0000","last_peered":"2023-09-27T23:30:29.402510+0000","last_clean":"2023-09-27T23:30:29.402510+0000","last_became_active":"2023-09-27T23:30:29.021867+0000","last_became_peered":"2023-09-27T23:30:29.021867+0000","last_unstale":"2023-09-27T23:30:29.402510+0000","last_undegraded":"2023-09-27T23:30:29.402510+0000","last_fullsized":"2023-09-27T23:30:29.402510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:37.446360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445578+0000","last_change":"2023-09-27T23:30:29.434529+0000","last_active":"2023-09-27T23:30:29.445578+0000","last_peered":"2023-09-27T23:30:29.445578+0000","last_clean":"2023-09-27T23:30:29.445578+0000","last_became_active":"2023-09-27T23:30:29.434185+0000","last_became_peered":"2023-09-27T23:30:29.434185+0000","last_unstale":"2023-09-27T23:30:29.445578+0000","last_undegraded":"2023-09-27T23:30:29.445578+0000","last_fullsized":"2023-09-27T23:30:29.445578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:09:06.448424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.404473+0000","last_change":"2023-09-27T23:30:29.039733+0000","last_active":"2023-09-27T23:30:29.404473+0000","last_peered":"2023-09-27T23:30:29.404473+0000","last_clean":"2023-09-27T23:30:29.404473+0000","last_became_active":"2023-09-27T23:30:29.039448+0000","last_became_peered":"2023-09-27T23:30:29.039448+0000","last_unstale":"2023-09-27T23:30:29.404473+0000","last_undegraded":"2023-09-27T23:30:29.404473+0000","last_fullsized":"2023-09-27T23:30:29.404473+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:15:03.014659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.436954+0000","last_change":"2023-09-27T23:30:29.423349+0000","last_active":"2023-09-27T23:30:29.436954+0000","last_peered":"2023-09-27T23:30:29.436954+0000","last_clean":"2023-09-27T23:30:29.436954+0000","last_became_active":"2023-09-27T23:30:29.422872+0000","last_became_peered":"2023-09-27T23:30:29.422872+0000","last_unstale":"2023-09-27T23:30:29.436954+0000","last_undegraded":"2023-09-27T23:30:29.436954+0000","last_fullsized":"2023-09-27T23:30:29.436954+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:16:33.345295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"114'3","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.447163+0000","last_change":"2023-09-27T23:30:29.043575+0000","last_active":"2023-09-27T23:30:29.447163+0000","last_peered":"2023-09-27T23:30:29.447163+0000","last_clean":"2023-09-27T23:30:29.447163+0000","last_became_active":"2023-09-27T23:30:29.043299+0000","last_became_peered":"2023-09-27T23:30:29.043299+0000","last_unstale":"2023-09-27T23:30:29.447163+0000","last_undegraded":"2023-09-27T23:30:29.447163+0000","last_fullsized":"2023-09-27T23:30:29.447163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:26:27.497772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.428615+0000","last_change":"2023-09-27T23:30:29.424145+0000","last_active":"2023-09-27T23:30:29.428615+0000","last_peered":"2023-09-27T23:30:29.428615+0000","last_clean":"2023-09-27T23:30:29.428615+0000","last_became_active":"2023-09-27T23:30:29.423793+0000","last_became_peered":"2023-09-27T23:30:29.423793+0000","last_unstale":"2023-09-27T23:30:29.428615+0000","last_undegraded":"2023-09-27T23:30:29.428615+0000","last_fullsized":"2023-09-27T23:30:29.428615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:40:10.514562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.449495+0000","last_change":"2023-09-27T23:30:29.040154+0000","last_active":"2023-09-27T23:30:29.449495+0000","last_peered":"2023-09-27T23:30:29.449495+0000","last_clean":"2023-09-27T23:30:29.449495+0000","last_became_active":"2023-09-27T23:30:29.039855+0000","last_became_peered":"2023-09-27T23:30:29.039855+0000","last_unstale":"2023-09-27T23:30:29.449495+0000","last_undegraded":"2023-09-27T23:30:29.449495+0000","last_fullsized":"2023-09-27T23:30:29.449495+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:26:42.757699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.426057+0000","last_change":"2023-09-27T23:30:29.418011+0000","last_active":"2023-09-27T23:30:29.426057+0000","last_peered":"2023-09-27T23:30:29.426057+0000","last_clean":"2023-09-27T23:30:29.426057+0000","last_became_active":"2023-09-27T23:30:29.417654+0000","last_became_peered":"2023-09-27T23:30:29.417654+0000","last_unstale":"2023-09-27T23:30:29.426057+0000","last_undegraded":"2023-09-27T23:30:29.426057+0000","last_fullsized":"2023-09-27T23:30:29.426057+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:27.509407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.444060+0000","last_change":"2023-09-27T23:30:29.040519+0000","last_active":"2023-09-27T23:30:29.444060+0000","last_peered":"2023-09-27T23:30:29.444060+0000","last_clean":"2023-09-27T23:30:29.444060+0000","last_became_active":"2023-09-27T23:30:29.038941+0000","last_became_peered":"2023-09-27T23:30:29.038941+0000","last_unstale":"2023-09-27T23:30:29.444060+0000","last_undegraded":"2023-09-27T23:30:29.444060+0000","last_fullsized":"2023-09-27T23:30:29.444060+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:48.020088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434186+0000","last_change":"2023-09-27T23:30:29.422077+0000","last_active":"2023-09-27T23:30:29.434186+0000","last_peered":"2023-09-27T23:30:29.434186+0000","last_clean":"2023-09-27T23:30:29.434186+0000","last_became_active":"2023-09-27T23:30:29.421784+0000","last_became_peered":"2023-09-27T23:30:29.421784+0000","last_unstale":"2023-09-27T23:30:29.434186+0000","last_undegraded":"2023-09-27T23:30:29.434186+0000","last_fullsized":"2023-09-27T23:30:29.434186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:35:59.674679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.414781+0000","last_change":"2023-09-27T23:30:29.039587+0000","last_active":"2023-09-27T23:30:29.414781+0000","last_peered":"2023-09-27T23:30:29.414781+0000","last_clean":"2023-09-27T23:30:29.414781+0000","last_became_active":"2023-09-27T23:30:29.039298+0000","last_became_peered":"2023-09-27T23:30:29.039298+0000","last_unstale":"2023-09-27T23:30:29.414781+0000","last_undegraded":"2023-09-27T23:30:29.414781+0000","last_fullsized":"2023-09-27T23:30:29.414781+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:34:14.672317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445131+0000","last_change":"2023-09-27T23:30:29.434466+0000","last_active":"2023-09-27T23:30:29.445131+0000","last_peered":"2023-09-27T23:30:29.445131+0000","last_clean":"2023-09-27T23:30:29.445131+0000","last_became_active":"2023-09-27T23:30:29.434133+0000","last_became_peered":"2023-09-27T23:30:29.434133+0000","last_unstale":"2023-09-27T23:30:29.445131+0000","last_undegraded":"2023-09-27T23:30:29.445131+0000","last_fullsized":"2023-09-27T23:30:29.445131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:46.166597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.425237+0000","last_change":"2023-09-27T23:30:29.040263+0000","last_active":"2023-09-27T23:30:29.425237+0000","last_peered":"2023-09-27T23:30:29.425237+0000","last_clean":"2023-09-27T23:30:29.425237+0000","last_became_active":"2023-09-27T23:30:29.039976+0000","last_became_peered":"2023-09-27T23:30:29.039976+0000","last_unstale":"2023-09-27T23:30:29.425237+0000","last_undegraded":"2023-09-27T23:30:29.425237+0000","last_fullsized":"2023-09-27T23:30:29.425237+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:13.841678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445625+0000","last_change":"2023-09-27T23:30:29.443610+0000","last_active":"2023-09-27T23:30:29.445625+0000","last_peered":"2023-09-27T23:30:29.445625+0000","last_clean":"2023-09-27T23:30:29.445625+0000","last_became_active":"2023-09-27T23:30:29.443355+0000","last_became_peered":"2023-09-27T23:30:29.443355+0000","last_unstale":"2023-09-27T23:30:29.445625+0000","last_undegraded":"2023-09-27T23:30:29.445625+0000","last_fullsized":"2023-09-27T23:30:29.445625+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:29.029841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437983+0000","last_change":"2023-09-27T23:30:29.031398+0000","last_active":"2023-09-27T23:30:29.437983+0000","last_peered":"2023-09-27T23:30:29.437983+0000","last_clean":"2023-09-27T23:30:29.437983+0000","last_became_active":"2023-09-27T23:30:29.031067+0000","last_became_peered":"2023-09-27T23:30:29.031067+0000","last_unstale":"2023-09-27T23:30:29.437983+0000","last_undegraded":"2023-09-27T23:30:29.437983+0000","last_fullsized":"2023-09-27T23:30:29.437983+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:24:54.823350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.427978+0000","last_change":"2023-09-27T23:30:29.423498+0000","last_active":"2023-09-27T23:30:29.427978+0000","last_peered":"2023-09-27T23:30:29.427978+0000","last_clean":"2023-09-27T23:30:29.427978+0000","last_became_active":"2023-09-27T23:30:29.423174+0000","last_became_peered":"2023-09-27T23:30:29.423174+0000","last_unstale":"2023-09-27T23:30:29.427978+0000","last_undegraded":"2023-09-27T23:30:29.427978+0000","last_fullsized":"2023-09-27T23:30:29.427978+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:26:43.770216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446099+0000","last_change":"2023-09-27T23:30:29.040980+0000","last_active":"2023-09-27T23:30:29.446099+0000","last_peered":"2023-09-27T23:30:29.446099+0000","last_clean":"2023-09-27T23:30:29.446099+0000","last_became_active":"2023-09-27T23:30:29.040677+0000","last_became_peered":"2023-09-27T23:30:29.040677+0000","last_unstale":"2023-09-27T23:30:29.446099+0000","last_undegraded":"2023-09-27T23:30:29.446099+0000","last_fullsized":"2023-09-27T23:30:29.446099+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:05:12.161490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.434246+0000","last_change":"2023-09-27T23:30:29.420956+0000","last_active":"2023-09-27T23:30:29.434246+0000","last_peered":"2023-09-27T23:30:29.434246+0000","last_clean":"2023-09-27T23:30:29.434246+0000","last_became_active":"2023-09-27T23:30:29.419860+0000","last_became_peered":"2023-09-27T23:30:29.419860+0000","last_unstale":"2023-09-27T23:30:29.434246+0000","last_undegraded":"2023-09-27T23:30:29.434246+0000","last_fullsized":"2023-09-27T23:30:29.434246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:10.232721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.409890+0000","last_change":"2023-09-27T23:30:29.038630+0000","last_active":"2023-09-27T23:30:29.409890+0000","last_peered":"2023-09-27T23:30:29.409890+0000","last_clean":"2023-09-27T23:30:29.409890+0000","last_became_active":"2023-09-27T23:30:29.038337+0000","last_became_peered":"2023-09-27T23:30:29.038337+0000","last_unstale":"2023-09-27T23:30:29.409890+0000","last_undegraded":"2023-09-27T23:30:29.409890+0000","last_fullsized":"2023-09-27T23:30:29.409890+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:00:07.215255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437190+0000","last_change":"2023-09-27T23:30:29.428257+0000","last_active":"2023-09-27T23:30:29.437190+0000","last_peered":"2023-09-27T23:30:29.437190+0000","last_clean":"2023-09-27T23:30:29.437190+0000","last_became_active":"2023-09-27T23:30:29.427972+0000","last_became_peered":"2023-09-27T23:30:29.427972+0000","last_unstale":"2023-09-27T23:30:29.437190+0000","last_undegraded":"2023-09-27T23:30:29.437190+0000","last_fullsized":"2023-09-27T23:30:29.437190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:15:35.465950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.406004+0000","last_change":"2023-09-27T23:30:29.044148+0000","last_active":"2023-09-27T23:30:29.406004+0000","last_peered":"2023-09-27T23:30:29.406004+0000","last_clean":"2023-09-27T23:30:29.406004+0000","last_became_active":"2023-09-27T23:30:29.043032+0000","last_became_peered":"2023-09-27T23:30:29.043032+0000","last_unstale":"2023-09-27T23:30:29.406004+0000","last_undegraded":"2023-09-27T23:30:29.406004+0000","last_fullsized":"2023-09-27T23:30:29.406004+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:05.569195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437200+0000","last_change":"2023-09-27T23:30:29.426542+0000","last_active":"2023-09-27T23:30:29.437200+0000","last_peered":"2023-09-27T23:30:29.437200+0000","last_clean":"2023-09-27T23:30:29.437200+0000","last_became_active":"2023-09-27T23:30:29.426187+0000","last_became_peered":"2023-09-27T23:30:29.426187+0000","last_unstale":"2023-09-27T23:30:29.437200+0000","last_undegraded":"2023-09-27T23:30:29.437200+0000","last_fullsized":"2023-09-27T23:30:29.437200+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:04:16.351723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.416587+0000","last_change":"2023-09-27T23:30:29.032606+0000","last_active":"2023-09-27T23:30:29.416587+0000","last_peered":"2023-09-27T23:30:29.416587+0000","last_clean":"2023-09-27T23:30:29.416587+0000","last_became_active":"2023-09-27T23:30:29.032331+0000","last_became_peered":"2023-09-27T23:30:29.032331+0000","last_unstale":"2023-09-27T23:30:29.416587+0000","last_undegraded":"2023-09-27T23:30:29.416587+0000","last_fullsized":"2023-09-27T23:30:29.416587+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:40:19.407365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445608+0000","last_change":"2023-09-27T23:30:29.443653+0000","last_active":"2023-09-27T23:30:29.445608+0000","last_peered":"2023-09-27T23:30:29.445608+0000","last_clean":"2023-09-27T23:30:29.445608+0000","last_became_active":"2023-09-27T23:30:29.443342+0000","last_became_peered":"2023-09-27T23:30:29.443342+0000","last_unstale":"2023-09-27T23:30:29.445608+0000","last_undegraded":"2023-09-27T23:30:29.445608+0000","last_fullsized":"2023-09-27T23:30:29.445608+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:30:09.079704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.405880+0000","last_change":"2023-09-27T23:30:29.043716+0000","last_active":"2023-09-27T23:30:29.405880+0000","last_peered":"2023-09-27T23:30:29.405880+0000","last_clean":"2023-09-27T23:30:29.405880+0000","last_became_active":"2023-09-27T23:30:29.043356+0000","last_became_peered":"2023-09-27T23:30:29.043356+0000","last_unstale":"2023-09-27T23:30:29.405880+0000","last_undegraded":"2023-09-27T23:30:29.405880+0000","last_fullsized":"2023-09-27T23:30:29.405880+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:08:15.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.446021+0000","last_change":"2023-09-27T23:30:29.435122+0000","last_active":"2023-09-27T23:30:29.446021+0000","last_peered":"2023-09-27T23:30:29.446021+0000","last_clean":"2023-09-27T23:30:29.446021+0000","last_became_active":"2023-09-27T23:30:29.434814+0000","last_became_peered":"2023-09-27T23:30:29.434814+0000","last_unstale":"2023-09-27T23:30:29.446021+0000","last_undegraded":"2023-09-27T23:30:29.446021+0000","last_fullsized":"2023-09-27T23:30:29.446021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:54.480949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.417071+0000","last_change":"2023-09-27T23:30:29.040276+0000","last_active":"2023-09-27T23:30:29.417071+0000","last_peered":"2023-09-27T23:30:29.417071+0000","last_clean":"2023-09-27T23:30:29.417071+0000","last_became_active":"2023-09-27T23:30:29.039872+0000","last_became_peered":"2023-09-27T23:30:29.039872+0000","last_unstale":"2023-09-27T23:30:29.417071+0000","last_undegraded":"2023-09-27T23:30:29.417071+0000","last_fullsized":"2023-09-27T23:30:29.417071+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:15:03.014659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445572+0000","last_change":"2023-09-27T23:30:29.443604+0000","last_active":"2023-09-27T23:30:29.445572+0000","last_peered":"2023-09-27T23:30:29.445572+0000","last_clean":"2023-09-27T23:30:29.445572+0000","last_became_active":"2023-09-27T23:30:29.443279+0000","last_became_peered":"2023-09-27T23:30:29.443279+0000","last_unstale":"2023-09-27T23:30:29.445572+0000","last_undegraded":"2023-09-27T23:30:29.445572+0000","last_fullsized":"2023-09-27T23:30:29.445572+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:28:09.452149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.404456+0000","last_change":"2023-09-27T23:30:29.037950+0000","last_active":"2023-09-27T23:30:29.404456+0000","last_peered":"2023-09-27T23:30:29.404456+0000","last_clean":"2023-09-27T23:30:29.404456+0000","last_became_active":"2023-09-27T23:30:29.037667+0000","last_became_peered":"2023-09-27T23:30:29.037667+0000","last_unstale":"2023-09-27T23:30:29.404456+0000","last_undegraded":"2023-09-27T23:30:29.404456+0000","last_fullsized":"2023-09-27T23:30:29.404456+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:14:57.904897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.437300+0000","last_change":"2023-09-27T23:30:29.426441+0000","last_active":"2023-09-27T23:30:29.437300+0000","last_peered":"2023-09-27T23:30:29.437300+0000","last_clean":"2023-09-27T23:30:29.437300+0000","last_became_active":"2023-09-27T23:30:29.426073+0000","last_became_peered":"2023-09-27T23:30:29.426073+0000","last_unstale":"2023-09-27T23:30:29.437300+0000","last_undegraded":"2023-09-27T23:30:29.437300+0000","last_fullsized":"2023-09-27T23:30:29.437300+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:24.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.413893+0000","last_change":"2023-09-27T23:30:29.044206+0000","last_active":"2023-09-27T23:30:29.413893+0000","last_peered":"2023-09-27T23:30:29.413893+0000","last_clean":"2023-09-27T23:30:29.413893+0000","last_became_active":"2023-09-27T23:30:29.043831+0000","last_became_peered":"2023-09-27T23:30:29.043831+0000","last_unstale":"2023-09-27T23:30:29.413893+0000","last_undegraded":"2023-09-27T23:30:29.413893+0000","last_fullsized":"2023-09-27T23:30:29.413893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:03:07.100864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.426664+0000","last_change":"2023-09-27T23:30:29.416702+0000","last_active":"2023-09-27T23:30:29.426664+0000","last_peered":"2023-09-27T23:30:29.426664+0000","last_clean":"2023-09-27T23:30:29.426664+0000","last_became_active":"2023-09-27T23:30:29.416380+0000","last_became_peered":"2023-09-27T23:30:29.416380+0000","last_unstale":"2023-09-27T23:30:29.426664+0000","last_undegraded":"2023-09-27T23:30:29.426664+0000","last_fullsized":"2023-09-27T23:30:29.426664+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:57.026518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.431495+0000","last_change":"2023-09-27T23:30:29.039847+0000","last_active":"2023-09-27T23:30:29.431495+0000","last_peered":"2023-09-27T23:30:29.431495+0000","last_clean":"2023-09-27T23:30:29.431495+0000","last_became_active":"2023-09-27T23:30:29.039540+0000","last_became_peered":"2023-09-27T23:30:29.039540+0000","last_unstale":"2023-09-27T23:30:29.431495+0000","last_undegraded":"2023-09-27T23:30:29.431495+0000","last_fullsized":"2023-09-27T23:30:29.431495+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:38:26.161321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"103'205","reported_seq":560,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.415545+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:30:29.415545+0000","last_peered":"2023-09-27T23:30:29.415545+0000","last_clean":"2023-09-27T23:30:29.415545+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:30:29.415545+0000","last_undegraded":"2023-09-27T23:30:29.415545+0000","last_fullsized":"2023-09-27T23:30:29.415545+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":100,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.423586+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:30:29.423586+0000","last_peered":"2023-09-27T23:30:29.423586+0000","last_clean":"2023-09-27T23:30:29.423586+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:30:29.423586+0000","last_undegraded":"2023-09-27T23:30:29.423586+0000","last_fullsized":"2023-09-27T23:30:29.423586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.448822+0000","last_change":"2023-09-27T23:30:29.447579+0000","last_active":"2023-09-27T23:30:29.448822+0000","last_peered":"2023-09-27T23:30:29.448822+0000","last_clean":"2023-09-27T23:30:29.448822+0000","last_became_active":"2023-09-27T23:30:29.446464+0000","last_became_peered":"2023-09-27T23:30:29.446464+0000","last_unstale":"2023-09-27T23:30:29.448822+0000","last_undegraded":"2023-09-27T23:30:29.448822+0000","last_fullsized":"2023-09-27T23:30:29.448822+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:25:24.056763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.406449+0000","last_change":"2023-09-27T23:30:29.043453+0000","last_active":"2023-09-27T23:30:29.406449+0000","last_peered":"2023-09-27T23:30:29.406449+0000","last_clean":"2023-09-27T23:30:29.406449+0000","last_became_active":"2023-09-27T23:30:29.043052+0000","last_became_peered":"2023-09-27T23:30:29.043052+0000","last_unstale":"2023-09-27T23:30:29.406449+0000","last_undegraded":"2023-09-27T23:30:29.406449+0000","last_fullsized":"2023-09-27T23:30:29.406449+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:28:26.496890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.415418+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:30:29.415418+0000","last_peered":"2023-09-27T23:30:29.415418+0000","last_clean":"2023-09-27T23:30:29.415418+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:30:29.415418+0000","last_undegraded":"2023-09-27T23:30:29.415418+0000","last_fullsized":"2023-09-27T23:30:29.415418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.438034+0000","last_change":"2023-09-27T23:30:29.430893+0000","last_active":"2023-09-27T23:30:29.438034+0000","last_peered":"2023-09-27T23:30:29.438034+0000","last_clean":"2023-09-27T23:30:29.438034+0000","last_became_active":"2023-09-27T23:30:29.430567+0000","last_became_peered":"2023-09-27T23:30:29.430567+0000","last_unstale":"2023-09-27T23:30:29.438034+0000","last_undegraded":"2023-09-27T23:30:29.438034+0000","last_fullsized":"2023-09-27T23:30:29.438034+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:48.502103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.438421+0000","last_change":"2023-09-27T23:30:29.026800+0000","last_active":"2023-09-27T23:30:29.438421+0000","last_peered":"2023-09-27T23:30:29.438421+0000","last_clean":"2023-09-27T23:30:29.438421+0000","last_became_active":"2023-09-27T23:30:29.026537+0000","last_became_peered":"2023-09-27T23:30:29.026537+0000","last_unstale":"2023-09-27T23:30:29.438421+0000","last_undegraded":"2023-09-27T23:30:29.438421+0000","last_fullsized":"2023-09-27T23:30:29.438421+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:14:57.904897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.414122+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:30:29.414122+0000","last_peered":"2023-09-27T23:30:29.414122+0000","last_clean":"2023-09-27T23:30:29.414122+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:30:29.414122+0000","last_undegraded":"2023-09-27T23:30:29.414122+0000","last_fullsized":"2023-09-27T23:30:29.414122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.448386+0000","last_change":"2023-09-27T23:30:29.446610+0000","last_active":"2023-09-27T23:30:29.448386+0000","last_peered":"2023-09-27T23:30:29.448386+0000","last_clean":"2023-09-27T23:30:29.448386+0000","last_became_active":"2023-09-27T23:30:29.446317+0000","last_became_peered":"2023-09-27T23:30:29.446317+0000","last_unstale":"2023-09-27T23:30:29.448386+0000","last_undegraded":"2023-09-27T23:30:29.448386+0000","last_fullsized":"2023-09-27T23:30:29.448386+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:55:23.042248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.445158+0000","last_change":"2023-09-27T23:30:29.046235+0000","last_active":"2023-09-27T23:30:29.445158+0000","last_peered":"2023-09-27T23:30:29.445158+0000","last_clean":"2023-09-27T23:30:29.445158+0000","last_became_active":"2023-09-27T23:30:29.045955+0000","last_became_peered":"2023-09-27T23:30:29.045955+0000","last_unstale":"2023-09-27T23:30:29.445158+0000","last_undegraded":"2023-09-27T23:30:29.445158+0000","last_fullsized":"2023-09-27T23:30:29.445158+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:39:54.902280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":100,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.423547+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:30:29.423547+0000","last_peered":"2023-09-27T23:30:29.423547+0000","last_clean":"2023-09-27T23:30:29.423547+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:30:29.423547+0000","last_undegraded":"2023-09-27T23:30:29.423547+0000","last_fullsized":"2023-09-27T23:30:29.423547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.442835+0000","last_change":"2023-09-27T23:30:29.437179+0000","last_active":"2023-09-27T23:30:29.442835+0000","last_peered":"2023-09-27T23:30:29.442835+0000","last_clean":"2023-09-27T23:30:29.442835+0000","last_became_active":"2023-09-27T23:30:29.436762+0000","last_became_peered":"2023-09-27T23:30:29.436762+0000","last_unstale":"2023-09-27T23:30:29.442835+0000","last_undegraded":"2023-09-27T23:30:29.442835+0000","last_fullsized":"2023-09-27T23:30:29.442835+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:41:06.581712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.412328+0000","last_change":"2023-09-27T23:30:29.044096+0000","last_active":"2023-09-27T23:30:29.412328+0000","last_peered":"2023-09-27T23:30:29.412328+0000","last_clean":"2023-09-27T23:30:29.412328+0000","last_became_active":"2023-09-27T23:30:29.043775+0000","last_became_peered":"2023-09-27T23:30:29.043775+0000","last_unstale":"2023-09-27T23:30:29.412328+0000","last_undegraded":"2023-09-27T23:30:29.412328+0000","last_fullsized":"2023-09-27T23:30:29.412328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:23.858912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.402923+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:30:29.402923+0000","last_peered":"2023-09-27T23:30:29.402923+0000","last_clean":"2023-09-27T23:30:29.402923+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:30:29.402923+0000","last_undegraded":"2023-09-27T23:30:29.402923+0000","last_fullsized":"2023-09-27T23:30:29.402923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.429889+0000","last_change":"2023-09-27T23:30:29.421109+0000","last_active":"2023-09-27T23:30:29.429889+0000","last_peered":"2023-09-27T23:30:29.429889+0000","last_clean":"2023-09-27T23:30:29.429889+0000","last_became_active":"2023-09-27T23:30:29.420831+0000","last_became_peered":"2023-09-27T23:30:29.420831+0000","last_unstale":"2023-09-27T23:30:29.429889+0000","last_undegraded":"2023-09-27T23:30:29.429889+0000","last_fullsized":"2023-09-27T23:30:29.429889+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:55:05.017898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.419676+0000","last_change":"2023-09-27T23:30:29.034195+0000","last_active":"2023-09-27T23:30:29.419676+0000","last_peered":"2023-09-27T23:30:29.419676+0000","last_clean":"2023-09-27T23:30:29.419676+0000","last_became_active":"2023-09-27T23:30:29.033891+0000","last_became_peered":"2023-09-27T23:30:29.033891+0000","last_unstale":"2023-09-27T23:30:29.419676+0000","last_undegraded":"2023-09-27T23:30:29.419676+0000","last_fullsized":"2023-09-27T23:30:29.419676+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:37:16.458361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.419180+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:30:29.419180+0000","last_peered":"2023-09-27T23:30:29.419180+0000","last_clean":"2023-09-27T23:30:29.419180+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:30:29.419180+0000","last_undegraded":"2023-09-27T23:30:29.419180+0000","last_fullsized":"2023-09-27T23:30:29.419180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.430232+0000","last_change":"2023-09-27T23:30:29.421562+0000","last_active":"2023-09-27T23:30:29.430232+0000","last_peered":"2023-09-27T23:30:29.430232+0000","last_clean":"2023-09-27T23:30:29.430232+0000","last_became_active":"2023-09-27T23:30:29.421234+0000","last_became_peered":"2023-09-27T23:30:29.421234+0000","last_unstale":"2023-09-27T23:30:29.430232+0000","last_undegraded":"2023-09-27T23:30:29.430232+0000","last_fullsized":"2023-09-27T23:30:29.430232+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:17:49.930694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.423482+0000","last_change":"2023-09-27T23:30:29.035612+0000","last_active":"2023-09-27T23:30:29.423482+0000","last_peered":"2023-09-27T23:30:29.423482+0000","last_clean":"2023-09-27T23:30:29.423482+0000","last_became_active":"2023-09-27T23:30:29.035021+0000","last_became_peered":"2023-09-27T23:30:29.035021+0000","last_unstale":"2023-09-27T23:30:29.423482+0000","last_undegraded":"2023-09-27T23:30:29.423482+0000","last_fullsized":"2023-09-27T23:30:29.423482+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:42.253439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":98,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.401712+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:30:29.401712+0000","last_peered":"2023-09-27T23:30:29.401712+0000","last_clean":"2023-09-27T23:30:29.401712+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:30:29.401712+0000","last_undegraded":"2023-09-27T23:30:29.401712+0000","last_fullsized":"2023-09-27T23:30:29.401712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.426595+0000","last_change":"2023-09-27T23:30:29.417174+0000","last_active":"2023-09-27T23:30:29.426595+0000","last_peered":"2023-09-27T23:30:29.426595+0000","last_clean":"2023-09-27T23:30:29.426595+0000","last_became_active":"2023-09-27T23:30:29.416889+0000","last_became_peered":"2023-09-27T23:30:29.416889+0000","last_unstale":"2023-09-27T23:30:29.426595+0000","last_undegraded":"2023-09-27T23:30:29.426595+0000","last_fullsized":"2023-09-27T23:30:29.426595+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:29.002998+0000","last_clean_scrub_stamp":"2023-09-27T23:30:29.002998+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:24:10.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.438432+0000","last_change":"2023-09-27T23:30:29.038140+0000","last_active":"2023-09-27T23:30:29.438432+0000","last_peered":"2023-09-27T23:30:29.438432+0000","last_clean":"2023-09-27T23:30:29.438432+0000","last_became_active":"2023-09-27T23:30:29.037827+0000","last_became_peered":"2023-09-27T23:30:29.037827+0000","last_unstale":"2023-09-27T23:30:29.438432+0000","last_undegraded":"2023-09-27T23:30:29.438432+0000","last_fullsized":"2023-09-27T23:30:29.438432+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:30:27.990481+0000","last_clean_scrub_stamp":"2023-09-27T23:30:27.990481+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:55:22.029731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-09-27T23:30:29.416356+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:30:29.416356+0000","last_peered":"2023-09-27T23:30:29.416356+0000","last_clean":"2023-09-27T23:30:29.416356+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:30:29.416356+0000","last_undegraded":"2023-09-27T23:30:29.416356+0000","last_fullsized":"2023-09-27T23:30:29.416356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509644,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26676,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":24704,"kb_avail":94345164,"statfs":{"total":96636764160,"available":96609447936,"internally_reserved":0,"allocated":2011136,"data_stored":1318711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25296896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.877,"15min":0.91100000000000003},"min":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.7949999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.712,"15min":1.712},"last":1.2270000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.89900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.179},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.337,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.56599999999999995,"5min":0.307,"15min":0.307},"max":{"1min":1.3129999999999999,"5min":2.681,"15min":2.681},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.53500000000000003},"max":{"1min":1.3520000000000001,"5min":1.718,"15min":1.718},"last":1.105}]},{"osd":3,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1000000000000001,"5min":1.29,"15min":1.421},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.052},"min":{"1min":0.58499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.95299999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.60499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3240000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.997,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.69599999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2450000000000001,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":1.012}]},{"osd":5,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0469999999999999,"15min":1.0289999999999999},"min":{"1min":0.79600000000000004,"5min":0.57499999999999996,"15min":0.48999999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.099},"min":{"1min":0.90300000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7190000000000001,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.056}]},{"osd":6,"last update":"Wed Sep 27 23:30:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.169,"15min":1.131},"min":{"1min":0.97299999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.141},{"interface":"front","average":{"1min":1.159,"5min":1.145,"15min":1.1279999999999999},"min":{"1min":0.97799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4139999999999999,"5min":2.548,"15min":2.548},"last":1.087}]}]},{"osd":6,"up_from":15,"seq":64424509643,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22256,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":94349584,"statfs":{"total":96636764160,"available":96613974016,"internally_reserved":0,"allocated":1417216,"data_stored":728855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.55500000000000005,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1279999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.022,"15min":1.0209999999999999},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2230000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.01}]},{"osd":1,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.88200000000000001},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.248,"5min":1.2889999999999999,"15min":1.323},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88900000000000001,"15min":0.89800000000000002},"min":{"1min":0.60799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.84599999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85999999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.222,"15min":1.353},"last":0.752},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.68799999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":1.157}]},{"osd":3,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94299999999999995,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.734},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.256,"5min":1.278,"15min":1.4970000000000001},"last":0.96499999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.61299999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.544},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.004,"5min":1.0369999999999999,"15min":1.0309999999999999},"min":{"1min":0.65100000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5980000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.0840000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.97699999999999998,"15min":0.97499999999999998},"min":{"1min":0.68700000000000006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.129},{"interface":"front","average":{"1min":1.163,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.85199999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3740000000000001,"5min":1.4259999999999999,"15min":1.49},"last":1.3169999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.143},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6719999999999999,"5min":1.6879999999999999,"15min":1.7390000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.147},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.6619999999999999},"last":1.218}]}]},{"osd":5,"up_from":15,"seq":64424509643,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36396,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94335444,"statfs":{"total":96636764160,"available":96599494656,"internally_reserved":0,"allocated":1413120,"data_stored":728301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1519999999999999,"5min":1.1890000000000001,"15min":1.2909999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.27,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.63700000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.2969999999999999,"15min":1.526},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.877,"15min":0.89000000000000001},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2390000000000001,"5min":1.242,"15min":1.331},"last":0.79700000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.52900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.97999999999999998,"15min":0.95099999999999996},"min":{"1min":0.63700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.276,"5min":1.369,"15min":1.5740000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2290000000000001,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":0.81699999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.88700000000000001},"min":{"1min":0.45200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.222,"5min":1.341,"15min":1.341},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.70599999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.022}]},{"osd":6,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0089999999999999,"15min":1.0069999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3799999999999999,"5min":1.429,"15min":1.5580000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3520000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.0680000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.123,"15min":1.123},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.6679999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2050000000000001,"15min":1.1890000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.633},"last":1.0469999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22524,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":21120,"kb_avail":94349316,"statfs":{"total":96636764160,"available":96613699584,"internally_reserved":0,"allocated":1429504,"data_stored":729891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21626880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2230000000000001,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.624,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.1639999999999999,"5min":1.45,"15min":1.45},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.52900000000000003},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.3919999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.329},"last":1.032}]},{"osd":2,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.91000000000000003},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.2430000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.331,"5min":1.3660000000000001,"15min":1.407},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.92500000000000004},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.167,"5min":1.256,"15min":1.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.018},"min":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":1.1439999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.90600000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.57599999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.78400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0489999999999999,"15min":1.0329999999999999},"min":{"1min":0.77300000000000002,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.149},"min":{"1min":0.97599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.514,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.236}]},{"osd":7,"last update":"Wed Sep 27 23:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.117,"15min":1.091},"min":{"1min":0.86899999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4770000000000001,"5min":1.627,"15min":1.627},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1759999999999999,"15min":1.155},"min":{"1min":0.97199999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.2609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22968,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":21568,"kb_avail":94348872,"statfs":{"total":96636764160,"available":96613244928,"internally_reserved":0,"allocated":1425408,"data_stored":729789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22085632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":1.1930000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2230000000000001,"5min":1.393,"15min":1.393},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0680000000000001,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82299999999999995,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3939999999999999},"last":0.62}]},{"osd":2,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":1.323,"15min":1.3440000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.446,"15min":1.5720000000000001},"last":0.85199999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.96899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.5349999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.0669999999999999,"15min":1.0720000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.89100000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.70699999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.4810000000000001,"15min":1.571},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.127,"15min":1.153},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.714,"15min":1.764},"last":0.751}]},{"osd":6,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1000000000000001,"15min":1.113},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2829999999999999,"5min":1.5329999999999999,"15min":1.587},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.2130000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.05}]},{"osd":7,"last update":"Wed Sep 27 23:30:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2130000000000001,"15min":1.232},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4119999999999999,"5min":1.655,"15min":1.655},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.177,"15min":1.196},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.556,"5min":1.629,"15min":1.671},"last":1.079}]}]},{"osd":2,"up_from":15,"seq":64424509644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21548,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":20160,"kb_avail":94350292,"statfs":{"total":96636764160,"available":96614699008,"internally_reserved":0,"allocated":1413120,"data_stored":728265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20643840},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97899999999999998,"5min":1.093,"15min":1.133},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.107,"5min":1.903,"15min":1.903},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.499,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.105,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.83199999999999996},"min":{"1min":0.439,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.032,"5min":1.391,"15min":1.391},"last":0.86099999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.78900000000000003,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.444,"15min":1.444},"last":1.1160000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0369999999999999,"15min":1.036},"min":{"1min":0.751,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.4239999999999999,"5min":1.4319999999999999,"15min":1.585},"last":1.242},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.038,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5620000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.2649999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.032,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4339999999999999,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.079},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1339999999999999,"15min":1.135},"min":{"1min":0.78700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.397,"5min":1.5169999999999999,"15min":1.645},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1619999999999999,"15min":1.163},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.45,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.514},{"interface":"front","average":{"1min":1.107,"5min":1.1679999999999999,"15min":1.159},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4359999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.212}]},{"osd":7,"last update":"Wed Sep 27 23:30:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.69699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.663,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.417},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2170000000000001},"min":{"1min":0.79700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.502,"5min":2.073,"15min":2.073},"last":1.296}]}]},{"osd":1,"up_from":14,"seq":60129542350,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":35520,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1409024,"data_stored":728231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36372480},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.47399999999999998,"15min":0.44700000000000001},"max":{"1min":1.123,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95799999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.56699999999999995,"5min":0.48199999999999998,"15min":0.45800000000000002},"max":{"1min":0.81999999999999995,"5min":1.2490000000000001,"15min":1.321},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1599999999999999,"5min":1.696,"15min":1.696},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.97599999999999998,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.431,"5min":1.464,"15min":1.464},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1180000000000001,"15min":1.127},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4410000000000001,"5min":1.5369999999999999,"15min":1.774},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.425,"5min":1.546,"15min":1.623},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1020000000000001,"15min":1.0980000000000001},"min":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88900000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.22,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.23,"5min":1.1890000000000001,"15min":1.1899999999999999},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.93400000000000005}]},{"osd":7,"last update":"Wed Sep 27 23:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.216,"15min":1.23},"min":{"1min":0.88300000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5609999999999999,"5min":1.877,"15min":1.877},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2030000000000001,"15min":1.2170000000000001},"min":{"1min":0.95299999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.55,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509645,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":2007040,"data_stored":1319153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":1.3,"15min":1.3},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.45600000000000002},"max":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.373},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99099999999999999,"5min":1.228,"15min":1.232},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.83999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.117,"5min":1.216,"15min":1.2769999999999999},"last":1.05},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"last":0.64300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.47},"last":1.2430000000000001},{"interface":"front","average":{"1min":0.995,"5min":1.022,"15min":1.0389999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.357,"5min":1.5589999999999999,"15min":1.587},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.081,"15min":1.091},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.272,"5min":1.5009999999999999,"15min":1.78},"last":1.028},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.113,"15min":1.137},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":4.6779999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.135}]},{"osd":6,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.147,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.4370000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.202,"15min":1.2150000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.419,"5min":1.73,"15min":1.8300000000000001},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:30:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2170000000000001,"15min":1.238},"min":{"1min":0.78500000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.482,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1919999999999999,"15min":1.222},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.5660000000000001,"5min":1.798,"15min":1.798},"last":1.3089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:30:37.215 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:30:37.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:30:37.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.371+0000 7f148f11c700 1 Processor -- start 2023-09-27T23:30:37.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.371+0000 7f148f11c700 1 -- start start 2023-09-27T23:30:37.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14880caa10 0x7f1488159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880cc040 0x7f14880ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14880cbbb0 con 0x7f14880ce2d0 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148806dc90 con 0x7f14880caa10 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14880cafa0 con 0x7f14880cc040 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14880caa10 0x7f1488159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14880caa10 0x7f1488159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34590/0 (socket says 172.21.15.64:34590) 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 -- 172.21.15.64:0/3286399186 learned_addr learned my addr 172.21.15.64:0/3286399186 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:37.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 -- 172.21.15.64:0/3286399186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880cc040 msgr2=0x7f14880ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148e91b700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880cc040 0x7f14880ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880cc040 0x7f14880ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 -- 172.21.15.64:0/3286399186 >> v1:172.21.15.64:6789/0 conn(0x7f14880ce2d0 legacy=0x7f1488158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 -- 172.21.15.64:0/3286399186 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880cb280 con 0x7f14880caa10 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148e91b700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880cc040 0x7f14880ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d919700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488159010 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f14840099b0 tx=0x7f1484009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5519f1bce03578e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d118700 1 -- 172.21.15.64:0/3286399186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400b860 con 0x7f14880caa10 2023-09-27T23:30:37.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d118700 1 -- 172.21.15.64:0/3286399186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148400b9c0 con 0x7f14880caa10 2023-09-27T23:30:37.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148d118700 1 -- 172.21.15.64:0/3286399186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400bd60 con 0x7f14880caa10 2023-09-27T23:30:37.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- 172.21.15.64:0/3286399186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 msgr2=0x7f1488159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488159010 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f14840099b0 tx=0x7f1484009250 comp rx=0 tx=0).stop 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- 172.21.15.64:0/3286399186 shutdown_connections 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880cc040 0x7f14880ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 --2- 172.21.15.64:0/3286399186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488159010 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- 172.21.15.64:0/3286399186 >> 172.21.15.64:0/3286399186 conn(0x7f1488067bb0 msgr2=0x7f14880cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- 172.21.15.64:0/3286399186 shutdown_connections 2023-09-27T23:30:37.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.375+0000 7f148f11c700 1 -- 172.21.15.64:0/3286399186 wait complete. 2023-09-27T23:30:37.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 Processor -- start 2023-09-27T23:30:37.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 -- start start 2023-09-27T23:30:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14880cc040 0x7f148815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880ce2d0 0x7f148815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14881586d0 con 0x7f14880caa10 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1488158c90 con 0x7f14880cc040 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14880c7b00 con 0x7f14880ce2d0 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34592/0 (socket says 172.21.15.64:34592) 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 -- 172.21.15.64:0/3290393699 learned_addr learned my addr 172.21.15.64:0/3290393699 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e91b700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880ce2d0 0x7f148815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148d919700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14880cc040 0x7f148815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 -- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880ce2d0 msgr2=0x7f148815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880ce2d0 0x7f148815ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 -- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14880cc040 msgr2=0x7f148815c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14880cc040 0x7f148815c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 -- 172.21.15.64:0/3290393699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840094f0 con 0x7f14880caa10 2023-09-27T23:30:37.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148e11a700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488146740 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f147800c510 tx=0x7f147800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478011590 con 0x7f14880caa10 2023-09-27T23:30:37.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478011710 con 0x7f14880caa10 2023-09-27T23:30:37.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.379+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14880c7de0 con 0x7f14880caa10 2023-09-27T23:30:37.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478011ad0 con 0x7f14880caa10 2023-09-27T23:30:37.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1488146c80 con 0x7f14880caa10 2023-09-27T23:30:37.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148804be40 con 0x7f14880caa10 2023-09-27T23:30:37.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f147801b3b0 con 0x7f14880caa10 2023-09-27T23:30:37.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f147e7fc700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f146c060a70 0x7f146c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f147807caf0 con 0x7f14880caa10 2023-09-27T23:30:37.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.383+0000 7f148d919700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f146c060a70 0x7f146c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.387+0000 7f148d919700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f146c060a70 0x7f146c062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14840099b0 tx=0x7f1484017000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.395+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f147804cb00 con 0x7f14880caa10 2023-09-27T23:30:37.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.619+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f14880cb560 con 0x7f14880caa10 2023-09-27T23:30:37.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.631+0000 7f147e7fc700 1 -- 172.21.15.64:0/3290393699 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1478017070 con 0x7f14880caa10 2023-09-27T23:30:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f146c060a70 msgr2=0x7f146c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f146c060a70 0x7f146c062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14840099b0 tx=0x7f1484017000 comp rx=0 tx=0).stop 2023-09-27T23:30:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 msgr2=0x7f1488146740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488146740 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f147800c510 tx=0x7f147800c8d0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 shutdown_connections 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f146c060a70 0x7f146c062f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14880ce2d0 0x7f148815ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14880cc040 0x7f148815c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 --2- 172.21.15.64:0/3290393699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14880caa10 0x7f1488146740 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 >> 172.21.15.64:0/3290393699 conn(0x7f1488067bb0 msgr2=0x7f1488155680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 shutdown_connections 2023-09-27T23:30:37.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.635+0000 7f148f11c700 1 -- 172.21.15.64:0/3290393699 wait complete. 2023-09-27T23:30:37.639 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:30:37.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:30:37.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 Processor -- start 2023-09-27T23:30:37.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- start start 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f080cb920 0x7f2f0815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2f0815cee0 0x7f2f0806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f08068dc0 con 0x7f2f08158d40 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080cc910 con 0x7f2f0815cee0 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080682a0 con 0x7f2f080cb920 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2f0815cee0 0x7f2f0806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2f0815cee0 0x7f2f0806c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34604/0 (socket says 172.21.15.64:34604) 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 -- 172.21.15.64:0/1575853174 learned_addr learned my addr 172.21.15.64:0/1575853174 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:37.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d1bd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2f08158d40 0x7f2f080cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52038/0 (socket says 172.21.15.64:52038) 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0c9bc700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f080cb920 0x7f2f0815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 -- 172.21.15.64:0/1575853174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f080cb920 msgr2=0x7f2f0815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f080cb920 0x7f2f0815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 -- 172.21.15.64:0/1575853174 >> v1:172.21.15.64:6789/0 conn(0x7f2f08158d40 legacy=0x7f2f080cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 -- 172.21.15.64:0/1575853174 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08068490 con 0x7f2f0815cee0 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0c9bc700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f080cb920 0x7f2f0815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0d9be700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f0815cee0 0x7f2f0806c7c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2f040099b0 tx=0x7f2f04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9358a7e4f5c43c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2efffff700 1 -- 172.21.15.64:0/1575853174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400b860 con 0x7f2f0815cee0 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2efffff700 1 -- 172.21.15.64:0/1575853174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0400b9c0 con 0x7f2f0815cee0 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2efffff700 1 -- 172.21.15.64:0/1575853174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400bd60 con 0x7f2f0815cee0 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1575853174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f0815cee0 msgr2=0x7f2f0806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f0815cee0 0x7f2f0806c7c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2f040099b0 tx=0x7f2f04009250 comp rx=0 tx=0).stop 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1575853174 shutdown_connections 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f0815cee0 0x7f2f0806c7c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1575853174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f080cb920 0x7f2f0815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1575853174 >> 172.21.15.64:0/1575853174 conn(0x7f2f080cdbc0 msgr2=0x7f2f080cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1575853174 shutdown_connections 2023-09-27T23:30:37.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.759+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1575853174 wait complete. 2023-09-27T23:30:37.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 Processor -- start 2023-09-27T23:30:37.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 -- start start 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 0x7f2f080af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f08158d40 0x7f2f080c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f0815cee0 0x7f2f080c3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c7fa0 con 0x7f2f080cb920 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c7420 con 0x7f2f08158d40 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c7700 con 0x7f2f0815cee0 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 0x7f2f080af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 0x7f2f080af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34606/0 (socket says 172.21.15.64:34606) 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 -- 172.21.15.64:0/1490055689 learned_addr learned my addr 172.21.15.64:0/1490055689 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:37.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0c9bc700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f08158d40 0x7f2f080c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d9be700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f0815cee0 0x7f2f080c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 -- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f0815cee0 msgr2=0x7f2f080c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f0815cee0 0x7f2f080c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 -- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f08158d40 msgr2=0x7f2f080c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f08158d40 0x7f2f080c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 -- 172.21.15.64:0/1490055689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040094f0 con 0x7f2f080cb920 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d9be700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f0815cee0 0x7f2f080c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0c9bc700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f08158d40 0x7f2f080c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0d1bd700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 0x7f2f080af4e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cdb0 tx=0x7f2f00001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00005df0 con 0x7f2f080cb920 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000a960 con 0x7f2f080cb920 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f080c79e0 con 0x7f2f080cb920 2023-09-27T23:30:37.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000ad20 con 0x7f2f080cb920 2023-09-27T23:30:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.763+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f080afa20 con 0x7f2f080cb920 2023-09-27T23:30:37.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.767+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f080afca0 con 0x7f2f080cb920 2023-09-27T23:30:37.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.767+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2f00007300 con 0x7f2f080cb920 2023-09-27T23:30:37.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.767+0000 7f2efd7fa700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2eec060a70 0x7f2eec062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:37.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.767+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2f0007dce0 con 0x7f2f080cb920 2023-09-27T23:30:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.767+0000 7f2f0c9bc700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2eec060a70 0x7f2eec062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:37.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.771+0000 7f2f0c9bc700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2eec060a70 0x7f2eec062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f080b1b40 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:37.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.775+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2f0004dcf0 con 0x7f2f080cb920 2023-09-27T23:30:37.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.975+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f080afca0 con 0x7f2f080cb920 2023-09-27T23:30:37.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.975+0000 7f2efd7fa700 1 -- 172.21.15.64:0/1490055689 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f2f080afca0 con 0x7f2f080cb920 2023-09-27T23:30:37.978 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:37.979 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:30:29.391491+0000","modified":"2023-09-27T23:30:31.410091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7605":{"gid":7605,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1829030850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1829030850},{"type":"v1","addr":"172.21.15.64:6835","nonce":1829030850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2eec060a70 msgr2=0x7f2eec062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2eec060a70 0x7f2eec062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f080b1b40 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 msgr2=0x7f2f080af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 0x7f2f080af4e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000cdb0 tx=0x7f2f00001b50 comp rx=0 tx=0).stop 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 shutdown_connections 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2eec060a70 0x7f2eec062f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f0815cee0 0x7f2f080c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f08158d40 0x7f2f080c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 --2- 172.21.15.64:0/1490055689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f080cb920 0x7f2f080af4e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:37.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 >> 172.21.15.64:0/1490055689 conn(0x7f2f080cdbc0 msgr2=0x7f2f080c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:37.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 shutdown_connections 2023-09-27T23:30:37.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:37.979+0000 7f2f0e1bf700 1 -- 172.21.15.64:0/1490055689 wait complete. 2023-09-27T23:30:37.982 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2023-09-27T23:30:38.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:30:29.391491+0000', 'modified': '2023-09-27T23:30:31.410091+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7605}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7605': {'gid': 7605, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/1829030850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 1829030850}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 1829030850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:30:38.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:30:38.396 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-27T23:30:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.515+0000 7f54b2e61700 1 Processor -- start 2023-09-27T23:30:38.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.515+0000 7f54b2e61700 1 -- start start 2023-09-27T23:30:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f54ac158910 0x7f54ac0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0accb0 con 0x7f54ac159530 2023-09-27T23:30:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0acf90 con 0x7f54ac159160 2023-09-27T23:30:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac158d70 con 0x7f54ac158910 2023-09-27T23:30:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f54ac159530 0x7f54ac158200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52040/0 (socket says 172.21.15.64:52040) 2023-09-27T23:30:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 -- 172.21.15.64:0/3092865237 learned_addr learned my addr 172.21.15.64:0/3092865237 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2660700 1 --2- 172.21.15.64:0/3092865237 >> v2:172.21.15.123:3300/0 conn(0x7f54ac158910 0x7f54ac0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336558720 0 0) 0x7f54ac0accb0 con 0x7f54ac159530 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5498003570 con 0x7f54ac159530 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572451303 0 0) 0x7f5498003570 con 0x7f54ac159530 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 >> v2:172.21.15.123:3300/0 conn(0x7f54ac158910 msgr2=0x7f54ac0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 --2- 172.21.15.64:0/3092865237 >> v2:172.21.15.123:3300/0 conn(0x7f54ac158910 0x7f54ac0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 >> v1:172.21.15.123:6789/0 conn(0x7f54ac159160 legacy=0x7f54ac0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0ad270 con 0x7f54ac159530 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f54a0001470 con 0x7f54ac159530 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 >> v1:172.21.15.64:6789/0 conn(0x7f54ac159530 legacy=0x7f54ac158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5498003a60 0x7f5498003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f5498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54980051c0 0x7f549800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549800c0a0 con 0x7f5498004810 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549800c210 con 0x7f5498003a60 2023-09-27T23:30:38.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549800c3d0 con 0x7f54980051c0 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f5498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5498003a60 0x7f5498003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2660700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54980051c0 0x7f549800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 -- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54980051c0 msgr2=0x7f549800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54980051c0 0x7f549800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 -- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5498003a60 msgr2=0x7f5498003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5498003a60 0x7f5498003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 -- 172.21.15.64:0/3092865237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498003570 con 0x7f5498004810 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2660700 1 -- 172.21.15.64:0/3092865237 reap_dead start 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b165e700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f5498004c80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f54a000c3c0 tx=0x7f54a0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967a3ac40b69ae8d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a000cc10 con 0x7f5498004810 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a000cd70 con 0x7f5498004810 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b0e5d700 1 -- 172.21.15.64:0/3092865237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0005b50 con 0x7f5498004810 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- 172.21.15.64:0/3092865237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 msgr2=0x7f5498004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f5498004c80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f54a000c3c0 tx=0x7f54a0001e50 comp rx=0 tx=0).stop 2023-09-27T23:30:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- 172.21.15.64:0/3092865237 shutdown_connections 2023-09-27T23:30:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 --2- 172.21.15.64:0/3092865237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f5498004c80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- 172.21.15.64:0/3092865237 >> 172.21.15.64:0/3092865237 conn(0x7f54ac067bb0 msgr2=0x7f54ac0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- 172.21.15.64:0/3092865237 shutdown_connections 2023-09-27T23:30:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.519+0000 7f54b2e61700 1 -- 172.21.15.64:0/3092865237 wait complete. 2023-09-27T23:30:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 Processor -- start 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 -- start start 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f54ac0a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac14a6b0 0x7f54ac0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac0c9dc0 0x7f54ac148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac14baf0 con 0x7f5498004810 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac14aee0 con 0x7f54ac0c9dc0 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2e61700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac14b1c0 con 0x7f54ac14a6b0 2023-09-27T23:30:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f54ac0a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f54ac0a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34622/0 (socket says 172.21.15.64:34622) 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 -- 172.21.15.64:0/4164163110 learned_addr learned my addr 172.21.15.64:0/4164163110 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b165e700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac14a6b0 0x7f54ac0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b2660700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac0c9dc0 0x7f54ac148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 -- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac14a6b0 msgr2=0x7f54ac0c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac14a6b0 0x7f54ac0c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 -- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac0c9dc0 msgr2=0x7f54ac148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac0c9dc0 0x7f54ac148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.523+0000 7f54b1e5f700 1 -- 172.21.15.64:0/4164163110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac14b4a0 con 0x7f5498004810 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f54ac0a59f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f54a8002950 tx=0x7f54a8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f549dffb700 1 -- 172.21.15.64:0/4164163110 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8006a30 con 0x7f5498004810 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f549dffb700 1 -- 172.21.15.64:0/4164163110 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8006b90 con 0x7f5498004810 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac14b780 con 0x7f5498004810 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f549dffb700 1 -- 172.21.15.64:0/4164163110 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8007030 con 0x7f5498004810 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac149070 con 0x7f5498004810 2023-09-27T23:30:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.527+0000 7f549dffb700 1 -- 172.21.15.64:0/4164163110 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f54a8007420 con 0x7f5498004810 2023-09-27T23:30:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f549dffb700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5490060a70 0x7f5490062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f549dffb700 1 -- 172.21.15.64:0/4164163110 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f54a807d250 con 0x7f5498004810 2023-09-27T23:30:38.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f54b165e700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5490060a70 0x7f5490062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f54b165e700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5490060a70 0x7f5490062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54a0007cd0 tx=0x7f54a0015000 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f54b636c740 1 Processor -- start 2023-09-27T23:30:38.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f54b636c740 1 -- start start 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.531+0000 7f54b636c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 0x200ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8f710 0x200d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x200d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f3af30 con 0x1d8fae0 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f36ed0 con 0x1dda790 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1efd0b0 con 0x1d8f710 2023-09-27T23:30:38.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 0x200ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 0x200ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34624/0 (socket says 172.21.15.64:34624) 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 -- 172.21.15.64:0/426610557 learned_addr learned my addr 172.21.15.64:0/426610557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b165e700 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8f710 0x200d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b2660700 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x200d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 -- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8f710 msgr2=0x200d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8f710 0x200d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 -- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 msgr2=0x200d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x200d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 -- 172.21.15.64:0/426610557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ddb9e0 con 0x1d8fae0 2023-09-27T23:30:38.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 0x200ce50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f54a807b420 tx=0x7f54a8053040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56b113a5ac703c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f5496ffd700 1 -- 172.21.15.64:0/426610557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80541b0 con 0x1d8fae0 2023-09-27T23:30:38.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f5496ffd700 1 -- 172.21.15.64:0/426610557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8057f00 con 0x1d8fae0 2023-09-27T23:30:38.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f5496ffd700 1 -- 172.21.15.64:0/426610557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80076d0 con 0x1d8fae0 2023-09-27T23:30:38.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- 172.21.15.64:0/426610557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 msgr2=0x200ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 0x200ce50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f54a807b420 tx=0x7f54a8053040 comp rx=0 tx=0).stop 2023-09-27T23:30:38.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- 172.21.15.64:0/426610557 shutdown_connections 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x200d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8f710 0x200d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 --2- 172.21.15.64:0/426610557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8fae0 0x200ce50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- 172.21.15.64:0/426610557 >> 172.21.15.64:0/426610557 conn(0x1d8feb0 msgr2=0x1fc9880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- 172.21.15.64:0/426610557 shutdown_connections 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- 172.21.15.64:0/426610557 wait complete. 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 -- start start 2023-09-27T23:30:38.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.535+0000 7f54b636c740 1 Processor -- start 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8f710 0x1fc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x1ff2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 0x1ff3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1efcd50 con 0x1dda790 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ed5970 con 0x1d8f710 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ef0430 con 0x1d8fae0 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 10 client.0 _ll_register_callbacks cb 0x1f80b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:30:38.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 0x1ff3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8f710 0x1fc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 0x1ff3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34626/0 (socket says 172.21.15.64:34626) 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 -- 172.21.15.64:0/3999853542 learned_addr learned my addr 172.21.15.64:0/3999853542 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b165e700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x1ff2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 -- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 msgr2=0x1ff2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x1ff2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 -- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8f710 msgr2=0x1fc9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8f710 0x1fc9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 -- 172.21.15.64:0/3999853542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac149070 con 0x1dda790 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8f710 0x1fc9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b2660700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 0x1ff3140 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0c83b0 tx=0x7f54a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dfaa167eb2fec4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f548bfff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f548bfff700 1 -- 172.21.15.64:0/3999853542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a400e030 con 0x1dda790 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f548bfff700 1 -- 172.21.15.64:0/3999853542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f54a4007cf0 con 0x1dda790 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ed5cc0 con 0x1dda790 2023-09-27T23:30:38.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f548bfff700 1 -- 172.21.15.64:0/3999853542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a4007eb0 con 0x1dda790 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 10 client.7770 fetch_fsmap learned FSMap version 39 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x1f05370 con 0x1dda790 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f548bfff700 1 -- 172.21.15.64:0/3999853542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ed5cc0 con 0x1dda790 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f548bfff700 1 -- 172.21.15.64:0/3999853542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f54a401e020 con 0x1dda790 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 10 client.7770 fetch_fsmap finished waiting for FSMap version 39 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 10 client.7770 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 20 client.7770 populate_metadata read hostname 'smithi064' 2023-09-27T23:30:38.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x20196b0 0x201ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 4 client.7770 mds_command: new command op to 7605 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:30:38.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 --> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ed93e0 con 0x20196b0 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.539+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x20196b0 0x201ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x20196b0 0x201ba90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4136840d3f561c3f server_cookie=f9bbc7f6479ec30e in_seq=0 out_seq=0 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f548bfff700 10 client.7770 ms_handle_connect on v2:172.21.15.64:6834/1829030850 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f548bfff700 1 -- 172.21.15.64:0/3999853542 <== mds.0 v2:172.21.15.64:6834/1829030850 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1ed93e0 con 0x20196b0 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f548bfff700 10 client.7770 handle_command_reply: tid=0 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 client.7770 shutdown 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 msgr2=0x1ff3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 0x1ff3140 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0c83b0 tx=0x7f54a400b040 comp rx=0 tx=0).stop 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 shutdown_connections 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x20196b0 0x201ba90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1dda790 0x1ff3140 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x1ff2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 --2- 172.21.15.64:0/3999853542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8f710 0x1fc9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f548bfff700 0 client.7770 ms_handle_reset on v2:172.21.15.64:6834/1829030850 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 >> 172.21.15.64:0/3999853542 conn(0x1d8feb0 msgr2=0x1fc1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 shutdown_connections 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 1 -- 172.21.15.64:0/3999853542 wait complete. 2023-09-27T23:30:38.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.543+0000 7f54b636c740 20 client.7770 trim_cache size 0 max 16384 2023-09-27T23:30:38.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 Processor -- start 2023-09-27T23:30:38.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 -- start start 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x1ff0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8fae0 0x1ff1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1dda790 0x1fc2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f3bef0 con 0x1d8f710 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa4bb0 con 0x1d8fae0 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f5e3b0 con 0x1dda790 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x1ff0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x1ff0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34630/0 (socket says 172.21.15.64:34630) 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 -- 172.21.15.64:0/2151295521 learned_addr learned my addr 172.21.15.64:0/2151295521 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b2660700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1dda790 0x1fc2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b165e700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8fae0 0x1ff1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 -- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1dda790 msgr2=0x1fc2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1dda790 0x1fc2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 -- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8fae0 msgr2=0x1ff1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8fae0 0x1ff1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 -- 172.21.15.64:0/2151295521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac1560e0 con 0x1d8f710 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b2660700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1dda790 0x1fc2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b165e700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8fae0 0x1ff1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x1ff0ac0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f54a8006fd0 tx=0x7f54a8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8706334f340e97e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.563+0000 7f54977fe700 1 -- 172.21.15.64:0/2151295521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80076d0 con 0x1d8f710 2023-09-27T23:30:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54977fe700 1 -- 172.21.15.64:0/2151295521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a804f060 con 0x1d8f710 2023-09-27T23:30:38.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54977fe700 1 -- 172.21.15.64:0/2151295521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8007190 con 0x1d8f710 2023-09-27T23:30:38.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- 172.21.15.64:0/2151295521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 msgr2=0x1ff0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x1ff0ac0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f54a8006fd0 tx=0x7f54a8014000 comp rx=0 tx=0).stop 2023-09-27T23:30:38.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- 172.21.15.64:0/2151295521 shutdown_connections 2023-09-27T23:30:38.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1dda790 0x1fc2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1d8fae0 0x1ff1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- 172.21.15.64:0/2151295521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x1ff0ac0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- 172.21.15.64:0/2151295521 >> 172.21.15.64:0/2151295521 conn(0x1d8feb0 msgr2=0x1ff2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- 172.21.15.64:0/2151295521 shutdown_connections 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- 172.21.15.64:0/2151295521 wait complete. 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- start start 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 Processor -- start 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x204b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x204ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x204bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa4bb0 con 0x1d8f710 2023-09-27T23:30:38.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f34890 con 0x1dda790 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f3bef0 con 0x1d8fae0 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b636c740 10 client.0 _ll_register_callbacks cb 0x1e530e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x204b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x204b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34632/0 (socket says 172.21.15.64:34632) 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 -- 172.21.15.64:0/1510673926 learned_addr learned my addr 172.21.15.64:0/1510673926 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b165e700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x204ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b2660700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x204bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 -- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 msgr2=0x204ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x204ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 -- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 msgr2=0x204bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.567+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x204bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b1e5f700 1 -- 172.21.15.64:0/1510673926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a8066040 con 0x1d8f710 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b1e5f700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x204b520 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f54a807b470 tx=0x7f54a8064070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0befc13723b0db server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 1 -- 172.21.15.64:0/1510673926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80076d0 con 0x1d8f710 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54a8057f00 con 0x1d8f710 2023-09-27T23:30:38.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 1 -- 172.21.15.64:0/1510673926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f54a8007190 con 0x1d8f710 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 1 -- 172.21.15.64:0/1510673926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a806b340 con 0x1d8f710 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 10 client.7776 fetch_fsmap learned FSMap version 39 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x1eebd80 con 0x1d8f710 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 1 -- 172.21.15.64:0/1510673926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8057f00 con 0x1d8f710 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 1 -- 172.21.15.64:0/1510673926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f54a8056070 con 0x1d8f710 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 10 client.7776 fetch_fsmap finished waiting for FSMap version 39 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 10 client.7776 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 20 client.7776 populate_metadata read hostname 'smithi064' 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x204c960 0x2072580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 4 client.7776 mds_command: new command op to 7605 tid=0[{"prefix": "status"}] 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 --> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1f0b410 con 0x204c960 2023-09-27T23:30:38.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b165e700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x204c960 0x2072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b165e700 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x204c960 0x2072580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42bc4599220ab447 server_cookie=cc610647ba45c94c in_seq=0 out_seq=0 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 10 client.7776 ms_handle_connect on v2:172.21.15.64:6834/1829030850 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 1 -- 172.21.15.64:0/1510673926 <== mds.0 v2:172.21.15.64:6834/1829030850 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x1f0b410 con 0x204c960 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 10 client.7776 handle_command_reply: tid=0 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 client.7776 shutdown 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 msgr2=0x204b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x204b520 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f54a807b470 tx=0x7f54a8064070 comp rx=0 tx=0).stop 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 shutdown_connections 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:6834/1829030850,v1:172.21.15.64:6835/1829030850] conn(0x204c960 0x2072580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1dda790 0x204bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1d8fae0 0x204ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f548bfff700 0 client.7776 ms_handle_reset on v2:172.21.15.64:6834/1829030850 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 --2- 172.21.15.64:0/1510673926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1d8f710 0x204b520 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 >> 172.21.15.64:0/1510673926 conn(0x1d8feb0 msgr2=0x1fc2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 shutdown_connections 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.571+0000 7f54b636c740 1 -- 172.21.15.64:0/1510673926 wait complete. 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b636c740 20 client.7776 trim_cache size 0 max 16384 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:30:38.579 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7605, 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.1784157769999997, 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 39, 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 115, 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 115, 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 13.734455128 2023-09-27T23:30:38.580 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5490060a70 msgr2=0x7f5490062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5490060a70 0x7f5490062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54a0007cd0 tx=0x7f54a0015000 comp rx=0 tx=0).stop 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 msgr2=0x7f54ac0a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f54ac0a59f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f54a8002950 tx=0x7f54a8007d30 comp rx=0 tx=0).stop 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 shutdown_connections 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5490060a70 0x7f5490062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac0c9dc0 0x7f54ac148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac14a6b0 0x7f54ac0c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 --2- 172.21.15.64:0/4164163110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5498004810 0x7f54ac0a59f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 >> 172.21.15.64:0/4164163110 conn(0x7f54ac067bb0 msgr2=0x7f54ac0cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 shutdown_connections 2023-09-27T23:30:38.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.575+0000 7f54b2e61700 1 -- 172.21.15.64:0/4164163110 wait complete. 2023-09-27T23:30:38.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:30:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 Processor -- start 2023-09-27T23:30:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 -- start start 2023-09-27T23:30:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc32006a540 0x7fc32006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3200ca5a0 con 0x7fc320158e70 2023-09-27T23:30:38.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32006dac0 con 0x7fc32006a540 2023-09-27T23:30:38.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc325fe0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32006d220 con 0x7fc3200cc470 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc32006a540 0x7fc32006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc3257df700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc32006a540 0x7fc32006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60706/0 (socket says 172.21.15.64:60706) 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc3257df700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49966/0 (socket says 172.21.15.64:49966) 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc324fde700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc320158e70 0x7fc3200afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37394/0 (socket says 172.21.15.64:37394) 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 -- 172.21.15.64:0/1950001220 learned_addr learned my addr 172.21.15.64:0/1950001220 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 -- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 msgr2=0x7fc3200cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 --2- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 -- 172.21.15.64:0/1950001220 >> v1:172.21.15.123:6789/0 conn(0x7fc320158e70 legacy=0x7fc3200afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 -- 172.21.15.64:0/1950001220 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32006d500 con 0x7fc32006a540 2023-09-27T23:30:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc3257df700 1 --2- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:38.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.702+0000 7fc31ffff700 1 --2- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32006c9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc3080099b0 tx=0x7fc308009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d20112e9c1009d8b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc31f7fe700 1 -- 172.21.15.64:0/1950001220 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308009ea0 con 0x7fc32006a540 2023-09-27T23:30:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc31f7fe700 1 -- 172.21.15.64:0/1950001220 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30800b860 con 0x7fc32006a540 2023-09-27T23:30:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc31f7fe700 1 -- 172.21.15.64:0/1950001220 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30800bc20 con 0x7fc32006a540 2023-09-27T23:30:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 -- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 msgr2=0x7fc32006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32006c9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc3080099b0 tx=0x7fc308009250 comp rx=0 tx=0).stop 2023-09-27T23:30:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 -- 172.21.15.64:0/1950001220 shutdown_connections 2023-09-27T23:30:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1950001220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32006c9d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 -- 172.21.15.64:0/1950001220 >> 172.21.15.64:0/1950001220 conn(0x7fc3200670e0 msgr2=0x7fc3200674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 -- 172.21.15.64:0/1950001220 shutdown_connections 2023-09-27T23:30:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 -- 172.21.15.64:0/1950001220 wait complete. 2023-09-27T23:30:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 Processor -- start 2023-09-27T23:30:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.706+0000 7fc325fe0700 1 -- start start 2023-09-27T23:30:38.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 0x7fc3200c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200b3a20 con 0x7fc320158e70 2023-09-27T23:30:38.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200b2e10 con 0x7fc32006a540 2023-09-27T23:30:38.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200b30f0 con 0x7fc3200cc470 2023-09-27T23:30:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 0x7fc3200c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 0x7fc3200c3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34642/0 (socket says 172.21.15.64:34642) 2023-09-27T23:30:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc324fde700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 -- 172.21.15.64:0/1733339994 learned_addr learned my addr 172.21.15.64:0/1733339994 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc31ffff700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 -- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 msgr2=0x7fc3200c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 -- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 msgr2=0x7fc32015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32015b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 -- 172.21.15.64:0/1733339994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc318009a20 con 0x7fc320158e70 2023-09-27T23:30:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc31ffff700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:38.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc324fde700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc3257df700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 0x7fc3200c3e90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc318009fc0 tx=0x7fc31800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31800cbd0 con 0x7fc320158e70 2023-09-27T23:30:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31800f040 con 0x7fc320158e70 2023-09-27T23:30:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3080094f0 con 0x7fc320158e70 2023-09-27T23:30:38.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318014480 con 0x7fc320158e70 2023-09-27T23:30:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.710+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3200b36b0 con 0x7fc320158e70 2023-09-27T23:30:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.714+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3200c3460 con 0x7fc320158e70 2023-09-27T23:30:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.714+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc318003020 con 0x7fc320158e70 2023-09-27T23:30:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.714+0000 7fc31cff9700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc300060aa0 0x7fc300062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.714+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc31807db60 con 0x7fc320158e70 2023-09-27T23:30:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.714+0000 7fc324fde700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc300060aa0 0x7fc300062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.714+0000 7fc324fde700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc300060aa0 0x7fc300062f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc310001710 tx=0x7fc310007eb0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:38.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.722+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc3200c3460 con 0x7fc320158e70 2023-09-27T23:30:38.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.918+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc3200c3460 con 0x7fc320158e70 2023-09-27T23:30:38.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.922+0000 7fc31cff9700 1 -- 172.21.15.64:0/1733339994 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc3200c3460 con 0x7fc320158e70 2023-09-27T23:30:38.924 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:30:38.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.922+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc300060aa0 msgr2=0x7fc300062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.922+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc300060aa0 0x7fc300062f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc310001710 tx=0x7fc310007eb0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.922+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 msgr2=0x7fc3200c3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.922+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 0x7fc3200c3e90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc318009fc0 tx=0x7fc31800b040 comp rx=0 tx=0).stop 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.922+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 shutdown_connections 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc300060aa0 0x7fc300062f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc320158e70 0x7fc3200c3e90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3200cc470 0x7fc3200c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 --2- 172.21.15.64:0/1733339994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc32006a540 0x7fc32015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 >> 172.21.15.64:0/1733339994 conn(0x7fc3200670e0 msgr2=0x7fc320158350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:38.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 shutdown_connections 2023-09-27T23:30:38.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:38.926+0000 7fc325fe0700 1 -- 172.21.15.64:0/1733339994 wait complete. 2023-09-27T23:30:38.943 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:30:38.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:30:38.944 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:30:38.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:38.945 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:30:38.953 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:30:38.953 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:30:38.953 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:30:38.953 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.953 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:30:38.953 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:30:38.954 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41545sec preferred_lft 41545sec 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:30:38.955 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:30:38.956 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:30:38.957 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:30:38.957 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.957 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:30:38.957 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:30:38.957 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:30:38.957 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:30:38.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:38.958 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:30:38.969 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:30:38.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:38.972 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:30:38.981 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:30:38.982 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:30:38.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:38.982 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:30:38.982 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:30:38.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:30:38.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:30:38.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:30:38.983 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:30:39.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:39.064 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:30:39.070 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:30:39.070 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:30:39.071 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:30:39.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:30:39.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:30:39.084 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:30:39.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:30:39.084 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:39.089 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:30:39.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:30:39.089 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:39.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:30:39.155 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:30:39.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:30:39.168 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:30:39.168 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:30:39.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:39.170 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:30:39.177 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:30:39.177 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:30:39.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:30:39.197 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:30:39.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:30:39.259 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:30:39.260 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:30:39.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:39.261 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:30:39.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:30:39.262+0000 7f3e75953380 -1 init, newargv = 0x5642868aa500 newargc=15 2023-09-27T23:30:39.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[52990]: starting ceph client 2023-09-27T23:30:39.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[52990]: starting fuse 2023-09-27T23:30:40.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:30:40.387 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:30:40.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:30:40.401 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:30:40.402 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:30:40.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:40.403 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:30:40.456 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:30:40.456 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:30:40.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:30:40.457 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:30:40.458 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:30:40.458 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:40.458 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:30:40.458 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:30:40.458 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:30:40.458 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:30:40.459 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:30:40.460 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:30:40.460 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:30:40.460 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:30:40.545 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.52990.asok 2023-09-27T23:30:40.729 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52990.asok 2023-09-27T23:30:40.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:40.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52990.asok status 2023-09-27T23:30:40.825 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "52990", 2023-09-27T23:30:40.826 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7785, 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 7785 2023-09-27T23:30:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3022511111 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3022511111 2023-09-27T23:30:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.7785 192.168.0.1:0/3022511111", 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3022511111", 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 39, 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 115, 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:30:40.829 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:30:40.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:40.835 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:30:40.835 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:40.844 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:30:40.844 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:40.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:30:40.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:30:40.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:30:41.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 Processor -- start 2023-09-27T23:30:41.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 -- start start 2023-09-27T23:30:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fad08158e20 0x7fad0806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad081591f0 0x7fad080ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0806b730 con 0x7fad0815ca80 2023-09-27T23:30:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0806ab20 con 0x7fad081591f0 2023-09-27T23:30:41.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0df1e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0806ae00 con 0x7fad08158e20 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad081591f0 0x7fad080ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad081591f0 0x7fad080ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34644/0 (socket says 172.21.15.64:34644) 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 -- 172.21.15.64:0/2331754206 learned_addr learned my addr 172.21.15.64:0/2331754206 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0cf1c700 1 --2- 172.21.15.64:0/2331754206 >> v2:172.21.15.123:3300/0 conn(0x7fad08158e20 0x7fad0806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 -- 172.21.15.64:0/2331754206 >> v2:172.21.15.123:3300/0 conn(0x7fad08158e20 msgr2=0x7fad0806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 --2- 172.21.15.64:0/2331754206 >> v2:172.21.15.123:3300/0 conn(0x7fad08158e20 0x7fad0806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 -- 172.21.15.64:0/2331754206 >> v1:172.21.15.64:6789/0 conn(0x7fad0815ca80 legacy=0x7fad0806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad077fe700 1 -- 172.21.15.64:0/2331754206 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035180951 0 0) 0x7fad0806b730 con 0x7fad0815ca80 2023-09-27T23:30:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad07fff700 1 -- 172.21.15.64:0/2331754206 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0806b0e0 con 0x7fad081591f0 2023-09-27T23:30:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.018+0000 7fad0cf1c700 1 --2- 172.21.15.64:0/2331754206 >> v2:172.21.15.123:3300/0 conn(0x7fad08158e20 0x7fad0806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad07fff700 1 --2- 172.21.15.64:0/2331754206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad080ce100 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fad0806e510 tx=0x7facf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e91596eb6c80818 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad077fe700 1 -- 172.21.15.64:0/2331754206 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000b860 con 0x7fad081591f0 2023-09-27T23:30:41.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad077fe700 1 -- 172.21.15.64:0/2331754206 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf000b9c0 con 0x7fad081591f0 2023-09-27T23:30:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad077fe700 1 -- 172.21.15.64:0/2331754206 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000bd60 con 0x7fad081591f0 2023-09-27T23:30:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 -- 172.21.15.64:0/2331754206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 msgr2=0x7fad080ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 --2- 172.21.15.64:0/2331754206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad080ce100 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fad0806e510 tx=0x7facf0009250 comp rx=0 tx=0).stop 2023-09-27T23:30:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 -- 172.21.15.64:0/2331754206 shutdown_connections 2023-09-27T23:30:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 --2- 172.21.15.64:0/2331754206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad080ce100 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 --2- 172.21.15.64:0/2331754206 >> v2:172.21.15.123:3300/0 conn(0x7fad08158e20 0x7fad0806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 -- 172.21.15.64:0/2331754206 >> 172.21.15.64:0/2331754206 conn(0x7fad08067bb0 msgr2=0x7fad080cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 -- 172.21.15.64:0/2331754206 shutdown_connections 2023-09-27T23:30:41.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 -- 172.21.15.64:0/2331754206 wait complete. 2023-09-27T23:30:41.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 Processor -- start 2023-09-27T23:30:41.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.022+0000 7fad0df1e700 1 -- start start 2023-09-27T23:30:41.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad08158e20 0x7fad0815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad08152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad0815ca80 0x7fad08160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad081595c0 con 0x7fad081591f0 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08152f50 con 0x7fad0815ca80 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad08159860 con 0x7fad08158e20 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad08152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad08152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34652/0 (socket says 172.21.15.64:34652) 2023-09-27T23:30:41.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 -- 172.21.15.64:0/3897718523 learned_addr learned my addr 172.21.15.64:0/3897718523 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0d71d700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad0815ca80 0x7fad08160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0cf1c700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad08158e20 0x7fad0815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 -- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad08158e20 msgr2=0x7fad0815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad08158e20 0x7fad0815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 -- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad0815ca80 msgr2=0x7fad08160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad0815ca80 0x7fad08160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 -- 172.21.15.64:0/3897718523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00094f0 con 0x7fad081591f0 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad07fff700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad08152410 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7facf00090b0 tx=0x7facf0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0011e60 con 0x7fad081591f0 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf001f070 con 0x7fad081591f0 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08159b40 con 0x7fad081591f0 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001a890 con 0x7fad081591f0 2023-09-27T23:30:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0815a100 con 0x7fad081591f0 2023-09-27T23:30:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.026+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0804be40 con 0x7fad081591f0 2023-09-27T23:30:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.030+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7facf001ac00 con 0x7fad081591f0 2023-09-27T23:30:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.030+0000 7fad04ff9700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face8060a70 0x7face8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.030+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7facf0021070 con 0x7fad081591f0 2023-09-27T23:30:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.030+0000 7fad0cf1c700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face8060a70 0x7face8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.030+0000 7fad0cf1c700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face8060a70 0x7face8062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7facf80076a0 tx=0x7facf80098f0 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:41.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.034+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7facf004f090 con 0x7fad081591f0 2023-09-27T23:30:41.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.226+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fad0806b3c0 con 0x7face8060a70 2023-09-27T23:30:41.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad04ff9700 1 -- 172.21.15.64:0/3897718523 <== mgr.6578 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fad0806b3c0 con 0x7face8060a70 2023-09-27T23:30:41.233 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:30:41.233 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:30:41.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:30:41.233 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:30:41.233 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face8060a70 msgr2=0x7face8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face8060a70 0x7face8062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7facf80076a0 tx=0x7facf80098f0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 msgr2=0x7fad08152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad08152410 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7facf00090b0 tx=0x7facf0017000 comp rx=0 tx=0).stop 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 shutdown_connections 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7face8060a70 0x7face8062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad0815ca80 0x7fad08160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad081591f0 0x7fad08152410 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 --2- 172.21.15.64:0/3897718523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad08158e20 0x7fad0815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 >> 172.21.15.64:0/3897718523 conn(0x7fad08067bb0 msgr2=0x7fad08154230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 shutdown_connections 2023-09-27T23:30:41.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.230+0000 7fad0df1e700 1 -- 172.21.15.64:0/3897718523 wait complete. 2023-09-27T23:30:41.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:30:41.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 Processor -- start 2023-09-27T23:30:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 -- start start 2023-09-27T23:30:41.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 0x7fd8300c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd83006de00 0x7fd83006b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83006d970 con 0x7fd83006c7c0 2023-09-27T23:30:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83006cd60 con 0x7fd83006de00 2023-09-27T23:30:41.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.354+0000 7fd8353c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83006d040 con 0x7fd8300c69a0 2023-09-27T23:30:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 0x7fd8300c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd83006de00 0x7fd83006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 0x7fd8300c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49978/0 (socket says 172.21.15.64:49978) 2023-09-27T23:30:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82ffff700 1 -- 172.21.15.64:0/891760012 learned_addr learned my addr 172.21.15.64:0/891760012 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:41.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd83006de00 0x7fd83006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:60746/0 (socket says 172.21.15.64:60746) 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd834bc3700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd83006c7c0 0x7fd83015cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:37398/0 (socket says 172.21.15.64:37398) 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 -- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 msgr2=0x7fd8300c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 --2- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 0x7fd8300c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 -- 172.21.15.64:0/891760012 >> v1:172.21.15.123:6789/0 conn(0x7fd83006c7c0 legacy=0x7fd83015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 -- 172.21.15.64:0/891760012 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83006d320 con 0x7fd83006de00 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82ffff700 1 --2- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 0x7fd8300c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:41.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82f7fe700 1 --2- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd83006de00 0x7fd83006b880 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd820009910 tx=0x7fd8200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af49a4f35ac0941 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:41.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82effd700 1 -- 172.21.15.64:0/891760012 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000b640 con 0x7fd83006de00 2023-09-27T23:30:41.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82effd700 1 -- 172.21.15.64:0/891760012 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82000b7a0 con 0x7fd83006de00 2023-09-27T23:30:41.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd82effd700 1 -- 172.21.15.64:0/891760012 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000bb60 con 0x7fd83006de00 2023-09-27T23:30:41.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 -- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd83006de00 msgr2=0x7fd83006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 --2- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd83006de00 0x7fd83006b880 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd820009910 tx=0x7fd8200091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 -- 172.21.15.64:0/891760012 shutdown_connections 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 --2- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd83006de00 0x7fd83006b880 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 --2- 172.21.15.64:0/891760012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8300c69a0 0x7fd8300c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 -- 172.21.15.64:0/891760012 >> 172.21.15.64:0/891760012 conn(0x7fd830067b50 msgr2=0x7fd8300cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 -- 172.21.15.64:0/891760012 shutdown_connections 2023-09-27T23:30:41.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.358+0000 7fd8353c4700 1 -- 172.21.15.64:0/891760012 wait complete. 2023-09-27T23:30:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.362+0000 7fd8353c4700 1 Processor -- start 2023-09-27T23:30:41.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.362+0000 7fd8353c4700 1 -- start start 2023-09-27T23:30:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.362+0000 7fd8353c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd83006c7c0 0x7fd830170400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.362+0000 7fd8353c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 0x7fd8300b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8300c69a0 0x7fd8300ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83015d9f0 con 0x7fd83006de00 2023-09-27T23:30:41.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83015cde0 con 0x7fd8300c69a0 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83015d0c0 con 0x7fd83006c7c0 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 0x7fd8300b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd82ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd83006c7c0 0x7fd830170400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8300c69a0 0x7fd8300ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd82ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd83006c7c0 0x7fd830170400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:49994/0 (socket says 172.21.15.64:49994) 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd82ffff700 1 -- 172.21.15.64:0/2979545081 learned_addr learned my addr 172.21.15.64:0/2979545081 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:41.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 0x7fd8300b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34664/0 (socket says 172.21.15.64:34664) 2023-09-27T23:30:41.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 -- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd83006c7c0 msgr2=0x7fd830170400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd83006c7c0 0x7fd830170400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 -- 172.21.15.64:0/2979545081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 msgr2=0x7fd8300b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:41.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 0x7fd8300b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 -- 172.21.15.64:0/2979545081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240094f0 con 0x7fd8300c69a0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd82f7fe700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 0x7fd8300b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd834bc3700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8300c69a0 0x7fd8300ad030 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd82800cc40 tx=0x7fd828001a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828005df0 con 0x7fd8300c69a0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd820009420 con 0x7fd8300c69a0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800a6a0 con 0x7fd8300c69a0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800aa20 con 0x7fd8300c69a0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83015d680 con 0x7fd8300c69a0 2023-09-27T23:30:41.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.366+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83015db60 con 0x7fd8300c69a0 2023-09-27T23:30:41.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.370+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 40) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd8280070f0 con 0x7fd8300c69a0 2023-09-27T23:30:41.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.370+0000 7fd817fff700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd81c060a70 0x7fd81c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:41.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.370+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd82807dd30 con 0x7fd8300c69a0 2023-09-27T23:30:41.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.370+0000 7fd82ffff700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd81c060a70 0x7fd81c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:41.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.370+0000 7fd82ffff700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd81c060a70 0x7fd81c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8300add20 tx=0x7fd824009250 comp rx=0 tx=0).ready entity=mgr.6578 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.378+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd82804dde0 con 0x7fd8300c69a0 2023-09-27T23:30:41.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:41.578+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd83004be40 con 0x7fd8300c69a0 2023-09-27T23:30:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.474+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 41) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fd82804e1b0 con 0x7fd8300c69a0 2023-09-27T23:30:42.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.478+0000 7fd817fff700 1 -- 172.21.15.64:0/2979545081 <== mon.1 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v41) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd82804fbe0 con 0x7fd8300c69a0 2023-09-27T23:30:42.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd81c060a70 msgr2=0x7fd81c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd81c060a70 0x7fd81c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8300add20 tx=0x7fd824009250 comp rx=0 tx=0).stop 2023-09-27T23:30:42.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8300c69a0 msgr2=0x7fd8300ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8300c69a0 0x7fd8300ad030 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd82800cc40 tx=0x7fd828001a50 comp rx=0 tx=0).stop 2023-09-27T23:30:42.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 shutdown_connections 2023-09-27T23:30:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd81c060a70 0x7fd81c062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8300c69a0 0x7fd8300ad030 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83006de00 0x7fd8300b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 --2- 172.21.15.64:0/2979545081 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd83006c7c0 0x7fd830170400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 >> 172.21.15.64:0/2979545081 conn(0x7fd830067b50 msgr2=0x7fd8300cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:42.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 shutdown_connections 2023-09-27T23:30:42.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.482+0000 7fd8353c4700 1 -- 172.21.15.64:0/2979545081 wait complete. 2023-09-27T23:30:42.499 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:30:42.597 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:42.597+0000 7f11e4bde000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:30:42.599 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:42.594+0000 7f140d61e000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:30:42.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.598+0000 7fe1d5e45700 1 Processor -- start 2023-09-27T23:30:42.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.598+0000 7fe1d5e45700 1 -- start start 2023-09-27T23:30:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1d01514a0 0x7fe1d0157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00cbee0 con 0x7fe1d006b7a0 2023-09-27T23:30:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d0158530 con 0x7fe1d00cae60 2023-09-27T23:30:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d00cc370 con 0x7fe1d01514a0 2023-09-27T23:30:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1d01514a0 0x7fe1d0157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1d01514a0 0x7fe1d0157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34702/0 (socket says 172.21.15.64:34702) 2023-09-27T23:30:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe1d006b7a0 0x7fe1d0158740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52052/0 (socket says 172.21.15.64:52052) 2023-09-27T23:30:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 -- 172.21.15.64:0/743500978 learned_addr learned my addr 172.21.15.64:0/743500978 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 -- 172.21.15.64:0/743500978 >> v1:172.21.15.123:6789/0 conn(0x7fe1d00cae60 legacy=0x7fe1d006b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 -- 172.21.15.64:0/743500978 >> v1:172.21.15.64:6789/0 conn(0x7fe1d006b7a0 legacy=0x7fe1d0158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 -- 172.21.15.64:0/743500978 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00cb2d0 con 0x7fe1d01514a0 2023-09-27T23:30:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1cf7fe700 1 --2- 172.21.15.64:0/743500978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0157ff0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00099b0 tx=0x7fe1c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba572c767910b3e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1ce7fc700 1 -- 172.21.15.64:0/743500978 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b860 con 0x7fe1d01514a0 2023-09-27T23:30:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1ce7fc700 1 -- 172.21.15.64:0/743500978 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000b9c0 con 0x7fe1d01514a0 2023-09-27T23:30:42.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1ce7fc700 1 -- 172.21.15.64:0/743500978 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000bd60 con 0x7fe1d01514a0 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- 172.21.15.64:0/743500978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 msgr2=0x7fe1d0157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/743500978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0157ff0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00099b0 tx=0x7fe1c0009250 comp rx=0 tx=0).stop 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- 172.21.15.64:0/743500978 shutdown_connections 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/743500978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0157ff0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- 172.21.15.64:0/743500978 >> 172.21.15.64:0/743500978 conn(0x7fe1d00cc580 msgr2=0x7fe1d00cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- 172.21.15.64:0/743500978 shutdown_connections 2023-09-27T23:30:42.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.602+0000 7fe1d5e45700 1 -- 172.21.15.64:0/743500978 wait complete. 2023-09-27T23:30:42.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 Processor -- start 2023-09-27T23:30:42.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 -- start start 2023-09-27T23:30:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d006b7a0 0x7fe1d006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00cae60 0x7fe1d00a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0169570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d016c920 con 0x7fe1d01514a0 2023-09-27T23:30:42.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d006d820 con 0x7fe1d00cae60 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d006db00 con 0x7fe1d006b7a0 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0169570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34710/0 (socket says 172.21.15.64:34710) 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 -- 172.21.15.64:0/3222042329 learned_addr learned my addr 172.21.15.64:0/3222042329 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1ceffd700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00cae60 0x7fe1d00a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cf7fe700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d006b7a0 0x7fe1d006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 -- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d006b7a0 msgr2=0x7fe1d006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d006b7a0 0x7fe1d006d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 -- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00cae60 msgr2=0x7fe1d00a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00cae60 0x7fe1d00a9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 -- 172.21.15.64:0/3222042329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00094f0 con 0x7fe1d01514a0 2023-09-27T23:30:42.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1cffff700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0169570 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800cd30 tx=0x7fe1c8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c800ce90 con 0x7fe1d01514a0 2023-09-27T23:30:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c8005df0 con 0x7fe1d01514a0 2023-09-27T23:30:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d006dde0 con 0x7fe1d01514a0 2023-09-27T23:30:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c800aaa0 con 0x7fe1d01514a0 2023-09-27T23:30:42.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.606+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d00a9960 con 0x7fe1d01514a0 2023-09-27T23:30:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.610+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d0071120 con 0x7fe1d01514a0 2023-09-27T23:30:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.610+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 41) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fe1c80070f0 con 0x7fe1d01514a0 2023-09-27T23:30:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.610+0000 7fe1b77fe700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1bc060ac0 0x7fe1bc062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.610+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe1c807d250 con 0x7fe1d01514a0 2023-09-27T23:30:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.610+0000 7fe1cf7fe700 1 -- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1bc060ac0 msgr2=0x7fe1bc062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:42.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.610+0000 7fe1cf7fe700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1bc060ac0 0x7fe1bc062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:42.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.618+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe1c804d280 con 0x7fe1d01514a0 2023-09-27T23:30:42.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.782+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe1d00cb5b0 con 0x7fe1d01514a0 2023-09-27T23:30:42.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.790+0000 7fe1b77fe700 1 -- 172.21.15.64:0/3222042329 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe1c80508a0 con 0x7fe1d01514a0 2023-09-27T23:30:42.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1bc060ac0 msgr2=0x7fe1bc062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:42.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1bc060ac0 0x7fe1bc062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 msgr2=0x7fe1d0169570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0169570 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800cd30 tx=0x7fe1c8001ae0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 shutdown_connections 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1bc060ac0 0x7fe1bc062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1d01514a0 0x7fe1d0169570 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00cae60 0x7fe1d00a9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 --2- 172.21.15.64:0/3222042329 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1d006b7a0 0x7fe1d006d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 >> 172.21.15.64:0/3222042329 conn(0x7fe1d00cc580 msgr2=0x7fe1d00b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 shutdown_connections 2023-09-27T23:30:42.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.794+0000 7fe1d5e45700 1 -- 172.21.15.64:0/3222042329 wait complete. 2023-09-27T23:30:42.808 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:30:42.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.918+0000 7f69957de700 1 Processor -- start 2023-09-27T23:30:42.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.918+0000 7f69957de700 1 -- start start 2023-09-27T23:30:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.918+0000 7f69957de700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f69901547a0 0x7f69900ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6990157fc0 0x7f6990158480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69900c7820 con 0x7f699006d050 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69900c7210 con 0x7f6990157fc0 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69900ae030 con 0x7f69901547a0 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6990157fc0 0x7f6990158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6990157fc0 0x7f6990158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34726/0 (socket says 172.21.15.64:34726) 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 -- 172.21.15.64:0/99806779 learned_addr learned my addr 172.21.15.64:0/99806779 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698ffff700 1 --2- 172.21.15.64:0/99806779 >> v2:172.21.15.123:3300/0 conn(0x7f69901547a0 0x7f69900ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 -- 172.21.15.64:0/99806779 >> v2:172.21.15.123:3300/0 conn(0x7f69901547a0 msgr2=0x7f69900ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 --2- 172.21.15.64:0/99806779 >> v2:172.21.15.123:3300/0 conn(0x7f69901547a0 0x7f69900ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 -- 172.21.15.64:0/99806779 >> v1:172.21.15.123:6789/0 conn(0x7f699006d050 legacy=0x7f69900c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 -- 172.21.15.64:0/99806779 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69900c74b0 con 0x7f6990157fc0 2023-09-27T23:30:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698f7fe700 1 --2- 172.21.15.64:0/99806779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6990157fc0 0x7f6990158480 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f69900cd960 tx=0x7f6984009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf60c3ab1653ea2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698effd700 1 -- 172.21.15.64:0/99806779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400b820 con 0x7f6990157fc0 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698effd700 1 -- 172.21.15.64:0/99806779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698400b980 con 0x7f6990157fc0 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f698effd700 1 -- 172.21.15.64:0/99806779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400bd20 con 0x7f6990157fc0 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- 172.21.15.64:0/99806779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6990157fc0 msgr2=0x7f6990158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 --2- 172.21.15.64:0/99806779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6990157fc0 0x7f6990158480 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f69900cd960 tx=0x7f6984009250 comp rx=0 tx=0).stop 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- 172.21.15.64:0/99806779 shutdown_connections 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 --2- 172.21.15.64:0/99806779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6990157fc0 0x7f6990158480 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 --2- 172.21.15.64:0/99806779 >> v2:172.21.15.123:3300/0 conn(0x7f69901547a0 0x7f69900ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- 172.21.15.64:0/99806779 >> 172.21.15.64:0/99806779 conn(0x7f69900ccb30 msgr2=0x7f69900ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:42.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- 172.21.15.64:0/99806779 shutdown_connections 2023-09-27T23:30:42.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- 172.21.15.64:0/99806779 wait complete. 2023-09-27T23:30:42.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 Processor -- start 2023-09-27T23:30:42.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.922+0000 7f69957de700 1 -- start start 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699006d050 0x7f69901573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 0x7f699014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6990157fc0 0x7f6990150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6990156310 con 0x7f69901547a0 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69901565f0 con 0x7f699006d050 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69901568d0 con 0x7f6990157fc0 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 0x7f699014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 0x7f699014eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34730/0 (socket says 172.21.15.64:34730) 2023-09-27T23:30:42.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 -- 172.21.15.64:0/1343287248 learned_addr learned my addr 172.21.15.64:0/1343287248 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698ffff700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699006d050 0x7f69901573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f6994fdd700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6990157fc0 0x7f6990150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 -- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6990157fc0 msgr2=0x7f6990150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6990157fc0 0x7f6990150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 -- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699006d050 msgr2=0x7f69901573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699006d050 0x7f69901573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 -- 172.21.15.64:0/1343287248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840094f0 con 0x7f69901547a0 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f6994fdd700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6990157fc0 0x7f6990150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:42.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f698f7fe700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 0x7f699014eb70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f69840030c0 tx=0x7f6984016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984011db0 con 0x7f69901547a0 2023-09-27T23:30:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6984020070 con 0x7f69901547a0 2023-09-27T23:30:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698401b6a0 con 0x7f69901547a0 2023-09-27T23:30:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6990156bb0 con 0x7f69901547a0 2023-09-27T23:30:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699014f0b0 con 0x7f69901547a0 2023-09-27T23:30:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.926+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699004be40 con 0x7f69901547a0 2023-09-27T23:30:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.930+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 41) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f6984022030 con 0x7f69901547a0 2023-09-27T23:30:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.930+0000 7f6977fff700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f697c060af0 0x7f697c062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.930+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f698407f010 con 0x7f69901547a0 2023-09-27T23:30:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.930+0000 7f698ffff700 1 -- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f697c060af0 msgr2=0x7f697c062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.930+0000 7f698ffff700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f697c060af0 0x7f697c062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:42.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:42.938+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f698404f040 con 0x7f69901547a0 2023-09-27T23:30:43.016 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.013+0000 7f11e4bde000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:30:43.050 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.046+0000 7f140d61e000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:30:43.063 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.061+0000 7f11e4bde000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:30:43.102 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.098+0000 7f140d61e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:30:43.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.102+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f699004be40 con 0x7f69901547a0 2023-09-27T23:30:43.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.110+0000 7f6977fff700 1 -- 172.21.15.64:0/1343287248 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6984052660 con 0x7f69901547a0 2023-09-27T23:30:43.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f697c060af0 msgr2=0x7f697c062fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f697c060af0 0x7f697c062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 msgr2=0x7f699014eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 0x7f699014eb70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f69840030c0 tx=0x7f6984016000 comp rx=0 tx=0).stop 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 shutdown_connections 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f697c060af0 0x7f697c062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6990157fc0 0x7f6990150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69901547a0 0x7f699014eb70 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 --2- 172.21.15.64:0/1343287248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f699006d050 0x7f69901573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 >> 172.21.15.64:0/1343287248 conn(0x7f69900ccb30 msgr2=0x7f69900b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 shutdown_connections 2023-09-27T23:30:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.114+0000 7f69957de700 1 -- 172.21.15.64:0/1343287248 wait complete. 2023-09-27T23:30:43.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-09-27T23:30:43.170 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.169+0000 7f11e4bde000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:43.214 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.210+0000 7f140d61e000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:43.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 Processor -- start 2023-09-27T23:30:43.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 -- start start 2023-09-27T23:30:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73841558d0 0x7f73841569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73840abe10 0x7f73840ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f73840adc80 0x7f73840c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7384156fa0 con 0x7f73840abe10 2023-09-27T23:30:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840abc60 con 0x7f73840adc80 2023-09-27T23:30:43.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7388bf2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840aeba0 con 0x7f73841558d0 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73840abe10 0x7f73840ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73840abe10 0x7f73840ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34738/0 (socket says 172.21.15.64:34738) 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 -- 172.21.15.64:0/2577305868 learned_addr learned my addr 172.21.15.64:0/2577305868 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f73837fe700 1 --2- 172.21.15.64:0/2577305868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73841558d0 0x7f73841569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7383fff700 1 --2- 172.21.15.64:0/2577305868 >> v2:172.21.15.123:3300/0 conn(0x7f73840adc80 0x7f73840c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 -- 172.21.15.64:0/2577305868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73841558d0 msgr2=0x7f73841569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 --2- 172.21.15.64:0/2577305868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73841558d0 0x7f73841569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 -- 172.21.15.64:0/2577305868 >> v2:172.21.15.123:3300/0 conn(0x7f73840adc80 msgr2=0x7f73840c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 --2- 172.21.15.64:0/2577305868 >> v2:172.21.15.123:3300/0 conn(0x7f73840adc80 0x7f73840c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.238+0000 7f7382ffd700 1 -- 172.21.15.64:0/2577305868 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840cbfb0 con 0x7f73840abe10 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7382ffd700 1 --2- 172.21.15.64:0/2577305868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73840abe10 0x7f73840ad740 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f73840b2b10 tx=0x7f7378009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170052475032b86a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:43.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f73827fc700 1 -- 172.21.15.64:0/2577305868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800b640 con 0x7f73840abe10 2023-09-27T23:30:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f73827fc700 1 -- 172.21.15.64:0/2577305868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737800b7a0 con 0x7f73840abe10 2023-09-27T23:30:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f73827fc700 1 -- 172.21.15.64:0/2577305868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800bb60 con 0x7f73840abe10 2023-09-27T23:30:43.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 -- 172.21.15.64:0/2577305868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73840abe10 msgr2=0x7f73840ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 --2- 172.21.15.64:0/2577305868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73840abe10 0x7f73840ad740 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f73840b2b10 tx=0x7f7378009250 comp rx=0 tx=0).stop 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 -- 172.21.15.64:0/2577305868 shutdown_connections 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 --2- 172.21.15.64:0/2577305868 >> v2:172.21.15.123:3300/0 conn(0x7f73840adc80 0x7f73840c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 --2- 172.21.15.64:0/2577305868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73840abe10 0x7f73840ad740 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 --2- 172.21.15.64:0/2577305868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73841558d0 0x7f73841569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 -- 172.21.15.64:0/2577305868 >> 172.21.15.64:0/2577305868 conn(0x7f738406b5b0 msgr2=0x7f738406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 -- 172.21.15.64:0/2577305868 shutdown_connections 2023-09-27T23:30:43.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 -- 172.21.15.64:0/2577305868 wait complete. 2023-09-27T23:30:43.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 Processor -- start 2023-09-27T23:30:43.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.242+0000 7f7388bf2700 1 -- start start 2023-09-27T23:30:43.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73840abe10 0x7f73840c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73840adc80 0x7f738414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 0x7f738414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840c50b0 con 0x7f73841558d0 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840c44a0 con 0x7f73840adc80 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840c4780 con 0x7f73840abe10 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 0x7f738414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 0x7f738414eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34742/0 (socket says 172.21.15.64:34742) 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 -- 172.21.15.64:0/1017115328 learned_addr learned my addr 172.21.15.64:0/1017115328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73840abe10 0x7f73840c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7382ffd700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73840adc80 0x7f738414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73840abe10 msgr2=0x7f73840c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73840abe10 0x7f73840c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73840adc80 msgr2=0x7f738414f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73840adc80 0x7f738414f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 -- 172.21.15.64:0/1017115328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780094f0 con 0x7f73841558d0 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7383fff700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 0x7f738414eb90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f737c009e90 tx=0x7f737c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00cb70 con 0x7f73841558d0 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c00f040 con 0x7f73841558d0 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73840c4a60 con 0x7f73841558d0 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c014480 con 0x7f73841558d0 2023-09-27T23:30:43.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73840c41b0 con 0x7f73841558d0 2023-09-27T23:30:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73840c5220 con 0x7f73841558d0 2023-09-27T23:30:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 41) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f737c014830 con 0x7f73841558d0 2023-09-27T23:30:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f736b7fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f737c07dc80 con 0x7f73841558d0 2023-09-27T23:30:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.246+0000 7f73837fe700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 msgr2=0x7f7370062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:43.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.250+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:30:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.258+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f737c04dba0 con 0x7f73841558d0 2023-09-27T23:30:43.364 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.365+0000 7f11e4bde000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:30:43.413 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.413+0000 7f11e4bde000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:43.420 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.418+0000 7f140d61e000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:30:43.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.438+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f738404be40 con 0x7f7370060ac0 2023-09-27T23:30:43.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.454+0000 7f73837fe700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 msgr2=0x7f7370062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:43.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.454+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:30:43.472 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.470+0000 7f140d61e000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:43.565 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.565+0000 7f11e4bde000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:30:43.621 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.621+0000 7f11e4bde000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:30:43.629 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.626+0000 7f140d61e000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:30:43.679 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.677+0000 7f11e4bde000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:30:43.687 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.682+0000 7f140d61e000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:30:43.748 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:43.746+0000 7f140d61e000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:30:43.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.858+0000 7f73837fe700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 msgr2=0x7f7370062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:43.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:43.858+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:30:43.974 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:43.973+0000 7f11e4bde000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:30:44.059 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.054+0000 7f140d61e000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:30:44.084 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.085+0000 7f11e4bde000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:30:44.175 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.170+0000 7f140d61e000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:30:44.362 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.361+0000 7f11e4bde000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:30:44.469 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.466+0000 7f140d61e000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:30:44.521 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.521+0000 7f11e4bde000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:44.585 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.585+0000 7f11e4bde000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:30:44.638 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.637+0000 7f11e4bde000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:30:44.639 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.634+0000 7f140d61e000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:30:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:44.662+0000 7f73837fe700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 msgr2=0x7f7370062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:30:44.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:44.662+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:30:44.691 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.689+0000 7f11e4bde000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:30:44.706 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.702+0000 7f140d61e000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:30:44.761 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.758+0000 7f140d61e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:30:44.817 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.814+0000 7f140d61e000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:30:44.865 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.865+0000 7f11e4bde000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:44.917 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.917+0000 7f11e4bde000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:30:44.980 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:44.981+0000 7f11e4bde000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:30:44.995 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:44.990+0000 7f140d61e000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:30:45.054 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:45.050+0000 7f140d61e000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:30:45.119 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:45.114+0000 7f140d61e000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:30:45.129 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.129+0000 7f11e4bde000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:30:45.190 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.189+0000 7f11e4bde000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:30:45.275 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:45.270+0000 7f140d61e000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:30:45.338 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:45.334+0000 7f140d61e000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:30:45.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.777+0000 7f11e4bde000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:30:45.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:45.810+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 42) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f737c07bef0 con 0x7f73841558d0 2023-09-27T23:30:45.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:45.814+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 msgr2=0x7f7370062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:30:45.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:45.814+0000 7f736b7fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:45.978 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:30:45.974+0000 7f140d61e000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:30:45.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a5589700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a5589700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a5589700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a5589700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.986 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a5589700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a858f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a858f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a858f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a858f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:45.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:30:45.985+0000 7f11a858f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:30:46.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.814+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 43) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f737c03cfc0 con 0x7f73841558d0 2023-09-27T23:30:46.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.814+0000 7f736b7fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:46.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.814+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f738404be40 con 0x7f7370060ac0 2023-09-27T23:30:46.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.822+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:46.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.822+0000 7f73837fe700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f73840b2930 tx=0x7f73740091a0 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:46.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.938+0000 7f736b7fe700 1 -- 172.21.15.64:0/1017115328 <== mgr.7765 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f738404be40 con 0x7f7370060ac0 2023-09-27T23:30:46.942 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /bad-path 2023-09-27T23:30:46.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 msgr2=0x7f7370062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:46.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f73840b2930 tx=0x7f73740091a0 comp rx=0 tx=0).stop 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 msgr2=0x7f738414eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 0x7f738414eb90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f737c009e90 tx=0x7f737c00b040 comp rx=0 tx=0).stop 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 shutdown_connections 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7370060ac0 0x7f7370062ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73841558d0 0x7f738414eb90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73840adc80 0x7f738414f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 --2- 172.21.15.64:0/1017115328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73840abe10 0x7f73840c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 >> 172.21.15.64:0/1017115328 conn(0x7f738406b5b0 msgr2=0x7f738406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:46.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 shutdown_connections 2023-09-27T23:30:46.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:46.942+0000 7f7388bf2700 1 -- 172.21.15.64:0/1017115328 wait complete. 2023-09-27T23:30:46.961 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-09-27T23:30:46.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-09-27T23:30:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.074+0000 7f58dd62b700 1 Processor -- start 2023-09-27T23:30:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.074+0000 7f58dd62b700 1 -- start start 2023-09-27T23:30:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58d815c760 0x7f58d80b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80b0ac0 con 0x7f58d80cb130 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d815cbe0 con 0x7f58d815c760 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d806b210 con 0x7f58d80cb500 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58d815c760 0x7f58d80b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d6ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58d80cb130 0x7f58d806a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47580/0 (socket says 172.21.15.64:47580) 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58d815c760 0x7f58d80b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36196/0 (socket says 172.21.15.64:36196) 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d6ffd700 1 -- 172.21.15.64:0/3946557469 learned_addr learned my addr 172.21.15.64:0/3946557469 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d67fc700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 -- 172.21.15.64:0/3946557469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 msgr2=0x7f58d815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:30:47.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 -- 172.21.15.64:0/3946557469 >> v1:172.21.15.64:6789/0 conn(0x7f58d80cb130 legacy=0x7f58d806a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:47.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d5ffb700 1 -- 172.21.15.64:0/3946557469 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490878669 0 0) 0x7f58d80b0ac0 con 0x7f58d80cb130 2023-09-27T23:30:47.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 -- 172.21.15.64:0/3946557469 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80ca810 con 0x7f58d815c760 2023-09-27T23:30:47.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d67fc700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:30:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d77fe700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d80b0f50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e60 tx=0x7f58d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22e538656250e67 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d5ffb700 1 -- 172.21.15.64:0/3946557469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000e070 con 0x7f58d815c760 2023-09-27T23:30:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d5ffb700 1 -- 172.21.15.64:0/3946557469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d000bc30 con 0x7f58d815c760 2023-09-27T23:30:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58d5ffb700 1 -- 172.21.15.64:0/3946557469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0012430 con 0x7f58d815c760 2023-09-27T23:30:47.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- 172.21.15.64:0/3946557469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 msgr2=0x7f58d80b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d80b0f50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e60 tx=0x7f58d0009780 comp rx=0 tx=0).stop 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- 172.21.15.64:0/3946557469 shutdown_connections 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d80b0f50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 --2- 172.21.15.64:0/3946557469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- 172.21.15.64:0/3946557469 >> 172.21.15.64:0/3946557469 conn(0x7f58d8067bb0 msgr2=0x7f58d80cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- 172.21.15.64:0/3946557469 shutdown_connections 2023-09-27T23:30:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.078+0000 7f58dd62b700 1 -- 172.21.15.64:0/3946557469 wait complete. 2023-09-27T23:30:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 Processor -- start 2023-09-27T23:30:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 -- start start 2023-09-27T23:30:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58d80cb130 0x7f58d80c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d81469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d815d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80b3440 con 0x7f58d815c760 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81578c0 con 0x7f58d80cb130 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58dd62b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c5430 con 0x7f58d80cb500 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36206/0 (socket says 172.21.15.64:36206) 2023-09-27T23:30:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58d77fe700 1 -- 172.21.15.64:0/4037521121 learned_addr learned my addr 172.21.15.64:0/4037521121 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:30:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58d6ffd700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58d80cb130 0x7f58d80c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.082+0000 7f58d67fc700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d81469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58d77fe700 1 -- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 msgr2=0x7f58d81469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58d77fe700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d81469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58d77fe700 1 -- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58d80cb130 msgr2=0x7f58d80c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58d77fe700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58d80cb130 0x7f58d80c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58d77fe700 1 -- 172.21.15.64:0/4037521121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009a20 con 0x7f58d815c760 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58d77fe700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d815d720 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f58d0013000 tx=0x7f58d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0027470 con 0x7f58d815c760 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0027630 con 0x7f58d815c760 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d815d160 con 0x7f58d815c760 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00279f0 con 0x7f58d815c760 2023-09-27T23:30:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.086+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d8146f10 con 0x7f58d815c760 2023-09-27T23:30:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.090+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 43) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f58d0024070 con 0x7f58d815c760 2023-09-27T23:30:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.090+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d8147130 con 0x7f58d815c760 2023-09-27T23:30:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.090+0000 7f58ceffd700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58b8060820 0x7f58b8062cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:30:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.090+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f58d007f4a0 con 0x7f58d815c760 2023-09-27T23:30:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.090+0000 7f58d6ffd700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58b8060820 0x7f58b8062cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:30:47.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.090+0000 7f58d6ffd700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58b8060820 0x7f58b8062cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58c0007fd0 tx=0x7f58c0007d30 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:30:47.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.102+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f58d004f730 con 0x7f58d815c760 2023-09-27T23:30:47.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.282+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58d80b3580 con 0x7f58b8060820 2023-09-27T23:30:47.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.298+0000 7f58ceffd700 1 -- 172.21.15.64:0/4037521121 <== mgr.7765 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7f58d80b3580 con 0x7f58b8060820 2023-09-27T23:30:47.301 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:30:47.301 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:30:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-09-27T23:30:47.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58b8060820 msgr2=0x7f58b8062cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:47.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58b8060820 0x7f58b8062cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58c0007fd0 tx=0x7f58c0007d30 comp rx=0 tx=0).stop 2023-09-27T23:30:47.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 msgr2=0x7f58d815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d815d720 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f58d0013000 tx=0x7f58d0019000 comp rx=0 tx=0).stop 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 shutdown_connections 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f58b8060820 0x7f58b8062cd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58d815c760 0x7f58d815d720 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58d80cb500 0x7f58d81469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 --2- 172.21.15.64:0/4037521121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58d80cb130 0x7f58d80c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 >> 172.21.15.64:0/4037521121 conn(0x7f58d8067bb0 msgr2=0x7f58d80b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:30:47.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 shutdown_connections 2023-09-27T23:30:47.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:30:47.302+0000 7f58dd62b700 1 -- 172.21.15.64:0/4037521121 wait complete. 2023-09-27T23:30:47.319 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:30:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-09-27T23:30:49.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:30:49.782 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:31:00.963 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:00.960+0000 7f1197d6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:00.963 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:00.960+0000 7f1197d6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:00.964 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:00.960+0000 7f1197d6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:00.964 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:00.960+0000 7f1197d6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:00.964 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:00.960+0000 7f1197d6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:19.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:31:19.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:31:47.320 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-09-27T23:31:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.441+0000 7f0d08cb7700 1 Processor -- start 2023-09-27T23:31:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.441+0000 7f0d08cb7700 1 -- start start 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d0406df10 0x7f0d0406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0406b420 con 0x7f0d0406ed80 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0415d720 con 0x7f0d04158da0 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040ca290 con 0x7f0d0406df10 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d02ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0d0406ed80 0x7f0d0406a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47732/0 (socket says 172.21.15.64:47732) 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d02ffd700 1 -- 172.21.15.64:0/1986153747 learned_addr learned my addr 172.21.15.64:0/1986153747 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d03fff700 1 --2- 172.21.15.64:0/1986153747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d0406df10 0x7f0d0406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014901628 0 0) 0x7f0d0406b420 con 0x7f0d0406ed80 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf0003570 con 0x7f0d0406ed80 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605723867 0 0) 0x7f0d0415d720 con 0x7f0d04158da0 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d0406b420 con 0x7f0d04158da0 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d03fff700 1 -- 172.21.15.64:0/1986153747 >> v1:172.21.15.123:6789/0 conn(0x7f0d04158da0 legacy=0x7f0d0406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d03fff700 1 -- 172.21.15.64:0/1986153747 >> v1:172.21.15.64:6789/0 conn(0x7f0d0406ed80 legacy=0x7f0d0406a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d03fff700 1 -- 172.21.15.64:0/1986153747 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040ca4a0 con 0x7f0d0406df10 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d03fff700 1 --2- 172.21.15.64:0/1986153747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d0406df10 0x7f0d0406aee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009910 tx=0x7f0cfc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5561847794facf38 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011630 con 0x7f0d0406df10 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc011790 con 0x7f0d0406df10 2023-09-27T23:31:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d027fc700 1 -- 172.21.15.64:0/1986153747 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011b10 con 0x7f0d0406df10 2023-09-27T23:31:47.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- 172.21.15.64:0/1986153747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d0406df10 msgr2=0x7f0d0406aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/1986153747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d0406df10 0x7f0d0406aee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009910 tx=0x7f0cfc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- 172.21.15.64:0/1986153747 shutdown_connections 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/1986153747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d0406df10 0x7f0d0406aee0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- 172.21.15.64:0/1986153747 >> 172.21.15.64:0/1986153747 conn(0x7f0d040cc760 msgr2=0x7f0d040ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- 172.21.15.64:0/1986153747 shutdown_connections 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.445+0000 7f0d08cb7700 1 -- 172.21.15.64:0/1986153747 wait complete. 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 Processor -- start 2023-09-27T23:31:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 -- start start 2023-09-27T23:31:47.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d0406df10 0x7f0d040b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 0x7f0d040b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d04158da0 0x7f0d040c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040c3560 con 0x7f0d0406ed80 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040c5f90 con 0x7f0d0406df10 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d08cb7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040c6270 con 0x7f0d04158da0 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d02ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 0x7f0d040b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d02ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 0x7f0d040b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57026/0 (socket says 172.21.15.64:57026) 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d03fff700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d04158da0 0x7f0d040c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d02ffd700 1 -- 172.21.15.64:0/3428946580 learned_addr learned my addr 172.21.15.64:0/3428946580 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d037fe700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d0406df10 0x7f0d040b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d03fff700 1 -- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d0406df10 msgr2=0x7f0d040b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d03fff700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d0406df10 0x7f0d040b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d03fff700 1 -- 172.21.15.64:0/3428946580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 msgr2=0x7f0d040b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d03fff700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 0x7f0d040b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d03fff700 1 -- 172.21.15.64:0/3428946580 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009420 con 0x7f0d04158da0 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d037fe700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d0406df10 0x7f0d040b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.449+0000 7f0d02ffd700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 0x7f0d040b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0d03fff700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d04158da0 0x7f0d040c3fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc016000 tx=0x7f0cfc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc025400 con 0x7f0d04158da0 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc025560 con 0x7f0d04158da0 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d040c3800 con 0x7f0d04158da0 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc025920 con 0x7f0d04158da0 2023-09-27T23:31:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d040b1400 con 0x7f0d04158da0 2023-09-27T23:31:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d040c3c80 con 0x7f0d04158da0 2023-09-27T23:31:47.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.453+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 45) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f0cfc0193b0 con 0x7f0d04158da0 2023-09-27T23:31:47.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.457+0000 7f0cfb7fe700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0ce4060be0 0x7f0ce4063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.457+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f0cfc07f180 con 0x7f0d04158da0 2023-09-27T23:31:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.457+0000 7f0d037fe700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0ce4060be0 0x7f0ce4063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.457+0000 7f0d037fe700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0ce4060be0 0x7f0ce4063090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000f80 tx=0x7f0cf4002680 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:47.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.469+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0cfc04ef90 con 0x7f0d04158da0 2023-09-27T23:31:47.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.657+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d0404be40 con 0x7f0ce4060be0 2023-09-27T23:31:47.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.669+0000 7f0cfb7fe700 1 -- 172.21.15.64:0/3428946580 <== mgr.7765 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f0d0404be40 con 0x7f0ce4060be0 2023-09-27T23:31:47.676 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:47.676 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:30:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0ce4060be0 msgr2=0x7f0ce4063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0ce4060be0 0x7f0ce4063090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000f80 tx=0x7f0cf4002680 comp rx=0 tx=0).stop 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d04158da0 msgr2=0x7f0d040c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d04158da0 0x7f0d040c3fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc016000 tx=0x7f0cfc017000 comp rx=0 tx=0).stop 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 shutdown_connections 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0ce4060be0 0x7f0ce4063090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d04158da0 0x7f0d040c3fe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0406ed80 0x7f0d040b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 --2- 172.21.15.64:0/3428946580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d0406df10 0x7f0d040b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 >> 172.21.15.64:0/3428946580 conn(0x7f0d040cc760 msgr2=0x7f0d0406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:47.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 shutdown_connections 2023-09-27T23:31:47.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.673+0000 7f0d08cb7700 1 -- 172.21.15.64:0/3428946580 wait complete. 2023-09-27T23:31:47.692 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:30:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-09-27T23:31:47.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-09-27T23:31:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.805+0000 7f65861c9700 1 Processor -- start 2023-09-27T23:31:47.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.805+0000 7f65861c9700 1 -- start start 2023-09-27T23:31:47.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65861c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 0x7f658006dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65861c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6580156d60 0x7f65800ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65861c9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800b0480 con 0x7f65800addf0 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65861c9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65800ae230 con 0x7f6580156d60 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65861c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658006c7c0 con 0x7f6580156990 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f65800addf0 0x7f65800c9970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47746/0 (socket says 172.21.15.64:47746) 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 -- 172.21.15.64:0/2206819619 learned_addr learned my addr 172.21.15.64:0/2206819619 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65849c6700 1 --2- 172.21.15.64:0/2206819619 >> v2:172.21.15.123:3300/0 conn(0x7f6580156d60 0x7f65800ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65851c7700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 0x7f658006dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852531651 0 0) 0x7f65800b0480 con 0x7f65800addf0 2023-09-27T23:31:47.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f656c007980 con 0x7f65800addf0 2023-09-27T23:31:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660281982 0 0) 0x7f656c007980 con 0x7f65800addf0 2023-09-27T23:31:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 msgr2=0x7f658006dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:47.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 0x7f658006dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 >> v2:172.21.15.123:3300/0 conn(0x7f6580156d60 msgr2=0x7f65800ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 --2- 172.21.15.64:0/2206819619 >> v2:172.21.15.123:3300/0 conn(0x7f6580156d60 0x7f65800ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658006d620 con 0x7f65800addf0 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f65780017f0 con 0x7f65800addf0 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 >> v1:172.21.15.64:6789/0 conn(0x7f65800addf0 legacy=0x7f65800c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 0x7f656c007e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c0083f0 0x7f656c008c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c009160 0x7f656c0095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800ae230 con 0x7f656c009160 2023-09-27T23:31:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658006c7c0 con 0x7f656c0083f0 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c00fea0 con 0x7f6580156990 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c009160 0x7f656c0095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65849c6700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c0083f0 0x7f656c008c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65851c7700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 0x7f656c007e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 -- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 msgr2=0x7f656c007e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580156990 0x7f656c007e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 -- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c0083f0 msgr2=0x7f656c008c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c0083f0 0x7f656c008c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 -- 172.21.15.64:0/2206819619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c007980 con 0x7f656c009160 2023-09-27T23:31:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65849c6700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c0083f0 0x7f656c008c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f65859c8700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c009160 0x7f656c0095d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f657800a970 tx=0x7f6578002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa64cd0ad6238f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578012440 con 0x7f656c009160 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65780125c0 con 0x7f656c009160 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.809+0000 7f6577fff700 1 -- 172.21.15.64:0/2206819619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578012980 con 0x7f656c009160 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 -- 172.21.15.64:0/2206819619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c009160 msgr2=0x7f656c0095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 --2- 172.21.15.64:0/2206819619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c009160 0x7f656c0095d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f657800a970 tx=0x7f6578002210 comp rx=0 tx=0).stop 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65859c8700 1 -- 172.21.15.64:0/2206819619 reap_dead start 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 -- 172.21.15.64:0/2206819619 shutdown_connections 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 -- 172.21.15.64:0/2206819619 >> 172.21.15.64:0/2206819619 conn(0x7f65800ce5e0 msgr2=0x7f65800ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 -- 172.21.15.64:0/2206819619 shutdown_connections 2023-09-27T23:31:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 -- 172.21.15.64:0/2206819619 wait complete. 2023-09-27T23:31:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.813+0000 7f65861c9700 1 Processor -- start 2023-09-27T23:31:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 -- start start 2023-09-27T23:31:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 0x7f6580155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6580069e90 0x7f65800b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580067ac0 0x7f658014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800c6590 con 0x7f6580155640 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800b0e00 con 0x7f6580069e90 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65800c5c60 con 0x7f6580067ac0 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 0x7f6580155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 0x7f6580155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57058/0 (socket says 172.21.15.64:57058) 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 -- 172.21.15.64:0/1947199959 learned_addr learned my addr 172.21.15.64:0/1947199959 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65859c8700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580067ac0 0x7f658014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65849c6700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6580069e90 0x7f65800b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 -- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580067ac0 msgr2=0x7f658014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580067ac0 0x7f658014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 -- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6580069e90 msgr2=0x7f65800b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6580069e90 0x7f65800b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 -- 172.21.15.64:0/1947199959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65800c5f40 con 0x7f6580155640 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65851c7700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 0x7f6580155a10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f657c011960 tx=0x7f657c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c015670 con 0x7f6580155640 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c0157f0 con 0x7f6580155640 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65800c6220 con 0x7f6580155640 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c015bb0 con 0x7f6580155640 2023-09-27T23:31:47.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.817+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580067f10 con 0x7f6580155640 2023-09-27T23:31:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.821+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 45) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f657c013410 con 0x7f6580155640 2023-09-27T23:31:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.821+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658004be40 con 0x7f6580155640 2023-09-27T23:31:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.821+0000 7f65757fa700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6564060be0 0x7f6564063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.821+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f657c07d900 con 0x7f6580155640 2023-09-27T23:31:47.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.821+0000 7f65849c6700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6564060be0 0x7f6564063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:47.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.825+0000 7f65849c6700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6564060be0 0x7f6564063090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6570009170 tx=0x7f6570008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:47.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:47.829+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f657c04d5b0 con 0x7f6580155640 2023-09-27T23:31:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.013+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f65800c66d0 con 0x7f6564060be0 2023-09-27T23:31:48.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.133+0000 7f65757fa700 1 -- 172.21.15.64:0/1947199959 <== mgr.7765 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f6570009a50 con 0x7f6564060be0 2023-09-27T23:31:48.139 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /bad-path 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6564060be0 msgr2=0x7f6564063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6564060be0 0x7f6564063090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6570009170 tx=0x7f6570008040 comp rx=0 tx=0).stop 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 msgr2=0x7f6580155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 0x7f6580155a10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f657c011960 tx=0x7f657c008040 comp rx=0 tx=0).stop 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 shutdown_connections 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6564060be0 0x7f6564063090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6580067ac0 0x7f658014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6580069e90 0x7f65800b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 --2- 172.21.15.64:0/1947199959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6580155640 0x7f6580155a10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 >> 172.21.15.64:0/1947199959 conn(0x7f65800ce5e0 msgr2=0x7f65801539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 shutdown_connections 2023-09-27T23:31:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.137+0000 7f65861c9700 1 -- 172.21.15.64:0/1947199959 wait complete. 2023-09-27T23:31:48.160 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-09-27T23:31:48.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:31:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.273+0000 7f9e19a8d700 1 Processor -- start 2023-09-27T23:31:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.273+0000 7f9e19a8d700 1 -- start start 2023-09-27T23:31:48.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e140c69a0 0x7f9e140c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406de00 0x7f9e1406b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1406d970 con 0x7f9e1406c7c0 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1406cd60 con 0x7f9e140c69a0 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1406d040 con 0x7f9e1406de00 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e140c69a0 0x7f9e140c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e140c69a0 0x7f9e140c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57074/0 (socket says 172.21.15.64:57074) 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 -- 172.21.15.64:0/1037434380 learned_addr learned my addr 172.21.15.64:0/1037434380 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:48.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e13fff700 1 --2- 172.21.15.64:0/1037434380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406de00 0x7f9e1406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 -- 172.21.15.64:0/1037434380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406de00 msgr2=0x7f9e1406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 --2- 172.21.15.64:0/1037434380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406de00 0x7f9e1406b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 -- 172.21.15.64:0/1037434380 >> v1:172.21.15.123:6789/0 conn(0x7f9e1406c7c0 legacy=0x7f9e1415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 -- 172.21.15.64:0/1037434380 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1406d320 con 0x7f9e140c69a0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e137fe700 1 -- 172.21.15.64:0/1037434380 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659296636 0 0) 0x7f9e1406d970 con 0x7f9e1406c7c0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e18a8b700 1 --2- 172.21.15.64:0/1037434380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e140c6d70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9e040099e0 tx=0x7f9e04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e795433ed2922b3f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e137fe700 1 -- 172.21.15.64:0/1037434380 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400b820 con 0x7f9e140c69a0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e137fe700 1 -- 172.21.15.64:0/1037434380 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0400b980 con 0x7f9e140c69a0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e137fe700 1 -- 172.21.15.64:0/1037434380 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400bd20 con 0x7f9e140c69a0 2023-09-27T23:31:48.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- 172.21.15.64:0/1037434380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 msgr2=0x7f9e140c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/1037434380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e140c6d70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9e040099e0 tx=0x7f9e04009250 comp rx=0 tx=0).stop 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- 172.21.15.64:0/1037434380 shutdown_connections 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/1037434380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406de00 0x7f9e1406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/1037434380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e140c6d70 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- 172.21.15.64:0/1037434380 >> 172.21.15.64:0/1037434380 conn(0x7f9e14067b50 msgr2=0x7f9e140cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- 172.21.15.64:0/1037434380 shutdown_connections 2023-09-27T23:31:48.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.277+0000 7f9e19a8d700 1 -- 172.21.15.64:0/1037434380 wait complete. 2023-09-27T23:31:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 Processor -- start 2023-09-27T23:31:48.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 -- start start 2023-09-27T23:31:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406c7c0 0x7f9e1416d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1406de00 0x7f9e1414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e1415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140afa50 con 0x7f9e140c69a0 2023-09-27T23:31:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1414c190 con 0x7f9e1406de00 2023-09-27T23:31:48.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e19a8d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e140afcf0 con 0x7f9e1406c7c0 2023-09-27T23:31:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e1415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e13fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1406de00 0x7f9e1414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e18a8b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406c7c0 0x7f9e1416d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e13fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1406de00 0x7f9e1414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:58880/0 (socket says 172.21.15.64:58880) 2023-09-27T23:31:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e1415aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57076/0 (socket says 172.21.15.64:57076) 2023-09-27T23:31:48.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e13fff700 1 -- 172.21.15.64:0/2433604382 learned_addr learned my addr 172.21.15.64:0/2433604382 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 -- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406c7c0 msgr2=0x7f9e1416d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406c7c0 0x7f9e1416d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 -- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1406de00 msgr2=0x7f9e1414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1406de00 0x7f9e1414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:48.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.281+0000 7f9e1928c700 1 -- 172.21.15.64:0/2433604382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040094f0 con 0x7f9e140c69a0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e1928c700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e1415aef0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c001560 tx=0x7f9e0c0016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c012630 con 0x7f9e140c69a0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c012790 con 0x7f9e140c69a0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e140affd0 con 0x7f9e140c69a0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c012b50 con 0x7f9e140c69a0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1415b9f0 con 0x7f9e140c69a0 2023-09-27T23:31:48.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e140b04b0 con 0x7f9e140c69a0 2023-09-27T23:31:48.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.285+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 45) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f9e0c003030 con 0x7f9e140c69a0 2023-09-27T23:31:48.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.289+0000 7f9e10ff9700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9df4060be0 0x7f9df4063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:48.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.289+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f9e0c07d550 con 0x7f9e140c69a0 2023-09-27T23:31:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.289+0000 7f9e18a8b700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9df4060be0 0x7f9df4063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:48.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.289+0000 7f9e18a8b700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9df4060be0 0x7f9df4063090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e040099e0 tx=0x7f9e04017000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:48.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.293+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9e0c04d2a0 con 0x7f9e140c69a0 2023-09-27T23:31:48.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:48.501+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f9e1404be40 con 0x7f9e140c69a0 2023-09-27T23:31:49.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.353+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f9e0c04d670 con 0x7f9e140c69a0 2023-09-27T23:31:49.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.357+0000 7f9e10ff9700 1 -- 172.21.15.64:0/2433604382 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v46) v1 ==== 93+0+0 (secure 0 0 0) 0x7f9e0c050d30 con 0x7f9e140c69a0 2023-09-27T23:31:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.357+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9df4060be0 msgr2=0x7f9df4063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.357+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9df4060be0 0x7f9df4063090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e040099e0 tx=0x7f9e04017000 comp rx=0 tx=0).stop 2023-09-27T23:31:49.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.357+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 msgr2=0x7f9e1415aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.357+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e1415aef0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c001560 tx=0x7f9e0c0016c0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.357+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 shutdown_connections 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9df4060be0 0x7f9df4063090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e140c69a0 0x7f9e1415aef0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1406de00 0x7f9e1414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 --2- 172.21.15.64:0/2433604382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e1406c7c0 0x7f9e1416d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 >> 172.21.15.64:0/2433604382 conn(0x7f9e14067b50 msgr2=0x7f9e1415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 shutdown_connections 2023-09-27T23:31:49.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.361+0000 7f9e19a8d700 1 -- 172.21.15.64:0/2433604382 wait complete. 2023-09-27T23:31:49.378 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:31:49.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:31:49.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:31:49.407 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:31:49.407 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:31:49.407 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:31:49.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:31:49.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[52990]: fuse finished with error 0 and tester_r 0 2023-09-27T23:31:49.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:31:49.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:31:49.444 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:31:49.477 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:49.475+0000 7f7443bfd000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:31:49.486 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:49.481+0000 7fb54387e000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:31:49.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:31:49.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:31:49.555 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:31:49.562 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:31:49.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:31:49.563 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:31:49.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:31:49.563 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:31:49.601 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:31:49.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:31:49.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:31:49.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:31:49.603 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:31:49.613 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:31:49.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:31:49.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:31:49.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe209355700 1 Processor -- start 2023-09-27T23:31:49.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe209355700 1 -- start start 2023-09-27T23:31:49.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe209355700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe204156770 0x7fe204156b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:49.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe209355700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2040ca8c0 con 0x7fe2040c8b50 2023-09-27T23:31:49.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe209355700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2040cb700 con 0x7fe20406b480 2023-09-27T23:31:49.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe209355700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20406bd20 con 0x7fe204156770 2023-09-27T23:31:49.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe2037fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe2040c8b50 0x7fe20406ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47754/0 (socket says 172.21.15.64:47754) 2023-09-27T23:31:49.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe2037fe700 1 -- 172.21.15.64:0/1099508506 learned_addr learned my addr 172.21.15.64:0/1099508506 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe203fff700 1 --2- 172.21.15.64:0/1099508506 >> v2:172.21.15.123:3300/0 conn(0x7fe204156770 0x7fe204156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe202ffd700 1 -- 172.21.15.64:0/1099508506 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511981415 0 0) 0x7fe2040ca8c0 con 0x7fe2040c8b50 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe202ffd700 1 -- 172.21.15.64:0/1099508506 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ec003570 con 0x7fe2040c8b50 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe203fff700 1 -- 172.21.15.64:0/1099508506 >> v1:172.21.15.123:6789/0 conn(0x7fe20406b480 legacy=0x7fe2040af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe203fff700 1 -- 172.21.15.64:0/1099508506 >> v1:172.21.15.64:6789/0 conn(0x7fe2040c8b50 legacy=0x7fe20406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe202ffd700 1 -- 172.21.15.64:0/1099508506 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 226572932 0 0) 0x7fe1ec003570 con 0x7fe2040c8b50 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.717+0000 7fe203fff700 1 -- 172.21.15.64:0/1099508506 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040caaf0 con 0x7fe204156770 2023-09-27T23:31:49.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe203fff700 1 --2- 172.21.15.64:0/1099508506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe204156770 0x7fe204156b40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009910 tx=0x7fe1f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25c69c0b8d6c6e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:49.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe202ffd700 1 -- 172.21.15.64:0/1099508506 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8001470 con 0x7fe204156770 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe202ffd700 1 -- 172.21.15.64:0/1099508506 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4011420 con 0x7fe204156770 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe202ffd700 1 -- 172.21.15.64:0/1099508506 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40117c0 con 0x7fe204156770 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 -- 172.21.15.64:0/1099508506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe204156770 msgr2=0x7fe204156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 --2- 172.21.15.64:0/1099508506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe204156770 0x7fe204156b40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009910 tx=0x7fe1f40091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 -- 172.21.15.64:0/1099508506 shutdown_connections 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 --2- 172.21.15.64:0/1099508506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe204156770 0x7fe204156b40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 -- 172.21.15.64:0/1099508506 >> 172.21.15.64:0/1099508506 conn(0x7fe204067bb0 msgr2=0x7fe2040add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 -- 172.21.15.64:0/1099508506 shutdown_connections 2023-09-27T23:31:49.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 -- 172.21.15.64:0/1099508506 wait complete. 2023-09-27T23:31:49.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 Processor -- start 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 -- start start 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20406b480 0x7fe204138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.721+0000 7fe209355700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2040c8b50 0x7fe20415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 0x7fe204138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20414fed0 con 0x7fe204156770 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204138ad0 con 0x7fe2040c8b50 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20414f2c0 con 0x7fe20406b480 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 0x7fe204138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:49.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 0x7fe204138dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57094/0 (socket says 172.21.15.64:57094) 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 -- 172.21.15.64:0/759855134 learned_addr learned my addr 172.21.15.64:0/759855134 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe2037fe700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2040c8b50 0x7fe20415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe203fff700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20406b480 0x7fe204138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 -- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20406b480 msgr2=0x7fe204138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20406b480 0x7fe204138590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 -- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2040c8b50 msgr2=0x7fe20415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2040c8b50 0x7fe20415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 -- 172.21.15.64:0/759855134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009420 con 0x7fe204156770 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe208b54700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 0x7fe204138dc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c840 tx=0x7fe1fc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc012630 con 0x7fe204156770 2023-09-27T23:31:49.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc0127b0 con 0x7fe204156770 2023-09-27T23:31:49.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc012b70 con 0x7fe204156770 2023-09-27T23:31:49.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20414f5a0 con 0x7fe204156770 2023-09-27T23:31:49.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20414fb60 con 0x7fe204156770 2023-09-27T23:31:49.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.725+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe204150080 con 0x7fe204156770 2023-09-27T23:31:49.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.729+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fe1fc024420 con 0x7fe204156770 2023-09-27T23:31:49.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.729+0000 7fe1ebfff700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 0x7fe1f0063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:49.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.729+0000 7fe203fff700 1 -- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 msgr2=0x7fe1f0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:49.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.729+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fe1fc07dbf0 con 0x7fe204156770 2023-09-27T23:31:49.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.729+0000 7fe203fff700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 0x7fe1f0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:49.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.733+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe1fc04d920 con 0x7fe204156770 2023-09-27T23:31:49.898 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:49.895+0000 7f7443bfd000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:31:49.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.929+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe20404be40 con 0x7fe204156770 2023-09-27T23:31:49.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.929+0000 7fe1ebfff700 1 -- 172.21.15.64:0/759855134 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fe1fc050f40 con 0x7fe204156770 2023-09-27T23:31:49.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:49.936 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:30:29.391491+0000","modified":"2023-09-27T23:30:31.410091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7605":{"gid":7605,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1829030850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1829030850},{"type":"v1","addr":"172.21.15.64:6835","nonce":1829030850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-09-27T23:31:49.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe203fff700 1 -- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 msgr2=0x7fe1f0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:49.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe203fff700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 0x7fe1f0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:31:49.937 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fb54387e000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:31:49.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 msgr2=0x7fe1f0063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:49.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 0x7fe1f0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 msgr2=0x7fe204138dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 0x7fe204138dc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c840 tx=0x7fe1fc001670 comp rx=0 tx=0).stop 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 shutdown_connections 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe1f0060b90 0x7fe1f0063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204156770 0x7fe204138dc0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2040c8b50 0x7fe20415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 --2- 172.21.15.64:0/759855134 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe20406b480 0x7fe204138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 >> 172.21.15.64:0/759855134 conn(0x7fe204067bb0 msgr2=0x7fe2040c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 shutdown_connections 2023-09-27T23:31:49.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:49.933+0000 7fe209355700 1 -- 172.21.15.64:0/759855134 wait complete. 2023-09-27T23:31:49.940 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2023-09-27T23:31:49.946 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:49.943+0000 7f7443bfd000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:31:49.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:31:49.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:31:49.989 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:31:49.990 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:49.985+0000 7fb54387e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:31:50.053 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.051+0000 7f7443bfd000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:31:50.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 Processor -- start 2023-09-27T23:31:50.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 -- start start 2023-09-27T23:31:50.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2c406e1d0 0x7fa2c4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2c406b210 0x7fa2c414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c415a260 con 0x7fa2c406ad70 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40afad0 con 0x7fa2c406b210 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40afdb0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2c406b210 0x7fa2c414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2c406b210 0x7fa2c414fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57100/0 (socket says 172.21.15.64:57100) 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 -- 172.21.15.64:0/1376464821 learned_addr learned my addr 172.21.15.64:0/1376464821 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2c9a17700 1 --2- 172.21.15.64:0/1376464821 >> v2:172.21.15.123:3300/0 conn(0x7fa2c406e1d0 0x7fa2c4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 -- 172.21.15.64:0/1376464821 >> v2:172.21.15.123:3300/0 conn(0x7fa2c406e1d0 msgr2=0x7fa2c4159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 --2- 172.21.15.64:0/1376464821 >> v2:172.21.15.123:3300/0 conn(0x7fa2c406e1d0 0x7fa2c4159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 -- 172.21.15.64:0/1376464821 >> v1:172.21.15.123:6789/0 conn(0x7fa2c406ad70 legacy=0x7fa2c414f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 -- 172.21.15.64:0/1376464821 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40b0090 con 0x7fa2c406b210 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2ca218700 1 --2- 172.21.15.64:0/1376464821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406b210 0x7fa2c414fbe0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40cd260 tx=0x7fa2bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f5d48833127e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2c8a15700 1 -- 172.21.15.64:0/1376464821 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00e070 con 0x7fa2c406b210 2023-09-27T23:31:50.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2c8a15700 1 -- 172.21.15.64:0/1376464821 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00ba20 con 0x7fa2c406b210 2023-09-27T23:31:50.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2c8a15700 1 -- 172.21.15.64:0/1376464821 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0092e0 con 0x7fa2c406b210 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 -- 172.21.15.64:0/1376464821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406b210 msgr2=0x7fa2c414fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.049+0000 7fa2caa19700 1 --2- 172.21.15.64:0/1376464821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406b210 0x7fa2c414fbe0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40cd260 tx=0x7fa2bc009840 comp rx=0 tx=0).stop 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- 172.21.15.64:0/1376464821 shutdown_connections 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 --2- 172.21.15.64:0/1376464821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406b210 0x7fa2c414fbe0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 --2- 172.21.15.64:0/1376464821 >> v2:172.21.15.123:3300/0 conn(0x7fa2c406e1d0 0x7fa2c4159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- 172.21.15.64:0/1376464821 >> 172.21.15.64:0/1376464821 conn(0x7fa2c4067bb0 msgr2=0x7fa2c40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- 172.21.15.64:0/1376464821 shutdown_connections 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- 172.21.15.64:0/1376464821 wait complete. 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 Processor -- start 2023-09-27T23:31:50.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- start start 2023-09-27T23:31:50.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2c406ad70 0x7fa2c40aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c406b210 0x7fa2c4159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 0x7fa2c40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41543f0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40ab160 con 0x7fa2c406b210 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2caa19700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40aa8c0 con 0x7fa2c406ad70 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 0x7fa2c40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 0x7fa2c40cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57104/0 (socket says 172.21.15.64:57104) 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 -- 172.21.15.64:0/2111747870 learned_addr learned my addr 172.21.15.64:0/2111747870 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2c9a17700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2c406ad70 0x7fa2c40aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2c9216700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c406b210 0x7fa2c4159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 -- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2c406ad70 msgr2=0x7fa2c40aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2c406ad70 0x7fa2c40aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 -- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c406b210 msgr2=0x7fa2c4159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c406b210 0x7fa2c4159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 -- 172.21.15.64:0/2111747870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc009ae0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2c9216700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c406b210 0x7fa2c4159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.053+0000 7fa2ca218700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 0x7fa2c40cb7d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0072f0 tx=0x7fa2bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc008bb0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc008d10 con 0x7fa2c406e1d0 2023-09-27T23:31:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40aaba0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0266b0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40cbd10 con 0x7fa2c406e1d0 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c41545a0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fa2bc003040 con 0x7fa2c406e1d0 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2b9ffb700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa2a8060bc0 0x7fa2a8063070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2c9a17700 1 -- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa2a8060bc0 msgr2=0x7fa2a8063070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa2bc07f4b0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.057+0000 7fa2c9a17700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa2a8060bc0 0x7fa2a8063070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:50.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.065+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa2c41545a0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.123 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.117+0000 7fb54387e000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:31:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.225+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2c404be40 con 0x7fa2c406e1d0 2023-09-27T23:31:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.225+0000 7fa2b9ffb700 1 -- 172.21.15.64:0/2111747870 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+17582 (secure 0 0 0) 0x7fa2bc04f0d0 con 0x7fa2c406e1d0 2023-09-27T23:31:50.231 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:50.231 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":116,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:45.799145+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:30:27.964715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-27T23:30:28.361565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:30:45.799008+0000","172.21.15.123:6832/33228":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/3430876092":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/3314489464":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/926402627":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1057705795":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1993249321":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/965278245":"2023-09-28T23:30:45.799008+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:31:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa2a8060bc0 msgr2=0x7fa2a8063070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa2a8060bc0 0x7fa2a8063070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 msgr2=0x7fa2c40cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 0x7fa2c40cb7d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0072f0 tx=0x7fa2bc018000 comp rx=0 tx=0).stop 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 shutdown_connections 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa2a8060bc0 0x7fa2a8063070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c406e1d0 0x7fa2c40cb7d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c406b210 0x7fa2c4159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 --2- 172.21.15.64:0/2111747870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2c406ad70 0x7fa2c40aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 >> 172.21.15.64:0/2111747870 conn(0x7fa2c4067bb0 msgr2=0x7fa2c40ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 shutdown_connections 2023-09-27T23:31:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.229+0000 7fa2caa19700 1 -- 172.21.15.64:0/2111747870 wait complete. 2023-09-27T23:31:50.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:30:27.964715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:30:28.361565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:31:50.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:31:50.248 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.247+0000 7f7443bfd000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:31:50.296 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.295+0000 7f7443bfd000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:31:50.344 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.337+0000 7fb54387e000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:31:50.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.389+0000 7f8af2a75700 1 Processor -- start 2023-09-27T23:31:50.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.389+0000 7f8af2a75700 1 -- start start 2023-09-27T23:31:50.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8aec0ae1c0 0x7f8aec0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8aec15cc00 0x7f8aec15d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec15d560 con 0x7f8aec0addf0 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0ae610 con 0x7f8aec15cc00 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0ac6b0 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8aec15cc00 0x7f8aec15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8aec15cc00 0x7f8aec15d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57106/0 (socket says 172.21.15.64:57106) 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 -- 172.21.15.64:0/3098493848 learned_addr learned my addr 172.21.15.64:0/3098493848 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af1272700 1 --2- 172.21.15.64:0/3098493848 >> v2:172.21.15.123:3300/0 conn(0x7f8aec0ae1c0 0x7f8aec0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 -- 172.21.15.64:0/3098493848 >> v2:172.21.15.123:3300/0 conn(0x7f8aec0ae1c0 msgr2=0x7f8aec0ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 --2- 172.21.15.64:0/3098493848 >> v2:172.21.15.123:3300/0 conn(0x7f8aec0ae1c0 0x7f8aec0ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 -- 172.21.15.64:0/3098493848 >> v1:172.21.15.123:6789/0 conn(0x7f8aec0addf0 legacy=0x7f8aec06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 -- 172.21.15.64:0/3098493848 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec0c68a0 con 0x7f8aec15cc00 2023-09-27T23:31:50.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2274700 1 --2- 172.21.15.64:0/3098493848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec15cc00 0x7f8aec15d020 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0c8ca0 tx=0x7f8ae4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bab0bc7c67a86e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af0a71700 1 -- 172.21.15.64:0/3098493848 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400e070 con 0x7f8aec15cc00 2023-09-27T23:31:50.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af0a71700 1 -- 172.21.15.64:0/3098493848 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae400ba50 con 0x7f8aec15cc00 2023-09-27T23:31:50.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af0a71700 1 -- 172.21.15.64:0/3098493848 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40092e0 con 0x7f8aec15cc00 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- 172.21.15.64:0/3098493848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec15cc00 msgr2=0x7f8aec15d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 --2- 172.21.15.64:0/3098493848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec15cc00 0x7f8aec15d020 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0c8ca0 tx=0x7f8ae4009840 comp rx=0 tx=0).stop 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- 172.21.15.64:0/3098493848 shutdown_connections 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 --2- 172.21.15.64:0/3098493848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec15cc00 0x7f8aec15d020 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 --2- 172.21.15.64:0/3098493848 >> v2:172.21.15.123:3300/0 conn(0x7f8aec0ae1c0 0x7f8aec0ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- 172.21.15.64:0/3098493848 >> 172.21.15.64:0/3098493848 conn(0x7f8aec0c8cd0 msgr2=0x7f8aec0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- 172.21.15.64:0/3098493848 shutdown_connections 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- 172.21.15.64:0/3098493848 wait complete. 2023-09-27T23:31:50.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 Processor -- start 2023-09-27T23:31:50.401 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.393+0000 7fb54387e000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:31:50.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.393+0000 7f8af2a75700 1 -- start start 2023-09-27T23:31:50.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec0addf0 0x7f8aec157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 0x7f8aec15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aec15cc00 0x7f8aec15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0ab050 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0aa440 con 0x7f8aec0addf0 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0aa720 con 0x7f8aec15cc00 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 0x7f8aec15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 0x7f8aec15bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57118/0 (socket says 172.21.15.64:57118) 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2274700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aec15cc00 0x7f8aec15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1a73700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec0addf0 0x7f8aec157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 -- 172.21.15.64:0/2250844304 learned_addr learned my addr 172.21.15.64:0/2250844304 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 -- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aec15cc00 msgr2=0x7f8aec15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aec15cc00 0x7f8aec15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 -- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec0addf0 msgr2=0x7f8aec157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec0addf0 0x7f8aec157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 -- 172.21.15.64:0/2250844304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4009ae0 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af1272700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 0x7f8aec15bc70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00d890 tx=0x7f8adc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0125f0 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc012770 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc012b30 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec0aaa00 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.397+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec0ab4e0 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.401+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec04be40 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.401+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f8adc01e460 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.401+0000 7f8ae1ffb700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ad0060b90 0x7f8ad0063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.401+0000 7f8af1a73700 1 -- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ad0060b90 msgr2=0x7f8ad0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:50.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.401+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f8adc07da10 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.401+0000 7f8af1a73700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ad0060b90 0x7f8ad0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:50.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.413+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8adc04d760 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.450 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.447+0000 7f7443bfd000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:31:50.507 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.503+0000 7f7443bfd000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:31:50.559 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.553+0000 7fb54387e000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:31:50.565 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.563+0000 7f7443bfd000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:31:50.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8aec0c6b80 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8ae1ffb700 1 -- 172.21.15.64:0/2250844304 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8adc050d80 con 0x7f8aec0ae1c0 2023-09-27T23:31:50.573 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:31:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ad0060b90 msgr2=0x7f8ad0063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ad0060b90 0x7f8ad0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 msgr2=0x7f8aec15bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 0x7f8aec15bc70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00d890 tx=0x7f8adc008040 comp rx=0 tx=0).stop 2023-09-27T23:31:50.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 shutdown_connections 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8ad0060b90 0x7f8ad0063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aec15cc00 0x7f8aec15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8aec0ae1c0 0x7f8aec15bc70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 --2- 172.21.15.64:0/2250844304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec0addf0 0x7f8aec157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.569+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 >> 172.21.15.64:0/2250844304 conn(0x7f8aec0c8cd0 msgr2=0x7f8aec158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.573+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 shutdown_connections 2023-09-27T23:31:50.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.573+0000 7f8af2a75700 1 -- 172.21.15.64:0/2250844304 wait complete. 2023-09-27T23:31:50.587 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:31:50.617 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.613+0000 7fb54387e000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:31:50.680 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.677+0000 7fb54387e000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:31:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.685+0000 7fedc6ae3700 1 Processor -- start 2023-09-27T23:31:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.685+0000 7fedc6ae3700 1 -- start start 2023-09-27T23:31:50.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fedc006db70 0x7fedc00ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00c68a0 con 0x7fedc0155e60 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00c6e60 con 0x7fedc006db70 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00ae4d0 con 0x7fedc0155a90 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc52e0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fedc006db70 0x7fedc00ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc62e2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fedc0155e60 0x7fedc00adca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47764/0 (socket says 172.21.15.64:47764) 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc62e2700 1 -- 172.21.15.64:0/686780436 learned_addr learned my addr 172.21.15.64:0/686780436 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc5ae1700 1 --2- 172.21.15.64:0/686780436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc4adf700 1 -- 172.21.15.64:0/686780436 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329652027 0 0) 0x7fedc00c68a0 con 0x7fedc0155e60 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc4adf700 1 -- 172.21.15.64:0/686780436 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003570 con 0x7fedc0155e60 2023-09-27T23:31:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc52e0700 1 -- 172.21.15.64:0/686780436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 msgr2=0x7fedc00ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc52e0700 1 --2- 172.21.15.64:0/686780436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc00ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc52e0700 1 -- 172.21.15.64:0/686780436 >> v1:172.21.15.64:6789/0 conn(0x7fedc0155e60 legacy=0x7fedc00adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc52e0700 1 -- 172.21.15.64:0/686780436 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00ae7b0 con 0x7fedc006db70 2023-09-27T23:31:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc52e0700 1 --2- 172.21.15.64:0/686780436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc00ad760 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fedb00098e0 tx=0x7fedb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9eca68877a5ab7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc4adf700 1 -- 172.21.15.64:0/686780436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0011630 con 0x7fedc006db70 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc4adf700 1 -- 172.21.15.64:0/686780436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0011790 con 0x7fedc006db70 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc4adf700 1 -- 172.21.15.64:0/686780436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0011b10 con 0x7fedc006db70 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- 172.21.15.64:0/686780436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 msgr2=0x7fedc00ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/686780436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc00ad760 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fedb00098e0 tx=0x7fedb00091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- 172.21.15.64:0/686780436 shutdown_connections 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/686780436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc00ad760 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/686780436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc00ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- 172.21.15.64:0/686780436 >> 172.21.15.64:0/686780436 conn(0x7fedc00c8cd0 msgr2=0x7fedc00c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- 172.21.15.64:0/686780436 shutdown_connections 2023-09-27T23:31:50.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.689+0000 7fedc6ae3700 1 -- 172.21.15.64:0/686780436 wait complete. 2023-09-27T23:31:50.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 Processor -- start 2023-09-27T23:31:50.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 -- start start 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc0153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc0166670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0155e60 0x7fedc00aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b0fb0 con 0x7fedc006db70 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b1290 con 0x7fedc0155e60 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc6ae3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b1570 con 0x7fedc0155a90 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc0153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc0153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57140/0 (socket says 172.21.15.64:57140) 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 -- 172.21.15.64:0/4005959109 learned_addr learned my addr 172.21.15.64:0/4005959109 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc52e0700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc0166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc62e2700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0155e60 0x7fedc00aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 -- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 msgr2=0x7fedc0166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc0166670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 -- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0155e60 msgr2=0x7fedc00aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0155e60 0x7fedc00aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 -- 172.21.15.64:0/4005959109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0009420 con 0x7fedc006db70 2023-09-27T23:31:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.693+0000 7fedc5ae1700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc0153830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00ec80 tx=0x7fedbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00c9d0 con 0x7fedc006db70 2023-09-27T23:31:50.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc00cb30 con 0x7fedc006db70 2023-09-27T23:31:50.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc013470 con 0x7fedc006db70 2023-09-27T23:31:50.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc00b1850 con 0x7fedc006db70 2023-09-27T23:31:50.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc00b0ad0 con 0x7fedc006db70 2023-09-27T23:31:50.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fedbc013820 con 0x7fedc006db70 2023-09-27T23:31:50.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc00b0d80 con 0x7fedc006db70 2023-09-27T23:31:50.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedb5ffb700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feda4060b90 0x7feda4063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fedbc07d8b0 con 0x7fedc006db70 2023-09-27T23:31:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedc52e0700 1 -- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feda4060b90 msgr2=0x7feda4063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:50.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.697+0000 7fedc52e0700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feda4060b90 0x7feda4063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.705+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fedbc04d600 con 0x7fedc006db70 2023-09-27T23:31:50.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fedc0166bb0 con 0x7fedc006db70 2023-09-27T23:31:50.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedb5ffb700 1 -- 172.21.15.64:0/4005959109 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v116) v1 ==== 92+0+10 (secure 0 0 0) 0x7fedbc050c20 con 0x7fedc006db70 2023-09-27T23:31:50.861 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:31:50.862 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.859+0000 7f7443bfd000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:31:50.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feda4060b90 msgr2=0x7feda4063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:50.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feda4060b90 0x7feda4063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 msgr2=0x7fedc0153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc0153830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00ec80 tx=0x7fedbc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 shutdown_connections 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feda4060b90 0x7feda4063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc0155e60 0x7fedc00aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fedc0155a90 0x7fedc0166670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 --2- 172.21.15.64:0/4005959109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc006db70 0x7fedc0153830 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 >> 172.21.15.64:0/4005959109 conn(0x7fedc00c8cd0 msgr2=0x7fedc00c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 shutdown_connections 2023-09-27T23:31:50.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.857+0000 7fedc6ae3700 1 -- 172.21.15.64:0/4005959109 wait complete. 2023-09-27T23:31:50.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:31:50.974 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:50.971+0000 7f7443bfd000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:31:50.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 Processor -- start 2023-09-27T23:31:50.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 -- start start 2023-09-27T23:31:50.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155a90 0x7fce580ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce58155e60 0x7fce580ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce580c68a0 con 0x7fce5806db70 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580c6e60 con 0x7fce58155e60 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce57fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580ae4d0 con 0x7fce58155a90 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce567fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fce5806db70 0x7fce580ad760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47778/0 (socket says 172.21.15.64:47778) 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce567fc700 1 -- 172.21.15.64:0/1313074385 learned_addr learned my addr 172.21.15.64:0/1313074385 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce577fe700 1 --2- 172.21.15.64:0/1313074385 >> v2:172.21.15.64:3300/0 conn(0x7fce58155e60 0x7fce580ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce56ffd700 1 --2- 172.21.15.64:0/1313074385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155a90 0x7fce580ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce55ffb700 1 -- 172.21.15.64:0/1313074385 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114007265 0 0) 0x7fce580c68a0 con 0x7fce5806db70 2023-09-27T23:31:50.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce55ffb700 1 -- 172.21.15.64:0/1313074385 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce44003570 con 0x7fce5806db70 2023-09-27T23:31:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.973+0000 7fce55ffb700 1 -- 172.21.15.64:0/1313074385 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574332900 0 0) 0x7fce44003570 con 0x7fce5806db70 2023-09-27T23:31:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce56ffd700 1 -- 172.21.15.64:0/1313074385 >> v2:172.21.15.64:3300/0 conn(0x7fce58155e60 msgr2=0x7fce580ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce56ffd700 1 --2- 172.21.15.64:0/1313074385 >> v2:172.21.15.64:3300/0 conn(0x7fce58155e60 0x7fce580ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce56ffd700 1 -- 172.21.15.64:0/1313074385 >> v1:172.21.15.64:6789/0 conn(0x7fce5806db70 legacy=0x7fce580ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce56ffd700 1 -- 172.21.15.64:0/1313074385 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580ae7b0 con 0x7fce58155a90 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce56ffd700 1 --2- 172.21.15.64:0/1313074385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155a90 0x7fce580ad220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce480098e0 tx=0x7fce480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605be422e050ce8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce55ffb700 1 -- 172.21.15.64:0/1313074385 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40001470 con 0x7fce58155a90 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce55ffb700 1 -- 172.21.15.64:0/1313074385 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48011420 con 0x7fce58155a90 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce55ffb700 1 -- 172.21.15.64:0/1313074385 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480117a0 con 0x7fce58155a90 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 -- 172.21.15.64:0/1313074385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155a90 msgr2=0x7fce580ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 --2- 172.21.15.64:0/1313074385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155a90 0x7fce580ad220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce480098e0 tx=0x7fce480091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 -- 172.21.15.64:0/1313074385 shutdown_connections 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 --2- 172.21.15.64:0/1313074385 >> v2:172.21.15.64:3300/0 conn(0x7fce58155e60 0x7fce580ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 --2- 172.21.15.64:0/1313074385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155a90 0x7fce580ad220 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 -- 172.21.15.64:0/1313074385 >> 172.21.15.64:0/1313074385 conn(0x7fce580c8cd0 msgr2=0x7fce580c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 -- 172.21.15.64:0/1313074385 shutdown_connections 2023-09-27T23:31:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 -- 172.21.15.64:0/1313074385 wait complete. 2023-09-27T23:31:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 Processor -- start 2023-09-27T23:31:50.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.977+0000 7fce57fff700 1 -- start start 2023-09-27T23:31:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce5806db70 0x7fce5814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 0x7fce580cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155e60 0x7fce5814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5814da50 con 0x7fce58155a90 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5814cba0 con 0x7fce5806db70 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5814ce40 con 0x7fce58155e60 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 0x7fce580cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce56ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce5806db70 0x7fce5814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce577fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155e60 0x7fce5814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 0x7fce580cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57158/0 (socket says 172.21.15.64:57158) 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce577fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155e60 0x7fce5814b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:57272/0 (socket says 172.21.15.64:57272) 2023-09-27T23:31:50.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 -- 172.21.15.64:0/451697789 learned_addr learned my addr 172.21.15.64:0/451697789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 -- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155e60 msgr2=0x7fce5814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155e60 0x7fce5814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 -- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce5806db70 msgr2=0x7fce5814c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce5806db70 0x7fce5814c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 -- 172.21.15.64:0/451697789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48009420 con 0x7fce58155a90 2023-09-27T23:31:50.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce567fc700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 0x7fce580cb210 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fce400011e0 tx=0x7fce40001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40005b30 con 0x7fce58155a90 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40005cb0 con 0x7fce58155a90 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5814d120 con 0x7fce58155a90 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40007340 con 0x7fce58155a90 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.981+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5814d6e0 con 0x7fce58155a90 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.985+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5814dbc0 con 0x7fce58155a90 2023-09-27T23:31:50.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.985+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fce40025430 con 0x7fce58155a90 2023-09-27T23:31:50.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.985+0000 7fce4effd700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fce38060b90 0x7fce38063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:50.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.985+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fce4007dbf0 con 0x7fce58155a90 2023-09-27T23:31:50.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.985+0000 7fce56ffd700 1 -- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fce38060b90 msgr2=0x7fce38063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.985+0000 7fce56ffd700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fce38060b90 0x7fce38063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:50.992 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:50.989+0000 7fb54387e000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:31:50.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:50.993+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fce4004d9a0 con 0x7fce58155a90 2023-09-27T23:31:51.109 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.105+0000 7fb54387e000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:31:51.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.149+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce580c6af0 con 0x7fce58155a90 2023-09-27T23:31:51.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.149+0000 7fce4effd700 1 -- 172.21.15.64:0/451697789 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce40050fc0 con 0x7fce58155a90 2023-09-27T23:31:51.155 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fce38060b90 msgr2=0x7fce38063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fce38060b90 0x7fce38063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 msgr2=0x7fce580cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 0x7fce580cb210 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fce400011e0 tx=0x7fce40001a10 comp rx=0 tx=0).stop 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 shutdown_connections 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fce38060b90 0x7fce38063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fce58155e60 0x7fce5814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce58155a90 0x7fce580cb210 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 --2- 172.21.15.64:0/451697789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce5806db70 0x7fce5814c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 >> 172.21.15.64:0/451697789 conn(0x7fce580c8cd0 msgr2=0x7fce580c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:51.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 shutdown_connections 2023-09-27T23:31:51.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.153+0000 7fce57fff700 1 -- 172.21.15.64:0/451697789 wait complete. 2023-09-27T23:31:51.168 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:31:51.256 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.255+0000 7f7443bfd000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:31:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 Processor -- start 2023-09-27T23:31:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- start start 2023-09-27T23:31:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76d4150200 0x7f76d40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f76d40ace30 0x7f76d40aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d40af250 con 0x7f76d40ae770 2023-09-27T23:31:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40af530 con 0x7f76d4150200 2023-09-27T23:31:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40ca8a0 con 0x7f76d40ace30 2023-09-27T23:31:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76d4150200 0x7f76d40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76d4150200 0x7f76d40c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57172/0 (socket says 172.21.15.64:57172) 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 -- 172.21.15.64:0/1916620401 learned_addr learned my addr 172.21.15.64:0/1916620401 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d94af700 1 --2- 172.21.15.64:0/1916620401 >> v2:172.21.15.123:3300/0 conn(0x7f76d40ace30 0x7f76d40aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 -- 172.21.15.64:0/1916620401 >> v2:172.21.15.123:3300/0 conn(0x7f76d40ace30 msgr2=0x7f76d40aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 --2- 172.21.15.64:0/1916620401 >> v2:172.21.15.123:3300/0 conn(0x7f76d40ace30 0x7f76d40aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 -- 172.21.15.64:0/1916620401 >> v1:172.21.15.123:6789/0 conn(0x7f76d40ae770 legacy=0x7f76d40ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 -- 172.21.15.64:0/1916620401 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40cab80 con 0x7f76d4150200 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d8cae700 1 --2- 172.21.15.64:0/1916620401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d4150200 0x7f76d40c7170 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f76d406f280 tx=0x7f76c800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5808d9b7fa2ecf0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d37fe700 1 -- 172.21.15.64:0/1916620401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c800dc10 con 0x7f76d4150200 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d37fe700 1 -- 172.21.15.64:0/1916620401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c800dd70 con 0x7f76d4150200 2023-09-27T23:31:51.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d37fe700 1 -- 172.21.15.64:0/1916620401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c80106c0 con 0x7f76d4150200 2023-09-27T23:31:51.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1916620401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d4150200 msgr2=0x7f76d40c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1916620401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d4150200 0x7f76d40c7170 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f76d406f280 tx=0x7f76c800b4f0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1916620401 shutdown_connections 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1916620401 >> v2:172.21.15.123:3300/0 conn(0x7f76d40ace30 0x7f76d40aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1916620401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d4150200 0x7f76d40c7170 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1916620401 >> 172.21.15.64:0/1916620401 conn(0x7f76d40ccb30 msgr2=0x7f76d40ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1916620401 shutdown_connections 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.269+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1916620401 wait complete. 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 Processor -- start 2023-09-27T23:31:51.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 -- start start 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ace30 0x7f76d40c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 0x7f76d414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d4150200 0x7f76d4069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41586c0 con 0x7f76d40ae770 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40c4b10 con 0x7f76d40ace30 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d9cb0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4157e20 con 0x7f76d4150200 2023-09-27T23:31:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 0x7f76d414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 0x7f76d414e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57178/0 (socket says 172.21.15.64:57178) 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d3fff700 1 -- 172.21.15.64:0/1610253368 learned_addr learned my addr 172.21.15.64:0/1610253368 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d8cae700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ace30 0x7f76d40c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.273+0000 7f76d94af700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d4150200 0x7f76d4069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d3fff700 1 -- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d4150200 msgr2=0x7f76d4069230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d3fff700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d4150200 0x7f76d4069230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d3fff700 1 -- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ace30 msgr2=0x7f76d40c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d3fff700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ace30 0x7f76d40c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d3fff700 1 -- 172.21.15.64:0/1610253368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c800b790 con 0x7f76d40ae770 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d3fff700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 0x7f76d414e9f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f76c000c5d0 tx=0x7f76c0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0011590 con 0x7f76d40ae770 2023-09-27T23:31:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c00116f0 con 0x7f76d40ae770 2023-09-27T23:31:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0011ab0 con 0x7f76d40ae770 2023-09-27T23:31:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d4158100 con 0x7f76d40ae770 2023-09-27T23:31:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d41589f0 con 0x7f76d40ae770 2023-09-27T23:31:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d404be40 con 0x7f76d40ae770 2023-09-27T23:31:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.277+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f76c001c460 con 0x7f76d40ae770 2023-09-27T23:31:51.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.281+0000 7f76d0ff9700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f76b8060b90 0x7f76b8063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.281+0000 7f76d8cae700 1 -- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f76b8060b90 msgr2=0x7f76b8063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:51.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.281+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f76c007d000 con 0x7f76d40ae770 2023-09-27T23:31:51.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.281+0000 7f76d8cae700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f76b8060b90 0x7f76b8063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:51.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.285+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f76c004cd50 con 0x7f76d40ae770 2023-09-27T23:31:51.408 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.405+0000 7fb54387e000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:31:51.419 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.415+0000 7f7443bfd000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:31:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.445+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76d406cbf0 con 0x7f76d40ae770 2023-09-27T23:31:51.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.445+0000 7f76d0ff9700 1 -- 172.21.15.64:0/1610253368 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76c0050370 con 0x7f76d40ae770 2023-09-27T23:31:51.451 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:31:51.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f76b8060b90 msgr2=0x7f76b8063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f76b8060b90 0x7f76b8063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 msgr2=0x7f76d414e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 0x7f76d414e9f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f76c000c5d0 tx=0x7f76c0007f60 comp rx=0 tx=0).stop 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 shutdown_connections 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f76b8060b90 0x7f76b8063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76d4150200 0x7f76d4069230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76d40ae770 0x7f76d414e9f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 --2- 172.21.15.64:0/1610253368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ace30 0x7f76d40c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 >> 172.21.15.64:0/1610253368 conn(0x7f76d40ccb30 msgr2=0x7f76d40b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 shutdown_connections 2023-09-27T23:31:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.449+0000 7f76d9cb0700 1 -- 172.21.15.64:0/1610253368 wait complete. 2023-09-27T23:31:51.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:31:51.482 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.479+0000 7f7443bfd000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:31:51.532 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.531+0000 7f7443bfd000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:31:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 Processor -- start 2023-09-27T23:31:51.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 -- start start 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f18ec0cb750 0x7f18ec0abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18ec0cbb20 0x7f18ec0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0c8730 con 0x7f18ec0aba90 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0cbf80 con 0x7f18ec0cb750 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18f2254700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0c7e70 con 0x7f18ec0cbb20 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18eaffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f18ec0aba90 0x7f18ec0c1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47794/0 (socket says 172.21.15.64:47794) 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18eb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f18ec0cb750 0x7f18ec0abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.561+0000 7f18eaffd700 1 -- 172.21.15.64:0/569828861 learned_addr learned my addr 172.21.15.64:0/569828861 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:51.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ebfff700 1 --2- 172.21.15.64:0/569828861 >> v2:172.21.15.123:3300/0 conn(0x7f18ec0cbb20 0x7f18ec0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ea7fc700 1 -- 172.21.15.64:0/569828861 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065317853 0 0) 0x7f18ec0c8730 con 0x7f18ec0aba90 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ea7fc700 1 -- 172.21.15.64:0/569828861 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18cc003570 con 0x7f18ec0aba90 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18eb7fe700 1 -- 172.21.15.64:0/569828861 >> v2:172.21.15.123:3300/0 conn(0x7f18ec0cbb20 msgr2=0x7f18ec0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18eb7fe700 1 --2- 172.21.15.64:0/569828861 >> v2:172.21.15.123:3300/0 conn(0x7f18ec0cbb20 0x7f18ec0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18eb7fe700 1 -- 172.21.15.64:0/569828861 >> v1:172.21.15.64:6789/0 conn(0x7f18ec0aba90 legacy=0x7f18ec0c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18eb7fe700 1 -- 172.21.15.64:0/569828861 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0c23a0 con 0x7f18ec0cb750 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ebfff700 1 --2- 172.21.15.64:0/569828861 >> v2:172.21.15.123:3300/0 conn(0x7f18ec0cbb20 0x7f18ec0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18eb7fe700 1 --2- 172.21.15.64:0/569828861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cb750 0x7f18ec0abff0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0c86a0 tx=0x7f18e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d8cb92cad7cd0d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:51.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ea7fc700 1 -- 172.21.15.64:0/569828861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4011630 con 0x7f18ec0cb750 2023-09-27T23:31:51.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ea7fc700 1 -- 172.21.15.64:0/569828861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4011790 con 0x7f18ec0cb750 2023-09-27T23:31:51.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18ea7fc700 1 -- 172.21.15.64:0/569828861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4011b10 con 0x7f18ec0cb750 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 -- 172.21.15.64:0/569828861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cb750 msgr2=0x7f18ec0abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 --2- 172.21.15.64:0/569828861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cb750 0x7f18ec0abff0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0c86a0 tx=0x7f18e40091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 -- 172.21.15.64:0/569828861 shutdown_connections 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 --2- 172.21.15.64:0/569828861 >> v2:172.21.15.123:3300/0 conn(0x7f18ec0cbb20 0x7f18ec0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 --2- 172.21.15.64:0/569828861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cb750 0x7f18ec0abff0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 -- 172.21.15.64:0/569828861 >> 172.21.15.64:0/569828861 conn(0x7f18ec0cd9c0 msgr2=0x7f18ec0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 -- 172.21.15.64:0/569828861 shutdown_connections 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 -- 172.21.15.64:0/569828861 wait complete. 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 Processor -- start 2023-09-27T23:31:51.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.565+0000 7f18f2254700 1 -- start start 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18ec0aba90 0x7f18ec14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18ec0cb750 0x7f18ec0a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 0x7f18ec0a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0aa040 con 0x7f18ec0cbb20 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0ae3e0 con 0x7f18ec0aba90 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0a9430 con 0x7f18ec0cb750 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 0x7f18ec0a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 0x7f18ec0a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57196/0 (socket says 172.21.15.64:57196) 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 -- 172.21.15.64:0/3465934779 learned_addr learned my addr 172.21.15.64:0/3465934779 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18eaffd700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18ec0cb750 0x7f18ec0a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18eb7fe700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18ec0aba90 0x7f18ec14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 -- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18ec0cb750 msgr2=0x7f18ec0a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18ec0cb750 0x7f18ec0a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 -- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18ec0aba90 msgr2=0x7f18ec14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18ec0aba90 0x7f18ec14dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 -- 172.21.15.64:0/3465934779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4009420 con 0x7f18ec0cbb20 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18ebfff700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 0x7f18ec0a8d90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00d890 tx=0x7f18dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0125f0 con 0x7f18ec0cbb20 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc012750 con 0x7f18ec0cbb20 2023-09-27T23:31:51.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec0a9710 con 0x7f18ec0cbb20 2023-09-27T23:31:51.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc012ad0 con 0x7f18ec0cbb20 2023-09-27T23:31:51.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec0a9cd0 con 0x7f18ec0cbb20 2023-09-27T23:31:51.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec071120 con 0x7f18ec0cbb20 2023-09-27T23:31:51.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.569+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f18dc013480 con 0x7f18ec0cbb20 2023-09-27T23:31:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.573+0000 7f18e37fe700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 0x7f18d0063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.573+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f18dc07d5a0 con 0x7f18ec0cbb20 2023-09-27T23:31:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.573+0000 7f18eb7fe700 1 -- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 msgr2=0x7f18d0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.573+0000 7f18eb7fe700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 0x7f18d0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:51.582 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.577+0000 7fb54387e000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:31:51.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.577+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f18dc050d80 con 0x7f18ec0cbb20 2023-09-27T23:31:51.587 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.583+0000 7f7443bfd000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:31:51.651 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.645+0000 7fb54387e000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:31:51.705 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.701+0000 7fb54387e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:31:51.762 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.759+0000 7f7443bfd000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:31:51.764 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.757+0000 7fb54387e000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:31:51.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.773+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18ec067f40 con 0x7f18ec0cbb20 2023-09-27T23:31:51.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.773+0000 7f18e37fe700 1 -- 172.21.15.64:0/3465934779 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f18dc050910 con 0x7f18ec0cbb20 2023-09-27T23:31:51.779 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:51.779 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:30:29.391491+0000","modified":"2023-09-27T23:30:31.410091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7605":{"gid":7605,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1829030850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1829030850},{"type":"v1","addr":"172.21.15.64:6835","nonce":1829030850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-09-27T23:31:51.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18eb7fe700 1 -- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 msgr2=0x7f18d0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:51.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18eb7fe700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 0x7f18d0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:31:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 msgr2=0x7f18d0063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:51.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 0x7f18d0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 msgr2=0x7f18ec0a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 0x7f18ec0a8d90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00d890 tx=0x7f18dc008040 comp rx=0 tx=0).stop 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 shutdown_connections 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f18d0060b90 0x7f18d0063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18ec0cbb20 0x7f18ec0a8d90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18ec0cb750 0x7f18ec0a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 --2- 172.21.15.64:0/3465934779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18ec0aba90 0x7f18ec14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 >> 172.21.15.64:0/3465934779 conn(0x7f18ec0cd9c0 msgr2=0x7f18ec14e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 shutdown_connections 2023-09-27T23:31:51.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.777+0000 7f18f2254700 1 -- 172.21.15.64:0/3465934779 wait complete. 2023-09-27T23:31:51.783 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2023-09-27T23:31:51.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:31:51.816 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.815+0000 7f7443bfd000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:31:51.878 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:51.875+0000 7f7443bfd000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:31:51.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 Processor -- start 2023-09-27T23:31:51.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 -- start start 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00acc30 0x7f24c00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24c00c8b60 0x7f24c015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00add10 con 0x7f24c00c8280 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c00cc920 con 0x7f24c00c8b60 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24c4cfd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00c9290 con 0x7f24c00acc30 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24beffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f24c00c8280 0x7f24c00ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47804/0 (socket says 172.21.15.64:47804) 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24beffd700 1 -- 172.21.15.64:0/3776192009 learned_addr learned my addr 172.21.15.64:0/3776192009 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:51.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bf7fe700 1 --2- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00acc30 0x7f24c00ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bffff700 1 --2- 172.21.15.64:0/3776192009 >> v2:172.21.15.123:3300/0 conn(0x7f24c00c8b60 0x7f24c015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24be7fc700 1 -- 172.21.15.64:0/3776192009 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790802513 0 0) 0x7f24c00add10 con 0x7f24c00c8280 2023-09-27T23:31:51.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24be7fc700 1 -- 172.21.15.64:0/3776192009 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a8003570 con 0x7f24c00c8280 2023-09-27T23:31:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bffff700 1 -- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00acc30 msgr2=0x7f24c00ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bffff700 1 --2- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00acc30 0x7f24c00ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bffff700 1 -- 172.21.15.64:0/3776192009 >> v1:172.21.15.64:6789/0 conn(0x7f24c00c8280 legacy=0x7f24c00ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bffff700 1 -- 172.21.15.64:0/3776192009 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00c86d0 con 0x7f24c00c8b60 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24bffff700 1 --2- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00c8b60 0x7f24c015a6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009e60 tx=0x7f24b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dd5d27f6a887463 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.889+0000 7f24be7fc700 1 -- 172.21.15.64:0/3776192009 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800e070 con 0x7f24c00c8b60 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24be7fc700 1 -- 172.21.15.64:0/3776192009 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800bb40 con 0x7f24c00c8b60 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24be7fc700 1 -- 172.21.15.64:0/3776192009 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8012430 con 0x7f24c00c8b60 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 -- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00c8b60 msgr2=0x7f24c015a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00c8b60 0x7f24c015a6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009e60 tx=0x7f24b8009780 comp rx=0 tx=0).stop 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 -- 172.21.15.64:0/3776192009 shutdown_connections 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00c8b60 0x7f24c015a6f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/3776192009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00acc30 0x7f24c00ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 -- 172.21.15.64:0/3776192009 >> 172.21.15.64:0/3776192009 conn(0x7f24c00cdbd0 msgr2=0x7f24c00cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 -- 172.21.15.64:0/3776192009 shutdown_connections 2023-09-27T23:31:51.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 -- 172.21.15.64:0/3776192009 wait complete. 2023-09-27T23:31:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 Processor -- start 2023-09-27T23:31:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.893+0000 7f24c4cfd700 1 -- start start 2023-09-27T23:31:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00acc30 0x7f24c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 0x7f24c00c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00c8b60 0x7f24c00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01575f0 con 0x7f24c00c8280 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0153490 con 0x7f24c00acc30 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0157840 con 0x7f24c00c8b60 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 0x7f24c00c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 0x7f24c00c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57212/0 (socket says 172.21.15.64:57212) 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 -- 172.21.15.64:0/160420452 learned_addr learned my addr 172.21.15.64:0/160420452 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24bf7fe700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00acc30 0x7f24c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24bffff700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00c8b60 0x7f24c00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 -- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00c8b60 msgr2=0x7f24c00c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00c8b60 0x7f24c00c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 -- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00acc30 msgr2=0x7f24c0068930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00acc30 0x7f24c0068930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 -- 172.21.15.64:0/160420452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009a20 con 0x7f24c00c8280 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24beffd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 0x7f24c00c6fe0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f24b400c7c0 tx=0x7f24b4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4005cd0 con 0x7f24c00c8280 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4005e30 con 0x7f24c00c8280 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c0157b20 con 0x7f24c00c8280 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4007320 con 0x7f24c00c8280 2023-09-27T23:31:51.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c01580e0 con 0x7f24c00c8280 2023-09-27T23:31:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c0158300 con 0x7f24c00c8280 2023-09-27T23:31:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f24b401b3b0 con 0x7f24c00c8280 2023-09-27T23:31:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.897+0000 7f24a77fe700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24ac060b90 0x7f24ac063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.901+0000 7f24bf7fe700 1 -- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24ac060b90 msgr2=0x7f24ac063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.901+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f24b407cb10 con 0x7f24c00c8280 2023-09-27T23:31:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.901+0000 7f24bf7fe700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24ac060b90 0x7f24ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:51.905+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f24b404c860 con 0x7f24c00c8280 2023-09-27T23:31:51.959 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:51.953+0000 7fb54387e000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:31:52.017 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:52.013+0000 7fb54387e000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:31:52.028 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.023+0000 7f7443bfd000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:31:52.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.065+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24c004be40 con 0x7f24c00c8280 2023-09-27T23:31:52.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.065+0000 7f24a77fe700 1 -- 172.21.15.64:0/160420452 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+17582 (secure 0 0 0) 0x7f24b4017020 con 0x7f24c00c8280 2023-09-27T23:31:52.071 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:52.071 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":116,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:30:45.799145+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:30:27.964715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-27T23:30:28.361565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:30:45.799008+0000","172.21.15.123:6832/33228":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/3430876092":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/3314489464":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/926402627":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1057705795":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1993249321":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/965278245":"2023-09-28T23:30:45.799008+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:31:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24ac060b90 msgr2=0x7f24ac063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24ac060b90 0x7f24ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 msgr2=0x7f24c00c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 0x7f24c00c6fe0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f24b400c7c0 tx=0x7f24b4007e00 comp rx=0 tx=0).stop 2023-09-27T23:31:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 shutdown_connections 2023-09-27T23:31:52.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24ac060b90 0x7f24ac063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24c00c8b60 0x7f24c00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c00c8280 0x7f24c00c6fe0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 --2- 172.21.15.64:0/160420452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c00acc30 0x7f24c0068930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 >> 172.21.15.64:0/160420452 conn(0x7f24c00cdbd0 msgr2=0x7f24c0155040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 shutdown_connections 2023-09-27T23:31:52.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.069+0000 7f24c4cfd700 1 -- 172.21.15.64:0/160420452 wait complete. 2023-09-27T23:31:52.084 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:31:52.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:31:52.085 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:52.081+0000 7fb54387e000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:31:52.087 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.083+0000 7f7443bfd000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:31:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 Processor -- start 2023-09-27T23:31:52.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 -- start start 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7760156c90 0x7f77600cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77600cda30 0x7f7760157060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600cd5a0 con 0x7f77600c90e0 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600cc990 con 0x7f7760156c90 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7766531700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600ccc70 con 0x7f77600cda30 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7760156c90 0x7f77600cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7760156c90 0x7f77600cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57226/0 (socket says 172.21.15.64:57226) 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 -- 172.21.15.64:0/259424177 learned_addr learned my addr 172.21.15.64:0/259424177 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:52.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 -- 172.21.15.64:0/259424177 >> v2:172.21.15.123:3300/0 conn(0x7f77600cda30 msgr2=0x7f7760157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7757fff700 1 --2- 172.21.15.64:0/259424177 >> v2:172.21.15.123:3300/0 conn(0x7f77600cda30 0x7f7760157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 --2- 172.21.15.64:0/259424177 >> v2:172.21.15.123:3300/0 conn(0x7f77600cda30 0x7f7760157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 -- 172.21.15.64:0/259424177 >> v1:172.21.15.123:6789/0 conn(0x7f77600c90e0 legacy=0x7f776015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 -- 172.21.15.64:0/259424177 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77600ccf50 con 0x7f7760156c90 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f7757fff700 1 --2- 172.21.15.64:0/259424177 >> v2:172.21.15.123:3300/0 conn(0x7f77600cda30 0x7f7760157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775ffff700 1 --2- 172.21.15.64:0/259424177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f77600cc3b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f77600638f0 tx=0x7f7750009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f792b2cfa1ebf036 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775f7fe700 1 -- 172.21.15.64:0/259424177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000b820 con 0x7f7760156c90 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775f7fe700 1 -- 172.21.15.64:0/259424177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000b980 con 0x7f7760156c90 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.177+0000 7f775f7fe700 1 -- 172.21.15.64:0/259424177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000bd20 con 0x7f7760156c90 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 -- 172.21.15.64:0/259424177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 msgr2=0x7f77600cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 --2- 172.21.15.64:0/259424177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f77600cc3b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f77600638f0 tx=0x7f7750009250 comp rx=0 tx=0).stop 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 -- 172.21.15.64:0/259424177 shutdown_connections 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 --2- 172.21.15.64:0/259424177 >> v2:172.21.15.123:3300/0 conn(0x7f77600cda30 0x7f7760157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 --2- 172.21.15.64:0/259424177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f77600cc3b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 -- 172.21.15.64:0/259424177 >> 172.21.15.64:0/259424177 conn(0x7f776006c7c0 msgr2=0x7f776006cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 -- 172.21.15.64:0/259424177 shutdown_connections 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 -- 172.21.15.64:0/259424177 wait complete. 2023-09-27T23:31:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 Processor -- start 2023-09-27T23:31:52.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.181+0000 7f7766531700 1 -- start start 2023-09-27T23:31:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77600c90e0 0x7f7760157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77600cda30 0x7f7760154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f7760154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760158520 con 0x7f7760156c90 2023-09-27T23:31:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601587c0 con 0x7f77600cda30 2023-09-27T23:31:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7760158aa0 con 0x7f77600c90e0 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f7760154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f7760154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57236/0 (socket says 172.21.15.64:57236) 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 -- 172.21.15.64:0/196840518 learned_addr learned my addr 172.21.15.64:0/196840518 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7757fff700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77600cda30 0x7f7760154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f775ffff700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77600c90e0 0x7f7760157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 -- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77600c90e0 msgr2=0x7f7760157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77600c90e0 0x7f7760157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 -- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77600cda30 msgr2=0x7f7760154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77600cda30 0x7f7760154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 -- 172.21.15.64:0/196840518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500094f0 con 0x7f7760156c90 2023-09-27T23:31:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7764ace700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f7760154d50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f775800bbb0 tx=0x7f7758009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:52.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800c630 con 0x7f7760156c90 2023-09-27T23:31:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775800c7b0 con 0x7f7760156c90 2023-09-27T23:31:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7760158d80 con 0x7f7760156c90 2023-09-27T23:31:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800cb70 con 0x7f7760156c90 2023-09-27T23:31:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77600c61c0 con 0x7f7760156c90 2023-09-27T23:31:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f7758012070 con 0x7f7760156c90 2023-09-27T23:31:52.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.185+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7760071120 con 0x7f7760156c90 2023-09-27T23:31:52.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.189+0000 7f775cff9700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 0x7f7740063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.189+0000 7f775ffff700 1 -- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 msgr2=0x7f7740063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:52.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.189+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f775807d030 con 0x7f7760156c90 2023-09-27T23:31:52.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.189+0000 7f775ffff700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 0x7f7740063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:52.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.193+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f775804cd80 con 0x7f7760156c90 2023-09-27T23:31:52.251 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:52.245+0000 7fb54387e000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:31:52.318 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:52.313+0000 7fb54387e000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:31:52.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.393+0000 7f775ffff700 1 -- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 msgr2=0x7f7740063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:52.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.393+0000 7f775ffff700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 0x7f7740063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:31:52.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.393+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77600cd230 con 0x7f7760156c90 2023-09-27T23:31:52.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.393+0000 7f775cff9700 1 -- 172.21.15.64:0/196840518 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f77580503a0 con 0x7f7760156c90 2023-09-27T23:31:52.399 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:52.400 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:30:29.391491+0000","modified":"2023-09-27T23:30:31.410091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7605":{"gid":7605,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1829030850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1829030850},{"type":"v1","addr":"172.21.15.64:6835","nonce":1829030850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-09-27T23:31:52.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 msgr2=0x7f7740063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:52.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 0x7f7740063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 msgr2=0x7f7760154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f7760154d50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f775800bbb0 tx=0x7f7758009840 comp rx=0 tx=0).stop 2023-09-27T23:31:52.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 shutdown_connections 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7740060b90 0x7f7740063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760156c90 0x7f7760154d50 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77600cda30 0x7f7760154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 --2- 172.21.15.64:0/196840518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77600c90e0 0x7f7760157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 >> 172.21.15.64:0/196840518 conn(0x7f776006c7c0 msgr2=0x7f776015af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 shutdown_connections 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.397+0000 7f7766531700 1 -- 172.21.15.64:0/196840518 wait complete. 2023-09-27T23:31:52.403 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2023-09-27T23:31:52.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:31:52.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 Processor -- start 2023-09-27T23:31:52.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 -- start start 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4998158e20 0x7f499806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 0x7f49980ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499806b730 con 0x7f499815ca80 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499806ab20 con 0x7f4998158e20 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4997fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499806ae00 con 0x7f49981591f0 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f49977fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f499815ca80 0x7f499806c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47814/0 (socket says 172.21.15.64:47814) 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f49977fe700 1 -- 172.21.15.64:0/2555502946 learned_addr learned my addr 172.21.15.64:0/2555502946 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f49967fc700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 0x7f49980ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297945899 0 0) 0x7f499806b730 con 0x7f499815ca80 2023-09-27T23:31:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4980003570 con 0x7f499815ca80 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4996ffd700 1 --2- 172.21.15.64:0/2555502946 >> v2:172.21.15.123:3300/0 conn(0x7f4998158e20 0x7f499806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1417523400 0 0) 0x7f4980003570 con 0x7f499815ca80 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 msgr2=0x7f49980ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 0x7f49980ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 >> v2:172.21.15.123:3300/0 conn(0x7f4998158e20 msgr2=0x7f499806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 --2- 172.21.15.64:0/2555502946 >> v2:172.21.15.123:3300/0 conn(0x7f4998158e20 0x7f499806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499806b0e0 con 0x7f499815ca80 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f49900017f0 con 0x7f499815ca80 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4996ffd700 1 --2- 172.21.15.64:0/2555502946 >> v2:172.21.15.123:3300/0 conn(0x7f4998158e20 0x7f499806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 >> v1:172.21.15.64:6789/0 conn(0x7f499815ca80 legacy=0x7f499806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4980003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980004810 0x7f4980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 0x7f49800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499806ab20 con 0x7f4980003a60 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499806ae00 con 0x7f4980004810 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498000bae0 con 0x7f49981591f0 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.517+0000 7f4996ffd700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4980003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f49967fc700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980004810 0x7f4980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4996ffd700 1 -- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 msgr2=0x7f49800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4996ffd700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49981591f0 0x7f49800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4996ffd700 1 -- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980004810 msgr2=0x7f4980004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4996ffd700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980004810 0x7f4980004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4996ffd700 1 -- 172.21.15.64:0/2555502946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980003570 con 0x7f4980003a60 2023-09-27T23:31:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4996ffd700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4980003e70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4988009ba0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2cee19b8d740c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800c070 con 0x7f4980003a60 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498800e5e0 con 0x7f4980003a60 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4995ffb700 1 -- 172.21.15.64:0/2555502946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988009510 con 0x7f4980003a60 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 -- 172.21.15.64:0/2555502946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 msgr2=0x7f4980003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 --2- 172.21.15.64:0/2555502946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4980003e70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4988009ba0 tx=0x7f4988008040 comp rx=0 tx=0).stop 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f49977fe700 1 -- 172.21.15.64:0/2555502946 reap_dead start 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 -- 172.21.15.64:0/2555502946 shutdown_connections 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 -- 172.21.15.64:0/2555502946 >> 172.21.15.64:0/2555502946 conn(0x7f4998067bb0 msgr2=0x7f49980cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 -- 172.21.15.64:0/2555502946 shutdown_connections 2023-09-27T23:31:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 -- 172.21.15.64:0/2555502946 wait complete. 2023-09-27T23:31:52.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 Processor -- start 2023-09-27T23:31:52.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.521+0000 7f4997fff700 1 -- start start 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4998155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980b4a10 0x7f49980c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f499815ec00 0x7f499815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499815f5e0 con 0x7f4980003a60 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4998155550 con 0x7f49980b4a10 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980c8eb0 con 0x7f499815ec00 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4998155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4998155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57252/0 (socket says 172.21.15.64:57252) 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 -- 172.21.15.64:0/749589195 learned_addr learned my addr 172.21.15.64:0/749589195 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f49967fc700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980b4a10 0x7f49980c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f49977fe700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f499815ec00 0x7f499815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 -- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f499815ec00 msgr2=0x7f499815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f499815ec00 0x7f499815f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 -- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980b4a10 msgr2=0x7f49980c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980b4a10 0x7f49980c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 -- 172.21.15.64:0/749589195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980c9190 con 0x7f4980003a60 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f49967fc700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980b4a10 0x7f49980c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4996ffd700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4998155010 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f498800e450 tx=0x7f498801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800ee60 con 0x7f4980003a60 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498801d440 con 0x7f4980003a60 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49980c9470 con 0x7f4980003a60 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498801d800 con 0x7f4980003a60 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499815a590 con 0x7f4980003a60 2023-09-27T23:31:52.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.525+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499804be40 con 0x7f4980003a60 2023-09-27T23:31:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.529+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 46) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f498801dbf0 con 0x7f4980003a60 2023-09-27T23:31:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.529+0000 7f497effd700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4984060bc0 0x7f4984063070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.529+0000 7f49967fc700 1 -- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4984060bc0 msgr2=0x7f4984063070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:31:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.529+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f498807f100 con 0x7f4980003a60 2023-09-27T23:31:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.529+0000 7f49967fc700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4984060bc0 0x7f4984063070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:31:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.537+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f498804ee50 con 0x7f4980003a60 2023-09-27T23:31:52.678 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.675+0000 7f7443bfd000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:31:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.709+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 47) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f498804f220 con 0x7f4980003a60 2023-09-27T23:31:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.713+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4984060bc0 msgr2=0x7f4984063070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:31:52.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.713+0000 7f497effd700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4984060bc0 0x7f4984063070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f499806b3c0 con 0x7f4980003a60 2023-09-27T23:31:52.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f497effd700 1 -- 172.21.15.64:0/749589195 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f4988054ad0 con 0x7f4980003a60 2023-09-27T23:31:52.737 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:52.737 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:30:29.391491+0000","modified":"2023-09-27T23:30:31.410091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7605":{"gid":7605,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/1829030850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1829030850},{"type":"v1","addr":"172.21.15.64:6835","nonce":1829030850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-09-27T23:31:52.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 msgr2=0x7f4998155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4998155010 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f498800e450 tx=0x7f498801a000 comp rx=0 tx=0).stop 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 shutdown_connections 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4984060bc0 0x7f4984063070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f499815ec00 0x7f499815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49980b4a10 0x7f49980c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 --2- 172.21.15.64:0/749589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4980003a60 0x7f4998155010 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 >> 172.21.15.64:0/749589195 conn(0x7f4998067bb0 msgr2=0x7f49980c4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 shutdown_connections 2023-09-27T23:31:52.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.733+0000 7f4997fff700 1 -- 172.21.15.64:0/749589195 wait complete. 2023-09-27T23:31:52.741 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2023-09-27T23:31:52.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740ce39700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740ce39700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740ce39700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740ce39700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740ce39700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740f63e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740f63e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740f63e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740f63e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:31:52.775+0000 7f740f63e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:31:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e25335700 1 Processor -- start 2023-09-27T23:31:52.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e25335700 1 -- start start 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e25335700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 0x7f6e2015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e25335700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e200c9240 con 0x7f6e200acdd0 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e25335700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e200ccb00 con 0x7f6e2015ac00 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e25335700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015b040 con 0x7f6e200c8360 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6e200acdd0 0x7f6e200c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47826/0 (socket says 172.21.15.64:47826) 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1ffff700 1 -- 172.21.15.64:0/2716953201 learned_addr learned my addr 172.21.15.64:0/2716953201 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e17fff700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 0x7f6e2015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259386595 0 0) 0x7f6e10001110 con 0x7f6e200acdd0 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e0c003570 con 0x7f6e200acdd0 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435540190 0 0) 0x7f6e0c003570 con 0x7f6e200acdd0 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 msgr2=0x7f6e2015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 0x7f6e2015a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 >> v1:172.21.15.123:6789/0 conn(0x7f6e2015ac00 legacy=0x7f6e200ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e200cbef0 con 0x7f6e200acdd0 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f6e10001600 con 0x7f6e200acdd0 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 >> v1:172.21.15.64:6789/0 conn(0x7f6e200acdd0 legacy=0x7f6e200c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e0c003a90 0x7f6e0c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 0x7f6e0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015b040 con 0x7f6e0c004d50 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c00bba0 con 0x7f6e0c003a90 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c00bd60 con 0x7f6e200c8360 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e24b34700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e1ffff700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e0c003a90 0x7f6e0c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e17fff700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 0x7f6e0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e24b34700 1 -- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 msgr2=0x7f6e0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e24b34700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c8360 0x7f6e0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e24b34700 1 -- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e0c003a90 msgr2=0x7f6e0c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e24b34700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e0c003a90 0x7f6e0c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.845+0000 7f6e24b34700 1 -- 172.21.15.64:0/2716953201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c003570 con 0x7f6e0c004d50 2023-09-27T23:31:52.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e24b34700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e0c0051c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6e18001fc0 tx=0x7f6e18001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2706bba7b9f9a03f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18001950 con 0x7f6e0c004d50 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1800c8b0 con 0x7f6e0c004d50 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e1f7fe700 1 -- 172.21.15.64:0/2716953201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800cc50 con 0x7f6e0c004d50 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- 172.21.15.64:0/2716953201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 msgr2=0x7f6e0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 --2- 172.21.15.64:0/2716953201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e0c0051c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6e18001fc0 tx=0x7f6e18001640 comp rx=0 tx=0).stop 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e24b34700 1 -- 172.21.15.64:0/2716953201 reap_dead start 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- 172.21.15.64:0/2716953201 shutdown_connections 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- 172.21.15.64:0/2716953201 >> 172.21.15.64:0/2716953201 conn(0x7f6e200cdbd0 msgr2=0x7f6e200cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- 172.21.15.64:0/2716953201 shutdown_connections 2023-09-27T23:31:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- 172.21.15.64:0/2716953201 wait complete. 2023-09-27T23:31:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 Processor -- start 2023-09-27T23:31:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- start start 2023-09-27T23:31:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e20141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20141c50 0x7f6e200c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c6760 0x7f6e200b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:52.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e200c7e10 con 0x7f6e0c004d50 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e200c7980 con 0x7f6e20141c50 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e25335700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e200c6d70 con 0x7f6e200c6760 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e20141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e20141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57270/0 (socket says 172.21.15.64:57270) 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e1ffff700 1 -- 172.21.15.64:0/1065907904 learned_addr learned my addr 172.21.15.64:0/1065907904 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e17fff700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20141c50 0x7f6e200c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.849+0000 7f6e24b34700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c6760 0x7f6e200b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1ffff700 1 -- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c6760 msgr2=0x7f6e200b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1ffff700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c6760 0x7f6e200b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1ffff700 1 -- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20141c50 msgr2=0x7f6e200c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1ffff700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20141c50 0x7f6e200c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:52.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1ffff700 1 -- 172.21.15.64:0/1065907904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e200c7050 con 0x7f6e0c004d50 2023-09-27T23:31:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1ffff700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e20141710 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000e800 tx=0x7f6e10000d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10010490 con 0x7f6e0c004d50 2023-09-27T23:31:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e10010040 con 0x7f6e0c004d50 2023-09-27T23:31:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e200c7330 con 0x7f6e0c004d50 2023-09-27T23:31:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10010840 con 0x7f6e0c004d50 2023-09-27T23:31:52.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e20155ab0 con 0x7f6e0c004d50 2023-09-27T23:31:52.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e200c6b30 con 0x7f6e0c004d50 2023-09-27T23:31:52.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 47) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f6e10010bb0 con 0x7f6e0c004d50 2023-09-27T23:31:52.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.853+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f6e1007bfc0 con 0x7f6e0c004d50 2023-09-27T23:31:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:52.861+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6e200c6b30 con 0x7f6e0c004d50 2023-09-27T23:31:52.972 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:31:52.965+0000 7fb54387e000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:31:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.021+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e200c9240 con 0x7f6e0c004d50 2023-09-27T23:31:53.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e1cff9700 1 -- 172.21.15.64:0/1065907904 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+18085 (secure 0 0 0) 0x7f6e200c9240 con 0x7f6e0c004d50 2023-09-27T23:31:53.029 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:31:53.029 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":117,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:31:52.699306+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:30:27.964715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-27T23:30:28.361565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2579397838":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/1252694671":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/589867559":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/1537566218":"2023-09-28T23:31:52.699200+0000","172.21.15.123:6833/33228":"2023-09-28T23:30:45.799008+0000","172.21.15.123:6832/33228":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/3430876092":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/2711804873":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/3314489464":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/4246203665":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/926402627":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1057705795":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1135488954":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/1993249321":"2023-09-28T23:30:45.799008+0000","172.21.15.123:0/1491530946":"2023-09-28T23:31:52.699200+0000","172.21.15.123:0/965278245":"2023-09-28T23:30:45.799008+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:31:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 msgr2=0x7f6e20141710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e20141710 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000e800 tx=0x7f6e10000d00 comp rx=0 tx=0).stop 2023-09-27T23:31:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 shutdown_connections 2023-09-27T23:31:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e200c6760 0x7f6e200b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20141c50 0x7f6e200c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 --2- 172.21.15.64:0/1065907904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e0c004d50 0x7f6e20141710 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 >> 172.21.15.64:0/1065907904 conn(0x7f6e200cdbd0 msgr2=0x7f6e2006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 shutdown_connections 2023-09-27T23:31:53.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.025+0000 7f6e25335700 1 -- 172.21.15.64:0/1065907904 wait complete. 2023-09-27T23:31:53.042 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:31:53.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 Processor -- start 2023-09-27T23:31:53.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 -- start start 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f58480c6cd0 0x7f584806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f584815cc20 0x7f584815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584806b470 con 0x7f58480c6900 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584806be40 con 0x7f58480c6cd0 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584d04b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584806e110 con 0x7f584815cc20 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f5847fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58480c6900 0x7f584806da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47830/0 (socket says 172.21.15.64:47830) 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f5847fff700 1 -- 172.21.15.64:0/3418706131 learned_addr learned my addr 172.21.15.64:0/3418706131 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f58477fe700 1 --2- 172.21.15.64:0/3418706131 >> v2:172.21.15.123:3300/0 conn(0x7f58480c6cd0 0x7f584806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:53.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584c84a700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f584815cc20 0x7f584815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f5846ffd700 1 -- 172.21.15.64:0/3418706131 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497864891 0 0) 0x7f584806b470 con 0x7f58480c6900 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f5846ffd700 1 -- 172.21.15.64:0/3418706131 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5828003570 con 0x7f58480c6900 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f58477fe700 1 -- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f584815cc20 msgr2=0x7f584815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f58477fe700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f584815cc20 0x7f584815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f58477fe700 1 -- 172.21.15.64:0/3418706131 >> v1:172.21.15.64:6789/0 conn(0x7f58480c6900 legacy=0x7f584806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f58477fe700 1 -- 172.21.15.64:0/3418706131 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848161d10 con 0x7f58480c6cd0 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f584c84a700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f584815cc20 0x7f584815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.145+0000 7f58477fe700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58480c6cd0 0x7f584806b900 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f58300098e0 tx=0x7f58300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=113c9b6d7f0e497c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f5846ffd700 1 -- 172.21.15.64:0/3418706131 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011630 con 0x7f58480c6cd0 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f5846ffd700 1 -- 172.21.15.64:0/3418706131 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830011790 con 0x7f58480c6cd0 2023-09-27T23:31:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f5846ffd700 1 -- 172.21.15.64:0/3418706131 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011b30 con 0x7f58480c6cd0 2023-09-27T23:31:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58480c6cd0 msgr2=0x7f584806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58480c6cd0 0x7f584806b900 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f58300098e0 tx=0x7f58300091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- 172.21.15.64:0/3418706131 shutdown_connections 2023-09-27T23:31:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f584815cc20 0x7f584815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 --2- 172.21.15.64:0/3418706131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58480c6cd0 0x7f584806b900 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- 172.21.15.64:0/3418706131 >> 172.21.15.64:0/3418706131 conn(0x7f5848067bb0 msgr2=0x7f58480cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- 172.21.15.64:0/3418706131 shutdown_connections 2023-09-27T23:31:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- 172.21.15.64:0/3418706131 wait complete. 2023-09-27T23:31:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 Processor -- start 2023-09-27T23:31:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- start start 2023-09-27T23:31:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58480c6900 0x7f584815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 0x7f5848159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f584815cc20 0x7f584815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b18e0 con 0x7f58480c6cd0 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b0cd0 con 0x7f584815cc20 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.149+0000 7f584d04b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b0fb0 con 0x7f58480c6900 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 0x7f5848159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 0x7f5848159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57286/0 (socket says 172.21.15.64:57286) 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 -- 172.21.15.64:0/3174354852 learned_addr learned my addr 172.21.15.64:0/3174354852 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f5847fff700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58480c6900 0x7f584815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f584c84a700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f584815cc20 0x7f584815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 -- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58480c6900 msgr2=0x7f584815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58480c6900 0x7f584815eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 -- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f584815cc20 msgr2=0x7f584815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f584815cc20 0x7f584815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 -- 172.21.15.64:0/3174354852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009420 con 0x7f58480c6cd0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f58477fe700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 0x7f5848159be0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5848064040 tx=0x7f5830017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011b10 con 0x7f58480c6cd0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58300253f0 con 0x7f58480c6cd0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830009420 con 0x7f58480c6cd0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300257b0 con 0x7f58480c6cd0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58480b1570 con 0x7f58480c6cd0 2023-09-27T23:31:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584804be40 con 0x7f58480c6cd0 2023-09-27T23:31:53.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.153+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 47) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f5830025ba0 con 0x7f58480c6cd0 2023-09-27T23:31:53.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.157+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f583007e620 con 0x7f58480c6cd0 2023-09-27T23:31:53.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.169+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f583004e840 con 0x7f58480c6cd0 2023-09-27T23:31:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.365+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f584816d6b0 con 0x7f58480c6cd0 2023-09-27T23:31:53.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.709+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 48) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f583004e2f0 con 0x7f58480c6cd0 2023-09-27T23:31:53.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.709+0000 7f583ffff700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f582c066dc0 0x7f582c0691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:53.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.713+0000 7f5847fff700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f582c066dc0 0x7f582c0691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:53.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:53.713+0000 7f5847fff700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f582c066dc0 0x7f582c0691a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5840005c50 tx=0x7f5840009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:54.361 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:31:54.357+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:31:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.373+0000 7f583ffff700 1 -- 172.21.15.64:0/3174354852 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f583002ecd0 con 0x7f58480c6cd0 2023-09-27T23:31:54.380 INFO:tasks.ceph.mds.c.smithi064.stderr: -6136> 2023-09-27T23:30:35.307+0000 7f60e121f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f582c066dc0 msgr2=0x7f582c0691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f582c066dc0 0x7f582c0691a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5840005c50 tx=0x7f5840009040 comp rx=0 tx=0).stop 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 msgr2=0x7f5848159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 0x7f5848159be0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5848064040 tx=0x7f5830017000 comp rx=0 tx=0).stop 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 shutdown_connections 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f582c066dc0 0x7f582c0691a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f584815cc20 0x7f584815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58480c6cd0 0x7f5848159be0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 --2- 172.21.15.64:0/3174354852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58480c6900 0x7f584815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 >> 172.21.15.64:0/3174354852 conn(0x7f5848067bb0 msgr2=0x7f58480c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 shutdown_connections 2023-09-27T23:31:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.377+0000 7f584d04b700 1 -- 172.21.15.64:0/3174354852 wait complete. 2023-09-27T23:31:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:31:54.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:31:54.410 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:31:54.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 Processor -- start 2023-09-27T23:31:54.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 -- start start 2023-09-27T23:31:54.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d10158c90 0x7f7d100caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d10159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d101595a0 0x7f7d10159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1006c850 con 0x7f7d101595a0 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1006b470 con 0x7f7d10158c90 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1651c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1006b750 con 0x7f7d1006aeb0 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d101595a0 0x7f7d10159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d101595a0 0x7f7d10159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57300/0 (socket says 172.21.15.64:57300) 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 -- 172.21.15.64:0/492827155 learned_addr learned my addr 172.21.15.64:0/492827155 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d14d19700 1 --2- 172.21.15.64:0/492827155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d10159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1551a700 1 --2- 172.21.15.64:0/492827155 >> v2:172.21.15.123:3300/0 conn(0x7f7d10158c90 0x7f7d100caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 -- 172.21.15.64:0/492827155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 msgr2=0x7f7d10159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:54.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 --2- 172.21.15.64:0/492827155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d10159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 -- 172.21.15.64:0/492827155 >> v2:172.21.15.123:3300/0 conn(0x7f7d10158c90 msgr2=0x7f7d100caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 --2- 172.21.15.64:0/492827155 >> v2:172.21.15.123:3300/0 conn(0x7f7d10158c90 0x7f7d100caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d15d1b700 1 -- 172.21.15.64:0/492827155 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1006ba30 con 0x7f7d101595a0 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.505+0000 7f7d1551a700 1 --2- 172.21.15.64:0/492827155 >> v2:172.21.15.123:3300/0 conn(0x7f7d10158c90 0x7f7d100caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d15d1b700 1 --2- 172.21.15.64:0/492827155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d10159a20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7d100698f0 tx=0x7f7d08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c902eea09b0b32f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d07fff700 1 -- 172.21.15.64:0/492827155 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800e070 con 0x7f7d101595a0 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d07fff700 1 -- 172.21.15.64:0/492827155 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0800ba20 con 0x7f7d101595a0 2023-09-27T23:31:54.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d07fff700 1 -- 172.21.15.64:0/492827155 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080092e0 con 0x7f7d101595a0 2023-09-27T23:31:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 -- 172.21.15.64:0/492827155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 msgr2=0x7f7d10159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 --2- 172.21.15.64:0/492827155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d10159a20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7d100698f0 tx=0x7f7d08009840 comp rx=0 tx=0).stop 2023-09-27T23:31:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 -- 172.21.15.64:0/492827155 shutdown_connections 2023-09-27T23:31:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 --2- 172.21.15.64:0/492827155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d10159a20 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 --2- 172.21.15.64:0/492827155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d10159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 --2- 172.21.15.64:0/492827155 >> v2:172.21.15.123:3300/0 conn(0x7f7d10158c90 0x7f7d100caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 -- 172.21.15.64:0/492827155 >> 172.21.15.64:0/492827155 conn(0x7f7d10067bb0 msgr2=0x7f7d100cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 -- 172.21.15.64:0/492827155 shutdown_connections 2023-09-27T23:31:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 -- 172.21.15.64:0/492827155 wait complete. 2023-09-27T23:31:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 Processor -- start 2023-09-27T23:31:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.509+0000 7f7d1651c700 1 -- start start 2023-09-27T23:31:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d100b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d10158c90 0x7f7d100b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d100b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100b4920 con 0x7f7d101595a0 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1016c910 con 0x7f7d10158c90 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101697d0 con 0x7f7d1006aeb0 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d100b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d100b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57314/0 (socket says 172.21.15.64:57314) 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 -- 172.21.15.64:0/1040737715 learned_addr learned my addr 172.21.15.64:0/1040737715 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1551a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d100b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d14d19700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d10158c90 0x7f7d100b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 -- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 msgr2=0x7f7d100b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d100b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 -- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d10158c90 msgr2=0x7f7d100b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d10158c90 0x7f7d100b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 -- 172.21.15.64:0/1040737715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08009ae0 con 0x7f7d101595a0 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d14d19700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d10158c90 0x7f7d100b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d15d1b700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d100b2d60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7d10069b70 tx=0x7f7d08018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080032f0 con 0x7f7d101595a0 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08008e60 con 0x7f7d101595a0 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10169ab0 con 0x7f7d101595a0 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0801c640 con 0x7f7d101595a0 2023-09-27T23:31:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1016a070 con 0x7f7d101595a0 2023-09-27T23:31:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.513+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1016ca80 con 0x7f7d101595a0 2023-09-27T23:31:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.517+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 48) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f7d08003040 con 0x7f7d101595a0 2023-09-27T23:31:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.517+0000 7f7d057fa700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7cf40607d0 0x7f7cf4062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.517+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f7d0807e830 con 0x7f7d101595a0 2023-09-27T23:31:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.517+0000 7f7d1551a700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7cf40607d0 0x7f7cf4062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.517+0000 7f7d1551a700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7cf40607d0 0x7f7cf4062c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006f00 tx=0x7f7d00008040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:54.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.521+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7d1016ca80 con 0x7f7d101595a0 2023-09-27T23:31:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:54.713+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7d1016ca80 con 0x7f7d101595a0 2023-09-27T23:31:55.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.373+0000 7f7d057fa700 1 -- 172.21.15.64:0/1040737715 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d1016ca80 con 0x7f7d101595a0 2023-09-27T23:31:55.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7cf40607d0 msgr2=0x7f7cf4062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7cf40607d0 0x7f7cf4062c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006f00 tx=0x7f7d00008040 comp rx=0 tx=0).stop 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 msgr2=0x7f7d100b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d100b2d60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7d10069b70 tx=0x7f7d08018000 comp rx=0 tx=0).stop 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 shutdown_connections 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7cf40607d0 0x7f7cf4062c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d101595a0 0x7f7d100b2d60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d10158c90 0x7f7d100b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 --2- 172.21.15.64:0/1040737715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d1006aeb0 0x7f7d100b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 >> 172.21.15.64:0/1040737715 conn(0x7f7d10067bb0 msgr2=0x7f7d100ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 shutdown_connections 2023-09-27T23:31:55.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.377+0000 7f7d1651c700 1 -- 172.21.15.64:0/1040737715 wait complete. 2023-09-27T23:31:55.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:31:55.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.513+0000 7fd4f654b700 1 Processor -- start 2023-09-27T23:31:55.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.513+0000 7fd4f654b700 1 -- start start 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd4f00cb130 0x7fd4f00cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f006aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4f015c220 0x7fd4f015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f006b030 con 0x7fd4f015c220 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f015cbe0 con 0x7fd4f00cb130 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00b0ac0 con 0x7fd4f006a6a0 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4f015c220 0x7fd4f015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4f015c220 0x7fd4f015c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34536/0 (socket says 172.21.15.64:34536) 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/1586487082 learned_addr learned my addr 172.21.15.64:0/1586487082 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:55.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4ef7fe700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f006aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4effff700 1 --2- 172.21.15.64:0/1586487082 >> v2:172.21.15.123:3300/0 conn(0x7fd4f00cb130 0x7fd4f00cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/1586487082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 msgr2=0x7fd4f006aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f006aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/1586487082 >> v2:172.21.15.123:3300/0 conn(0x7fd4f00cb130 msgr2=0x7fd4f00cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 --2- 172.21.15.64:0/1586487082 >> v2:172.21.15.123:3300/0 conn(0x7fd4f00cb130 0x7fd4f00cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/1586487082 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f006b210 con 0x7fd4f015c220 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4ef7fe700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f006aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:31:55.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f4ae8700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f015c6a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd4f006b400 tx=0x7fd4e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7928d5ef6b492a41 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:55.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4eeffd700 1 -- 172.21.15.64:0/1586487082 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800e070 con 0x7fd4f015c220 2023-09-27T23:31:55.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4eeffd700 1 -- 172.21.15.64:0/1586487082 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e800ba10 con 0x7fd4f015c220 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4eeffd700 1 -- 172.21.15.64:0/1586487082 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800bdd0 con 0x7fd4f015c220 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- 172.21.15.64:0/1586487082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 msgr2=0x7fd4f015c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f015c6a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd4f006b400 tx=0x7fd4e8009840 comp rx=0 tx=0).stop 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- 172.21.15.64:0/1586487082 shutdown_connections 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f015c6a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- 172.21.15.64:0/1586487082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f006aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 --2- 172.21.15.64:0/1586487082 >> v2:172.21.15.123:3300/0 conn(0x7fd4f00cb130 0x7fd4f00cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- 172.21.15.64:0/1586487082 >> 172.21.15.64:0/1586487082 conn(0x7fd4f0067bb0 msgr2=0x7fd4f00cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- 172.21.15.64:0/1586487082 shutdown_connections 2023-09-27T23:31:55.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.517+0000 7fd4f654b700 1 -- 172.21.15.64:0/1586487082 wait complete. 2023-09-27T23:31:55.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 Processor -- start 2023-09-27T23:31:55.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 -- start start 2023-09-27T23:31:55.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f0169180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f00cb130 0x7fd4f014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f00c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0155530 con 0x7fd4f015c220 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f016a550 con 0x7fd4f00cb130 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f654b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01696c0 con 0x7fd4f006a6a0 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f00c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f00c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34552/0 (socket says 172.21.15.64:34552) 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/3191272665 learned_addr learned my addr 172.21.15.64:0/3191272665 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4effff700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f0169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4ef7fe700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f00cb130 0x7fd4f014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 msgr2=0x7fd4f0169180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f0169180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f00cb130 msgr2=0x7fd4f014aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.521+0000 7fd4f4ae8700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f00cb130 0x7fd4f014aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4f4ae8700 1 -- 172.21.15.64:0/3191272665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8009ae0 con 0x7fd4f015c220 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4f4ae8700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f00c57b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009ab0 tx=0x7fd4e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80032f0 con 0x7fd4f015c220 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e800be80 con 0x7fd4f015c220 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f0169940 con 0x7fd4f015c220 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e801e430 con 0x7fd4f015c220 2023-09-27T23:31:55.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f0169f00 con 0x7fd4f015c220 2023-09-27T23:31:55.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f015c050 con 0x7fd4f015c220 2023-09-27T23:31:55.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.525+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 48) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fd4e801e7a0 con 0x7fd4f015c220 2023-09-27T23:31:55.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.529+0000 7fd4e7fff700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd4d00607d0 0x7fd4d0062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:55.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.529+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fd4e807f200 con 0x7fd4f015c220 2023-09-27T23:31:55.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.529+0000 7fd4effff700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd4d00607d0 0x7fd4d0062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:55.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.529+0000 7fd4effff700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd4d00607d0 0x7fd4d0062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0008040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:55.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.533+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd4e804f370 con 0x7fd4f015c220 2023-09-27T23:31:55.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.697+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4f015c050 con 0x7fd4f015c220 2023-09-27T23:31:55.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:55.725+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd4e804f740 con 0x7fd4f015c220 2023-09-27T23:31:56.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.377+0000 7fd4e7fff700 1 -- 172.21.15.64:0/3191272665 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4e80648d0 con 0x7fd4f015c220 2023-09-27T23:31:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.385+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd4d00607d0 msgr2=0x7fd4d0062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.385+0000 7fd4f654b700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd4d00607d0 0x7fd4d0062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0008040 comp rx=0 tx=0).stop 2023-09-27T23:31:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.385+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 msgr2=0x7fd4f00c57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:56.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.385+0000 7fd4f654b700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f00c57b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009ab0 tx=0x7fd4e8017000 comp rx=0 tx=0).stop 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 shutdown_connections 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd4d00607d0 0x7fd4d0062c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4f015c220 0x7fd4f00c57b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f00cb130 0x7fd4f014aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 --2- 172.21.15.64:0/3191272665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f006a6a0 0x7fd4f0169180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 >> 172.21.15.64:0/3191272665 conn(0x7fd4f0067bb0 msgr2=0x7fd4f00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 shutdown_connections 2023-09-27T23:31:56.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.389+0000 7fd4f654b700 1 -- 172.21.15.64:0/3191272665 wait complete. 2023-09-27T23:31:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:31:56.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:31:56.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9958cc1700 1 Processor -- start 2023-09-27T23:31:56.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9958cc1700 1 -- start start 2023-09-27T23:31:56.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9958cc1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f995415c490 0x7f99541596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:56.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9958cc1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540add70 con 0x7f995415c860 2023-09-27T23:31:56.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9958cc1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9954159fa0 con 0x7f995406d760 2023-09-27T23:31:56.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9958cc1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995415a280 con 0x7f995415c490 2023-09-27T23:31:56.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f995259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f995415c490 0x7f99541596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:56.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9951d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f995415c860 0x7f995406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33418/0 (socket says 172.21.15.64:33418) 2023-09-27T23:31:56.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f9951d9b700 1 -- 172.21.15.64:0/3621187335 learned_addr learned my addr 172.21.15.64:0/3621187335 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:56.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f995259c700 1 -- 172.21.15.64:0/3621187335 >> v1:172.21.15.123:6789/0 conn(0x7f995406d760 legacy=0x7f99540ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:56.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f995259c700 1 -- 172.21.15.64:0/3621187335 >> v1:172.21.15.64:6789/0 conn(0x7f995415c860 legacy=0x7f995406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:56.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f995259c700 1 -- 172.21.15.64:0/3621187335 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995415a560 con 0x7f995415c490 2023-09-27T23:31:56.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.509+0000 7f995259c700 1 --2- 172.21.15.64:0/3621187335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f99541596b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9944009910 tx=0x7f99440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741b50aa0a2e0e7b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:56.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f995159a700 1 -- 172.21.15.64:0/3621187335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944009eb0 con 0x7f995415c490 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f995159a700 1 -- 172.21.15.64:0/3621187335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994400b760 con 0x7f995415c490 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f995159a700 1 -- 172.21.15.64:0/3621187335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400bb00 con 0x7f995415c490 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- 172.21.15.64:0/3621187335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 msgr2=0x7f99541596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3621187335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f99541596b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9944009910 tx=0x7f99440091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- 172.21.15.64:0/3621187335 shutdown_connections 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3621187335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f99541596b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- 172.21.15.64:0/3621187335 >> 172.21.15.64:0/3621187335 conn(0x7f99540ccc20 msgr2=0x7f99540cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- 172.21.15.64:0/3621187335 shutdown_connections 2023-09-27T23:31:56.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- 172.21.15.64:0/3621187335 wait complete. 2023-09-27T23:31:56.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 Processor -- start 2023-09-27T23:31:56.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- start start 2023-09-27T23:31:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f995406d760 0x7f99540ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f995414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995415c860 0x7f995416cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540a9950 con 0x7f995415c490 2023-09-27T23:31:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540ab6b0 con 0x7f995415c860 2023-09-27T23:31:56.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9958cc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995414eb60 con 0x7f995406d760 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.513+0000 7f9951d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f995414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f995414e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34568/0 (socket says 172.21.15.64:34568) 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 -- 172.21.15.64:0/3033909522 learned_addr learned my addr 172.21.15.64:0/3033909522 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f995259c700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f995406d760 0x7f99540ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9952d9d700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995415c860 0x7f995416cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 -- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f995406d760 msgr2=0x7f99540ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f995406d760 0x7f99540ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 -- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995415c860 msgr2=0x7f995416cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995415c860 0x7f995416cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 -- 172.21.15.64:0/3033909522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944009420 con 0x7f995415c490 2023-09-27T23:31:56.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9951d9b700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f995414e620 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f994800c840 tx=0x7f9948007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948005b30 con 0x7f995415c490 2023-09-27T23:31:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9948005c90 con 0x7f995415c490 2023-09-27T23:31:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99540b0d40 con 0x7f995415c490 2023-09-27T23:31:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948007320 con 0x7f995415c490 2023-09-27T23:31:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99540a9300 con 0x7f995415c490 2023-09-27T23:31:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99540a9ac0 con 0x7f995415c490 2023-09-27T23:31:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f994801b3b0 con 0x7f995415c490 2023-09-27T23:31:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f99427fc700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9934060aa0 0x7f9934062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6714+0+0 (secure 0 0 0) 0x7f994807c790 con 0x7f995415c490 2023-09-27T23:31:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.517+0000 7f995259c700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9934060aa0 0x7f9934062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.521+0000 7f995259c700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9934060aa0 0x7f9934062f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99440093f0 tx=0x7f9944016000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.521+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f994804c670 con 0x7f995415c490 2023-09-27T23:31:56.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:56.685+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9954071120 con 0x7f995415c490 2023-09-27T23:31:57.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.393+0000 7f99427fc700 1 -- 172.21.15.64:0/3033909522 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) v1 ==== 169+0+0 (secure 0 0 0) 0x7f994804fc90 con 0x7f995415c490 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9934060aa0 msgr2=0x7f9934062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9934060aa0 0x7f9934062f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99440093f0 tx=0x7f9944016000 comp rx=0 tx=0).stop 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 msgr2=0x7f995414e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f995414e620 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f994800c840 tx=0x7f9948007f30 comp rx=0 tx=0).stop 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 shutdown_connections 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9934060aa0 0x7f9934062f50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995415c860 0x7f995416cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f995415c490 0x7f995414e620 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 --2- 172.21.15.64:0/3033909522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f995406d760 0x7f99540ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 >> 172.21.15.64:0/3033909522 conn(0x7f99540ccc20 msgr2=0x7f9954069a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 shutdown_connections 2023-09-27T23:31:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.397+0000 7f9958cc1700 1 -- 172.21.15.64:0/3033909522 wait complete. 2023-09-27T23:31:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:31:57.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:31:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58929c1700 1 Processor -- start 2023-09-27T23:31:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58929c1700 1 -- start start 2023-09-27T23:31:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58929c1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f588c06baa0 0x7f588c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58929c1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c15ae30 con 0x7f588c0adab0 2023-09-27T23:31:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58929c1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0abf70 con 0x7f588c154780 2023-09-27T23:31:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58929c1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c154b50 con 0x7f588c06baa0 2023-09-27T23:31:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58921c0700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f588c0adab0 0x7f588c0adeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33428/0 (socket says 172.21.15.64:33428) 2023-09-27T23:31:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58921c0700 1 -- 172.21.15.64:0/1046982257 learned_addr learned my addr 172.21.15.64:0/1046982257 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58911be700 1 --2- 172.21.15.64:0/1046982257 >> v2:172.21.15.123:3300/0 conn(0x7f588c06baa0 0x7f588c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58909bd700 1 -- 172.21.15.64:0/1046982257 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35818551 0 0) 0x7f588c15ae30 con 0x7f588c0adab0 2023-09-27T23:31:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.525+0000 7f58909bd700 1 -- 172.21.15.64:0/1046982257 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5874003570 con 0x7f588c0adab0 2023-09-27T23:31:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58911be700 1 -- 172.21.15.64:0/1046982257 >> v1:172.21.15.123:6789/0 conn(0x7f588c154780 legacy=0x7f588c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58911be700 1 -- 172.21.15.64:0/1046982257 >> v1:172.21.15.64:6789/0 conn(0x7f588c0adab0 legacy=0x7f588c0adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58909bd700 1 -- 172.21.15.64:0/1046982257 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026199049 0 0) 0x7f5874003570 con 0x7f588c0adab0 2023-09-27T23:31:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58911be700 1 -- 172.21.15.64:0/1046982257 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c0ac210 con 0x7f588c06baa0 2023-09-27T23:31:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58911be700 1 --2- 172.21.15.64:0/1046982257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c06baa0 0x7f588c15b2c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5880009e60 tx=0x7f5880009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56159819ef3dac3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58909bd700 1 -- 172.21.15.64:0/1046982257 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000e070 con 0x7f588c06baa0 2023-09-27T23:31:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58909bd700 1 -- 172.21.15.64:0/1046982257 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588000b850 con 0x7f588c06baa0 2023-09-27T23:31:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58909bd700 1 -- 172.21.15.64:0/1046982257 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884001470 con 0x7f588c06baa0 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 -- 172.21.15.64:0/1046982257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c06baa0 msgr2=0x7f588c15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 --2- 172.21.15.64:0/1046982257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c06baa0 0x7f588c15b2c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5880009e60 tx=0x7f5880009780 comp rx=0 tx=0).stop 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 -- 172.21.15.64:0/1046982257 shutdown_connections 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 --2- 172.21.15.64:0/1046982257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c06baa0 0x7f588c15b2c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 -- 172.21.15.64:0/1046982257 >> 172.21.15.64:0/1046982257 conn(0x7f588c067b50 msgr2=0x7f588c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 -- 172.21.15.64:0/1046982257 shutdown_connections 2023-09-27T23:31:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 -- 172.21.15.64:0/1046982257 wait complete. 2023-09-27T23:31:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 Processor -- start 2023-09-27T23:31:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.529+0000 7f58929c1700 1 -- start start 2023-09-27T23:31:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 0x7f588c0c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c0adab0 0x7f588c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f588c154780 0x7f588c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0af7b0 con 0x7f588c06baa0 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c167fd0 con 0x7f588c0adab0 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0ae9f0 con 0x7f588c154780 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 0x7f588c0c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 0x7f588c0c1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34584/0 (socket says 172.21.15.64:34584) 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 -- 172.21.15.64:0/786305690 learned_addr learned my addr 172.21.15.64:0/786305690 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58911be700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c0adab0 0x7f588c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58921c0700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f588c154780 0x7f588c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 -- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f588c154780 msgr2=0x7f588c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f588c154780 0x7f588c0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 -- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c0adab0 msgr2=0x7f588c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c0adab0 0x7f588c0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 -- 172.21.15.64:0/786305690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5880009a20 con 0x7f588c06baa0 2023-09-27T23:31:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58919bf700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 0x7f588c0c1ba0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f587c00c4e0 tx=0x7f587c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c011590 con 0x7f588c06baa0 2023-09-27T23:31:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c011710 con 0x7f588c06baa0 2023-09-27T23:31:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c0aecd0 con 0x7f588c06baa0 2023-09-27T23:31:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c011ad0 con 0x7f588c06baa0 2023-09-27T23:31:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c0af290 con 0x7f588c06baa0 2023-09-27T23:31:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c06e620 con 0x7f588c06baa0 2023-09-27T23:31:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.533+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f587c01b3b0 con 0x7f588c06baa0 2023-09-27T23:31:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.537+0000 7f5889ffb700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5878060aa0 0x7f5878062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.537+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f587c07c5e0 con 0x7f588c06baa0 2023-09-27T23:31:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.537+0000 7f58911be700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5878060aa0 0x7f5878062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.537+0000 7f58911be700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5878060aa0 0x7f5878062f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f588c06be70 tx=0x7f5880017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:57.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.545+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f588c06e620 con 0x7f588c06baa0 2023-09-27T23:31:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.705+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f588c06e620 con 0x7f588c06baa0 2023-09-27T23:31:57.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f5889ffb700 1 -- 172.21.15.64:0/786305690 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v37) v1 ==== 115+0+0 (secure 0 0 0) 0x7f588c06e620 con 0x7f588c06baa0 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5878060aa0 msgr2=0x7f5878062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5878060aa0 0x7f5878062f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f588c06be70 tx=0x7f5880017000 comp rx=0 tx=0).stop 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 msgr2=0x7f588c0c1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 0x7f588c0c1ba0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f587c00c4e0 tx=0x7f587c00c7f0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 shutdown_connections 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5878060aa0 0x7f5878062f50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f588c154780 0x7f588c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f588c0adab0 0x7f588c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 --2- 172.21.15.64:0/786305690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c06baa0 0x7f588c0c1ba0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 >> 172.21.15.64:0/786305690 conn(0x7f588c067b50 msgr2=0x7f588c0c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:57.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 shutdown_connections 2023-09-27T23:31:57.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.717+0000 7f58929c1700 1 -- 172.21.15.64:0/786305690 wait complete. 2023-09-27T23:31:57.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:31:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 Processor -- start 2023-09-27T23:31:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 -- start start 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fddf8155cd0 0x7fddf80c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fddf81560a0 0x7fddf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf806d500 con 0x7fddf80ca2b0 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf806c8f0 con 0x7fddf8155cd0 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddfceba700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf806cbd0 con 0x7fddf81560a0 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fddf8155cd0 0x7fddf80c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf6d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fddf80ca2b0 0x7fddf80af400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33436/0 (socket says 172.21.15.64:33436) 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf6d9d700 1 -- 172.21.15.64:0/2279572961 learned_addr learned my addr 172.21.15.64:0/2279572961 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 --2- 172.21.15.64:0/2279572961 >> v2:172.21.15.64:3300/0 conn(0x7fddf8155cd0 0x7fddf80c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34590/0 (socket says 172.21.15.64:34590) 2023-09-27T23:31:57.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddeffff700 1 --2- 172.21.15.64:0/2279572961 >> v2:172.21.15.123:3300/0 conn(0x7fddf81560a0 0x7fddf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 -- 172.21.15.64:0/2279572961 >> v2:172.21.15.123:3300/0 conn(0x7fddf81560a0 msgr2=0x7fddf806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 --2- 172.21.15.64:0/2279572961 >> v2:172.21.15.123:3300/0 conn(0x7fddf81560a0 0x7fddf806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 -- 172.21.15.64:0/2279572961 >> v1:172.21.15.64:6789/0 conn(0x7fddf80ca2b0 legacy=0x7fddf80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf5d9b700 1 -- 172.21.15.64:0/2279572961 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48614547 0 0) 0x7fddf806d500 con 0x7fddf80ca2b0 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 -- 172.21.15.64:0/2279572961 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf806ceb0 con 0x7fddf8155cd0 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf659c700 1 --2- 172.21.15.64:0/2279572961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf80c9d70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fddf80af160 tx=0x7fdde8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05abc92c8bc2b85 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf5d9b700 1 -- 172.21.15.64:0/2279572961 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8005c10 con 0x7fddf8155cd0 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf5d9b700 1 -- 172.21.15.64:0/2279572961 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde8005d70 con 0x7fddf8155cd0 2023-09-27T23:31:57.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.845+0000 7fddf5d9b700 1 -- 172.21.15.64:0/2279572961 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8006260 con 0x7fddf8155cd0 2023-09-27T23:31:57.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- 172.21.15.64:0/2279572961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 msgr2=0x7fddf80c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 --2- 172.21.15.64:0/2279572961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf80c9d70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fddf80af160 tx=0x7fdde8009120 comp rx=0 tx=0).stop 2023-09-27T23:31:57.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- 172.21.15.64:0/2279572961 shutdown_connections 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 --2- 172.21.15.64:0/2279572961 >> v2:172.21.15.123:3300/0 conn(0x7fddf81560a0 0x7fddf806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 --2- 172.21.15.64:0/2279572961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf80c9d70 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- 172.21.15.64:0/2279572961 >> 172.21.15.64:0/2279572961 conn(0x7fddf8068b40 msgr2=0x7fddf8068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- 172.21.15.64:0/2279572961 shutdown_connections 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- 172.21.15.64:0/2279572961 wait complete. 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 Processor -- start 2023-09-27T23:31:57.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- start start 2023-09-27T23:31:57.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddf80ca2b0 0x7fddf8147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf8163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddf81560a0 0x7fddf81597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf8146ae0 con 0x7fddf8155cd0 2023-09-27T23:31:57.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf814a330 con 0x7fddf80ca2b0 2023-09-27T23:31:57.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddfceba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf814a610 con 0x7fddf81560a0 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf8163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf8163ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34594/0 (socket says 172.21.15.64:34594) 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddeffff700 1 -- 172.21.15.64:0/2951187265 learned_addr learned my addr 172.21.15.64:0/2951187265 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.849+0000 7fddf659c700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddf80ca2b0 0x7fddf8147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddf6d9d700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddf81560a0 0x7fddf81597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddeffff700 1 -- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddf81560a0 msgr2=0x7fddf81597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddeffff700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddf81560a0 0x7fddf81597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddeffff700 1 -- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddf80ca2b0 msgr2=0x7fddf8147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddeffff700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddf80ca2b0 0x7fddf8147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddeffff700 1 -- 172.21.15.64:0/2951187265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80093c0 con 0x7fddf8155cd0 2023-09-27T23:31:57.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddeffff700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf8163ed0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d860 tx=0x7fdde0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0009610 con 0x7fddf8155cd0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde0009770 con 0x7fddf8155cd0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf814a8f0 con 0x7fddf8155cd0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0009b30 con 0x7fddf8155cd0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf814aeb0 con 0x7fddf8155cd0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf8071120 con 0x7fddf8155cd0 2023-09-27T23:31:57.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fdde001e460 con 0x7fddf8155cd0 2023-09-27T23:31:57.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddee7fc700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddd8060a70 0x7fddd8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:57.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.853+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fdde007d750 con 0x7fddf8155cd0 2023-09-27T23:31:57.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.857+0000 7fddf659c700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddd8060a70 0x7fddd8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:57.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.857+0000 7fddf659c700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddd8060a70 0x7fddd8062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009160 tx=0x7fdde8016000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:57.861+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdde004d7d0 con 0x7fddf8155cd0 2023-09-27T23:31:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.037+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fddf806f4f0 con 0x7fddf8155cd0 2023-09-27T23:31:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddee7fc700 1 -- 172.21.15.64:0/2951187265 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v38) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdde0050df0 con 0x7fddf8155cd0 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddd8060a70 msgr2=0x7fddd8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddd8060a70 0x7fddd8062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009160 tx=0x7fdde8016000 comp rx=0 tx=0).stop 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 msgr2=0x7fddf8163ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf8163ed0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d860 tx=0x7fdde0008040 comp rx=0 tx=0).stop 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 shutdown_connections 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fddd8060a70 0x7fddd8062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddf81560a0 0x7fddf81597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddf8155cd0 0x7fddf8163ed0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 --2- 172.21.15.64:0/2951187265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddf80ca2b0 0x7fddf8147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 >> 172.21.15.64:0/2951187265 conn(0x7fddf8068b40 msgr2=0x7fddf8156670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 shutdown_connections 2023-09-27T23:31:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.045+0000 7fddfceba700 1 -- 172.21.15.64:0/2951187265 wait complete. 2023-09-27T23:31:58.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:31:58.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 Processor -- start 2023-09-27T23:31:58.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 -- start start 2023-09-27T23:31:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8d8153430 0x7ff8d815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff8d8153800 0x7ff8d8159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d806c850 con 0x7ff8d80ce790 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d815d1c0 con 0x7ff8d8153430 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8deda4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d815d4a0 con 0x7ff8d8153800 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8dcb40700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8d8153430 0x7ff8d815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8dd341700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff8d80ce790 0x7ff8d8067980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33440/0 (socket says 172.21.15.64:33440) 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.169+0000 7ff8dd341700 1 -- 172.21.15.64:0/3453852486 learned_addr learned my addr 172.21.15.64:0/3453852486 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8d7fff700 1 --2- 172.21.15.64:0/3453852486 >> v2:172.21.15.123:3300/0 conn(0x7ff8d8153800 0x7ff8d8159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8d77fe700 1 -- 172.21.15.64:0/3453852486 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484299041 0 0) 0x7ff8d806c850 con 0x7ff8d80ce790 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8d77fe700 1 -- 172.21.15.64:0/3453852486 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c4003570 con 0x7ff8d80ce790 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8dcb40700 1 -- 172.21.15.64:0/3453852486 >> v2:172.21.15.123:3300/0 conn(0x7ff8d8153800 msgr2=0x7ff8d8159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8dcb40700 1 --2- 172.21.15.64:0/3453852486 >> v2:172.21.15.123:3300/0 conn(0x7ff8d8153800 0x7ff8d8159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8dcb40700 1 -- 172.21.15.64:0/3453852486 >> v1:172.21.15.64:6789/0 conn(0x7ff8d80ce790 legacy=0x7ff8d8067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8dcb40700 1 -- 172.21.15.64:0/3453852486 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d815d780 con 0x7ff8d8153430 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8dcb40700 1 --2- 172.21.15.64:0/3453852486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d815cbe0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80cac50 tx=0x7ff8c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7da06f43697e9f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8d77fe700 1 -- 172.21.15.64:0/3453852486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8011630 con 0x7ff8d8153430 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8d77fe700 1 -- 172.21.15.64:0/3453852486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c8011790 con 0x7ff8d8153430 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8d77fe700 1 -- 172.21.15.64:0/3453852486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8011b30 con 0x7ff8d8153430 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 -- 172.21.15.64:0/3453852486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 msgr2=0x7ff8d815cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 --2- 172.21.15.64:0/3453852486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d815cbe0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80cac50 tx=0x7ff8c80091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 -- 172.21.15.64:0/3453852486 shutdown_connections 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 --2- 172.21.15.64:0/3453852486 >> v2:172.21.15.123:3300/0 conn(0x7ff8d8153800 0x7ff8d8159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 --2- 172.21.15.64:0/3453852486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d815cbe0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 -- 172.21.15.64:0/3453852486 >> 172.21.15.64:0/3453852486 conn(0x7ff8d80cd720 msgr2=0x7ff8d806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:58.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 -- 172.21.15.64:0/3453852486 shutdown_connections 2023-09-27T23:31:58.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.173+0000 7ff8deda4700 1 -- 172.21.15.64:0/3453852486 wait complete. 2023-09-27T23:31:58.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 Processor -- start 2023-09-27T23:31:58.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 -- start start 2023-09-27T23:31:58.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8d80ce790 0x7ff8d816d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8d8153800 0x7ff8d815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d816ccb0 con 0x7ff8d8153430 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80b1b30 con 0x7ff8d80ce790 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8deda4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d816c660 con 0x7ff8d8153800 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d80b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d80b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34614/0 (socket says 172.21.15.64:34614) 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8d7fff700 1 -- 172.21.15.64:0/2547963460 learned_addr learned my addr 172.21.15.64:0/2547963460 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8dd341700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8d8153800 0x7ff8d815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.177+0000 7ff8dcb40700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8d80ce790 0x7ff8d816d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d7fff700 1 -- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8d8153800 msgr2=0x7ff8d815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d7fff700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8d8153800 0x7ff8d815e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d7fff700 1 -- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8d80ce790 msgr2=0x7ff8d816d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d7fff700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8d80ce790 0x7ff8d816d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d7fff700 1 -- 172.21.15.64:0/2547963460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009420 con 0x7ff8d8153430 2023-09-27T23:31:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d7fff700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d80b11c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e930 tx=0x7ff8c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0013680 con 0x7ff8d8153430 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00137e0 con 0x7ff8d8153430 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d816c940 con 0x7ff8d8153430 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0013b80 con 0x7ff8d8153430 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d80b1700 con 0x7ff8d8153430 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d80690b0 con 0x7ff8d8153430 2023-09-27T23:31:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff8c000c3e0 con 0x7ff8d8153430 2023-09-27T23:31:58.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.181+0000 7ff8d4ff9700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff8b8060a70 0x7ff8b8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.185+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7ff8c007d280 con 0x7ff8d8153430 2023-09-27T23:31:58.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.185+0000 7ff8dcb40700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff8b8060a70 0x7ff8b8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.185+0000 7ff8dcb40700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff8b8060a70 0x7ff8b8062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80cac50 tx=0x7ff8c8017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:58.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.193+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff8d80690b0 con 0x7ff8d8153430 2023-09-27T23:31:58.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.357+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff8d80690b0 con 0x7ff8d8153430 2023-09-27T23:31:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.361+0000 7ff8d4ff9700 1 -- 172.21.15.64:0/2547963460 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff8d80690b0 con 0x7ff8d8153430 2023-09-27T23:31:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff8b8060a70 msgr2=0x7ff8b8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff8b8060a70 0x7ff8b8062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80cac50 tx=0x7ff8c8017000 comp rx=0 tx=0).stop 2023-09-27T23:31:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 msgr2=0x7ff8d80b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d80b11c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e930 tx=0x7ff8c00091a0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 shutdown_connections 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff8b8060a70 0x7ff8b8062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8d8153800 0x7ff8d815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d8153430 0x7ff8d80b11c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 --2- 172.21.15.64:0/2547963460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8d80ce790 0x7ff8d816d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 >> 172.21.15.64:0/2547963460 conn(0x7ff8d80cd720 msgr2=0x7ff8d80cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 shutdown_connections 2023-09-27T23:31:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.365+0000 7ff8deda4700 1 -- 172.21.15.64:0/2547963460 wait complete. 2023-09-27T23:31:58.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-09-27T23:31:58.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.493+0000 7f521b59e700 1 Processor -- start 2023-09-27T23:31:58.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.493+0000 7f521b59e700 1 -- start start 2023-09-27T23:31:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f521c06b480 0x7f521c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c0ca8c0 con 0x7f521c156b40 2023-09-27T23:31:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c0cb700 con 0x7f521c156770 2023-09-27T23:31:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c156fa0 con 0x7f521c06b480 2023-09-27T23:31:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5213fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f521c156b40 0x7f521c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33454/0 (socket says 172.21.15.64:33454) 2023-09-27T23:31:58.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5213fff700 1 -- 172.21.15.64:0/3992410166 learned_addr learned my addr 172.21.15.64:0/3992410166 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 --2- 172.21.15.64:0/3992410166 >> v2:172.21.15.123:3300/0 conn(0x7f521c06b480 0x7f521c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893222622 0 0) 0x7f521c0ca8c0 con 0x7f521c156b40 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51fc003570 con 0x7f521c156b40 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1400437693 0 0) 0x7f51fc003570 con 0x7f521c156b40 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 >> v2:172.21.15.123:3300/0 conn(0x7f521c06b480 msgr2=0x7f521c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 --2- 172.21.15.64:0/3992410166 >> v2:172.21.15.123:3300/0 conn(0x7f521c06b480 0x7f521c161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 >> v1:172.21.15.123:6789/0 conn(0x7f521c156770 legacy=0x7f521c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c06bd20 con 0x7f521c156b40 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f5204001470 con 0x7f521c156b40 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 >> v1:172.21.15.64:6789/0 conn(0x7f521c156b40 legacy=0x7f521c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51fc003a60 0x7f51fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51fc004810 0x7f51fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f51fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c156fa0 con 0x7f51fc0051c0 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc00c160 con 0x7f51fc004810 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc00c320 con 0x7f51fc003a60 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f51fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5213fff700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51fc004810 0x7f51fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521a59c700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51fc003a60 0x7f51fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 -- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51fc003a60 msgr2=0x7f51fc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51fc003a60 0x7f51fc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 -- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51fc004810 msgr2=0x7f51fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51fc004810 0x7f51fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 -- 172.21.15.64:0/3992410166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc003570 con 0x7f51fc0051c0 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 -- 172.21.15.64:0/3992410166 reap_dead start 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521ad9d700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f51fc00ba60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f521400e210 tx=0x7f521400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25807c699709d99 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400eac0 con 0x7f51fc0051c0 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400f040 con 0x7f51fc0051c0 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f5219d9b700 1 -- 172.21.15.64:0/3992410166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400ee70 con 0x7f51fc0051c0 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- 172.21.15.64:0/3992410166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 msgr2=0x7f51fc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f51fc00ba60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f521400e210 tx=0x7f521400b040 comp rx=0 tx=0).stop 2023-09-27T23:31:58.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- 172.21.15.64:0/3992410166 shutdown_connections 2023-09-27T23:31:58.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 --2- 172.21.15.64:0/3992410166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f51fc00ba60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- 172.21.15.64:0/3992410166 >> 172.21.15.64:0/3992410166 conn(0x7f521c067bb0 msgr2=0x7f521c0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:58.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- 172.21.15.64:0/3992410166 shutdown_connections 2023-09-27T23:31:58.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.497+0000 7f521b59e700 1 -- 172.21.15.64:0/3992410166 wait complete. 2023-09-27T23:31:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 Processor -- start 2023-09-27T23:31:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 -- start start 2023-09-27T23:31:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f521c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c158480 0x7f521c146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f521c0c7890 0x7f521c0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c159520 con 0x7f51fc0051c0 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c158910 con 0x7f521c158480 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c158bf0 con 0x7f521c0c7890 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f521c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f521c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34640/0 (socket says 172.21.15.64:34640) 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.501+0000 7f521a59c700 1 -- 172.21.15.64:0/3180587056 learned_addr learned my addr 172.21.15.64:0/3180587056 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f5213fff700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c158480 0x7f521c146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521ad9d700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f521c0c7890 0x7f521c0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521a59c700 1 -- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f521c0c7890 msgr2=0x7f521c0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521a59c700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f521c0c7890 0x7f521c0c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521a59c700 1 -- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c158480 msgr2=0x7f521c146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521a59c700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c158480 0x7f521c146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521a59c700 1 -- 172.21.15.64:0/3180587056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c158ed0 con 0x7f51fc0051c0 2023-09-27T23:31:58.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521a59c700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f521c1599b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f520c00e580 tx=0x7f520c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c002470 con 0x7f51fc0051c0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c010040 con 0x7f51fc0051c0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c1591b0 con 0x7f51fc0051c0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c002800 con 0x7f51fc0051c0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0c7cf0 con 0x7f51fc0051c0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c04be40 con 0x7f51fc0051c0 2023-09-27T23:31:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.505+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f520c002b70 con 0x7f51fc0051c0 2023-09-27T23:31:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.509+0000 7f52127fc700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5200060a70 0x7f5200062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.509+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f520c07c660 con 0x7f51fc0051c0 2023-09-27T23:31:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.509+0000 7f5213fff700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5200060a70 0x7f5200062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.509+0000 7f5213fff700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5200060a70 0x7f5200062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f521c16a6e0 tx=0x7f5204001e50 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:58.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.513+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f520c04bbb0 con 0x7f51fc0051c0 2023-09-27T23:31:58.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:58.733+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f521c0caaf0 con 0x7f51fc0051c0 2023-09-27T23:31:59.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.401+0000 7f52127fc700 1 -- 172.21.15.64:0/3180587056 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v680) v1 ==== 183+0+0 (secure 0 0 0) 0x7f520c050110 con 0x7f51fc0051c0 2023-09-27T23:31:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5200060a70 msgr2=0x7f5200062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5200060a70 0x7f5200062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f521c16a6e0 tx=0x7f5204001e50 comp rx=0 tx=0).stop 2023-09-27T23:31:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 msgr2=0x7f521c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f521c1599b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f520c00e580 tx=0x7f520c009040 comp rx=0 tx=0).stop 2023-09-27T23:31:59.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 shutdown_connections 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5200060a70 0x7f5200062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f521c0c7890 0x7f521c0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c158480 0x7f521c146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 --2- 172.21.15.64:0/3180587056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51fc0051c0 0x7f521c1599b0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 >> 172.21.15.64:0/3180587056 conn(0x7f521c067bb0 msgr2=0x7f521c0ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 shutdown_connections 2023-09-27T23:31:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.405+0000 7f521b59e700 1 -- 172.21.15.64:0/3180587056 wait complete. 2023-09-27T23:31:59.425 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-09-27T23:31:59.425 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-09-27T23:31:59.425 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:31:59.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-09-27T23:31:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.529+0000 7fa2bd032700 1 Processor -- start 2023-09-27T23:31:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.529+0000 7fa2bd032700 1 -- start start 2023-09-27T23:31:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2b813fb40 0x7fa2b8159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80c22e0 con 0x7fa2b815a690 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80cc8e0 con 0x7fa2b8159c10 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b80cc6d0 con 0x7fa2b813fb40 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2b813fb40 0x7fa2b8159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2affff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa2b815a690 0x7fa2b80c2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33458/0 (socket says 172.21.15.64:33458) 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2b813fb40 0x7fa2b8159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34650/0 (socket says 172.21.15.64:34650) 2023-09-27T23:31:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2affff700 1 -- 172.21.15.64:0/1081837336 learned_addr learned my addr 172.21.15.64:0/1081837336 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:31:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b659c700 1 -- 172.21.15.64:0/1081837336 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925776573 0 0) 0x7fa2b80cc8e0 con 0x7fa2b8159c10 2023-09-27T23:31:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b759e700 1 -- 172.21.15.64:0/1081837336 >> v1:172.21.15.123:6789/0 conn(0x7fa2b8159c10 legacy=0x7fa2b813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b759e700 1 -- 172.21.15.64:0/1081837336 >> v1:172.21.15.64:6789/0 conn(0x7fa2b815a690 legacy=0x7fa2b80c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b759e700 1 -- 172.21.15.64:0/1081837336 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80ccbc0 con 0x7fa2b813fb40 2023-09-27T23:31:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b759e700 1 --2- 172.21.15.64:0/1081837336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b8159fe0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009e60 tx=0x7fa2b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5210254dfc4a74 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:59.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b659c700 1 -- 172.21.15.64:0/1081837336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000e070 con 0x7fa2b813fb40 2023-09-27T23:31:59.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b659c700 1 -- 172.21.15.64:0/1081837336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b000bb30 con 0x7fa2b813fb40 2023-09-27T23:31:59.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2b659c700 1 -- 172.21.15.64:0/1081837336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000bed0 con 0x7fa2b813fb40 2023-09-27T23:31:59.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- 172.21.15.64:0/1081837336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 msgr2=0x7fa2b8159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1081837336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b8159fe0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009e60 tx=0x7fa2b0009780 comp rx=0 tx=0).stop 2023-09-27T23:31:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- 172.21.15.64:0/1081837336 shutdown_connections 2023-09-27T23:31:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1081837336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b8159fe0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- 172.21.15.64:0/1081837336 >> 172.21.15.64:0/1081837336 conn(0x7fa2b80cd9c0 msgr2=0x7fa2b80c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:31:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- 172.21.15.64:0/1081837336 shutdown_connections 2023-09-27T23:31:59.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.533+0000 7fa2bd032700 1 -- 172.21.15.64:0/1081837336 wait complete. 2023-09-27T23:31:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 Processor -- start 2023-09-27T23:31:59.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 -- start start 2023-09-27T23:31:59.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b80ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2b8159c10 0x7fa2b813e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b815a690 0x7fa2b80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b813ec80 con 0x7fa2b813fb40 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80ae990 con 0x7fa2b815a690 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2bd032700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80add80 con 0x7fa2b8159c10 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2b6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b80ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2b6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b80ad6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34660/0 (socket says 172.21.15.64:34660) 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2b6d9d700 1 -- 172.21.15.64:0/1836572112 learned_addr learned my addr 172.21.15.64:0/1836572112 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2affff700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2b8159c10 0x7fa2b813e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:59.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.537+0000 7fa2b759e700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b815a690 0x7fa2b80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2b6d9d700 1 -- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2b8159c10 msgr2=0x7fa2b813e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2b6d9d700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2b8159c10 0x7fa2b813e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2b6d9d700 1 -- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b815a690 msgr2=0x7fa2b80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2b6d9d700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b815a690 0x7fa2b80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2b6d9d700 1 -- 172.21.15.64:0/1836572112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0009a20 con 0x7fa2b813fb40 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2b6d9d700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b80ad6a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800c5e0 tx=0x7fa2a8007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011590 con 0x7fa2b813fb40 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a80116f0 con 0x7fa2b813fb40 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011ab0 con 0x7fa2b813fb40 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b80ae060 con 0x7fa2b813fb40 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b80ae620 con 0x7fa2b813fb40 2023-09-27T23:31:59.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b8071120 con 0x7fa2b813fb40 2023-09-27T23:31:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.541+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa2a801b3b0 con 0x7fa2b813fb40 2023-09-27T23:31:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.545+0000 7fa2aeffd700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa298060a70 0x7fa298062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:31:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.545+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fa2a807c590 con 0x7fa2b813fb40 2023-09-27T23:31:59.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.545+0000 7fa2affff700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa298060a70 0x7fa298062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:31:59.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.545+0000 7fa2affff700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa298060a70 0x7fa298062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80b2bd0 tx=0x7fa2a0009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:31:59.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.549+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa2a804c610 con 0x7fa2b813fb40 2023-09-27T23:31:59.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:31:59.773+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fa2b80c1c90 con 0x7fa2b813fb40 2023-09-27T23:32:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.401+0000 7fa2aeffd700 1 -- 172.21.15.64:0/1836572112 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v681) v1 ==== 151+0+0 (secure 0 0 0) 0x7fa2a80170d0 con 0x7fa2b813fb40 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa298060a70 msgr2=0x7fa298062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa298060a70 0x7fa298062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80b2bd0 tx=0x7fa2a0009040 comp rx=0 tx=0).stop 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 msgr2=0x7fa2b80ad6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b80ad6a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800c5e0 tx=0x7fa2a8007e60 comp rx=0 tx=0).stop 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 shutdown_connections 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa298060a70 0x7fa298062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b815a690 0x7fa2b80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2b8159c10 0x7fa2b813e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 --2- 172.21.15.64:0/1836572112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b813fb40 0x7fa2b80ad6a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 >> 172.21.15.64:0/1836572112 conn(0x7fa2b80cd9c0 msgr2=0x7fa2b8067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 shutdown_connections 2023-09-27T23:32:00.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.409+0000 7fa2bd032700 1 -- 172.21.15.64:0/1836572112 wait complete. 2023-09-27T23:32:00.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:32:00.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.545+0000 7f81c5b79700 1 Processor -- start 2023-09-27T23:32:00.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- start start 2023-09-27T23:32:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f81c00cb970 0x7f81c006acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81c00cecd0 0x7f81c006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c00cbdf0 con 0x7f81c0158e20 2023-09-27T23:32:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c00cc0d0 con 0x7f81c00cecd0 2023-09-27T23:32:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c00cc3b0 con 0x7f81c00cb970 2023-09-27T23:32:00.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81c00cecd0 0x7f81c006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bf7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f81c0158e20 0x7f81c00ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33460/0 (socket says 172.21.15.64:33460) 2023-09-27T23:32:00.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81c00cecd0 0x7f81c006b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34674/0 (socket says 172.21.15.64:34674) 2023-09-27T23:32:00.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bf7fe700 1 -- 172.21.15.64:0/214213348 learned_addr learned my addr 172.21.15.64:0/214213348 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:00.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81beffd700 1 --2- 172.21.15.64:0/214213348 >> v2:172.21.15.123:3300/0 conn(0x7f81c00cb970 0x7f81c006acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 -- 172.21.15.64:0/214213348 >> v2:172.21.15.123:3300/0 conn(0x7f81c00cb970 msgr2=0x7f81c006acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:00.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 --2- 172.21.15.64:0/214213348 >> v2:172.21.15.123:3300/0 conn(0x7f81c00cb970 0x7f81c006acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 -- 172.21.15.64:0/214213348 >> v1:172.21.15.64:6789/0 conn(0x7f81c0158e20 legacy=0x7f81c00ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81be7fc700 1 -- 172.21.15.64:0/214213348 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473220692 0 0) 0x7f81c00cbdf0 con 0x7f81c0158e20 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 -- 172.21.15.64:0/214213348 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00cc690 con 0x7f81c00cecd0 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81bffff700 1 --2- 172.21.15.64:0/214213348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c00cecd0 0x7f81c006b200 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009e60 tx=0x7f81b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567274840eb6996d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81be7fc700 1 -- 172.21.15.64:0/214213348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b800e070 con 0x7f81c00cecd0 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81be7fc700 1 -- 172.21.15.64:0/214213348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b800bb20 con 0x7f81c00cecd0 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81be7fc700 1 -- 172.21.15.64:0/214213348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b8012430 con 0x7f81c00cecd0 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- 172.21.15.64:0/214213348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c00cecd0 msgr2=0x7f81c006b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 --2- 172.21.15.64:0/214213348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c00cecd0 0x7f81c006b200 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009e60 tx=0x7f81b8009780 comp rx=0 tx=0).stop 2023-09-27T23:32:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- 172.21.15.64:0/214213348 shutdown_connections 2023-09-27T23:32:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 --2- 172.21.15.64:0/214213348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c00cecd0 0x7f81c006b200 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 --2- 172.21.15.64:0/214213348 >> v2:172.21.15.123:3300/0 conn(0x7f81c00cb970 0x7f81c006acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- 172.21.15.64:0/214213348 >> 172.21.15.64:0/214213348 conn(0x7f81c0067bb0 msgr2=0x7f81c00d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- 172.21.15.64:0/214213348 shutdown_connections 2023-09-27T23:32:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.549+0000 7f81c5b79700 1 -- 172.21.15.64:0/214213348 wait complete. 2023-09-27T23:32:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 Processor -- start 2023-09-27T23:32:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 -- start start 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81c00cecd0 0x7f81c014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 0x7f81c0150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81c0150e70 0x7f81c0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c7230 con 0x7f81c0158e20 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c7510 con 0x7f81c0150e70 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81c5b79700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00c77f0 con 0x7f81c00cecd0 2023-09-27T23:32:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 0x7f81c0150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 0x7f81c0150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34676/0 (socket says 172.21.15.64:34676) 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 -- 172.21.15.64:0/2744405846 learned_addr learned my addr 172.21.15.64:0/2744405846 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81bffff700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81c0150e70 0x7f81c0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81bf7fe700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81c00cecd0 0x7f81c014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 -- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81c00cecd0 msgr2=0x7f81c014cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81c00cecd0 0x7f81c014cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 -- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81c0150e70 msgr2=0x7f81c0152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81c0150e70 0x7f81c0152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 -- 172.21.15.64:0/2744405846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b8009a20 con 0x7f81c0158e20 2023-09-27T23:32:00.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81bf7fe700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81c00cecd0 0x7f81c014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.553+0000 7f81beffd700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 0x7f81c0150930 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e900 tx=0x7f81b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000eea0 con 0x7f81c0158e20 2023-09-27T23:32:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000c850 con 0x7f81c0158e20 2023-09-27T23:32:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c00c7ad0 con 0x7f81c0158e20 2023-09-27T23:32:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000cc10 con 0x7f81c0158e20 2023-09-27T23:32:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c014d0f0 con 0x7f81c0158e20 2023-09-27T23:32:00.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c00cf510 con 0x7f81c0158e20 2023-09-27T23:32:00.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f81b0013470 con 0x7f81c0158e20 2023-09-27T23:32:00.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81b77fe700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81a0060a70 0x7f81a0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f81b007d2d0 con 0x7f81c0158e20 2023-09-27T23:32:00.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.557+0000 7f81bf7fe700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81a0060a70 0x7f81a0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.561+0000 7f81bf7fe700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81a0060a70 0x7f81a0062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f81a80018d0 tx=0x7f81a8009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:00.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.565+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f81c00cf510 con 0x7f81c0158e20 2023-09-27T23:32:00.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.729+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f81c00cf510 con 0x7f81c0158e20 2023-09-27T23:32:00.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.737+0000 7f81b77fe700 1 -- 172.21.15.64:0/2744405846 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81c00cf510 con 0x7f81c0158e20 2023-09-27T23:32:00.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81a0060a70 msgr2=0x7f81a0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81a0060a70 0x7f81a0062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f81a80018d0 tx=0x7f81a8009040 comp rx=0 tx=0).stop 2023-09-27T23:32:00.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 msgr2=0x7f81c0150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 0x7f81c0150930 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e900 tx=0x7f81b00091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 shutdown_connections 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f81a0060a70 0x7f81a0062f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81c0150e70 0x7f81c0152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c0158e20 0x7f81c0150930 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 --2- 172.21.15.64:0/2744405846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81c00cecd0 0x7f81c014cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 >> 172.21.15.64:0/2744405846 conn(0x7f81c0067bb0 msgr2=0x7f81c006abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 shutdown_connections 2023-09-27T23:32:00.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.741+0000 7f81c5b79700 1 -- 172.21.15.64:0/2744405846 wait complete. 2023-09-27T23:32:00.760 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:32:00.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:32:00.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:32:00.794 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:32:00.794 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:32:00.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:32:00.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.901+0000 7fa891039700 1 Processor -- start 2023-09-27T23:32:00.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.901+0000 7fa891039700 1 -- start start 2023-09-27T23:32:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.901+0000 7fa891039700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c13fb40 0x7fa88c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.901+0000 7fa891039700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0c22e0 con 0x7fa88c159c10 2023-09-27T23:32:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.901+0000 7fa891039700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0cc8e0 con 0x7fa88c15a690 2023-09-27T23:32:00.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.901+0000 7fa891039700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0cc6d0 con 0x7fa88c13fb40 2023-09-27T23:32:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88ad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa88c159c10 0x7fa88c13f430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33470/0 (socket says 172.21.15.64:33470) 2023-09-27T23:32:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88ad9d700 1 -- 172.21.15.64:0/1570467755 learned_addr learned my addr 172.21.15.64:0/1570467755 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88b59e700 1 --2- 172.21.15.64:0/1570467755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c13fb40 0x7fa88c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88a59c700 1 -- 172.21.15.64:0/1570467755 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122385175 0 0) 0x7fa88c0c22e0 con 0x7fa88c159c10 2023-09-27T23:32:00.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88a59c700 1 -- 172.21.15.64:0/1570467755 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa878003570 con 0x7fa88c159c10 2023-09-27T23:32:00.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88b59e700 1 -- 172.21.15.64:0/1570467755 >> v1:172.21.15.123:6789/0 conn(0x7fa88c15a690 legacy=0x7fa88c0c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88a59c700 1 -- 172.21.15.64:0/1570467755 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016782613 0 0) 0x7fa88c0cc8e0 con 0x7fa88c15a690 2023-09-27T23:32:00.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88b59e700 1 -- 172.21.15.64:0/1570467755 >> v1:172.21.15.64:6789/0 conn(0x7fa88c159c10 legacy=0x7fa88c13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88b59e700 1 -- 172.21.15.64:0/1570467755 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0ccbc0 con 0x7fa88c13fb40 2023-09-27T23:32:00.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88b59e700 1 --2- 172.21.15.64:0/1570467755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c13fb40 0x7fa88c159fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa884009e90 tx=0x7fa884009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bcc824cefccd87d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88a59c700 1 -- 172.21.15.64:0/1570467755 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400e070 con 0x7fa88c13fb40 2023-09-27T23:32:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88a59c700 1 -- 172.21.15.64:0/1570467755 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88400bc30 con 0x7fa88c13fb40 2023-09-27T23:32:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa88a59c700 1 -- 172.21.15.64:0/1570467755 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884012430 con 0x7fa88c13fb40 2023-09-27T23:32:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 -- 172.21.15.64:0/1570467755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c13fb40 msgr2=0x7fa88c159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 --2- 172.21.15.64:0/1570467755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c13fb40 0x7fa88c159fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa884009e90 tx=0x7fa884009780 comp rx=0 tx=0).stop 2023-09-27T23:32:00.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 -- 172.21.15.64:0/1570467755 shutdown_connections 2023-09-27T23:32:00.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 --2- 172.21.15.64:0/1570467755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c13fb40 0x7fa88c159fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 -- 172.21.15.64:0/1570467755 >> 172.21.15.64:0/1570467755 conn(0x7fa88c0cd9c0 msgr2=0x7fa88c0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:00.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 -- 172.21.15.64:0/1570467755 shutdown_connections 2023-09-27T23:32:00.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.905+0000 7fa891039700 1 -- 172.21.15.64:0/1570467755 wait complete. 2023-09-27T23:32:00.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 Processor -- start 2023-09-27T23:32:00.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 -- start start 2023-09-27T23:32:00.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c13fb40 0x7fa88c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c159c10 0x7fa88c0adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 0x7fa88c16c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0ae160 con 0x7fa88c15a690 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0ae350 con 0x7fa88c13fb40 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa891039700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0ae630 con 0x7fa88c159c10 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 0x7fa88c16c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 0x7fa88c16c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34688/0 (socket says 172.21.15.64:34688) 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c13fb40 0x7fa88c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 -- 172.21.15.64:0/1555740774 learned_addr learned my addr 172.21.15.64:0/1555740774 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa883fff700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c159c10 0x7fa88c0adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 -- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c159c10 msgr2=0x7fa88c0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c159c10 0x7fa88c0adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 -- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c13fb40 msgr2=0x7fa88c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c13fb40 0x7fa88c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 -- 172.21.15.64:0/1555740774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa884009a20 con 0x7fa88c15a690 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88ad9d700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c13fb40 0x7fa88c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa88b59e700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 0x7fa88c16c3b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa8840099f0 tx=0x7fa884019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.909+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884027470 con 0x7fa88c15a690 2023-09-27T23:32:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c0ae910 con 0x7fa88c15a690 2023-09-27T23:32:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884027610 con 0x7fa88c15a690 2023-09-27T23:32:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840279d0 con 0x7fa88c15a690 2023-09-27T23:32:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c0aeed0 con 0x7fa88c15a690 2023-09-27T23:32:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c169690 con 0x7fa88c15a690 2023-09-27T23:32:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa884024070 con 0x7fa88c15a690 2023-09-27T23:32:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa882ffd700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa86c060a70 0x7fa86c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fa88407f840 con 0x7fa88c15a690 2023-09-27T23:32:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.913+0000 7fa88ad9d700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa86c060a70 0x7fa86c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.917+0000 7fa88ad9d700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa86c060a70 0x7fa86c062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa87c007fd0 tx=0x7fa87c009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:00.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:00.921+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa88c169690 con 0x7fa88c15a690 2023-09-27T23:32:01.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa88c169660 con 0x7fa88c15a690 2023-09-27T23:32:01.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa882ffd700 1 -- 172.21.15.64:0/1555740774 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fa88402c070 con 0x7fa88c15a690 2023-09-27T23:32:01.129 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:01.129 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3704265371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3704265371},{"type":"v1","addr":"172.21.15.123:6837","nonce":3704265371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1415600633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1415600633},{"type":"v1","addr":"172.21.15.123:6835","nonce":1415600633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7599,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2770860021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2770860021},{"type":"v1","addr":"172.21.15.64:6833","nonce":2770860021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1424559867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1424559867},{"type":"v1","addr":"172.21.15.64:6835","nonce":1424559867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[]} 2023-09-27T23:32:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa86c060a70 msgr2=0x7fa86c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa86c060a70 0x7fa86c062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa87c007fd0 tx=0x7fa87c009040 comp rx=0 tx=0).stop 2023-09-27T23:32:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 msgr2=0x7fa88c16c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 0x7fa88c16c3b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa8840099f0 tx=0x7fa884019000 comp rx=0 tx=0).stop 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 shutdown_connections 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa86c060a70 0x7fa86c062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa88c15a690 0x7fa88c16c3b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa88c159c10 0x7fa88c0adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 --2- 172.21.15.64:0/1555740774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa88c13fb40 0x7fa88c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 >> 172.21.15.64:0/1555740774 conn(0x7fa88c0cd9c0 msgr2=0x7fa88c0b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 shutdown_connections 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.125+0000 7fa891039700 1 -- 172.21.15.64:0/1555740774 wait complete. 2023-09-27T23:32:01.133 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 41 2023-09-27T23:32:01.144 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:32:01.144 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:32:01.145 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:32:01.145 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:32:01.145 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:32:01.145 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:32:01.145 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:32:01.145 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:32:01.146 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:32:01.146 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:32:01.146 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:32:01.146 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:32:01.146 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:32:01.146 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:32:01.146 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:32:01.147 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:32:01.155 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:32:01.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:32:01.159 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:32:01.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:32:01.161 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:32:01.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:32:01.162 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:32:01.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:32:01.163 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:32:01.164 DEBUG:teuthology.parallel:result is None 2023-09-27T23:32:01.164 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:32:01.165 DEBUG:teuthology.parallel:result is None 2023-09-27T23:32:01.201 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:32:01.202 DEBUG:teuthology.parallel:result is None 2023-09-27T23:32:01.203 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:32:01.203 DEBUG:teuthology.parallel:result is None 2023-09-27T23:32:01.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:32:01.249 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:32:01.250 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:32:01.279 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:32:01.292 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:32:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ada0aa700 1 Processor -- start 2023-09-27T23:32:01.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ada0aa700 1 -- start start 2023-09-27T23:32:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ada0aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad4156ec0 0x7f2ad406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ada0aa700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad406d500 con 0x7f2ad406ded0 2023-09-27T23:32:01.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ada0aa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad406c8f0 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ada0aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad406cbd0 con 0x7f2ad4156ec0 2023-09-27T23:32:01.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad2ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2ad406ded0 0x7f2ad40c9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33474/0 (socket says 172.21.15.64:33474) 2023-09-27T23:32:01.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/1526045358 learned_addr learned my addr 172.21.15.64:0/1526045358 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:01.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/1526045358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad4156ec0 0x7f2ad406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:01.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750574164 0 0) 0x7f2ad406d500 con 0x7f2ad406ded0 2023-09-27T23:32:01.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2abc003570 con 0x7f2ad406ded0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627727698 0 0) 0x7f2ad406c8f0 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad406d500 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 816921588 0 0) 0x7f2abc003570 con 0x7f2ad406ded0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad37fe700 1 -- 172.21.15.64:0/1526045358 >> v1:172.21.15.123:6789/0 conn(0x7f2ad40ca6b0 legacy=0x7f2ad40ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad37fe700 1 -- 172.21.15.64:0/1526045358 >> v1:172.21.15.64:6789/0 conn(0x7f2ad406ded0 legacy=0x7f2ad40c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad37fe700 1 -- 172.21.15.64:0/1526045358 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad406ceb0 con 0x7f2ad4156ec0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/1526045358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad4156ec0 0x7f2ad406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009910 tx=0x7f2ac40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80d960b2fb539ace server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80013f0 con 0x7f2ad4156ec0 2023-09-27T23:32:01.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac4011420 con 0x7f2ad4156ec0 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.325+0000 7f2ad27fc700 1 -- 172.21.15.64:0/1526045358 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40117c0 con 0x7f2ad4156ec0 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/1526045358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad4156ec0 msgr2=0x7f2ad406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/1526045358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad4156ec0 0x7f2ad406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009910 tx=0x7f2ac40091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/1526045358 shutdown_connections 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/1526045358 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad4156ec0 0x7f2ad406d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/1526045358 >> 172.21.15.64:0/1526045358 conn(0x7f2ad4069d80 msgr2=0x7f2ad40ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/1526045358 shutdown_connections 2023-09-27T23:32:01.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/1526045358 wait complete. 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 Processor -- start 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- start start 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ded0 0x7f2ad40c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 0x7f2ad4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4156ec0 0x7f2ad414f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41586d0 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4167350 con 0x7f2ad4156ec0 2023-09-27T23:32:01.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41581e0 con 0x7f2ad406ded0 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 0x7f2ad4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 0x7f2ad4157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34724/0 (socket says 172.21.15.64:34724) 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/175546051 learned_addr learned my addr 172.21.15.64:0/175546051 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ded0 0x7f2ad40c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad3fff700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4156ec0 0x7f2ad414f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ded0 msgr2=0x7f2ad40c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ded0 0x7f2ad40c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4156ec0 msgr2=0x7f2ad414f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4156ec0 0x7f2ad414f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:01.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 -- 172.21.15.64:0/175546051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4009420 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ad2ffd700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 0x7f2ad4157ca0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800cb40 tx=0x7f2ac8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8007150 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac80072d0 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad4167ba0 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8007690 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.329+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad41583f0 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.333+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad4167d50 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.333+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2ac8004a40 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.333+0000 7f2abb7fe700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060a70 0x7f2ac0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:01.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.333+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6300+0+0 (secure 0 0 0) 0x7f2ac807c810 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.333+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060a70 0x7f2ac0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:01.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.337+0000 7f2ad37fe700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060a70 0x7f2ac0062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40030c0 tx=0x7f2ac4016000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:01.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.341+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2ad4167d50 con 0x7f2ad40ca6b0 2023-09-27T23:32:01.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:01.513+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2ad4167d50 con 0x7f2ad40ca6b0 2023-09-27T23:32:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.421+0000 7f2abb7fe700 1 -- 172.21.15.64:0/175546051 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v125) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2ad4167d50 con 0x7f2ad40ca6b0 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060a70 msgr2=0x7f2ac0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060a70 0x7f2ac0062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40030c0 tx=0x7f2ac4016000 comp rx=0 tx=0).stop 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 msgr2=0x7f2ad4157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 0x7f2ad4157ca0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800cb40 tx=0x7f2ac8001a30 comp rx=0 tx=0).stop 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 shutdown_connections 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2ac0060a70 0x7f2ac0062f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad4156ec0 0x7f2ad414f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad40ca6b0 0x7f2ad4157ca0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 --2- 172.21.15.64:0/175546051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ad406ded0 0x7f2ad40c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 >> 172.21.15.64:0/175546051 conn(0x7f2ad4069d80 msgr2=0x7f2ad4156210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:02.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 shutdown_connections 2023-09-27T23:32:02.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.425+0000 7f2ada0aa700 1 -- 172.21.15.64:0/175546051 wait complete. 2023-09-27T23:32:02.433 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:32:02.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:32:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90a140700 1 Processor -- start 2023-09-27T23:32:02.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90a140700 1 -- start start 2023-09-27T23:32:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90a140700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc90406d760 0x7fc9041591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90a140700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc904159f90 con 0x7fc904159700 2023-09-27T23:32:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90a140700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90415a270 con 0x7fc904158df0 2023-09-27T23:32:02.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90a140700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040c7930 con 0x7fc90406d760 2023-09-27T23:32:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90993f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc904159700 0x7fc9040c8130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33486/0 (socket says 172.21.15.64:33486) 2023-09-27T23:32:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90913e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc904158df0 0x7fc90406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:34362/0 (socket says 172.21.15.64:34362) 2023-09-27T23:32:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90893d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc90406d760 0x7fc9041591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc90993f700 1 -- 172.21.15.64:0/4023043587 learned_addr learned my addr 172.21.15.64:0/4023043587 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.553+0000 7fc903fff700 1 -- 172.21.15.64:0/4023043587 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307719356 0 0) 0x7fc904159f90 con 0x7fc904159700 2023-09-27T23:32:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc903fff700 1 -- 172.21.15.64:0/4023043587 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ec003570 con 0x7fc904159700 2023-09-27T23:32:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc903fff700 1 -- 172.21.15.64:0/4023043587 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768945495 0 0) 0x7fc90415a270 con 0x7fc904158df0 2023-09-27T23:32:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90893d700 1 -- 172.21.15.64:0/4023043587 >> v1:172.21.15.123:6789/0 conn(0x7fc904158df0 legacy=0x7fc90406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90893d700 1 -- 172.21.15.64:0/4023043587 >> v1:172.21.15.64:6789/0 conn(0x7fc904159700 legacy=0x7fc9040c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90893d700 1 -- 172.21.15.64:0/4023043587 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040c7c10 con 0x7fc90406d760 2023-09-27T23:32:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90893d700 1 --2- 172.21.15.64:0/4023043587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc90406d760 0x7fc9041591c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80098e0 tx=0x7fc8f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8fc37526967120 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc903fff700 1 -- 172.21.15.64:0/4023043587 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800b640 con 0x7fc90406d760 2023-09-27T23:32:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc903fff700 1 -- 172.21.15.64:0/4023043587 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f800b7a0 con 0x7fc90406d760 2023-09-27T23:32:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc903fff700 1 -- 172.21.15.64:0/4023043587 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800bb40 con 0x7fc90406d760 2023-09-27T23:32:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 -- 172.21.15.64:0/4023043587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc90406d760 msgr2=0x7fc9041591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 --2- 172.21.15.64:0/4023043587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc90406d760 0x7fc9041591c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80098e0 tx=0x7fc8f80091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 -- 172.21.15.64:0/4023043587 shutdown_connections 2023-09-27T23:32:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 --2- 172.21.15.64:0/4023043587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc90406d760 0x7fc9041591c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 -- 172.21.15.64:0/4023043587 >> 172.21.15.64:0/4023043587 conn(0x7fc90406c080 msgr2=0x7fc9040ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 -- 172.21.15.64:0/4023043587 shutdown_connections 2023-09-27T23:32:02.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.557+0000 7fc90a140700 1 -- 172.21.15.64:0/4023043587 wait complete. 2023-09-27T23:32:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 Processor -- start 2023-09-27T23:32:02.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 -- start start 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 0x7fc9040cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc904158df0 0x7fc9041582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc904159700 0x7fc904156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040c6980 con 0x7fc90406d760 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040af880 con 0x7fc904159700 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040cc570 con 0x7fc904158df0 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 0x7fc9040cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 0x7fc9040cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34736/0 (socket says 172.21.15.64:34736) 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 -- 172.21.15.64:0/3586538291 learned_addr learned my addr 172.21.15.64:0/3586538291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90893d700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc904158df0 0x7fc9041582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90993f700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc904159700 0x7fc904156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 -- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc904158df0 msgr2=0x7fc9041582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc904158df0 0x7fc9041582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 -- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc904159700 msgr2=0x7fc904156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc904159700 0x7fc904156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 -- 172.21.15.64:0/3586538291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8009420 con 0x7fc90406d760 2023-09-27T23:32:02.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90993f700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc904159700 0x7fc904156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90913e700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 0x7fc9040cc030 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00cdd0 tx=0x7fc8fc001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc005df0 con 0x7fc90406d760 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc00a960 con 0x7fc90406d760 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00ad20 con 0x7fc90406d760 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040c6b70 con 0x7fc90406d760 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.561+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040afb10 con 0x7fc90406d760 2023-09-27T23:32:02.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.565+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90404be40 con 0x7fc90406d760 2023-09-27T23:32:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.565+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc8fc007300 con 0x7fc90406d760 2023-09-27T23:32:02.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.565+0000 7fc9017fa700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8f0060a70 0x7fc8f0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.565+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc8fc07cfe0 con 0x7fc90406d760 2023-09-27T23:32:02.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.565+0000 7fc90893d700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8f0060a70 0x7fc8f0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.569+0000 7fc90893d700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8f0060a70 0x7fc8f0062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80098e0 tx=0x7fc8f8017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:02.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.573+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc8fc04d3e0 con 0x7fc90406d760 2023-09-27T23:32:02.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.793+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc90415d8c0 con 0x7fc90406d760 2023-09-27T23:32:02.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.793+0000 7fc9017fa700 1 -- 172.21.15.64:0/3586538291 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc8fc050a00 con 0x7fc90406d760 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:32:02.799 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.800 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:32:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:32:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:32:02.801 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.802 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:32:02.803 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:32:02.805 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.806 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.808 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.809 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:32:02.810 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:32:02.811 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:32:02.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.813 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.814 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:32:02.815 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:32:02.816 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8f0060a70 msgr2=0x7fc8f0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8f0060a70 0x7fc8f0062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80098e0 tx=0x7fc8f8017000 comp rx=0 tx=0).stop 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 msgr2=0x7fc9040cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 0x7fc9040cc030 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00cdd0 tx=0x7fc8fc001bc0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 shutdown_connections 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc8f0060a70 0x7fc8f0062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc904159700 0x7fc904156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc904158df0 0x7fc9041582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 --2- 172.21.15.64:0/3586538291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc90406d760 0x7fc9040cc030 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 >> 172.21.15.64:0/3586538291 conn(0x7fc90406c080 msgr2=0x7fc9040c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:02.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 shutdown_connections 2023-09-27T23:32:02.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.797+0000 7fc90a140700 1 -- 172.21.15.64:0/3586538291 wait complete. 2023-09-27T23:32:02.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:32:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 Processor -- start 2023-09-27T23:32:02.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 -- start start 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b401552f0 0x7f6b401556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b4006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4006d140 con 0x7f6b400ae520 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400c8710 con 0x7f6b401552f0 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b46e24700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4006a9d0 con 0x7f6b4006c7c0 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.921+0000 7f6b44bc0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b401552f0 0x7f6b401556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3ffff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6b400ae520 0x7f6b400c8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:34366/0 (socket says 172.21.15.64:34366) 2023-09-27T23:32:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3ffff700 1 -- 172.21.15.64:0/3490429820 learned_addr learned my addr 172.21.15.64:0/3490429820 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b453c1700 1 --2- 172.21.15.64:0/3490429820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b4006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3f7fe700 1 -- 172.21.15.64:0/3490429820 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394983257 0 0) 0x7f6b4006d140 con 0x7f6b400ae520 2023-09-27T23:32:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3f7fe700 1 -- 172.21.15.64:0/3490429820 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2c003570 con 0x7f6b400ae520 2023-09-27T23:32:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b44bc0700 1 -- 172.21.15.64:0/3490429820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 msgr2=0x7f6b4006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b44bc0700 1 --2- 172.21.15.64:0/3490429820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b4006cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b44bc0700 1 -- 172.21.15.64:0/3490429820 >> v1:172.21.15.123:6789/0 conn(0x7f6b400ae520 legacy=0x7f6b400c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b44bc0700 1 -- 172.21.15.64:0/3490429820 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4006acb0 con 0x7f6b401552f0 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b44bc0700 1 --2- 172.21.15.64:0/3490429820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b401556c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6b300098e0 tx=0x7f6b300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae06cb8dc84e55b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3f7fe700 1 -- 172.21.15.64:0/3490429820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011630 con 0x7f6b401552f0 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3f7fe700 1 -- 172.21.15.64:0/3490429820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30011790 con 0x7f6b401552f0 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b3f7fe700 1 -- 172.21.15.64:0/3490429820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011b10 con 0x7f6b401552f0 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 -- 172.21.15.64:0/3490429820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 msgr2=0x7f6b401556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3490429820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b401556c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6b300098e0 tx=0x7f6b300091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 -- 172.21.15.64:0/3490429820 shutdown_connections 2023-09-27T23:32:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3490429820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b4006cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3490429820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b401556c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 -- 172.21.15.64:0/3490429820 >> 172.21.15.64:0/3490429820 conn(0x7f6b400c8850 msgr2=0x7f6b400c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 -- 172.21.15.64:0/3490429820 shutdown_connections 2023-09-27T23:32:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 -- 172.21.15.64:0/3490429820 wait complete. 2023-09-27T23:32:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.925+0000 7f6b46e24700 1 Processor -- start 2023-09-27T23:32:02.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 -- start start 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b40152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b400ae520 0x7f6b40169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b40156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40155d30 con 0x7f6b401552f0 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40156010 con 0x7f6b400ae520 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401562f0 con 0x7f6b4006c7c0 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b40156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b40156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34766/0 (socket says 172.21.15.64:34766) 2023-09-27T23:32:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 -- 172.21.15.64:0/3116030086 learned_addr learned my addr 172.21.15.64:0/3116030086 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b3ffff700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b400ae520 0x7f6b40169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b44bc0700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b40152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 -- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 msgr2=0x7f6b40152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b40152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 -- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b400ae520 msgr2=0x7f6b40169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b400ae520 0x7f6b40169f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 -- 172.21.15.64:0/3116030086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b30009420 con 0x7f6b401552f0 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b453c1700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b40156dd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009df0 tx=0x7f6b3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800caa0 con 0x7f6b401552f0 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3800f040 con 0x7f6b401552f0 2023-09-27T23:32:02.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b401565d0 con 0x7f6b401552f0 2023-09-27T23:32:02.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800ce70 con 0x7f6b401552f0 2023-09-27T23:32:02.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.929+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b401528b0 con 0x7f6b401552f0 2023-09-27T23:32:02.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.933+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b40156ad0 con 0x7f6b401552f0 2023-09-27T23:32:02.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.933+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6b38014690 con 0x7f6b401552f0 2023-09-27T23:32:02.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.933+0000 7f6b3cff9700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b20060a70 0x7f6b20062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:02.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.933+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6b3807d6c0 con 0x7f6b401552f0 2023-09-27T23:32:02.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.933+0000 7f6b44bc0700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b20060a70 0x7f6b20062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:02.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.933+0000 7f6b44bc0700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b20060a70 0x7f6b20062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b300091e0 tx=0x7f6b30017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:02.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:02.941+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6b380513c0 con 0x7f6b401552f0 2023-09-27T23:32:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.105+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b40071120 con 0x7f6b401552f0 2023-09-27T23:32:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.105+0000 7f6b3cff9700 1 -- 172.21.15.64:0/3116030086 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13865 (secure 0 0 0) 0x7f6b38050f50 con 0x7f6b401552f0 2023-09-27T23:32:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:03.111 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":125,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:32:02.411485+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b20060a70 msgr2=0x7f6b20062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b20060a70 0x7f6b20062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b300091e0 tx=0x7f6b30017000 comp rx=0 tx=0).stop 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 msgr2=0x7f6b40156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b40156dd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009df0 tx=0x7f6b3800b040 comp rx=0 tx=0).stop 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 shutdown_connections 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b20060a70 0x7f6b20062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b401552f0 0x7f6b40156dd0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b400ae520 0x7f6b40169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 --2- 172.21.15.64:0/3116030086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b4006c7c0 0x7f6b40152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 >> 172.21.15.64:0/3116030086 conn(0x7f6b400c8850 msgr2=0x7f6b400c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 shutdown_connections 2023-09-27T23:32:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.109+0000 7f6b46e24700 1 -- 172.21.15.64:0/3116030086 wait complete. 2023-09-27T23:32:03.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:32:03.126 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:32:03.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.233+0000 7fb33727d700 1 Processor -- start 2023-09-27T23:32:03.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.233+0000 7fb33727d700 1 -- start start 2023-09-27T23:32:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc1e0 0x7fb3300c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3300cc5f0 0x7fb3300c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33006d750 con 0x7fb330158fd0 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300cbaf0 con 0x7fb3300cc5f0 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33006cb40 con 0x7fb3300cc1e0 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3300cc5f0 0x7fb3300c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb3300cc5f0 0x7fb3300c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34774/0 (socket says 172.21.15.64:34774) 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 -- 172.21.15.64:0/971272533 learned_addr learned my addr 172.21.15.64:0/971272533 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb335a7a700 1 --2- 172.21.15.64:0/971272533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc1e0 0x7fb3300c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 -- 172.21.15.64:0/971272533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc1e0 msgr2=0x7fb3300c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 --2- 172.21.15.64:0/971272533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc1e0 0x7fb3300c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 -- 172.21.15.64:0/971272533 >> v1:172.21.15.64:6789/0 conn(0x7fb330158fd0 legacy=0x7fb33006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 -- 172.21.15.64:0/971272533 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33006ce20 con 0x7fb3300cc5f0 2023-09-27T23:32:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb336a7c700 1 --2- 172.21.15.64:0/971272533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc5f0 0x7fb3300c8960 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb328009f20 tx=0x7fb328009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b032f0d900031d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb335279700 1 -- 172.21.15.64:0/971272533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800e070 con 0x7fb3300cc5f0 2023-09-27T23:32:03.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb335279700 1 -- 172.21.15.64:0/971272533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32800ba80 con 0x7fb3300cc5f0 2023-09-27T23:32:03.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb335279700 1 -- 172.21.15.64:0/971272533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280092e0 con 0x7fb3300cc5f0 2023-09-27T23:32:03.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- 172.21.15.64:0/971272533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc5f0 msgr2=0x7fb3300c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 --2- 172.21.15.64:0/971272533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc5f0 0x7fb3300c8960 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb328009f20 tx=0x7fb328009840 comp rx=0 tx=0).stop 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- 172.21.15.64:0/971272533 shutdown_connections 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 --2- 172.21.15.64:0/971272533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc5f0 0x7fb3300c8960 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 --2- 172.21.15.64:0/971272533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc1e0 0x7fb3300c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- 172.21.15.64:0/971272533 >> 172.21.15.64:0/971272533 conn(0x7fb3300cdc00 msgr2=0x7fb3300ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- 172.21.15.64:0/971272533 shutdown_connections 2023-09-27T23:32:03.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.237+0000 7fb33727d700 1 -- 172.21.15.64:0/971272533 wait complete. 2023-09-27T23:32:03.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 Processor -- start 2023-09-27T23:32:03.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 -- start start 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 0x7fb3300ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc5f0 0x7fb33015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330158fd0 0x7fb3301526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300ad6b0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c9360 con 0x7fb330158fd0 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33727d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c9640 con 0x7fb3300cc5f0 2023-09-27T23:32:03.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33627b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 0x7fb3300ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33627b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 0x7fb3300ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34778/0 (socket says 172.21.15.64:34778) 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb33627b700 1 -- 172.21.15.64:0/3609056341 learned_addr learned my addr 172.21.15.64:0/3609056341 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb335a7a700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc5f0 0x7fb33015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.241+0000 7fb336a7c700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330158fd0 0x7fb3301526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33627b700 1 -- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc5f0 msgr2=0x7fb33015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33627b700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc5f0 0x7fb33015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33627b700 1 -- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330158fd0 msgr2=0x7fb3301526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33627b700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330158fd0 0x7fb3301526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33627b700 1 -- 172.21.15.64:0/3609056341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009ae0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb335a7a700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc5f0 0x7fb33015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:03.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb336a7c700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330158fd0 0x7fb3301526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33627b700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 0x7fb3300ca770 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c4e0 tx=0x7fb32c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011590 con 0x7fb3300cc1e0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3300c9920 con 0x7fb3300cc1e0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c011710 con 0x7fb3300cc1e0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011ad0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3300cacb0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33004be40 con 0x7fb3300cc1e0 2023-09-27T23:32:03.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.245+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb32c01b3b0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.249+0000 7fb3227fc700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb314060a70 0x7fb314062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.249+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fb32c07c350 con 0x7fb3300cc1e0 2023-09-27T23:32:03.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.249+0000 7fb335a7a700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb314060a70 0x7fb314062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.249+0000 7fb335a7a700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb314060a70 0x7fb314062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3301586d0 tx=0x7fb324008040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.253+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb32c04c6d0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.413+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3300ad7f0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb3227fc700 1 -- 172.21.15.64:0/3609056341 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb32c0170a0 con 0x7fb3300cc1e0 2023-09-27T23:32:03.420 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb314060a70 msgr2=0x7fb314062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb314060a70 0x7fb314062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3301586d0 tx=0x7fb324008040 comp rx=0 tx=0).stop 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 msgr2=0x7fb3300ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 0x7fb3300ca770 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c4e0 tx=0x7fb32c00c8a0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 shutdown_connections 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb314060a70 0x7fb314062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330158fd0 0x7fb3301526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3300cc5f0 0x7fb33015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 --2- 172.21.15.64:0/3609056341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3300cc1e0 0x7fb3300ca770 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 >> 172.21.15.64:0/3609056341 conn(0x7fb3300cdc00 msgr2=0x7fb330153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:03.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 shutdown_connections 2023-09-27T23:32:03.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.417+0000 7fb33727d700 1 -- 172.21.15.64:0/3609056341 wait complete. 2023-09-27T23:32:03.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:32:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 Processor -- start 2023-09-27T23:32:03.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 -- start start 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8980ce2d0 0x7fa8980ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 0x7fa898159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980cbbb0 con 0x7fa8980caa10 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89806dc90 con 0x7fa8980ce2d0 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89dfcf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980cafa0 con 0x7fa8980cc040 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89d7ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 0x7fa898159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89d7ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 0x7fa898159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:44644/0 (socket says 172.21.15.64:44644) 2023-09-27T23:32:03.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89d7ce700 1 -- 172.21.15.64:0/2555893755 learned_addr learned my addr 172.21.15.64:0/2555893755 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:03.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.541+0000 7fa89cfcd700 1 --2- 172.21.15.64:0/2555893755 >> v2:172.21.15.123:3300/0 conn(0x7fa8980ce2d0 0x7fa8980ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89d7ce700 1 -- 172.21.15.64:0/2555893755 >> v2:172.21.15.123:3300/0 conn(0x7fa8980ce2d0 msgr2=0x7fa8980ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89d7ce700 1 --2- 172.21.15.64:0/2555893755 >> v2:172.21.15.123:3300/0 conn(0x7fa8980ce2d0 0x7fa8980ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89d7ce700 1 -- 172.21.15.64:0/2555893755 >> v1:172.21.15.123:6789/0 conn(0x7fa8980caa10 legacy=0x7fa898158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89d7ce700 1 -- 172.21.15.64:0/2555893755 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980cb280 con 0x7fa8980cc040 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89d7ce700 1 --2- 172.21.15.64:0/2555893755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 0x7fa898159010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa8900099b0 tx=0x7fa890009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd38b068b4d513b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa8977fe700 1 -- 172.21.15.64:0/2555893755 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890009ea0 con 0x7fa8980cc040 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa8977fe700 1 -- 172.21.15.64:0/2555893755 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89000b860 con 0x7fa8980cc040 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa8977fe700 1 -- 172.21.15.64:0/2555893755 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89000bc00 con 0x7fa8980cc040 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 -- 172.21.15.64:0/2555893755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 msgr2=0x7fa898159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/2555893755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 0x7fa898159010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa8900099b0 tx=0x7fa890009250 comp rx=0 tx=0).stop 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 -- 172.21.15.64:0/2555893755 shutdown_connections 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/2555893755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980cc040 0x7fa898159010 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/2555893755 >> v2:172.21.15.123:3300/0 conn(0x7fa8980ce2d0 0x7fa8980ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 -- 172.21.15.64:0/2555893755 >> 172.21.15.64:0/2555893755 conn(0x7fa898067bb0 msgr2=0x7fa8980cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 -- 172.21.15.64:0/2555893755 shutdown_connections 2023-09-27T23:32:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 -- 172.21.15.64:0/2555893755 wait complete. 2023-09-27T23:32:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 Processor -- start 2023-09-27T23:32:03.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.545+0000 7fa89dfcf700 1 -- start start 2023-09-27T23:32:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8980caa10 0x7fa8980c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 0x7fa898155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980ce2d0 0x7fa898146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89816da40 con 0x7fa8980cc040 2023-09-27T23:32:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980c7350 con 0x7fa8980caa10 2023-09-27T23:32:03.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89816cb50 con 0x7fa8980ce2d0 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa897fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 0x7fa898155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980ce2d0 0x7fa898146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa897fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 0x7fa898155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34790/0 (socket says 172.21.15.64:34790) 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89cfcd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8980caa10 0x7fa8980c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa897fff700 1 -- 172.21.15.64:0/4195777312 learned_addr learned my addr 172.21.15.64:0/4195777312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 -- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8980caa10 msgr2=0x7fa8980c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8980caa10 0x7fa8980c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 -- 172.21.15.64:0/4195777312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 msgr2=0x7fa898155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 0x7fa898155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 -- 172.21.15.64:0/4195777312 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900094f0 con 0x7fa8980ce2d0 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89cfcd700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8980caa10 0x7fa8980c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:03.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa897fff700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 0x7fa898155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89d7ce700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980ce2d0 0x7fa898146410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8980696f0 tx=0x7fa890017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89000b9d0 con 0x7fa8980ce2d0 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89000bb50 con 0x7fa8980ce2d0 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89816ce30 con 0x7fa8980ce2d0 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890025670 con 0x7fa8980ce2d0 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.549+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89816d3f0 con 0x7fa8980ce2d0 2023-09-27T23:32:03.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.553+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89804be40 con 0x7fa8980ce2d0 2023-09-27T23:32:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.553+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa890023070 con 0x7fa8980ce2d0 2023-09-27T23:32:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.553+0000 7fa894ff9700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa884060a70 0x7fa884062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.553+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fa89007e4a0 con 0x7fa8980ce2d0 2023-09-27T23:32:03.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.553+0000 7fa89cfcd700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa884060a70 0x7fa884062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.553+0000 7fa89cfcd700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa884060a70 0x7fa884062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0113e0 tx=0x7fa88c0091a0 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.565+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa89004e780 con 0x7fa8980ce2d0 2023-09-27T23:32:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.733+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8980cb560 con 0x7fa8980ce2d0 2023-09-27T23:32:03.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.733+0000 7fa894ff9700 1 -- 172.21.15.64:0/4195777312 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v125) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa890051da0 con 0x7fa8980ce2d0 2023-09-27T23:32:03.738 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa884060a70 msgr2=0x7fa884062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa884060a70 0x7fa884062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0113e0 tx=0x7fa88c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980ce2d0 msgr2=0x7fa898146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980ce2d0 0x7fa898146410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8980696f0 tx=0x7fa890017000 comp rx=0 tx=0).stop 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 shutdown_connections 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa884060a70 0x7fa884062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8980ce2d0 0x7fa898146410 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8980cc040 0x7fa898155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 --2- 172.21.15.64:0/4195777312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8980caa10 0x7fa8980c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 >> 172.21.15.64:0/4195777312 conn(0x7fa898067bb0 msgr2=0x7fa898062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 shutdown_connections 2023-09-27T23:32:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.737+0000 7fa89dfcf700 1 -- 172.21.15.64:0/4195777312 wait complete. 2023-09-27T23:32:03.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:32:03.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 Processor -- start 2023-09-27T23:32:03.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- start start 2023-09-27T23:32:03.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f390c159340 0x7f390c067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c068c70 con 0x7f390c158f70 2023-09-27T23:32:03.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c068060 con 0x7f390c069100 2023-09-27T23:32:03.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c068340 con 0x7f390c159340 2023-09-27T23:32:03.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3910819700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f390c158f70 0x7f390c0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33512/0 (socket says 172.21.15.64:33512) 2023-09-27T23:32:03.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3910819700 1 -- 172.21.15.64:0/2354761319 learned_addr learned my addr 172.21.15.64:0/2354761319 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:03.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f391101a700 1 --2- 172.21.15.64:0/2354761319 >> v2:172.21.15.64:3300/0 conn(0x7f390c159340 0x7f390c067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f390b7fe700 1 -- 172.21.15.64:0/2354761319 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546861557 0 0) 0x7f390c068c70 con 0x7f390c158f70 2023-09-27T23:32:03.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f391101a700 1 -- 172.21.15.64:0/2354761319 >> v1:172.21.15.123:6789/0 conn(0x7f390c069100 legacy=0x7f390c0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f391101a700 1 -- 172.21.15.64:0/2354761319 >> v1:172.21.15.64:6789/0 conn(0x7f390c158f70 legacy=0x7f390c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f391101a700 1 -- 172.21.15.64:0/2354761319 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c068620 con 0x7f390c159340 2023-09-27T23:32:03.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f391101a700 1 --2- 172.21.15.64:0/2354761319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c159340 0x7f390c067980 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3904009890 tx=0x7f39040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16649dd675dabc20 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f390b7fe700 1 -- 172.21.15.64:0/2354761319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904011630 con 0x7f390c159340 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f390b7fe700 1 -- 172.21.15.64:0/2354761319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904011790 con 0x7f390c159340 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f390b7fe700 1 -- 172.21.15.64:0/2354761319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904011b30 con 0x7f390c159340 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- 172.21.15.64:0/2354761319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c159340 msgr2=0x7f390c067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 --2- 172.21.15.64:0/2354761319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c159340 0x7f390c067980 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3904009890 tx=0x7f39040091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- 172.21.15.64:0/2354761319 shutdown_connections 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 --2- 172.21.15.64:0/2354761319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c159340 0x7f390c067980 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.853+0000 7f3912a7d700 1 -- 172.21.15.64:0/2354761319 >> 172.21.15.64:0/2354761319 conn(0x7f390c0ccc40 msgr2=0x7f390c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 -- 172.21.15.64:0/2354761319 shutdown_connections 2023-09-27T23:32:03.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 -- 172.21.15.64:0/2354761319 wait complete. 2023-09-27T23:32:03.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 Processor -- start 2023-09-27T23:32:03.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 -- start start 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 0x7f390c157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c158f70 0x7f390c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f390c159340 0x7f390c169df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0c8810 con 0x7f390c069100 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c157dc0 con 0x7f390c158f70 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.857+0000 7f3912a7d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0c8250 con 0x7f390c159340 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 0x7f390c157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 0x7f390c157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34810/0 (socket says 172.21.15.64:34810) 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 -- 172.21.15.64:0/3964596624 learned_addr learned my addr 172.21.15.64:0/3964596624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f390bfff700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c158f70 0x7f390c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f391101a700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f390c159340 0x7f390c169df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 -- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f390c159340 msgr2=0x7f390c169df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f390c159340 0x7f390c169df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 -- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c158f70 msgr2=0x7f390c155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:03.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c158f70 0x7f390c155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:03.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 -- 172.21.15.64:0/3964596624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009420 con 0x7f390c069100 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3910819700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 0x7f390c157880 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f38f400c810 tx=0x7f38f4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4005c00 con 0x7f390c069100 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c155590 con 0x7f390c069100 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c0c84a0 con 0x7f390c069100 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4005d60 con 0x7f390c069100 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4007320 con 0x7f390c069100 2023-09-27T23:32:03.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.861+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c071120 con 0x7f390c069100 2023-09-27T23:32:03.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.865+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f38f401b3b0 con 0x7f390c069100 2023-09-27T23:32:03.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.865+0000 7f3908ff9700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38ec060a70 0x7f38ec062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:03.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.865+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f38f407c3f0 con 0x7f390c069100 2023-09-27T23:32:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.865+0000 7f390bfff700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38ec060a70 0x7f38ec062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.865+0000 7f390bfff700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38ec060a70 0x7f38ec062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007fd0 tx=0x7f38fc009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:03.869+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f38f404c770 con 0x7f390c069100 2023-09-27T23:32:04.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.037+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f390c068900 con 0x7f390c069100 2023-09-27T23:32:04.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.037+0000 7f3908ff9700 1 -- 172.21.15.64:0/3964596624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) v1 ==== 84+0+126 (secure 0 0 0) 0x7f38f4017090 con 0x7f390c069100 2023-09-27T23:32:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:04.044 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38ec060a70 msgr2=0x7f38ec062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38ec060a70 0x7f38ec062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007fd0 tx=0x7f38fc009040 comp rx=0 tx=0).stop 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 msgr2=0x7f390c157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 0x7f390c157880 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f38f400c810 tx=0x7f38f4007d90 comp rx=0 tx=0).stop 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 shutdown_connections 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f38ec060a70 0x7f38ec062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f390c159340 0x7f390c169df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c158f70 0x7f390c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 --2- 172.21.15.64:0/3964596624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f390c069100 0x7f390c157880 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 >> 172.21.15.64:0/3964596624 conn(0x7f390c0ccc40 msgr2=0x7f390c0c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 shutdown_connections 2023-09-27T23:32:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.041+0000 7f3912a7d700 1 -- 172.21.15.64:0/3964596624 wait complete. 2023-09-27T23:32:04.063 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:32:04.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:32:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.177+0000 7f3fd14d2700 1 Processor -- start 2023-09-27T23:32:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.177+0000 7f3fd14d2700 1 -- start start 2023-09-27T23:32:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3fcc158da0 0x7f3fcc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc0caca0 0x7f3fcc06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc15d720 con 0x7f3fcc06df10 2023-09-27T23:32:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc0ca290 con 0x7f3fcc158da0 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0ca4a0 con 0x7f3fcc0caca0 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3fcc06df10 0x7f3fcc06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33526/0 (socket says 172.21.15.64:33526) 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcb7fe700 1 -- 172.21.15.64:0/2911130477 learned_addr learned my addr 172.21.15.64:0/2911130477 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/2911130477 >> v2:172.21.15.123:3300/0 conn(0x7f3fcc158da0 0x7f3fcc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd0cd1700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc0caca0 0x7f3fcc06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcaffd700 1 -- 172.21.15.64:0/2911130477 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174449229 0 0) 0x7f3fcc15d720 con 0x7f3fcc06df10 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcaffd700 1 -- 172.21.15.64:0/2911130477 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fb8003570 con 0x7f3fcc06df10 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcbfff700 1 -- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc0caca0 msgr2=0x7f3fcc06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc0caca0 0x7f3fcc06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcbfff700 1 -- 172.21.15.64:0/2911130477 >> v1:172.21.15.64:6789/0 conn(0x7f3fcc06df10 legacy=0x7f3fcc06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcbfff700 1 -- 172.21.15.64:0/2911130477 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc0ca780 con 0x7f3fcc158da0 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd0cd1700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc0caca0 0x7f3fcc06ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc06e670 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0098e0 tx=0x7f3fbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=793487776a83139d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcaffd700 1 -- 172.21.15.64:0/2911130477 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011630 con 0x7f3fcc158da0 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcaffd700 1 -- 172.21.15.64:0/2911130477 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc011790 con 0x7f3fcc158da0 2023-09-27T23:32:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fcaffd700 1 -- 172.21.15.64:0/2911130477 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011b30 con 0x7f3fcc158da0 2023-09-27T23:32:04.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 msgr2=0x7f3fcc06e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc06e670 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0098e0 tx=0x7f3fbc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- 172.21.15.64:0/2911130477 shutdown_connections 2023-09-27T23:32:04.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc0caca0 0x7f3fcc06ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/2911130477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc06e670 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- 172.21.15.64:0/2911130477 >> 172.21.15.64:0/2911130477 conn(0x7f3fcc0cc760 msgr2=0x7f3fcc0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:04.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- 172.21.15.64:0/2911130477 shutdown_connections 2023-09-27T23:32:04.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.181+0000 7f3fd14d2700 1 -- 172.21.15.64:0/2911130477 wait complete. 2023-09-27T23:32:04.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 Processor -- start 2023-09-27T23:32:04.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 -- start start 2023-09-27T23:32:04.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc06df10 0x7f3fcc0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0caca0 0x7f3fcc0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0b2aa0 con 0x7f3fcc0caca0 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c7f40 con 0x7f3fcc158da0 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd14d2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c8220 con 0x7f3fcc06df10 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fcb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0caca0 0x7f3fcc0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fcb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0caca0 0x7f3fcc0c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34814/0 (socket says 172.21.15.64:34814) 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fcbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc06df10 0x7f3fcc0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fcb7fe700 1 -- 172.21.15.64:0/1104139726 learned_addr learned my addr 172.21.15.64:0/1104139726 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.185+0000 7f3fd0cd1700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fcbfff700 1 -- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 msgr2=0x7f3fcc150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fcbfff700 1 -- 172.21.15.64:0/1104139726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0caca0 msgr2=0x7f3fcc0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0caca0 0x7f3fcc0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fcbfff700 1 -- 172.21.15.64:0/1104139726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4009a20 con 0x7f3fcc06df10 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fd0cd1700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fcbfff700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc06df10 0x7f3fcc0c9500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc016000 tx=0x7f3fbc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011b10 con 0x7f3fcc06df10 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc0253f0 con 0x7f3fcc06df10 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc009420 con 0x7f3fcc06df10 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0257b0 con 0x7f3fcc06df10 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc0b2cf0 con 0x7f3fcc06df10 2023-09-27T23:32:04.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc04be40 con 0x7f3fcc06df10 2023-09-27T23:32:04.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.189+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3fbc0193b0 con 0x7f3fcc06df10 2023-09-27T23:32:04.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.193+0000 7f3fc3fff700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fac060a70 0x7f3fac062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.193+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3fbc07e970 con 0x7f3fcc06df10 2023-09-27T23:32:04.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.193+0000 7f3fcb7fe700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fac060a70 0x7f3fac062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.193+0000 7f3fcb7fe700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fac060a70 0x7f3fac062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0b00d0 tx=0x7f3fb4009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:04.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.201+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3fbc04edb0 con 0x7f3fcc06df10 2023-09-27T23:32:04.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.361+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3fcc0c8360 con 0x7f3fcc06df10 2023-09-27T23:32:04.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.457+0000 7f3fc3fff700 1 -- 172.21.15.64:0/1104139726 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3fcc0c8360 con 0x7f3fcc06df10 2023-09-27T23:32:04.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fac060a70 msgr2=0x7f3fac062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fac060a70 0x7f3fac062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0b00d0 tx=0x7f3fb4009040 comp rx=0 tx=0).stop 2023-09-27T23:32:04.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc06df10 msgr2=0x7f3fcc0c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc06df10 0x7f3fcc0c9500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc016000 tx=0x7f3fbc017000 comp rx=0 tx=0).stop 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 shutdown_connections 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3fac060a70 0x7f3fac062f20 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0b00d0 tx=0x7f3fb4009040 comp rx=0 tx=0).stop 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc158da0 0x7f3fcc150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fcc0caca0 0x7f3fcc0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 --2- 172.21.15.64:0/1104139726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fcc06df10 0x7f3fcc0c9500 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 >> 172.21.15.64:0/1104139726 conn(0x7f3fcc0cc760 msgr2=0x7f3fcc157610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 shutdown_connections 2023-09-27T23:32:04.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.461+0000 7f3fd14d2700 1 -- 172.21.15.64:0/1104139726 wait complete. 2023-09-27T23:32:04.467 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:32:04.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:32:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.581+0000 7f69529ba700 1 Processor -- start 2023-09-27T23:32:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.581+0000 7f69529ba700 1 -- start start 2023-09-27T23:32:04.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f694c0cc040 0x7f694c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c0cbbb0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c06dc90 con 0x7f694c0cc040 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0cafa0 con 0x7f694c0caa10 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f694c0cc040 0x7f694c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f694c0cc040 0x7f694c0ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34822/0 (socket says 172.21.15.64:34822) 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 -- 172.21.15.64:0/1430550153 learned_addr learned my addr 172.21.15.64:0/1430550153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69519b8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f694c0ce2d0 0x7f694c158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33530/0 (socket says 172.21.15.64:33530) 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69511b7700 1 --2- 172.21.15.64:0/1430550153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 -- 172.21.15.64:0/1430550153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 msgr2=0x7f694c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 --2- 172.21.15.64:0/1430550153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 -- 172.21.15.64:0/1430550153 >> v1:172.21.15.64:6789/0 conn(0x7f694c0ce2d0 legacy=0x7f694c158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 -- 172.21.15.64:0/1430550153 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0cb280 con 0x7f694c0cc040 2023-09-27T23:32:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69521b9700 1 --2- 172.21.15.64:0/1430550153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0cc040 0x7f694c0ce6a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6944009f20 tx=0x7f6944009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377b67962faeb137 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69509b6700 1 -- 172.21.15.64:0/1430550153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694400e070 con 0x7f694c0cc040 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69509b6700 1 -- 172.21.15.64:0/1430550153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694400ba80 con 0x7f694c0cc040 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69509b6700 1 -- 172.21.15.64:0/1430550153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69440092e0 con 0x7f694c0cc040 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- 172.21.15.64:0/1430550153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0cc040 msgr2=0x7f694c0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 --2- 172.21.15.64:0/1430550153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0cc040 0x7f694c0ce6a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6944009f20 tx=0x7f6944009840 comp rx=0 tx=0).stop 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- 172.21.15.64:0/1430550153 shutdown_connections 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 --2- 172.21.15.64:0/1430550153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0cc040 0x7f694c0ce6a0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 --2- 172.21.15.64:0/1430550153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- 172.21.15.64:0/1430550153 >> 172.21.15.64:0/1430550153 conn(0x7f694c067bb0 msgr2=0x7f694c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- 172.21.15.64:0/1430550153 shutdown_connections 2023-09-27T23:32:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.585+0000 7f69529ba700 1 -- 172.21.15.64:0/1430550153 wait complete. 2023-09-27T23:32:04.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.589+0000 7f69529ba700 1 Processor -- start 2023-09-27T23:32:04.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.589+0000 7f69529ba700 1 -- start start 2023-09-27T23:32:04.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c0cc040 0x7f694c1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 0x7f694c158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c154fc0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c142e50 con 0x7f694c0cc040 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c15b9b0 con 0x7f694c0caa10 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 0x7f694c158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 0x7f694c158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34832/0 (socket says 172.21.15.64:34832) 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 -- 172.21.15.64:0/3115676424 learned_addr learned my addr 172.21.15.64:0/3115676424 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69519b8700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69511b7700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c0cc040 0x7f694c1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 -- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 msgr2=0x7f694c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 -- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c0cc040 msgr2=0x7f694c1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c0cc040 0x7f694c1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 -- 172.21.15.64:0/3115676424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944009ae0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69521b9700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 0x7f694c158dc0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f694400b6c0 tx=0x7f6944018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69440032f0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6944008d10 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c15bc90 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944026690 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.593+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c142ae0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.597+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c04be40 con 0x7f694c0ce2d0 2023-09-27T23:32:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.597+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6944003040 con 0x7f694c0ce2d0 2023-09-27T23:32:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.597+0000 7f693dffb700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6930060a70 0x7f6930062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.597+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f694407efa0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.597+0000 7f69519b8700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6930060a70 0x7f6930062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.597+0000 7f69519b8700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6930060a70 0x7f6930062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f694c155e80 tx=0x7f6948007eb0 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.601+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f69440aebb0 con 0x7f694c0ce2d0 2023-09-27T23:32:04.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:04.765+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f694c142f90 con 0x7f694c0ce2d0 2023-09-27T23:32:05.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.457+0000 7f693dffb700 1 -- 172.21.15.64:0/3115676424 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) v1 ==== 170+0+0 (secure 0 0 0) 0x7f694404f0a0 con 0x7f694c0ce2d0 2023-09-27T23:32:05.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.465+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6930060a70 msgr2=0x7f6930062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.465+0000 7f69529ba700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6930060a70 0x7f6930062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f694c155e80 tx=0x7f6948007eb0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.465+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 msgr2=0x7f694c158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 0x7f694c158dc0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f694400b6c0 tx=0x7f6944018000 comp rx=0 tx=0).stop 2023-09-27T23:32:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 shutdown_connections 2023-09-27T23:32:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6930060a70 0x7f6930062f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694c0ce2d0 0x7f694c158dc0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c0cc040 0x7f694c1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 --2- 172.21.15.64:0/3115676424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f694c0caa10 0x7f694c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 >> 172.21.15.64:0/3115676424 conn(0x7f694c067bb0 msgr2=0x7f694c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:05.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 shutdown_connections 2023-09-27T23:32:05.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.469+0000 7f69529ba700 1 -- 172.21.15.64:0/3115676424 wait complete. 2023-09-27T23:32:05.486 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:32:05.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:32:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.609+0000 7f8834808700 1 Processor -- start 2023-09-27T23:32:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.609+0000 7f8834808700 1 -- start start 2023-09-27T23:32:05.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f883006c7c0 0x7f883015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883015da20 con 0x7f883006b420 2023-09-27T23:32:05.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883006da40 con 0x7f883006b050 2023-09-27T23:32:05.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883006cc00 con 0x7f883006c7c0 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f883006c7c0 0x7f883015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f883006c7c0 0x7f883015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50088/0 (socket says 172.21.15.64:50088) 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 -- 172.21.15.64:0/1921833975 learned_addr learned my addr 172.21.15.64:0/1921833975 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 -- 172.21.15.64:0/1921833975 >> v1:172.21.15.123:6789/0 conn(0x7f883006b050 legacy=0x7f88301594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 -- 172.21.15.64:0/1921833975 >> v1:172.21.15.64:6789/0 conn(0x7f883006b420 legacy=0x7f8830159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 -- 172.21.15.64:0/1921833975 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883006cec0 con 0x7f883006c7c0 2023-09-27T23:32:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882ffff700 1 --2- 172.21.15.64:0/1921833975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006c7c0 0x7f883015d4e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f20 tx=0x7f8828009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5dcb7b8640db40 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882effd700 1 -- 172.21.15.64:0/1921833975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800e070 con 0x7f883006c7c0 2023-09-27T23:32:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882effd700 1 -- 172.21.15.64:0/1921833975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882800ba80 con 0x7f883006c7c0 2023-09-27T23:32:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f882effd700 1 -- 172.21.15.64:0/1921833975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280092e0 con 0x7f883006c7c0 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- 172.21.15.64:0/1921833975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006c7c0 msgr2=0x7f883015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 --2- 172.21.15.64:0/1921833975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006c7c0 0x7f883015d4e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f20 tx=0x7f8828009840 comp rx=0 tx=0).stop 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- 172.21.15.64:0/1921833975 shutdown_connections 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 --2- 172.21.15.64:0/1921833975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006c7c0 0x7f883015d4e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- 172.21.15.64:0/1921833975 >> 172.21.15.64:0/1921833975 conn(0x7f88300c8cd0 msgr2=0x7f88300c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- 172.21.15.64:0/1921833975 shutdown_connections 2023-09-27T23:32:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.613+0000 7f8834808700 1 -- 172.21.15.64:0/1921833975 wait complete. 2023-09-27T23:32:05.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 Processor -- start 2023-09-27T23:32:05.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 -- start start 2023-09-27T23:32:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 0x7f88300b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f883006b420 0x7f88300c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f883006c7c0 0x7f8830157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88300c3060 con 0x7f883006b050 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883016d630 con 0x7f883006b420 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8834808700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883016cd40 con 0x7f883006c7c0 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 0x7f88300b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 0x7f88300b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50100/0 (socket says 172.21.15.64:50100) 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 -- 172.21.15.64:0/639790169 learned_addr learned my addr 172.21.15.64:0/639790169 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882ffff700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f883006c7c0 0x7f8830157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f8826ffd700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f883006b420 0x7f88300c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 -- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f883006c7c0 msgr2=0x7f8830157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f883006c7c0 0x7f8830157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 -- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f883006b420 msgr2=0x7f88300c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.617+0000 7f882f7fe700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f883006b420 0x7f88300c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f882f7fe700 1 -- 172.21.15.64:0/639790169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828009ae0 con 0x7f883006b050 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f882ffff700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f883006c7c0 0x7f8830157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f882f7fe700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 0x7f88300b0220 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f882000c590 tx=0x7f882000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011590 con 0x7f883006b050 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820011710 con 0x7f883006b050 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883016cfe0 con 0x7f883006b050 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011ad0 con 0x7f883006b050 2023-09-27T23:32:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88300b08b0 con 0x7f883006b050 2023-09-27T23:32:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883004be40 con 0x7f883006b050 2023-09-27T23:32:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.621+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f882001b3b0 con 0x7f883006b050 2023-09-27T23:32:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.625+0000 7f8827fff700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8810060a70 0x7f8810062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.625+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f882007c4e0 con 0x7f883006b050 2023-09-27T23:32:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.625+0000 7f8826ffd700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8810060a70 0x7f8810062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.625+0000 7f8826ffd700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8810060a70 0x7f8810062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88300a8b90 tx=0x7f8818007ec0 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:05.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.629+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f882004c570 con 0x7f883006b050 2023-09-27T23:32:05.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.829+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f883006d1a0 con 0x7f883006b050 2023-09-27T23:32:05.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.837+0000 7f8827fff700 1 -- 172.21.15.64:0/639790169 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7f88200170c0 con 0x7f883006b050 2023-09-27T23:32:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.857+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8810060a70 msgr2=0x7f8810062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.857+0000 7f8834808700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8810060a70 0x7f8810062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88300a8b90 tx=0x7f8818007ec0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.857+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 msgr2=0x7f88300b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.857+0000 7f8834808700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 0x7f88300b0220 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f882000c590 tx=0x7f882000c8a0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 shutdown_connections 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8810060a70 0x7f8810062f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f883006c7c0 0x7f8830157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f883006b420 0x7f88300c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 --2- 172.21.15.64:0/639790169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f883006b050 0x7f88300b0220 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 >> 172.21.15.64:0/639790169 conn(0x7f88300c8cd0 msgr2=0x7f8830156630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:05.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 shutdown_connections 2023-09-27T23:32:05.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.865+0000 7f8834808700 1 -- 172.21.15.64:0/639790169 wait complete. 2023-09-27T23:32:05.876 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 16 and data pool 17 2023-09-27T23:32:05.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:32:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 Processor -- start 2023-09-27T23:32:05.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 -- start start 2023-09-27T23:32:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff1100b3600 0x7ff1100cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff1100afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100cd500 con 0x7ff1100b03b0 2023-09-27T23:32:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100c8f30 con 0x7ff1100b3600 2023-09-27T23:32:05.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff11723c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b39d0 con 0x7ff1100ae3e0 2023-09-27T23:32:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff1157d9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1100b03b0 0x7ff110161140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44432/0 (socket says 172.21.15.64:44432) 2023-09-27T23:32:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff1157d9700 1 -- 172.21.15.64:0/3410918995 learned_addr learned my addr 172.21.15.64:0/3410918995 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff114fd8700 1 --2- 172.21.15.64:0/3410918995 >> v2:172.21.15.123:3300/0 conn(0x7ff1100b3600 0x7ff1100cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10ffff700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff1100afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10f7fe700 1 -- 172.21.15.64:0/3410918995 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983232910 0 0) 0x7ff1100cd500 con 0x7ff1100b03b0 2023-09-27T23:32:05.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10f7fe700 1 -- 172.21.15.64:0/3410918995 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc003570 con 0x7ff1100b03b0 2023-09-27T23:32:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff114fd8700 1 -- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 msgr2=0x7ff1100afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff114fd8700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff1100afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff114fd8700 1 -- 172.21.15.64:0/3410918995 >> v1:172.21.15.64:6789/0 conn(0x7ff1100b03b0 legacy=0x7ff110161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff114fd8700 1 -- 172.21.15.64:0/3410918995 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100ccc60 con 0x7ff1100b3600 2023-09-27T23:32:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10ffff700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff1100afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:05.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff114fd8700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b3600 0x7ff1100cc580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80098e0 tx=0x7ff0f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf2310b9d1b568ba server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10f7fe700 1 -- 172.21.15.64:0/3410918995 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8011630 con 0x7ff1100b3600 2023-09-27T23:32:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10f7fe700 1 -- 172.21.15.64:0/3410918995 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8011790 con 0x7ff1100b3600 2023-09-27T23:32:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.977+0000 7ff10f7fe700 1 -- 172.21.15.64:0/3410918995 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8011b30 con 0x7ff1100b3600 2023-09-27T23:32:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b3600 msgr2=0x7ff1100cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b3600 0x7ff1100cc580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80098e0 tx=0x7ff0f80091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- 172.21.15.64:0/3410918995 shutdown_connections 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff1100afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 --2- 172.21.15.64:0/3410918995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b3600 0x7ff1100cc580 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- 172.21.15.64:0/3410918995 >> 172.21.15.64:0/3410918995 conn(0x7ff11006c7c0 msgr2=0x7ff11006cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- 172.21.15.64:0/3410918995 shutdown_connections 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- 172.21.15.64:0/3410918995 wait complete. 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 Processor -- start 2023-09-27T23:32:05.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- start start 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff11015b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b03b0 0x7ff11006ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 0x7ff11015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110069b60 con 0x7ff1100b3600 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110069e40 con 0x7ff1100b03b0 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff11723c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11006a120 con 0x7ff1100ae3e0 2023-09-27T23:32:05.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff1157d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 0x7ff11015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff1157d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 0x7ff11015ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50108/0 (socket says 172.21.15.64:50108) 2023-09-27T23:32:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff1157d9700 1 -- 172.21.15.64:0/2524823680 learned_addr learned my addr 172.21.15.64:0/2524823680 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff10ffff700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b03b0 0x7ff11006ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.981+0000 7ff114fd8700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff11015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff1157d9700 1 -- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 msgr2=0x7ff11015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff1157d9700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff11015b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff1157d9700 1 -- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b03b0 msgr2=0x7ff11006ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff1157d9700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b03b0 0x7ff11006ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff1157d9700 1 -- 172.21.15.64:0/2524823680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108009a20 con 0x7ff1100b3600 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff1157d9700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 0x7ff11015ebe0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff1000010e0 tx=0x7ff100007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100005b30 con 0x7ff1100b3600 2023-09-27T23:32:05.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100005c90 con 0x7ff1100b3600 2023-09-27T23:32:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f8009420 con 0x7ff1100b3600 2023-09-27T23:32:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000072e0 con 0x7ff1100b3600 2023-09-27T23:32:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11006a6e0 con 0x7ff1100b3600 2023-09-27T23:32:05.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.985+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff110071120 con 0x7ff1100b3600 2023-09-27T23:32:05.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.989+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff100012400 con 0x7ff1100b3600 2023-09-27T23:32:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.989+0000 7ff10cff9700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff0f0060a70 0x7ff0f0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.989+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff10007cb90 con 0x7ff1100b3600 2023-09-27T23:32:05.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.989+0000 7ff114fd8700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff0f0060a70 0x7ff0f0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:05.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.989+0000 7ff114fd8700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff0f0060a70 0x7ff0f0062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1100c4ea0 tx=0x7ff0f8017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:05.993+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff10004ccf0 con 0x7ff1100b3600 2023-09-27T23:32:06.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.153+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1100ccf40 con 0x7ff1100b3600 2023-09-27T23:32:06.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.153+0000 7ff10cff9700 1 -- 172.21.15.64:0/2524823680 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17098 (secure 0 0 0) 0x7ff100021090 con 0x7ff1100b3600 2023-09-27T23:32:06.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:06.160 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":128,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:32:05.833061+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:32:04.410437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-09-27T23:32:04.813612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff0f0060a70 msgr2=0x7ff0f0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff0f0060a70 0x7ff0f0062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1100c4ea0 tx=0x7ff0f8017000 comp rx=0 tx=0).stop 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 msgr2=0x7ff11015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 0x7ff11015ebe0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff1000010e0 tx=0x7ff100007dd0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 shutdown_connections 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff0f0060a70 0x7ff0f0062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1100b3600 0x7ff11015ebe0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1100b03b0 0x7ff11006ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 --2- 172.21.15.64:0/2524823680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1100ae3e0 0x7ff11015b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 >> 172.21.15.64:0/2524823680 conn(0x7ff11006c7c0 msgr2=0x7ff1100c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:06.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 shutdown_connections 2023-09-27T23:32:06.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.157+0000 7ff11723c700 1 -- 172.21.15.64:0/2524823680 wait complete. 2023-09-27T23:32:06.174 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:32:06.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.273+0000 7f7274bdd700 1 Processor -- start 2023-09-27T23:32:06.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.273+0000 7f7274bdd700 1 -- start start 2023-09-27T23:32:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.273+0000 7f7274bdd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72700ce810 0x7f727006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.273+0000 7f7274bdd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727006b730 con 0x7f72701591f0 2023-09-27T23:32:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.273+0000 7f7274bdd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727006ab20 con 0x7f7270158e20 2023-09-27T23:32:06.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.273+0000 7f7274bdd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727006ae00 con 0x7f72700ce810 2023-09-27T23:32:06.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f72701591f0 0x7f72700ca950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44436/0 (socket says 172.21.15.64:44436) 2023-09-27T23:32:06.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726effd700 1 -- 172.21.15.64:0/797182213 learned_addr learned my addr 172.21.15.64:0/797182213 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:06.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726ffff700 1 --2- 172.21.15.64:0/797182213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72700ce810 0x7f727006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359338997 0 0) 0x7f727006b730 con 0x7f72701591f0 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c003570 con 0x7f72701591f0 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350576933 0 0) 0x7f727006ab20 con 0x7f7270158e20 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f727006b730 con 0x7f7270158e20 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726ffff700 1 -- 172.21.15.64:0/797182213 >> v1:172.21.15.123:6789/0 conn(0x7f7270158e20 legacy=0x7f72700ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726ffff700 1 -- 172.21.15.64:0/797182213 >> v1:172.21.15.64:6789/0 conn(0x7f72701591f0 legacy=0x7f72700ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726ffff700 1 -- 172.21.15.64:0/797182213 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727006b0e0 con 0x7f72700ce810 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726ffff700 1 --2- 172.21.15.64:0/797182213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72700ce810 0x7f727006a540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7268009910 tx=0x7f72680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3951f45881baa92d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268011630 con 0x7f72700ce810 2023-09-27T23:32:06.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268011790 con 0x7f72700ce810 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f726e7fc700 1 -- 172.21.15.64:0/797182213 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268011b30 con 0x7f72700ce810 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 -- 172.21.15.64:0/797182213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72700ce810 msgr2=0x7f727006a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 --2- 172.21.15.64:0/797182213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72700ce810 0x7f727006a540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7268009910 tx=0x7f72680091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 -- 172.21.15.64:0/797182213 shutdown_connections 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 --2- 172.21.15.64:0/797182213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72700ce810 0x7f727006a540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 -- 172.21.15.64:0/797182213 >> 172.21.15.64:0/797182213 conn(0x7f7270067bb0 msgr2=0x7f72700cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 -- 172.21.15.64:0/797182213 shutdown_connections 2023-09-27T23:32:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 -- 172.21.15.64:0/797182213 wait complete. 2023-09-27T23:32:06.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.277+0000 7f7274bdd700 1 Processor -- start 2023-09-27T23:32:06.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 -- start start 2023-09-27T23:32:06.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 0x7f727016cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7270158e20 0x7f727014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72701591f0 0x7f727014c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700c6e70 con 0x7f72700ce810 2023-09-27T23:32:06.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727016d8f0 con 0x7f72701591f0 2023-09-27T23:32:06.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727016dbd0 con 0x7f7270158e20 2023-09-27T23:32:06.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 0x7f727016cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 0x7f727016cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50112/0 (socket says 172.21.15.64:50112) 2023-09-27T23:32:06.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726f7fe700 1 -- 172.21.15.64:0/742946419 learned_addr learned my addr 172.21.15.64:0/742946419 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:06.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7270158e20 0x7f727014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726ffff700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72701591f0 0x7f727014c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 -- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72701591f0 msgr2=0x7f727014c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72701591f0 0x7f727014c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 -- 172.21.15.64:0/742946419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 msgr2=0x7f727016cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 0x7f727016cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 -- 172.21.15.64:0/742946419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268009420 con 0x7f7270158e20 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726ffff700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72701591f0 0x7f727014c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726f7fe700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 0x7f727016cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f726effd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7270158e20 0x7f727014efd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f725800c560 tx=0x7f7258007d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258005b30 con 0x7f7270158e20 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258005cb0 con 0x7f7270158e20 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72700c7080 con 0x7f7270158e20 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258007320 con 0x7f7270158e20 2023-09-27T23:32:06.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.281+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727014e820 con 0x7f7270158e20 2023-09-27T23:32:06.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.285+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727004be40 con 0x7f7270158e20 2023-09-27T23:32:06.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.285+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f725801b3b0 con 0x7f7270158e20 2023-09-27T23:32:06.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.285+0000 7f72677fe700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7250060a70 0x7f7250062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.285+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f725807c580 con 0x7f7270158e20 2023-09-27T23:32:06.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.285+0000 7f726f7fe700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7250060a70 0x7f7250062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.285+0000 7f726f7fe700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7250060a70 0x7f7250062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7260007fd0 tx=0x7f7260002820 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.293+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f725804c510 con 0x7f7270158e20 2023-09-27T23:32:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.457+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f727016dd60 con 0x7f7270158e20 2023-09-27T23:32:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.457+0000 7f72677fe700 1 -- 172.21.15.64:0/742946419 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f725804fb30 con 0x7f7270158e20 2023-09-27T23:32:06.464 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:06.464 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":128,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:32:05.833061+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:32:04.410437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-09-27T23:32:04.813612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7250060a70 msgr2=0x7f7250062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7250060a70 0x7f7250062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7260007fd0 tx=0x7f7260002820 comp rx=0 tx=0).stop 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7270158e20 msgr2=0x7f727014efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7270158e20 0x7f727014efd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f725800c560 tx=0x7f7258007d70 comp rx=0 tx=0).stop 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 shutdown_connections 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7250060a70 0x7f7250062f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72701591f0 0x7f727014c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7270158e20 0x7f727014efd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 --2- 172.21.15.64:0/742946419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72700ce810 0x7f727016cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 >> 172.21.15.64:0/742946419 conn(0x7f7270067bb0 msgr2=0x7f72700b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 shutdown_connections 2023-09-27T23:32:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.461+0000 7f7274bdd700 1 -- 172.21.15.64:0/742946419 wait complete. 2023-09-27T23:32:06.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:32:06.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 Processor -- start 2023-09-27T23:32:06.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 -- start start 2023-09-27T23:32:06.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87380acf30 0x7f87381557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f873806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8738156780 con 0x7f87380cc0d0 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87381532a0 con 0x7f87380acf30 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873ddfb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8738158f90 con 0x7f87380ad300 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873d5fa700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f87380cc0d0 0x7f8738159420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44444/0 (socket says 172.21.15.64:44444) 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873d5fa700 1 -- 172.21.15.64:0/1228840310 learned_addr learned my addr 172.21.15.64:0/1228840310 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f8737fff700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f873806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.589+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1228840310 >> v2:172.21.15.123:3300/0 conn(0x7f87380acf30 0x7f87381557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562683333 0 0) 0x7f8738156780 con 0x7f87380cc0d0 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8720003570 con 0x7f87380cc0d0 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1580468364 0 0) 0x7f8720003570 con 0x7f87380cc0d0 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 msgr2=0x7f873806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:06.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f873806d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 >> v2:172.21.15.123:3300/0 conn(0x7f87380acf30 msgr2=0x7f87381557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 --2- 172.21.15.64:0/1228840310 >> v2:172.21.15.123:3300/0 conn(0x7f87380acf30 0x7f87381557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738155ee0 con 0x7f87380cc0d0 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f87300017f0 con 0x7f87380cc0d0 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 >> v1:172.21.15.64:6789/0 conn(0x7f87380cc0d0 legacy=0x7f8738159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8720003a60 0x7f8720003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720004810 0x7f8720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f87200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738158f90 con 0x7f8720004810 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872000ba60 con 0x7f8720003a60 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872000bba0 con 0x7f87380ad300 2023-09-27T23:32:06.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f8737fff700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720004810 0x7f8720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8720003a60 0x7f8720003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873d5fa700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f87200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 -- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 msgr2=0x7f87200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f87200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 -- 172.21.15.64:0/1228840310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720004810 msgr2=0x7f8720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720004810 0x7f8720004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 -- 172.21.15.64:0/1228840310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8720003570 con 0x7f8720003a60 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873d5fa700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87380ad300 0x7f87200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8720003a60 0x7f8720003e70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f872c009c30 tx=0x7f872c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae3871213e59b36 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00e710 con 0x7f8720003a60 2023-09-27T23:32:06.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00e870 con 0x7f8720003a60 2023-09-27T23:32:06.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f87377fe700 1 -- 172.21.15.64:0/1228840310 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00ec30 con 0x7f8720003a60 2023-09-27T23:32:06.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873ddfb700 1 -- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8720003a60 msgr2=0x7f8720003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1228840310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8720003a60 0x7f8720003e70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f872c009c30 tx=0x7f872c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873d5fa700 1 -- 172.21.15.64:0/1228840310 reap_dead start 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873ddfb700 1 -- 172.21.15.64:0/1228840310 shutdown_connections 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873ddfb700 1 -- 172.21.15.64:0/1228840310 >> 172.21.15.64:0/1228840310 conn(0x7f8738067b50 msgr2=0x7f87380cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873ddfb700 1 -- 172.21.15.64:0/1228840310 shutdown_connections 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.593+0000 7f873ddfb700 1 -- 172.21.15.64:0/1228840310 wait complete. 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 Processor -- start 2023-09-27T23:32:06.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 -- start start 2023-09-27T23:32:06.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 0x7f8738137f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8738157f30 0x7f87381583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87380ae950 0x7f87380aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738138470 con 0x7f8720003a60 2023-09-27T23:32:06.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87380af300 con 0x7f87380ae950 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873ddfb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87381668c0 con 0x7f8738157f30 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 0x7f8738137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 0x7f8738137f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50126/0 (socket says 172.21.15.64:50126) 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 -- 172.21.15.64:0/1209777638 learned_addr learned my addr 172.21.15.64:0/1209777638 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873d5fa700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87380ae950 0x7f87380aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f8737fff700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8738157f30 0x7f87381583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 -- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8738157f30 msgr2=0x7f87381583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8738157f30 0x7f87381583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 -- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87380ae950 msgr2=0x7f87380aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87380ae950 0x7f87380aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f873cdf9700 1 -- 172.21.15.64:0/1209777638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738165d40 con 0x7f8720003a60 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.597+0000 7f8737fff700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8738157f30 0x7f87381583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f873cdf9700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 0x7f8738137f30 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f872c000df0 tx=0x7f872c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c025070 con 0x7f8720003a60 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00ebf0 con 0x7f8720003a60 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8738166020 con 0x7f8720003a60 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c020260 con 0x7f8720003a60 2023-09-27T23:32:06.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87381665e0 con 0x7f8720003a60 2023-09-27T23:32:06.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873804be40 con 0x7f8720003a60 2023-09-27T23:32:06.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.601+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f872c01e660 con 0x7f8720003a60 2023-09-27T23:32:06.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.605+0000 7f8734ff9700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8724060a70 0x7f8724062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:06.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.605+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f872c080af0 con 0x7f8720003a60 2023-09-27T23:32:06.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.605+0000 7f8737fff700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8724060a70 0x7f8724062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:06.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.605+0000 7f8737fff700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8724060a70 0x7f8724062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87280093f0 tx=0x7f8728008040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:06.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.613+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f872c050b20 con 0x7f8720003a60 2023-09-27T23:32:06.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:06.813+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f873806e0f0 con 0x7f8720003a60 2023-09-27T23:32:07.227 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:32:07.225+0000 7fd06eac6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:32:07.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f8734ff9700 1 -- 172.21.15.64:0/1209777638 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f872c054140 con 0x7f8720003a60 2023-09-27T23:32:07.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8724060a70 msgr2=0x7f8724062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:07.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8724060a70 0x7f8724062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f87280093f0 tx=0x7f8728008040 comp rx=0 tx=0).stop 2023-09-27T23:32:07.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 msgr2=0x7f8738137f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 0x7f8738137f30 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f872c000df0 tx=0x7f872c01a000 comp rx=0 tx=0).stop 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 shutdown_connections 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8724060a70 0x7f8724062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87380ae950 0x7f87380aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8738157f30 0x7f87381583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 --2- 172.21.15.64:0/1209777638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8720003a60 0x7f8738137f30 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.505+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 >> 172.21.15.64:0/1209777638 conn(0x7f8738067b50 msgr2=0x7f87380c6580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.509+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 shutdown_connections 2023-09-27T23:32:07.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.509+0000 7f873ddfb700 1 -- 172.21.15.64:0/1209777638 wait complete. 2023-09-27T23:32:07.529 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:32:07.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:32:07.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 Processor -- start 2023-09-27T23:32:07.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 -- start start 2023-09-27T23:32:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 0x7fe1a00c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe1a00ac720 0x7fe1a00acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a00af4c0 con 0x7fe1a0156e80 2023-09-27T23:32:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00ad0e0 con 0x7fe1a00ac720 2023-09-27T23:32:07.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00cae80 con 0x7fe1a0150420 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 0x7fe1a00c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 0x7fe1a00c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:35158/0 (socket says 172.21.15.64:35158) 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 -- 172.21.15.64:0/1014052879 learned_addr learned my addr 172.21.15.64:0/1014052879 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19dd9b700 1 --1- 172.21.15.64:0/1014052879 >> v1:172.21.15.123:6789/0 conn(0x7fe1a0156e80 0x7fe1a00ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:55562/0 (socket says 172.21.15.64:55562) 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19ed9d700 1 --2- 172.21.15.64:0/1014052879 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00ac720 0x7fe1a00acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19d59a700 1 -- 172.21.15.64:0/1014052879 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993560070 0 0) 0x7fe1a00af4c0 con 0x7fe1a0156e80 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19d59a700 1 -- 172.21.15.64:0/1014052879 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe18c003570 con 0x7fe1a0156e80 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 -- 172.21.15.64:0/1014052879 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00ac720 msgr2=0x7fe1a00acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 --2- 172.21.15.64:0/1014052879 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00ac720 0x7fe1a00acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 -- 172.21.15.64:0/1014052879 >> v1:172.21.15.123:6789/0 conn(0x7fe1a0156e80 legacy=0x7fe1a00ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 -- 172.21.15.64:0/1014052879 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00af1e0 con 0x7fe1a0150420 2023-09-27T23:32:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19e59c700 1 --2- 172.21.15.64:0/1014052879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 0x7fe1a00c7170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1880099b0 tx=0x7fe188009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1eef77d57bbd88c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19d59a700 1 -- 172.21.15.64:0/1014052879 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18800b680 con 0x7fe1a0150420 2023-09-27T23:32:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19d59a700 1 -- 172.21.15.64:0/1014052879 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18800b7e0 con 0x7fe1a0150420 2023-09-27T23:32:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.648+0000 7fe19d59a700 1 -- 172.21.15.64:0/1014052879 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18800bb60 con 0x7fe1a0150420 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 -- 172.21.15.64:0/1014052879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 msgr2=0x7fe1a00c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1014052879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 0x7fe1a00c7170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1880099b0 tx=0x7fe188009250 comp rx=0 tx=0).stop 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 -- 172.21.15.64:0/1014052879 shutdown_connections 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1014052879 >> v2:172.21.15.123:3300/0 conn(0x7fe1a00ac720 0x7fe1a00acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1014052879 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a0150420 0x7fe1a00c7170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 -- 172.21.15.64:0/1014052879 >> 172.21.15.64:0/1014052879 conn(0x7fe1a00ccb30 msgr2=0x7fe1a00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 -- 172.21.15.64:0/1014052879 shutdown_connections 2023-09-27T23:32:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 -- 172.21.15.64:0/1014052879 wait complete. 2023-09-27T23:32:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 Processor -- start 2023-09-27T23:32:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.652+0000 7fe19f59e700 1 -- start start 2023-09-27T23:32:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a00ac720 0x7fe1a0150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a0150420 0x7fe1a016cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 0x7fe1a00670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a016d490 con 0x7fe1a0156e80 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0069040 con 0x7fe1a0150420 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0068a70 con 0x7fe1a00ac720 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 0x7fe1a00670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 0x7fe1a00670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50132/0 (socket says 172.21.15.64:50132) 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 -- 172.21.15.64:0/1248910848 learned_addr learned my addr 172.21.15.64:0/1248910848 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19e59c700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a00ac720 0x7fe1a0150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19dd9b700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a0150420 0x7fe1a016cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 -- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a00ac720 msgr2=0x7fe1a0150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a00ac720 0x7fe1a0150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 -- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a0150420 msgr2=0x7fe1a016cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a0150420 0x7fe1a016cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 -- 172.21.15.64:0/1248910848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880094f0 con 0x7fe1a0156e80 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19ed9d700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 0x7fe1a00670e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe198009e90 tx=0x7fe19800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800cb70 con 0x7fe1a0156e80 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a0067620 con 0x7fe1a0156e80 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19800f040 con 0x7fe1a0156e80 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198014480 con 0x7fe1a0156e80 2023-09-27T23:32:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.656+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a0153600 con 0x7fe1a0156e80 2023-09-27T23:32:07.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.660+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a004be40 con 0x7fe1a0156e80 2023-09-27T23:32:07.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.660+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe198014830 con 0x7fe1a0156e80 2023-09-27T23:32:07.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.660+0000 7fe1967fc700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe180060a70 0x7fe180062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.660+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe19807dcc0 con 0x7fe1a0156e80 2023-09-27T23:32:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.660+0000 7fe19e59c700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe180060a70 0x7fe180062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.660+0000 7fe19e59c700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe180060a70 0x7fe180062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe188009290 tx=0x7fe188017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:07.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.668+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe198084080 con 0x7fe1a0156e80 2023-09-27T23:32:07.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:07.868+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe1a00ca830 con 0x7fe1a0156e80 2023-09-27T23:32:08.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.508+0000 7fe1967fc700 1 -- 172.21.15.64:0/1248910848 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe1980180c0 con 0x7fe1a0156e80 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe180060a70 msgr2=0x7fe180062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe180060a70 0x7fe180062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe188009290 tx=0x7fe188017000 comp rx=0 tx=0).stop 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 msgr2=0x7fe1a00670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 0x7fe1a00670e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe198009e90 tx=0x7fe19800b040 comp rx=0 tx=0).stop 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 shutdown_connections 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe180060a70 0x7fe180062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1a0156e80 0x7fe1a00670e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a0150420 0x7fe1a016cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 --2- 172.21.15.64:0/1248910848 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a00ac720 0x7fe1a0150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 >> 172.21.15.64:0/1248910848 conn(0x7fe1a00ccb30 msgr2=0x7fe1a014f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:08.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 shutdown_connections 2023-09-27T23:32:08.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.512+0000 7fe19f59e700 1 -- 172.21.15.64:0/1248910848 wait complete. 2023-09-27T23:32:08.535 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:32:08.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:32:08.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 Processor -- start 2023-09-27T23:32:08.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 -- start start 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29e406b990 0x7f29e40c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29e415a5e0 0x7f29e40c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e40cb030 con 0x7f29e415a210 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e41599a0 con 0x7f29e415a5e0 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e40cb290 con 0x7f29e406b990 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29e415a5e0 0x7f29e40c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29e415a5e0 0x7f29e40c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50136/0 (socket says 172.21.15.64:50136) 2023-09-27T23:32:08.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 -- 172.21.15.64:0/3287688400 learned_addr learned my addr 172.21.15.64:0/3287688400 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:08.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29e9767700 1 --2- 172.21.15.64:0/3287688400 >> v2:172.21.15.123:3300/0 conn(0x7f29e406b990 0x7f29e40c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:08.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 -- 172.21.15.64:0/3287688400 >> v2:172.21.15.123:3300/0 conn(0x7f29e406b990 msgr2=0x7f29e40c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:08.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 --2- 172.21.15.64:0/3287688400 >> v2:172.21.15.123:3300/0 conn(0x7f29e406b990 0x7f29e40c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 -- 172.21.15.64:0/3287688400 >> v1:172.21.15.123:6789/0 conn(0x7f29e415a210 legacy=0x7f29e406b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 -- 172.21.15.64:0/3287688400 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e40cb570 con 0x7f29e415a5e0 2023-09-27T23:32:08.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ea769700 1 --2- 172.21.15.64:0/3287688400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e415a5e0 0x7f29e40c95f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f29e40b2760 tx=0x7f29dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2f69421c92ec58 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29e8f66700 1 -- 172.21.15.64:0/3287688400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00e070 con 0x7f29e415a5e0 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29e8f66700 1 -- 172.21.15.64:0/3287688400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc00ba20 con 0x7f29e415a5e0 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29e8f66700 1 -- 172.21.15.64:0/3287688400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0092e0 con 0x7f29e415a5e0 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 -- 172.21.15.64:0/3287688400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e415a5e0 msgr2=0x7f29e40c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.640+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/3287688400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e415a5e0 0x7f29e40c95f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f29e40b2760 tx=0x7f29dc009840 comp rx=0 tx=0).stop 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- 172.21.15.64:0/3287688400 shutdown_connections 2023-09-27T23:32:08.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/3287688400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e415a5e0 0x7f29e40c95f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/3287688400 >> v2:172.21.15.123:3300/0 conn(0x7f29e406b990 0x7f29e40c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- 172.21.15.64:0/3287688400 >> 172.21.15.64:0/3287688400 conn(0x7f29e4067410 msgr2=0x7f29e40677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:08.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- 172.21.15.64:0/3287688400 shutdown_connections 2023-09-27T23:32:08.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- 172.21.15.64:0/3287688400 wait complete. 2023-09-27T23:32:08.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 Processor -- start 2023-09-27T23:32:08.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- start start 2023-09-27T23:32:08.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 0x7f29e40ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e415a210 0x7f29e4147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415a5e0 0x7f29e415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40acea0 con 0x7f29e406b990 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b0210 con 0x7f29e415a5e0 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40ac510 con 0x7f29e415a210 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 0x7f29e40ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 0x7f29e40ac960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50138/0 (socket says 172.21.15.64:50138) 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 -- 172.21.15.64:0/1029922760 learned_addr learned my addr 172.21.15.64:0/1029922760 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9767700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e415a210 0x7f29e4147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ea769700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415a5e0 0x7f29e415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 -- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e415a210 msgr2=0x7f29e4147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e415a210 0x7f29e4147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 -- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415a5e0 msgr2=0x7f29e415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415a5e0 0x7f29e415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 -- 172.21.15.64:0/1029922760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009ae0 con 0x7f29e406b990 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9767700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e415a210 0x7f29e4147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29e9f68700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 0x7f29e40ac960 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c590 tx=0x7f29d400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011590 con 0x7f29e406b990 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4011710 con 0x7f29e406b990 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e40b0460 con 0x7f29e406b990 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011ad0 con 0x7f29e406b990 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.644+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e40b0a20 con 0x7f29e406b990 2023-09-27T23:32:08.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.648+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e406e9d0 con 0x7f29e406b990 2023-09-27T23:32:08.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.648+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f29d401b3b0 con 0x7f29e406b990 2023-09-27T23:32:08.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.648+0000 7f29d9ffb700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29c8060a70 0x7f29c8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.648+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f29d407c6d0 con 0x7f29e406b990 2023-09-27T23:32:08.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.648+0000 7f29e9767700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29c8060a70 0x7f29c8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:08.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.652+0000 7f29e9767700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29c8060a70 0x7f29c8062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29e4153fc0 tx=0x7f29e0008040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.656+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f29e406e9d0 con 0x7f29e406b990 2023-09-27T23:32:08.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.864+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29e4071120 con 0x7f29e406b990 2023-09-27T23:32:08.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.864+0000 7f29d9ffb700 1 -- 172.21.15.64:0/1029922760 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f29d404c660 con 0x7f29e406b990 2023-09-27T23:32:08.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:08.871 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:32:05.833505+0000","modified":"2023-09-27T23:32:08.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8151},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8151":{"gid":8151,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4228640199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4228640199},{"type":"v1","addr":"172.21.15.64:6835","nonce":4228640199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-09-27T23:32:08.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29c8060a70 msgr2=0x7f29c8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29c8060a70 0x7f29c8062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29e4153fc0 tx=0x7f29e0008040 comp rx=0 tx=0).stop 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 msgr2=0x7f29e40ac960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 0x7f29e40ac960 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c590 tx=0x7f29d400c8a0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 shutdown_connections 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f29c8060a70 0x7f29c8062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415a5e0 0x7f29e415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e415a210 0x7f29e4147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 --2- 172.21.15.64:0/1029922760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29e406b990 0x7f29e40ac960 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 >> 172.21.15.64:0/1029922760 conn(0x7f29e4067410 msgr2=0x7f29e4149380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 shutdown_connections 2023-09-27T23:32:08.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.868+0000 7f29ec1cc700 1 -- 172.21.15.64:0/1029922760 wait complete. 2023-09-27T23:32:08.875 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-09-27T23:32:08.886 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:32:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fee04e39700 1 Processor -- start 2023-09-27T23:32:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fee04e39700 1 -- start start 2023-09-27T23:32:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fee04e39700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee0006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fee04e39700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0006b470 con 0x7fee000c6900 2023-09-27T23:32:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fee04e39700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0006e110 con 0x7fee000c6cd0 2023-09-27T23:32:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fee04e39700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0015d7b0 con 0x7fee0015d330 2023-09-27T23:32:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fedff7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fee000c6900 0x7fee0006da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44452/0 (socket says 172.21.15.64:44452) 2023-09-27T23:32:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fedff7fe700 1 -- 172.21.15.64:0/4228407270 learned_addr learned my addr 172.21.15.64:0/4228407270 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fedfffff700 1 --2- 172.21.15.64:0/4228407270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee0006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369783606 0 0) 0x7fee0006b470 con 0x7fee000c6900 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.992+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedec003570 con 0x7fee000c6900 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407429854 0 0) 0x7fee0006e110 con 0x7fee000c6cd0 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee0006b470 con 0x7fee000c6cd0 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfffff700 1 -- 172.21.15.64:0/4228407270 >> v1:172.21.15.123:6789/0 conn(0x7fee000c6cd0 legacy=0x7fee0015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfffff700 1 -- 172.21.15.64:0/4228407270 >> v1:172.21.15.64:6789/0 conn(0x7fee000c6900 legacy=0x7fee0006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfffff700 1 -- 172.21.15.64:0/4228407270 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00161d10 con 0x7fee0015d330 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfffff700 1 --2- 172.21.15.64:0/4228407270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee0006b900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009e90 tx=0x7fedf8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c3ad83c45e1bd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800e070 con 0x7fee0015d330 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf800bb80 con 0x7fee0015d330 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fedfe7fc700 1 -- 172.21.15.64:0/4228407270 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8012430 con 0x7fee0015d330 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 -- 172.21.15.64:0/4228407270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 msgr2=0x7fee0006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 --2- 172.21.15.64:0/4228407270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee0006b900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009e90 tx=0x7fedf8009780 comp rx=0 tx=0).stop 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 -- 172.21.15.64:0/4228407270 shutdown_connections 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 --2- 172.21.15.64:0/4228407270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee0006b900 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 -- 172.21.15.64:0/4228407270 >> 172.21.15.64:0/4228407270 conn(0x7fee00067bb0 msgr2=0x7fee000cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 -- 172.21.15.64:0/4228407270 shutdown_connections 2023-09-27T23:32:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 -- 172.21.15.64:0/4228407270 wait complete. 2023-09-27T23:32:09.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 Processor -- start 2023-09-27T23:32:09.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:08.996+0000 7fee04e39700 1 -- start start 2023-09-27T23:32:09.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 0x7fee00160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee000c6cd0 0x7fee00157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee00157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000ade90 con 0x7fee000c6900 2023-09-27T23:32:09.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000af6c0 con 0x7fee000c6cd0 2023-09-27T23:32:09.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee000af9a0 con 0x7fee0015d330 2023-09-27T23:32:09.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 0x7fee00160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 0x7fee00160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50140/0 (socket says 172.21.15.64:50140) 2023-09-27T23:32:09.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 -- 172.21.15.64:0/758823851 learned_addr learned my addr 172.21.15.64:0/758823851 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:09.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedfeffd700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee000c6cd0 0x7fee00157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedfffff700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee00157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 -- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 msgr2=0x7fee00157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee00157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 -- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee000c6cd0 msgr2=0x7fee00157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee000c6cd0 0x7fee00157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 -- 172.21.15.64:0/758823851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf8009a20 con 0x7fee000c6900 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedff7fe700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 0x7fee00160260 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fedf000c790 tx=0x7fedf0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0005b30 con 0x7fee000c6900 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf0005cd0 con 0x7fee000c6900 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0007320 con 0x7fee000c6900 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee000afc80 con 0x7fee000c6900 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.000+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0016d530 con 0x7fee000c6900 2023-09-27T23:32:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.004+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0004be40 con 0x7fee000c6900 2023-09-27T23:32:09.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.004+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fedf0012400 con 0x7fee000c6900 2023-09-27T23:32:09.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.004+0000 7fedf77fe700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fede0060a70 0x7fede0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.004+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fedf007c4b0 con 0x7fee000c6900 2023-09-27T23:32:09.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.004+0000 7fedfeffd700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fede0060a70 0x7fede0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.004+0000 7fedfeffd700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fede0060a70 0x7fede0062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fd0 tx=0x7fede8007d50 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.016+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fedf0049bb0 con 0x7fee000c6900 2023-09-27T23:32:09.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.176+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee000cc490 con 0x7fee000c6900 2023-09-27T23:32:09.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fedf77fe700 1 -- 172.21.15.64:0/758823851 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+17085 (secure 0 0 0) 0x7fedf0017090 con 0x7fee000c6900 2023-09-27T23:32:09.185 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:09.185 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":129,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:32:06.836568+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:32:04.410437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-09-27T23:32:04.813612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:32:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fede0060a70 msgr2=0x7fede0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fede0060a70 0x7fede0062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fd0 tx=0x7fede8007d50 comp rx=0 tx=0).stop 2023-09-27T23:32:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 msgr2=0x7fee00160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 0x7fee00160260 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fedf000c790 tx=0x7fedf0007e80 comp rx=0 tx=0).stop 2023-09-27T23:32:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 shutdown_connections 2023-09-27T23:32:09.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fede0060a70 0x7fede0062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee0015d330 0x7fee00157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee000c6cd0 0x7fee00157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 --2- 172.21.15.64:0/758823851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee000c6900 0x7fee00160260 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 >> 172.21.15.64:0/758823851 conn(0x7fee00067bb0 msgr2=0x7fee00159820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 shutdown_connections 2023-09-27T23:32:09.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.180+0000 7fee04e39700 1 -- 172.21.15.64:0/758823851 wait complete. 2023-09-27T23:32:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:32:09.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:32:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 Processor -- start 2023-09-27T23:32:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 -- start start 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff8158cb0 0x7f7ff80b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ff806b4f0 0x7f7ff806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff806d470 con 0x7f7ff80a8850 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff806c860 con 0x7f7ff806b4f0 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffdca8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff806cb40 con 0x7f7ff8158cb0 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffcca6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff8158cb0 0x7f7ff80b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ff7fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7ff80a8850 0x7f7ff806ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:55600/0 (socket says 172.21.15.64:55600) 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ff7fff700 1 -- 172.21.15.64:0/4078744989 learned_addr learned my addr 172.21.15.64:0/4078744989 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.300+0000 7f7ffd4a7700 1 --2- 172.21.15.64:0/4078744989 >> v2:172.21.15.123:3300/0 conn(0x7f7ff806b4f0 0x7f7ff806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ff77fe700 1 -- 172.21.15.64:0/4078744989 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074165317 0 0) 0x7f7ff806d470 con 0x7f7ff80a8850 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ff77fe700 1 -- 172.21.15.64:0/4078744989 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe4003570 con 0x7f7ff80a8850 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff8158cb0 msgr2=0x7f7ff80b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffd4a7700 1 --2- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff8158cb0 0x7f7ff80b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/4078744989 >> v1:172.21.15.123:6789/0 conn(0x7f7ff80a8850 legacy=0x7f7ff806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/4078744989 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff806ce20 con 0x7f7ff806b4f0 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffcca6700 1 --2- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff8158cb0 0x7f7ff80b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffd4a7700 1 --2- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff806b4f0 0x7f7ff806d900 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009910 tx=0x7f7ff00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dd619c30f1391ce server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ff77fe700 1 -- 172.21.15.64:0/4078744989 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0011630 con 0x7f7ff806b4f0 2023-09-27T23:32:09.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ff77fe700 1 -- 172.21.15.64:0/4078744989 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0011790 con 0x7f7ff806b4f0 2023-09-27T23:32:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ff77fe700 1 -- 172.21.15.64:0/4078744989 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0011b30 con 0x7f7ff806b4f0 2023-09-27T23:32:09.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 -- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff806b4f0 msgr2=0x7f7ff806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff806b4f0 0x7f7ff806d900 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009910 tx=0x7f7ff00091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 -- 172.21.15.64:0/4078744989 shutdown_connections 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff806b4f0 0x7f7ff806d900 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/4078744989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff8158cb0 0x7f7ff80b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 -- 172.21.15.64:0/4078744989 >> 172.21.15.64:0/4078744989 conn(0x7f7ff80684d0 msgr2=0x7f7ff80688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 -- 172.21.15.64:0/4078744989 shutdown_connections 2023-09-27T23:32:09.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.304+0000 7f7ffdca8700 1 -- 172.21.15.64:0/4078744989 wait complete. 2023-09-27T23:32:09.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 Processor -- start 2023-09-27T23:32:09.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 -- start start 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff806b4f0 0x7f7ff8149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff80a8850 0x7f7ff8159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 0x7f7ff8169300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff816a2a0 con 0x7f7ff8158cb0 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8159590 con 0x7f7ff80a8850 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.308+0000 7f7ffdca8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8169ce0 con 0x7f7ff806b4f0 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 0x7f7ff8169300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffcca6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff806b4f0 0x7f7ff8149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff80a8850 0x7f7ff8159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 0x7f7ff8169300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50150/0 (socket says 172.21.15.64:50150) 2023-09-27T23:32:09.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/639748895 learned_addr learned my addr 172.21.15.64:0/639748895 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff806b4f0 msgr2=0x7f7ff8149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff806b4f0 0x7f7ff8149c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff80a8850 msgr2=0x7f7ff8159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff80a8850 0x7f7ff8159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 -- 172.21.15.64:0/639748895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8009420 con 0x7f7ff8158cb0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffd4a7700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 0x7f7ff8169300 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0013000 tx=0x7f7ff0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0011a50 con 0x7f7ff8158cb0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0025420 con 0x7f7ff8158cb0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff0009420 con 0x7f7ff8158cb0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00257e0 con 0x7f7ff8158cb0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff8159220 con 0x7f7ff8158cb0 2023-09-27T23:32:09.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff8159700 con 0x7f7ff8158cb0 2023-09-27T23:32:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7ff0023070 con 0x7f7ff8158cb0 2023-09-27T23:32:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.312+0000 7f7ff4ff9700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7fd8060a70 0x7f7fd8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.316+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7ff007e6d0 con 0x7f7ff8158cb0 2023-09-27T23:32:09.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.316+0000 7f7ffcca6700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7fd8060a70 0x7f7fd8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.316+0000 7f7ffcca6700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7fd8060a70 0x7f7fd8062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009f40 tx=0x7f7fe80091a0 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.324+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7ff004e700 con 0x7f7ff8158cb0 2023-09-27T23:32:09.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.480+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ff804be40 con 0x7f7fd8060a70 2023-09-27T23:32:09.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.500+0000 7f7ff4ff9700 1 -- 172.21.15.64:0/639748895 <== mgr.6971 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303530 (secure 0 0 0) 0x7f7ff804be40 con 0x7f7fd8060a70 2023-09-27T23:32:09.510 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7fd8060a70 msgr2=0x7f7fd8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7fd8060a70 0x7f7fd8062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009f40 tx=0x7f7fe80091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 msgr2=0x7f7ff8169300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 0x7f7ff8169300 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0013000 tx=0x7f7ff0017000 comp rx=0 tx=0).stop 2023-09-27T23:32:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 shutdown_connections 2023-09-27T23:32:09.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7fd8060a70 0x7f7fd8062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff8158cb0 0x7f7ff8169300 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ff80a8850 0x7f7ff8159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 --2- 172.21.15.64:0/639748895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ff806b4f0 0x7f7ff8149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 >> 172.21.15.64:0/639748895 conn(0x7f7ff80684d0 msgr2=0x7f7ff8157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 shutdown_connections 2023-09-27T23:32:09.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.504+0000 7f7ffdca8700 1 -- 172.21.15.64:0/639748895 wait complete. 2023-09-27T23:32:09.514 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:32:09.527 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:32:08.730336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":389,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":251,"ondisk_log_size":251,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229568,"kb_used_data":13504,"kb_used_omap":0,"kb_used_meta":216000,"kb_avail":754745152,"statfs":{"total":773094113280,"available":772859035648,"internally_reserved":0,"allocated":13828096,"data_stored":7721154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":221184000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":104,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.280290"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863665+0000","last_change":"2023-09-27T23:32:05.860722+0000","last_active":"2023-09-27T23:32:05.863665+0000","last_peered":"2023-09-27T23:32:05.863665+0000","last_clean":"2023-09-27T23:32:05.863665+0000","last_became_active":"2023-09-27T23:32:05.860381+0000","last_became_peered":"2023-09-27T23:32:05.860381+0000","last_unstale":"2023-09-27T23:32:05.863665+0000","last_undegraded":"2023-09-27T23:32:05.863665+0000","last_fullsized":"2023-09-27T23:32:05.863665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879763+0000","last_change":"2023-09-27T23:32:05.873945+0000","last_active":"2023-09-27T23:32:05.879763+0000","last_peered":"2023-09-27T23:32:05.879763+0000","last_clean":"2023-09-27T23:32:05.879763+0000","last_became_active":"2023-09-27T23:32:05.872503+0000","last_became_peered":"2023-09-27T23:32:05.872503+0000","last_unstale":"2023-09-27T23:32:05.879763+0000","last_undegraded":"2023-09-27T23:32:05.879763+0000","last_fullsized":"2023-09-27T23:32:05.879763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:06.052849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.866846+0000","last_change":"2023-09-27T23:32:05.863147+0000","last_active":"2023-09-27T23:32:05.866846+0000","last_peered":"2023-09-27T23:32:05.866846+0000","last_clean":"2023-09-27T23:32:05.866846+0000","last_became_active":"2023-09-27T23:32:05.862602+0000","last_became_peered":"2023-09-27T23:32:05.862602+0000","last_unstale":"2023-09-27T23:32:05.866846+0000","last_undegraded":"2023-09-27T23:32:05.866846+0000","last_fullsized":"2023-09-27T23:32:05.866846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:01.411281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867813+0000","last_change":"2023-09-27T23:32:05.861426+0000","last_active":"2023-09-27T23:32:05.867813+0000","last_peered":"2023-09-27T23:32:05.867813+0000","last_clean":"2023-09-27T23:32:05.867813+0000","last_became_active":"2023-09-27T23:32:05.861004+0000","last_became_peered":"2023-09-27T23:32:05.861004+0000","last_unstale":"2023-09-27T23:32:05.867813+0000","last_undegraded":"2023-09-27T23:32:05.867813+0000","last_fullsized":"2023-09-27T23:32:05.867813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:09.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881110+0000","last_change":"2023-09-27T23:32:05.878031+0000","last_active":"2023-09-27T23:32:05.881110+0000","last_peered":"2023-09-27T23:32:05.881110+0000","last_clean":"2023-09-27T23:32:05.881110+0000","last_became_active":"2023-09-27T23:32:05.877499+0000","last_became_peered":"2023-09-27T23:32:05.877499+0000","last_unstale":"2023-09-27T23:32:05.881110+0000","last_undegraded":"2023-09-27T23:32:05.881110+0000","last_fullsized":"2023-09-27T23:32:05.881110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:24.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865013+0000","last_change":"2023-09-27T23:32:05.861704+0000","last_active":"2023-09-27T23:32:05.865013+0000","last_peered":"2023-09-27T23:32:05.865013+0000","last_clean":"2023-09-27T23:32:05.865013+0000","last_became_active":"2023-09-27T23:32:05.861317+0000","last_became_peered":"2023-09-27T23:32:05.861317+0000","last_unstale":"2023-09-27T23:32:05.865013+0000","last_undegraded":"2023-09-27T23:32:05.865013+0000","last_fullsized":"2023-09-27T23:32:05.865013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881099+0000","last_change":"2023-09-27T23:32:05.877897+0000","last_active":"2023-09-27T23:32:05.881099+0000","last_peered":"2023-09-27T23:32:05.881099+0000","last_clean":"2023-09-27T23:32:05.881099+0000","last_became_active":"2023-09-27T23:32:05.877494+0000","last_became_peered":"2023-09-27T23:32:05.877494+0000","last_unstale":"2023-09-27T23:32:05.881099+0000","last_undegraded":"2023-09-27T23:32:05.881099+0000","last_fullsized":"2023-09-27T23:32:05.881099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:55.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879623+0000","last_change":"2023-09-27T23:32:05.872800+0000","last_active":"2023-09-27T23:32:05.879623+0000","last_peered":"2023-09-27T23:32:05.879623+0000","last_clean":"2023-09-27T23:32:05.879623+0000","last_became_active":"2023-09-27T23:32:05.872486+0000","last_became_peered":"2023-09-27T23:32:05.872486+0000","last_unstale":"2023-09-27T23:32:05.879623+0000","last_undegraded":"2023-09-27T23:32:05.879623+0000","last_fullsized":"2023-09-27T23:32:05.879623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:57.553491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873468+0000","last_change":"2023-09-27T23:32:05.872253+0000","last_active":"2023-09-27T23:32:05.873468+0000","last_peered":"2023-09-27T23:32:05.873468+0000","last_clean":"2023-09-27T23:32:05.873468+0000","last_became_active":"2023-09-27T23:32:05.871935+0000","last_became_peered":"2023-09-27T23:32:05.871935+0000","last_unstale":"2023-09-27T23:32:05.873468+0000","last_undegraded":"2023-09-27T23:32:05.873468+0000","last_fullsized":"2023-09-27T23:32:05.873468+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:00.996007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865348+0000","last_change":"2023-09-27T23:32:05.863475+0000","last_active":"2023-09-27T23:32:05.865348+0000","last_peered":"2023-09-27T23:32:05.865348+0000","last_clean":"2023-09-27T23:32:05.865348+0000","last_became_active":"2023-09-27T23:32:05.863171+0000","last_became_peered":"2023-09-27T23:32:05.863171+0000","last_unstale":"2023-09-27T23:32:05.865348+0000","last_undegraded":"2023-09-27T23:32:05.865348+0000","last_fullsized":"2023-09-27T23:32:05.865348+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862603+0000","last_change":"2023-09-27T23:32:05.859721+0000","last_active":"2023-09-27T23:32:05.862603+0000","last_peered":"2023-09-27T23:32:05.862603+0000","last_clean":"2023-09-27T23:32:05.862603+0000","last_became_active":"2023-09-27T23:32:05.859426+0000","last_became_peered":"2023-09-27T23:32:05.859426+0000","last_unstale":"2023-09-27T23:32:05.862603+0000","last_undegraded":"2023-09-27T23:32:05.862603+0000","last_fullsized":"2023-09-27T23:32:05.862603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:39.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879745+0000","last_change":"2023-09-27T23:32:05.874339+0000","last_active":"2023-09-27T23:32:05.879745+0000","last_peered":"2023-09-27T23:32:05.879745+0000","last_clean":"2023-09-27T23:32:05.879745+0000","last_became_active":"2023-09-27T23:32:05.872716+0000","last_became_peered":"2023-09-27T23:32:05.872716+0000","last_unstale":"2023-09-27T23:32:05.879745+0000","last_undegraded":"2023-09-27T23:32:05.879745+0000","last_fullsized":"2023-09-27T23:32:05.879745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:21.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871701+0000","last_change":"2023-09-27T23:32:05.870769+0000","last_active":"2023-09-27T23:32:05.871701+0000","last_peered":"2023-09-27T23:32:05.871701+0000","last_clean":"2023-09-27T23:32:05.871701+0000","last_became_active":"2023-09-27T23:32:05.870469+0000","last_became_peered":"2023-09-27T23:32:05.870469+0000","last_unstale":"2023-09-27T23:32:05.871701+0000","last_undegraded":"2023-09-27T23:32:05.871701+0000","last_fullsized":"2023-09-27T23:32:05.871701+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:01:42.856864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867672+0000","last_change":"2023-09-27T23:32:05.865682+0000","last_active":"2023-09-27T23:32:05.867672+0000","last_peered":"2023-09-27T23:32:05.867672+0000","last_clean":"2023-09-27T23:32:05.867672+0000","last_became_active":"2023-09-27T23:32:05.865377+0000","last_became_peered":"2023-09-27T23:32:05.865377+0000","last_unstale":"2023-09-27T23:32:05.867672+0000","last_undegraded":"2023-09-27T23:32:05.867672+0000","last_fullsized":"2023-09-27T23:32:05.867672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868870+0000","last_change":"2023-09-27T23:32:05.860566+0000","last_active":"2023-09-27T23:32:05.868870+0000","last_peered":"2023-09-27T23:32:05.868870+0000","last_clean":"2023-09-27T23:32:05.868870+0000","last_became_active":"2023-09-27T23:32:05.860209+0000","last_became_peered":"2023-09-27T23:32:05.860209+0000","last_unstale":"2023-09-27T23:32:05.868870+0000","last_undegraded":"2023-09-27T23:32:05.868870+0000","last_fullsized":"2023-09-27T23:32:05.868870+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:42.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875841+0000","last_change":"2023-09-27T23:32:05.873642+0000","last_active":"2023-09-27T23:32:05.875841+0000","last_peered":"2023-09-27T23:32:05.875841+0000","last_clean":"2023-09-27T23:32:05.875841+0000","last_became_active":"2023-09-27T23:32:05.873228+0000","last_became_peered":"2023-09-27T23:32:05.873228+0000","last_unstale":"2023-09-27T23:32:05.875841+0000","last_undegraded":"2023-09-27T23:32:05.875841+0000","last_fullsized":"2023-09-27T23:32:05.875841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:26:39.116439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871647+0000","last_change":"2023-09-27T23:32:05.867910+0000","last_active":"2023-09-27T23:32:05.871647+0000","last_peered":"2023-09-27T23:32:05.871647+0000","last_clean":"2023-09-27T23:32:05.871647+0000","last_became_active":"2023-09-27T23:32:05.867511+0000","last_became_peered":"2023-09-27T23:32:05.867511+0000","last_unstale":"2023-09-27T23:32:05.871647+0000","last_undegraded":"2023-09-27T23:32:05.871647+0000","last_fullsized":"2023-09-27T23:32:05.871647+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:31:21.387831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865328+0000","last_change":"2023-09-27T23:32:05.863332+0000","last_active":"2023-09-27T23:32:05.865328+0000","last_peered":"2023-09-27T23:32:05.865328+0000","last_clean":"2023-09-27T23:32:05.865328+0000","last_became_active":"2023-09-27T23:32:05.863052+0000","last_became_peered":"2023-09-27T23:32:05.863052+0000","last_unstale":"2023-09-27T23:32:05.865328+0000","last_undegraded":"2023-09-27T23:32:05.865328+0000","last_fullsized":"2023-09-27T23:32:05.865328+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873422+0000","last_change":"2023-09-27T23:32:05.871483+0000","last_active":"2023-09-27T23:32:05.873422+0000","last_peered":"2023-09-27T23:32:05.873422+0000","last_clean":"2023-09-27T23:32:05.873422+0000","last_became_active":"2023-09-27T23:32:05.871087+0000","last_became_peered":"2023-09-27T23:32:05.871087+0000","last_unstale":"2023-09-27T23:32:05.873422+0000","last_undegraded":"2023-09-27T23:32:05.873422+0000","last_fullsized":"2023-09-27T23:32:05.873422+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:16.072601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881812+0000","last_change":"2023-09-27T23:32:05.879569+0000","last_active":"2023-09-27T23:32:05.881812+0000","last_peered":"2023-09-27T23:32:05.881812+0000","last_clean":"2023-09-27T23:32:05.881812+0000","last_became_active":"2023-09-27T23:32:05.879296+0000","last_became_peered":"2023-09-27T23:32:05.879296+0000","last_unstale":"2023-09-27T23:32:05.881812+0000","last_undegraded":"2023-09-27T23:32:05.881812+0000","last_fullsized":"2023-09-27T23:32:05.881812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:42.688153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876663+0000","last_change":"2023-09-27T23:32:05.875209+0000","last_active":"2023-09-27T23:32:05.876663+0000","last_peered":"2023-09-27T23:32:05.876663+0000","last_clean":"2023-09-27T23:32:05.876663+0000","last_became_active":"2023-09-27T23:32:05.874886+0000","last_became_peered":"2023-09-27T23:32:05.874886+0000","last_unstale":"2023-09-27T23:32:05.876663+0000","last_undegraded":"2023-09-27T23:32:05.876663+0000","last_fullsized":"2023-09-27T23:32:05.876663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868346+0000","last_change":"2023-09-27T23:32:05.865108+0000","last_active":"2023-09-27T23:32:05.868346+0000","last_peered":"2023-09-27T23:32:05.868346+0000","last_clean":"2023-09-27T23:32:05.868346+0000","last_became_active":"2023-09-27T23:32:05.864839+0000","last_became_peered":"2023-09-27T23:32:05.864839+0000","last_unstale":"2023-09-27T23:32:05.868346+0000","last_undegraded":"2023-09-27T23:32:05.868346+0000","last_fullsized":"2023-09-27T23:32:05.868346+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:24:55.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865011+0000","last_change":"2023-09-27T23:32:05.859858+0000","last_active":"2023-09-27T23:32:05.865011+0000","last_peered":"2023-09-27T23:32:05.865011+0000","last_clean":"2023-09-27T23:32:05.865011+0000","last_became_active":"2023-09-27T23:32:05.859569+0000","last_became_peered":"2023-09-27T23:32:05.859569+0000","last_unstale":"2023-09-27T23:32:05.865011+0000","last_undegraded":"2023-09-27T23:32:05.865011+0000","last_fullsized":"2023-09-27T23:32:05.865011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:40.168536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871871+0000","last_change":"2023-09-27T23:32:05.867437+0000","last_active":"2023-09-27T23:32:05.871871+0000","last_peered":"2023-09-27T23:32:05.871871+0000","last_clean":"2023-09-27T23:32:05.871871+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871871+0000","last_undegraded":"2023-09-27T23:32:05.871871+0000","last_fullsized":"2023-09-27T23:32:05.871871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:22.332265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877163+0000","last_change":"2023-09-27T23:32:05.474605+0000","last_active":"2023-09-27T23:32:05.877163+0000","last_peered":"2023-09-27T23:32:05.877163+0000","last_clean":"2023-09-27T23:32:05.877163+0000","last_became_active":"2023-09-27T23:32:05.474269+0000","last_became_peered":"2023-09-27T23:32:05.474269+0000","last_unstale":"2023-09-27T23:32:05.877163+0000","last_undegraded":"2023-09-27T23:32:05.877163+0000","last_fullsized":"2023-09-27T23:32:05.877163+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864812+0000","last_change":"2023-09-27T23:32:05.858996+0000","last_active":"2023-09-27T23:32:05.864812+0000","last_peered":"2023-09-27T23:32:05.864812+0000","last_clean":"2023-09-27T23:32:05.864812+0000","last_became_active":"2023-09-27T23:32:05.858668+0000","last_became_peered":"2023-09-27T23:32:05.858668+0000","last_unstale":"2023-09-27T23:32:05.864812+0000","last_undegraded":"2023-09-27T23:32:05.864812+0000","last_fullsized":"2023-09-27T23:32:05.864812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:35:43.781059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862048+0000","last_change":"2023-09-27T23:32:05.477638+0000","last_active":"2023-09-27T23:32:05.862048+0000","last_peered":"2023-09-27T23:32:05.862048+0000","last_clean":"2023-09-27T23:32:05.862048+0000","last_became_active":"2023-09-27T23:32:05.477296+0000","last_became_peered":"2023-09-27T23:32:05.477296+0000","last_unstale":"2023-09-27T23:32:05.862048+0000","last_undegraded":"2023-09-27T23:32:05.862048+0000","last_fullsized":"2023-09-27T23:32:05.862048+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:24.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881733+0000","last_change":"2023-09-27T23:32:05.879475+0000","last_active":"2023-09-27T23:32:05.881733+0000","last_peered":"2023-09-27T23:32:05.881733+0000","last_clean":"2023-09-27T23:32:05.881733+0000","last_became_active":"2023-09-27T23:32:05.877744+0000","last_became_peered":"2023-09-27T23:32:05.877744+0000","last_unstale":"2023-09-27T23:32:05.881733+0000","last_undegraded":"2023-09-27T23:32:05.881733+0000","last_fullsized":"2023-09-27T23:32:05.881733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:18.120648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862399+0000","last_change":"2023-09-27T23:32:05.478152+0000","last_active":"2023-09-27T23:32:05.862399+0000","last_peered":"2023-09-27T23:32:05.862399+0000","last_clean":"2023-09-27T23:32:05.862399+0000","last_became_active":"2023-09-27T23:32:05.477854+0000","last_became_peered":"2023-09-27T23:32:05.477854+0000","last_unstale":"2023-09-27T23:32:05.862399+0000","last_undegraded":"2023-09-27T23:32:05.862399+0000","last_fullsized":"2023-09-27T23:32:05.862399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865090+0000","last_change":"2023-09-27T23:32:05.861730+0000","last_active":"2023-09-27T23:32:05.865090+0000","last_peered":"2023-09-27T23:32:05.865090+0000","last_clean":"2023-09-27T23:32:05.865090+0000","last_became_active":"2023-09-27T23:32:05.861021+0000","last_became_peered":"2023-09-27T23:32:05.861021+0000","last_unstale":"2023-09-27T23:32:05.865090+0000","last_undegraded":"2023-09-27T23:32:05.865090+0000","last_fullsized":"2023-09-27T23:32:05.865090+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862148+0000","last_change":"2023-09-27T23:32:05.474408+0000","last_active":"2023-09-27T23:32:05.862148+0000","last_peered":"2023-09-27T23:32:05.862148+0000","last_clean":"2023-09-27T23:32:05.862148+0000","last_became_active":"2023-09-27T23:32:05.473091+0000","last_became_peered":"2023-09-27T23:32:05.473091+0000","last_unstale":"2023-09-27T23:32:05.862148+0000","last_undegraded":"2023-09-27T23:32:05.862148+0000","last_fullsized":"2023-09-27T23:32:05.862148+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:20.064087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855524+0000","last_change":"2023-09-27T23:32:05.486286+0000","last_active":"2023-09-27T23:32:05.855524+0000","last_peered":"2023-09-27T23:32:05.855524+0000","last_clean":"2023-09-27T23:32:05.855524+0000","last_became_active":"2023-09-27T23:32:05.485968+0000","last_became_peered":"2023-09-27T23:32:05.485968+0000","last_unstale":"2023-09-27T23:32:05.855524+0000","last_undegraded":"2023-09-27T23:32:05.855524+0000","last_fullsized":"2023-09-27T23:32:05.855524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:20.522455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.883531+0000","last_change":"2023-09-27T23:32:05.882768+0000","last_active":"2023-09-27T23:32:05.883531+0000","last_peered":"2023-09-27T23:32:05.883531+0000","last_clean":"2023-09-27T23:32:05.883531+0000","last_became_active":"2023-09-27T23:32:05.882322+0000","last_became_peered":"2023-09-27T23:32:05.882322+0000","last_unstale":"2023-09-27T23:32:05.883531+0000","last_undegraded":"2023-09-27T23:32:05.883531+0000","last_fullsized":"2023-09-27T23:32:05.883531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:46.976392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877676+0000","last_change":"2023-09-27T23:32:05.875211+0000","last_active":"2023-09-27T23:32:05.877676+0000","last_peered":"2023-09-27T23:32:05.877676+0000","last_clean":"2023-09-27T23:32:05.877676+0000","last_became_active":"2023-09-27T23:32:05.874899+0000","last_became_peered":"2023-09-27T23:32:05.874899+0000","last_unstale":"2023-09-27T23:32:05.877676+0000","last_undegraded":"2023-09-27T23:32:05.877676+0000","last_fullsized":"2023-09-27T23:32:05.877676+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:38.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.858932+0000","last_change":"2023-09-27T23:32:05.474634+0000","last_active":"2023-09-27T23:32:05.858932+0000","last_peered":"2023-09-27T23:32:05.858932+0000","last_clean":"2023-09-27T23:32:05.858932+0000","last_became_active":"2023-09-27T23:32:05.474314+0000","last_became_peered":"2023-09-27T23:32:05.474314+0000","last_unstale":"2023-09-27T23:32:05.858932+0000","last_undegraded":"2023-09-27T23:32:05.858932+0000","last_fullsized":"2023-09-27T23:32:05.858932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:05.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885276+0000","last_change":"2023-09-27T23:32:05.884516+0000","last_active":"2023-09-27T23:32:05.885276+0000","last_peered":"2023-09-27T23:32:05.885276+0000","last_clean":"2023-09-27T23:32:05.885276+0000","last_became_active":"2023-09-27T23:32:05.884276+0000","last_became_peered":"2023-09-27T23:32:05.884276+0000","last_unstale":"2023-09-27T23:32:05.885276+0000","last_undegraded":"2023-09-27T23:32:05.885276+0000","last_fullsized":"2023-09-27T23:32:05.885276+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:14.206863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871586+0000","last_change":"2023-09-27T23:32:05.483320+0000","last_active":"2023-09-27T23:32:05.871586+0000","last_peered":"2023-09-27T23:32:05.871586+0000","last_clean":"2023-09-27T23:32:05.871586+0000","last_became_active":"2023-09-27T23:32:05.482969+0000","last_became_peered":"2023-09-27T23:32:05.482969+0000","last_unstale":"2023-09-27T23:32:05.871586+0000","last_undegraded":"2023-09-27T23:32:05.871586+0000","last_fullsized":"2023-09-27T23:32:05.871586+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:56.542035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881069+0000","last_change":"2023-09-27T23:32:05.877251+0000","last_active":"2023-09-27T23:32:05.881069+0000","last_peered":"2023-09-27T23:32:05.881069+0000","last_clean":"2023-09-27T23:32:05.881069+0000","last_became_active":"2023-09-27T23:32:05.876868+0000","last_became_peered":"2023-09-27T23:32:05.876868+0000","last_unstale":"2023-09-27T23:32:05.881069+0000","last_undegraded":"2023-09-27T23:32:05.881069+0000","last_fullsized":"2023-09-27T23:32:05.881069+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:10:56.379380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879641+0000","last_change":"2023-09-27T23:32:05.873902+0000","last_active":"2023-09-27T23:32:05.879641+0000","last_peered":"2023-09-27T23:32:05.879641+0000","last_clean":"2023-09-27T23:32:05.879641+0000","last_became_active":"2023-09-27T23:32:05.872995+0000","last_became_peered":"2023-09-27T23:32:05.872995+0000","last_unstale":"2023-09-27T23:32:05.879641+0000","last_undegraded":"2023-09-27T23:32:05.879641+0000","last_fullsized":"2023-09-27T23:32:05.879641+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:35.151434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864787+0000","last_change":"2023-09-27T23:32:05.477932+0000","last_active":"2023-09-27T23:32:05.864787+0000","last_peered":"2023-09-27T23:32:05.864787+0000","last_clean":"2023-09-27T23:32:05.864787+0000","last_became_active":"2023-09-27T23:32:05.477556+0000","last_became_peered":"2023-09-27T23:32:05.477556+0000","last_unstale":"2023-09-27T23:32:05.864787+0000","last_undegraded":"2023-09-27T23:32:05.864787+0000","last_fullsized":"2023-09-27T23:32:05.864787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:23:33.015795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847377+0000","last_change":"2023-09-27T23:32:05.473937+0000","last_active":"2023-09-27T23:32:05.847377+0000","last_peered":"2023-09-27T23:32:05.847377+0000","last_clean":"2023-09-27T23:32:05.847377+0000","last_became_active":"2023-09-27T23:32:05.473535+0000","last_became_peered":"2023-09-27T23:32:05.473535+0000","last_unstale":"2023-09-27T23:32:05.847377+0000","last_undegraded":"2023-09-27T23:32:05.847377+0000","last_fullsized":"2023-09-27T23:32:05.847377+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:59.468491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.880779+0000","last_change":"2023-09-27T23:32:05.877232+0000","last_active":"2023-09-27T23:32:05.880779+0000","last_peered":"2023-09-27T23:32:05.880779+0000","last_clean":"2023-09-27T23:32:05.880779+0000","last_became_active":"2023-09-27T23:32:05.876906+0000","last_became_peered":"2023-09-27T23:32:05.876906+0000","last_unstale":"2023-09-27T23:32:05.880779+0000","last_undegraded":"2023-09-27T23:32:05.880779+0000","last_fullsized":"2023-09-27T23:32:05.880779+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:58.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848215+0000","last_change":"2023-09-27T23:32:05.478424+0000","last_active":"2023-09-27T23:32:05.848215+0000","last_peered":"2023-09-27T23:32:05.848215+0000","last_clean":"2023-09-27T23:32:05.848215+0000","last_became_active":"2023-09-27T23:32:05.478120+0000","last_became_peered":"2023-09-27T23:32:05.478120+0000","last_unstale":"2023-09-27T23:32:05.848215+0000","last_undegraded":"2023-09-27T23:32:05.848215+0000","last_fullsized":"2023-09-27T23:32:05.848215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:15.570089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877156+0000","last_change":"2023-09-27T23:32:05.875429+0000","last_active":"2023-09-27T23:32:05.877156+0000","last_peered":"2023-09-27T23:32:05.877156+0000","last_clean":"2023-09-27T23:32:05.877156+0000","last_became_active":"2023-09-27T23:32:05.875127+0000","last_became_peered":"2023-09-27T23:32:05.875127+0000","last_unstale":"2023-09-27T23:32:05.877156+0000","last_undegraded":"2023-09-27T23:32:05.877156+0000","last_fullsized":"2023-09-27T23:32:05.877156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:21.585472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875964+0000","last_change":"2023-09-27T23:32:05.873357+0000","last_active":"2023-09-27T23:32:05.875964+0000","last_peered":"2023-09-27T23:32:05.875964+0000","last_clean":"2023-09-27T23:32:05.875964+0000","last_became_active":"2023-09-27T23:32:05.873037+0000","last_became_peered":"2023-09-27T23:32:05.873037+0000","last_unstale":"2023-09-27T23:32:05.875964+0000","last_undegraded":"2023-09-27T23:32:05.875964+0000","last_fullsized":"2023-09-27T23:32:05.875964+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:54:10.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882131+0000","last_change":"2023-09-27T23:32:05.487770+0000","last_active":"2023-09-27T23:32:05.882131+0000","last_peered":"2023-09-27T23:32:05.882131+0000","last_clean":"2023-09-27T23:32:05.882131+0000","last_became_active":"2023-09-27T23:32:05.487423+0000","last_became_peered":"2023-09-27T23:32:05.487423+0000","last_unstale":"2023-09-27T23:32:05.882131+0000","last_undegraded":"2023-09-27T23:32:05.882131+0000","last_fullsized":"2023-09-27T23:32:05.882131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:34.139978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:06.303516+0000","last_change":"2023-09-27T23:32:05.473896+0000","last_active":"2023-09-27T23:32:06.303516+0000","last_peered":"2023-09-27T23:32:06.303516+0000","last_clean":"2023-09-27T23:32:06.303516+0000","last_became_active":"2023-09-27T23:32:05.473665+0000","last_became_peered":"2023-09-27T23:32:05.473665+0000","last_unstale":"2023-09-27T23:32:06.303516+0000","last_undegraded":"2023-09-27T23:32:06.303516+0000","last_fullsized":"2023-09-27T23:32:06.303516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:17.109191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876032+0000","last_change":"2023-09-27T23:32:05.874148+0000","last_active":"2023-09-27T23:32:05.876032+0000","last_peered":"2023-09-27T23:32:05.876032+0000","last_clean":"2023-09-27T23:32:05.876032+0000","last_became_active":"2023-09-27T23:32:05.873853+0000","last_became_peered":"2023-09-27T23:32:05.873853+0000","last_unstale":"2023-09-27T23:32:05.876032+0000","last_undegraded":"2023-09-27T23:32:05.876032+0000","last_fullsized":"2023-09-27T23:32:05.876032+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:16:14.012071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846161+0000","last_change":"2023-09-27T23:32:05.482613+0000","last_active":"2023-09-27T23:32:05.846161+0000","last_peered":"2023-09-27T23:32:05.846161+0000","last_clean":"2023-09-27T23:32:05.846161+0000","last_became_active":"2023-09-27T23:32:05.482252+0000","last_became_peered":"2023-09-27T23:32:05.482252+0000","last_unstale":"2023-09-27T23:32:05.846161+0000","last_undegraded":"2023-09-27T23:32:05.846161+0000","last_fullsized":"2023-09-27T23:32:05.846161+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:05.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869011+0000","last_change":"2023-09-27T23:32:05.866830+0000","last_active":"2023-09-27T23:32:05.869011+0000","last_peered":"2023-09-27T23:32:05.869011+0000","last_clean":"2023-09-27T23:32:05.869011+0000","last_became_active":"2023-09-27T23:32:05.866498+0000","last_became_peered":"2023-09-27T23:32:05.866498+0000","last_unstale":"2023-09-27T23:32:05.869011+0000","last_undegraded":"2023-09-27T23:32:05.869011+0000","last_fullsized":"2023-09-27T23:32:05.869011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865072+0000","last_change":"2023-09-27T23:32:05.860941+0000","last_active":"2023-09-27T23:32:05.865072+0000","last_peered":"2023-09-27T23:32:05.865072+0000","last_clean":"2023-09-27T23:32:05.865072+0000","last_became_active":"2023-09-27T23:32:05.860543+0000","last_became_peered":"2023-09-27T23:32:05.860543+0000","last_unstale":"2023-09-27T23:32:05.865072+0000","last_undegraded":"2023-09-27T23:32:05.865072+0000","last_fullsized":"2023-09-27T23:32:05.865072+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.852958+0000","last_change":"2023-09-27T23:32:05.479760+0000","last_active":"2023-09-27T23:32:05.852958+0000","last_peered":"2023-09-27T23:32:05.852958+0000","last_clean":"2023-09-27T23:32:05.852958+0000","last_became_active":"2023-09-27T23:32:05.479474+0000","last_became_peered":"2023-09-27T23:32:05.479474+0000","last_unstale":"2023-09-27T23:32:05.852958+0000","last_undegraded":"2023-09-27T23:32:05.852958+0000","last_fullsized":"2023-09-27T23:32:05.852958+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:43.445603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881743+0000","last_change":"2023-09-27T23:32:05.879523+0000","last_active":"2023-09-27T23:32:05.881743+0000","last_peered":"2023-09-27T23:32:05.881743+0000","last_clean":"2023-09-27T23:32:05.881743+0000","last_became_active":"2023-09-27T23:32:05.878009+0000","last_became_peered":"2023-09-27T23:32:05.878009+0000","last_unstale":"2023-09-27T23:32:05.881743+0000","last_undegraded":"2023-09-27T23:32:05.881743+0000","last_fullsized":"2023-09-27T23:32:05.881743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:21:40.330475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847190+0000","last_change":"2023-09-27T23:32:05.473857+0000","last_active":"2023-09-27T23:32:05.847190+0000","last_peered":"2023-09-27T23:32:05.847190+0000","last_clean":"2023-09-27T23:32:05.847190+0000","last_became_active":"2023-09-27T23:32:05.473507+0000","last_became_peered":"2023-09-27T23:32:05.473507+0000","last_unstale":"2023-09-27T23:32:05.847190+0000","last_undegraded":"2023-09-27T23:32:05.847190+0000","last_fullsized":"2023-09-27T23:32:05.847190+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:19.244182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869149+0000","last_change":"2023-09-27T23:32:05.861010+0000","last_active":"2023-09-27T23:32:05.869149+0000","last_peered":"2023-09-27T23:32:05.869149+0000","last_clean":"2023-09-27T23:32:05.869149+0000","last_became_active":"2023-09-27T23:32:05.860737+0000","last_became_peered":"2023-09-27T23:32:05.860737+0000","last_unstale":"2023-09-27T23:32:05.869149+0000","last_undegraded":"2023-09-27T23:32:05.869149+0000","last_fullsized":"2023-09-27T23:32:05.869149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:58:21.738937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"117'227","reported_seq":650,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864897+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:32:05.864897+0000","last_peered":"2023-09-27T23:32:05.864897+0000","last_clean":"2023-09-27T23:32:05.864897+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:32:05.864897+0000","last_undegraded":"2023-09-27T23:32:05.864897+0000","last_fullsized":"2023-09-27T23:32:05.864897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":227,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":110,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-09-27T23:32:01.316522+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:32:01.316522+0000","last_peered":"2023-09-27T23:32:01.316522+0000","last_clean":"2023-09-27T23:32:01.316522+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:32:01.316522+0000","last_undegraded":"2023-09-27T23:32:01.316522+0000","last_fullsized":"2023-09-27T23:32:01.316522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869078+0000","last_change":"2023-09-27T23:32:05.862823+0000","last_active":"2023-09-27T23:32:05.869078+0000","last_peered":"2023-09-27T23:32:05.869078+0000","last_clean":"2023-09-27T23:32:05.869078+0000","last_became_active":"2023-09-27T23:32:05.862677+0000","last_became_peered":"2023-09-27T23:32:05.862677+0000","last_unstale":"2023-09-27T23:32:05.869078+0000","last_undegraded":"2023-09-27T23:32:05.869078+0000","last_fullsized":"2023-09-27T23:32:05.869078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:01.586755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846271+0000","last_change":"2023-09-27T23:32:05.487459+0000","last_active":"2023-09-27T23:32:05.846271+0000","last_peered":"2023-09-27T23:32:05.846271+0000","last_clean":"2023-09-27T23:32:05.846271+0000","last_became_active":"2023-09-27T23:32:05.487130+0000","last_became_peered":"2023-09-27T23:32:05.487130+0000","last_unstale":"2023-09-27T23:32:05.846271+0000","last_undegraded":"2023-09-27T23:32:05.846271+0000","last_fullsized":"2023-09-27T23:32:05.846271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:57.985464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864781+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:32:05.864781+0000","last_peered":"2023-09-27T23:32:05.864781+0000","last_clean":"2023-09-27T23:32:05.864781+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:32:05.864781+0000","last_undegraded":"2023-09-27T23:32:05.864781+0000","last_fullsized":"2023-09-27T23:32:05.864781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871725+0000","last_change":"2023-09-27T23:32:05.864506+0000","last_active":"2023-09-27T23:32:05.871725+0000","last_peered":"2023-09-27T23:32:05.871725+0000","last_clean":"2023-09-27T23:32:05.871725+0000","last_became_active":"2023-09-27T23:32:05.864207+0000","last_became_peered":"2023-09-27T23:32:05.864207+0000","last_unstale":"2023-09-27T23:32:05.871725+0000","last_undegraded":"2023-09-27T23:32:05.871725+0000","last_fullsized":"2023-09-27T23:32:05.871725+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:53:46.038154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875524+0000","last_change":"2023-09-27T23:32:05.478414+0000","last_active":"2023-09-27T23:32:05.875524+0000","last_peered":"2023-09-27T23:32:05.875524+0000","last_clean":"2023-09-27T23:32:05.875524+0000","last_became_active":"2023-09-27T23:32:05.478088+0000","last_became_peered":"2023-09-27T23:32:05.478088+0000","last_unstale":"2023-09-27T23:32:05.875524+0000","last_undegraded":"2023-09-27T23:32:05.875524+0000","last_fullsized":"2023-09-27T23:32:05.875524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:21.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847252+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:32:05.847252+0000","last_peered":"2023-09-27T23:32:05.847252+0000","last_clean":"2023-09-27T23:32:05.847252+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:32:05.847252+0000","last_undegraded":"2023-09-27T23:32:05.847252+0000","last_fullsized":"2023-09-27T23:32:05.847252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871490+0000","last_change":"2023-09-27T23:32:05.864496+0000","last_active":"2023-09-27T23:32:05.871490+0000","last_peered":"2023-09-27T23:32:05.871490+0000","last_clean":"2023-09-27T23:32:05.871490+0000","last_became_active":"2023-09-27T23:32:05.863828+0000","last_became_peered":"2023-09-27T23:32:05.863828+0000","last_unstale":"2023-09-27T23:32:05.871490+0000","last_undegraded":"2023-09-27T23:32:05.871490+0000","last_fullsized":"2023-09-27T23:32:05.871490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:46.777842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882372+0000","last_change":"2023-09-27T23:32:05.485399+0000","last_active":"2023-09-27T23:32:05.882372+0000","last_peered":"2023-09-27T23:32:05.882372+0000","last_clean":"2023-09-27T23:32:05.882372+0000","last_became_active":"2023-09-27T23:32:05.485073+0000","last_became_peered":"2023-09-27T23:32:05.485073+0000","last_unstale":"2023-09-27T23:32:05.882372+0000","last_undegraded":"2023-09-27T23:32:05.882372+0000","last_fullsized":"2023-09-27T23:32:05.882372+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:20.429282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":110,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-09-27T23:32:01.316522+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:32:01.316522+0000","last_peered":"2023-09-27T23:32:01.316522+0000","last_clean":"2023-09-27T23:32:01.316522+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:32:01.316522+0000","last_undegraded":"2023-09-27T23:32:01.316522+0000","last_fullsized":"2023-09-27T23:32:01.316522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876013+0000","last_change":"2023-09-27T23:32:05.874103+0000","last_active":"2023-09-27T23:32:05.876013+0000","last_peered":"2023-09-27T23:32:05.876013+0000","last_clean":"2023-09-27T23:32:05.876013+0000","last_became_active":"2023-09-27T23:32:05.873806+0000","last_became_peered":"2023-09-27T23:32:05.873806+0000","last_unstale":"2023-09-27T23:32:05.876013+0000","last_undegraded":"2023-09-27T23:32:05.876013+0000","last_fullsized":"2023-09-27T23:32:05.876013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:06:14.340339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.845266+0000","last_change":"2023-09-27T23:32:05.474807+0000","last_active":"2023-09-27T23:32:05.845266+0000","last_peered":"2023-09-27T23:32:05.845266+0000","last_clean":"2023-09-27T23:32:05.845266+0000","last_became_active":"2023-09-27T23:32:05.474510+0000","last_became_peered":"2023-09-27T23:32:05.474510+0000","last_unstale":"2023-09-27T23:32:05.845266+0000","last_undegraded":"2023-09-27T23:32:05.845266+0000","last_fullsized":"2023-09-27T23:32:05.845266+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:23.368573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.849356+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:32:05.849356+0000","last_peered":"2023-09-27T23:32:05.849356+0000","last_clean":"2023-09-27T23:32:05.849356+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:32:05.849356+0000","last_undegraded":"2023-09-27T23:32:05.849356+0000","last_fullsized":"2023-09-27T23:32:05.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847066+0000","last_change":"2023-09-27T23:32:05.477781+0000","last_active":"2023-09-27T23:32:05.847066+0000","last_peered":"2023-09-27T23:32:05.847066+0000","last_clean":"2023-09-27T23:32:05.847066+0000","last_became_active":"2023-09-27T23:32:05.477416+0000","last_became_peered":"2023-09-27T23:32:05.477416+0000","last_unstale":"2023-09-27T23:32:05.847066+0000","last_undegraded":"2023-09-27T23:32:05.847066+0000","last_fullsized":"2023-09-27T23:32:05.847066+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:35.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863310+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:32:05.863310+0000","last_peered":"2023-09-27T23:32:05.863310+0000","last_clean":"2023-09-27T23:32:05.863310+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:32:05.863310+0000","last_undegraded":"2023-09-27T23:32:05.863310+0000","last_fullsized":"2023-09-27T23:32:05.863310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848976+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:32:05.848976+0000","last_peered":"2023-09-27T23:32:05.848976+0000","last_clean":"2023-09-27T23:32:05.848976+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:32:05.848976+0000","last_undegraded":"2023-09-27T23:32:05.848976+0000","last_fullsized":"2023-09-27T23:32:05.848976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869005+0000","last_change":"2023-09-27T23:32:05.862414+0000","last_active":"2023-09-27T23:32:05.869005+0000","last_peered":"2023-09-27T23:32:05.869005+0000","last_clean":"2023-09-27T23:32:05.869005+0000","last_became_active":"2023-09-27T23:32:05.862102+0000","last_became_peered":"2023-09-27T23:32:05.862102+0000","last_unstale":"2023-09-27T23:32:05.869005+0000","last_undegraded":"2023-09-27T23:32:05.869005+0000","last_fullsized":"2023-09-27T23:32:05.869005+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:40.054520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847811+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:32:05.847811+0000","last_peered":"2023-09-27T23:32:05.847811+0000","last_clean":"2023-09-27T23:32:05.847811+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:32:05.847811+0000","last_undegraded":"2023-09-27T23:32:05.847811+0000","last_fullsized":"2023-09-27T23:32:05.847811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":104,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":40,"acting":40,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509663,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":25024,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":2191360,"data_stored":1415956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25624576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.52200000000000002,"15min":0.51100000000000001},"max":{"1min":1.2470000000000001,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1819999999999999,"5min":1.5489999999999999,"15min":1.712},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91900000000000004,"15min":0.89700000000000002},"min":{"1min":0.66000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1759999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95699999999999996,"15min":0.93899999999999995},"min":{"1min":0.69599999999999995,"5min":0.66400000000000003,"15min":0.28299999999999997},"max":{"1min":1.2430000000000001,"5min":1.337,"15min":1.7589999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.90200000000000002,"15min":0.91500000000000004},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.307},"max":{"1min":1.238,"5min":1.3129999999999999,"15min":2.681},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.97599999999999998,"15min":0.97899999999999998},"min":{"1min":0.77700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.405,"15min":1.718},"last":0.997}]},{"osd":3,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.93500000000000005},"min":{"1min":0.71099999999999997,"5min":0.63400000000000001,"15min":0.53200000000000003},"max":{"1min":1.151,"5min":1.204,"15min":1.421},"last":1.042},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.747,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.409,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.073}]},{"osd":4,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.117,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97499999999999998,"15min":0.98499999999999999},"min":{"1min":0.65200000000000002,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.3029999999999999,"5min":1.3240000000000001,"15min":2.8879999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.67000000000000004,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.3540000000000001,"5min":1.79,"15min":1.79},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.252,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.139},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.839,"15min":1.839},"last":1.246},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.107,"15min":1.1259999999999999},"min":{"1min":0.96599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":2.548},"last":1.218}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27100,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":25536,"kb_avail":94344740,"statfs":{"total":96636764160,"available":96609013760,"internally_reserved":0,"allocated":1593344,"data_stored":825070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26148864},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89200000000000002,"15min":0.94399999999999995},"min":{"1min":0.629,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.2130000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.78500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.246,"5min":1.321,"15min":1.5149999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.88},"min":{"1min":0.56299999999999994,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.214,"5min":1.248,"15min":1.323},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.258,"5min":1.403,"15min":1.403},"last":0.93899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.875},"min":{"1min":0.72099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2150000000000001,"5min":1.222,"15min":1.353},"last":1.238},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91100000000000003,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.246,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.629,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2170000000000001,"5min":1.51,"15min":1.734},"last":1.073},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.72499999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1619999999999999,"5min":1.278,"15min":1.4970000000000001},"last":1.4279999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1100000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.274},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2929999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.81499999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2669999999999999,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.343},{"interface":"front","average":{"1min":1.119,"5min":1.1319999999999999,"15min":1.1539999999999999},"min":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3169999999999999,"5min":1.4259999999999999,"15min":1.49},"last":1.121}]},{"osd":7,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.81399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3580000000000001,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1259999999999999,"15min":1.145},"min":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.6619999999999999},"last":1.4750000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509661,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36968,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35456,"kb_avail":94334872,"statfs":{"total":96636764160,"available":96598908928,"internally_reserved":0,"allocated":1540096,"data_stored":797189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36306944},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.236,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.1599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.526},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.874,"15min":0.88300000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2190000000000001,"5min":1.242,"15min":1.331},"last":1.24}]},{"osd":2,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.73999999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.474,"15min":1.474},"last":0.874},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97099999999999997,"15min":0.95099999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.5740000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.97599999999999998,"15min":0.95599999999999996},"min":{"1min":0.70499999999999996,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":1.23,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.999,"15min":1.004},"min":{"1min":0.80800000000000005,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.323,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.56200000000000006,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.3220000000000001,"5min":1.341,"15min":1.341},"last":1.099},{"interface":"front","average":{"1min":1.109,"5min":1.038,"15min":1.0209999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.613,"15min":1.613},"last":1.218}]},{"osd":6,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0089999999999999,"15min":1.01},"min":{"1min":0.70499999999999996,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.5580000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.1020000000000001},"min":{"1min":0.876,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.331}]},{"osd":7,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1619999999999999,"15min":1.1379999999999999},"min":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.446,"5min":1.665,"15min":1.6679999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.198,"15min":1.1890000000000001},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.508,"5min":1.528,"15min":1.633},"last":1.28}]}]},{"osd":4,"up_from":15,"seq":64424509663,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23000,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94348840,"statfs":{"total":96636764160,"available":96613212160,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.876,"15min":0.89300000000000002},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.27,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.54400000000000004},"max":{"1min":1.212,"5min":1.343,"15min":1.45},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89500000000000002,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.262,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.73899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90200000000000002,"15min":0.91100000000000003},"min":{"1min":0.65600000000000003,"5min":0.57499999999999996,"15min":0.497},"max":{"1min":1.2410000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97999999999999998,"15min":0.96199999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.252,"5min":1.331,"15min":1.407},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92000000000000004},"min":{"1min":0.70999999999999996,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2150000000000001,"5min":1.272,"15min":1.2829999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.03,"15min":1.026},"min":{"1min":0.79900000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3660000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4610000000000001,"5min":1.609,"15min":1.609},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0229999999999999,"15min":1.01},"min":{"1min":0.66800000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0389999999999999,"15min":1.04},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3169999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.161,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.1499999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0940000000000001,"15min":1.091},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.427,"5min":1.4770000000000001,"15min":1.627},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.145,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.85899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.181}]}]},{"osd":3,"up_from":15,"seq":64424509662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":1605632,"data_stored":826606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.70199999999999996},"min":{"1min":0.40400000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.012,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.109,"5min":1.323,"15min":1.3440000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.57499999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.61499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.753,"5min":0.505,"15min":0.505},"max":{"1min":1.298,"5min":1.605,"15min":1.605},"last":1.163}]},{"osd":5,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0820000000000001,"15min":1.111},"min":{"1min":0.85099999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3999999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.232},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1160000000000001,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.714,"15min":1.764},"last":1.431}]},{"osd":6,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.105,"15min":1.1160000000000001},"min":{"1min":0.81200000000000006,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1950000000000001,"15min":1.2150000000000001},"min":{"1min":0.94899999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.488,"5min":1.6599999999999999,"15min":1.712},"last":1.3200000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.2370000000000001},"min":{"1min":0.79800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.173,"15min":1.202},"min":{"1min":0.88,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.143}]}]},{"osd":2,"up_from":15,"seq":64424509663,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22168,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94349672,"statfs":{"total":96636764160,"available":96614064128,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.903},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.097,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":0.98499999999999999,"5min":1.228,"15min":1.391},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":1.2949999999999999,"5min":1.49,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.93400000000000005},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.399,"5min":1.399,"15min":1.444},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0349999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.40600000000000003},"max":{"1min":1.417,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0629999999999999,"15min":1.0600000000000001},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":1.635,"15min":1.7390000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0269999999999999,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":0.89300000000000002,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.92500000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.722,"5min":1.722,"15min":1.9450000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.125,"15min":1.1539999999999999},"min":{"1min":0.83099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.623,"5min":1.623,"15min":2.0129999999999999},"last":1.0029999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2110000000000001,"15min":1.212},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1910000000000001,"15min":1.212},"min":{"1min":0.76000000000000001,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.5389999999999999,"5min":1.556,"15min":2.073},"last":1.0600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542369,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35840,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":1597440,"data_stored":825124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36700160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.503,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.123,"15min":1.3560000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95199999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.73299999999999998},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.2490000000000001,"15min":1.321},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.88200000000000001},"min":{"1min":0.60499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1439999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.83599999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.81299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2110000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.696,"15min":1.696},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97999999999999998,"15min":0.97299999999999998},"min":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.282,"5min":1.464,"15min":1.464},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.121,"15min":1.131},"min":{"1min":0.75600000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.774},"last":1.196}]},{"osd":5,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0860000000000001},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5109999999999999,"5min":1.546,"15min":1.623},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.88900000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.395,"5min":1.589,"15min":1.589},"last":1.2569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.089},{"interface":"front","average":{"1min":1.107,"5min":1.165,"15min":1.1850000000000001},"min":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.554,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4179999999999999,"5min":1.877,"15min":1.877},"last":1.407},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.194,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4410000000000001,"5min":1.5529999999999999,"15min":1.6339999999999999},"last":1.141}]}]},{"osd":0,"up_from":15,"seq":64424509663,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":2121728,"data_stored":1381209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.874},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.751,"15min":0.78900000000000003},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1970000000000001,"5min":1.3,"15min":1.3},"last":0.95999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.73099999999999998,"15min":0.73199999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":1.2150000000000001,"15min":1.373},"last":0.625},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.232},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0640000000000001,"5min":1.216,"15min":1.2769999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.85299999999999998,"15min":0.88300000000000001},"min":{"1min":0.46800000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":1.3160000000000001,"15min":1.333},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97299999999999998,"15min":1.0049999999999999},"min":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.47},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.997,"15min":1.032},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.206,"5min":1.4239999999999999,"15min":1.587},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.75700000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.468},{"interface":"front","average":{"1min":1.087,"5min":1.103,"15min":1.133},"min":{"1min":0.82799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.452,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.98999999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.76800000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3740000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1819999999999999,"15min":1.2110000000000001},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.52,"5min":1.73,"15min":1.8300000000000001},"last":1.417}]},{"osd":7,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.204,"15min":1.232},"min":{"1min":0.78800000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.4490000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.181,"15min":1.22},"min":{"1min":0.92600000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":1.103}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:32:09.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:32:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.636+0000 7fb0448f7700 1 Processor -- start 2023-09-27T23:32:09.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.636+0000 7fb0448f7700 1 -- start start 2023-09-27T23:32:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb0400c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb04006b900 0x7fb04006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04006b470 con 0x7fb04015cc20 2023-09-27T23:32:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04015d7b0 con 0x7fb04006b900 2023-09-27T23:32:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb040161d10 con 0x7fb0400c6900 2023-09-27T23:32:09.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb04006b900 0x7fb04006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb04006b900 0x7fb04006da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50164/0 (socket says 172.21.15.64:50164) 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 -- 172.21.15.64:0/1015535034 learned_addr learned my addr 172.21.15.64:0/1015535034 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03f7fe700 1 --2- 172.21.15.64:0/1015535034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb0400c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 -- 172.21.15.64:0/1015535034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 msgr2=0x7fb0400c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 --2- 172.21.15.64:0/1015535034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb0400c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 -- 172.21.15.64:0/1015535034 >> v1:172.21.15.123:6789/0 conn(0x7fb04015cc20 legacy=0x7fb04015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 -- 172.21.15.64:0/1015535034 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04006bd70 con 0x7fb04006b900 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03effd700 1 --2- 172.21.15.64:0/1015535034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04006da00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb0300099e0 tx=0x7fb030009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d27f0943ddb215 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03e7fc700 1 -- 172.21.15.64:0/1015535034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000b820 con 0x7fb04006b900 2023-09-27T23:32:09.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03e7fc700 1 -- 172.21.15.64:0/1015535034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000b980 con 0x7fb04006b900 2023-09-27T23:32:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb03e7fc700 1 -- 172.21.15.64:0/1015535034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000bd20 con 0x7fb04006b900 2023-09-27T23:32:09.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- 172.21.15.64:0/1015535034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 msgr2=0x7fb04006da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 --2- 172.21.15.64:0/1015535034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04006da00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb0300099e0 tx=0x7fb030009250 comp rx=0 tx=0).stop 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- 172.21.15.64:0/1015535034 shutdown_connections 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 --2- 172.21.15.64:0/1015535034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04006da00 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 --2- 172.21.15.64:0/1015535034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb0400c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- 172.21.15.64:0/1015535034 >> 172.21.15.64:0/1015535034 conn(0x7fb040067bb0 msgr2=0x7fb0400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- 172.21.15.64:0/1015535034 shutdown_connections 2023-09-27T23:32:09.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.640+0000 7fb0448f7700 1 -- 172.21.15.64:0/1015535034 wait complete. 2023-09-27T23:32:09.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 Processor -- start 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 -- start start 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04014fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb040153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb04015cc20 0x7fb0400af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04015eae0 con 0x7fb04006b900 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04015edc0 con 0x7fb04015cc20 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04015f0a0 con 0x7fb0400c6900 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04014fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04014fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50180/0 (socket says 172.21.15.64:50180) 2023-09-27T23:32:09.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb040153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 -- 172.21.15.64:0/3887608334 learned_addr learned my addr 172.21.15.64:0/3887608334 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03ffff700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb04015cc20 0x7fb0400af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 -- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 msgr2=0x7fb040153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb040153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 -- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb04015cc20 msgr2=0x7fb0400af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb04015cc20 0x7fb0400af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 -- 172.21.15.64:0/3887608334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300094f0 con 0x7fb04006b900 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb03f7fe700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04014fe90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb03400ec80 tx=0x7fb0340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03400c850 con 0x7fb04006b900 2023-09-27T23:32:09.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03400c9b0 con 0x7fb04006b900 2023-09-27T23:32:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04015f380 con 0x7fb04006b900 2023-09-27T23:32:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03400cd50 con 0x7fb04006b900 2023-09-27T23:32:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.644+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04015e740 con 0x7fb04006b900 2023-09-27T23:32:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.648+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04004be40 con 0x7fb04006b900 2023-09-27T23:32:09.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.648+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb034013680 con 0x7fb04006b900 2023-09-27T23:32:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.648+0000 7fb0277fe700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb02c060a70 0x7fb02c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.648+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb03407d350 con 0x7fb04006b900 2023-09-27T23:32:09.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.648+0000 7fb03effd700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb02c060a70 0x7fb02c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.652+0000 7fb03effd700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb02c060a70 0x7fb02c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0300099e0 tx=0x7fb030017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.660+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb03404d380 con 0x7fb04006b900 2023-09-27T23:32:09.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.816+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb04006a580 con 0x7fb02c060a70 2023-09-27T23:32:09.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.832+0000 7fb0277fe700 1 -- 172.21.15.64:0/3887608334 <== mgr.6971 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303530 (secure 0 0 0) 0x7fb04006a580 con 0x7fb02c060a70 2023-09-27T23:32:09.844 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb02c060a70 msgr2=0x7fb02c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb02c060a70 0x7fb02c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0300099e0 tx=0x7fb030017000 comp rx=0 tx=0).stop 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 msgr2=0x7fb04014fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04014fe90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb03400ec80 tx=0x7fb0340091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 shutdown_connections 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb02c060a70 0x7fb02c062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb04015cc20 0x7fb0400af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0400c6900 0x7fb040153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 --2- 172.21.15.64:0/3887608334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb04006b900 0x7fb04014fe90 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 >> 172.21.15.64:0/3887608334 conn(0x7fb040067bb0 msgr2=0x7fb0400cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 shutdown_connections 2023-09-27T23:32:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.840+0000 7fb0448f7700 1 -- 172.21.15.64:0/3887608334 wait complete. 2023-09-27T23:32:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:32:09.860 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:32:08.730336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":389,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":251,"ondisk_log_size":251,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229568,"kb_used_data":13504,"kb_used_omap":0,"kb_used_meta":216000,"kb_avail":754745152,"statfs":{"total":773094113280,"available":772859035648,"internally_reserved":0,"allocated":13828096,"data_stored":7721154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":221184000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":104,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.280290"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863665+0000","last_change":"2023-09-27T23:32:05.860722+0000","last_active":"2023-09-27T23:32:05.863665+0000","last_peered":"2023-09-27T23:32:05.863665+0000","last_clean":"2023-09-27T23:32:05.863665+0000","last_became_active":"2023-09-27T23:32:05.860381+0000","last_became_peered":"2023-09-27T23:32:05.860381+0000","last_unstale":"2023-09-27T23:32:05.863665+0000","last_undegraded":"2023-09-27T23:32:05.863665+0000","last_fullsized":"2023-09-27T23:32:05.863665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879763+0000","last_change":"2023-09-27T23:32:05.873945+0000","last_active":"2023-09-27T23:32:05.879763+0000","last_peered":"2023-09-27T23:32:05.879763+0000","last_clean":"2023-09-27T23:32:05.879763+0000","last_became_active":"2023-09-27T23:32:05.872503+0000","last_became_peered":"2023-09-27T23:32:05.872503+0000","last_unstale":"2023-09-27T23:32:05.879763+0000","last_undegraded":"2023-09-27T23:32:05.879763+0000","last_fullsized":"2023-09-27T23:32:05.879763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:06.052849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.866846+0000","last_change":"2023-09-27T23:32:05.863147+0000","last_active":"2023-09-27T23:32:05.866846+0000","last_peered":"2023-09-27T23:32:05.866846+0000","last_clean":"2023-09-27T23:32:05.866846+0000","last_became_active":"2023-09-27T23:32:05.862602+0000","last_became_peered":"2023-09-27T23:32:05.862602+0000","last_unstale":"2023-09-27T23:32:05.866846+0000","last_undegraded":"2023-09-27T23:32:05.866846+0000","last_fullsized":"2023-09-27T23:32:05.866846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:01.411281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867813+0000","last_change":"2023-09-27T23:32:05.861426+0000","last_active":"2023-09-27T23:32:05.867813+0000","last_peered":"2023-09-27T23:32:05.867813+0000","last_clean":"2023-09-27T23:32:05.867813+0000","last_became_active":"2023-09-27T23:32:05.861004+0000","last_became_peered":"2023-09-27T23:32:05.861004+0000","last_unstale":"2023-09-27T23:32:05.867813+0000","last_undegraded":"2023-09-27T23:32:05.867813+0000","last_fullsized":"2023-09-27T23:32:05.867813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:09.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881110+0000","last_change":"2023-09-27T23:32:05.878031+0000","last_active":"2023-09-27T23:32:05.881110+0000","last_peered":"2023-09-27T23:32:05.881110+0000","last_clean":"2023-09-27T23:32:05.881110+0000","last_became_active":"2023-09-27T23:32:05.877499+0000","last_became_peered":"2023-09-27T23:32:05.877499+0000","last_unstale":"2023-09-27T23:32:05.881110+0000","last_undegraded":"2023-09-27T23:32:05.881110+0000","last_fullsized":"2023-09-27T23:32:05.881110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:24.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865013+0000","last_change":"2023-09-27T23:32:05.861704+0000","last_active":"2023-09-27T23:32:05.865013+0000","last_peered":"2023-09-27T23:32:05.865013+0000","last_clean":"2023-09-27T23:32:05.865013+0000","last_became_active":"2023-09-27T23:32:05.861317+0000","last_became_peered":"2023-09-27T23:32:05.861317+0000","last_unstale":"2023-09-27T23:32:05.865013+0000","last_undegraded":"2023-09-27T23:32:05.865013+0000","last_fullsized":"2023-09-27T23:32:05.865013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881099+0000","last_change":"2023-09-27T23:32:05.877897+0000","last_active":"2023-09-27T23:32:05.881099+0000","last_peered":"2023-09-27T23:32:05.881099+0000","last_clean":"2023-09-27T23:32:05.881099+0000","last_became_active":"2023-09-27T23:32:05.877494+0000","last_became_peered":"2023-09-27T23:32:05.877494+0000","last_unstale":"2023-09-27T23:32:05.881099+0000","last_undegraded":"2023-09-27T23:32:05.881099+0000","last_fullsized":"2023-09-27T23:32:05.881099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:55.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879623+0000","last_change":"2023-09-27T23:32:05.872800+0000","last_active":"2023-09-27T23:32:05.879623+0000","last_peered":"2023-09-27T23:32:05.879623+0000","last_clean":"2023-09-27T23:32:05.879623+0000","last_became_active":"2023-09-27T23:32:05.872486+0000","last_became_peered":"2023-09-27T23:32:05.872486+0000","last_unstale":"2023-09-27T23:32:05.879623+0000","last_undegraded":"2023-09-27T23:32:05.879623+0000","last_fullsized":"2023-09-27T23:32:05.879623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:57.553491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873468+0000","last_change":"2023-09-27T23:32:05.872253+0000","last_active":"2023-09-27T23:32:05.873468+0000","last_peered":"2023-09-27T23:32:05.873468+0000","last_clean":"2023-09-27T23:32:05.873468+0000","last_became_active":"2023-09-27T23:32:05.871935+0000","last_became_peered":"2023-09-27T23:32:05.871935+0000","last_unstale":"2023-09-27T23:32:05.873468+0000","last_undegraded":"2023-09-27T23:32:05.873468+0000","last_fullsized":"2023-09-27T23:32:05.873468+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:00.996007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865348+0000","last_change":"2023-09-27T23:32:05.863475+0000","last_active":"2023-09-27T23:32:05.865348+0000","last_peered":"2023-09-27T23:32:05.865348+0000","last_clean":"2023-09-27T23:32:05.865348+0000","last_became_active":"2023-09-27T23:32:05.863171+0000","last_became_peered":"2023-09-27T23:32:05.863171+0000","last_unstale":"2023-09-27T23:32:05.865348+0000","last_undegraded":"2023-09-27T23:32:05.865348+0000","last_fullsized":"2023-09-27T23:32:05.865348+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862603+0000","last_change":"2023-09-27T23:32:05.859721+0000","last_active":"2023-09-27T23:32:05.862603+0000","last_peered":"2023-09-27T23:32:05.862603+0000","last_clean":"2023-09-27T23:32:05.862603+0000","last_became_active":"2023-09-27T23:32:05.859426+0000","last_became_peered":"2023-09-27T23:32:05.859426+0000","last_unstale":"2023-09-27T23:32:05.862603+0000","last_undegraded":"2023-09-27T23:32:05.862603+0000","last_fullsized":"2023-09-27T23:32:05.862603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:39.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879745+0000","last_change":"2023-09-27T23:32:05.874339+0000","last_active":"2023-09-27T23:32:05.879745+0000","last_peered":"2023-09-27T23:32:05.879745+0000","last_clean":"2023-09-27T23:32:05.879745+0000","last_became_active":"2023-09-27T23:32:05.872716+0000","last_became_peered":"2023-09-27T23:32:05.872716+0000","last_unstale":"2023-09-27T23:32:05.879745+0000","last_undegraded":"2023-09-27T23:32:05.879745+0000","last_fullsized":"2023-09-27T23:32:05.879745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:21.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871701+0000","last_change":"2023-09-27T23:32:05.870769+0000","last_active":"2023-09-27T23:32:05.871701+0000","last_peered":"2023-09-27T23:32:05.871701+0000","last_clean":"2023-09-27T23:32:05.871701+0000","last_became_active":"2023-09-27T23:32:05.870469+0000","last_became_peered":"2023-09-27T23:32:05.870469+0000","last_unstale":"2023-09-27T23:32:05.871701+0000","last_undegraded":"2023-09-27T23:32:05.871701+0000","last_fullsized":"2023-09-27T23:32:05.871701+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:01:42.856864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867672+0000","last_change":"2023-09-27T23:32:05.865682+0000","last_active":"2023-09-27T23:32:05.867672+0000","last_peered":"2023-09-27T23:32:05.867672+0000","last_clean":"2023-09-27T23:32:05.867672+0000","last_became_active":"2023-09-27T23:32:05.865377+0000","last_became_peered":"2023-09-27T23:32:05.865377+0000","last_unstale":"2023-09-27T23:32:05.867672+0000","last_undegraded":"2023-09-27T23:32:05.867672+0000","last_fullsized":"2023-09-27T23:32:05.867672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868870+0000","last_change":"2023-09-27T23:32:05.860566+0000","last_active":"2023-09-27T23:32:05.868870+0000","last_peered":"2023-09-27T23:32:05.868870+0000","last_clean":"2023-09-27T23:32:05.868870+0000","last_became_active":"2023-09-27T23:32:05.860209+0000","last_became_peered":"2023-09-27T23:32:05.860209+0000","last_unstale":"2023-09-27T23:32:05.868870+0000","last_undegraded":"2023-09-27T23:32:05.868870+0000","last_fullsized":"2023-09-27T23:32:05.868870+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:42.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875841+0000","last_change":"2023-09-27T23:32:05.873642+0000","last_active":"2023-09-27T23:32:05.875841+0000","last_peered":"2023-09-27T23:32:05.875841+0000","last_clean":"2023-09-27T23:32:05.875841+0000","last_became_active":"2023-09-27T23:32:05.873228+0000","last_became_peered":"2023-09-27T23:32:05.873228+0000","last_unstale":"2023-09-27T23:32:05.875841+0000","last_undegraded":"2023-09-27T23:32:05.875841+0000","last_fullsized":"2023-09-27T23:32:05.875841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:26:39.116439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871647+0000","last_change":"2023-09-27T23:32:05.867910+0000","last_active":"2023-09-27T23:32:05.871647+0000","last_peered":"2023-09-27T23:32:05.871647+0000","last_clean":"2023-09-27T23:32:05.871647+0000","last_became_active":"2023-09-27T23:32:05.867511+0000","last_became_peered":"2023-09-27T23:32:05.867511+0000","last_unstale":"2023-09-27T23:32:05.871647+0000","last_undegraded":"2023-09-27T23:32:05.871647+0000","last_fullsized":"2023-09-27T23:32:05.871647+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:31:21.387831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865328+0000","last_change":"2023-09-27T23:32:05.863332+0000","last_active":"2023-09-27T23:32:05.865328+0000","last_peered":"2023-09-27T23:32:05.865328+0000","last_clean":"2023-09-27T23:32:05.865328+0000","last_became_active":"2023-09-27T23:32:05.863052+0000","last_became_peered":"2023-09-27T23:32:05.863052+0000","last_unstale":"2023-09-27T23:32:05.865328+0000","last_undegraded":"2023-09-27T23:32:05.865328+0000","last_fullsized":"2023-09-27T23:32:05.865328+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873422+0000","last_change":"2023-09-27T23:32:05.871483+0000","last_active":"2023-09-27T23:32:05.873422+0000","last_peered":"2023-09-27T23:32:05.873422+0000","last_clean":"2023-09-27T23:32:05.873422+0000","last_became_active":"2023-09-27T23:32:05.871087+0000","last_became_peered":"2023-09-27T23:32:05.871087+0000","last_unstale":"2023-09-27T23:32:05.873422+0000","last_undegraded":"2023-09-27T23:32:05.873422+0000","last_fullsized":"2023-09-27T23:32:05.873422+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:16.072601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881812+0000","last_change":"2023-09-27T23:32:05.879569+0000","last_active":"2023-09-27T23:32:05.881812+0000","last_peered":"2023-09-27T23:32:05.881812+0000","last_clean":"2023-09-27T23:32:05.881812+0000","last_became_active":"2023-09-27T23:32:05.879296+0000","last_became_peered":"2023-09-27T23:32:05.879296+0000","last_unstale":"2023-09-27T23:32:05.881812+0000","last_undegraded":"2023-09-27T23:32:05.881812+0000","last_fullsized":"2023-09-27T23:32:05.881812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:42.688153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876663+0000","last_change":"2023-09-27T23:32:05.875209+0000","last_active":"2023-09-27T23:32:05.876663+0000","last_peered":"2023-09-27T23:32:05.876663+0000","last_clean":"2023-09-27T23:32:05.876663+0000","last_became_active":"2023-09-27T23:32:05.874886+0000","last_became_peered":"2023-09-27T23:32:05.874886+0000","last_unstale":"2023-09-27T23:32:05.876663+0000","last_undegraded":"2023-09-27T23:32:05.876663+0000","last_fullsized":"2023-09-27T23:32:05.876663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868346+0000","last_change":"2023-09-27T23:32:05.865108+0000","last_active":"2023-09-27T23:32:05.868346+0000","last_peered":"2023-09-27T23:32:05.868346+0000","last_clean":"2023-09-27T23:32:05.868346+0000","last_became_active":"2023-09-27T23:32:05.864839+0000","last_became_peered":"2023-09-27T23:32:05.864839+0000","last_unstale":"2023-09-27T23:32:05.868346+0000","last_undegraded":"2023-09-27T23:32:05.868346+0000","last_fullsized":"2023-09-27T23:32:05.868346+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:24:55.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865011+0000","last_change":"2023-09-27T23:32:05.859858+0000","last_active":"2023-09-27T23:32:05.865011+0000","last_peered":"2023-09-27T23:32:05.865011+0000","last_clean":"2023-09-27T23:32:05.865011+0000","last_became_active":"2023-09-27T23:32:05.859569+0000","last_became_peered":"2023-09-27T23:32:05.859569+0000","last_unstale":"2023-09-27T23:32:05.865011+0000","last_undegraded":"2023-09-27T23:32:05.865011+0000","last_fullsized":"2023-09-27T23:32:05.865011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:40.168536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871871+0000","last_change":"2023-09-27T23:32:05.867437+0000","last_active":"2023-09-27T23:32:05.871871+0000","last_peered":"2023-09-27T23:32:05.871871+0000","last_clean":"2023-09-27T23:32:05.871871+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871871+0000","last_undegraded":"2023-09-27T23:32:05.871871+0000","last_fullsized":"2023-09-27T23:32:05.871871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:22.332265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877163+0000","last_change":"2023-09-27T23:32:05.474605+0000","last_active":"2023-09-27T23:32:05.877163+0000","last_peered":"2023-09-27T23:32:05.877163+0000","last_clean":"2023-09-27T23:32:05.877163+0000","last_became_active":"2023-09-27T23:32:05.474269+0000","last_became_peered":"2023-09-27T23:32:05.474269+0000","last_unstale":"2023-09-27T23:32:05.877163+0000","last_undegraded":"2023-09-27T23:32:05.877163+0000","last_fullsized":"2023-09-27T23:32:05.877163+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864812+0000","last_change":"2023-09-27T23:32:05.858996+0000","last_active":"2023-09-27T23:32:05.864812+0000","last_peered":"2023-09-27T23:32:05.864812+0000","last_clean":"2023-09-27T23:32:05.864812+0000","last_became_active":"2023-09-27T23:32:05.858668+0000","last_became_peered":"2023-09-27T23:32:05.858668+0000","last_unstale":"2023-09-27T23:32:05.864812+0000","last_undegraded":"2023-09-27T23:32:05.864812+0000","last_fullsized":"2023-09-27T23:32:05.864812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:35:43.781059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862048+0000","last_change":"2023-09-27T23:32:05.477638+0000","last_active":"2023-09-27T23:32:05.862048+0000","last_peered":"2023-09-27T23:32:05.862048+0000","last_clean":"2023-09-27T23:32:05.862048+0000","last_became_active":"2023-09-27T23:32:05.477296+0000","last_became_peered":"2023-09-27T23:32:05.477296+0000","last_unstale":"2023-09-27T23:32:05.862048+0000","last_undegraded":"2023-09-27T23:32:05.862048+0000","last_fullsized":"2023-09-27T23:32:05.862048+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:24.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881733+0000","last_change":"2023-09-27T23:32:05.879475+0000","last_active":"2023-09-27T23:32:05.881733+0000","last_peered":"2023-09-27T23:32:05.881733+0000","last_clean":"2023-09-27T23:32:05.881733+0000","last_became_active":"2023-09-27T23:32:05.877744+0000","last_became_peered":"2023-09-27T23:32:05.877744+0000","last_unstale":"2023-09-27T23:32:05.881733+0000","last_undegraded":"2023-09-27T23:32:05.881733+0000","last_fullsized":"2023-09-27T23:32:05.881733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:18.120648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862399+0000","last_change":"2023-09-27T23:32:05.478152+0000","last_active":"2023-09-27T23:32:05.862399+0000","last_peered":"2023-09-27T23:32:05.862399+0000","last_clean":"2023-09-27T23:32:05.862399+0000","last_became_active":"2023-09-27T23:32:05.477854+0000","last_became_peered":"2023-09-27T23:32:05.477854+0000","last_unstale":"2023-09-27T23:32:05.862399+0000","last_undegraded":"2023-09-27T23:32:05.862399+0000","last_fullsized":"2023-09-27T23:32:05.862399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865090+0000","last_change":"2023-09-27T23:32:05.861730+0000","last_active":"2023-09-27T23:32:05.865090+0000","last_peered":"2023-09-27T23:32:05.865090+0000","last_clean":"2023-09-27T23:32:05.865090+0000","last_became_active":"2023-09-27T23:32:05.861021+0000","last_became_peered":"2023-09-27T23:32:05.861021+0000","last_unstale":"2023-09-27T23:32:05.865090+0000","last_undegraded":"2023-09-27T23:32:05.865090+0000","last_fullsized":"2023-09-27T23:32:05.865090+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862148+0000","last_change":"2023-09-27T23:32:05.474408+0000","last_active":"2023-09-27T23:32:05.862148+0000","last_peered":"2023-09-27T23:32:05.862148+0000","last_clean":"2023-09-27T23:32:05.862148+0000","last_became_active":"2023-09-27T23:32:05.473091+0000","last_became_peered":"2023-09-27T23:32:05.473091+0000","last_unstale":"2023-09-27T23:32:05.862148+0000","last_undegraded":"2023-09-27T23:32:05.862148+0000","last_fullsized":"2023-09-27T23:32:05.862148+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:20.064087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855524+0000","last_change":"2023-09-27T23:32:05.486286+0000","last_active":"2023-09-27T23:32:05.855524+0000","last_peered":"2023-09-27T23:32:05.855524+0000","last_clean":"2023-09-27T23:32:05.855524+0000","last_became_active":"2023-09-27T23:32:05.485968+0000","last_became_peered":"2023-09-27T23:32:05.485968+0000","last_unstale":"2023-09-27T23:32:05.855524+0000","last_undegraded":"2023-09-27T23:32:05.855524+0000","last_fullsized":"2023-09-27T23:32:05.855524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:20.522455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.883531+0000","last_change":"2023-09-27T23:32:05.882768+0000","last_active":"2023-09-27T23:32:05.883531+0000","last_peered":"2023-09-27T23:32:05.883531+0000","last_clean":"2023-09-27T23:32:05.883531+0000","last_became_active":"2023-09-27T23:32:05.882322+0000","last_became_peered":"2023-09-27T23:32:05.882322+0000","last_unstale":"2023-09-27T23:32:05.883531+0000","last_undegraded":"2023-09-27T23:32:05.883531+0000","last_fullsized":"2023-09-27T23:32:05.883531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:46.976392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877676+0000","last_change":"2023-09-27T23:32:05.875211+0000","last_active":"2023-09-27T23:32:05.877676+0000","last_peered":"2023-09-27T23:32:05.877676+0000","last_clean":"2023-09-27T23:32:05.877676+0000","last_became_active":"2023-09-27T23:32:05.874899+0000","last_became_peered":"2023-09-27T23:32:05.874899+0000","last_unstale":"2023-09-27T23:32:05.877676+0000","last_undegraded":"2023-09-27T23:32:05.877676+0000","last_fullsized":"2023-09-27T23:32:05.877676+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:38.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.858932+0000","last_change":"2023-09-27T23:32:05.474634+0000","last_active":"2023-09-27T23:32:05.858932+0000","last_peered":"2023-09-27T23:32:05.858932+0000","last_clean":"2023-09-27T23:32:05.858932+0000","last_became_active":"2023-09-27T23:32:05.474314+0000","last_became_peered":"2023-09-27T23:32:05.474314+0000","last_unstale":"2023-09-27T23:32:05.858932+0000","last_undegraded":"2023-09-27T23:32:05.858932+0000","last_fullsized":"2023-09-27T23:32:05.858932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:05.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885276+0000","last_change":"2023-09-27T23:32:05.884516+0000","last_active":"2023-09-27T23:32:05.885276+0000","last_peered":"2023-09-27T23:32:05.885276+0000","last_clean":"2023-09-27T23:32:05.885276+0000","last_became_active":"2023-09-27T23:32:05.884276+0000","last_became_peered":"2023-09-27T23:32:05.884276+0000","last_unstale":"2023-09-27T23:32:05.885276+0000","last_undegraded":"2023-09-27T23:32:05.885276+0000","last_fullsized":"2023-09-27T23:32:05.885276+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:14.206863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871586+0000","last_change":"2023-09-27T23:32:05.483320+0000","last_active":"2023-09-27T23:32:05.871586+0000","last_peered":"2023-09-27T23:32:05.871586+0000","last_clean":"2023-09-27T23:32:05.871586+0000","last_became_active":"2023-09-27T23:32:05.482969+0000","last_became_peered":"2023-09-27T23:32:05.482969+0000","last_unstale":"2023-09-27T23:32:05.871586+0000","last_undegraded":"2023-09-27T23:32:05.871586+0000","last_fullsized":"2023-09-27T23:32:05.871586+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:56.542035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881069+0000","last_change":"2023-09-27T23:32:05.877251+0000","last_active":"2023-09-27T23:32:05.881069+0000","last_peered":"2023-09-27T23:32:05.881069+0000","last_clean":"2023-09-27T23:32:05.881069+0000","last_became_active":"2023-09-27T23:32:05.876868+0000","last_became_peered":"2023-09-27T23:32:05.876868+0000","last_unstale":"2023-09-27T23:32:05.881069+0000","last_undegraded":"2023-09-27T23:32:05.881069+0000","last_fullsized":"2023-09-27T23:32:05.881069+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:10:56.379380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879641+0000","last_change":"2023-09-27T23:32:05.873902+0000","last_active":"2023-09-27T23:32:05.879641+0000","last_peered":"2023-09-27T23:32:05.879641+0000","last_clean":"2023-09-27T23:32:05.879641+0000","last_became_active":"2023-09-27T23:32:05.872995+0000","last_became_peered":"2023-09-27T23:32:05.872995+0000","last_unstale":"2023-09-27T23:32:05.879641+0000","last_undegraded":"2023-09-27T23:32:05.879641+0000","last_fullsized":"2023-09-27T23:32:05.879641+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:35.151434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864787+0000","last_change":"2023-09-27T23:32:05.477932+0000","last_active":"2023-09-27T23:32:05.864787+0000","last_peered":"2023-09-27T23:32:05.864787+0000","last_clean":"2023-09-27T23:32:05.864787+0000","last_became_active":"2023-09-27T23:32:05.477556+0000","last_became_peered":"2023-09-27T23:32:05.477556+0000","last_unstale":"2023-09-27T23:32:05.864787+0000","last_undegraded":"2023-09-27T23:32:05.864787+0000","last_fullsized":"2023-09-27T23:32:05.864787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:23:33.015795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847377+0000","last_change":"2023-09-27T23:32:05.473937+0000","last_active":"2023-09-27T23:32:05.847377+0000","last_peered":"2023-09-27T23:32:05.847377+0000","last_clean":"2023-09-27T23:32:05.847377+0000","last_became_active":"2023-09-27T23:32:05.473535+0000","last_became_peered":"2023-09-27T23:32:05.473535+0000","last_unstale":"2023-09-27T23:32:05.847377+0000","last_undegraded":"2023-09-27T23:32:05.847377+0000","last_fullsized":"2023-09-27T23:32:05.847377+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:59.468491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.880779+0000","last_change":"2023-09-27T23:32:05.877232+0000","last_active":"2023-09-27T23:32:05.880779+0000","last_peered":"2023-09-27T23:32:05.880779+0000","last_clean":"2023-09-27T23:32:05.880779+0000","last_became_active":"2023-09-27T23:32:05.876906+0000","last_became_peered":"2023-09-27T23:32:05.876906+0000","last_unstale":"2023-09-27T23:32:05.880779+0000","last_undegraded":"2023-09-27T23:32:05.880779+0000","last_fullsized":"2023-09-27T23:32:05.880779+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:58.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848215+0000","last_change":"2023-09-27T23:32:05.478424+0000","last_active":"2023-09-27T23:32:05.848215+0000","last_peered":"2023-09-27T23:32:05.848215+0000","last_clean":"2023-09-27T23:32:05.848215+0000","last_became_active":"2023-09-27T23:32:05.478120+0000","last_became_peered":"2023-09-27T23:32:05.478120+0000","last_unstale":"2023-09-27T23:32:05.848215+0000","last_undegraded":"2023-09-27T23:32:05.848215+0000","last_fullsized":"2023-09-27T23:32:05.848215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:15.570089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877156+0000","last_change":"2023-09-27T23:32:05.875429+0000","last_active":"2023-09-27T23:32:05.877156+0000","last_peered":"2023-09-27T23:32:05.877156+0000","last_clean":"2023-09-27T23:32:05.877156+0000","last_became_active":"2023-09-27T23:32:05.875127+0000","last_became_peered":"2023-09-27T23:32:05.875127+0000","last_unstale":"2023-09-27T23:32:05.877156+0000","last_undegraded":"2023-09-27T23:32:05.877156+0000","last_fullsized":"2023-09-27T23:32:05.877156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:21.585472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875964+0000","last_change":"2023-09-27T23:32:05.873357+0000","last_active":"2023-09-27T23:32:05.875964+0000","last_peered":"2023-09-27T23:32:05.875964+0000","last_clean":"2023-09-27T23:32:05.875964+0000","last_became_active":"2023-09-27T23:32:05.873037+0000","last_became_peered":"2023-09-27T23:32:05.873037+0000","last_unstale":"2023-09-27T23:32:05.875964+0000","last_undegraded":"2023-09-27T23:32:05.875964+0000","last_fullsized":"2023-09-27T23:32:05.875964+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:54:10.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882131+0000","last_change":"2023-09-27T23:32:05.487770+0000","last_active":"2023-09-27T23:32:05.882131+0000","last_peered":"2023-09-27T23:32:05.882131+0000","last_clean":"2023-09-27T23:32:05.882131+0000","last_became_active":"2023-09-27T23:32:05.487423+0000","last_became_peered":"2023-09-27T23:32:05.487423+0000","last_unstale":"2023-09-27T23:32:05.882131+0000","last_undegraded":"2023-09-27T23:32:05.882131+0000","last_fullsized":"2023-09-27T23:32:05.882131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:34.139978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:06.303516+0000","last_change":"2023-09-27T23:32:05.473896+0000","last_active":"2023-09-27T23:32:06.303516+0000","last_peered":"2023-09-27T23:32:06.303516+0000","last_clean":"2023-09-27T23:32:06.303516+0000","last_became_active":"2023-09-27T23:32:05.473665+0000","last_became_peered":"2023-09-27T23:32:05.473665+0000","last_unstale":"2023-09-27T23:32:06.303516+0000","last_undegraded":"2023-09-27T23:32:06.303516+0000","last_fullsized":"2023-09-27T23:32:06.303516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:17.109191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876032+0000","last_change":"2023-09-27T23:32:05.874148+0000","last_active":"2023-09-27T23:32:05.876032+0000","last_peered":"2023-09-27T23:32:05.876032+0000","last_clean":"2023-09-27T23:32:05.876032+0000","last_became_active":"2023-09-27T23:32:05.873853+0000","last_became_peered":"2023-09-27T23:32:05.873853+0000","last_unstale":"2023-09-27T23:32:05.876032+0000","last_undegraded":"2023-09-27T23:32:05.876032+0000","last_fullsized":"2023-09-27T23:32:05.876032+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:16:14.012071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846161+0000","last_change":"2023-09-27T23:32:05.482613+0000","last_active":"2023-09-27T23:32:05.846161+0000","last_peered":"2023-09-27T23:32:05.846161+0000","last_clean":"2023-09-27T23:32:05.846161+0000","last_became_active":"2023-09-27T23:32:05.482252+0000","last_became_peered":"2023-09-27T23:32:05.482252+0000","last_unstale":"2023-09-27T23:32:05.846161+0000","last_undegraded":"2023-09-27T23:32:05.846161+0000","last_fullsized":"2023-09-27T23:32:05.846161+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:05.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869011+0000","last_change":"2023-09-27T23:32:05.866830+0000","last_active":"2023-09-27T23:32:05.869011+0000","last_peered":"2023-09-27T23:32:05.869011+0000","last_clean":"2023-09-27T23:32:05.869011+0000","last_became_active":"2023-09-27T23:32:05.866498+0000","last_became_peered":"2023-09-27T23:32:05.866498+0000","last_unstale":"2023-09-27T23:32:05.869011+0000","last_undegraded":"2023-09-27T23:32:05.869011+0000","last_fullsized":"2023-09-27T23:32:05.869011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865072+0000","last_change":"2023-09-27T23:32:05.860941+0000","last_active":"2023-09-27T23:32:05.865072+0000","last_peered":"2023-09-27T23:32:05.865072+0000","last_clean":"2023-09-27T23:32:05.865072+0000","last_became_active":"2023-09-27T23:32:05.860543+0000","last_became_peered":"2023-09-27T23:32:05.860543+0000","last_unstale":"2023-09-27T23:32:05.865072+0000","last_undegraded":"2023-09-27T23:32:05.865072+0000","last_fullsized":"2023-09-27T23:32:05.865072+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.852958+0000","last_change":"2023-09-27T23:32:05.479760+0000","last_active":"2023-09-27T23:32:05.852958+0000","last_peered":"2023-09-27T23:32:05.852958+0000","last_clean":"2023-09-27T23:32:05.852958+0000","last_became_active":"2023-09-27T23:32:05.479474+0000","last_became_peered":"2023-09-27T23:32:05.479474+0000","last_unstale":"2023-09-27T23:32:05.852958+0000","last_undegraded":"2023-09-27T23:32:05.852958+0000","last_fullsized":"2023-09-27T23:32:05.852958+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:43.445603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881743+0000","last_change":"2023-09-27T23:32:05.879523+0000","last_active":"2023-09-27T23:32:05.881743+0000","last_peered":"2023-09-27T23:32:05.881743+0000","last_clean":"2023-09-27T23:32:05.881743+0000","last_became_active":"2023-09-27T23:32:05.878009+0000","last_became_peered":"2023-09-27T23:32:05.878009+0000","last_unstale":"2023-09-27T23:32:05.881743+0000","last_undegraded":"2023-09-27T23:32:05.881743+0000","last_fullsized":"2023-09-27T23:32:05.881743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:21:40.330475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847190+0000","last_change":"2023-09-27T23:32:05.473857+0000","last_active":"2023-09-27T23:32:05.847190+0000","last_peered":"2023-09-27T23:32:05.847190+0000","last_clean":"2023-09-27T23:32:05.847190+0000","last_became_active":"2023-09-27T23:32:05.473507+0000","last_became_peered":"2023-09-27T23:32:05.473507+0000","last_unstale":"2023-09-27T23:32:05.847190+0000","last_undegraded":"2023-09-27T23:32:05.847190+0000","last_fullsized":"2023-09-27T23:32:05.847190+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:19.244182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869149+0000","last_change":"2023-09-27T23:32:05.861010+0000","last_active":"2023-09-27T23:32:05.869149+0000","last_peered":"2023-09-27T23:32:05.869149+0000","last_clean":"2023-09-27T23:32:05.869149+0000","last_became_active":"2023-09-27T23:32:05.860737+0000","last_became_peered":"2023-09-27T23:32:05.860737+0000","last_unstale":"2023-09-27T23:32:05.869149+0000","last_undegraded":"2023-09-27T23:32:05.869149+0000","last_fullsized":"2023-09-27T23:32:05.869149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:58:21.738937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"117'227","reported_seq":650,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864897+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:32:05.864897+0000","last_peered":"2023-09-27T23:32:05.864897+0000","last_clean":"2023-09-27T23:32:05.864897+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:32:05.864897+0000","last_undegraded":"2023-09-27T23:32:05.864897+0000","last_fullsized":"2023-09-27T23:32:05.864897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":227,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":110,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-09-27T23:32:01.316522+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:32:01.316522+0000","last_peered":"2023-09-27T23:32:01.316522+0000","last_clean":"2023-09-27T23:32:01.316522+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:32:01.316522+0000","last_undegraded":"2023-09-27T23:32:01.316522+0000","last_fullsized":"2023-09-27T23:32:01.316522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869078+0000","last_change":"2023-09-27T23:32:05.862823+0000","last_active":"2023-09-27T23:32:05.869078+0000","last_peered":"2023-09-27T23:32:05.869078+0000","last_clean":"2023-09-27T23:32:05.869078+0000","last_became_active":"2023-09-27T23:32:05.862677+0000","last_became_peered":"2023-09-27T23:32:05.862677+0000","last_unstale":"2023-09-27T23:32:05.869078+0000","last_undegraded":"2023-09-27T23:32:05.869078+0000","last_fullsized":"2023-09-27T23:32:05.869078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:01.586755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846271+0000","last_change":"2023-09-27T23:32:05.487459+0000","last_active":"2023-09-27T23:32:05.846271+0000","last_peered":"2023-09-27T23:32:05.846271+0000","last_clean":"2023-09-27T23:32:05.846271+0000","last_became_active":"2023-09-27T23:32:05.487130+0000","last_became_peered":"2023-09-27T23:32:05.487130+0000","last_unstale":"2023-09-27T23:32:05.846271+0000","last_undegraded":"2023-09-27T23:32:05.846271+0000","last_fullsized":"2023-09-27T23:32:05.846271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:57.985464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864781+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:32:05.864781+0000","last_peered":"2023-09-27T23:32:05.864781+0000","last_clean":"2023-09-27T23:32:05.864781+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:32:05.864781+0000","last_undegraded":"2023-09-27T23:32:05.864781+0000","last_fullsized":"2023-09-27T23:32:05.864781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871725+0000","last_change":"2023-09-27T23:32:05.864506+0000","last_active":"2023-09-27T23:32:05.871725+0000","last_peered":"2023-09-27T23:32:05.871725+0000","last_clean":"2023-09-27T23:32:05.871725+0000","last_became_active":"2023-09-27T23:32:05.864207+0000","last_became_peered":"2023-09-27T23:32:05.864207+0000","last_unstale":"2023-09-27T23:32:05.871725+0000","last_undegraded":"2023-09-27T23:32:05.871725+0000","last_fullsized":"2023-09-27T23:32:05.871725+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:53:46.038154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875524+0000","last_change":"2023-09-27T23:32:05.478414+0000","last_active":"2023-09-27T23:32:05.875524+0000","last_peered":"2023-09-27T23:32:05.875524+0000","last_clean":"2023-09-27T23:32:05.875524+0000","last_became_active":"2023-09-27T23:32:05.478088+0000","last_became_peered":"2023-09-27T23:32:05.478088+0000","last_unstale":"2023-09-27T23:32:05.875524+0000","last_undegraded":"2023-09-27T23:32:05.875524+0000","last_fullsized":"2023-09-27T23:32:05.875524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:21.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847252+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:32:05.847252+0000","last_peered":"2023-09-27T23:32:05.847252+0000","last_clean":"2023-09-27T23:32:05.847252+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:32:05.847252+0000","last_undegraded":"2023-09-27T23:32:05.847252+0000","last_fullsized":"2023-09-27T23:32:05.847252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871490+0000","last_change":"2023-09-27T23:32:05.864496+0000","last_active":"2023-09-27T23:32:05.871490+0000","last_peered":"2023-09-27T23:32:05.871490+0000","last_clean":"2023-09-27T23:32:05.871490+0000","last_became_active":"2023-09-27T23:32:05.863828+0000","last_became_peered":"2023-09-27T23:32:05.863828+0000","last_unstale":"2023-09-27T23:32:05.871490+0000","last_undegraded":"2023-09-27T23:32:05.871490+0000","last_fullsized":"2023-09-27T23:32:05.871490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:46.777842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882372+0000","last_change":"2023-09-27T23:32:05.485399+0000","last_active":"2023-09-27T23:32:05.882372+0000","last_peered":"2023-09-27T23:32:05.882372+0000","last_clean":"2023-09-27T23:32:05.882372+0000","last_became_active":"2023-09-27T23:32:05.485073+0000","last_became_peered":"2023-09-27T23:32:05.485073+0000","last_unstale":"2023-09-27T23:32:05.882372+0000","last_undegraded":"2023-09-27T23:32:05.882372+0000","last_fullsized":"2023-09-27T23:32:05.882372+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:20.429282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":110,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-09-27T23:32:01.316522+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:32:01.316522+0000","last_peered":"2023-09-27T23:32:01.316522+0000","last_clean":"2023-09-27T23:32:01.316522+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:32:01.316522+0000","last_undegraded":"2023-09-27T23:32:01.316522+0000","last_fullsized":"2023-09-27T23:32:01.316522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876013+0000","last_change":"2023-09-27T23:32:05.874103+0000","last_active":"2023-09-27T23:32:05.876013+0000","last_peered":"2023-09-27T23:32:05.876013+0000","last_clean":"2023-09-27T23:32:05.876013+0000","last_became_active":"2023-09-27T23:32:05.873806+0000","last_became_peered":"2023-09-27T23:32:05.873806+0000","last_unstale":"2023-09-27T23:32:05.876013+0000","last_undegraded":"2023-09-27T23:32:05.876013+0000","last_fullsized":"2023-09-27T23:32:05.876013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:06:14.340339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.845266+0000","last_change":"2023-09-27T23:32:05.474807+0000","last_active":"2023-09-27T23:32:05.845266+0000","last_peered":"2023-09-27T23:32:05.845266+0000","last_clean":"2023-09-27T23:32:05.845266+0000","last_became_active":"2023-09-27T23:32:05.474510+0000","last_became_peered":"2023-09-27T23:32:05.474510+0000","last_unstale":"2023-09-27T23:32:05.845266+0000","last_undegraded":"2023-09-27T23:32:05.845266+0000","last_fullsized":"2023-09-27T23:32:05.845266+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:23.368573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.849356+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:32:05.849356+0000","last_peered":"2023-09-27T23:32:05.849356+0000","last_clean":"2023-09-27T23:32:05.849356+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:32:05.849356+0000","last_undegraded":"2023-09-27T23:32:05.849356+0000","last_fullsized":"2023-09-27T23:32:05.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847066+0000","last_change":"2023-09-27T23:32:05.477781+0000","last_active":"2023-09-27T23:32:05.847066+0000","last_peered":"2023-09-27T23:32:05.847066+0000","last_clean":"2023-09-27T23:32:05.847066+0000","last_became_active":"2023-09-27T23:32:05.477416+0000","last_became_peered":"2023-09-27T23:32:05.477416+0000","last_unstale":"2023-09-27T23:32:05.847066+0000","last_undegraded":"2023-09-27T23:32:05.847066+0000","last_fullsized":"2023-09-27T23:32:05.847066+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:35.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863310+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:32:05.863310+0000","last_peered":"2023-09-27T23:32:05.863310+0000","last_clean":"2023-09-27T23:32:05.863310+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:32:05.863310+0000","last_undegraded":"2023-09-27T23:32:05.863310+0000","last_fullsized":"2023-09-27T23:32:05.863310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848976+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:32:05.848976+0000","last_peered":"2023-09-27T23:32:05.848976+0000","last_clean":"2023-09-27T23:32:05.848976+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:32:05.848976+0000","last_undegraded":"2023-09-27T23:32:05.848976+0000","last_fullsized":"2023-09-27T23:32:05.848976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869005+0000","last_change":"2023-09-27T23:32:05.862414+0000","last_active":"2023-09-27T23:32:05.869005+0000","last_peered":"2023-09-27T23:32:05.869005+0000","last_clean":"2023-09-27T23:32:05.869005+0000","last_became_active":"2023-09-27T23:32:05.862102+0000","last_became_peered":"2023-09-27T23:32:05.862102+0000","last_unstale":"2023-09-27T23:32:05.869005+0000","last_undegraded":"2023-09-27T23:32:05.869005+0000","last_fullsized":"2023-09-27T23:32:05.869005+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:40.054520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847811+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:32:05.847811+0000","last_peered":"2023-09-27T23:32:05.847811+0000","last_clean":"2023-09-27T23:32:05.847811+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:32:05.847811+0000","last_undegraded":"2023-09-27T23:32:05.847811+0000","last_fullsized":"2023-09-27T23:32:05.847811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":104,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":40,"acting":40,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509663,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":25024,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":2191360,"data_stored":1415956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25624576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.52200000000000002,"15min":0.51100000000000001},"max":{"1min":1.2470000000000001,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1819999999999999,"5min":1.5489999999999999,"15min":1.712},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91900000000000004,"15min":0.89700000000000002},"min":{"1min":0.66000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1759999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95699999999999996,"15min":0.93899999999999995},"min":{"1min":0.69599999999999995,"5min":0.66400000000000003,"15min":0.28299999999999997},"max":{"1min":1.2430000000000001,"5min":1.337,"15min":1.7589999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.90200000000000002,"15min":0.91500000000000004},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.307},"max":{"1min":1.238,"5min":1.3129999999999999,"15min":2.681},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.97599999999999998,"15min":0.97899999999999998},"min":{"1min":0.77700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.405,"15min":1.718},"last":0.997}]},{"osd":3,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.93500000000000005},"min":{"1min":0.71099999999999997,"5min":0.63400000000000001,"15min":0.53200000000000003},"max":{"1min":1.151,"5min":1.204,"15min":1.421},"last":1.042},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.747,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.409,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.073}]},{"osd":4,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.117,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97499999999999998,"15min":0.98499999999999999},"min":{"1min":0.65200000000000002,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.3029999999999999,"5min":1.3240000000000001,"15min":2.8879999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.67000000000000004,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.3540000000000001,"5min":1.79,"15min":1.79},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.252,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.139},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.839,"15min":1.839},"last":1.246},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.107,"15min":1.1259999999999999},"min":{"1min":0.96599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":2.548},"last":1.218}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27100,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":25536,"kb_avail":94344740,"statfs":{"total":96636764160,"available":96609013760,"internally_reserved":0,"allocated":1593344,"data_stored":825070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26148864},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89200000000000002,"15min":0.94399999999999995},"min":{"1min":0.629,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.2130000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.78500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.246,"5min":1.321,"15min":1.5149999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.88},"min":{"1min":0.56299999999999994,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.214,"5min":1.248,"15min":1.323},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.258,"5min":1.403,"15min":1.403},"last":0.93899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.875},"min":{"1min":0.72099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2150000000000001,"5min":1.222,"15min":1.353},"last":1.238},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91100000000000003,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.246,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.629,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2170000000000001,"5min":1.51,"15min":1.734},"last":1.073},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.72499999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1619999999999999,"5min":1.278,"15min":1.4970000000000001},"last":1.4279999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1100000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.274},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2929999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.81499999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2669999999999999,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.343},{"interface":"front","average":{"1min":1.119,"5min":1.1319999999999999,"15min":1.1539999999999999},"min":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3169999999999999,"5min":1.4259999999999999,"15min":1.49},"last":1.121}]},{"osd":7,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.81399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3580000000000001,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1259999999999999,"15min":1.145},"min":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.6619999999999999},"last":1.4750000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509661,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36968,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35456,"kb_avail":94334872,"statfs":{"total":96636764160,"available":96598908928,"internally_reserved":0,"allocated":1540096,"data_stored":797189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36306944},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.236,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.1599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.526},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.874,"15min":0.88300000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2190000000000001,"5min":1.242,"15min":1.331},"last":1.24}]},{"osd":2,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.73999999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.474,"15min":1.474},"last":0.874},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97099999999999997,"15min":0.95099999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.5740000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.97599999999999998,"15min":0.95599999999999996},"min":{"1min":0.70499999999999996,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":1.23,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.999,"15min":1.004},"min":{"1min":0.80800000000000005,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.323,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.56200000000000006,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.3220000000000001,"5min":1.341,"15min":1.341},"last":1.099},{"interface":"front","average":{"1min":1.109,"5min":1.038,"15min":1.0209999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.613,"15min":1.613},"last":1.218}]},{"osd":6,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0089999999999999,"15min":1.01},"min":{"1min":0.70499999999999996,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.5580000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.1020000000000001},"min":{"1min":0.876,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.331}]},{"osd":7,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1619999999999999,"15min":1.1379999999999999},"min":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.446,"5min":1.665,"15min":1.6679999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.198,"15min":1.1890000000000001},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.508,"5min":1.528,"15min":1.633},"last":1.28}]}]},{"osd":4,"up_from":15,"seq":64424509663,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23000,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94348840,"statfs":{"total":96636764160,"available":96613212160,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.876,"15min":0.89300000000000002},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.27,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.54400000000000004},"max":{"1min":1.212,"5min":1.343,"15min":1.45},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89500000000000002,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.262,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.73899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90200000000000002,"15min":0.91100000000000003},"min":{"1min":0.65600000000000003,"5min":0.57499999999999996,"15min":0.497},"max":{"1min":1.2410000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97999999999999998,"15min":0.96199999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.252,"5min":1.331,"15min":1.407},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92000000000000004},"min":{"1min":0.70999999999999996,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2150000000000001,"5min":1.272,"15min":1.2829999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.03,"15min":1.026},"min":{"1min":0.79900000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3660000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4610000000000001,"5min":1.609,"15min":1.609},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0229999999999999,"15min":1.01},"min":{"1min":0.66800000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0389999999999999,"15min":1.04},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3169999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.161,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.1499999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0940000000000001,"15min":1.091},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.427,"5min":1.4770000000000001,"15min":1.627},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.145,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.85899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.181}]}]},{"osd":3,"up_from":15,"seq":64424509662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":1605632,"data_stored":826606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.70199999999999996},"min":{"1min":0.40400000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.012,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.109,"5min":1.323,"15min":1.3440000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.57499999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.61499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.753,"5min":0.505,"15min":0.505},"max":{"1min":1.298,"5min":1.605,"15min":1.605},"last":1.163}]},{"osd":5,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0820000000000001,"15min":1.111},"min":{"1min":0.85099999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3999999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.232},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1160000000000001,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.714,"15min":1.764},"last":1.431}]},{"osd":6,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.105,"15min":1.1160000000000001},"min":{"1min":0.81200000000000006,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1950000000000001,"15min":1.2150000000000001},"min":{"1min":0.94899999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.488,"5min":1.6599999999999999,"15min":1.712},"last":1.3200000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.2370000000000001},"min":{"1min":0.79800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.173,"15min":1.202},"min":{"1min":0.88,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.143}]}]},{"osd":2,"up_from":15,"seq":64424509663,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22168,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94349672,"statfs":{"total":96636764160,"available":96614064128,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.903},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.097,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":0.98499999999999999,"5min":1.228,"15min":1.391},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":1.2949999999999999,"5min":1.49,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.93400000000000005},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.399,"5min":1.399,"15min":1.444},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0349999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.40600000000000003},"max":{"1min":1.417,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0629999999999999,"15min":1.0600000000000001},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":1.635,"15min":1.7390000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0269999999999999,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":0.89300000000000002,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.92500000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.722,"5min":1.722,"15min":1.9450000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.125,"15min":1.1539999999999999},"min":{"1min":0.83099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.623,"5min":1.623,"15min":2.0129999999999999},"last":1.0029999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2110000000000001,"15min":1.212},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1910000000000001,"15min":1.212},"min":{"1min":0.76000000000000001,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.5389999999999999,"5min":1.556,"15min":2.073},"last":1.0600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542369,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35840,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":1597440,"data_stored":825124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36700160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.503,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.123,"15min":1.3560000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95199999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.73299999999999998},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.2490000000000001,"15min":1.321},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.88200000000000001},"min":{"1min":0.60499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1439999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.83599999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.81299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2110000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.696,"15min":1.696},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97999999999999998,"15min":0.97299999999999998},"min":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.282,"5min":1.464,"15min":1.464},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.121,"15min":1.131},"min":{"1min":0.75600000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.774},"last":1.196}]},{"osd":5,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0860000000000001},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5109999999999999,"5min":1.546,"15min":1.623},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.88900000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.395,"5min":1.589,"15min":1.589},"last":1.2569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.089},{"interface":"front","average":{"1min":1.107,"5min":1.165,"15min":1.1850000000000001},"min":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.554,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4179999999999999,"5min":1.877,"15min":1.877},"last":1.407},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.194,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4410000000000001,"5min":1.5529999999999999,"15min":1.6339999999999999},"last":1.141}]}]},{"osd":0,"up_from":15,"seq":64424509663,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":2121728,"data_stored":1381209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.874},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.751,"15min":0.78900000000000003},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1970000000000001,"5min":1.3,"15min":1.3},"last":0.95999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.73099999999999998,"15min":0.73199999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":1.2150000000000001,"15min":1.373},"last":0.625},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.232},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0640000000000001,"5min":1.216,"15min":1.2769999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.85299999999999998,"15min":0.88300000000000001},"min":{"1min":0.46800000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":1.3160000000000001,"15min":1.333},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97299999999999998,"15min":1.0049999999999999},"min":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.47},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.997,"15min":1.032},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.206,"5min":1.4239999999999999,"15min":1.587},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.75700000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.468},{"interface":"front","average":{"1min":1.087,"5min":1.103,"15min":1.133},"min":{"1min":0.82799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.452,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.98999999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.76800000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3740000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1819999999999999,"15min":1.2110000000000001},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.52,"5min":1.73,"15min":1.8300000000000001},"last":1.417}]},{"osd":7,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.204,"15min":1.232},"min":{"1min":0.78800000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.4490000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.181,"15min":1.22},"min":{"1min":0.92600000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":1.103}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:32:09.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:32:09.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9782cd7700 1 Processor -- start 2023-09-27T23:32:09.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9782cd7700 1 -- start start 2023-09-27T23:32:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9782cd7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f977c159160 0x7f977c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9782cd7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c0c63a0 con 0x7f977c0ae210 2023-09-27T23:32:09.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9782cd7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c0accb0 con 0x7f977c0c5fd0 2023-09-27T23:32:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9782cd7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c06a7b0 con 0x7f977c159160 2023-09-27T23:32:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f97824d6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f977c0ae210 0x7f977c158200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44456/0 (socket says 172.21.15.64:44456) 2023-09-27T23:32:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f97824d6700 1 -- 172.21.15.64:0/678734162 learned_addr learned my addr 172.21.15.64:0/678734162 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f97814d4700 1 --2- 172.21.15.64:0/678734162 >> v2:172.21.15.123:3300/0 conn(0x7f977c159160 0x7f977c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211323680 0 0) 0x7f977c0c63a0 con 0x7f977c0ae210 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.968+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9768003570 con 0x7f977c0ae210 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174543511 0 0) 0x7f977c0accb0 con 0x7f977c0c5fd0 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f977c0c63a0 con 0x7f977c0c5fd0 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889246634 0 0) 0x7f9768003570 con 0x7f977c0ae210 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 >> v2:172.21.15.123:3300/0 conn(0x7f977c159160 msgr2=0x7f977c0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:09.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 --2- 172.21.15.64:0/678734162 >> v2:172.21.15.123:3300/0 conn(0x7f977c159160 0x7f977c0adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 >> v1:172.21.15.123:6789/0 conn(0x7f977c0c5fd0 legacy=0x7f977c0cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0acf90 con 0x7f977c0ae210 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f97740017f0 con 0x7f977c0ae210 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 >> v1:172.21.15.64:6789/0 conn(0x7f977c0ae210 legacy=0x7f977c158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768003bf0 0x7f9768004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9768004810 0x7f9768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97680051c0 0x7f976800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c06a7b0 con 0x7f9768004810 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976800c0d0 con 0x7f9768003bf0 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976800c290 con 0x7f97680051c0 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9768004810 0x7f9768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97824d6700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97680051c0 0x7f976800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9781cd5700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768003bf0 0x7f9768004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 -- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97680051c0 msgr2=0x7f976800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97680051c0 0x7f976800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 -- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768003bf0 msgr2=0x7f9768004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768003bf0 0x7f9768004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97824d6700 1 -- 172.21.15.64:0/678734162 reap_dead start 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 -- 172.21.15.64:0/678734162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768003570 con 0x7f9768004810 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9781cd5700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768003bf0 0x7f9768004000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f97814d4700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9768004810 0x7f9768004c80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f976c009c00 tx=0x7f976c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a831e6f627f690 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00e710 con 0x7f9768004810 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c00e870 con 0x7f9768004810 2023-09-27T23:32:09.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9780cd3700 1 -- 172.21.15.64:0/678734162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00ec30 con 0x7f9768004810 2023-09-27T23:32:09.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 -- 172.21.15.64:0/678734162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9768004810 msgr2=0x7f9768004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9768004810 0x7f9768004c80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f976c009c00 tx=0x7f976c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 -- 172.21.15.64:0/678734162 shutdown_connections 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 --2- 172.21.15.64:0/678734162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9768004810 0x7f9768004c80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 -- 172.21.15.64:0/678734162 >> 172.21.15.64:0/678734162 conn(0x7f977c067bb0 msgr2=0x7f977c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 -- 172.21.15.64:0/678734162 shutdown_connections 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.972+0000 7f9782cd7700 1 -- 172.21.15.64:0/678734162 wait complete. 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 Processor -- start 2023-09-27T23:32:09.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 -- start start 2023-09-27T23:32:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768004810 0x7f977c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 0x7f977c1470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f977c147630 0x7f977c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c147ab0 con 0x7f977c0caf60 2023-09-27T23:32:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a54e0 con 0x7f9768004810 2023-09-27T23:32:09.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a4440 con 0x7f977c147630 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 0x7f977c1470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 0x7f977c1470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50200/0 (socket says 172.21.15.64:50200) 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 -- 172.21.15.64:0/3546730821 learned_addr learned my addr 172.21.15.64:0/3546730821 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9781cd5700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768004810 0x7f977c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97824d6700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f977c147630 0x7f977c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 -- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f977c147630 msgr2=0x7f977c0b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f977c147630 0x7f977c0b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 -- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768004810 msgr2=0x7f977c0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768004810 0x7f977c0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 -- 172.21.15.64:0/3546730821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0a4720 con 0x7f977c0caf60 2023-09-27T23:32:09.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97824d6700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f977c147630 0x7f977c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f97814d4700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 0x7f977c1470f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f976c00e4b0 tx=0x7f976c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c022070 con 0x7f977c0caf60 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c013cd0 con 0x7f977c0caf60 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0276c0 con 0x7f977c0caf60 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c0a4a00 con 0x7f977c0caf60 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.976+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c0a4fc0 con 0x7f977c0caf60 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.980+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c147a30 con 0x7f977c0caf60 2023-09-27T23:32:09.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.980+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f976c027ab0 con 0x7f977c0caf60 2023-09-27T23:32:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.980+0000 7f9771ffb700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9760060a70 0x7f9760062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.980+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f976c07ea90 con 0x7f977c0caf60 2023-09-27T23:32:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.980+0000 7f9781cd5700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9760060a70 0x7f9760062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.984+0000 7f9781cd5700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9760060a70 0x7f9760062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9778007fd0 tx=0x7f9778009040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:09.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:09.992+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f976c04ebc0 con 0x7f977c0caf60 2023-09-27T23:32:10.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.144+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f977c04be40 con 0x7f9760060a70 2023-09-27T23:32:10.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.160+0000 7f9771ffb700 1 -- 172.21.15.64:0/3546730821 <== mgr.6971 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303530 (secure 0 0 0) 0x7f977c04be40 con 0x7f9760060a70 2023-09-27T23:32:10.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9760060a70 msgr2=0x7f9760062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9760060a70 0x7f9760062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9778007fd0 tx=0x7f9778009040 comp rx=0 tx=0).stop 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 msgr2=0x7f977c1470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 0x7f977c1470f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f976c00e4b0 tx=0x7f976c01a000 comp rx=0 tx=0).stop 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 shutdown_connections 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9760060a70 0x7f9760062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f977c147630 0x7f977c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f977c0caf60 0x7f977c1470f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 --2- 172.21.15.64:0/3546730821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9768004810 0x7f977c0caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 >> 172.21.15.64:0/3546730821 conn(0x7f977c067bb0 msgr2=0x7f977c06acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:10.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 shutdown_connections 2023-09-27T23:32:10.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.164+0000 7f9782cd7700 1 -- 172.21.15.64:0/3546730821 wait complete. 2023-09-27T23:32:10.173 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:32:10.184 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:32:08.730336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":389,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":251,"ondisk_log_size":251,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229568,"kb_used_data":13504,"kb_used_omap":0,"kb_used_meta":216000,"kb_avail":754745152,"statfs":{"total":773094113280,"available":772859035648,"internally_reserved":0,"allocated":13828096,"data_stored":7721154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":221184000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":104,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.280290"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863665+0000","last_change":"2023-09-27T23:32:05.860722+0000","last_active":"2023-09-27T23:32:05.863665+0000","last_peered":"2023-09-27T23:32:05.863665+0000","last_clean":"2023-09-27T23:32:05.863665+0000","last_became_active":"2023-09-27T23:32:05.860381+0000","last_became_peered":"2023-09-27T23:32:05.860381+0000","last_unstale":"2023-09-27T23:32:05.863665+0000","last_undegraded":"2023-09-27T23:32:05.863665+0000","last_fullsized":"2023-09-27T23:32:05.863665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879763+0000","last_change":"2023-09-27T23:32:05.873945+0000","last_active":"2023-09-27T23:32:05.879763+0000","last_peered":"2023-09-27T23:32:05.879763+0000","last_clean":"2023-09-27T23:32:05.879763+0000","last_became_active":"2023-09-27T23:32:05.872503+0000","last_became_peered":"2023-09-27T23:32:05.872503+0000","last_unstale":"2023-09-27T23:32:05.879763+0000","last_undegraded":"2023-09-27T23:32:05.879763+0000","last_fullsized":"2023-09-27T23:32:05.879763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:06.052849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.866846+0000","last_change":"2023-09-27T23:32:05.863147+0000","last_active":"2023-09-27T23:32:05.866846+0000","last_peered":"2023-09-27T23:32:05.866846+0000","last_clean":"2023-09-27T23:32:05.866846+0000","last_became_active":"2023-09-27T23:32:05.862602+0000","last_became_peered":"2023-09-27T23:32:05.862602+0000","last_unstale":"2023-09-27T23:32:05.866846+0000","last_undegraded":"2023-09-27T23:32:05.866846+0000","last_fullsized":"2023-09-27T23:32:05.866846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:01.411281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867813+0000","last_change":"2023-09-27T23:32:05.861426+0000","last_active":"2023-09-27T23:32:05.867813+0000","last_peered":"2023-09-27T23:32:05.867813+0000","last_clean":"2023-09-27T23:32:05.867813+0000","last_became_active":"2023-09-27T23:32:05.861004+0000","last_became_peered":"2023-09-27T23:32:05.861004+0000","last_unstale":"2023-09-27T23:32:05.867813+0000","last_undegraded":"2023-09-27T23:32:05.867813+0000","last_fullsized":"2023-09-27T23:32:05.867813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:09.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881110+0000","last_change":"2023-09-27T23:32:05.878031+0000","last_active":"2023-09-27T23:32:05.881110+0000","last_peered":"2023-09-27T23:32:05.881110+0000","last_clean":"2023-09-27T23:32:05.881110+0000","last_became_active":"2023-09-27T23:32:05.877499+0000","last_became_peered":"2023-09-27T23:32:05.877499+0000","last_unstale":"2023-09-27T23:32:05.881110+0000","last_undegraded":"2023-09-27T23:32:05.881110+0000","last_fullsized":"2023-09-27T23:32:05.881110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:24.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865013+0000","last_change":"2023-09-27T23:32:05.861704+0000","last_active":"2023-09-27T23:32:05.865013+0000","last_peered":"2023-09-27T23:32:05.865013+0000","last_clean":"2023-09-27T23:32:05.865013+0000","last_became_active":"2023-09-27T23:32:05.861317+0000","last_became_peered":"2023-09-27T23:32:05.861317+0000","last_unstale":"2023-09-27T23:32:05.865013+0000","last_undegraded":"2023-09-27T23:32:05.865013+0000","last_fullsized":"2023-09-27T23:32:05.865013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881099+0000","last_change":"2023-09-27T23:32:05.877897+0000","last_active":"2023-09-27T23:32:05.881099+0000","last_peered":"2023-09-27T23:32:05.881099+0000","last_clean":"2023-09-27T23:32:05.881099+0000","last_became_active":"2023-09-27T23:32:05.877494+0000","last_became_peered":"2023-09-27T23:32:05.877494+0000","last_unstale":"2023-09-27T23:32:05.881099+0000","last_undegraded":"2023-09-27T23:32:05.881099+0000","last_fullsized":"2023-09-27T23:32:05.881099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:55.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879623+0000","last_change":"2023-09-27T23:32:05.872800+0000","last_active":"2023-09-27T23:32:05.879623+0000","last_peered":"2023-09-27T23:32:05.879623+0000","last_clean":"2023-09-27T23:32:05.879623+0000","last_became_active":"2023-09-27T23:32:05.872486+0000","last_became_peered":"2023-09-27T23:32:05.872486+0000","last_unstale":"2023-09-27T23:32:05.879623+0000","last_undegraded":"2023-09-27T23:32:05.879623+0000","last_fullsized":"2023-09-27T23:32:05.879623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:57.553491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873468+0000","last_change":"2023-09-27T23:32:05.872253+0000","last_active":"2023-09-27T23:32:05.873468+0000","last_peered":"2023-09-27T23:32:05.873468+0000","last_clean":"2023-09-27T23:32:05.873468+0000","last_became_active":"2023-09-27T23:32:05.871935+0000","last_became_peered":"2023-09-27T23:32:05.871935+0000","last_unstale":"2023-09-27T23:32:05.873468+0000","last_undegraded":"2023-09-27T23:32:05.873468+0000","last_fullsized":"2023-09-27T23:32:05.873468+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:00.996007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865348+0000","last_change":"2023-09-27T23:32:05.863475+0000","last_active":"2023-09-27T23:32:05.865348+0000","last_peered":"2023-09-27T23:32:05.865348+0000","last_clean":"2023-09-27T23:32:05.865348+0000","last_became_active":"2023-09-27T23:32:05.863171+0000","last_became_peered":"2023-09-27T23:32:05.863171+0000","last_unstale":"2023-09-27T23:32:05.865348+0000","last_undegraded":"2023-09-27T23:32:05.865348+0000","last_fullsized":"2023-09-27T23:32:05.865348+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862603+0000","last_change":"2023-09-27T23:32:05.859721+0000","last_active":"2023-09-27T23:32:05.862603+0000","last_peered":"2023-09-27T23:32:05.862603+0000","last_clean":"2023-09-27T23:32:05.862603+0000","last_became_active":"2023-09-27T23:32:05.859426+0000","last_became_peered":"2023-09-27T23:32:05.859426+0000","last_unstale":"2023-09-27T23:32:05.862603+0000","last_undegraded":"2023-09-27T23:32:05.862603+0000","last_fullsized":"2023-09-27T23:32:05.862603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:39.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879745+0000","last_change":"2023-09-27T23:32:05.874339+0000","last_active":"2023-09-27T23:32:05.879745+0000","last_peered":"2023-09-27T23:32:05.879745+0000","last_clean":"2023-09-27T23:32:05.879745+0000","last_became_active":"2023-09-27T23:32:05.872716+0000","last_became_peered":"2023-09-27T23:32:05.872716+0000","last_unstale":"2023-09-27T23:32:05.879745+0000","last_undegraded":"2023-09-27T23:32:05.879745+0000","last_fullsized":"2023-09-27T23:32:05.879745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:21.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871701+0000","last_change":"2023-09-27T23:32:05.870769+0000","last_active":"2023-09-27T23:32:05.871701+0000","last_peered":"2023-09-27T23:32:05.871701+0000","last_clean":"2023-09-27T23:32:05.871701+0000","last_became_active":"2023-09-27T23:32:05.870469+0000","last_became_peered":"2023-09-27T23:32:05.870469+0000","last_unstale":"2023-09-27T23:32:05.871701+0000","last_undegraded":"2023-09-27T23:32:05.871701+0000","last_fullsized":"2023-09-27T23:32:05.871701+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:01:42.856864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867672+0000","last_change":"2023-09-27T23:32:05.865682+0000","last_active":"2023-09-27T23:32:05.867672+0000","last_peered":"2023-09-27T23:32:05.867672+0000","last_clean":"2023-09-27T23:32:05.867672+0000","last_became_active":"2023-09-27T23:32:05.865377+0000","last_became_peered":"2023-09-27T23:32:05.865377+0000","last_unstale":"2023-09-27T23:32:05.867672+0000","last_undegraded":"2023-09-27T23:32:05.867672+0000","last_fullsized":"2023-09-27T23:32:05.867672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868870+0000","last_change":"2023-09-27T23:32:05.860566+0000","last_active":"2023-09-27T23:32:05.868870+0000","last_peered":"2023-09-27T23:32:05.868870+0000","last_clean":"2023-09-27T23:32:05.868870+0000","last_became_active":"2023-09-27T23:32:05.860209+0000","last_became_peered":"2023-09-27T23:32:05.860209+0000","last_unstale":"2023-09-27T23:32:05.868870+0000","last_undegraded":"2023-09-27T23:32:05.868870+0000","last_fullsized":"2023-09-27T23:32:05.868870+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:42.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875841+0000","last_change":"2023-09-27T23:32:05.873642+0000","last_active":"2023-09-27T23:32:05.875841+0000","last_peered":"2023-09-27T23:32:05.875841+0000","last_clean":"2023-09-27T23:32:05.875841+0000","last_became_active":"2023-09-27T23:32:05.873228+0000","last_became_peered":"2023-09-27T23:32:05.873228+0000","last_unstale":"2023-09-27T23:32:05.875841+0000","last_undegraded":"2023-09-27T23:32:05.875841+0000","last_fullsized":"2023-09-27T23:32:05.875841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:26:39.116439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871647+0000","last_change":"2023-09-27T23:32:05.867910+0000","last_active":"2023-09-27T23:32:05.871647+0000","last_peered":"2023-09-27T23:32:05.871647+0000","last_clean":"2023-09-27T23:32:05.871647+0000","last_became_active":"2023-09-27T23:32:05.867511+0000","last_became_peered":"2023-09-27T23:32:05.867511+0000","last_unstale":"2023-09-27T23:32:05.871647+0000","last_undegraded":"2023-09-27T23:32:05.871647+0000","last_fullsized":"2023-09-27T23:32:05.871647+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:31:21.387831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865328+0000","last_change":"2023-09-27T23:32:05.863332+0000","last_active":"2023-09-27T23:32:05.865328+0000","last_peered":"2023-09-27T23:32:05.865328+0000","last_clean":"2023-09-27T23:32:05.865328+0000","last_became_active":"2023-09-27T23:32:05.863052+0000","last_became_peered":"2023-09-27T23:32:05.863052+0000","last_unstale":"2023-09-27T23:32:05.865328+0000","last_undegraded":"2023-09-27T23:32:05.865328+0000","last_fullsized":"2023-09-27T23:32:05.865328+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873422+0000","last_change":"2023-09-27T23:32:05.871483+0000","last_active":"2023-09-27T23:32:05.873422+0000","last_peered":"2023-09-27T23:32:05.873422+0000","last_clean":"2023-09-27T23:32:05.873422+0000","last_became_active":"2023-09-27T23:32:05.871087+0000","last_became_peered":"2023-09-27T23:32:05.871087+0000","last_unstale":"2023-09-27T23:32:05.873422+0000","last_undegraded":"2023-09-27T23:32:05.873422+0000","last_fullsized":"2023-09-27T23:32:05.873422+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:16.072601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881812+0000","last_change":"2023-09-27T23:32:05.879569+0000","last_active":"2023-09-27T23:32:05.881812+0000","last_peered":"2023-09-27T23:32:05.881812+0000","last_clean":"2023-09-27T23:32:05.881812+0000","last_became_active":"2023-09-27T23:32:05.879296+0000","last_became_peered":"2023-09-27T23:32:05.879296+0000","last_unstale":"2023-09-27T23:32:05.881812+0000","last_undegraded":"2023-09-27T23:32:05.881812+0000","last_fullsized":"2023-09-27T23:32:05.881812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:42.688153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876663+0000","last_change":"2023-09-27T23:32:05.875209+0000","last_active":"2023-09-27T23:32:05.876663+0000","last_peered":"2023-09-27T23:32:05.876663+0000","last_clean":"2023-09-27T23:32:05.876663+0000","last_became_active":"2023-09-27T23:32:05.874886+0000","last_became_peered":"2023-09-27T23:32:05.874886+0000","last_unstale":"2023-09-27T23:32:05.876663+0000","last_undegraded":"2023-09-27T23:32:05.876663+0000","last_fullsized":"2023-09-27T23:32:05.876663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868346+0000","last_change":"2023-09-27T23:32:05.865108+0000","last_active":"2023-09-27T23:32:05.868346+0000","last_peered":"2023-09-27T23:32:05.868346+0000","last_clean":"2023-09-27T23:32:05.868346+0000","last_became_active":"2023-09-27T23:32:05.864839+0000","last_became_peered":"2023-09-27T23:32:05.864839+0000","last_unstale":"2023-09-27T23:32:05.868346+0000","last_undegraded":"2023-09-27T23:32:05.868346+0000","last_fullsized":"2023-09-27T23:32:05.868346+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:24:55.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865011+0000","last_change":"2023-09-27T23:32:05.859858+0000","last_active":"2023-09-27T23:32:05.865011+0000","last_peered":"2023-09-27T23:32:05.865011+0000","last_clean":"2023-09-27T23:32:05.865011+0000","last_became_active":"2023-09-27T23:32:05.859569+0000","last_became_peered":"2023-09-27T23:32:05.859569+0000","last_unstale":"2023-09-27T23:32:05.865011+0000","last_undegraded":"2023-09-27T23:32:05.865011+0000","last_fullsized":"2023-09-27T23:32:05.865011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:40.168536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871871+0000","last_change":"2023-09-27T23:32:05.867437+0000","last_active":"2023-09-27T23:32:05.871871+0000","last_peered":"2023-09-27T23:32:05.871871+0000","last_clean":"2023-09-27T23:32:05.871871+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871871+0000","last_undegraded":"2023-09-27T23:32:05.871871+0000","last_fullsized":"2023-09-27T23:32:05.871871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:22.332265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877163+0000","last_change":"2023-09-27T23:32:05.474605+0000","last_active":"2023-09-27T23:32:05.877163+0000","last_peered":"2023-09-27T23:32:05.877163+0000","last_clean":"2023-09-27T23:32:05.877163+0000","last_became_active":"2023-09-27T23:32:05.474269+0000","last_became_peered":"2023-09-27T23:32:05.474269+0000","last_unstale":"2023-09-27T23:32:05.877163+0000","last_undegraded":"2023-09-27T23:32:05.877163+0000","last_fullsized":"2023-09-27T23:32:05.877163+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864812+0000","last_change":"2023-09-27T23:32:05.858996+0000","last_active":"2023-09-27T23:32:05.864812+0000","last_peered":"2023-09-27T23:32:05.864812+0000","last_clean":"2023-09-27T23:32:05.864812+0000","last_became_active":"2023-09-27T23:32:05.858668+0000","last_became_peered":"2023-09-27T23:32:05.858668+0000","last_unstale":"2023-09-27T23:32:05.864812+0000","last_undegraded":"2023-09-27T23:32:05.864812+0000","last_fullsized":"2023-09-27T23:32:05.864812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:35:43.781059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862048+0000","last_change":"2023-09-27T23:32:05.477638+0000","last_active":"2023-09-27T23:32:05.862048+0000","last_peered":"2023-09-27T23:32:05.862048+0000","last_clean":"2023-09-27T23:32:05.862048+0000","last_became_active":"2023-09-27T23:32:05.477296+0000","last_became_peered":"2023-09-27T23:32:05.477296+0000","last_unstale":"2023-09-27T23:32:05.862048+0000","last_undegraded":"2023-09-27T23:32:05.862048+0000","last_fullsized":"2023-09-27T23:32:05.862048+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:24.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881733+0000","last_change":"2023-09-27T23:32:05.879475+0000","last_active":"2023-09-27T23:32:05.881733+0000","last_peered":"2023-09-27T23:32:05.881733+0000","last_clean":"2023-09-27T23:32:05.881733+0000","last_became_active":"2023-09-27T23:32:05.877744+0000","last_became_peered":"2023-09-27T23:32:05.877744+0000","last_unstale":"2023-09-27T23:32:05.881733+0000","last_undegraded":"2023-09-27T23:32:05.881733+0000","last_fullsized":"2023-09-27T23:32:05.881733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:18.120648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862399+0000","last_change":"2023-09-27T23:32:05.478152+0000","last_active":"2023-09-27T23:32:05.862399+0000","last_peered":"2023-09-27T23:32:05.862399+0000","last_clean":"2023-09-27T23:32:05.862399+0000","last_became_active":"2023-09-27T23:32:05.477854+0000","last_became_peered":"2023-09-27T23:32:05.477854+0000","last_unstale":"2023-09-27T23:32:05.862399+0000","last_undegraded":"2023-09-27T23:32:05.862399+0000","last_fullsized":"2023-09-27T23:32:05.862399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865090+0000","last_change":"2023-09-27T23:32:05.861730+0000","last_active":"2023-09-27T23:32:05.865090+0000","last_peered":"2023-09-27T23:32:05.865090+0000","last_clean":"2023-09-27T23:32:05.865090+0000","last_became_active":"2023-09-27T23:32:05.861021+0000","last_became_peered":"2023-09-27T23:32:05.861021+0000","last_unstale":"2023-09-27T23:32:05.865090+0000","last_undegraded":"2023-09-27T23:32:05.865090+0000","last_fullsized":"2023-09-27T23:32:05.865090+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862148+0000","last_change":"2023-09-27T23:32:05.474408+0000","last_active":"2023-09-27T23:32:05.862148+0000","last_peered":"2023-09-27T23:32:05.862148+0000","last_clean":"2023-09-27T23:32:05.862148+0000","last_became_active":"2023-09-27T23:32:05.473091+0000","last_became_peered":"2023-09-27T23:32:05.473091+0000","last_unstale":"2023-09-27T23:32:05.862148+0000","last_undegraded":"2023-09-27T23:32:05.862148+0000","last_fullsized":"2023-09-27T23:32:05.862148+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:20.064087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855524+0000","last_change":"2023-09-27T23:32:05.486286+0000","last_active":"2023-09-27T23:32:05.855524+0000","last_peered":"2023-09-27T23:32:05.855524+0000","last_clean":"2023-09-27T23:32:05.855524+0000","last_became_active":"2023-09-27T23:32:05.485968+0000","last_became_peered":"2023-09-27T23:32:05.485968+0000","last_unstale":"2023-09-27T23:32:05.855524+0000","last_undegraded":"2023-09-27T23:32:05.855524+0000","last_fullsized":"2023-09-27T23:32:05.855524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:20.522455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.883531+0000","last_change":"2023-09-27T23:32:05.882768+0000","last_active":"2023-09-27T23:32:05.883531+0000","last_peered":"2023-09-27T23:32:05.883531+0000","last_clean":"2023-09-27T23:32:05.883531+0000","last_became_active":"2023-09-27T23:32:05.882322+0000","last_became_peered":"2023-09-27T23:32:05.882322+0000","last_unstale":"2023-09-27T23:32:05.883531+0000","last_undegraded":"2023-09-27T23:32:05.883531+0000","last_fullsized":"2023-09-27T23:32:05.883531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:46.976392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877676+0000","last_change":"2023-09-27T23:32:05.875211+0000","last_active":"2023-09-27T23:32:05.877676+0000","last_peered":"2023-09-27T23:32:05.877676+0000","last_clean":"2023-09-27T23:32:05.877676+0000","last_became_active":"2023-09-27T23:32:05.874899+0000","last_became_peered":"2023-09-27T23:32:05.874899+0000","last_unstale":"2023-09-27T23:32:05.877676+0000","last_undegraded":"2023-09-27T23:32:05.877676+0000","last_fullsized":"2023-09-27T23:32:05.877676+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:38.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.858932+0000","last_change":"2023-09-27T23:32:05.474634+0000","last_active":"2023-09-27T23:32:05.858932+0000","last_peered":"2023-09-27T23:32:05.858932+0000","last_clean":"2023-09-27T23:32:05.858932+0000","last_became_active":"2023-09-27T23:32:05.474314+0000","last_became_peered":"2023-09-27T23:32:05.474314+0000","last_unstale":"2023-09-27T23:32:05.858932+0000","last_undegraded":"2023-09-27T23:32:05.858932+0000","last_fullsized":"2023-09-27T23:32:05.858932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:05.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885276+0000","last_change":"2023-09-27T23:32:05.884516+0000","last_active":"2023-09-27T23:32:05.885276+0000","last_peered":"2023-09-27T23:32:05.885276+0000","last_clean":"2023-09-27T23:32:05.885276+0000","last_became_active":"2023-09-27T23:32:05.884276+0000","last_became_peered":"2023-09-27T23:32:05.884276+0000","last_unstale":"2023-09-27T23:32:05.885276+0000","last_undegraded":"2023-09-27T23:32:05.885276+0000","last_fullsized":"2023-09-27T23:32:05.885276+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:14.206863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871586+0000","last_change":"2023-09-27T23:32:05.483320+0000","last_active":"2023-09-27T23:32:05.871586+0000","last_peered":"2023-09-27T23:32:05.871586+0000","last_clean":"2023-09-27T23:32:05.871586+0000","last_became_active":"2023-09-27T23:32:05.482969+0000","last_became_peered":"2023-09-27T23:32:05.482969+0000","last_unstale":"2023-09-27T23:32:05.871586+0000","last_undegraded":"2023-09-27T23:32:05.871586+0000","last_fullsized":"2023-09-27T23:32:05.871586+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:56.542035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881069+0000","last_change":"2023-09-27T23:32:05.877251+0000","last_active":"2023-09-27T23:32:05.881069+0000","last_peered":"2023-09-27T23:32:05.881069+0000","last_clean":"2023-09-27T23:32:05.881069+0000","last_became_active":"2023-09-27T23:32:05.876868+0000","last_became_peered":"2023-09-27T23:32:05.876868+0000","last_unstale":"2023-09-27T23:32:05.881069+0000","last_undegraded":"2023-09-27T23:32:05.881069+0000","last_fullsized":"2023-09-27T23:32:05.881069+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:10:56.379380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879641+0000","last_change":"2023-09-27T23:32:05.873902+0000","last_active":"2023-09-27T23:32:05.879641+0000","last_peered":"2023-09-27T23:32:05.879641+0000","last_clean":"2023-09-27T23:32:05.879641+0000","last_became_active":"2023-09-27T23:32:05.872995+0000","last_became_peered":"2023-09-27T23:32:05.872995+0000","last_unstale":"2023-09-27T23:32:05.879641+0000","last_undegraded":"2023-09-27T23:32:05.879641+0000","last_fullsized":"2023-09-27T23:32:05.879641+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:35.151434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864787+0000","last_change":"2023-09-27T23:32:05.477932+0000","last_active":"2023-09-27T23:32:05.864787+0000","last_peered":"2023-09-27T23:32:05.864787+0000","last_clean":"2023-09-27T23:32:05.864787+0000","last_became_active":"2023-09-27T23:32:05.477556+0000","last_became_peered":"2023-09-27T23:32:05.477556+0000","last_unstale":"2023-09-27T23:32:05.864787+0000","last_undegraded":"2023-09-27T23:32:05.864787+0000","last_fullsized":"2023-09-27T23:32:05.864787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:23:33.015795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847377+0000","last_change":"2023-09-27T23:32:05.473937+0000","last_active":"2023-09-27T23:32:05.847377+0000","last_peered":"2023-09-27T23:32:05.847377+0000","last_clean":"2023-09-27T23:32:05.847377+0000","last_became_active":"2023-09-27T23:32:05.473535+0000","last_became_peered":"2023-09-27T23:32:05.473535+0000","last_unstale":"2023-09-27T23:32:05.847377+0000","last_undegraded":"2023-09-27T23:32:05.847377+0000","last_fullsized":"2023-09-27T23:32:05.847377+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:59.468491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.880779+0000","last_change":"2023-09-27T23:32:05.877232+0000","last_active":"2023-09-27T23:32:05.880779+0000","last_peered":"2023-09-27T23:32:05.880779+0000","last_clean":"2023-09-27T23:32:05.880779+0000","last_became_active":"2023-09-27T23:32:05.876906+0000","last_became_peered":"2023-09-27T23:32:05.876906+0000","last_unstale":"2023-09-27T23:32:05.880779+0000","last_undegraded":"2023-09-27T23:32:05.880779+0000","last_fullsized":"2023-09-27T23:32:05.880779+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:58.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848215+0000","last_change":"2023-09-27T23:32:05.478424+0000","last_active":"2023-09-27T23:32:05.848215+0000","last_peered":"2023-09-27T23:32:05.848215+0000","last_clean":"2023-09-27T23:32:05.848215+0000","last_became_active":"2023-09-27T23:32:05.478120+0000","last_became_peered":"2023-09-27T23:32:05.478120+0000","last_unstale":"2023-09-27T23:32:05.848215+0000","last_undegraded":"2023-09-27T23:32:05.848215+0000","last_fullsized":"2023-09-27T23:32:05.848215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:15.570089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877156+0000","last_change":"2023-09-27T23:32:05.875429+0000","last_active":"2023-09-27T23:32:05.877156+0000","last_peered":"2023-09-27T23:32:05.877156+0000","last_clean":"2023-09-27T23:32:05.877156+0000","last_became_active":"2023-09-27T23:32:05.875127+0000","last_became_peered":"2023-09-27T23:32:05.875127+0000","last_unstale":"2023-09-27T23:32:05.877156+0000","last_undegraded":"2023-09-27T23:32:05.877156+0000","last_fullsized":"2023-09-27T23:32:05.877156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:21.585472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875964+0000","last_change":"2023-09-27T23:32:05.873357+0000","last_active":"2023-09-27T23:32:05.875964+0000","last_peered":"2023-09-27T23:32:05.875964+0000","last_clean":"2023-09-27T23:32:05.875964+0000","last_became_active":"2023-09-27T23:32:05.873037+0000","last_became_peered":"2023-09-27T23:32:05.873037+0000","last_unstale":"2023-09-27T23:32:05.875964+0000","last_undegraded":"2023-09-27T23:32:05.875964+0000","last_fullsized":"2023-09-27T23:32:05.875964+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:54:10.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882131+0000","last_change":"2023-09-27T23:32:05.487770+0000","last_active":"2023-09-27T23:32:05.882131+0000","last_peered":"2023-09-27T23:32:05.882131+0000","last_clean":"2023-09-27T23:32:05.882131+0000","last_became_active":"2023-09-27T23:32:05.487423+0000","last_became_peered":"2023-09-27T23:32:05.487423+0000","last_unstale":"2023-09-27T23:32:05.882131+0000","last_undegraded":"2023-09-27T23:32:05.882131+0000","last_fullsized":"2023-09-27T23:32:05.882131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:34.139978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:06.303516+0000","last_change":"2023-09-27T23:32:05.473896+0000","last_active":"2023-09-27T23:32:06.303516+0000","last_peered":"2023-09-27T23:32:06.303516+0000","last_clean":"2023-09-27T23:32:06.303516+0000","last_became_active":"2023-09-27T23:32:05.473665+0000","last_became_peered":"2023-09-27T23:32:05.473665+0000","last_unstale":"2023-09-27T23:32:06.303516+0000","last_undegraded":"2023-09-27T23:32:06.303516+0000","last_fullsized":"2023-09-27T23:32:06.303516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:17.109191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876032+0000","last_change":"2023-09-27T23:32:05.874148+0000","last_active":"2023-09-27T23:32:05.876032+0000","last_peered":"2023-09-27T23:32:05.876032+0000","last_clean":"2023-09-27T23:32:05.876032+0000","last_became_active":"2023-09-27T23:32:05.873853+0000","last_became_peered":"2023-09-27T23:32:05.873853+0000","last_unstale":"2023-09-27T23:32:05.876032+0000","last_undegraded":"2023-09-27T23:32:05.876032+0000","last_fullsized":"2023-09-27T23:32:05.876032+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:16:14.012071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846161+0000","last_change":"2023-09-27T23:32:05.482613+0000","last_active":"2023-09-27T23:32:05.846161+0000","last_peered":"2023-09-27T23:32:05.846161+0000","last_clean":"2023-09-27T23:32:05.846161+0000","last_became_active":"2023-09-27T23:32:05.482252+0000","last_became_peered":"2023-09-27T23:32:05.482252+0000","last_unstale":"2023-09-27T23:32:05.846161+0000","last_undegraded":"2023-09-27T23:32:05.846161+0000","last_fullsized":"2023-09-27T23:32:05.846161+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:05.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869011+0000","last_change":"2023-09-27T23:32:05.866830+0000","last_active":"2023-09-27T23:32:05.869011+0000","last_peered":"2023-09-27T23:32:05.869011+0000","last_clean":"2023-09-27T23:32:05.869011+0000","last_became_active":"2023-09-27T23:32:05.866498+0000","last_became_peered":"2023-09-27T23:32:05.866498+0000","last_unstale":"2023-09-27T23:32:05.869011+0000","last_undegraded":"2023-09-27T23:32:05.869011+0000","last_fullsized":"2023-09-27T23:32:05.869011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865072+0000","last_change":"2023-09-27T23:32:05.860941+0000","last_active":"2023-09-27T23:32:05.865072+0000","last_peered":"2023-09-27T23:32:05.865072+0000","last_clean":"2023-09-27T23:32:05.865072+0000","last_became_active":"2023-09-27T23:32:05.860543+0000","last_became_peered":"2023-09-27T23:32:05.860543+0000","last_unstale":"2023-09-27T23:32:05.865072+0000","last_undegraded":"2023-09-27T23:32:05.865072+0000","last_fullsized":"2023-09-27T23:32:05.865072+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.852958+0000","last_change":"2023-09-27T23:32:05.479760+0000","last_active":"2023-09-27T23:32:05.852958+0000","last_peered":"2023-09-27T23:32:05.852958+0000","last_clean":"2023-09-27T23:32:05.852958+0000","last_became_active":"2023-09-27T23:32:05.479474+0000","last_became_peered":"2023-09-27T23:32:05.479474+0000","last_unstale":"2023-09-27T23:32:05.852958+0000","last_undegraded":"2023-09-27T23:32:05.852958+0000","last_fullsized":"2023-09-27T23:32:05.852958+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:43.445603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881743+0000","last_change":"2023-09-27T23:32:05.879523+0000","last_active":"2023-09-27T23:32:05.881743+0000","last_peered":"2023-09-27T23:32:05.881743+0000","last_clean":"2023-09-27T23:32:05.881743+0000","last_became_active":"2023-09-27T23:32:05.878009+0000","last_became_peered":"2023-09-27T23:32:05.878009+0000","last_unstale":"2023-09-27T23:32:05.881743+0000","last_undegraded":"2023-09-27T23:32:05.881743+0000","last_fullsized":"2023-09-27T23:32:05.881743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:21:40.330475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847190+0000","last_change":"2023-09-27T23:32:05.473857+0000","last_active":"2023-09-27T23:32:05.847190+0000","last_peered":"2023-09-27T23:32:05.847190+0000","last_clean":"2023-09-27T23:32:05.847190+0000","last_became_active":"2023-09-27T23:32:05.473507+0000","last_became_peered":"2023-09-27T23:32:05.473507+0000","last_unstale":"2023-09-27T23:32:05.847190+0000","last_undegraded":"2023-09-27T23:32:05.847190+0000","last_fullsized":"2023-09-27T23:32:05.847190+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:19.244182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869149+0000","last_change":"2023-09-27T23:32:05.861010+0000","last_active":"2023-09-27T23:32:05.869149+0000","last_peered":"2023-09-27T23:32:05.869149+0000","last_clean":"2023-09-27T23:32:05.869149+0000","last_became_active":"2023-09-27T23:32:05.860737+0000","last_became_peered":"2023-09-27T23:32:05.860737+0000","last_unstale":"2023-09-27T23:32:05.869149+0000","last_undegraded":"2023-09-27T23:32:05.869149+0000","last_fullsized":"2023-09-27T23:32:05.869149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:58:21.738937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"117'227","reported_seq":650,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864897+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:32:05.864897+0000","last_peered":"2023-09-27T23:32:05.864897+0000","last_clean":"2023-09-27T23:32:05.864897+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:32:05.864897+0000","last_undegraded":"2023-09-27T23:32:05.864897+0000","last_fullsized":"2023-09-27T23:32:05.864897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":227,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":110,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-09-27T23:32:01.316522+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:32:01.316522+0000","last_peered":"2023-09-27T23:32:01.316522+0000","last_clean":"2023-09-27T23:32:01.316522+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:32:01.316522+0000","last_undegraded":"2023-09-27T23:32:01.316522+0000","last_fullsized":"2023-09-27T23:32:01.316522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869078+0000","last_change":"2023-09-27T23:32:05.862823+0000","last_active":"2023-09-27T23:32:05.869078+0000","last_peered":"2023-09-27T23:32:05.869078+0000","last_clean":"2023-09-27T23:32:05.869078+0000","last_became_active":"2023-09-27T23:32:05.862677+0000","last_became_peered":"2023-09-27T23:32:05.862677+0000","last_unstale":"2023-09-27T23:32:05.869078+0000","last_undegraded":"2023-09-27T23:32:05.869078+0000","last_fullsized":"2023-09-27T23:32:05.869078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:01.586755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846271+0000","last_change":"2023-09-27T23:32:05.487459+0000","last_active":"2023-09-27T23:32:05.846271+0000","last_peered":"2023-09-27T23:32:05.846271+0000","last_clean":"2023-09-27T23:32:05.846271+0000","last_became_active":"2023-09-27T23:32:05.487130+0000","last_became_peered":"2023-09-27T23:32:05.487130+0000","last_unstale":"2023-09-27T23:32:05.846271+0000","last_undegraded":"2023-09-27T23:32:05.846271+0000","last_fullsized":"2023-09-27T23:32:05.846271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:57.985464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864781+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:32:05.864781+0000","last_peered":"2023-09-27T23:32:05.864781+0000","last_clean":"2023-09-27T23:32:05.864781+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:32:05.864781+0000","last_undegraded":"2023-09-27T23:32:05.864781+0000","last_fullsized":"2023-09-27T23:32:05.864781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871725+0000","last_change":"2023-09-27T23:32:05.864506+0000","last_active":"2023-09-27T23:32:05.871725+0000","last_peered":"2023-09-27T23:32:05.871725+0000","last_clean":"2023-09-27T23:32:05.871725+0000","last_became_active":"2023-09-27T23:32:05.864207+0000","last_became_peered":"2023-09-27T23:32:05.864207+0000","last_unstale":"2023-09-27T23:32:05.871725+0000","last_undegraded":"2023-09-27T23:32:05.871725+0000","last_fullsized":"2023-09-27T23:32:05.871725+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:53:46.038154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875524+0000","last_change":"2023-09-27T23:32:05.478414+0000","last_active":"2023-09-27T23:32:05.875524+0000","last_peered":"2023-09-27T23:32:05.875524+0000","last_clean":"2023-09-27T23:32:05.875524+0000","last_became_active":"2023-09-27T23:32:05.478088+0000","last_became_peered":"2023-09-27T23:32:05.478088+0000","last_unstale":"2023-09-27T23:32:05.875524+0000","last_undegraded":"2023-09-27T23:32:05.875524+0000","last_fullsized":"2023-09-27T23:32:05.875524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:21.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847252+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:32:05.847252+0000","last_peered":"2023-09-27T23:32:05.847252+0000","last_clean":"2023-09-27T23:32:05.847252+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:32:05.847252+0000","last_undegraded":"2023-09-27T23:32:05.847252+0000","last_fullsized":"2023-09-27T23:32:05.847252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871490+0000","last_change":"2023-09-27T23:32:05.864496+0000","last_active":"2023-09-27T23:32:05.871490+0000","last_peered":"2023-09-27T23:32:05.871490+0000","last_clean":"2023-09-27T23:32:05.871490+0000","last_became_active":"2023-09-27T23:32:05.863828+0000","last_became_peered":"2023-09-27T23:32:05.863828+0000","last_unstale":"2023-09-27T23:32:05.871490+0000","last_undegraded":"2023-09-27T23:32:05.871490+0000","last_fullsized":"2023-09-27T23:32:05.871490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:46.777842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882372+0000","last_change":"2023-09-27T23:32:05.485399+0000","last_active":"2023-09-27T23:32:05.882372+0000","last_peered":"2023-09-27T23:32:05.882372+0000","last_clean":"2023-09-27T23:32:05.882372+0000","last_became_active":"2023-09-27T23:32:05.485073+0000","last_became_peered":"2023-09-27T23:32:05.485073+0000","last_unstale":"2023-09-27T23:32:05.882372+0000","last_undegraded":"2023-09-27T23:32:05.882372+0000","last_fullsized":"2023-09-27T23:32:05.882372+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:20.429282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":110,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-09-27T23:32:01.316522+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:32:01.316522+0000","last_peered":"2023-09-27T23:32:01.316522+0000","last_clean":"2023-09-27T23:32:01.316522+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:32:01.316522+0000","last_undegraded":"2023-09-27T23:32:01.316522+0000","last_fullsized":"2023-09-27T23:32:01.316522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876013+0000","last_change":"2023-09-27T23:32:05.874103+0000","last_active":"2023-09-27T23:32:05.876013+0000","last_peered":"2023-09-27T23:32:05.876013+0000","last_clean":"2023-09-27T23:32:05.876013+0000","last_became_active":"2023-09-27T23:32:05.873806+0000","last_became_peered":"2023-09-27T23:32:05.873806+0000","last_unstale":"2023-09-27T23:32:05.876013+0000","last_undegraded":"2023-09-27T23:32:05.876013+0000","last_fullsized":"2023-09-27T23:32:05.876013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:06:14.340339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.845266+0000","last_change":"2023-09-27T23:32:05.474807+0000","last_active":"2023-09-27T23:32:05.845266+0000","last_peered":"2023-09-27T23:32:05.845266+0000","last_clean":"2023-09-27T23:32:05.845266+0000","last_became_active":"2023-09-27T23:32:05.474510+0000","last_became_peered":"2023-09-27T23:32:05.474510+0000","last_unstale":"2023-09-27T23:32:05.845266+0000","last_undegraded":"2023-09-27T23:32:05.845266+0000","last_fullsized":"2023-09-27T23:32:05.845266+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:23.368573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.849356+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:32:05.849356+0000","last_peered":"2023-09-27T23:32:05.849356+0000","last_clean":"2023-09-27T23:32:05.849356+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:32:05.849356+0000","last_undegraded":"2023-09-27T23:32:05.849356+0000","last_fullsized":"2023-09-27T23:32:05.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847066+0000","last_change":"2023-09-27T23:32:05.477781+0000","last_active":"2023-09-27T23:32:05.847066+0000","last_peered":"2023-09-27T23:32:05.847066+0000","last_clean":"2023-09-27T23:32:05.847066+0000","last_became_active":"2023-09-27T23:32:05.477416+0000","last_became_peered":"2023-09-27T23:32:05.477416+0000","last_unstale":"2023-09-27T23:32:05.847066+0000","last_undegraded":"2023-09-27T23:32:05.847066+0000","last_fullsized":"2023-09-27T23:32:05.847066+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:35.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863310+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:32:05.863310+0000","last_peered":"2023-09-27T23:32:05.863310+0000","last_clean":"2023-09-27T23:32:05.863310+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:32:05.863310+0000","last_undegraded":"2023-09-27T23:32:05.863310+0000","last_fullsized":"2023-09-27T23:32:05.863310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:05.445440+0000","last_change":"2023-09-27T23:32:05.445440+0000","last_active":"2023-09-27T23:32:05.445440+0000","last_peered":"2023-09-27T23:32:05.445440+0000","last_clean":"2023-09-27T23:32:05.445440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:05.445440+0000","last_undegraded":"2023-09-27T23:32:05.445440+0000","last_fullsized":"2023-09-27T23:32:05.445440+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848976+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:32:05.848976+0000","last_peered":"2023-09-27T23:32:05.848976+0000","last_clean":"2023-09-27T23:32:05.848976+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:32:05.848976+0000","last_undegraded":"2023-09-27T23:32:05.848976+0000","last_fullsized":"2023-09-27T23:32:05.848976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869005+0000","last_change":"2023-09-27T23:32:05.862414+0000","last_active":"2023-09-27T23:32:05.869005+0000","last_peered":"2023-09-27T23:32:05.869005+0000","last_clean":"2023-09-27T23:32:05.869005+0000","last_became_active":"2023-09-27T23:32:05.862102+0000","last_became_peered":"2023-09-27T23:32:05.862102+0000","last_unstale":"2023-09-27T23:32:05.869005+0000","last_undegraded":"2023-09-27T23:32:05.869005+0000","last_fullsized":"2023-09-27T23:32:05.869005+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:40.054520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:32:04.433983+0000","last_change":"2023-09-27T23:32:04.433983+0000","last_active":"2023-09-27T23:32:04.433983+0000","last_peered":"2023-09-27T23:32:04.433983+0000","last_clean":"2023-09-27T23:32:04.433983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:32:04.433983+0000","last_undegraded":"2023-09-27T23:32:04.433983+0000","last_fullsized":"2023-09-27T23:32:04.433983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847811+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:32:05.847811+0000","last_peered":"2023-09-27T23:32:05.847811+0000","last_clean":"2023-09-27T23:32:05.847811+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:32:05.847811+0000","last_undegraded":"2023-09-27T23:32:05.847811+0000","last_fullsized":"2023-09-27T23:32:05.847811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":104,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":40,"acting":40,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509663,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":25024,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":2191360,"data_stored":1415956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25624576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.52200000000000002,"15min":0.51100000000000001},"max":{"1min":1.2470000000000001,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1819999999999999,"5min":1.5489999999999999,"15min":1.712},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91900000000000004,"15min":0.89700000000000002},"min":{"1min":0.66000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1759999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95699999999999996,"15min":0.93899999999999995},"min":{"1min":0.69599999999999995,"5min":0.66400000000000003,"15min":0.28299999999999997},"max":{"1min":1.2430000000000001,"5min":1.337,"15min":1.7589999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.90200000000000002,"15min":0.91500000000000004},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.307},"max":{"1min":1.238,"5min":1.3129999999999999,"15min":2.681},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.97599999999999998,"15min":0.97899999999999998},"min":{"1min":0.77700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.405,"15min":1.718},"last":0.997}]},{"osd":3,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.93500000000000005},"min":{"1min":0.71099999999999997,"5min":0.63400000000000001,"15min":0.53200000000000003},"max":{"1min":1.151,"5min":1.204,"15min":1.421},"last":1.042},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.747,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.409,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.073}]},{"osd":4,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.117,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97499999999999998,"15min":0.98499999999999999},"min":{"1min":0.65200000000000002,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.3029999999999999,"5min":1.3240000000000001,"15min":2.8879999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.67000000000000004,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.3540000000000001,"5min":1.79,"15min":1.79},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.252,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.139},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.839,"15min":1.839},"last":1.246},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.107,"15min":1.1259999999999999},"min":{"1min":0.96599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":2.548},"last":1.218}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27100,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":25536,"kb_avail":94344740,"statfs":{"total":96636764160,"available":96609013760,"internally_reserved":0,"allocated":1593344,"data_stored":825070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26148864},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89200000000000002,"15min":0.94399999999999995},"min":{"1min":0.629,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.2130000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.78500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.246,"5min":1.321,"15min":1.5149999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.88},"min":{"1min":0.56299999999999994,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.214,"5min":1.248,"15min":1.323},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.258,"5min":1.403,"15min":1.403},"last":0.93899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.875},"min":{"1min":0.72099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2150000000000001,"5min":1.222,"15min":1.353},"last":1.238},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91100000000000003,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.246,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.629,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2170000000000001,"5min":1.51,"15min":1.734},"last":1.073},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.72499999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1619999999999999,"5min":1.278,"15min":1.4970000000000001},"last":1.4279999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1100000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.274},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2929999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.81499999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2669999999999999,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.343},{"interface":"front","average":{"1min":1.119,"5min":1.1319999999999999,"15min":1.1539999999999999},"min":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3169999999999999,"5min":1.4259999999999999,"15min":1.49},"last":1.121}]},{"osd":7,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.81399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3580000000000001,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1259999999999999,"15min":1.145},"min":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.6619999999999999},"last":1.4750000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509661,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36968,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35456,"kb_avail":94334872,"statfs":{"total":96636764160,"available":96598908928,"internally_reserved":0,"allocated":1540096,"data_stored":797189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36306944},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.236,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.1599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.526},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.874,"15min":0.88300000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2190000000000001,"5min":1.242,"15min":1.331},"last":1.24}]},{"osd":2,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.73999999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.474,"15min":1.474},"last":0.874},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97099999999999997,"15min":0.95099999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.5740000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.97599999999999998,"15min":0.95599999999999996},"min":{"1min":0.70499999999999996,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":1.23,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.999,"15min":1.004},"min":{"1min":0.80800000000000005,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.323,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.56200000000000006,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.3220000000000001,"5min":1.341,"15min":1.341},"last":1.099},{"interface":"front","average":{"1min":1.109,"5min":1.038,"15min":1.0209999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.613,"15min":1.613},"last":1.218}]},{"osd":6,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0089999999999999,"15min":1.01},"min":{"1min":0.70499999999999996,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.5580000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.1020000000000001},"min":{"1min":0.876,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.331}]},{"osd":7,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1619999999999999,"15min":1.1379999999999999},"min":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.446,"5min":1.665,"15min":1.6679999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.198,"15min":1.1890000000000001},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.508,"5min":1.528,"15min":1.633},"last":1.28}]}]},{"osd":4,"up_from":15,"seq":64424509663,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23000,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94348840,"statfs":{"total":96636764160,"available":96613212160,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.876,"15min":0.89300000000000002},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.27,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.54400000000000004},"max":{"1min":1.212,"5min":1.343,"15min":1.45},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89500000000000002,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.262,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.73899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90200000000000002,"15min":0.91100000000000003},"min":{"1min":0.65600000000000003,"5min":0.57499999999999996,"15min":0.497},"max":{"1min":1.2410000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97999999999999998,"15min":0.96199999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.252,"5min":1.331,"15min":1.407},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92000000000000004},"min":{"1min":0.70999999999999996,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2150000000000001,"5min":1.272,"15min":1.2829999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.03,"15min":1.026},"min":{"1min":0.79900000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3660000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4610000000000001,"5min":1.609,"15min":1.609},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0229999999999999,"15min":1.01},"min":{"1min":0.66800000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0389999999999999,"15min":1.04},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3169999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.161,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.1499999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0940000000000001,"15min":1.091},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.427,"5min":1.4770000000000001,"15min":1.627},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.145,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.85899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.181}]}]},{"osd":3,"up_from":15,"seq":64424509662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":1605632,"data_stored":826606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.70199999999999996},"min":{"1min":0.40400000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.012,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.109,"5min":1.323,"15min":1.3440000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.57499999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.61499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.753,"5min":0.505,"15min":0.505},"max":{"1min":1.298,"5min":1.605,"15min":1.605},"last":1.163}]},{"osd":5,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0820000000000001,"15min":1.111},"min":{"1min":0.85099999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3999999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.232},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1160000000000001,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.714,"15min":1.764},"last":1.431}]},{"osd":6,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.105,"15min":1.1160000000000001},"min":{"1min":0.81200000000000006,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1950000000000001,"15min":1.2150000000000001},"min":{"1min":0.94899999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.488,"5min":1.6599999999999999,"15min":1.712},"last":1.3200000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.2370000000000001},"min":{"1min":0.79800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.173,"15min":1.202},"min":{"1min":0.88,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.143}]}]},{"osd":2,"up_from":15,"seq":64424509663,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22168,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94349672,"statfs":{"total":96636764160,"available":96614064128,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.903},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.097,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":0.98499999999999999,"5min":1.228,"15min":1.391},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":1.2949999999999999,"5min":1.49,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.93400000000000005},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.399,"5min":1.399,"15min":1.444},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0349999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.40600000000000003},"max":{"1min":1.417,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0629999999999999,"15min":1.0600000000000001},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":1.635,"15min":1.7390000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0269999999999999,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":0.89300000000000002,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.92500000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.722,"5min":1.722,"15min":1.9450000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.125,"15min":1.1539999999999999},"min":{"1min":0.83099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.623,"5min":1.623,"15min":2.0129999999999999},"last":1.0029999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2110000000000001,"15min":1.212},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1910000000000001,"15min":1.212},"min":{"1min":0.76000000000000001,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.5389999999999999,"5min":1.556,"15min":2.073},"last":1.0600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542369,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35840,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":1597440,"data_stored":825124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36700160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.503,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.123,"15min":1.3560000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95199999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.73299999999999998},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.2490000000000001,"15min":1.321},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.88200000000000001},"min":{"1min":0.60499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1439999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.83599999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.81299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2110000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.696,"15min":1.696},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97999999999999998,"15min":0.97299999999999998},"min":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.282,"5min":1.464,"15min":1.464},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.121,"15min":1.131},"min":{"1min":0.75600000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.774},"last":1.196}]},{"osd":5,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0860000000000001},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5109999999999999,"5min":1.546,"15min":1.623},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.88900000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.395,"5min":1.589,"15min":1.589},"last":1.2569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.089},{"interface":"front","average":{"1min":1.107,"5min":1.165,"15min":1.1850000000000001},"min":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.554,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4179999999999999,"5min":1.877,"15min":1.877},"last":1.407},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.194,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4410000000000001,"5min":1.5529999999999999,"15min":1.6339999999999999},"last":1.141}]}]},{"osd":0,"up_from":15,"seq":64424509663,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":2121728,"data_stored":1381209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.874},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.751,"15min":0.78900000000000003},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1970000000000001,"5min":1.3,"15min":1.3},"last":0.95999999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.73099999999999998,"15min":0.73199999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":1.2150000000000001,"15min":1.373},"last":0.625},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.232},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0640000000000001,"5min":1.216,"15min":1.2769999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.85299999999999998,"15min":0.88300000000000001},"min":{"1min":0.46800000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":1.3160000000000001,"15min":1.333},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97299999999999998,"15min":1.0049999999999999},"min":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.47},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.997,"15min":1.032},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.206,"5min":1.4239999999999999,"15min":1.587},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.089},"min":{"1min":0.75700000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.468},{"interface":"front","average":{"1min":1.087,"5min":1.103,"15min":1.133},"min":{"1min":0.82799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.452,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.98999999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.76800000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3740000000000001,"5min":1.5269999999999999,"15min":1.7729999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1819999999999999,"15min":1.2110000000000001},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.52,"5min":1.73,"15min":1.8300000000000001},"last":1.417}]},{"osd":7,"last update":"Wed Sep 27 23:31:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.204,"15min":1.232},"min":{"1min":0.78800000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.4490000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.181,"15min":1.22},"min":{"1min":0.92600000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":1.103}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:32:10.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-09-27T23:32:10.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-09-27T23:32:10.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-09-27T23:32:10.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-09-27T23:32:10.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-09-27T23:32:10.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-09-27T23:32:10.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-09-27T23:32:10.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-09-27T23:32:10.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-09-27T23:32:10.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-09-27T23:32:10.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:05.445440+0000', 'last_change': '2023-09-27T23:32:05.445440+0000', 'last_active': '2023-09-27T23:32:05.445440+0000', 'last_peered': '2023-09-27T23:32:05.445440+0000', 'last_clean': '2023-09-27T23:32:05.445440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:05.445440+0000', 'last_undegraded': '2023-09-27T23:32:05.445440+0000', 'last_fullsized': '2023-09-27T23:32:05.445440+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:05.445440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-09-27T23:32:10.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-09-27T23:32:10.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:32:04.433983+0000', 'last_change': '2023-09-27T23:32:04.433983+0000', 'last_active': '2023-09-27T23:32:04.433983+0000', 'last_peered': '2023-09-27T23:32:04.433983+0000', 'last_clean': '2023-09-27T23:32:04.433983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:32:04.433983+0000', 'last_undegraded': '2023-09-27T23:32:04.433983+0000', 'last_fullsized': '2023-09-27T23:32:04.433983+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'last_clean_scrub_stamp': '2023-09-27T23:32:04.433983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:32:10.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:32:10.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 Processor -- start 2023-09-27T23:32:10.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 -- start start 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ecc06e1d0 0x7f0ecc159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0ecc06ad70 0x7f0ecc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc15a260 con 0x7f0ecc14f4d0 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc06b750 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed2c5c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc0afad0 con 0x7f0ecc06ad70 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ecc06e1d0 0x7f0ecc159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ecc06e1d0 0x7f0ecc159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50212/0 (socket says 172.21.15.64:50212) 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 -- 172.21.15.64:0/73444989 learned_addr learned my addr 172.21.15.64:0/73444989 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1459700 1 --2- 172.21.15.64:0/73444989 >> v2:172.21.15.123:3300/0 conn(0x7f0ecc06ad70 0x7f0ecc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 -- 172.21.15.64:0/73444989 >> v2:172.21.15.123:3300/0 conn(0x7f0ecc06ad70 msgr2=0x7f0ecc06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:10.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 --2- 172.21.15.64:0/73444989 >> v2:172.21.15.123:3300/0 conn(0x7f0ecc06ad70 0x7f0ecc06b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 -- 172.21.15.64:0/73444989 >> v1:172.21.15.64:6789/0 conn(0x7f0ecc14f4d0 legacy=0x7f0ecc14f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 -- 172.21.15.64:0/73444989 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0afdb0 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1459700 1 --2- 172.21.15.64:0/73444989 >> v2:172.21.15.123:3300/0 conn(0x7f0ecc06ad70 0x7f0ecc06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed1c5a700 1 --2- 172.21.15.64:0/73444989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc159d20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0af460 tx=0x7f0ec4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b4f7d721236389 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed0c58700 1 -- 172.21.15.64:0/73444989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400e070 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed0c58700 1 -- 172.21.15.64:0/73444989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400ba50 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.312+0000 7f0ed0c58700 1 -- 172.21.15.64:0/73444989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40092e0 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/73444989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 msgr2=0x7f0ecc159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/73444989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc159d20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0af460 tx=0x7f0ec4009840 comp rx=0 tx=0).stop 2023-09-27T23:32:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/73444989 shutdown_connections 2023-09-27T23:32:10.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/73444989 >> v2:172.21.15.123:3300/0 conn(0x7f0ecc06ad70 0x7f0ecc06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/73444989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc159d20 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/73444989 >> 172.21.15.64:0/73444989 conn(0x7f0ecc067bb0 msgr2=0x7f0ecc0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/73444989 shutdown_connections 2023-09-27T23:32:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/73444989 wait complete. 2023-09-27T23:32:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 Processor -- start 2023-09-27T23:32:10.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- start start 2023-09-27T23:32:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ecc06ad70 0x7f0ecc154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ecc14f4d0 0x7f0ecc14eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc16d4c0 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc16c8b0 con 0x7f0ecc06ad70 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed2c5c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc16cb90 con 0x7f0ecc14f4d0 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed1459700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed1459700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc14e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50224/0 (socket says 172.21.15.64:50224) 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed1459700 1 -- 172.21.15.64:0/3766872049 learned_addr learned my addr 172.21.15.64:0/3766872049 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed1c5a700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ecc06ad70 0x7f0ecc154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.316+0000 7f0ed245b700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ecc14f4d0 0x7f0ecc14eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed1459700 1 -- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ecc14f4d0 msgr2=0x7f0ecc14eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed1459700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ecc14f4d0 0x7f0ecc14eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed1459700 1 -- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ecc06ad70 msgr2=0x7f0ecc154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed1459700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ecc06ad70 0x7f0ecc154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed1459700 1 -- 172.21.15.64:0/3766872049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4009ae0 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed1459700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc14e990 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c830 tx=0x7f0ebc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:10.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc011600 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc011760 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc16ce70 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc011b20 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc154880 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0ebc018360 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.320+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc154aa0 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.324+0000 7f0ec1ffb700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0eb0060a70 0x7f0eb0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.324+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0ebc07d150 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.324+0000 7f0ed1c5a700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0eb0060a70 0x7f0eb0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.324+0000 7f0ed1c5a700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0eb0060a70 0x7f0eb0062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009880 tx=0x7f0ec4018000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.332+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0ebc04d070 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.544+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0ecc04be40 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.544+0000 7f0ec1ffb700 1 -- 172.21.15.64:0/3766872049 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3289 (secure 0 0 0) 0x7f0ebc050690 con 0x7f0ecc06e1d0 2023-09-27T23:32:10.550 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:10.551 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1129,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":74},{"state_name":"unknown","count":31}],"num_pgs":105,"num_pools":4,"num_objects":15,"data_bytes":590472,"bytes_used":235077632,"bytes_avail":772859035648,"bytes_total":773094113280,"unknown_pgs_ratio":0.29523810744285583,"write_bytes_sec":1674,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":46,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8151}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-09-27T23:31:54.726214+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0eb0060a70 msgr2=0x7f0eb0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0eb0060a70 0x7f0eb0062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009880 tx=0x7f0ec4018000 comp rx=0 tx=0).stop 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 msgr2=0x7f0ecc14e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc14e990 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c830 tx=0x7f0ebc008040 comp rx=0 tx=0).stop 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 shutdown_connections 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0eb0060a70 0x7f0eb0062f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ecc14f4d0 0x7f0ecc14eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc06e1d0 0x7f0ecc14e990 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 --2- 172.21.15.64:0/3766872049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ecc06ad70 0x7f0ecc154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 >> 172.21.15.64:0/3766872049 conn(0x7f0ecc067bb0 msgr2=0x7f0ecc158700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:10.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 shutdown_connections 2023-09-27T23:32:10.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.548+0000 7f0ed2c5c700 1 -- 172.21.15.64:0/3766872049 wait complete. 2023-09-27T23:32:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:32:10.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:32:10.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.660+0000 7fcd66378700 1 Processor -- start 2023-09-27T23:32:10.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.660+0000 7fcd66378700 1 -- start start 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd601502b0 0x7fcd60068060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd600c5900 0x7fcd600c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd600c6630 con 0x7fcd60067c90 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd600c6910 con 0x7fcd600c5900 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600c6bf0 con 0x7fcd601502b0 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd64915700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd600c5900 0x7fcd600c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd601502b0 0x7fcd60068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd64915700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd600c5900 0x7fcd600c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50234/0 (socket says 172.21.15.64:50234) 2023-09-27T23:32:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd64915700 1 -- 172.21.15.64:0/882071330 learned_addr learned my addr 172.21.15.64:0/882071330 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5ffff700 1 --1- 172.21.15.64:0/882071330 >> v1:172.21.15.123:6789/0 conn(0x7fcd60067c90 0x7fcd6014fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:55626/0 (socket says 172.21.15.64:55626) 2023-09-27T23:32:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5f7fe700 1 -- 172.21.15.64:0/882071330 >> v2:172.21.15.64:3300/0 conn(0x7fcd600c5900 msgr2=0x7fcd600c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5f7fe700 1 --2- 172.21.15.64:0/882071330 >> v2:172.21.15.64:3300/0 conn(0x7fcd600c5900 0x7fcd600c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5effd700 1 -- 172.21.15.64:0/882071330 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920096899 0 0) 0x7fcd600c6630 con 0x7fcd60067c90 2023-09-27T23:32:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5f7fe700 1 -- 172.21.15.64:0/882071330 >> v1:172.21.15.123:6789/0 conn(0x7fcd60067c90 legacy=0x7fcd6014fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5f7fe700 1 -- 172.21.15.64:0/882071330 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd600c6ed0 con 0x7fcd601502b0 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5f7fe700 1 --2- 172.21.15.64:0/882071330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd601502b0 0x7fcd60068060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098e0 tx=0x7fcd500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a42d1c7c8df6db server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5effd700 1 -- 172.21.15.64:0/882071330 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000b640 con 0x7fcd601502b0 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5effd700 1 -- 172.21.15.64:0/882071330 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000b7a0 con 0x7fcd601502b0 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd5effd700 1 -- 172.21.15.64:0/882071330 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000bb40 con 0x7fcd601502b0 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- 172.21.15.64:0/882071330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd601502b0 msgr2=0x7fcd60068060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 --2- 172.21.15.64:0/882071330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd601502b0 0x7fcd60068060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098e0 tx=0x7fcd500091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- 172.21.15.64:0/882071330 shutdown_connections 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 --2- 172.21.15.64:0/882071330 >> v2:172.21.15.64:3300/0 conn(0x7fcd600c5900 0x7fcd600c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 --2- 172.21.15.64:0/882071330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd601502b0 0x7fcd60068060 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- 172.21.15.64:0/882071330 >> 172.21.15.64:0/882071330 conn(0x7fcd600cd6d0 msgr2=0x7fcd6006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- 172.21.15.64:0/882071330 shutdown_connections 2023-09-27T23:32:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.664+0000 7fcd66378700 1 -- 172.21.15.64:0/882071330 wait complete. 2023-09-27T23:32:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 Processor -- start 2023-09-27T23:32:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 -- start start 2023-09-27T23:32:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 0x7fcd600b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd600c5900 0x7fcd600b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd601502b0 0x7fcd60152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600b4e40 con 0x7fcd60067c90 2023-09-27T23:32:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600b5120 con 0x7fcd601502b0 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600b5400 con 0x7fcd600c5900 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 0x7fcd600b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 0x7fcd600b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50242/0 (socket says 172.21.15.64:50242) 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 -- 172.21.15.64:0/3609452484 learned_addr learned my addr 172.21.15.64:0/3609452484 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5f7fe700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd600c5900 0x7fcd600b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd64915700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd601502b0 0x7fcd60152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 -- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd600c5900 msgr2=0x7fcd600b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd600c5900 0x7fcd600b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 -- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd601502b0 msgr2=0x7fcd60152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd601502b0 0x7fcd60152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 -- 172.21.15.64:0/3609452484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009420 con 0x7fcd60067c90 2023-09-27T23:32:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd5ffff700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 0x7fcd600b40d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcd540011e0 tx=0x7fcd54007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54005a70 con 0x7fcd60067c90 2023-09-27T23:32:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd54005bd0 con 0x7fcd60067c90 2023-09-27T23:32:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd600b56e0 con 0x7fcd60067c90 2023-09-27T23:32:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd540070d0 con 0x7fcd60067c90 2023-09-27T23:32:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.668+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd60151090 con 0x7fcd60067c90 2023-09-27T23:32:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.672+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd60071120 con 0x7fcd60067c90 2023-09-27T23:32:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.672+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fcd540074c0 con 0x7fcd60067c90 2023-09-27T23:32:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.672+0000 7fcd47fff700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd4c060aa0 0x7fcd4c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.672+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fcd5407c490 con 0x7fcd60067c90 2023-09-27T23:32:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.672+0000 7fcd5f7fe700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd4c060aa0 0x7fcd4c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.676+0000 7fcd5f7fe700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd4c060aa0 0x7fcd4c062f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd500091e0 tx=0x7fcd50017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:10.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.680+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcd54049bb0 con 0x7fcd60067c90 2023-09-27T23:32:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.836+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd600c78c0 con 0x7fcd4c060aa0 2023-09-27T23:32:10.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.852+0000 7fcd47fff700 1 -- 172.21.15.64:0/3609452484 <== mgr.6971 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308106 (secure 0 0 0) 0x7fcd600c78c0 con 0x7fcd4c060aa0 2023-09-27T23:32:10.865 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd4c060aa0 msgr2=0x7fcd4c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd4c060aa0 0x7fcd4c062f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd500091e0 tx=0x7fcd50017000 comp rx=0 tx=0).stop 2023-09-27T23:32:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 msgr2=0x7fcd600b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:10.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 0x7fcd600b40d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcd540011e0 tx=0x7fcd54007e10 comp rx=0 tx=0).stop 2023-09-27T23:32:10.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 shutdown_connections 2023-09-27T23:32:10.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcd4c060aa0 0x7fcd4c062f50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd601502b0 0x7fcd60152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd600c5900 0x7fcd600b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 --2- 172.21.15.64:0/3609452484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd60067c90 0x7fcd600b40d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:10.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 >> 172.21.15.64:0/3609452484 conn(0x7fcd600cd6d0 msgr2=0x7fcd600ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:10.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 shutdown_connections 2023-09-27T23:32:10.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:10.860+0000 7fcd66378700 1 -- 172.21.15.64:0/3609452484 wait complete. 2023-09-27T23:32:10.869 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:32:10.884 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-27T23:32:10.731150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":404,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230088,"kb_used_data":13640,"kb_used_omap":0,"kb_used_meta":216384,"kb_avail":754744632,"statfs":{"total":773094113280,"available":772858503168,"internally_reserved":0,"allocated":13967360,"data_stored":7785348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":221577216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001957"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885283+0000","last_change":"2023-09-27T23:32:05.874869+0000","last_active":"2023-09-27T23:32:05.885283+0000","last_peered":"2023-09-27T23:32:05.885283+0000","last_clean":"2023-09-27T23:32:05.885283+0000","last_became_active":"2023-09-27T23:32:05.874146+0000","last_became_peered":"2023-09-27T23:32:05.874146+0000","last_unstale":"2023-09-27T23:32:05.885283+0000","last_undegraded":"2023-09-27T23:32:05.885283+0000","last_fullsized":"2023-09-27T23:32:05.885283+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863665+0000","last_change":"2023-09-27T23:32:05.860722+0000","last_active":"2023-09-27T23:32:05.863665+0000","last_peered":"2023-09-27T23:32:05.863665+0000","last_clean":"2023-09-27T23:32:05.863665+0000","last_became_active":"2023-09-27T23:32:05.860381+0000","last_became_peered":"2023-09-27T23:32:05.860381+0000","last_unstale":"2023-09-27T23:32:05.863665+0000","last_undegraded":"2023-09-27T23:32:05.863665+0000","last_fullsized":"2023-09-27T23:32:05.863665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879763+0000","last_change":"2023-09-27T23:32:05.873945+0000","last_active":"2023-09-27T23:32:05.879763+0000","last_peered":"2023-09-27T23:32:05.879763+0000","last_clean":"2023-09-27T23:32:05.879763+0000","last_became_active":"2023-09-27T23:32:05.872503+0000","last_became_peered":"2023-09-27T23:32:05.872503+0000","last_unstale":"2023-09-27T23:32:05.879763+0000","last_undegraded":"2023-09-27T23:32:05.879763+0000","last_fullsized":"2023-09-27T23:32:05.879763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:06.052849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885427+0000","last_change":"2023-09-27T23:32:05.878460+0000","last_active":"2023-09-27T23:32:05.885427+0000","last_peered":"2023-09-27T23:32:05.885427+0000","last_clean":"2023-09-27T23:32:05.885427+0000","last_became_active":"2023-09-27T23:32:05.877778+0000","last_became_peered":"2023-09-27T23:32:05.877778+0000","last_unstale":"2023-09-27T23:32:05.885427+0000","last_undegraded":"2023-09-27T23:32:05.885427+0000","last_fullsized":"2023-09-27T23:32:05.885427+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:51:08.482614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.866846+0000","last_change":"2023-09-27T23:32:05.863147+0000","last_active":"2023-09-27T23:32:05.866846+0000","last_peered":"2023-09-27T23:32:05.866846+0000","last_clean":"2023-09-27T23:32:05.866846+0000","last_became_active":"2023-09-27T23:32:05.862602+0000","last_became_peered":"2023-09-27T23:32:05.862602+0000","last_unstale":"2023-09-27T23:32:05.866846+0000","last_undegraded":"2023-09-27T23:32:05.866846+0000","last_fullsized":"2023-09-27T23:32:05.866846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:01.411281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867813+0000","last_change":"2023-09-27T23:32:05.861426+0000","last_active":"2023-09-27T23:32:05.867813+0000","last_peered":"2023-09-27T23:32:05.867813+0000","last_clean":"2023-09-27T23:32:05.867813+0000","last_became_active":"2023-09-27T23:32:05.861004+0000","last_became_peered":"2023-09-27T23:32:05.861004+0000","last_unstale":"2023-09-27T23:32:05.867813+0000","last_undegraded":"2023-09-27T23:32:05.867813+0000","last_fullsized":"2023-09-27T23:32:05.867813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:09.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.870308+0000","last_change":"2023-09-27T23:32:05.865614+0000","last_active":"2023-09-27T23:32:05.870308+0000","last_peered":"2023-09-27T23:32:05.870308+0000","last_clean":"2023-09-27T23:32:05.870308+0000","last_became_active":"2023-09-27T23:32:05.861505+0000","last_became_peered":"2023-09-27T23:32:05.861505+0000","last_unstale":"2023-09-27T23:32:05.870308+0000","last_undegraded":"2023-09-27T23:32:05.870308+0000","last_fullsized":"2023-09-27T23:32:05.870308+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:36.735279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881110+0000","last_change":"2023-09-27T23:32:05.878031+0000","last_active":"2023-09-27T23:32:05.881110+0000","last_peered":"2023-09-27T23:32:05.881110+0000","last_clean":"2023-09-27T23:32:05.881110+0000","last_became_active":"2023-09-27T23:32:05.877499+0000","last_became_peered":"2023-09-27T23:32:05.877499+0000","last_unstale":"2023-09-27T23:32:05.881110+0000","last_undegraded":"2023-09-27T23:32:05.881110+0000","last_fullsized":"2023-09-27T23:32:05.881110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:24.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885247+0000","last_change":"2023-09-27T23:32:05.874547+0000","last_active":"2023-09-27T23:32:05.885247+0000","last_peered":"2023-09-27T23:32:05.885247+0000","last_clean":"2023-09-27T23:32:05.885247+0000","last_became_active":"2023-09-27T23:32:05.874003+0000","last_became_peered":"2023-09-27T23:32:05.874003+0000","last_unstale":"2023-09-27T23:32:05.885247+0000","last_undegraded":"2023-09-27T23:32:05.885247+0000","last_fullsized":"2023-09-27T23:32:05.885247+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885163+0000","last_change":"2023-09-27T23:32:05.873890+0000","last_active":"2023-09-27T23:32:05.885163+0000","last_peered":"2023-09-27T23:32:05.885163+0000","last_clean":"2023-09-27T23:32:05.885163+0000","last_became_active":"2023-09-27T23:32:05.870764+0000","last_became_peered":"2023-09-27T23:32:05.870764+0000","last_unstale":"2023-09-27T23:32:05.885163+0000","last_undegraded":"2023-09-27T23:32:05.885163+0000","last_fullsized":"2023-09-27T23:32:05.885163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865013+0000","last_change":"2023-09-27T23:32:05.861704+0000","last_active":"2023-09-27T23:32:05.865013+0000","last_peered":"2023-09-27T23:32:05.865013+0000","last_clean":"2023-09-27T23:32:05.865013+0000","last_became_active":"2023-09-27T23:32:05.861317+0000","last_became_peered":"2023-09-27T23:32:05.861317+0000","last_unstale":"2023-09-27T23:32:05.865013+0000","last_undegraded":"2023-09-27T23:32:05.865013+0000","last_fullsized":"2023-09-27T23:32:05.865013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885469+0000","last_change":"2023-09-27T23:32:05.878755+0000","last_active":"2023-09-27T23:32:05.885469+0000","last_peered":"2023-09-27T23:32:05.885469+0000","last_clean":"2023-09-27T23:32:05.885469+0000","last_became_active":"2023-09-27T23:32:05.878132+0000","last_became_peered":"2023-09-27T23:32:05.878132+0000","last_unstale":"2023-09-27T23:32:05.885469+0000","last_undegraded":"2023-09-27T23:32:05.885469+0000","last_fullsized":"2023-09-27T23:32:05.885469+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:11:14.766160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881099+0000","last_change":"2023-09-27T23:32:05.877897+0000","last_active":"2023-09-27T23:32:05.881099+0000","last_peered":"2023-09-27T23:32:05.881099+0000","last_clean":"2023-09-27T23:32:05.881099+0000","last_became_active":"2023-09-27T23:32:05.877494+0000","last_became_peered":"2023-09-27T23:32:05.877494+0000","last_unstale":"2023-09-27T23:32:05.881099+0000","last_undegraded":"2023-09-27T23:32:05.881099+0000","last_fullsized":"2023-09-27T23:32:05.881099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:55.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879623+0000","last_change":"2023-09-27T23:32:05.872800+0000","last_active":"2023-09-27T23:32:05.879623+0000","last_peered":"2023-09-27T23:32:05.879623+0000","last_clean":"2023-09-27T23:32:05.879623+0000","last_became_active":"2023-09-27T23:32:05.872486+0000","last_became_peered":"2023-09-27T23:32:05.872486+0000","last_unstale":"2023-09-27T23:32:05.879623+0000","last_undegraded":"2023-09-27T23:32:05.879623+0000","last_fullsized":"2023-09-27T23:32:05.879623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:57.553491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873468+0000","last_change":"2023-09-27T23:32:05.872253+0000","last_active":"2023-09-27T23:32:05.873468+0000","last_peered":"2023-09-27T23:32:05.873468+0000","last_clean":"2023-09-27T23:32:05.873468+0000","last_became_active":"2023-09-27T23:32:05.871935+0000","last_became_peered":"2023-09-27T23:32:05.871935+0000","last_unstale":"2023-09-27T23:32:05.873468+0000","last_undegraded":"2023-09-27T23:32:05.873468+0000","last_fullsized":"2023-09-27T23:32:05.873468+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:00.996007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885494+0000","last_change":"2023-09-27T23:32:05.878992+0000","last_active":"2023-09-27T23:32:05.885494+0000","last_peered":"2023-09-27T23:32:05.885494+0000","last_clean":"2023-09-27T23:32:05.885494+0000","last_became_active":"2023-09-27T23:32:05.878488+0000","last_became_peered":"2023-09-27T23:32:05.878488+0000","last_unstale":"2023-09-27T23:32:05.885494+0000","last_undegraded":"2023-09-27T23:32:05.885494+0000","last_fullsized":"2023-09-27T23:32:05.885494+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865348+0000","last_change":"2023-09-27T23:32:05.863475+0000","last_active":"2023-09-27T23:32:05.865348+0000","last_peered":"2023-09-27T23:32:05.865348+0000","last_clean":"2023-09-27T23:32:05.865348+0000","last_became_active":"2023-09-27T23:32:05.863171+0000","last_became_peered":"2023-09-27T23:32:05.863171+0000","last_unstale":"2023-09-27T23:32:05.865348+0000","last_undegraded":"2023-09-27T23:32:05.865348+0000","last_fullsized":"2023-09-27T23:32:05.865348+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862603+0000","last_change":"2023-09-27T23:32:05.859721+0000","last_active":"2023-09-27T23:32:05.862603+0000","last_peered":"2023-09-27T23:32:05.862603+0000","last_clean":"2023-09-27T23:32:05.862603+0000","last_became_active":"2023-09-27T23:32:05.859426+0000","last_became_peered":"2023-09-27T23:32:05.859426+0000","last_unstale":"2023-09-27T23:32:05.862603+0000","last_undegraded":"2023-09-27T23:32:05.862603+0000","last_fullsized":"2023-09-27T23:32:05.862603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:39.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879745+0000","last_change":"2023-09-27T23:32:05.874339+0000","last_active":"2023-09-27T23:32:05.879745+0000","last_peered":"2023-09-27T23:32:05.879745+0000","last_clean":"2023-09-27T23:32:05.879745+0000","last_became_active":"2023-09-27T23:32:05.872716+0000","last_became_peered":"2023-09-27T23:32:05.872716+0000","last_unstale":"2023-09-27T23:32:05.879745+0000","last_undegraded":"2023-09-27T23:32:05.879745+0000","last_fullsized":"2023-09-27T23:32:05.879745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:21.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871701+0000","last_change":"2023-09-27T23:32:05.870769+0000","last_active":"2023-09-27T23:32:05.871701+0000","last_peered":"2023-09-27T23:32:05.871701+0000","last_clean":"2023-09-27T23:32:05.871701+0000","last_became_active":"2023-09-27T23:32:05.870469+0000","last_became_peered":"2023-09-27T23:32:05.870469+0000","last_unstale":"2023-09-27T23:32:05.871701+0000","last_undegraded":"2023-09-27T23:32:05.871701+0000","last_fullsized":"2023-09-27T23:32:05.871701+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:01:42.856864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867672+0000","last_change":"2023-09-27T23:32:05.865682+0000","last_active":"2023-09-27T23:32:05.867672+0000","last_peered":"2023-09-27T23:32:05.867672+0000","last_clean":"2023-09-27T23:32:05.867672+0000","last_became_active":"2023-09-27T23:32:05.865377+0000","last_became_peered":"2023-09-27T23:32:05.865377+0000","last_unstale":"2023-09-27T23:32:05.867672+0000","last_undegraded":"2023-09-27T23:32:05.867672+0000","last_fullsized":"2023-09-27T23:32:05.867672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868870+0000","last_change":"2023-09-27T23:32:05.860566+0000","last_active":"2023-09-27T23:32:05.868870+0000","last_peered":"2023-09-27T23:32:05.868870+0000","last_clean":"2023-09-27T23:32:05.868870+0000","last_became_active":"2023-09-27T23:32:05.860209+0000","last_became_peered":"2023-09-27T23:32:05.860209+0000","last_unstale":"2023-09-27T23:32:05.868870+0000","last_undegraded":"2023-09-27T23:32:05.868870+0000","last_fullsized":"2023-09-27T23:32:05.868870+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:42.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875841+0000","last_change":"2023-09-27T23:32:05.873642+0000","last_active":"2023-09-27T23:32:05.875841+0000","last_peered":"2023-09-27T23:32:05.875841+0000","last_clean":"2023-09-27T23:32:05.875841+0000","last_became_active":"2023-09-27T23:32:05.873228+0000","last_became_peered":"2023-09-27T23:32:05.873228+0000","last_unstale":"2023-09-27T23:32:05.875841+0000","last_undegraded":"2023-09-27T23:32:05.875841+0000","last_fullsized":"2023-09-27T23:32:05.875841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:26:39.116439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871647+0000","last_change":"2023-09-27T23:32:05.867910+0000","last_active":"2023-09-27T23:32:05.871647+0000","last_peered":"2023-09-27T23:32:05.871647+0000","last_clean":"2023-09-27T23:32:05.871647+0000","last_became_active":"2023-09-27T23:32:05.867511+0000","last_became_peered":"2023-09-27T23:32:05.867511+0000","last_unstale":"2023-09-27T23:32:05.871647+0000","last_undegraded":"2023-09-27T23:32:05.871647+0000","last_fullsized":"2023-09-27T23:32:05.871647+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:31:21.387831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865328+0000","last_change":"2023-09-27T23:32:05.863332+0000","last_active":"2023-09-27T23:32:05.865328+0000","last_peered":"2023-09-27T23:32:05.865328+0000","last_clean":"2023-09-27T23:32:05.865328+0000","last_became_active":"2023-09-27T23:32:05.863052+0000","last_became_peered":"2023-09-27T23:32:05.863052+0000","last_unstale":"2023-09-27T23:32:05.865328+0000","last_undegraded":"2023-09-27T23:32:05.865328+0000","last_fullsized":"2023-09-27T23:32:05.865328+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885048+0000","last_change":"2023-09-27T23:32:05.873577+0000","last_active":"2023-09-27T23:32:05.885048+0000","last_peered":"2023-09-27T23:32:05.885048+0000","last_clean":"2023-09-27T23:32:05.885048+0000","last_became_active":"2023-09-27T23:32:05.870063+0000","last_became_peered":"2023-09-27T23:32:05.870063+0000","last_unstale":"2023-09-27T23:32:05.885048+0000","last_undegraded":"2023-09-27T23:32:05.885048+0000","last_fullsized":"2023-09-27T23:32:05.885048+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:01:38.998159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873422+0000","last_change":"2023-09-27T23:32:05.871483+0000","last_active":"2023-09-27T23:32:05.873422+0000","last_peered":"2023-09-27T23:32:05.873422+0000","last_clean":"2023-09-27T23:32:05.873422+0000","last_became_active":"2023-09-27T23:32:05.871087+0000","last_became_peered":"2023-09-27T23:32:05.871087+0000","last_unstale":"2023-09-27T23:32:05.873422+0000","last_undegraded":"2023-09-27T23:32:05.873422+0000","last_fullsized":"2023-09-27T23:32:05.873422+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:16.072601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881812+0000","last_change":"2023-09-27T23:32:05.879569+0000","last_active":"2023-09-27T23:32:05.881812+0000","last_peered":"2023-09-27T23:32:05.881812+0000","last_clean":"2023-09-27T23:32:05.881812+0000","last_became_active":"2023-09-27T23:32:05.879296+0000","last_became_peered":"2023-09-27T23:32:05.879296+0000","last_unstale":"2023-09-27T23:32:05.881812+0000","last_undegraded":"2023-09-27T23:32:05.881812+0000","last_fullsized":"2023-09-27T23:32:05.881812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:42.688153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876663+0000","last_change":"2023-09-27T23:32:05.875209+0000","last_active":"2023-09-27T23:32:05.876663+0000","last_peered":"2023-09-27T23:32:05.876663+0000","last_clean":"2023-09-27T23:32:05.876663+0000","last_became_active":"2023-09-27T23:32:05.874886+0000","last_became_peered":"2023-09-27T23:32:05.874886+0000","last_unstale":"2023-09-27T23:32:05.876663+0000","last_undegraded":"2023-09-27T23:32:05.876663+0000","last_fullsized":"2023-09-27T23:32:05.876663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871939+0000","last_change":"2023-09-27T23:32:05.868005+0000","last_active":"2023-09-27T23:32:05.871939+0000","last_peered":"2023-09-27T23:32:05.871939+0000","last_clean":"2023-09-27T23:32:05.871939+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871939+0000","last_undegraded":"2023-09-27T23:32:05.871939+0000","last_fullsized":"2023-09-27T23:32:05.871939+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:25.597622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885078+0000","last_change":"2023-09-27T23:32:05.873691+0000","last_active":"2023-09-27T23:32:05.885078+0000","last_peered":"2023-09-27T23:32:05.885078+0000","last_clean":"2023-09-27T23:32:05.885078+0000","last_became_active":"2023-09-27T23:32:05.870676+0000","last_became_peered":"2023-09-27T23:32:05.870676+0000","last_unstale":"2023-09-27T23:32:05.885078+0000","last_undegraded":"2023-09-27T23:32:05.885078+0000","last_fullsized":"2023-09-27T23:32:05.885078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868346+0000","last_change":"2023-09-27T23:32:05.865108+0000","last_active":"2023-09-27T23:32:05.868346+0000","last_peered":"2023-09-27T23:32:05.868346+0000","last_clean":"2023-09-27T23:32:05.868346+0000","last_became_active":"2023-09-27T23:32:05.864839+0000","last_became_peered":"2023-09-27T23:32:05.864839+0000","last_unstale":"2023-09-27T23:32:05.868346+0000","last_undegraded":"2023-09-27T23:32:05.868346+0000","last_fullsized":"2023-09-27T23:32:05.868346+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:24:55.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865011+0000","last_change":"2023-09-27T23:32:05.859858+0000","last_active":"2023-09-27T23:32:05.865011+0000","last_peered":"2023-09-27T23:32:05.865011+0000","last_clean":"2023-09-27T23:32:05.865011+0000","last_became_active":"2023-09-27T23:32:05.859569+0000","last_became_peered":"2023-09-27T23:32:05.859569+0000","last_unstale":"2023-09-27T23:32:05.865011+0000","last_undegraded":"2023-09-27T23:32:05.865011+0000","last_fullsized":"2023-09-27T23:32:05.865011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:40.168536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"128'4","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877659+0000","last_change":"2023-09-27T23:32:05.473760+0000","last_active":"2023-09-27T23:32:05.877659+0000","last_peered":"2023-09-27T23:32:05.877659+0000","last_clean":"2023-09-27T23:32:05.877659+0000","last_became_active":"2023-09-27T23:32:05.473462+0000","last_became_peered":"2023-09-27T23:32:05.473462+0000","last_unstale":"2023-09-27T23:32:05.877659+0000","last_undegraded":"2023-09-27T23:32:05.877659+0000","last_fullsized":"2023-09-27T23:32:05.877659+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:52.366185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871871+0000","last_change":"2023-09-27T23:32:05.867437+0000","last_active":"2023-09-27T23:32:05.871871+0000","last_peered":"2023-09-27T23:32:05.871871+0000","last_clean":"2023-09-27T23:32:05.871871+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871871+0000","last_undegraded":"2023-09-27T23:32:05.871871+0000","last_fullsized":"2023-09-27T23:32:05.871871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:22.332265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877163+0000","last_change":"2023-09-27T23:32:05.474605+0000","last_active":"2023-09-27T23:32:05.877163+0000","last_peered":"2023-09-27T23:32:05.877163+0000","last_clean":"2023-09-27T23:32:05.877163+0000","last_became_active":"2023-09-27T23:32:05.474269+0000","last_became_peered":"2023-09-27T23:32:05.474269+0000","last_unstale":"2023-09-27T23:32:05.877163+0000","last_undegraded":"2023-09-27T23:32:05.877163+0000","last_fullsized":"2023-09-27T23:32:05.877163+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864812+0000","last_change":"2023-09-27T23:32:05.858996+0000","last_active":"2023-09-27T23:32:05.864812+0000","last_peered":"2023-09-27T23:32:05.864812+0000","last_clean":"2023-09-27T23:32:05.864812+0000","last_became_active":"2023-09-27T23:32:05.858668+0000","last_became_peered":"2023-09-27T23:32:05.858668+0000","last_unstale":"2023-09-27T23:32:05.864812+0000","last_undegraded":"2023-09-27T23:32:05.864812+0000","last_fullsized":"2023-09-27T23:32:05.864812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:35:43.781059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862048+0000","last_change":"2023-09-27T23:32:05.477638+0000","last_active":"2023-09-27T23:32:05.862048+0000","last_peered":"2023-09-27T23:32:05.862048+0000","last_clean":"2023-09-27T23:32:05.862048+0000","last_became_active":"2023-09-27T23:32:05.477296+0000","last_became_peered":"2023-09-27T23:32:05.477296+0000","last_unstale":"2023-09-27T23:32:05.862048+0000","last_undegraded":"2023-09-27T23:32:05.862048+0000","last_fullsized":"2023-09-27T23:32:05.862048+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:24.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881733+0000","last_change":"2023-09-27T23:32:05.879475+0000","last_active":"2023-09-27T23:32:05.881733+0000","last_peered":"2023-09-27T23:32:05.881733+0000","last_clean":"2023-09-27T23:32:05.881733+0000","last_became_active":"2023-09-27T23:32:05.877744+0000","last_became_peered":"2023-09-27T23:32:05.877744+0000","last_unstale":"2023-09-27T23:32:05.881733+0000","last_undegraded":"2023-09-27T23:32:05.881733+0000","last_fullsized":"2023-09-27T23:32:05.881733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:18.120648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862399+0000","last_change":"2023-09-27T23:32:05.478152+0000","last_active":"2023-09-27T23:32:05.862399+0000","last_peered":"2023-09-27T23:32:05.862399+0000","last_clean":"2023-09-27T23:32:05.862399+0000","last_became_active":"2023-09-27T23:32:05.477854+0000","last_became_peered":"2023-09-27T23:32:05.477854+0000","last_unstale":"2023-09-27T23:32:05.862399+0000","last_undegraded":"2023-09-27T23:32:05.862399+0000","last_fullsized":"2023-09-27T23:32:05.862399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.884996+0000","last_change":"2023-09-27T23:32:05.870784+0000","last_active":"2023-09-27T23:32:05.884996+0000","last_peered":"2023-09-27T23:32:05.884996+0000","last_clean":"2023-09-27T23:32:05.884996+0000","last_became_active":"2023-09-27T23:32:05.870489+0000","last_became_peered":"2023-09-27T23:32:05.870489+0000","last_unstale":"2023-09-27T23:32:05.884996+0000","last_undegraded":"2023-09-27T23:32:05.884996+0000","last_fullsized":"2023-09-27T23:32:05.884996+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864176+0000","last_change":"2023-09-27T23:32:05.484779+0000","last_active":"2023-09-27T23:32:05.864176+0000","last_peered":"2023-09-27T23:32:05.864176+0000","last_clean":"2023-09-27T23:32:05.864176+0000","last_became_active":"2023-09-27T23:32:05.484492+0000","last_became_peered":"2023-09-27T23:32:05.484492+0000","last_unstale":"2023-09-27T23:32:05.864176+0000","last_undegraded":"2023-09-27T23:32:05.864176+0000","last_fullsized":"2023-09-27T23:32:05.864176+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:32:29.817112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865090+0000","last_change":"2023-09-27T23:32:05.861730+0000","last_active":"2023-09-27T23:32:05.865090+0000","last_peered":"2023-09-27T23:32:05.865090+0000","last_clean":"2023-09-27T23:32:05.865090+0000","last_became_active":"2023-09-27T23:32:05.861021+0000","last_became_peered":"2023-09-27T23:32:05.861021+0000","last_unstale":"2023-09-27T23:32:05.865090+0000","last_undegraded":"2023-09-27T23:32:05.865090+0000","last_fullsized":"2023-09-27T23:32:05.865090+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862148+0000","last_change":"2023-09-27T23:32:05.474408+0000","last_active":"2023-09-27T23:32:05.862148+0000","last_peered":"2023-09-27T23:32:05.862148+0000","last_clean":"2023-09-27T23:32:05.862148+0000","last_became_active":"2023-09-27T23:32:05.473091+0000","last_became_peered":"2023-09-27T23:32:05.473091+0000","last_unstale":"2023-09-27T23:32:05.862148+0000","last_undegraded":"2023-09-27T23:32:05.862148+0000","last_fullsized":"2023-09-27T23:32:05.862148+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:20.064087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885105+0000","last_change":"2023-09-27T23:32:05.870585+0000","last_active":"2023-09-27T23:32:05.885105+0000","last_peered":"2023-09-27T23:32:05.885105+0000","last_clean":"2023-09-27T23:32:05.885105+0000","last_became_active":"2023-09-27T23:32:05.870282+0000","last_became_peered":"2023-09-27T23:32:05.870282+0000","last_unstale":"2023-09-27T23:32:05.885105+0000","last_undegraded":"2023-09-27T23:32:05.885105+0000","last_fullsized":"2023-09-27T23:32:05.885105+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855524+0000","last_change":"2023-09-27T23:32:05.486286+0000","last_active":"2023-09-27T23:32:05.855524+0000","last_peered":"2023-09-27T23:32:05.855524+0000","last_clean":"2023-09-27T23:32:05.855524+0000","last_became_active":"2023-09-27T23:32:05.485968+0000","last_became_peered":"2023-09-27T23:32:05.485968+0000","last_unstale":"2023-09-27T23:32:05.855524+0000","last_undegraded":"2023-09-27T23:32:05.855524+0000","last_fullsized":"2023-09-27T23:32:05.855524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:20.522455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.870329+0000","last_change":"2023-09-27T23:32:05.865877+0000","last_active":"2023-09-27T23:32:05.870329+0000","last_peered":"2023-09-27T23:32:05.870329+0000","last_clean":"2023-09-27T23:32:05.870329+0000","last_became_active":"2023-09-27T23:32:05.865294+0000","last_became_peered":"2023-09-27T23:32:05.865294+0000","last_unstale":"2023-09-27T23:32:05.870329+0000","last_undegraded":"2023-09-27T23:32:05.870329+0000","last_fullsized":"2023-09-27T23:32:05.870329+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:54:00.479947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.851579+0000","last_change":"2023-09-27T23:32:05.484134+0000","last_active":"2023-09-27T23:32:05.851579+0000","last_peered":"2023-09-27T23:32:05.851579+0000","last_clean":"2023-09-27T23:32:05.851579+0000","last_became_active":"2023-09-27T23:32:05.483856+0000","last_became_peered":"2023-09-27T23:32:05.483856+0000","last_unstale":"2023-09-27T23:32:05.851579+0000","last_undegraded":"2023-09-27T23:32:05.851579+0000","last_fullsized":"2023-09-27T23:32:05.851579+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:40.992540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.883531+0000","last_change":"2023-09-27T23:32:05.882768+0000","last_active":"2023-09-27T23:32:05.883531+0000","last_peered":"2023-09-27T23:32:05.883531+0000","last_clean":"2023-09-27T23:32:05.883531+0000","last_became_active":"2023-09-27T23:32:05.882322+0000","last_became_peered":"2023-09-27T23:32:05.882322+0000","last_unstale":"2023-09-27T23:32:05.883531+0000","last_undegraded":"2023-09-27T23:32:05.883531+0000","last_fullsized":"2023-09-27T23:32:05.883531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:46.976392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881529+0000","last_change":"2023-09-27T23:32:05.486576+0000","last_active":"2023-09-27T23:32:05.881529+0000","last_peered":"2023-09-27T23:32:05.881529+0000","last_clean":"2023-09-27T23:32:05.881529+0000","last_became_active":"2023-09-27T23:32:05.484863+0000","last_became_peered":"2023-09-27T23:32:05.484863+0000","last_unstale":"2023-09-27T23:32:05.881529+0000","last_undegraded":"2023-09-27T23:32:05.881529+0000","last_fullsized":"2023-09-27T23:32:05.881529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:20.574016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877676+0000","last_change":"2023-09-27T23:32:05.875211+0000","last_active":"2023-09-27T23:32:05.877676+0000","last_peered":"2023-09-27T23:32:05.877676+0000","last_clean":"2023-09-27T23:32:05.877676+0000","last_became_active":"2023-09-27T23:32:05.874899+0000","last_became_peered":"2023-09-27T23:32:05.874899+0000","last_unstale":"2023-09-27T23:32:05.877676+0000","last_undegraded":"2023-09-27T23:32:05.877676+0000","last_fullsized":"2023-09-27T23:32:05.877676+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:38.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.858932+0000","last_change":"2023-09-27T23:32:05.474634+0000","last_active":"2023-09-27T23:32:05.858932+0000","last_peered":"2023-09-27T23:32:05.858932+0000","last_clean":"2023-09-27T23:32:05.858932+0000","last_became_active":"2023-09-27T23:32:05.474314+0000","last_became_peered":"2023-09-27T23:32:05.474314+0000","last_unstale":"2023-09-27T23:32:05.858932+0000","last_undegraded":"2023-09-27T23:32:05.858932+0000","last_fullsized":"2023-09-27T23:32:05.858932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:05.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885276+0000","last_change":"2023-09-27T23:32:05.884516+0000","last_active":"2023-09-27T23:32:05.885276+0000","last_peered":"2023-09-27T23:32:05.885276+0000","last_clean":"2023-09-27T23:32:05.885276+0000","last_became_active":"2023-09-27T23:32:05.884276+0000","last_became_peered":"2023-09-27T23:32:05.884276+0000","last_unstale":"2023-09-27T23:32:05.885276+0000","last_undegraded":"2023-09-27T23:32:05.885276+0000","last_fullsized":"2023-09-27T23:32:05.885276+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:14.206863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871586+0000","last_change":"2023-09-27T23:32:05.483320+0000","last_active":"2023-09-27T23:32:05.871586+0000","last_peered":"2023-09-27T23:32:05.871586+0000","last_clean":"2023-09-27T23:32:05.871586+0000","last_became_active":"2023-09-27T23:32:05.482969+0000","last_became_peered":"2023-09-27T23:32:05.482969+0000","last_unstale":"2023-09-27T23:32:05.871586+0000","last_undegraded":"2023-09-27T23:32:05.871586+0000","last_fullsized":"2023-09-27T23:32:05.871586+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:56.542035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881069+0000","last_change":"2023-09-27T23:32:05.877251+0000","last_active":"2023-09-27T23:32:05.881069+0000","last_peered":"2023-09-27T23:32:05.881069+0000","last_clean":"2023-09-27T23:32:05.881069+0000","last_became_active":"2023-09-27T23:32:05.876868+0000","last_became_peered":"2023-09-27T23:32:05.876868+0000","last_unstale":"2023-09-27T23:32:05.881069+0000","last_undegraded":"2023-09-27T23:32:05.881069+0000","last_fullsized":"2023-09-27T23:32:05.881069+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:10:56.379380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.854712+0000","last_change":"2023-09-27T23:32:05.487592+0000","last_active":"2023-09-27T23:32:05.854712+0000","last_peered":"2023-09-27T23:32:05.854712+0000","last_clean":"2023-09-27T23:32:05.854712+0000","last_became_active":"2023-09-27T23:32:05.487244+0000","last_became_peered":"2023-09-27T23:32:05.487244+0000","last_unstale":"2023-09-27T23:32:05.854712+0000","last_undegraded":"2023-09-27T23:32:05.854712+0000","last_fullsized":"2023-09-27T23:32:05.854712+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:07:33.532568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879641+0000","last_change":"2023-09-27T23:32:05.873902+0000","last_active":"2023-09-27T23:32:05.879641+0000","last_peered":"2023-09-27T23:32:05.879641+0000","last_clean":"2023-09-27T23:32:05.879641+0000","last_became_active":"2023-09-27T23:32:05.872995+0000","last_became_peered":"2023-09-27T23:32:05.872995+0000","last_unstale":"2023-09-27T23:32:05.879641+0000","last_undegraded":"2023-09-27T23:32:05.879641+0000","last_fullsized":"2023-09-27T23:32:05.879641+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:35.151434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864787+0000","last_change":"2023-09-27T23:32:05.477932+0000","last_active":"2023-09-27T23:32:05.864787+0000","last_peered":"2023-09-27T23:32:05.864787+0000","last_clean":"2023-09-27T23:32:05.864787+0000","last_became_active":"2023-09-27T23:32:05.477556+0000","last_became_peered":"2023-09-27T23:32:05.477556+0000","last_unstale":"2023-09-27T23:32:05.864787+0000","last_undegraded":"2023-09-27T23:32:05.864787+0000","last_fullsized":"2023-09-27T23:32:05.864787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:23:33.015795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885492+0000","last_change":"2023-09-27T23:32:05.878808+0000","last_active":"2023-09-27T23:32:05.885492+0000","last_peered":"2023-09-27T23:32:05.885492+0000","last_clean":"2023-09-27T23:32:05.885492+0000","last_became_active":"2023-09-27T23:32:05.878462+0000","last_became_peered":"2023-09-27T23:32:05.878462+0000","last_unstale":"2023-09-27T23:32:05.885492+0000","last_undegraded":"2023-09-27T23:32:05.885492+0000","last_fullsized":"2023-09-27T23:32:05.885492+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847377+0000","last_change":"2023-09-27T23:32:05.473937+0000","last_active":"2023-09-27T23:32:05.847377+0000","last_peered":"2023-09-27T23:32:05.847377+0000","last_clean":"2023-09-27T23:32:05.847377+0000","last_became_active":"2023-09-27T23:32:05.473535+0000","last_became_peered":"2023-09-27T23:32:05.473535+0000","last_unstale":"2023-09-27T23:32:05.847377+0000","last_undegraded":"2023-09-27T23:32:05.847377+0000","last_fullsized":"2023-09-27T23:32:05.847377+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:59.468491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.880779+0000","last_change":"2023-09-27T23:32:05.877232+0000","last_active":"2023-09-27T23:32:05.880779+0000","last_peered":"2023-09-27T23:32:05.880779+0000","last_clean":"2023-09-27T23:32:05.880779+0000","last_became_active":"2023-09-27T23:32:05.876906+0000","last_became_peered":"2023-09-27T23:32:05.876906+0000","last_unstale":"2023-09-27T23:32:05.880779+0000","last_undegraded":"2023-09-27T23:32:05.880779+0000","last_fullsized":"2023-09-27T23:32:05.880779+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:58.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848215+0000","last_change":"2023-09-27T23:32:05.478424+0000","last_active":"2023-09-27T23:32:05.848215+0000","last_peered":"2023-09-27T23:32:05.848215+0000","last_clean":"2023-09-27T23:32:05.848215+0000","last_became_active":"2023-09-27T23:32:05.478120+0000","last_became_peered":"2023-09-27T23:32:05.478120+0000","last_unstale":"2023-09-27T23:32:05.848215+0000","last_undegraded":"2023-09-27T23:32:05.848215+0000","last_fullsized":"2023-09-27T23:32:05.848215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:15.570089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877156+0000","last_change":"2023-09-27T23:32:05.875429+0000","last_active":"2023-09-27T23:32:05.877156+0000","last_peered":"2023-09-27T23:32:05.877156+0000","last_clean":"2023-09-27T23:32:05.877156+0000","last_became_active":"2023-09-27T23:32:05.875127+0000","last_became_peered":"2023-09-27T23:32:05.875127+0000","last_unstale":"2023-09-27T23:32:05.877156+0000","last_undegraded":"2023-09-27T23:32:05.877156+0000","last_fullsized":"2023-09-27T23:32:05.877156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:21.585472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.851675+0000","last_change":"2023-09-27T23:32:05.481075+0000","last_active":"2023-09-27T23:32:05.851675+0000","last_peered":"2023-09-27T23:32:05.851675+0000","last_clean":"2023-09-27T23:32:05.851675+0000","last_became_active":"2023-09-27T23:32:05.480382+0000","last_became_peered":"2023-09-27T23:32:05.480382+0000","last_unstale":"2023-09-27T23:32:05.851675+0000","last_undegraded":"2023-09-27T23:32:05.851675+0000","last_fullsized":"2023-09-27T23:32:05.851675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:51.337572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875964+0000","last_change":"2023-09-27T23:32:05.873357+0000","last_active":"2023-09-27T23:32:05.875964+0000","last_peered":"2023-09-27T23:32:05.875964+0000","last_clean":"2023-09-27T23:32:05.875964+0000","last_became_active":"2023-09-27T23:32:05.873037+0000","last_became_peered":"2023-09-27T23:32:05.873037+0000","last_unstale":"2023-09-27T23:32:05.875964+0000","last_undegraded":"2023-09-27T23:32:05.875964+0000","last_fullsized":"2023-09-27T23:32:05.875964+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:54:10.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882131+0000","last_change":"2023-09-27T23:32:05.487770+0000","last_active":"2023-09-27T23:32:05.882131+0000","last_peered":"2023-09-27T23:32:05.882131+0000","last_clean":"2023-09-27T23:32:05.882131+0000","last_became_active":"2023-09-27T23:32:05.487423+0000","last_became_peered":"2023-09-27T23:32:05.487423+0000","last_unstale":"2023-09-27T23:32:05.882131+0000","last_undegraded":"2023-09-27T23:32:05.882131+0000","last_fullsized":"2023-09-27T23:32:05.882131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:34.139978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885219+0000","last_change":"2023-09-27T23:32:05.874286+0000","last_active":"2023-09-27T23:32:05.885219+0000","last_peered":"2023-09-27T23:32:05.885219+0000","last_clean":"2023-09-27T23:32:05.885219+0000","last_became_active":"2023-09-27T23:32:05.870718+0000","last_became_peered":"2023-09-27T23:32:05.870718+0000","last_unstale":"2023-09-27T23:32:05.885219+0000","last_undegraded":"2023-09-27T23:32:05.885219+0000","last_fullsized":"2023-09-27T23:32:05.885219+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:06.303516+0000","last_change":"2023-09-27T23:32:05.473896+0000","last_active":"2023-09-27T23:32:06.303516+0000","last_peered":"2023-09-27T23:32:06.303516+0000","last_clean":"2023-09-27T23:32:06.303516+0000","last_became_active":"2023-09-27T23:32:05.473665+0000","last_became_peered":"2023-09-27T23:32:05.473665+0000","last_unstale":"2023-09-27T23:32:06.303516+0000","last_undegraded":"2023-09-27T23:32:06.303516+0000","last_fullsized":"2023-09-27T23:32:06.303516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:17.109191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885451+0000","last_change":"2023-09-27T23:32:05.878658+0000","last_active":"2023-09-27T23:32:05.885451+0000","last_peered":"2023-09-27T23:32:05.885451+0000","last_clean":"2023-09-27T23:32:05.885451+0000","last_became_active":"2023-09-27T23:32:05.877911+0000","last_became_peered":"2023-09-27T23:32:05.877911+0000","last_unstale":"2023-09-27T23:32:05.885451+0000","last_undegraded":"2023-09-27T23:32:05.885451+0000","last_fullsized":"2023-09-27T23:32:05.885451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:39.054756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855548+0000","last_change":"2023-09-27T23:32:05.480279+0000","last_active":"2023-09-27T23:32:05.855548+0000","last_peered":"2023-09-27T23:32:05.855548+0000","last_clean":"2023-09-27T23:32:05.855548+0000","last_became_active":"2023-09-27T23:32:05.479931+0000","last_became_peered":"2023-09-27T23:32:05.479931+0000","last_unstale":"2023-09-27T23:32:05.855548+0000","last_undegraded":"2023-09-27T23:32:05.855548+0000","last_fullsized":"2023-09-27T23:32:05.855548+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:07.847247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876032+0000","last_change":"2023-09-27T23:32:05.874148+0000","last_active":"2023-09-27T23:32:05.876032+0000","last_peered":"2023-09-27T23:32:05.876032+0000","last_clean":"2023-09-27T23:32:05.876032+0000","last_became_active":"2023-09-27T23:32:05.873853+0000","last_became_peered":"2023-09-27T23:32:05.873853+0000","last_unstale":"2023-09-27T23:32:05.876032+0000","last_undegraded":"2023-09-27T23:32:05.876032+0000","last_fullsized":"2023-09-27T23:32:05.876032+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:16:14.012071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846161+0000","last_change":"2023-09-27T23:32:05.482613+0000","last_active":"2023-09-27T23:32:05.846161+0000","last_peered":"2023-09-27T23:32:05.846161+0000","last_clean":"2023-09-27T23:32:05.846161+0000","last_became_active":"2023-09-27T23:32:05.482252+0000","last_became_peered":"2023-09-27T23:32:05.482252+0000","last_unstale":"2023-09-27T23:32:05.846161+0000","last_undegraded":"2023-09-27T23:32:05.846161+0000","last_fullsized":"2023-09-27T23:32:05.846161+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:05.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869011+0000","last_change":"2023-09-27T23:32:05.866830+0000","last_active":"2023-09-27T23:32:05.869011+0000","last_peered":"2023-09-27T23:32:05.869011+0000","last_clean":"2023-09-27T23:32:05.869011+0000","last_became_active":"2023-09-27T23:32:05.866498+0000","last_became_peered":"2023-09-27T23:32:05.866498+0000","last_unstale":"2023-09-27T23:32:05.869011+0000","last_undegraded":"2023-09-27T23:32:05.869011+0000","last_fullsized":"2023-09-27T23:32:05.869011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855259+0000","last_change":"2023-09-27T23:32:05.480057+0000","last_active":"2023-09-27T23:32:05.855259+0000","last_peered":"2023-09-27T23:32:05.855259+0000","last_clean":"2023-09-27T23:32:05.855259+0000","last_became_active":"2023-09-27T23:32:05.479684+0000","last_became_peered":"2023-09-27T23:32:05.479684+0000","last_unstale":"2023-09-27T23:32:05.855259+0000","last_undegraded":"2023-09-27T23:32:05.855259+0000","last_fullsized":"2023-09-27T23:32:05.855259+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:04:23.100428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865072+0000","last_change":"2023-09-27T23:32:05.860941+0000","last_active":"2023-09-27T23:32:05.865072+0000","last_peered":"2023-09-27T23:32:05.865072+0000","last_clean":"2023-09-27T23:32:05.865072+0000","last_became_active":"2023-09-27T23:32:05.860543+0000","last_became_peered":"2023-09-27T23:32:05.860543+0000","last_unstale":"2023-09-27T23:32:05.865072+0000","last_undegraded":"2023-09-27T23:32:05.865072+0000","last_fullsized":"2023-09-27T23:32:05.865072+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.852958+0000","last_change":"2023-09-27T23:32:05.479760+0000","last_active":"2023-09-27T23:32:05.852958+0000","last_peered":"2023-09-27T23:32:05.852958+0000","last_clean":"2023-09-27T23:32:05.852958+0000","last_became_active":"2023-09-27T23:32:05.479474+0000","last_became_peered":"2023-09-27T23:32:05.479474+0000","last_unstale":"2023-09-27T23:32:05.852958+0000","last_undegraded":"2023-09-27T23:32:05.852958+0000","last_fullsized":"2023-09-27T23:32:05.852958+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:43.445603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881743+0000","last_change":"2023-09-27T23:32:05.879523+0000","last_active":"2023-09-27T23:32:05.881743+0000","last_peered":"2023-09-27T23:32:05.881743+0000","last_clean":"2023-09-27T23:32:05.881743+0000","last_became_active":"2023-09-27T23:32:05.878009+0000","last_became_peered":"2023-09-27T23:32:05.878009+0000","last_unstale":"2023-09-27T23:32:05.881743+0000","last_undegraded":"2023-09-27T23:32:05.881743+0000","last_fullsized":"2023-09-27T23:32:05.881743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:21:40.330475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847190+0000","last_change":"2023-09-27T23:32:05.473857+0000","last_active":"2023-09-27T23:32:05.847190+0000","last_peered":"2023-09-27T23:32:05.847190+0000","last_clean":"2023-09-27T23:32:05.847190+0000","last_became_active":"2023-09-27T23:32:05.473507+0000","last_became_peered":"2023-09-27T23:32:05.473507+0000","last_unstale":"2023-09-27T23:32:05.847190+0000","last_undegraded":"2023-09-27T23:32:05.847190+0000","last_fullsized":"2023-09-27T23:32:05.847190+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:19.244182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885101+0000","last_change":"2023-09-27T23:32:05.873758+0000","last_active":"2023-09-27T23:32:05.885101+0000","last_peered":"2023-09-27T23:32:05.885101+0000","last_clean":"2023-09-27T23:32:05.885101+0000","last_became_active":"2023-09-27T23:32:05.870310+0000","last_became_peered":"2023-09-27T23:32:05.870310+0000","last_unstale":"2023-09-27T23:32:05.885101+0000","last_undegraded":"2023-09-27T23:32:05.885101+0000","last_fullsized":"2023-09-27T23:32:05.885101+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:35.596502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.851773+0000","last_change":"2023-09-27T23:32:05.482571+0000","last_active":"2023-09-27T23:32:05.851773+0000","last_peered":"2023-09-27T23:32:05.851773+0000","last_clean":"2023-09-27T23:32:05.851773+0000","last_became_active":"2023-09-27T23:32:05.480775+0000","last_became_peered":"2023-09-27T23:32:05.480775+0000","last_unstale":"2023-09-27T23:32:05.851773+0000","last_undegraded":"2023-09-27T23:32:05.851773+0000","last_fullsized":"2023-09-27T23:32:05.851773+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:31.045851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869149+0000","last_change":"2023-09-27T23:32:05.861010+0000","last_active":"2023-09-27T23:32:05.869149+0000","last_peered":"2023-09-27T23:32:05.869149+0000","last_clean":"2023-09-27T23:32:05.869149+0000","last_became_active":"2023-09-27T23:32:05.860737+0000","last_became_peered":"2023-09-27T23:32:05.860737+0000","last_unstale":"2023-09-27T23:32:05.869149+0000","last_undegraded":"2023-09-27T23:32:05.869149+0000","last_fullsized":"2023-09-27T23:32:05.869149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:58:21.738937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.856839+0000","last_change":"2023-09-27T23:32:05.487870+0000","last_active":"2023-09-27T23:32:05.856839+0000","last_peered":"2023-09-27T23:32:05.856839+0000","last_clean":"2023-09-27T23:32:05.856839+0000","last_became_active":"2023-09-27T23:32:05.487469+0000","last_became_peered":"2023-09-27T23:32:05.487469+0000","last_unstale":"2023-09-27T23:32:05.856839+0000","last_undegraded":"2023-09-27T23:32:05.856839+0000","last_fullsized":"2023-09-27T23:32:05.856839+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:25:27.956056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"117'227","reported_seq":650,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864897+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:32:05.864897+0000","last_peered":"2023-09-27T23:32:05.864897+0000","last_clean":"2023-09-27T23:32:05.864897+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:32:05.864897+0000","last_undegraded":"2023-09-27T23:32:05.864897+0000","last_fullsized":"2023-09-27T23:32:05.864897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":227,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":114,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855351+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:32:05.855351+0000","last_peered":"2023-09-27T23:32:05.855351+0000","last_clean":"2023-09-27T23:32:05.855351+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:32:05.855351+0000","last_undegraded":"2023-09-27T23:32:05.855351+0000","last_fullsized":"2023-09-27T23:32:05.855351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869078+0000","last_change":"2023-09-27T23:32:05.862823+0000","last_active":"2023-09-27T23:32:05.869078+0000","last_peered":"2023-09-27T23:32:05.869078+0000","last_clean":"2023-09-27T23:32:05.869078+0000","last_became_active":"2023-09-27T23:32:05.862677+0000","last_became_peered":"2023-09-27T23:32:05.862677+0000","last_unstale":"2023-09-27T23:32:05.869078+0000","last_undegraded":"2023-09-27T23:32:05.869078+0000","last_fullsized":"2023-09-27T23:32:05.869078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:01.586755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846271+0000","last_change":"2023-09-27T23:32:05.487459+0000","last_active":"2023-09-27T23:32:05.846271+0000","last_peered":"2023-09-27T23:32:05.846271+0000","last_clean":"2023-09-27T23:32:05.846271+0000","last_became_active":"2023-09-27T23:32:05.487130+0000","last_became_peered":"2023-09-27T23:32:05.487130+0000","last_unstale":"2023-09-27T23:32:05.846271+0000","last_undegraded":"2023-09-27T23:32:05.846271+0000","last_fullsized":"2023-09-27T23:32:05.846271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:57.985464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864781+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:32:05.864781+0000","last_peered":"2023-09-27T23:32:05.864781+0000","last_clean":"2023-09-27T23:32:05.864781+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:32:05.864781+0000","last_undegraded":"2023-09-27T23:32:05.864781+0000","last_fullsized":"2023-09-27T23:32:05.864781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871725+0000","last_change":"2023-09-27T23:32:05.864506+0000","last_active":"2023-09-27T23:32:05.871725+0000","last_peered":"2023-09-27T23:32:05.871725+0000","last_clean":"2023-09-27T23:32:05.871725+0000","last_became_active":"2023-09-27T23:32:05.864207+0000","last_became_peered":"2023-09-27T23:32:05.864207+0000","last_unstale":"2023-09-27T23:32:05.871725+0000","last_undegraded":"2023-09-27T23:32:05.871725+0000","last_fullsized":"2023-09-27T23:32:05.871725+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:53:46.038154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875524+0000","last_change":"2023-09-27T23:32:05.478414+0000","last_active":"2023-09-27T23:32:05.875524+0000","last_peered":"2023-09-27T23:32:05.875524+0000","last_clean":"2023-09-27T23:32:05.875524+0000","last_became_active":"2023-09-27T23:32:05.478088+0000","last_became_peered":"2023-09-27T23:32:05.478088+0000","last_unstale":"2023-09-27T23:32:05.875524+0000","last_undegraded":"2023-09-27T23:32:05.875524+0000","last_fullsized":"2023-09-27T23:32:05.875524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:21.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847252+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:32:05.847252+0000","last_peered":"2023-09-27T23:32:05.847252+0000","last_clean":"2023-09-27T23:32:05.847252+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:32:05.847252+0000","last_undegraded":"2023-09-27T23:32:05.847252+0000","last_fullsized":"2023-09-27T23:32:05.847252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871490+0000","last_change":"2023-09-27T23:32:05.864496+0000","last_active":"2023-09-27T23:32:05.871490+0000","last_peered":"2023-09-27T23:32:05.871490+0000","last_clean":"2023-09-27T23:32:05.871490+0000","last_became_active":"2023-09-27T23:32:05.863828+0000","last_became_peered":"2023-09-27T23:32:05.863828+0000","last_unstale":"2023-09-27T23:32:05.871490+0000","last_undegraded":"2023-09-27T23:32:05.871490+0000","last_fullsized":"2023-09-27T23:32:05.871490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:46.777842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882372+0000","last_change":"2023-09-27T23:32:05.485399+0000","last_active":"2023-09-27T23:32:05.882372+0000","last_peered":"2023-09-27T23:32:05.882372+0000","last_clean":"2023-09-27T23:32:05.882372+0000","last_became_active":"2023-09-27T23:32:05.485073+0000","last_became_peered":"2023-09-27T23:32:05.485073+0000","last_unstale":"2023-09-27T23:32:05.882372+0000","last_undegraded":"2023-09-27T23:32:05.882372+0000","last_fullsized":"2023-09-27T23:32:05.882372+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:20.429282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":114,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.857942+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:32:05.857942+0000","last_peered":"2023-09-27T23:32:05.857942+0000","last_clean":"2023-09-27T23:32:05.857942+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:32:05.857942+0000","last_undegraded":"2023-09-27T23:32:05.857942+0000","last_fullsized":"2023-09-27T23:32:05.857942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876013+0000","last_change":"2023-09-27T23:32:05.874103+0000","last_active":"2023-09-27T23:32:05.876013+0000","last_peered":"2023-09-27T23:32:05.876013+0000","last_clean":"2023-09-27T23:32:05.876013+0000","last_became_active":"2023-09-27T23:32:05.873806+0000","last_became_peered":"2023-09-27T23:32:05.873806+0000","last_unstale":"2023-09-27T23:32:05.876013+0000","last_undegraded":"2023-09-27T23:32:05.876013+0000","last_fullsized":"2023-09-27T23:32:05.876013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:06:14.340339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.845266+0000","last_change":"2023-09-27T23:32:05.474807+0000","last_active":"2023-09-27T23:32:05.845266+0000","last_peered":"2023-09-27T23:32:05.845266+0000","last_clean":"2023-09-27T23:32:05.845266+0000","last_became_active":"2023-09-27T23:32:05.474510+0000","last_became_peered":"2023-09-27T23:32:05.474510+0000","last_unstale":"2023-09-27T23:32:05.845266+0000","last_undegraded":"2023-09-27T23:32:05.845266+0000","last_fullsized":"2023-09-27T23:32:05.845266+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:23.368573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.849356+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:32:05.849356+0000","last_peered":"2023-09-27T23:32:05.849356+0000","last_clean":"2023-09-27T23:32:05.849356+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:32:05.849356+0000","last_undegraded":"2023-09-27T23:32:05.849356+0000","last_fullsized":"2023-09-27T23:32:05.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.870438+0000","last_change":"2023-09-27T23:32:05.865477+0000","last_active":"2023-09-27T23:32:05.870438+0000","last_peered":"2023-09-27T23:32:05.870438+0000","last_clean":"2023-09-27T23:32:05.870438+0000","last_became_active":"2023-09-27T23:32:05.865162+0000","last_became_peered":"2023-09-27T23:32:05.865162+0000","last_unstale":"2023-09-27T23:32:05.870438+0000","last_undegraded":"2023-09-27T23:32:05.870438+0000","last_fullsized":"2023-09-27T23:32:05.870438+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:27.464013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847066+0000","last_change":"2023-09-27T23:32:05.477781+0000","last_active":"2023-09-27T23:32:05.847066+0000","last_peered":"2023-09-27T23:32:05.847066+0000","last_clean":"2023-09-27T23:32:05.847066+0000","last_became_active":"2023-09-27T23:32:05.477416+0000","last_became_peered":"2023-09-27T23:32:05.477416+0000","last_unstale":"2023-09-27T23:32:05.847066+0000","last_undegraded":"2023-09-27T23:32:05.847066+0000","last_fullsized":"2023-09-27T23:32:05.847066+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:35.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863310+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:32:05.863310+0000","last_peered":"2023-09-27T23:32:05.863310+0000","last_clean":"2023-09-27T23:32:05.863310+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:32:05.863310+0000","last_undegraded":"2023-09-27T23:32:05.863310+0000","last_fullsized":"2023-09-27T23:32:05.863310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.874160+0000","last_change":"2023-09-27T23:32:05.870982+0000","last_active":"2023-09-27T23:32:05.874160+0000","last_peered":"2023-09-27T23:32:05.874160+0000","last_clean":"2023-09-27T23:32:05.874160+0000","last_became_active":"2023-09-27T23:32:05.870670+0000","last_became_peered":"2023-09-27T23:32:05.870670+0000","last_unstale":"2023-09-27T23:32:05.874160+0000","last_undegraded":"2023-09-27T23:32:05.874160+0000","last_fullsized":"2023-09-27T23:32:05.874160+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:06.714208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"128'6","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885835+0000","last_change":"2023-09-27T23:32:05.486457+0000","last_active":"2023-09-27T23:32:05.885835+0000","last_peered":"2023-09-27T23:32:05.885835+0000","last_clean":"2023-09-27T23:32:05.885835+0000","last_became_active":"2023-09-27T23:32:05.485076+0000","last_became_peered":"2023-09-27T23:32:05.485076+0000","last_unstale":"2023-09-27T23:32:05.885835+0000","last_undegraded":"2023-09-27T23:32:05.885835+0000","last_fullsized":"2023-09-27T23:32:05.885835+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:41.676696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848976+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:32:05.848976+0000","last_peered":"2023-09-27T23:32:05.848976+0000","last_clean":"2023-09-27T23:32:05.848976+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:32:05.848976+0000","last_undegraded":"2023-09-27T23:32:05.848976+0000","last_fullsized":"2023-09-27T23:32:05.848976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869005+0000","last_change":"2023-09-27T23:32:05.862414+0000","last_active":"2023-09-27T23:32:05.869005+0000","last_peered":"2023-09-27T23:32:05.869005+0000","last_clean":"2023-09-27T23:32:05.869005+0000","last_became_active":"2023-09-27T23:32:05.862102+0000","last_became_peered":"2023-09-27T23:32:05.862102+0000","last_unstale":"2023-09-27T23:32:05.869005+0000","last_undegraded":"2023-09-27T23:32:05.869005+0000","last_fullsized":"2023-09-27T23:32:05.869005+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:40.054520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"128'3","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881576+0000","last_change":"2023-09-27T23:32:05.483339+0000","last_active":"2023-09-27T23:32:05.881576+0000","last_peered":"2023-09-27T23:32:05.881576+0000","last_clean":"2023-09-27T23:32:05.881576+0000","last_became_active":"2023-09-27T23:32:05.483058+0000","last_became_peered":"2023-09-27T23:32:05.483058+0000","last_unstale":"2023-09-27T23:32:05.881576+0000","last_undegraded":"2023-09-27T23:32:05.881576+0000","last_fullsized":"2023-09-27T23:32:05.881576+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:45.766386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847811+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:32:05.847811+0000","last_peered":"2023-09-27T23:32:05.847811+0000","last_clean":"2023-09-27T23:32:05.847811+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:32:05.847811+0000","last_undegraded":"2023-09-27T23:32:05.847811+0000","last_fullsized":"2023-09-27T23:32:05.847811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509663,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":25024,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":2191360,"data_stored":1415956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25624576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.52200000000000002,"15min":0.51100000000000001},"max":{"1min":1.2470000000000001,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1819999999999999,"5min":1.5489999999999999,"15min":1.712},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91900000000000004,"15min":0.89700000000000002},"min":{"1min":0.66000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1759999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95699999999999996,"15min":0.93899999999999995},"min":{"1min":0.69599999999999995,"5min":0.66400000000000003,"15min":0.28299999999999997},"max":{"1min":1.2430000000000001,"5min":1.337,"15min":1.7589999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.90200000000000002,"15min":0.91500000000000004},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.307},"max":{"1min":1.238,"5min":1.3129999999999999,"15min":2.681},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.97599999999999998,"15min":0.97899999999999998},"min":{"1min":0.77700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.405,"15min":1.718},"last":0.997}]},{"osd":3,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.93500000000000005},"min":{"1min":0.71099999999999997,"5min":0.63400000000000001,"15min":0.53200000000000003},"max":{"1min":1.151,"5min":1.204,"15min":1.421},"last":1.042},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.747,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.409,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.073}]},{"osd":4,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.117,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97499999999999998,"15min":0.98499999999999999},"min":{"1min":0.65200000000000002,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.3029999999999999,"5min":1.3240000000000001,"15min":2.8879999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.67000000000000004,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.3540000000000001,"5min":1.79,"15min":1.79},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.252,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.139},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.839,"15min":1.839},"last":1.246},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.107,"15min":1.1259999999999999},"min":{"1min":0.96599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":2.548},"last":1.218}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27100,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":25536,"kb_avail":94344740,"statfs":{"total":96636764160,"available":96609013760,"internally_reserved":0,"allocated":1593344,"data_stored":825070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26148864},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89200000000000002,"15min":0.94399999999999995},"min":{"1min":0.629,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.2130000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.78500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.246,"5min":1.321,"15min":1.5149999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.88},"min":{"1min":0.56299999999999994,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.214,"5min":1.248,"15min":1.323},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.258,"5min":1.403,"15min":1.403},"last":0.93899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.875},"min":{"1min":0.72099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2150000000000001,"5min":1.222,"15min":1.353},"last":1.238},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91100000000000003,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.246,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.629,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2170000000000001,"5min":1.51,"15min":1.734},"last":1.073},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.72499999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1619999999999999,"5min":1.278,"15min":1.4970000000000001},"last":1.4279999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1100000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.274},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2929999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.81499999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2669999999999999,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.343},{"interface":"front","average":{"1min":1.119,"5min":1.1319999999999999,"15min":1.1539999999999999},"min":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3169999999999999,"5min":1.4259999999999999,"15min":1.49},"last":1.121}]},{"osd":7,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.81399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3580000000000001,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1259999999999999,"15min":1.145},"min":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.6619999999999999},"last":1.4750000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509662,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35584,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1597440,"data_stored":825576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36438016},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.236,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.84299999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.526},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.874,"15min":0.88300000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2190000000000001,"5min":1.242,"15min":1.331},"last":0.79200000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.73999999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.474,"15min":1.474},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97099999999999997,"15min":0.95099999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.5740000000000001},"last":0.91600000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.97599999999999998,"15min":0.95599999999999996},"min":{"1min":0.70499999999999996,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":1.23,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.999,"15min":1.004},"min":{"1min":0.80800000000000005,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.323,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.56200000000000006,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.3220000000000001,"5min":1.341,"15min":1.341},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.038,"15min":1.0209999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.613,"15min":1.613},"last":1.0669999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0089999999999999,"15min":1.01},"min":{"1min":0.70499999999999996,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.5580000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.1020000000000001},"min":{"1min":0.876,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.117}]},{"osd":7,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1619999999999999,"15min":1.1379999999999999},"min":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.446,"5min":1.665,"15min":1.6679999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.198,"15min":1.1890000000000001},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.508,"5min":1.528,"15min":1.633},"last":1.1519999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509663,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23000,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94348840,"statfs":{"total":96636764160,"available":96613212160,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.876,"15min":0.89300000000000002},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.27,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.54400000000000004},"max":{"1min":1.212,"5min":1.343,"15min":1.45},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89500000000000002,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.262,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.73899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90200000000000002,"15min":0.91100000000000003},"min":{"1min":0.65600000000000003,"5min":0.57499999999999996,"15min":0.497},"max":{"1min":1.2410000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97999999999999998,"15min":0.96199999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.252,"5min":1.331,"15min":1.407},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92000000000000004},"min":{"1min":0.70999999999999996,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2150000000000001,"5min":1.272,"15min":1.2829999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.03,"15min":1.026},"min":{"1min":0.79900000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3660000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4610000000000001,"5min":1.609,"15min":1.609},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0229999999999999,"15min":1.01},"min":{"1min":0.66800000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0389999999999999,"15min":1.04},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3169999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.161,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.1499999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0940000000000001,"15min":1.091},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.427,"5min":1.4770000000000001,"15min":1.627},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.145,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.85899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.181}]}]},{"osd":3,"up_from":15,"seq":64424509662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":1605632,"data_stored":826606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.70199999999999996},"min":{"1min":0.40400000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.012,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.109,"5min":1.323,"15min":1.3440000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.57499999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.61499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.753,"5min":0.505,"15min":0.505},"max":{"1min":1.298,"5min":1.605,"15min":1.605},"last":1.163}]},{"osd":5,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0820000000000001,"15min":1.111},"min":{"1min":0.85099999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3999999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.232},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1160000000000001,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.714,"15min":1.764},"last":1.431}]},{"osd":6,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.105,"15min":1.1160000000000001},"min":{"1min":0.81200000000000006,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1950000000000001,"15min":1.2150000000000001},"min":{"1min":0.94899999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.488,"5min":1.6599999999999999,"15min":1.712},"last":1.3200000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.2370000000000001},"min":{"1min":0.79800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.173,"15min":1.202},"min":{"1min":0.88,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.143}]}]},{"osd":2,"up_from":15,"seq":64424509663,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22168,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94349672,"statfs":{"total":96636764160,"available":96614064128,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.903},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.097,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":0.98499999999999999,"5min":1.228,"15min":1.391},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":1.2949999999999999,"5min":1.49,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.93400000000000005},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.399,"5min":1.399,"15min":1.444},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0349999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.40600000000000003},"max":{"1min":1.417,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0629999999999999,"15min":1.0600000000000001},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":1.635,"15min":1.7390000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0269999999999999,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":0.89300000000000002,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.92500000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.722,"5min":1.722,"15min":1.9450000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.125,"15min":1.1539999999999999},"min":{"1min":0.83099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.623,"5min":1.623,"15min":2.0129999999999999},"last":1.0029999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2110000000000001,"15min":1.212},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1910000000000001,"15min":1.212},"min":{"1min":0.76000000000000001,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.5389999999999999,"5min":1.556,"15min":2.073},"last":1.0600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542369,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35840,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":1597440,"data_stored":825124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36700160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.503,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.123,"15min":1.3560000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95199999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.73299999999999998},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.2490000000000001,"15min":1.321},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.88200000000000001},"min":{"1min":0.60499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1439999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.83599999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.81299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2110000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.696,"15min":1.696},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97999999999999998,"15min":0.97299999999999998},"min":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.282,"5min":1.464,"15min":1.464},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.121,"15min":1.131},"min":{"1min":0.75600000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.774},"last":1.196}]},{"osd":5,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0860000000000001},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5109999999999999,"5min":1.546,"15min":1.623},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.88900000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.395,"5min":1.589,"15min":1.589},"last":1.2569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.089},{"interface":"front","average":{"1min":1.107,"5min":1.165,"15min":1.1850000000000001},"min":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.554,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4179999999999999,"5min":1.877,"15min":1.877},"last":1.407},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.194,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4410000000000001,"5min":1.5529999999999999,"15min":1.6339999999999999},"last":1.141}]}]},{"osd":0,"up_from":15,"seq":64424509664,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28464,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":26304,"kb_avail":94343376,"statfs":{"total":96636764160,"available":96607617024,"internally_reserved":0,"allocated":2203648,"data_stored":1417016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26935296},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.874,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.145,"5min":1.3,"15min":1.3},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.58899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":1.228,"15min":1.228},"last":0.63}]},{"osd":3,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.82899999999999996},"min":{"1min":0.5,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.012,"5min":1.1220000000000001,"15min":1.2769999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.53200000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.25,"5min":1.3160000000000001,"15min":1.325},"last":1.044}]},{"osd":4,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98099999999999998,"15min":1.0029999999999999},"min":{"1min":0.55300000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.47},"last":1.1100000000000001},{"interface":"front","average":{"1min":1,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.323,"5min":1.4239999999999999,"15min":1.587},"last":1.413}]},{"osd":5,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.091},"min":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5680000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.111,"15min":1.131},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.46,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.375}]},{"osd":6,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.113,"15min":1.155},"min":{"1min":0.71399999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3440000000000001,"5min":1.5269999999999999,"15min":1.5940000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.179,"15min":1.21},"min":{"1min":0.75700000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.502,"5min":1.5289999999999999,"15min":1.8300000000000001},"last":1.649}]},{"osd":7,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.204,"15min":1.228},"min":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.8109999999999999},"last":1.476},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1879999999999999,"15min":1.22},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.464,"5min":1.798,"15min":1.798},"last":1.603}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:32:13.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:32:14.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 Processor -- start 2023-09-27T23:32:14.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- start start 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1bb406d650 0x7f1bb4068870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40ad070 con 0x7f1bb406d280 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40ac780 con 0x7f1bb415aa10 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb4068e90 con 0x7f1bb406d650 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb2ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1bb406d650 0x7f1bb4068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1bb406d280 0x7f1bb4068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44474/0 (socket says 172.21.15.64:44474) 2023-09-27T23:32:14.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb37fe700 1 -- 172.21.15.64:0/1851109158 learned_addr learned my addr 172.21.15.64:0/1851109158 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb2ffd700 1 -- 172.21.15.64:0/1851109158 >> v1:172.21.15.123:6789/0 conn(0x7f1bb415aa10 legacy=0x7f1bb415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb2ffd700 1 -- 172.21.15.64:0/1851109158 >> v1:172.21.15.64:6789/0 conn(0x7f1bb406d280 legacy=0x7f1bb4068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb27fc700 1 -- 172.21.15.64:0/1851109158 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910369004 0 0) 0x7f1bb40ad070 con 0x7f1bb406d280 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb2ffd700 1 -- 172.21.15.64:0/1851109158 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40aca20 con 0x7f1bb406d650 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb2ffd700 1 --2- 172.21.15.64:0/1851109158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d650 0x7f1bb4068870 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009940 tx=0x7f1ba40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7b970a2406238f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb27fc700 1 -- 172.21.15.64:0/1851109158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400b870 con 0x7f1bb406d650 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb27fc700 1 -- 172.21.15.64:0/1851109158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba400b9d0 con 0x7f1bb406d650 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb27fc700 1 -- 172.21.15.64:0/1851109158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400bd70 con 0x7f1bb406d650 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1851109158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d650 msgr2=0x7f1bb4068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1851109158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d650 0x7f1bb4068870 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009940 tx=0x7f1ba40091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1851109158 shutdown_connections 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1851109158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d650 0x7f1bb4068870 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1851109158 >> 172.21.15.64:0/1851109158 conn(0x7f1bb406c080 msgr2=0x7f1bb40ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1851109158 shutdown_connections 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.008+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1851109158 wait complete. 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 Processor -- start 2023-09-27T23:32:14.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 -- start start 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 0x7f1bb414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb406d650 0x7f1bb41543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bb415aa10 0x7f1bb4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb414bfd0 con 0x7f1bb406d280 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40b2080 con 0x7f1bb406d650 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40b2510 con 0x7f1bb415aa10 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 0x7f1bb414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 0x7f1bb414ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50252/0 (socket says 172.21.15.64:50252) 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 -- 172.21.15.64:0/1034981263 learned_addr learned my addr 172.21.15.64:0/1034981263 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb2ffd700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb406d650 0x7f1bb41543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb3fff700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bb415aa10 0x7f1bb4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 -- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bb415aa10 msgr2=0x7f1bb4158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bb415aa10 0x7f1bb4158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 -- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb406d650 msgr2=0x7f1bb41543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb406d650 0x7f1bb41543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 -- 172.21.15.64:0/1034981263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba4009420 con 0x7f1bb406d280 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb2ffd700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb406d650 0x7f1bb41543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb3fff700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bb415aa10 0x7f1bb4158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb37fe700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 0x7f1bb414ba90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c5b0 tx=0x7f1ba8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8005b30 con 0x7f1bb406d280 2023-09-27T23:32:14.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8005c90 con 0x7f1bb406d280 2023-09-27T23:32:14.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb40b1470 con 0x7f1bb406d280 2023-09-27T23:32:14.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8007300 con 0x7f1bb406d280 2023-09-27T23:32:14.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.012+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb40b1a30 con 0x7f1bb406d280 2023-09-27T23:32:14.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.016+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb404be40 con 0x7f1bb406d280 2023-09-27T23:32:14.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.020+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1ba80076b0 con 0x7f1bb406d280 2023-09-27T23:32:14.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.020+0000 7f1b9b7fe700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ba0060a70 0x7f1ba0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.020+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1ba807cb10 con 0x7f1bb406d280 2023-09-27T23:32:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.020+0000 7f1bb2ffd700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ba0060a70 0x7f1ba0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.020+0000 7f1bb2ffd700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ba0060a70 0x7f1ba0062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40030c0 tx=0x7f1ba4016000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.024+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1ba804cb20 con 0x7f1bb406d280 2023-09-27T23:32:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.184+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bb40b21c0 con 0x7f1ba0060a70 2023-09-27T23:32:14.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.204+0000 7f1b9b7fe700 1 -- 172.21.15.64:0/1034981263 <== mgr.6971 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308106 (secure 0 0 0) 0x7f1ba400bb40 con 0x7f1ba0060a70 2023-09-27T23:32:14.214 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:14.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ba0060a70 msgr2=0x7f1ba0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ba0060a70 0x7f1ba0062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40030c0 tx=0x7f1ba4016000 comp rx=0 tx=0).stop 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 msgr2=0x7f1bb414ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 0x7f1bb414ba90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c5b0 tx=0x7f1ba8001070 comp rx=0 tx=0).stop 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 shutdown_connections 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ba0060a70 0x7f1ba0062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bb415aa10 0x7f1bb4158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb406d650 0x7f1bb41543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 --2- 172.21.15.64:0/1034981263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bb406d280 0x7f1bb414ba90 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 >> 172.21.15.64:0/1034981263 conn(0x7f1bb406c080 msgr2=0x7f1bb4151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 shutdown_connections 2023-09-27T23:32:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.208+0000 7f1bb8fae700 1 -- 172.21.15.64:0/1034981263 wait complete. 2023-09-27T23:32:14.218 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:32:14.229 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:32:12.731950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":404,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230088,"kb_used_data":13640,"kb_used_omap":0,"kb_used_meta":216384,"kb_avail":754744632,"statfs":{"total":773094113280,"available":772858503168,"internally_reserved":0,"allocated":13967360,"data_stored":7785348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":221577216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.275893"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885283+0000","last_change":"2023-09-27T23:32:05.874869+0000","last_active":"2023-09-27T23:32:05.885283+0000","last_peered":"2023-09-27T23:32:05.885283+0000","last_clean":"2023-09-27T23:32:05.885283+0000","last_became_active":"2023-09-27T23:32:05.874146+0000","last_became_peered":"2023-09-27T23:32:05.874146+0000","last_unstale":"2023-09-27T23:32:05.885283+0000","last_undegraded":"2023-09-27T23:32:05.885283+0000","last_fullsized":"2023-09-27T23:32:05.885283+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863665+0000","last_change":"2023-09-27T23:32:05.860722+0000","last_active":"2023-09-27T23:32:05.863665+0000","last_peered":"2023-09-27T23:32:05.863665+0000","last_clean":"2023-09-27T23:32:05.863665+0000","last_became_active":"2023-09-27T23:32:05.860381+0000","last_became_peered":"2023-09-27T23:32:05.860381+0000","last_unstale":"2023-09-27T23:32:05.863665+0000","last_undegraded":"2023-09-27T23:32:05.863665+0000","last_fullsized":"2023-09-27T23:32:05.863665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879763+0000","last_change":"2023-09-27T23:32:05.873945+0000","last_active":"2023-09-27T23:32:05.879763+0000","last_peered":"2023-09-27T23:32:05.879763+0000","last_clean":"2023-09-27T23:32:05.879763+0000","last_became_active":"2023-09-27T23:32:05.872503+0000","last_became_peered":"2023-09-27T23:32:05.872503+0000","last_unstale":"2023-09-27T23:32:05.879763+0000","last_undegraded":"2023-09-27T23:32:05.879763+0000","last_fullsized":"2023-09-27T23:32:05.879763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:06.052849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885427+0000","last_change":"2023-09-27T23:32:05.878460+0000","last_active":"2023-09-27T23:32:05.885427+0000","last_peered":"2023-09-27T23:32:05.885427+0000","last_clean":"2023-09-27T23:32:05.885427+0000","last_became_active":"2023-09-27T23:32:05.877778+0000","last_became_peered":"2023-09-27T23:32:05.877778+0000","last_unstale":"2023-09-27T23:32:05.885427+0000","last_undegraded":"2023-09-27T23:32:05.885427+0000","last_fullsized":"2023-09-27T23:32:05.885427+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:51:08.482614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.866846+0000","last_change":"2023-09-27T23:32:05.863147+0000","last_active":"2023-09-27T23:32:05.866846+0000","last_peered":"2023-09-27T23:32:05.866846+0000","last_clean":"2023-09-27T23:32:05.866846+0000","last_became_active":"2023-09-27T23:32:05.862602+0000","last_became_peered":"2023-09-27T23:32:05.862602+0000","last_unstale":"2023-09-27T23:32:05.866846+0000","last_undegraded":"2023-09-27T23:32:05.866846+0000","last_fullsized":"2023-09-27T23:32:05.866846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:16:01.411281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867813+0000","last_change":"2023-09-27T23:32:05.861426+0000","last_active":"2023-09-27T23:32:05.867813+0000","last_peered":"2023-09-27T23:32:05.867813+0000","last_clean":"2023-09-27T23:32:05.867813+0000","last_became_active":"2023-09-27T23:32:05.861004+0000","last_became_peered":"2023-09-27T23:32:05.861004+0000","last_unstale":"2023-09-27T23:32:05.867813+0000","last_undegraded":"2023-09-27T23:32:05.867813+0000","last_fullsized":"2023-09-27T23:32:05.867813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:09.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.870308+0000","last_change":"2023-09-27T23:32:05.865614+0000","last_active":"2023-09-27T23:32:05.870308+0000","last_peered":"2023-09-27T23:32:05.870308+0000","last_clean":"2023-09-27T23:32:05.870308+0000","last_became_active":"2023-09-27T23:32:05.861505+0000","last_became_peered":"2023-09-27T23:32:05.861505+0000","last_unstale":"2023-09-27T23:32:05.870308+0000","last_undegraded":"2023-09-27T23:32:05.870308+0000","last_fullsized":"2023-09-27T23:32:05.870308+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:36.735279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881110+0000","last_change":"2023-09-27T23:32:05.878031+0000","last_active":"2023-09-27T23:32:05.881110+0000","last_peered":"2023-09-27T23:32:05.881110+0000","last_clean":"2023-09-27T23:32:05.881110+0000","last_became_active":"2023-09-27T23:32:05.877499+0000","last_became_peered":"2023-09-27T23:32:05.877499+0000","last_unstale":"2023-09-27T23:32:05.881110+0000","last_undegraded":"2023-09-27T23:32:05.881110+0000","last_fullsized":"2023-09-27T23:32:05.881110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:24.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885247+0000","last_change":"2023-09-27T23:32:05.874547+0000","last_active":"2023-09-27T23:32:05.885247+0000","last_peered":"2023-09-27T23:32:05.885247+0000","last_clean":"2023-09-27T23:32:05.885247+0000","last_became_active":"2023-09-27T23:32:05.874003+0000","last_became_peered":"2023-09-27T23:32:05.874003+0000","last_unstale":"2023-09-27T23:32:05.885247+0000","last_undegraded":"2023-09-27T23:32:05.885247+0000","last_fullsized":"2023-09-27T23:32:05.885247+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885163+0000","last_change":"2023-09-27T23:32:05.873890+0000","last_active":"2023-09-27T23:32:05.885163+0000","last_peered":"2023-09-27T23:32:05.885163+0000","last_clean":"2023-09-27T23:32:05.885163+0000","last_became_active":"2023-09-27T23:32:05.870764+0000","last_became_peered":"2023-09-27T23:32:05.870764+0000","last_unstale":"2023-09-27T23:32:05.885163+0000","last_undegraded":"2023-09-27T23:32:05.885163+0000","last_fullsized":"2023-09-27T23:32:05.885163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865013+0000","last_change":"2023-09-27T23:32:05.861704+0000","last_active":"2023-09-27T23:32:05.865013+0000","last_peered":"2023-09-27T23:32:05.865013+0000","last_clean":"2023-09-27T23:32:05.865013+0000","last_became_active":"2023-09-27T23:32:05.861317+0000","last_became_peered":"2023-09-27T23:32:05.861317+0000","last_unstale":"2023-09-27T23:32:05.865013+0000","last_undegraded":"2023-09-27T23:32:05.865013+0000","last_fullsized":"2023-09-27T23:32:05.865013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885469+0000","last_change":"2023-09-27T23:32:05.878755+0000","last_active":"2023-09-27T23:32:05.885469+0000","last_peered":"2023-09-27T23:32:05.885469+0000","last_clean":"2023-09-27T23:32:05.885469+0000","last_became_active":"2023-09-27T23:32:05.878132+0000","last_became_peered":"2023-09-27T23:32:05.878132+0000","last_unstale":"2023-09-27T23:32:05.885469+0000","last_undegraded":"2023-09-27T23:32:05.885469+0000","last_fullsized":"2023-09-27T23:32:05.885469+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:11:14.766160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881099+0000","last_change":"2023-09-27T23:32:05.877897+0000","last_active":"2023-09-27T23:32:05.881099+0000","last_peered":"2023-09-27T23:32:05.881099+0000","last_clean":"2023-09-27T23:32:05.881099+0000","last_became_active":"2023-09-27T23:32:05.877494+0000","last_became_peered":"2023-09-27T23:32:05.877494+0000","last_unstale":"2023-09-27T23:32:05.881099+0000","last_undegraded":"2023-09-27T23:32:05.881099+0000","last_fullsized":"2023-09-27T23:32:05.881099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:55.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879623+0000","last_change":"2023-09-27T23:32:05.872800+0000","last_active":"2023-09-27T23:32:05.879623+0000","last_peered":"2023-09-27T23:32:05.879623+0000","last_clean":"2023-09-27T23:32:05.879623+0000","last_became_active":"2023-09-27T23:32:05.872486+0000","last_became_peered":"2023-09-27T23:32:05.872486+0000","last_unstale":"2023-09-27T23:32:05.879623+0000","last_undegraded":"2023-09-27T23:32:05.879623+0000","last_fullsized":"2023-09-27T23:32:05.879623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:57.553491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873468+0000","last_change":"2023-09-27T23:32:05.872253+0000","last_active":"2023-09-27T23:32:05.873468+0000","last_peered":"2023-09-27T23:32:05.873468+0000","last_clean":"2023-09-27T23:32:05.873468+0000","last_became_active":"2023-09-27T23:32:05.871935+0000","last_became_peered":"2023-09-27T23:32:05.871935+0000","last_unstale":"2023-09-27T23:32:05.873468+0000","last_undegraded":"2023-09-27T23:32:05.873468+0000","last_fullsized":"2023-09-27T23:32:05.873468+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:00.996007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885494+0000","last_change":"2023-09-27T23:32:05.878992+0000","last_active":"2023-09-27T23:32:05.885494+0000","last_peered":"2023-09-27T23:32:05.885494+0000","last_clean":"2023-09-27T23:32:05.885494+0000","last_became_active":"2023-09-27T23:32:05.878488+0000","last_became_peered":"2023-09-27T23:32:05.878488+0000","last_unstale":"2023-09-27T23:32:05.885494+0000","last_undegraded":"2023-09-27T23:32:05.885494+0000","last_fullsized":"2023-09-27T23:32:05.885494+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865348+0000","last_change":"2023-09-27T23:32:05.863475+0000","last_active":"2023-09-27T23:32:05.865348+0000","last_peered":"2023-09-27T23:32:05.865348+0000","last_clean":"2023-09-27T23:32:05.865348+0000","last_became_active":"2023-09-27T23:32:05.863171+0000","last_became_peered":"2023-09-27T23:32:05.863171+0000","last_unstale":"2023-09-27T23:32:05.865348+0000","last_undegraded":"2023-09-27T23:32:05.865348+0000","last_fullsized":"2023-09-27T23:32:05.865348+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:56.191065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862603+0000","last_change":"2023-09-27T23:32:05.859721+0000","last_active":"2023-09-27T23:32:05.862603+0000","last_peered":"2023-09-27T23:32:05.862603+0000","last_clean":"2023-09-27T23:32:05.862603+0000","last_became_active":"2023-09-27T23:32:05.859426+0000","last_became_peered":"2023-09-27T23:32:05.859426+0000","last_unstale":"2023-09-27T23:32:05.862603+0000","last_undegraded":"2023-09-27T23:32:05.862603+0000","last_fullsized":"2023-09-27T23:32:05.862603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:34:39.568162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879745+0000","last_change":"2023-09-27T23:32:05.874339+0000","last_active":"2023-09-27T23:32:05.879745+0000","last_peered":"2023-09-27T23:32:05.879745+0000","last_clean":"2023-09-27T23:32:05.879745+0000","last_became_active":"2023-09-27T23:32:05.872716+0000","last_became_peered":"2023-09-27T23:32:05.872716+0000","last_unstale":"2023-09-27T23:32:05.879745+0000","last_undegraded":"2023-09-27T23:32:05.879745+0000","last_fullsized":"2023-09-27T23:32:05.879745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:21.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871701+0000","last_change":"2023-09-27T23:32:05.870769+0000","last_active":"2023-09-27T23:32:05.871701+0000","last_peered":"2023-09-27T23:32:05.871701+0000","last_clean":"2023-09-27T23:32:05.871701+0000","last_became_active":"2023-09-27T23:32:05.870469+0000","last_became_peered":"2023-09-27T23:32:05.870469+0000","last_unstale":"2023-09-27T23:32:05.871701+0000","last_undegraded":"2023-09-27T23:32:05.871701+0000","last_fullsized":"2023-09-27T23:32:05.871701+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:01:42.856864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.867672+0000","last_change":"2023-09-27T23:32:05.865682+0000","last_active":"2023-09-27T23:32:05.867672+0000","last_peered":"2023-09-27T23:32:05.867672+0000","last_clean":"2023-09-27T23:32:05.867672+0000","last_became_active":"2023-09-27T23:32:05.865377+0000","last_became_peered":"2023-09-27T23:32:05.865377+0000","last_unstale":"2023-09-27T23:32:05.867672+0000","last_undegraded":"2023-09-27T23:32:05.867672+0000","last_fullsized":"2023-09-27T23:32:05.867672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:18:44.847390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868870+0000","last_change":"2023-09-27T23:32:05.860566+0000","last_active":"2023-09-27T23:32:05.868870+0000","last_peered":"2023-09-27T23:32:05.868870+0000","last_clean":"2023-09-27T23:32:05.868870+0000","last_became_active":"2023-09-27T23:32:05.860209+0000","last_became_peered":"2023-09-27T23:32:05.860209+0000","last_unstale":"2023-09-27T23:32:05.868870+0000","last_undegraded":"2023-09-27T23:32:05.868870+0000","last_fullsized":"2023-09-27T23:32:05.868870+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:42.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875841+0000","last_change":"2023-09-27T23:32:05.873642+0000","last_active":"2023-09-27T23:32:05.875841+0000","last_peered":"2023-09-27T23:32:05.875841+0000","last_clean":"2023-09-27T23:32:05.875841+0000","last_became_active":"2023-09-27T23:32:05.873228+0000","last_became_peered":"2023-09-27T23:32:05.873228+0000","last_unstale":"2023-09-27T23:32:05.875841+0000","last_undegraded":"2023-09-27T23:32:05.875841+0000","last_fullsized":"2023-09-27T23:32:05.875841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:26:39.116439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871647+0000","last_change":"2023-09-27T23:32:05.867910+0000","last_active":"2023-09-27T23:32:05.871647+0000","last_peered":"2023-09-27T23:32:05.871647+0000","last_clean":"2023-09-27T23:32:05.871647+0000","last_became_active":"2023-09-27T23:32:05.867511+0000","last_became_peered":"2023-09-27T23:32:05.867511+0000","last_unstale":"2023-09-27T23:32:05.871647+0000","last_undegraded":"2023-09-27T23:32:05.871647+0000","last_fullsized":"2023-09-27T23:32:05.871647+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:31:21.387831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865328+0000","last_change":"2023-09-27T23:32:05.863332+0000","last_active":"2023-09-27T23:32:05.865328+0000","last_peered":"2023-09-27T23:32:05.865328+0000","last_clean":"2023-09-27T23:32:05.865328+0000","last_became_active":"2023-09-27T23:32:05.863052+0000","last_became_peered":"2023-09-27T23:32:05.863052+0000","last_unstale":"2023-09-27T23:32:05.865328+0000","last_undegraded":"2023-09-27T23:32:05.865328+0000","last_fullsized":"2023-09-27T23:32:05.865328+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885048+0000","last_change":"2023-09-27T23:32:05.873577+0000","last_active":"2023-09-27T23:32:05.885048+0000","last_peered":"2023-09-27T23:32:05.885048+0000","last_clean":"2023-09-27T23:32:05.885048+0000","last_became_active":"2023-09-27T23:32:05.870063+0000","last_became_peered":"2023-09-27T23:32:05.870063+0000","last_unstale":"2023-09-27T23:32:05.885048+0000","last_undegraded":"2023-09-27T23:32:05.885048+0000","last_fullsized":"2023-09-27T23:32:05.885048+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:01:38.998159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.873422+0000","last_change":"2023-09-27T23:32:05.871483+0000","last_active":"2023-09-27T23:32:05.873422+0000","last_peered":"2023-09-27T23:32:05.873422+0000","last_clean":"2023-09-27T23:32:05.873422+0000","last_became_active":"2023-09-27T23:32:05.871087+0000","last_became_peered":"2023-09-27T23:32:05.871087+0000","last_unstale":"2023-09-27T23:32:05.873422+0000","last_undegraded":"2023-09-27T23:32:05.873422+0000","last_fullsized":"2023-09-27T23:32:05.873422+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:16.072601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881812+0000","last_change":"2023-09-27T23:32:05.879569+0000","last_active":"2023-09-27T23:32:05.881812+0000","last_peered":"2023-09-27T23:32:05.881812+0000","last_clean":"2023-09-27T23:32:05.881812+0000","last_became_active":"2023-09-27T23:32:05.879296+0000","last_became_peered":"2023-09-27T23:32:05.879296+0000","last_unstale":"2023-09-27T23:32:05.881812+0000","last_undegraded":"2023-09-27T23:32:05.881812+0000","last_fullsized":"2023-09-27T23:32:05.881812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:42.688153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876663+0000","last_change":"2023-09-27T23:32:05.875209+0000","last_active":"2023-09-27T23:32:05.876663+0000","last_peered":"2023-09-27T23:32:05.876663+0000","last_clean":"2023-09-27T23:32:05.876663+0000","last_became_active":"2023-09-27T23:32:05.874886+0000","last_became_peered":"2023-09-27T23:32:05.874886+0000","last_unstale":"2023-09-27T23:32:05.876663+0000","last_undegraded":"2023-09-27T23:32:05.876663+0000","last_fullsized":"2023-09-27T23:32:05.876663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:42:37.523447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871939+0000","last_change":"2023-09-27T23:32:05.868005+0000","last_active":"2023-09-27T23:32:05.871939+0000","last_peered":"2023-09-27T23:32:05.871939+0000","last_clean":"2023-09-27T23:32:05.871939+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871939+0000","last_undegraded":"2023-09-27T23:32:05.871939+0000","last_fullsized":"2023-09-27T23:32:05.871939+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:25.597622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885078+0000","last_change":"2023-09-27T23:32:05.873691+0000","last_active":"2023-09-27T23:32:05.885078+0000","last_peered":"2023-09-27T23:32:05.885078+0000","last_clean":"2023-09-27T23:32:05.885078+0000","last_became_active":"2023-09-27T23:32:05.870676+0000","last_became_peered":"2023-09-27T23:32:05.870676+0000","last_unstale":"2023-09-27T23:32:05.885078+0000","last_undegraded":"2023-09-27T23:32:05.885078+0000","last_fullsized":"2023-09-27T23:32:05.885078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.868346+0000","last_change":"2023-09-27T23:32:05.865108+0000","last_active":"2023-09-27T23:32:05.868346+0000","last_peered":"2023-09-27T23:32:05.868346+0000","last_clean":"2023-09-27T23:32:05.868346+0000","last_became_active":"2023-09-27T23:32:05.864839+0000","last_became_peered":"2023-09-27T23:32:05.864839+0000","last_unstale":"2023-09-27T23:32:05.868346+0000","last_undegraded":"2023-09-27T23:32:05.868346+0000","last_fullsized":"2023-09-27T23:32:05.868346+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:24:55.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865011+0000","last_change":"2023-09-27T23:32:05.859858+0000","last_active":"2023-09-27T23:32:05.865011+0000","last_peered":"2023-09-27T23:32:05.865011+0000","last_clean":"2023-09-27T23:32:05.865011+0000","last_became_active":"2023-09-27T23:32:05.859569+0000","last_became_peered":"2023-09-27T23:32:05.859569+0000","last_unstale":"2023-09-27T23:32:05.865011+0000","last_undegraded":"2023-09-27T23:32:05.865011+0000","last_fullsized":"2023-09-27T23:32:05.865011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:40.168536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"128'4","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877659+0000","last_change":"2023-09-27T23:32:05.473760+0000","last_active":"2023-09-27T23:32:05.877659+0000","last_peered":"2023-09-27T23:32:05.877659+0000","last_clean":"2023-09-27T23:32:05.877659+0000","last_became_active":"2023-09-27T23:32:05.473462+0000","last_became_peered":"2023-09-27T23:32:05.473462+0000","last_unstale":"2023-09-27T23:32:05.877659+0000","last_undegraded":"2023-09-27T23:32:05.877659+0000","last_fullsized":"2023-09-27T23:32:05.877659+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:52.366185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871871+0000","last_change":"2023-09-27T23:32:05.867437+0000","last_active":"2023-09-27T23:32:05.871871+0000","last_peered":"2023-09-27T23:32:05.871871+0000","last_clean":"2023-09-27T23:32:05.871871+0000","last_became_active":"2023-09-27T23:32:05.867174+0000","last_became_peered":"2023-09-27T23:32:05.867174+0000","last_unstale":"2023-09-27T23:32:05.871871+0000","last_undegraded":"2023-09-27T23:32:05.871871+0000","last_fullsized":"2023-09-27T23:32:05.871871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:22.332265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877163+0000","last_change":"2023-09-27T23:32:05.474605+0000","last_active":"2023-09-27T23:32:05.877163+0000","last_peered":"2023-09-27T23:32:05.877163+0000","last_clean":"2023-09-27T23:32:05.877163+0000","last_became_active":"2023-09-27T23:32:05.474269+0000","last_became_peered":"2023-09-27T23:32:05.474269+0000","last_unstale":"2023-09-27T23:32:05.877163+0000","last_undegraded":"2023-09-27T23:32:05.877163+0000","last_fullsized":"2023-09-27T23:32:05.877163+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864812+0000","last_change":"2023-09-27T23:32:05.858996+0000","last_active":"2023-09-27T23:32:05.864812+0000","last_peered":"2023-09-27T23:32:05.864812+0000","last_clean":"2023-09-27T23:32:05.864812+0000","last_became_active":"2023-09-27T23:32:05.858668+0000","last_became_peered":"2023-09-27T23:32:05.858668+0000","last_unstale":"2023-09-27T23:32:05.864812+0000","last_undegraded":"2023-09-27T23:32:05.864812+0000","last_fullsized":"2023-09-27T23:32:05.864812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:35:43.781059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862048+0000","last_change":"2023-09-27T23:32:05.477638+0000","last_active":"2023-09-27T23:32:05.862048+0000","last_peered":"2023-09-27T23:32:05.862048+0000","last_clean":"2023-09-27T23:32:05.862048+0000","last_became_active":"2023-09-27T23:32:05.477296+0000","last_became_peered":"2023-09-27T23:32:05.477296+0000","last_unstale":"2023-09-27T23:32:05.862048+0000","last_undegraded":"2023-09-27T23:32:05.862048+0000","last_fullsized":"2023-09-27T23:32:05.862048+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:58:24.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881733+0000","last_change":"2023-09-27T23:32:05.879475+0000","last_active":"2023-09-27T23:32:05.881733+0000","last_peered":"2023-09-27T23:32:05.881733+0000","last_clean":"2023-09-27T23:32:05.881733+0000","last_became_active":"2023-09-27T23:32:05.877744+0000","last_became_peered":"2023-09-27T23:32:05.877744+0000","last_unstale":"2023-09-27T23:32:05.881733+0000","last_undegraded":"2023-09-27T23:32:05.881733+0000","last_fullsized":"2023-09-27T23:32:05.881733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:18.120648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862399+0000","last_change":"2023-09-27T23:32:05.478152+0000","last_active":"2023-09-27T23:32:05.862399+0000","last_peered":"2023-09-27T23:32:05.862399+0000","last_clean":"2023-09-27T23:32:05.862399+0000","last_became_active":"2023-09-27T23:32:05.477854+0000","last_became_peered":"2023-09-27T23:32:05.477854+0000","last_unstale":"2023-09-27T23:32:05.862399+0000","last_undegraded":"2023-09-27T23:32:05.862399+0000","last_fullsized":"2023-09-27T23:32:05.862399+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:26.452556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.884996+0000","last_change":"2023-09-27T23:32:05.870784+0000","last_active":"2023-09-27T23:32:05.884996+0000","last_peered":"2023-09-27T23:32:05.884996+0000","last_clean":"2023-09-27T23:32:05.884996+0000","last_became_active":"2023-09-27T23:32:05.870489+0000","last_became_peered":"2023-09-27T23:32:05.870489+0000","last_unstale":"2023-09-27T23:32:05.884996+0000","last_undegraded":"2023-09-27T23:32:05.884996+0000","last_fullsized":"2023-09-27T23:32:05.884996+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:19.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864176+0000","last_change":"2023-09-27T23:32:05.484779+0000","last_active":"2023-09-27T23:32:05.864176+0000","last_peered":"2023-09-27T23:32:05.864176+0000","last_clean":"2023-09-27T23:32:05.864176+0000","last_became_active":"2023-09-27T23:32:05.484492+0000","last_became_peered":"2023-09-27T23:32:05.484492+0000","last_unstale":"2023-09-27T23:32:05.864176+0000","last_undegraded":"2023-09-27T23:32:05.864176+0000","last_fullsized":"2023-09-27T23:32:05.864176+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:32:29.817112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865090+0000","last_change":"2023-09-27T23:32:05.861730+0000","last_active":"2023-09-27T23:32:05.865090+0000","last_peered":"2023-09-27T23:32:05.865090+0000","last_clean":"2023-09-27T23:32:05.865090+0000","last_became_active":"2023-09-27T23:32:05.861021+0000","last_became_peered":"2023-09-27T23:32:05.861021+0000","last_unstale":"2023-09-27T23:32:05.865090+0000","last_undegraded":"2023-09-27T23:32:05.865090+0000","last_fullsized":"2023-09-27T23:32:05.865090+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.862148+0000","last_change":"2023-09-27T23:32:05.474408+0000","last_active":"2023-09-27T23:32:05.862148+0000","last_peered":"2023-09-27T23:32:05.862148+0000","last_clean":"2023-09-27T23:32:05.862148+0000","last_became_active":"2023-09-27T23:32:05.473091+0000","last_became_peered":"2023-09-27T23:32:05.473091+0000","last_unstale":"2023-09-27T23:32:05.862148+0000","last_undegraded":"2023-09-27T23:32:05.862148+0000","last_fullsized":"2023-09-27T23:32:05.862148+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:49:20.064087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885105+0000","last_change":"2023-09-27T23:32:05.870585+0000","last_active":"2023-09-27T23:32:05.885105+0000","last_peered":"2023-09-27T23:32:05.885105+0000","last_clean":"2023-09-27T23:32:05.885105+0000","last_became_active":"2023-09-27T23:32:05.870282+0000","last_became_peered":"2023-09-27T23:32:05.870282+0000","last_unstale":"2023-09-27T23:32:05.885105+0000","last_undegraded":"2023-09-27T23:32:05.885105+0000","last_fullsized":"2023-09-27T23:32:05.885105+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:12.442888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855524+0000","last_change":"2023-09-27T23:32:05.486286+0000","last_active":"2023-09-27T23:32:05.855524+0000","last_peered":"2023-09-27T23:32:05.855524+0000","last_clean":"2023-09-27T23:32:05.855524+0000","last_became_active":"2023-09-27T23:32:05.485968+0000","last_became_peered":"2023-09-27T23:32:05.485968+0000","last_unstale":"2023-09-27T23:32:05.855524+0000","last_undegraded":"2023-09-27T23:32:05.855524+0000","last_fullsized":"2023-09-27T23:32:05.855524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:20.522455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.870329+0000","last_change":"2023-09-27T23:32:05.865877+0000","last_active":"2023-09-27T23:32:05.870329+0000","last_peered":"2023-09-27T23:32:05.870329+0000","last_clean":"2023-09-27T23:32:05.870329+0000","last_became_active":"2023-09-27T23:32:05.865294+0000","last_became_peered":"2023-09-27T23:32:05.865294+0000","last_unstale":"2023-09-27T23:32:05.870329+0000","last_undegraded":"2023-09-27T23:32:05.870329+0000","last_fullsized":"2023-09-27T23:32:05.870329+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:54:00.479947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.851579+0000","last_change":"2023-09-27T23:32:05.484134+0000","last_active":"2023-09-27T23:32:05.851579+0000","last_peered":"2023-09-27T23:32:05.851579+0000","last_clean":"2023-09-27T23:32:05.851579+0000","last_became_active":"2023-09-27T23:32:05.483856+0000","last_became_peered":"2023-09-27T23:32:05.483856+0000","last_unstale":"2023-09-27T23:32:05.851579+0000","last_undegraded":"2023-09-27T23:32:05.851579+0000","last_fullsized":"2023-09-27T23:32:05.851579+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:40.992540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.883531+0000","last_change":"2023-09-27T23:32:05.882768+0000","last_active":"2023-09-27T23:32:05.883531+0000","last_peered":"2023-09-27T23:32:05.883531+0000","last_clean":"2023-09-27T23:32:05.883531+0000","last_became_active":"2023-09-27T23:32:05.882322+0000","last_became_peered":"2023-09-27T23:32:05.882322+0000","last_unstale":"2023-09-27T23:32:05.883531+0000","last_undegraded":"2023-09-27T23:32:05.883531+0000","last_fullsized":"2023-09-27T23:32:05.883531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:40:46.976392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881529+0000","last_change":"2023-09-27T23:32:05.486576+0000","last_active":"2023-09-27T23:32:05.881529+0000","last_peered":"2023-09-27T23:32:05.881529+0000","last_clean":"2023-09-27T23:32:05.881529+0000","last_became_active":"2023-09-27T23:32:05.484863+0000","last_became_peered":"2023-09-27T23:32:05.484863+0000","last_unstale":"2023-09-27T23:32:05.881529+0000","last_undegraded":"2023-09-27T23:32:05.881529+0000","last_fullsized":"2023-09-27T23:32:05.881529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:20.574016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877676+0000","last_change":"2023-09-27T23:32:05.875211+0000","last_active":"2023-09-27T23:32:05.877676+0000","last_peered":"2023-09-27T23:32:05.877676+0000","last_clean":"2023-09-27T23:32:05.877676+0000","last_became_active":"2023-09-27T23:32:05.874899+0000","last_became_peered":"2023-09-27T23:32:05.874899+0000","last_unstale":"2023-09-27T23:32:05.877676+0000","last_undegraded":"2023-09-27T23:32:05.877676+0000","last_fullsized":"2023-09-27T23:32:05.877676+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:26:38.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.858932+0000","last_change":"2023-09-27T23:32:05.474634+0000","last_active":"2023-09-27T23:32:05.858932+0000","last_peered":"2023-09-27T23:32:05.858932+0000","last_clean":"2023-09-27T23:32:05.858932+0000","last_became_active":"2023-09-27T23:32:05.474314+0000","last_became_peered":"2023-09-27T23:32:05.474314+0000","last_unstale":"2023-09-27T23:32:05.858932+0000","last_undegraded":"2023-09-27T23:32:05.858932+0000","last_fullsized":"2023-09-27T23:32:05.858932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:05.702752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885276+0000","last_change":"2023-09-27T23:32:05.884516+0000","last_active":"2023-09-27T23:32:05.885276+0000","last_peered":"2023-09-27T23:32:05.885276+0000","last_clean":"2023-09-27T23:32:05.885276+0000","last_became_active":"2023-09-27T23:32:05.884276+0000","last_became_peered":"2023-09-27T23:32:05.884276+0000","last_unstale":"2023-09-27T23:32:05.885276+0000","last_undegraded":"2023-09-27T23:32:05.885276+0000","last_fullsized":"2023-09-27T23:32:05.885276+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:45:14.206863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871586+0000","last_change":"2023-09-27T23:32:05.483320+0000","last_active":"2023-09-27T23:32:05.871586+0000","last_peered":"2023-09-27T23:32:05.871586+0000","last_clean":"2023-09-27T23:32:05.871586+0000","last_became_active":"2023-09-27T23:32:05.482969+0000","last_became_peered":"2023-09-27T23:32:05.482969+0000","last_unstale":"2023-09-27T23:32:05.871586+0000","last_undegraded":"2023-09-27T23:32:05.871586+0000","last_fullsized":"2023-09-27T23:32:05.871586+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:55:56.542035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881069+0000","last_change":"2023-09-27T23:32:05.877251+0000","last_active":"2023-09-27T23:32:05.881069+0000","last_peered":"2023-09-27T23:32:05.881069+0000","last_clean":"2023-09-27T23:32:05.881069+0000","last_became_active":"2023-09-27T23:32:05.876868+0000","last_became_peered":"2023-09-27T23:32:05.876868+0000","last_unstale":"2023-09-27T23:32:05.881069+0000","last_undegraded":"2023-09-27T23:32:05.881069+0000","last_fullsized":"2023-09-27T23:32:05.881069+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:10:56.379380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.854712+0000","last_change":"2023-09-27T23:32:05.487592+0000","last_active":"2023-09-27T23:32:05.854712+0000","last_peered":"2023-09-27T23:32:05.854712+0000","last_clean":"2023-09-27T23:32:05.854712+0000","last_became_active":"2023-09-27T23:32:05.487244+0000","last_became_peered":"2023-09-27T23:32:05.487244+0000","last_unstale":"2023-09-27T23:32:05.854712+0000","last_undegraded":"2023-09-27T23:32:05.854712+0000","last_fullsized":"2023-09-27T23:32:05.854712+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:07:33.532568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.879641+0000","last_change":"2023-09-27T23:32:05.873902+0000","last_active":"2023-09-27T23:32:05.879641+0000","last_peered":"2023-09-27T23:32:05.879641+0000","last_clean":"2023-09-27T23:32:05.879641+0000","last_became_active":"2023-09-27T23:32:05.872995+0000","last_became_peered":"2023-09-27T23:32:05.872995+0000","last_unstale":"2023-09-27T23:32:05.879641+0000","last_undegraded":"2023-09-27T23:32:05.879641+0000","last_fullsized":"2023-09-27T23:32:05.879641+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:35.151434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864787+0000","last_change":"2023-09-27T23:32:05.477932+0000","last_active":"2023-09-27T23:32:05.864787+0000","last_peered":"2023-09-27T23:32:05.864787+0000","last_clean":"2023-09-27T23:32:05.864787+0000","last_became_active":"2023-09-27T23:32:05.477556+0000","last_became_peered":"2023-09-27T23:32:05.477556+0000","last_unstale":"2023-09-27T23:32:05.864787+0000","last_undegraded":"2023-09-27T23:32:05.864787+0000","last_fullsized":"2023-09-27T23:32:05.864787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:23:33.015795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885492+0000","last_change":"2023-09-27T23:32:05.878808+0000","last_active":"2023-09-27T23:32:05.885492+0000","last_peered":"2023-09-27T23:32:05.885492+0000","last_clean":"2023-09-27T23:32:05.885492+0000","last_became_active":"2023-09-27T23:32:05.878462+0000","last_became_peered":"2023-09-27T23:32:05.878462+0000","last_unstale":"2023-09-27T23:32:05.885492+0000","last_undegraded":"2023-09-27T23:32:05.885492+0000","last_fullsized":"2023-09-27T23:32:05.885492+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:27:55.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847377+0000","last_change":"2023-09-27T23:32:05.473937+0000","last_active":"2023-09-27T23:32:05.847377+0000","last_peered":"2023-09-27T23:32:05.847377+0000","last_clean":"2023-09-27T23:32:05.847377+0000","last_became_active":"2023-09-27T23:32:05.473535+0000","last_became_peered":"2023-09-27T23:32:05.473535+0000","last_unstale":"2023-09-27T23:32:05.847377+0000","last_undegraded":"2023-09-27T23:32:05.847377+0000","last_fullsized":"2023-09-27T23:32:05.847377+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:53:59.468491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.880779+0000","last_change":"2023-09-27T23:32:05.877232+0000","last_active":"2023-09-27T23:32:05.880779+0000","last_peered":"2023-09-27T23:32:05.880779+0000","last_clean":"2023-09-27T23:32:05.880779+0000","last_became_active":"2023-09-27T23:32:05.876906+0000","last_became_peered":"2023-09-27T23:32:05.876906+0000","last_unstale":"2023-09-27T23:32:05.880779+0000","last_undegraded":"2023-09-27T23:32:05.880779+0000","last_fullsized":"2023-09-27T23:32:05.880779+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:58.996920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848215+0000","last_change":"2023-09-27T23:32:05.478424+0000","last_active":"2023-09-27T23:32:05.848215+0000","last_peered":"2023-09-27T23:32:05.848215+0000","last_clean":"2023-09-27T23:32:05.848215+0000","last_became_active":"2023-09-27T23:32:05.478120+0000","last_became_peered":"2023-09-27T23:32:05.478120+0000","last_unstale":"2023-09-27T23:32:05.848215+0000","last_undegraded":"2023-09-27T23:32:05.848215+0000","last_fullsized":"2023-09-27T23:32:05.848215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:02:15.570089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.877156+0000","last_change":"2023-09-27T23:32:05.875429+0000","last_active":"2023-09-27T23:32:05.877156+0000","last_peered":"2023-09-27T23:32:05.877156+0000","last_clean":"2023-09-27T23:32:05.877156+0000","last_became_active":"2023-09-27T23:32:05.875127+0000","last_became_peered":"2023-09-27T23:32:05.875127+0000","last_unstale":"2023-09-27T23:32:05.877156+0000","last_undegraded":"2023-09-27T23:32:05.877156+0000","last_fullsized":"2023-09-27T23:32:05.877156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:11:21.585472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.851675+0000","last_change":"2023-09-27T23:32:05.481075+0000","last_active":"2023-09-27T23:32:05.851675+0000","last_peered":"2023-09-27T23:32:05.851675+0000","last_clean":"2023-09-27T23:32:05.851675+0000","last_became_active":"2023-09-27T23:32:05.480382+0000","last_became_peered":"2023-09-27T23:32:05.480382+0000","last_unstale":"2023-09-27T23:32:05.851675+0000","last_undegraded":"2023-09-27T23:32:05.851675+0000","last_fullsized":"2023-09-27T23:32:05.851675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:51.337572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875964+0000","last_change":"2023-09-27T23:32:05.873357+0000","last_active":"2023-09-27T23:32:05.875964+0000","last_peered":"2023-09-27T23:32:05.875964+0000","last_clean":"2023-09-27T23:32:05.875964+0000","last_became_active":"2023-09-27T23:32:05.873037+0000","last_became_peered":"2023-09-27T23:32:05.873037+0000","last_unstale":"2023-09-27T23:32:05.875964+0000","last_undegraded":"2023-09-27T23:32:05.875964+0000","last_fullsized":"2023-09-27T23:32:05.875964+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:54:10.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882131+0000","last_change":"2023-09-27T23:32:05.487770+0000","last_active":"2023-09-27T23:32:05.882131+0000","last_peered":"2023-09-27T23:32:05.882131+0000","last_clean":"2023-09-27T23:32:05.882131+0000","last_became_active":"2023-09-27T23:32:05.487423+0000","last_became_peered":"2023-09-27T23:32:05.487423+0000","last_unstale":"2023-09-27T23:32:05.882131+0000","last_undegraded":"2023-09-27T23:32:05.882131+0000","last_fullsized":"2023-09-27T23:32:05.882131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:07:34.139978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885219+0000","last_change":"2023-09-27T23:32:05.874286+0000","last_active":"2023-09-27T23:32:05.885219+0000","last_peered":"2023-09-27T23:32:05.885219+0000","last_clean":"2023-09-27T23:32:05.885219+0000","last_became_active":"2023-09-27T23:32:05.870718+0000","last_became_peered":"2023-09-27T23:32:05.870718+0000","last_unstale":"2023-09-27T23:32:05.885219+0000","last_undegraded":"2023-09-27T23:32:05.885219+0000","last_fullsized":"2023-09-27T23:32:05.885219+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:29:40.248653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"128'1","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:06.303516+0000","last_change":"2023-09-27T23:32:05.473896+0000","last_active":"2023-09-27T23:32:06.303516+0000","last_peered":"2023-09-27T23:32:06.303516+0000","last_clean":"2023-09-27T23:32:06.303516+0000","last_became_active":"2023-09-27T23:32:05.473665+0000","last_became_peered":"2023-09-27T23:32:05.473665+0000","last_unstale":"2023-09-27T23:32:06.303516+0000","last_undegraded":"2023-09-27T23:32:06.303516+0000","last_fullsized":"2023-09-27T23:32:06.303516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:05:17.109191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885451+0000","last_change":"2023-09-27T23:32:05.878658+0000","last_active":"2023-09-27T23:32:05.885451+0000","last_peered":"2023-09-27T23:32:05.885451+0000","last_clean":"2023-09-27T23:32:05.885451+0000","last_became_active":"2023-09-27T23:32:05.877911+0000","last_became_peered":"2023-09-27T23:32:05.877911+0000","last_unstale":"2023-09-27T23:32:05.885451+0000","last_undegraded":"2023-09-27T23:32:05.885451+0000","last_fullsized":"2023-09-27T23:32:05.885451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:39.054756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855548+0000","last_change":"2023-09-27T23:32:05.480279+0000","last_active":"2023-09-27T23:32:05.855548+0000","last_peered":"2023-09-27T23:32:05.855548+0000","last_clean":"2023-09-27T23:32:05.855548+0000","last_became_active":"2023-09-27T23:32:05.479931+0000","last_became_peered":"2023-09-27T23:32:05.479931+0000","last_unstale":"2023-09-27T23:32:05.855548+0000","last_undegraded":"2023-09-27T23:32:05.855548+0000","last_fullsized":"2023-09-27T23:32:05.855548+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:07.847247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876032+0000","last_change":"2023-09-27T23:32:05.874148+0000","last_active":"2023-09-27T23:32:05.876032+0000","last_peered":"2023-09-27T23:32:05.876032+0000","last_clean":"2023-09-27T23:32:05.876032+0000","last_became_active":"2023-09-27T23:32:05.873853+0000","last_became_peered":"2023-09-27T23:32:05.873853+0000","last_unstale":"2023-09-27T23:32:05.876032+0000","last_undegraded":"2023-09-27T23:32:05.876032+0000","last_fullsized":"2023-09-27T23:32:05.876032+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:16:14.012071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846161+0000","last_change":"2023-09-27T23:32:05.482613+0000","last_active":"2023-09-27T23:32:05.846161+0000","last_peered":"2023-09-27T23:32:05.846161+0000","last_clean":"2023-09-27T23:32:05.846161+0000","last_became_active":"2023-09-27T23:32:05.482252+0000","last_became_peered":"2023-09-27T23:32:05.482252+0000","last_unstale":"2023-09-27T23:32:05.846161+0000","last_undegraded":"2023-09-27T23:32:05.846161+0000","last_fullsized":"2023-09-27T23:32:05.846161+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:05.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869011+0000","last_change":"2023-09-27T23:32:05.866830+0000","last_active":"2023-09-27T23:32:05.869011+0000","last_peered":"2023-09-27T23:32:05.869011+0000","last_clean":"2023-09-27T23:32:05.869011+0000","last_became_active":"2023-09-27T23:32:05.866498+0000","last_became_peered":"2023-09-27T23:32:05.866498+0000","last_unstale":"2023-09-27T23:32:05.869011+0000","last_undegraded":"2023-09-27T23:32:05.869011+0000","last_fullsized":"2023-09-27T23:32:05.869011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:35.884351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855259+0000","last_change":"2023-09-27T23:32:05.480057+0000","last_active":"2023-09-27T23:32:05.855259+0000","last_peered":"2023-09-27T23:32:05.855259+0000","last_clean":"2023-09-27T23:32:05.855259+0000","last_became_active":"2023-09-27T23:32:05.479684+0000","last_became_peered":"2023-09-27T23:32:05.479684+0000","last_unstale":"2023-09-27T23:32:05.855259+0000","last_undegraded":"2023-09-27T23:32:05.855259+0000","last_fullsized":"2023-09-27T23:32:05.855259+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:04:23.100428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.865072+0000","last_change":"2023-09-27T23:32:05.860941+0000","last_active":"2023-09-27T23:32:05.865072+0000","last_peered":"2023-09-27T23:32:05.865072+0000","last_clean":"2023-09-27T23:32:05.865072+0000","last_became_active":"2023-09-27T23:32:05.860543+0000","last_became_peered":"2023-09-27T23:32:05.860543+0000","last_unstale":"2023-09-27T23:32:05.865072+0000","last_undegraded":"2023-09-27T23:32:05.865072+0000","last_fullsized":"2023-09-27T23:32:05.865072+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:48.667352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.852958+0000","last_change":"2023-09-27T23:32:05.479760+0000","last_active":"2023-09-27T23:32:05.852958+0000","last_peered":"2023-09-27T23:32:05.852958+0000","last_clean":"2023-09-27T23:32:05.852958+0000","last_became_active":"2023-09-27T23:32:05.479474+0000","last_became_peered":"2023-09-27T23:32:05.479474+0000","last_unstale":"2023-09-27T23:32:05.852958+0000","last_undegraded":"2023-09-27T23:32:05.852958+0000","last_fullsized":"2023-09-27T23:32:05.852958+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:43.445603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881743+0000","last_change":"2023-09-27T23:32:05.879523+0000","last_active":"2023-09-27T23:32:05.881743+0000","last_peered":"2023-09-27T23:32:05.881743+0000","last_clean":"2023-09-27T23:32:05.881743+0000","last_became_active":"2023-09-27T23:32:05.878009+0000","last_became_peered":"2023-09-27T23:32:05.878009+0000","last_unstale":"2023-09-27T23:32:05.881743+0000","last_undegraded":"2023-09-27T23:32:05.881743+0000","last_fullsized":"2023-09-27T23:32:05.881743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:21:40.330475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847190+0000","last_change":"2023-09-27T23:32:05.473857+0000","last_active":"2023-09-27T23:32:05.847190+0000","last_peered":"2023-09-27T23:32:05.847190+0000","last_clean":"2023-09-27T23:32:05.847190+0000","last_became_active":"2023-09-27T23:32:05.473507+0000","last_became_peered":"2023-09-27T23:32:05.473507+0000","last_unstale":"2023-09-27T23:32:05.847190+0000","last_undegraded":"2023-09-27T23:32:05.847190+0000","last_fullsized":"2023-09-27T23:32:05.847190+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:19.244182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885101+0000","last_change":"2023-09-27T23:32:05.873758+0000","last_active":"2023-09-27T23:32:05.885101+0000","last_peered":"2023-09-27T23:32:05.885101+0000","last_clean":"2023-09-27T23:32:05.885101+0000","last_became_active":"2023-09-27T23:32:05.870310+0000","last_became_peered":"2023-09-27T23:32:05.870310+0000","last_unstale":"2023-09-27T23:32:05.885101+0000","last_undegraded":"2023-09-27T23:32:05.885101+0000","last_fullsized":"2023-09-27T23:32:05.885101+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:35.596502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.851773+0000","last_change":"2023-09-27T23:32:05.482571+0000","last_active":"2023-09-27T23:32:05.851773+0000","last_peered":"2023-09-27T23:32:05.851773+0000","last_clean":"2023-09-27T23:32:05.851773+0000","last_became_active":"2023-09-27T23:32:05.480775+0000","last_became_peered":"2023-09-27T23:32:05.480775+0000","last_unstale":"2023-09-27T23:32:05.851773+0000","last_undegraded":"2023-09-27T23:32:05.851773+0000","last_fullsized":"2023-09-27T23:32:05.851773+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:31.045851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869149+0000","last_change":"2023-09-27T23:32:05.861010+0000","last_active":"2023-09-27T23:32:05.869149+0000","last_peered":"2023-09-27T23:32:05.869149+0000","last_clean":"2023-09-27T23:32:05.869149+0000","last_became_active":"2023-09-27T23:32:05.860737+0000","last_became_peered":"2023-09-27T23:32:05.860737+0000","last_unstale":"2023-09-27T23:32:05.869149+0000","last_undegraded":"2023-09-27T23:32:05.869149+0000","last_fullsized":"2023-09-27T23:32:05.869149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:58:21.738937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.856839+0000","last_change":"2023-09-27T23:32:05.487870+0000","last_active":"2023-09-27T23:32:05.856839+0000","last_peered":"2023-09-27T23:32:05.856839+0000","last_clean":"2023-09-27T23:32:05.856839+0000","last_became_active":"2023-09-27T23:32:05.487469+0000","last_became_peered":"2023-09-27T23:32:05.487469+0000","last_unstale":"2023-09-27T23:32:05.856839+0000","last_undegraded":"2023-09-27T23:32:05.856839+0000","last_fullsized":"2023-09-27T23:32:05.856839+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:25:27.956056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"117'227","reported_seq":650,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864897+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:32:05.864897+0000","last_peered":"2023-09-27T23:32:05.864897+0000","last_clean":"2023-09-27T23:32:05.864897+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:32:05.864897+0000","last_undegraded":"2023-09-27T23:32:05.864897+0000","last_fullsized":"2023-09-27T23:32:05.864897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":227,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":114,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.855351+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:32:05.855351+0000","last_peered":"2023-09-27T23:32:05.855351+0000","last_clean":"2023-09-27T23:32:05.855351+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:32:05.855351+0000","last_undegraded":"2023-09-27T23:32:05.855351+0000","last_fullsized":"2023-09-27T23:32:05.855351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869078+0000","last_change":"2023-09-27T23:32:05.862823+0000","last_active":"2023-09-27T23:32:05.869078+0000","last_peered":"2023-09-27T23:32:05.869078+0000","last_clean":"2023-09-27T23:32:05.869078+0000","last_became_active":"2023-09-27T23:32:05.862677+0000","last_became_peered":"2023-09-27T23:32:05.862677+0000","last_unstale":"2023-09-27T23:32:05.869078+0000","last_undegraded":"2023-09-27T23:32:05.869078+0000","last_fullsized":"2023-09-27T23:32:05.869078+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:32:01.586755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.846271+0000","last_change":"2023-09-27T23:32:05.487459+0000","last_active":"2023-09-27T23:32:05.846271+0000","last_peered":"2023-09-27T23:32:05.846271+0000","last_clean":"2023-09-27T23:32:05.846271+0000","last_became_active":"2023-09-27T23:32:05.487130+0000","last_became_peered":"2023-09-27T23:32:05.487130+0000","last_unstale":"2023-09-27T23:32:05.846271+0000","last_undegraded":"2023-09-27T23:32:05.846271+0000","last_fullsized":"2023-09-27T23:32:05.846271+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:53:57.985464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.864781+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:32:05.864781+0000","last_peered":"2023-09-27T23:32:05.864781+0000","last_clean":"2023-09-27T23:32:05.864781+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:32:05.864781+0000","last_undegraded":"2023-09-27T23:32:05.864781+0000","last_fullsized":"2023-09-27T23:32:05.864781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871725+0000","last_change":"2023-09-27T23:32:05.864506+0000","last_active":"2023-09-27T23:32:05.871725+0000","last_peered":"2023-09-27T23:32:05.871725+0000","last_clean":"2023-09-27T23:32:05.871725+0000","last_became_active":"2023-09-27T23:32:05.864207+0000","last_became_peered":"2023-09-27T23:32:05.864207+0000","last_unstale":"2023-09-27T23:32:05.871725+0000","last_undegraded":"2023-09-27T23:32:05.871725+0000","last_fullsized":"2023-09-27T23:32:05.871725+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:53:46.038154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.875524+0000","last_change":"2023-09-27T23:32:05.478414+0000","last_active":"2023-09-27T23:32:05.875524+0000","last_peered":"2023-09-27T23:32:05.875524+0000","last_clean":"2023-09-27T23:32:05.875524+0000","last_became_active":"2023-09-27T23:32:05.478088+0000","last_became_peered":"2023-09-27T23:32:05.478088+0000","last_unstale":"2023-09-27T23:32:05.875524+0000","last_undegraded":"2023-09-27T23:32:05.875524+0000","last_fullsized":"2023-09-27T23:32:05.875524+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:21.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847252+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:32:05.847252+0000","last_peered":"2023-09-27T23:32:05.847252+0000","last_clean":"2023-09-27T23:32:05.847252+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:32:05.847252+0000","last_undegraded":"2023-09-27T23:32:05.847252+0000","last_fullsized":"2023-09-27T23:32:05.847252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.871490+0000","last_change":"2023-09-27T23:32:05.864496+0000","last_active":"2023-09-27T23:32:05.871490+0000","last_peered":"2023-09-27T23:32:05.871490+0000","last_clean":"2023-09-27T23:32:05.871490+0000","last_became_active":"2023-09-27T23:32:05.863828+0000","last_became_peered":"2023-09-27T23:32:05.863828+0000","last_unstale":"2023-09-27T23:32:05.871490+0000","last_undegraded":"2023-09-27T23:32:05.871490+0000","last_fullsized":"2023-09-27T23:32:05.871490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:46.777842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.882372+0000","last_change":"2023-09-27T23:32:05.485399+0000","last_active":"2023-09-27T23:32:05.882372+0000","last_peered":"2023-09-27T23:32:05.882372+0000","last_clean":"2023-09-27T23:32:05.882372+0000","last_became_active":"2023-09-27T23:32:05.485073+0000","last_became_peered":"2023-09-27T23:32:05.485073+0000","last_unstale":"2023-09-27T23:32:05.882372+0000","last_undegraded":"2023-09-27T23:32:05.882372+0000","last_fullsized":"2023-09-27T23:32:05.882372+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:20.429282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":114,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.857942+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:32:05.857942+0000","last_peered":"2023-09-27T23:32:05.857942+0000","last_clean":"2023-09-27T23:32:05.857942+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:32:05.857942+0000","last_undegraded":"2023-09-27T23:32:05.857942+0000","last_fullsized":"2023-09-27T23:32:05.857942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.876013+0000","last_change":"2023-09-27T23:32:05.874103+0000","last_active":"2023-09-27T23:32:05.876013+0000","last_peered":"2023-09-27T23:32:05.876013+0000","last_clean":"2023-09-27T23:32:05.876013+0000","last_became_active":"2023-09-27T23:32:05.873806+0000","last_became_peered":"2023-09-27T23:32:05.873806+0000","last_unstale":"2023-09-27T23:32:05.876013+0000","last_undegraded":"2023-09-27T23:32:05.876013+0000","last_fullsized":"2023-09-27T23:32:05.876013+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:06:14.340339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.845266+0000","last_change":"2023-09-27T23:32:05.474807+0000","last_active":"2023-09-27T23:32:05.845266+0000","last_peered":"2023-09-27T23:32:05.845266+0000","last_clean":"2023-09-27T23:32:05.845266+0000","last_became_active":"2023-09-27T23:32:05.474510+0000","last_became_peered":"2023-09-27T23:32:05.474510+0000","last_unstale":"2023-09-27T23:32:05.845266+0000","last_undegraded":"2023-09-27T23:32:05.845266+0000","last_fullsized":"2023-09-27T23:32:05.845266+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:54:23.368573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.849356+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:32:05.849356+0000","last_peered":"2023-09-27T23:32:05.849356+0000","last_clean":"2023-09-27T23:32:05.849356+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:32:05.849356+0000","last_undegraded":"2023-09-27T23:32:05.849356+0000","last_fullsized":"2023-09-27T23:32:05.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.870438+0000","last_change":"2023-09-27T23:32:05.865477+0000","last_active":"2023-09-27T23:32:05.870438+0000","last_peered":"2023-09-27T23:32:05.870438+0000","last_clean":"2023-09-27T23:32:05.870438+0000","last_became_active":"2023-09-27T23:32:05.865162+0000","last_became_peered":"2023-09-27T23:32:05.865162+0000","last_unstale":"2023-09-27T23:32:05.870438+0000","last_undegraded":"2023-09-27T23:32:05.870438+0000","last_fullsized":"2023-09-27T23:32:05.870438+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:29:27.464013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847066+0000","last_change":"2023-09-27T23:32:05.477781+0000","last_active":"2023-09-27T23:32:05.847066+0000","last_peered":"2023-09-27T23:32:05.847066+0000","last_clean":"2023-09-27T23:32:05.847066+0000","last_became_active":"2023-09-27T23:32:05.477416+0000","last_became_peered":"2023-09-27T23:32:05.477416+0000","last_unstale":"2023-09-27T23:32:05.847066+0000","last_undegraded":"2023-09-27T23:32:05.847066+0000","last_fullsized":"2023-09-27T23:32:05.847066+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:35.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.863310+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:32:05.863310+0000","last_peered":"2023-09-27T23:32:05.863310+0000","last_clean":"2023-09-27T23:32:05.863310+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:32:05.863310+0000","last_undegraded":"2023-09-27T23:32:05.863310+0000","last_fullsized":"2023-09-27T23:32:05.863310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.874160+0000","last_change":"2023-09-27T23:32:05.870982+0000","last_active":"2023-09-27T23:32:05.874160+0000","last_peered":"2023-09-27T23:32:05.874160+0000","last_clean":"2023-09-27T23:32:05.874160+0000","last_became_active":"2023-09-27T23:32:05.870670+0000","last_became_peered":"2023-09-27T23:32:05.870670+0000","last_unstale":"2023-09-27T23:32:05.874160+0000","last_undegraded":"2023-09-27T23:32:05.874160+0000","last_fullsized":"2023-09-27T23:32:05.874160+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:51:06.714208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"128'6","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.885835+0000","last_change":"2023-09-27T23:32:05.486457+0000","last_active":"2023-09-27T23:32:05.885835+0000","last_peered":"2023-09-27T23:32:05.885835+0000","last_clean":"2023-09-27T23:32:05.885835+0000","last_became_active":"2023-09-27T23:32:05.485076+0000","last_became_peered":"2023-09-27T23:32:05.485076+0000","last_unstale":"2023-09-27T23:32:05.885835+0000","last_undegraded":"2023-09-27T23:32:05.885835+0000","last_fullsized":"2023-09-27T23:32:05.885835+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:00:41.676696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.848976+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:32:05.848976+0000","last_peered":"2023-09-27T23:32:05.848976+0000","last_clean":"2023-09-27T23:32:05.848976+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:32:05.848976+0000","last_undegraded":"2023-09-27T23:32:05.848976+0000","last_fullsized":"2023-09-27T23:32:05.848976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.869005+0000","last_change":"2023-09-27T23:32:05.862414+0000","last_active":"2023-09-27T23:32:05.869005+0000","last_peered":"2023-09-27T23:32:05.869005+0000","last_clean":"2023-09-27T23:32:05.869005+0000","last_became_active":"2023-09-27T23:32:05.862102+0000","last_became_peered":"2023-09-27T23:32:05.862102+0000","last_unstale":"2023-09-27T23:32:05.869005+0000","last_undegraded":"2023-09-27T23:32:05.869005+0000","last_fullsized":"2023-09-27T23:32:05.869005+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:05.445440+0000","last_clean_scrub_stamp":"2023-09-27T23:32:05.445440+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:40.054520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"128'3","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.881576+0000","last_change":"2023-09-27T23:32:05.483339+0000","last_active":"2023-09-27T23:32:05.881576+0000","last_peered":"2023-09-27T23:32:05.881576+0000","last_clean":"2023-09-27T23:32:05.881576+0000","last_became_active":"2023-09-27T23:32:05.483058+0000","last_became_peered":"2023-09-27T23:32:05.483058+0000","last_unstale":"2023-09-27T23:32:05.881576+0000","last_undegraded":"2023-09-27T23:32:05.881576+0000","last_fullsized":"2023-09-27T23:32:05.881576+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:32:04.433983+0000","last_clean_scrub_stamp":"2023-09-27T23:32:04.433983+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:45:45.766386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-27T23:32:05.847811+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:32:05.847811+0000","last_peered":"2023-09-27T23:32:05.847811+0000","last_clean":"2023-09-27T23:32:05.847811+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:32:05.847811+0000","last_undegraded":"2023-09-27T23:32:05.847811+0000","last_fullsized":"2023-09-27T23:32:05.847811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509663,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":25024,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":2191360,"data_stored":1415956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25624576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.52200000000000002,"15min":0.51100000000000001},"max":{"1min":1.2470000000000001,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1819999999999999,"5min":1.5489999999999999,"15min":1.712},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91900000000000004,"15min":0.89700000000000002},"min":{"1min":0.66000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1759999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95699999999999996,"15min":0.93899999999999995},"min":{"1min":0.69599999999999995,"5min":0.66400000000000003,"15min":0.28299999999999997},"max":{"1min":1.2430000000000001,"5min":1.337,"15min":1.7589999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.90200000000000002,"15min":0.91500000000000004},"min":{"1min":0.72299999999999998,"5min":0.56599999999999995,"15min":0.307},"max":{"1min":1.238,"5min":1.3129999999999999,"15min":2.681},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.97599999999999998,"15min":0.97899999999999998},"min":{"1min":0.77700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.405,"15min":1.718},"last":0.997}]},{"osd":3,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.93500000000000005},"min":{"1min":0.71099999999999997,"5min":0.63400000000000001,"15min":0.53200000000000003},"max":{"1min":1.151,"5min":1.204,"15min":1.421},"last":1.042},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.747,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.409,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.073}]},{"osd":4,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.117,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97499999999999998,"15min":0.98499999999999999},"min":{"1min":0.65200000000000002,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.3029999999999999,"5min":1.3240000000000001,"15min":2.8879999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.67000000000000004,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.3540000000000001,"5min":1.79,"15min":1.79},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.252,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:32:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1719999999999999,"15min":1.139},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.839,"15min":1.839},"last":1.246},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.107,"15min":1.1259999999999999},"min":{"1min":0.96599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.639,"5min":1.639,"15min":2.548},"last":1.218}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27100,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":25536,"kb_avail":94344740,"statfs":{"total":96636764160,"available":96609013760,"internally_reserved":0,"allocated":1593344,"data_stored":825070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26148864},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89200000000000002,"15min":0.94399999999999995},"min":{"1min":0.629,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.2130000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.78500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.246,"5min":1.321,"15min":1.5149999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.88},"min":{"1min":0.56299999999999994,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.214,"5min":1.248,"15min":1.323},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.258,"5min":1.403,"15min":1.403},"last":0.93899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.875},"min":{"1min":0.72099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2150000000000001,"5min":1.222,"15min":1.353},"last":1.238},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91100000000000003,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.246,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.92300000000000004},"min":{"1min":0.629,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2170000000000001,"5min":1.51,"15min":1.734},"last":1.073},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.72499999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1619999999999999,"5min":1.278,"15min":1.4970000000000001},"last":1.4279999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1100000000000001,"5min":1.3280000000000001,"15min":1.544},"last":1.274},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2929999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.81499999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2669999999999999,"5min":1.4450000000000001,"15min":1.5820000000000001},"last":1.343},{"interface":"front","average":{"1min":1.119,"5min":1.1319999999999999,"15min":1.1539999999999999},"min":{"1min":0.69099999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3169999999999999,"5min":1.4259999999999999,"15min":1.49},"last":1.121}]},{"osd":7,"last update":"Wed Sep 27 23:31:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.81399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3580000000000001,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1259999999999999,"15min":1.145},"min":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.423,"5min":1.423,"15min":1.6619999999999999},"last":1.4750000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509662,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35584,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1597440,"data_stored":825576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36438016},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.86799999999999999,"15min":0.871},"min":{"1min":0.61199999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.236,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.84299999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86199999999999999,"15min":0.88500000000000001},"min":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.526},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.874,"15min":0.88300000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2190000000000001,"5min":1.242,"15min":1.331},"last":0.79200000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90800000000000003,"15min":0.90600000000000003},"min":{"1min":0.73999999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3520000000000001,"5min":1.474,"15min":1.474},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97099999999999997,"15min":0.95099999999999996},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.5740000000000001},"last":0.91600000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.97599999999999998,"15min":0.95599999999999996},"min":{"1min":0.70499999999999996,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":1.23,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.999,"15min":1.004},"min":{"1min":0.80800000000000005,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.323,"5min":1.3720000000000001,"15min":1.6200000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.56200000000000006,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.3220000000000001,"5min":1.341,"15min":1.341},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.038,"15min":1.0209999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.613,"15min":1.613},"last":1.0669999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0089999999999999,"15min":1.01},"min":{"1min":0.70499999999999996,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.5580000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.1020000000000001},"min":{"1min":0.876,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.117}]},{"osd":7,"last update":"Wed Sep 27 23:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1619999999999999,"15min":1.1379999999999999},"min":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.446,"5min":1.665,"15min":1.6679999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.198,"15min":1.1890000000000001},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.508,"5min":1.528,"15min":1.633},"last":1.1519999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509663,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23000,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":94348840,"statfs":{"total":96636764160,"available":96613212160,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.876,"15min":0.89300000000000002},"min":{"1min":0.56200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.27,"5min":1.2789999999999999,"15min":1.3080000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.54400000000000004},"max":{"1min":1.212,"5min":1.343,"15min":1.45},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89500000000000002,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.262,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.73899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90200000000000002,"15min":0.91100000000000003},"min":{"1min":0.65600000000000003,"5min":0.57499999999999996,"15min":0.497},"max":{"1min":1.2410000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97999999999999998,"15min":0.96199999999999997},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.252,"5min":1.331,"15min":1.407},"last":0.91100000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92000000000000004},"min":{"1min":0.70999999999999996,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2150000000000001,"5min":1.272,"15min":1.2829999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.03,"15min":1.026},"min":{"1min":0.79900000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3660000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.94499999999999995,"15min":0.91700000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4610000000000001,"5min":1.609,"15min":1.609},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0229999999999999,"15min":1.01},"min":{"1min":0.66800000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0389999999999999,"15min":1.04},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.60099999999999998},"max":{"1min":1.3169999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.161,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.1499999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0940000000000001,"15min":1.091},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.427,"5min":1.4770000000000001,"15min":1.627},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.145,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.85899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7929999999999999,"15min":1.859},"last":1.181}]}]},{"osd":3,"up_from":15,"seq":64424509662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":26048,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":1605632,"data_stored":826606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26673152},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.70199999999999996},"min":{"1min":0.40400000000000003,"5min":0.218,"15min":0.218},"max":{"1min":1.012,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.2170000000000001,"15min":1.2869999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81100000000000005,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.109,"5min":1.323,"15min":1.3440000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.57499999999999996,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.446,"15min":1.5720000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.61499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.753,"5min":0.505,"15min":0.505},"max":{"1min":1.298,"5min":1.605,"15min":1.605},"last":1.163}]},{"osd":5,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0820000000000001,"15min":1.111},"min":{"1min":0.85099999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3999999999999999,"5min":1.4810000000000001,"15min":1.571},"last":1.232},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1160000000000001,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.714,"15min":1.764},"last":1.431}]},{"osd":6,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.105,"15min":1.1160000000000001},"min":{"1min":0.81200000000000006,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1950000000000001,"15min":1.2150000000000001},"min":{"1min":0.94899999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.488,"5min":1.6599999999999999,"15min":1.712},"last":1.3200000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:31:15 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.2370000000000001},"min":{"1min":0.79800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.173,"15min":1.202},"min":{"1min":0.88,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.143}]}]},{"osd":2,"up_from":15,"seq":64424509663,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22168,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":20608,"kb_avail":94349672,"statfs":{"total":96636764160,"available":96614064128,"internally_reserved":0,"allocated":1589248,"data_stored":825000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21102592},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.903},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.097,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":0.98499999999999999,"5min":1.228,"15min":1.391},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":1.2949999999999999,"5min":1.49,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.93400000000000005},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.399,"5min":1.399,"15min":1.444},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0309999999999999,"15min":1.0349999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.40600000000000003},"max":{"1min":1.417,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0629999999999999,"15min":1.0600000000000001},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":1.635,"15min":1.7390000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0269999999999999,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":0.89300000000000002,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.92500000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.722,"5min":1.722,"15min":1.9450000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.125,"15min":1.1539999999999999},"min":{"1min":0.83099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.623,"5min":1.623,"15min":2.0129999999999999},"last":1.0029999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:04 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2110000000000001,"15min":1.212},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1910000000000001,"15min":1.212},"min":{"1min":0.76000000000000001,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.5389999999999999,"5min":1.556,"15min":2.073},"last":1.0600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542369,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":35840,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":1597440,"data_stored":825124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36700160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.503,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.123,"15min":1.3560000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60299999999999998,"5min":0.52100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95199999999999996,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.72999999999999998,"15min":0.73299999999999998},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.2490000000000001,"15min":1.321},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.88200000000000001},"min":{"1min":0.60499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1439999999999999,"5min":1.3819999999999999,"15min":1.399},"last":0.83599999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.81299999999999994,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2110000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.024,"5min":1.696,"15min":1.696},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97999999999999998,"15min":0.97299999999999998},"min":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.282,"5min":1.464,"15min":1.464},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.121,"15min":1.131},"min":{"1min":0.75600000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.774},"last":1.196}]},{"osd":5,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0860000000000001},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5109999999999999,"5min":1.546,"15min":1.623},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.88900000000000001,"5min":0.78400000000000003,"15min":0.72499999999999998},"max":{"1min":1.395,"5min":1.589,"15min":1.589},"last":1.2569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.70999999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.089},{"interface":"front","average":{"1min":1.107,"5min":1.165,"15min":1.1850000000000001},"min":{"1min":0.93400000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.554,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.345}]},{"osd":7,"last update":"Wed Sep 27 23:31:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4179999999999999,"5min":1.877,"15min":1.877},"last":1.407},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.194,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.4410000000000001,"5min":1.5529999999999999,"15min":1.6339999999999999},"last":1.141}]}]},{"osd":0,"up_from":15,"seq":64424509664,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28464,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":26304,"kb_avail":94343376,"statfs":{"total":96636764160,"available":96607617024,"internally_reserved":0,"allocated":2203648,"data_stored":1417016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26935296},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.874,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.145,"5min":1.3,"15min":1.3},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91600000000000004,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.58899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":1.228,"15min":1.228},"last":0.63}]},{"osd":3,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.82899999999999996},"min":{"1min":0.5,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.012,"5min":1.1220000000000001,"15min":1.2769999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.53200000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.25,"5min":1.3160000000000001,"15min":1.325},"last":1.044}]},{"osd":4,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98099999999999998,"15min":1.0029999999999999},"min":{"1min":0.55300000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.47},"last":1.1100000000000001},{"interface":"front","average":{"1min":1,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.323,"5min":1.4239999999999999,"15min":1.587},"last":1.413}]},{"osd":5,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0700000000000001,"15min":1.091},"min":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5680000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.111,"15min":1.131},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.46,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.375}]},{"osd":6,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.113,"15min":1.155},"min":{"1min":0.71399999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3440000000000001,"5min":1.5269999999999999,"15min":1.5940000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.179,"15min":1.21},"min":{"1min":0.75700000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.502,"5min":1.5289999999999999,"15min":1.8300000000000001},"last":1.649}]},{"osd":7,"last update":"Wed Sep 27 23:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.204,"15min":1.228},"min":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.8109999999999999},"last":1.476},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1879999999999999,"15min":1.22},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.464,"5min":1.798,"15min":1.798},"last":1.603}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:32:14.235 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:32:14.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:32:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 Processor -- start 2023-09-27T23:32:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 -- start start 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c41514a0 0x7fb7c4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7c40cae60 0x7fb7c406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40cbee0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c406ba50 con 0x7fb7c40cae60 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7ca0a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4158530 con 0x7fb7c41514a0 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7c2ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7c40cae60 0x7fb7c406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7c3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb7c406b5d0 0x7fb7c415ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44490/0 (socket says 172.21.15.64:44490) 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.348+0000 7fb7c3fff700 1 -- 172.21.15.64:0/277488065 learned_addr learned my addr 172.21.15.64:0/277488065 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c37fe700 1 --2- 172.21.15.64:0/277488065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c41514a0 0x7fb7c4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c27fc700 1 -- 172.21.15.64:0/277488065 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592039344 0 0) 0x7fb7c40cbee0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c2ffd700 1 -- 172.21.15.64:0/277488065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c41514a0 msgr2=0x7fb7c4157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c2ffd700 1 --2- 172.21.15.64:0/277488065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c41514a0 0x7fb7c4157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c2ffd700 1 -- 172.21.15.64:0/277488065 >> v1:172.21.15.64:6789/0 conn(0x7fb7c406b5d0 legacy=0x7fb7c415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c2ffd700 1 -- 172.21.15.64:0/277488065 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c40cc370 con 0x7fb7c40cae60 2023-09-27T23:32:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c2ffd700 1 --2- 172.21.15.64:0/277488065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c40cae60 0x7fb7c406b090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40098e0 tx=0x7fb7b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3577a5292e1ab4a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c27fc700 1 -- 172.21.15.64:0/277488065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400b640 con 0x7fb7c40cae60 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c27fc700 1 -- 172.21.15.64:0/277488065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b400b7a0 con 0x7fb7c40cae60 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7c27fc700 1 -- 172.21.15.64:0/277488065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400bb40 con 0x7fb7c40cae60 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/277488065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c40cae60 msgr2=0x7fb7c406b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/277488065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c40cae60 0x7fb7c406b090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40098e0 tx=0x7fb7b40091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/277488065 shutdown_connections 2023-09-27T23:32:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/277488065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c40cae60 0x7fb7c406b090 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/277488065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c41514a0 0x7fb7c4157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/277488065 >> 172.21.15.64:0/277488065 conn(0x7fb7c40cc580 msgr2=0x7fb7c40cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/277488065 shutdown_connections 2023-09-27T23:32:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.352+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/277488065 wait complete. 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 Processor -- start 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 -- start start 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 0x7fb7c41383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c40cae60 0x7fb7c415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c41514a0 0x7fb7c40b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415c020 con 0x7fb7c406b5d0 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415b170 con 0x7fb7c41514a0 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415b410 con 0x7fb7c40cae60 2023-09-27T23:32:14.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 0x7fb7c41383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 0x7fb7c41383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50276/0 (socket says 172.21.15.64:50276) 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 -- 172.21.15.64:0/4159100591 learned_addr learned my addr 172.21.15.64:0/4159100591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c3fff700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c41514a0 0x7fb7c40b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c2ffd700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c40cae60 0x7fb7c415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 -- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c40cae60 msgr2=0x7fb7c415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c40cae60 0x7fb7c415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 -- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c41514a0 msgr2=0x7fb7c40b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c41514a0 0x7fb7c40b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 -- 172.21.15.64:0/4159100591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4009420 con 0x7fb7c406b5d0 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c2ffd700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c40cae60 0x7fb7c415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7c37fe700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 0x7fb7c41383f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800e930 tx=0x7fb7b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8013680 con 0x7fb7c406b5d0 2023-09-27T23:32:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b80137e0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c415b6f0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8013b80 con 0x7fb7c406b5d0 2023-09-27T23:32:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.356+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c415bcb0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.360+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb7b800c460 con 0x7fb7c406b5d0 2023-09-27T23:32:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.360+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c4071120 con 0x7fb7c406b5d0 2023-09-27T23:32:14.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.360+0000 7fb7ab7fe700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb7b0060a70 0x7fb7b0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.360+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb7b807d6b0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.360+0000 7fb7c2ffd700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb7b0060a70 0x7fb7b0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.364+0000 7fb7c2ffd700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb7b0060a70 0x7fb7b0062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40093f0 tx=0x7fb7b4017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.368+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb7b804d640 con 0x7fb7c406b5d0 2023-09-27T23:32:14.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.588+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb7c40cb2d0 con 0x7fb7c406b5d0 2023-09-27T23:32:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ab7fe700 1 -- 172.21.15.64:0/4159100591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb7b8050c60 con 0x7fb7c406b5d0 2023-09-27T23:32:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb7b0060a70 msgr2=0x7fb7b0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb7b0060a70 0x7fb7b0062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40093f0 tx=0x7fb7b4017000 comp rx=0 tx=0).stop 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 msgr2=0x7fb7c41383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 0x7fb7c41383f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800e930 tx=0x7fb7b80091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 shutdown_connections 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb7b0060a70 0x7fb7b0062f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c41514a0 0x7fb7c40b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7c40cae60 0x7fb7c415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 --2- 172.21.15.64:0/4159100591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7c406b5d0 0x7fb7c41383f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 >> 172.21.15.64:0/4159100591 conn(0x7fb7c40cc580 msgr2=0x7fb7c40abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 shutdown_connections 2023-09-27T23:32:14.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.600+0000 7fb7ca0a3700 1 -- 172.21.15.64:0/4159100591 wait complete. 2023-09-27T23:32:14.609 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:32:14.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:32:14.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c688e4700 1 Processor -- start 2023-09-27T23:32:14.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c688e4700 1 -- start start 2023-09-27T23:32:14.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c688e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c688e4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6406d990 con 0x7f2c64155960 2023-09-27T23:32:14.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c688e4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6406cd80 con 0x7f2c6406c7c0 2023-09-27T23:32:14.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c688e4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406d060 con 0x7f2c6406de20 2023-09-27T23:32:14.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c637fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2c64155960 0x7f2c6415a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34074/0 (socket says 172.21.15.64:34074) 2023-09-27T23:32:14.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c637fe700 1 -- 172.21.15.64:0/379745322 learned_addr learned my addr 172.21.15.64:0/379745322 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:14.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c62ffd700 1 --2- 172.21.15.64:0/379745322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546131869 0 0) 0x7f2c6406d990 con 0x7f2c64155960 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50003570 con 0x7f2c64155960 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251149702 0 0) 0x7f2c6406cd80 con 0x7f2c6406c7c0 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c6406d990 con 0x7f2c6406c7c0 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c62ffd700 1 -- 172.21.15.64:0/379745322 >> v1:172.21.15.123:6789/0 conn(0x7f2c6406c7c0 legacy=0x7f2c640aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 216126896 0 0) 0x7f2c50003570 con 0x7f2c64155960 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.724+0000 7f2c62ffd700 1 -- 172.21.15.64:0/379745322 >> v1:172.21.15.64:6789/0 conn(0x7f2c64155960 legacy=0x7f2c6415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c62ffd700 1 -- 172.21.15.64:0/379745322 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6406d340 con 0x7f2c6406de20 2023-09-27T23:32:14.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c62ffd700 1 --2- 172.21.15.64:0/379745322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c77f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009e90 tx=0x7f2c54009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6caba83240b1ec2a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400e070 con 0x7f2c6406de20 2023-09-27T23:32:14.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400b870 con 0x7f2c6406de20 2023-09-27T23:32:14.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c627fc700 1 -- 172.21.15.64:0/379745322 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c001470 con 0x7f2c6406de20 2023-09-27T23:32:14.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 -- 172.21.15.64:0/379745322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 msgr2=0x7f2c640c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 --2- 172.21.15.64:0/379745322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c77f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009e90 tx=0x7f2c54009780 comp rx=0 tx=0).stop 2023-09-27T23:32:14.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 -- 172.21.15.64:0/379745322 shutdown_connections 2023-09-27T23:32:14.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 --2- 172.21.15.64:0/379745322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c77f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 -- 172.21.15.64:0/379745322 >> 172.21.15.64:0/379745322 conn(0x7f2c64067bb0 msgr2=0x7f2c640cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:14.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 -- 172.21.15.64:0/379745322 shutdown_connections 2023-09-27T23:32:14.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 -- 172.21.15.64:0/379745322 wait complete. 2023-09-27T23:32:14.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 Processor -- start 2023-09-27T23:32:14.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.728+0000 7f2c688e4700 1 -- start start 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c6406c7c0 0x7f2c6414c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 0x7f2c640c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64155dd0 con 0x7f2c64155960 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c641560b0 con 0x7f2c6406c7c0 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64156390 con 0x7f2c6406de20 2023-09-27T23:32:14.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 0x7f2c640c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 0x7f2c640c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52506/0 (socket says 172.21.15.64:52506) 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 -- 172.21.15.64:0/3872755118 learned_addr learned my addr 172.21.15.64:0/3872755118 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c637fe700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c6406c7c0 0x7f2c6414c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c62ffd700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 -- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 msgr2=0x7f2c640c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 -- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c6406c7c0 msgr2=0x7f2c6414c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c6406c7c0 0x7f2c6414c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 -- 172.21.15.64:0/3872755118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54009a20 con 0x7f2c64155960 2023-09-27T23:32:14.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c63fff700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 0x7f2c640c9a40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c590 tx=0x7f2c5c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c016070 con 0x7f2c64155960 2023-09-27T23:32:14.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c011590 con 0x7f2c64155960 2023-09-27T23:32:14.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64156670 con 0x7f2c64155960 2023-09-27T23:32:14.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011970 con 0x7f2c64155960 2023-09-27T23:32:14.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.732+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6414c580 con 0x7f2c64155960 2023-09-27T23:32:14.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.736+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6414c740 con 0x7f2c64155960 2023-09-27T23:32:14.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.736+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2c5c012420 con 0x7f2c64155960 2023-09-27T23:32:14.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.736+0000 7f2c5b7fe700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c44060a70 0x7f2c44062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:14.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.736+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2c5c07cc10 con 0x7f2c64155960 2023-09-27T23:32:14.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.736+0000 7f2c637fe700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c44060a70 0x7f2c44062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:14.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.736+0000 7f2c637fe700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c44060a70 0x7f2c44062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c6414fdd0 tx=0x7f2c4c000bc0 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:14.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.744+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2c5c04cba0 con 0x7f2c64155960 2023-09-27T23:32:14.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.932+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c6404be40 con 0x7f2c64155960 2023-09-27T23:32:14.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.932+0000 7f2c5b7fe700 1 -- 172.21.15.64:0/3872755118 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f2c5c0501c0 con 0x7f2c64155960 2023-09-27T23:32:14.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:14.940 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:32:05.833505+0000","modified":"2023-09-27T23:32:08.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8151},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8151":{"gid":8151,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4228640199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4228640199},{"type":"v1","addr":"172.21.15.64:6835","nonce":4228640199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-09-27T23:32:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c44060a70 msgr2=0x7f2c44062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c44060a70 0x7f2c44062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c6414fdd0 tx=0x7f2c4c000bc0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 msgr2=0x7f2c640c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 0x7f2c640c9a40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c590 tx=0x7f2c5c007d80 comp rx=0 tx=0).stop 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 shutdown_connections 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2c44060a70 0x7f2c44062f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c64155960 0x7f2c640c9a40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c6406de20 0x7f2c640c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 --2- 172.21.15.64:0/3872755118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c6406c7c0 0x7f2c6414c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 >> 172.21.15.64:0/3872755118 conn(0x7f2c64067bb0 msgr2=0x7f2c640c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 shutdown_connections 2023-09-27T23:32:14.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:14.936+0000 7f2c688e4700 1 -- 172.21.15.64:0/3872755118 wait complete. 2023-09-27T23:32:14.944 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-09-27T23:32:15.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:32:05.833505+0000', 'modified': '2023-09-27T23:32:08.504707+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8151}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8151': {'gid': 8151, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/4228640199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 4228640199}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 4228640199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:32:15.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:32:15.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-27T23:32:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 Processor -- start 2023-09-27T23:32:15.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 -- start start 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 0x7fd94006b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006baa0 0x7fd9400b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94006b0d0 con 0x7fd940158cc0 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94006be90 con 0x7fd94006dc00 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9458b9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400b4790 con 0x7fd94006baa0 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9367fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 0x7fd94006b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd93effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd940158cc0 0x7fd94006d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34080/0 (socket says 172.21.15.64:34080) 2023-09-27T23:32:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd9367fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 0x7fd94006b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52512/0 (socket says 172.21.15.64:52512) 2023-09-27T23:32:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd93effd700 1 -- 172.21.15.64:0/2609562316 learned_addr learned my addr 172.21.15.64:0/2609562316 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.468+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/2609562316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006baa0 0x7fd9400b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93e7fc700 1 -- 172.21.15.64:0/2609562316 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322047991 0 0) 0x7fd94006b0d0 con 0x7fd940158cc0 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93e7fc700 1 -- 172.21.15.64:0/2609562316 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd920003570 con 0x7fd940158cc0 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93f7fe700 1 -- 172.21.15.64:0/2609562316 >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 msgr2=0x7fd94006b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/2609562316 >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 0x7fd94006b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93f7fe700 1 -- 172.21.15.64:0/2609562316 >> v1:172.21.15.64:6789/0 conn(0x7fd940158cc0 legacy=0x7fd94006d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93f7fe700 1 -- 172.21.15.64:0/2609562316 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94006a7e0 con 0x7fd94006baa0 2023-09-27T23:32:15.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2609562316 >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 0x7fd94006b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/2609562316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006baa0 0x7fd9400b3870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd938009e60 tx=0x7fd938009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6b6aeee4c593dd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93e7fc700 1 -- 172.21.15.64:0/2609562316 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800e070 con 0x7fd94006baa0 2023-09-27T23:32:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93e7fc700 1 -- 172.21.15.64:0/2609562316 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800bab0 con 0x7fd94006baa0 2023-09-27T23:32:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd93e7fc700 1 -- 172.21.15.64:0/2609562316 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800be30 con 0x7fd94006baa0 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 -- 172.21.15.64:0/2609562316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006baa0 msgr2=0x7fd9400b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2609562316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006baa0 0x7fd9400b3870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd938009e60 tx=0x7fd938009780 comp rx=0 tx=0).stop 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 -- 172.21.15.64:0/2609562316 shutdown_connections 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2609562316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006baa0 0x7fd9400b3870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2609562316 >> v2:172.21.15.64:3300/0 conn(0x7fd94006dc00 0x7fd94006b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 -- 172.21.15.64:0/2609562316 >> 172.21.15.64:0/2609562316 conn(0x7fd940067bb0 msgr2=0x7fd9400cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 -- 172.21.15.64:0/2609562316 shutdown_connections 2023-09-27T23:32:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.472+0000 7fd9458b9700 1 -- 172.21.15.64:0/2609562316 wait complete. 2023-09-27T23:32:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 Processor -- start 2023-09-27T23:32:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 -- start start 2023-09-27T23:32:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 0x7fd9401593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006dc00 0x7fd940154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd940158cc0 0x7fd9400ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9401599a0 con 0x7fd94006baa0 2023-09-27T23:32:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940159c80 con 0x7fd940158cc0 2023-09-27T23:32:15.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9458b9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940159f60 con 0x7fd94006dc00 2023-09-27T23:32:15.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 0x7fd9401593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 0x7fd9401593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52528/0 (socket says 172.21.15.64:52528) 2023-09-27T23:32:15.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd93effd700 1 -- 172.21.15.64:0/2175709939 learned_addr learned my addr 172.21.15.64:0/2175709939 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006dc00 0x7fd940154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.476+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd940158cc0 0x7fd9400ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd93effd700 1 -- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006dc00 msgr2=0x7fd940154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd93effd700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006dc00 0x7fd940154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd93effd700 1 -- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd940158cc0 msgr2=0x7fd9400ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd93effd700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd940158cc0 0x7fd9400ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd93effd700 1 -- 172.21.15.64:0/2175709939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd928009420 con 0x7fd94006baa0 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd93effd700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 0x7fd9401593c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd9300011e0 tx=0x7fd930007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9377fe700 1 -- 172.21.15.64:0/2175709939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930005b30 con 0x7fd94006baa0 2023-09-27T23:32:15.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9377fe700 1 -- 172.21.15.64:0/2175709939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930005c90 con 0x7fd94006baa0 2023-09-27T23:32:15.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd938009a20 con 0x7fd94006baa0 2023-09-27T23:32:15.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9377fe700 1 -- 172.21.15.64:0/2175709939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930007320 con 0x7fd94006baa0 2023-09-27T23:32:15.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94015d000 con 0x7fd94006baa0 2023-09-27T23:32:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9377fe700 1 -- 172.21.15.64:0/2175709939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd930007690 con 0x7fd94006baa0 2023-09-27T23:32:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.480+0000 7fd9377fe700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd924060a70 0x7fd924062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd9377fe700 1 -- 172.21.15.64:0/2175709939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd93007c540 con 0x7fd94006baa0 2023-09-27T23:32:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd924060a70 0x7fd924062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd924060a70 0x7fd924062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9400b1d20 tx=0x7fd9280091a0 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 Processor -- start 2023-09-27T23:32:15.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 -- start start 2023-09-27T23:32:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 0x2136e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9710 0x2137390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x21378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2064f30 con 0x1eb9ae0 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2060ed0 con 0x1f04790 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.484+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x20270b0 con 0x1eb9710 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 0x2136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 0x2136e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52530/0 (socket says 172.21.15.64:52530) 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 -- 172.21.15.64:0/491911197 learned_addr learned my addr 172.21.15.64:0/491911197 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd9367fc700 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9710 0x2137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x21378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 -- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9710 msgr2=0x2137390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9710 0x2137390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 -- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 msgr2=0x21378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x21378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 -- 172.21.15.64:0/491911197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f059e0 con 0x1eb9ae0 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x21378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd93effd700 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 0x2136e50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd9300070a0 tx=0x7fd93004c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d39205b0e1447b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd92ffff700 1 -- 172.21.15.64:0/491911197 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930057170 con 0x1eb9ae0 2023-09-27T23:32:15.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd92ffff700 1 -- 172.21.15.64:0/491911197 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930005e00 con 0x1eb9ae0 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd92ffff700 1 -- 172.21.15.64:0/491911197 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93004e2d0 con 0x1eb9ae0 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 -- 172.21.15.64:0/491911197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 msgr2=0x2136e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 0x2136e50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd9300070a0 tx=0x7fd93004c600 comp rx=0 tx=0).stop 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 -- 172.21.15.64:0/491911197 shutdown_connections 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x21378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9710 0x2137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 --2- 172.21.15.64:0/491911197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9ae0 0x2136e50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 -- 172.21.15.64:0/491911197 >> 172.21.15.64:0/491911197 conn(0x1eb9eb0 msgr2=0x20f3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 -- 172.21.15.64:0/491911197 shutdown_connections 2023-09-27T23:32:15.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 -- 172.21.15.64:0/491911197 wait complete. 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 -- start start 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.488+0000 7fd947b62740 1 Processor -- start 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x20f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9ae0 0x211cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x211d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2026d50 con 0x1eb9710 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fff970 con 0x1f04790 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x2064100 con 0x1eb9ae0 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 10 client.0 _ll_register_callbacks cb 0x20aab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:32:15.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x20f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x20f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52532/0 (socket says 172.21.15.64:52532) 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 -- 172.21.15.64:0/2553080858 learned_addr learned my addr 172.21.15.64:0/2553080858 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9ae0 0x211cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x211d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 -- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9ae0 msgr2=0x211cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9ae0 0x211cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 -- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 msgr2=0x211d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x211d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 -- 172.21.15.64:0/2553080858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94015d000 con 0x1eb9710 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x211d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd93effd700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x20f3740 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd93004c690 tx=0x7fd930062000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b154d1f329f6cc54 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 1 -- 172.21.15.64:0/2553080858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930069040 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 1 -- 172.21.15.64:0/2553080858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd930057170 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1fffcc0 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 1 -- 172.21.15.64:0/2553080858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd93007a260 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 10 client.8316 fetch_fsmap learned FSMap version 46 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x202f370 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 1 -- 172.21.15.64:0/2553080858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1fffcc0 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 1 -- 172.21.15.64:0/2553080858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd930050430 con 0x1eb9710 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 10 client.8316 fetch_fsmap finished waiting for FSMap version 46 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 10 client.8316 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 20 client.8316 populate_metadata read hostname 'smithi064' 2023-09-27T23:32:15.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 4 client.8316 mds_command: new command op to 8151 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:32:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 --> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x20033e0 con 0x21436b0 2023-09-27T23:32:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 10 client.8316 ms_handle_connect on v2:172.21.15.64:6834/4228640199 2023-09-27T23:32:15.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd9367fc700 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2145a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e36f48a9b079ae4 server_cookie=16983dcbefa199cc in_seq=0 out_seq=0 2023-09-27T23:32:15.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 1 -- 172.21.15.64:0/2553080858 <== mds.0 v2:172.21.15.64:6834/4228640199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x20033e0 con 0x21436b0 2023-09-27T23:32:15.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd92d7fa700 10 client.8316 handle_command_reply: tid=0 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.492+0000 7fd947b62740 1 client.8316 shutdown 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 msgr2=0x20f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x20f3740 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd93004c690 tx=0x7fd930062000 comp rx=0 tx=0).stop 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 shutdown_connections 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2145a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1f04790 0x211d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd92d7fa700 0 client.8316 ms_handle_reset on v2:172.21.15.64:6834/4228640199 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1eb9ae0 0x211cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 --2- 172.21.15.64:0/2553080858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x20f3740 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 >> 172.21.15.64:0/2553080858 conn(0x1eb9eb0 msgr2=0x20ebed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 shutdown_connections 2023-09-27T23:32:15.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 1 -- 172.21.15.64:0/2553080858 wait complete. 2023-09-27T23:32:15.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.496+0000 7fd947b62740 20 client.8316 trim_cache size 0 max 16384 2023-09-27T23:32:15.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 Processor -- start 2023-09-27T23:32:15.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 -- start start 2023-09-27T23:32:15.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2146510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2034f40 con 0x1eb9710 2023-09-27T23:32:15.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x20cebb0 con 0x1eb9ae0 2023-09-27T23:32:15.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x205cb50 con 0x1f04790 2023-09-27T23:32:15.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52538/0 (socket says 172.21.15.64:52538) 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 -- 172.21.15.64:0/69247571 learned_addr learned my addr 172.21.15.64:0/69247571 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd9367fc700 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 -- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 msgr2=0x2146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 -- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 msgr2=0x2145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2145fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 -- 172.21.15.64:0/69247571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd940154490 con 0x1eb9710 2023-09-27T23:32:15.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd93effd700 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2145a90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd93000cb40 tx=0x7fd930075000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71df4eb92fd72986 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd934ff9700 1 -- 172.21.15.64:0/69247571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930069040 con 0x1eb9710 2023-09-27T23:32:15.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd934ff9700 1 -- 172.21.15.64:0/69247571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93007a500 con 0x1eb9710 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.516+0000 7fd934ff9700 1 -- 172.21.15.64:0/69247571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9300508d0 con 0x1eb9710 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- 172.21.15.64:0/69247571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 msgr2=0x2145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2145a90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd93000cb40 tx=0x7fd930075000 comp rx=0 tx=0).stop 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- 172.21.15.64:0/69247571 shutdown_connections 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- 172.21.15.64:0/69247571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2145a90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- 172.21.15.64:0/69247571 >> 172.21.15.64:0/69247571 conn(0x1eb9eb0 msgr2=0x20ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- 172.21.15.64:0/69247571 shutdown_connections 2023-09-27T23:32:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- 172.21.15.64:0/69247571 wait complete. 2023-09-27T23:32:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- start start 2023-09-27T23:32:15.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 Processor -- start 2023-09-27T23:32:15.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2118540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2118a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2118fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2036550 con 0x1eb9710 2023-09-27T23:32:15.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x20883b0 con 0x1eb9ae0 2023-09-27T23:32:15.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x2065490 con 0x1f04790 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd947b62740 10 client.0 _ll_register_callbacks cb 0x1f7d0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2118540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2118540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52554/0 (socket says 172.21.15.64:52554) 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd9367fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2118a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 -- 172.21.15.64:0/1656785080 learned_addr learned my addr 172.21.15.64:0/1656785080 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93f7fe700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2118fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 -- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 msgr2=0x2118fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2118fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 -- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 msgr2=0x2118a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2118a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 -- 172.21.15.64:0/1656785080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x202f370 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd9367fc700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd93effd700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2118540 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd93004c690 tx=0x7fd930009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d51fcc80e2322d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd92d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd92d7fa700 1 -- 172.21.15.64:0/1656785080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930069040 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.520+0000 7fd92d7fa700 1 -- 172.21.15.64:0/1656785080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd930053420 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd930005e00 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 1 -- 172.21.15.64:0/1656785080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9300508d0 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 10 client.8322 fetch_fsmap learned FSMap version 46 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x2015d80 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 1 -- 172.21.15.64:0/1656785080 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930005e00 con 0x1eb9710 2023-09-27T23:32:15.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 1 -- 172.21.15.64:0/1656785080 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd93005f050 con 0x1eb9710 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 10 client.8322 fetch_fsmap finished waiting for FSMap version 46 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 10 client.8322 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 20 client.8322 populate_metadata read hostname 'smithi064' 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2119950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 4 client.8322 mds_command: new command op to 8151 tid=0[{"prefix": "status"}] 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 --> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2035410 con 0x21436b0 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9367fc700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2119950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 10 client.8322 ms_handle_connect on v2:172.21.15.64:6834/4228640199 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9367fc700 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2119950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=205532c5f19c620d server_cookie=ffaab8d243f0fcbc in_seq=0 out_seq=0 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 1 -- 172.21.15.64:0/1656785080 <== mds.0 v2:172.21.15.64:6834/4228640199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x2035410 con 0x21436b0 2023-09-27T23:32:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 10 client.8322 handle_command_reply: tid=0 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 client.8322 shutdown 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 msgr2=0x2118540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2118540 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd93004c690 tx=0x7fd930009040 comp rx=0 tx=0).stop 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 shutdown_connections 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:6834/4228640199,v1:172.21.15.64:6835/4228640199] conn(0x21436b0 0x2119950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1f04790 0x2118fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1eb9ae0 0x2118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 --2- 172.21.15.64:0/1656785080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1eb9710 0x2118540 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 >> 172.21.15.64:0/1656785080 conn(0x1eb9eb0 msgr2=0x211bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd92d7fa700 0 client.8322 ms_handle_reset on v2:172.21.15.64:6834/4228640199 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 shutdown_connections 2023-09-27T23:32:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 1 -- 172.21.15.64:0/1656785080 wait complete. 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd947b62740 20 client.8322 trim_cache size 0 max 16384 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 8151, 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:32:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.6921473490000007, 2023-09-27T23:32:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 46, 2023-09-27T23:32:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 129, 2023-09-27T23:32:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 129, 2023-09-27T23:32:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 14.235594130000001 2023-09-27T23:32:15.537 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd924060a70 msgr2=0x7fd924062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd924060a70 0x7fd924062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9400b1d20 tx=0x7fd9280091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 msgr2=0x7fd9401593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 0x7fd9401593c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd9300011e0 tx=0x7fd930007df0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 shutdown_connections 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd924060a70 0x7fd924062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd940158cc0 0x7fd9400ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd94006dc00 0x7fd940154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 --2- 172.21.15.64:0/2175709939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd94006baa0 0x7fd9401593c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 >> 172.21.15.64:0/2175709939 conn(0x7fd940067bb0 msgr2=0x7fd940157c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.524+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 shutdown_connections 2023-09-27T23:32:15.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.528+0000 7fd9458b9700 1 -- 172.21.15.64:0/2175709939 wait complete. 2023-09-27T23:32:15.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:32:15.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 Processor -- start 2023-09-27T23:32:15.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 -- start start 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60e80c6900 0x7f60e80c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e806b900 0x7f60e806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e806b470 con 0x7f60e815cc20 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e815d7b0 con 0x7f60e80c6900 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60ecee5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8161d10 con 0x7f60e806b900 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60e80c6900 0x7f60e80c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60e80c6900 0x7f60e80c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52562/0 (socket says 172.21.15.64:52562) 2023-09-27T23:32:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 -- 172.21.15.64:0/3612505732 learned_addr learned my addr 172.21.15.64:0/3612505732 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e6ffd700 1 --2- 172.21.15.64:0/3612505732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e806b900 0x7f60e806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 -- 172.21.15.64:0/3612505732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e806b900 msgr2=0x7f60e806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 --2- 172.21.15.64:0/3612505732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e806b900 0x7f60e806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 -- 172.21.15.64:0/3612505732 >> v1:172.21.15.123:6789/0 conn(0x7f60e815cc20 legacy=0x7f60e815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e67fc700 1 -- 172.21.15.64:0/3612505732 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255243852 0 0) 0x7f60e806b470 con 0x7f60e815cc20 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.656+0000 7f60e77fe700 1 -- 172.21.15.64:0/3612505732 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e806bd70 con 0x7f60e80c6900 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60e77fe700 1 --2- 172.21.15.64:0/3612505732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e80c6cd0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f60d80099e0 tx=0x7f60d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b7e0ff0d4d3f5a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60e67fc700 1 -- 172.21.15.64:0/3612505732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800b820 con 0x7f60e80c6900 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60e67fc700 1 -- 172.21.15.64:0/3612505732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d800b980 con 0x7f60e80c6900 2023-09-27T23:32:15.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60e67fc700 1 -- 172.21.15.64:0/3612505732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800bd20 con 0x7f60e80c6900 2023-09-27T23:32:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 -- 172.21.15.64:0/3612505732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 msgr2=0x7f60e80c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 --2- 172.21.15.64:0/3612505732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e80c6cd0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f60d80099e0 tx=0x7f60d8009250 comp rx=0 tx=0).stop 2023-09-27T23:32:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 -- 172.21.15.64:0/3612505732 shutdown_connections 2023-09-27T23:32:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 --2- 172.21.15.64:0/3612505732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e806b900 0x7f60e806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 --2- 172.21.15.64:0/3612505732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e80c6cd0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 -- 172.21.15.64:0/3612505732 >> 172.21.15.64:0/3612505732 conn(0x7f60e8067bb0 msgr2=0x7f60e80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 -- 172.21.15.64:0/3612505732 shutdown_connections 2023-09-27T23:32:15.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.660+0000 7f60ecee5700 1 -- 172.21.15.64:0/3612505732 wait complete. 2023-09-27T23:32:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 Processor -- start 2023-09-27T23:32:15.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 -- start start 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e806b900 0x7f60e8170650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e815cc20 0x7f60e816db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e815f0e0 con 0x7f60e80c6900 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e815f3c0 con 0x7f60e806b900 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60ecee5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e815f6a0 con 0x7f60e815cc20 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52566/0 (socket says 172.21.15.64:52566) 2023-09-27T23:32:15.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 -- 172.21.15.64:0/1152861720 learned_addr learned my addr 172.21.15.64:0/1152861720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e77fe700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e806b900 0x7f60e8170650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e7fff700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e815cc20 0x7f60e816db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 -- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e815cc20 msgr2=0x7f60e816db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e815cc20 0x7f60e816db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 -- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e806b900 msgr2=0x7f60e8170650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e806b900 0x7f60e8170650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.664+0000 7f60e6ffd700 1 -- 172.21.15.64:0/1152861720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80094f0 con 0x7f60e80c6900 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60e6ffd700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e815e5d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009ef0 tx=0x7f60d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d000cc00 con 0x7f60e80c6900 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d000f040 con 0x7f60e80c6900 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e815f980 con 0x7f60e80c6900 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0014480 con 0x7f60e80c6900 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e8170b90 con 0x7f60e80c6900 2023-09-27T23:32:15.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e804be40 con 0x7f60e80c6900 2023-09-27T23:32:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f60d00147f0 con 0x7f60e80c6900 2023-09-27T23:32:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60df7fe700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f60c8060aa0 0x7f60c8062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f60d007dc30 con 0x7f60e80c6900 2023-09-27T23:32:15.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.668+0000 7f60e77fe700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f60c8060aa0 0x7f60c8062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:15.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.672+0000 7f60e77fe700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f60c8060aa0 0x7f60c8062f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60d80099e0 tx=0x7f60d8017000 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.680+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f60d004dab0 con 0x7f60e80c6900 2023-09-27T23:32:15.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.884+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60e816d880 con 0x7f60e80c6900 2023-09-27T23:32:15.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.884+0000 7f60df7fe700 1 -- 172.21.15.64:0/1152861720 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f60d00510d0 con 0x7f60e80c6900 2023-09-27T23:32:15.890 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f60c8060aa0 msgr2=0x7f60c8062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f60c8060aa0 0x7f60c8062f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60d80099e0 tx=0x7f60d8017000 comp rx=0 tx=0).stop 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 msgr2=0x7f60e815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e815e5d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009ef0 tx=0x7f60d000b040 comp rx=0 tx=0).stop 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 shutdown_connections 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f60c8060aa0 0x7f60c8062f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60e815cc20 0x7f60e816db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60e80c6900 0x7f60e815e5d0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 --2- 172.21.15.64:0/1152861720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e806b900 0x7f60e8170650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 >> 172.21.15.64:0/1152861720 conn(0x7f60e8067bb0 msgr2=0x7f60e8159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:15.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 shutdown_connections 2023-09-27T23:32:15.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:15.888+0000 7f60ecee5700 1 -- 172.21.15.64:0/1152861720 wait complete. 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:32:15.905 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:32:15.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:32:15.906 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:32:15.913 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:32:15.914 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41448sec preferred_lft 41448sec 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:32:15.915 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:32:15.916 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:32:15.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:32:15.917 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:32:15.930 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:32:15.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:32:15.932 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:32:15.984 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:32:15.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:32:15.986 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:32:15.986 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:32:15.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:32:15.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:32:15.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:32:15.986 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:32:16.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:32:16.026 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:32:16.032 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:32:16.032 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:32:16.032 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:32:16.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:32:16.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:32:16.046 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:32:16.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:32:16.046 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:32:16.052 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:32:16.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:32:16.052 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:32:16.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:32:16.071 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:32:16.071 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:32:16.086 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:32:16.087 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:32:16.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:16.088 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:32:16.135 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:32:16.135 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:32:16.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:32:16.153 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:32:16.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:32:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:32:16.167 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:32:16.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:16.168 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:32:16.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:32:16.220+0000 7fd94fef5380 -1 init, newargv = 0x55b88f5f1240 newargc=15 2023-09-27T23:32:16.225 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[54820]: starting ceph client 2023-09-27T23:32:16.234 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[54820]: starting fuse 2023-09-27T23:32:17.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:32:17.293 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:32:17.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:32:17.309 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:32:17.310 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:32:17.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:17.310 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:32:17.359 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:32:17.359 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:32:17.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:32:17.359 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:32:17.359 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:32:17.359 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:32:17.359 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:32:17.359 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:32:17.360 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:32:17.361 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:32:17.449 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.54820.asok 2023-09-27T23:32:17.633 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54820.asok 2023-09-27T23:32:17.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:17.633 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54820.asok status 2023-09-27T23:32:17.728 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "54820", 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:32:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 8224, 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 8224 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:32:17.730 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2612544970 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2612544970 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.8224 192.168.0.1:0/2612544970", 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2612544970", 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:32:17.731 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 46, 2023-09-27T23:32:17.732 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 129, 2023-09-27T23:32:17.732 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:32:17.732 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:32:17.732 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:32:17.732 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:32:17.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:17.738 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:32:17.738 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:32:17.744 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:32:17.744 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:32:17.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:17.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:32:17.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:32:17.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 Processor -- start 2023-09-27T23:32:17.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 -- start start 2023-09-27T23:32:17.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e5015d050 0x7f5e5006a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:17.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e50160e60 0x7f5e500af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:17.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5006ba50 con 0x7f5e500afd20 2023-09-27T23:32:17.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5006bee0 con 0x7f5e5015d050 2023-09-27T23:32:17.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e549d6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e500af600 con 0x7f5e50160e60 2023-09-27T23:32:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e4e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e5015d050 0x7f5e5006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e4ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5e500afd20 0x7f5e5006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34096/0 (socket says 172.21.15.64:34096) 2023-09-27T23:32:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e4ed9d700 1 -- 172.21.15.64:0/929525755 learned_addr learned my addr 172.21.15.64:0/929525755 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.904+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/929525755 >> v2:172.21.15.123:3300/0 conn(0x7f5e50160e60 0x7f5e500af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4d59a700 1 -- 172.21.15.64:0/929525755 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579432874 0 0) 0x7f5e5006ba50 con 0x7f5e500afd20 2023-09-27T23:32:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4d59a700 1 -- 172.21.15.64:0/929525755 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003570 con 0x7f5e500afd20 2023-09-27T23:32:17.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4e59c700 1 -- 172.21.15.64:0/929525755 >> v2:172.21.15.123:3300/0 conn(0x7f5e50160e60 msgr2=0x7f5e500af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/929525755 >> v2:172.21.15.123:3300/0 conn(0x7f5e50160e60 0x7f5e500af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4e59c700 1 -- 172.21.15.64:0/929525755 >> v1:172.21.15.64:6789/0 conn(0x7f5e500afd20 legacy=0x7f5e5006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4e59c700 1 -- 172.21.15.64:0/929525755 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500b0140 con 0x7f5e5015d050 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/929525755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e5015d050 0x7f5e5006a740 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5e5006a150 tx=0x7f5e440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efedda7a91919f27 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4d59a700 1 -- 172.21.15.64:0/929525755 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44011630 con 0x7f5e5015d050 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4d59a700 1 -- 172.21.15.64:0/929525755 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e440117b0 con 0x7f5e5015d050 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e4d59a700 1 -- 172.21.15.64:0/929525755 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44011b70 con 0x7f5e5015d050 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 -- 172.21.15.64:0/929525755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e5015d050 msgr2=0x7f5e5006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 --2- 172.21.15.64:0/929525755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e5015d050 0x7f5e5006a740 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5e5006a150 tx=0x7f5e440091a0 comp rx=0 tx=0).stop 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 -- 172.21.15.64:0/929525755 shutdown_connections 2023-09-27T23:32:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 --2- 172.21.15.64:0/929525755 >> v2:172.21.15.123:3300/0 conn(0x7f5e50160e60 0x7f5e500af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 --2- 172.21.15.64:0/929525755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e5015d050 0x7f5e5006a740 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 -- 172.21.15.64:0/929525755 >> 172.21.15.64:0/929525755 conn(0x7f5e50067b60 msgr2=0x7f5e500ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 -- 172.21.15.64:0/929525755 shutdown_connections 2023-09-27T23:32:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 -- 172.21.15.64:0/929525755 wait complete. 2023-09-27T23:32:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 Processor -- start 2023-09-27T23:32:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.908+0000 7f5e549d6700 1 -- start start 2023-09-27T23:32:17.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 0x7f5e50149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:17.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e5015d050 0x7f5e500c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e50160e60 0x7f5e500c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5014c0c0 con 0x7f5e500afd20 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5014b4b0 con 0x7f5e5015d050 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5014b790 con 0x7f5e50160e60 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 0x7f5e50149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 0x7f5e50149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52580/0 (socket says 172.21.15.64:52580) 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 -- 172.21.15.64:0/2183737488 learned_addr learned my addr 172.21.15.64:0/2183737488 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e5015d050 0x7f5e500c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e50160e60 0x7f5e500c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 -- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e50160e60 msgr2=0x7f5e500c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e50160e60 0x7f5e500c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:17.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 -- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e5015d050 msgr2=0x7f5e500c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e5015d050 0x7f5e500c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 -- 172.21.15.64:0/2183737488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44009420 con 0x7f5e500afd20 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e5015d050 0x7f5e500c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e50160e60 0x7f5e500c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 0x7f5e50149000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5e44013000 tx=0x7f5e44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4401f070 con 0x7f5e500afd20 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44000c90 con 0x7f5e500afd20 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5014ba70 con 0x7f5e500afd20 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44011b50 con 0x7f5e500afd20 2023-09-27T23:32:17.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e50149540 con 0x7f5e500afd20 2023-09-27T23:32:17.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.912+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e50071120 con 0x7f5e500afd20 2023-09-27T23:32:17.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.916+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5e44023070 con 0x7f5e500afd20 2023-09-27T23:32:17.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.916+0000 7f5e3e7fc700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e30060a70 0x7f5e30062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:17.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.916+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5e4407e910 con 0x7f5e500afd20 2023-09-27T23:32:17.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.916+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e30060a70 0x7f5e30062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:17.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.916+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e30060a70 0x7f5e30062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e500af9f0 tx=0x7f5e40008040 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:17.924+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5e4404e9c0 con 0x7f5e500afd20 2023-09-27T23:32:18.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.108+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e5014c200 con 0x7f5e30060a70 2023-09-27T23:32:18.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.112+0000 7f5e3e7fc700 1 -- 172.21.15.64:0/2183737488 <== mgr.6971 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5e40009aa0 con 0x7f5e30060a70 2023-09-27T23:32:18.118 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:32:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:32:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:32:18.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:32:18.119 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:32:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e30060a70 msgr2=0x7f5e30062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e30060a70 0x7f5e30062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e500af9f0 tx=0x7f5e40008040 comp rx=0 tx=0).stop 2023-09-27T23:32:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 msgr2=0x7f5e50149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 0x7f5e50149000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5e44013000 tx=0x7f5e44017000 comp rx=0 tx=0).stop 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 shutdown_connections 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e30060a70 0x7f5e30062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e50160e60 0x7f5e500c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e5015d050 0x7f5e500c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 --2- 172.21.15.64:0/2183737488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e500afd20 0x7f5e50149000 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 >> 172.21.15.64:0/2183737488 conn(0x7f5e50067b60 msgr2=0x7f5e500c7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 shutdown_connections 2023-09-27T23:32:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.116+0000 7f5e549d6700 1 -- 172.21.15.64:0/2183737488 wait complete. 2023-09-27T23:32:18.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:32:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 Processor -- start 2023-09-27T23:32:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- start start 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6db4158fd0 0x7f6db406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6db40cc610 0x7f6db40c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db406d750 con 0x7f6db40cc1e0 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40cbaf0 con 0x7f6db4158fd0 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db406e120 con 0x7f6db40cc610 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6db4158fd0 0x7f6db406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6db4158fd0 0x7f6db406dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52588/0 (socket says 172.21.15.64:52588) 2023-09-27T23:32:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 -- 172.21.15.64:0/252571452 learned_addr learned my addr 172.21.15.64:0/252571452 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:18.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db96ed700 1 --2- 172.21.15.64:0/252571452 >> v2:172.21.15.123:3300/0 conn(0x7f6db40cc610 0x7f6db40c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 -- 172.21.15.64:0/252571452 >> v2:172.21.15.123:3300/0 conn(0x7f6db40cc610 msgr2=0x7f6db40c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 --2- 172.21.15.64:0/252571452 >> v2:172.21.15.123:3300/0 conn(0x7f6db40cc610 0x7f6db40c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db37fe700 1 -- 172.21.15.64:0/252571452 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945533282 0 0) 0x7f6db406d750 con 0x7f6db40cc1e0 2023-09-27T23:32:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 -- 172.21.15.64:0/252571452 >> v1:172.21.15.123:6789/0 conn(0x7f6db40cc1e0 legacy=0x7f6db40c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 -- 172.21.15.64:0/252571452 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db406cb40 con 0x7f6db4158fd0 2023-09-27T23:32:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db96ed700 1 --2- 172.21.15.64:0/252571452 >> v2:172.21.15.123:3300/0 conn(0x7f6db40cc610 0x7f6db40c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db8eec700 1 --2- 172.21.15.64:0/252571452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158fd0 0x7f6db406dbe0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6db40b0780 tx=0x7f6da8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70599769d262be21 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db37fe700 1 -- 172.21.15.64:0/252571452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800b860 con 0x7f6db4158fd0 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db37fe700 1 -- 172.21.15.64:0/252571452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800b9c0 con 0x7f6db4158fd0 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db37fe700 1 -- 172.21.15.64:0/252571452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800bd60 con 0x7f6db4158fd0 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- 172.21.15.64:0/252571452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158fd0 msgr2=0x7f6db406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 --2- 172.21.15.64:0/252571452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158fd0 0x7f6db406dbe0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6db40b0780 tx=0x7f6da8009250 comp rx=0 tx=0).stop 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- 172.21.15.64:0/252571452 shutdown_connections 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 --2- 172.21.15.64:0/252571452 >> v2:172.21.15.123:3300/0 conn(0x7f6db40cc610 0x7f6db40c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 --2- 172.21.15.64:0/252571452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db4158fd0 0x7f6db406dbe0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- 172.21.15.64:0/252571452 >> 172.21.15.64:0/252571452 conn(0x7f6db40cdc00 msgr2=0x7f6db40ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- 172.21.15.64:0/252571452 shutdown_connections 2023-09-27T23:32:18.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.236+0000 7f6db9eee700 1 -- 172.21.15.64:0/252571452 wait complete. 2023-09-27T23:32:18.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 Processor -- start 2023-09-27T23:32:18.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 -- start start 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 0x7f6db4166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db40cc610 0x7f6db4169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6db4158fd0 0x7f6db41689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40af360 con 0x7f6db40cc1e0 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db416b510 con 0x7f6db40cc610 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db9eee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db416a210 con 0x7f6db4158fd0 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db8eec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 0x7f6db4166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db8eec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 0x7f6db4166ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52594/0 (socket says 172.21.15.64:52594) 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db8eec700 1 -- 172.21.15.64:0/3457710776 learned_addr learned my addr 172.21.15.64:0/3457710776 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.240+0000 7f6db3fff700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db40cc610 0x7f6db4169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:18.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db96ed700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6db4158fd0 0x7f6db41689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:18.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db8eec700 1 -- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6db4158fd0 msgr2=0x7f6db41689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:18.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db8eec700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6db4158fd0 0x7f6db41689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db8eec700 1 -- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db40cc610 msgr2=0x7f6db4169cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:18.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db8eec700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db40cc610 0x7f6db4169cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:18.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db8eec700 1 -- 172.21.15.64:0/3457710776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80094f0 con 0x7f6db40cc1e0 2023-09-27T23:32:18.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db8eec700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 0x7f6db4166ee0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6da80090b0 tx=0x7f6da8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8026480 con 0x7f6db40cc1e0 2023-09-27T23:32:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da801f070 con 0x7f6db40cc1e0 2023-09-27T23:32:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db416a900 con 0x7f6db40cc1e0 2023-09-27T23:32:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8026830 con 0x7f6db40cc1e0 2023-09-27T23:32:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db416aec0 con 0x7f6db40cc1e0 2023-09-27T23:32:18.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db404be40 con 0x7f6db40cc1e0 2023-09-27T23:32:18.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.244+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 49) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6da801a430 con 0x7f6db40cc1e0 2023-09-27T23:32:18.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.248+0000 7f6db0ff9700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d98060a70 0x7f6d98062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:18.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.248+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6da8021070 con 0x7f6db40cc1e0 2023-09-27T23:32:18.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.248+0000 7f6db3fff700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d98060a70 0x7f6d98062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:18.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.248+0000 7f6db3fff700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d98060a70 0x7f6d98062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6db4166e90 tx=0x7f6da0007d30 comp rx=0 tx=0).ready entity=mgr.6971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:18.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.256+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6da804ef90 con 0x7f6db40cc1e0 2023-09-27T23:32:18.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:18.452+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6db406ce20 con 0x7f6db40cc1e0 2023-09-27T23:32:19.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.236+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 50) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f6da804f360 con 0x7f6db40cc1e0 2023-09-27T23:32:19.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.240+0000 7f6db0ff9700 1 -- 172.21.15.64:0/3457710776 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v50) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6da80644f0 con 0x7f6db40cc1e0 2023-09-27T23:32:19.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d98060a70 msgr2=0x7f6d98062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d98060a70 0x7f6d98062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6db4166e90 tx=0x7f6da0007d30 comp rx=0 tx=0).stop 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 msgr2=0x7f6db4166ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 0x7f6db4166ee0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6da80090b0 tx=0x7f6da8017000 comp rx=0 tx=0).stop 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 shutdown_connections 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d98060a70 0x7f6d98062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6db4158fd0 0x7f6db41689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6db40cc610 0x7f6db4169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 --2- 172.21.15.64:0/3457710776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6db40cc1e0 0x7f6db4166ee0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 >> 172.21.15.64:0/3457710776 conn(0x7f6db40cdc00 msgr2=0x7f6db40c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 shutdown_connections 2023-09-27T23:32:19.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.244+0000 7f6db9eee700 1 -- 172.21.15.64:0/3457710776 wait complete. 2023-09-27T23:32:19.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:32:19.358 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:19.355+0000 7fefb5d68000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:32:19.373 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:19.368+0000 7fcf12127000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:32:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.368+0000 7f6613ecb700 1 Processor -- start 2023-09-27T23:32:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- start start 2023-09-27T23:32:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f660c06cfa0 0x7f660c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f660c069a20 0x7f660c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c0ab6c0 con 0x7f660c158ad0 2023-09-27T23:32:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c0ac240 con 0x7f660c06cfa0 2023-09-27T23:32:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c0ab910 con 0x7f660c069a20 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f660c06cfa0 0x7f660c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f660c06cfa0 0x7f660c06d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52626/0 (socket says 172.21.15.64:52626) 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 -- 172.21.15.64:0/3350776608 learned_addr learned my addr 172.21.15.64:0/3350776608 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611466700 1 --2- 172.21.15.64:0/3350776608 >> v2:172.21.15.123:3300/0 conn(0x7f660c069a20 0x7f660c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 -- 172.21.15.64:0/3350776608 >> v2:172.21.15.123:3300/0 conn(0x7f660c069a20 msgr2=0x7f660c0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 --2- 172.21.15.64:0/3350776608 >> v2:172.21.15.123:3300/0 conn(0x7f660c069a20 0x7f660c0b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 -- 172.21.15.64:0/3350776608 >> v1:172.21.15.123:6789/0 conn(0x7f660c158ad0 legacy=0x7f660c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 -- 172.21.15.64:0/3350776608 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0abbf0 con 0x7f660c06cfa0 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6611c67700 1 --2- 172.21.15.64:0/3350776608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c06d370 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f660c0b0000 tx=0x7f6608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e875697246e99188 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6610c65700 1 -- 172.21.15.64:0/3350776608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800b820 con 0x7f660c06cfa0 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6610c65700 1 -- 172.21.15.64:0/3350776608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660800b980 con 0x7f660c06cfa0 2023-09-27T23:32:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6610c65700 1 -- 172.21.15.64:0/3350776608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800bd20 con 0x7f660c06cfa0 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- 172.21.15.64:0/3350776608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 msgr2=0x7f660c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 --2- 172.21.15.64:0/3350776608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c06d370 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f660c0b0000 tx=0x7f6608009250 comp rx=0 tx=0).stop 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- 172.21.15.64:0/3350776608 shutdown_connections 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 --2- 172.21.15.64:0/3350776608 >> v2:172.21.15.123:3300/0 conn(0x7f660c069a20 0x7f660c0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 --2- 172.21.15.64:0/3350776608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c06d370 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- 172.21.15.64:0/3350776608 >> 172.21.15.64:0/3350776608 conn(0x7f660c0cd870 msgr2=0x7f660c069310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- 172.21.15.64:0/3350776608 shutdown_connections 2023-09-27T23:32:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.372+0000 7f6613ecb700 1 -- 172.21.15.64:0/3350776608 wait complete. 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 Processor -- start 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- start start 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f660c069a20 0x7f660c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660c158ad0 0x7f660c152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0c6500 con 0x7f660c06cfa0 2023-09-27T23:32:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c15cf00 con 0x7f660c158ad0 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c15d1a0 con 0x7f660c069a20 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52634/0 (socket says 172.21.15.64:52634) 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 -- 172.21.15.64:0/2629818328 learned_addr learned my addr 172.21.15.64:0/2629818328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611c67700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f660c069a20 0x7f660c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6612468700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660c158ad0 0x7f660c152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 -- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f660c069a20 msgr2=0x7f660c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f660c069a20 0x7f660c0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 -- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660c158ad0 msgr2=0x7f660c152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660c158ad0 0x7f660c152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 -- 172.21.15.64:0/2629818328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66080094f0 con 0x7f660c06cfa0 2023-09-27T23:32:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6612468700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660c158ad0 0x7f660c152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611c67700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f660c069a20 0x7f660c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6611466700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c15e4e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00d860 tx=0x7f65fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0125f0 con 0x7f660c06cfa0 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc012750 con 0x7f660c06cfa0 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c15d480 con 0x7f660c06cfa0 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc012ad0 con 0x7f660c06cfa0 2023-09-27T23:32:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c15da40 con 0x7f660c06cfa0 2023-09-27T23:32:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.376+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c071120 con 0x7f660c06cfa0 2023-09-27T23:32:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.380+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 50) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f65fc01e460 con 0x7f660c06cfa0 2023-09-27T23:32:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.380+0000 7f6601ffb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65f0060af0 0x7f65f0062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.380+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f65fc07d7f0 con 0x7f660c06cfa0 2023-09-27T23:32:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.380+0000 7f6611c67700 1 -- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65f0060af0 msgr2=0x7f65f0062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:32:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.380+0000 7f6611c67700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65f0060af0 0x7f65f0062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:32:19.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.388+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f65fc0512b0 con 0x7f660c06cfa0 2023-09-27T23:32:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.548+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f660c0abed0 con 0x7f660c06cfa0 2023-09-27T23:32:19.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.552+0000 7f6601ffb700 1 -- 172.21.15.64:0/2629818328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f65fc050e40 con 0x7f660c06cfa0 2023-09-27T23:32:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65f0060af0 msgr2=0x7f65f0062fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:32:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65f0060af0 0x7f65f0062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 msgr2=0x7f660c15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c15e4e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00d860 tx=0x7f65fc008040 comp rx=0 tx=0).stop 2023-09-27T23:32:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 shutdown_connections 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f65f0060af0 0x7f65f0062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660c158ad0 0x7f660c152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f660c06cfa0 0x7f660c15e4e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 --2- 172.21.15.64:0/2629818328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f660c069a20 0x7f660c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 >> 172.21.15.64:0/2629818328 conn(0x7f660c0cd870 msgr2=0x7f660c154630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 shutdown_connections 2023-09-27T23:32:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.556+0000 7f6613ecb700 1 -- 172.21.15.64:0/2629818328 wait complete. 2023-09-27T23:32:19.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:32:19.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.692+0000 7f9172c4a700 1 Processor -- start 2023-09-27T23:32:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.692+0000 7f9172c4a700 1 -- start start 2023-09-27T23:32:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f916c155a90 0x7f916c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f916c06db70 0x7f916c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0c68a0 con 0x7f916c155e60 2023-09-27T23:32:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0c6e60 con 0x7f916c155a90 2023-09-27T23:32:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0ae4d0 con 0x7f916c06db70 2023-09-27T23:32:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f916c155a90 0x7f916c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f916c155a90 0x7f916c0ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52644/0 (socket says 172.21.15.64:52644) 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 -- 172.21.15.64:0/3807872558 learned_addr learned my addr 172.21.15.64:0/3807872558 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171447700 1 --2- 172.21.15.64:0/3807872558 >> v2:172.21.15.123:3300/0 conn(0x7f916c06db70 0x7f916c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 -- 172.21.15.64:0/3807872558 >> v2:172.21.15.123:3300/0 conn(0x7f916c06db70 msgr2=0x7f916c0ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 --2- 172.21.15.64:0/3807872558 >> v2:172.21.15.123:3300/0 conn(0x7f916c06db70 0x7f916c0ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 -- 172.21.15.64:0/3807872558 >> v1:172.21.15.123:6789/0 conn(0x7f916c155e60 legacy=0x7f916c0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 -- 172.21.15.64:0/3807872558 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0ae7b0 con 0x7f916c155a90 2023-09-27T23:32:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9171c48700 1 --2- 172.21.15.64:0/3807872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155a90 0x7f916c0ad220 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f916c06a810 tx=0x7f9168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa4aa9908e9632d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9170c46700 1 -- 172.21.15.64:0/3807872558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168009e70 con 0x7f916c155a90 2023-09-27T23:32:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9170c46700 1 -- 172.21.15.64:0/3807872558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916800b860 con 0x7f916c155a90 2023-09-27T23:32:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9170c46700 1 -- 172.21.15.64:0/3807872558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800bc00 con 0x7f916c155a90 2023-09-27T23:32:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- 172.21.15.64:0/3807872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155a90 msgr2=0x7f916c0ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 --2- 172.21.15.64:0/3807872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155a90 0x7f916c0ad220 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f916c06a810 tx=0x7f9168009250 comp rx=0 tx=0).stop 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- 172.21.15.64:0/3807872558 shutdown_connections 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 --2- 172.21.15.64:0/3807872558 >> v2:172.21.15.123:3300/0 conn(0x7f916c06db70 0x7f916c0ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 --2- 172.21.15.64:0/3807872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155a90 0x7f916c0ad220 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- 172.21.15.64:0/3807872558 >> 172.21.15.64:0/3807872558 conn(0x7f916c0c8cd0 msgr2=0x7f916c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- 172.21.15.64:0/3807872558 shutdown_connections 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.696+0000 7f9172c4a700 1 -- 172.21.15.64:0/3807872558 wait complete. 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 Processor -- start 2023-09-27T23:32:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 -- start start 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f916c06db70 0x7f916c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f916c155a90 0x7f916c0a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 0x7f916c166c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c154a50 con 0x7f916c155e60 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0a80c0 con 0x7f916c06db70 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0a4580 con 0x7f916c155a90 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 0x7f916c166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 0x7f916c166c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52656/0 (socket says 172.21.15.64:52656) 2023-09-27T23:32:19.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 -- 172.21.15.64:0/91899560 learned_addr learned my addr 172.21.15.64:0/91899560 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9171c48700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f916c06db70 0x7f916c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9171447700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f916c155a90 0x7f916c0a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 -- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f916c155a90 msgr2=0x7f916c0a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f916c155a90 0x7f916c0a7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 -- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f916c06db70 msgr2=0x7f916c154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f916c06db70 0x7f916c154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 -- 172.21.15.64:0/91899560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680094f0 con 0x7f916c155e60 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172449700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 0x7f916c166c60 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f916400c870 tx=0x7f9164001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164012630 con 0x7f916c155e60 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640127b0 con 0x7f916c155e60 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0a4860 con 0x7f916c155e60 2023-09-27T23:32:19.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164012b70 con 0x7f916c155e60 2023-09-27T23:32:19.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.700+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0a4e20 con 0x7f916c155e60 2023-09-27T23:32:19.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.704+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0a5040 con 0x7f916c155e60 2023-09-27T23:32:19.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.704+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 50) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f9164003030 con 0x7f916c155e60 2023-09-27T23:32:19.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.704+0000 7f9161ffb700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9150060ac0 0x7f9150062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:19.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.704+0000 7f9171c48700 1 -- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9150060ac0 msgr2=0x7f9150062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:32:19.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.704+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f916407d0f0 con 0x7f916c155e60 2023-09-27T23:32:19.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.704+0000 7f9171c48700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9150060ac0 0x7f9150062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:32:19.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.712+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f916404d080 con 0x7f916c155e60 2023-09-27T23:32:19.777 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:19.775+0000 7fefb5d68000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:32:19.825 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:19.816+0000 7fcf12127000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:32:19.829 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:19.823+0000 7fefb5d68000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:32:19.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.872+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f916c04be40 con 0x7f916c155e60 2023-09-27T23:32:19.880 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:19.872+0000 7fcf12127000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:32:19.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9161ffb700 1 -- 172.21.15.64:0/91899560 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f91640506a0 con 0x7f916c155e60 2023-09-27T23:32:19.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9150060ac0 msgr2=0x7f9150062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:32:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9150060ac0 0x7f9150062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 msgr2=0x7f916c166c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 0x7f916c166c60 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f916400c870 tx=0x7f9164001670 comp rx=0 tx=0).stop 2023-09-27T23:32:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 shutdown_connections 2023-09-27T23:32:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9150060ac0 0x7f9150062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c155e60 0x7f916c166c60 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f916c155a90 0x7f916c0a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 --2- 172.21.15.64:0/91899560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f916c06db70 0x7f916c154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:19.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 >> 172.21.15.64:0/91899560 conn(0x7f916c0c8cd0 msgr2=0x7f916c0af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:19.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 shutdown_connections 2023-09-27T23:32:19.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:19.880+0000 7f9172c4a700 1 -- 172.21.15.64:0/91899560 wait complete. 2023-09-27T23:32:19.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:19.900 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:32:19.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-09-27T23:32:19.932 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:19.927+0000 7fefb5d68000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:32:19.990 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:19.984+0000 7fcf12127000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:32:20.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.028+0000 7f904a9c7700 1 Processor -- start 2023-09-27T23:32:20.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.028+0000 7f904a9c7700 1 -- start start 2023-09-27T23:32:20.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a9c7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f90440c7700 0x7f90440abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:20.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a9c7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440cbfb0 con 0x7f90441558d0 2023-09-27T23:32:20.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a9c7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440cc290 con 0x7f90441569b0 2023-09-27T23:32:20.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a9c7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9044156fa0 con 0x7f90440c7700 2023-09-27T23:32:20.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90499c5700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90441558d0 0x7f90440ad740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34106/0 (socket says 172.21.15.64:34106) 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90499c5700 1 -- 172.21.15.64:0/3173686200 learned_addr learned my addr 172.21.15.64:0/3173686200 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a1c6700 1 --2- 172.21.15.64:0/3173686200 >> v2:172.21.15.123:3300/0 conn(0x7f90440c7700 0x7f90440abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90489c3700 1 -- 172.21.15.64:0/3173686200 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371052703 0 0) 0x7f90440cbfb0 con 0x7f90441558d0 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90489c3700 1 -- 172.21.15.64:0/3173686200 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9030003570 con 0x7f90441558d0 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a1c6700 1 -- 172.21.15.64:0/3173686200 >> v1:172.21.15.123:6789/0 conn(0x7f90441569b0 legacy=0x7f90440c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a1c6700 1 -- 172.21.15.64:0/3173686200 >> v1:172.21.15.64:6789/0 conn(0x7f90441558d0 legacy=0x7f90440ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a1c6700 1 -- 172.21.15.64:0/3173686200 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440cc570 con 0x7f90440c7700 2023-09-27T23:32:20.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a1c6700 1 --2- 172.21.15.64:0/3173686200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90440c7700 0x7f90440abe10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f903c009e60 tx=0x7f903c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166234dab971ffd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:20.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90489c3700 1 -- 172.21.15.64:0/3173686200 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00e070 con 0x7f90440c7700 2023-09-27T23:32:20.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90489c3700 1 -- 172.21.15.64:0/3173686200 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c00bb60 con 0x7f90440c7700 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f90489c3700 1 -- 172.21.15.64:0/3173686200 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c012430 con 0x7f90440c7700 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a9c7700 1 -- 172.21.15.64:0/3173686200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90440c7700 msgr2=0x7f90440abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.032+0000 7f904a9c7700 1 --2- 172.21.15.64:0/3173686200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90440c7700 0x7f90440abe10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f903c009e60 tx=0x7f903c009780 comp rx=0 tx=0).stop 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- 172.21.15.64:0/3173686200 shutdown_connections 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 --2- 172.21.15.64:0/3173686200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90440c7700 0x7f90440abe10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- 172.21.15.64:0/3173686200 >> 172.21.15.64:0/3173686200 conn(0x7f904406b5b0 msgr2=0x7f904406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- 172.21.15.64:0/3173686200 shutdown_connections 2023-09-27T23:32:20.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- 172.21.15.64:0/3173686200 wait complete. 2023-09-27T23:32:20.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 Processor -- start 2023-09-27T23:32:20.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- start start 2023-09-27T23:32:20.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 0x7f9044154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:20.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90441558d0 0x7f9044166ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90441569b0 0x7f9044167220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440c68d0 con 0x7f90440c7700 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90441678c0 con 0x7f90441558d0 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f904a9c7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9044167ba0 con 0x7f90441569b0 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f90499c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 0x7f9044154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f90499c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 0x7f9044154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52668/0 (socket says 172.21.15.64:52668) 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f90499c5700 1 -- 172.21.15.64:0/2405807597 learned_addr learned my addr 172.21.15.64:0/2405807597 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.036+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90441558d0 0x7f9044166ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f904a1c6700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90441569b0 0x7f9044167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90499c5700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90441569b0 msgr2=0x7f9044167220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:20.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90499c5700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90441569b0 0x7f9044167220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90499c5700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90441558d0 msgr2=0x7f9044166ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90499c5700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90441558d0 0x7f9044166ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90499c5700 1 -- 172.21.15.64:0/2405807597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c009a20 con 0x7f90440c7700 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90441558d0 0x7f9044166ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f90499c5700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 0x7f9044154170 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f904000c7c0 tx=0x7f9040007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040005b30 con 0x7f90440c7700 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040005c90 con 0x7f90440c7700 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440b09a0 con 0x7f90440c7700 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040007340 con 0x7f90440c7700 2023-09-27T23:32:20.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440b0f60 con 0x7f90440c7700 2023-09-27T23:32:20.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904406c230 con 0x7f90440c7700 2023-09-27T23:32:20.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 50) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f904001b400 con 0x7f90440c7700 2023-09-27T23:32:20.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.040+0000 7f9039ffb700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:20.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.044+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f904007cd20 con 0x7f90440c7700 2023-09-27T23:32:20.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.044+0000 7f90491c4700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 msgr2=0x7f9028062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:32:20.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.044+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:32:20.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:32:20.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:32:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.052+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f904406c230 con 0x7f90440c7700 2023-09-27T23:32:20.127 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.123+0000 7fefb5d68000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:32:20.176 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.175+0000 7fefb5d68000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:32:20.200 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.192+0000 7fcf12127000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:32:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.232+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9044167a30 con 0x7f9028060ac0 2023-09-27T23:32:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.248+0000 7f90491c4700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 msgr2=0x7f9028062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:32:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.248+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:32:20.254 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.248+0000 7fcf12127000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:32:20.328 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.327+0000 7fefb5d68000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:32:20.386 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.383+0000 7fefb5d68000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:32:20.411 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.404+0000 7fcf12127000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:32:20.445 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.443+0000 7fefb5d68000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:32:20.470 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.464+0000 7fcf12127000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:32:20.532 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.524+0000 7fcf12127000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:32:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.652+0000 7f90491c4700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 msgr2=0x7f9028062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:32:20.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:20.652+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:32:20.741 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.739+0000 7fefb5d68000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:32:20.841 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.836+0000 7fcf12127000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:32:20.853 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:20.851+0000 7fefb5d68000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:32:20.957 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:20.952+0000 7fcf12127000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:32:21.135 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.131+0000 7fefb5d68000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:32:21.254 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.248+0000 7fcf12127000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:32:21.299 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.295+0000 7fefb5d68000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:32:21.364 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.363+0000 7fefb5d68000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:32:21.416 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.411+0000 7fefb5d68000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:32:21.436 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.428+0000 7fcf12127000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:32:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:21.456+0000 7f90491c4700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 msgr2=0x7f9028062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:32:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:21.456+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:32:21.470 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.467+0000 7fefb5d68000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:32:21.508 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.504+0000 7fcf12127000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:32:21.567 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.560+0000 7fcf12127000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:32:21.628 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.620+0000 7fcf12127000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:32:21.644 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.643+0000 7fefb5d68000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:32:21.698 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.695+0000 7fefb5d68000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:32:21.760 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.755+0000 7fefb5d68000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:32:21.824 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.820+0000 7fcf12127000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:32:21.886 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.880+0000 7fcf12127000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:32:21.912 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.911+0000 7fefb5d68000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:32:21.954 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:21.948+0000 7fcf12127000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:32:21.972 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:21.971+0000 7fefb5d68000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:32:22.124 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:22.116+0000 7fcf12127000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:32:22.190 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:22.184+0000 7fcf12127000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:32:22.562 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.559+0000 7fefb5d68000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:32:22.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:22.592+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 51) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f904004d120 con 0x7f90440c7700 2023-09-27T23:32:22.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:22.592+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 msgr2=0x7f9028062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:32:22.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:22.592+0000 7f9039ffb700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:22.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef78f18700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef78f18700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef78f18700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef78f18700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef78f18700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.745 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef76713700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.746 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef76713700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.746 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef76713700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.746 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef76713700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.746 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:32:22.743+0000 7fef76713700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:32:22.849 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:32:22.844+0000 7fcf12127000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:32:23.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.596+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 52) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f904007af60 con 0x7f90440c7700 2023-09-27T23:32:23.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.596+0000 7f9039ffb700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.596+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9044167a30 con 0x7f9028060ac0 2023-09-27T23:32:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.596+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.600+0000 7f90491c4700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f90440b0930 tx=0x7f9034001130 comp rx=0 tx=0).ready entity=mgr.7271 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.696+0000 7f9039ffb700 1 -- 172.21.15.64:0/2405807597 <== mgr.7271 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9044167a30 con 0x7f9028060ac0 2023-09-27T23:32:23.701 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path snap_test_dir1 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 msgr2=0x7f9028062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f90440b0930 tx=0x7f9034001130 comp rx=0 tx=0).stop 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 msgr2=0x7f9044154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 0x7f9044154170 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f904000c7c0 tx=0x7f9040007e00 comp rx=0 tx=0).stop 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 shutdown_connections 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9028060ac0 0x7f9028062f70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90441569b0 0x7f9044167220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90441558d0 0x7f9044166ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 --2- 172.21.15.64:0/2405807597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90440c7700 0x7f9044154170 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 >> 172.21.15.64:0/2405807597 conn(0x7f904406b5b0 msgr2=0x7f9044154ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 shutdown_connections 2023-09-27T23:32:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.700+0000 7f904a9c7700 1 -- 172.21.15.64:0/2405807597 wait complete. 2023-09-27T23:32:23.721 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-09-27T23:32:23.722 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-09-27-23_33_00 in ~37s... 2023-09-27T23:32:23.722 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-09-27T23:32:23.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-09-27T23:32:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 Processor -- start 2023-09-27T23:32:23.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 -- start start 2023-09-27T23:32:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 0x7f82cc0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0c8db0 0x7f82cc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82cc0c9230 con 0x7f82cc06ade0 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc15b680 con 0x7f82cc0ad9f0 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82cb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc06da90 con 0x7f82cc0c8db0 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.844+0000 7f82ca59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 0x7f82cc0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c9d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0c8db0 0x7f82cc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82ca59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 0x7f82cc0addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52670/0 (socket says 172.21.15.64:52670) 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82ca59c700 1 -- 172.21.15.64:0/2626104975 learned_addr learned my addr 172.21.15.64:0/2626104975 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:23.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cad9d700 1 --1- 172.21.15.64:0/2626104975 >> v1:172.21.15.123:6789/0 conn(0x7f82cc06ade0 0x7f82cc0b0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:54988/0 (socket says 172.21.15.64:54988) 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c9d9b700 1 -- 172.21.15.64:0/2626104975 >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 msgr2=0x7f82cc0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c9d9b700 1 --2- 172.21.15.64:0/2626104975 >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 0x7f82cc0addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c9d9b700 1 -- 172.21.15.64:0/2626104975 >> v1:172.21.15.123:6789/0 conn(0x7f82cc06ade0 legacy=0x7f82cc0b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c9d9b700 1 -- 172.21.15.64:0/2626104975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc06a620 con 0x7f82cc0c8db0 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82ca59c700 1 --2- 172.21.15.64:0/2626104975 >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 0x7f82cc0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c9d9b700 1 --2- 172.21.15.64:0/2626104975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0c8db0 0x7f82cc15b140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009e60 tx=0x7f82b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ac880d6f7ffa46 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c959a700 1 -- 172.21.15.64:0/2626104975 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b400e070 con 0x7f82cc0c8db0 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c959a700 1 -- 172.21.15.64:0/2626104975 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b400bb60 con 0x7f82cc0c8db0 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82c959a700 1 -- 172.21.15.64:0/2626104975 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4012430 con 0x7f82cc0c8db0 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 -- 172.21.15.64:0/2626104975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0c8db0 msgr2=0x7f82cc15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:23.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 --2- 172.21.15.64:0/2626104975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0c8db0 0x7f82cc15b140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009e60 tx=0x7f82b4009780 comp rx=0 tx=0).stop 2023-09-27T23:32:23.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 -- 172.21.15.64:0/2626104975 shutdown_connections 2023-09-27T23:32:23.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 --2- 172.21.15.64:0/2626104975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0c8db0 0x7f82cc15b140 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 --2- 172.21.15.64:0/2626104975 >> v2:172.21.15.64:3300/0 conn(0x7f82cc0ad9f0 0x7f82cc0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 -- 172.21.15.64:0/2626104975 >> 172.21.15.64:0/2626104975 conn(0x7f82cc067bb0 msgr2=0x7f82cc0c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:23.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 -- 172.21.15.64:0/2626104975 shutdown_connections 2023-09-27T23:32:23.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.848+0000 7f82cb59e700 1 -- 172.21.15.64:0/2626104975 wait complete. 2023-09-27T23:32:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 Processor -- start 2023-09-27T23:32:23.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 -- start start 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc06ade0 0x7f82cc166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0ad9f0 0x7f82cc0a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 0x7f82cc0a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0a6c30 con 0x7f82cc0c8db0 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0a7b40 con 0x7f82cc06ade0 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc13c3c0 con 0x7f82cc0ad9f0 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 0x7f82cc0a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 0x7f82cc0a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52680/0 (socket says 172.21.15.64:52680) 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 -- 172.21.15.64:0/4256652837 learned_addr learned my addr 172.21.15.64:0/4256652837 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:32:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82ca59c700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc06ade0 0x7f82cc166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82c9d9b700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0ad9f0 0x7f82cc0a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 -- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0ad9f0 msgr2=0x7f82cc0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0ad9f0 0x7f82cc0a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 -- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc06ade0 msgr2=0x7f82cc166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc06ade0 0x7f82cc166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 -- 172.21.15.64:0/4256652837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc0094f0 con 0x7f82cc0c8db0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cad9d700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 0x7f82cc0a7600 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f82c400c7c0 tx=0x7f82c4007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4005a70 con 0x7f82cc0c8db0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4005bf0 con 0x7f82cc0c8db0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 -- 172.21.15.64:0/4256652837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b4009a20 con 0x7f82cc0c8db0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40070d0 con 0x7f82cc0c8db0 2023-09-27T23:32:23.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.852+0000 7f82cb59e700 1 -- 172.21.15.64:0/4256652837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc0a65e0 con 0x7f82cc0c8db0 2023-09-27T23:32:23.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.856+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a40052b0 con 0x7f82cc0c8db0 2023-09-27T23:32:23.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.856+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 52) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f82c4007440 con 0x7f82cc0c8db0 2023-09-27T23:32:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.856+0000 7f82c27fc700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f82ac0607f0 0x7f82ac062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:32:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.856+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f82c407c380 con 0x7f82cc0c8db0 2023-09-27T23:32:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.856+0000 7f82ca59c700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f82ac0607f0 0x7f82ac062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:32:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.860+0000 7f82ca59c700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f82ac0607f0 0x7f82ac062ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006d20 tx=0x7f82bc009250 comp rx=0 tx=0).ready entity=mgr.7271 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:32:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:23.860+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f82c404c540 con 0x7f82cc0c8db0 2023-09-27T23:32:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.044+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82a4000bc0 con 0x7f82ac0607f0 2023-09-27T23:32:24.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.060+0000 7f82c27fc700 1 -- 172.21.15.64:0/4256652837 <== mgr.7271 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f82a4000bc0 con 0x7f82ac0607f0 2023-09-27T23:32:24.067 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:32:24.067 INFO:teuthology.orchestra.run.smithi064.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f82ac0607f0 msgr2=0x7f82ac062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f82ac0607f0 0x7f82ac062ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006d20 tx=0x7f82bc009250 comp rx=0 tx=0).stop 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 msgr2=0x7f82cc0a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 0x7f82cc0a7600 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f82c400c7c0 tx=0x7f82c4007eb0 comp rx=0 tx=0).stop 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 shutdown_connections 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f82ac0607f0 0x7f82ac062ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82cc0c8db0 0x7f82cc0a7600 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82cc0ad9f0 0x7f82cc0a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:24.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 --2- 172.21.15.64:0/4256652837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc06ade0 0x7f82cc166ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:32:24.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 >> 172.21.15.64:0/4256652837 conn(0x7f82cc067bb0 msgr2=0x7f82cc13cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:32:24.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 shutdown_connections 2023-09-27T23:32:24.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:32:24.064+0000 7f82abfff700 1 -- 172.21.15.64:0/4256652837 wait complete. 2023-09-27T23:32:24.084 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-09-27T23:32:24.084 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-09-27T23:32:24.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:24.084 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:24.100 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:24.101 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:24.101 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:25.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:25.103 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:25.121 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:25.121 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:25.121 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:26.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:26.122 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:26.141 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:26.142 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:26.142 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:27.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:27.143 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:27.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:27.160 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:27.160 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:28.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:28.162 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:28.179 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:28.179 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:28.179 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:29.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:29.181 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:29.198 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:29.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:29.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:30.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:30.200 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:30.220 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:30.220 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:30.220 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:31.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:31.221 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:31.238 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:31.238 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:31.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:32.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:32.239 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:32.258 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:32.258 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:32.258 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:33.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:33.259 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:33.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:33.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:33.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:34.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:34.278 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:34.294 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:34.294 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:34.295 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:35.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:35.296 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:35.315 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:35.315 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:35.315 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:36.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:36.317 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:36.335 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:36.335 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:36.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:37.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:37.337 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:37.355 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:37.356 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:37.356 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:38.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:38.357 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:38.374 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:38.374 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:38.374 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:39.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:39.375 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:39.393 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:39.393 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:39.393 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:40.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:40.394 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:40.411 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:40.411 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:40.411 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:41.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:41.413 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:41.430 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:41.430 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:41.430 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:42.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:42.431 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:42.448 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:42.449 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:42.449 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:43.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:43.450 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:43.467 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:43.467 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:43.467 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:44.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:44.468 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:44.486 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:44.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:44.486 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:45.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:45.487 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:45.507 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:45.507 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:45.508 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:46.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:46.509 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:46.527 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:46.527 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:46.527 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:47.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:47.528 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:47.546 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:47.546 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:47.546 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:48.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:48.547 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:48.565 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:48.565 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:48.565 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:49.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:49.566 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:49.584 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:49.584 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:49.584 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:50.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:32:50.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:32:50.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:50.586 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:50.606 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:50.606 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:50.606 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:51.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:51.607 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:51.625 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:51.625 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:51.626 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:52.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:52.626 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:52.644 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:52.645 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:52.645 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:53.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:53.646 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:53.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:53.665 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:53.665 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:54.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:54.665 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:54.684 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:54.684 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:54.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:55.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:55.686 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:55.705 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:55.705 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:55.705 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:56.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:56.706 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:56.725 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:56.725 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:56.725 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:57.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:57.726 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:57.744 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:57.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:57.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:58.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:58.746 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:58.764 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:58.764 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:58.764 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:32:59.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:32:59.766 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:32:59.784 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:32:59.784 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:32:59.784 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:33:00.720 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:00.714+0000 7fef68ef8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:00.720 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:00.714+0000 7fef68ef8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:00.720 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:00.714+0000 7fef68ef8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:00.720 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:00.714+0000 7fef68ef8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:00.720 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:00.714+0000 7fef68ef8700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:00.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:00.787 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:33:00.805 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_33_00_UTC 2023-09-27T23:33:00.806 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_33_00_UTC'} 2023-09-27T23:33:00.806 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-09-27-23_33_00_UTC'} 2023-09-27T23:33:00.806 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:33:00.807 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-09-27-23_33_00_UTC'] 2023-09-27T23:33:00.807 DEBUG:tasks.cephfs.test_snap_schedules:exec=1695857543.7219963, now = 1695857580.8071392, timo = 37 2023-09-27T23:33:01.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-09-27T23:33:01.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.931+0000 7f650ad0d700 1 Processor -- start 2023-09-27T23:33:01.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.931+0000 7f650ad0d700 1 -- start start 2023-09-27T23:33:01.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6504158da0 0x7f650406e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:01.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650415d720 con 0x7f650406df10 2023-09-27T23:33:01.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040ca4a0 con 0x7f65040caca0 2023-09-27T23:33:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040ca290 con 0x7f6504158da0 2023-09-27T23:33:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6509d0b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6504158da0 0x7f650406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650950a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f650406df10 0x7f650406a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43644/0 (socket says 172.21.15.64:43644) 2023-09-27T23:33:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650950a700 1 -- 172.21.15.64:0/3722882248 learned_addr learned my addr 172.21.15.64:0/3722882248 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6508d09700 1 -- 172.21.15.64:0/3722882248 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627943149 0 0) 0x7f650415d720 con 0x7f650406df10 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6508d09700 1 -- 172.21.15.64:0/3722882248 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f0003570 con 0x7f650406df10 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6509d0b700 1 -- 172.21.15.64:0/3722882248 >> v1:172.21.15.123:6789/0 conn(0x7f65040caca0 legacy=0x7f650406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6509d0b700 1 -- 172.21.15.64:0/3722882248 >> v1:172.21.15.64:6789/0 conn(0x7f650406df10 legacy=0x7f650406a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6509d0b700 1 -- 172.21.15.64:0/3722882248 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040ca780 con 0x7f6504158da0 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6509d0b700 1 --2- 172.21.15.64:0/3722882248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650406e670 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f65000098e0 tx=0x7f65000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6001df71b5e35608 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6508d09700 1 -- 172.21.15.64:0/3722882248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650000b640 con 0x7f6504158da0 2023-09-27T23:33:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6508d09700 1 -- 172.21.15.64:0/3722882248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650000b7a0 con 0x7f6504158da0 2023-09-27T23:33:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f6508d09700 1 -- 172.21.15.64:0/3722882248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650000bb20 con 0x7f6504158da0 2023-09-27T23:33:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- 172.21.15.64:0/3722882248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 msgr2=0x7f650406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 --2- 172.21.15.64:0/3722882248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650406e670 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f65000098e0 tx=0x7f65000091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- 172.21.15.64:0/3722882248 shutdown_connections 2023-09-27T23:33:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 --2- 172.21.15.64:0/3722882248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650406e670 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- 172.21.15.64:0/3722882248 >> 172.21.15.64:0/3722882248 conn(0x7f65040cc760 msgr2=0x7f65040ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- 172.21.15.64:0/3722882248 shutdown_connections 2023-09-27T23:33:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.935+0000 7f650ad0d700 1 -- 172.21.15.64:0/3722882248 wait complete. 2023-09-27T23:33:01.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.939+0000 7f650ad0d700 1 Processor -- start 2023-09-27T23:33:01.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.939+0000 7f650ad0d700 1 -- start start 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f650406df10 0x7f65040c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65040caca0 0x7f6504154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650416d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504169ae0 con 0x7f6504158da0 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504169dc0 con 0x7f650406df10 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650416a0a0 con 0x7f65040caca0 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650416d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:01.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650416d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43012/0 (socket says 172.21.15.64:43012) 2023-09-27T23:33:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 -- 172.21.15.64:0/488324100 learned_addr learned my addr 172.21.15.64:0/488324100 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650950a700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65040caca0 0x7f6504154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:01.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f6509d0b700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f650406df10 0x7f65040c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 -- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65040caca0 msgr2=0x7f6504154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65040caca0 0x7f6504154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 -- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f650406df10 msgr2=0x7f65040c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f650406df10 0x7f65040c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 -- 172.21.15.64:0/488324100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500009420 con 0x7f6504158da0 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650a50c700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650416d0c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c8e0 tx=0x7f64fc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc012630 con 0x7f6504158da0 2023-09-27T23:33:01.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc012790 con 0x7f6504158da0 2023-09-27T23:33:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650416a380 con 0x7f6504158da0 2023-09-27T23:33:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc012b50 con 0x7f6504158da0 2023-09-27T23:33:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.943+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040c93c0 con 0x7f6504158da0 2023-09-27T23:33:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.947+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650404be40 con 0x7f6504158da0 2023-09-27T23:33:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.947+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 54) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f64fc01a3a0 con 0x7f6504158da0 2023-09-27T23:33:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.947+0000 7f64f9ffb700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64e8060be0 0x7f64e8063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.947+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f64fc07d640 con 0x7f6504158da0 2023-09-27T23:33:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.947+0000 7f6509d0b700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64e8060be0 0x7f64e8063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:01.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.951+0000 7f6509d0b700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64e8060be0 0x7f64e8063090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65000093f0 tx=0x7f6500017000 comp rx=0 tx=0).ready entity=mgr.7271 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:01.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:01.959+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f64fc050e20 con 0x7f6504158da0 2023-09-27T23:33:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.147+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f65040ae950 con 0x7f64e8060be0 2023-09-27T23:33:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.271+0000 7f64f9ffb700 1 -- 172.21.15.64:0/488324100 <== mgr.7271 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f65040ae950 con 0x7f64e8060be0 2023-09-27T23:33:02.274 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path snap_test_dir1 2023-09-27T23:33:02.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64e8060be0 msgr2=0x7f64e8063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64e8060be0 0x7f64e8063090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65000093f0 tx=0x7f6500017000 comp rx=0 tx=0).stop 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 msgr2=0x7f650416d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650416d0c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c8e0 tx=0x7f64fc001710 comp rx=0 tx=0).stop 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 shutdown_connections 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f64e8060be0 0x7f64e8063090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6504158da0 0x7f650416d0c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65040caca0 0x7f6504154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 --2- 172.21.15.64:0/488324100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f650406df10 0x7f65040c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 >> 172.21.15.64:0/488324100 conn(0x7f65040cc760 msgr2=0x7f65040ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 shutdown_connections 2023-09-27T23:33:02.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.275+0000 7f650ad0d700 1 -- 172.21.15.64:0/488324100 wait complete. 2023-09-27T23:33:02.296 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-09-27T23:33:02.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:02.296 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:33:02.311 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_33_00_UTC 2023-09-27T23:33:02.311 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-09-27-23_33_00_UTC 2023-09-27T23:33:02.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:02.312 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-09-27-23_33_00_UTC) 2023-09-27T23:33:02.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:02.333 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-09-27T23:33:02.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:33:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f4225408700 1 Processor -- start 2023-09-27T23:33:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f4225408700 1 -- start start 2023-09-27T23:33:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f4225408700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42200c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f4225408700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422006d750 con 0x7f4220158fd0 2023-09-27T23:33:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f4225408700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200cbaf0 con 0x7f42200cc5f0 2023-09-27T23:33:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f4225408700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422006cb40 con 0x7f42200cc1e0 2023-09-27T23:33:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4220158fd0 0x7f422006dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43660/0 (socket says 172.21.15.64:43660) 2023-09-27T23:33:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421ffff700 1 -- 172.21.15.64:0/746376307 learned_addr learned my addr 172.21.15.64:0/746376307 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f42177fe700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42200c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873094417 0 0) 0x7f422006d750 con 0x7f4220158fd0 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f420c003570 con 0x7f4220158fd0 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892223552 0 0) 0x7f42200cbaf0 con 0x7f42200cc5f0 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f422006d750 con 0x7f42200cc5f0 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223458842 0 0) 0x7f420c003570 con 0x7f4220158fd0 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 msgr2=0x7f42200c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42200c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 >> v1:172.21.15.123:6789/0 conn(0x7f42200cc5f0 legacy=0x7f42200c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422006ce20 con 0x7f4220158fd0 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f4210001470 con 0x7f4220158fd0 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 >> v1:172.21.15.64:6789/0 conn(0x7f4220158fd0 legacy=0x7f422006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420c003c20 0x7f420c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f420c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422006cb40 con 0x7f420c003c20 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00bb40 con 0x7f420c004810 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.467+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00bd00 con 0x7f42200cc1e0 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f421ffff700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420c003c20 0x7f420c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f420c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f42177fe700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 -- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420c004810 msgr2=0x7f420c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f420c004810 0x7f420c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 -- 172.21.15.64:0/746376307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420c003c20 msgr2=0x7f420c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420c003c20 0x7f420c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 -- 172.21.15.64:0/746376307 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c003570 con 0x7f42200cc1e0 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f421ffff700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f420c003c20 0x7f420c004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f420c0051c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4218002120 tx=0x7f42180017a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35bf4121e79df442 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218001ab0 con 0x7f42200cc1e0 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421800c980 con 0x7f42200cc1e0 2023-09-27T23:33:02.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f421f7fe700 1 -- 172.21.15.64:0/746376307 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800cd20 con 0x7f42200cc1e0 2023-09-27T23:33:02.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4225408700 1 -- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 msgr2=0x7f420c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4225408700 1 --2- 172.21.15.64:0/746376307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f420c0051c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4218002120 tx=0x7f42180017a0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4224c07700 1 -- 172.21.15.64:0/746376307 reap_dead start 2023-09-27T23:33:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4225408700 1 -- 172.21.15.64:0/746376307 shutdown_connections 2023-09-27T23:33:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4225408700 1 -- 172.21.15.64:0/746376307 >> 172.21.15.64:0/746376307 conn(0x7f42200cdc00 msgr2=0x7f42200ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4225408700 1 -- 172.21.15.64:0/746376307 shutdown_connections 2023-09-27T23:33:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.471+0000 7f4225408700 1 -- 172.21.15.64:0/746376307 wait complete. 2023-09-27T23:33:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 Processor -- start 2023-09-27T23:33:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 -- start start 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42201512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4220151920 0x7f42200c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 0x7f42201581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42201586e0 con 0x7f4220157d20 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42200c9470 con 0x7f4220151920 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4225408700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4220169410 con 0x7f42200cc1e0 2023-09-27T23:33:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 0x7f42201581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 0x7f42201581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43038/0 (socket says 172.21.15.64:43038) 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 -- 172.21.15.64:0/4195503747 learned_addr learned my addr 172.21.15.64:0/4195503747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f421ffff700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42201512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f42177fe700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4220151920 0x7f42200c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 -- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 msgr2=0x7f42201512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42201512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 -- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4220151920 msgr2=0x7f42200c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4220151920 0x7f42200c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.475+0000 7f4224c07700 1 -- 172.21.15.64:0/4195503747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42201696f0 con 0x7f4220157d20 2023-09-27T23:33:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f4224c07700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 0x7f42201581a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4218003000 tx=0x7f4218016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218001ab0 con 0x7f4220157d20 2023-09-27T23:33:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42201699d0 con 0x7f4220157d20 2023-09-27T23:33:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421800c6d0 con 0x7f4220157d20 2023-09-27T23:33:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421801f650 con 0x7f4220157d20 2023-09-27T23:33:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220169f90 con 0x7f4220157d20 2023-09-27T23:33:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422004be40 con 0x7f4220157d20 2023-09-27T23:33:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 54) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f4218025020 con 0x7f4220157d20 2023-09-27T23:33:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421cff9700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4200060be0 0x7f4200063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f421807f960 con 0x7f4220157d20 2023-09-27T23:33:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.479+0000 7f421ffff700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4200060be0 0x7f4200063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.483+0000 7f421ffff700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4200060be0 0x7f4200063090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f42100010e0 tx=0x7f4210001e10 comp rx=0 tx=0).ready entity=mgr.7271 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:02.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.487+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f421804f770 con 0x7f4220157d20 2023-09-27T23:33:02.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:02.691+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f42200c5c80 con 0x7f4220157d20 2023-09-27T23:33:03.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.299+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f4218003030 con 0x7f4220157d20 2023-09-27T23:33:03.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.303+0000 7f421cff9700 1 -- 172.21.15.64:0/4195503747 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v55) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4218053200 con 0x7f4220157d20 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.303+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4200060be0 msgr2=0x7f4200063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.303+0000 7f4225408700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4200060be0 0x7f4200063090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f42100010e0 tx=0x7f4210001e10 comp rx=0 tx=0).stop 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.303+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 msgr2=0x7f42201581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.303+0000 7f4225408700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 0x7f42201581a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4218003000 tx=0x7f4218016000 comp rx=0 tx=0).stop 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 shutdown_connections 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4200060be0 0x7f4200063090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4220157d20 0x7f42201581a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4220151920 0x7f42200c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 --2- 172.21.15.64:0/4195503747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42200cc1e0 0x7f42201512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 >> 172.21.15.64:0/4195503747 conn(0x7f42200cdc00 msgr2=0x7f42200b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 shutdown_connections 2023-09-27T23:33:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.307+0000 7f4225408700 1 -- 172.21.15.64:0/4195503747 wait complete. 2023-09-27T23:33:03.322 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:33:03.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:03.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:03.351 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:33:03.351 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:33:03.351 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:33:03.351 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:03.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[54820]: fuse finished with error 0 and tester_r 0 2023-09-27T23:33:03.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:33:03.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:03.394 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:33:03.420 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:03.418+0000 7fab55331000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:33:03.423 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:03.419+0000 7f4fca4d9000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:33:03.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:33:03.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:03.519 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:33:03.527 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:33:03.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:33:03.528 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:33:03.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:03.529 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:33:03.566 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:33:03.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:33:03.567 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:33:03.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:03.567 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:33:03.577 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:33:03.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:33:03.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 Processor -- start 2023-09-27T23:33:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 -- start start 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c840cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c840cda30 0x7f8c84157060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c840cd5a0 con 0x7f8c840c90e0 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c840cc990 con 0x7f8c840cda30 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c89fd4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840ccc70 con 0x7f8c84156c90 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c840cda30 0x7f8c84157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c840cda30 0x7f8c84157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43048/0 (socket says 172.21.15.64:43048) 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 -- 172.21.15.64:0/1836942677 learned_addr learned my addr 172.21.15.64:0/1836942677 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c837fe700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c840cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 -- 172.21.15.64:0/1836942677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 msgr2=0x7f8c840cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c840cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 -- 172.21.15.64:0/1836942677 >> v1:172.21.15.123:6789/0 conn(0x7f8c840c90e0 legacy=0x7f8c8415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c82ffd700 1 -- 172.21.15.64:0/1836942677 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840ccf50 con 0x7f8c840cda30 2023-09-27T23:33:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.679+0000 7f8c837fe700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c840cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c82ffd700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c84157060 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8c780099b0 tx=0x7f8c78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f19be4c49d3e1d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c827fc700 1 -- 172.21.15.64:0/1836942677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800b860 con 0x7f8c840cda30 2023-09-27T23:33:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c827fc700 1 -- 172.21.15.64:0/1836942677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800b9c0 con 0x7f8c840cda30 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c827fc700 1 -- 172.21.15.64:0/1836942677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800bd60 con 0x7f8c840cda30 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 -- 172.21.15.64:0/1836942677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 msgr2=0x7f8c84157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c84157060 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8c780099b0 tx=0x7f8c78009250 comp rx=0 tx=0).stop 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 -- 172.21.15.64:0/1836942677 shutdown_connections 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c84157060 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/1836942677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c840cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 -- 172.21.15.64:0/1836942677 >> 172.21.15.64:0/1836942677 conn(0x7f8c8406c7c0 msgr2=0x7f8c8406cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 -- 172.21.15.64:0/1836942677 shutdown_connections 2023-09-27T23:33:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.683+0000 7f8c89fd4700 1 -- 172.21.15.64:0/1836942677 wait complete. 2023-09-27T23:33:03.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 Processor -- start 2023-09-27T23:33:03.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 -- start start 2023-09-27T23:33:03.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c840c90e0 0x7f8c840ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c840ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c84158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8415be60 con 0x7f8c840cda30 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840ad780 con 0x7f8c840c90e0 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8415c0b0 con 0x7f8c84156c90 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c840ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c840ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43052/0 (socket says 172.21.15.64:43052) 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 -- 172.21.15.64:0/89283541 learned_addr learned my addr 172.21.15.64:0/89283541 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c837fe700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c840c90e0 0x7f8c840ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c83fff700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c84158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:03.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 -- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 msgr2=0x7f8c84158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c84158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 -- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c840c90e0 msgr2=0x7f8c840ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c840c90e0 0x7f8c840ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 -- 172.21.15.64:0/89283541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780094f0 con 0x7f8c840cda30 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c83fff700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c84158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c82ffd700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c840ae210 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8c780090b0 tx=0x7f8c78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7801a450 con 0x7f8c840cda30 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7801a5b0 con 0x7f8c840cda30 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8415c390 con 0x7f8c840cda30 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7801a970 con 0x7f8c840cda30 2023-09-27T23:33:03.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.687+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8415c950 con 0x7f8c840cda30 2023-09-27T23:33:03.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.691+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8415cb40 con 0x7f8c840cda30 2023-09-27T23:33:03.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.691+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f8c7801ad20 con 0x7f8c840cda30 2023-09-27T23:33:03.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.691+0000 7f8c6b7fe700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 0x7f8c70063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:03.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.691+0000 7f8c837fe700 1 -- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 msgr2=0x7f8c70063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:03.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.691+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f8c7807eab0 con 0x7f8c840cda30 2023-09-27T23:33:03.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.691+0000 7f8c837fe700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 0x7f8c70063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:03.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.699+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8c7804e800 con 0x7f8c840cda30 2023-09-27T23:33:03.842 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:03.838+0000 7fab55331000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:33:03.867 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:03.863+0000 7f4fca4d9000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:33:03.889 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:03.886+0000 7fab55331000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:33:03.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.891+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c84071120 con 0x7f8c840cda30 2023-09-27T23:33:03.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.891+0000 7f8c6b7fe700 1 -- 172.21.15.64:0/89283541 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f8c78020090 con 0x7f8c840cda30 2023-09-27T23:33:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:03.895 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:32:05.833505+0000","modified":"2023-09-27T23:32:08.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8151},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8151":{"gid":8151,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4228640199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4228640199},{"type":"v1","addr":"172.21.15.64:6835","nonce":4228640199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-09-27T23:33:03.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c837fe700 1 -- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 msgr2=0x7f8c70063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:03.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c837fe700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 0x7f8c70063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:33:03.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 msgr2=0x7f8c70063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:03.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 0x7f8c70063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 msgr2=0x7f8c840ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c840ae210 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8c780090b0 tx=0x7f8c78017000 comp rx=0 tx=0).stop 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 shutdown_connections 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c70060b90 0x7f8c70063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c84156c90 0x7f8c84158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c840cda30 0x7f8c840ae210 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 --2- 172.21.15.64:0/89283541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c840c90e0 0x7f8c840ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 >> 172.21.15.64:0/89283541 conn(0x7f8c8406c7c0 msgr2=0x7f8c840cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 shutdown_connections 2023-09-27T23:33:03.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:03.895+0000 7f8c89fd4700 1 -- 172.21.15.64:0/89283541 wait complete. 2023-09-27T23:33:03.899 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-09-27T23:33:03.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:03.921 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:03.919+0000 7f4fca4d9000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:33:03.996 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:03.994+0000 7fab55331000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 Processor -- start 2023-09-27T23:33:04.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 -- start start 2023-09-27T23:33:04.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f503415d140 0x7f50340670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f503406d050 0x7f503406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50340cd540 con 0x7f5034158df0 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503406d9f0 con 0x7f503415d140 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50340c7ed0 con 0x7f503406d050 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f503415d140 0x7f50340670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f503415d140 0x7f50340670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43058/0 (socket says 172.21.15.64:43058) 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 -- 172.21.15.64:0/2383975732 learned_addr learned my addr 172.21.15.64:0/2383975732 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503a51a700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f503406d050 0x7f503406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5038d17700 1 -- 172.21.15.64:0/2383975732 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383511264 0 0) 0x7f50340cd540 con 0x7f5034158df0 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 -- 172.21.15.64:0/2383975732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f503406d050 msgr2=0x7f503406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f503406d050 0x7f503406d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 -- 172.21.15.64:0/2383975732 >> v1:172.21.15.123:6789/0 conn(0x7f5034158df0 legacy=0x7f503415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 -- 172.21.15.64:0/2383975732 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503415d610 con 0x7f503415d140 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503a51a700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f503406d050 0x7f503406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5039518700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340670e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f502c009f20 tx=0x7f502c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4151f8b0d3f57e1e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5038d17700 1 -- 172.21.15.64:0/2383975732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00e070 con 0x7f503415d140 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5038d17700 1 -- 172.21.15.64:0/2383975732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c00b9d0 con 0x7f503415d140 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f5038d17700 1 -- 172.21.15.64:0/2383975732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0092e0 con 0x7f503415d140 2023-09-27T23:33:04.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 -- 172.21.15.64:0/2383975732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 msgr2=0x7f50340670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340670e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f502c009f20 tx=0x7f502c009840 comp rx=0 tx=0).stop 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.011+0000 7f503ad1b700 1 -- 172.21.15.64:0/2383975732 shutdown_connections 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f503406d050 0x7f503406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 --2- 172.21.15.64:0/2383975732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340670e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- 172.21.15.64:0/2383975732 >> 172.21.15.64:0/2383975732 conn(0x7f503406c080 msgr2=0x7f50340ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- 172.21.15.64:0/2383975732 shutdown_connections 2023-09-27T23:33:04.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- 172.21.15.64:0/2383975732 wait complete. 2023-09-27T23:33:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 Processor -- start 2023-09-27T23:33:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- start start 2023-09-27T23:33:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503406d050 0x7f50340c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5034158df0 0x7f50340af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50340b17d0 con 0x7f503415d140 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50340b1ab0 con 0x7f503406d050 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50340b1d90 con 0x7f5034158df0 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f5039d19700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503406d050 0x7f50340c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43066/0 (socket says 172.21.15.64:43066) 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 -- 172.21.15.64:0/1131609995 learned_addr learned my addr 172.21.15.64:0/1131609995 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f5039518700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5034158df0 0x7f50340af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 -- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5034158df0 msgr2=0x7f50340af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5034158df0 0x7f50340af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 -- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503406d050 msgr2=0x7f50340c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503406d050 0x7f50340c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 -- 172.21.15.64:0/1131609995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028009420 con 0x7f503415d140 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f5039d19700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503406d050 0x7f50340c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503a51a700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340b0330 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f502800ea20 tx=0x7f50280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800ee60 con 0x7f503415d140 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c009ae0 con 0x7f503415d140 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800c850 con 0x7f503415d140 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800cc10 con 0x7f503415d140 2023-09-27T23:33:04.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.015+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50340afee0 con 0x7f503415d140 2023-09-27T23:33:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.019+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50340c6600 con 0x7f503415d140 2023-09-27T23:33:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.019+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f5028013470 con 0x7f503415d140 2023-09-27T23:33:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.019+0000 7f5025ffb700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5018060b90 0x7f5018063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.019+0000 7f5039d19700 1 -- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5018060b90 msgr2=0x7f5018063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.019+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f502807d4e0 con 0x7f503415d140 2023-09-27T23:33:04.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.019+0000 7f5039d19700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5018060b90 0x7f5018063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:04.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.023+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f502804d190 con 0x7f503415d140 2023-09-27T23:33:04.089 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.087+0000 7f4fca4d9000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:04.190 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.186+0000 7fab55331000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:33:04.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.203+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f503404be40 con 0x7f503415d140 2023-09-27T23:33:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f5025ffb700 1 -- 172.21.15.64:0/1131609995 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17584 (secure 0 0 0) 0x7f50280507b0 con 0x7f503415d140 2023-09-27T23:33:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:04.209 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":130,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:32:22.582544+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:32:04.410437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-09-27T23:32:04.813612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3012318414":"2023-09-28T23:32:22.582464+0000","172.21.15.123:6832/33228":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4117106670":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/1295929548":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4175644346":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3675917224":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4076639863":"2023-09-28T23:32:22.582464+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5018060b90 msgr2=0x7f5018063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5018060b90 0x7f5018063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 msgr2=0x7f50340b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340b0330 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f502800ea20 tx=0x7f50280091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 shutdown_connections 2023-09-27T23:33:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5018060b90 0x7f5018063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f503415d140 0x7f50340b0330 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5034158df0 0x7f50340af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 --2- 172.21.15.64:0/1131609995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503406d050 0x7f50340c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 >> 172.21.15.64:0/1131609995 conn(0x7f503406c080 msgr2=0x7f50340cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 shutdown_connections 2023-09-27T23:33:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.207+0000 7f503ad1b700 1 -- 172.21.15.64:0/1131609995 wait complete. 2023-09-27T23:33:04.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:32:04.410437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:32:04.813612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:33:04.223 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:33:04.238 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.234+0000 7fab55331000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:04.295 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.291+0000 7f4fca4d9000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:33:04.348 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.343+0000 7f4fca4d9000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 Processor -- start 2023-09-27T23:33:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 -- start start 2023-09-27T23:33:04.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fac06b2c0 0x7f1fac06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06d680 0x7f1fac0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac06c7c0 con 0x7f1fac06d210 2023-09-27T23:33:04.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac06cd80 con 0x7f1fac06b2c0 2023-09-27T23:33:04.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fb1b75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0b0940 con 0x7f1fac06d680 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fac06b2c0 0x7f1fac06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fac06b2c0 0x7f1fac06b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43070/0 (socket says 172.21.15.64:43070) 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1faaffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1fac06d210 0x7f1fac0afbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43680/0 (socket says 172.21.15.64:43680) 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fab7fe700 1 -- 172.21.15.64:0/1434299241 learned_addr learned my addr 172.21.15.64:0/1434299241 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.379+0000 7f1fabfff700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06d680 0x7f1fac0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fab7fe700 1 -- 172.21.15.64:0/1434299241 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06d680 msgr2=0x7f1fac0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fab7fe700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06d680 0x7f1fac0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fab7fe700 1 -- 172.21.15.64:0/1434299241 >> v1:172.21.15.64:6789/0 conn(0x7f1fac06d210 legacy=0x7f1fac0afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fab7fe700 1 -- 172.21.15.64:0/1434299241 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0b0c20 con 0x7f1fac06b2c0 2023-09-27T23:33:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fabfff700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06d680 0x7f1fac0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fab7fe700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06b2c0 0x7f1fac06b690 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0099b0 tx=0x7f1f9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f75e06549e825f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1faa7fc700 1 -- 172.21.15.64:0/1434299241 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00b860 con 0x7f1fac06b2c0 2023-09-27T23:33:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1faa7fc700 1 -- 172.21.15.64:0/1434299241 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c00b9c0 con 0x7f1fac06b2c0 2023-09-27T23:33:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1faa7fc700 1 -- 172.21.15.64:0/1434299241 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00bd60 con 0x7f1fac06b2c0 2023-09-27T23:33:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1434299241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06b2c0 msgr2=0x7f1fac06b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06b2c0 0x7f1fac06b690 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0099b0 tx=0x7f1f9c009250 comp rx=0 tx=0).stop 2023-09-27T23:33:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1434299241 shutdown_connections 2023-09-27T23:33:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06d680 0x7f1fac0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1434299241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06b2c0 0x7f1fac06b690 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1434299241 >> 172.21.15.64:0/1434299241 conn(0x7f1fac0cc580 msgr2=0x7f1fac0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1434299241 shutdown_connections 2023-09-27T23:33:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.383+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1434299241 wait complete. 2023-09-27T23:33:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 Processor -- start 2023-09-27T23:33:04.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 -- start start 2023-09-27T23:33:04.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06b2c0 0x7f1fac13f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 0x7f1fac15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac06d680 0x7f1fac1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0aa600 con 0x7f1fac06d210 2023-09-27T23:33:04.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac16a810 con 0x7f1fac06d680 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac16aa60 con 0x7f1fac06b2c0 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 0x7f1fac15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 0x7f1fac15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43078/0 (socket says 172.21.15.64:43078) 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 -- 172.21.15.64:0/1307143163 learned_addr learned my addr 172.21.15.64:0/1307143163 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fabfff700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac06d680 0x7f1fac1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fab7fe700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06b2c0 0x7f1fac13f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 -- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06b2c0 msgr2=0x7f1fac13f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06b2c0 0x7f1fac13f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 -- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac06d680 msgr2=0x7f1fac1530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac06d680 0x7f1fac1530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 -- 172.21.15.64:0/1307143163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0094f0 con 0x7f1fac06d210 2023-09-27T23:33:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1faaffd700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 0x7f1fac15c3f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c7f0 tx=0x7f1fa0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:04.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0005a70 con 0x7f1fac06d210 2023-09-27T23:33:04.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0005bf0 con 0x7f1fac06d210 2023-09-27T23:33:04.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac1513f0 con 0x7f1fac06d210 2023-09-27T23:33:04.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00070d0 con 0x7f1fac06d210 2023-09-27T23:33:04.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.387+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac1519b0 con 0x7f1fac06d210 2023-09-27T23:33:04.393 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.386+0000 7fab55331000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:33:04.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.391+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac0aa7b0 con 0x7f1fac06d210 2023-09-27T23:33:04.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.391+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f1fa0007440 con 0x7f1fac06d210 2023-09-27T23:33:04.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.391+0000 7f1f937fe700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f98060b90 0x7f1f98063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.391+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f1fa007cb40 con 0x7f1fac06d210 2023-09-27T23:33:04.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.391+0000 7f1fab7fe700 1 -- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f98060b90 msgr2=0x7f1f98063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:04.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.391+0000 7f1fab7fe700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f98060b90 0x7f1f98063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:04.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.399+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1fa004c890 con 0x7f1fac06d210 2023-09-27T23:33:04.448 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.446+0000 7fab55331000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:33:04.503 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.499+0000 7f4fca4d9000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:33:04.506 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.502+0000 7fab55331000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:33:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.555+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fac0aa7b0 con 0x7f1fac06d210 2023-09-27T23:33:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1f937fe700 1 -- 172.21.15.64:0/1307143163 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fac0aa7b0 con 0x7f1fac06d210 2023-09-27T23:33:04.562 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:33:04.562 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f4fca4d9000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:33:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f98060b90 msgr2=0x7f1f98063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f98060b90 0x7f1f98063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 msgr2=0x7f1fac15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 0x7f1fac15c3f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c7f0 tx=0x7f1fa0007f30 comp rx=0 tx=0).stop 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 shutdown_connections 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f98060b90 0x7f1f98063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac06d680 0x7f1fac1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac06d210 0x7f1fac15c3f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 --2- 172.21.15.64:0/1307143163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac06b2c0 0x7f1fac13f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 >> 172.21.15.64:0/1307143163 conn(0x7f1fac0cc580 msgr2=0x7f1fac157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 shutdown_connections 2023-09-27T23:33:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.559+0000 7f1fb1b75700 1 -- 172.21.15.64:0/1307143163 wait complete. 2023-09-27T23:33:04.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:33:04.649 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.647+0000 7f4fca4d9000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:33:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f759e700 1 Processor -- start 2023-09-27T23:33:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f759e700 1 -- start start 2023-09-27T23:33:04.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9f80ae6c0 0x7fc9f806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f80ad020 con 0x7fc9f8155520 2023-09-27T23:33:04.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f80c81e0 con 0x7fc9f806b360 2023-09-27T23:33:04.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f806b7e0 con 0x7fc9f80ae6c0 2023-09-27T23:33:04.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc9f8155520 0x7fc9f80c7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39146/0 (socket says 172.21.15.64:39146) 2023-09-27T23:33:04.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.695+0000 7fc9f659c700 1 -- 172.21.15.64:0/3971754017 learned_addr learned my addr 172.21.15.64:0/3971754017 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:04.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 --2- 172.21.15.64:0/3971754017 >> v2:172.21.15.123:3300/0 conn(0x7fc9f80ae6c0 0x7fc9f806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293467622 0 0) 0x7fc9f80ad020 con 0x7fc9f8155520 2023-09-27T23:33:04.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e0003570 con 0x7fc9f8155520 2023-09-27T23:33:04.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569201586 0 0) 0x7fc9e0003570 con 0x7fc9f8155520 2023-09-27T23:33:04.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 >> v2:172.21.15.123:3300/0 conn(0x7fc9f80ae6c0 msgr2=0x7fc9f806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:04.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 --2- 172.21.15.64:0/3971754017 >> v2:172.21.15.123:3300/0 conn(0x7fc9f80ae6c0 0x7fc9f806ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 >> v1:172.21.15.123:6789/0 conn(0x7fc9f806b360 legacy=0x7fc9f80ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80aca10 con 0x7fc9f8155520 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fc9e8001470 con 0x7fc9f8155520 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 >> v1:172.21.15.64:6789/0 conn(0x7fc9f8155520 legacy=0x7fc9f80c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0003a60 0x7fc9e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e0004810 0x7fc9e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9e00051c0 0x7fc9e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f806b7e0 con 0x7fc9e0004810 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e000c0d0 con 0x7fc9e0003a60 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e000c290 con 0x7fc9e00051c0 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e0004810 0x7fc9e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f659c700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0003a60 0x7fc9e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f6d9d700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9e00051c0 0x7fc9e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 -- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9e00051c0 msgr2=0x7fc9e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9e00051c0 0x7fc9e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 -- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0003a60 msgr2=0x7fc9e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0003a60 0x7fc9e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 -- 172.21.15.64:0/3971754017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0003570 con 0x7fc9e0004810 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f6d9d700 1 -- 172.21.15.64:0/3971754017 reap_dead start 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f5d9b700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e0004810 0x7fc9e0004c80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80aea90 tx=0x7fc9ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfccfc06763b9305 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00e6e0 con 0x7fc9e0004810 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec00e840 con 0x7fc9e0004810 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f559a700 1 -- 172.21.15.64:0/3971754017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00ec00 con 0x7fc9e0004810 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 -- 172.21.15.64:0/3971754017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e0004810 msgr2=0x7fc9e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e0004810 0x7fc9e0004c80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80aea90 tx=0x7fc9ec0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 -- 172.21.15.64:0/3971754017 shutdown_connections 2023-09-27T23:33:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 --2- 172.21.15.64:0/3971754017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e0004810 0x7fc9e0004c80 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 -- 172.21.15.64:0/3971754017 >> 172.21.15.64:0/3971754017 conn(0x7fc9f80c8cd0 msgr2=0x7fc9f80c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 -- 172.21.15.64:0/3971754017 shutdown_connections 2023-09-27T23:33:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.699+0000 7fc9f759e700 1 -- 172.21.15.64:0/3971754017 wait complete. 2023-09-27T23:33:04.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 Processor -- start 2023-09-27T23:33:04.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 -- start start 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0004810 0x7fc9f814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9f80b15a0 0x7fc9f80c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 0x7fc9f80c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80af0f0 con 0x7fc9f80c3850 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c1ab0 con 0x7fc9e0004810 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c1d90 con 0x7fc9f80b15a0 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 0x7fc9f80c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 0x7fc9f80c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42114/0 (socket says 172.21.15.64:42114) 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 -- 172.21.15.64:0/822580584 learned_addr learned my addr 172.21.15.64:0/822580584 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f659c700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0004810 0x7fc9f814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f5d9b700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9f80b15a0 0x7fc9f80c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 -- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9f80b15a0 msgr2=0x7fc9f80c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9f80b15a0 0x7fc9f80c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 -- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0004810 msgr2=0x7fc9f814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0004810 0x7fc9f814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 -- 172.21.15.64:0/822580584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80c2070 con 0x7fc9f80c3850 2023-09-27T23:33:04.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.703+0000 7fc9f6d9d700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 0x7fc9f80c3cf0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0007fd0 tx=0x7fc9f0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00032e0 con 0x7fc9f80c3850 2023-09-27T23:33:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f0001580 con 0x7fc9f80c3850 2023-09-27T23:33:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f80c46f0 con 0x7fc9f80c3850 2023-09-27T23:33:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0001940 con 0x7fc9f80c3850 2023-09-27T23:33:04.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f814df40 con 0x7fc9f80c3850 2023-09-27T23:33:04.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f80c4530 con 0x7fc9f80c3850 2023-09-27T23:33:04.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fc9f0001cb0 con 0x7fc9f80c3850 2023-09-27T23:33:04.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9e67fc700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc9d8060c90 0x7fc9d8063140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:04.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fc9f007ccd0 con 0x7fc9f80c3850 2023-09-27T23:33:04.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9f659c700 1 -- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc9d8060c90 msgr2=0x7fc9d8063140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:04.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.707+0000 7fc9f659c700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc9d8060c90 0x7fc9d8063140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:04.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.715+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc9f004caa0 con 0x7fc9f80c3850 2023-09-27T23:33:04.802 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.798+0000 7fab55331000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:33:04.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.875+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9f804be40 con 0x7fc9f80c3850 2023-09-27T23:33:04.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9e67fc700 1 -- 172.21.15.64:0/822580584 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v130) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc9f00190a0 con 0x7fc9f80c3850 2023-09-27T23:33:04.881 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc9d8060c90 msgr2=0x7fc9d8063140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc9d8060c90 0x7fc9d8063140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 msgr2=0x7fc9f80c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 0x7fc9f80c3cf0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0007fd0 tx=0x7fc9f0009040 comp rx=0 tx=0).stop 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 shutdown_connections 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc9d8060c90 0x7fc9d8063140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f80c3850 0x7fc9f80c3cf0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9f80b15a0 0x7fc9f80c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 --2- 172.21.15.64:0/822580584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9e0004810 0x7fc9f814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:04.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 >> 172.21.15.64:0/822580584 conn(0x7fc9f80c8cd0 msgr2=0x7fc9f806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:04.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 shutdown_connections 2023-09-27T23:33:04.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.879+0000 7fc9f759e700 1 -- 172.21.15.64:0/822580584 wait complete. 2023-09-27T23:33:04.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:33:04.914 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:04.910+0000 7fab55331000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:33:04.980 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:04.975+0000 7f4fca4d9000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:33:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.995+0000 7f159b5f2700 1 Processor -- start 2023-09-27T23:33:04.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.995+0000 7f159b5f2700 1 -- start start 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15940ab870 0x7f15940afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15940b0e40 con 0x7f15940b08d0 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15940b1120 con 0x7f159415a180 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15940b1400 con 0x7f15940ab870 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f1598b8d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f15940b08d0 0x7f1594159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39152/0 (socket says 172.21.15.64:39152) 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f1598b8d700 1 -- 172.21.15.64:0/3713239794 learned_addr learned my addr 172.21.15.64:0/3713239794 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:05.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159938e700 1 --2- 172.21.15.64:0/3713239794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15940ab870 0x7f15940afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829388980 0 0) 0x7f15940b0e40 con 0x7f15940b08d0 2023-09-27T23:33:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1580003570 con 0x7f15940b08d0 2023-09-27T23:33:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602600512 0 0) 0x7f15940b1120 con 0x7f159415a180 2023-09-27T23:33:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15940b0e40 con 0x7f159415a180 2023-09-27T23:33:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034274787 0 0) 0x7f1580003570 con 0x7f15940b08d0 2023-09-27T23:33:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159938e700 1 -- 172.21.15.64:0/3713239794 >> v1:172.21.15.123:6789/0 conn(0x7f159415a180 legacy=0x7f159406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159938e700 1 -- 172.21.15.64:0/3713239794 >> v1:172.21.15.64:6789/0 conn(0x7f15940b08d0 legacy=0x7f1594159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159938e700 1 -- 172.21.15.64:0/3713239794 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940b16e0 con 0x7f15940ab870 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159938e700 1 --2- 172.21.15.64:0/3713239794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15940ab870 0x7f15940afdb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1590009910 tx=0x7f15900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e70af1916302f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584001470 con 0x7f15940ab870 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590011420 con 0x7f15940ab870 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f158bfff700 1 -- 172.21.15.64:0/3713239794 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900117a0 con 0x7f15940ab870 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- 172.21.15.64:0/3713239794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15940ab870 msgr2=0x7f15940afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 --2- 172.21.15.64:0/3713239794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15940ab870 0x7f15940afdb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1590009910 tx=0x7f15900091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- 172.21.15.64:0/3713239794 shutdown_connections 2023-09-27T23:33:05.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 --2- 172.21.15.64:0/3713239794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15940ab870 0x7f15940afdb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- 172.21.15.64:0/3713239794 >> 172.21.15.64:0/3713239794 conn(0x7f15940cc580 msgr2=0x7f15940cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- 172.21.15.64:0/3713239794 shutdown_connections 2023-09-27T23:33:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:04.999+0000 7f159b5f2700 1 -- 172.21.15.64:0/3713239794 wait complete. 2023-09-27T23:33:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 Processor -- start 2023-09-27T23:33:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 -- start start 2023-09-27T23:33:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 0x7f15940cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15940b08d0 0x7f15940cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159415a180 0x7f15940c2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940c3a10 con 0x7f15940ab870 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940c2e00 con 0x7f15940b08d0 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15940c30e0 con 0x7f159415a180 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 0x7f15940cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 0x7f15940cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42122/0 (socket says 172.21.15.64:42122) 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 -- 172.21.15.64:0/4054333665 learned_addr learned my addr 172.21.15.64:0/4054333665 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f1598b8d700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15940b08d0 0x7f15940cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f1599b8f700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159415a180 0x7f15940c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 -- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159415a180 msgr2=0x7f15940c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159415a180 0x7f15940c2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 -- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15940b08d0 msgr2=0x7f15940cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15940b08d0 0x7f15940cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 -- 172.21.15.64:0/4054333665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590009420 con 0x7f15940ab870 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159938e700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 0x7f15940cccc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f15900091e0 tx=0x7f1590016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159001caf0 con 0x7f15940ab870 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159001cc50 con 0x7f15940ab870 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15940c33c0 con 0x7f15940ab870 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.003+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159001a450 con 0x7f15940ab870 2023-09-27T23:33:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15941519c0 con 0x7f15940ab870 2023-09-27T23:33:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f159001a820 con 0x7f15940ab870 2023-09-27T23:33:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15940c3840 con 0x7f15940ab870 2023-09-27T23:33:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f15897fa700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1578060b90 0x7f1578063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f159007ee20 con 0x7f15940ab870 2023-09-27T23:33:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f1598b8d700 1 -- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1578060b90 msgr2=0x7f1578063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.007+0000 7f1598b8d700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1578060b90 0x7f1578063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:05.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.015+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f159004eaa0 con 0x7f15940ab870 2023-09-27T23:33:05.097 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.095+0000 7f4fca4d9000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:33:05.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.171+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1594071120 con 0x7f15940ab870 2023-09-27T23:33:05.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.171+0000 7f15897fa700 1 -- 172.21.15.64:0/4054333665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15900520c0 con 0x7f15940ab870 2023-09-27T23:33:05.175 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:33:05.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1578060b90 msgr2=0x7f1578063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:05.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1578060b90 0x7f1578063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 msgr2=0x7f15940cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 0x7f15940cccc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f15900091e0 tx=0x7f1590016000 comp rx=0 tx=0).stop 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 shutdown_connections 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1578060b90 0x7f1578063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159415a180 0x7f15940c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15940b08d0 0x7f15940cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 --2- 172.21.15.64:0/4054333665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15940ab870 0x7f15940cccc0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 >> 172.21.15.64:0/4054333665 conn(0x7f15940cc580 msgr2=0x7f1594152510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 shutdown_connections 2023-09-27T23:33:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.175+0000 7f159b5f2700 1 -- 172.21.15.64:0/4054333665 wait complete. 2023-09-27T23:33:05.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:33:05.197 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.194+0000 7fab55331000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:33:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.303+0000 7f19cb59e700 1 Processor -- start 2023-09-27T23:33:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.303+0000 7f19cb59e700 1 -- start start 2023-09-27T23:33:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19cc1547a0 0x7f19cc0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19cc157fc0 0x7f19cc158480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19cc06da10 con 0x7f19cc1547a0 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19cc0c7820 con 0x7f19cc157fc0 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc0c7210 con 0x7f19cc06d050 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19cc1547a0 0x7f19cc0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19cc1547a0 0x7f19cc0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42124/0 (socket says 172.21.15.64:42124) 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cad9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 -- 172.21.15.64:0/683699070 learned_addr learned my addr 172.21.15.64:0/683699070 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19c9d9b700 1 --2- 172.21.15.64:0/683699070 >> v2:172.21.15.123:3300/0 conn(0x7f19cc157fc0 0x7f19cc158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 -- 172.21.15.64:0/683699070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 msgr2=0x7f19cc06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 --2- 172.21.15.64:0/683699070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 -- 172.21.15.64:0/683699070 >> v2:172.21.15.123:3300/0 conn(0x7f19cc157fc0 msgr2=0x7f19cc158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 --2- 172.21.15.64:0/683699070 >> v2:172.21.15.123:3300/0 conn(0x7f19cc157fc0 0x7f19cc158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 -- 172.21.15.64:0/683699070 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc0ae030 con 0x7f19cc1547a0 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19ca59c700 1 --2- 172.21.15.64:0/683699070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc1547a0 0x7f19cc0ad730 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0cd930 tx=0x7f19bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242852f2961ab9fc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19c959a700 1 -- 172.21.15.64:0/683699070 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00b640 con 0x7f19cc1547a0 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19c959a700 1 -- 172.21.15.64:0/683699070 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00b7a0 con 0x7f19cc1547a0 2023-09-27T23:33:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19c959a700 1 -- 172.21.15.64:0/683699070 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00bb60 con 0x7f19cc1547a0 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- 172.21.15.64:0/683699070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc1547a0 msgr2=0x7f19cc0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- 172.21.15.64:0/683699070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc1547a0 0x7f19cc0ad730 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0cd930 tx=0x7f19bc009250 comp rx=0 tx=0).stop 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- 172.21.15.64:0/683699070 shutdown_connections 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- 172.21.15.64:0/683699070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- 172.21.15.64:0/683699070 >> v2:172.21.15.123:3300/0 conn(0x7f19cc157fc0 0x7f19cc158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 --2- 172.21.15.64:0/683699070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc1547a0 0x7f19cc0ad730 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- 172.21.15.64:0/683699070 >> 172.21.15.64:0/683699070 conn(0x7f19cc0ccb30 msgr2=0x7f19cc0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- 172.21.15.64:0/683699070 shutdown_connections 2023-09-27T23:33:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.307+0000 7f19cb59e700 1 -- 172.21.15.64:0/683699070 wait complete. 2023-09-27T23:33:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 Processor -- start 2023-09-27T23:33:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 -- start start 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19cc1547a0 0x7f19cc156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 0x7f19cc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc0c5160 con 0x7f19cc157fc0 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc157560 con 0x7f19cc1547a0 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc157840 con 0x7f19cc06d050 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 0x7f19cc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 0x7f19cc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42132/0 (socket says 172.21.15.64:42132) 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19ca59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 -- 172.21.15.64:0/1107546130 learned_addr learned my addr 172.21.15.64:0/1107546130 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19c9d9b700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19cc1547a0 0x7f19cc156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 -- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 msgr2=0x7f19cc147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 -- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19cc1547a0 msgr2=0x7f19cc156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19cc1547a0 0x7f19cc156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 -- 172.21.15.64:0/1107546130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0094f0 con 0x7f19cc157fc0 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19ca59c700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cad9d700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 0x7f19cc154b70 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009df0 tx=0x7f19c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400caa0 con 0x7f19cc157fc0 2023-09-27T23:33:05.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400f040 con 0x7f19cc157fc0 2023-09-27T23:33:05.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc0c4b10 con 0x7f19cc157fc0 2023-09-27T23:33:05.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400ce70 con 0x7f19cc157fc0 2023-09-27T23:33:05.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.311+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc147da0 con 0x7f19cc157fc0 2023-09-27T23:33:05.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.315+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc0c5310 con 0x7f19cc157fc0 2023-09-27T23:33:05.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.315+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f19c4014690 con 0x7f19cc157fc0 2023-09-27T23:33:05.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.315+0000 7f19ba7fc700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19ac060b90 0x7f19ac063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.315+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f19c407e110 con 0x7f19cc157fc0 2023-09-27T23:33:05.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.315+0000 7f19ca59c700 1 -- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19ac060b90 msgr2=0x7f19ac063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:05.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.315+0000 7f19ca59c700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19ac060b90 0x7f19ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:05.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.323+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f19cc0c5310 con 0x7f19cc157fc0 2023-09-27T23:33:05.361 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.358+0000 7fab55331000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:05.422 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.419+0000 7f4fca4d9000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:33:05.425 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.422+0000 7fab55331000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:33:05.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.471+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19cc04be40 con 0x7f19cc157fc0 2023-09-27T23:33:05.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.471+0000 7f19ba7fc700 1 -- 172.21.15.64:0/1107546130 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19c40517e0 con 0x7f19cc157fc0 2023-09-27T23:33:05.476 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:33:05.478 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.474+0000 7fab55331000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:33:05.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19ac060b90 msgr2=0x7f19ac063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19ac060b90 0x7f19ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 msgr2=0x7f19cc154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 0x7f19cc154b70 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009df0 tx=0x7f19c400b040 comp rx=0 tx=0).stop 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 shutdown_connections 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f19ac060b90 0x7f19ac063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19cc157fc0 0x7f19cc154b70 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19cc1547a0 0x7f19cc156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 --2- 172.21.15.64:0/1107546130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19cc06d050 0x7f19cc147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 >> 172.21.15.64:0/1107546130 conn(0x7f19cc0ccb30 msgr2=0x7f19cc148170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.475+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 shutdown_connections 2023-09-27T23:33:05.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.479+0000 7f19cb59e700 1 -- 172.21.15.64:0/1107546130 wait complete. 2023-09-27T23:33:05.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:05.531 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.526+0000 7fab55331000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:33:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 Processor -- start 2023-09-27T23:33:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 -- start start 2023-09-27T23:33:05.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 0x7facb806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 0x7facb815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80ad070 con 0x7facb8068160 2023-09-27T23:33:05.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb80ac780 con 0x7facb806d280 2023-09-27T23:33:05.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbdb08700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb8068e90 con 0x7facb815aa10 2023-09-27T23:33:05.604 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.599+0000 7f4fca4d9000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 0x7facb815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbcb06700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 0x7facb806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 0x7facb815ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:60016/0 (socket says 172.21.15.64:60016) 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbcb06700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 0x7facb806d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:38022/0 (socket says 172.21.15.64:38022) 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 -- 172.21.15.64:0/882681044 learned_addr learned my addr 172.21.15.64:0/882681044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 -- 172.21.15.64:0/882681044 >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 msgr2=0x7facb806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 --2- 172.21.15.64:0/882681044 >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 0x7facb806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 -- 172.21.15.64:0/882681044 >> v1:172.21.15.123:6789/0 conn(0x7facb8068160 legacy=0x7facb80685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbd307700 1 -- 172.21.15.64:0/882681044 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80aca20 con 0x7facb815aa10 2023-09-27T23:33:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.599+0000 7facbcb06700 1 --2- 172.21.15.64:0/882681044 >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 0x7facb806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:05.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbd307700 1 --2- 172.21.15.64:0/882681044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 0x7facb815ae70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7facb0009f20 tx=0x7facb0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489f5b7b89e1f54a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facb77fe700 1 -- 172.21.15.64:0/882681044 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000e070 con 0x7facb815aa10 2023-09-27T23:33:05.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facb77fe700 1 -- 172.21.15.64:0/882681044 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb000ba60 con 0x7facb815aa10 2023-09-27T23:33:05.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facb77fe700 1 -- 172.21.15.64:0/882681044 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb00092e0 con 0x7facb815aa10 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 -- 172.21.15.64:0/882681044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 msgr2=0x7facb815ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 --2- 172.21.15.64:0/882681044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 0x7facb815ae70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7facb0009f20 tx=0x7facb0009840 comp rx=0 tx=0).stop 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 -- 172.21.15.64:0/882681044 shutdown_connections 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 --2- 172.21.15.64:0/882681044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb815aa10 0x7facb815ae70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 --2- 172.21.15.64:0/882681044 >> v2:172.21.15.123:3300/0 conn(0x7facb806d280 0x7facb806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 -- 172.21.15.64:0/882681044 >> 172.21.15.64:0/882681044 conn(0x7facb806c080 msgr2=0x7facb80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 -- 172.21.15.64:0/882681044 shutdown_connections 2023-09-27T23:33:05.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 -- 172.21.15.64:0/882681044 wait complete. 2023-09-27T23:33:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 Processor -- start 2023-09-27T23:33:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.603+0000 7facbdb08700 1 -- start start 2023-09-27T23:33:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 0x7facb8148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb806d280 0x7facb80b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb815aa10 0x7facb80b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb8149b80 con 0x7facb8068160 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb8149e60 con 0x7facb815aa10 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb814a140 con 0x7facb806d280 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 0x7facb8148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 0x7facb8148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42136/0 (socket says 172.21.15.64:42136) 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 -- 172.21.15.64:0/720286980 learned_addr learned my addr 172.21.15.64:0/720286980 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbd307700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb815aa10 0x7facb80b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facb7fff700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb806d280 0x7facb80b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 -- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb806d280 msgr2=0x7facb80b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb806d280 0x7facb80b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 -- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb815aa10 msgr2=0x7facb80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb815aa10 0x7facb80b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 -- 172.21.15.64:0/720286980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac009420 con 0x7facb8068160 2023-09-27T23:33:05.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbcb06700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 0x7facb8148740 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7facac00ec80 tx=0x7facac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00c920 con 0x7facb8068160 2023-09-27T23:33:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac00ca80 con 0x7facb8068160 2023-09-27T23:33:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb0009ae0 con 0x7facb8068160 2023-09-27T23:33:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00ce00 con 0x7facb8068160 2023-09-27T23:33:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.607+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb814a700 con 0x7facb8068160 2023-09-27T23:33:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.611+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb804be40 con 0x7facb8068160 2023-09-27T23:33:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.611+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7facac013680 con 0x7facb8068160 2023-09-27T23:33:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.611+0000 7facb4ff9700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 0x7fac9c063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.611+0000 7facb7fff700 1 -- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 msgr2=0x7fac9c063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.611+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7facac07d860 con 0x7facb8068160 2023-09-27T23:33:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.611+0000 7facb7fff700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 0x7fac9c063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.619+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7facac04d5b0 con 0x7facb8068160 2023-09-27T23:33:05.673 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.671+0000 7f4fca4d9000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:33:05.707 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.702+0000 7fab55331000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:05.732 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.731+0000 7f4fca4d9000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:33:05.761 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.758+0000 7fab55331000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:33:05.793 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.791+0000 7f4fca4d9000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:33:05.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.811+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facb8063210 con 0x7facb8068160 2023-09-27T23:33:05.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facb7fff700 1 -- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 msgr2=0x7fac9c063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facb7fff700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 0x7fac9c063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:33:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facb4ff9700 1 -- 172.21.15.64:0/720286980 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4602 (secure 0 0 0) 0x7facac050bd0 con 0x7facb8068160 2023-09-27T23:33:05.817 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:05.817 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:32:05.833505+0000","modified":"2023-09-27T23:32:08.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8151},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8151":{"gid":8151,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4228640199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4228640199},{"type":"v1","addr":"172.21.15.64:6835","nonce":4228640199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-09-27T23:33:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 msgr2=0x7fac9c063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 0x7fac9c063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 msgr2=0x7facb8148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 0x7facb8148740 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7facac00ec80 tx=0x7facac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 shutdown_connections 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fac9c060b90 0x7fac9c063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb815aa10 0x7facb80b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facb806d280 0x7facb80b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 --2- 172.21.15.64:0/720286980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facb8068160 0x7facb8148740 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 >> 172.21.15.64:0/720286980 conn(0x7facb806c080 msgr2=0x7facb80cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 shutdown_connections 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.815+0000 7facbdb08700 1 -- 172.21.15.64:0/720286980 wait complete. 2023-09-27T23:33:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-09-27T23:33:05.822 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.818+0000 7fab55331000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:33:05.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.935+0000 7f31b9397700 1 Processor -- start 2023-09-27T23:33:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- start start 2023-09-27T23:33:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31b41552f0 0x7f31b41556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31b406c7c0 0x7f31b406cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b406d0f0 con 0x7f31b40ae520 2023-09-27T23:33:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b40c8710 con 0x7f31b41552f0 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b40c1b90 con 0x7f31b406c7c0 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b2ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31b41552f0 0x7f31b41556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31abfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f31b40ae520 0x7f31b40c8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39158/0 (socket says 172.21.15.64:39158) 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31abfff700 1 -- 172.21.15.64:0/2813036157 learned_addr learned my addr 172.21.15.64:0/2813036157 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b37fe700 1 --2- 172.21.15.64:0/2813036157 >> v2:172.21.15.123:3300/0 conn(0x7f31b406c7c0 0x7f31b406cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b2ffd700 1 -- 172.21.15.64:0/2813036157 >> v2:172.21.15.123:3300/0 conn(0x7f31b406c7c0 msgr2=0x7f31b406cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b2ffd700 1 --2- 172.21.15.64:0/2813036157 >> v2:172.21.15.123:3300/0 conn(0x7f31b406c7c0 0x7f31b406cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b2ffd700 1 -- 172.21.15.64:0/2813036157 >> v1:172.21.15.64:6789/0 conn(0x7f31b40ae520 legacy=0x7f31b40c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b27fc700 1 -- 172.21.15.64:0/2813036157 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264328477 0 0) 0x7f31b406d0f0 con 0x7f31b40ae520 2023-09-27T23:33:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b2ffd700 1 -- 172.21.15.64:0/2813036157 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b406a9d0 con 0x7f31b41552f0 2023-09-27T23:33:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b2ffd700 1 --2- 172.21.15.64:0/2813036157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b41552f0 0x7f31b41556c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f31b406e2b0 tx=0x7f31a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5c054d9352df81 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b27fc700 1 -- 172.21.15.64:0/2813036157 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400b6b0 con 0x7f31b41552f0 2023-09-27T23:33:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b27fc700 1 -- 172.21.15.64:0/2813036157 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a400b810 con 0x7f31b41552f0 2023-09-27T23:33:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b27fc700 1 -- 172.21.15.64:0/2813036157 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400bbb0 con 0x7f31b41552f0 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- 172.21.15.64:0/2813036157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b41552f0 msgr2=0x7f31b41556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 --2- 172.21.15.64:0/2813036157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b41552f0 0x7f31b41556c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f31b406e2b0 tx=0x7f31a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- 172.21.15.64:0/2813036157 shutdown_connections 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 --2- 172.21.15.64:0/2813036157 >> v2:172.21.15.123:3300/0 conn(0x7f31b406c7c0 0x7f31b406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 --2- 172.21.15.64:0/2813036157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b41552f0 0x7f31b41556c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- 172.21.15.64:0/2813036157 >> 172.21.15.64:0/2813036157 conn(0x7f31b40c8850 msgr2=0x7f31b40c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:05.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- 172.21.15.64:0/2813036157 shutdown_connections 2023-09-27T23:33:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.939+0000 7f31b9397700 1 -- 172.21.15.64:0/2813036157 wait complete. 2023-09-27T23:33:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 Processor -- start 2023-09-27T23:33:05.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 -- start start 2023-09-27T23:33:05.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 0x7f31b40af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b40ae520 0x7f31b40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41552f0 0x7f31b4141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40afaf0 con 0x7f31b406c7c0 2023-09-27T23:33:05.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40c5870 con 0x7f31b41552f0 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40b05a0 con 0x7f31b40ae520 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 0x7f31b40af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 0x7f31b40af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42160/0 (socket says 172.21.15.64:42160) 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 -- 172.21.15.64:0/1527763888 learned_addr learned my addr 172.21.15.64:0/1527763888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b37fe700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41552f0 0x7f31b4141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31abfff700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b40ae520 0x7f31b40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 -- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b40ae520 msgr2=0x7f31b40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b40ae520 0x7f31b40c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 -- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41552f0 msgr2=0x7f31b4141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41552f0 0x7f31b4141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 -- 172.21.15.64:0/1527763888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4009420 con 0x7f31b406c7c0 2023-09-27T23:33:05.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b2ffd700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 0x7f31b40af5b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f31b406a560 tx=0x7f31a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4000d10 con 0x7f31b406c7c0 2023-09-27T23:33:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a400bb90 con 0x7f31b406c7c0 2023-09-27T23:33:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b4156af0 con 0x7f31b406c7c0 2023-09-27T23:33:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4011650 con 0x7f31b406c7c0 2023-09-27T23:33:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.943+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b40affe0 con 0x7f31b406c7c0 2023-09-27T23:33:05.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.947+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b4071120 con 0x7f31b406c7c0 2023-09-27T23:33:05.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.947+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f31a4011a80 con 0x7f31b406c7c0 2023-09-27T23:33:05.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.947+0000 7f31aaffd700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3194060b90 0x7f3194063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:05.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.947+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f31a407ec70 con 0x7f31b406c7c0 2023-09-27T23:33:05.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.947+0000 7f31abfff700 1 -- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3194060b90 msgr2=0x7f3194063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:05.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.947+0000 7f31abfff700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3194060b90 0x7f3194063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:05.955+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f31a404e9c0 con 0x7f31b406c7c0 2023-09-27T23:33:05.970 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:05.966+0000 7fab55331000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:33:06.000 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:05.995+0000 7f4fca4d9000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:06.031 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.026+0000 7fab55331000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:33:06.061 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:06.059+0000 7f4fca4d9000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:33:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.123+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31b40642a0 con 0x7f31b406c7c0 2023-09-27T23:33:06.129 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f4fca4d9000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:33:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31aaffd700 1 -- 172.21.15.64:0/1527763888 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17584 (secure 0 0 0) 0x7f31a4015090 con 0x7f31b406c7c0 2023-09-27T23:33:06.130 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:06.130 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":130,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:32:22.582544+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:32:04.410437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-09-27T23:32:04.813612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3012318414":"2023-09-28T23:32:22.582464+0000","172.21.15.123:6832/33228":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4117106670":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/1295929548":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4175644346":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3675917224":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4076639863":"2023-09-28T23:32:22.582464+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3194060b90 msgr2=0x7f3194063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3194060b90 0x7f3194063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 msgr2=0x7f31b40af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 0x7f31b40af5b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f31b406a560 tx=0x7f31a4017000 comp rx=0 tx=0).stop 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 shutdown_connections 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3194060b90 0x7f3194063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41552f0 0x7f31b4141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b40ae520 0x7f31b40c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 --2- 172.21.15.64:0/1527763888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b406c7c0 0x7f31b40af5b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 >> 172.21.15.64:0/1527763888 conn(0x7f31b40c8850 msgr2=0x7f31b40670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 shutdown_connections 2023-09-27T23:33:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.127+0000 7f31b9397700 1 -- 172.21.15.64:0/1527763888 wait complete. 2023-09-27T23:33:06.144 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:33:06.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:06.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 Processor -- start 2023-09-27T23:33:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 -- start start 2023-09-27T23:33:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a3015cf70 0x7f7a300afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a3015d340 0x7f7a3006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3006cf40 con 0x7f7a300cbd00 2023-09-27T23:33:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300b0890 con 0x7f7a3015d340 2023-09-27T23:33:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.247+0000 7f7a2ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b21d0 con 0x7f7a3015cf70 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2e7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a3015d340 0x7f7a3006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7a300cbd00 0x7f7a30160d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39166/0 (socket says 172.21.15.64:39166) 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2f7fe700 1 -- 172.21.15.64:0/1798293510 learned_addr learned my addr 172.21.15.64:0/1798293510 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2effd700 1 --2- 172.21.15.64:0/1798293510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a3015cf70 0x7f7a300afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2e7fc700 1 -- 172.21.15.64:0/1798293510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a3015cf70 msgr2=0x7f7a300afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2e7fc700 1 --2- 172.21.15.64:0/1798293510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a3015cf70 0x7f7a300afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2e7fc700 1 -- 172.21.15.64:0/1798293510 >> v1:172.21.15.64:6789/0 conn(0x7f7a300cbd00 legacy=0x7f7a30160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2dffb700 1 -- 172.21.15.64:0/1798293510 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752881252 0 0) 0x7f7a3006cf40 con 0x7f7a300cbd00 2023-09-27T23:33:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2e7fc700 1 -- 172.21.15.64:0/1798293510 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300b0330 con 0x7f7a3015d340 2023-09-27T23:33:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2e7fc700 1 --2- 172.21.15.64:0/1798293510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a3006ad70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7a240098e0 tx=0x7f7a240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff6e26b933b19a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2dffb700 1 -- 172.21.15.64:0/1798293510 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24009e80 con 0x7f7a3015d340 2023-09-27T23:33:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2dffb700 1 -- 172.21.15.64:0/1798293510 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2400b860 con 0x7f7a3015d340 2023-09-27T23:33:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2dffb700 1 -- 172.21.15.64:0/1798293510 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400bbe0 con 0x7f7a3015d340 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- 172.21.15.64:0/1798293510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 msgr2=0x7f7a3006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/1798293510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a3006ad70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7a240098e0 tx=0x7f7a240091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- 172.21.15.64:0/1798293510 shutdown_connections 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/1798293510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a3006ad70 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/1798293510 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a3015cf70 0x7f7a300afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- 172.21.15.64:0/1798293510 >> 172.21.15.64:0/1798293510 conn(0x7f7a30067bb0 msgr2=0x7f7a300add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- 172.21.15.64:0/1798293510 shutdown_connections 2023-09-27T23:33:06.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- 172.21.15.64:0/1798293510 wait complete. 2023-09-27T23:33:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 Processor -- start 2023-09-27T23:33:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- start start 2023-09-27T23:33:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a300cbd00 0x7f7a3014c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3015cf70 0x7f7a3015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a301563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3014ce50 con 0x7f7a3015d340 2023-09-27T23:33:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30153f30 con 0x7f7a3015cf70 2023-09-27T23:33:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3015ad90 con 0x7f7a300cbd00 2023-09-27T23:33:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.251+0000 7f7a2f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a301563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a301563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42174/0 (socket says 172.21.15.64:42174) 2023-09-27T23:33:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 -- 172.21.15.64:0/4123580451 learned_addr learned my addr 172.21.15.64:0/4123580451 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2effd700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a300cbd00 0x7f7a3014c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2e7fc700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3015cf70 0x7f7a3015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 -- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a300cbd00 msgr2=0x7f7a3014c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a300cbd00 0x7f7a3014c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 -- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3015cf70 msgr2=0x7f7a3015a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3015cf70 0x7f7a3015a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 -- 172.21.15.64:0/4123580451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24009420 con 0x7f7a3015d340 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2f7fe700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a301563e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800cc90 tx=0x7f7a28001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800a870 con 0x7f7a3015d340 2023-09-27T23:33:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2800a9d0 con 0x7f7a3015d340 2023-09-27T23:33:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3015b070 con 0x7f7a3015d340 2023-09-27T23:33:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800ad90 con 0x7f7a3015d340 2023-09-27T23:33:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a30153bc0 con 0x7f7a3015d340 2023-09-27T23:33:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f7a28007300 con 0x7f7a3015d340 2023-09-27T23:33:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a301540c0 con 0x7f7a3015d340 2023-09-27T23:33:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a16ffd700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 0x7f7a1c063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f7a2807d830 con 0x7f7a3015d340 2023-09-27T23:33:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2effd700 1 -- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 msgr2=0x7f7a1c063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.255+0000 7f7a2effd700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 0x7f7a1c063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.263+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a2804d570 con 0x7f7a3015d340 2023-09-27T23:33:06.292 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:06.287+0000 7f4fca4d9000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:33:06.359 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:06.355+0000 7f4fca4d9000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:33:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.459+0000 7f7a2effd700 1 -- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 msgr2=0x7f7a1c063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.459+0000 7f7a2effd700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 0x7f7a1c063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:33:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.459+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a3004be40 con 0x7f7a3015d340 2023-09-27T23:33:06.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.463+0000 7f7a16ffd700 1 -- 172.21.15.64:0/4123580451 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f7a28050b90 con 0x7f7a3015d340 2023-09-27T23:33:06.465 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:06.465 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:32:05.833505+0000","modified":"2023-09-27T23:32:08.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8151},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8151":{"gid":8151,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4228640199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4228640199},{"type":"v1","addr":"172.21.15.64:6835","nonce":4228640199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-09-27T23:33:06.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 msgr2=0x7f7a1c063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 0x7f7a1c063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 msgr2=0x7f7a301563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a301563e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800cc90 tx=0x7f7a28001a20 comp rx=0 tx=0).stop 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 shutdown_connections 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a1c060b90 0x7f7a1c063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3015d340 0x7f7a301563e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3015cf70 0x7f7a3015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 --2- 172.21.15.64:0/4123580451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a300cbd00 0x7f7a3014c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 >> 172.21.15.64:0/4123580451 conn(0x7f7a30067bb0 msgr2=0x7f7a30159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:06.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 shutdown_connections 2023-09-27T23:33:06.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.467+0000 7f7a2ffff700 1 -- 172.21.15.64:0/4123580451 wait complete. 2023-09-27T23:33:06.471 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-09-27T23:33:06.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:06.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e9a234700 1 Processor -- start 2023-09-27T23:33:06.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e9a234700 1 -- start start 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e9a234700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e941591f0 0x7f0e9406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e9a234700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9406b730 con 0x7f0e940ce810 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e9a234700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9406ab20 con 0x7f0e94158e20 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e9a234700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9406ae00 con 0x7f0e941591f0 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e98a31700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e941591f0 0x7f0e9406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e99a33700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0e940ce810 0x7f0e940ca950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39174/0 (socket says 172.21.15.64:39174) 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e99a33700 1 -- 172.21.15.64:0/3093034421 learned_addr learned my addr 172.21.15.64:0/3093034421 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e8bfff700 1 -- 172.21.15.64:0/3093034421 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879636322 0 0) 0x7f0e9406b730 con 0x7f0e940ce810 2023-09-27T23:33:06.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e8bfff700 1 -- 172.21.15.64:0/3093034421 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e80003570 con 0x7f0e940ce810 2023-09-27T23:33:06.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e98a31700 1 -- 172.21.15.64:0/3093034421 >> v1:172.21.15.123:6789/0 conn(0x7f0e94158e20 legacy=0x7f0e940ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e8bfff700 1 -- 172.21.15.64:0/3093034421 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996700552 0 0) 0x7f0e80003570 con 0x7f0e940ce810 2023-09-27T23:33:06.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e98a31700 1 -- 172.21.15.64:0/3093034421 >> v1:172.21.15.64:6789/0 conn(0x7f0e940ce810 legacy=0x7f0e940ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.579+0000 7f0e98a31700 1 -- 172.21.15.64:0/3093034421 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9406b0e0 con 0x7f0e941591f0 2023-09-27T23:33:06.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e98a31700 1 --2- 172.21.15.64:0/3093034421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e941591f0 0x7f0e9406a540 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0e840098e0 tx=0x7f0e840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910cc2b07552f8d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:06.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e8bfff700 1 -- 172.21.15.64:0/3093034421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0017f0 con 0x7f0e941591f0 2023-09-27T23:33:06.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e8bfff700 1 -- 172.21.15.64:0/3093034421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e84011420 con 0x7f0e941591f0 2023-09-27T23:33:06.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e8bfff700 1 -- 172.21.15.64:0/3093034421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e840117c0 con 0x7f0e941591f0 2023-09-27T23:33:06.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 -- 172.21.15.64:0/3093034421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e941591f0 msgr2=0x7f0e9406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 --2- 172.21.15.64:0/3093034421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e941591f0 0x7f0e9406a540 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0e840098e0 tx=0x7f0e840091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 -- 172.21.15.64:0/3093034421 shutdown_connections 2023-09-27T23:33:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 --2- 172.21.15.64:0/3093034421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e941591f0 0x7f0e9406a540 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 -- 172.21.15.64:0/3093034421 >> 172.21.15.64:0/3093034421 conn(0x7f0e94067bb0 msgr2=0x7f0e940cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 -- 172.21.15.64:0/3093034421 shutdown_connections 2023-09-27T23:33:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 -- 172.21.15.64:0/3093034421 wait complete. 2023-09-27T23:33:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 Processor -- start 2023-09-27T23:33:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.583+0000 7f0e9a234700 1 -- start start 2023-09-27T23:33:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 0x7f0e9416d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e94158e20 0x7f0e94150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e941591f0 0x7f0e94169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9416a670 con 0x7f0e940ce810 2023-09-27T23:33:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9416d1c0 con 0x7f0e941591f0 2023-09-27T23:33:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9416db90 con 0x7f0e94158e20 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 0x7f0e9416d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 0x7f0e9416d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42190/0 (socket says 172.21.15.64:42190) 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 -- 172.21.15.64:0/4237451424 learned_addr learned my addr 172.21.15.64:0/4237451424 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99a33700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e941591f0 0x7f0e94169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e98a31700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e94158e20 0x7f0e94150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 -- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e94158e20 msgr2=0x7f0e94150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e94158e20 0x7f0e94150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 -- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e941591f0 msgr2=0x7f0e94169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e941591f0 0x7f0e94169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 -- 172.21.15.64:0/4237451424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84009420 con 0x7f0e940ce810 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e99232700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 0x7f0e9416d650 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c5c0 tx=0x7f0e9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90011590 con 0x7f0e940ce810 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e90011710 con 0x7f0e940ce810 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9416a0b0 con 0x7f0e940ce810 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90011ad0 con 0x7f0e940ce810 2023-09-27T23:33:06.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9416ce50 con 0x7f0e940ce810 2023-09-27T23:33:06.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9404be40 con 0x7f0e940ce810 2023-09-27T23:33:06.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 55) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f0e9001b3b0 con 0x7f0e940ce810 2023-09-27T23:33:06.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e897fa700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e78060b90 0x7f0e78063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e98a31700 1 -- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e78060b90 msgr2=0x7f0e78063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:06.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f0e9007c8a0 con 0x7f0e940ce810 2023-09-27T23:33:06.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.587+0000 7f0e98a31700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e78060b90 0x7f0e78063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:06.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.595+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0e9004c5f0 con 0x7f0e940ce810 2023-09-27T23:33:06.626 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.622+0000 7fab55331000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:33:06.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.663+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 56) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f0e9004c9c0 con 0x7f0e940ce810 2023-09-27T23:33:06.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.663+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e78060b90 msgr2=0x7f0e78063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:06.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.663+0000 7f0e897fa700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e78060b90 0x7f0e78063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.728 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab1fd70700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.728 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab1fd70700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.728 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab1fd70700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.728 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab1fd70700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.729 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab1fd70700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.729 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab22d76700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.729 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab22d76700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.729 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab22d76700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.729 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab22d76700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.729 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:06.726+0000 7fab22d76700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:06.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e9406b3c0 con 0x7f0e940ce810 2023-09-27T23:33:06.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e897fa700 1 -- 172.21.15.64:0/4237451424 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f0e90069410 con 0x7f0e940ce810 2023-09-27T23:33:06.814 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:06.814 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:32:05.833505+0000","modified":"2023-09-27T23:32:08.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8151},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8151":{"gid":8151,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/4228640199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4228640199},{"type":"v1","addr":"172.21.15.64:6835","nonce":4228640199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-09-27T23:33:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 msgr2=0x7f0e9416d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 0x7f0e9416d650 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c5c0 tx=0x7f0e9000c8d0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 shutdown_connections 2023-09-27T23:33:06.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0e78060b90 0x7f0e78063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e941591f0 0x7f0e94169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e94158e20 0x7f0e94150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 --2- 172.21.15.64:0/4237451424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940ce810 0x7f0e9416d650 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 >> 172.21.15.64:0/4237451424 conn(0x7f0e94067bb0 msgr2=0x7f0e94153740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 shutdown_connections 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.811+0000 7f0e9a234700 1 -- 172.21.15.64:0/4237451424 wait complete. 2023-09-27T23:33:06.817 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 46 2023-09-27T23:33:06.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 Processor -- start 2023-09-27T23:33:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- start start 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c40cc3b0 0x7fe0c40cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c4157060 0x7fe0c415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40cd5a0 con 0x7fe0c4156c90 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c4157470 con 0x7fe0c40cc3b0 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40cc990 con 0x7fe0c4157060 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c40cc3b0 0x7fe0c40cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c40cc3b0 0x7fe0c40cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42204/0 (socket says 172.21.15.64:42204) 2023-09-27T23:33:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/3688563846 learned_addr learned my addr 172.21.15.64:0/3688563846 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c2d9d700 1 --2- 172.21.15.64:0/3688563846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c4157060 0x7fe0c415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/3688563846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c4157060 msgr2=0x7fe0c415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 --2- 172.21.15.64:0/3688563846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c4157060 0x7fe0c415ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/3688563846 >> v1:172.21.15.123:6789/0 conn(0x7fe0c4156c90 legacy=0x7fe0c40c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/3688563846 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40ccc70 con 0x7fe0c40cc3b0 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c1d9b700 1 --2- 172.21.15.64:0/3688563846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c40cc3b0 0x7fe0c40cda30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0099b0 tx=0x7fe0ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ee08a6873cace server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c159a700 1 -- 172.21.15.64:0/3688563846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00b820 con 0x7fe0c40cc3b0 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c159a700 1 -- 172.21.15.64:0/3688563846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00b980 con 0x7fe0c40cc3b0 2023-09-27T23:33:06.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c159a700 1 -- 172.21.15.64:0/3688563846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00bd20 con 0x7fe0c40cc3b0 2023-09-27T23:33:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- 172.21.15.64:0/3688563846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c40cc3b0 msgr2=0x7fe0c40cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 --2- 172.21.15.64:0/3688563846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c40cc3b0 0x7fe0c40cda30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0099b0 tx=0x7fe0ac009250 comp rx=0 tx=0).stop 2023-09-27T23:33:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- 172.21.15.64:0/3688563846 shutdown_connections 2023-09-27T23:33:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 --2- 172.21.15.64:0/3688563846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c4157060 0x7fe0c415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 --2- 172.21.15.64:0/3688563846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c40cc3b0 0x7fe0c40cda30 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- 172.21.15.64:0/3688563846 >> 172.21.15.64:0/3688563846 conn(0x7fe0c406c7c0 msgr2=0x7fe0c406cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- 172.21.15.64:0/3688563846 shutdown_connections 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.931+0000 7fe0c89de700 1 -- 172.21.15.64:0/3688563846 wait complete. 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 Processor -- start 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- start start 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c40cc3b0 0x7fe0c4159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 0x7fe0c40c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c4157060 0x7fe0c4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c4157650 con 0x7fe0c4156c90 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c4157930 con 0x7fe0c4157060 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c4157c10 con 0x7fe0c40cc3b0 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 0x7fe0c40c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 0x7fe0c40c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42220/0 (socket says 172.21.15.64:42220) 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/2730224026 learned_addr learned my addr 172.21.15.64:0/2730224026 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c2d9d700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c4157060 0x7fe0c4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c259c700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c40cc3b0 0x7fe0c4159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:06.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c40cc3b0 msgr2=0x7fe0c4159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c40cc3b0 0x7fe0c4159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c4157060 msgr2=0x7fe0c4154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c4157060 0x7fe0c4154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 -- 172.21.15.64:0/2730224026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0094f0 con 0x7fe0c4156c90 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c1d9b700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 0x7fe0c40c7850 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009290 tx=0x7fe0ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac01b470 con 0x7fe0c4156c90 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c4157ef0 con 0x7fe0c4156c90 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac01b5f0 con 0x7fe0c4156c90 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac01b9b0 con 0x7fe0c4156c90 2023-09-27T23:33:06.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c40c7d90 con 0x7fe0c4156c90 2023-09-27T23:33:06.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.935+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c4071120 con 0x7fe0c4156c90 2023-09-27T23:33:06.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.939+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 56) v1 ==== 79970+0+0 (secure 0 0 0) 0x7fe0ac02f480 con 0x7fe0c4156c90 2023-09-27T23:33:06.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.939+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 7122+0+0 (secure 0 0 0) 0x7fe0ac07ef20 con 0x7fe0c4156c90 2023-09-27T23:33:06.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:06.943+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe0ac04f130 con 0x7fe0c4156c90 2023-09-27T23:33:07.031 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:07.027+0000 7f4fca4d9000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:33:07.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.103+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0c40ccf50 con 0x7fe0c4156c90 2023-09-27T23:33:07.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0ba7fc700 1 -- 172.21.15.64:0/2730224026 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+18196 (secure 0 0 0) 0x7fe0ac02a020 con 0x7fe0c4156c90 2023-09-27T23:33:07.109 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:07.109 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":131,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:06.646697+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:32:04.410437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-09-27T23:32:04.813612+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/1587055090":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/2325830561":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/950592668":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/2755290719":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/232223077":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/2810382411":"2023-09-28T23:33:06.646576+0000","172.21.15.123:6833/33228":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3012318414":"2023-09-28T23:32:22.582464+0000","172.21.15.123:6832/33228":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3924250679":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/4117106670":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/1295929548":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/274584241":"2023-09-28T23:33:06.646576+0000","172.21.15.123:0/4175644346":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/3675917224":"2023-09-28T23:32:22.582464+0000","172.21.15.123:0/4076639863":"2023-09-28T23:32:22.582464+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 msgr2=0x7fe0c40c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 0x7fe0c40c7850 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009290 tx=0x7fe0ac016000 comp rx=0 tx=0).stop 2023-09-27T23:33:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 shutdown_connections 2023-09-27T23:33:07.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c4157060 0x7fe0c4154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c4156c90 0x7fe0c40c7850 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 --2- 172.21.15.64:0/2730224026 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c40cc3b0 0x7fe0c4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 >> 172.21.15.64:0/2730224026 conn(0x7fe0c406c7c0 msgr2=0x7fe0c40c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 shutdown_connections 2023-09-27T23:33:07.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.107+0000 7fe0c89de700 1 -- 172.21.15.64:0/2730224026 wait complete. 2023-09-27T23:33:07.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:33:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 Processor -- start 2023-09-27T23:33:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 -- start start 2023-09-27T23:33:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b90155960 0x7f6b9015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b900c7f00 0x7f6b900aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9006d990 con 0x7f6b9006de20 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9015a570 con 0x7f6b90155960 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b948a4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9006cd80 con 0x7f6b900c7f00 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b8f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b90155960 0x7f6b9015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b8f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b90155960 0x7f6b9015a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42228/0 (socket says 172.21.15.64:42228) 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b8f7fe700 1 -- 172.21.15.64:0/2275985337 learned_addr learned my addr 172.21.15.64:0/2275985337 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.235+0000 7f6b8ffff700 1 --2- 172.21.15.64:0/2275985337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b900c7f00 0x7f6b900aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8f7fe700 1 -- 172.21.15.64:0/2275985337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b900c7f00 msgr2=0x7f6b900aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8f7fe700 1 --2- 172.21.15.64:0/2275985337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b900c7f00 0x7f6b900aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8f7fe700 1 -- 172.21.15.64:0/2275985337 >> v1:172.21.15.123:6789/0 conn(0x7f6b9006de20 legacy=0x7f6b900c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8f7fe700 1 -- 172.21.15.64:0/2275985337 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9006d060 con 0x7f6b90155960 2023-09-27T23:33:07.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8f7fe700 1 --2- 172.21.15.64:0/2275985337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b90155960 0x7f6b9015a030 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6b800099b0 tx=0x7f6b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac86ffd6f42dabe server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:07.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8e7fc700 1 -- 172.21.15.64:0/2275985337 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000b860 con 0x7f6b90155960 2023-09-27T23:33:07.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8e7fc700 1 -- 172.21.15.64:0/2275985337 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8000b9c0 con 0x7f6b90155960 2023-09-27T23:33:07.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8e7fc700 1 -- 172.21.15.64:0/2275985337 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000bd60 con 0x7f6b90155960 2023-09-27T23:33:07.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- 172.21.15.64:0/2275985337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b90155960 msgr2=0x7f6b9015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:07.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 --2- 172.21.15.64:0/2275985337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b90155960 0x7f6b9015a030 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6b800099b0 tx=0x7f6b80009250 comp rx=0 tx=0).stop 2023-09-27T23:33:07.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- 172.21.15.64:0/2275985337 shutdown_connections 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 --2- 172.21.15.64:0/2275985337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b900c7f00 0x7f6b900aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 --2- 172.21.15.64:0/2275985337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b90155960 0x7f6b9015a030 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- 172.21.15.64:0/2275985337 >> 172.21.15.64:0/2275985337 conn(0x7f6b90067bb0 msgr2=0x7f6b900cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- 172.21.15.64:0/2275985337 shutdown_connections 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- 172.21.15.64:0/2275985337 wait complete. 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 Processor -- start 2023-09-27T23:33:07.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- start start 2023-09-27T23:33:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b9006de20 0x7f6b90165740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 0x7f6b900c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:07.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b900c7ad0 0x7f6b901410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90154c00 con 0x7f6b900c7f00 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90165c80 con 0x7f6b900c7ad0 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b948a4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90156210 con 0x7f6b9006de20 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 0x7f6b900c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 0x7f6b900c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42234/0 (socket says 172.21.15.64:42234) 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8effd700 1 -- 172.21.15.64:0/4134665484 learned_addr learned my addr 172.21.15.64:0/4134665484 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8ffff700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b900c7ad0 0x7f6b901410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.239+0000 7f6b8f7fe700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b9006de20 0x7f6b90165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b8effd700 1 -- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b9006de20 msgr2=0x7f6b90165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:07.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b8effd700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b9006de20 0x7f6b90165740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b8effd700 1 -- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b900c7ad0 msgr2=0x7f6b901410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b8effd700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b900c7ad0 0x7f6b901410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b8effd700 1 -- 172.21.15.64:0/4134665484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800094f0 con 0x7f6b900c7f00 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b8effd700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 0x7f6b900c7590 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800c790 tx=0x7f6b78007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78011590 con 0x7f6b900c7f00 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b78011710 con 0x7f6b900c7f00 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b90165f30 con 0x7f6b900c7f00 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78011ad0 con 0x7f6b900c7f00 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90166570 con 0x7f6b900c7f00 2023-09-27T23:33:07.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9004be40 con 0x7f6b900c7f00 2023-09-27T23:33:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.243+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 56) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f6b7801b3b0 con 0x7f6b900c7f00 2023-09-27T23:33:07.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.247+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 7122+0+0 (secure 0 0 0) 0x7f6b7807c0e0 con 0x7f6b900c7f00 2023-09-27T23:33:07.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.251+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6b7804c2f0 con 0x7f6b900c7f00 2023-09-27T23:33:07.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.439+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b9006d370 con 0x7f6b900c7f00 2023-09-27T23:33:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.663+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 57) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f6b7804bda0 con 0x7f6b900c7f00 2023-09-27T23:33:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.663+0000 7f6b877fe700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b70066ea0 0x7f6b70069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.667+0000 7f6b8f7fe700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b70066ea0 0x7f6b70069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:07.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:07.671+0000 7f6b8f7fe700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b70066ea0 0x7f6b70069280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b800099b0 tx=0x7f6b80017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:08.301 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:33:08.299+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:33:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.315+0000 7f6b877fe700 1 -- 172.21.15.64:0/4134665484 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b780690d0 con 0x7f6b900c7f00 2023-09-27T23:33:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b70066ea0 msgr2=0x7f6b70069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b70066ea0 0x7f6b70069280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b800099b0 tx=0x7f6b80017000 comp rx=0 tx=0).stop 2023-09-27T23:33:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 msgr2=0x7f6b900c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 0x7f6b900c7590 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800c790 tx=0x7f6b78007f40 comp rx=0 tx=0).stop 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 shutdown_connections 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6b70066ea0 0x7f6b70069280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b900c7ad0 0x7f6b901410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b900c7f00 0x7f6b900c7590 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 --2- 172.21.15.64:0/4134665484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b9006de20 0x7f6b90165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 >> 172.21.15.64:0/4134665484 conn(0x7f6b90067bb0 msgr2=0x7f6b9014e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 shutdown_connections 2023-09-27T23:33:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.319+0000 7f6b948a4700 1 -- 172.21.15.64:0/4134665484 wait complete. 2023-09-27T23:33:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:33:08.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:33:08.360 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:33:08.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 Processor -- start 2023-09-27T23:33:08.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 -- start start 2023-09-27T23:33:08.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 0x7fa0540a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:08.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa05415c8f0 0x7fa05406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:08.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540c6ab0 con 0x7fa05406ace0 2023-09-27T23:33:08.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05406b520 con 0x7fa05415c8f0 2023-09-27T23:33:08.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa059078700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540cb910 con 0x7fa05415c520 2023-09-27T23:33:08.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 0x7fa0540a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa05415c8f0 0x7fa05406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 0x7fa0540a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:60090/0 (socket says 172.21.15.64:60090) 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05359e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa05406ace0 0x7fa0540ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:39314/0 (socket says 172.21.15.64:39314) 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05359e700 1 -- 172.21.15.64:0/261506247 learned_addr learned my addr 172.21.15.64:0/261506247 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa05415c8f0 0x7fa05406a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:38112/0 (socket says 172.21.15.64:38112) 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05259c700 1 -- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 msgr2=0x7fa0540a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:08.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05259c700 1 --2- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 0x7fa0540a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05259c700 1 -- 172.21.15.64:0/261506247 >> v1:172.21.15.123:6789/0 conn(0x7fa05406ace0 legacy=0x7fa0540ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa05259c700 1 -- 172.21.15.64:0/261506247 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05406b240 con 0x7fa05415c8f0 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.439+0000 7fa052d9d700 1 --2- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 0x7fa0540a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa05259c700 1 --2- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa05406a7a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa0480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126f8660e600cf9a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa051d9b700 1 -- 172.21.15.64:0/261506247 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800b6d0 con 0x7fa05415c8f0 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa051d9b700 1 -- 172.21.15.64:0/261506247 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04800b830 con 0x7fa05415c8f0 2023-09-27T23:33:08.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa051d9b700 1 -- 172.21.15.64:0/261506247 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800bbd0 con 0x7fa05415c8f0 2023-09-27T23:33:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 -- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 msgr2=0x7fa05406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:08.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 --2- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa05406a7a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa0480091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 -- 172.21.15.64:0/261506247 shutdown_connections 2023-09-27T23:33:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 --2- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa05406a7a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 --2- 172.21.15.64:0/261506247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05415c520 0x7fa0540a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 -- 172.21.15.64:0/261506247 >> 172.21.15.64:0/261506247 conn(0x7fa054067bb0 msgr2=0x7fa0540cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 -- 172.21.15.64:0/261506247 shutdown_connections 2023-09-27T23:33:08.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 -- 172.21.15.64:0/261506247 wait complete. 2023-09-27T23:33:08.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 Processor -- start 2023-09-27T23:33:08.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.443+0000 7fa059078700 1 -- start start 2023-09-27T23:33:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05406ace0 0x7fa0540c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 0x7fa054148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa0540c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c1d10 con 0x7fa05415c520 2023-09-27T23:33:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c1100 con 0x7fa05415c8f0 2023-09-27T23:33:08.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c13e0 con 0x7fa05406ace0 2023-09-27T23:33:08.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 0x7fa054148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:08.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 0x7fa054148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42258/0 (socket says 172.21.15.64:42258) 2023-09-27T23:33:08.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 -- 172.21.15.64:0/1045431574 learned_addr learned my addr 172.21.15.64:0/1045431574 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:08.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa052d9d700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05406ace0 0x7fa0540c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:08.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05359e700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa0540c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 -- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05406ace0 msgr2=0x7fa0540c0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05406ace0 0x7fa0540c0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 -- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 msgr2=0x7fa0540c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa0540c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 -- 172.21.15.64:0/1045431574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044009420 con 0x7fa05415c520 2023-09-27T23:33:08.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa05259c700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 0x7fa054148670 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa048013000 tx=0x7fa048017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04801bcc0 con 0x7fa05415c520 2023-09-27T23:33:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04801be60 con 0x7fa05415c520 2023-09-27T23:33:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa048009420 con 0x7fa05415c520 2023-09-27T23:33:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048011670 con 0x7fa05415c520 2023-09-27T23:33:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.447+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0540c19a0 con 0x7fa05415c520 2023-09-27T23:33:08.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.451+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa054071120 con 0x7fa05415c520 2023-09-27T23:33:08.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.451+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 57) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fa0480119e0 con 0x7fa05415c520 2023-09-27T23:33:08.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.451+0000 7fa03affd700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0400607a0 0x7fa040062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:08.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.451+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 7180+0+0 (secure 0 0 0) 0x7fa04807e760 con 0x7fa05415c520 2023-09-27T23:33:08.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.451+0000 7fa052d9d700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0400607a0 0x7fa040062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:08.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.455+0000 7fa052d9d700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0400607a0 0x7fa040062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa044009f40 tx=0x7fa0440091a0 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:08.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.463+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa04804e830 con 0x7fa05415c520 2023-09-27T23:33:08.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:08.663+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa0540c9cc0 con 0x7fa05415c520 2023-09-27T23:33:09.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa03affd700 1 -- 172.21.15.64:0/1045431574 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa048051e50 con 0x7fa05415c520 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0400607a0 msgr2=0x7fa040062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0400607a0 0x7fa040062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa044009f40 tx=0x7fa0440091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 msgr2=0x7fa054148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 0x7fa054148670 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa048013000 tx=0x7fa048017000 comp rx=0 tx=0).stop 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 shutdown_connections 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa0400607a0 0x7fa040062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa05415c8f0 0x7fa0540c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa05415c520 0x7fa054148670 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 --2- 172.21.15.64:0/1045431574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa05406ace0 0x7fa0540c0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 >> 172.21.15.64:0/1045431574 conn(0x7fa054067bb0 msgr2=0x7fa0540c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:09.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 shutdown_connections 2023-09-27T23:33:09.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.319+0000 7fa059078700 1 -- 172.21.15.64:0/1045431574 wait complete. 2023-09-27T23:33:09.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:33:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 Processor -- start 2023-09-27T23:33:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 -- start start 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07940af190 0x7f079406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f07940cac50 0x7f07940af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940cb3f0 con 0x7f07940aedc0 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07940cc140 con 0x7f07940af190 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079b1fd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07940af610 con 0x7f07940cac50 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f07999fa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07940af190 0x7f079406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079a1fb700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f07940aedc0 0x7f079406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39186/0 (socket says 172.21.15.64:39186) 2023-09-27T23:33:09.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079a1fb700 1 -- 172.21.15.64:0/3494424333 learned_addr learned my addr 172.21.15.64:0/3494424333 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:09.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.455+0000 7f079a9fc700 1 --2- 172.21.15.64:0/3494424333 >> v2:172.21.15.123:3300/0 conn(0x7f07940cac50 0x7f07940af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07999fa700 1 -- 172.21.15.64:0/3494424333 >> v2:172.21.15.123:3300/0 conn(0x7f07940cac50 msgr2=0x7f07940af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07999fa700 1 --2- 172.21.15.64:0/3494424333 >> v2:172.21.15.123:3300/0 conn(0x7f07940cac50 0x7f07940af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07999fa700 1 -- 172.21.15.64:0/3494424333 >> v1:172.21.15.64:6789/0 conn(0x7f07940aedc0 legacy=0x7f079406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07999fa700 1 -- 172.21.15.64:0/3494424333 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079406d220 con 0x7f07940af190 2023-09-27T23:33:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07999fa700 1 --2- 172.21.15.64:0/3494424333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940af190 0x7f079406af50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f079406a0e0 tx=0x7f07840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c144b6700a07e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07991f9700 1 -- 172.21.15.64:0/3494424333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784011630 con 0x7f07940af190 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07991f9700 1 -- 172.21.15.64:0/3494424333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784011790 con 0x7f07940af190 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f07991f9700 1 -- 172.21.15.64:0/3494424333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784011b30 con 0x7f07940af190 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 -- 172.21.15.64:0/3494424333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940af190 msgr2=0x7f079406af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 --2- 172.21.15.64:0/3494424333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940af190 0x7f079406af50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f079406a0e0 tx=0x7f07840091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 -- 172.21.15.64:0/3494424333 shutdown_connections 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 --2- 172.21.15.64:0/3494424333 >> v2:172.21.15.123:3300/0 conn(0x7f07940cac50 0x7f07940af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 --2- 172.21.15.64:0/3494424333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940af190 0x7f079406af50 secure :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f079406a0e0 tx=0x7f07840091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 -- 172.21.15.64:0/3494424333 >> 172.21.15.64:0/3494424333 conn(0x7f07940cc760 msgr2=0x7f07940ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 -- 172.21.15.64:0/3494424333 shutdown_connections 2023-09-27T23:33:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.459+0000 7f079b1fd700 1 -- 172.21.15.64:0/3494424333 wait complete. 2023-09-27T23:33:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 Processor -- start 2023-09-27T23:33:09.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 -- start start 2023-09-27T23:33:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 0x7f079415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07940cac50 0x7f07940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07940b2aa0 0x7f07940b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c50d0 con 0x7f07940aedc0 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079415a900 con 0x7f07940cac50 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079b1fd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940ceb20 con 0x7f07940b2aa0 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 0x7f079415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 0x7f079415a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42284/0 (socket says 172.21.15.64:42284) 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 -- 172.21.15.64:0/2897009101 learned_addr learned my addr 172.21.15.64:0/2897009101 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:09.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f07999fa700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07940cac50 0x7f07940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a9fc700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07940b2aa0 0x7f07940b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 -- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07940b2aa0 msgr2=0x7f07940b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07940b2aa0 0x7f07940b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 -- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07940cac50 msgr2=0x7f07940b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07940cac50 0x7f07940b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a1fb700 1 -- 172.21.15.64:0/2897009101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009420 con 0x7f07940aedc0 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f079a9fc700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07940b2aa0 0x7f07940b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.463+0000 7f07999fa700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07940cac50 0x7f07940b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f079a1fb700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 0x7f079415a3c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f079000c7f0 tx=0x7f0790007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790005b30 con 0x7f07940aedc0 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790005cb0 con 0x7f07940aedc0 2023-09-27T23:33:09.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790004a80 con 0x7f07940aedc0 2023-09-27T23:33:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07940cee00 con 0x7f07940aedc0 2023-09-27T23:33:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07940c4cf0 con 0x7f07940aedc0 2023-09-27T23:33:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079404be40 con 0x7f07940aedc0 2023-09-27T23:33:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.467+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 57) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f07900070f0 con 0x7f07940aedc0 2023-09-27T23:33:09.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.471+0000 7f078a7fc700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07780607a0 0x7f0778062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:09.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.471+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 7180+0+0 (secure 0 0 0) 0x7f079007c3d0 con 0x7f07940aedc0 2023-09-27T23:33:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.471+0000 7f07999fa700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07780607a0 0x7f0778062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:09.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.471+0000 7f07999fa700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07780607a0 0x7f0778062c50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f07940af560 tx=0x7f0784017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:09.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.475+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f079004c520 con 0x7f07940aedc0 2023-09-27T23:33:09.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.639+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f079406c8f0 con 0x7f07940aedc0 2023-09-27T23:33:09.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:09.675+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f079007c0c0 con 0x7f07940aedc0 2023-09-27T23:33:10.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.327+0000 7f078a7fc700 1 -- 172.21.15.64:0/2897009101 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7f079004ffb0 con 0x7f07940aedc0 2023-09-27T23:33:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.331+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07780607a0 msgr2=0x7f0778062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.331+0000 7f079b1fd700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07780607a0 0x7f0778062c50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f07940af560 tx=0x7f0784017000 comp rx=0 tx=0).stop 2023-09-27T23:33:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.331+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 msgr2=0x7f079415a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.331+0000 7f079b1fd700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 0x7f079415a3c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f079000c7f0 tx=0x7f0790007f70 comp rx=0 tx=0).stop 2023-09-27T23:33:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 shutdown_connections 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f07780607a0 0x7f0778062c50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07940b2aa0 0x7f07940b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07940cac50 0x7f07940b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 --2- 172.21.15.64:0/2897009101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07940aedc0 0x7f079415a3c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 >> 172.21.15.64:0/2897009101 conn(0x7f07940cc760 msgr2=0x7f079415e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 shutdown_connections 2023-09-27T23:33:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.335+0000 7f079b1fd700 1 -- 172.21.15.64:0/2897009101 wait complete. 2023-09-27T23:33:10.340 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:33:10.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:33:10.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 Processor -- start 2023-09-27T23:33:10.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 -- start start 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9d01569b0 0x7fb9d00abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d00c7950 con 0x7fb9d01558d0 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d0156fa0 con 0x7fb9d01569b0 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d69dd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00abc60 con 0x7fb9d00c6ff0 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d51da700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9d01569b0 0x7fb9d00abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d61dc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d59db700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb9d01558d0 0x7fb9d00ad740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:39316/0 (socket says 172.21.15.64:39316) 2023-09-27T23:33:10.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d59db700 1 -- 172.21.15.64:0/3208803890 learned_addr learned my addr 172.21.15.64:0/3208803890 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d51da700 1 -- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 msgr2=0x7fb9d00c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d51da700 1 --2- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d51da700 1 -- 172.21.15.64:0/3208803890 >> v1:172.21.15.123:6789/0 conn(0x7fb9d01558d0 legacy=0x7fb9d00ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d51da700 1 -- 172.21.15.64:0/3208803890 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00cbfb0 con 0x7fb9d01569b0 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d49d9700 1 -- 172.21.15.64:0/3208803890 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612834171 0 0) 0x7fb9d00c7950 con 0x7fb9d01558d0 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d61dc700 1 --2- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d51da700 1 --2- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00abe10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000bb90 tx=0x7fb9c000b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92c901fd4a4d462 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d49d9700 1 -- 172.21.15.64:0/3208803890 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000db40 con 0x7fb9d01569b0 2023-09-27T23:33:10.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d49d9700 1 -- 172.21.15.64:0/3208803890 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c000dca0 con 0x7fb9d01569b0 2023-09-27T23:33:10.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.463+0000 7fb9d49d9700 1 -- 172.21.15.64:0/3208803890 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00196b0 con 0x7fb9d01569b0 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 -- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 msgr2=0x7fb9d00abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00abe10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000bb90 tx=0x7fb9c000b430 comp rx=0 tx=0).stop 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 -- 172.21.15.64:0/3208803890 shutdown_connections 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/3208803890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00abe10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 -- 172.21.15.64:0/3208803890 >> 172.21.15.64:0/3208803890 conn(0x7fb9d006b5b0 msgr2=0x7fb9d006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 -- 172.21.15.64:0/3208803890 shutdown_connections 2023-09-27T23:33:10.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 -- 172.21.15.64:0/3208803890 wait complete. 2023-09-27T23:33:10.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 Processor -- start 2023-09-27T23:33:10.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.467+0000 7fb9d69dd700 1 -- start start 2023-09-27T23:33:10.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00a8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 0x7fb9d015f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d0159d00 con 0x7fb9d01558d0 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00a8e40 con 0x7fb9d01569b0 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00a9120 con 0x7fb9d00c6ff0 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 0x7fb9d015f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 0x7fb9d015f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42292/0 (socket says 172.21.15.64:42292) 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 -- 172.21.15.64:0/2355493416 learned_addr learned my addr 172.21.15.64:0/2355493416 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:10.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d59db700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d61dc700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 -- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 msgr2=0x7fb9d00a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 -- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 msgr2=0x7fb9d00c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 -- 172.21.15.64:0/2355493416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8009a20 con 0x7fb9d01558d0 2023-09-27T23:33:10.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d59db700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:10.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d51da700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 0x7fb9d015f0f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000bfd0 tx=0x7fb9c001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:10.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0010470 con 0x7fb9d01558d0 2023-09-27T23:33:10.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c00105f0 con 0x7fb9d01558d0 2023-09-27T23:33:10.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c000b6d0 con 0x7fb9d01558d0 2023-09-27T23:33:10.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00109b0 con 0x7fb9d01558d0 2023-09-27T23:33:10.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.471+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d00a9760 con 0x7fb9d01558d0 2023-09-27T23:33:10.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.475+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d00aa7d0 con 0x7fb9d01558d0 2023-09-27T23:33:10.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.475+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb9c0024420 con 0x7fb9d01558d0 2023-09-27T23:33:10.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.475+0000 7fb9c5ffb700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb9b4060a70 0x7fb9b4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:10.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.475+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6742+0+0 (secure 0 0 0) 0x7fb9c0020070 con 0x7fb9d01558d0 2023-09-27T23:33:10.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.475+0000 7fb9d59db700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb9b4060a70 0x7fb9b4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:10.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.475+0000 7fb9d59db700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb9b4060a70 0x7fb9b4062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0011e0 tx=0x7fb9cc009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:10.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.483+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb9c004f3f0 con 0x7fb9d01558d0 2023-09-27T23:33:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:10.651+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb9d004be40 con 0x7fb9d01558d0 2023-09-27T23:33:11.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.243+0000 7fb9c5ffb700 1 -- 172.21.15.64:0/2355493416 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb9c0052a10 con 0x7fb9d01558d0 2023-09-27T23:33:11.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.251+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb9b4060a70 msgr2=0x7fb9b4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.251+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb9b4060a70 0x7fb9b4062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0011e0 tx=0x7fb9cc009040 comp rx=0 tx=0).stop 2023-09-27T23:33:11.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.251+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 msgr2=0x7fb9d015f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.251+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 0x7fb9d015f0f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000bfd0 tx=0x7fb9c001b000 comp rx=0 tx=0).stop 2023-09-27T23:33:11.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 shutdown_connections 2023-09-27T23:33:11.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb9b4060a70 0x7fb9b4062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d01569b0 0x7fb9d00c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9d01558d0 0x7fb9d015f0f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 --2- 172.21.15.64:0/2355493416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9d00c6ff0 0x7fb9d00a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 >> 172.21.15.64:0/2355493416 conn(0x7fb9d006b5b0 msgr2=0x7fb9d0068460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:11.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 shutdown_connections 2023-09-27T23:33:11.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.255+0000 7fb9d69dd700 1 -- 172.21.15.64:0/2355493416 wait complete. 2023-09-27T23:33:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:33:11.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:33:11.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 Processor -- start 2023-09-27T23:33:11.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 -- start start 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c13f2d0 0x7f2a3c0ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a3c13f6a0 0x7f2a3c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2a3c0c7780 0x7f2a3c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0c8140 con 0x7f2a3c13f6a0 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0adc60 con 0x7f2a3c0c7780 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a42bd1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15a5c0 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a3c13f6a0 0x7f2a3c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a3c13f6a0 0x7f2a3c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42306/0 (socket says 172.21.15.64:42306) 2023-09-27T23:33:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 -- 172.21.15.64:0/1224204565 learned_addr learned my addr 172.21.15.64:0/1224204565 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a423d0700 1 --2- 172.21.15.64:0/1224204565 >> v2:172.21.15.123:3300/0 conn(0x7f2a3c0c7780 0x7f2a3c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a41bcf700 1 --2- 172.21.15.64:0/1224204565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c13f2d0 0x7f2a3c0ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 -- 172.21.15.64:0/1224204565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c13f2d0 msgr2=0x7f2a3c0ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:11.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 --2- 172.21.15.64:0/1224204565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c13f2d0 0x7f2a3c0ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 -- 172.21.15.64:0/1224204565 >> v2:172.21.15.123:3300/0 conn(0x7f2a3c0c7780 msgr2=0x7f2a3c0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:11.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 --2- 172.21.15.64:0/1224204565 >> v2:172.21.15.123:3300/0 conn(0x7f2a3c0c7780 0x7f2a3c0c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 -- 172.21.15.64:0/1224204565 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0ad050 con 0x7f2a3c13f6a0 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a423d0700 1 --2- 172.21.15.64:0/1224204565 >> v2:172.21.15.123:3300/0 conn(0x7f2a3c0c7780 0x7f2a3c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a413ce700 1 --2- 172.21.15.64:0/1224204565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c15a080 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009f20 tx=0x7f2a38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba04d5fe3f0d884 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a40bcd700 1 -- 172.21.15.64:0/1224204565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800e070 con 0x7f2a3c13f6a0 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a40bcd700 1 -- 172.21.15.64:0/1224204565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3800ba40 con 0x7f2a3c13f6a0 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.383+0000 7f2a40bcd700 1 -- 172.21.15.64:0/1224204565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380092e0 con 0x7f2a3c13f6a0 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 -- 172.21.15.64:0/1224204565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f6a0 msgr2=0x7f2a3c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/1224204565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c15a080 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009f20 tx=0x7f2a38009840 comp rx=0 tx=0).stop 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 -- 172.21.15.64:0/1224204565 shutdown_connections 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/1224204565 >> v2:172.21.15.123:3300/0 conn(0x7f2a3c0c7780 0x7f2a3c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/1224204565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c15a080 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/1224204565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c13f2d0 0x7f2a3c0ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 -- 172.21.15.64:0/1224204565 >> 172.21.15.64:0/1224204565 conn(0x7f2a3c067b50 msgr2=0x7f2a3c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:11.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 -- 172.21.15.64:0/1224204565 shutdown_connections 2023-09-27T23:33:11.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 -- 172.21.15.64:0/1224204565 wait complete. 2023-09-27T23:33:11.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 Processor -- start 2023-09-27T23:33:11.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 -- start start 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 0x7f2a3c13a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 0x7f2a3c1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.387+0000 7f2a42bd1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a42bd1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c157e30 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a42bd1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1560c0 con 0x7f2a3c13f6a0 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a42bd1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c156310 con 0x7f2a3c0c7780 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 0x7f2a3c1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a41bcf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 0x7f2a3c13a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 0x7f2a3c1573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42320/0 (socket says 172.21.15.64:42320) 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a41bcf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 0x7f2a3c13a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:60138/0 (socket says 172.21.15.64:60138) 2023-09-27T23:33:11.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a423d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 -- 172.21.15.64:0/934962120 learned_addr learned my addr 172.21.15.64:0/934962120 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 -- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 msgr2=0x7f2a3c13a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 0x7f2a3c13a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 -- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a3c13f6a0 msgr2=0x7f2a3c1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c1578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 -- 172.21.15.64:0/934962120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a38009ae0 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a41bcf700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 0x7f2a3c13a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a413ce700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 0x7f2a3c1573b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b6c0 tx=0x7f2a38018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38007820 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c1565f0 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c156e90 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a38007b90 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38007d00 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.391+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c0052b0 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.395+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2a38003040 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.395+0000 7f2a31ffb700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a20060a70 0x7f2a20062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.395+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2a3807eb80 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.395+0000 7f2a41bcf700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a20060a70 0x7f2a20062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.395+0000 7f2a41bcf700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a20060a70 0x7f2a20062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c006fd0 tx=0x7f2a2c008040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:11.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.403+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2a3804ead0 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.567+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f2a1c005600 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.579+0000 7f2a31ffb700 1 -- 172.21.15.64:0/934962120 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v43) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2a38007e60 con 0x7f2a3c13f2d0 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.579+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a20060a70 msgr2=0x7f2a20062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.579+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a20060a70 0x7f2a20062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c006fd0 tx=0x7f2a2c008040 comp rx=0 tx=0).stop 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 msgr2=0x7f2a3c1573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 0x7f2a3c1573b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b6c0 tx=0x7f2a38018000 comp rx=0 tx=0).stop 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 shutdown_connections 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a20060a70 0x7f2a20062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a3c13f6a0 0x7f2a3c1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a3c13f2d0 0x7f2a3c1573b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 --2- 172.21.15.64:0/934962120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a3c0c7780 0x7f2a3c13a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 >> 172.21.15.64:0/934962120 conn(0x7f2a3c067b50 msgr2=0x7f2a3c139040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 shutdown_connections 2023-09-27T23:33:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.583+0000 7f2a42bd1700 1 -- 172.21.15.64:0/934962120 wait complete. 2023-09-27T23:33:11.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:33:11.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 Processor -- start 2023-09-27T23:33:11.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 -- start start 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f13c8158c90 0x7f13c80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13c806aeb0 0x7f13c8159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c806c850 con 0x7f13c81595a0 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c806b470 con 0x7f13c806aeb0 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cdc3d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c806b750 con 0x7f13c8158c90 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13c7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13c806aeb0 0x7f13c8159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cd43c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f13c81595a0 0x7f13c815c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39198/0 (socket says 172.21.15.64:39198) 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13cd43c700 1 -- 172.21.15.64:0/3658998840 learned_addr learned my addr 172.21.15.64:0/3658998840 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:11.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.707+0000 7f13ccc3b700 1 --2- 172.21.15.64:0/3658998840 >> v2:172.21.15.123:3300/0 conn(0x7f13c8158c90 0x7f13c80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c7fff700 1 -- 172.21.15.64:0/3658998840 >> v2:172.21.15.123:3300/0 conn(0x7f13c8158c90 msgr2=0x7f13c80caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:11.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c7fff700 1 --2- 172.21.15.64:0/3658998840 >> v2:172.21.15.123:3300/0 conn(0x7f13c8158c90 0x7f13c80caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c7fff700 1 -- 172.21.15.64:0/3658998840 >> v1:172.21.15.64:6789/0 conn(0x7f13c81595a0 legacy=0x7f13c815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c77fe700 1 -- 172.21.15.64:0/3658998840 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137725064 0 0) 0x7f13c806c850 con 0x7f13c81595a0 2023-09-27T23:33:11.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c7fff700 1 -- 172.21.15.64:0/3658998840 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c806ba30 con 0x7f13c806aeb0 2023-09-27T23:33:11.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c7fff700 1 --2- 172.21.15.64:0/3658998840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c806aeb0 0x7f13c8159060 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f13c80698f0 tx=0x7f13b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c60774430eaf679 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:11.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c77fe700 1 -- 172.21.15.64:0/3658998840 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000b6b0 con 0x7f13c806aeb0 2023-09-27T23:33:11.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c77fe700 1 -- 172.21.15.64:0/3658998840 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000b810 con 0x7f13c806aeb0 2023-09-27T23:33:11.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13c77fe700 1 -- 172.21.15.64:0/3658998840 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000bbb0 con 0x7f13c806aeb0 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3658998840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c806aeb0 msgr2=0x7f13c8159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3658998840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c806aeb0 0x7f13c8159060 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f13c80698f0 tx=0x7f13b00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3658998840 shutdown_connections 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3658998840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c806aeb0 0x7f13c8159060 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3658998840 >> v2:172.21.15.123:3300/0 conn(0x7f13c8158c90 0x7f13c80caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3658998840 >> 172.21.15.64:0/3658998840 conn(0x7f13c8067bb0 msgr2=0x7f13c80cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3658998840 shutdown_connections 2023-09-27T23:33:11.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.711+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3658998840 wait complete. 2023-09-27T23:33:11.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 Processor -- start 2023-09-27T23:33:11.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 -- start start 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13c806aeb0 0x7f13c80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 0x7f13c8157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13c81595a0 0x7f13c8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8169280 con 0x7f13c8158c90 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80c9b80 con 0x7f13c81595a0 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8169d90 con 0x7f13c806aeb0 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 0x7f13c8157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 0x7f13c8157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42330/0 (socket says 172.21.15.64:42330) 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 -- 172.21.15.64:0/3328734584 learned_addr learned my addr 172.21.15.64:0/3328734584 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:11.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cd43c700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13c81595a0 0x7f13c8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13ccc3b700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13c806aeb0 0x7f13c80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 -- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13c806aeb0 msgr2=0x7f13c80aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13c806aeb0 0x7f13c80aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 -- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13c81595a0 msgr2=0x7f13c8158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13c81595a0 0x7f13c8158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 -- 172.21.15.64:0/3328734584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0009420 con 0x7f13c8158c90 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c7fff700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 0x7f13c8157ae0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f13b000b0e0 tx=0x7f13b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0000d10 con 0x7f13c8158c90 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000b980 con 0x7f13c8158c90 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c816a220 con 0x7f13c8158c90 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0025420 con 0x7f13c8158c90 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.715+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c8169740 con 0x7f13c8158c90 2023-09-27T23:33:11.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.719+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c804be40 con 0x7f13c8158c90 2023-09-27T23:33:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.719+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f13b0023070 con 0x7f13c8158c90 2023-09-27T23:33:11.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.719+0000 7f13c4ff9700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f13a8060a70 0x7f13a8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:11.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.719+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f13b007e750 con 0x7f13c8158c90 2023-09-27T23:33:11.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.719+0000 7f13ccc3b700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f13a8060a70 0x7f13a8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:11.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.719+0000 7f13ccc3b700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f13a8060a70 0x7f13a8062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009fd0 tx=0x7f13c0009880 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:11.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.727+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f13b004e710 con 0x7f13c8158c90 2023-09-27T23:33:11.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.887+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f13c806bd10 con 0x7f13c8158c90 2023-09-27T23:33:11.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.891+0000 7f13c4ff9700 1 -- 172.21.15.64:0/3328734584 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7f13b0051d30 con 0x7f13c8158c90 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f13a8060a70 msgr2=0x7f13a8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f13a8060a70 0x7f13a8062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009fd0 tx=0x7f13c0009880 comp rx=0 tx=0).stop 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 msgr2=0x7f13c8157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 0x7f13c8157ae0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f13b000b0e0 tx=0x7f13b0017000 comp rx=0 tx=0).stop 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 shutdown_connections 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f13a8060a70 0x7f13a8062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13c81595a0 0x7f13c8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13c8158c90 0x7f13c8157ae0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 --2- 172.21.15.64:0/3328734584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13c806aeb0 0x7f13c80aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 >> 172.21.15.64:0/3328734584 conn(0x7f13c8067bb0 msgr2=0x7f13c80b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 shutdown_connections 2023-09-27T23:33:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:11.895+0000 7f13cdc3d700 1 -- 172.21.15.64:0/3328734584 wait complete. 2023-09-27T23:33:11.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:33:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.019+0000 7fe565cd6700 1 Processor -- start 2023-09-27T23:33:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.019+0000 7fe565cd6700 1 -- start start 2023-09-27T23:33:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe565cd6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560158c90 0x7fe5600caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe565cd6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56006c850 con 0x7fe560159770 2023-09-27T23:33:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe565cd6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56006b470 con 0x7fe56006aeb0 2023-09-27T23:33:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe565cd6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56006b750 con 0x7fe560158c90 2023-09-27T23:33:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe5654d5700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe560159770 0x7fe56006cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39212/0 (socket says 172.21.15.64:39212) 2023-09-27T23:33:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe5654d5700 1 -- 172.21.15.64:0/1558075021 learned_addr learned my addr 172.21.15.64:0/1558075021 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe564cd4700 1 --2- 172.21.15.64:0/1558075021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560158c90 0x7fe5600caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970755029 0 0) 0x7fe56006c850 con 0x7fe560159770 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe54c003570 con 0x7fe560159770 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855865190 0 0) 0x7fe56006b470 con 0x7fe56006aeb0 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe56006c850 con 0x7fe56006aeb0 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294875024 0 0) 0x7fe54c003570 con 0x7fe560159770 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe564cd4700 1 -- 172.21.15.64:0/1558075021 >> v1:172.21.15.123:6789/0 conn(0x7fe56006aeb0 legacy=0x7fe560159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe564cd4700 1 -- 172.21.15.64:0/1558075021 >> v1:172.21.15.64:6789/0 conn(0x7fe560159770 legacy=0x7fe56006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe564cd4700 1 -- 172.21.15.64:0/1558075021 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56006ba30 con 0x7fe560158c90 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe564cd4700 1 --2- 172.21.15.64:0/1558075021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560158c90 0x7fe5600caf40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe550009910 tx=0x7fe5500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd5d1d9ce314b5d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580017f0 con 0x7fe560158c90 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe550011420 con 0x7fe560158c90 2023-09-27T23:33:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.023+0000 7fe55f7fe700 1 -- 172.21.15.64:0/1558075021 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500117c0 con 0x7fe560158c90 2023-09-27T23:33:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 -- 172.21.15.64:0/1558075021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560158c90 msgr2=0x7fe5600caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1558075021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560158c90 0x7fe5600caf40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe550009910 tx=0x7fe5500091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 -- 172.21.15.64:0/1558075021 shutdown_connections 2023-09-27T23:33:12.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1558075021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560158c90 0x7fe5600caf40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 -- 172.21.15.64:0/1558075021 >> 172.21.15.64:0/1558075021 conn(0x7fe560067bb0 msgr2=0x7fe5600cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 -- 172.21.15.64:0/1558075021 shutdown_connections 2023-09-27T23:33:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 -- 172.21.15.64:0/1558075021 wait complete. 2023-09-27T23:33:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 Processor -- start 2023-09-27T23:33:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.027+0000 7fe565cd6700 1 -- start start 2023-09-27T23:33:12.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 0x7fe5601579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe560158c90 0x7fe560155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560159770 0x7fe5600c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe560156790 con 0x7fe56006aeb0 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601559a0 con 0x7fe560158c90 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe560155c10 con 0x7fe560159770 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 0x7fe5601579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 0x7fe5601579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42340/0 (socket says 172.21.15.64:42340) 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 -- 172.21.15.64:0/1959918851 learned_addr learned my addr 172.21.15.64:0/1959918851 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe5654d5700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560159770 0x7fe5600c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe55ffff700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe560158c90 0x7fe560155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 -- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560159770 msgr2=0x7fe5600c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560159770 0x7fe5600c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 -- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe560158c90 msgr2=0x7fe560155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe560158c90 0x7fe560155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 -- 172.21.15.64:0/1959918851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe550009420 con 0x7fe56006aeb0 2023-09-27T23:33:12.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe5654d5700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560159770 0x7fe5600c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:33:12.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe55ffff700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe560158c90 0x7fe560155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:33:12.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe564cd4700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 0x7fe5601579a0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fe550009860 tx=0x7fe550016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55001cc90 con 0x7fe56006aeb0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55001cdf0 con 0x7fe56006aeb0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe560155ef0 con 0x7fe56006aeb0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55001a6a0 con 0x7fe56006aeb0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5601564b0 con 0x7fe56006aeb0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.031+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56004be40 con 0x7fe56006aeb0 2023-09-27T23:33:12.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.035+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe55001aa90 con 0x7fe56006aeb0 2023-09-27T23:33:12.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.035+0000 7fe55cff9700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe540060a70 0x7fe540062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.035+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe55007ef20 con 0x7fe56006aeb0 2023-09-27T23:33:12.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.035+0000 7fe55ffff700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe540060a70 0x7fe540062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.035+0000 7fe55ffff700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe540060a70 0x7fe540062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe548007fd0 tx=0x7fe548009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.043+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe55004ef80 con 0x7fe56006aeb0 2023-09-27T23:33:12.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.203+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe56006bd10 con 0x7fe56006aeb0 2023-09-27T23:33:12.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe55cff9700 1 -- 172.21.15.64:0/1959918851 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe550019ed0 con 0x7fe56006aeb0 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe540060a70 msgr2=0x7fe540062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe540060a70 0x7fe540062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe548007fd0 tx=0x7fe548009040 comp rx=0 tx=0).stop 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 msgr2=0x7fe5601579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 0x7fe5601579a0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fe550009860 tx=0x7fe550016000 comp rx=0 tx=0).stop 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 shutdown_connections 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe540060a70 0x7fe540062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe560159770 0x7fe5600c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe560158c90 0x7fe560155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 --2- 172.21.15.64:0/1959918851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe56006aeb0 0x7fe5601579a0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 >> 172.21.15.64:0/1959918851 conn(0x7fe560067bb0 msgr2=0x7fe560159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 shutdown_connections 2023-09-27T23:33:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.215+0000 7fe565cd6700 1 -- 172.21.15.64:0/1959918851 wait complete. 2023-09-27T23:33:12.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-09-27T23:33:12.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a67b9700 1 Processor -- start 2023-09-27T23:33:12.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a67b9700 1 -- start start 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a67b9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59a0156b40 0x7f59a006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a67b9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a00ca8c0 con 0x7f59a006b2b0 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a67b9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a00cb700 con 0x7f59a0156770 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a67b9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a0156fa0 con 0x7f59a0156b40 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a5fb8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f59a006b2b0 0x7f59a00af120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39224/0 (socket says 172.21.15.64:39224) 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a5fb8700 1 -- 172.21.15.64:0/1301730870 learned_addr learned my addr 172.21.15.64:0/1301730870 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f59a4fb6700 1 --2- 172.21.15.64:0/1301730870 >> v2:172.21.15.123:3300/0 conn(0x7f59a0156b40 0x7f59a006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587154266 0 0) 0x7f59a00ca8c0 con 0x7f59a006b2b0 2023-09-27T23:33:12.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f598c003570 con 0x7f59a006b2b0 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.339+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047399537 0 0) 0x7f598c003570 con 0x7f59a006b2b0 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 >> v2:172.21.15.123:3300/0 conn(0x7f59a0156b40 msgr2=0x7f59a006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 --2- 172.21.15.64:0/1301730870 >> v2:172.21.15.123:3300/0 conn(0x7f59a0156b40 0x7f59a006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 >> v1:172.21.15.123:6789/0 conn(0x7f59a0156770 legacy=0x7f59a00c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a006b6e0 con 0x7f59a006b2b0 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f599c001470 con 0x7f59a006b2b0 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 >> v1:172.21.15.64:6789/0 conn(0x7f59a006b2b0 legacy=0x7f59a00af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f598c003a90 0x7f598c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f598c004810 0x7f598c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c005210 0x7f598c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a0156fa0 con 0x7f598c003a90 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c00c1b0 con 0x7f598c005210 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c00c370 con 0x7f598c004810 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f598c003a90 0x7f598c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a4fb6700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f598c004810 0x7f598c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a5fb8700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c005210 0x7f598c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 -- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f598c004810 msgr2=0x7f598c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f598c004810 0x7f598c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 -- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c005210 msgr2=0x7f598c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c005210 0x7f598c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 -- 172.21.15.64:0/1301730870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c003570 con 0x7f598c003a90 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a5fb8700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c005210 0x7f598c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a5fb8700 1 -- 172.21.15.64:0/1301730870 reap_dead start 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a57b7700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f598c003a90 0x7f598c003ea0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5998001f90 tx=0x7f5998001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764acf0af9943bc3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998001950 con 0x7f598c003a90 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599800c8b0 con 0x7f598c003a90 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f5993fff700 1 -- 172.21.15.64:0/1301730870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800cc50 con 0x7f598c003a90 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 -- 172.21.15.64:0/1301730870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f598c003a90 msgr2=0x7f598c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f598c003a90 0x7f598c003ea0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5998001f90 tx=0x7f5998001640 comp rx=0 tx=0).stop 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 -- 172.21.15.64:0/1301730870 shutdown_connections 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 --2- 172.21.15.64:0/1301730870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f598c003a90 0x7f598c003ea0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 -- 172.21.15.64:0/1301730870 >> 172.21.15.64:0/1301730870 conn(0x7f59a0067bb0 msgr2=0x7f59a00add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 -- 172.21.15.64:0/1301730870 shutdown_connections 2023-09-27T23:33:12.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 -- 172.21.15.64:0/1301730870 wait complete. 2023-09-27T23:33:12.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 Processor -- start 2023-09-27T23:33:12.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.343+0000 7f59a67b9700 1 -- start start 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c003a90 0x7f59a014fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a0169d60 0x7f59a014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 0x7f59a00c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a016d300 con 0x7f59a00c64f0 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00c6e80 con 0x7f598c003a90 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01502d0 con 0x7f59a0169d60 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 0x7f59a00c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 0x7f59a00c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42356/0 (socket says 172.21.15.64:42356) 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 -- 172.21.15.64:0/4040528484 learned_addr learned my addr 172.21.15.64:0/4040528484 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a4fb6700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a0169d60 0x7f59a014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a57b7700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c003a90 0x7f59a014fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 -- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a0169d60 msgr2=0x7f59a014ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a0169d60 0x7f59a014ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 -- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c003a90 msgr2=0x7f59a014fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c003a90 0x7f59a014fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 -- 172.21.15.64:0/4040528484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a016c9d0 con 0x7f59a00c64f0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a4fb6700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a0169d60 0x7f59a014ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a5fb8700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 0x7f59a00c6940 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f59a0156f10 tx=0x7f599c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007360 con 0x7f59a00c64f0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c0074c0 con 0x7f59a00c64f0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a016ccb0 con 0x7f59a00c64f0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c004830 con 0x7f59a00c64f0 2023-09-27T23:33:12.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a016d790 con 0x7f59a00c64f0 2023-09-27T23:33:12.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a016d470 con 0x7f59a00c64f0 2023-09-27T23:33:12.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.347+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f599c004ba0 con 0x7f59a00c64f0 2023-09-27T23:33:12.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.351+0000 7f59917fa700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5984060a70 0x7f5984062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:12.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.351+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f599c07c630 con 0x7f59a00c64f0 2023-09-27T23:33:12.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.351+0000 7f59a57b7700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5984060a70 0x7f5984062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:12.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.351+0000 7f59a57b7700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5984060a70 0x7f5984062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5998004290 tx=0x7f5998015000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:12.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.359+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f599c04c580 con 0x7f59a00c64f0 2023-09-27T23:33:12.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:12.579+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f59a004be40 con 0x7f59a00c64f0 2023-09-27T23:33:13.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59917fa700 1 -- 172.21.15.64:0/4040528484 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v732) v1 ==== 148+0+0 (secure 0 0 0) 0x7f599c04fba0 con 0x7f59a00c64f0 2023-09-27T23:33:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5984060a70 msgr2=0x7f5984062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5984060a70 0x7f5984062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5998004290 tx=0x7f5998015000 comp rx=0 tx=0).stop 2023-09-27T23:33:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 msgr2=0x7f59a00c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 0x7f59a00c6940 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f59a0156f10 tx=0x7f599c009040 comp rx=0 tx=0).stop 2023-09-27T23:33:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 shutdown_connections 2023-09-27T23:33:13.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5984060a70 0x7f5984062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a00c64f0 0x7f59a00c6940 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59a0169d60 0x7f59a014ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 --2- 172.21.15.64:0/4040528484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f598c003a90 0x7f59a014fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 >> 172.21.15.64:0/4040528484 conn(0x7f59a0067bb0 msgr2=0x7f59a0154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 shutdown_connections 2023-09-27T23:33:13.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.351+0000 7f59a67b9700 1 -- 172.21.15.64:0/4040528484 wait complete. 2023-09-27T23:33:13.376 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-09-27T23:33:13.376 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-09-27T23:33:13.376 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:33:13.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-09-27T23:33:13.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9abb91700 1 Processor -- start 2023-09-27T23:33:13.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9abb91700 1 -- start start 2023-09-27T23:33:13.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9abb91700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9a4158f70 0x7ff9a4067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:13.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9abb91700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a4068c70 con 0x7ff9a4069100 2023-09-27T23:33:13.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9abb91700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a4068060 con 0x7ff9a4159340 2023-09-27T23:33:13.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9abb91700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a4068340 con 0x7ff9a4158f70 2023-09-27T23:33:13.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9a992d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9a4158f70 0x7ff9a4067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:13.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9a912c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff9a4069100 0x7ff9a40cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39228/0 (socket says 172.21.15.64:39228) 2023-09-27T23:33:13.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9a992d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff9a4158f70 0x7ff9a4067980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42370/0 (socket says 172.21.15.64:42370) 2023-09-27T23:33:13.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.487+0000 7ff9a912c700 1 -- 172.21.15.64:0/3842193731 learned_addr learned my addr 172.21.15.64:0/3842193731 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:13.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a992d700 1 -- 172.21.15.64:0/3842193731 >> v1:172.21.15.123:6789/0 conn(0x7ff9a4159340 legacy=0x7ff9a40cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a992d700 1 -- 172.21.15.64:0/3842193731 >> v1:172.21.15.64:6789/0 conn(0x7ff9a4069100 legacy=0x7ff9a40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a892b700 1 -- 172.21.15.64:0/3842193731 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017844466 0 0) 0x7ff9a4068c70 con 0x7ff9a4069100 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a992d700 1 -- 172.21.15.64:0/3842193731 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4068620 con 0x7ff9a4158f70 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a992d700 1 --2- 172.21.15.64:0/3842193731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4158f70 0x7ff9a4067980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009e90 tx=0x7ff98c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481c4d3253b03704 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a892b700 1 -- 172.21.15.64:0/3842193731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00e070 con 0x7ff9a4158f70 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a892b700 1 -- 172.21.15.64:0/3842193731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c00bbf0 con 0x7ff9a4158f70 2023-09-27T23:33:13.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9a892b700 1 -- 172.21.15.64:0/3842193731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c012430 con 0x7ff9a4158f70 2023-09-27T23:33:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 -- 172.21.15.64:0/3842193731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4158f70 msgr2=0x7ff9a4067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 --2- 172.21.15.64:0/3842193731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4158f70 0x7ff9a4067980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009e90 tx=0x7ff98c009780 comp rx=0 tx=0).stop 2023-09-27T23:33:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 -- 172.21.15.64:0/3842193731 shutdown_connections 2023-09-27T23:33:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 --2- 172.21.15.64:0/3842193731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4158f70 0x7ff9a4067980 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 -- 172.21.15.64:0/3842193731 >> 172.21.15.64:0/3842193731 conn(0x7ff9a40ccc40 msgr2=0x7ff9a40cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:13.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 -- 172.21.15.64:0/3842193731 shutdown_connections 2023-09-27T23:33:13.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.491+0000 7ff9abb91700 1 -- 172.21.15.64:0/3842193731 wait complete. 2023-09-27T23:33:13.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 Processor -- start 2023-09-27T23:33:13.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 -- start start 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a4069100 0x7ff9a416cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9a4158f70 0x7ff9a415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 0x7ff9a414d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a416dc50 con 0x7ff9a4159340 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40afee0 con 0x7ff9a4069100 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a416d690 con 0x7ff9a4158f70 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 0x7ff9a414d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9a992d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a4069100 0x7ff9a416cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:13.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 0x7ff9a414d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42378/0 (socket says 172.21.15.64:42378) 2023-09-27T23:33:13.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9a992d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a4069100 0x7ff9a416cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:38218/0 (socket says 172.21.15.64:38218) 2023-09-27T23:33:13.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 -- 172.21.15.64:0/4253580908 learned_addr learned my addr 172.21.15.64:0/4253580908 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:13.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9a912c700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9a4158f70 0x7ff9a415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 -- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9a4158f70 msgr2=0x7ff9a415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9a4158f70 0x7ff9a415cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 -- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a4069100 msgr2=0x7ff9a416cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a4069100 0x7ff9a416cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 -- 172.21.15.64:0/4253580908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c009a20 con 0x7ff9a4159340 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9aa12e700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 0x7ff9a414d850 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c4c0 tx=0x7ff99c007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c00ce30 con 0x7ff9a4159340 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c011590 con 0x7ff9a4159340 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a416d970 con 0x7ff9a4159340 2023-09-27T23:33:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c011950 con 0x7ff9a4159340 2023-09-27T23:33:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.495+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a416d280 con 0x7ff9a4159340 2023-09-27T23:33:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.499+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a40b0070 con 0x7ff9a4159340 2023-09-27T23:33:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.499+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff99c011cc0 con 0x7ff9a4159340 2023-09-27T23:33:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.499+0000 7ff9a1ffb700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff984060a70 0x7ff984062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.499+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff99c07c3c0 con 0x7ff9a4159340 2023-09-27T23:33:13.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.499+0000 7ff9a992d700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff984060a70 0x7ff984062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:13.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.503+0000 7ff9a992d700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff984060a70 0x7ff984062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009fd0 tx=0x7ff98c019000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.507+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff99c04c420 con 0x7ff9a4159340 2023-09-27T23:33:13.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:13.727+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7ff9a4071120 con 0x7ff9a4159340 2023-09-27T23:33:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.351+0000 7ff9a1ffb700 1 -- 172.21.15.64:0/4253580908 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v733) v1 ==== 177+0+0 (secure 0 0 0) 0x7ff99c0170f0 con 0x7ff9a4159340 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff984060a70 msgr2=0x7ff984062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff984060a70 0x7ff984062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009fd0 tx=0x7ff98c019000 comp rx=0 tx=0).stop 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 msgr2=0x7ff9a414d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 0x7ff9a414d850 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c4c0 tx=0x7ff99c007f20 comp rx=0 tx=0).stop 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 shutdown_connections 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff984060a70 0x7ff984062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a4159340 0x7ff9a414d850 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9a4158f70 0x7ff9a415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 --2- 172.21.15.64:0/4253580908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a4069100 0x7ff9a416cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 >> 172.21.15.64:0/4253580908 conn(0x7ff9a40ccc40 msgr2=0x7ff9a40caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 shutdown_connections 2023-09-27T23:33:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.359+0000 7ff9abb91700 1 -- 172.21.15.64:0/4253580908 wait complete. 2023-09-27T23:33:14.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:33:14.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.495+0000 7fb76e445700 1 Processor -- start 2023-09-27T23:33:14.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.495+0000 7fb76e445700 1 -- start start 2023-09-27T23:33:14.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768158e20 0x7fb7680ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb7680cb970 0x7fb7680ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7680ca2c0 0x7fb76806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7680cbdf0 con 0x7fb7680ca2c0 2023-09-27T23:33:14.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7680cc0d0 con 0x7fb7680cb970 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7680cc3b0 con 0x7fb768158e20 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7680ca2c0 0x7fb76806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7680ca2c0 0x7fb76806c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42390/0 (socket says 172.21.15.64:42390) 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 -- 172.21.15.64:0/4183533262 learned_addr learned my addr 172.21.15.64:0/4183533262 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb767fff700 1 --2- 172.21.15.64:0/4183533262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768158e20 0x7fb7680ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb7677fe700 1 --2- 172.21.15.64:0/4183533262 >> v2:172.21.15.123:3300/0 conn(0x7fb7680cb970 0x7fb7680ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 -- 172.21.15.64:0/4183533262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768158e20 msgr2=0x7fb7680ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 --2- 172.21.15.64:0/4183533262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768158e20 0x7fb7680ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 -- 172.21.15.64:0/4183533262 >> v2:172.21.15.123:3300/0 conn(0x7fb7680cb970 msgr2=0x7fb7680ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 --2- 172.21.15.64:0/4183533262 >> v2:172.21.15.123:3300/0 conn(0x7fb7680cb970 0x7fb7680ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 -- 172.21.15.64:0/4183533262 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7680cc690 con 0x7fb7680ca2c0 2023-09-27T23:33:14.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76c9e2700 1 --2- 172.21.15.64:0/4183533262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7680ca2c0 0x7fb76806c7c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb768067fb0 tx=0x7fb760009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff51e57db61fa874 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb766ffd700 1 -- 172.21.15.64:0/4183533262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000e070 con 0x7fb7680ca2c0 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb766ffd700 1 -- 172.21.15.64:0/4183533262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76000b970 con 0x7fb7680ca2c0 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb766ffd700 1 -- 172.21.15.64:0/4183533262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7600092e0 con 0x7fb7680ca2c0 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 -- 172.21.15.64:0/4183533262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7680ca2c0 msgr2=0x7fb76806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- 172.21.15.64:0/4183533262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7680ca2c0 0x7fb76806c7c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb768067fb0 tx=0x7fb760009840 comp rx=0 tx=0).stop 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 -- 172.21.15.64:0/4183533262 shutdown_connections 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- 172.21.15.64:0/4183533262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7680ca2c0 0x7fb76806c7c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- 172.21.15.64:0/4183533262 >> v2:172.21.15.123:3300/0 conn(0x7fb7680cb970 0x7fb7680ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 --2- 172.21.15.64:0/4183533262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768158e20 0x7fb7680ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.499+0000 7fb76e445700 1 -- 172.21.15.64:0/4183533262 >> 172.21.15.64:0/4183533262 conn(0x7fb768067bb0 msgr2=0x7fb7680d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:14.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 -- 172.21.15.64:0/4183533262 shutdown_connections 2023-09-27T23:33:14.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 -- 172.21.15.64:0/4183533262 wait complete. 2023-09-27T23:33:14.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 Processor -- start 2023-09-27T23:33:14.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 -- start start 2023-09-27T23:33:14.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7680ca2c0 0x7fb76814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 0x7fb768159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb76815b010 0x7fb76815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76815a760 con 0x7fb768158e20 2023-09-27T23:33:14.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7680c85d0 con 0x7fb76815b010 2023-09-27T23:33:14.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.503+0000 7fb76e445700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76814f570 con 0x7fb7680ca2c0 2023-09-27T23:33:14.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 0x7fb768159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 0x7fb768159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42394/0 (socket says 172.21.15.64:42394) 2023-09-27T23:33:14.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 -- 172.21.15.64:0/2175548096 learned_addr learned my addr 172.21.15.64:0/2175548096 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:14.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb767fff700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7680ca2c0 0x7fb76814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb76c9e2700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb76815b010 0x7fb76815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 -- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7680ca2c0 msgr2=0x7fb76814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7680ca2c0 0x7fb76814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 -- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb76815b010 msgr2=0x7fb76815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb76815b010 0x7fb76815eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 -- 172.21.15.64:0/2175548096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb760009ae0 con 0x7fb768158e20 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb767fff700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7680ca2c0 0x7fb76814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb7677fe700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 0x7fb768159300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00ec80 tx=0x7fb75c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00c850 con 0x7fb768158e20 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75c00c9b0 con 0x7fb768158e20 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb768159b50 con 0x7fb768158e20 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00cd50 con 0x7fb768158e20 2023-09-27T23:33:14.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76815a110 con 0x7fb768158e20 2023-09-27T23:33:14.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.507+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7680c8740 con 0x7fb768158e20 2023-09-27T23:33:14.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.511+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb75c013680 con 0x7fb768158e20 2023-09-27T23:33:14.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.511+0000 7fb74ffff700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb754060a70 0x7fb754062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.511+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb75c07d2f0 con 0x7fb768158e20 2023-09-27T23:33:14.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.511+0000 7fb767fff700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb754060a70 0x7fb754062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.511+0000 7fb767fff700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb754060a70 0x7fb754062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb758011480 tx=0x7fb7580091a0 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:14.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.519+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb7680c8740 con 0x7fb768158e20 2023-09-27T23:33:14.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.687+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7680c8740 con 0x7fb768158e20 2023-09-27T23:33:14.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.695+0000 7fb74ffff700 1 -- 172.21.15.64:0/2175548096 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb7680c8740 con 0x7fb768158e20 2023-09-27T23:33:14.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb754060a70 msgr2=0x7fb754062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb754060a70 0x7fb754062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb758011480 tx=0x7fb7580091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 msgr2=0x7fb768159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 0x7fb768159300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00ec80 tx=0x7fb75c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 shutdown_connections 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb754060a70 0x7fb754062f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb76815b010 0x7fb76815eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb768158e20 0x7fb768159300 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 --2- 172.21.15.64:0/2175548096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb7680ca2c0 0x7fb76814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 >> 172.21.15.64:0/2175548096 conn(0x7fb768067bb0 msgr2=0x7fb768151ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 shutdown_connections 2023-09-27T23:33:14.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.699+0000 7fb76e445700 1 -- 172.21.15.64:0/2175548096 wait complete. 2023-09-27T23:33:14.716 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:33:14.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:14.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:14.750 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:33:14.750 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:33:14.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:14.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.851+0000 7f8059bf8700 1 Processor -- start 2023-09-27T23:33:14.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.851+0000 7f8059bf8700 1 -- start start 2023-09-27T23:33:14.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8054158e20 0x7f805406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80540ce810 0x7f80540ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805406b730 con 0x7f80541591f0 2023-09-27T23:33:14.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805406ab20 con 0x7f8054158e20 2023-09-27T23:33:14.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805406ae00 con 0x7f80540ce810 2023-09-27T23:33:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8058bf6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8054158e20 0x7f805406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8053fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f80541591f0 0x7f80540ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:46308/0 (socket says 172.21.15.64:46308) 2023-09-27T23:33:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8053fff700 1 -- 172.21.15.64:0/920376928 learned_addr learned my addr 172.21.15.64:0/920376928 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f80593f7700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80540ce810 0x7f80540ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8058bf6700 1 -- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80540ce810 msgr2=0x7f80540ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8058bf6700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80540ce810 0x7f80540ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8058bf6700 1 -- 172.21.15.64:0/920376928 >> v1:172.21.15.123:6789/0 conn(0x7f80541591f0 legacy=0x7f80540ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8058bf6700 1 -- 172.21.15.64:0/920376928 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805406b0e0 con 0x7f8054158e20 2023-09-27T23:33:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f80593f7700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80540ce810 0x7f80540ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8058bf6700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8054158e20 0x7f805406a540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1f30306aaf8fba server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f80537fe700 1 -- 172.21.15.64:0/920376928 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00b640 con 0x7f8054158e20 2023-09-27T23:33:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f80537fe700 1 -- 172.21.15.64:0/920376928 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c00b7a0 con 0x7f8054158e20 2023-09-27T23:33:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f80537fe700 1 -- 172.21.15.64:0/920376928 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00bb40 con 0x7f8054158e20 2023-09-27T23:33:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 -- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8054158e20 msgr2=0x7f805406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8054158e20 0x7f805406a540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 -- 172.21.15.64:0/920376928 shutdown_connections 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80540ce810 0x7f80540ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 --2- 172.21.15.64:0/920376928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8054158e20 0x7f805406a540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.855+0000 7f8059bf8700 1 -- 172.21.15.64:0/920376928 >> 172.21.15.64:0/920376928 conn(0x7f8054067bb0 msgr2=0x7f80540cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 -- 172.21.15.64:0/920376928 shutdown_connections 2023-09-27T23:33:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 -- 172.21.15.64:0/920376928 wait complete. 2023-09-27T23:33:14.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 Processor -- start 2023-09-27T23:33:14.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 -- start start 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80540ce810 0x7f8054169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 0x7f80541497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80541591f0 0x7f8054157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8054155970 con 0x7f8054158e20 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805416a3e0 con 0x7f80540ce810 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8059bf8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8054154d60 con 0x7f80541591f0 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8053fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 0x7f80541497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8053fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 0x7f80541497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44926/0 (socket says 172.21.15.64:44926) 2023-09-27T23:33:14.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8053fff700 1 -- 172.21.15.64:0/3626419060 learned_addr learned my addr 172.21.15.64:0/3626419060 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f80593f7700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80541591f0 0x7f8054157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.859+0000 7f8058bf6700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80540ce810 0x7f8054169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8053fff700 1 -- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80541591f0 msgr2=0x7f8054157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8053fff700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80541591f0 0x7f8054157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8053fff700 1 -- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80540ce810 msgr2=0x7f8054169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8053fff700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80540ce810 0x7f8054169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8053fff700 1 -- 172.21.15.64:0/3626419060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c009a20 con 0x7f8054158e20 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f80593f7700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80541591f0 0x7f8054157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8053fff700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 0x7f80541497f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f804400c5c0 tx=0x7f8044007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:14.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044005b30 con 0x7f8054158e20 2023-09-27T23:33:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c009420 con 0x7f8054158e20 2023-09-27T23:33:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8044005cb0 con 0x7f8054158e20 2023-09-27T23:33:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044007320 con 0x7f8054158e20 2023-09-27T23:33:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80541553a0 con 0x7f8054158e20 2023-09-27T23:33:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8054155ae0 con 0x7f8054158e20 2023-09-27T23:33:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f804401b3b0 con 0x7f8054158e20 2023-09-27T23:33:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8050ff9700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8034060a70 0x7f8034062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f804407c430 con 0x7f8054158e20 2023-09-27T23:33:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.863+0000 7f8058bf6700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8034060a70 0x7f8034062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:14.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.867+0000 7f8058bf6700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8034060a70 0x7f8034062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f803c0091e0 tx=0x7f803c017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:14.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:14.875+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f804404c550 con 0x7f8054158e20 2023-09-27T23:33:15.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f805404be40 con 0x7f8054158e20 2023-09-27T23:33:15.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8050ff9700 1 -- 172.21.15.64:0/3626419060 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f804404fb70 con 0x7f8054158e20 2023-09-27T23:33:15.078 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:15.078 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7118,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1406608655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1406608655},{"type":"v1","addr":"172.21.15.64:6833","nonce":1406608655}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2637090403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2637090403},{"type":"v1","addr":"172.21.15.123:6836","nonce":2637090403}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/783024701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":783024701},{"type":"v1","addr":"172.21.15.123:6837","nonce":783024701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/4190628750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":4190628750},{"type":"v1","addr":"172.21.15.64:6835","nonce":4190628750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[]} 2023-09-27T23:33:15.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8034060a70 msgr2=0x7f8034062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:15.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8034060a70 0x7f8034062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f803c0091e0 tx=0x7f803c017000 comp rx=0 tx=0).stop 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 msgr2=0x7f80541497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 0x7f80541497f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f804400c5c0 tx=0x7f8044007f90 comp rx=0 tx=0).stop 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 shutdown_connections 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8034060a70 0x7f8034062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80541591f0 0x7f8054157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8054158e20 0x7f80541497f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 --2- 172.21.15.64:0/3626419060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80540ce810 0x7f8054169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 >> 172.21.15.64:0/3626419060 conn(0x7f8054067bb0 msgr2=0x7f80540b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 shutdown_connections 2023-09-27T23:33:15.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.075+0000 7f8059bf8700 1 -- 172.21.15.64:0/3626419060 wait complete. 2023-09-27T23:33:15.082 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 48 2023-09-27T23:33:15.092 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:33:15.093 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:33:15.093 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:33:15.093 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:15.093 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:33:15.093 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:33:15.093 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:33:15.094 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:15.094 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:33:15.094 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:33:15.094 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:33:15.094 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:15.094 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:33:15.094 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:33:15.095 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:33:15.095 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:15.103 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:33:15.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:33:15.109 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:33:15.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:33:15.114 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:33:15.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:33:15.115 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:33:15.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:33:15.157 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:33:15.157 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:15.158 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:33:15.159 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:33:15.159 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:15.159 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:15.159 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:33:15.160 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:15.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:33:15.229 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:33:15.232 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:33:15.237 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:33:15.238 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:33:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 Processor -- start 2023-09-27T23:33:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 -- start start 2023-09-27T23:33:15.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcf6806be10 0x7fcf680b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf68154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf680af530 con 0x7fcf680b0810 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf68159470 con 0x7fcf6806be10 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6e0e6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68159660 con 0x7fcf680af9c0 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.275+0000 7fcf6d0e4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcf680b0810 0x7fcf68158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37496/0 (socket says 172.21.15.64:37496) 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6d0e4700 1 -- 172.21.15.64:0/1208053415 learned_addr learned my addr 172.21.15.64:0/1208053415 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6c8e3700 1 --2- 172.21.15.64:0/1208053415 >> v2:172.21.15.123:3300/0 conn(0x7fcf6806be10 0x7fcf680b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6d8e5700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf68154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf5ffff700 1 -- 172.21.15.64:0/1208053415 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037146947 0 0) 0x7fcf680af530 con 0x7fcf680b0810 2023-09-27T23:33:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf5ffff700 1 -- 172.21.15.64:0/1208053415 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf54003570 con 0x7fcf680b0810 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6c8e3700 1 -- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 msgr2=0x7fcf68154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6c8e3700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf68154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6c8e3700 1 -- 172.21.15.64:0/1208053415 >> v1:172.21.15.64:6789/0 conn(0x7fcf680b0810 legacy=0x7fcf68158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6c8e3700 1 -- 172.21.15.64:0/1208053415 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68159940 con 0x7fcf6806be10 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6d8e5700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf68154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6c8e3700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6806be10 0x7fcf680b0be0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcf640098e0 tx=0x7fcf640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b6001a40aaf193 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf5ffff700 1 -- 172.21.15.64:0/1208053415 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64011630 con 0x7fcf6806be10 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf5ffff700 1 -- 172.21.15.64:0/1208053415 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64011790 con 0x7fcf6806be10 2023-09-27T23:33:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf5ffff700 1 -- 172.21.15.64:0/1208053415 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64011b10 con 0x7fcf6806be10 2023-09-27T23:33:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6806be10 msgr2=0x7fcf680b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6806be10 0x7fcf680b0be0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcf640098e0 tx=0x7fcf640091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/1208053415 shutdown_connections 2023-09-27T23:33:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf68154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/1208053415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6806be10 0x7fcf680b0be0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/1208053415 >> 172.21.15.64:0/1208053415 conn(0x7fcf68067bb0 msgr2=0x7fcf680add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/1208053415 shutdown_connections 2023-09-27T23:33:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.279+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/1208053415 wait complete. 2023-09-27T23:33:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 Processor -- start 2023-09-27T23:33:15.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 -- start start 2023-09-27T23:33:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 0x7fcf68142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf681572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf680b0810 0x7fcf68157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:15.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68168150 con 0x7fcf6806be10 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68168db0 con 0x7fcf680b0810 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68168480 con 0x7fcf680af9c0 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 0x7fcf68142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 0x7fcf68142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44990/0 (socket says 172.21.15.64:44990) 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 -- 172.21.15.64:0/100966095 learned_addr learned my addr 172.21.15.64:0/100966095 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6c8e3700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf681572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d8e5700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf680b0810 0x7fcf68157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 -- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 msgr2=0x7fcf681572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf681572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 -- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf680b0810 msgr2=0x7fcf68157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf680b0810 0x7fcf68157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 -- 172.21.15.64:0/100966095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf60009a20 con 0x7fcf6806be10 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6d0e4700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 0x7fcf68142b90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c590 tx=0x7fcf58007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580063b0 con 0x7fcf6806be10 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58006510 con 0x7fcf6806be10 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf64009420 con 0x7fcf6806be10 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58004a80 con 0x7fcf6806be10 2023-09-27T23:33:15.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.283+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf68168a40 con 0x7fcf6806be10 2023-09-27T23:33:15.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.287+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6804be40 con 0x7fcf6806be10 2023-09-27T23:33:15.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.287+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fcf5801b3b0 con 0x7fcf6806be10 2023-09-27T23:33:15.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.287+0000 7fcf5d7fa700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf4c060aa0 0x7fcf4c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:15.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.287+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6300+0+0 (secure 0 0 0) 0x7fcf5807c570 con 0x7fcf6806be10 2023-09-27T23:33:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.287+0000 7fcf6c8e3700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf4c060aa0 0x7fcf4c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:15.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.287+0000 7fcf6c8e3700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf4c060aa0 0x7fcf4c062f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806c250 tx=0x7fcf64017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:15.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.295+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcf5804c5f0 con 0x7fcf6806be10 2023-09-27T23:33:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:15.467+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcf68159c20 con 0x7fcf6806be10 2023-09-27T23:33:16.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.367+0000 7fcf5d7fa700 1 -- 172.21.15.64:0/100966095 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcf5804fc10 con 0x7fcf6806be10 2023-09-27T23:33:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf4c060aa0 msgr2=0x7fcf4c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf4c060aa0 0x7fcf4c062f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806c250 tx=0x7fcf64017000 comp rx=0 tx=0).stop 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 msgr2=0x7fcf68142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 0x7fcf68142b90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c590 tx=0x7fcf58007dd0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 shutdown_connections 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf4c060aa0 0x7fcf4c062f50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf680b0810 0x7fcf68157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf680af9c0 0x7fcf681572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 --2- 172.21.15.64:0/100966095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf6806be10 0x7fcf68142b90 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 >> 172.21.15.64:0/100966095 conn(0x7fcf68067bb0 msgr2=0x7fcf68153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 shutdown_connections 2023-09-27T23:33:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.371+0000 7fcf6e0e6700 1 -- 172.21.15.64:0/100966095 wait complete. 2023-09-27T23:33:16.376 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:33:16.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:33:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.495+0000 7f8dc26bd700 1 Processor -- start 2023-09-27T23:33:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.495+0000 7f8dc26bd700 1 -- start start 2023-09-27T23:33:16.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8dbc06a580 0x7f8dbc06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06d210 0x7f8dbc06d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc06c7c0 con 0x7f8dbc0b1410 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc06af40 con 0x7f8dbc06a580 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc06cd80 con 0x7f8dbc06d210 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8dbc06a580 0x7f8dbc06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc16bb700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8dbc0b1410 0x7f8dbc0ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37512/0 (socket says 172.21.15.64:37512) 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8dbc06a580 0x7f8dbc06aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44994/0 (socket says 172.21.15.64:44994) 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc16bb700 1 -- 172.21.15.64:0/818668750 learned_addr learned my addr 172.21.15.64:0/818668750 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc1ebc700 1 --2- 172.21.15.64:0/818668750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06d210 0x7f8dbc06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 -- 172.21.15.64:0/818668750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06d210 msgr2=0x7f8dbc06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 --2- 172.21.15.64:0/818668750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06d210 0x7f8dbc06d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 -- 172.21.15.64:0/818668750 >> v1:172.21.15.64:6789/0 conn(0x7f8dbc0b1410 legacy=0x7f8dbc0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 -- 172.21.15.64:0/818668750 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc06b8f0 con 0x7f8dbc06a580 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc0eba700 1 --2- 172.21.15.64:0/818668750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06a580 0x7f8dbc06aa00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098e0 tx=0x7f8db00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c807aa83be86f91 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8daffff700 1 -- 172.21.15.64:0/818668750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000b640 con 0x7f8dbc06a580 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8daffff700 1 -- 172.21.15.64:0/818668750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db000b7a0 con 0x7f8dbc06a580 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8daffff700 1 -- 172.21.15.64:0/818668750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000bb40 con 0x7f8dbc06a580 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- 172.21.15.64:0/818668750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06a580 msgr2=0x7f8dbc06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/818668750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06a580 0x7f8dbc06aa00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098e0 tx=0x7f8db00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- 172.21.15.64:0/818668750 shutdown_connections 2023-09-27T23:33:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/818668750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06d210 0x7f8dbc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/818668750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06a580 0x7f8dbc06aa00 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- 172.21.15.64:0/818668750 >> 172.21.15.64:0/818668750 conn(0x7f8dbc067bb0 msgr2=0x7f8dbc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- 172.21.15.64:0/818668750 shutdown_connections 2023-09-27T23:33:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.499+0000 7f8dc26bd700 1 -- 172.21.15.64:0/818668750 wait complete. 2023-09-27T23:33:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.503+0000 7f8dc26bd700 1 Processor -- start 2023-09-27T23:33:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.503+0000 7f8dc26bd700 1 -- start start 2023-09-27T23:33:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06a580 0x7f8dbc1671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 0x7f8dbc0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dbc0b1410 0x7f8dbc1484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c4430 con 0x7f8dbc06d210 2023-09-27T23:33:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc148a30 con 0x7f8dbc0b1410 2023-09-27T23:33:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c08f0 con 0x7f8dbc06a580 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 0x7f8dbc0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 0x7f8dbc0c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45004/0 (socket says 172.21.15.64:45004) 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 -- 172.21.15.64:0/2321694345 learned_addr learned my addr 172.21.15.64:0/2321694345 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc16bb700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06a580 0x7f8dbc1671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc1ebc700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dbc0b1410 0x7f8dbc1484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 -- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06a580 msgr2=0x7f8dbc1671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06a580 0x7f8dbc1671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 -- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dbc0b1410 msgr2=0x7f8dbc1484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dbc0b1410 0x7f8dbc1484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 -- 172.21.15.64:0/2321694345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0009420 con 0x7f8dbc06d210 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc16bb700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06a580 0x7f8dbc1671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc1ebc700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dbc0b1410 0x7f8dbc1484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc0eba700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 0x7f8dbc0c3ef0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b6c0 tx=0x7f8db0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000bb20 con 0x7f8dbc06d210 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0011420 con 0x7f8dbc06d210 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc167810 con 0x7f8dbc06d210 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0011800 con 0x7f8dbc06d210 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc167dd0 con 0x7f8dbc06d210 2023-09-27T23:33:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.507+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc04be40 con 0x7f8dbc06d210 2023-09-27T23:33:16.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.511+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8db0011c30 con 0x7f8dbc06d210 2023-09-27T23:33:16.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.511+0000 7f8dad7fa700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8da0060a70 0x7f8da0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.511+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f8db007e9b0 con 0x7f8dbc06d210 2023-09-27T23:33:16.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.511+0000 7f8dc16bb700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8da0060a70 0x7f8da0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.511+0000 7f8dc16bb700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8da0060a70 0x7f8da0062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1583a0 tx=0x7f8db8009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:16.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.519+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8db004ec90 con 0x7f8dbc06d210 2023-09-27T23:33:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.735+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8dbc0c0a30 con 0x7f8dbc06d210 2023-09-27T23:33:16.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.735+0000 7f8dad7fa700 1 -- 172.21.15.64:0/2321694345 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8db00522b0 con 0x7f8dbc06d210 2023-09-27T23:33:16.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:16.743 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.744 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:16.745 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:16.745 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.748 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:33:16.748 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.749 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.750 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.751 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:33:16.753 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.754 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.755 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:33:16.756 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.757 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:33:16.758 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:33:16.760 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.761 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:33:16.762 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:16.763 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.763 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:16.763 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:33:16.763 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:33:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8da0060a70 msgr2=0x7f8da0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8da0060a70 0x7f8da0062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1583a0 tx=0x7f8db8009040 comp rx=0 tx=0).stop 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 msgr2=0x7f8dbc0c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 0x7f8dbc0c3ef0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b6c0 tx=0x7f8db0017000 comp rx=0 tx=0).stop 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 shutdown_connections 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8da0060a70 0x7f8da0062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dbc0b1410 0x7f8dbc1484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dbc06d210 0x7f8dbc0c3ef0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 --2- 172.21.15.64:0/2321694345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dbc06a580 0x7f8dbc1671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 >> 172.21.15.64:0/2321694345 conn(0x7f8dbc067bb0 msgr2=0x7f8dbc156420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 shutdown_connections 2023-09-27T23:33:16.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.739+0000 7f8dc26bd700 1 -- 172.21.15.64:0/2321694345 wait complete. 2023-09-27T23:33:16.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662fef8700 1 Processor -- start 2023-09-27T23:33:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662fef8700 1 -- start start 2023-09-27T23:33:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662fef8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f662815d320 0x7f662815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662fef8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280c7190 con 0x7f66280aeba0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662fef8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280c1b50 con 0x7f66280c15b0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662fef8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c1e30 con 0x7f662815d320 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662e495700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f66280aeba0 0x7f66280680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37520/0 (socket says 172.21.15.64:37520) 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662e495700 1 -- 172.21.15.64:0/346280112 learned_addr learned my addr 172.21.15.64:0/346280112 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662dc94700 1 --2- 172.21.15.64:0/346280112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f662815d320 0x7f662815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148393767 0 0) 0x7f66280c7190 con 0x7f66280aeba0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6614003570 con 0x7f66280aeba0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706186461 0 0) 0x7f66280c1b50 con 0x7f66280c15b0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66280c7190 con 0x7f66280c15b0 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662dc94700 1 -- 172.21.15.64:0/346280112 >> v1:172.21.15.123:6789/0 conn(0x7f66280c15b0 legacy=0x7f662806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662dc94700 1 -- 172.21.15.64:0/346280112 >> v1:172.21.15.64:6789/0 conn(0x7f66280aeba0 legacy=0x7f66280680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662dc94700 1 -- 172.21.15.64:0/346280112 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280c6e20 con 0x7f662815d320 2023-09-27T23:33:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662dc94700 1 --2- 172.21.15.64:0/346280112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f662815d320 0x7f662815d6f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f661c009910 tx=0x7f661c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471aaf193551b7c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011630 con 0x7f662815d320 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c011790 con 0x7f662815d320 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.879+0000 7f662cc92700 1 -- 172.21.15.64:0/346280112 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011b30 con 0x7f662815d320 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- 172.21.15.64:0/346280112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f662815d320 msgr2=0x7f662815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 --2- 172.21.15.64:0/346280112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f662815d320 0x7f662815d6f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f661c009910 tx=0x7f661c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- 172.21.15.64:0/346280112 shutdown_connections 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 --2- 172.21.15.64:0/346280112 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f662815d320 0x7f662815d6f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- 172.21.15.64:0/346280112 >> 172.21.15.64:0/346280112 conn(0x7f66280c8850 msgr2=0x7f66280c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- 172.21.15.64:0/346280112 shutdown_connections 2023-09-27T23:33:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- 172.21.15.64:0/346280112 wait complete. 2023-09-27T23:33:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 Processor -- start 2023-09-27T23:33:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- start start 2023-09-27T23:33:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 0x7f6628158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66280c15b0 0x7f66280c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f662815d320 0x7f662816ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628159ab0 con 0x7f66280c15b0 2023-09-27T23:33:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628159020 con 0x7f662815d320 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662fef8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662816d880 con 0x7f66280aeba0 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662d493700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66280c15b0 0x7f66280c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662dc94700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 0x7f6628158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662d493700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66280c15b0 0x7f66280c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45014/0 (socket says 172.21.15.64:45014) 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662dc94700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 0x7f6628158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38720/0 (socket says 172.21.15.64:38720) 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662d493700 1 -- 172.21.15.64:0/40840165 learned_addr learned my addr 172.21.15.64:0/40840165 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.883+0000 7f662e495700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f662815d320 0x7f662816ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662dc94700 1 -- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f662815d320 msgr2=0x7f662816ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662dc94700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f662815d320 0x7f662816ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662dc94700 1 -- 172.21.15.64:0/40840165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66280c15b0 msgr2=0x7f66280c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662dc94700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66280c15b0 0x7f66280c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662dc94700 1 -- 172.21.15.64:0/40840165 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c009420 con 0x7f66280aeba0 2023-09-27T23:33:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662dc94700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 0x7f6628158ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f661c016000 tx=0x7f661c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c01f070 con 0x7f66280aeba0 2023-09-27T23:33:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c002f70 con 0x7f66280aeba0 2023-09-27T23:33:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66280c4f10 con 0x7f66280aeba0 2023-09-27T23:33:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011900 con 0x7f66280aeba0 2023-09-27T23:33:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281594f0 con 0x7f66280aeba0 2023-09-27T23:33:16.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.887+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628071120 con 0x7f66280aeba0 2023-09-27T23:33:16.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.891+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f661c023070 con 0x7f66280aeba0 2023-09-27T23:33:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.891+0000 7f6619ffb700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f660c060aa0 0x7f660c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.891+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f661c07dd70 con 0x7f66280aeba0 2023-09-27T23:33:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.891+0000 7f662d493700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f660c060aa0 0x7f660c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:16.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.891+0000 7f662d493700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f660c060aa0 0x7f660c062f50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f66280ae8b0 tx=0x7f6620002820 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:16.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:16.899+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f661c04e170 con 0x7f66280aeba0 2023-09-27T23:33:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.059+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66280642a0 con 0x7f66280aeba0 2023-09-27T23:33:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.059+0000 7f6619ffb700 1 -- 172.21.15.64:0/40840165 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+13865 (secure 0 0 0) 0x7f661c051790 con 0x7f66280aeba0 2023-09-27T23:33:17.064 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:17.064 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":139,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:16.359417+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f660c060aa0 msgr2=0x7f660c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f660c060aa0 0x7f660c062f50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f66280ae8b0 tx=0x7f6620002820 comp rx=0 tx=0).stop 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 msgr2=0x7f6628158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 0x7f6628158ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f661c016000 tx=0x7f661c017000 comp rx=0 tx=0).stop 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 shutdown_connections 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f660c060aa0 0x7f660c062f50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f662815d320 0x7f662816ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66280c15b0 0x7f66280c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 --2- 172.21.15.64:0/40840165 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66280aeba0 0x7f6628158ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 >> 172.21.15.64:0/40840165 conn(0x7f66280c8850 msgr2=0x7f662806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 shutdown_connections 2023-09-27T23:33:17.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.063+0000 7f662fef8700 1 -- 172.21.15.64:0/40840165 wait complete. 2023-09-27T23:33:17.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:33:17.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:33:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 Processor -- start 2023-09-27T23:33:17.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 -- start start 2023-09-27T23:33:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94a0154780 0x7f94a015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94a00adab0 0x7f94a00adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a015ae30 con 0x7f94a006baa0 2023-09-27T23:33:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00abf70 con 0x7f94a00adab0 2023-09-27T23:33:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f94a48df700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a0154b50 con 0x7f94a0154780 2023-09-27T23:33:17.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94a00adab0 0x7f94a00adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94a00adab0 0x7f94a00adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45016/0 (socket says 172.21.15.64:45016) 2023-09-27T23:33:17.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f94a006baa0 0x7f94a0157b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37526/0 (socket says 172.21.15.64:37526) 2023-09-27T23:33:17.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 -- 172.21.15.64:0/1959891772 learned_addr learned my addr 172.21.15.64:0/1959891772 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:17.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949f7fe700 1 --2- 172.21.15.64:0/1959891772 >> v2:172.21.15.123:3300/0 conn(0x7f94a0154780 0x7f94a015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 -- 172.21.15.64:0/1959891772 >> v2:172.21.15.123:3300/0 conn(0x7f94a0154780 msgr2=0x7f94a015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 --2- 172.21.15.64:0/1959891772 >> v2:172.21.15.123:3300/0 conn(0x7f94a0154780 0x7f94a015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 -- 172.21.15.64:0/1959891772 >> v1:172.21.15.64:6789/0 conn(0x7f94a006baa0 legacy=0x7f94a0157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949e7fc700 1 -- 172.21.15.64:0/1959891772 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854860976 0 0) 0x7f94a015ae30 con 0x7f94a006baa0 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 -- 172.21.15.64:0/1959891772 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00ac210 con 0x7f94a00adab0 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949ffff700 1 --2- 172.21.15.64:0/1959891772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00adeb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f94a006a3f0 tx=0x7f9498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6305f154aa9cb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949e7fc700 1 -- 172.21.15.64:0/1959891772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949800b820 con 0x7f94a00adab0 2023-09-27T23:33:17.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949e7fc700 1 -- 172.21.15.64:0/1959891772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949800b980 con 0x7f94a00adab0 2023-09-27T23:33:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.195+0000 7f949e7fc700 1 -- 172.21.15.64:0/1959891772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949800bd20 con 0x7f94a00adab0 2023-09-27T23:33:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 -- 172.21.15.64:0/1959891772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 msgr2=0x7f94a00adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 --2- 172.21.15.64:0/1959891772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00adeb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f94a006a3f0 tx=0x7f9498009250 comp rx=0 tx=0).stop 2023-09-27T23:33:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 -- 172.21.15.64:0/1959891772 shutdown_connections 2023-09-27T23:33:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 --2- 172.21.15.64:0/1959891772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00adeb0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 --2- 172.21.15.64:0/1959891772 >> v2:172.21.15.123:3300/0 conn(0x7f94a0154780 0x7f94a015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 -- 172.21.15.64:0/1959891772 >> 172.21.15.64:0/1959891772 conn(0x7f94a0067b50 msgr2=0x7f94a00cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 -- 172.21.15.64:0/1959891772 shutdown_connections 2023-09-27T23:33:17.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.199+0000 7f94a48df700 1 -- 172.21.15.64:0/1959891772 wait complete. 2023-09-27T23:33:17.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 Processor -- start 2023-09-27T23:33:17.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 -- start start 2023-09-27T23:33:17.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a006baa0 0x7f94a0141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94a0154780 0x7f94a016b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00b18f0 con 0x7f94a00adab0 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0142340 con 0x7f94a006baa0 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0142620 con 0x7f94a0154780 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45022/0 (socket says 172.21.15.64:45022) 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 -- 172.21.15.64:0/1715147104 learned_addr learned my addr 172.21.15.64:0/1715147104 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949ffff700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94a0154780 0x7f94a016b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949f7fe700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a006baa0 0x7f94a0141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 -- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94a0154780 msgr2=0x7f94a016b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94a0154780 0x7f94a016b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 -- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a006baa0 msgr2=0x7f94a0141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a006baa0 0x7f94a0141cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 -- 172.21.15.64:0/1715147104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980094f0 con 0x7f94a00adab0 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949f7fe700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a006baa0 0x7f94a0141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949ffff700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94a0154780 0x7f94a016b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f949effd700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00b13b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f948800cd00 tx=0x7f9488001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488005df0 con 0x7f94a00adab0 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948800a960 con 0x7f94a00adab0 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800ad00 con 0x7f94a00adab0 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0142900 con 0x7f94a00adab0 2023-09-27T23:33:17.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.203+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0142ec0 con 0x7f94a00adab0 2023-09-27T23:33:17.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.207+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a004be40 con 0x7f94a00adab0 2023-09-27T23:33:17.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.207+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9488007300 con 0x7f94a00adab0 2023-09-27T23:33:17.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.207+0000 7f94977fe700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9480060a70 0x7f9480062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.207+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f948807cfd0 con 0x7f94a00adab0 2023-09-27T23:33:17.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.207+0000 7f949f7fe700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9480060a70 0x7f9480062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.207+0000 7f949f7fe700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9480060a70 0x7f9480062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.215+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f948804d330 con 0x7f94a00adab0 2023-09-27T23:33:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.379+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94a00cc1a0 con 0x7f94a00adab0 2023-09-27T23:33:17.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.379+0000 7f94977fe700 1 -- 172.21.15.64:0/1715147104 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f948800ae60 con 0x7f94a00adab0 2023-09-27T23:33:17.384 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9480060a70 msgr2=0x7f9480062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9480060a70 0x7f9480062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).stop 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 msgr2=0x7f94a00b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00b13b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f948800cd00 tx=0x7f9488001ac0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 shutdown_connections 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9480060a70 0x7f9480062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94a0154780 0x7f94a016b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94a00adab0 0x7f94a00b13b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 --2- 172.21.15.64:0/1715147104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a006baa0 0x7f94a0141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 >> 172.21.15.64:0/1715147104 conn(0x7f94a0067b50 msgr2=0x7f94a00c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 shutdown_connections 2023-09-27T23:33:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.387+0000 7f94a48df700 1 -- 172.21.15.64:0/1715147104 wait complete. 2023-09-27T23:33:17.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:33:17.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.523+0000 7f21d5f27700 1 Processor -- start 2023-09-27T23:33:17.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.523+0000 7f21d5f27700 1 -- start start 2023-09-27T23:33:17.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.523+0000 7f21d5f27700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21d0142680 0x7f21d00c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d00cc7e0 con 0x7f21d00c8f40 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d00cbbf0 con 0x7f21d006d760 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00cbeb0 con 0x7f21d0142680 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d4f25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21d0142680 0x7f21d00c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d4f25700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21d0142680 0x7f21d00c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45026/0 (socket says 172.21.15.64:45026) 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f21d00c8f40 0x7f21d006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37540/0 (socket says 172.21.15.64:37540) 2023-09-27T23:33:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cffff700 1 -- 172.21.15.64:0/313170657 learned_addr learned my addr 172.21.15.64:0/313170657 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584962656 0 0) 0x7f21d00cc7e0 con 0x7f21d00c8f40 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b8003570 con 0x7f21d00c8f40 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689832528 0 0) 0x7f21d00cbbf0 con 0x7f21d006d760 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d00cc7e0 con 0x7f21d006d760 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d4f25700 1 -- 172.21.15.64:0/313170657 >> v1:172.21.15.123:6789/0 conn(0x7f21d006d760 legacy=0x7f21d00ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d4f25700 1 -- 172.21.15.64:0/313170657 >> v1:172.21.15.64:6789/0 conn(0x7f21d00c8f40 legacy=0x7f21d006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d4f25700 1 -- 172.21.15.64:0/313170657 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00cc190 con 0x7f21d0142680 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d4f25700 1 --2- 172.21.15.64:0/313170657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d0142680 0x7f21d00c8a00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f21c00098e0 tx=0x7f21c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4798c285ddf0a1d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0011630 con 0x7f21d0142680 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c0011790 con 0x7f21d0142680 2023-09-27T23:33:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21cf7fe700 1 -- 172.21.15.64:0/313170657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0011b30 con 0x7f21d0142680 2023-09-27T23:33:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- 172.21.15.64:0/313170657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d0142680 msgr2=0x7f21d00c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 --2- 172.21.15.64:0/313170657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d0142680 0x7f21d00c8a00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f21c00098e0 tx=0x7f21c00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- 172.21.15.64:0/313170657 shutdown_connections 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 --2- 172.21.15.64:0/313170657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d0142680 0x7f21d00c8a00 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- 172.21.15.64:0/313170657 >> 172.21.15.64:0/313170657 conn(0x7f21d00ccb30 msgr2=0x7f21d00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- 172.21.15.64:0/313170657 shutdown_connections 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.527+0000 7f21d5f27700 1 -- 172.21.15.64:0/313170657 wait complete. 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 Processor -- start 2023-09-27T23:33:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 -- start start 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 0x7f21d00c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21d00c8f40 0x7f21d00c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21d0142680 0x7f21d00c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00c3be0 con 0x7f21d006d760 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0069160 con 0x7f21d0142680 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0069440 con 0x7f21d00c8f40 2023-09-27T23:33:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 0x7f21d00c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 0x7f21d00c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45038/0 (socket says 172.21.15.64:45038) 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 -- 172.21.15.64:0/3020411333 learned_addr learned my addr 172.21.15.64:0/3020411333 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21cffff700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21d00c8f40 0x7f21d00c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5726700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21d0142680 0x7f21d00c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 -- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21d00c8f40 msgr2=0x7f21d00c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21d00c8f40 0x7f21d00c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 -- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21d0142680 msgr2=0x7f21d00c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21d0142680 0x7f21d00c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 -- 172.21.15.64:0/3020411333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009420 con 0x7f21d006d760 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d4f25700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 0x7f21d00c45f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f21d006b3a0 tx=0x7f21c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c001f070 con 0x7f21d006d760 2023-09-27T23:33:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d00c3df0 con 0x7f21d006d760 2023-09-27T23:33:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c0000c90 con 0x7f21d006d760 2023-09-27T23:33:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0025420 con 0x7f21d006d760 2023-09-27T23:33:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.531+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d00c68c0 con 0x7f21d006d760 2023-09-27T23:33:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.535+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d004be40 con 0x7f21d006d760 2023-09-27T23:33:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.535+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f21c0023070 con 0x7f21d006d760 2023-09-27T23:33:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.535+0000 7f21ccff9700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f21bc060a70 0x7f21bc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.535+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f21c007e1d0 con 0x7f21d006d760 2023-09-27T23:33:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.535+0000 7f21cffff700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f21bc060a70 0x7f21bc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.535+0000 7f21cffff700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f21bc060a70 0x7f21bc062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f21d0141600 tx=0x7f21c4009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.543+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f21c004e4b0 con 0x7f21d006d760 2023-09-27T23:33:17.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.707+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f21d00cc470 con 0x7f21d006d760 2023-09-27T23:33:17.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.707+0000 7f21ccff9700 1 -- 172.21.15.64:0/3020411333 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7f21c002a020 con 0x7f21d006d760 2023-09-27T23:33:17.711 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:33:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f21bc060a70 msgr2=0x7f21bc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f21bc060a70 0x7f21bc062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f21d0141600 tx=0x7f21c4009040 comp rx=0 tx=0).stop 2023-09-27T23:33:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 msgr2=0x7f21d00c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 0x7f21d00c45f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f21d006b3a0 tx=0x7f21c0017000 comp rx=0 tx=0).stop 2023-09-27T23:33:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 shutdown_connections 2023-09-27T23:33:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f21bc060a70 0x7f21bc062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21d0142680 0x7f21d00c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21d00c8f40 0x7f21d00c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 --2- 172.21.15.64:0/3020411333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d006d760 0x7f21d00c45f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 >> 172.21.15.64:0/3020411333 conn(0x7f21d00ccb30 msgr2=0x7f21d013c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 shutdown_connections 2023-09-27T23:33:17.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.711+0000 7f21d5f27700 1 -- 172.21.15.64:0/3020411333 wait complete. 2023-09-27T23:33:17.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:33:17.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.823+0000 7f7a7f59e700 1 Processor -- start 2023-09-27T23:33:17.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.823+0000 7f7a7f59e700 1 -- start start 2023-09-27T23:33:17.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a800670e0 0x7f7a80159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a800abce0 0x7f7a800c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a800c1910 con 0x7f7a800ab910 2023-09-27T23:33:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a80159cf0 con 0x7f7a800670e0 2023-09-27T23:33:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a80159fd0 con 0x7f7a800abce0 2023-09-27T23:33:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a800670e0 0x7f7a80159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a800670e0 0x7f7a80159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45054/0 (socket says 172.21.15.64:45054) 2023-09-27T23:33:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 -- 172.21.15.64:0/4094651027 learned_addr learned my addr 172.21.15.64:0/4094651027 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:17.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7ed9d700 1 --2- 172.21.15.64:0/4094651027 >> v2:172.21.15.123:3300/0 conn(0x7f7a800abce0 0x7f7a800c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 -- 172.21.15.64:0/4094651027 >> v2:172.21.15.123:3300/0 conn(0x7f7a800abce0 msgr2=0x7f7a800c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:17.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 --2- 172.21.15.64:0/4094651027 >> v2:172.21.15.123:3300/0 conn(0x7f7a800abce0 0x7f7a800c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 -- 172.21.15.64:0/4094651027 >> v1:172.21.15.123:6789/0 conn(0x7f7a800ab910 legacy=0x7f7a800cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 -- 172.21.15.64:0/4094651027 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8015a2b0 con 0x7f7a800670e0 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7dd9b700 1 --2- 172.21.15.64:0/4094651027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a80159710 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7a800c1880 tx=0x7f7a74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0852e5b0d46b6a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7d59a700 1 -- 172.21.15.64:0/4094651027 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74009e70 con 0x7f7a800670e0 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7d59a700 1 -- 172.21.15.64:0/4094651027 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7400b860 con 0x7f7a800670e0 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7d59a700 1 -- 172.21.15.64:0/4094651027 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7400bc00 con 0x7f7a800670e0 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- 172.21.15.64:0/4094651027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 msgr2=0x7f7a80159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/4094651027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a80159710 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7a800c1880 tx=0x7f7a74009250 comp rx=0 tx=0).stop 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- 172.21.15.64:0/4094651027 shutdown_connections 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/4094651027 >> v2:172.21.15.123:3300/0 conn(0x7f7a800abce0 0x7f7a800c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/4094651027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800670e0 0x7f7a80159710 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- 172.21.15.64:0/4094651027 >> 172.21.15.64:0/4094651027 conn(0x7f7a8006c080 msgr2=0x7f7a800ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:17.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- 172.21.15.64:0/4094651027 shutdown_connections 2023-09-27T23:33:17.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.827+0000 7f7a7f59e700 1 -- 172.21.15.64:0/4094651027 wait complete. 2023-09-27T23:33:17.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 Processor -- start 2023-09-27T23:33:17.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 -- start start 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800670e0 0x7f7a800b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a800c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 0x7f7a8014b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800b0820 con 0x7f7a800abce0 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8014a5f0 con 0x7f7a800ab910 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8014a8d0 con 0x7f7a800670e0 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 0x7f7a8014b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 0x7f7a8014b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45064/0 (socket says 172.21.15.64:45064) 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 -- 172.21.15.64:0/426230131 learned_addr learned my addr 172.21.15.64:0/426230131 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:17.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7e59c700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800670e0 0x7f7a800b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7dd9b700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a800c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 -- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800670e0 msgr2=0x7f7a800b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800670e0 0x7f7a800b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 -- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 msgr2=0x7f7a800c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a800c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.831+0000 7f7a7ed9d700 1 -- 172.21.15.64:0/426230131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740094f0 con 0x7f7a800abce0 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a7e59c700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800670e0 0x7f7a800b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a7ed9d700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 0x7f7a8014b690 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e90 tx=0x7f7a7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800cc20 con 0x7f7a800abce0 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7800f040 con 0x7f7a800abce0 2023-09-27T23:33:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8014abb0 con 0x7f7a800abce0 2023-09-27T23:33:17.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78014480 con 0x7f7a800abce0 2023-09-27T23:33:17.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8014b170 con 0x7f7a800abce0 2023-09-27T23:33:17.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8014b390 con 0x7f7a800abce0 2023-09-27T23:33:17.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7a78014830 con 0x7f7a800abce0 2023-09-27T23:33:17.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a6e7fc700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060a70 0x7f7a60062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:17.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7a7807d920 con 0x7f7a800abce0 2023-09-27T23:33:17.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.835+0000 7f7a7e59c700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060a70 0x7f7a60062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:17.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.839+0000 7f7a7e59c700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060a70 0x7f7a60062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a70001780 tx=0x7f7a70009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:17.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:17.843+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a78051620 con 0x7f7a800abce0 2023-09-27T23:33:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7a8004be40 con 0x7f7a800abce0 2023-09-27T23:33:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a6e7fc700 1 -- 172.21.15.64:0/426230131 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7a78018090 con 0x7f7a800abce0 2023-09-27T23:33:18.014 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:18.014 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:33:18.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:18.014 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:33:18.014 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:33:18.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:18.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:18.015 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:33:18.015 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:33:18.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:18.015 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:33:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060a70 msgr2=0x7f7a60062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060a70 0x7f7a60062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a70001780 tx=0x7f7a70009040 comp rx=0 tx=0).stop 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 msgr2=0x7f7a8014b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 0x7f7a8014b690 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e90 tx=0x7f7a7800b040 comp rx=0 tx=0).stop 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 shutdown_connections 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a60060a70 0x7f7a60062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a800abce0 0x7f7a8014b690 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.011+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a800ab910 0x7f7a800c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.015+0000 7f7a7f59e700 1 --2- 172.21.15.64:0/426230131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a800670e0 0x7f7a800b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.015+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 >> 172.21.15.64:0/426230131 conn(0x7f7a8006c080 msgr2=0x7f7a801588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.015+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 shutdown_connections 2023-09-27T23:33:18.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.015+0000 7f7a7f59e700 1 -- 172.21.15.64:0/426230131 wait complete. 2023-09-27T23:33:18.028 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:33:18.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:33:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.139+0000 7ff51bfff700 1 Processor -- start 2023-09-27T23:33:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 -- start start 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c0addc0 0x7ff51c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c06da90 con 0x7ff51c0ad9f0 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c06a620 con 0x7ff51c15b140 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c06dd30 con 0x7ff51c0addc0 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51affd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff51c0ad9f0 0x7ff51c06ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37546/0 (socket says 172.21.15.64:37546) 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51affd700 1 -- 172.21.15.64:0/3834904094 learned_addr learned my addr 172.21.15.64:0/3834904094 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51a7fc700 1 --2- 172.21.15.64:0/3834904094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c0addc0 0x7ff51c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920274552 0 0) 0x7ff51c06da90 con 0x7ff51c0ad9f0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff508003570 con 0x7ff51c0ad9f0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289197920 0 0) 0x7ff51c06a620 con 0x7ff51c15b140 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff51c06da90 con 0x7ff51c15b140 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005421838 0 0) 0x7ff508003570 con 0x7ff51c0ad9f0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3834904094 >> v1:172.21.15.123:6789/0 conn(0x7ff51c15b140 legacy=0x7ff51c0b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3834904094 >> v1:172.21.15.64:6789/0 conn(0x7ff51c0ad9f0 legacy=0x7ff51c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3834904094 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c06a8c0 con 0x7ff51c0addc0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51a7fc700 1 --2- 172.21.15.64:0/3834904094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c0addc0 0x7ff51c0c8db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff504009910 tx=0x7ff5040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f066b9800ebac7a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c001470 con 0x7ff51c0addc0 2023-09-27T23:33:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff504011420 con 0x7ff51c0addc0 2023-09-27T23:33:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff519ffb700 1 -- 172.21.15.64:0/3834904094 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5040117a0 con 0x7ff51c0addc0 2023-09-27T23:33:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 -- 172.21.15.64:0/3834904094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c0addc0 msgr2=0x7ff51c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.143+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3834904094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c0addc0 0x7ff51c0c8db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff504009910 tx=0x7ff5040091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- 172.21.15.64:0/3834904094 shutdown_connections 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3834904094 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c0addc0 0x7ff51c0c8db0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- 172.21.15.64:0/3834904094 >> 172.21.15.64:0/3834904094 conn(0x7ff51c067bb0 msgr2=0x7ff51c0c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- 172.21.15.64:0/3834904094 shutdown_connections 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- 172.21.15.64:0/3834904094 wait complete. 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 Processor -- start 2023-09-27T23:33:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- start start 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff51c0ad9f0 0x7ff51c0a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 0x7ff51c0a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c15b140 0x7ff51c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0a8a60 con 0x7ff51c0addc0 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0a7e50 con 0x7ff51c0ad9f0 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0a8130 con 0x7ff51c15b140 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 0x7ff51c0a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 0x7ff51c0a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45076/0 (socket says 172.21.15.64:45076) 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3149624346 learned_addr learned my addr 172.21.15.64:0/3149624346 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51affd700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff51c0ad9f0 0x7ff51c0a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51b7fe700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c15b140 0x7ff51c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c15b140 msgr2=0x7ff51c1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51a7fc700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c15b140 0x7ff51c1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.147+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff51c0ad9f0 msgr2=0x7ff51c0a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff51a7fc700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff51c0ad9f0 0x7ff51c0a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff51a7fc700 1 -- 172.21.15.64:0/3149624346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504009420 con 0x7ff51c0addc0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff51a7fc700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 0x7ff51c0a9800 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff504009860 tx=0x7ff504016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50401caf0 con 0x7ff51c0addc0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c0a8410 con 0x7ff51c0addc0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50401cc50 con 0x7ff51c0addc0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50401a450 con 0x7ff51c0addc0 2023-09-27T23:33:18.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c0a7b20 con 0x7ff51c0addc0 2023-09-27T23:33:18.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.151+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c064040 con 0x7ff51c0addc0 2023-09-27T23:33:18.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.155+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff50401a840 con 0x7ff51c0addc0 2023-09-27T23:33:18.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.155+0000 7ff512ffd700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4fc060a70 0x7ff4fc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.155+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7ff50407e690 con 0x7ff51c0addc0 2023-09-27T23:33:18.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.155+0000 7ff51affd700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4fc060a70 0x7ff4fc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.155+0000 7ff51affd700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4fc060a70 0x7ff4fc062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff50c001ed0 tx=0x7ff50c000bc0 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:18.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.159+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff50404ea10 con 0x7ff51c0addc0 2023-09-27T23:33:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.323+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff51c0a8ba0 con 0x7ff51c0addc0 2023-09-27T23:33:18.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.391+0000 7ff512ffd700 1 -- 172.21.15.64:0/3149624346 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff504052030 con 0x7ff51c0addc0 2023-09-27T23:33:18.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4fc060a70 msgr2=0x7ff4fc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4fc060a70 0x7ff4fc062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff50c001ed0 tx=0x7ff50c000bc0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 msgr2=0x7ff51c0a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 0x7ff51c0a9800 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff504009860 tx=0x7ff504016000 comp rx=0 tx=0).stop 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 shutdown_connections 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff4fc060a70 0x7ff4fc062f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff51c15b140 0x7ff51c1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff51c0addc0 0x7ff51c0a9800 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3149624346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff51c0ad9f0 0x7ff51c0a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 >> 172.21.15.64:0/3149624346 conn(0x7ff51c067bb0 msgr2=0x7ff51c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 shutdown_connections 2023-09-27T23:33:18.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.403+0000 7ff51bfff700 1 -- 172.21.15.64:0/3149624346 wait complete. 2023-09-27T23:33:18.407 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:33:18.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:33:18.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.523+0000 7ffb46eb9700 1 Processor -- start 2023-09-27T23:33:18.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.523+0000 7ffb46eb9700 1 -- start start 2023-09-27T23:33:18.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb46eb9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb40158e20 0x7ffb400ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb46eb9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4006d5e0 con 0x7ffb400ca2c0 2023-09-27T23:33:18.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb46eb9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb400cbdf0 con 0x7ffb400cb970 2023-09-27T23:33:18.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb46eb9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb400cc0d0 con 0x7ffb40158e20 2023-09-27T23:33:18.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb45456700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffb400ca2c0 0x7ffb4006ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37550/0 (socket says 172.21.15.64:37550) 2023-09-27T23:33:18.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb45456700 1 -- 172.21.15.64:0/1867656806 learned_addr learned my addr 172.21.15.64:0/1867656806 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:18.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb44c55700 1 --2- 172.21.15.64:0/1867656806 >> v2:172.21.15.123:3300/0 conn(0x7ffb40158e20 0x7ffb400ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395074979 0 0) 0x7ffb4006d5e0 con 0x7ffb400ca2c0 2023-09-27T23:33:18.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb30003570 con 0x7ffb400ca2c0 2023-09-27T23:33:18.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513638924 0 0) 0x7ffb400cbdf0 con 0x7ffb400cb970 2023-09-27T23:33:18.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb4006d5e0 con 0x7ffb400cb970 2023-09-27T23:33:18.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3034072057 0 0) 0x7ffb30003570 con 0x7ffb400ca2c0 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 >> v2:172.21.15.123:3300/0 conn(0x7ffb40158e20 msgr2=0x7ffb400ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 --2- 172.21.15.64:0/1867656806 >> v2:172.21.15.123:3300/0 conn(0x7ffb40158e20 0x7ffb400ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 >> v1:172.21.15.123:6789/0 conn(0x7ffb400cb970 legacy=0x7ffb4006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400cc3b0 con 0x7ffb400ca2c0 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7ffb38001770 con 0x7ffb400ca2c0 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 >> v1:172.21.15.64:6789/0 conn(0x7ffb400ca2c0 legacy=0x7ffb4006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb30003c40 0x7ffb30004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb30004810 0x7ffb30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30005210 0x7ffb3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cc0d0 con 0x7ffb30004810 2023-09-27T23:33:18.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3000c1b0 con 0x7ffb30005210 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3000c370 con 0x7ffb30003c40 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb30004810 0x7ffb30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb45456700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30005210 0x7ffb3000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb44c55700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb30003c40 0x7ffb30004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 -- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb30003c40 msgr2=0x7ffb30004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb30003c40 0x7ffb30004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 -- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30005210 msgr2=0x7ffb3000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30005210 0x7ffb3000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 -- 172.21.15.64:0/1867656806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30003570 con 0x7ffb30004810 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb45456700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30005210 0x7ffb3000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb45456700 1 -- 172.21.15.64:0/1867656806 reap_dead start 2023-09-27T23:33:18.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3ffff700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb30004810 0x7ffb30004c80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ffb4006a570 tx=0x7ffb2c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd51ddac1e421cc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c00c5d0 con 0x7ffb30004810 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c00c730 con 0x7ffb30004810 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.527+0000 7ffb3f7fe700 1 -- 172.21.15.64:0/1867656806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c00c8c0 con 0x7ffb30004810 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- 172.21.15.64:0/1867656806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb30004810 msgr2=0x7ffb30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb30004810 0x7ffb30004c80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ffb4006a570 tx=0x7ffb2c007d30 comp rx=0 tx=0).stop 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- 172.21.15.64:0/1867656806 shutdown_connections 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/1867656806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb30004810 0x7ffb30004c80 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- 172.21.15.64:0/1867656806 >> 172.21.15.64:0/1867656806 conn(0x7ffb40067bb0 msgr2=0x7ffb400d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- 172.21.15.64:0/1867656806 shutdown_connections 2023-09-27T23:33:18.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- 172.21.15.64:0/1867656806 wait complete. 2023-09-27T23:33:18.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 Processor -- start 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- start start 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30004810 0x7ffb400c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb400c99f0 0x7ffb4014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 0x7ffb4015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400c8e40 con 0x7ffb4015b080 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4015ba10 con 0x7ffb30004810 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400c8230 con 0x7ffb400c99f0 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 0x7ffb4015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 0x7ffb4015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45082/0 (socket says 172.21.15.64:45082) 2023-09-27T23:33:18.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 -- 172.21.15.64:0/3918099986 learned_addr learned my addr 172.21.15.64:0/3918099986 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb44c55700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30004810 0x7ffb400c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb3ffff700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb400c99f0 0x7ffb4014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 -- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb400c99f0 msgr2=0x7ffb4014ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb400c99f0 0x7ffb4014ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 -- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30004810 msgr2=0x7ffb400c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30004810 0x7ffb400c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 -- 172.21.15.64:0/3918099986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400c8510 con 0x7ffb4015b080 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb44c55700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30004810 0x7ffb400c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb45456700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 0x7ffb4015b4d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ffb38002250 tx=0x7ffb38002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38007af0 con 0x7ffb4015b080 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb380133f0 con 0x7ffb4015b080 2023-09-27T23:33:18.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb400c87f0 con 0x7ffb4015b080 2023-09-27T23:33:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.531+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38013770 con 0x7ffb4015b080 2023-09-27T23:33:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.535+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb400c9e60 con 0x7ffb4015b080 2023-09-27T23:33:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.535+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40071120 con 0x7ffb4015b080 2023-09-27T23:33:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.535+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ffb38013ae0 con 0x7ffb4015b080 2023-09-27T23:33:18.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.535+0000 7ffb3cff9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb24060aa0 0x7ffb24062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.535+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5896+0+0 (secure 0 0 0) 0x7ffb3807d710 con 0x7ffb4015b080 2023-09-27T23:33:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.535+0000 7ffb44c55700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb24060aa0 0x7ffb24062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:18.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.539+0000 7ffb44c55700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb24060aa0 0x7ffb24062f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006dc0 tx=0x7ffb34008040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:18.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.543+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ffb380513b0 con 0x7ffb4015b080 2023-09-27T23:33:18.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:18.711+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ffb400c6d20 con 0x7ffb4015b080 2023-09-27T23:33:19.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.411+0000 7ffb3cff9700 1 -- 172.21.15.64:0/3918099986 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) v1 ==== 170+0+0 (secure 0 0 0) 0x7ffb38050f40 con 0x7ffb4015b080 2023-09-27T23:33:19.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb24060aa0 msgr2=0x7ffb24062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb24060aa0 0x7ffb24062f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb34006dc0 tx=0x7ffb34008040 comp rx=0 tx=0).stop 2023-09-27T23:33:19.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 msgr2=0x7ffb4015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 0x7ffb4015b4d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ffb38002250 tx=0x7ffb38002040 comp rx=0 tx=0).stop 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 shutdown_connections 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ffb24060aa0 0x7ffb24062f50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb4015b080 0x7ffb4015b4d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb400c99f0 0x7ffb4014ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 --2- 172.21.15.64:0/3918099986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb30004810 0x7ffb400c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 >> 172.21.15.64:0/3918099986 conn(0x7ffb40067bb0 msgr2=0x7ffb40155ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 shutdown_connections 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.435+0000 7ffb46eb9700 1 -- 172.21.15.64:0/3918099986 wait complete. 2023-09-27T23:33:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:33:19.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:33:19.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 Processor -- start 2023-09-27T23:33:19.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 -- start start 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28280c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f282806de00 0x7f282806b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282806d970 con 0x7f282806c7c0 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282806cd60 con 0x7f282806de00 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282c890700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282806d040 con 0x7f28280c69a0 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282759e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f282806c7c0 0x7f282815cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37564/0 (socket says 172.21.15.64:37564) 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282759e700 1 -- 172.21.15.64:0/3674567248 learned_addr learned my addr 172.21.15.64:0/3674567248 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:19.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f2826d9d700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28280c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f282659c700 1 --2- 172.21.15.64:0/3674567248 >> v2:172.21.15.123:3300/0 conn(0x7f282806de00 0x7f282806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941014068 0 0) 0x7f282806d970 con 0x7f282806c7c0 2023-09-27T23:33:19.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.551+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2810003570 con 0x7f282806c7c0 2023-09-27T23:33:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539570701 0 0) 0x7f2810003570 con 0x7f282806c7c0 2023-09-27T23:33:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 msgr2=0x7f28280c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28280c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 >> v2:172.21.15.123:3300/0 conn(0x7f282806de00 msgr2=0x7f282806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:19.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 --2- 172.21.15.64:0/3674567248 >> v2:172.21.15.123:3300/0 conn(0x7f282806de00 0x7f282806b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282806d320 con 0x7f282806c7c0 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f28200017f0 con 0x7f282806c7c0 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 >> v1:172.21.15.64:6789/0 conn(0x7f282806c7c0 legacy=0x7f282815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2810003a60 0x7f2810003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f2810004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282806cd60 con 0x7f2810004810 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282806d040 con 0x7f2810003a60 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281000ba60 con 0x7f28280c69a0 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f2810004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2826d9d700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2810003a60 0x7f2810003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282759e700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 -- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 msgr2=0x7f28100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 -- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2810003a60 msgr2=0x7f2810003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2810003a60 0x7f2810003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 -- 172.21.15.64:0/3674567248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810003570 con 0x7f2810004810 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282759e700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28280c69a0 0x7f28100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282659c700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f2810004c80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f281c009c30 tx=0x7f281c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6833e6feb10332 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c00e710 con 0x7f2810004810 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c00e870 con 0x7f2810004810 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f2825d9b700 1 -- 172.21.15.64:0/3674567248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c00ec30 con 0x7f2810004810 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282c890700 1 -- 172.21.15.64:0/3674567248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 msgr2=0x7f2810004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282c890700 1 --2- 172.21.15.64:0/3674567248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f2810004c80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f281c009c30 tx=0x7f281c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282759e700 1 -- 172.21.15.64:0/3674567248 reap_dead start 2023-09-27T23:33:19.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282c890700 1 -- 172.21.15.64:0/3674567248 shutdown_connections 2023-09-27T23:33:19.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282c890700 1 -- 172.21.15.64:0/3674567248 >> 172.21.15.64:0/3674567248 conn(0x7f2828067b50 msgr2=0x7f28280cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:19.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282c890700 1 -- 172.21.15.64:0/3674567248 shutdown_connections 2023-09-27T23:33:19.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.555+0000 7f282c890700 1 -- 172.21.15.64:0/3674567248 wait complete. 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 Processor -- start 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 -- start start 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f282815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f282815be00 0x7f28281499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2828149ee0 0x7f2828147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282814af70 con 0x7f2810004810 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282814a360 con 0x7f282815be00 2023-09-27T23:33:19.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282814a640 con 0x7f2828149ee0 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f282815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f282815b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45106/0 (socket says 172.21.15.64:45106) 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 -- 172.21.15.64:0/2807864913 learned_addr learned my addr 172.21.15.64:0/2807864913 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282659c700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f282815be00 0x7f28281499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282759e700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2828149ee0 0x7f2828147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 -- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2828149ee0 msgr2=0x7f2828147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2828149ee0 0x7f2828147350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 -- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f282815be00 msgr2=0x7f28281499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f282815be00 0x7f28281499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 -- 172.21.15.64:0/2807864913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282814a920 con 0x7f2810004810 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282659c700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f282815be00 0x7f28281499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f2826d9d700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f282815b8c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2818011990 tx=0x7f2818008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:19.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818015670 con 0x7f2810004810 2023-09-27T23:33:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28180157d0 con 0x7f2810004810 2023-09-27T23:33:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282814ac00 con 0x7f2810004810 2023-09-27T23:33:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818015b90 con 0x7f2810004810 2023-09-27T23:33:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.559+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282815a470 con 0x7f2810004810 2023-09-27T23:33:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.563+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282804be40 con 0x7f2810004810 2023-09-27T23:33:19.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.563+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2818013410 con 0x7f2810004810 2023-09-27T23:33:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.563+0000 7f280effd700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2814060b40 0x7f2814062ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.563+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f281807d520 con 0x7f2810004810 2023-09-27T23:33:19.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.563+0000 7f282659c700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2814060b40 0x7f2814062ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.567+0000 7f282659c700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2814060b40 0x7f2814062ff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f281c0091e0 tx=0x7f281c01a000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:19.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.571+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f281804d610 con 0x7f2810004810 2023-09-27T23:33:19.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.771+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f282815a470 con 0x7f2810004810 2023-09-27T23:33:19.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.787+0000 7f280effd700 1 -- 172.21.15.64:0/2807864913 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7f282815a470 con 0x7f2810004810 2023-09-27T23:33:19.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2814060b40 msgr2=0x7f2814062ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2814060b40 0x7f2814062ff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f281c0091e0 tx=0x7f281c01a000 comp rx=0 tx=0).stop 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 msgr2=0x7f282815b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f282815b8c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2818011990 tx=0x7f2818008040 comp rx=0 tx=0).stop 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 shutdown_connections 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2814060b40 0x7f2814062ff0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2828149ee0 0x7f2828147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f282815be00 0x7f28281499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 --2- 172.21.15.64:0/2807864913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2810004810 0x7f282815b8c0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 >> 172.21.15.64:0/2807864913 conn(0x7f2828067b50 msgr2=0x7f282806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 shutdown_connections 2023-09-27T23:33:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.799+0000 7f282c890700 1 -- 172.21.15.64:0/2807864913 wait complete. 2023-09-27T23:33:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 18 and data pool 19 2023-09-27T23:33:19.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:19.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.919+0000 7fc62ffff700 1 Processor -- start 2023-09-27T23:33:19.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.919+0000 7fc62ffff700 1 -- start start 2023-09-27T23:33:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc630155080 0x7fc6300b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6300afb70 con 0x7fc6300af120 2023-09-27T23:33:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6300afe50 con 0x7fc63006b480 2023-09-27T23:33:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6300b0130 con 0x7fc630155080 2023-09-27T23:33:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc627fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc6300af120 0x7fc63006ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37566/0 (socket says 172.21.15.64:37566) 2023-09-27T23:33:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc627fff700 1 -- 172.21.15.64:0/2745912453 learned_addr learned my addr 172.21.15.64:0/2745912453 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62effd700 1 --2- 172.21.15.64:0/2745912453 >> v2:172.21.15.123:3300/0 conn(0x7fc630155080 0x7fc6300b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62e7fc700 1 -- 172.21.15.64:0/2745912453 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588848425 0 0) 0x7fc6300afb70 con 0x7fc6300af120 2023-09-27T23:33:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62e7fc700 1 -- 172.21.15.64:0/2745912453 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc61c003570 con 0x7fc6300af120 2023-09-27T23:33:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62effd700 1 -- 172.21.15.64:0/2745912453 >> v1:172.21.15.123:6789/0 conn(0x7fc63006b480 legacy=0x7fc6300c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62effd700 1 -- 172.21.15.64:0/2745912453 >> v1:172.21.15.64:6789/0 conn(0x7fc6300af120 legacy=0x7fc63006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62effd700 1 -- 172.21.15.64:0/2745912453 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6300b0410 con 0x7fc630155080 2023-09-27T23:33:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62effd700 1 --2- 172.21.15.64:0/2745912453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300b0c10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc6200098e0 tx=0x7fc6200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59281f674de82c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62e7fc700 1 -- 172.21.15.64:0/2745912453 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62000b640 con 0x7fc630155080 2023-09-27T23:33:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62e7fc700 1 -- 172.21.15.64:0/2745912453 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62000b7a0 con 0x7fc630155080 2023-09-27T23:33:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62e7fc700 1 -- 172.21.15.64:0/2745912453 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62000bb40 con 0x7fc630155080 2023-09-27T23:33:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- 172.21.15.64:0/2745912453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 msgr2=0x7fc6300b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 --2- 172.21.15.64:0/2745912453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300b0c10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc6200098e0 tx=0x7fc6200091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- 172.21.15.64:0/2745912453 shutdown_connections 2023-09-27T23:33:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 --2- 172.21.15.64:0/2745912453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300b0c10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- 172.21.15.64:0/2745912453 >> 172.21.15.64:0/2745912453 conn(0x7fc630067bb0 msgr2=0x7fc6300add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- 172.21.15.64:0/2745912453 shutdown_connections 2023-09-27T23:33:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.923+0000 7fc62ffff700 1 -- 172.21.15.64:0/2745912453 wait complete. 2023-09-27T23:33:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 Processor -- start 2023-09-27T23:33:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 -- start start 2023-09-27T23:33:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 0x7fc630166700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6300af120 0x7fc6301677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc630168e60 con 0x7fc63006b480 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc630166c40 con 0x7fc630155080 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc630167d30 con 0x7fc6300af120 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 0x7fc630166700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 0x7fc630166700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45114/0 (socket says 172.21.15.64:45114) 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62effd700 1 -- 172.21.15.64:0/407902769 learned_addr learned my addr 172.21.15.64:0/407902769 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc627fff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6300af120 0x7fc6301677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.927+0000 7fc62f7fe700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62effd700 1 -- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6300af120 msgr2=0x7fc6301677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62effd700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6300af120 0x7fc6301677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62effd700 1 -- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 msgr2=0x7fc6300acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62effd700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62effd700 1 -- 172.21.15.64:0/407902769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc620009420 con 0x7fc63006b480 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62effd700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 0x7fc630166700 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc620009f80 tx=0x7fc620017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620000d10 con 0x7fc63006b480 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62000bb20 con 0x7fc63006b480 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc630167f70 con 0x7fc63006b480 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620011650 con 0x7fc63006b480 2023-09-27T23:33:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc630168530 con 0x7fc63006b480 2023-09-27T23:33:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63004be40 con 0x7fc63006b480 2023-09-27T23:33:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.931+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc620011a00 con 0x7fc63006b480 2023-09-27T23:33:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.935+0000 7fc626ffd700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc610060aa0 0x7fc610062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:19.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.935+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc62007e690 con 0x7fc63006b480 2023-09-27T23:33:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.935+0000 7fc627fff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc610060aa0 0x7fc610062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:19.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.935+0000 7fc627fff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc610060aa0 0x7fc610062f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc6300ae3c0 tx=0x7fc618009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:19.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:19.939+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc62004e6c0 con 0x7fc63006b480 2023-09-27T23:33:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.103+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6300b06f0 con 0x7fc63006b480 2023-09-27T23:33:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.103+0000 7fc626ffd700 1 -- 172.21.15.64:0/407902769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17098 (secure 0 0 0) 0x7fc620011cb0 con 0x7fc63006b480 2023-09-27T23:33:20.108 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:20.108 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":142,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:19.774756+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:18.367224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:18.752273+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc610060aa0 msgr2=0x7fc610062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc610060aa0 0x7fc610062f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc6300ae3c0 tx=0x7fc618009040 comp rx=0 tx=0).stop 2023-09-27T23:33:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 msgr2=0x7fc630166700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 0x7fc630166700 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc620009f80 tx=0x7fc620017000 comp rx=0 tx=0).stop 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 shutdown_connections 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc610060aa0 0x7fc610062f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc630155080 0x7fc6300acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6300af120 0x7fc6301677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 --2- 172.21.15.64:0/407902769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63006b480 0x7fc630166700 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 >> 172.21.15.64:0/407902769 conn(0x7fc630067bb0 msgr2=0x7fc6301532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 shutdown_connections 2023-09-27T23:33:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.107+0000 7fc62ffff700 1 -- 172.21.15.64:0/407902769 wait complete. 2023-09-27T23:33:20.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:20.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:33:20.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:33:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 Processor -- start 2023-09-27T23:33:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 -- start start 2023-09-27T23:33:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a1c155330 0x7f4a1c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a1c159aa0 0x7f4a1c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c06b0f0 con 0x7f4a1c155700 2023-09-27T23:33:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c159ea0 con 0x7f4a1c159aa0 2023-09-27T23:33:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a21fc5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c06b340 con 0x7f4a1c155330 2023-09-27T23:33:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a1c159aa0 0x7f4a1c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a1c159aa0 0x7f4a1c0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45122/0 (socket says 172.21.15.64:45122) 2023-09-27T23:33:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 -- 172.21.15.64:0/1914840075 learned_addr learned my addr 172.21.15.64:0/1914840075 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1b7fe700 1 --2- 172.21.15.64:0/1914840075 >> v2:172.21.15.123:3300/0 conn(0x7f4a1c155330 0x7f4a1c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 -- 172.21.15.64:0/1914840075 >> v2:172.21.15.123:3300/0 conn(0x7f4a1c155330 msgr2=0x7f4a1c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:20.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 --2- 172.21.15.64:0/1914840075 >> v2:172.21.15.123:3300/0 conn(0x7f4a1c155330 0x7f4a1c0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 -- 172.21.15.64:0/1914840075 >> v1:172.21.15.123:6789/0 conn(0x7f4a1c155700 legacy=0x7f4a1c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 -- 172.21.15.64:0/1914840075 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c06b620 con 0x7f4a1c159aa0 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1b7fe700 1 --2- 172.21.15.64:0/1914840075 >> v2:172.21.15.123:3300/0 conn(0x7f4a1c155330 0x7f4a1c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1bfff700 1 --2- 172.21.15.64:0/1914840075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c0cba60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c067480 tx=0x7f4a1400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be97bde6be1fe48 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1a7fc700 1 -- 172.21.15.64:0/1914840075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400dc50 con 0x7f4a1c159aa0 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1a7fc700 1 -- 172.21.15.64:0/1914840075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400ddb0 con 0x7f4a1c159aa0 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.227+0000 7f4a1a7fc700 1 -- 172.21.15.64:0/1914840075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140196d0 con 0x7f4a1c159aa0 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 -- 172.21.15.64:0/1914840075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c159aa0 msgr2=0x7f4a1c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/1914840075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c0cba60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c067480 tx=0x7f4a1400b4f0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 -- 172.21.15.64:0/1914840075 shutdown_connections 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/1914840075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c0cba60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/1914840075 >> v2:172.21.15.123:3300/0 conn(0x7f4a1c155330 0x7f4a1c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 -- 172.21.15.64:0/1914840075 >> 172.21.15.64:0/1914840075 conn(0x7f4a1c0c90f0 msgr2=0x7f4a1c0c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 -- 172.21.15.64:0/1914840075 shutdown_connections 2023-09-27T23:33:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 -- 172.21.15.64:0/1914840075 wait complete. 2023-09-27T23:33:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 Processor -- start 2023-09-27T23:33:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.231+0000 7f4a21fc5700 1 -- start start 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 0x7f4a1c0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c155700 0x7f4a1c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c169860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b0f80 con 0x7f4a1c155330 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0ae520 con 0x7f4a1c159aa0 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0ad910 con 0x7f4a1c155700 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 0x7f4a1c0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 0x7f4a1c0ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45132/0 (socket says 172.21.15.64:45132) 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 -- 172.21.15.64:0/668137893 learned_addr learned my addr 172.21.15.64:0/668137893 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1bfff700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1affd700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c155700 0x7f4a1c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 -- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c155700 msgr2=0x7f4a1c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c155700 0x7f4a1c0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 -- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c159aa0 msgr2=0x7f4a1c169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 -- 172.21.15.64:0/668137893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1400b790 con 0x7f4a1c155330 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a1b7fe700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 0x7f4a1c0ad2a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00d180 tx=0x7f4a0c0099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00dc40 con 0x7f4a1c155330 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c00f040 con 0x7f4a1c155330 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0adbf0 con 0x7f4a1c155330 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c015440 con 0x7f4a1c155330 2023-09-27T23:33:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0ae1b0 con 0x7f4a1c155330 2023-09-27T23:33:20.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.235+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c071120 con 0x7f4a1c155330 2023-09-27T23:33:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.239+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4a0c015830 con 0x7f4a1c155330 2023-09-27T23:33:20.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.239+0000 7f4a137fe700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49fc060a70 0x7f49fc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.239+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4a0c07d570 con 0x7f4a1c155330 2023-09-27T23:33:20.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.239+0000 7f4a1affd700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49fc060a70 0x7f49fc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.239+0000 7f4a1affd700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49fc060a70 0x7f49fc062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c158050 tx=0x7f4a04007ec0 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:20.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.247+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4a0c04cbb0 con 0x7f4a1c155330 2023-09-27T23:33:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.407+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a1c158e60 con 0x7f4a1c155330 2023-09-27T23:33:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.411+0000 7f4a137fe700 1 -- 172.21.15.64:0/668137893 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f4a0c04d4a0 con 0x7f4a1c155330 2023-09-27T23:33:20.415 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:20.415 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":142,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:19.774756+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:18.367224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:18.752273+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49fc060a70 msgr2=0x7f49fc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49fc060a70 0x7f49fc062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c158050 tx=0x7f4a04007ec0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 msgr2=0x7f4a1c0ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 0x7f4a1c0ad2a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00d180 tx=0x7f4a0c0099f0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 shutdown_connections 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49fc060a70 0x7f49fc062f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c159aa0 0x7f4a1c169860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c155700 0x7f4a1c0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 --2- 172.21.15.64:0/668137893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a1c155330 0x7f4a1c0ad2a0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 >> 172.21.15.64:0/668137893 conn(0x7f4a1c0c90f0 msgr2=0x7f4a1c0c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 shutdown_connections 2023-09-27T23:33:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.415+0000 7f4a21fc5700 1 -- 172.21.15.64:0/668137893 wait complete. 2023-09-27T23:33:20.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:33:20.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 Processor -- start 2023-09-27T23:33:20.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 -- start start 2023-09-27T23:33:20.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d500caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6d50159770 0x7f6d5006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5006c850 con 0x7f6d5006aeb0 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5006b470 con 0x7f6d50159770 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d558f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5006b750 con 0x7f6d50158c90 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6d5006aeb0 0x7f6d50159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37570/0 (socket says 172.21.15.64:37570) 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4ffff700 1 -- 172.21.15.64:0/60113213 learned_addr learned my addr 172.21.15.64:0/60113213 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d550f2700 1 --2- 172.21.15.64:0/60113213 >> v2:172.21.15.123:3300/0 conn(0x7f6d50159770 0x7f6d5006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d548f1700 1 --2- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d500caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4f7fe700 1 -- 172.21.15.64:0/60113213 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737109206 0 0) 0x7f6d5006c850 con 0x7f6d5006aeb0 2023-09-27T23:33:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4f7fe700 1 -- 172.21.15.64:0/60113213 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d40003570 con 0x7f6d5006aeb0 2023-09-27T23:33:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d550f2700 1 -- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 msgr2=0x7f6d500caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d550f2700 1 --2- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d500caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d550f2700 1 -- 172.21.15.64:0/60113213 >> v1:172.21.15.64:6789/0 conn(0x7f6d5006aeb0 legacy=0x7f6d50159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d550f2700 1 -- 172.21.15.64:0/60113213 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5006ba30 con 0x7f6d50159770 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d550f2700 1 --2- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d5006cce0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009e60 tx=0x7f6d48009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef108a9891f56c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4f7fe700 1 -- 172.21.15.64:0/60113213 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800e070 con 0x7f6d50159770 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4f7fe700 1 -- 172.21.15.64:0/60113213 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4800bab0 con 0x7f6d50159770 2023-09-27T23:33:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.595+0000 7f6d4f7fe700 1 -- 172.21.15.64:0/60113213 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800be30 con 0x7f6d50159770 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 msgr2=0x7f6d5006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 --2- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d5006cce0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009e60 tx=0x7f6d48009780 comp rx=0 tx=0).stop 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- 172.21.15.64:0/60113213 shutdown_connections 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 --2- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d5006cce0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 --2- 172.21.15.64:0/60113213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d500caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- 172.21.15.64:0/60113213 >> 172.21.15.64:0/60113213 conn(0x7f6d50067bb0 msgr2=0x7f6d500cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- 172.21.15.64:0/60113213 shutdown_connections 2023-09-27T23:33:20.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- 172.21.15.64:0/60113213 wait complete. 2023-09-27T23:33:20.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 Processor -- start 2023-09-27T23:33:20.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- start start 2023-09-27T23:33:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 0x7f6d50153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d501575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d501548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50169090 con 0x7f6d5006aeb0 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50157e50 con 0x7f6d50159770 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.599+0000 7f6d558f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50158130 con 0x7f6d50158c90 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 0x7f6d50153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 0x7f6d50153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45144/0 (socket says 172.21.15.64:45144) 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 -- 172.21.15.64:0/2455457995 learned_addr learned my addr 172.21.15.64:0/2455457995 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d4ffff700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d501575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d550f2700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d501548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 -- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 msgr2=0x7f6d501575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d501575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 -- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 msgr2=0x7f6d501548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d501548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 -- 172.21.15.64:0/2455457995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d48009a20 con 0x7f6d5006aeb0 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d548f1700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 0x7f6d50153a20 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400ec80 tx=0x7f6d440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400c850 con 0x7f6d5006aeb0 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4400c9b0 con 0x7f6d5006aeb0 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d50158410 con 0x7f6d5006aeb0 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400cd50 con 0x7f6d5006aeb0 2023-09-27T23:33:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d500b4520 con 0x7f6d5006aeb0 2023-09-27T23:33:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6d44013680 con 0x7f6d5006aeb0 2023-09-27T23:33:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.603+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5004be40 con 0x7f6d5006aeb0 2023-09-27T23:33:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.607+0000 7f6d4cff9700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d34060a70 0x7f6d34062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.607+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6d4407d330 con 0x7f6d5006aeb0 2023-09-27T23:33:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.607+0000 7f6d4ffff700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d34060a70 0x7f6d34062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:20.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.607+0000 7f6d4ffff700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d34060a70 0x7f6d34062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d5015f170 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:20.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.611+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6d4404d360 con 0x7f6d5006aeb0 2023-09-27T23:33:20.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:20.811+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6d5006bd10 con 0x7f6d5006aeb0 2023-09-27T23:33:21.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.799+0000 7f6d4cff9700 1 -- 172.21.15.64:0/2455457995 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6d44050980 con 0x7f6d5006aeb0 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d34060a70 msgr2=0x7f6d34062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d34060a70 0x7f6d34062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d5015f170 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 msgr2=0x7f6d50153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 0x7f6d50153a20 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400ec80 tx=0x7f6d440091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 shutdown_connections 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6d34060a70 0x7f6d34062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d50159770 0x7f6d501548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d50158c90 0x7f6d501575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 --2- 172.21.15.64:0/2455457995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d5006aeb0 0x7f6d50153a20 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 >> 172.21.15.64:0/2455457995 conn(0x7f6d50067bb0 msgr2=0x7f6d500caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:21.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 shutdown_connections 2023-09-27T23:33:21.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.803+0000 7f6d558f3700 1 -- 172.21.15.64:0/2455457995 wait complete. 2023-09-27T23:33:21.825 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:33:21.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:33:21.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.939+0000 7f6804946700 1 Processor -- start 2023-09-27T23:33:21.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.939+0000 7f6804946700 1 -- start start 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f680015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680015ae30 con 0x7f680006baa0 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000abf70 con 0x7f6800154780 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6800154b50 con 0x7f68000ae1c0 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67f659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f680006baa0 0x7f68000adab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37578/0 (socket says 172.21.15.64:37578) 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67f659c700 1 -- 172.21.15.64:0/1492860824 learned_addr learned my addr 172.21.15.64:0/1492860824 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:21.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67ff59e700 1 --2- 172.21.15.64:0/1492860824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f680015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67fe59c700 1 -- 172.21.15.64:0/1492860824 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292986436 0 0) 0x7f680015ae30 con 0x7f680006baa0 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67fe59c700 1 -- 172.21.15.64:0/1492860824 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec003570 con 0x7f680006baa0 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67fe59c700 1 -- 172.21.15.64:0/1492860824 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176998880 0 0) 0x7f68000abf70 con 0x7f6800154780 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67ff59e700 1 -- 172.21.15.64:0/1492860824 >> v1:172.21.15.123:6789/0 conn(0x7f6800154780 legacy=0x7f6800157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67ff59e700 1 -- 172.21.15.64:0/1492860824 >> v1:172.21.15.64:6789/0 conn(0x7f680006baa0 legacy=0x7f68000adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67ff59e700 1 -- 172.21.15.64:0/1492860824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000ac210 con 0x7f68000ae1c0 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67ff59e700 1 --2- 172.21.15.64:0/1492860824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f680015b2c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e90 tx=0x7f67f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d87a56352e5e1d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67fe59c700 1 -- 172.21.15.64:0/1492860824 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e070 con 0x7f68000ae1c0 2023-09-27T23:33:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67fe59c700 1 -- 172.21.15.64:0/1492860824 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800bc30 con 0x7f68000ae1c0 2023-09-27T23:33:21.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f67fe59c700 1 -- 172.21.15.64:0/1492860824 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8012430 con 0x7f68000ae1c0 2023-09-27T23:33:21.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- 172.21.15.64:0/1492860824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 msgr2=0x7f680015b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 --2- 172.21.15.64:0/1492860824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f680015b2c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e90 tx=0x7f67f8009780 comp rx=0 tx=0).stop 2023-09-27T23:33:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- 172.21.15.64:0/1492860824 shutdown_connections 2023-09-27T23:33:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 --2- 172.21.15.64:0/1492860824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f680015b2c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- 172.21.15.64:0/1492860824 >> 172.21.15.64:0/1492860824 conn(0x7f6800067b50 msgr2=0x7f68000cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- 172.21.15.64:0/1492860824 shutdown_connections 2023-09-27T23:33:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.943+0000 7f6804946700 1 -- 172.21.15.64:0/1492860824 wait complete. 2023-09-27T23:33:21.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.947+0000 7f6804946700 1 Processor -- start 2023-09-27T23:33:21.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.947+0000 7f6804946700 1 -- start start 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680006baa0 0x7f6800169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f68001594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 0x7f6800153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001599f0 con 0x7f6800154780 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68000c67c0 con 0x7f680006baa0 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68001538c0 con 0x7f68000ae1c0 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 0x7f6800153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67fed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680006baa0 0x7f6800169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 0x7f6800153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45150/0 (socket says 172.21.15.64:45150) 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 -- 172.21.15.64:0/3530563679 learned_addr learned my addr 172.21.15.64:0/3530563679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67f659c700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f68001594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 -- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 msgr2=0x7f68001594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f68001594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 -- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680006baa0 msgr2=0x7f6800169680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680006baa0 0x7f6800169680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 -- 172.21.15.64:0/3530563679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009a20 con 0x7f6800154780 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67ff59e700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 0x7f6800153380 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f67f800b6c0 tx=0x7f67f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8026480 con 0x7f6800154780 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f8026600 con 0x7f6800154780 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6800159bf0 con 0x7f6800154780 2023-09-27T23:33:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80269c0 con 0x7f6800154780 2023-09-27T23:33:21.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.951+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68000c6cf0 con 0x7f6800154780 2023-09-27T23:33:21.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.955+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680004be40 con 0x7f6800154780 2023-09-27T23:33:21.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.955+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f67f801c400 con 0x7f6800154780 2023-09-27T23:33:21.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.955+0000 7f67f77fe700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f67e0060a70 0x7f67e0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:21.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.955+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f67f807f2f0 con 0x7f6800154780 2023-09-27T23:33:21.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.955+0000 7f67fed9d700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f67e0060a70 0x7f67e0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:21.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.955+0000 7f67fed9d700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f67e0060a70 0x7f67e0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6800153b30 tx=0x7f67f0009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:21.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:21.959+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f67f8052ca0 con 0x7f6800154780 2023-09-27T23:33:22.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.155+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f68000cc1a0 con 0x7f6800154780 2023-09-27T23:33:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.803+0000 7f67f77fe700 1 -- 172.21.15.64:0/3530563679 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f67f8052830 con 0x7f6800154780 2023-09-27T23:33:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f67e0060a70 msgr2=0x7f67e0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f67e0060a70 0x7f67e0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6800153b30 tx=0x7f67f0009040 comp rx=0 tx=0).stop 2023-09-27T23:33:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 msgr2=0x7f6800153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 0x7f6800153380 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f67f800b6c0 tx=0x7f67f8019000 comp rx=0 tx=0).stop 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 shutdown_connections 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f67e0060a70 0x7f67e0062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800154780 0x7f6800153380 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68000ae1c0 0x7f68001594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 --2- 172.21.15.64:0/3530563679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680006baa0 0x7f6800169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 >> 172.21.15.64:0/3530563679 conn(0x7f6800067b50 msgr2=0x7f68000c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 shutdown_connections 2023-09-27T23:33:22.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.807+0000 7f6804946700 1 -- 172.21.15.64:0/3530563679 wait complete. 2023-09-27T23:33:22.828 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:33:22.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:22.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 Processor -- start 2023-09-27T23:33:22.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 -- start start 2023-09-27T23:33:22.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb898067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:22.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb898159340 0x7fb8980cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:22.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb898068c70 con 0x7fb898158f70 2023-09-27T23:33:22.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb898068060 con 0x7fb898159340 2023-09-27T23:33:22.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89fb26700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb898068340 con 0x7fb898069100 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb898159340 0x7fb8980cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb898159340 0x7fb8980cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45152/0 (socket says 172.21.15.64:45152) 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 -- 172.21.15.64:0/2620657905 learned_addr learned my addr 172.21.15.64:0/2620657905 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89d0c1700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb898067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 -- 172.21.15.64:0/2620657905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 msgr2=0x7fb898067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb898067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89c8c0700 1 -- 172.21.15.64:0/2620657905 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615119645 0 0) 0x7fb898068c70 con 0x7fb898158f70 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 -- 172.21.15.64:0/2620657905 >> v1:172.21.15.123:6789/0 conn(0x7fb898158f70 legacy=0x7fb8980cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 -- 172.21.15.64:0/2620657905 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898068620 con 0x7fb898159340 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89d0c1700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb898067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:22.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89e0c3700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898159340 0x7fb8980cb3e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb8900099b0 tx=0x7fb890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703fe275a35b51d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89c8c0700 1 -- 172.21.15.64:0/2620657905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000b860 con 0x7fb898159340 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89c8c0700 1 -- 172.21.15.64:0/2620657905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000b9c0 con 0x7fb898159340 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.939+0000 7fb89c8c0700 1 -- 172.21.15.64:0/2620657905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000bd60 con 0x7fb898159340 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 -- 172.21.15.64:0/2620657905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898159340 msgr2=0x7fb8980cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898159340 0x7fb8980cb3e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb8900099b0 tx=0x7fb890009250 comp rx=0 tx=0).stop 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 -- 172.21.15.64:0/2620657905 shutdown_connections 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898159340 0x7fb8980cb3e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2620657905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb898067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 -- 172.21.15.64:0/2620657905 >> 172.21.15.64:0/2620657905 conn(0x7fb8980ccc40 msgr2=0x7fb8980cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 -- 172.21.15.64:0/2620657905 shutdown_connections 2023-09-27T23:33:22.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 -- 172.21.15.64:0/2620657905 wait complete. 2023-09-27T23:33:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.943+0000 7fb89fb26700 1 Processor -- start 2023-09-27T23:33:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 -- start start 2023-09-27T23:33:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb89814dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 0x7fb8980cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb898159340 0x7fb8980cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980b5f60 con 0x7fb898158f70 2023-09-27T23:33:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980b3e60 con 0x7fb898159340 2023-09-27T23:33:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980b55c0 con 0x7fb898069100 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 0x7fb8980cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 0x7fb8980cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45158/0 (socket says 172.21.15.64:45158) 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 -- 172.21.15.64:0/2918059279 learned_addr learned my addr 172.21.15.64:0/2918059279 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89e0c3700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb898159340 0x7fb8980cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d8c2700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb89814dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 -- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 msgr2=0x7fb89814dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb89814dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 -- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb898159340 msgr2=0x7fb8980cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb898159340 0x7fb8980cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 -- 172.21.15.64:0/2918059279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900094f0 con 0x7fb898158f70 2023-09-27T23:33:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89d0c1700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 0x7fb8980cb3e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e8d0 tx=0x7fb880008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000ee70 con 0x7fb898158f70 2023-09-27T23:33:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000c5e0 con 0x7fb898158f70 2023-09-27T23:33:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8980b58a0 con 0x7fb898158f70 2023-09-27T23:33:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000c980 con 0x7fb898158f70 2023-09-27T23:33:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.947+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8980b5270 con 0x7fb898158f70 2023-09-27T23:33:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.951+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8980b60a0 con 0x7fb898158f70 2023-09-27T23:33:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.951+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb880013410 con 0x7fb898158f70 2023-09-27T23:33:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.951+0000 7fb895ffb700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb878060a70 0x7fb878062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.951+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb88007d100 con 0x7fb898158f70 2023-09-27T23:33:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.951+0000 7fb89d8c2700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb878060a70 0x7fb878062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.955+0000 7fb89d8c2700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb878060a70 0x7fb878062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb888001560 tx=0x7fb888009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:22.959+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb8980b60a0 con 0x7fb898158f70 2023-09-27T23:33:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.163+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8980b60a0 con 0x7fb898158f70 2023-09-27T23:33:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.163+0000 7fb895ffb700 1 -- 172.21.15.64:0/2918059279 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb8980b60a0 con 0x7fb898158f70 2023-09-27T23:33:23.168 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:23.168 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:19.775714+0000","modified":"2023-09-27T23:33:22.797158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2286000796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2286000796},{"type":"v1","addr":"172.21.15.64:6835","nonce":2286000796}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-09-27T23:33:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb878060a70 msgr2=0x7fb878062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb878060a70 0x7fb878062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb888001560 tx=0x7fb888009040 comp rx=0 tx=0).stop 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 msgr2=0x7fb8980cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 0x7fb8980cb3e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e8d0 tx=0x7fb880008040 comp rx=0 tx=0).stop 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 shutdown_connections 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb878060a70 0x7fb878062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb898159340 0x7fb8980cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb898158f70 0x7fb8980cb3e0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 --2- 172.21.15.64:0/2918059279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb898069100 0x7fb89814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 >> 172.21.15.64:0/2918059279 conn(0x7fb8980ccc40 msgr2=0x7fb89815ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 shutdown_connections 2023-09-27T23:33:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.167+0000 7fb89fb26700 1 -- 172.21.15.64:0/2918059279 wait complete. 2023-09-27T23:33:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-09-27T23:33:23.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:23.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.287+0000 7fc466a0c700 1 Processor -- start 2023-09-27T23:33:23.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- start start 2023-09-27T23:33:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4601552f0 0x7fc4601556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc46006c7c0 0x7fc46006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46006d120 con 0x7fc4600ae520 2023-09-27T23:33:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4600c8710 con 0x7fc46006c7c0 2023-09-27T23:33:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600c1b90 con 0x7fc4601552f0 2023-09-27T23:33:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc4600ae520 0x7fc4600c8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37590/0 (socket says 172.21.15.64:37590) 2023-09-27T23:33:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45f7fe700 1 -- 172.21.15.64:0/1598512695 learned_addr learned my addr 172.21.15.64:0/1598512695 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc464fa9700 1 --2- 172.21.15.64:0/1598512695 >> v2:172.21.15.64:3300/0 conn(0x7fc46006c7c0 0x7fc46006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45ffff700 1 --2- 172.21.15.64:0/1598512695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4601552f0 0x7fc4601556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45effd700 1 -- 172.21.15.64:0/1598512695 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219849894 0 0) 0x7fc46006d120 con 0x7fc4600ae520 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45effd700 1 -- 172.21.15.64:0/1598512695 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc44c003570 con 0x7fc4600ae520 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc464fa9700 1 -- 172.21.15.64:0/1598512695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4601552f0 msgr2=0x7fc4601556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc464fa9700 1 --2- 172.21.15.64:0/1598512695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4601552f0 0x7fc4601556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc464fa9700 1 -- 172.21.15.64:0/1598512695 >> v1:172.21.15.64:6789/0 conn(0x7fc4600ae520 legacy=0x7fc4600c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc464fa9700 1 -- 172.21.15.64:0/1598512695 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46006a9d0 con 0x7fc46006c7c0 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc464fa9700 1 --2- 172.21.15.64:0/1598512695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46006c7c0 0x7fc46006cbe0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e60 tx=0x7fc458009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57e450c13d8e76b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45effd700 1 -- 172.21.15.64:0/1598512695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800e070 con 0x7fc46006c7c0 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45effd700 1 -- 172.21.15.64:0/1598512695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45800bab0 con 0x7fc46006c7c0 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc45effd700 1 -- 172.21.15.64:0/1598512695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800be70 con 0x7fc46006c7c0 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- 172.21.15.64:0/1598512695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46006c7c0 msgr2=0x7fc46006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 --2- 172.21.15.64:0/1598512695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46006c7c0 0x7fc46006cbe0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e60 tx=0x7fc458009780 comp rx=0 tx=0).stop 2023-09-27T23:33:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- 172.21.15.64:0/1598512695 shutdown_connections 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 --2- 172.21.15.64:0/1598512695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46006c7c0 0x7fc46006cbe0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 --2- 172.21.15.64:0/1598512695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4601552f0 0x7fc4601556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- 172.21.15.64:0/1598512695 >> 172.21.15.64:0/1598512695 conn(0x7fc4600c8850 msgr2=0x7fc4600c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- 172.21.15.64:0/1598512695 shutdown_connections 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.291+0000 7fc466a0c700 1 -- 172.21.15.64:0/1598512695 wait complete. 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 Processor -- start 2023-09-27T23:33:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 -- start start 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc46006c7c0 0x7fc460143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4600ae520 0x7fc460067e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 0x7fc460068360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc460169f00 con 0x7fc4601552f0 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600c3960 con 0x7fc4600ae520 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc460144140 con 0x7fc46006c7c0 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 0x7fc460068360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 0x7fc460068360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45166/0 (socket says 172.21.15.64:45166) 2023-09-27T23:33:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 -- 172.21.15.64:0/917222997 learned_addr learned my addr 172.21.15.64:0/917222997 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc45ffff700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc46006c7c0 0x7fc460143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc45f7fe700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4600ae520 0x7fc460067e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 -- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc46006c7c0 msgr2=0x7fc460143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc46006c7c0 0x7fc460143c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 -- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4600ae520 msgr2=0x7fc460067e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4600ae520 0x7fc460067e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 -- 172.21.15.64:0/917222997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458009a20 con 0x7fc4601552f0 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc464fa9700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 0x7fc460068360 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc45800b610 tx=0x7fc458019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458012dd0 con 0x7fc4601552f0 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc458027470 con 0x7fc4601552f0 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4600c3030 con 0x7fc4601552f0 2023-09-27T23:33:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.295+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458027830 con 0x7fc4601552f0 2023-09-27T23:33:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.299+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4600c35f0 con 0x7fc4601552f0 2023-09-27T23:33:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.299+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc460071120 con 0x7fc4601552f0 2023-09-27T23:33:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.299+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc458024070 con 0x7fc4601552f0 2023-09-27T23:33:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.299+0000 7fc457fff700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc440060a70 0x7fc440062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.299+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc45807fa00 con 0x7fc4601552f0 2023-09-27T23:33:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.299+0000 7fc45ffff700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc440060a70 0x7fc440062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.303+0000 7fc45ffff700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc440060a70 0x7fc440062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc450009db0 tx=0x7fc4500091a0 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.307+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc45804fae0 con 0x7fc4601552f0 2023-09-27T23:33:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.471+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4600c3aa0 con 0x7fc4601552f0 2023-09-27T23:33:23.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.471+0000 7fc457fff700 1 -- 172.21.15.64:0/917222997 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+17085 (secure 0 0 0) 0x7fc458053100 con 0x7fc4601552f0 2023-09-27T23:33:23.476 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:23.476 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":143,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:20.779985+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:18.367224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:18.752273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:23.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc440060a70 msgr2=0x7fc440062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc440060a70 0x7fc440062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc450009db0 tx=0x7fc4500091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 msgr2=0x7fc460068360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 0x7fc460068360 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc45800b610 tx=0x7fc458019000 comp rx=0 tx=0).stop 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 shutdown_connections 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc440060a70 0x7fc440062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4601552f0 0x7fc460068360 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4600ae520 0x7fc460067e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 --2- 172.21.15.64:0/917222997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc46006c7c0 0x7fc460143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 >> 172.21.15.64:0/917222997 conn(0x7fc4600c8850 msgr2=0x7fc4600670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 shutdown_connections 2023-09-27T23:33:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.475+0000 7fc466a0c700 1 -- 172.21.15.64:0/917222997 wait complete. 2023-09-27T23:33:23.494 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:33:23.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:23.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.595+0000 7feb63fff700 1 Processor -- start 2023-09-27T23:33:23.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.595+0000 7feb63fff700 1 -- start start 2023-09-27T23:33:23.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb640cad10 0x7feb640cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb6406df20 0x7feb6415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6406b2c0 con 0x7feb6406df20 2023-09-27T23:33:23.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6406a6b0 con 0x7feb640cad10 2023-09-27T23:33:23.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6406a990 con 0x7feb6406b750 2023-09-27T23:33:23.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb6406df20 0x7feb6415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb6406df20 0x7feb6415bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45168/0 (socket says 172.21.15.64:45168) 2023-09-27T23:33:23.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 -- 172.21.15.64:0/272902184 learned_addr learned my addr 172.21.15.64:0/272902184 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:23.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb62ffd700 1 --2- 172.21.15.64:0/272902184 >> v2:172.21.15.123:3300/0 conn(0x7feb640cad10 0x7feb640cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb627fc700 1 --2- 172.21.15.64:0/272902184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 -- 172.21.15.64:0/272902184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 msgr2=0x7feb6406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 --2- 172.21.15.64:0/272902184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6406d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 -- 172.21.15.64:0/272902184 >> v2:172.21.15.123:3300/0 conn(0x7feb640cad10 msgr2=0x7feb640cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 --2- 172.21.15.64:0/272902184 >> v2:172.21.15.123:3300/0 conn(0x7feb640cad10 0x7feb640cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 -- 172.21.15.64:0/272902184 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6406ac70 con 0x7feb6406df20 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb637fe700 1 --2- 172.21.15.64:0/272902184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6406df20 0x7feb6415bd00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7feb64069f90 tx=0x7feb5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bef0196c0f7b3c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb61ffb700 1 -- 172.21.15.64:0/272902184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c00e070 con 0x7feb6406df20 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb61ffb700 1 -- 172.21.15.64:0/272902184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c00b970 con 0x7feb6406df20 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb61ffb700 1 -- 172.21.15.64:0/272902184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0092e0 con 0x7feb6406df20 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- 172.21.15.64:0/272902184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6406df20 msgr2=0x7feb6415bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- 172.21.15.64:0/272902184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6406df20 0x7feb6415bd00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7feb64069f90 tx=0x7feb5c009840 comp rx=0 tx=0).stop 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- 172.21.15.64:0/272902184 shutdown_connections 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- 172.21.15.64:0/272902184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6406df20 0x7feb6415bd00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- 172.21.15.64:0/272902184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 --2- 172.21.15.64:0/272902184 >> v2:172.21.15.123:3300/0 conn(0x7feb640cad10 0x7feb640cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- 172.21.15.64:0/272902184 >> 172.21.15.64:0/272902184 conn(0x7feb64067bb0 msgr2=0x7feb640cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- 172.21.15.64:0/272902184 shutdown_connections 2023-09-27T23:33:23.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.599+0000 7feb63fff700 1 -- 172.21.15.64:0/272902184 wait complete. 2023-09-27T23:33:23.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.603+0000 7feb63fff700 1 Processor -- start 2023-09-27T23:33:23.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.603+0000 7feb63fff700 1 -- start start 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6406df20 0x7feb64169e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 0x7feb641586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb64156630 con 0x7feb640cad10 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6416a3a0 con 0x7feb6406df20 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb64158c30 con 0x7feb6406b750 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 0x7feb641586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 0x7feb641586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45180/0 (socket says 172.21.15.64:45180) 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 -- 172.21.15.64:0/4012788689 learned_addr learned my addr 172.21.15.64:0/4012788689 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:23.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb62ffd700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb627fc700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6406df20 0x7feb64169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 -- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 msgr2=0x7feb6415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 -- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6406df20 msgr2=0x7feb64169e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6406df20 0x7feb64169e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 -- 172.21.15.64:0/4012788689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c009ae0 con 0x7feb640cad10 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb627fc700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6406df20 0x7feb64169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb62ffd700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb637fe700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 0x7feb641586f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009880 tx=0x7feb5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c008bc0 con 0x7feb640cad10 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c008d40 con 0x7feb640cad10 2023-09-27T23:33:23.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6415b270 con 0x7feb640cad10 2023-09-27T23:33:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0266b0 con 0x7feb640cad10 2023-09-27T23:33:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb64155fe0 con 0x7feb640cad10 2023-09-27T23:33:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.607+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb641567e0 con 0x7feb640cad10 2023-09-27T23:33:23.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.611+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7feb5c003040 con 0x7feb640cad10 2023-09-27T23:33:23.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.611+0000 7feb4affd700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb50060a70 0x7feb50062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.611+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7feb5c07f0d0 con 0x7feb640cad10 2023-09-27T23:33:23.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.611+0000 7feb62ffd700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb50060a70 0x7feb50062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.611+0000 7feb62ffd700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb50060a70 0x7feb50062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb58009ce0 tx=0x7feb58008040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.615+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7feb641567e0 con 0x7feb640cad10 2023-09-27T23:33:23.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.775+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb6404be40 con 0x7feb50060a70 2023-09-27T23:33:23.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.795+0000 7feb4affd700 1 -- 172.21.15.64:0/4012788689 <== mgr.7358 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300604 (secure 0 0 0) 0x7feb6404be40 con 0x7feb50060a70 2023-09-27T23:33:23.802 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb50060a70 msgr2=0x7feb50062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb50060a70 0x7feb50062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb58009ce0 tx=0x7feb58008040 comp rx=0 tx=0).stop 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 msgr2=0x7feb641586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 0x7feb641586f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009880 tx=0x7feb5c018000 comp rx=0 tx=0).stop 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 shutdown_connections 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7feb50060a70 0x7feb50062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb640cad10 0x7feb641586f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6406df20 0x7feb64169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 --2- 172.21.15.64:0/4012788689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb6406b750 0x7feb6415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.799+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 >> 172.21.15.64:0/4012788689 conn(0x7feb64067bb0 msgr2=0x7feb6415bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.803+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 shutdown_connections 2023-09-27T23:33:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.803+0000 7feb63fff700 1 -- 172.21.15.64:0/4012788689 wait complete. 2023-09-27T23:33:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:23.818 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:22.682895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592016,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":429,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":118,"acting":118,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234468,"kb_used_data":14820,"kb_used_omap":0,"kb_used_meta":219584,"kb_avail":754740252,"statfs":{"total":773094113280,"available":772854018048,"internally_reserved":0,"allocated":15175680,"data_stored":8450576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224854016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.289603"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816721+0000","last_change":"2023-09-27T23:33:19.810193+0000","last_active":"2023-09-27T23:33:19.816721+0000","last_peered":"2023-09-27T23:33:19.816721+0000","last_clean":"2023-09-27T23:33:19.816721+0000","last_became_active":"2023-09-27T23:33:19.809935+0000","last_became_peered":"2023-09-27T23:33:19.809935+0000","last_unstale":"2023-09-27T23:33:19.816721+0000","last_undegraded":"2023-09-27T23:33:19.816721+0000","last_fullsized":"2023-09-27T23:33:19.816721+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:32:44.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816570+0000","last_change":"2023-09-27T23:33:19.811311+0000","last_active":"2023-09-27T23:33:19.816570+0000","last_peered":"2023-09-27T23:33:19.816570+0000","last_clean":"2023-09-27T23:33:19.816570+0000","last_became_active":"2023-09-27T23:33:19.809810+0000","last_became_peered":"2023-09-27T23:33:19.809810+0000","last_unstale":"2023-09-27T23:33:19.816570+0000","last_undegraded":"2023-09-27T23:33:19.816570+0000","last_fullsized":"2023-09-27T23:33:19.816570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:06.194398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825973+0000","last_change":"2023-09-27T23:33:19.817990+0000","last_active":"2023-09-27T23:33:19.825973+0000","last_peered":"2023-09-27T23:33:19.825973+0000","last_clean":"2023-09-27T23:33:19.825973+0000","last_became_active":"2023-09-27T23:33:19.809746+0000","last_became_peered":"2023-09-27T23:33:19.809746+0000","last_unstale":"2023-09-27T23:33:19.825973+0000","last_undegraded":"2023-09-27T23:33:19.825973+0000","last_fullsized":"2023-09-27T23:33:19.825973+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.812775+0000","last_change":"2023-09-27T23:33:19.801379+0000","last_active":"2023-09-27T23:33:19.812775+0000","last_peered":"2023-09-27T23:33:19.812775+0000","last_clean":"2023-09-27T23:33:19.812775+0000","last_became_active":"2023-09-27T23:33:19.800971+0000","last_became_peered":"2023-09-27T23:33:19.800971+0000","last_unstale":"2023-09-27T23:33:19.812775+0000","last_undegraded":"2023-09-27T23:33:19.812775+0000","last_fullsized":"2023-09-27T23:33:19.812775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:53.280332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826231+0000","last_change":"2023-09-27T23:33:19.819831+0000","last_active":"2023-09-27T23:33:19.826231+0000","last_peered":"2023-09-27T23:33:19.826231+0000","last_clean":"2023-09-27T23:33:19.826231+0000","last_became_active":"2023-09-27T23:33:19.819582+0000","last_became_peered":"2023-09-27T23:33:19.819582+0000","last_unstale":"2023-09-27T23:33:19.826231+0000","last_undegraded":"2023-09-27T23:33:19.826231+0000","last_fullsized":"2023-09-27T23:33:19.826231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:59.618721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826213+0000","last_change":"2023-09-27T23:33:19.818540+0000","last_active":"2023-09-27T23:33:19.826213+0000","last_peered":"2023-09-27T23:33:19.826213+0000","last_clean":"2023-09-27T23:33:19.826213+0000","last_became_active":"2023-09-27T23:33:19.818222+0000","last_became_peered":"2023-09-27T23:33:19.818222+0000","last_unstale":"2023-09-27T23:33:19.826213+0000","last_undegraded":"2023-09-27T23:33:19.826213+0000","last_fullsized":"2023-09-27T23:33:19.826213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813361+0000","last_change":"2023-09-27T23:33:19.806217+0000","last_active":"2023-09-27T23:33:19.813361+0000","last_peered":"2023-09-27T23:33:19.813361+0000","last_clean":"2023-09-27T23:33:19.813361+0000","last_became_active":"2023-09-27T23:33:19.805338+0000","last_became_peered":"2023-09-27T23:33:19.805338+0000","last_unstale":"2023-09-27T23:33:19.813361+0000","last_undegraded":"2023-09-27T23:33:19.813361+0000","last_fullsized":"2023-09-27T23:33:19.813361+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.311666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826156+0000","last_change":"2023-09-27T23:33:19.810568+0000","last_active":"2023-09-27T23:33:19.826156+0000","last_peered":"2023-09-27T23:33:19.826156+0000","last_clean":"2023-09-27T23:33:19.826156+0000","last_became_active":"2023-09-27T23:33:19.810237+0000","last_became_peered":"2023-09-27T23:33:19.810237+0000","last_unstale":"2023-09-27T23:33:19.826156+0000","last_undegraded":"2023-09-27T23:33:19.826156+0000","last_fullsized":"2023-09-27T23:33:19.826156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816810+0000","last_change":"2023-09-27T23:33:19.810731+0000","last_active":"2023-09-27T23:33:19.816810+0000","last_peered":"2023-09-27T23:33:19.816810+0000","last_clean":"2023-09-27T23:33:19.816810+0000","last_became_active":"2023-09-27T23:33:19.810477+0000","last_became_peered":"2023-09-27T23:33:19.810477+0000","last_unstale":"2023-09-27T23:33:19.816810+0000","last_undegraded":"2023-09-27T23:33:19.816810+0000","last_fullsized":"2023-09-27T23:33:19.816810+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:19:28.503800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809513+0000","last_change":"2023-09-27T23:33:19.805356+0000","last_active":"2023-09-27T23:33:19.809513+0000","last_peered":"2023-09-27T23:33:19.809513+0000","last_clean":"2023-09-27T23:33:19.809513+0000","last_became_active":"2023-09-27T23:33:19.805064+0000","last_became_peered":"2023-09-27T23:33:19.805064+0000","last_unstale":"2023-09-27T23:33:19.809513+0000","last_undegraded":"2023-09-27T23:33:19.809513+0000","last_fullsized":"2023-09-27T23:33:19.809513+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:37.422975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827932+0000","last_change":"2023-09-27T23:33:19.822764+0000","last_active":"2023-09-27T23:33:19.827932+0000","last_peered":"2023-09-27T23:33:19.827932+0000","last_clean":"2023-09-27T23:33:19.827932+0000","last_became_active":"2023-09-27T23:33:19.822456+0000","last_became_peered":"2023-09-27T23:33:19.822456+0000","last_unstale":"2023-09-27T23:33:19.827932+0000","last_undegraded":"2023-09-27T23:33:19.827932+0000","last_fullsized":"2023-09-27T23:33:19.827932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826283+0000","last_change":"2023-09-27T23:33:19.818051+0000","last_active":"2023-09-27T23:33:19.826283+0000","last_peered":"2023-09-27T23:33:19.826283+0000","last_clean":"2023-09-27T23:33:19.826283+0000","last_became_active":"2023-09-27T23:33:19.809795+0000","last_became_peered":"2023-09-27T23:33:19.809795+0000","last_unstale":"2023-09-27T23:33:19.826283+0000","last_undegraded":"2023-09-27T23:33:19.826283+0000","last_fullsized":"2023-09-27T23:33:19.826283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.804587+0000","last_change":"2023-09-27T23:33:19.798526+0000","last_active":"2023-09-27T23:33:19.804587+0000","last_peered":"2023-09-27T23:33:19.804587+0000","last_clean":"2023-09-27T23:33:19.804587+0000","last_became_active":"2023-09-27T23:33:19.798226+0000","last_became_peered":"2023-09-27T23:33:19.798226+0000","last_unstale":"2023-09-27T23:33:19.804587+0000","last_undegraded":"2023-09-27T23:33:19.804587+0000","last_fullsized":"2023-09-27T23:33:19.804587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:13:28.298592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815678+0000","last_change":"2023-09-27T23:33:19.805843+0000","last_active":"2023-09-27T23:33:19.815678+0000","last_peered":"2023-09-27T23:33:19.815678+0000","last_clean":"2023-09-27T23:33:19.815678+0000","last_became_active":"2023-09-27T23:33:19.805571+0000","last_became_peered":"2023-09-27T23:33:19.805571+0000","last_unstale":"2023-09-27T23:33:19.815678+0000","last_undegraded":"2023-09-27T23:33:19.815678+0000","last_fullsized":"2023-09-27T23:33:19.815678+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:49.075742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.823630+0000","last_change":"2023-09-27T23:33:19.819123+0000","last_active":"2023-09-27T23:33:19.823630+0000","last_peered":"2023-09-27T23:33:19.823630+0000","last_clean":"2023-09-27T23:33:19.823630+0000","last_became_active":"2023-09-27T23:33:19.818840+0000","last_became_peered":"2023-09-27T23:33:19.818840+0000","last_unstale":"2023-09-27T23:33:19.823630+0000","last_undegraded":"2023-09-27T23:33:19.823630+0000","last_fullsized":"2023-09-27T23:33:19.823630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:00.849910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827890+0000","last_change":"2023-09-27T23:33:19.822665+0000","last_active":"2023-09-27T23:33:19.827890+0000","last_peered":"2023-09-27T23:33:19.827890+0000","last_clean":"2023-09-27T23:33:19.827890+0000","last_became_active":"2023-09-27T23:33:19.822350+0000","last_became_peered":"2023-09-27T23:33:19.822350+0000","last_unstale":"2023-09-27T23:33:19.827890+0000","last_undegraded":"2023-09-27T23:33:19.827890+0000","last_fullsized":"2023-09-27T23:33:19.827890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:30:36.041799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827909+0000","last_change":"2023-09-27T23:33:19.822700+0000","last_active":"2023-09-27T23:33:19.827909+0000","last_peered":"2023-09-27T23:33:19.827909+0000","last_clean":"2023-09-27T23:33:19.827909+0000","last_became_active":"2023-09-27T23:33:19.822405+0000","last_became_peered":"2023-09-27T23:33:19.822405+0000","last_unstale":"2023-09-27T23:33:19.827909+0000","last_undegraded":"2023-09-27T23:33:19.827909+0000","last_fullsized":"2023-09-27T23:33:19.827909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:22.750767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825923+0000","last_change":"2023-09-27T23:33:19.810331+0000","last_active":"2023-09-27T23:33:19.825923+0000","last_peered":"2023-09-27T23:33:19.825923+0000","last_clean":"2023-09-27T23:33:19.825923+0000","last_became_active":"2023-09-27T23:33:19.810041+0000","last_became_peered":"2023-09-27T23:33:19.810041+0000","last_unstale":"2023-09-27T23:33:19.825923+0000","last_undegraded":"2023-09-27T23:33:19.825923+0000","last_fullsized":"2023-09-27T23:33:19.825923+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814781+0000","last_change":"2023-09-27T23:33:19.436486+0000","last_active":"2023-09-27T23:33:19.814781+0000","last_peered":"2023-09-27T23:33:19.814781+0000","last_clean":"2023-09-27T23:33:19.814781+0000","last_became_active":"2023-09-27T23:33:19.436056+0000","last_became_peered":"2023-09-27T23:33:19.436056+0000","last_unstale":"2023-09-27T23:33:19.814781+0000","last_undegraded":"2023-09-27T23:33:19.814781+0000","last_fullsized":"2023-09-27T23:33:19.814781+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809160+0000","last_change":"2023-09-27T23:33:19.427279+0000","last_active":"2023-09-27T23:33:19.809160+0000","last_peered":"2023-09-27T23:33:19.809160+0000","last_clean":"2023-09-27T23:33:19.809160+0000","last_became_active":"2023-09-27T23:33:19.426915+0000","last_became_peered":"2023-09-27T23:33:19.426915+0000","last_unstale":"2023-09-27T23:33:19.809160+0000","last_undegraded":"2023-09-27T23:33:19.809160+0000","last_fullsized":"2023-09-27T23:33:19.809160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:54.341042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816372+0000","last_change":"2023-09-27T23:33:19.433429+0000","last_active":"2023-09-27T23:33:19.816372+0000","last_peered":"2023-09-27T23:33:19.816372+0000","last_clean":"2023-09-27T23:33:19.816372+0000","last_became_active":"2023-09-27T23:33:19.433030+0000","last_became_peered":"2023-09-27T23:33:19.433030+0000","last_unstale":"2023-09-27T23:33:19.816372+0000","last_undegraded":"2023-09-27T23:33:19.816372+0000","last_fullsized":"2023-09-27T23:33:19.816372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:13:30.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826212+0000","last_change":"2023-09-27T23:33:19.819783+0000","last_active":"2023-09-27T23:33:19.826212+0000","last_peered":"2023-09-27T23:33:19.826212+0000","last_clean":"2023-09-27T23:33:19.826212+0000","last_became_active":"2023-09-27T23:33:19.819564+0000","last_became_peered":"2023-09-27T23:33:19.819564+0000","last_unstale":"2023-09-27T23:33:19.826212+0000","last_undegraded":"2023-09-27T23:33:19.826212+0000","last_fullsized":"2023-09-27T23:33:19.826212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:37.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813572+0000","last_change":"2023-09-27T23:33:19.809091+0000","last_active":"2023-09-27T23:33:19.813572+0000","last_peered":"2023-09-27T23:33:19.813572+0000","last_clean":"2023-09-27T23:33:19.813572+0000","last_became_active":"2023-09-27T23:33:19.808763+0000","last_became_peered":"2023-09-27T23:33:19.808763+0000","last_unstale":"2023-09-27T23:33:19.813572+0000","last_undegraded":"2023-09-27T23:33:19.813572+0000","last_fullsized":"2023-09-27T23:33:19.813572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:41.828993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788226+0000","last_change":"2023-09-27T23:33:19.428241+0000","last_active":"2023-09-27T23:33:19.788226+0000","last_peered":"2023-09-27T23:33:19.788226+0000","last_clean":"2023-09-27T23:33:19.788226+0000","last_became_active":"2023-09-27T23:33:19.427908+0000","last_became_peered":"2023-09-27T23:33:19.427908+0000","last_unstale":"2023-09-27T23:33:19.788226+0000","last_undegraded":"2023-09-27T23:33:19.788226+0000","last_fullsized":"2023-09-27T23:33:19.788226+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:49.753965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824232+0000","last_change":"2023-09-27T23:33:19.808815+0000","last_active":"2023-09-27T23:33:19.824232+0000","last_peered":"2023-09-27T23:33:19.824232+0000","last_clean":"2023-09-27T23:33:19.824232+0000","last_became_active":"2023-09-27T23:33:19.808532+0000","last_became_peered":"2023-09-27T23:33:19.808532+0000","last_unstale":"2023-09-27T23:33:19.824232+0000","last_undegraded":"2023-09-27T23:33:19.824232+0000","last_fullsized":"2023-09-27T23:33:19.824232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815149+0000","last_change":"2023-09-27T23:33:19.435202+0000","last_active":"2023-09-27T23:33:19.815149+0000","last_peered":"2023-09-27T23:33:19.815149+0000","last_clean":"2023-09-27T23:33:19.815149+0000","last_became_active":"2023-09-27T23:33:19.432656+0000","last_became_peered":"2023-09-27T23:33:19.432656+0000","last_unstale":"2023-09-27T23:33:19.815149+0000","last_undegraded":"2023-09-27T23:33:19.815149+0000","last_fullsized":"2023-09-27T23:33:19.815149+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817212+0000","last_change":"2023-09-27T23:33:19.811187+0000","last_active":"2023-09-27T23:33:19.817212+0000","last_peered":"2023-09-27T23:33:19.817212+0000","last_clean":"2023-09-27T23:33:19.817212+0000","last_became_active":"2023-09-27T23:33:19.810921+0000","last_became_peered":"2023-09-27T23:33:19.810921+0000","last_unstale":"2023-09-27T23:33:19.817212+0000","last_undegraded":"2023-09-27T23:33:19.817212+0000","last_fullsized":"2023-09-27T23:33:19.817212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:58.020074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819500+0000","last_change":"2023-09-27T23:33:19.816975+0000","last_active":"2023-09-27T23:33:19.819500+0000","last_peered":"2023-09-27T23:33:19.819500+0000","last_clean":"2023-09-27T23:33:19.819500+0000","last_became_active":"2023-09-27T23:33:19.816573+0000","last_became_peered":"2023-09-27T23:33:19.816573+0000","last_unstale":"2023-09-27T23:33:19.819500+0000","last_undegraded":"2023-09-27T23:33:19.819500+0000","last_fullsized":"2023-09-27T23:33:19.819500+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:46:14.483620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790958+0000","last_change":"2023-09-27T23:33:19.435177+0000","last_active":"2023-09-27T23:33:19.790958+0000","last_peered":"2023-09-27T23:33:19.790958+0000","last_clean":"2023-09-27T23:33:19.790958+0000","last_became_active":"2023-09-27T23:33:19.432614+0000","last_became_peered":"2023-09-27T23:33:19.432614+0000","last_unstale":"2023-09-27T23:33:19.790958+0000","last_undegraded":"2023-09-27T23:33:19.790958+0000","last_fullsized":"2023-09-27T23:33:19.790958+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:07.369858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816845+0000","last_change":"2023-09-27T23:33:19.810499+0000","last_active":"2023-09-27T23:33:19.816845+0000","last_peered":"2023-09-27T23:33:19.816845+0000","last_clean":"2023-09-27T23:33:19.816845+0000","last_became_active":"2023-09-27T23:33:19.810241+0000","last_became_peered":"2023-09-27T23:33:19.810241+0000","last_unstale":"2023-09-27T23:33:19.816845+0000","last_undegraded":"2023-09-27T23:33:19.816845+0000","last_fullsized":"2023-09-27T23:33:19.816845+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:52:01.755843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816627+0000","last_change":"2023-09-27T23:33:19.436255+0000","last_active":"2023-09-27T23:33:19.816627+0000","last_peered":"2023-09-27T23:33:19.816627+0000","last_clean":"2023-09-27T23:33:19.816627+0000","last_became_active":"2023-09-27T23:33:19.435903+0000","last_became_peered":"2023-09-27T23:33:19.435903+0000","last_unstale":"2023-09-27T23:33:19.816627+0000","last_undegraded":"2023-09-27T23:33:19.816627+0000","last_fullsized":"2023-09-27T23:33:19.816627+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:01.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813432+0000","last_change":"2023-09-27T23:33:19.809133+0000","last_active":"2023-09-27T23:33:19.813432+0000","last_peered":"2023-09-27T23:33:19.813432+0000","last_clean":"2023-09-27T23:33:19.813432+0000","last_became_active":"2023-09-27T23:33:19.806861+0000","last_became_peered":"2023-09-27T23:33:19.806861+0000","last_unstale":"2023-09-27T23:33:19.813432+0000","last_undegraded":"2023-09-27T23:33:19.813432+0000","last_fullsized":"2023-09-27T23:33:19.813432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:46:39.679217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815834+0000","last_change":"2023-09-27T23:33:19.434317+0000","last_active":"2023-09-27T23:33:19.815834+0000","last_peered":"2023-09-27T23:33:19.815834+0000","last_clean":"2023-09-27T23:33:19.815834+0000","last_became_active":"2023-09-27T23:33:19.434002+0000","last_became_peered":"2023-09-27T23:33:19.434002+0000","last_unstale":"2023-09-27T23:33:19.815834+0000","last_undegraded":"2023-09-27T23:33:19.815834+0000","last_fullsized":"2023-09-27T23:33:19.815834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827942+0000","last_change":"2023-09-27T23:33:19.822884+0000","last_active":"2023-09-27T23:33:19.827942+0000","last_peered":"2023-09-27T23:33:19.827942+0000","last_clean":"2023-09-27T23:33:19.827942+0000","last_became_active":"2023-09-27T23:33:19.822472+0000","last_became_peered":"2023-09-27T23:33:19.822472+0000","last_unstale":"2023-09-27T23:33:19.827942+0000","last_undegraded":"2023-09-27T23:33:19.827942+0000","last_fullsized":"2023-09-27T23:33:19.827942+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:52:33.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809717+0000","last_change":"2023-09-27T23:33:19.804614+0000","last_active":"2023-09-27T23:33:19.809717+0000","last_peered":"2023-09-27T23:33:19.809717+0000","last_clean":"2023-09-27T23:33:19.809717+0000","last_became_active":"2023-09-27T23:33:19.804328+0000","last_became_peered":"2023-09-27T23:33:19.804328+0000","last_unstale":"2023-09-27T23:33:19.809717+0000","last_undegraded":"2023-09-27T23:33:19.809717+0000","last_fullsized":"2023-09-27T23:33:19.809717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:06.058928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788038+0000","last_change":"2023-09-27T23:33:19.422966+0000","last_active":"2023-09-27T23:33:19.788038+0000","last_peered":"2023-09-27T23:33:19.788038+0000","last_clean":"2023-09-27T23:33:19.788038+0000","last_became_active":"2023-09-27T23:33:19.422649+0000","last_became_peered":"2023-09-27T23:33:19.422649+0000","last_unstale":"2023-09-27T23:33:19.788038+0000","last_undegraded":"2023-09-27T23:33:19.788038+0000","last_fullsized":"2023-09-27T23:33:19.788038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:04:02.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.786802+0000","last_change":"2023-09-27T23:33:19.424649+0000","last_active":"2023-09-27T23:33:19.786802+0000","last_peered":"2023-09-27T23:33:19.786802+0000","last_clean":"2023-09-27T23:33:19.786802+0000","last_became_active":"2023-09-27T23:33:19.424230+0000","last_became_peered":"2023-09-27T23:33:19.424230+0000","last_unstale":"2023-09-27T23:33:19.786802+0000","last_undegraded":"2023-09-27T23:33:19.786802+0000","last_fullsized":"2023-09-27T23:33:19.786802+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:46:54.150917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814735+0000","last_change":"2023-09-27T23:33:19.807080+0000","last_active":"2023-09-27T23:33:19.814735+0000","last_peered":"2023-09-27T23:33:19.814735+0000","last_clean":"2023-09-27T23:33:19.814735+0000","last_became_active":"2023-09-27T23:33:19.806755+0000","last_became_peered":"2023-09-27T23:33:19.806755+0000","last_unstale":"2023-09-27T23:33:19.814735+0000","last_undegraded":"2023-09-27T23:33:19.814735+0000","last_fullsized":"2023-09-27T23:33:19.814735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:03:57.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806466+0000","last_change":"2023-09-27T23:33:19.804496+0000","last_active":"2023-09-27T23:33:19.806466+0000","last_peered":"2023-09-27T23:33:19.806466+0000","last_clean":"2023-09-27T23:33:19.806466+0000","last_became_active":"2023-09-27T23:33:19.804186+0000","last_became_peered":"2023-09-27T23:33:19.804186+0000","last_unstale":"2023-09-27T23:33:19.806466+0000","last_undegraded":"2023-09-27T23:33:19.806466+0000","last_fullsized":"2023-09-27T23:33:19.806466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:59:09.351560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"131'249","reported_seq":736,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257356+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:15.257356+0000","last_peered":"2023-09-27T23:33:15.257356+0000","last_clean":"2023-09-27T23:33:15.257356+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:15.257356+0000","last_undegraded":"2023-09-27T23:33:15.257356+0000","last_fullsized":"2023-09-27T23:33:15.257356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":249,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790633+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:19.790633+0000","last_peered":"2023-09-27T23:33:19.790633+0000","last_clean":"2023-09-27T23:33:19.790633+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:19.790633+0000","last_undegraded":"2023-09-27T23:33:19.790633+0000","last_fullsized":"2023-09-27T23:33:19.790633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826305+0000","last_change":"2023-09-27T23:33:19.818550+0000","last_active":"2023-09-27T23:33:19.826305+0000","last_peered":"2023-09-27T23:33:19.826305+0000","last_clean":"2023-09-27T23:33:19.826305+0000","last_became_active":"2023-09-27T23:33:19.818313+0000","last_became_peered":"2023-09-27T23:33:19.818313+0000","last_unstale":"2023-09-27T23:33:19.826305+0000","last_undegraded":"2023-09-27T23:33:19.826305+0000","last_fullsized":"2023-09-27T23:33:19.826305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825230+0000","last_change":"2023-09-27T23:33:19.439141+0000","last_active":"2023-09-27T23:33:19.825230+0000","last_peered":"2023-09-27T23:33:19.825230+0000","last_clean":"2023-09-27T23:33:19.825230+0000","last_became_active":"2023-09-27T23:33:19.438828+0000","last_became_peered":"2023-09-27T23:33:19.438828+0000","last_unstale":"2023-09-27T23:33:19.825230+0000","last_undegraded":"2023-09-27T23:33:19.825230+0000","last_fullsized":"2023-09-27T23:33:19.825230+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:33.463327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257347+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:15.257347+0000","last_peered":"2023-09-27T23:33:15.257347+0000","last_clean":"2023-09-27T23:33:15.257347+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:15.257347+0000","last_undegraded":"2023-09-27T23:33:15.257347+0000","last_fullsized":"2023-09-27T23:33:15.257347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816597+0000","last_change":"2023-09-27T23:33:19.812801+0000","last_active":"2023-09-27T23:33:19.816597+0000","last_peered":"2023-09-27T23:33:19.816597+0000","last_clean":"2023-09-27T23:33:19.816597+0000","last_became_active":"2023-09-27T23:33:19.812506+0000","last_became_peered":"2023-09-27T23:33:19.812506+0000","last_unstale":"2023-09-27T23:33:19.816597+0000","last_undegraded":"2023-09-27T23:33:19.816597+0000","last_fullsized":"2023-09-27T23:33:19.816597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:16:22.761586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788828+0000","last_change":"2023-09-27T23:33:19.424648+0000","last_active":"2023-09-27T23:33:19.788828+0000","last_peered":"2023-09-27T23:33:19.788828+0000","last_clean":"2023-09-27T23:33:19.788828+0000","last_became_active":"2023-09-27T23:33:19.424271+0000","last_became_peered":"2023-09-27T23:33:19.424271+0000","last_unstale":"2023-09-27T23:33:19.788828+0000","last_undegraded":"2023-09-27T23:33:19.788828+0000","last_fullsized":"2023-09-27T23:33:19.788828+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:25.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798546+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:19.798546+0000","last_peered":"2023-09-27T23:33:19.798546+0000","last_clean":"2023-09-27T23:33:19.798546+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:19.798546+0000","last_undegraded":"2023-09-27T23:33:19.798546+0000","last_fullsized":"2023-09-27T23:33:19.798546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817238+0000","last_change":"2023-09-27T23:33:19.813654+0000","last_active":"2023-09-27T23:33:19.817238+0000","last_peered":"2023-09-27T23:33:19.817238+0000","last_clean":"2023-09-27T23:33:19.817238+0000","last_became_active":"2023-09-27T23:33:19.813363+0000","last_became_peered":"2023-09-27T23:33:19.813363+0000","last_unstale":"2023-09-27T23:33:19.817238+0000","last_undegraded":"2023-09-27T23:33:19.817238+0000","last_fullsized":"2023-09-27T23:33:19.817238+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:09.337371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798813+0000","last_change":"2023-09-27T23:33:19.432418+0000","last_active":"2023-09-27T23:33:19.798813+0000","last_peered":"2023-09-27T23:33:19.798813+0000","last_clean":"2023-09-27T23:33:19.798813+0000","last_became_active":"2023-09-27T23:33:19.428641+0000","last_became_peered":"2023-09-27T23:33:19.428641+0000","last_unstale":"2023-09-27T23:33:19.798813+0000","last_undegraded":"2023-09-27T23:33:19.798813+0000","last_fullsized":"2023-09-27T23:33:19.798813+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790099+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:19.790099+0000","last_peered":"2023-09-27T23:33:19.790099+0000","last_clean":"2023-09-27T23:33:19.790099+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:19.790099+0000","last_undegraded":"2023-09-27T23:33:19.790099+0000","last_fullsized":"2023-09-27T23:33:19.790099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828563+0000","last_change":"2023-09-27T23:33:19.430328+0000","last_active":"2023-09-27T23:33:19.828563+0000","last_peered":"2023-09-27T23:33:19.828563+0000","last_clean":"2023-09-27T23:33:19.828563+0000","last_became_active":"2023-09-27T23:33:19.430033+0000","last_became_peered":"2023-09-27T23:33:19.430033+0000","last_unstale":"2023-09-27T23:33:19.828563+0000","last_undegraded":"2023-09-27T23:33:19.828563+0000","last_fullsized":"2023-09-27T23:33:19.828563+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:48.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789643+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:19.789643+0000","last_peered":"2023-09-27T23:33:19.789643+0000","last_clean":"2023-09-27T23:33:19.789643+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:19.789643+0000","last_undegraded":"2023-09-27T23:33:19.789643+0000","last_fullsized":"2023-09-27T23:33:19.789643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822697+0000","last_change":"2023-09-27T23:33:19.438806+0000","last_active":"2023-09-27T23:33:19.822697+0000","last_peered":"2023-09-27T23:33:19.822697+0000","last_clean":"2023-09-27T23:33:19.822697+0000","last_became_active":"2023-09-27T23:33:19.438541+0000","last_became_peered":"2023-09-27T23:33:19.438541+0000","last_unstale":"2023-09-27T23:33:19.822697+0000","last_undegraded":"2023-09-27T23:33:19.822697+0000","last_fullsized":"2023-09-27T23:33:19.822697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:29.683418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257370+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:15.257370+0000","last_peered":"2023-09-27T23:33:15.257370+0000","last_clean":"2023-09-27T23:33:15.257370+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:15.257370+0000","last_undegraded":"2023-09-27T23:33:15.257370+0000","last_fullsized":"2023-09-27T23:33:15.257370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817658+0000","last_change":"2023-09-27T23:33:19.815111+0000","last_active":"2023-09-27T23:33:19.817658+0000","last_peered":"2023-09-27T23:33:19.817658+0000","last_clean":"2023-09-27T23:33:19.817658+0000","last_became_active":"2023-09-27T23:33:19.814866+0000","last_became_peered":"2023-09-27T23:33:19.814866+0000","last_unstale":"2023-09-27T23:33:19.817658+0000","last_undegraded":"2023-09-27T23:33:19.817658+0000","last_fullsized":"2023-09-27T23:33:19.817658+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:30.614318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788544+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:19.788544+0000","last_peered":"2023-09-27T23:33:19.788544+0000","last_clean":"2023-09-27T23:33:19.788544+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:19.788544+0000","last_undegraded":"2023-09-27T23:33:19.788544+0000","last_fullsized":"2023-09-27T23:33:19.788544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816760+0000","last_change":"2023-09-27T23:33:19.807831+0000","last_active":"2023-09-27T23:33:19.816760+0000","last_peered":"2023-09-27T23:33:19.816760+0000","last_clean":"2023-09-27T23:33:19.816760+0000","last_became_active":"2023-09-27T23:33:19.807647+0000","last_became_peered":"2023-09-27T23:33:19.807647+0000","last_unstale":"2023-09-27T23:33:19.816760+0000","last_undegraded":"2023-09-27T23:33:19.816760+0000","last_fullsized":"2023-09-27T23:33:19.816760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:34.467334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792775+0000","last_change":"2023-09-27T23:33:19.438063+0000","last_active":"2023-09-27T23:33:19.792775+0000","last_peered":"2023-09-27T23:33:19.792775+0000","last_clean":"2023-09-27T23:33:19.792775+0000","last_became_active":"2023-09-27T23:33:19.437740+0000","last_became_peered":"2023-09-27T23:33:19.437740+0000","last_unstale":"2023-09-27T23:33:19.792775+0000","last_undegraded":"2023-09-27T23:33:19.792775+0000","last_fullsized":"2023-09-27T23:33:19.792775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:01.021179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796971+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:19.796971+0000","last_peered":"2023-09-27T23:33:19.796971+0000","last_clean":"2023-09-27T23:33:19.796971+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:19.796971+0000","last_undegraded":"2023-09-27T23:33:19.796971+0000","last_fullsized":"2023-09-27T23:33:19.796971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509676,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":2314240,"data_stored":1484913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":1.397},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":1.486}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":2.6850000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.0469999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":2.5680000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":2.7589999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.5600000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.704},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":2.8250000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26368,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1781760,"data_stored":923892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27000832},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":1.155},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":1.018},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.345},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.4079999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509676,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37776,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94334064,"statfs":{"total":96636764160,"available":96598081536,"internally_reserved":0,"allocated":1777664,"data_stored":922468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":1.242}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.995},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.4950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509676,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":23320,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":21632,"kb_avail":94348520,"statfs":{"total":96636764160,"available":96612884480,"internally_reserved":0,"allocated":1720320,"data_stored":894545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22151168},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":1.004}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.03},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.624},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.139},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.1160000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.087}]}]},{"osd":3,"up_from":15,"seq":64424509675,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1720320,"data_stored":894545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67000000000000004,"15min":0.69399999999999995},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.218},"max":{"1min":0.82199999999999995,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.441,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1779999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.1419999999999999,"15min":1.2869999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.40000000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.266,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.04,"5min":1.323,"15min":1.3440000000000001},"last":0.878},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.40500000000000003},"max":{"1min":1.093,"5min":1.446,"15min":1.5720000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":0.96599999999999997},"min":{"1min":0.68899999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2430000000000001,"5min":1.613,"15min":1.613},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.073,"15min":1.0760000000000001},"min":{"1min":0.76200000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.4430000000000001,"5min":1.605,"15min":1.605},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0960000000000001,"15min":1.1140000000000001},"min":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.66000000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.571},"last":1.29},{"interface":"front","average":{"1min":1.109,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.69999999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.431,"5min":1.714,"15min":1.764},"last":1.0940000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.1160000000000001},"min":{"1min":0.753,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.353,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.91600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5509999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.23}]},{"osd":7,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2090000000000001,"15min":1.2310000000000001},"min":{"1min":0.84099999999999997,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.4390000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.018},{"interface":"front","average":{"1min":1.145,"5min":1.165,"15min":1.196},"min":{"1min":0.72899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3680000000000001,"5min":1.837,"15min":1.837},"last":1.194}]}]},{"osd":2,"up_from":15,"seq":64424509677,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22672,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":94349168,"statfs":{"total":96636764160,"available":96613548032,"internally_reserved":0,"allocated":1777664,"data_stored":922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.498},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.95999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.01},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37896,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":36160,"kb_avail":94333944,"statfs":{"total":96636764160,"available":96597958656,"internally_reserved":0,"allocated":1769472,"data_stored":922356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37027840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":1.026}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.171},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.302},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.119}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.27},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2470000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509677,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28892,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94342948,"statfs":{"total":96636764160,"available":96607178752,"internally_reserved":0,"allocated":2314240,"data_stored":1484913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.81000000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.002},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.095}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:23.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:23.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 Processor -- start 2023-09-27T23:33:23.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 -- start start 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4db406a580 0x7f4db406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4db406d210 0x7f4db406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db406c7c0 con 0x7f4db40b1410 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db406af40 con 0x7f4db406d210 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dbad5d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db406cd80 con 0x7f4db406a580 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dba55c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4db406d210 0x7f4db406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dba55c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4db406d210 0x7f4db406d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45184/0 (socket says 172.21.15.64:45184) 2023-09-27T23:33:23.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4dba55c700 1 -- 172.21.15.64:0/2216093329 learned_addr learned my addr 172.21.15.64:0/2216093329 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:23.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.927+0000 7f4db955a700 1 --2- 172.21.15.64:0/2216093329 >> v2:172.21.15.123:3300/0 conn(0x7f4db406a580 0x7f4db406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dba55c700 1 -- 172.21.15.64:0/2216093329 >> v2:172.21.15.123:3300/0 conn(0x7f4db406a580 msgr2=0x7f4db406aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dba55c700 1 --2- 172.21.15.64:0/2216093329 >> v2:172.21.15.123:3300/0 conn(0x7f4db406a580 0x7f4db406aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dba55c700 1 -- 172.21.15.64:0/2216093329 >> v1:172.21.15.123:6789/0 conn(0x7f4db40b1410 legacy=0x7f4db40ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dba55c700 1 -- 172.21.15.64:0/2216093329 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db406b8f0 con 0x7f4db406d210 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dba55c700 1 --2- 172.21.15.64:0/2216093329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db406d210 0x7f4db406d620 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f4db4068040 tx=0x7f4dac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d08af61d64444f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4db8d59700 1 -- 172.21.15.64:0/2216093329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00e070 con 0x7f4db406d210 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4db8d59700 1 -- 172.21.15.64:0/2216093329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac00ba20 con 0x7f4db406d210 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4db8d59700 1 -- 172.21.15.64:0/2216093329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0092e0 con 0x7f4db406d210 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 -- 172.21.15.64:0/2216093329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db406d210 msgr2=0x7f4db406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/2216093329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db406d210 0x7f4db406d620 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f4db4068040 tx=0x7f4dac009840 comp rx=0 tx=0).stop 2023-09-27T23:33:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 -- 172.21.15.64:0/2216093329 shutdown_connections 2023-09-27T23:33:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/2216093329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db406d210 0x7f4db406d620 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/2216093329 >> v2:172.21.15.123:3300/0 conn(0x7f4db406a580 0x7f4db406aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 -- 172.21.15.64:0/2216093329 >> 172.21.15.64:0/2216093329 conn(0x7f4db4067bb0 msgr2=0x7f4db40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 -- 172.21.15.64:0/2216093329 shutdown_connections 2023-09-27T23:33:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.931+0000 7f4dbad5d700 1 -- 172.21.15.64:0/2216093329 wait complete. 2023-09-27T23:33:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 Processor -- start 2023-09-27T23:33:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 -- start start 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 0x7f4db4168b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db406d210 0x7f4db40c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 0x7f4db41495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4169d30 con 0x7f4db40b1410 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40c49b0 con 0x7f4db406d210 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dbad5d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4169120 con 0x7f4db406a580 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dba55c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 0x7f4db41495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4db9d5b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 0x7f4db4168b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4db955a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db406d210 0x7f4db40c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dba55c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 0x7f4db41495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45190/0 (socket says 172.21.15.64:45190) 2023-09-27T23:33:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4db9d5b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 0x7f4db4168b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38948/0 (socket says 172.21.15.64:38948) 2023-09-27T23:33:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.935+0000 7f4dba55c700 1 -- 172.21.15.64:0/3449917039 learned_addr learned my addr 172.21.15.64:0/3449917039 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dba55c700 1 -- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 msgr2=0x7f4db4168b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dba55c700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 0x7f4db4168b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dba55c700 1 -- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db406d210 msgr2=0x7f4db40c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dba55c700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db406d210 0x7f4db40c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dba55c700 1 -- 172.21.15.64:0/3449917039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009ae0 con 0x7f4db40b1410 2023-09-27T23:33:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4db955a700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db406d210 0x7f4db40c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4db9d5b700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 0x7f4db4168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dba55c700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 0x7f4db41495b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00b8c0 tx=0x7f4dac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac008b00 con 0x7f4db40b1410 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac008c60 con 0x7f4db40b1410 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac026460 con 0x7f4db40b1410 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db4169400 con 0x7f4db40b1410 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db41699c0 con 0x7f4db40b1410 2023-09-27T23:33:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db4169ee0 con 0x7f4db40b1410 2023-09-27T23:33:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.939+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4dac003040 con 0x7f4db40b1410 2023-09-27T23:33:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.943+0000 7f4da5ffb700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d98060aa0 0x7f4d98062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.943+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4dac07f120 con 0x7f4db40b1410 2023-09-27T23:33:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.943+0000 7f4db9d5b700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d98060aa0 0x7f4d98062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.943+0000 7f4db9d5b700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d98060aa0 0x7f4d98062f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4db0000c00 tx=0x7f4db0009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:23.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:23.951+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4db4169ee0 con 0x7f4db40b1410 2023-09-27T23:33:24.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.107+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4db40cc140 con 0x7f4d98060aa0 2023-09-27T23:33:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.127+0000 7f4da5ffb700 1 -- 172.21.15.64:0/3449917039 <== mgr.7358 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300604 (secure 0 0 0) 0x7f4db40cc140 con 0x7f4d98060aa0 2023-09-27T23:33:24.135 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d98060aa0 msgr2=0x7f4d98062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d98060aa0 0x7f4d98062f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4db0000c00 tx=0x7f4db0009040 comp rx=0 tx=0).stop 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 msgr2=0x7f4db41495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 0x7f4db41495b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00b8c0 tx=0x7f4dac018000 comp rx=0 tx=0).stop 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 shutdown_connections 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4d98060aa0 0x7f4d98062f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db40b1410 0x7f4db41495b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db406d210 0x7f4db40c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 --2- 172.21.15.64:0/3449917039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4db406a580 0x7f4db4168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 >> 172.21.15.64:0/3449917039 conn(0x7f4db4067bb0 msgr2=0x7f4db406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 shutdown_connections 2023-09-27T23:33:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.135+0000 7f4dbad5d700 1 -- 172.21.15.64:0/3449917039 wait complete. 2023-09-27T23:33:24.139 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:24.152 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:22.682895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592016,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":429,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":118,"acting":118,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234468,"kb_used_data":14820,"kb_used_omap":0,"kb_used_meta":219584,"kb_avail":754740252,"statfs":{"total":773094113280,"available":772854018048,"internally_reserved":0,"allocated":15175680,"data_stored":8450576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224854016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.289603"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816721+0000","last_change":"2023-09-27T23:33:19.810193+0000","last_active":"2023-09-27T23:33:19.816721+0000","last_peered":"2023-09-27T23:33:19.816721+0000","last_clean":"2023-09-27T23:33:19.816721+0000","last_became_active":"2023-09-27T23:33:19.809935+0000","last_became_peered":"2023-09-27T23:33:19.809935+0000","last_unstale":"2023-09-27T23:33:19.816721+0000","last_undegraded":"2023-09-27T23:33:19.816721+0000","last_fullsized":"2023-09-27T23:33:19.816721+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:32:44.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816570+0000","last_change":"2023-09-27T23:33:19.811311+0000","last_active":"2023-09-27T23:33:19.816570+0000","last_peered":"2023-09-27T23:33:19.816570+0000","last_clean":"2023-09-27T23:33:19.816570+0000","last_became_active":"2023-09-27T23:33:19.809810+0000","last_became_peered":"2023-09-27T23:33:19.809810+0000","last_unstale":"2023-09-27T23:33:19.816570+0000","last_undegraded":"2023-09-27T23:33:19.816570+0000","last_fullsized":"2023-09-27T23:33:19.816570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:06.194398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825973+0000","last_change":"2023-09-27T23:33:19.817990+0000","last_active":"2023-09-27T23:33:19.825973+0000","last_peered":"2023-09-27T23:33:19.825973+0000","last_clean":"2023-09-27T23:33:19.825973+0000","last_became_active":"2023-09-27T23:33:19.809746+0000","last_became_peered":"2023-09-27T23:33:19.809746+0000","last_unstale":"2023-09-27T23:33:19.825973+0000","last_undegraded":"2023-09-27T23:33:19.825973+0000","last_fullsized":"2023-09-27T23:33:19.825973+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.812775+0000","last_change":"2023-09-27T23:33:19.801379+0000","last_active":"2023-09-27T23:33:19.812775+0000","last_peered":"2023-09-27T23:33:19.812775+0000","last_clean":"2023-09-27T23:33:19.812775+0000","last_became_active":"2023-09-27T23:33:19.800971+0000","last_became_peered":"2023-09-27T23:33:19.800971+0000","last_unstale":"2023-09-27T23:33:19.812775+0000","last_undegraded":"2023-09-27T23:33:19.812775+0000","last_fullsized":"2023-09-27T23:33:19.812775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:53.280332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826231+0000","last_change":"2023-09-27T23:33:19.819831+0000","last_active":"2023-09-27T23:33:19.826231+0000","last_peered":"2023-09-27T23:33:19.826231+0000","last_clean":"2023-09-27T23:33:19.826231+0000","last_became_active":"2023-09-27T23:33:19.819582+0000","last_became_peered":"2023-09-27T23:33:19.819582+0000","last_unstale":"2023-09-27T23:33:19.826231+0000","last_undegraded":"2023-09-27T23:33:19.826231+0000","last_fullsized":"2023-09-27T23:33:19.826231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:59.618721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826213+0000","last_change":"2023-09-27T23:33:19.818540+0000","last_active":"2023-09-27T23:33:19.826213+0000","last_peered":"2023-09-27T23:33:19.826213+0000","last_clean":"2023-09-27T23:33:19.826213+0000","last_became_active":"2023-09-27T23:33:19.818222+0000","last_became_peered":"2023-09-27T23:33:19.818222+0000","last_unstale":"2023-09-27T23:33:19.826213+0000","last_undegraded":"2023-09-27T23:33:19.826213+0000","last_fullsized":"2023-09-27T23:33:19.826213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813361+0000","last_change":"2023-09-27T23:33:19.806217+0000","last_active":"2023-09-27T23:33:19.813361+0000","last_peered":"2023-09-27T23:33:19.813361+0000","last_clean":"2023-09-27T23:33:19.813361+0000","last_became_active":"2023-09-27T23:33:19.805338+0000","last_became_peered":"2023-09-27T23:33:19.805338+0000","last_unstale":"2023-09-27T23:33:19.813361+0000","last_undegraded":"2023-09-27T23:33:19.813361+0000","last_fullsized":"2023-09-27T23:33:19.813361+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.311666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826156+0000","last_change":"2023-09-27T23:33:19.810568+0000","last_active":"2023-09-27T23:33:19.826156+0000","last_peered":"2023-09-27T23:33:19.826156+0000","last_clean":"2023-09-27T23:33:19.826156+0000","last_became_active":"2023-09-27T23:33:19.810237+0000","last_became_peered":"2023-09-27T23:33:19.810237+0000","last_unstale":"2023-09-27T23:33:19.826156+0000","last_undegraded":"2023-09-27T23:33:19.826156+0000","last_fullsized":"2023-09-27T23:33:19.826156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816810+0000","last_change":"2023-09-27T23:33:19.810731+0000","last_active":"2023-09-27T23:33:19.816810+0000","last_peered":"2023-09-27T23:33:19.816810+0000","last_clean":"2023-09-27T23:33:19.816810+0000","last_became_active":"2023-09-27T23:33:19.810477+0000","last_became_peered":"2023-09-27T23:33:19.810477+0000","last_unstale":"2023-09-27T23:33:19.816810+0000","last_undegraded":"2023-09-27T23:33:19.816810+0000","last_fullsized":"2023-09-27T23:33:19.816810+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:19:28.503800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809513+0000","last_change":"2023-09-27T23:33:19.805356+0000","last_active":"2023-09-27T23:33:19.809513+0000","last_peered":"2023-09-27T23:33:19.809513+0000","last_clean":"2023-09-27T23:33:19.809513+0000","last_became_active":"2023-09-27T23:33:19.805064+0000","last_became_peered":"2023-09-27T23:33:19.805064+0000","last_unstale":"2023-09-27T23:33:19.809513+0000","last_undegraded":"2023-09-27T23:33:19.809513+0000","last_fullsized":"2023-09-27T23:33:19.809513+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:37.422975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827932+0000","last_change":"2023-09-27T23:33:19.822764+0000","last_active":"2023-09-27T23:33:19.827932+0000","last_peered":"2023-09-27T23:33:19.827932+0000","last_clean":"2023-09-27T23:33:19.827932+0000","last_became_active":"2023-09-27T23:33:19.822456+0000","last_became_peered":"2023-09-27T23:33:19.822456+0000","last_unstale":"2023-09-27T23:33:19.827932+0000","last_undegraded":"2023-09-27T23:33:19.827932+0000","last_fullsized":"2023-09-27T23:33:19.827932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826283+0000","last_change":"2023-09-27T23:33:19.818051+0000","last_active":"2023-09-27T23:33:19.826283+0000","last_peered":"2023-09-27T23:33:19.826283+0000","last_clean":"2023-09-27T23:33:19.826283+0000","last_became_active":"2023-09-27T23:33:19.809795+0000","last_became_peered":"2023-09-27T23:33:19.809795+0000","last_unstale":"2023-09-27T23:33:19.826283+0000","last_undegraded":"2023-09-27T23:33:19.826283+0000","last_fullsized":"2023-09-27T23:33:19.826283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.804587+0000","last_change":"2023-09-27T23:33:19.798526+0000","last_active":"2023-09-27T23:33:19.804587+0000","last_peered":"2023-09-27T23:33:19.804587+0000","last_clean":"2023-09-27T23:33:19.804587+0000","last_became_active":"2023-09-27T23:33:19.798226+0000","last_became_peered":"2023-09-27T23:33:19.798226+0000","last_unstale":"2023-09-27T23:33:19.804587+0000","last_undegraded":"2023-09-27T23:33:19.804587+0000","last_fullsized":"2023-09-27T23:33:19.804587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:13:28.298592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815678+0000","last_change":"2023-09-27T23:33:19.805843+0000","last_active":"2023-09-27T23:33:19.815678+0000","last_peered":"2023-09-27T23:33:19.815678+0000","last_clean":"2023-09-27T23:33:19.815678+0000","last_became_active":"2023-09-27T23:33:19.805571+0000","last_became_peered":"2023-09-27T23:33:19.805571+0000","last_unstale":"2023-09-27T23:33:19.815678+0000","last_undegraded":"2023-09-27T23:33:19.815678+0000","last_fullsized":"2023-09-27T23:33:19.815678+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:49.075742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.823630+0000","last_change":"2023-09-27T23:33:19.819123+0000","last_active":"2023-09-27T23:33:19.823630+0000","last_peered":"2023-09-27T23:33:19.823630+0000","last_clean":"2023-09-27T23:33:19.823630+0000","last_became_active":"2023-09-27T23:33:19.818840+0000","last_became_peered":"2023-09-27T23:33:19.818840+0000","last_unstale":"2023-09-27T23:33:19.823630+0000","last_undegraded":"2023-09-27T23:33:19.823630+0000","last_fullsized":"2023-09-27T23:33:19.823630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:00.849910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827890+0000","last_change":"2023-09-27T23:33:19.822665+0000","last_active":"2023-09-27T23:33:19.827890+0000","last_peered":"2023-09-27T23:33:19.827890+0000","last_clean":"2023-09-27T23:33:19.827890+0000","last_became_active":"2023-09-27T23:33:19.822350+0000","last_became_peered":"2023-09-27T23:33:19.822350+0000","last_unstale":"2023-09-27T23:33:19.827890+0000","last_undegraded":"2023-09-27T23:33:19.827890+0000","last_fullsized":"2023-09-27T23:33:19.827890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:30:36.041799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827909+0000","last_change":"2023-09-27T23:33:19.822700+0000","last_active":"2023-09-27T23:33:19.827909+0000","last_peered":"2023-09-27T23:33:19.827909+0000","last_clean":"2023-09-27T23:33:19.827909+0000","last_became_active":"2023-09-27T23:33:19.822405+0000","last_became_peered":"2023-09-27T23:33:19.822405+0000","last_unstale":"2023-09-27T23:33:19.827909+0000","last_undegraded":"2023-09-27T23:33:19.827909+0000","last_fullsized":"2023-09-27T23:33:19.827909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:22.750767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825923+0000","last_change":"2023-09-27T23:33:19.810331+0000","last_active":"2023-09-27T23:33:19.825923+0000","last_peered":"2023-09-27T23:33:19.825923+0000","last_clean":"2023-09-27T23:33:19.825923+0000","last_became_active":"2023-09-27T23:33:19.810041+0000","last_became_peered":"2023-09-27T23:33:19.810041+0000","last_unstale":"2023-09-27T23:33:19.825923+0000","last_undegraded":"2023-09-27T23:33:19.825923+0000","last_fullsized":"2023-09-27T23:33:19.825923+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814781+0000","last_change":"2023-09-27T23:33:19.436486+0000","last_active":"2023-09-27T23:33:19.814781+0000","last_peered":"2023-09-27T23:33:19.814781+0000","last_clean":"2023-09-27T23:33:19.814781+0000","last_became_active":"2023-09-27T23:33:19.436056+0000","last_became_peered":"2023-09-27T23:33:19.436056+0000","last_unstale":"2023-09-27T23:33:19.814781+0000","last_undegraded":"2023-09-27T23:33:19.814781+0000","last_fullsized":"2023-09-27T23:33:19.814781+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809160+0000","last_change":"2023-09-27T23:33:19.427279+0000","last_active":"2023-09-27T23:33:19.809160+0000","last_peered":"2023-09-27T23:33:19.809160+0000","last_clean":"2023-09-27T23:33:19.809160+0000","last_became_active":"2023-09-27T23:33:19.426915+0000","last_became_peered":"2023-09-27T23:33:19.426915+0000","last_unstale":"2023-09-27T23:33:19.809160+0000","last_undegraded":"2023-09-27T23:33:19.809160+0000","last_fullsized":"2023-09-27T23:33:19.809160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:54.341042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816372+0000","last_change":"2023-09-27T23:33:19.433429+0000","last_active":"2023-09-27T23:33:19.816372+0000","last_peered":"2023-09-27T23:33:19.816372+0000","last_clean":"2023-09-27T23:33:19.816372+0000","last_became_active":"2023-09-27T23:33:19.433030+0000","last_became_peered":"2023-09-27T23:33:19.433030+0000","last_unstale":"2023-09-27T23:33:19.816372+0000","last_undegraded":"2023-09-27T23:33:19.816372+0000","last_fullsized":"2023-09-27T23:33:19.816372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:13:30.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826212+0000","last_change":"2023-09-27T23:33:19.819783+0000","last_active":"2023-09-27T23:33:19.826212+0000","last_peered":"2023-09-27T23:33:19.826212+0000","last_clean":"2023-09-27T23:33:19.826212+0000","last_became_active":"2023-09-27T23:33:19.819564+0000","last_became_peered":"2023-09-27T23:33:19.819564+0000","last_unstale":"2023-09-27T23:33:19.826212+0000","last_undegraded":"2023-09-27T23:33:19.826212+0000","last_fullsized":"2023-09-27T23:33:19.826212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:37.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813572+0000","last_change":"2023-09-27T23:33:19.809091+0000","last_active":"2023-09-27T23:33:19.813572+0000","last_peered":"2023-09-27T23:33:19.813572+0000","last_clean":"2023-09-27T23:33:19.813572+0000","last_became_active":"2023-09-27T23:33:19.808763+0000","last_became_peered":"2023-09-27T23:33:19.808763+0000","last_unstale":"2023-09-27T23:33:19.813572+0000","last_undegraded":"2023-09-27T23:33:19.813572+0000","last_fullsized":"2023-09-27T23:33:19.813572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:41.828993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788226+0000","last_change":"2023-09-27T23:33:19.428241+0000","last_active":"2023-09-27T23:33:19.788226+0000","last_peered":"2023-09-27T23:33:19.788226+0000","last_clean":"2023-09-27T23:33:19.788226+0000","last_became_active":"2023-09-27T23:33:19.427908+0000","last_became_peered":"2023-09-27T23:33:19.427908+0000","last_unstale":"2023-09-27T23:33:19.788226+0000","last_undegraded":"2023-09-27T23:33:19.788226+0000","last_fullsized":"2023-09-27T23:33:19.788226+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:49.753965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824232+0000","last_change":"2023-09-27T23:33:19.808815+0000","last_active":"2023-09-27T23:33:19.824232+0000","last_peered":"2023-09-27T23:33:19.824232+0000","last_clean":"2023-09-27T23:33:19.824232+0000","last_became_active":"2023-09-27T23:33:19.808532+0000","last_became_peered":"2023-09-27T23:33:19.808532+0000","last_unstale":"2023-09-27T23:33:19.824232+0000","last_undegraded":"2023-09-27T23:33:19.824232+0000","last_fullsized":"2023-09-27T23:33:19.824232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815149+0000","last_change":"2023-09-27T23:33:19.435202+0000","last_active":"2023-09-27T23:33:19.815149+0000","last_peered":"2023-09-27T23:33:19.815149+0000","last_clean":"2023-09-27T23:33:19.815149+0000","last_became_active":"2023-09-27T23:33:19.432656+0000","last_became_peered":"2023-09-27T23:33:19.432656+0000","last_unstale":"2023-09-27T23:33:19.815149+0000","last_undegraded":"2023-09-27T23:33:19.815149+0000","last_fullsized":"2023-09-27T23:33:19.815149+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817212+0000","last_change":"2023-09-27T23:33:19.811187+0000","last_active":"2023-09-27T23:33:19.817212+0000","last_peered":"2023-09-27T23:33:19.817212+0000","last_clean":"2023-09-27T23:33:19.817212+0000","last_became_active":"2023-09-27T23:33:19.810921+0000","last_became_peered":"2023-09-27T23:33:19.810921+0000","last_unstale":"2023-09-27T23:33:19.817212+0000","last_undegraded":"2023-09-27T23:33:19.817212+0000","last_fullsized":"2023-09-27T23:33:19.817212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:58.020074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819500+0000","last_change":"2023-09-27T23:33:19.816975+0000","last_active":"2023-09-27T23:33:19.819500+0000","last_peered":"2023-09-27T23:33:19.819500+0000","last_clean":"2023-09-27T23:33:19.819500+0000","last_became_active":"2023-09-27T23:33:19.816573+0000","last_became_peered":"2023-09-27T23:33:19.816573+0000","last_unstale":"2023-09-27T23:33:19.819500+0000","last_undegraded":"2023-09-27T23:33:19.819500+0000","last_fullsized":"2023-09-27T23:33:19.819500+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:46:14.483620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790958+0000","last_change":"2023-09-27T23:33:19.435177+0000","last_active":"2023-09-27T23:33:19.790958+0000","last_peered":"2023-09-27T23:33:19.790958+0000","last_clean":"2023-09-27T23:33:19.790958+0000","last_became_active":"2023-09-27T23:33:19.432614+0000","last_became_peered":"2023-09-27T23:33:19.432614+0000","last_unstale":"2023-09-27T23:33:19.790958+0000","last_undegraded":"2023-09-27T23:33:19.790958+0000","last_fullsized":"2023-09-27T23:33:19.790958+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:07.369858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816845+0000","last_change":"2023-09-27T23:33:19.810499+0000","last_active":"2023-09-27T23:33:19.816845+0000","last_peered":"2023-09-27T23:33:19.816845+0000","last_clean":"2023-09-27T23:33:19.816845+0000","last_became_active":"2023-09-27T23:33:19.810241+0000","last_became_peered":"2023-09-27T23:33:19.810241+0000","last_unstale":"2023-09-27T23:33:19.816845+0000","last_undegraded":"2023-09-27T23:33:19.816845+0000","last_fullsized":"2023-09-27T23:33:19.816845+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:52:01.755843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816627+0000","last_change":"2023-09-27T23:33:19.436255+0000","last_active":"2023-09-27T23:33:19.816627+0000","last_peered":"2023-09-27T23:33:19.816627+0000","last_clean":"2023-09-27T23:33:19.816627+0000","last_became_active":"2023-09-27T23:33:19.435903+0000","last_became_peered":"2023-09-27T23:33:19.435903+0000","last_unstale":"2023-09-27T23:33:19.816627+0000","last_undegraded":"2023-09-27T23:33:19.816627+0000","last_fullsized":"2023-09-27T23:33:19.816627+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:01.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813432+0000","last_change":"2023-09-27T23:33:19.809133+0000","last_active":"2023-09-27T23:33:19.813432+0000","last_peered":"2023-09-27T23:33:19.813432+0000","last_clean":"2023-09-27T23:33:19.813432+0000","last_became_active":"2023-09-27T23:33:19.806861+0000","last_became_peered":"2023-09-27T23:33:19.806861+0000","last_unstale":"2023-09-27T23:33:19.813432+0000","last_undegraded":"2023-09-27T23:33:19.813432+0000","last_fullsized":"2023-09-27T23:33:19.813432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:46:39.679217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815834+0000","last_change":"2023-09-27T23:33:19.434317+0000","last_active":"2023-09-27T23:33:19.815834+0000","last_peered":"2023-09-27T23:33:19.815834+0000","last_clean":"2023-09-27T23:33:19.815834+0000","last_became_active":"2023-09-27T23:33:19.434002+0000","last_became_peered":"2023-09-27T23:33:19.434002+0000","last_unstale":"2023-09-27T23:33:19.815834+0000","last_undegraded":"2023-09-27T23:33:19.815834+0000","last_fullsized":"2023-09-27T23:33:19.815834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827942+0000","last_change":"2023-09-27T23:33:19.822884+0000","last_active":"2023-09-27T23:33:19.827942+0000","last_peered":"2023-09-27T23:33:19.827942+0000","last_clean":"2023-09-27T23:33:19.827942+0000","last_became_active":"2023-09-27T23:33:19.822472+0000","last_became_peered":"2023-09-27T23:33:19.822472+0000","last_unstale":"2023-09-27T23:33:19.827942+0000","last_undegraded":"2023-09-27T23:33:19.827942+0000","last_fullsized":"2023-09-27T23:33:19.827942+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:52:33.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809717+0000","last_change":"2023-09-27T23:33:19.804614+0000","last_active":"2023-09-27T23:33:19.809717+0000","last_peered":"2023-09-27T23:33:19.809717+0000","last_clean":"2023-09-27T23:33:19.809717+0000","last_became_active":"2023-09-27T23:33:19.804328+0000","last_became_peered":"2023-09-27T23:33:19.804328+0000","last_unstale":"2023-09-27T23:33:19.809717+0000","last_undegraded":"2023-09-27T23:33:19.809717+0000","last_fullsized":"2023-09-27T23:33:19.809717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:06.058928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788038+0000","last_change":"2023-09-27T23:33:19.422966+0000","last_active":"2023-09-27T23:33:19.788038+0000","last_peered":"2023-09-27T23:33:19.788038+0000","last_clean":"2023-09-27T23:33:19.788038+0000","last_became_active":"2023-09-27T23:33:19.422649+0000","last_became_peered":"2023-09-27T23:33:19.422649+0000","last_unstale":"2023-09-27T23:33:19.788038+0000","last_undegraded":"2023-09-27T23:33:19.788038+0000","last_fullsized":"2023-09-27T23:33:19.788038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:04:02.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.786802+0000","last_change":"2023-09-27T23:33:19.424649+0000","last_active":"2023-09-27T23:33:19.786802+0000","last_peered":"2023-09-27T23:33:19.786802+0000","last_clean":"2023-09-27T23:33:19.786802+0000","last_became_active":"2023-09-27T23:33:19.424230+0000","last_became_peered":"2023-09-27T23:33:19.424230+0000","last_unstale":"2023-09-27T23:33:19.786802+0000","last_undegraded":"2023-09-27T23:33:19.786802+0000","last_fullsized":"2023-09-27T23:33:19.786802+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:46:54.150917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814735+0000","last_change":"2023-09-27T23:33:19.807080+0000","last_active":"2023-09-27T23:33:19.814735+0000","last_peered":"2023-09-27T23:33:19.814735+0000","last_clean":"2023-09-27T23:33:19.814735+0000","last_became_active":"2023-09-27T23:33:19.806755+0000","last_became_peered":"2023-09-27T23:33:19.806755+0000","last_unstale":"2023-09-27T23:33:19.814735+0000","last_undegraded":"2023-09-27T23:33:19.814735+0000","last_fullsized":"2023-09-27T23:33:19.814735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:03:57.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806466+0000","last_change":"2023-09-27T23:33:19.804496+0000","last_active":"2023-09-27T23:33:19.806466+0000","last_peered":"2023-09-27T23:33:19.806466+0000","last_clean":"2023-09-27T23:33:19.806466+0000","last_became_active":"2023-09-27T23:33:19.804186+0000","last_became_peered":"2023-09-27T23:33:19.804186+0000","last_unstale":"2023-09-27T23:33:19.806466+0000","last_undegraded":"2023-09-27T23:33:19.806466+0000","last_fullsized":"2023-09-27T23:33:19.806466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:59:09.351560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"131'249","reported_seq":736,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257356+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:15.257356+0000","last_peered":"2023-09-27T23:33:15.257356+0000","last_clean":"2023-09-27T23:33:15.257356+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:15.257356+0000","last_undegraded":"2023-09-27T23:33:15.257356+0000","last_fullsized":"2023-09-27T23:33:15.257356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":249,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790633+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:19.790633+0000","last_peered":"2023-09-27T23:33:19.790633+0000","last_clean":"2023-09-27T23:33:19.790633+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:19.790633+0000","last_undegraded":"2023-09-27T23:33:19.790633+0000","last_fullsized":"2023-09-27T23:33:19.790633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826305+0000","last_change":"2023-09-27T23:33:19.818550+0000","last_active":"2023-09-27T23:33:19.826305+0000","last_peered":"2023-09-27T23:33:19.826305+0000","last_clean":"2023-09-27T23:33:19.826305+0000","last_became_active":"2023-09-27T23:33:19.818313+0000","last_became_peered":"2023-09-27T23:33:19.818313+0000","last_unstale":"2023-09-27T23:33:19.826305+0000","last_undegraded":"2023-09-27T23:33:19.826305+0000","last_fullsized":"2023-09-27T23:33:19.826305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825230+0000","last_change":"2023-09-27T23:33:19.439141+0000","last_active":"2023-09-27T23:33:19.825230+0000","last_peered":"2023-09-27T23:33:19.825230+0000","last_clean":"2023-09-27T23:33:19.825230+0000","last_became_active":"2023-09-27T23:33:19.438828+0000","last_became_peered":"2023-09-27T23:33:19.438828+0000","last_unstale":"2023-09-27T23:33:19.825230+0000","last_undegraded":"2023-09-27T23:33:19.825230+0000","last_fullsized":"2023-09-27T23:33:19.825230+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:33.463327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257347+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:15.257347+0000","last_peered":"2023-09-27T23:33:15.257347+0000","last_clean":"2023-09-27T23:33:15.257347+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:15.257347+0000","last_undegraded":"2023-09-27T23:33:15.257347+0000","last_fullsized":"2023-09-27T23:33:15.257347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816597+0000","last_change":"2023-09-27T23:33:19.812801+0000","last_active":"2023-09-27T23:33:19.816597+0000","last_peered":"2023-09-27T23:33:19.816597+0000","last_clean":"2023-09-27T23:33:19.816597+0000","last_became_active":"2023-09-27T23:33:19.812506+0000","last_became_peered":"2023-09-27T23:33:19.812506+0000","last_unstale":"2023-09-27T23:33:19.816597+0000","last_undegraded":"2023-09-27T23:33:19.816597+0000","last_fullsized":"2023-09-27T23:33:19.816597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:16:22.761586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788828+0000","last_change":"2023-09-27T23:33:19.424648+0000","last_active":"2023-09-27T23:33:19.788828+0000","last_peered":"2023-09-27T23:33:19.788828+0000","last_clean":"2023-09-27T23:33:19.788828+0000","last_became_active":"2023-09-27T23:33:19.424271+0000","last_became_peered":"2023-09-27T23:33:19.424271+0000","last_unstale":"2023-09-27T23:33:19.788828+0000","last_undegraded":"2023-09-27T23:33:19.788828+0000","last_fullsized":"2023-09-27T23:33:19.788828+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:25.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798546+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:19.798546+0000","last_peered":"2023-09-27T23:33:19.798546+0000","last_clean":"2023-09-27T23:33:19.798546+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:19.798546+0000","last_undegraded":"2023-09-27T23:33:19.798546+0000","last_fullsized":"2023-09-27T23:33:19.798546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817238+0000","last_change":"2023-09-27T23:33:19.813654+0000","last_active":"2023-09-27T23:33:19.817238+0000","last_peered":"2023-09-27T23:33:19.817238+0000","last_clean":"2023-09-27T23:33:19.817238+0000","last_became_active":"2023-09-27T23:33:19.813363+0000","last_became_peered":"2023-09-27T23:33:19.813363+0000","last_unstale":"2023-09-27T23:33:19.817238+0000","last_undegraded":"2023-09-27T23:33:19.817238+0000","last_fullsized":"2023-09-27T23:33:19.817238+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:09.337371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798813+0000","last_change":"2023-09-27T23:33:19.432418+0000","last_active":"2023-09-27T23:33:19.798813+0000","last_peered":"2023-09-27T23:33:19.798813+0000","last_clean":"2023-09-27T23:33:19.798813+0000","last_became_active":"2023-09-27T23:33:19.428641+0000","last_became_peered":"2023-09-27T23:33:19.428641+0000","last_unstale":"2023-09-27T23:33:19.798813+0000","last_undegraded":"2023-09-27T23:33:19.798813+0000","last_fullsized":"2023-09-27T23:33:19.798813+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790099+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:19.790099+0000","last_peered":"2023-09-27T23:33:19.790099+0000","last_clean":"2023-09-27T23:33:19.790099+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:19.790099+0000","last_undegraded":"2023-09-27T23:33:19.790099+0000","last_fullsized":"2023-09-27T23:33:19.790099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828563+0000","last_change":"2023-09-27T23:33:19.430328+0000","last_active":"2023-09-27T23:33:19.828563+0000","last_peered":"2023-09-27T23:33:19.828563+0000","last_clean":"2023-09-27T23:33:19.828563+0000","last_became_active":"2023-09-27T23:33:19.430033+0000","last_became_peered":"2023-09-27T23:33:19.430033+0000","last_unstale":"2023-09-27T23:33:19.828563+0000","last_undegraded":"2023-09-27T23:33:19.828563+0000","last_fullsized":"2023-09-27T23:33:19.828563+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:48.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789643+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:19.789643+0000","last_peered":"2023-09-27T23:33:19.789643+0000","last_clean":"2023-09-27T23:33:19.789643+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:19.789643+0000","last_undegraded":"2023-09-27T23:33:19.789643+0000","last_fullsized":"2023-09-27T23:33:19.789643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822697+0000","last_change":"2023-09-27T23:33:19.438806+0000","last_active":"2023-09-27T23:33:19.822697+0000","last_peered":"2023-09-27T23:33:19.822697+0000","last_clean":"2023-09-27T23:33:19.822697+0000","last_became_active":"2023-09-27T23:33:19.438541+0000","last_became_peered":"2023-09-27T23:33:19.438541+0000","last_unstale":"2023-09-27T23:33:19.822697+0000","last_undegraded":"2023-09-27T23:33:19.822697+0000","last_fullsized":"2023-09-27T23:33:19.822697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:29.683418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257370+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:15.257370+0000","last_peered":"2023-09-27T23:33:15.257370+0000","last_clean":"2023-09-27T23:33:15.257370+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:15.257370+0000","last_undegraded":"2023-09-27T23:33:15.257370+0000","last_fullsized":"2023-09-27T23:33:15.257370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817658+0000","last_change":"2023-09-27T23:33:19.815111+0000","last_active":"2023-09-27T23:33:19.817658+0000","last_peered":"2023-09-27T23:33:19.817658+0000","last_clean":"2023-09-27T23:33:19.817658+0000","last_became_active":"2023-09-27T23:33:19.814866+0000","last_became_peered":"2023-09-27T23:33:19.814866+0000","last_unstale":"2023-09-27T23:33:19.817658+0000","last_undegraded":"2023-09-27T23:33:19.817658+0000","last_fullsized":"2023-09-27T23:33:19.817658+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:30.614318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788544+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:19.788544+0000","last_peered":"2023-09-27T23:33:19.788544+0000","last_clean":"2023-09-27T23:33:19.788544+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:19.788544+0000","last_undegraded":"2023-09-27T23:33:19.788544+0000","last_fullsized":"2023-09-27T23:33:19.788544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816760+0000","last_change":"2023-09-27T23:33:19.807831+0000","last_active":"2023-09-27T23:33:19.816760+0000","last_peered":"2023-09-27T23:33:19.816760+0000","last_clean":"2023-09-27T23:33:19.816760+0000","last_became_active":"2023-09-27T23:33:19.807647+0000","last_became_peered":"2023-09-27T23:33:19.807647+0000","last_unstale":"2023-09-27T23:33:19.816760+0000","last_undegraded":"2023-09-27T23:33:19.816760+0000","last_fullsized":"2023-09-27T23:33:19.816760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:34.467334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792775+0000","last_change":"2023-09-27T23:33:19.438063+0000","last_active":"2023-09-27T23:33:19.792775+0000","last_peered":"2023-09-27T23:33:19.792775+0000","last_clean":"2023-09-27T23:33:19.792775+0000","last_became_active":"2023-09-27T23:33:19.437740+0000","last_became_peered":"2023-09-27T23:33:19.437740+0000","last_unstale":"2023-09-27T23:33:19.792775+0000","last_undegraded":"2023-09-27T23:33:19.792775+0000","last_fullsized":"2023-09-27T23:33:19.792775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:01.021179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796971+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:19.796971+0000","last_peered":"2023-09-27T23:33:19.796971+0000","last_clean":"2023-09-27T23:33:19.796971+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:19.796971+0000","last_undegraded":"2023-09-27T23:33:19.796971+0000","last_fullsized":"2023-09-27T23:33:19.796971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509676,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":2314240,"data_stored":1484913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":1.397},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":1.486}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":2.6850000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.0469999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":2.5680000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":2.7589999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.5600000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.704},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":2.8250000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26368,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1781760,"data_stored":923892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27000832},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":1.155},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":1.018},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.345},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.4079999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509676,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37776,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94334064,"statfs":{"total":96636764160,"available":96598081536,"internally_reserved":0,"allocated":1777664,"data_stored":922468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":1.242}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.995},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.4950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509676,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":23320,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":21632,"kb_avail":94348520,"statfs":{"total":96636764160,"available":96612884480,"internally_reserved":0,"allocated":1720320,"data_stored":894545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22151168},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":1.004}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.03},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.624},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.139},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.1160000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.087}]}]},{"osd":3,"up_from":15,"seq":64424509675,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1720320,"data_stored":894545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67000000000000004,"15min":0.69399999999999995},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.218},"max":{"1min":0.82199999999999995,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.441,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1779999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.1419999999999999,"15min":1.2869999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.40000000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.266,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.04,"5min":1.323,"15min":1.3440000000000001},"last":0.878},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.40500000000000003},"max":{"1min":1.093,"5min":1.446,"15min":1.5720000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":0.96599999999999997},"min":{"1min":0.68899999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2430000000000001,"5min":1.613,"15min":1.613},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.073,"15min":1.0760000000000001},"min":{"1min":0.76200000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.4430000000000001,"5min":1.605,"15min":1.605},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0960000000000001,"15min":1.1140000000000001},"min":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.66000000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.571},"last":1.29},{"interface":"front","average":{"1min":1.109,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.69999999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.431,"5min":1.714,"15min":1.764},"last":1.0940000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.1160000000000001},"min":{"1min":0.753,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.353,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.91600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5509999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.23}]},{"osd":7,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2090000000000001,"15min":1.2310000000000001},"min":{"1min":0.84099999999999997,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.4390000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.018},{"interface":"front","average":{"1min":1.145,"5min":1.165,"15min":1.196},"min":{"1min":0.72899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3680000000000001,"5min":1.837,"15min":1.837},"last":1.194}]}]},{"osd":2,"up_from":15,"seq":64424509677,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22672,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":94349168,"statfs":{"total":96636764160,"available":96613548032,"internally_reserved":0,"allocated":1777664,"data_stored":922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.498},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.95999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.01},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37896,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":36160,"kb_avail":94333944,"statfs":{"total":96636764160,"available":96597958656,"internally_reserved":0,"allocated":1769472,"data_stored":922356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37027840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":1.026}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.171},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.302},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.119}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.27},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2470000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509677,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28892,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94342948,"statfs":{"total":96636764160,"available":96607178752,"internally_reserved":0,"allocated":2314240,"data_stored":1484913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.81000000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.002},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.095}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:24.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:24.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 Processor -- start 2023-09-27T23:33:24.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 -- start start 2023-09-27T23:33:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb806de20 0x7f2eb80c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2eb806c7c0 0x7f2eb80c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb806d990 con 0x7f2eb8155960 2023-09-27T23:33:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb806cd80 con 0x7f2eb806de20 2023-09-27T23:33:24.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.267+0000 7f2ebe4c0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb806d060 con 0x7f2eb806c7c0 2023-09-27T23:33:24.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb806de20 0x7f2eb80c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb806de20 0x7f2eb80c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45206/0 (socket says 172.21.15.64:45206) 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 -- 172.21.15.64:0/3706454763 learned_addr learned my addr 172.21.15.64:0/3706454763 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebdcbf700 1 --2- 172.21.15.64:0/3706454763 >> v2:172.21.15.123:3300/0 conn(0x7f2eb806c7c0 0x7f2eb80c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 -- 172.21.15.64:0/3706454763 >> v2:172.21.15.123:3300/0 conn(0x7f2eb806c7c0 msgr2=0x7f2eb80c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 --2- 172.21.15.64:0/3706454763 >> v2:172.21.15.123:3300/0 conn(0x7f2eb806c7c0 0x7f2eb80c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 -- 172.21.15.64:0/3706454763 >> v1:172.21.15.64:6789/0 conn(0x7f2eb8155960 legacy=0x7f2eb815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 -- 172.21.15.64:0/3706454763 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb806d340 con 0x7f2eb806de20 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2eabfff700 1 -- 172.21.15.64:0/3706454763 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987745486 0 0) 0x7f2eb806d990 con 0x7f2eb8155960 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebdcbf700 1 --2- 172.21.15.64:0/3706454763 >> v2:172.21.15.123:3300/0 conn(0x7f2eb806c7c0 0x7f2eb80c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebccbd700 1 --2- 172.21.15.64:0/3706454763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806de20 0x7f2eb80c77f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806a380 tx=0x7f2eac00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219b9250d2d8228a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2eabfff700 1 -- 172.21.15.64:0/3706454763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00dc50 con 0x7f2eb806de20 2023-09-27T23:33:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2eabfff700 1 -- 172.21.15.64:0/3706454763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac00ddb0 con 0x7f2eb806de20 2023-09-27T23:33:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2eabfff700 1 -- 172.21.15.64:0/3706454763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0196d0 con 0x7f2eb806de20 2023-09-27T23:33:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/3706454763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806de20 msgr2=0x7f2eb80c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/3706454763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806de20 0x7f2eb80c77f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806a380 tx=0x7f2eac00b4f0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/3706454763 shutdown_connections 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/3706454763 >> v2:172.21.15.123:3300/0 conn(0x7f2eb806c7c0 0x7f2eb80c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/3706454763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806de20 0x7f2eb80c77f0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/3706454763 >> 172.21.15.64:0/3706454763 conn(0x7f2eb8067bb0 msgr2=0x7f2eb80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/3706454763 shutdown_connections 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.271+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/3706454763 wait complete. 2023-09-27T23:33:24.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 Processor -- start 2023-09-27T23:33:24.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 -- start start 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 0x7f2eb8169f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8155960 0x7f2eb816c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80c4c70 0x7f2eb80c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb8154870 con 0x7f2eb806c7c0 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c5610 con 0x7f2eb80c4c70 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb816a460 con 0x7f2eb8155960 2023-09-27T23:33:24.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 0x7f2eb8169f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 0x7f2eb8169f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45208/0 (socket says 172.21.15.64:45208) 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 -- 172.21.15.64:0/2315813497 learned_addr learned my addr 172.21.15.64:0/2315813497 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebccbd700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8155960 0x7f2eb816c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebdcbf700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80c4c70 0x7f2eb80c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 -- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8155960 msgr2=0x7f2eb816c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8155960 0x7f2eb816c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 -- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80c4c70 msgr2=0x7f2eb80c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80c4c70 0x7f2eb80c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 -- 172.21.15.64:0/2315813497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac00b790 con 0x7f2eb806c7c0 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebd4be700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 0x7f2eb8169f20 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c530 tx=0x7f2eb400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011590 con 0x7f2eb806c7c0 2023-09-27T23:33:24.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4011710 con 0x7f2eb806c7c0 2023-09-27T23:33:24.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb8153c60 con 0x7f2eb806c7c0 2023-09-27T23:33:24.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011ad0 con 0x7f2eb806c7c0 2023-09-27T23:33:24.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.275+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb8154220 con 0x7f2eb806c7c0 2023-09-27T23:33:24.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.279+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb804be40 con 0x7f2eb806c7c0 2023-09-27T23:33:24.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.279+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2eb401b3b0 con 0x7f2eb806c7c0 2023-09-27T23:33:24.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.279+0000 7f2ea97fa700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2e9c060a70 0x7f2e9c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.279+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2eb407c590 con 0x7f2eb806c7c0 2023-09-27T23:33:24.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.279+0000 7f2ebccbd700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2e9c060a70 0x7f2e9c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.279+0000 7f2ebccbd700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2e9c060a70 0x7f2e9c062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806a380 tx=0x7f2eac01b000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.287+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2eb404c5c0 con 0x7f2eb806c7c0 2023-09-27T23:33:24.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.447+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2eb806d620 con 0x7f2e9c060a70 2023-09-27T23:33:24.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.463+0000 7f2ea97fa700 1 -- 172.21.15.64:0/2315813497 <== mgr.7358 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300604 (secure 0 0 0) 0x7f2eb806d620 con 0x7f2e9c060a70 2023-09-27T23:33:24.471 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2e9c060a70 msgr2=0x7f2e9c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2e9c060a70 0x7f2e9c062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806a380 tx=0x7f2eac01b000 comp rx=0 tx=0).stop 2023-09-27T23:33:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 msgr2=0x7f2eb8169f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 0x7f2eb8169f20 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c530 tx=0x7f2eb400c840 comp rx=0 tx=0).stop 2023-09-27T23:33:24.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 shutdown_connections 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2e9c060a70 0x7f2e9c062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80c4c70 0x7f2eb80c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8155960 0x7f2eb816c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 --2- 172.21.15.64:0/2315813497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb806c7c0 0x7f2eb8169f20 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 >> 172.21.15.64:0/2315813497 conn(0x7f2eb8067bb0 msgr2=0x7f2eb80cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 shutdown_connections 2023-09-27T23:33:24.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.471+0000 7f2ebe4c0700 1 -- 172.21.15.64:0/2315813497 wait complete. 2023-09-27T23:33:24.476 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:24.488 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:22.682895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592016,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":429,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":118,"acting":118,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234468,"kb_used_data":14820,"kb_used_omap":0,"kb_used_meta":219584,"kb_avail":754740252,"statfs":{"total":773094113280,"available":772854018048,"internally_reserved":0,"allocated":15175680,"data_stored":8450576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224854016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.289603"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816721+0000","last_change":"2023-09-27T23:33:19.810193+0000","last_active":"2023-09-27T23:33:19.816721+0000","last_peered":"2023-09-27T23:33:19.816721+0000","last_clean":"2023-09-27T23:33:19.816721+0000","last_became_active":"2023-09-27T23:33:19.809935+0000","last_became_peered":"2023-09-27T23:33:19.809935+0000","last_unstale":"2023-09-27T23:33:19.816721+0000","last_undegraded":"2023-09-27T23:33:19.816721+0000","last_fullsized":"2023-09-27T23:33:19.816721+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:32:44.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816570+0000","last_change":"2023-09-27T23:33:19.811311+0000","last_active":"2023-09-27T23:33:19.816570+0000","last_peered":"2023-09-27T23:33:19.816570+0000","last_clean":"2023-09-27T23:33:19.816570+0000","last_became_active":"2023-09-27T23:33:19.809810+0000","last_became_peered":"2023-09-27T23:33:19.809810+0000","last_unstale":"2023-09-27T23:33:19.816570+0000","last_undegraded":"2023-09-27T23:33:19.816570+0000","last_fullsized":"2023-09-27T23:33:19.816570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:06.194398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825973+0000","last_change":"2023-09-27T23:33:19.817990+0000","last_active":"2023-09-27T23:33:19.825973+0000","last_peered":"2023-09-27T23:33:19.825973+0000","last_clean":"2023-09-27T23:33:19.825973+0000","last_became_active":"2023-09-27T23:33:19.809746+0000","last_became_peered":"2023-09-27T23:33:19.809746+0000","last_unstale":"2023-09-27T23:33:19.825973+0000","last_undegraded":"2023-09-27T23:33:19.825973+0000","last_fullsized":"2023-09-27T23:33:19.825973+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.812775+0000","last_change":"2023-09-27T23:33:19.801379+0000","last_active":"2023-09-27T23:33:19.812775+0000","last_peered":"2023-09-27T23:33:19.812775+0000","last_clean":"2023-09-27T23:33:19.812775+0000","last_became_active":"2023-09-27T23:33:19.800971+0000","last_became_peered":"2023-09-27T23:33:19.800971+0000","last_unstale":"2023-09-27T23:33:19.812775+0000","last_undegraded":"2023-09-27T23:33:19.812775+0000","last_fullsized":"2023-09-27T23:33:19.812775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:53.280332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826231+0000","last_change":"2023-09-27T23:33:19.819831+0000","last_active":"2023-09-27T23:33:19.826231+0000","last_peered":"2023-09-27T23:33:19.826231+0000","last_clean":"2023-09-27T23:33:19.826231+0000","last_became_active":"2023-09-27T23:33:19.819582+0000","last_became_peered":"2023-09-27T23:33:19.819582+0000","last_unstale":"2023-09-27T23:33:19.826231+0000","last_undegraded":"2023-09-27T23:33:19.826231+0000","last_fullsized":"2023-09-27T23:33:19.826231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:59.618721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826213+0000","last_change":"2023-09-27T23:33:19.818540+0000","last_active":"2023-09-27T23:33:19.826213+0000","last_peered":"2023-09-27T23:33:19.826213+0000","last_clean":"2023-09-27T23:33:19.826213+0000","last_became_active":"2023-09-27T23:33:19.818222+0000","last_became_peered":"2023-09-27T23:33:19.818222+0000","last_unstale":"2023-09-27T23:33:19.826213+0000","last_undegraded":"2023-09-27T23:33:19.826213+0000","last_fullsized":"2023-09-27T23:33:19.826213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813361+0000","last_change":"2023-09-27T23:33:19.806217+0000","last_active":"2023-09-27T23:33:19.813361+0000","last_peered":"2023-09-27T23:33:19.813361+0000","last_clean":"2023-09-27T23:33:19.813361+0000","last_became_active":"2023-09-27T23:33:19.805338+0000","last_became_peered":"2023-09-27T23:33:19.805338+0000","last_unstale":"2023-09-27T23:33:19.813361+0000","last_undegraded":"2023-09-27T23:33:19.813361+0000","last_fullsized":"2023-09-27T23:33:19.813361+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.311666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826156+0000","last_change":"2023-09-27T23:33:19.810568+0000","last_active":"2023-09-27T23:33:19.826156+0000","last_peered":"2023-09-27T23:33:19.826156+0000","last_clean":"2023-09-27T23:33:19.826156+0000","last_became_active":"2023-09-27T23:33:19.810237+0000","last_became_peered":"2023-09-27T23:33:19.810237+0000","last_unstale":"2023-09-27T23:33:19.826156+0000","last_undegraded":"2023-09-27T23:33:19.826156+0000","last_fullsized":"2023-09-27T23:33:19.826156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816810+0000","last_change":"2023-09-27T23:33:19.810731+0000","last_active":"2023-09-27T23:33:19.816810+0000","last_peered":"2023-09-27T23:33:19.816810+0000","last_clean":"2023-09-27T23:33:19.816810+0000","last_became_active":"2023-09-27T23:33:19.810477+0000","last_became_peered":"2023-09-27T23:33:19.810477+0000","last_unstale":"2023-09-27T23:33:19.816810+0000","last_undegraded":"2023-09-27T23:33:19.816810+0000","last_fullsized":"2023-09-27T23:33:19.816810+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:19:28.503800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809513+0000","last_change":"2023-09-27T23:33:19.805356+0000","last_active":"2023-09-27T23:33:19.809513+0000","last_peered":"2023-09-27T23:33:19.809513+0000","last_clean":"2023-09-27T23:33:19.809513+0000","last_became_active":"2023-09-27T23:33:19.805064+0000","last_became_peered":"2023-09-27T23:33:19.805064+0000","last_unstale":"2023-09-27T23:33:19.809513+0000","last_undegraded":"2023-09-27T23:33:19.809513+0000","last_fullsized":"2023-09-27T23:33:19.809513+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:37.422975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827932+0000","last_change":"2023-09-27T23:33:19.822764+0000","last_active":"2023-09-27T23:33:19.827932+0000","last_peered":"2023-09-27T23:33:19.827932+0000","last_clean":"2023-09-27T23:33:19.827932+0000","last_became_active":"2023-09-27T23:33:19.822456+0000","last_became_peered":"2023-09-27T23:33:19.822456+0000","last_unstale":"2023-09-27T23:33:19.827932+0000","last_undegraded":"2023-09-27T23:33:19.827932+0000","last_fullsized":"2023-09-27T23:33:19.827932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826283+0000","last_change":"2023-09-27T23:33:19.818051+0000","last_active":"2023-09-27T23:33:19.826283+0000","last_peered":"2023-09-27T23:33:19.826283+0000","last_clean":"2023-09-27T23:33:19.826283+0000","last_became_active":"2023-09-27T23:33:19.809795+0000","last_became_peered":"2023-09-27T23:33:19.809795+0000","last_unstale":"2023-09-27T23:33:19.826283+0000","last_undegraded":"2023-09-27T23:33:19.826283+0000","last_fullsized":"2023-09-27T23:33:19.826283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.804587+0000","last_change":"2023-09-27T23:33:19.798526+0000","last_active":"2023-09-27T23:33:19.804587+0000","last_peered":"2023-09-27T23:33:19.804587+0000","last_clean":"2023-09-27T23:33:19.804587+0000","last_became_active":"2023-09-27T23:33:19.798226+0000","last_became_peered":"2023-09-27T23:33:19.798226+0000","last_unstale":"2023-09-27T23:33:19.804587+0000","last_undegraded":"2023-09-27T23:33:19.804587+0000","last_fullsized":"2023-09-27T23:33:19.804587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:13:28.298592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815678+0000","last_change":"2023-09-27T23:33:19.805843+0000","last_active":"2023-09-27T23:33:19.815678+0000","last_peered":"2023-09-27T23:33:19.815678+0000","last_clean":"2023-09-27T23:33:19.815678+0000","last_became_active":"2023-09-27T23:33:19.805571+0000","last_became_peered":"2023-09-27T23:33:19.805571+0000","last_unstale":"2023-09-27T23:33:19.815678+0000","last_undegraded":"2023-09-27T23:33:19.815678+0000","last_fullsized":"2023-09-27T23:33:19.815678+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:49.075742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.823630+0000","last_change":"2023-09-27T23:33:19.819123+0000","last_active":"2023-09-27T23:33:19.823630+0000","last_peered":"2023-09-27T23:33:19.823630+0000","last_clean":"2023-09-27T23:33:19.823630+0000","last_became_active":"2023-09-27T23:33:19.818840+0000","last_became_peered":"2023-09-27T23:33:19.818840+0000","last_unstale":"2023-09-27T23:33:19.823630+0000","last_undegraded":"2023-09-27T23:33:19.823630+0000","last_fullsized":"2023-09-27T23:33:19.823630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:00.849910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827890+0000","last_change":"2023-09-27T23:33:19.822665+0000","last_active":"2023-09-27T23:33:19.827890+0000","last_peered":"2023-09-27T23:33:19.827890+0000","last_clean":"2023-09-27T23:33:19.827890+0000","last_became_active":"2023-09-27T23:33:19.822350+0000","last_became_peered":"2023-09-27T23:33:19.822350+0000","last_unstale":"2023-09-27T23:33:19.827890+0000","last_undegraded":"2023-09-27T23:33:19.827890+0000","last_fullsized":"2023-09-27T23:33:19.827890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:30:36.041799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827909+0000","last_change":"2023-09-27T23:33:19.822700+0000","last_active":"2023-09-27T23:33:19.827909+0000","last_peered":"2023-09-27T23:33:19.827909+0000","last_clean":"2023-09-27T23:33:19.827909+0000","last_became_active":"2023-09-27T23:33:19.822405+0000","last_became_peered":"2023-09-27T23:33:19.822405+0000","last_unstale":"2023-09-27T23:33:19.827909+0000","last_undegraded":"2023-09-27T23:33:19.827909+0000","last_fullsized":"2023-09-27T23:33:19.827909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:22.750767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825923+0000","last_change":"2023-09-27T23:33:19.810331+0000","last_active":"2023-09-27T23:33:19.825923+0000","last_peered":"2023-09-27T23:33:19.825923+0000","last_clean":"2023-09-27T23:33:19.825923+0000","last_became_active":"2023-09-27T23:33:19.810041+0000","last_became_peered":"2023-09-27T23:33:19.810041+0000","last_unstale":"2023-09-27T23:33:19.825923+0000","last_undegraded":"2023-09-27T23:33:19.825923+0000","last_fullsized":"2023-09-27T23:33:19.825923+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814781+0000","last_change":"2023-09-27T23:33:19.436486+0000","last_active":"2023-09-27T23:33:19.814781+0000","last_peered":"2023-09-27T23:33:19.814781+0000","last_clean":"2023-09-27T23:33:19.814781+0000","last_became_active":"2023-09-27T23:33:19.436056+0000","last_became_peered":"2023-09-27T23:33:19.436056+0000","last_unstale":"2023-09-27T23:33:19.814781+0000","last_undegraded":"2023-09-27T23:33:19.814781+0000","last_fullsized":"2023-09-27T23:33:19.814781+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809160+0000","last_change":"2023-09-27T23:33:19.427279+0000","last_active":"2023-09-27T23:33:19.809160+0000","last_peered":"2023-09-27T23:33:19.809160+0000","last_clean":"2023-09-27T23:33:19.809160+0000","last_became_active":"2023-09-27T23:33:19.426915+0000","last_became_peered":"2023-09-27T23:33:19.426915+0000","last_unstale":"2023-09-27T23:33:19.809160+0000","last_undegraded":"2023-09-27T23:33:19.809160+0000","last_fullsized":"2023-09-27T23:33:19.809160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:54.341042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816372+0000","last_change":"2023-09-27T23:33:19.433429+0000","last_active":"2023-09-27T23:33:19.816372+0000","last_peered":"2023-09-27T23:33:19.816372+0000","last_clean":"2023-09-27T23:33:19.816372+0000","last_became_active":"2023-09-27T23:33:19.433030+0000","last_became_peered":"2023-09-27T23:33:19.433030+0000","last_unstale":"2023-09-27T23:33:19.816372+0000","last_undegraded":"2023-09-27T23:33:19.816372+0000","last_fullsized":"2023-09-27T23:33:19.816372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:13:30.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826212+0000","last_change":"2023-09-27T23:33:19.819783+0000","last_active":"2023-09-27T23:33:19.826212+0000","last_peered":"2023-09-27T23:33:19.826212+0000","last_clean":"2023-09-27T23:33:19.826212+0000","last_became_active":"2023-09-27T23:33:19.819564+0000","last_became_peered":"2023-09-27T23:33:19.819564+0000","last_unstale":"2023-09-27T23:33:19.826212+0000","last_undegraded":"2023-09-27T23:33:19.826212+0000","last_fullsized":"2023-09-27T23:33:19.826212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:37.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813572+0000","last_change":"2023-09-27T23:33:19.809091+0000","last_active":"2023-09-27T23:33:19.813572+0000","last_peered":"2023-09-27T23:33:19.813572+0000","last_clean":"2023-09-27T23:33:19.813572+0000","last_became_active":"2023-09-27T23:33:19.808763+0000","last_became_peered":"2023-09-27T23:33:19.808763+0000","last_unstale":"2023-09-27T23:33:19.813572+0000","last_undegraded":"2023-09-27T23:33:19.813572+0000","last_fullsized":"2023-09-27T23:33:19.813572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:41.828993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788226+0000","last_change":"2023-09-27T23:33:19.428241+0000","last_active":"2023-09-27T23:33:19.788226+0000","last_peered":"2023-09-27T23:33:19.788226+0000","last_clean":"2023-09-27T23:33:19.788226+0000","last_became_active":"2023-09-27T23:33:19.427908+0000","last_became_peered":"2023-09-27T23:33:19.427908+0000","last_unstale":"2023-09-27T23:33:19.788226+0000","last_undegraded":"2023-09-27T23:33:19.788226+0000","last_fullsized":"2023-09-27T23:33:19.788226+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:49.753965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824232+0000","last_change":"2023-09-27T23:33:19.808815+0000","last_active":"2023-09-27T23:33:19.824232+0000","last_peered":"2023-09-27T23:33:19.824232+0000","last_clean":"2023-09-27T23:33:19.824232+0000","last_became_active":"2023-09-27T23:33:19.808532+0000","last_became_peered":"2023-09-27T23:33:19.808532+0000","last_unstale":"2023-09-27T23:33:19.824232+0000","last_undegraded":"2023-09-27T23:33:19.824232+0000","last_fullsized":"2023-09-27T23:33:19.824232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815149+0000","last_change":"2023-09-27T23:33:19.435202+0000","last_active":"2023-09-27T23:33:19.815149+0000","last_peered":"2023-09-27T23:33:19.815149+0000","last_clean":"2023-09-27T23:33:19.815149+0000","last_became_active":"2023-09-27T23:33:19.432656+0000","last_became_peered":"2023-09-27T23:33:19.432656+0000","last_unstale":"2023-09-27T23:33:19.815149+0000","last_undegraded":"2023-09-27T23:33:19.815149+0000","last_fullsized":"2023-09-27T23:33:19.815149+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817212+0000","last_change":"2023-09-27T23:33:19.811187+0000","last_active":"2023-09-27T23:33:19.817212+0000","last_peered":"2023-09-27T23:33:19.817212+0000","last_clean":"2023-09-27T23:33:19.817212+0000","last_became_active":"2023-09-27T23:33:19.810921+0000","last_became_peered":"2023-09-27T23:33:19.810921+0000","last_unstale":"2023-09-27T23:33:19.817212+0000","last_undegraded":"2023-09-27T23:33:19.817212+0000","last_fullsized":"2023-09-27T23:33:19.817212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:58.020074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819500+0000","last_change":"2023-09-27T23:33:19.816975+0000","last_active":"2023-09-27T23:33:19.819500+0000","last_peered":"2023-09-27T23:33:19.819500+0000","last_clean":"2023-09-27T23:33:19.819500+0000","last_became_active":"2023-09-27T23:33:19.816573+0000","last_became_peered":"2023-09-27T23:33:19.816573+0000","last_unstale":"2023-09-27T23:33:19.819500+0000","last_undegraded":"2023-09-27T23:33:19.819500+0000","last_fullsized":"2023-09-27T23:33:19.819500+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:46:14.483620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790958+0000","last_change":"2023-09-27T23:33:19.435177+0000","last_active":"2023-09-27T23:33:19.790958+0000","last_peered":"2023-09-27T23:33:19.790958+0000","last_clean":"2023-09-27T23:33:19.790958+0000","last_became_active":"2023-09-27T23:33:19.432614+0000","last_became_peered":"2023-09-27T23:33:19.432614+0000","last_unstale":"2023-09-27T23:33:19.790958+0000","last_undegraded":"2023-09-27T23:33:19.790958+0000","last_fullsized":"2023-09-27T23:33:19.790958+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:07.369858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816845+0000","last_change":"2023-09-27T23:33:19.810499+0000","last_active":"2023-09-27T23:33:19.816845+0000","last_peered":"2023-09-27T23:33:19.816845+0000","last_clean":"2023-09-27T23:33:19.816845+0000","last_became_active":"2023-09-27T23:33:19.810241+0000","last_became_peered":"2023-09-27T23:33:19.810241+0000","last_unstale":"2023-09-27T23:33:19.816845+0000","last_undegraded":"2023-09-27T23:33:19.816845+0000","last_fullsized":"2023-09-27T23:33:19.816845+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:52:01.755843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816627+0000","last_change":"2023-09-27T23:33:19.436255+0000","last_active":"2023-09-27T23:33:19.816627+0000","last_peered":"2023-09-27T23:33:19.816627+0000","last_clean":"2023-09-27T23:33:19.816627+0000","last_became_active":"2023-09-27T23:33:19.435903+0000","last_became_peered":"2023-09-27T23:33:19.435903+0000","last_unstale":"2023-09-27T23:33:19.816627+0000","last_undegraded":"2023-09-27T23:33:19.816627+0000","last_fullsized":"2023-09-27T23:33:19.816627+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:01.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813432+0000","last_change":"2023-09-27T23:33:19.809133+0000","last_active":"2023-09-27T23:33:19.813432+0000","last_peered":"2023-09-27T23:33:19.813432+0000","last_clean":"2023-09-27T23:33:19.813432+0000","last_became_active":"2023-09-27T23:33:19.806861+0000","last_became_peered":"2023-09-27T23:33:19.806861+0000","last_unstale":"2023-09-27T23:33:19.813432+0000","last_undegraded":"2023-09-27T23:33:19.813432+0000","last_fullsized":"2023-09-27T23:33:19.813432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:46:39.679217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815834+0000","last_change":"2023-09-27T23:33:19.434317+0000","last_active":"2023-09-27T23:33:19.815834+0000","last_peered":"2023-09-27T23:33:19.815834+0000","last_clean":"2023-09-27T23:33:19.815834+0000","last_became_active":"2023-09-27T23:33:19.434002+0000","last_became_peered":"2023-09-27T23:33:19.434002+0000","last_unstale":"2023-09-27T23:33:19.815834+0000","last_undegraded":"2023-09-27T23:33:19.815834+0000","last_fullsized":"2023-09-27T23:33:19.815834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827942+0000","last_change":"2023-09-27T23:33:19.822884+0000","last_active":"2023-09-27T23:33:19.827942+0000","last_peered":"2023-09-27T23:33:19.827942+0000","last_clean":"2023-09-27T23:33:19.827942+0000","last_became_active":"2023-09-27T23:33:19.822472+0000","last_became_peered":"2023-09-27T23:33:19.822472+0000","last_unstale":"2023-09-27T23:33:19.827942+0000","last_undegraded":"2023-09-27T23:33:19.827942+0000","last_fullsized":"2023-09-27T23:33:19.827942+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:52:33.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809717+0000","last_change":"2023-09-27T23:33:19.804614+0000","last_active":"2023-09-27T23:33:19.809717+0000","last_peered":"2023-09-27T23:33:19.809717+0000","last_clean":"2023-09-27T23:33:19.809717+0000","last_became_active":"2023-09-27T23:33:19.804328+0000","last_became_peered":"2023-09-27T23:33:19.804328+0000","last_unstale":"2023-09-27T23:33:19.809717+0000","last_undegraded":"2023-09-27T23:33:19.809717+0000","last_fullsized":"2023-09-27T23:33:19.809717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:06.058928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788038+0000","last_change":"2023-09-27T23:33:19.422966+0000","last_active":"2023-09-27T23:33:19.788038+0000","last_peered":"2023-09-27T23:33:19.788038+0000","last_clean":"2023-09-27T23:33:19.788038+0000","last_became_active":"2023-09-27T23:33:19.422649+0000","last_became_peered":"2023-09-27T23:33:19.422649+0000","last_unstale":"2023-09-27T23:33:19.788038+0000","last_undegraded":"2023-09-27T23:33:19.788038+0000","last_fullsized":"2023-09-27T23:33:19.788038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:04:02.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.786802+0000","last_change":"2023-09-27T23:33:19.424649+0000","last_active":"2023-09-27T23:33:19.786802+0000","last_peered":"2023-09-27T23:33:19.786802+0000","last_clean":"2023-09-27T23:33:19.786802+0000","last_became_active":"2023-09-27T23:33:19.424230+0000","last_became_peered":"2023-09-27T23:33:19.424230+0000","last_unstale":"2023-09-27T23:33:19.786802+0000","last_undegraded":"2023-09-27T23:33:19.786802+0000","last_fullsized":"2023-09-27T23:33:19.786802+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:46:54.150917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814735+0000","last_change":"2023-09-27T23:33:19.807080+0000","last_active":"2023-09-27T23:33:19.814735+0000","last_peered":"2023-09-27T23:33:19.814735+0000","last_clean":"2023-09-27T23:33:19.814735+0000","last_became_active":"2023-09-27T23:33:19.806755+0000","last_became_peered":"2023-09-27T23:33:19.806755+0000","last_unstale":"2023-09-27T23:33:19.814735+0000","last_undegraded":"2023-09-27T23:33:19.814735+0000","last_fullsized":"2023-09-27T23:33:19.814735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:03:57.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806466+0000","last_change":"2023-09-27T23:33:19.804496+0000","last_active":"2023-09-27T23:33:19.806466+0000","last_peered":"2023-09-27T23:33:19.806466+0000","last_clean":"2023-09-27T23:33:19.806466+0000","last_became_active":"2023-09-27T23:33:19.804186+0000","last_became_peered":"2023-09-27T23:33:19.804186+0000","last_unstale":"2023-09-27T23:33:19.806466+0000","last_undegraded":"2023-09-27T23:33:19.806466+0000","last_fullsized":"2023-09-27T23:33:19.806466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:59:09.351560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"131'249","reported_seq":736,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257356+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:15.257356+0000","last_peered":"2023-09-27T23:33:15.257356+0000","last_clean":"2023-09-27T23:33:15.257356+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:15.257356+0000","last_undegraded":"2023-09-27T23:33:15.257356+0000","last_fullsized":"2023-09-27T23:33:15.257356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":249,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790633+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:19.790633+0000","last_peered":"2023-09-27T23:33:19.790633+0000","last_clean":"2023-09-27T23:33:19.790633+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:19.790633+0000","last_undegraded":"2023-09-27T23:33:19.790633+0000","last_fullsized":"2023-09-27T23:33:19.790633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826305+0000","last_change":"2023-09-27T23:33:19.818550+0000","last_active":"2023-09-27T23:33:19.826305+0000","last_peered":"2023-09-27T23:33:19.826305+0000","last_clean":"2023-09-27T23:33:19.826305+0000","last_became_active":"2023-09-27T23:33:19.818313+0000","last_became_peered":"2023-09-27T23:33:19.818313+0000","last_unstale":"2023-09-27T23:33:19.826305+0000","last_undegraded":"2023-09-27T23:33:19.826305+0000","last_fullsized":"2023-09-27T23:33:19.826305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825230+0000","last_change":"2023-09-27T23:33:19.439141+0000","last_active":"2023-09-27T23:33:19.825230+0000","last_peered":"2023-09-27T23:33:19.825230+0000","last_clean":"2023-09-27T23:33:19.825230+0000","last_became_active":"2023-09-27T23:33:19.438828+0000","last_became_peered":"2023-09-27T23:33:19.438828+0000","last_unstale":"2023-09-27T23:33:19.825230+0000","last_undegraded":"2023-09-27T23:33:19.825230+0000","last_fullsized":"2023-09-27T23:33:19.825230+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:33.463327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257347+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:15.257347+0000","last_peered":"2023-09-27T23:33:15.257347+0000","last_clean":"2023-09-27T23:33:15.257347+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:15.257347+0000","last_undegraded":"2023-09-27T23:33:15.257347+0000","last_fullsized":"2023-09-27T23:33:15.257347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816597+0000","last_change":"2023-09-27T23:33:19.812801+0000","last_active":"2023-09-27T23:33:19.816597+0000","last_peered":"2023-09-27T23:33:19.816597+0000","last_clean":"2023-09-27T23:33:19.816597+0000","last_became_active":"2023-09-27T23:33:19.812506+0000","last_became_peered":"2023-09-27T23:33:19.812506+0000","last_unstale":"2023-09-27T23:33:19.816597+0000","last_undegraded":"2023-09-27T23:33:19.816597+0000","last_fullsized":"2023-09-27T23:33:19.816597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:16:22.761586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788828+0000","last_change":"2023-09-27T23:33:19.424648+0000","last_active":"2023-09-27T23:33:19.788828+0000","last_peered":"2023-09-27T23:33:19.788828+0000","last_clean":"2023-09-27T23:33:19.788828+0000","last_became_active":"2023-09-27T23:33:19.424271+0000","last_became_peered":"2023-09-27T23:33:19.424271+0000","last_unstale":"2023-09-27T23:33:19.788828+0000","last_undegraded":"2023-09-27T23:33:19.788828+0000","last_fullsized":"2023-09-27T23:33:19.788828+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:25.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798546+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:19.798546+0000","last_peered":"2023-09-27T23:33:19.798546+0000","last_clean":"2023-09-27T23:33:19.798546+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:19.798546+0000","last_undegraded":"2023-09-27T23:33:19.798546+0000","last_fullsized":"2023-09-27T23:33:19.798546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817238+0000","last_change":"2023-09-27T23:33:19.813654+0000","last_active":"2023-09-27T23:33:19.817238+0000","last_peered":"2023-09-27T23:33:19.817238+0000","last_clean":"2023-09-27T23:33:19.817238+0000","last_became_active":"2023-09-27T23:33:19.813363+0000","last_became_peered":"2023-09-27T23:33:19.813363+0000","last_unstale":"2023-09-27T23:33:19.817238+0000","last_undegraded":"2023-09-27T23:33:19.817238+0000","last_fullsized":"2023-09-27T23:33:19.817238+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:09.337371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798813+0000","last_change":"2023-09-27T23:33:19.432418+0000","last_active":"2023-09-27T23:33:19.798813+0000","last_peered":"2023-09-27T23:33:19.798813+0000","last_clean":"2023-09-27T23:33:19.798813+0000","last_became_active":"2023-09-27T23:33:19.428641+0000","last_became_peered":"2023-09-27T23:33:19.428641+0000","last_unstale":"2023-09-27T23:33:19.798813+0000","last_undegraded":"2023-09-27T23:33:19.798813+0000","last_fullsized":"2023-09-27T23:33:19.798813+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790099+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:19.790099+0000","last_peered":"2023-09-27T23:33:19.790099+0000","last_clean":"2023-09-27T23:33:19.790099+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:19.790099+0000","last_undegraded":"2023-09-27T23:33:19.790099+0000","last_fullsized":"2023-09-27T23:33:19.790099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828563+0000","last_change":"2023-09-27T23:33:19.430328+0000","last_active":"2023-09-27T23:33:19.828563+0000","last_peered":"2023-09-27T23:33:19.828563+0000","last_clean":"2023-09-27T23:33:19.828563+0000","last_became_active":"2023-09-27T23:33:19.430033+0000","last_became_peered":"2023-09-27T23:33:19.430033+0000","last_unstale":"2023-09-27T23:33:19.828563+0000","last_undegraded":"2023-09-27T23:33:19.828563+0000","last_fullsized":"2023-09-27T23:33:19.828563+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:48.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789643+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:19.789643+0000","last_peered":"2023-09-27T23:33:19.789643+0000","last_clean":"2023-09-27T23:33:19.789643+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:19.789643+0000","last_undegraded":"2023-09-27T23:33:19.789643+0000","last_fullsized":"2023-09-27T23:33:19.789643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:19.389362+0000","last_change":"2023-09-27T23:33:19.389362+0000","last_active":"2023-09-27T23:33:19.389362+0000","last_peered":"2023-09-27T23:33:19.389362+0000","last_clean":"2023-09-27T23:33:19.389362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:19.389362+0000","last_undegraded":"2023-09-27T23:33:19.389362+0000","last_fullsized":"2023-09-27T23:33:19.389362+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822697+0000","last_change":"2023-09-27T23:33:19.438806+0000","last_active":"2023-09-27T23:33:19.822697+0000","last_peered":"2023-09-27T23:33:19.822697+0000","last_clean":"2023-09-27T23:33:19.822697+0000","last_became_active":"2023-09-27T23:33:19.438541+0000","last_became_peered":"2023-09-27T23:33:19.438541+0000","last_unstale":"2023-09-27T23:33:19.822697+0000","last_undegraded":"2023-09-27T23:33:19.822697+0000","last_fullsized":"2023-09-27T23:33:19.822697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:29.683418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-09-27T23:33:15.257370+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:15.257370+0000","last_peered":"2023-09-27T23:33:15.257370+0000","last_clean":"2023-09-27T23:33:15.257370+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:15.257370+0000","last_undegraded":"2023-09-27T23:33:15.257370+0000","last_fullsized":"2023-09-27T23:33:15.257370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817658+0000","last_change":"2023-09-27T23:33:19.815111+0000","last_active":"2023-09-27T23:33:19.817658+0000","last_peered":"2023-09-27T23:33:19.817658+0000","last_clean":"2023-09-27T23:33:19.817658+0000","last_became_active":"2023-09-27T23:33:19.814866+0000","last_became_peered":"2023-09-27T23:33:19.814866+0000","last_unstale":"2023-09-27T23:33:19.817658+0000","last_undegraded":"2023-09-27T23:33:19.817658+0000","last_fullsized":"2023-09-27T23:33:19.817658+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:30.614318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:18.385355+0000","last_change":"2023-09-27T23:33:18.385355+0000","last_active":"2023-09-27T23:33:18.385355+0000","last_peered":"2023-09-27T23:33:18.385355+0000","last_clean":"2023-09-27T23:33:18.385355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:18.385355+0000","last_undegraded":"2023-09-27T23:33:18.385355+0000","last_fullsized":"2023-09-27T23:33:18.385355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788544+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:19.788544+0000","last_peered":"2023-09-27T23:33:19.788544+0000","last_clean":"2023-09-27T23:33:19.788544+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:19.788544+0000","last_undegraded":"2023-09-27T23:33:19.788544+0000","last_fullsized":"2023-09-27T23:33:19.788544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816760+0000","last_change":"2023-09-27T23:33:19.807831+0000","last_active":"2023-09-27T23:33:19.816760+0000","last_peered":"2023-09-27T23:33:19.816760+0000","last_clean":"2023-09-27T23:33:19.816760+0000","last_became_active":"2023-09-27T23:33:19.807647+0000","last_became_peered":"2023-09-27T23:33:19.807647+0000","last_unstale":"2023-09-27T23:33:19.816760+0000","last_undegraded":"2023-09-27T23:33:19.816760+0000","last_fullsized":"2023-09-27T23:33:19.816760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:34.467334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792775+0000","last_change":"2023-09-27T23:33:19.438063+0000","last_active":"2023-09-27T23:33:19.792775+0000","last_peered":"2023-09-27T23:33:19.792775+0000","last_clean":"2023-09-27T23:33:19.792775+0000","last_became_active":"2023-09-27T23:33:19.437740+0000","last_became_peered":"2023-09-27T23:33:19.437740+0000","last_unstale":"2023-09-27T23:33:19.792775+0000","last_undegraded":"2023-09-27T23:33:19.792775+0000","last_fullsized":"2023-09-27T23:33:19.792775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:01.021179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796971+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:19.796971+0000","last_peered":"2023-09-27T23:33:19.796971+0000","last_clean":"2023-09-27T23:33:19.796971+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:19.796971+0000","last_undegraded":"2023-09-27T23:33:19.796971+0000","last_fullsized":"2023-09-27T23:33:19.796971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1648,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":32,"acting":32,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509676,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":2314240,"data_stored":1484913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":1.397},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":1.486}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":2.6850000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.0469999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":2.5680000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":2.7589999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.5600000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.704},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":2.8250000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26368,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1781760,"data_stored":923892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27000832},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":1.155},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":1.018},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.345},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.4079999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509676,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37776,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94334064,"statfs":{"total":96636764160,"available":96598081536,"internally_reserved":0,"allocated":1777664,"data_stored":922468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":1.242}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.995},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.4950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509676,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":23320,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":21632,"kb_avail":94348520,"statfs":{"total":96636764160,"available":96612884480,"internally_reserved":0,"allocated":1720320,"data_stored":894545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22151168},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":1.004}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.03},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.624},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.98299999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.139},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.1160000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.087}]}]},{"osd":3,"up_from":15,"seq":64424509675,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1720320,"data_stored":894545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67000000000000004,"15min":0.69399999999999995},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.218},"max":{"1min":0.82199999999999995,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.441,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1779999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.1419999999999999,"15min":1.2869999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.40000000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.266,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.86299999999999999},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.04,"5min":1.323,"15min":1.3440000000000001},"last":0.878},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.40500000000000003},"max":{"1min":1.093,"5min":1.446,"15min":1.5720000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":0.96599999999999997},"min":{"1min":0.68899999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2430000000000001,"5min":1.613,"15min":1.613},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.073,"15min":1.0760000000000001},"min":{"1min":0.76200000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.4430000000000001,"5min":1.605,"15min":1.605},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0960000000000001,"15min":1.1140000000000001},"min":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.66000000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.571},"last":1.29},{"interface":"front","average":{"1min":1.109,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.69999999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.431,"5min":1.714,"15min":1.764},"last":1.0940000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.1160000000000001},"min":{"1min":0.753,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.353,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.91600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5509999999999999,"5min":1.6599999999999999,"15min":1.712},"last":1.23}]},{"osd":7,"last update":"Wed Sep 27 23:32:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2090000000000001,"15min":1.2310000000000001},"min":{"1min":0.84099999999999997,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.4390000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.018},{"interface":"front","average":{"1min":1.145,"5min":1.165,"15min":1.196},"min":{"1min":0.72899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3680000000000001,"5min":1.837,"15min":1.837},"last":1.194}]}]},{"osd":2,"up_from":15,"seq":64424509677,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22672,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":94349168,"statfs":{"total":96636764160,"available":96613548032,"internally_reserved":0,"allocated":1777664,"data_stored":922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.498},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.95999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.01},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37896,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":36160,"kb_avail":94333944,"statfs":{"total":96636764160,"available":96597958656,"internally_reserved":0,"allocated":1769472,"data_stored":922356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37027840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":1.026}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.171},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.302},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.119}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.27},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2470000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509677,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28892,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94342948,"statfs":{"total":96636764160,"available":96607178752,"internally_reserved":0,"allocated":2314240,"data_stored":1484913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.81000000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.002},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.095}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:24.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-09-27T23:33:24.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-09-27T23:33:24.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-09-27T23:33:24.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-09-27T23:33:24.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2023-09-27T23:33:24.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-09-27T23:33:24.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2023-09-27T23:33:24.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-09-27T23:33:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-09-27T23:33:24.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2023-09-27T23:33:24.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2023-09-27T23:33:24.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-09-27T23:33:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-09-27T23:33:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:19.389362+0000', 'last_change': '2023-09-27T23:33:19.389362+0000', 'last_active': '2023-09-27T23:33:19.389362+0000', 'last_peered': '2023-09-27T23:33:19.389362+0000', 'last_clean': '2023-09-27T23:33:19.389362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:19.389362+0000', 'last_undegraded': '2023-09-27T23:33:19.389362+0000', 'last_fullsized': '2023-09-27T23:33:19.389362+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:19.389362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2023-09-27T23:33:24.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:18.385355+0000', 'last_change': '2023-09-27T23:33:18.385355+0000', 'last_active': '2023-09-27T23:33:18.385355+0000', 'last_peered': '2023-09-27T23:33:18.385355+0000', 'last_clean': '2023-09-27T23:33:18.385355+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:18.385355+0000', 'last_undegraded': '2023-09-27T23:33:18.385355+0000', 'last_fullsized': '2023-09-27T23:33:18.385355+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:18.385355+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:33:24.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:33:24.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.603+0000 7f0941aa2700 1 Processor -- start 2023-09-27T23:33:24.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.603+0000 7f0941aa2700 1 -- start start 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f0941aa2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f093c06be10 0x7f093c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f0941aa2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0af530 con 0x7f093c0b0810 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f0941aa2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c159660 con 0x7f093c0af9c0 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f0941aa2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c159470 con 0x7f093c06be10 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f0940aa0700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f093c0b0810 0x7f093c158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60600/0 (socket says 172.21.15.64:60600) 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f0940aa0700 1 -- 172.21.15.64:0/789632420 learned_addr learned my addr 172.21.15.64:0/789632420 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:24.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093bfff700 1 --2- 172.21.15.64:0/789632420 >> v2:172.21.15.64:3300/0 conn(0x7f093c06be10 0x7f093c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093b7fe700 1 -- 172.21.15.64:0/789632420 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570580457 0 0) 0x7f093c0af530 con 0x7f093c0b0810 2023-09-27T23:33:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093b7fe700 1 -- 172.21.15.64:0/789632420 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f092c004680 con 0x7f093c0b0810 2023-09-27T23:33:24.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093b7fe700 1 -- 172.21.15.64:0/789632420 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181840807 0 0) 0x7f093c159660 con 0x7f093c0af9c0 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093bfff700 1 -- 172.21.15.64:0/789632420 >> v1:172.21.15.123:6789/0 conn(0x7f093c0af9c0 legacy=0x7f093c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093bfff700 1 -- 172.21.15.64:0/789632420 >> v1:172.21.15.64:6789/0 conn(0x7f093c0b0810 legacy=0x7f093c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093bfff700 1 -- 172.21.15.64:0/789632420 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c159940 con 0x7f093c06be10 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093bfff700 1 --2- 172.21.15.64:0/789632420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c06be10 0x7f093c0b0be0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f09280098e0 tx=0x7f09280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9234cd14483726f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.607+0000 7f093b7fe700 1 -- 172.21.15.64:0/789632420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800b6d0 con 0x7f093c06be10 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f093b7fe700 1 -- 172.21.15.64:0/789632420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092800b830 con 0x7f093c06be10 2023-09-27T23:33:24.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f093b7fe700 1 -- 172.21.15.64:0/789632420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800bbd0 con 0x7f093c06be10 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 -- 172.21.15.64:0/789632420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c06be10 msgr2=0x7f093c0b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 --2- 172.21.15.64:0/789632420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c06be10 0x7f093c0b0be0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f09280098e0 tx=0x7f09280091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 -- 172.21.15.64:0/789632420 shutdown_connections 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 --2- 172.21.15.64:0/789632420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c06be10 0x7f093c0b0be0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 -- 172.21.15.64:0/789632420 >> 172.21.15.64:0/789632420 conn(0x7f093c067bb0 msgr2=0x7f093c0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 -- 172.21.15.64:0/789632420 shutdown_connections 2023-09-27T23:33:24.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 -- 172.21.15.64:0/789632420 wait complete. 2023-09-27T23:33:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 Processor -- start 2023-09-27T23:33:24.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 -- start start 2023-09-27T23:33:24.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c06be10 0x7f093c16a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.611+0000 7f0941aa2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 0x7f093c16c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f093c0b0810 0x7f093c16cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0cb4f0 con 0x7f093c0af9c0 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0cc010 con 0x7f093c06be10 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c16a7b0 con 0x7f093c0b0810 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 0x7f093c16c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 0x7f093c16c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54226/0 (socket says 172.21.15.64:54226) 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 -- 172.21.15.64:0/755306912 learned_addr learned my addr 172.21.15.64:0/755306912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0940aa0700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c06be10 0x7f093c16a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f09412a1700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f093c0b0810 0x7f093c16cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 -- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f093c0b0810 msgr2=0x7f093c16cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f093c0b0810 0x7f093c16cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 -- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c06be10 msgr2=0x7f093c16a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c06be10 0x7f093c16a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 -- 172.21.15.64:0/755306912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928009420 con 0x7f093c0af9c0 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f093bfff700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 0x7f093c16c6a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0928013000 tx=0x7f0928017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092801f070 con 0x7f093c0af9c0 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928000c90 con 0x7f093c0af9c0 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800bbd0 con 0x7f093c0af9c0 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c0c6f30 con 0x7f093c0af9c0 2023-09-27T23:33:24.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0cb9c0 con 0x7f093c0af9c0 2023-09-27T23:33:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.615+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c0cc180 con 0x7f093c0af9c0 2023-09-27T23:33:24.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.619+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0928023070 con 0x7f093c0af9c0 2023-09-27T23:33:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.619+0000 7f0938ff9700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0920060aa0 0x7f0920062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.619+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f092807e360 con 0x7f093c0af9c0 2023-09-27T23:33:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.619+0000 7f0940aa0700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0920060aa0 0x7f0920062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.619+0000 7f0940aa0700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0920060aa0 0x7f0920062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f093c0c8250 tx=0x7f0930009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.627+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f092804e2f0 con 0x7f093c0af9c0 2023-09-27T23:33:24.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.851+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f093c04be40 con 0x7f093c0af9c0 2023-09-27T23:33:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0938ff9700 1 -- 172.21.15.64:0/755306912 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3214 (secure 0 0 0) 0x7f0928015ca0 con 0x7f093c0af9c0 2023-09-27T23:33:24.856 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:24.857 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1203,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":143,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":240984064,"bytes_avail":772853129216,"bytes_total":773094113280,"write_bytes_sec":2217,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":54,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8649}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-09-27T23:33:08.678726+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0920060aa0 msgr2=0x7f0920062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0920060aa0 0x7f0920062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f093c0c8250 tx=0x7f0930009040 comp rx=0 tx=0).stop 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 msgr2=0x7f093c16c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 0x7f093c16c6a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0928013000 tx=0x7f0928017000 comp rx=0 tx=0).stop 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 shutdown_connections 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0920060aa0 0x7f0920062f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f093c0b0810 0x7f093c16cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f093c0af9c0 0x7f093c16c6a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 --2- 172.21.15.64:0/755306912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f093c06be10 0x7f093c16a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 >> 172.21.15.64:0/755306912 conn(0x7f093c067bb0 msgr2=0x7f093c153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 shutdown_connections 2023-09-27T23:33:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.855+0000 7f0941aa2700 1 -- 172.21.15.64:0/755306912 wait complete. 2023-09-27T23:33:24.870 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:33:24.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:24.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 Processor -- start 2023-09-27T23:33:24.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 -- start start 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc140ae0 0x7f9fdc0680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fdc068630 0x7f9fdc0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc06e150 con 0x7f9fdc140710 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc068ab0 con 0x7f9fdc068630 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fe12a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc06d540 con 0x7f9fdc140ae0 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fdb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fdc068630 0x7f9fdc0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fda7fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc140ae0 0x7f9fdc0680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fdb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fdc068630 0x7f9fdc0ac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54236/0 (socket says 172.21.15.64:54236) 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fdb7fe700 1 -- 172.21.15.64:0/742321096 learned_addr learned my addr 172.21.15.64:0/742321096 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:24.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fdaffd700 1 --1- 172.21.15.64:0/742321096 >> v1:172.21.15.123:6789/0 conn(0x7f9fdc140710 0x7f9fdc0c70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:44082/0 (socket says 172.21.15.64:44082) 2023-09-27T23:33:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fda7fc700 1 -- 172.21.15.64:0/742321096 >> v2:172.21.15.64:3300/0 conn(0x7f9fdc068630 msgr2=0x7f9fdc0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:24.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fda7fc700 1 --2- 172.21.15.64:0/742321096 >> v2:172.21.15.64:3300/0 conn(0x7f9fdc068630 0x7f9fdc0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fda7fc700 1 -- 172.21.15.64:0/742321096 >> v1:172.21.15.123:6789/0 conn(0x7f9fdc140710 legacy=0x7f9fdc0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fda7fc700 1 -- 172.21.15.64:0/742321096 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc06d820 con 0x7f9fdc140ae0 2023-09-27T23:33:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fda7fc700 1 --2- 172.21.15.64:0/742321096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc140ae0 0x7f9fdc0680f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009e60 tx=0x7f9fd4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f60c6dc77beafda server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fd9ffb700 1 -- 172.21.15.64:0/742321096 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400e070 con 0x7f9fdc140ae0 2023-09-27T23:33:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fd9ffb700 1 -- 172.21.15.64:0/742321096 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd400bb60 con 0x7f9fdc140ae0 2023-09-27T23:33:24.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.967+0000 7f9fd9ffb700 1 -- 172.21.15.64:0/742321096 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4012430 con 0x7f9fdc140ae0 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- 172.21.15.64:0/742321096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc140ae0 msgr2=0x7f9fdc0680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/742321096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc140ae0 0x7f9fdc0680f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009e60 tx=0x7f9fd4009780 comp rx=0 tx=0).stop 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- 172.21.15.64:0/742321096 shutdown_connections 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/742321096 >> v2:172.21.15.64:3300/0 conn(0x7f9fdc068630 0x7f9fdc0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/742321096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc140ae0 0x7f9fdc0680f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- 172.21.15.64:0/742321096 >> 172.21.15.64:0/742321096 conn(0x7f9fdc0c8850 msgr2=0x7f9fdc0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- 172.21.15.64:0/742321096 shutdown_connections 2023-09-27T23:33:24.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- 172.21.15.64:0/742321096 wait complete. 2023-09-27T23:33:24.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 Processor -- start 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- start start 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 0x7f9fdc155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fdc140710 0x7f9fdc13de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc140ae0 0x7f9fdc0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0aeba0 con 0x7f9fdc140ae0 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0af720 con 0x7f9fdc140710 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fe12a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1562d0 con 0x7f9fdc068630 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc140ae0 0x7f9fdc0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 0x7f9fdc155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 0x7f9fdc155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:33590/0 (socket says 172.21.15.64:33590) 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 -- 172.21.15.64:0/723220280 learned_addr learned my addr 172.21.15.64:0/723220280 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:24.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fda7fc700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fdc140710 0x7f9fdc13de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 -- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fdc140710 msgr2=0x7f9fdc13de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fdc140710 0x7f9fdc13de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 -- 172.21.15.64:0/723220280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc140ae0 msgr2=0x7f9fdc0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc140ae0 0x7f9fdc0ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.971+0000 7f9fdaffd700 1 -- 172.21.15.64:0/723220280 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4009a20 con 0x7f9fdc068630 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fdaffd700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 0x7f9fdc155d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400c5c0 tx=0x7f9fc4007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4005a70 con 0x7f9fdc068630 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4005bd0 con 0x7f9fdc068630 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40070d0 con 0x7f9fdc068630 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0aedf0 con 0x7f9fdc068630 2023-09-27T23:33:24.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0af3b0 con 0x7f9fdc068630 2023-09-27T23:33:24.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9fc4007440 con 0x7f9fdc068630 2023-09-27T23:33:24.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.975+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc071120 con 0x7f9fdc068630 2023-09-27T23:33:24.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.979+0000 7f9fd2ffd700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fc0060a70 0x7f9fc0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:24.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.979+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9fc407c3d0 con 0x7f9fdc068630 2023-09-27T23:33:24.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.979+0000 7f9fda7fc700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fc0060a70 0x7f9fc0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:24.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.979+0000 7f9fda7fc700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fc0060a70 0x7f9fc0062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40097c0 tx=0x7f9fd4019000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:24.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:24.991+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9fc404c400 con 0x7f9fdc068630 2023-09-27T23:33:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.147+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9fdc0af860 con 0x7f9fc0060a70 2023-09-27T23:33:25.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.167+0000 7f9fd2ffd700 1 -- 172.21.15.64:0/723220280 <== mgr.7358 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308005 (secure 0 0 0) 0x7f9fd400be90 con 0x7f9fc0060a70 2023-09-27T23:33:25.173 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:25.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fc0060a70 msgr2=0x7f9fc0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:25.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fc0060a70 0x7f9fc0062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40097c0 tx=0x7f9fd4019000 comp rx=0 tx=0).stop 2023-09-27T23:33:25.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 msgr2=0x7f9fdc155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 0x7f9fdc155d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400c5c0 tx=0x7f9fc4007f90 comp rx=0 tx=0).stop 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 shutdown_connections 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9fc0060a70 0x7f9fc0062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc140ae0 0x7f9fdc0ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fdc140710 0x7f9fdc13de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 --2- 172.21.15.64:0/723220280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc068630 0x7f9fdc155d90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 >> 172.21.15.64:0/723220280 conn(0x7f9fdc0c8850 msgr2=0x7f9fdc0aaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 shutdown_connections 2023-09-27T23:33:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:25.171+0000 7f9fe12a2700 1 -- 172.21.15.64:0/723220280 wait complete. 2023-09-27T23:33:25.177 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:25.189 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-27T23:33:24.683850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235336,"kb_used_data":15048,"kb_used_omap":0,"kb_used_meta":220224,"kb_avail":754739384,"statfs":{"total":773094113280,"available":772853129216,"internally_reserved":0,"allocated":15409152,"data_stored":8564196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":225509376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002167"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816721+0000","last_change":"2023-09-27T23:33:19.810193+0000","last_active":"2023-09-27T23:33:19.816721+0000","last_peered":"2023-09-27T23:33:19.816721+0000","last_clean":"2023-09-27T23:33:19.816721+0000","last_became_active":"2023-09-27T23:33:19.809935+0000","last_became_peered":"2023-09-27T23:33:19.809935+0000","last_unstale":"2023-09-27T23:33:19.816721+0000","last_undegraded":"2023-09-27T23:33:19.816721+0000","last_fullsized":"2023-09-27T23:33:19.816721+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:32:44.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816570+0000","last_change":"2023-09-27T23:33:19.811311+0000","last_active":"2023-09-27T23:33:19.816570+0000","last_peered":"2023-09-27T23:33:19.816570+0000","last_clean":"2023-09-27T23:33:19.816570+0000","last_became_active":"2023-09-27T23:33:19.809810+0000","last_became_peered":"2023-09-27T23:33:19.809810+0000","last_unstale":"2023-09-27T23:33:19.816570+0000","last_undegraded":"2023-09-27T23:33:19.816570+0000","last_fullsized":"2023-09-27T23:33:19.816570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:06.194398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825973+0000","last_change":"2023-09-27T23:33:19.817990+0000","last_active":"2023-09-27T23:33:19.825973+0000","last_peered":"2023-09-27T23:33:19.825973+0000","last_clean":"2023-09-27T23:33:19.825973+0000","last_became_active":"2023-09-27T23:33:19.809746+0000","last_became_peered":"2023-09-27T23:33:19.809746+0000","last_unstale":"2023-09-27T23:33:19.825973+0000","last_undegraded":"2023-09-27T23:33:19.825973+0000","last_fullsized":"2023-09-27T23:33:19.825973+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.812775+0000","last_change":"2023-09-27T23:33:19.801379+0000","last_active":"2023-09-27T23:33:19.812775+0000","last_peered":"2023-09-27T23:33:19.812775+0000","last_clean":"2023-09-27T23:33:19.812775+0000","last_became_active":"2023-09-27T23:33:19.800971+0000","last_became_peered":"2023-09-27T23:33:19.800971+0000","last_unstale":"2023-09-27T23:33:19.812775+0000","last_undegraded":"2023-09-27T23:33:19.812775+0000","last_fullsized":"2023-09-27T23:33:19.812775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:53.280332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815298+0000","last_change":"2023-09-27T23:33:19.811325+0000","last_active":"2023-09-27T23:33:19.815298+0000","last_peered":"2023-09-27T23:33:19.815298+0000","last_clean":"2023-09-27T23:33:19.815298+0000","last_became_active":"2023-09-27T23:33:19.811029+0000","last_became_peered":"2023-09-27T23:33:19.811029+0000","last_unstale":"2023-09-27T23:33:19.815298+0000","last_undegraded":"2023-09-27T23:33:19.815298+0000","last_fullsized":"2023-09-27T23:33:19.815298+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:12:46.263357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815037+0000","last_change":"2023-09-27T23:33:19.807530+0000","last_active":"2023-09-27T23:33:19.815037+0000","last_peered":"2023-09-27T23:33:19.815037+0000","last_clean":"2023-09-27T23:33:19.815037+0000","last_became_active":"2023-09-27T23:33:19.807070+0000","last_became_peered":"2023-09-27T23:33:19.807070+0000","last_unstale":"2023-09-27T23:33:19.815037+0000","last_undegraded":"2023-09-27T23:33:19.815037+0000","last_fullsized":"2023-09-27T23:33:19.815037+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:42.285344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825591+0000","last_change":"2023-09-27T23:33:19.823986+0000","last_active":"2023-09-27T23:33:19.825591+0000","last_peered":"2023-09-27T23:33:19.825591+0000","last_clean":"2023-09-27T23:33:19.825591+0000","last_became_active":"2023-09-27T23:33:19.823794+0000","last_became_peered":"2023-09-27T23:33:19.823794+0000","last_unstale":"2023-09-27T23:33:19.825591+0000","last_undegraded":"2023-09-27T23:33:19.825591+0000","last_fullsized":"2023-09-27T23:33:19.825591+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:41.867018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826231+0000","last_change":"2023-09-27T23:33:19.819831+0000","last_active":"2023-09-27T23:33:19.826231+0000","last_peered":"2023-09-27T23:33:19.826231+0000","last_clean":"2023-09-27T23:33:19.826231+0000","last_became_active":"2023-09-27T23:33:19.819582+0000","last_became_peered":"2023-09-27T23:33:19.819582+0000","last_unstale":"2023-09-27T23:33:19.826231+0000","last_undegraded":"2023-09-27T23:33:19.826231+0000","last_fullsized":"2023-09-27T23:33:19.826231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:59.618721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826213+0000","last_change":"2023-09-27T23:33:19.818540+0000","last_active":"2023-09-27T23:33:19.826213+0000","last_peered":"2023-09-27T23:33:19.826213+0000","last_clean":"2023-09-27T23:33:19.826213+0000","last_became_active":"2023-09-27T23:33:19.818222+0000","last_became_peered":"2023-09-27T23:33:19.818222+0000","last_unstale":"2023-09-27T23:33:19.826213+0000","last_undegraded":"2023-09-27T23:33:19.826213+0000","last_fullsized":"2023-09-27T23:33:19.826213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813361+0000","last_change":"2023-09-27T23:33:19.806217+0000","last_active":"2023-09-27T23:33:19.813361+0000","last_peered":"2023-09-27T23:33:19.813361+0000","last_clean":"2023-09-27T23:33:19.813361+0000","last_became_active":"2023-09-27T23:33:19.805338+0000","last_became_peered":"2023-09-27T23:33:19.805338+0000","last_unstale":"2023-09-27T23:33:19.813361+0000","last_undegraded":"2023-09-27T23:33:19.813361+0000","last_fullsized":"2023-09-27T23:33:19.813361+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.311666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809763+0000","last_change":"2023-09-27T23:33:19.806347+0000","last_active":"2023-09-27T23:33:19.809763+0000","last_peered":"2023-09-27T23:33:19.809763+0000","last_clean":"2023-09-27T23:33:19.809763+0000","last_became_active":"2023-09-27T23:33:19.805988+0000","last_became_peered":"2023-09-27T23:33:19.805988+0000","last_unstale":"2023-09-27T23:33:19.809763+0000","last_undegraded":"2023-09-27T23:33:19.809763+0000","last_fullsized":"2023-09-27T23:33:19.809763+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:29:12.943601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826156+0000","last_change":"2023-09-27T23:33:19.810568+0000","last_active":"2023-09-27T23:33:19.826156+0000","last_peered":"2023-09-27T23:33:19.826156+0000","last_clean":"2023-09-27T23:33:19.826156+0000","last_became_active":"2023-09-27T23:33:19.810237+0000","last_became_peered":"2023-09-27T23:33:19.810237+0000","last_unstale":"2023-09-27T23:33:19.826156+0000","last_undegraded":"2023-09-27T23:33:19.826156+0000","last_fullsized":"2023-09-27T23:33:19.826156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816810+0000","last_change":"2023-09-27T23:33:19.810731+0000","last_active":"2023-09-27T23:33:19.816810+0000","last_peered":"2023-09-27T23:33:19.816810+0000","last_clean":"2023-09-27T23:33:19.816810+0000","last_became_active":"2023-09-27T23:33:19.810477+0000","last_became_peered":"2023-09-27T23:33:19.810477+0000","last_unstale":"2023-09-27T23:33:19.816810+0000","last_undegraded":"2023-09-27T23:33:19.816810+0000","last_fullsized":"2023-09-27T23:33:19.816810+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:19:28.503800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809513+0000","last_change":"2023-09-27T23:33:19.805356+0000","last_active":"2023-09-27T23:33:19.809513+0000","last_peered":"2023-09-27T23:33:19.809513+0000","last_clean":"2023-09-27T23:33:19.809513+0000","last_became_active":"2023-09-27T23:33:19.805064+0000","last_became_peered":"2023-09-27T23:33:19.805064+0000","last_unstale":"2023-09-27T23:33:19.809513+0000","last_undegraded":"2023-09-27T23:33:19.809513+0000","last_fullsized":"2023-09-27T23:33:19.809513+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:37.422975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827932+0000","last_change":"2023-09-27T23:33:19.822764+0000","last_active":"2023-09-27T23:33:19.827932+0000","last_peered":"2023-09-27T23:33:19.827932+0000","last_clean":"2023-09-27T23:33:19.827932+0000","last_became_active":"2023-09-27T23:33:19.822456+0000","last_became_peered":"2023-09-27T23:33:19.822456+0000","last_unstale":"2023-09-27T23:33:19.827932+0000","last_undegraded":"2023-09-27T23:33:19.827932+0000","last_fullsized":"2023-09-27T23:33:19.827932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824854+0000","last_change":"2023-09-27T23:33:19.818152+0000","last_active":"2023-09-27T23:33:19.824854+0000","last_peered":"2023-09-27T23:33:19.824854+0000","last_clean":"2023-09-27T23:33:19.824854+0000","last_became_active":"2023-09-27T23:33:19.817283+0000","last_became_peered":"2023-09-27T23:33:19.817283+0000","last_unstale":"2023-09-27T23:33:19.824854+0000","last_undegraded":"2023-09-27T23:33:19.824854+0000","last_fullsized":"2023-09-27T23:33:19.824854+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:36:20.392671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816834+0000","last_change":"2023-09-27T23:33:19.812064+0000","last_active":"2023-09-27T23:33:19.816834+0000","last_peered":"2023-09-27T23:33:19.816834+0000","last_clean":"2023-09-27T23:33:19.816834+0000","last_became_active":"2023-09-27T23:33:19.811676+0000","last_became_peered":"2023-09-27T23:33:19.811676+0000","last_unstale":"2023-09-27T23:33:19.816834+0000","last_undegraded":"2023-09-27T23:33:19.816834+0000","last_fullsized":"2023-09-27T23:33:19.816834+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:33:54.723421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824872+0000","last_change":"2023-09-27T23:33:19.818291+0000","last_active":"2023-09-27T23:33:19.824872+0000","last_peered":"2023-09-27T23:33:19.824872+0000","last_clean":"2023-09-27T23:33:19.824872+0000","last_became_active":"2023-09-27T23:33:19.817949+0000","last_became_peered":"2023-09-27T23:33:19.817949+0000","last_unstale":"2023-09-27T23:33:19.824872+0000","last_undegraded":"2023-09-27T23:33:19.824872+0000","last_fullsized":"2023-09-27T23:33:19.824872+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:31:08.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816188+0000","last_change":"2023-09-27T23:33:19.811585+0000","last_active":"2023-09-27T23:33:19.816188+0000","last_peered":"2023-09-27T23:33:19.816188+0000","last_clean":"2023-09-27T23:33:19.816188+0000","last_became_active":"2023-09-27T23:33:19.810377+0000","last_became_peered":"2023-09-27T23:33:19.810377+0000","last_unstale":"2023-09-27T23:33:19.816188+0000","last_undegraded":"2023-09-27T23:33:19.816188+0000","last_fullsized":"2023-09-27T23:33:19.816188+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:42.285344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824811+0000","last_change":"2023-09-27T23:33:19.817725+0000","last_active":"2023-09-27T23:33:19.824811+0000","last_peered":"2023-09-27T23:33:19.824811+0000","last_clean":"2023-09-27T23:33:19.824811+0000","last_became_active":"2023-09-27T23:33:19.817387+0000","last_became_peered":"2023-09-27T23:33:19.817387+0000","last_unstale":"2023-09-27T23:33:19.824811+0000","last_undegraded":"2023-09-27T23:33:19.824811+0000","last_fullsized":"2023-09-27T23:33:19.824811+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:34.864941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.808788+0000","last_change":"2023-09-27T23:33:19.803176+0000","last_active":"2023-09-27T23:33:19.808788+0000","last_peered":"2023-09-27T23:33:19.808788+0000","last_clean":"2023-09-27T23:33:19.808788+0000","last_became_active":"2023-09-27T23:33:19.802860+0000","last_became_peered":"2023-09-27T23:33:19.802860+0000","last_unstale":"2023-09-27T23:33:19.808788+0000","last_undegraded":"2023-09-27T23:33:19.808788+0000","last_fullsized":"2023-09-27T23:33:19.808788+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.807080+0000","last_change":"2023-09-27T23:33:19.805641+0000","last_active":"2023-09-27T23:33:19.807080+0000","last_peered":"2023-09-27T23:33:19.807080+0000","last_clean":"2023-09-27T23:33:19.807080+0000","last_became_active":"2023-09-27T23:33:19.805378+0000","last_became_peered":"2023-09-27T23:33:19.805378+0000","last_unstale":"2023-09-27T23:33:19.807080+0000","last_undegraded":"2023-09-27T23:33:19.807080+0000","last_fullsized":"2023-09-27T23:33:19.807080+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:25:59.533646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826283+0000","last_change":"2023-09-27T23:33:19.818051+0000","last_active":"2023-09-27T23:33:19.826283+0000","last_peered":"2023-09-27T23:33:19.826283+0000","last_clean":"2023-09-27T23:33:19.826283+0000","last_became_active":"2023-09-27T23:33:19.809795+0000","last_became_peered":"2023-09-27T23:33:19.809795+0000","last_unstale":"2023-09-27T23:33:19.826283+0000","last_undegraded":"2023-09-27T23:33:19.826283+0000","last_fullsized":"2023-09-27T23:33:19.826283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814736+0000","last_change":"2023-09-27T23:33:19.808749+0000","last_active":"2023-09-27T23:33:19.814736+0000","last_peered":"2023-09-27T23:33:19.814736+0000","last_clean":"2023-09-27T23:33:19.814736+0000","last_became_active":"2023-09-27T23:33:19.808480+0000","last_became_peered":"2023-09-27T23:33:19.808480+0000","last_unstale":"2023-09-27T23:33:19.814736+0000","last_undegraded":"2023-09-27T23:33:19.814736+0000","last_fullsized":"2023-09-27T23:33:19.814736+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:34:58.552571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.804587+0000","last_change":"2023-09-27T23:33:19.798526+0000","last_active":"2023-09-27T23:33:19.804587+0000","last_peered":"2023-09-27T23:33:19.804587+0000","last_clean":"2023-09-27T23:33:19.804587+0000","last_became_active":"2023-09-27T23:33:19.798226+0000","last_became_peered":"2023-09-27T23:33:19.798226+0000","last_unstale":"2023-09-27T23:33:19.804587+0000","last_undegraded":"2023-09-27T23:33:19.804587+0000","last_fullsized":"2023-09-27T23:33:19.804587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:13:28.298592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815408+0000","last_change":"2023-09-27T23:33:19.805590+0000","last_active":"2023-09-27T23:33:19.815408+0000","last_peered":"2023-09-27T23:33:19.815408+0000","last_clean":"2023-09-27T23:33:19.815408+0000","last_became_active":"2023-09-27T23:33:19.805319+0000","last_became_peered":"2023-09-27T23:33:19.805319+0000","last_unstale":"2023-09-27T23:33:19.815408+0000","last_undegraded":"2023-09-27T23:33:19.815408+0000","last_fullsized":"2023-09-27T23:33:19.815408+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:33:39.999069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815678+0000","last_change":"2023-09-27T23:33:19.805843+0000","last_active":"2023-09-27T23:33:19.815678+0000","last_peered":"2023-09-27T23:33:19.815678+0000","last_clean":"2023-09-27T23:33:19.815678+0000","last_became_active":"2023-09-27T23:33:19.805571+0000","last_became_peered":"2023-09-27T23:33:19.805571+0000","last_unstale":"2023-09-27T23:33:19.815678+0000","last_undegraded":"2023-09-27T23:33:19.815678+0000","last_fullsized":"2023-09-27T23:33:19.815678+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:49.075742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.823630+0000","last_change":"2023-09-27T23:33:19.819123+0000","last_active":"2023-09-27T23:33:19.823630+0000","last_peered":"2023-09-27T23:33:19.823630+0000","last_clean":"2023-09-27T23:33:19.823630+0000","last_became_active":"2023-09-27T23:33:19.818840+0000","last_became_peered":"2023-09-27T23:33:19.818840+0000","last_unstale":"2023-09-27T23:33:19.823630+0000","last_undegraded":"2023-09-27T23:33:19.823630+0000","last_fullsized":"2023-09-27T23:33:19.823630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:00.849910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827890+0000","last_change":"2023-09-27T23:33:19.822665+0000","last_active":"2023-09-27T23:33:19.827890+0000","last_peered":"2023-09-27T23:33:19.827890+0000","last_clean":"2023-09-27T23:33:19.827890+0000","last_became_active":"2023-09-27T23:33:19.822350+0000","last_became_peered":"2023-09-27T23:33:19.822350+0000","last_unstale":"2023-09-27T23:33:19.827890+0000","last_undegraded":"2023-09-27T23:33:19.827890+0000","last_fullsized":"2023-09-27T23:33:19.827890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:30:36.041799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827909+0000","last_change":"2023-09-27T23:33:19.822700+0000","last_active":"2023-09-27T23:33:19.827909+0000","last_peered":"2023-09-27T23:33:19.827909+0000","last_clean":"2023-09-27T23:33:19.827909+0000","last_became_active":"2023-09-27T23:33:19.822405+0000","last_became_peered":"2023-09-27T23:33:19.822405+0000","last_unstale":"2023-09-27T23:33:19.827909+0000","last_undegraded":"2023-09-27T23:33:19.827909+0000","last_fullsized":"2023-09-27T23:33:19.827909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:22.750767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814349+0000","last_change":"2023-09-27T23:33:19.806164+0000","last_active":"2023-09-27T23:33:19.814349+0000","last_peered":"2023-09-27T23:33:19.814349+0000","last_clean":"2023-09-27T23:33:19.814349+0000","last_became_active":"2023-09-27T23:33:19.805693+0000","last_became_peered":"2023-09-27T23:33:19.805693+0000","last_unstale":"2023-09-27T23:33:19.814349+0000","last_undegraded":"2023-09-27T23:33:19.814349+0000","last_fullsized":"2023-09-27T23:33:19.814349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:43.937071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.820374+0000","last_change":"2023-09-27T23:33:19.816258+0000","last_active":"2023-09-27T23:33:19.820374+0000","last_peered":"2023-09-27T23:33:19.820374+0000","last_clean":"2023-09-27T23:33:19.820374+0000","last_became_active":"2023-09-27T23:33:19.816064+0000","last_became_peered":"2023-09-27T23:33:19.816064+0000","last_unstale":"2023-09-27T23:33:19.820374+0000","last_undegraded":"2023-09-27T23:33:19.820374+0000","last_fullsized":"2023-09-27T23:33:19.820374+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:34:58.552571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.820067+0000","last_change":"2023-09-27T23:33:19.815313+0000","last_active":"2023-09-27T23:33:19.820067+0000","last_peered":"2023-09-27T23:33:19.820067+0000","last_clean":"2023-09-27T23:33:19.820067+0000","last_became_active":"2023-09-27T23:33:19.815025+0000","last_became_peered":"2023-09-27T23:33:19.815025+0000","last_unstale":"2023-09-27T23:33:19.820067+0000","last_undegraded":"2023-09-27T23:33:19.820067+0000","last_fullsized":"2023-09-27T23:33:19.820067+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:19:49.113506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"142'4","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.808019+0000","last_change":"2023-09-27T23:33:19.431509+0000","last_active":"2023-09-27T23:33:19.808019+0000","last_peered":"2023-09-27T23:33:19.808019+0000","last_clean":"2023-09-27T23:33:19.808019+0000","last_became_active":"2023-09-27T23:33:19.431232+0000","last_became_peered":"2023-09-27T23:33:19.431232+0000","last_unstale":"2023-09-27T23:33:19.808019+0000","last_undegraded":"2023-09-27T23:33:19.808019+0000","last_fullsized":"2023-09-27T23:33:19.808019+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:29.610311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825923+0000","last_change":"2023-09-27T23:33:19.810331+0000","last_active":"2023-09-27T23:33:19.825923+0000","last_peered":"2023-09-27T23:33:19.825923+0000","last_clean":"2023-09-27T23:33:19.825923+0000","last_became_active":"2023-09-27T23:33:19.810041+0000","last_became_peered":"2023-09-27T23:33:19.810041+0000","last_unstale":"2023-09-27T23:33:19.825923+0000","last_undegraded":"2023-09-27T23:33:19.825923+0000","last_fullsized":"2023-09-27T23:33:19.825923+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814781+0000","last_change":"2023-09-27T23:33:19.436486+0000","last_active":"2023-09-27T23:33:19.814781+0000","last_peered":"2023-09-27T23:33:19.814781+0000","last_clean":"2023-09-27T23:33:19.814781+0000","last_became_active":"2023-09-27T23:33:19.436056+0000","last_became_peered":"2023-09-27T23:33:19.436056+0000","last_unstale":"2023-09-27T23:33:19.814781+0000","last_undegraded":"2023-09-27T23:33:19.814781+0000","last_fullsized":"2023-09-27T23:33:19.814781+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815222+0000","last_change":"2023-09-27T23:33:19.812688+0000","last_active":"2023-09-27T23:33:19.815222+0000","last_peered":"2023-09-27T23:33:19.815222+0000","last_clean":"2023-09-27T23:33:19.815222+0000","last_became_active":"2023-09-27T23:33:19.812408+0000","last_became_peered":"2023-09-27T23:33:19.812408+0000","last_unstale":"2023-09-27T23:33:19.815222+0000","last_undegraded":"2023-09-27T23:33:19.815222+0000","last_fullsized":"2023-09-27T23:33:19.815222+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815969+0000","last_change":"2023-09-27T23:33:19.436742+0000","last_active":"2023-09-27T23:33:19.815969+0000","last_peered":"2023-09-27T23:33:19.815969+0000","last_clean":"2023-09-27T23:33:19.815969+0000","last_became_active":"2023-09-27T23:33:19.436395+0000","last_became_peered":"2023-09-27T23:33:19.436395+0000","last_unstale":"2023-09-27T23:33:19.815969+0000","last_undegraded":"2023-09-27T23:33:19.815969+0000","last_fullsized":"2023-09-27T23:33:19.815969+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:16:48.236035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.808536+0000","last_change":"2023-09-27T23:33:19.802405+0000","last_active":"2023-09-27T23:33:19.808536+0000","last_peered":"2023-09-27T23:33:19.808536+0000","last_clean":"2023-09-27T23:33:19.808536+0000","last_became_active":"2023-09-27T23:33:19.802115+0000","last_became_peered":"2023-09-27T23:33:19.802115+0000","last_unstale":"2023-09-27T23:33:19.808536+0000","last_undegraded":"2023-09-27T23:33:19.808536+0000","last_fullsized":"2023-09-27T23:33:19.808536+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809160+0000","last_change":"2023-09-27T23:33:19.427279+0000","last_active":"2023-09-27T23:33:19.809160+0000","last_peered":"2023-09-27T23:33:19.809160+0000","last_clean":"2023-09-27T23:33:19.809160+0000","last_became_active":"2023-09-27T23:33:19.426915+0000","last_became_peered":"2023-09-27T23:33:19.426915+0000","last_unstale":"2023-09-27T23:33:19.809160+0000","last_undegraded":"2023-09-27T23:33:19.809160+0000","last_fullsized":"2023-09-27T23:33:19.809160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:54.341042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819812+0000","last_change":"2023-09-27T23:33:19.811218+0000","last_active":"2023-09-27T23:33:19.819812+0000","last_peered":"2023-09-27T23:33:19.819812+0000","last_clean":"2023-09-27T23:33:19.819812+0000","last_became_active":"2023-09-27T23:33:19.810963+0000","last_became_peered":"2023-09-27T23:33:19.810963+0000","last_unstale":"2023-09-27T23:33:19.819812+0000","last_undegraded":"2023-09-27T23:33:19.819812+0000","last_fullsized":"2023-09-27T23:33:19.819812+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:49:04.290739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816372+0000","last_change":"2023-09-27T23:33:19.433429+0000","last_active":"2023-09-27T23:33:19.816372+0000","last_peered":"2023-09-27T23:33:19.816372+0000","last_clean":"2023-09-27T23:33:19.816372+0000","last_became_active":"2023-09-27T23:33:19.433030+0000","last_became_peered":"2023-09-27T23:33:19.433030+0000","last_unstale":"2023-09-27T23:33:19.816372+0000","last_undegraded":"2023-09-27T23:33:19.816372+0000","last_fullsized":"2023-09-27T23:33:19.816372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:13:30.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814913+0000","last_change":"2023-09-27T23:33:19.810319+0000","last_active":"2023-09-27T23:33:19.814913+0000","last_peered":"2023-09-27T23:33:19.814913+0000","last_clean":"2023-09-27T23:33:19.814913+0000","last_became_active":"2023-09-27T23:33:19.810027+0000","last_became_peered":"2023-09-27T23:33:19.810027+0000","last_unstale":"2023-09-27T23:33:19.814913+0000","last_undegraded":"2023-09-27T23:33:19.814913+0000","last_fullsized":"2023-09-27T23:33:19.814913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:59.536848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824684+0000","last_change":"2023-09-27T23:33:19.434695+0000","last_active":"2023-09-27T23:33:19.824684+0000","last_peered":"2023-09-27T23:33:19.824684+0000","last_clean":"2023-09-27T23:33:19.824684+0000","last_became_active":"2023-09-27T23:33:19.434312+0000","last_became_peered":"2023-09-27T23:33:19.434312+0000","last_unstale":"2023-09-27T23:33:19.824684+0000","last_undegraded":"2023-09-27T23:33:19.824684+0000","last_fullsized":"2023-09-27T23:33:19.824684+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:58.614714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826212+0000","last_change":"2023-09-27T23:33:19.819783+0000","last_active":"2023-09-27T23:33:19.826212+0000","last_peered":"2023-09-27T23:33:19.826212+0000","last_clean":"2023-09-27T23:33:19.826212+0000","last_became_active":"2023-09-27T23:33:19.819564+0000","last_became_peered":"2023-09-27T23:33:19.819564+0000","last_unstale":"2023-09-27T23:33:19.826212+0000","last_undegraded":"2023-09-27T23:33:19.826212+0000","last_fullsized":"2023-09-27T23:33:19.826212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:37.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796561+0000","last_change":"2023-09-27T23:33:19.429166+0000","last_active":"2023-09-27T23:33:19.796561+0000","last_peered":"2023-09-27T23:33:19.796561+0000","last_clean":"2023-09-27T23:33:19.796561+0000","last_became_active":"2023-09-27T23:33:19.428861+0000","last_became_peered":"2023-09-27T23:33:19.428861+0000","last_unstale":"2023-09-27T23:33:19.796561+0000","last_undegraded":"2023-09-27T23:33:19.796561+0000","last_fullsized":"2023-09-27T23:33:19.796561+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:10:34.625899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813572+0000","last_change":"2023-09-27T23:33:19.809091+0000","last_active":"2023-09-27T23:33:19.813572+0000","last_peered":"2023-09-27T23:33:19.813572+0000","last_clean":"2023-09-27T23:33:19.813572+0000","last_became_active":"2023-09-27T23:33:19.808763+0000","last_became_peered":"2023-09-27T23:33:19.808763+0000","last_unstale":"2023-09-27T23:33:19.813572+0000","last_undegraded":"2023-09-27T23:33:19.813572+0000","last_fullsized":"2023-09-27T23:33:19.813572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:41.828993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788226+0000","last_change":"2023-09-27T23:33:19.428241+0000","last_active":"2023-09-27T23:33:19.788226+0000","last_peered":"2023-09-27T23:33:19.788226+0000","last_clean":"2023-09-27T23:33:19.788226+0000","last_became_active":"2023-09-27T23:33:19.427908+0000","last_became_peered":"2023-09-27T23:33:19.427908+0000","last_unstale":"2023-09-27T23:33:19.788226+0000","last_undegraded":"2023-09-27T23:33:19.788226+0000","last_fullsized":"2023-09-27T23:33:19.788226+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:49.753965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824232+0000","last_change":"2023-09-27T23:33:19.808815+0000","last_active":"2023-09-27T23:33:19.824232+0000","last_peered":"2023-09-27T23:33:19.824232+0000","last_clean":"2023-09-27T23:33:19.824232+0000","last_became_active":"2023-09-27T23:33:19.808532+0000","last_became_peered":"2023-09-27T23:33:19.808532+0000","last_unstale":"2023-09-27T23:33:19.824232+0000","last_undegraded":"2023-09-27T23:33:19.824232+0000","last_fullsized":"2023-09-27T23:33:19.824232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815149+0000","last_change":"2023-09-27T23:33:19.435202+0000","last_active":"2023-09-27T23:33:19.815149+0000","last_peered":"2023-09-27T23:33:19.815149+0000","last_clean":"2023-09-27T23:33:19.815149+0000","last_became_active":"2023-09-27T23:33:19.432656+0000","last_became_peered":"2023-09-27T23:33:19.432656+0000","last_unstale":"2023-09-27T23:33:19.815149+0000","last_undegraded":"2023-09-27T23:33:19.815149+0000","last_fullsized":"2023-09-27T23:33:19.815149+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817212+0000","last_change":"2023-09-27T23:33:19.811187+0000","last_active":"2023-09-27T23:33:19.817212+0000","last_peered":"2023-09-27T23:33:19.817212+0000","last_clean":"2023-09-27T23:33:19.817212+0000","last_became_active":"2023-09-27T23:33:19.810921+0000","last_became_peered":"2023-09-27T23:33:19.810921+0000","last_unstale":"2023-09-27T23:33:19.817212+0000","last_undegraded":"2023-09-27T23:33:19.817212+0000","last_fullsized":"2023-09-27T23:33:19.817212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:58.020074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.821969+0000","last_change":"2023-09-27T23:33:19.427776+0000","last_active":"2023-09-27T23:33:19.821969+0000","last_peered":"2023-09-27T23:33:19.821969+0000","last_clean":"2023-09-27T23:33:19.821969+0000","last_became_active":"2023-09-27T23:33:19.427494+0000","last_became_peered":"2023-09-27T23:33:19.427494+0000","last_unstale":"2023-09-27T23:33:19.821969+0000","last_undegraded":"2023-09-27T23:33:19.821969+0000","last_fullsized":"2023-09-27T23:33:19.821969+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:30.740099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815316+0000","last_change":"2023-09-27T23:33:19.805962+0000","last_active":"2023-09-27T23:33:19.815316+0000","last_peered":"2023-09-27T23:33:19.815316+0000","last_clean":"2023-09-27T23:33:19.815316+0000","last_became_active":"2023-09-27T23:33:19.805667+0000","last_became_peered":"2023-09-27T23:33:19.805667+0000","last_unstale":"2023-09-27T23:33:19.815316+0000","last_undegraded":"2023-09-27T23:33:19.815316+0000","last_fullsized":"2023-09-27T23:33:19.815316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:30:21.924258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822497+0000","last_change":"2023-09-27T23:33:19.438960+0000","last_active":"2023-09-27T23:33:19.822497+0000","last_peered":"2023-09-27T23:33:19.822497+0000","last_clean":"2023-09-27T23:33:19.822497+0000","last_became_active":"2023-09-27T23:33:19.438634+0000","last_became_peered":"2023-09-27T23:33:19.438634+0000","last_unstale":"2023-09-27T23:33:19.822497+0000","last_undegraded":"2023-09-27T23:33:19.822497+0000","last_fullsized":"2023-09-27T23:33:19.822497+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:09:47.783943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819500+0000","last_change":"2023-09-27T23:33:19.816975+0000","last_active":"2023-09-27T23:33:19.819500+0000","last_peered":"2023-09-27T23:33:19.819500+0000","last_clean":"2023-09-27T23:33:19.819500+0000","last_became_active":"2023-09-27T23:33:19.816573+0000","last_became_peered":"2023-09-27T23:33:19.816573+0000","last_unstale":"2023-09-27T23:33:19.819500+0000","last_undegraded":"2023-09-27T23:33:19.819500+0000","last_fullsized":"2023-09-27T23:33:19.819500+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:46:14.483620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790958+0000","last_change":"2023-09-27T23:33:19.435177+0000","last_active":"2023-09-27T23:33:19.790958+0000","last_peered":"2023-09-27T23:33:19.790958+0000","last_clean":"2023-09-27T23:33:19.790958+0000","last_became_active":"2023-09-27T23:33:19.432614+0000","last_became_peered":"2023-09-27T23:33:19.432614+0000","last_unstale":"2023-09-27T23:33:19.790958+0000","last_undegraded":"2023-09-27T23:33:19.790958+0000","last_fullsized":"2023-09-27T23:33:19.790958+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:07.369858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816845+0000","last_change":"2023-09-27T23:33:19.810499+0000","last_active":"2023-09-27T23:33:19.816845+0000","last_peered":"2023-09-27T23:33:19.816845+0000","last_clean":"2023-09-27T23:33:19.816845+0000","last_became_active":"2023-09-27T23:33:19.810241+0000","last_became_peered":"2023-09-27T23:33:19.810241+0000","last_unstale":"2023-09-27T23:33:19.816845+0000","last_undegraded":"2023-09-27T23:33:19.816845+0000","last_fullsized":"2023-09-27T23:33:19.816845+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:52:01.755843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816627+0000","last_change":"2023-09-27T23:33:19.436255+0000","last_active":"2023-09-27T23:33:19.816627+0000","last_peered":"2023-09-27T23:33:19.816627+0000","last_clean":"2023-09-27T23:33:19.816627+0000","last_became_active":"2023-09-27T23:33:19.435903+0000","last_became_peered":"2023-09-27T23:33:19.435903+0000","last_unstale":"2023-09-27T23:33:19.816627+0000","last_undegraded":"2023-09-27T23:33:19.816627+0000","last_fullsized":"2023-09-27T23:33:19.816627+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:01.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826471+0000","last_change":"2023-09-27T23:33:19.824931+0000","last_active":"2023-09-27T23:33:19.826471+0000","last_peered":"2023-09-27T23:33:19.826471+0000","last_clean":"2023-09-27T23:33:19.826471+0000","last_became_active":"2023-09-27T23:33:19.824656+0000","last_became_peered":"2023-09-27T23:33:19.824656+0000","last_unstale":"2023-09-27T23:33:19.826471+0000","last_undegraded":"2023-09-27T23:33:19.826471+0000","last_fullsized":"2023-09-27T23:33:19.826471+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:18:51.294221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.803050+0000","last_change":"2023-09-27T23:33:19.430029+0000","last_active":"2023-09-27T23:33:19.803050+0000","last_peered":"2023-09-27T23:33:19.803050+0000","last_clean":"2023-09-27T23:33:19.803050+0000","last_became_active":"2023-09-27T23:33:19.429572+0000","last_became_peered":"2023-09-27T23:33:19.429572+0000","last_unstale":"2023-09-27T23:33:19.803050+0000","last_undegraded":"2023-09-27T23:33:19.803050+0000","last_fullsized":"2023-09-27T23:33:19.803050+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:36.384182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813432+0000","last_change":"2023-09-27T23:33:19.809133+0000","last_active":"2023-09-27T23:33:19.813432+0000","last_peered":"2023-09-27T23:33:19.813432+0000","last_clean":"2023-09-27T23:33:19.813432+0000","last_became_active":"2023-09-27T23:33:19.806861+0000","last_became_peered":"2023-09-27T23:33:19.806861+0000","last_unstale":"2023-09-27T23:33:19.813432+0000","last_undegraded":"2023-09-27T23:33:19.813432+0000","last_fullsized":"2023-09-27T23:33:19.813432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:46:39.679217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.795787+0000","last_change":"2023-09-27T23:33:19.432595+0000","last_active":"2023-09-27T23:33:19.795787+0000","last_peered":"2023-09-27T23:33:19.795787+0000","last_clean":"2023-09-27T23:33:19.795787+0000","last_became_active":"2023-09-27T23:33:19.432271+0000","last_became_peered":"2023-09-27T23:33:19.432271+0000","last_unstale":"2023-09-27T23:33:19.795787+0000","last_undegraded":"2023-09-27T23:33:19.795787+0000","last_fullsized":"2023-09-27T23:33:19.795787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:16.879676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814323+0000","last_change":"2023-09-27T23:33:19.805969+0000","last_active":"2023-09-27T23:33:19.814323+0000","last_peered":"2023-09-27T23:33:19.814323+0000","last_clean":"2023-09-27T23:33:19.814323+0000","last_became_active":"2023-09-27T23:33:19.805294+0000","last_became_peered":"2023-09-27T23:33:19.805294+0000","last_unstale":"2023-09-27T23:33:19.814323+0000","last_undegraded":"2023-09-27T23:33:19.814323+0000","last_fullsized":"2023-09-27T23:33:19.814323+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.794866+0000","last_change":"2023-09-27T23:33:19.427838+0000","last_active":"2023-09-27T23:33:19.794866+0000","last_peered":"2023-09-27T23:33:19.794866+0000","last_clean":"2023-09-27T23:33:19.794866+0000","last_became_active":"2023-09-27T23:33:19.427524+0000","last_became_peered":"2023-09-27T23:33:19.427524+0000","last_unstale":"2023-09-27T23:33:19.794866+0000","last_undegraded":"2023-09-27T23:33:19.794866+0000","last_fullsized":"2023-09-27T23:33:19.794866+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:07:22.859377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816803+0000","last_change":"2023-09-27T23:33:19.811653+0000","last_active":"2023-09-27T23:33:19.816803+0000","last_peered":"2023-09-27T23:33:19.816803+0000","last_clean":"2023-09-27T23:33:19.816803+0000","last_became_active":"2023-09-27T23:33:19.811327+0000","last_became_peered":"2023-09-27T23:33:19.811327+0000","last_unstale":"2023-09-27T23:33:19.816803+0000","last_undegraded":"2023-09-27T23:33:19.816803+0000","last_fullsized":"2023-09-27T23:33:19.816803+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:25:37.144907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815834+0000","last_change":"2023-09-27T23:33:19.434317+0000","last_active":"2023-09-27T23:33:19.815834+0000","last_peered":"2023-09-27T23:33:19.815834+0000","last_clean":"2023-09-27T23:33:19.815834+0000","last_became_active":"2023-09-27T23:33:19.434002+0000","last_became_peered":"2023-09-27T23:33:19.434002+0000","last_unstale":"2023-09-27T23:33:19.815834+0000","last_undegraded":"2023-09-27T23:33:19.815834+0000","last_fullsized":"2023-09-27T23:33:19.815834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827942+0000","last_change":"2023-09-27T23:33:19.822884+0000","last_active":"2023-09-27T23:33:19.827942+0000","last_peered":"2023-09-27T23:33:19.827942+0000","last_clean":"2023-09-27T23:33:19.827942+0000","last_became_active":"2023-09-27T23:33:19.822472+0000","last_became_peered":"2023-09-27T23:33:19.822472+0000","last_unstale":"2023-09-27T23:33:19.827942+0000","last_undegraded":"2023-09-27T23:33:19.827942+0000","last_fullsized":"2023-09-27T23:33:19.827942+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:52:33.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793144+0000","last_change":"2023-09-27T23:33:19.430327+0000","last_active":"2023-09-27T23:33:19.793144+0000","last_peered":"2023-09-27T23:33:19.793144+0000","last_clean":"2023-09-27T23:33:19.793144+0000","last_became_active":"2023-09-27T23:33:19.430043+0000","last_became_peered":"2023-09-27T23:33:19.430043+0000","last_unstale":"2023-09-27T23:33:19.793144+0000","last_undegraded":"2023-09-27T23:33:19.793144+0000","last_fullsized":"2023-09-27T23:33:19.793144+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:57.016067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809717+0000","last_change":"2023-09-27T23:33:19.804614+0000","last_active":"2023-09-27T23:33:19.809717+0000","last_peered":"2023-09-27T23:33:19.809717+0000","last_clean":"2023-09-27T23:33:19.809717+0000","last_became_active":"2023-09-27T23:33:19.804328+0000","last_became_peered":"2023-09-27T23:33:19.804328+0000","last_unstale":"2023-09-27T23:33:19.809717+0000","last_undegraded":"2023-09-27T23:33:19.809717+0000","last_fullsized":"2023-09-27T23:33:19.809717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:06.058928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788038+0000","last_change":"2023-09-27T23:33:19.422966+0000","last_active":"2023-09-27T23:33:19.788038+0000","last_peered":"2023-09-27T23:33:19.788038+0000","last_clean":"2023-09-27T23:33:19.788038+0000","last_became_active":"2023-09-27T23:33:19.422649+0000","last_became_peered":"2023-09-27T23:33:19.422649+0000","last_unstale":"2023-09-27T23:33:19.788038+0000","last_undegraded":"2023-09-27T23:33:19.788038+0000","last_fullsized":"2023-09-27T23:33:19.788038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:04:02.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.820346+0000","last_change":"2023-09-27T23:33:19.815980+0000","last_active":"2023-09-27T23:33:19.820346+0000","last_peered":"2023-09-27T23:33:19.820346+0000","last_clean":"2023-09-27T23:33:19.820346+0000","last_became_active":"2023-09-27T23:33:19.815700+0000","last_became_peered":"2023-09-27T23:33:19.815700+0000","last_unstale":"2023-09-27T23:33:19.820346+0000","last_undegraded":"2023-09-27T23:33:19.820346+0000","last_fullsized":"2023-09-27T23:33:19.820346+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:41:11.095627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793216+0000","last_change":"2023-09-27T23:33:19.434165+0000","last_active":"2023-09-27T23:33:19.793216+0000","last_peered":"2023-09-27T23:33:19.793216+0000","last_clean":"2023-09-27T23:33:19.793216+0000","last_became_active":"2023-09-27T23:33:19.433869+0000","last_became_peered":"2023-09-27T23:33:19.433869+0000","last_unstale":"2023-09-27T23:33:19.793216+0000","last_undegraded":"2023-09-27T23:33:19.793216+0000","last_fullsized":"2023-09-27T23:33:19.793216+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:52.914410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814499+0000","last_change":"2023-09-27T23:33:19.808152+0000","last_active":"2023-09-27T23:33:19.814499+0000","last_peered":"2023-09-27T23:33:19.814499+0000","last_clean":"2023-09-27T23:33:19.814499+0000","last_became_active":"2023-09-27T23:33:19.807830+0000","last_became_peered":"2023-09-27T23:33:19.807830+0000","last_unstale":"2023-09-27T23:33:19.814499+0000","last_undegraded":"2023-09-27T23:33:19.814499+0000","last_fullsized":"2023-09-27T23:33:19.814499+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.786802+0000","last_change":"2023-09-27T23:33:19.424649+0000","last_active":"2023-09-27T23:33:19.786802+0000","last_peered":"2023-09-27T23:33:19.786802+0000","last_clean":"2023-09-27T23:33:19.786802+0000","last_became_active":"2023-09-27T23:33:19.424230+0000","last_became_peered":"2023-09-27T23:33:19.424230+0000","last_unstale":"2023-09-27T23:33:19.786802+0000","last_undegraded":"2023-09-27T23:33:19.786802+0000","last_fullsized":"2023-09-27T23:33:19.786802+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:46:54.150917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814735+0000","last_change":"2023-09-27T23:33:19.807080+0000","last_active":"2023-09-27T23:33:19.814735+0000","last_peered":"2023-09-27T23:33:19.814735+0000","last_clean":"2023-09-27T23:33:19.814735+0000","last_became_active":"2023-09-27T23:33:19.806755+0000","last_became_peered":"2023-09-27T23:33:19.806755+0000","last_unstale":"2023-09-27T23:33:19.814735+0000","last_undegraded":"2023-09-27T23:33:19.814735+0000","last_fullsized":"2023-09-27T23:33:19.814735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:03:57.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793447+0000","last_change":"2023-09-27T23:33:19.433717+0000","last_active":"2023-09-27T23:33:19.793447+0000","last_peered":"2023-09-27T23:33:19.793447+0000","last_clean":"2023-09-27T23:33:19.793447+0000","last_became_active":"2023-09-27T23:33:19.433373+0000","last_became_peered":"2023-09-27T23:33:19.433373+0000","last_unstale":"2023-09-27T23:33:19.793447+0000","last_undegraded":"2023-09-27T23:33:19.793447+0000","last_fullsized":"2023-09-27T23:33:19.793447+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:05.142294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828131+0000","last_change":"2023-09-27T23:33:19.821633+0000","last_active":"2023-09-27T23:33:19.828131+0000","last_peered":"2023-09-27T23:33:19.828131+0000","last_clean":"2023-09-27T23:33:19.828131+0000","last_became_active":"2023-09-27T23:33:19.821291+0000","last_became_peered":"2023-09-27T23:33:19.821291+0000","last_unstale":"2023-09-27T23:33:19.828131+0000","last_undegraded":"2023-09-27T23:33:19.828131+0000","last_fullsized":"2023-09-27T23:33:19.828131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:46:20.943175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.795895+0000","last_change":"2023-09-27T23:33:19.432266+0000","last_active":"2023-09-27T23:33:19.795895+0000","last_peered":"2023-09-27T23:33:19.795895+0000","last_clean":"2023-09-27T23:33:19.795895+0000","last_became_active":"2023-09-27T23:33:19.430799+0000","last_became_peered":"2023-09-27T23:33:19.430799+0000","last_unstale":"2023-09-27T23:33:19.795895+0000","last_undegraded":"2023-09-27T23:33:19.795895+0000","last_fullsized":"2023-09-27T23:33:19.795895+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:28.682181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806466+0000","last_change":"2023-09-27T23:33:19.804496+0000","last_active":"2023-09-27T23:33:19.806466+0000","last_peered":"2023-09-27T23:33:19.806466+0000","last_clean":"2023-09-27T23:33:19.806466+0000","last_became_active":"2023-09-27T23:33:19.804186+0000","last_became_peered":"2023-09-27T23:33:19.804186+0000","last_unstale":"2023-09-27T23:33:19.806466+0000","last_undegraded":"2023-09-27T23:33:19.806466+0000","last_fullsized":"2023-09-27T23:33:19.806466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:59:09.351560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.794961+0000","last_change":"2023-09-27T23:33:19.430707+0000","last_active":"2023-09-27T23:33:19.794961+0000","last_peered":"2023-09-27T23:33:19.794961+0000","last_clean":"2023-09-27T23:33:19.794961+0000","last_became_active":"2023-09-27T23:33:19.430423+0000","last_became_peered":"2023-09-27T23:33:19.430423+0000","last_unstale":"2023-09-27T23:33:19.794961+0000","last_undegraded":"2023-09-27T23:33:19.794961+0000","last_fullsized":"2023-09-27T23:33:19.794961+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:58.532840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814478+0000","last_change":"2023-09-27T23:33:19.804146+0000","last_active":"2023-09-27T23:33:19.814478+0000","last_peered":"2023-09-27T23:33:19.814478+0000","last_clean":"2023-09-27T23:33:19.814478+0000","last_became_active":"2023-09-27T23:33:19.803861+0000","last_became_peered":"2023-09-27T23:33:19.803861+0000","last_unstale":"2023-09-27T23:33:19.814478+0000","last_undegraded":"2023-09-27T23:33:19.814478+0000","last_fullsized":"2023-09-27T23:33:19.814478+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806946+0000","last_change":"2023-09-27T23:33:19.436051+0000","last_active":"2023-09-27T23:33:19.806946+0000","last_peered":"2023-09-27T23:33:19.806946+0000","last_clean":"2023-09-27T23:33:19.806946+0000","last_became_active":"2023-09-27T23:33:19.435770+0000","last_became_peered":"2023-09-27T23:33:19.435770+0000","last_unstale":"2023-09-27T23:33:19.806946+0000","last_undegraded":"2023-09-27T23:33:19.806946+0000","last_fullsized":"2023-09-27T23:33:19.806946+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:14:12.478687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"131'249","reported_seq":740,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792090+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:19.792090+0000","last_peered":"2023-09-27T23:33:19.792090+0000","last_clean":"2023-09-27T23:33:19.792090+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:19.792090+0000","last_undegraded":"2023-09-27T23:33:19.792090+0000","last_fullsized":"2023-09-27T23:33:19.792090+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":249,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790633+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:19.790633+0000","last_peered":"2023-09-27T23:33:19.790633+0000","last_clean":"2023-09-27T23:33:19.790633+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:19.790633+0000","last_undegraded":"2023-09-27T23:33:19.790633+0000","last_fullsized":"2023-09-27T23:33:19.790633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826305+0000","last_change":"2023-09-27T23:33:19.818550+0000","last_active":"2023-09-27T23:33:19.826305+0000","last_peered":"2023-09-27T23:33:19.826305+0000","last_clean":"2023-09-27T23:33:19.826305+0000","last_became_active":"2023-09-27T23:33:19.818313+0000","last_became_peered":"2023-09-27T23:33:19.818313+0000","last_unstale":"2023-09-27T23:33:19.826305+0000","last_undegraded":"2023-09-27T23:33:19.826305+0000","last_fullsized":"2023-09-27T23:33:19.826305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825230+0000","last_change":"2023-09-27T23:33:19.439141+0000","last_active":"2023-09-27T23:33:19.825230+0000","last_peered":"2023-09-27T23:33:19.825230+0000","last_clean":"2023-09-27T23:33:19.825230+0000","last_became_active":"2023-09-27T23:33:19.438828+0000","last_became_peered":"2023-09-27T23:33:19.438828+0000","last_unstale":"2023-09-27T23:33:19.825230+0000","last_undegraded":"2023-09-27T23:33:19.825230+0000","last_fullsized":"2023-09-27T23:33:19.825230+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:33.463327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789544+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:19.789544+0000","last_peered":"2023-09-27T23:33:19.789544+0000","last_clean":"2023-09-27T23:33:19.789544+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:19.789544+0000","last_undegraded":"2023-09-27T23:33:19.789544+0000","last_fullsized":"2023-09-27T23:33:19.789544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816597+0000","last_change":"2023-09-27T23:33:19.812801+0000","last_active":"2023-09-27T23:33:19.816597+0000","last_peered":"2023-09-27T23:33:19.816597+0000","last_clean":"2023-09-27T23:33:19.816597+0000","last_became_active":"2023-09-27T23:33:19.812506+0000","last_became_peered":"2023-09-27T23:33:19.812506+0000","last_unstale":"2023-09-27T23:33:19.816597+0000","last_undegraded":"2023-09-27T23:33:19.816597+0000","last_fullsized":"2023-09-27T23:33:19.816597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:16:22.761586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788828+0000","last_change":"2023-09-27T23:33:19.424648+0000","last_active":"2023-09-27T23:33:19.788828+0000","last_peered":"2023-09-27T23:33:19.788828+0000","last_clean":"2023-09-27T23:33:19.788828+0000","last_became_active":"2023-09-27T23:33:19.424271+0000","last_became_peered":"2023-09-27T23:33:19.424271+0000","last_unstale":"2023-09-27T23:33:19.788828+0000","last_undegraded":"2023-09-27T23:33:19.788828+0000","last_fullsized":"2023-09-27T23:33:19.788828+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:25.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798546+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:19.798546+0000","last_peered":"2023-09-27T23:33:19.798546+0000","last_clean":"2023-09-27T23:33:19.798546+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:19.798546+0000","last_undegraded":"2023-09-27T23:33:19.798546+0000","last_fullsized":"2023-09-27T23:33:19.798546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817238+0000","last_change":"2023-09-27T23:33:19.813654+0000","last_active":"2023-09-27T23:33:19.817238+0000","last_peered":"2023-09-27T23:33:19.817238+0000","last_clean":"2023-09-27T23:33:19.817238+0000","last_became_active":"2023-09-27T23:33:19.813363+0000","last_became_peered":"2023-09-27T23:33:19.813363+0000","last_unstale":"2023-09-27T23:33:19.817238+0000","last_undegraded":"2023-09-27T23:33:19.817238+0000","last_fullsized":"2023-09-27T23:33:19.817238+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:09.337371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798813+0000","last_change":"2023-09-27T23:33:19.432418+0000","last_active":"2023-09-27T23:33:19.798813+0000","last_peered":"2023-09-27T23:33:19.798813+0000","last_clean":"2023-09-27T23:33:19.798813+0000","last_became_active":"2023-09-27T23:33:19.428641+0000","last_became_peered":"2023-09-27T23:33:19.428641+0000","last_unstale":"2023-09-27T23:33:19.798813+0000","last_undegraded":"2023-09-27T23:33:19.798813+0000","last_fullsized":"2023-09-27T23:33:19.798813+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790099+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:19.790099+0000","last_peered":"2023-09-27T23:33:19.790099+0000","last_clean":"2023-09-27T23:33:19.790099+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:19.790099+0000","last_undegraded":"2023-09-27T23:33:19.790099+0000","last_fullsized":"2023-09-27T23:33:19.790099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815270+0000","last_change":"2023-09-27T23:33:19.809693+0000","last_active":"2023-09-27T23:33:19.815270+0000","last_peered":"2023-09-27T23:33:19.815270+0000","last_clean":"2023-09-27T23:33:19.815270+0000","last_became_active":"2023-09-27T23:33:19.809423+0000","last_became_peered":"2023-09-27T23:33:19.809423+0000","last_unstale":"2023-09-27T23:33:19.815270+0000","last_undegraded":"2023-09-27T23:33:19.815270+0000","last_fullsized":"2023-09-27T23:33:19.815270+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828563+0000","last_change":"2023-09-27T23:33:19.430328+0000","last_active":"2023-09-27T23:33:19.828563+0000","last_peered":"2023-09-27T23:33:19.828563+0000","last_clean":"2023-09-27T23:33:19.828563+0000","last_became_active":"2023-09-27T23:33:19.430033+0000","last_became_peered":"2023-09-27T23:33:19.430033+0000","last_unstale":"2023-09-27T23:33:19.828563+0000","last_undegraded":"2023-09-27T23:33:19.828563+0000","last_fullsized":"2023-09-27T23:33:19.828563+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:48.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789643+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:19.789643+0000","last_peered":"2023-09-27T23:33:19.789643+0000","last_clean":"2023-09-27T23:33:19.789643+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:19.789643+0000","last_undegraded":"2023-09-27T23:33:19.789643+0000","last_fullsized":"2023-09-27T23:33:19.789643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824588+0000","last_change":"2023-09-27T23:33:19.823201+0000","last_active":"2023-09-27T23:33:19.824588+0000","last_peered":"2023-09-27T23:33:19.824588+0000","last_clean":"2023-09-27T23:33:19.824588+0000","last_became_active":"2023-09-27T23:33:19.822894+0000","last_became_peered":"2023-09-27T23:33:19.822894+0000","last_unstale":"2023-09-27T23:33:19.824588+0000","last_undegraded":"2023-09-27T23:33:19.824588+0000","last_fullsized":"2023-09-27T23:33:19.824588+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822697+0000","last_change":"2023-09-27T23:33:19.438806+0000","last_active":"2023-09-27T23:33:19.822697+0000","last_peered":"2023-09-27T23:33:19.822697+0000","last_clean":"2023-09-27T23:33:19.822697+0000","last_became_active":"2023-09-27T23:33:19.438541+0000","last_became_peered":"2023-09-27T23:33:19.438541+0000","last_unstale":"2023-09-27T23:33:19.822697+0000","last_undegraded":"2023-09-27T23:33:19.822697+0000","last_fullsized":"2023-09-27T23:33:19.822697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:29.683418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793521+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:19.793521+0000","last_peered":"2023-09-27T23:33:19.793521+0000","last_clean":"2023-09-27T23:33:19.793521+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:19.793521+0000","last_undegraded":"2023-09-27T23:33:19.793521+0000","last_fullsized":"2023-09-27T23:33:19.793521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817658+0000","last_change":"2023-09-27T23:33:19.815111+0000","last_active":"2023-09-27T23:33:19.817658+0000","last_peered":"2023-09-27T23:33:19.817658+0000","last_clean":"2023-09-27T23:33:19.817658+0000","last_became_active":"2023-09-27T23:33:19.814866+0000","last_became_peered":"2023-09-27T23:33:19.814866+0000","last_unstale":"2023-09-27T23:33:19.817658+0000","last_undegraded":"2023-09-27T23:33:19.817658+0000","last_fullsized":"2023-09-27T23:33:19.817658+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:30.614318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.800186+0000","last_change":"2023-09-27T23:33:19.433336+0000","last_active":"2023-09-27T23:33:19.800186+0000","last_peered":"2023-09-27T23:33:19.800186+0000","last_clean":"2023-09-27T23:33:19.800186+0000","last_became_active":"2023-09-27T23:33:19.433032+0000","last_became_peered":"2023-09-27T23:33:19.433032+0000","last_unstale":"2023-09-27T23:33:19.800186+0000","last_undegraded":"2023-09-27T23:33:19.800186+0000","last_fullsized":"2023-09-27T23:33:19.800186+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:56.817223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788544+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:19.788544+0000","last_peered":"2023-09-27T23:33:19.788544+0000","last_clean":"2023-09-27T23:33:19.788544+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:19.788544+0000","last_undegraded":"2023-09-27T23:33:19.788544+0000","last_fullsized":"2023-09-27T23:33:19.788544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816760+0000","last_change":"2023-09-27T23:33:19.807831+0000","last_active":"2023-09-27T23:33:19.816760+0000","last_peered":"2023-09-27T23:33:19.816760+0000","last_clean":"2023-09-27T23:33:19.816760+0000","last_became_active":"2023-09-27T23:33:19.807647+0000","last_became_peered":"2023-09-27T23:33:19.807647+0000","last_unstale":"2023-09-27T23:33:19.816760+0000","last_undegraded":"2023-09-27T23:33:19.816760+0000","last_fullsized":"2023-09-27T23:33:19.816760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:34.467334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792775+0000","last_change":"2023-09-27T23:33:19.438063+0000","last_active":"2023-09-27T23:33:19.792775+0000","last_peered":"2023-09-27T23:33:19.792775+0000","last_clean":"2023-09-27T23:33:19.792775+0000","last_became_active":"2023-09-27T23:33:19.437740+0000","last_became_peered":"2023-09-27T23:33:19.437740+0000","last_unstale":"2023-09-27T23:33:19.792775+0000","last_undegraded":"2023-09-27T23:33:19.792775+0000","last_fullsized":"2023-09-27T23:33:19.792775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:01.021179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796971+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:19.796971+0000","last_peered":"2023-09-27T23:33:19.796971+0000","last_clean":"2023-09-27T23:33:19.796971+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:19.796971+0000","last_undegraded":"2023-09-27T23:33:19.796971+0000","last_fullsized":"2023-09-27T23:33:19.796971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509677,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":2332,"kb_used_omap":0,"kb_used_meta":25408,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":2387968,"data_stored":1514870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26017792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.77900000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":0.82099999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0429999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":0.86899999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.083},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":0.73899999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26368,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1781760,"data_stored":923892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27000832},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":1.155},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":1.018},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.345},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.4079999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509676,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37776,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94334064,"statfs":{"total":96636764160,"available":96598081536,"internally_reserved":0,"allocated":1777664,"data_stored":922468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":1.242}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.995},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.4950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509677,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23500,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":21760,"kb_avail":94348340,"statfs":{"total":96636764160,"available":96612700160,"internally_reserved":0,"allocated":1773568,"data_stored":922426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22282240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":0.83699999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":1.135}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.036},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.107}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.228}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.1699999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509676,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28560,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94343280,"statfs":{"total":96636764160,"available":96607518720,"internally_reserved":0,"allocated":1777664,"data_stored":922516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.034}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.161}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":1.1899999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.006},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.5389999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.3},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.3220000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.1100000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509677,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22672,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":94349168,"statfs":{"total":96636764160,"available":96613548032,"internally_reserved":0,"allocated":1777664,"data_stored":922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.498},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.95999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.01},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37896,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":36160,"kb_avail":94333944,"statfs":{"total":96636764160,"available":96597958656,"internally_reserved":0,"allocated":1769472,"data_stored":922356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37027840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":1.026}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.171},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.302},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.119}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.27},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2470000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509678,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29068,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":26752,"kb_avail":94342772,"statfs":{"total":96636764160,"available":96606998528,"internally_reserved":0,"allocated":2363392,"data_stored":1512724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27394048},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.624},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.998}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":0.94199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.105},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.341}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.5269999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.28}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:25.712 INFO:tasks.ceph.mds.c.smithi064.stderr:2023-09-27T23:33:25.707+0000 7ff44206f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:33:28.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:28.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 Processor -- start 2023-09-27T23:33:28.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 -- start start 2023-09-27T23:33:28.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc55c0cb920 0x7fc55c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc55c15cee0 0x7fc55c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c068dc0 con 0x7fc55c158d40 2023-09-27T23:33:28.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c0cc910 con 0x7fc55c15cee0 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c0682a0 con 0x7fc55c0cb920 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc55c15cee0 0x7fc55c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc55c15cee0 0x7fc55c06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54252/0 (socket says 172.21.15.64:54252) 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 -- 172.21.15.64:0/319445324 learned_addr learned my addr 172.21.15.64:0/319445324 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc55b7fe700 1 --2- 172.21.15.64:0/319445324 >> v2:172.21.15.123:3300/0 conn(0x7fc55c0cb920 0x7fc55c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 -- 172.21.15.64:0/319445324 >> v2:172.21.15.123:3300/0 conn(0x7fc55c0cb920 msgr2=0x7fc55c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 --2- 172.21.15.64:0/319445324 >> v2:172.21.15.123:3300/0 conn(0x7fc55c0cb920 0x7fc55c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 -- 172.21.15.64:0/319445324 >> v1:172.21.15.64:6789/0 conn(0x7fc55c158d40 legacy=0x7fc55c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:28.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 -- 172.21.15.64:0/319445324 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c068490 con 0x7fc55c15cee0 2023-09-27T23:33:28.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc560e02700 1 --2- 172.21.15.64:0/319445324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c15cee0 0x7fc55c06c7c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0b3bc0 tx=0x7fc554009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23843ceaef9ff5f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc55affd700 1 -- 172.21.15.64:0/319445324 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400e070 con 0x7fc55c15cee0 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc55affd700 1 -- 172.21.15.64:0/319445324 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55400ba50 con 0x7fc55c15cee0 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc55affd700 1 -- 172.21.15.64:0/319445324 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5540092e0 con 0x7fc55c15cee0 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 -- 172.21.15.64:0/319445324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c15cee0 msgr2=0x7fc55c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 --2- 172.21.15.64:0/319445324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c15cee0 0x7fc55c06c7c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0b3bc0 tx=0x7fc554009840 comp rx=0 tx=0).stop 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 -- 172.21.15.64:0/319445324 shutdown_connections 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 --2- 172.21.15.64:0/319445324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c15cee0 0x7fc55c06c7c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.315+0000 7fc561603700 1 --2- 172.21.15.64:0/319445324 >> v2:172.21.15.123:3300/0 conn(0x7fc55c0cb920 0x7fc55c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.319+0000 7fc561603700 1 -- 172.21.15.64:0/319445324 >> 172.21.15.64:0/319445324 conn(0x7fc55c0cdbc0 msgr2=0x7fc55c0cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:28.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.319+0000 7fc561603700 1 -- 172.21.15.64:0/319445324 shutdown_connections 2023-09-27T23:33:28.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.319+0000 7fc561603700 1 -- 172.21.15.64:0/319445324 wait complete. 2023-09-27T23:33:28.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.319+0000 7fc561603700 1 Processor -- start 2023-09-27T23:33:28.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.319+0000 7fc561603700 1 -- start start 2023-09-27T23:33:28.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc55c0cb920 0x7fc55c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 0x7fc55c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc55c15cee0 0x7fc55c16ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c6650 con 0x7fc55c158d40 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c154860 con 0x7fc55c0cb920 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c159870 con 0x7fc55c15cee0 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 0x7fc55c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 0x7fc55c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54254/0 (socket says 172.21.15.64:54254) 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 -- 172.21.15.64:0/877598315 learned_addr learned my addr 172.21.15.64:0/877598315 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:28.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55bfff700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc55c0cb920 0x7fc55c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc560e02700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc55c15cee0 0x7fc55c16ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 -- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc55c15cee0 msgr2=0x7fc55c16ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc55c15cee0 0x7fc55c16ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 -- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc55c0cb920 msgr2=0x7fc55c15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc55c0cb920 0x7fc55c15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 -- 172.21.15.64:0/877598315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc554009ae0 con 0x7fc55c158d40 2023-09-27T23:33:28.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc55b7fe700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 0x7fc55c1572e0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc55000d8c0 tx=0x7fc550008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:28.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5500125f0 con 0x7fc55c158d40 2023-09-27T23:33:28.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc550012750 con 0x7fc55c158d40 2023-09-27T23:33:28.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550012af0 con 0x7fc55c158d40 2023-09-27T23:33:28.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c0c6860 con 0x7fc55c158d40 2023-09-27T23:33:28.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c0c6e20 con 0x7fc55c158d40 2023-09-27T23:33:28.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c04be40 con 0x7fc55c158d40 2023-09-27T23:33:28.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.323+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc55001e460 con 0x7fc55c158d40 2023-09-27T23:33:28.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.327+0000 7fc543fff700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc548060a70 0x7fc548062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.327+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc55007d6a0 con 0x7fc55c158d40 2023-09-27T23:33:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.327+0000 7fc55bfff700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc548060a70 0x7fc548062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.327+0000 7fc55bfff700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc548060a70 0x7fc548062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc54c003aa0 tx=0x7fc54c000bc0 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.335+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc55004d6d0 con 0x7fc55c158d40 2023-09-27T23:33:28.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.495+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc55c0671b0 con 0x7fc548060a70 2023-09-27T23:33:28.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.511+0000 7fc543fff700 1 -- 172.21.15.64:0/877598315 <== mgr.7358 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308005 (secure 0 0 0) 0x7fc55c0671b0 con 0x7fc548060a70 2023-09-27T23:33:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:28.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc548060a70 msgr2=0x7fc548062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc548060a70 0x7fc548062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc54c003aa0 tx=0x7fc54c000bc0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 msgr2=0x7fc55c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 0x7fc55c1572e0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc55000d8c0 tx=0x7fc550008040 comp rx=0 tx=0).stop 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 shutdown_connections 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc548060a70 0x7fc548062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc55c15cee0 0x7fc55c16ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c158d40 0x7fc55c1572e0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 --2- 172.21.15.64:0/877598315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc55c0cb920 0x7fc55c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 >> 172.21.15.64:0/877598315 conn(0x7fc55c0cdbc0 msgr2=0x7fc55c0ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 shutdown_connections 2023-09-27T23:33:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.515+0000 7fc561603700 1 -- 172.21.15.64:0/877598315 wait complete. 2023-09-27T23:33:28.521 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:28.532 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:33:26.684687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235336,"kb_used_data":15048,"kb_used_omap":0,"kb_used_meta":220224,"kb_avail":754739384,"statfs":{"total":773094113280,"available":772853129216,"internally_reserved":0,"allocated":15409152,"data_stored":8564196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":225509376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.275610"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816721+0000","last_change":"2023-09-27T23:33:19.810193+0000","last_active":"2023-09-27T23:33:19.816721+0000","last_peered":"2023-09-27T23:33:19.816721+0000","last_clean":"2023-09-27T23:33:19.816721+0000","last_became_active":"2023-09-27T23:33:19.809935+0000","last_became_peered":"2023-09-27T23:33:19.809935+0000","last_unstale":"2023-09-27T23:33:19.816721+0000","last_undegraded":"2023-09-27T23:33:19.816721+0000","last_fullsized":"2023-09-27T23:33:19.816721+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:32:44.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816570+0000","last_change":"2023-09-27T23:33:19.811311+0000","last_active":"2023-09-27T23:33:19.816570+0000","last_peered":"2023-09-27T23:33:19.816570+0000","last_clean":"2023-09-27T23:33:19.816570+0000","last_became_active":"2023-09-27T23:33:19.809810+0000","last_became_peered":"2023-09-27T23:33:19.809810+0000","last_unstale":"2023-09-27T23:33:19.816570+0000","last_undegraded":"2023-09-27T23:33:19.816570+0000","last_fullsized":"2023-09-27T23:33:19.816570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:06.194398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825973+0000","last_change":"2023-09-27T23:33:19.817990+0000","last_active":"2023-09-27T23:33:19.825973+0000","last_peered":"2023-09-27T23:33:19.825973+0000","last_clean":"2023-09-27T23:33:19.825973+0000","last_became_active":"2023-09-27T23:33:19.809746+0000","last_became_peered":"2023-09-27T23:33:19.809746+0000","last_unstale":"2023-09-27T23:33:19.825973+0000","last_undegraded":"2023-09-27T23:33:19.825973+0000","last_fullsized":"2023-09-27T23:33:19.825973+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.812775+0000","last_change":"2023-09-27T23:33:19.801379+0000","last_active":"2023-09-27T23:33:19.812775+0000","last_peered":"2023-09-27T23:33:19.812775+0000","last_clean":"2023-09-27T23:33:19.812775+0000","last_became_active":"2023-09-27T23:33:19.800971+0000","last_became_peered":"2023-09-27T23:33:19.800971+0000","last_unstale":"2023-09-27T23:33:19.812775+0000","last_undegraded":"2023-09-27T23:33:19.812775+0000","last_fullsized":"2023-09-27T23:33:19.812775+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:08:53.280332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815298+0000","last_change":"2023-09-27T23:33:19.811325+0000","last_active":"2023-09-27T23:33:19.815298+0000","last_peered":"2023-09-27T23:33:19.815298+0000","last_clean":"2023-09-27T23:33:19.815298+0000","last_became_active":"2023-09-27T23:33:19.811029+0000","last_became_peered":"2023-09-27T23:33:19.811029+0000","last_unstale":"2023-09-27T23:33:19.815298+0000","last_undegraded":"2023-09-27T23:33:19.815298+0000","last_fullsized":"2023-09-27T23:33:19.815298+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:12:46.263357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815037+0000","last_change":"2023-09-27T23:33:19.807530+0000","last_active":"2023-09-27T23:33:19.815037+0000","last_peered":"2023-09-27T23:33:19.815037+0000","last_clean":"2023-09-27T23:33:19.815037+0000","last_became_active":"2023-09-27T23:33:19.807070+0000","last_became_peered":"2023-09-27T23:33:19.807070+0000","last_unstale":"2023-09-27T23:33:19.815037+0000","last_undegraded":"2023-09-27T23:33:19.815037+0000","last_fullsized":"2023-09-27T23:33:19.815037+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:42.285344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825591+0000","last_change":"2023-09-27T23:33:19.823986+0000","last_active":"2023-09-27T23:33:19.825591+0000","last_peered":"2023-09-27T23:33:19.825591+0000","last_clean":"2023-09-27T23:33:19.825591+0000","last_became_active":"2023-09-27T23:33:19.823794+0000","last_became_peered":"2023-09-27T23:33:19.823794+0000","last_unstale":"2023-09-27T23:33:19.825591+0000","last_undegraded":"2023-09-27T23:33:19.825591+0000","last_fullsized":"2023-09-27T23:33:19.825591+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:41.867018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826231+0000","last_change":"2023-09-27T23:33:19.819831+0000","last_active":"2023-09-27T23:33:19.826231+0000","last_peered":"2023-09-27T23:33:19.826231+0000","last_clean":"2023-09-27T23:33:19.826231+0000","last_became_active":"2023-09-27T23:33:19.819582+0000","last_became_peered":"2023-09-27T23:33:19.819582+0000","last_unstale":"2023-09-27T23:33:19.826231+0000","last_undegraded":"2023-09-27T23:33:19.826231+0000","last_fullsized":"2023-09-27T23:33:19.826231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:59.618721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826213+0000","last_change":"2023-09-27T23:33:19.818540+0000","last_active":"2023-09-27T23:33:19.826213+0000","last_peered":"2023-09-27T23:33:19.826213+0000","last_clean":"2023-09-27T23:33:19.826213+0000","last_became_active":"2023-09-27T23:33:19.818222+0000","last_became_peered":"2023-09-27T23:33:19.818222+0000","last_unstale":"2023-09-27T23:33:19.826213+0000","last_undegraded":"2023-09-27T23:33:19.826213+0000","last_fullsized":"2023-09-27T23:33:19.826213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813361+0000","last_change":"2023-09-27T23:33:19.806217+0000","last_active":"2023-09-27T23:33:19.813361+0000","last_peered":"2023-09-27T23:33:19.813361+0000","last_clean":"2023-09-27T23:33:19.813361+0000","last_became_active":"2023-09-27T23:33:19.805338+0000","last_became_peered":"2023-09-27T23:33:19.805338+0000","last_unstale":"2023-09-27T23:33:19.813361+0000","last_undegraded":"2023-09-27T23:33:19.813361+0000","last_fullsized":"2023-09-27T23:33:19.813361+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.311666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809763+0000","last_change":"2023-09-27T23:33:19.806347+0000","last_active":"2023-09-27T23:33:19.809763+0000","last_peered":"2023-09-27T23:33:19.809763+0000","last_clean":"2023-09-27T23:33:19.809763+0000","last_became_active":"2023-09-27T23:33:19.805988+0000","last_became_peered":"2023-09-27T23:33:19.805988+0000","last_unstale":"2023-09-27T23:33:19.809763+0000","last_undegraded":"2023-09-27T23:33:19.809763+0000","last_fullsized":"2023-09-27T23:33:19.809763+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:29:12.943601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826156+0000","last_change":"2023-09-27T23:33:19.810568+0000","last_active":"2023-09-27T23:33:19.826156+0000","last_peered":"2023-09-27T23:33:19.826156+0000","last_clean":"2023-09-27T23:33:19.826156+0000","last_became_active":"2023-09-27T23:33:19.810237+0000","last_became_peered":"2023-09-27T23:33:19.810237+0000","last_unstale":"2023-09-27T23:33:19.826156+0000","last_undegraded":"2023-09-27T23:33:19.826156+0000","last_fullsized":"2023-09-27T23:33:19.826156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816810+0000","last_change":"2023-09-27T23:33:19.810731+0000","last_active":"2023-09-27T23:33:19.816810+0000","last_peered":"2023-09-27T23:33:19.816810+0000","last_clean":"2023-09-27T23:33:19.816810+0000","last_became_active":"2023-09-27T23:33:19.810477+0000","last_became_peered":"2023-09-27T23:33:19.810477+0000","last_unstale":"2023-09-27T23:33:19.816810+0000","last_undegraded":"2023-09-27T23:33:19.816810+0000","last_fullsized":"2023-09-27T23:33:19.816810+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:19:28.503800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809513+0000","last_change":"2023-09-27T23:33:19.805356+0000","last_active":"2023-09-27T23:33:19.809513+0000","last_peered":"2023-09-27T23:33:19.809513+0000","last_clean":"2023-09-27T23:33:19.809513+0000","last_became_active":"2023-09-27T23:33:19.805064+0000","last_became_peered":"2023-09-27T23:33:19.805064+0000","last_unstale":"2023-09-27T23:33:19.809513+0000","last_undegraded":"2023-09-27T23:33:19.809513+0000","last_fullsized":"2023-09-27T23:33:19.809513+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:49:37.422975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827932+0000","last_change":"2023-09-27T23:33:19.822764+0000","last_active":"2023-09-27T23:33:19.827932+0000","last_peered":"2023-09-27T23:33:19.827932+0000","last_clean":"2023-09-27T23:33:19.827932+0000","last_became_active":"2023-09-27T23:33:19.822456+0000","last_became_peered":"2023-09-27T23:33:19.822456+0000","last_unstale":"2023-09-27T23:33:19.827932+0000","last_undegraded":"2023-09-27T23:33:19.827932+0000","last_fullsized":"2023-09-27T23:33:19.827932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824854+0000","last_change":"2023-09-27T23:33:19.818152+0000","last_active":"2023-09-27T23:33:19.824854+0000","last_peered":"2023-09-27T23:33:19.824854+0000","last_clean":"2023-09-27T23:33:19.824854+0000","last_became_active":"2023-09-27T23:33:19.817283+0000","last_became_peered":"2023-09-27T23:33:19.817283+0000","last_unstale":"2023-09-27T23:33:19.824854+0000","last_undegraded":"2023-09-27T23:33:19.824854+0000","last_fullsized":"2023-09-27T23:33:19.824854+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:36:20.392671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816834+0000","last_change":"2023-09-27T23:33:19.812064+0000","last_active":"2023-09-27T23:33:19.816834+0000","last_peered":"2023-09-27T23:33:19.816834+0000","last_clean":"2023-09-27T23:33:19.816834+0000","last_became_active":"2023-09-27T23:33:19.811676+0000","last_became_peered":"2023-09-27T23:33:19.811676+0000","last_unstale":"2023-09-27T23:33:19.816834+0000","last_undegraded":"2023-09-27T23:33:19.816834+0000","last_fullsized":"2023-09-27T23:33:19.816834+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:33:54.723421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824872+0000","last_change":"2023-09-27T23:33:19.818291+0000","last_active":"2023-09-27T23:33:19.824872+0000","last_peered":"2023-09-27T23:33:19.824872+0000","last_clean":"2023-09-27T23:33:19.824872+0000","last_became_active":"2023-09-27T23:33:19.817949+0000","last_became_peered":"2023-09-27T23:33:19.817949+0000","last_unstale":"2023-09-27T23:33:19.824872+0000","last_undegraded":"2023-09-27T23:33:19.824872+0000","last_fullsized":"2023-09-27T23:33:19.824872+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:31:08.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816188+0000","last_change":"2023-09-27T23:33:19.811585+0000","last_active":"2023-09-27T23:33:19.816188+0000","last_peered":"2023-09-27T23:33:19.816188+0000","last_clean":"2023-09-27T23:33:19.816188+0000","last_became_active":"2023-09-27T23:33:19.810377+0000","last_became_peered":"2023-09-27T23:33:19.810377+0000","last_unstale":"2023-09-27T23:33:19.816188+0000","last_undegraded":"2023-09-27T23:33:19.816188+0000","last_fullsized":"2023-09-27T23:33:19.816188+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:42.285344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824811+0000","last_change":"2023-09-27T23:33:19.817725+0000","last_active":"2023-09-27T23:33:19.824811+0000","last_peered":"2023-09-27T23:33:19.824811+0000","last_clean":"2023-09-27T23:33:19.824811+0000","last_became_active":"2023-09-27T23:33:19.817387+0000","last_became_peered":"2023-09-27T23:33:19.817387+0000","last_unstale":"2023-09-27T23:33:19.824811+0000","last_undegraded":"2023-09-27T23:33:19.824811+0000","last_fullsized":"2023-09-27T23:33:19.824811+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:55:34.864941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.808788+0000","last_change":"2023-09-27T23:33:19.803176+0000","last_active":"2023-09-27T23:33:19.808788+0000","last_peered":"2023-09-27T23:33:19.808788+0000","last_clean":"2023-09-27T23:33:19.808788+0000","last_became_active":"2023-09-27T23:33:19.802860+0000","last_became_peered":"2023-09-27T23:33:19.802860+0000","last_unstale":"2023-09-27T23:33:19.808788+0000","last_undegraded":"2023-09-27T23:33:19.808788+0000","last_fullsized":"2023-09-27T23:33:19.808788+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:37:54.875615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.807080+0000","last_change":"2023-09-27T23:33:19.805641+0000","last_active":"2023-09-27T23:33:19.807080+0000","last_peered":"2023-09-27T23:33:19.807080+0000","last_clean":"2023-09-27T23:33:19.807080+0000","last_became_active":"2023-09-27T23:33:19.805378+0000","last_became_peered":"2023-09-27T23:33:19.805378+0000","last_unstale":"2023-09-27T23:33:19.807080+0000","last_undegraded":"2023-09-27T23:33:19.807080+0000","last_fullsized":"2023-09-27T23:33:19.807080+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:25:59.533646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826283+0000","last_change":"2023-09-27T23:33:19.818051+0000","last_active":"2023-09-27T23:33:19.826283+0000","last_peered":"2023-09-27T23:33:19.826283+0000","last_clean":"2023-09-27T23:33:19.826283+0000","last_became_active":"2023-09-27T23:33:19.809795+0000","last_became_peered":"2023-09-27T23:33:19.809795+0000","last_unstale":"2023-09-27T23:33:19.826283+0000","last_undegraded":"2023-09-27T23:33:19.826283+0000","last_fullsized":"2023-09-27T23:33:19.826283+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814736+0000","last_change":"2023-09-27T23:33:19.808749+0000","last_active":"2023-09-27T23:33:19.814736+0000","last_peered":"2023-09-27T23:33:19.814736+0000","last_clean":"2023-09-27T23:33:19.814736+0000","last_became_active":"2023-09-27T23:33:19.808480+0000","last_became_peered":"2023-09-27T23:33:19.808480+0000","last_unstale":"2023-09-27T23:33:19.814736+0000","last_undegraded":"2023-09-27T23:33:19.814736+0000","last_fullsized":"2023-09-27T23:33:19.814736+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:34:58.552571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.804587+0000","last_change":"2023-09-27T23:33:19.798526+0000","last_active":"2023-09-27T23:33:19.804587+0000","last_peered":"2023-09-27T23:33:19.804587+0000","last_clean":"2023-09-27T23:33:19.804587+0000","last_became_active":"2023-09-27T23:33:19.798226+0000","last_became_peered":"2023-09-27T23:33:19.798226+0000","last_unstale":"2023-09-27T23:33:19.804587+0000","last_undegraded":"2023-09-27T23:33:19.804587+0000","last_fullsized":"2023-09-27T23:33:19.804587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:13:28.298592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815408+0000","last_change":"2023-09-27T23:33:19.805590+0000","last_active":"2023-09-27T23:33:19.815408+0000","last_peered":"2023-09-27T23:33:19.815408+0000","last_clean":"2023-09-27T23:33:19.815408+0000","last_became_active":"2023-09-27T23:33:19.805319+0000","last_became_peered":"2023-09-27T23:33:19.805319+0000","last_unstale":"2023-09-27T23:33:19.815408+0000","last_undegraded":"2023-09-27T23:33:19.815408+0000","last_fullsized":"2023-09-27T23:33:19.815408+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:33:39.999069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815678+0000","last_change":"2023-09-27T23:33:19.805843+0000","last_active":"2023-09-27T23:33:19.815678+0000","last_peered":"2023-09-27T23:33:19.815678+0000","last_clean":"2023-09-27T23:33:19.815678+0000","last_became_active":"2023-09-27T23:33:19.805571+0000","last_became_peered":"2023-09-27T23:33:19.805571+0000","last_unstale":"2023-09-27T23:33:19.815678+0000","last_undegraded":"2023-09-27T23:33:19.815678+0000","last_fullsized":"2023-09-27T23:33:19.815678+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:49.075742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.823630+0000","last_change":"2023-09-27T23:33:19.819123+0000","last_active":"2023-09-27T23:33:19.823630+0000","last_peered":"2023-09-27T23:33:19.823630+0000","last_clean":"2023-09-27T23:33:19.823630+0000","last_became_active":"2023-09-27T23:33:19.818840+0000","last_became_peered":"2023-09-27T23:33:19.818840+0000","last_unstale":"2023-09-27T23:33:19.823630+0000","last_undegraded":"2023-09-27T23:33:19.823630+0000","last_fullsized":"2023-09-27T23:33:19.823630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:00.849910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827890+0000","last_change":"2023-09-27T23:33:19.822665+0000","last_active":"2023-09-27T23:33:19.827890+0000","last_peered":"2023-09-27T23:33:19.827890+0000","last_clean":"2023-09-27T23:33:19.827890+0000","last_became_active":"2023-09-27T23:33:19.822350+0000","last_became_peered":"2023-09-27T23:33:19.822350+0000","last_unstale":"2023-09-27T23:33:19.827890+0000","last_undegraded":"2023-09-27T23:33:19.827890+0000","last_fullsized":"2023-09-27T23:33:19.827890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:30:36.041799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827909+0000","last_change":"2023-09-27T23:33:19.822700+0000","last_active":"2023-09-27T23:33:19.827909+0000","last_peered":"2023-09-27T23:33:19.827909+0000","last_clean":"2023-09-27T23:33:19.827909+0000","last_became_active":"2023-09-27T23:33:19.822405+0000","last_became_peered":"2023-09-27T23:33:19.822405+0000","last_unstale":"2023-09-27T23:33:19.827909+0000","last_undegraded":"2023-09-27T23:33:19.827909+0000","last_fullsized":"2023-09-27T23:33:19.827909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:59:22.750767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814349+0000","last_change":"2023-09-27T23:33:19.806164+0000","last_active":"2023-09-27T23:33:19.814349+0000","last_peered":"2023-09-27T23:33:19.814349+0000","last_clean":"2023-09-27T23:33:19.814349+0000","last_became_active":"2023-09-27T23:33:19.805693+0000","last_became_peered":"2023-09-27T23:33:19.805693+0000","last_unstale":"2023-09-27T23:33:19.814349+0000","last_undegraded":"2023-09-27T23:33:19.814349+0000","last_fullsized":"2023-09-27T23:33:19.814349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:43.937071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.820374+0000","last_change":"2023-09-27T23:33:19.816258+0000","last_active":"2023-09-27T23:33:19.820374+0000","last_peered":"2023-09-27T23:33:19.820374+0000","last_clean":"2023-09-27T23:33:19.820374+0000","last_became_active":"2023-09-27T23:33:19.816064+0000","last_became_peered":"2023-09-27T23:33:19.816064+0000","last_unstale":"2023-09-27T23:33:19.820374+0000","last_undegraded":"2023-09-27T23:33:19.820374+0000","last_fullsized":"2023-09-27T23:33:19.820374+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:34:58.552571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.820067+0000","last_change":"2023-09-27T23:33:19.815313+0000","last_active":"2023-09-27T23:33:19.820067+0000","last_peered":"2023-09-27T23:33:19.820067+0000","last_clean":"2023-09-27T23:33:19.820067+0000","last_became_active":"2023-09-27T23:33:19.815025+0000","last_became_peered":"2023-09-27T23:33:19.815025+0000","last_unstale":"2023-09-27T23:33:19.820067+0000","last_undegraded":"2023-09-27T23:33:19.820067+0000","last_fullsized":"2023-09-27T23:33:19.820067+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:19:49.113506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"142'4","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.808019+0000","last_change":"2023-09-27T23:33:19.431509+0000","last_active":"2023-09-27T23:33:19.808019+0000","last_peered":"2023-09-27T23:33:19.808019+0000","last_clean":"2023-09-27T23:33:19.808019+0000","last_became_active":"2023-09-27T23:33:19.431232+0000","last_became_peered":"2023-09-27T23:33:19.431232+0000","last_unstale":"2023-09-27T23:33:19.808019+0000","last_undegraded":"2023-09-27T23:33:19.808019+0000","last_fullsized":"2023-09-27T23:33:19.808019+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:29.610311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825923+0000","last_change":"2023-09-27T23:33:19.810331+0000","last_active":"2023-09-27T23:33:19.825923+0000","last_peered":"2023-09-27T23:33:19.825923+0000","last_clean":"2023-09-27T23:33:19.825923+0000","last_became_active":"2023-09-27T23:33:19.810041+0000","last_became_peered":"2023-09-27T23:33:19.810041+0000","last_unstale":"2023-09-27T23:33:19.825923+0000","last_undegraded":"2023-09-27T23:33:19.825923+0000","last_fullsized":"2023-09-27T23:33:19.825923+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814781+0000","last_change":"2023-09-27T23:33:19.436486+0000","last_active":"2023-09-27T23:33:19.814781+0000","last_peered":"2023-09-27T23:33:19.814781+0000","last_clean":"2023-09-27T23:33:19.814781+0000","last_became_active":"2023-09-27T23:33:19.436056+0000","last_became_peered":"2023-09-27T23:33:19.436056+0000","last_unstale":"2023-09-27T23:33:19.814781+0000","last_undegraded":"2023-09-27T23:33:19.814781+0000","last_fullsized":"2023-09-27T23:33:19.814781+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815222+0000","last_change":"2023-09-27T23:33:19.812688+0000","last_active":"2023-09-27T23:33:19.815222+0000","last_peered":"2023-09-27T23:33:19.815222+0000","last_clean":"2023-09-27T23:33:19.815222+0000","last_became_active":"2023-09-27T23:33:19.812408+0000","last_became_peered":"2023-09-27T23:33:19.812408+0000","last_unstale":"2023-09-27T23:33:19.815222+0000","last_undegraded":"2023-09-27T23:33:19.815222+0000","last_fullsized":"2023-09-27T23:33:19.815222+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:17.883683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815969+0000","last_change":"2023-09-27T23:33:19.436742+0000","last_active":"2023-09-27T23:33:19.815969+0000","last_peered":"2023-09-27T23:33:19.815969+0000","last_clean":"2023-09-27T23:33:19.815969+0000","last_became_active":"2023-09-27T23:33:19.436395+0000","last_became_peered":"2023-09-27T23:33:19.436395+0000","last_unstale":"2023-09-27T23:33:19.815969+0000","last_undegraded":"2023-09-27T23:33:19.815969+0000","last_fullsized":"2023-09-27T23:33:19.815969+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:16:48.236035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.808536+0000","last_change":"2023-09-27T23:33:19.802405+0000","last_active":"2023-09-27T23:33:19.808536+0000","last_peered":"2023-09-27T23:33:19.808536+0000","last_clean":"2023-09-27T23:33:19.808536+0000","last_became_active":"2023-09-27T23:33:19.802115+0000","last_became_peered":"2023-09-27T23:33:19.802115+0000","last_unstale":"2023-09-27T23:33:19.808536+0000","last_undegraded":"2023-09-27T23:33:19.808536+0000","last_fullsized":"2023-09-27T23:33:19.808536+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:57.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809160+0000","last_change":"2023-09-27T23:33:19.427279+0000","last_active":"2023-09-27T23:33:19.809160+0000","last_peered":"2023-09-27T23:33:19.809160+0000","last_clean":"2023-09-27T23:33:19.809160+0000","last_became_active":"2023-09-27T23:33:19.426915+0000","last_became_peered":"2023-09-27T23:33:19.426915+0000","last_unstale":"2023-09-27T23:33:19.809160+0000","last_undegraded":"2023-09-27T23:33:19.809160+0000","last_fullsized":"2023-09-27T23:33:19.809160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:54.341042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819812+0000","last_change":"2023-09-27T23:33:19.811218+0000","last_active":"2023-09-27T23:33:19.819812+0000","last_peered":"2023-09-27T23:33:19.819812+0000","last_clean":"2023-09-27T23:33:19.819812+0000","last_became_active":"2023-09-27T23:33:19.810963+0000","last_became_peered":"2023-09-27T23:33:19.810963+0000","last_unstale":"2023-09-27T23:33:19.819812+0000","last_undegraded":"2023-09-27T23:33:19.819812+0000","last_fullsized":"2023-09-27T23:33:19.819812+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:49:04.290739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816372+0000","last_change":"2023-09-27T23:33:19.433429+0000","last_active":"2023-09-27T23:33:19.816372+0000","last_peered":"2023-09-27T23:33:19.816372+0000","last_clean":"2023-09-27T23:33:19.816372+0000","last_became_active":"2023-09-27T23:33:19.433030+0000","last_became_peered":"2023-09-27T23:33:19.433030+0000","last_unstale":"2023-09-27T23:33:19.816372+0000","last_undegraded":"2023-09-27T23:33:19.816372+0000","last_fullsized":"2023-09-27T23:33:19.816372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:13:30.439502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814913+0000","last_change":"2023-09-27T23:33:19.810319+0000","last_active":"2023-09-27T23:33:19.814913+0000","last_peered":"2023-09-27T23:33:19.814913+0000","last_clean":"2023-09-27T23:33:19.814913+0000","last_became_active":"2023-09-27T23:33:19.810027+0000","last_became_peered":"2023-09-27T23:33:19.810027+0000","last_unstale":"2023-09-27T23:33:19.814913+0000","last_undegraded":"2023-09-27T23:33:19.814913+0000","last_fullsized":"2023-09-27T23:33:19.814913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:59.536848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824684+0000","last_change":"2023-09-27T23:33:19.434695+0000","last_active":"2023-09-27T23:33:19.824684+0000","last_peered":"2023-09-27T23:33:19.824684+0000","last_clean":"2023-09-27T23:33:19.824684+0000","last_became_active":"2023-09-27T23:33:19.434312+0000","last_became_peered":"2023-09-27T23:33:19.434312+0000","last_unstale":"2023-09-27T23:33:19.824684+0000","last_undegraded":"2023-09-27T23:33:19.824684+0000","last_fullsized":"2023-09-27T23:33:19.824684+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:47:58.614714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826212+0000","last_change":"2023-09-27T23:33:19.819783+0000","last_active":"2023-09-27T23:33:19.826212+0000","last_peered":"2023-09-27T23:33:19.826212+0000","last_clean":"2023-09-27T23:33:19.826212+0000","last_became_active":"2023-09-27T23:33:19.819564+0000","last_became_peered":"2023-09-27T23:33:19.819564+0000","last_unstale":"2023-09-27T23:33:19.826212+0000","last_undegraded":"2023-09-27T23:33:19.826212+0000","last_fullsized":"2023-09-27T23:33:19.826212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:37.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796561+0000","last_change":"2023-09-27T23:33:19.429166+0000","last_active":"2023-09-27T23:33:19.796561+0000","last_peered":"2023-09-27T23:33:19.796561+0000","last_clean":"2023-09-27T23:33:19.796561+0000","last_became_active":"2023-09-27T23:33:19.428861+0000","last_became_peered":"2023-09-27T23:33:19.428861+0000","last_unstale":"2023-09-27T23:33:19.796561+0000","last_undegraded":"2023-09-27T23:33:19.796561+0000","last_fullsized":"2023-09-27T23:33:19.796561+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:10:34.625899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813572+0000","last_change":"2023-09-27T23:33:19.809091+0000","last_active":"2023-09-27T23:33:19.813572+0000","last_peered":"2023-09-27T23:33:19.813572+0000","last_clean":"2023-09-27T23:33:19.813572+0000","last_became_active":"2023-09-27T23:33:19.808763+0000","last_became_peered":"2023-09-27T23:33:19.808763+0000","last_unstale":"2023-09-27T23:33:19.813572+0000","last_undegraded":"2023-09-27T23:33:19.813572+0000","last_fullsized":"2023-09-27T23:33:19.813572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:41.828993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788226+0000","last_change":"2023-09-27T23:33:19.428241+0000","last_active":"2023-09-27T23:33:19.788226+0000","last_peered":"2023-09-27T23:33:19.788226+0000","last_clean":"2023-09-27T23:33:19.788226+0000","last_became_active":"2023-09-27T23:33:19.427908+0000","last_became_peered":"2023-09-27T23:33:19.427908+0000","last_unstale":"2023-09-27T23:33:19.788226+0000","last_undegraded":"2023-09-27T23:33:19.788226+0000","last_fullsized":"2023-09-27T23:33:19.788226+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:49.753965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824232+0000","last_change":"2023-09-27T23:33:19.808815+0000","last_active":"2023-09-27T23:33:19.824232+0000","last_peered":"2023-09-27T23:33:19.824232+0000","last_clean":"2023-09-27T23:33:19.824232+0000","last_became_active":"2023-09-27T23:33:19.808532+0000","last_became_peered":"2023-09-27T23:33:19.808532+0000","last_unstale":"2023-09-27T23:33:19.824232+0000","last_undegraded":"2023-09-27T23:33:19.824232+0000","last_fullsized":"2023-09-27T23:33:19.824232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815149+0000","last_change":"2023-09-27T23:33:19.435202+0000","last_active":"2023-09-27T23:33:19.815149+0000","last_peered":"2023-09-27T23:33:19.815149+0000","last_clean":"2023-09-27T23:33:19.815149+0000","last_became_active":"2023-09-27T23:33:19.432656+0000","last_became_peered":"2023-09-27T23:33:19.432656+0000","last_unstale":"2023-09-27T23:33:19.815149+0000","last_undegraded":"2023-09-27T23:33:19.815149+0000","last_fullsized":"2023-09-27T23:33:19.815149+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817212+0000","last_change":"2023-09-27T23:33:19.811187+0000","last_active":"2023-09-27T23:33:19.817212+0000","last_peered":"2023-09-27T23:33:19.817212+0000","last_clean":"2023-09-27T23:33:19.817212+0000","last_became_active":"2023-09-27T23:33:19.810921+0000","last_became_peered":"2023-09-27T23:33:19.810921+0000","last_unstale":"2023-09-27T23:33:19.817212+0000","last_undegraded":"2023-09-27T23:33:19.817212+0000","last_fullsized":"2023-09-27T23:33:19.817212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:58.020074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.821969+0000","last_change":"2023-09-27T23:33:19.427776+0000","last_active":"2023-09-27T23:33:19.821969+0000","last_peered":"2023-09-27T23:33:19.821969+0000","last_clean":"2023-09-27T23:33:19.821969+0000","last_became_active":"2023-09-27T23:33:19.427494+0000","last_became_peered":"2023-09-27T23:33:19.427494+0000","last_unstale":"2023-09-27T23:33:19.821969+0000","last_undegraded":"2023-09-27T23:33:19.821969+0000","last_fullsized":"2023-09-27T23:33:19.821969+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:54:30.740099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815316+0000","last_change":"2023-09-27T23:33:19.805962+0000","last_active":"2023-09-27T23:33:19.815316+0000","last_peered":"2023-09-27T23:33:19.815316+0000","last_clean":"2023-09-27T23:33:19.815316+0000","last_became_active":"2023-09-27T23:33:19.805667+0000","last_became_peered":"2023-09-27T23:33:19.805667+0000","last_unstale":"2023-09-27T23:33:19.815316+0000","last_undegraded":"2023-09-27T23:33:19.815316+0000","last_fullsized":"2023-09-27T23:33:19.815316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:30:21.924258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822497+0000","last_change":"2023-09-27T23:33:19.438960+0000","last_active":"2023-09-27T23:33:19.822497+0000","last_peered":"2023-09-27T23:33:19.822497+0000","last_clean":"2023-09-27T23:33:19.822497+0000","last_became_active":"2023-09-27T23:33:19.438634+0000","last_became_peered":"2023-09-27T23:33:19.438634+0000","last_unstale":"2023-09-27T23:33:19.822497+0000","last_undegraded":"2023-09-27T23:33:19.822497+0000","last_fullsized":"2023-09-27T23:33:19.822497+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:09:47.783943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.819500+0000","last_change":"2023-09-27T23:33:19.816975+0000","last_active":"2023-09-27T23:33:19.819500+0000","last_peered":"2023-09-27T23:33:19.819500+0000","last_clean":"2023-09-27T23:33:19.819500+0000","last_became_active":"2023-09-27T23:33:19.816573+0000","last_became_peered":"2023-09-27T23:33:19.816573+0000","last_unstale":"2023-09-27T23:33:19.819500+0000","last_undegraded":"2023-09-27T23:33:19.819500+0000","last_fullsized":"2023-09-27T23:33:19.819500+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:46:14.483620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790958+0000","last_change":"2023-09-27T23:33:19.435177+0000","last_active":"2023-09-27T23:33:19.790958+0000","last_peered":"2023-09-27T23:33:19.790958+0000","last_clean":"2023-09-27T23:33:19.790958+0000","last_became_active":"2023-09-27T23:33:19.432614+0000","last_became_peered":"2023-09-27T23:33:19.432614+0000","last_unstale":"2023-09-27T23:33:19.790958+0000","last_undegraded":"2023-09-27T23:33:19.790958+0000","last_fullsized":"2023-09-27T23:33:19.790958+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:25:07.369858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816845+0000","last_change":"2023-09-27T23:33:19.810499+0000","last_active":"2023-09-27T23:33:19.816845+0000","last_peered":"2023-09-27T23:33:19.816845+0000","last_clean":"2023-09-27T23:33:19.816845+0000","last_became_active":"2023-09-27T23:33:19.810241+0000","last_became_peered":"2023-09-27T23:33:19.810241+0000","last_unstale":"2023-09-27T23:33:19.816845+0000","last_undegraded":"2023-09-27T23:33:19.816845+0000","last_fullsized":"2023-09-27T23:33:19.816845+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:52:01.755843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816627+0000","last_change":"2023-09-27T23:33:19.436255+0000","last_active":"2023-09-27T23:33:19.816627+0000","last_peered":"2023-09-27T23:33:19.816627+0000","last_clean":"2023-09-27T23:33:19.816627+0000","last_became_active":"2023-09-27T23:33:19.435903+0000","last_became_peered":"2023-09-27T23:33:19.435903+0000","last_unstale":"2023-09-27T23:33:19.816627+0000","last_undegraded":"2023-09-27T23:33:19.816627+0000","last_fullsized":"2023-09-27T23:33:19.816627+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:01.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826471+0000","last_change":"2023-09-27T23:33:19.824931+0000","last_active":"2023-09-27T23:33:19.826471+0000","last_peered":"2023-09-27T23:33:19.826471+0000","last_clean":"2023-09-27T23:33:19.826471+0000","last_became_active":"2023-09-27T23:33:19.824656+0000","last_became_peered":"2023-09-27T23:33:19.824656+0000","last_unstale":"2023-09-27T23:33:19.826471+0000","last_undegraded":"2023-09-27T23:33:19.826471+0000","last_fullsized":"2023-09-27T23:33:19.826471+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:18:51.294221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.803050+0000","last_change":"2023-09-27T23:33:19.430029+0000","last_active":"2023-09-27T23:33:19.803050+0000","last_peered":"2023-09-27T23:33:19.803050+0000","last_clean":"2023-09-27T23:33:19.803050+0000","last_became_active":"2023-09-27T23:33:19.429572+0000","last_became_peered":"2023-09-27T23:33:19.429572+0000","last_unstale":"2023-09-27T23:33:19.803050+0000","last_undegraded":"2023-09-27T23:33:19.803050+0000","last_fullsized":"2023-09-27T23:33:19.803050+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:30:36.384182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.813432+0000","last_change":"2023-09-27T23:33:19.809133+0000","last_active":"2023-09-27T23:33:19.813432+0000","last_peered":"2023-09-27T23:33:19.813432+0000","last_clean":"2023-09-27T23:33:19.813432+0000","last_became_active":"2023-09-27T23:33:19.806861+0000","last_became_peered":"2023-09-27T23:33:19.806861+0000","last_unstale":"2023-09-27T23:33:19.813432+0000","last_undegraded":"2023-09-27T23:33:19.813432+0000","last_fullsized":"2023-09-27T23:33:19.813432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:46:39.679217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.795787+0000","last_change":"2023-09-27T23:33:19.432595+0000","last_active":"2023-09-27T23:33:19.795787+0000","last_peered":"2023-09-27T23:33:19.795787+0000","last_clean":"2023-09-27T23:33:19.795787+0000","last_became_active":"2023-09-27T23:33:19.432271+0000","last_became_peered":"2023-09-27T23:33:19.432271+0000","last_unstale":"2023-09-27T23:33:19.795787+0000","last_undegraded":"2023-09-27T23:33:19.795787+0000","last_fullsized":"2023-09-27T23:33:19.795787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:20:16.879676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814323+0000","last_change":"2023-09-27T23:33:19.805969+0000","last_active":"2023-09-27T23:33:19.814323+0000","last_peered":"2023-09-27T23:33:19.814323+0000","last_clean":"2023-09-27T23:33:19.814323+0000","last_became_active":"2023-09-27T23:33:19.805294+0000","last_became_peered":"2023-09-27T23:33:19.805294+0000","last_unstale":"2023-09-27T23:33:19.814323+0000","last_undegraded":"2023-09-27T23:33:19.814323+0000","last_fullsized":"2023-09-27T23:33:19.814323+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:26.938542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.794866+0000","last_change":"2023-09-27T23:33:19.427838+0000","last_active":"2023-09-27T23:33:19.794866+0000","last_peered":"2023-09-27T23:33:19.794866+0000","last_clean":"2023-09-27T23:33:19.794866+0000","last_became_active":"2023-09-27T23:33:19.427524+0000","last_became_peered":"2023-09-27T23:33:19.427524+0000","last_unstale":"2023-09-27T23:33:19.794866+0000","last_undegraded":"2023-09-27T23:33:19.794866+0000","last_fullsized":"2023-09-27T23:33:19.794866+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:07:22.859377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816803+0000","last_change":"2023-09-27T23:33:19.811653+0000","last_active":"2023-09-27T23:33:19.816803+0000","last_peered":"2023-09-27T23:33:19.816803+0000","last_clean":"2023-09-27T23:33:19.816803+0000","last_became_active":"2023-09-27T23:33:19.811327+0000","last_became_peered":"2023-09-27T23:33:19.811327+0000","last_unstale":"2023-09-27T23:33:19.816803+0000","last_undegraded":"2023-09-27T23:33:19.816803+0000","last_fullsized":"2023-09-27T23:33:19.816803+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:25:37.144907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815834+0000","last_change":"2023-09-27T23:33:19.434317+0000","last_active":"2023-09-27T23:33:19.815834+0000","last_peered":"2023-09-27T23:33:19.815834+0000","last_clean":"2023-09-27T23:33:19.815834+0000","last_became_active":"2023-09-27T23:33:19.434002+0000","last_became_peered":"2023-09-27T23:33:19.434002+0000","last_unstale":"2023-09-27T23:33:19.815834+0000","last_undegraded":"2023-09-27T23:33:19.815834+0000","last_fullsized":"2023-09-27T23:33:19.815834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:17:04.919326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.827942+0000","last_change":"2023-09-27T23:33:19.822884+0000","last_active":"2023-09-27T23:33:19.827942+0000","last_peered":"2023-09-27T23:33:19.827942+0000","last_clean":"2023-09-27T23:33:19.827942+0000","last_became_active":"2023-09-27T23:33:19.822472+0000","last_became_peered":"2023-09-27T23:33:19.822472+0000","last_unstale":"2023-09-27T23:33:19.827942+0000","last_undegraded":"2023-09-27T23:33:19.827942+0000","last_fullsized":"2023-09-27T23:33:19.827942+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:52:33.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793144+0000","last_change":"2023-09-27T23:33:19.430327+0000","last_active":"2023-09-27T23:33:19.793144+0000","last_peered":"2023-09-27T23:33:19.793144+0000","last_clean":"2023-09-27T23:33:19.793144+0000","last_became_active":"2023-09-27T23:33:19.430043+0000","last_became_peered":"2023-09-27T23:33:19.430043+0000","last_unstale":"2023-09-27T23:33:19.793144+0000","last_undegraded":"2023-09-27T23:33:19.793144+0000","last_fullsized":"2023-09-27T23:33:19.793144+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:21:57.016067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.809717+0000","last_change":"2023-09-27T23:33:19.804614+0000","last_active":"2023-09-27T23:33:19.809717+0000","last_peered":"2023-09-27T23:33:19.809717+0000","last_clean":"2023-09-27T23:33:19.809717+0000","last_became_active":"2023-09-27T23:33:19.804328+0000","last_became_peered":"2023-09-27T23:33:19.804328+0000","last_unstale":"2023-09-27T23:33:19.809717+0000","last_undegraded":"2023-09-27T23:33:19.809717+0000","last_fullsized":"2023-09-27T23:33:19.809717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:06.058928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788038+0000","last_change":"2023-09-27T23:33:19.422966+0000","last_active":"2023-09-27T23:33:19.788038+0000","last_peered":"2023-09-27T23:33:19.788038+0000","last_clean":"2023-09-27T23:33:19.788038+0000","last_became_active":"2023-09-27T23:33:19.422649+0000","last_became_peered":"2023-09-27T23:33:19.422649+0000","last_unstale":"2023-09-27T23:33:19.788038+0000","last_undegraded":"2023-09-27T23:33:19.788038+0000","last_fullsized":"2023-09-27T23:33:19.788038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:04:02.441972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.820346+0000","last_change":"2023-09-27T23:33:19.815980+0000","last_active":"2023-09-27T23:33:19.820346+0000","last_peered":"2023-09-27T23:33:19.820346+0000","last_clean":"2023-09-27T23:33:19.820346+0000","last_became_active":"2023-09-27T23:33:19.815700+0000","last_became_peered":"2023-09-27T23:33:19.815700+0000","last_unstale":"2023-09-27T23:33:19.820346+0000","last_undegraded":"2023-09-27T23:33:19.820346+0000","last_fullsized":"2023-09-27T23:33:19.820346+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:41:11.095627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793216+0000","last_change":"2023-09-27T23:33:19.434165+0000","last_active":"2023-09-27T23:33:19.793216+0000","last_peered":"2023-09-27T23:33:19.793216+0000","last_clean":"2023-09-27T23:33:19.793216+0000","last_became_active":"2023-09-27T23:33:19.433869+0000","last_became_peered":"2023-09-27T23:33:19.433869+0000","last_unstale":"2023-09-27T23:33:19.793216+0000","last_undegraded":"2023-09-27T23:33:19.793216+0000","last_fullsized":"2023-09-27T23:33:19.793216+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:52.914410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814499+0000","last_change":"2023-09-27T23:33:19.808152+0000","last_active":"2023-09-27T23:33:19.814499+0000","last_peered":"2023-09-27T23:33:19.814499+0000","last_clean":"2023-09-27T23:33:19.814499+0000","last_became_active":"2023-09-27T23:33:19.807830+0000","last_became_peered":"2023-09-27T23:33:19.807830+0000","last_unstale":"2023-09-27T23:33:19.814499+0000","last_undegraded":"2023-09-27T23:33:19.814499+0000","last_fullsized":"2023-09-27T23:33:19.814499+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:28:28.899181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.786802+0000","last_change":"2023-09-27T23:33:19.424649+0000","last_active":"2023-09-27T23:33:19.786802+0000","last_peered":"2023-09-27T23:33:19.786802+0000","last_clean":"2023-09-27T23:33:19.786802+0000","last_became_active":"2023-09-27T23:33:19.424230+0000","last_became_peered":"2023-09-27T23:33:19.424230+0000","last_unstale":"2023-09-27T23:33:19.786802+0000","last_undegraded":"2023-09-27T23:33:19.786802+0000","last_fullsized":"2023-09-27T23:33:19.786802+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:46:54.150917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814735+0000","last_change":"2023-09-27T23:33:19.807080+0000","last_active":"2023-09-27T23:33:19.814735+0000","last_peered":"2023-09-27T23:33:19.814735+0000","last_clean":"2023-09-27T23:33:19.814735+0000","last_became_active":"2023-09-27T23:33:19.806755+0000","last_became_peered":"2023-09-27T23:33:19.806755+0000","last_unstale":"2023-09-27T23:33:19.814735+0000","last_undegraded":"2023-09-27T23:33:19.814735+0000","last_fullsized":"2023-09-27T23:33:19.814735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:03:57.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793447+0000","last_change":"2023-09-27T23:33:19.433717+0000","last_active":"2023-09-27T23:33:19.793447+0000","last_peered":"2023-09-27T23:33:19.793447+0000","last_clean":"2023-09-27T23:33:19.793447+0000","last_became_active":"2023-09-27T23:33:19.433373+0000","last_became_peered":"2023-09-27T23:33:19.433373+0000","last_unstale":"2023-09-27T23:33:19.793447+0000","last_undegraded":"2023-09-27T23:33:19.793447+0000","last_fullsized":"2023-09-27T23:33:19.793447+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:37:05.142294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828131+0000","last_change":"2023-09-27T23:33:19.821633+0000","last_active":"2023-09-27T23:33:19.828131+0000","last_peered":"2023-09-27T23:33:19.828131+0000","last_clean":"2023-09-27T23:33:19.828131+0000","last_became_active":"2023-09-27T23:33:19.821291+0000","last_became_peered":"2023-09-27T23:33:19.821291+0000","last_unstale":"2023-09-27T23:33:19.828131+0000","last_undegraded":"2023-09-27T23:33:19.828131+0000","last_fullsized":"2023-09-27T23:33:19.828131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:46:20.943175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.795895+0000","last_change":"2023-09-27T23:33:19.432266+0000","last_active":"2023-09-27T23:33:19.795895+0000","last_peered":"2023-09-27T23:33:19.795895+0000","last_clean":"2023-09-27T23:33:19.795895+0000","last_became_active":"2023-09-27T23:33:19.430799+0000","last_became_peered":"2023-09-27T23:33:19.430799+0000","last_unstale":"2023-09-27T23:33:19.795895+0000","last_undegraded":"2023-09-27T23:33:19.795895+0000","last_fullsized":"2023-09-27T23:33:19.795895+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:09:28.682181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806466+0000","last_change":"2023-09-27T23:33:19.804496+0000","last_active":"2023-09-27T23:33:19.806466+0000","last_peered":"2023-09-27T23:33:19.806466+0000","last_clean":"2023-09-27T23:33:19.806466+0000","last_became_active":"2023-09-27T23:33:19.804186+0000","last_became_peered":"2023-09-27T23:33:19.804186+0000","last_unstale":"2023-09-27T23:33:19.806466+0000","last_undegraded":"2023-09-27T23:33:19.806466+0000","last_fullsized":"2023-09-27T23:33:19.806466+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:59:09.351560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.794961+0000","last_change":"2023-09-27T23:33:19.430707+0000","last_active":"2023-09-27T23:33:19.794961+0000","last_peered":"2023-09-27T23:33:19.794961+0000","last_clean":"2023-09-27T23:33:19.794961+0000","last_became_active":"2023-09-27T23:33:19.430423+0000","last_became_peered":"2023-09-27T23:33:19.430423+0000","last_unstale":"2023-09-27T23:33:19.794961+0000","last_undegraded":"2023-09-27T23:33:19.794961+0000","last_fullsized":"2023-09-27T23:33:19.794961+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:58.532840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.814478+0000","last_change":"2023-09-27T23:33:19.804146+0000","last_active":"2023-09-27T23:33:19.814478+0000","last_peered":"2023-09-27T23:33:19.814478+0000","last_clean":"2023-09-27T23:33:19.814478+0000","last_became_active":"2023-09-27T23:33:19.803861+0000","last_became_peered":"2023-09-27T23:33:19.803861+0000","last_unstale":"2023-09-27T23:33:19.814478+0000","last_undegraded":"2023-09-27T23:33:19.814478+0000","last_fullsized":"2023-09-27T23:33:19.814478+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:18:43.577553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.806946+0000","last_change":"2023-09-27T23:33:19.436051+0000","last_active":"2023-09-27T23:33:19.806946+0000","last_peered":"2023-09-27T23:33:19.806946+0000","last_clean":"2023-09-27T23:33:19.806946+0000","last_became_active":"2023-09-27T23:33:19.435770+0000","last_became_peered":"2023-09-27T23:33:19.435770+0000","last_unstale":"2023-09-27T23:33:19.806946+0000","last_undegraded":"2023-09-27T23:33:19.806946+0000","last_fullsized":"2023-09-27T23:33:19.806946+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:14:12.478687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"131'249","reported_seq":740,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792090+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:19.792090+0000","last_peered":"2023-09-27T23:33:19.792090+0000","last_clean":"2023-09-27T23:33:19.792090+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:19.792090+0000","last_undegraded":"2023-09-27T23:33:19.792090+0000","last_fullsized":"2023-09-27T23:33:19.792090+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":249,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790633+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:19.790633+0000","last_peered":"2023-09-27T23:33:19.790633+0000","last_clean":"2023-09-27T23:33:19.790633+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:19.790633+0000","last_undegraded":"2023-09-27T23:33:19.790633+0000","last_fullsized":"2023-09-27T23:33:19.790633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.826305+0000","last_change":"2023-09-27T23:33:19.818550+0000","last_active":"2023-09-27T23:33:19.826305+0000","last_peered":"2023-09-27T23:33:19.826305+0000","last_clean":"2023-09-27T23:33:19.826305+0000","last_became_active":"2023-09-27T23:33:19.818313+0000","last_became_peered":"2023-09-27T23:33:19.818313+0000","last_unstale":"2023-09-27T23:33:19.826305+0000","last_undegraded":"2023-09-27T23:33:19.826305+0000","last_fullsized":"2023-09-27T23:33:19.826305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"142'8","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.825230+0000","last_change":"2023-09-27T23:33:19.439141+0000","last_active":"2023-09-27T23:33:19.825230+0000","last_peered":"2023-09-27T23:33:19.825230+0000","last_clean":"2023-09-27T23:33:19.825230+0000","last_became_active":"2023-09-27T23:33:19.438828+0000","last_became_peered":"2023-09-27T23:33:19.438828+0000","last_unstale":"2023-09-27T23:33:19.825230+0000","last_undegraded":"2023-09-27T23:33:19.825230+0000","last_fullsized":"2023-09-27T23:33:19.825230+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:33.463327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789544+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:19.789544+0000","last_peered":"2023-09-27T23:33:19.789544+0000","last_clean":"2023-09-27T23:33:19.789544+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:19.789544+0000","last_undegraded":"2023-09-27T23:33:19.789544+0000","last_fullsized":"2023-09-27T23:33:19.789544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816597+0000","last_change":"2023-09-27T23:33:19.812801+0000","last_active":"2023-09-27T23:33:19.816597+0000","last_peered":"2023-09-27T23:33:19.816597+0000","last_clean":"2023-09-27T23:33:19.816597+0000","last_became_active":"2023-09-27T23:33:19.812506+0000","last_became_peered":"2023-09-27T23:33:19.812506+0000","last_unstale":"2023-09-27T23:33:19.816597+0000","last_undegraded":"2023-09-27T23:33:19.816597+0000","last_fullsized":"2023-09-27T23:33:19.816597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:16:22.761586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788828+0000","last_change":"2023-09-27T23:33:19.424648+0000","last_active":"2023-09-27T23:33:19.788828+0000","last_peered":"2023-09-27T23:33:19.788828+0000","last_clean":"2023-09-27T23:33:19.788828+0000","last_became_active":"2023-09-27T23:33:19.424271+0000","last_became_peered":"2023-09-27T23:33:19.424271+0000","last_unstale":"2023-09-27T23:33:19.788828+0000","last_undegraded":"2023-09-27T23:33:19.788828+0000","last_fullsized":"2023-09-27T23:33:19.788828+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:25.699402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798546+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:19.798546+0000","last_peered":"2023-09-27T23:33:19.798546+0000","last_clean":"2023-09-27T23:33:19.798546+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:19.798546+0000","last_undegraded":"2023-09-27T23:33:19.798546+0000","last_fullsized":"2023-09-27T23:33:19.798546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817238+0000","last_change":"2023-09-27T23:33:19.813654+0000","last_active":"2023-09-27T23:33:19.817238+0000","last_peered":"2023-09-27T23:33:19.817238+0000","last_clean":"2023-09-27T23:33:19.817238+0000","last_became_active":"2023-09-27T23:33:19.813363+0000","last_became_peered":"2023-09-27T23:33:19.813363+0000","last_unstale":"2023-09-27T23:33:19.817238+0000","last_undegraded":"2023-09-27T23:33:19.817238+0000","last_fullsized":"2023-09-27T23:33:19.817238+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:30:09.337371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.798813+0000","last_change":"2023-09-27T23:33:19.432418+0000","last_active":"2023-09-27T23:33:19.798813+0000","last_peered":"2023-09-27T23:33:19.798813+0000","last_clean":"2023-09-27T23:33:19.798813+0000","last_became_active":"2023-09-27T23:33:19.428641+0000","last_became_peered":"2023-09-27T23:33:19.428641+0000","last_unstale":"2023-09-27T23:33:19.798813+0000","last_undegraded":"2023-09-27T23:33:19.798813+0000","last_fullsized":"2023-09-27T23:33:19.798813+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:38:31.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.790099+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:19.790099+0000","last_peered":"2023-09-27T23:33:19.790099+0000","last_clean":"2023-09-27T23:33:19.790099+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:19.790099+0000","last_undegraded":"2023-09-27T23:33:19.790099+0000","last_fullsized":"2023-09-27T23:33:19.790099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.815270+0000","last_change":"2023-09-27T23:33:19.809693+0000","last_active":"2023-09-27T23:33:19.815270+0000","last_peered":"2023-09-27T23:33:19.815270+0000","last_clean":"2023-09-27T23:33:19.815270+0000","last_became_active":"2023-09-27T23:33:19.809423+0000","last_became_peered":"2023-09-27T23:33:19.809423+0000","last_unstale":"2023-09-27T23:33:19.815270+0000","last_undegraded":"2023-09-27T23:33:19.815270+0000","last_fullsized":"2023-09-27T23:33:19.815270+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:21:41.409607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"142'6","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.828563+0000","last_change":"2023-09-27T23:33:19.430328+0000","last_active":"2023-09-27T23:33:19.828563+0000","last_peered":"2023-09-27T23:33:19.828563+0000","last_clean":"2023-09-27T23:33:19.828563+0000","last_became_active":"2023-09-27T23:33:19.430033+0000","last_became_peered":"2023-09-27T23:33:19.430033+0000","last_unstale":"2023-09-27T23:33:19.828563+0000","last_undegraded":"2023-09-27T23:33:19.828563+0000","last_fullsized":"2023-09-27T23:33:19.828563+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:49:48.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.789643+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:19.789643+0000","last_peered":"2023-09-27T23:33:19.789643+0000","last_clean":"2023-09-27T23:33:19.789643+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:19.789643+0000","last_undegraded":"2023-09-27T23:33:19.789643+0000","last_fullsized":"2023-09-27T23:33:19.789643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.824588+0000","last_change":"2023-09-27T23:33:19.823201+0000","last_active":"2023-09-27T23:33:19.824588+0000","last_peered":"2023-09-27T23:33:19.824588+0000","last_clean":"2023-09-27T23:33:19.824588+0000","last_became_active":"2023-09-27T23:33:19.822894+0000","last_became_peered":"2023-09-27T23:33:19.822894+0000","last_unstale":"2023-09-27T23:33:19.824588+0000","last_undegraded":"2023-09-27T23:33:19.824588+0000","last_fullsized":"2023-09-27T23:33:19.824588+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:09:40.094617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"142'3","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.822697+0000","last_change":"2023-09-27T23:33:19.438806+0000","last_active":"2023-09-27T23:33:19.822697+0000","last_peered":"2023-09-27T23:33:19.822697+0000","last_clean":"2023-09-27T23:33:19.822697+0000","last_became_active":"2023-09-27T23:33:19.438541+0000","last_became_peered":"2023-09-27T23:33:19.438541+0000","last_unstale":"2023-09-27T23:33:19.822697+0000","last_undegraded":"2023-09-27T23:33:19.822697+0000","last_fullsized":"2023-09-27T23:33:19.822697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:52:29.683418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.793521+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:19.793521+0000","last_peered":"2023-09-27T23:33:19.793521+0000","last_clean":"2023-09-27T23:33:19.793521+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:19.793521+0000","last_undegraded":"2023-09-27T23:33:19.793521+0000","last_fullsized":"2023-09-27T23:33:19.793521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.817658+0000","last_change":"2023-09-27T23:33:19.815111+0000","last_active":"2023-09-27T23:33:19.817658+0000","last_peered":"2023-09-27T23:33:19.817658+0000","last_clean":"2023-09-27T23:33:19.817658+0000","last_became_active":"2023-09-27T23:33:19.814866+0000","last_became_peered":"2023-09-27T23:33:19.814866+0000","last_unstale":"2023-09-27T23:33:19.817658+0000","last_undegraded":"2023-09-27T23:33:19.817658+0000","last_fullsized":"2023-09-27T23:33:19.817658+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:36:30.614318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.800186+0000","last_change":"2023-09-27T23:33:19.433336+0000","last_active":"2023-09-27T23:33:19.800186+0000","last_peered":"2023-09-27T23:33:19.800186+0000","last_clean":"2023-09-27T23:33:19.800186+0000","last_became_active":"2023-09-27T23:33:19.433032+0000","last_became_peered":"2023-09-27T23:33:19.433032+0000","last_unstale":"2023-09-27T23:33:19.800186+0000","last_undegraded":"2023-09-27T23:33:19.800186+0000","last_fullsized":"2023-09-27T23:33:19.800186+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:56.817223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.788544+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:19.788544+0000","last_peered":"2023-09-27T23:33:19.788544+0000","last_clean":"2023-09-27T23:33:19.788544+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:19.788544+0000","last_undegraded":"2023-09-27T23:33:19.788544+0000","last_fullsized":"2023-09-27T23:33:19.788544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.816760+0000","last_change":"2023-09-27T23:33:19.807831+0000","last_active":"2023-09-27T23:33:19.816760+0000","last_peered":"2023-09-27T23:33:19.816760+0000","last_clean":"2023-09-27T23:33:19.816760+0000","last_became_active":"2023-09-27T23:33:19.807647+0000","last_became_peered":"2023-09-27T23:33:19.807647+0000","last_unstale":"2023-09-27T23:33:19.816760+0000","last_undegraded":"2023-09-27T23:33:19.816760+0000","last_fullsized":"2023-09-27T23:33:19.816760+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:19.389362+0000","last_clean_scrub_stamp":"2023-09-27T23:33:19.389362+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:36:34.467334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.792775+0000","last_change":"2023-09-27T23:33:19.438063+0000","last_active":"2023-09-27T23:33:19.792775+0000","last_peered":"2023-09-27T23:33:19.792775+0000","last_clean":"2023-09-27T23:33:19.792775+0000","last_became_active":"2023-09-27T23:33:19.437740+0000","last_became_peered":"2023-09-27T23:33:19.437740+0000","last_unstale":"2023-09-27T23:33:19.792775+0000","last_undegraded":"2023-09-27T23:33:19.792775+0000","last_fullsized":"2023-09-27T23:33:19.792775+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:18.385355+0000","last_clean_scrub_stamp":"2023-09-27T23:33:18.385355+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:01.021179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-09-27T23:33:19.796971+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:19.796971+0000","last_peered":"2023-09-27T23:33:19.796971+0000","last_clean":"2023-09-27T23:33:19.796971+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:19.796971+0000","last_undegraded":"2023-09-27T23:33:19.796971+0000","last_fullsized":"2023-09-27T23:33:19.796971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509677,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":2332,"kb_used_omap":0,"kb_used_meta":25408,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":2387968,"data_stored":1514870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26017792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.77900000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":0.82099999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0429999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":0.86899999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.083},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":0.73899999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26368,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1781760,"data_stored":923892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27000832},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":1.155},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":1.018},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.345},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.4079999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509676,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37776,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":36032,"kb_avail":94334064,"statfs":{"total":96636764160,"available":96598081536,"internally_reserved":0,"allocated":1777664,"data_stored":922468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36896768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":1.242}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.995},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.4950000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509677,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23500,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":21760,"kb_avail":94348340,"statfs":{"total":96636764160,"available":96612700160,"internally_reserved":0,"allocated":1773568,"data_stored":922426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22282240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":0.83699999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":1.135}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.036},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.107}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.228}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.1699999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509676,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28560,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94343280,"statfs":{"total":96636764160,"available":96607518720,"internally_reserved":0,"allocated":1777664,"data_stored":922516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.034}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.161}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":1.1899999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.006},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.5389999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.3},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.3220000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.1100000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509677,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22672,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":94349168,"statfs":{"total":96636764160,"available":96613548032,"internally_reserved":0,"allocated":1777664,"data_stored":922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.64300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.498},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":0.93000000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":0.95999999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.01},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542383,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37896,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":36160,"kb_avail":94333944,"statfs":{"total":96636764160,"available":96597958656,"internally_reserved":0,"allocated":1769472,"data_stored":922356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37027840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":1.026}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.171},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.302},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.119}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.27},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2470000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509678,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29068,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":26752,"kb_avail":94342772,"statfs":{"total":96636764160,"available":96606998528,"internally_reserved":0,"allocated":2363392,"data_stored":1512724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27394048},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.624},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.998}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":0.94199999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.105},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.341}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.5269999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.28}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:28.539 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:33:28.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:33:28.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde7173b700 1 Processor -- start 2023-09-27T23:33:28.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde7173b700 1 -- start start 2023-09-27T23:33:28.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde7173b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde7173b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0aff50 con 0x7fde6c155420 2023-09-27T23:33:28.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde7173b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0af380 con 0x7fde6c06b5d0 2023-09-27T23:33:28.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde7173b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0c95b0 con 0x7fde6c1557f0 2023-09-27T23:33:28.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde6affd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fde6c155420 0x7fde6c06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60614/0 (socket says 172.21.15.64:60614) 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde6affd700 1 -- 172.21.15.64:0/848093052 learned_addr learned my addr 172.21.15.64:0/848093052 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.663+0000 7fde6a7fc700 1 --2- 172.21.15.64:0/848093052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247203410 0 0) 0x7fde6c0aff50 con 0x7fde6c155420 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde54003570 con 0x7fde6c155420 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723822119 0 0) 0x7fde6c0af380 con 0x7fde6c06b5d0 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde6c0aff50 con 0x7fde6c06b5d0 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde6a7fc700 1 -- 172.21.15.64:0/848093052 >> v1:172.21.15.123:6789/0 conn(0x7fde6c06b5d0 legacy=0x7fde6c0b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde6a7fc700 1 -- 172.21.15.64:0/848093052 >> v1:172.21.15.64:6789/0 conn(0x7fde6c155420 legacy=0x7fde6c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde6a7fc700 1 -- 172.21.15.64:0/848093052 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0af620 con 0x7fde6c1557f0 2023-09-27T23:33:28.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde6a7fc700 1 --2- 172.21.15.64:0/848093052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c06b090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde60009910 tx=0x7fde600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1963431e220a5e03 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60011630 con 0x7fde6c1557f0 2023-09-27T23:33:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde60011790 con 0x7fde6c1557f0 2023-09-27T23:33:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde69ffb700 1 -- 172.21.15.64:0/848093052 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60011b30 con 0x7fde6c1557f0 2023-09-27T23:33:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 -- 172.21.15.64:0/848093052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 msgr2=0x7fde6c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 --2- 172.21.15.64:0/848093052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c06b090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde60009910 tx=0x7fde600091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 -- 172.21.15.64:0/848093052 shutdown_connections 2023-09-27T23:33:28.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 --2- 172.21.15.64:0/848093052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c06b090 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 -- 172.21.15.64:0/848093052 >> 172.21.15.64:0/848093052 conn(0x7fde6c0cc580 msgr2=0x7fde6c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:28.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 -- 172.21.15.64:0/848093052 shutdown_connections 2023-09-27T23:33:28.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.667+0000 7fde7173b700 1 -- 172.21.15.64:0/848093052 wait complete. 2023-09-27T23:33:28.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 Processor -- start 2023-09-27T23:33:28.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 -- start start 2023-09-27T23:33:28.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 0x7fde6c14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6c155420 0x7fde6c0aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c13ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c15afa0 con 0x7fde6c06b5d0 2023-09-27T23:33:28.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0ac740 con 0x7fde6c155420 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde7173b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0abee0 con 0x7fde6c1557f0 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 0x7fde6c14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 0x7fde6c14cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54258/0 (socket says 172.21.15.64:54258) 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 -- 172.21.15.64:0/2784466865 learned_addr learned my addr 172.21.15.64:0/2784466865 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6a7fc700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6c155420 0x7fde6c0aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6b7fe700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c13ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 -- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 msgr2=0x7fde6c13ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c13ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 -- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6c155420 msgr2=0x7fde6c0aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6c155420 0x7fde6c0aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.671+0000 7fde6affd700 1 -- 172.21.15.64:0/2784466865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60009420 con 0x7fde6c06b5d0 2023-09-27T23:33:28.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde6affd700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 0x7fde6c14cee0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00c7c0 tx=0x7fde5c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c005b30 con 0x7fde6c06b5d0 2023-09-27T23:33:28.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c005cd0 con 0x7fde6c06b5d0 2023-09-27T23:33:28.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c0ac180 con 0x7fde6c06b5d0 2023-09-27T23:33:28.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c007320 con 0x7fde6c06b5d0 2023-09-27T23:33:28.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c14d420 con 0x7fde6c06b5d0 2023-09-27T23:33:28.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c15ad70 con 0x7fde6c06b5d0 2023-09-27T23:33:28.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fde5c012400 con 0x7fde6c06b5d0 2023-09-27T23:33:28.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.675+0000 7fde52ffd700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde58060a70 0x7fde58062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:28.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.679+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fde5c07c490 con 0x7fde6c06b5d0 2023-09-27T23:33:28.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.679+0000 7fde6a7fc700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde58060a70 0x7fde58062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:28.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.679+0000 7fde6a7fc700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde58060a70 0x7fde58062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde60009fd0 tx=0x7fde60017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:28.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.687+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fde5c049bb0 con 0x7fde6c06b5d0 2023-09-27T23:33:28.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.903+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde6c071120 con 0x7fde6c06b5d0 2023-09-27T23:33:28.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde52ffd700 1 -- 172.21.15.64:0/2784466865 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7fde5c04ffa0 con 0x7fde6c06b5d0 2023-09-27T23:33:28.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde58060a70 msgr2=0x7fde58062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde58060a70 0x7fde58062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde60009fd0 tx=0x7fde60017000 comp rx=0 tx=0).stop 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 msgr2=0x7fde6c14cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 0x7fde6c14cee0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00c7c0 tx=0x7fde5c007e80 comp rx=0 tx=0).stop 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 shutdown_connections 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fde58060a70 0x7fde58062f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde6c1557f0 0x7fde6c13ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde6c155420 0x7fde6c0aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 --2- 172.21.15.64:0/2784466865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde6c06b5d0 0x7fde6c14cee0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:28.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 >> 172.21.15.64:0/2784466865 conn(0x7fde6c0cc580 msgr2=0x7fde6c150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 shutdown_connections 2023-09-27T23:33:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:28.915+0000 7fde7173b700 1 -- 172.21.15.64:0/2784466865 wait complete. 2023-09-27T23:33:28.921 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:33:28.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:29.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.031+0000 7fea09d61700 1 Processor -- start 2023-09-27T23:33:29.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.031+0000 7fea09d61700 1 -- start start 2023-09-27T23:33:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fea0413f890 0x7fea040aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea040680f0 0x7fea04068510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0406c9f0 con 0x7fea040c2980 2023-09-27T23:33:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040c7f20 con 0x7fea0413f890 2023-09-27T23:33:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea04068a50 con 0x7fea040680f0 2023-09-27T23:33:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea037fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fea0413f890 0x7fea040aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea02ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fea040c2980 0x7fea040c1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60624/0 (socket says 172.21.15.64:60624) 2023-09-27T23:33:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea02ffd700 1 -- 172.21.15.64:0/3816170203 learned_addr learned my addr 172.21.15.64:0/3816170203 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea03fff700 1 --2- 172.21.15.64:0/3816170203 >> v2:172.21.15.123:3300/0 conn(0x7fea040680f0 0x7fea04068510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea037fe700 1 -- 172.21.15.64:0/3816170203 >> v2:172.21.15.123:3300/0 conn(0x7fea040680f0 msgr2=0x7fea04068510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea037fe700 1 --2- 172.21.15.64:0/3816170203 >> v2:172.21.15.123:3300/0 conn(0x7fea040680f0 0x7fea04068510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea027fc700 1 -- 172.21.15.64:0/3816170203 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370357567 0 0) 0x7fea0406c9f0 con 0x7fea040c2980 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea037fe700 1 -- 172.21.15.64:0/3816170203 >> v1:172.21.15.64:6789/0 conn(0x7fea040c2980 legacy=0x7fea040c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea037fe700 1 -- 172.21.15.64:0/3816170203 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040c1c50 con 0x7fea0413f890 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea037fe700 1 --2- 172.21.15.64:0/3816170203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0413f890 0x7fea040aeba0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fea0406a6a0 tx=0x7fe9f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fdca1aefc893fd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea027fc700 1 -- 172.21.15.64:0/3816170203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8011630 con 0x7fea0413f890 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea027fc700 1 -- 172.21.15.64:0/3816170203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8011790 con 0x7fea0413f890 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea027fc700 1 -- 172.21.15.64:0/3816170203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8011b30 con 0x7fea0413f890 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- 172.21.15.64:0/3816170203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0413f890 msgr2=0x7fea040aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 --2- 172.21.15.64:0/3816170203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0413f890 0x7fea040aeba0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fea0406a6a0 tx=0x7fe9f80091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- 172.21.15.64:0/3816170203 shutdown_connections 2023-09-27T23:33:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 --2- 172.21.15.64:0/3816170203 >> v2:172.21.15.123:3300/0 conn(0x7fea040680f0 0x7fea04068510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 --2- 172.21.15.64:0/3816170203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0413f890 0x7fea040aeba0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- 172.21.15.64:0/3816170203 >> 172.21.15.64:0/3816170203 conn(0x7fea040c8850 msgr2=0x7fea040c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- 172.21.15.64:0/3816170203 shutdown_connections 2023-09-27T23:33:29.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.035+0000 7fea09d61700 1 -- 172.21.15.64:0/3816170203 wait complete. 2023-09-27T23:33:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.039+0000 7fea09d61700 1 Processor -- start 2023-09-27T23:33:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.039+0000 7fea09d61700 1 -- start start 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040680f0 0x7fea0413ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 0x7fea041586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0413f890 0x7fea0415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c4bb0 con 0x7fea040c2980 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0413f0f0 con 0x7fea0413f890 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c3fe0 con 0x7fea040680f0 2023-09-27T23:33:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 0x7fea041586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 0x7fea041586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54268/0 (socket says 172.21.15.64:54268) 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 -- 172.21.15.64:0/3220501996 learned_addr learned my addr 172.21.15.64:0/3220501996 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea03fff700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0413f890 0x7fea0415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea037fe700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040680f0 0x7fea0413ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 -- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040680f0 msgr2=0x7fea0413ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040680f0 0x7fea0413ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 -- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0413f890 msgr2=0x7fea0415b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0413f890 0x7fea0415b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 -- 172.21.15.64:0/3220501996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8009420 con 0x7fea040c2980 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea037fe700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040680f0 0x7fea0413ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea02ffd700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 0x7fea041586e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400c810 tx=0x7fe9f4007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4005c00 con 0x7fea040c2980 2023-09-27T23:33:29.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4005d60 con 0x7fea040c2980 2023-09-27T23:33:29.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0415c900 con 0x7fea040c2980 2023-09-27T23:33:29.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40072e0 con 0x7fea040c2980 2023-09-27T23:33:29.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea040c4560 con 0x7fea040c2980 2023-09-27T23:33:29.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.043+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea040c4d60 con 0x7fea040c2980 2023-09-27T23:33:29.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.047+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe9f4007690 con 0x7fea040c2980 2023-09-27T23:33:29.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.047+0000 7fe9eb7fe700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9f0060aa0 0x7fe9f0062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.047+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe9f407c510 con 0x7fea040c2980 2023-09-27T23:33:29.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.047+0000 7fea037fe700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9f0060aa0 0x7fe9f0062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.047+0000 7fea037fe700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9f0060aa0 0x7fe9f0062f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80093f0 tx=0x7fe9f8017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.051+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe9f4049bb0 con 0x7fea040c2980 2023-09-27T23:33:29.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.247+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea04071120 con 0x7fea040c2980 2023-09-27T23:33:29.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.247+0000 7fe9eb7fe700 1 -- 172.21.15.64:0/3220501996 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fea040c4d60 con 0x7fea040c2980 2023-09-27T23:33:29.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:29.250 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:19.775714+0000","modified":"2023-09-27T23:33:22.797158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2286000796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2286000796},{"type":"v1","addr":"172.21.15.64:6835","nonce":2286000796}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9f0060aa0 msgr2=0x7fe9f0062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9f0060aa0 0x7fe9f0062f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80093f0 tx=0x7fe9f8017000 comp rx=0 tx=0).stop 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 msgr2=0x7fea041586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 0x7fea041586e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400c810 tx=0x7fe9f4007db0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 shutdown_connections 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe9f0060aa0 0x7fe9f0062f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0413f890 0x7fea0415b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea040c2980 0x7fea041586e0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 --2- 172.21.15.64:0/3220501996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040680f0 0x7fea0413ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 >> 172.21.15.64:0/3220501996 conn(0x7fea040c8850 msgr2=0x7fea040af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 shutdown_connections 2023-09-27T23:33:29.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.251+0000 7fea09d61700 1 -- 172.21.15.64:0/3220501996 wait complete. 2023-09-27T23:33:29.254 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-09-27T23:33:29.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:33:19.775714+0000', 'modified': '2023-09-27T23:33:22.797158+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8649}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8649': {'gid': 8649, 'name': 'c', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/2286000796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 2286000796}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 2286000796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:33:29.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:33:29.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-27T23:33:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7730f6700 1 Processor -- start 2023-09-27T23:33:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7730f6700 1 -- start start 2023-09-27T23:33:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7730f6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe76c06aa00 0x7fe76c06d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7730f6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c06c7c0 con 0x7fe76c06a580 2023-09-27T23:33:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7730f6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c06cd80 con 0x7fe76c0b1410 2023-09-27T23:33:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7730f6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c06b8f0 con 0x7fe76c06aa00 2023-09-27T23:33:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7718f3700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe76c06a580 0x7fe76c06d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60636/0 (socket says 172.21.15.64:60636) 2023-09-27T23:33:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7718f3700 1 -- 172.21.15.64:0/4176475221 learned_addr learned my addr 172.21.15.64:0/4176475221 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7728f5700 1 --2- 172.21.15.64:0/4176475221 >> v2:172.21.15.123:3300/0 conn(0x7fe76c06aa00 0x7fe76c06d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106934545 0 0) 0x7fe76c06c7c0 con 0x7fe76c06a580 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe758003570 con 0x7fe76c06a580 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017305969 0 0) 0x7fe758003570 con 0x7fe76c06a580 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 >> v2:172.21.15.123:3300/0 conn(0x7fe76c06aa00 msgr2=0x7fe76c06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 --2- 172.21.15.64:0/4176475221 >> v2:172.21.15.123:3300/0 conn(0x7fe76c06aa00 0x7fe76c06d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 >> v1:172.21.15.123:6789/0 conn(0x7fe76c0b1410 legacy=0x7fe76c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0ac1e0 con 0x7fe76c06a580 2023-09-27T23:33:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fe75c001470 con 0x7fe76c06a580 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 >> v1:172.21.15.64:6789/0 conn(0x7fe76c06a580 legacy=0x7fe76c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe758003a60 0x7fe758003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe758004810 0x7fe758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7580051c0 0x7fe75800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c06b8f0 con 0x7fe758003a60 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75800c0d0 con 0x7fe7580051c0 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.783+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75800c290 con 0x7fe758004810 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7720f4700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe758003a60 0x7fe758003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7718f3700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe758004810 0x7fe758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7580051c0 0x7fe75800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 -- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe758004810 msgr2=0x7fe758004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe758004810 0x7fe758004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 -- 172.21.15.64:0/4176475221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe758003a60 msgr2=0x7fe758003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe758003a60 0x7fe758003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 -- 172.21.15.64:0/4176475221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758003570 con 0x7fe7580051c0 2023-09-27T23:33:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7720f4700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe758003a60 0x7fe758003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 -- 172.21.15.64:0/4176475221 reap_dead start 2023-09-27T23:33:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7728f5700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7580051c0 0x7fe75800ba60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e210 tx=0x7fe76400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c6e26c315c67dc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76400eac0 con 0x7fe7580051c0 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76400f040 con 0x7fe7580051c0 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7710f2700 1 -- 172.21.15.64:0/4176475221 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76400ee70 con 0x7fe7580051c0 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 -- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7580051c0 msgr2=0x7fe75800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7580051c0 0x7fe75800ba60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e210 tx=0x7fe76400b040 comp rx=0 tx=0).stop 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 -- 172.21.15.64:0/4176475221 shutdown_connections 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 --2- 172.21.15.64:0/4176475221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7580051c0 0x7fe75800ba60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 -- 172.21.15.64:0/4176475221 >> 172.21.15.64:0/4176475221 conn(0x7fe76c067bb0 msgr2=0x7fe76c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 -- 172.21.15.64:0/4176475221 shutdown_connections 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 -- 172.21.15.64:0/4176475221 wait complete. 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 Processor -- start 2023-09-27T23:33:29.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.787+0000 7fe7730f6700 1 -- start start 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7580051c0 0x7fe76c1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 0x7fe76c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c156470 0x7fe76c0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c167060 con 0x7fe76c1596f0 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1568e0 con 0x7fe76c156470 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c153850 con 0x7fe7580051c0 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 0x7fe76c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 0x7fe76c155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54296/0 (socket says 172.21.15.64:54296) 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 -- 172.21.15.64:0/2106372897 learned_addr learned my addr 172.21.15.64:0/2106372897 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7728f5700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c156470 0x7fe76c0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7720f4700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7580051c0 0x7fe76c1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 -- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7580051c0 msgr2=0x7fe76c1483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7580051c0 0x7fe76c1483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 -- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c156470 msgr2=0x7fe76c0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c156470 0x7fe76c0c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 -- 172.21.15.64:0/2106372897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c1582f0 con 0x7fe76c1596f0 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7720f4700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7580051c0 0x7fe76c1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7718f3700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 0x7fe76c155f30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe75c001c00 tx=0x7fe76c06ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7627fc700 1 -- 172.21.15.64:0/2106372897 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c018030 con 0x7fe76c1596f0 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7627fc700 1 -- 172.21.15.64:0/2106372897 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00f8d0 con 0x7fe76c1596f0 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c1585d0 con 0x7fe76c1596f0 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7627fc700 1 -- 172.21.15.64:0/2106372897 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c001470 con 0x7fe76c1596f0 2023-09-27T23:33:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c158b90 con 0x7fe76c1596f0 2023-09-27T23:33:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7627fc700 1 -- 172.21.15.64:0/2106372897 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe75c01b070 con 0x7fe76c1596f0 2023-09-27T23:33:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.791+0000 7fe7627fc700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe750060aa0 0x7fe750062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7627fc700 1 -- 172.21.15.64:0/2106372897 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe75c019070 con 0x7fe76c1596f0 2023-09-27T23:33:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe750060aa0 0x7fe750062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe750060aa0 0x7fe750062f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe768000c00 tx=0x7fe768009040 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 Processor -- start 2023-09-27T23:33:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 -- start start 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x28ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x28ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2819f30 con 0x266eae0 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2815ed0 con 0x26b9790 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x27dc0b0 con 0x266e710 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28ebe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54308/0 (socket says 172.21.15.64:54308) 2023-09-27T23:33:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 -- 172.21.15.64:0/275257589 learned_addr learned my addr 172.21.15.64:0/275257589 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7728f5700 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x28ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7718f3700 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x28ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 -- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 msgr2=0x28ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x28ec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 -- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 msgr2=0x28ec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x28ec8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7720f4700 1 -- 172.21.15.64:0/275257589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26ba9e0 con 0x266eae0 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.795+0000 7fe7718f3700 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x28ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe7720f4700 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28ebe50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe768001ec0 tx=0x7fe768015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b099378f73d526d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe74f7fe700 1 -- 172.21.15.64:0/275257589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768014650 con 0x266eae0 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe74f7fe700 1 -- 172.21.15.64:0/275257589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680147b0 con 0x266eae0 2023-09-27T23:33:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe74f7fe700 1 -- 172.21.15.64:0/275257589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768014b30 con 0x266eae0 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- 172.21.15.64:0/275257589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 msgr2=0x28ebe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28ebe50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe768001ec0 tx=0x7fe768015000 comp rx=0 tx=0).stop 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- 172.21.15.64:0/275257589 shutdown_connections 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x28ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x28ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- 172.21.15.64:0/275257589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28ebe50 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- 172.21.15.64:0/275257589 >> 172.21.15.64:0/275257589 conn(0x266eeb0 msgr2=0x28a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- 172.21.15.64:0/275257589 shutdown_connections 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- 172.21.15.64:0/275257589 wait complete. 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- start start 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 Processor -- start 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x28a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x26b9790 0x28d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x27dbd50 con 0x266eae0 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x27b4970 con 0x266e710 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x2819100 con 0x26b9790 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe776601740 10 client.0 _ll_register_callbacks cb 0x285fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:33:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe7718f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe7718f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54310/0 (socket says 172.21.15.64:54310) 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe7718f3700 1 -- 172.21.15.64:0/1528498099 learned_addr learned my addr 172.21.15.64:0/1528498099 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe7728f5700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x26b9790 0x28d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.799+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x28a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7718f3700 1 -- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x26b9790 msgr2=0x28d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7718f3700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x26b9790 0x28d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7718f3700 1 -- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 msgr2=0x28a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7718f3700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x28a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7718f3700 1 -- 172.21.15.64:0/1528498099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76800f040 con 0x266eae0 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x28a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7728f5700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x26b9790 0x28d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7718f3700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28d1c00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe75c07b2c0 tx=0x7fe75c053040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19b4da02fa109cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 1 -- 172.21.15.64:0/1528498099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0661b0 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 1 -- 172.21.15.64:0/1528498099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe75c04f160 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27b4cc0 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 1 -- 172.21.15.64:0/1528498099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe75c051270 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 10 client.8832 fetch_fsmap learned FSMap version 54 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x27e4370 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 1 -- 172.21.15.64:0/1528498099 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27b4cc0 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 1 -- 172.21.15.64:0/1528498099 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 54) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe75c0502b0 con 0x266eae0 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 10 client.8832 fetch_fsmap finished waiting for FSMap version 54 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 10 client.8832 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 20 client.8832 populate_metadata read hostname 'smithi064' 2023-09-27T23:33:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x28f86b0 0x28faa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 4 client.8832 mds_command: new command op to 8649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:33:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 --> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x27b83e0 con 0x28f86b0 2023-09-27T23:33:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x28f86b0 0x28faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 10 client.8832 ms_handle_connect on v2:172.21.15.64:6834/2286000796 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x28f86b0 0x28faa90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54753a2066a2cc20 server_cookie=b0d68bfb67a8ebc8 in_seq=0 out_seq=0 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 1 -- 172.21.15.64:0/1528498099 <== mds.0 v2:172.21.15.64:6834/2286000796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x27b83e0 con 0x28f86b0 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 10 client.8832 handle_command_reply: tid=0 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 client.8832 shutdown 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 msgr2=0x28d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28d1c00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe75c07b2c0 tx=0x7fe75c053040 comp rx=0 tx=0).stop 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 shutdown_connections 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x28f86b0 0x28faa90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x26b9790 0x28d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x28d1c00 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe74cff9700 0 client.8832 ms_handle_reset on v2:172.21.15.64:6834/2286000796 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 --2- 172.21.15.64:0/1528498099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x28a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 >> 172.21.15.64:0/1528498099 conn(0x266eeb0 msgr2=0x28a0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 shutdown_connections 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.803+0000 7fe776601740 1 -- 172.21.15.64:0/1528498099 wait complete. 2023-09-27T23:33:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.807+0000 7fe776601740 20 client.8832 trim_cache size 0 max 16384 2023-09-27T23:33:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 Processor -- start 2023-09-27T23:33:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 -- start start 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x2945cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x2946210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x2946750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x28156e0 con 0x266eae0 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x27c4e00 con 0x26b9790 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x27bf510 con 0x266e710 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x2946210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x2946210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54312/0 (socket says 172.21.15.64:54312) 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 -- 172.21.15.64:0/601625154 learned_addr learned my addr 172.21.15.64:0/601625154 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7728f5700 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x2946750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7720f4700 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x2945cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 -- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 msgr2=0x2945cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x2945cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 -- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 msgr2=0x2946750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x2946750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 -- 172.21.15.64:0/601625154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c159110 con 0x266eae0 2023-09-27T23:33:29.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe7718f3700 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x2946210 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe75c050870 tx=0x7fe75c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebaa62864fd2f91 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe74ffff700 1 -- 172.21.15.64:0/601625154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0661b0 con 0x266eae0 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe74ffff700 1 -- 172.21.15.64:0/601625154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c057e20 con 0x266eae0 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.827+0000 7fe74ffff700 1 -- 172.21.15.64:0/601625154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c04d100 con 0x266eae0 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- 172.21.15.64:0/601625154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 msgr2=0x2946210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x2946210 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe75c050870 tx=0x7fe75c020000 comp rx=0 tx=0).stop 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- 172.21.15.64:0/601625154 shutdown_connections 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x26b9790 0x2946750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x266eae0 0x2946210 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- 172.21.15.64:0/601625154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266e710 0x2945cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- 172.21.15.64:0/601625154 >> 172.21.15.64:0/601625154 conn(0x266eeb0 msgr2=0x28cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- 172.21.15.64:0/601625154 shutdown_connections 2023-09-27T23:33:29.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- 172.21.15.64:0/601625154 wait complete. 2023-09-27T23:33:29.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- start start 2023-09-27T23:33:29.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 Processor -- start 2023-09-27T23:33:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x2904970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266eae0 0x2904eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 0x29053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2813890 con 0x26b9790 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x27e9040 con 0x266e710 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x27eb550 con 0x266eae0 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 10 client.0 _ll_register_callbacks cb 0x27320e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 0x29053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 0x29053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54318/0 (socket says 172.21.15.64:54318) 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 -- 172.21.15.64:0/1134641427 learned_addr learned my addr 172.21.15.64:0/1134641427 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x2904970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7718f3700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266eae0 0x2904eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 -- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266eae0 msgr2=0x2904eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266eae0 0x2904eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 -- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 msgr2=0x2904970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x2904970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 -- 172.21.15.64:0/1134641427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x27e4370 con 0x26b9790 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe7728f5700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 0x29053f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0ae680 tx=0x7fe764019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e6a15dad3f40d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe74cff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe74cff9700 1 -- 172.21.15.64:0/1134641427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764023040 con 0x26b9790 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7680147b0 con 0x26b9790 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe74cff9700 1 -- 172.21.15.64:0/1134641427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe764014cd0 con 0x26b9790 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.831+0000 7fe74cff9700 1 -- 172.21.15.64:0/1134641427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe764014e90 con 0x26b9790 2023-09-27T23:33:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 10 client.8838 fetch_fsmap learned FSMap version 54 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x27cad80 con 0x26b9790 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe74cff9700 1 -- 172.21.15.64:0/1134641427 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764032070 con 0x26b9790 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe74cff9700 1 -- 172.21.15.64:0/1134641427 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 54) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe764029680 con 0x26b9790 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 10 client.8838 fetch_fsmap finished waiting for FSMap version 54 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 10 client.8838 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 20 client.8838 populate_metadata read hostname 'smithi064' 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x294ff50 0x2952330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 4 client.8838 mds_command: new command op to 8649 tid=0[{"prefix": "status"}] 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 --> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] -- command(tid 0: {"prefix": "status"}) v1 -- 0x27ea410 con 0x294ff50 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x294ff50 0x2952330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe74cff9700 10 client.8838 ms_handle_connect on v2:172.21.15.64:6834/2286000796 2023-09-27T23:33:29.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7720f4700 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x294ff50 0x2952330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f9c98f945a7757 server_cookie=f52f8014ae4992b5 in_seq=0 out_seq=0 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe74cff9700 1 -- 172.21.15.64:0/1134641427 <== mds.0 v2:172.21.15.64:6834/2286000796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x27ea410 con 0x294ff50 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe74cff9700 10 client.8838 handle_command_reply: tid=0 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 client.8838 shutdown 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 msgr2=0x29053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 0x29053f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0ae680 tx=0x7fe764019000 comp rx=0 tx=0).stop 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 shutdown_connections 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:6834/2286000796,v1:172.21.15.64:6835/2286000796] conn(0x294ff50 0x2952330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe74cff9700 0 client.8838 ms_handle_reset on v2:172.21.15.64:6834/2286000796 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x26b9790 0x29053f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x266eae0 0x2904eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 --2- 172.21.15.64:0/1134641427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x266e710 0x2904970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 >> 172.21.15.64:0/1134641427 conn(0x266eeb0 msgr2=0x28d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 shutdown_connections 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 1 -- 172.21.15.64:0/1134641427 wait complete. 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe776601740 20 client.8838 trim_cache size 0 max 16384 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 8649, 2023-09-27T23:33:29.843 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 10.057258358, 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 54, 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 143, 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 143, 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 14.603966271999999 2023-09-27T23:33:29.844 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:33:29.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe750060aa0 msgr2=0x7fe750062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe750060aa0 0x7fe750062f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe768000c00 tx=0x7fe768009040 comp rx=0 tx=0).stop 2023-09-27T23:33:29.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 msgr2=0x7fe76c155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 0x7fe76c155f30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe75c001c00 tx=0x7fe76c06ccf0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 shutdown_connections 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe750060aa0 0x7fe750062f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c156470 0x7fe76c0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c1596f0 0x7fe76c155f30 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 --2- 172.21.15.64:0/2106372897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7580051c0 0x7fe76c1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.835+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 >> 172.21.15.64:0/2106372897 conn(0x7fe76c067bb0 msgr2=0x7fe76c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.839+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 shutdown_connections 2023-09-27T23:33:29.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.839+0000 7fe7730f6700 1 -- 172.21.15.64:0/2106372897 wait complete. 2023-09-27T23:33:29.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:33:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 Processor -- start 2023-09-27T23:33:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 -- start start 2023-09-27T23:33:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f489806d1f0 0x7f489806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f48980c1d20 0x7f48980c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48980c92c0 con 0x7f489806d1f0 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48980c86b0 con 0x7f48980c1d20 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489f93e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48980c8990 con 0x7f48980cbdb0 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f489806d1f0 0x7f489806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f489806d1f0 0x7f489806d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54320/0 (socket says 172.21.15.64:54320) 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 -- 172.21.15.64:0/2991460416 learned_addr learned my addr 172.21.15.64:0/2991460416 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489dedb700 1 --2- 172.21.15.64:0/2991460416 >> v2:172.21.15.123:3300/0 conn(0x7f48980c1d20 0x7f48980c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489ced9700 1 --2- 172.21.15.64:0/2991460416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 -- 172.21.15.64:0/2991460416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 msgr2=0x7f48980c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 --2- 172.21.15.64:0/2991460416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 -- 172.21.15.64:0/2991460416 >> v2:172.21.15.123:3300/0 conn(0x7f48980c1d20 msgr2=0x7f48980c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 --2- 172.21.15.64:0/2991460416 >> v2:172.21.15.123:3300/0 conn(0x7f48980c1d20 0x7f48980c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 -- 172.21.15.64:0/2991460416 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48980c8c70 con 0x7f489806d1f0 2023-09-27T23:33:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f489d6da700 1 --2- 172.21.15.64:0/2991460416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489806d1f0 0x7f489806d5c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f48980cc200 tx=0x7f4894009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630ebcd02ce2b1fa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f488ffff700 1 -- 172.21.15.64:0/2991460416 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400b640 con 0x7f489806d1f0 2023-09-27T23:33:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f488ffff700 1 -- 172.21.15.64:0/2991460416 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489400b7a0 con 0x7f489806d1f0 2023-09-27T23:33:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.971+0000 7f488ffff700 1 -- 172.21.15.64:0/2991460416 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400bb60 con 0x7f489806d1f0 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 -- 172.21.15.64:0/2991460416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489806d1f0 msgr2=0x7f489806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 --2- 172.21.15.64:0/2991460416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489806d1f0 0x7f489806d5c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f48980cc200 tx=0x7f4894009250 comp rx=0 tx=0).stop 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 -- 172.21.15.64:0/2991460416 shutdown_connections 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 --2- 172.21.15.64:0/2991460416 >> v2:172.21.15.123:3300/0 conn(0x7f48980c1d20 0x7f48980c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 --2- 172.21.15.64:0/2991460416 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 --2- 172.21.15.64:0/2991460416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f489806d1f0 0x7f489806d5c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 -- 172.21.15.64:0/2991460416 >> 172.21.15.64:0/2991460416 conn(0x7f48980cd9c0 msgr2=0x7f48980c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 -- 172.21.15.64:0/2991460416 shutdown_connections 2023-09-27T23:33:29.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 -- 172.21.15.64:0/2991460416 wait complete. 2023-09-27T23:33:29.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.975+0000 7f489f93e700 1 Processor -- start 2023-09-27T23:33:29.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 -- start start 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489806d1f0 0x7f489815e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 0x7f4898148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489815cfa0 con 0x7f48980c1d20 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489815d280 con 0x7f489806d1f0 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489815d560 con 0x7f48980cbdb0 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 0x7f4898148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 0x7f4898148880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54326/0 (socket says 172.21.15.64:54326) 2023-09-27T23:33:29.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 -- 172.21.15.64:0/3098704749 learned_addr learned my addr 172.21.15.64:0/3098704749 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:29.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489d6da700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489806d1f0 0x7f489815e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489dedb700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 -- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 msgr2=0x7f48980caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 -- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489806d1f0 msgr2=0x7f489815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489806d1f0 0x7f489815e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 -- 172.21.15.64:0/3098704749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940094f0 con 0x7f48980c1d20 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489ced9700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 0x7f4898148880 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4890009e70 tx=0x7f489000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000caa0 con 0x7f48980c1d20 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489000f040 con 0x7f48980c1d20 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489815d840 con 0x7f48980c1d20 2023-09-27T23:33:29.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000ce70 con 0x7f48980c1d20 2023-09-27T23:33:29.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.979+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4898148dc0 con 0x7f48980c1d20 2023-09-27T23:33:29.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.983+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489815dd40 con 0x7f48980c1d20 2023-09-27T23:33:29.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.983+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4890014690 con 0x7f48980c1d20 2023-09-27T23:33:29.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.983+0000 7f488d7fa700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4884060a70 0x7f4884062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:29.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.983+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f489007db20 con 0x7f48980c1d20 2023-09-27T23:33:29.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.983+0000 7f489d6da700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4884060a70 0x7f4884062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:29.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.983+0000 7f489d6da700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4884060a70 0x7f4884062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48980cc200 tx=0x7f4894017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:29.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:29.991+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f489004daf0 con 0x7f48980c1d20 2023-09-27T23:33:30.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.191+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4898071120 con 0x7f48980c1d20 2023-09-27T23:33:30.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.191+0000 7f488d7fa700 1 -- 172.21.15.64:0/3098704749 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4890018070 con 0x7f48980c1d20 2023-09-27T23:33:30.194 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:33:30.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.191+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4884060a70 msgr2=0x7f4884062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:30.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.191+0000 7f489f93e700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4884060a70 0x7f4884062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48980cc200 tx=0x7f4894017000 comp rx=0 tx=0).stop 2023-09-27T23:33:30.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.191+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 msgr2=0x7f4898148880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:30.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.191+0000 7f489f93e700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 0x7f4898148880 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4890009e70 tx=0x7f489000b040 comp rx=0 tx=0).stop 2023-09-27T23:33:30.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 shutdown_connections 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4884060a70 0x7f4884062f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48980cbdb0 0x7f48980caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48980c1d20 0x7f4898148880 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 --2- 172.21.15.64:0/3098704749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489806d1f0 0x7f489815e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 >> 172.21.15.64:0/3098704749 conn(0x7f48980cd9c0 msgr2=0x7f48980c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 shutdown_connections 2023-09-27T23:33:30.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:30.195+0000 7f489f93e700 1 -- 172.21.15.64:0/3098704749 wait complete. 2023-09-27T23:33:30.208 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:33:30.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:33:30.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:33:30.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:33:30.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:30.208 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:33:30.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:33:30.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:33:30.209 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:33:30.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:30.209 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:33:30.217 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:33:30.217 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:33:30.217 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:33:30.217 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.217 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:33:30.217 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41374sec preferred_lft 41374sec 2023-09-27T23:33:30.218 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.219 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:33:30.220 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:33:30.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:30.221 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:33:30.234 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:33:30.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:30.236 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:33:30.292 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:33:30.292 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:33:30.292 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:33:30.293 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:33:30.293 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:33:30.293 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:33:30.293 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:33:30.293 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:33:30.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:30.293 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:33:30.293 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:33:30.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:33:30.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:33:30.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:33:30.294 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:33:30.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:30.330 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:33:30.379 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:33:30.379 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:33:30.379 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:33:30.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:30.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:33:30.397 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:33:30.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:33:30.397 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:30.404 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:33:30.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:33:30.404 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:30.410 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:33:30.469 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:33:30.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:33:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:33:30.486 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:33:30.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:30.486 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:33:30.493 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:33:30.493 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:33:30.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:33:30.510 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:33:30.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:33:30.568 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:33:30.569 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:33:30.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:30.569 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:33:30.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:33:30.579+0000 7fe1ef55d380 -1 init, newargv = 0x560b5baaadf0 newargc=15 2023-09-27T23:33:30.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[56703]: starting ceph client 2023-09-27T23:33:30.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[56703]: starting fuse 2023-09-27T23:33:31.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:33:31.693 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:33:31.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:33:31.704 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:33:31.705 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:33:31.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:31.706 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:33:31.755 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:33:31.755 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:33:31.755 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:33:31.756 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:33:31.757 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:33:31.757 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:33:31.757 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:33:31.757 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:33:31.757 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:33:31.757 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:33:31.841 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.56703.asok 2023-09-27T23:33:32.027 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56703.asok 2023-09-27T23:33:32.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:32.027 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56703.asok status 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:33:32.120 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "56703", 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 7643, 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 7643 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:32.121 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3166310824 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3166310824 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:32.122 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.7643 192.168.0.1:0/3166310824", 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3166310824", 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 54, 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 143, 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:33:32.123 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:33:32.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:32.129 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:33:32.129 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:32.135 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:33:32.135 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:32.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:32.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:32.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:33:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.303+0000 7fc35522c700 1 Processor -- start 2023-09-27T23:33:32.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.303+0000 7fc35522c700 1 -- start start 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc35006a540 0x7fc35006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc3500cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500ca5a0 con 0x7fc350158e70 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35006dac0 con 0x7fc35006a540 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35006d220 con 0x7fc3500cc470 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc35006a540 0x7fc35006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc35006a540 0x7fc35006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54338/0 (socket says 172.21.15.64:54338) 2023-09-27T23:33:32.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 -- 172.21.15.64:0/2655204536 learned_addr learned my addr 172.21.15.64:0/2655204536 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34ffff700 1 --1- 172.21.15.64:0/2655204536 >> v1:172.21.15.64:6789/0 conn(0x7fc350158e70 0x7fc3500afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60638/0 (socket says 172.21.15.64:60638) 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc354a2b700 1 --2- 172.21.15.64:0/2655204536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc3500cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 -- 172.21.15.64:0/2655204536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 msgr2=0x7fc3500cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 --2- 172.21.15.64:0/2655204536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc3500cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 -- 172.21.15.64:0/2655204536 >> v1:172.21.15.64:6789/0 conn(0x7fc350158e70 legacy=0x7fc3500afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 -- 172.21.15.64:0/2655204536 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35006d500 con 0x7fc35006a540 2023-09-27T23:33:32.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34f7fe700 1 --2- 172.21.15.64:0/2655204536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc35006c9d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099b0 tx=0x7fc338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231d4466a076e072 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34effd700 1 -- 172.21.15.64:0/2655204536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800b860 con 0x7fc35006a540 2023-09-27T23:33:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34effd700 1 -- 172.21.15.64:0/2655204536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800b9c0 con 0x7fc35006a540 2023-09-27T23:33:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc34effd700 1 -- 172.21.15.64:0/2655204536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800bd80 con 0x7fc35006a540 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- 172.21.15.64:0/2655204536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 msgr2=0x7fc35006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 --2- 172.21.15.64:0/2655204536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc35006c9d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099b0 tx=0x7fc338009250 comp rx=0 tx=0).stop 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- 172.21.15.64:0/2655204536 shutdown_connections 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 --2- 172.21.15.64:0/2655204536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc3500cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 --2- 172.21.15.64:0/2655204536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc35006c9d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- 172.21.15.64:0/2655204536 >> 172.21.15.64:0/2655204536 conn(0x7fc3500670e0 msgr2=0x7fc3500674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- 172.21.15.64:0/2655204536 shutdown_connections 2023-09-27T23:33:32.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.307+0000 7fc35522c700 1 -- 172.21.15.64:0/2655204536 wait complete. 2023-09-27T23:33:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 Processor -- start 2023-09-27T23:33:32.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 -- start start 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc3500b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc35015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc350158e70 0x7fc350157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500b3170 con 0x7fc35006a540 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500b3450 con 0x7fc350158e70 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500b3730 con 0x7fc3500cc470 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc3500b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc3500b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54344/0 (socket says 172.21.15.64:54344) 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 -- 172.21.15.64:0/729111916 learned_addr learned my addr 172.21.15.64:0/729111916 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34f7fe700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc35015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc354a2b700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc350158e70 0x7fc350157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 -- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 msgr2=0x7fc35015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc35015b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 -- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc350158e70 msgr2=0x7fc350157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc350158e70 0x7fc350157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 -- 172.21.15.64:0/729111916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380094f0 con 0x7fc35006a540 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc34ffff700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc3500b4210 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c7c0 tx=0x7fc340007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340005a70 con 0x7fc35006a540 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340005bd0 con 0x7fc35006a540 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3500b3a10 con 0x7fc35006a540 2023-09-27T23:33:32.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400070d0 con 0x7fc35006a540 2023-09-27T23:33:32.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.311+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35015beb0 con 0x7fc35006a540 2023-09-27T23:33:32.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.315+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35004be40 con 0x7fc35006a540 2023-09-27T23:33:32.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.315+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc340007440 con 0x7fc35006a540 2023-09-27T23:33:32.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.319+0000 7fc347fff700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc330060a70 0x7fc330062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.319+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc34007c9d0 con 0x7fc35006a540 2023-09-27T23:33:32.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.319+0000 7fc34f7fe700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc330060a70 0x7fc330062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.319+0000 7fc34f7fe700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc330060a70 0x7fc330062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099b0 tx=0x7fc338017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:32.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.323+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc34004ca80 con 0x7fc35006a540 2023-09-27T23:33:32.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.511+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc35006bb10 con 0x7fc330060a70 2023-09-27T23:33:32.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc347fff700 1 -- 172.21.15.64:0/729111916 <== mgr.7358 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc35006bb10 con 0x7fc330060a70 2023-09-27T23:33:32.517 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:33:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:33:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:32.518 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:33:32.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc330060a70 msgr2=0x7fc330062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc330060a70 0x7fc330062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099b0 tx=0x7fc338017000 comp rx=0 tx=0).stop 2023-09-27T23:33:32.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 msgr2=0x7fc3500b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc3500b4210 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c7c0 tx=0x7fc340007f30 comp rx=0 tx=0).stop 2023-09-27T23:33:32.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 shutdown_connections 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc330060a70 0x7fc330062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc350158e70 0x7fc350157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc3500cc470 0x7fc35015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 --2- 172.21.15.64:0/729111916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc35006a540 0x7fc3500b4210 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 >> 172.21.15.64:0/729111916 conn(0x7fc3500670e0 msgr2=0x7fc3501571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 shutdown_connections 2023-09-27T23:33:32.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.515+0000 7fc35522c700 1 -- 172.21.15.64:0/729111916 wait complete. 2023-09-27T23:33:32.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:33:32.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23359e700 1 Processor -- start 2023-09-27T23:33:32.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23359e700 1 -- start start 2023-09-27T23:33:32.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb2340c69a0 0x7fb2340c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23406d970 con 0x7fb23406c7c0 2023-09-27T23:33:32.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23406cd60 con 0x7fb23406de00 2023-09-27T23:33:32.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23406d040 con 0x7fb2340c69a0 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb232d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb23406c7c0 0x7fb23415cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60654/0 (socket says 172.21.15.64:60654) 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb232d9d700 1 -- 172.21.15.64:0/46057718 learned_addr learned my addr 172.21.15.64:0/46057718 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23259c700 1 --2- 172.21.15.64:0/46057718 >> v2:172.21.15.123:3300/0 conn(0x7fb2340c69a0 0x7fb2340c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23159a700 1 -- 172.21.15.64:0/46057718 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826489833 0 0) 0x7fb23406d970 con 0x7fb23406c7c0 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23159a700 1 -- 172.21.15.64:0/46057718 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c003570 con 0x7fb23406c7c0 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23259c700 1 -- 172.21.15.64:0/46057718 >> v1:172.21.15.123:6789/0 conn(0x7fb23406de00 legacy=0x7fb23406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23159a700 1 -- 172.21.15.64:0/46057718 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296500675 0 0) 0x7fb23406cd60 con 0x7fb23406de00 2023-09-27T23:33:32.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23259c700 1 -- 172.21.15.64:0/46057718 >> v1:172.21.15.64:6789/0 conn(0x7fb23406c7c0 legacy=0x7fb23415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23259c700 1 -- 172.21.15.64:0/46057718 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23406d320 con 0x7fb2340c69a0 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23259c700 1 --2- 172.21.15.64:0/46057718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2340c69a0 0x7fb2340c6d70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb2280098e0 tx=0x7fb2280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52014dd05f338a7c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23159a700 1 -- 172.21.15.64:0/46057718 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0017f0 con 0x7fb2340c69a0 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23159a700 1 -- 172.21.15.64:0/46057718 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228011420 con 0x7fb2340c69a0 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.643+0000 7fb23159a700 1 -- 172.21.15.64:0/46057718 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280117a0 con 0x7fb2340c69a0 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- 172.21.15.64:0/46057718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2340c69a0 msgr2=0x7fb2340c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 --2- 172.21.15.64:0/46057718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2340c69a0 0x7fb2340c6d70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb2280098e0 tx=0x7fb2280091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- 172.21.15.64:0/46057718 shutdown_connections 2023-09-27T23:33:32.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 --2- 172.21.15.64:0/46057718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2340c69a0 0x7fb2340c6d70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- 172.21.15.64:0/46057718 >> 172.21.15.64:0/46057718 conn(0x7fb234067b50 msgr2=0x7fb2340cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:32.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- 172.21.15.64:0/46057718 shutdown_connections 2023-09-27T23:33:32.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- 172.21.15.64:0/46057718 wait complete. 2023-09-27T23:33:32.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 Processor -- start 2023-09-27T23:33:32.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- start start 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb23406c7c0 0x7fb234151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23406de00 0x7fb23415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 0x7fb23415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23415dd60 con 0x7fb2340c69a0 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23415e040 con 0x7fb23406de00 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23415e320 con 0x7fb23406c7c0 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 0x7fb23415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 0x7fb23415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54352/0 (socket says 172.21.15.64:54352) 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 -- 172.21.15.64:0/1390713642 learned_addr learned my addr 172.21.15.64:0/1390713642 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb231d9b700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23406de00 0x7fb23415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb23259c700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb23406c7c0 0x7fb234151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 -- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb23406c7c0 msgr2=0x7fb234151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb23406c7c0 0x7fb234151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 -- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23406de00 msgr2=0x7fb23415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23406de00 0x7fb23415d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb232d9d700 1 -- 172.21.15.64:0/1390713642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009420 con 0x7fb2340c69a0 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.647+0000 7fb231d9b700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23406de00 0x7fb23415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb232d9d700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 0x7fb23415a330 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00ccf0 tx=0x7fb22c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c004dc0 con 0x7fb2340c69a0 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23415e600 con 0x7fb2340c69a0 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c00a9b0 con 0x7fb2340c69a0 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2340b0830 con 0x7fb2340c69a0 2023-09-27T23:33:32.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00ad30 con 0x7fb2340c69a0 2023-09-27T23:33:32.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 58) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb22c007360 con 0x7fb2340c69a0 2023-09-27T23:33:32.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23404be40 con 0x7fb2340c69a0 2023-09-27T23:33:32.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb2227fc700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb214060a70 0x7fb214062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:32.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb23259c700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb214060a70 0x7fb214062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:32.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.651+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb22c07ce30 con 0x7fb2340c69a0 2023-09-27T23:33:32.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.655+0000 7fb23259c700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb214060a70 0x7fb214062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2280093f0 tx=0x7fb228017000 comp rx=0 tx=0).ready entity=mgr.7358 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:32.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.659+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb22c04ce80 con 0x7fb2340c69a0 2023-09-27T23:33:32.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:32.859+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb23406d600 con 0x7fb2340c69a0 2023-09-27T23:33:33.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.299+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 59) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fb22c04d250 con 0x7fb2340c69a0 2023-09-27T23:33:33.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.303+0000 7fb2227fc700 1 -- 172.21.15.64:0/1390713642 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v59) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb22c050b40 con 0x7fb2340c69a0 2023-09-27T23:33:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb214060a70 msgr2=0x7fb214062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb214060a70 0x7fb214062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2280093f0 tx=0x7fb228017000 comp rx=0 tx=0).stop 2023-09-27T23:33:33.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 msgr2=0x7fb23415a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 0x7fb23415a330 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00ccf0 tx=0x7fb22c007e10 comp rx=0 tx=0).stop 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 shutdown_connections 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb214060a70 0x7fb214062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340c69a0 0x7fb23415a330 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23406de00 0x7fb23415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 --2- 172.21.15.64:0/1390713642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb23406c7c0 0x7fb234151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 >> 172.21.15.64:0/1390713642 conn(0x7fb234067b50 msgr2=0x7fb234160100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 shutdown_connections 2023-09-27T23:33:33.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.307+0000 7fb23359e700 1 -- 172.21.15.64:0/1390713642 wait complete. 2023-09-27T23:33:33.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:33:33.421 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:33.421+0000 7fe711fdf000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:33:33.430 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:33.427+0000 7f3ca2a22000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:33:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 Processor -- start 2023-09-27T23:33:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 -- start start 2023-09-27T23:33:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca441592a0 0x7fca440cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4406d050 0x7fca4406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4406d9c0 con 0x7fca44158ed0 2023-09-27T23:33:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440cca20 con 0x7fca441592a0 2023-09-27T23:33:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.439+0000 7fca48c4c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca440ca280 con 0x7fca4406d050 2023-09-27T23:33:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca42d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4406d050 0x7fca4406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca41d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca441592a0 0x7fca440cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca4259c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fca44158ed0 0x7fca4415cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:44118/0 (socket says 172.21.15.64:44118) 2023-09-27T23:33:33.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca4259c700 1 -- 172.21.15.64:0/2741751996 learned_addr learned my addr 172.21.15.64:0/2741751996 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca42d9d700 1 -- 172.21.15.64:0/2741751996 >> v2:172.21.15.123:3300/0 conn(0x7fca441592a0 msgr2=0x7fca440cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca42d9d700 1 --2- 172.21.15.64:0/2741751996 >> v2:172.21.15.123:3300/0 conn(0x7fca441592a0 0x7fca440cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca42d9d700 1 -- 172.21.15.64:0/2741751996 >> v1:172.21.15.123:6789/0 conn(0x7fca44158ed0 legacy=0x7fca4415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca42d9d700 1 -- 172.21.15.64:0/2741751996 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4415d7c0 con 0x7fca4406d050 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca41d9b700 1 --2- 172.21.15.64:0/2741751996 >> v2:172.21.15.123:3300/0 conn(0x7fca441592a0 0x7fca440cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca42d9d700 1 --2- 172.21.15.64:0/2741751996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4406d050 0x7fca4406d480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009e60 tx=0x7fca3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3feb3727d73bc59 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca4159a700 1 -- 172.21.15.64:0/2741751996 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00e070 con 0x7fca4406d050 2023-09-27T23:33:33.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca4159a700 1 -- 172.21.15.64:0/2741751996 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00bb60 con 0x7fca4406d050 2023-09-27T23:33:33.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca4159a700 1 -- 172.21.15.64:0/2741751996 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c012430 con 0x7fca4406d050 2023-09-27T23:33:33.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 -- 172.21.15.64:0/2741751996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4406d050 msgr2=0x7fca4406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2741751996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4406d050 0x7fca4406d480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009e60 tx=0x7fca3c009780 comp rx=0 tx=0).stop 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 -- 172.21.15.64:0/2741751996 shutdown_connections 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2741751996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4406d050 0x7fca4406d480 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2741751996 >> v2:172.21.15.123:3300/0 conn(0x7fca441592a0 0x7fca440cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 -- 172.21.15.64:0/2741751996 >> 172.21.15.64:0/2741751996 conn(0x7fca440cd8f0 msgr2=0x7fca440c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 -- 172.21.15.64:0/2741751996 shutdown_connections 2023-09-27T23:33:33.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.443+0000 7fca48c4c700 1 -- 172.21.15.64:0/2741751996 wait complete. 2023-09-27T23:33:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 Processor -- start 2023-09-27T23:33:33.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 -- start start 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 0x7fca44169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca44158ed0 0x7fca44153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca441592a0 0x7fca44157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44156230 con 0x7fca4406d050 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44155620 con 0x7fca44158ed0 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca44155900 con 0x7fca441592a0 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 0x7fca44169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 0x7fca44169520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54390/0 (socket says 172.21.15.64:54390) 2023-09-27T23:33:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 -- 172.21.15.64:0/2205934289 learned_addr learned my addr 172.21.15.64:0/2205934289 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca42d9d700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca441592a0 0x7fca44157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca41d9b700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca44158ed0 0x7fca44153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 -- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca441592a0 msgr2=0x7fca44157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca441592a0 0x7fca44157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 -- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca44158ed0 msgr2=0x7fca44153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca44158ed0 0x7fca44153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 -- 172.21.15.64:0/2205934289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38009420 con 0x7fca4406d050 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca4259c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 0x7fca44169520 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c7c0 tx=0x7fca34007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34005b30 con 0x7fca4406d050 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34005c90 con 0x7fca4406d050 2023-09-27T23:33:33.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c009a20 con 0x7fca4406d050 2023-09-27T23:33:33.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.447+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34004a80 con 0x7fca4406d050 2023-09-27T23:33:33.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca44155ec0 con 0x7fca4406d050 2023-09-27T23:33:33.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44071120 con 0x7fca4406d050 2023-09-27T23:33:33.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 59) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fca340070d0 con 0x7fca4406d050 2023-09-27T23:33:33.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca327fc700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca24060ac0 0x7fca24062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca41d9b700 1 -- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca24060ac0 msgr2=0x7fca24062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:33.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fca3407c470 con 0x7fca4406d050 2023-09-27T23:33:33.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.451+0000 7fca41d9b700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca24060ac0 0x7fca24062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:33.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.459+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fca34049bb0 con 0x7fca4406d050 2023-09-27T23:33:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.623+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fca4406ddd0 con 0x7fca4406d050 2023-09-27T23:33:33.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.631+0000 7fca327fc700 1 -- 172.21.15.64:0/2205934289 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7fca3404c4a0 con 0x7fca4406d050 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca24060ac0 msgr2=0x7fca24062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca24060ac0 0x7fca24062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 msgr2=0x7fca44169520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 0x7fca44169520 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c7c0 tx=0x7fca34007f70 comp rx=0 tx=0).stop 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 shutdown_connections 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca24060ac0 0x7fca24062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca441592a0 0x7fca44157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca44158ed0 0x7fca44153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 --2- 172.21.15.64:0/2205934289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4406d050 0x7fca44169520 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 >> 172.21.15.64:0/2205934289 conn(0x7fca440cd8f0 msgr2=0x7fca440ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 shutdown_connections 2023-09-27T23:33:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.635+0000 7fca48c4c700 1 -- 172.21.15.64:0/2205934289 wait complete. 2023-09-27T23:33:33.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:33:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 Processor -- start 2023-09-27T23:33:33.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 -- start start 2023-09-27T23:33:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5eb81547a0 0x7f5eb80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5eb806d050 0x7f5eb806d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80c7820 con 0x7f5eb8157fc0 2023-09-27T23:33:33.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb806d9d0 con 0x7f5eb81547a0 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebd7fb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb8158b90 con 0x7f5eb806d050 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5eb81547a0 0x7f5eb80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5eb81547a0 0x7f5eb80ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54400/0 (socket says 172.21.15.64:54400) 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2286990635 learned_addr learned my addr 172.21.15.64:0/2286990635 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eaffff700 1 --1- 172.21.15.64:0/2286990635 >> v1:172.21.15.64:6789/0 conn(0x7f5eb8157fc0 0x7f5eb8158480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60668/0 (socket says 172.21.15.64:60668) 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5ebcffa700 1 --2- 172.21.15.64:0/2286990635 >> v2:172.21.15.123:3300/0 conn(0x7f5eb806d050 0x7f5eb806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2286990635 >> v2:172.21.15.123:3300/0 conn(0x7f5eb806d050 msgr2=0x7f5eb806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 --2- 172.21.15.64:0/2286990635 >> v2:172.21.15.123:3300/0 conn(0x7f5eb806d050 0x7f5eb806d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2286990635 >> v1:172.21.15.64:6789/0 conn(0x7f5eb8157fc0 legacy=0x7f5eb8158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2286990635 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80c7210 con 0x7f5eb81547a0 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.763+0000 7f5eb7fff700 1 --2- 172.21.15.64:0/2286990635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb81547a0 0x7f5eb80ad730 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b2b10 tx=0x7f5ea8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfde0af355e38a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:33.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb77fe700 1 -- 172.21.15.64:0/2286990635 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800b760 con 0x7f5eb81547a0 2023-09-27T23:33:33.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb77fe700 1 -- 172.21.15.64:0/2286990635 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800b8c0 con 0x7f5eb81547a0 2023-09-27T23:33:33.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb77fe700 1 -- 172.21.15.64:0/2286990635 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800bc60 con 0x7f5eb81547a0 2023-09-27T23:33:33.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2286990635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb81547a0 msgr2=0x7f5eb80ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2286990635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb81547a0 0x7f5eb80ad730 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b2b10 tx=0x7f5ea8009250 comp rx=0 tx=0).stop 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2286990635 shutdown_connections 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2286990635 >> v2:172.21.15.123:3300/0 conn(0x7f5eb806d050 0x7f5eb806d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2286990635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb81547a0 0x7f5eb80ad730 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2286990635 >> 172.21.15.64:0/2286990635 conn(0x7f5eb80ccb30 msgr2=0x7f5eb80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2286990635 shutdown_connections 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2286990635 wait complete. 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 Processor -- start 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- start start 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 0x7f5eb80cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5eb81547a0 0x7f5eb814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb8157fc0 0x7f5eb8067ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80caab0 con 0x7f5eb806d050 2023-09-27T23:33:33.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80cbd30 con 0x7f5eb8157fc0 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebd7fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8069b00 con 0x7f5eb81547a0 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 0x7f5eb80cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 0x7f5eb80cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54416/0 (socket says 172.21.15.64:54416) 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2801478525 learned_addr learned my addr 172.21.15.64:0/2801478525 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eaffff700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5eb81547a0 0x7f5eb814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5ebcffa700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb8157fc0 0x7f5eb8067ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5eb81547a0 msgr2=0x7f5eb814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5eb81547a0 0x7f5eb814f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb8157fc0 msgr2=0x7f5eb8067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb8157fc0 0x7f5eb8067ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 -- 172.21.15.64:0/2801478525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea80094f0 con 0x7f5eb806d050 2023-09-27T23:33:33.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.767+0000 7f5eb7fff700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 0x7f5eb80cc1c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80030c0 tx=0x7f5ea8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:33.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8006bc0 con 0x7f5eb806d050 2023-09-27T23:33:33.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8011e60 con 0x7f5eb806d050 2023-09-27T23:33:33.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb8069de0 con 0x7f5eb806d050 2023-09-27T23:33:33.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea801c8e0 con 0x7f5eb806d050 2023-09-27T23:33:33.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb80cb6b0 con 0x7f5eb806d050 2023-09-27T23:33:33.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80cc700 con 0x7f5eb806d050 2023-09-27T23:33:33.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 59) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f5ea801cc90 con 0x7f5eb806d050 2023-09-27T23:33:33.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eb4ff9700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e98060ac0 0x7f5e98062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:33.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5ea807f350 con 0x7f5eb806d050 2023-09-27T23:33:33.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eaffff700 1 -- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e98060ac0 msgr2=0x7f5e98062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:33.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.771+0000 7f5eaffff700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e98060ac0 0x7f5e98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:33.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.779+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5ea804f380 con 0x7f5eb806d050 2023-09-27T23:33:33.841 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:33.841+0000 7fe711fdf000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:33:33.881 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:33.879+0000 7f3ca2a22000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:33:33.890 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:33.889+0000 7fe711fdf000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:33:33.933 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:33.931+0000 7f3ca2a22000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:33:33.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.939+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f5eb804be40 con 0x7f5eb806d050 2023-09-27T23:33:33.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.947+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/2801478525 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5ea80529a0 con 0x7f5eb806d050 2023-09-27T23:33:33.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e98060ac0 msgr2=0x7f5e98062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:33.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e98060ac0 0x7f5e98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 msgr2=0x7f5eb80cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:33.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 0x7f5eb80cc1c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80030c0 tx=0x7f5ea8016000 comp rx=0 tx=0).stop 2023-09-27T23:33:33.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 shutdown_connections 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5e98060ac0 0x7f5e98062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5eb8157fc0 0x7f5eb8067ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5eb81547a0 0x7f5eb814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 --2- 172.21.15.64:0/2801478525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5eb806d050 0x7f5eb80cc1c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 >> 172.21.15.64:0/2801478525 conn(0x7f5eb80ccb30 msgr2=0x7f5eb80b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 shutdown_connections 2023-09-27T23:33:33.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:33.951+0000 7f5ebd7fb700 1 -- 172.21.15.64:0/2801478525 wait complete. 2023-09-27T23:33:33.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:33.966 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:33:33.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-09-27T23:33:33.996 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:33.997+0000 7fe711fdf000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:34.046 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.043+0000 7f3ca2a22000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:34.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 Processor -- start 2023-09-27T23:33:34.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- start start 2023-09-27T23:33:34.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7f80aea30 0x7fc7f80aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:34.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80caf00 con 0x7fc7f813f830 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80cc200 con 0x7fc7f80c7780 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f80cb160 con 0x7fc7f80aea30 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc7f813f830 0x7fc7f80ca7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60674/0 (socket says 172.21.15.64:60674) 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f77fe700 1 -- 172.21.15.64:0/1268067219 learned_addr learned my addr 172.21.15.64:0/1268067219 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7eeffd700 1 --2- 172.21.15.64:0/1268067219 >> v2:172.21.15.64:3300/0 conn(0x7fc7f80aea30 0x7fc7f80aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f6ffd700 1 -- 172.21.15.64:0/1268067219 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598807415 0 0) 0x7fc7f80caf00 con 0x7fc7f813f830 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f6ffd700 1 -- 172.21.15.64:0/1268067219 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e4003570 con 0x7fc7f813f830 2023-09-27T23:33:34.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7eeffd700 1 -- 172.21.15.64:0/1268067219 >> v1:172.21.15.123:6789/0 conn(0x7fc7f80c7780 legacy=0x7fc7f80c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:34.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7eeffd700 1 -- 172.21.15.64:0/1268067219 >> v1:172.21.15.64:6789/0 conn(0x7fc7f813f830 legacy=0x7fc7f80ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:34.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7eeffd700 1 -- 172.21.15.64:0/1268067219 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80cb440 con 0x7fc7f80aea30 2023-09-27T23:33:34.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7eeffd700 1 --2- 172.21.15.64:0/1268067219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f80aea30 0x7fc7f80aeed0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00098e0 tx=0x7fc7e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a804c072ffee20a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:34.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f6ffd700 1 -- 172.21.15.64:0/1268067219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000b640 con 0x7fc7f80aea30 2023-09-27T23:33:34.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f6ffd700 1 -- 172.21.15.64:0/1268067219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e000b7a0 con 0x7fc7f80aea30 2023-09-27T23:33:34.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7f6ffd700 1 -- 172.21.15.64:0/1268067219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000bb40 con 0x7fc7f80aea30 2023-09-27T23:33:34.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/1268067219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f80aea30 msgr2=0x7fc7f80aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:34.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/1268067219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f80aea30 0x7fc7f80aeed0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00098e0 tx=0x7fc7e00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:34.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/1268067219 shutdown_connections 2023-09-27T23:33:34.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/1268067219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f80aea30 0x7fc7f80aeed0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:34.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/1268067219 >> 172.21.15.64:0/1268067219 conn(0x7fc7f80ccc20 msgr2=0x7fc7f80cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.091+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/1268067219 shutdown_connections 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/1268067219 wait complete. 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 Processor -- start 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- start start 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80aea30 0x7fc7f8166060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f80c7780 0x7fc7f8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 0x7fc7f80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80b0c30 con 0x7fc7f813f830 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80b0950 con 0x7fc7f80aea30 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8067f80 con 0x7fc7f80c7780 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 0x7fc7f80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 0x7fc7f80c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54430/0 (socket says 172.21.15.64:54430) 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 -- 172.21.15.64:0/3864242528 learned_addr learned my addr 172.21.15.64:0/3864242528 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80aea30 0x7fc7f8166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7eeffd700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f80c7780 0x7fc7f8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f80c7780 msgr2=0x7fc7f8155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:34.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f80c7780 0x7fc7f8155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80aea30 msgr2=0x7fc7f8166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80aea30 0x7fc7f8166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 -- 172.21.15.64:0/3864242528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0009420 con 0x7fc7f813f830 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7f7fff700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 0x7fc7f80c8210 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000c840 tx=0x7fc7f0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0012630 con 0x7fc7f813f830 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f00127b0 con 0x7fc7f813f830 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0012b70 con 0x7fc7f813f830 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8068260 con 0x7fc7f813f830 2023-09-27T23:33:34.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.095+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f81665a0 con 0x7fc7f813f830 2023-09-27T23:33:34.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.099+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f8068760 con 0x7fc7f813f830 2023-09-27T23:33:34.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.099+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 59) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fc7f001a3a0 con 0x7fc7f813f830 2023-09-27T23:33:34.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.099+0000 7fc7effff700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:34.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.099+0000 7fc7f77fe700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 msgr2=0x7fc7d8062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:34.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.099+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc7f007d2b0 con 0x7fc7f813f830 2023-09-27T23:33:34.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.099+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:34.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.103+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc7f004d2e0 con 0x7fc7f813f830 2023-09-27T23:33:34.191 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.189+0000 7fe711fdf000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:33:34.240 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.241+0000 7fe711fdf000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:34.250 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.247+0000 7f3ca2a22000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:33:34.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.283+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7f8067e00 con 0x7fc7d8060ac0 2023-09-27T23:33:34.302 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.299+0000 7f3ca2a22000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.303+0000 7fc7f77fe700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 msgr2=0x7fc7d8062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.303+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:33:34.394 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.393+0000 7fe711fdf000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:33:34.450 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.449+0000 7fe711fdf000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:33:34.458 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.455+0000 7f3ca2a22000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:33:34.510 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.509+0000 7fe711fdf000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:33:34.517 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.515+0000 7f3ca2a22000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:33:34.578 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.575+0000 7f3ca2a22000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:33:34.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.707+0000 7fc7f77fe700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 msgr2=0x7fc7d8062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:34.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:34.707+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:33:34.806 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.805+0000 7fe711fdf000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:33:34.887 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.883+0000 7f3ca2a22000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:33:34.918 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:34.917+0000 7fe711fdf000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:33:35.003 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:34.999+0000 7f3ca2a22000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:33:35.200 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.201+0000 7fe711fdf000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:33:35.293 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.291+0000 7f3ca2a22000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:33:35.363 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.361+0000 7fe711fdf000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:35.427 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.425+0000 7fe711fdf000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:33:35.463 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.459+0000 7f3ca2a22000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:35.481 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.481+0000 7fe711fdf000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:33:35.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:35.511+0000 7fc7f77fe700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 msgr2=0x7fc7d8062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:35.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:35.511+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:33:35.530 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.527+0000 7f3ca2a22000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:33:35.536 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.537+0000 7fe711fdf000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:33:35.584 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.583+0000 7f3ca2a22000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:33:35.644 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.639+0000 7f3ca2a22000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:33:35.710 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.709+0000 7fe711fdf000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:35.764 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.765+0000 7fe711fdf000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:33:35.821 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.819+0000 7f3ca2a22000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:35.829 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.829+0000 7fe711fdf000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:33:35.878 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.875+0000 7f3ca2a22000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:33:35.943 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:35.939+0000 7f3ca2a22000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:33:35.979 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:35.977+0000 7fe711fdf000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:33:36.040 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.041+0000 7fe711fdf000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:33:36.100 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:36.099+0000 7f3ca2a22000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:33:36.164 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:36.159+0000 7f3ca2a22000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:33:36.634 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.633+0000 7fe711fdf000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:33:36.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:36.667+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 60) v1 ==== 79987+0+0 (secure 0 0 0) 0x7fc7f004d6b0 con 0x7fc7f813f830 2023-09-27T23:33:36.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:36.667+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 msgr2=0x7fc7d8062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:36.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:36.667+0000 7fc7effff700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:36.787 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:36.783+0000 7f3ca2a22000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:33:36.833 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d2189700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.833 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d2189700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.833 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d2189700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.833 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d2189700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.833 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d2189700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.834 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d5990700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.834 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d5990700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.834 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d5990700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.834 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d5990700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:36.834 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:36.833+0000 7fe6d5990700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.671+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 61) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fc7f007b4f0 con 0x7fc7f813f830 2023-09-27T23:33:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.675+0000 7fc7effff700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.675+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7d8069590 con 0x7fc7d8060ac0 2023-09-27T23:33:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.679+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.683+0000 7fc7f77fe700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062ef0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8007fd0 tx=0x7fc7e8007d50 comp rx=0 tx=0).ready entity=mgr.8743 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:37.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.771+0000 7fc7effff700 1 -- 172.21.15.64:0/3864242528 <== mgr.8743 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc7d8069590 con 0x7fc7d8060ac0 2023-09-27T23:33:37.775 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path snap_test_dir1 2023-09-27T23:33:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 msgr2=0x7fc7d8062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062ef0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8007fd0 tx=0x7fc7e8007d50 comp rx=0 tx=0).stop 2023-09-27T23:33:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 msgr2=0x7fc7f80c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:37.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 0x7fc7f80c8210 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000c840 tx=0x7fc7f0001670 comp rx=0 tx=0).stop 2023-09-27T23:33:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 shutdown_connections 2023-09-27T23:33:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fc7d8060ac0 0x7fc7d8062ef0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7f813f830 0x7fc7f80c8210 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7f80c7780 0x7fc7f8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 --2- 172.21.15.64:0/3864242528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80aea30 0x7fc7f8166060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 >> 172.21.15.64:0/3864242528 conn(0x7fc7f80ccc20 msgr2=0x7fc7f806d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 shutdown_connections 2023-09-27T23:33:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.775+0000 7fc7fdbbb700 1 -- 172.21.15.64:0/3864242528 wait complete. 2023-09-27T23:33:37.797 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-09-27T23:33:37.798 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-09-27T23:33:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.919+0000 7fdb07fff700 1 Processor -- start 2023-09-27T23:33:37.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.919+0000 7fdb07fff700 1 -- start start 2023-09-27T23:33:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb08158740 0x7fdb0806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb0806e7b0 0x7fdb0806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0806ded0 con 0x7fdb08158b10 2023-09-27T23:33:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080c82f0 con 0x7fdb08158740 2023-09-27T23:33:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080c7fa0 con 0x7fdb0806e7b0 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb08158740 0x7fdb0806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdb08158740 0x7fdb0806e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59986/0 (socket says 172.21.15.64:59986) 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 -- 172.21.15.64:0/501627211 learned_addr learned my addr 172.21.15.64:0/501627211 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb067fc700 1 --2- 172.21.15.64:0/501627211 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb0806e7b0 0x7fdb0806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 -- 172.21.15.64:0/501627211 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb0806e7b0 msgr2=0x7fdb0806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 --2- 172.21.15.64:0/501627211 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb0806e7b0 0x7fdb0806d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 -- 172.21.15.64:0/501627211 >> v1:172.21.15.123:6789/0 conn(0x7fdb08158b10 legacy=0x7fdb0806ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 -- 172.21.15.64:0/501627211 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080c70f0 con 0x7fdb08158740 2023-09-27T23:33:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb06ffd700 1 --2- 172.21.15.64:0/501627211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb08158740 0x7fdb0806e270 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800bc40 tx=0x7fdaf800b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf97bdfb2817c0fb server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb05ffb700 1 -- 172.21.15.64:0/501627211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf800db40 con 0x7fdb08158740 2023-09-27T23:33:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb05ffb700 1 -- 172.21.15.64:0/501627211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf800dca0 con 0x7fdb08158740 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb05ffb700 1 -- 172.21.15.64:0/501627211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8017640 con 0x7fdb08158740 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- 172.21.15.64:0/501627211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb08158740 msgr2=0x7fdb0806e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 --2- 172.21.15.64:0/501627211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb08158740 0x7fdb0806e270 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800bc40 tx=0x7fdaf800b4e0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- 172.21.15.64:0/501627211 shutdown_connections 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 --2- 172.21.15.64:0/501627211 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb0806e7b0 0x7fdb0806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 --2- 172.21.15.64:0/501627211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb08158740 0x7fdb0806e270 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- 172.21.15.64:0/501627211 >> 172.21.15.64:0/501627211 conn(0x7fdb080c8c00 msgr2=0x7fdb080c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- 172.21.15.64:0/501627211 shutdown_connections 2023-09-27T23:33:37.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.923+0000 7fdb07fff700 1 -- 172.21.15.64:0/501627211 wait complete. 2023-09-27T23:33:37.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 Processor -- start 2023-09-27T23:33:37.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 -- start start 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 0x7fdb080c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb08158740 0x7fdb080c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb08158b10 0x7fdb08164e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080c6ec0 con 0x7fdb0806e7b0 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080c32d0 con 0x7fdb08158b10 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb07fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080c6620 con 0x7fdb08158740 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 0x7fdb080c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 0x7fdb080c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59998/0 (socket says 172.21.15.64:59998) 2023-09-27T23:33:37.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 -- 172.21.15.64:0/3985340932 learned_addr learned my addr 172.21.15.64:0/3985340932 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb067fc700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb08158740 0x7fdb080c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb077fe700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb08158b10 0x7fdb08164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 -- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb08158740 msgr2=0x7fdb080c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb08158740 0x7fdb080c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 -- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb08158b10 msgr2=0x7fdb08164e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb08158b10 0x7fdb08164e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:37.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.927+0000 7fdb06ffd700 1 -- 172.21.15.64:0/3985340932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf800b780 con 0x7fdb0806e7b0 2023-09-27T23:33:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdb06ffd700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 0x7fdb080c5890 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800d780 tx=0x7fdaf8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8022610 con 0x7fdb0806e7b0 2023-09-27T23:33:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8022790 con 0x7fdb0806e7b0 2023-09-27T23:33:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb080c6900 con 0x7fdb0806e7b0 2023-09-27T23:33:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8022b50 con 0x7fdb0806e7b0 2023-09-27T23:33:37.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb080c6310 con 0x7fdb0806e7b0 2023-09-27T23:33:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0804be40 con 0x7fdb0806e7b0 2023-09-27T23:33:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 61) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fdaf8015070 con 0x7fdb0806e7b0 2023-09-27T23:33:37.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdaeeffd700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdaf40607f0 0x7fdaf4062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fdaf8080c70 con 0x7fdb0806e7b0 2023-09-27T23:33:37.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.931+0000 7fdb067fc700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdaf40607f0 0x7fdaf4062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:37.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.935+0000 7fdb067fc700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdaf40607f0 0x7fdaf4062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdb08162a80 tx=0x7fdafc008040 comp rx=0 tx=0).ready entity=mgr.8743 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:37.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:37.939+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdaf8050f00 con 0x7fdb0806e7b0 2023-09-27T23:33:38.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.119+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb080c3410 con 0x7fdaf40607f0 2023-09-27T23:33:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.247+0000 7fdaeeffd700 1 -- 172.21.15.64:0/3985340932 <== mgr.8743 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdafc0107b0 con 0x7fdaf40607f0 2023-09-27T23:33:38.252 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path snap_test_dir1 2023-09-27T23:33:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.251+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdaf40607f0 msgr2=0x7fdaf4062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.251+0000 7fdb07fff700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdaf40607f0 0x7fdaf4062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdb08162a80 tx=0x7fdafc008040 comp rx=0 tx=0).stop 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.251+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 msgr2=0x7fdb080c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.251+0000 7fdb07fff700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 0x7fdb080c5890 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800d780 tx=0x7fdaf8019000 comp rx=0 tx=0).stop 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 shutdown_connections 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fdaf40607f0 0x7fdaf4062ca0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb08158b10 0x7fdb08164e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb08158740 0x7fdb080c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 --2- 172.21.15.64:0/3985340932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb0806e7b0 0x7fdb080c5890 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 >> 172.21.15.64:0/3985340932 conn(0x7fdb080c8c00 msgr2=0x7fdb080670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 shutdown_connections 2023-09-27T23:33:38.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.255+0000 7fdb07fff700 1 -- 172.21.15.64:0/3985340932 wait complete. 2023-09-27T23:33:38.273 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-09-27T23:33:38.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-09-27T23:33:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.387+0000 7f62c3fd8700 1 Processor -- start 2023-09-27T23:33:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.387+0000 7f62c3fd8700 1 -- start start 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62bc0cb130 0x7f62bc0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62bc06a6a0 0x7f62bc06aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc0b0ac0 con 0x7f62bc15c220 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc06b030 con 0x7f62bc0cb130 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc06b210 con 0x7f62bc06a6a0 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62bc0cb130 0x7f62bc0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62bc0cb130 0x7f62bc0cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60004/0 (socket says 172.21.15.64:60004) 2023-09-27T23:33:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 -- 172.21.15.64:0/1012989164 learned_addr learned my addr 172.21.15.64:0/1012989164 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1573700 1 --2- 172.21.15.64:0/1012989164 >> v2:172.21.15.123:3300/0 conn(0x7f62bc06a6a0 0x7f62bc06aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 -- 172.21.15.64:0/1012989164 >> v2:172.21.15.123:3300/0 conn(0x7f62bc06a6a0 msgr2=0x7f62bc06aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 --2- 172.21.15.64:0/1012989164 >> v2:172.21.15.123:3300/0 conn(0x7f62bc06a6a0 0x7f62bc06aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 -- 172.21.15.64:0/1012989164 >> v1:172.21.15.123:6789/0 conn(0x7f62bc15c220 legacy=0x7f62bc06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 -- 172.21.15.64:0/1012989164 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc0ca810 con 0x7f62bc0cb130 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c1d74700 1 --2- 172.21.15.64:0/1012989164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc0cb130 0x7f62bc0cb500 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0cc930 tx=0x7f62ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f92133fe013b289 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c0d72700 1 -- 172.21.15.64:0/1012989164 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac009e70 con 0x7f62bc0cb130 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c0d72700 1 -- 172.21.15.64:0/1012989164 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00b860 con 0x7f62bc0cb130 2023-09-27T23:33:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c0d72700 1 -- 172.21.15.64:0/1012989164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00bc00 con 0x7f62bc0cb130 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- 172.21.15.64:0/1012989164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc0cb130 msgr2=0x7f62bc0cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 --2- 172.21.15.64:0/1012989164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc0cb130 0x7f62bc0cb500 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0cc930 tx=0x7f62ac009250 comp rx=0 tx=0).stop 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- 172.21.15.64:0/1012989164 shutdown_connections 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 --2- 172.21.15.64:0/1012989164 >> v2:172.21.15.123:3300/0 conn(0x7f62bc06a6a0 0x7f62bc06aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 --2- 172.21.15.64:0/1012989164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc0cb130 0x7f62bc0cb500 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- 172.21.15.64:0/1012989164 >> 172.21.15.64:0/1012989164 conn(0x7f62bc067bb0 msgr2=0x7f62bc0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:38.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- 172.21.15.64:0/1012989164 shutdown_connections 2023-09-27T23:33:38.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.391+0000 7f62c3fd8700 1 -- 172.21.15.64:0/1012989164 wait complete. 2023-09-27T23:33:38.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 Processor -- start 2023-09-27T23:33:38.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 -- start start 2023-09-27T23:33:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62bc06a6a0 0x7f62bc157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc0cb130 0x7f62bc0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc15c220 0x7f62bc0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0c8810 con 0x7f62bc15c220 2023-09-27T23:33:38.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0c7e40 con 0x7f62bc0cb130 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c3fd8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0c7230 con 0x7f62bc06a6a0 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc15c220 0x7f62bc0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc15c220 0x7f62bc0af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60014/0 (socket says 172.21.15.64:60014) 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 -- 172.21.15.64:0/189502835 learned_addr learned my addr 172.21.15.64:0/189502835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c1573700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc0cb130 0x7f62bc0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c1d74700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62bc06a6a0 0x7f62bc157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 -- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62bc06a6a0 msgr2=0x7f62bc157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62bc06a6a0 0x7f62bc157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 -- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc0cb130 msgr2=0x7f62bc0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc0cb130 0x7f62bc0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c2575700 1 -- 172.21.15.64:0/189502835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0094f0 con 0x7f62bc15c220 2023-09-27T23:33:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.395+0000 7f62c1573700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc0cb130 0x7f62bc0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62c2575700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62bc15c220 0x7f62bc0af270 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f62b400c8e0 tx=0x7f62b4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400ce80 con 0x7f62bc15c220 2023-09-27T23:33:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b400a7e0 con 0x7f62bc15c220 2023-09-27T23:33:38.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62c3fd8700 1 -- 172.21.15.64:0/189502835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc0c7510 con 0x7f62bc15c220 2023-09-27T23:33:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400ab80 con 0x7f62bc15c220 2023-09-27T23:33:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62c3fd8700 1 -- 172.21.15.64:0/189502835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc0c7ad0 con 0x7f62bc15c220 2023-09-27T23:33:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62c3fd8700 1 -- 172.21.15.64:0/189502835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc0c7fd0 con 0x7f62bc15c220 2023-09-27T23:33:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 61) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f62b4012420 con 0x7f62bc15c220 2023-09-27T23:33:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62b1ffb700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a00607f0 0x7f62a0062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.399+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f62b407d190 con 0x7f62bc15c220 2023-09-27T23:33:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.403+0000 7f62c1d74700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a00607f0 0x7f62a0062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.403+0000 7f62c1d74700 1 --2- 172.21.15.64:0/189502835 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f62a00607f0 0x7f62a0062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0cc930 tx=0x7f62ac017000 comp rx=0 tx=0).ready entity=mgr.8743 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.411+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f62b404d420 con 0x7f62bc15c220 2023-09-27T23:33:38.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.587+0000 7f62c3fd8700 1 -- 172.21.15.64:0/189502835 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f62bc15bc80 con 0x7f62a00607f0 2023-09-27T23:33:38.603 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:38.601+0000 7fe6dc99e700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-09-27T23:33:38.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.599+0000 7f62b1ffb700 1 -- 172.21.15.64:0/189502835 <== mgr.8743 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f62bc15bc80 con 0x7f62a00607f0 2023-09-27T23:33:38.605 INFO:teuthology.orchestra.run.smithi064.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-09-27T23:33:38.609 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-09-27T23:33:38.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:33:38.610 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-09-27T23:33:38.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:33:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.731+0000 7fb704176700 1 Processor -- start 2023-09-27T23:33:38.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.731+0000 7fb704176700 1 -- start start 2023-09-27T23:33:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6fc158e20 0x7fb6fc0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6fc0ca2c0 0x7fb6fc0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc0cbdf0 con 0x7fb6fc0cb970 2023-09-27T23:33:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc0cc0d0 con 0x7fb6fc0ca2c0 2023-09-27T23:33:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc0cc3b0 con 0x7fb6fc158e20 2023-09-27T23:33:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb702713700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6fc0ca2c0 0x7fb6fc0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb701711700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb6fc0cb970 0x7fb6fc06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37464/0 (socket says 172.21.15.64:37464) 2023-09-27T23:33:38.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb701711700 1 -- 172.21.15.64:0/560070216 learned_addr learned my addr 172.21.15.64:0/560070216 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb701f12700 1 --2- 172.21.15.64:0/560070216 >> v2:172.21.15.123:3300/0 conn(0x7fb6fc158e20 0x7fb6fc0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb702713700 1 -- 172.21.15.64:0/560070216 >> v2:172.21.15.123:3300/0 conn(0x7fb6fc158e20 msgr2=0x7fb6fc0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb702713700 1 --2- 172.21.15.64:0/560070216 >> v2:172.21.15.123:3300/0 conn(0x7fb6fc158e20 0x7fb6fc0ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb702713700 1 -- 172.21.15.64:0/560070216 >> v1:172.21.15.64:6789/0 conn(0x7fb6fc0cb970 legacy=0x7fb6fc06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb702713700 1 -- 172.21.15.64:0/560070216 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0cc690 con 0x7fb6fc0ca2c0 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb702713700 1 --2- 172.21.15.64:0/560070216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0ceb00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009e60 tx=0x7fb6f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1d230fbd5fd0c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb700f10700 1 -- 172.21.15.64:0/560070216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400e070 con 0x7fb6fc0ca2c0 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb700f10700 1 -- 172.21.15.64:0/560070216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f400bbf0 con 0x7fb6fc0ca2c0 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb700f10700 1 -- 172.21.15.64:0/560070216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4012430 con 0x7fb6fc0ca2c0 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- 172.21.15.64:0/560070216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc0ca2c0 msgr2=0x7fb6fc0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 --2- 172.21.15.64:0/560070216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0ceb00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009e60 tx=0x7fb6f4009780 comp rx=0 tx=0).stop 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- 172.21.15.64:0/560070216 shutdown_connections 2023-09-27T23:33:38.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 --2- 172.21.15.64:0/560070216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0ceb00 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 --2- 172.21.15.64:0/560070216 >> v2:172.21.15.123:3300/0 conn(0x7fb6fc158e20 0x7fb6fc0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- 172.21.15.64:0/560070216 >> 172.21.15.64:0/560070216 conn(0x7fb6fc067bb0 msgr2=0x7fb6fc0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- 172.21.15.64:0/560070216 shutdown_connections 2023-09-27T23:33:38.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.735+0000 7fb704176700 1 -- 172.21.15.64:0/560070216 wait complete. 2023-09-27T23:33:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 Processor -- start 2023-09-27T23:33:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 -- start start 2023-09-27T23:33:38.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6fc158e20 0x7fb6fc14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 0x7fb6fc06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0b4910 con 0x7fb6fc150930 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc1524c0 con 0x7fb6fc0ca2c0 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc151310 con 0x7fb6fc158e20 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 0x7fb6fc06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 0x7fb6fc06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60044/0 (socket says 172.21.15.64:60044) 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 -- 172.21.15.64:0/980258928 learned_addr learned my addr 172.21.15.64:0/980258928 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb701711700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6fc158e20 0x7fb6fc14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb701f12700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 -- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6fc158e20 msgr2=0x7fb6fc14aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6fc158e20 0x7fb6fc14aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 -- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6fc0ca2c0 msgr2=0x7fb6fc0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 -- 172.21.15.64:0/980258928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f4009a20 con 0x7fb6fc150930 2023-09-27T23:33:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb702713700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 0x7fb6fc06c7c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc069a80 tx=0x7fb6f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4027470 con 0x7fb6fc150930 2023-09-27T23:33:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4027610 con 0x7fb6fc150930 2023-09-27T23:33:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6fc1515f0 con 0x7fb6fc150930 2023-09-27T23:33:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40279d0 con 0x7fb6fc150930 2023-09-27T23:33:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.739+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc151bb0 con 0x7fb6fc150930 2023-09-27T23:33:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.743+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6fc152670 con 0x7fb6fc150930 2023-09-27T23:33:38.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.743+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 61) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fb6f4024070 con 0x7fb6fc150930 2023-09-27T23:33:38.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.743+0000 7fb6f1ffb700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb6e0060820 0x7fb6e0062cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:38.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.743+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fb6f407f7a0 con 0x7fb6fc150930 2023-09-27T23:33:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.743+0000 7fb701f12700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb6e0060820 0x7fb6e0062cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.743+0000 7fb701f12700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb6e0060820 0x7fb6e0062cd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc153930 tx=0x7fb6ec008040 comp rx=0 tx=0).ready entity=mgr.8743 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:38.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.751+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb6fc152670 con 0x7fb6fc150930 2023-09-27T23:33:38.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:38.943+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb6fc152630 con 0x7fb6fc150930 2023-09-27T23:33:39.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.679+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fb6f4003040 con 0x7fb6fc150930 2023-09-27T23:33:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.683+0000 7fb6f1ffb700 1 -- 172.21.15.64:0/980258928 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v62) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb6f40534c0 con 0x7fb6fc150930 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb6e0060820 msgr2=0x7fb6e0062cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb6e0060820 0x7fb6e0062cd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc153930 tx=0x7fb6ec008040 comp rx=0 tx=0).stop 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 msgr2=0x7fb6fc06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 0x7fb6fc06c7c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc069a80 tx=0x7fb6f4019000 comp rx=0 tx=0).stop 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 shutdown_connections 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fb6e0060820 0x7fb6e0062cd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6fc150930 0x7fb6fc06c7c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6fc158e20 0x7fb6fc14aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 --2- 172.21.15.64:0/980258928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6fc0ca2c0 0x7fb6fc0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 >> 172.21.15.64:0/980258928 conn(0x7fb6fc067bb0 msgr2=0x7fb6fc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:39.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 shutdown_connections 2023-09-27T23:33:39.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:39.687+0000 7fb704176700 1 -- 172.21.15.64:0/980258928 wait complete. 2023-09-27T23:33:39.706 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:33:39.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:39.720 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:39.737 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:33:39.737 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:33:39.737 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:33:39.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:33:39.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[56703]: fuse finished with error 0 and tester_r 0 2023-09-27T23:33:39.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:33:39.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:39.778 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:33:39.808 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:39.809+0000 7f20b0d4b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:33:39.812 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:39.811+0000 7faf43c7c000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:33:39.899 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:33:39.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:39.900 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:33:39.907 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:33:39.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:33:39.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:33:39.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:39.910 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:33:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:33:39.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:33:39.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:33:39.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:33:39.917 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:33:39.923 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:33:39.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:33:39.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 Processor -- start 2023-09-27T23:33:40.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 -- start start 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d600ca420 0x7f5d60159f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600cb8a0 con 0x7f5d60159ba0 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600cac90 con 0x7f5d6015a4b0 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600caf70 con 0x7f5d600ca420 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6619f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5d60159ba0 0x7f5d600670e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37480/0 (socket says 172.21.15.64:37480) 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6619f700 1 -- 172.21.15.64:0/4191404451 learned_addr learned my addr 172.21.15.64:0/4191404451 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:40.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6599e700 1 --2- 172.21.15.64:0/4191404451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d600ca420 0x7f5d60159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6519d700 1 -- 172.21.15.64:0/4191404451 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844763424 0 0) 0x7f5d600cb8a0 con 0x7f5d60159ba0 2023-09-27T23:33:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6519d700 1 -- 172.21.15.64:0/4191404451 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d4c003570 con 0x7f5d60159ba0 2023-09-27T23:33:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6519d700 1 -- 172.21.15.64:0/4191404451 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3217240877 0 0) 0x7f5d4c003570 con 0x7f5d60159ba0 2023-09-27T23:33:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6599e700 1 -- 172.21.15.64:0/4191404451 >> v1:172.21.15.123:6789/0 conn(0x7f5d6015a4b0 legacy=0x7f5d601558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6599e700 1 -- 172.21.15.64:0/4191404451 >> v1:172.21.15.64:6789/0 conn(0x7f5d60159ba0 legacy=0x7f5d600670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6599e700 1 -- 172.21.15.64:0/4191404451 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600cb250 con 0x7f5d600ca420 2023-09-27T23:33:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6599e700 1 --2- 172.21.15.64:0/4191404451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d600ca420 0x7f5d60159f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009910 tx=0x7f5d500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dd9ba1dcb2de96a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6519d700 1 -- 172.21.15.64:0/4191404451 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c001470 con 0x7f5d600ca420 2023-09-27T23:33:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6519d700 1 -- 172.21.15.64:0/4191404451 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d50011420 con 0x7f5d600ca420 2023-09-27T23:33:40.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d6519d700 1 -- 172.21.15.64:0/4191404451 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d500117a0 con 0x7f5d600ca420 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 -- 172.21.15.64:0/4191404451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d600ca420 msgr2=0x7f5d60159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.063+0000 7f5d68403700 1 --2- 172.21.15.64:0/4191404451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d600ca420 0x7f5d60159f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009910 tx=0x7f5d500091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- 172.21.15.64:0/4191404451 shutdown_connections 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 --2- 172.21.15.64:0/4191404451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d600ca420 0x7f5d60159f70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- 172.21.15.64:0/4191404451 >> 172.21.15.64:0/4191404451 conn(0x7f5d600cd990 msgr2=0x7f5d600ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- 172.21.15.64:0/4191404451 shutdown_connections 2023-09-27T23:33:40.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- 172.21.15.64:0/4191404451 wait complete. 2023-09-27T23:33:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 Processor -- start 2023-09-27T23:33:40.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- start start 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d600ca420 0x7f5d6014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d60159ba0 0x7f5d600c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 0x7f5d60148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60150a00 con 0x7f5d6015a4b0 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6014b2a0 con 0x7f5d600ca420 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600c61a0 con 0x7f5d60159ba0 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 0x7f5d60148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 0x7f5d60148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60062/0 (socket says 172.21.15.64:60062) 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 -- 172.21.15.64:0/3393641555 learned_addr learned my addr 172.21.15.64:0/3393641555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:40.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d6599e700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d60159ba0 0x7f5d600c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d6619f700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d600ca420 0x7f5d6014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 -- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d60159ba0 msgr2=0x7f5d600c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d60159ba0 0x7f5d600c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 -- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d600ca420 msgr2=0x7f5d6014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d600ca420 0x7f5d6014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 -- 172.21.15.64:0/3393641555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d50009420 con 0x7f5d6015a4b0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d669a0700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 0x7f5d60148b20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800c840 tx=0x7f5d58001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58012630 con 0x7f5d6015a4b0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d580127b0 con 0x7f5d6015a4b0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58012b70 con 0x7f5d6015a4b0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6014b000 con 0x7f5d6015a4b0 2023-09-27T23:33:40.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.067+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d60150720 con 0x7f5d6015a4b0 2023-09-27T23:33:40.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.071+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d600c6360 con 0x7f5d6015a4b0 2023-09-27T23:33:40.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.071+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f5d5801a3a0 con 0x7f5d6015a4b0 2023-09-27T23:33:40.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.071+0000 7f5d567fc700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 0x7f5d44062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.071+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f5d5807d560 con 0x7f5d6015a4b0 2023-09-27T23:33:40.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.071+0000 7f5d6619f700 1 -- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 msgr2=0x7f5d44062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:40.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.071+0000 7f5d6619f700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 0x7f5d44062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:40.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.075+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5d5804d2c0 con 0x7f5d6015a4b0 2023-09-27T23:33:40.232 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.233+0000 7f20b0d4b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:33:40.262 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.259+0000 7faf43c7c000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:33:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.275+0000 7f5d6619f700 1 -- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 msgr2=0x7f5d44062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.275+0000 7f5d6619f700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 0x7f5d44062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:33:40.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.275+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d60071120 con 0x7f5d6015a4b0 2023-09-27T23:33:40.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.275+0000 7f5d567fc700 1 -- 172.21.15.64:0/3393641555 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f5d580508e0 con 0x7f5d6015a4b0 2023-09-27T23:33:40.278 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:40.278 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:19.775714+0000","modified":"2023-09-27T23:33:22.797158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2286000796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2286000796},{"type":"v1","addr":"172.21.15.64:6835","nonce":2286000796}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 msgr2=0x7f5d44062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 0x7f5d44062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 msgr2=0x7f5d60148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 0x7f5d60148b20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800c840 tx=0x7f5d58001670 comp rx=0 tx=0).stop 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 shutdown_connections 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5d44060b00 0x7f5d44062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d6015a4b0 0x7f5d60148b20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d60159ba0 0x7f5d600c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 --2- 172.21.15.64:0/3393641555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d600ca420 0x7f5d6014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 >> 172.21.15.64:0/3393641555 conn(0x7f5d600cd990 msgr2=0x7f5d60153080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 shutdown_connections 2023-09-27T23:33:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.279+0000 7f5d68403700 1 -- 172.21.15.64:0/3393641555 wait complete. 2023-09-27T23:33:40.282 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.281+0000 7f20b0d4b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:33:40.282 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-09-27T23:33:40.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:40.317 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.311+0000 7faf43c7c000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:33:40.386 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.385+0000 7f20b0d4b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:40.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12de3f700 1 Processor -- start 2023-09-27T23:33:40.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12de3f700 1 -- start start 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12de3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe128159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12de3f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12815a260 con 0x7fe12806ad70 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12de3f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1280afad0 con 0x7fe12806b1e0 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12de3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280afdb0 con 0x7fe12806e1d0 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe127fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe12806ad70 0x7fe12814f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37492/0 (socket says 172.21.15.64:37492) 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe127fff700 1 -- 172.21.15.64:0/3432507291 learned_addr learned my addr 172.21.15.64:0/3432507291 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe12ce3d700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe128159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534974479 0 0) 0x7fe12815a260 con 0x7fe12806ad70 2023-09-27T23:33:40.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe118003570 con 0x7fe12806ad70 2023-09-27T23:33:40.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.395+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380991959 0 0) 0x7fe118003570 con 0x7fe12806ad70 2023-09-27T23:33:40.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 msgr2=0x7fe128159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe128159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 >> v1:172.21.15.123:6789/0 conn(0x7fe12806b1e0 legacy=0x7fe12814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280b0090 con 0x7fe12806ad70 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fe114001470 con 0x7fe12806ad70 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12ce3d700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe128159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 >> v1:172.21.15.64:6789/0 conn(0x7fe12806ad70 legacy=0x7fe12814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe118003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe118003fd0 0x7fe118004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118004d50 0x7fe1180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280afdb0 con 0x7fe118003fd0 2023-09-27T23:33:40.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11800bb40 con 0x7fe118004d50 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11800bd00 con 0x7fe12806e1d0 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe118003fd0 0x7fe118004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12ce3d700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe118003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12d63e700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118004d50 0x7fe1180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 -- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 msgr2=0x7fe118003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe12806e1d0 0x7fe118003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 -- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118004d50 msgr2=0x7fe1180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118004d50 0x7fe1180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 -- 172.21.15.64:0/3432507291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c009420 con 0x7fe118003fd0 2023-09-27T23:33:40.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe127fff700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe118003fd0 0x7fe118004810 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c2d0 tx=0x7fe114001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd17fa708e4ce44 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400cb60 con 0x7fe118003fd0 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11400ccc0 con 0x7fe118003fd0 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe1277fe700 1 -- 172.21.15.64:0/3432507291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114005b50 con 0x7fe118003fd0 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 -- 172.21.15.64:0/3432507291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe118003fd0 msgr2=0x7fe118004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 --2- 172.21.15.64:0/3432507291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe118003fd0 0x7fe118004810 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c2d0 tx=0x7fe114001df0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12d63e700 1 -- 172.21.15.64:0/3432507291 reap_dead start 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 -- 172.21.15.64:0/3432507291 shutdown_connections 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 -- 172.21.15.64:0/3432507291 >> 172.21.15.64:0/3432507291 conn(0x7fe128067bb0 msgr2=0x7fe1280add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 -- 172.21.15.64:0/3432507291 shutdown_connections 2023-09-27T23:33:40.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 -- 172.21.15.64:0/3432507291 wait complete. 2023-09-27T23:33:40.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 Processor -- start 2023-09-27T23:33:40.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.399+0000 7fe12de3f700 1 -- start start 2023-09-27T23:33:40.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118003fd0 0x7fe128140d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 0x7fe128150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe128168d70 0x7fe1281691e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe128141270 con 0x7fe128152050 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280aa7f0 con 0x7fe118003fd0 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280ab370 con 0x7fe128168d70 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 0x7fe128150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 0x7fe128150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60080/0 (socket says 172.21.15.64:60080) 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 -- 172.21.15.64:0/4239951278 learned_addr learned my addr 172.21.15.64:0/4239951278 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12d63e700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe128168d70 0x7fe1281691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12ce3d700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118003fd0 0x7fe128140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 -- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe128168d70 msgr2=0x7fe1281691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe128168d70 0x7fe1281691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 -- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118003fd0 msgr2=0x7fe128140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118003fd0 0x7fe128140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 -- 172.21.15.64:0/4239951278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280aaa40 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe127fff700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 0x7fe128150dd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c8c0 tx=0x7fe114015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140070d0 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114007270 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1280aad20 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114007630 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe128151310 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1280ab520 con 0x7fe128152050 2023-09-27T23:33:40.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fe114019030 con 0x7fe128152050 2023-09-27T23:33:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe124ff9700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe10c060b00 0x7fe10c062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fe11407e7d0 con 0x7fe128152050 2023-09-27T23:33:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12ce3d700 1 -- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe10c060b00 msgr2=0x7fe10c062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:40.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.403+0000 7fe12ce3d700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe10c060b00 0x7fe10c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:40.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.415+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe1280ab520 con 0x7fe128152050 2023-09-27T23:33:40.431 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.427+0000 7faf43c7c000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:40.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.571+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1280ab4e0 con 0x7fe128152050 2023-09-27T23:33:40.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.571+0000 7fe124ff9700 1 -- 172.21.15.64:0/4239951278 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17583 (secure 0 0 0) 0x7fe11404e530 con 0x7fe128152050 2023-09-27T23:33:40.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:40.575 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":144,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:36.654857+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:18.367224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:18.752273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:33:36.654780+0000","172.21.15.123:6832/33228":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/1464378569":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3916275130":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3298374072":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/2311414465":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/588895895":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3890296918":"2023-09-28T23:33:36.654780+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe10c060b00 msgr2=0x7fe10c062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe10c060b00 0x7fe10c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 msgr2=0x7fe128150dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 0x7fe128150dd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c8c0 tx=0x7fe114015000 comp rx=0 tx=0).stop 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 shutdown_connections 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe10c060b00 0x7fe10c062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe128168d70 0x7fe1281691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe128152050 0x7fe128150dd0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 --2- 172.21.15.64:0/4239951278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe118003fd0 0x7fe128140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 >> 172.21.15.64:0/4239951278 conn(0x7fe128067bb0 msgr2=0x7fe1280a90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 shutdown_connections 2023-09-27T23:33:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.575+0000 7fe12de3f700 1 -- 172.21.15.64:0/4239951278 wait complete. 2023-09-27T23:33:40.581 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.581+0000 7f20b0d4b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:33:40.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:33:18.367224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:33:18.752273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:33:40.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:33:40.629 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.629+0000 7f20b0d4b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:40.640 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.639+0000 7faf43c7c000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:33:40.692 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.691+0000 7faf43c7c000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:40.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 Processor -- start 2023-09-27T23:33:40.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 -- start start 2023-09-27T23:33:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f524c0cb970 0x7f524c06acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f524c0cecd0 0x7f524c06b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c0cbdf0 con 0x7f524c158e20 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c0cc0d0 con 0x7f524c0cb970 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c0cc3b0 con 0x7f524c0cecd0 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f524c0cb970 0x7f524c06acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f524c0cb970 0x7f524c06acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60094/0 (socket says 172.21.15.64:60094) 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 -- 172.21.15.64:0/2536069535 learned_addr learned my addr 172.21.15.64:0/2536069535 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f525107b700 1 --2- 172.21.15.64:0/2536069535 >> v2:172.21.15.123:3300/0 conn(0x7f524c0cecd0 0x7f524c06b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 -- 172.21.15.64:0/2536069535 >> v2:172.21.15.123:3300/0 conn(0x7f524c0cecd0 msgr2=0x7f524c06b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 --2- 172.21.15.64:0/2536069535 >> v2:172.21.15.123:3300/0 conn(0x7f524c0cecd0 0x7f524c06b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 -- 172.21.15.64:0/2536069535 >> v1:172.21.15.123:6789/0 conn(0x7f524c158e20 legacy=0x7f524c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 -- 172.21.15.64:0/2536069535 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0cc690 con 0x7f524c0cb970 2023-09-27T23:33:40.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524bfff700 1 --2- 172.21.15.64:0/2536069535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cb970 0x7f524c06acc0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f52380099b0 tx=0x7f5238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f32807a5c8c434 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:40.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524b7fe700 1 -- 172.21.15.64:0/2536069535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238009ea0 con 0x7f524c0cb970 2023-09-27T23:33:40.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524b7fe700 1 -- 172.21.15.64:0/2536069535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523800b820 con 0x7f524c0cb970 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f524b7fe700 1 -- 172.21.15.64:0/2536069535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523800bbc0 con 0x7f524c0cb970 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.695+0000 7f5252ade700 1 -- 172.21.15.64:0/2536069535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cb970 msgr2=0x7f524c06acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 --2- 172.21.15.64:0/2536069535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cb970 0x7f524c06acc0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f52380099b0 tx=0x7f5238009250 comp rx=0 tx=0).stop 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- 172.21.15.64:0/2536069535 shutdown_connections 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 --2- 172.21.15.64:0/2536069535 >> v2:172.21.15.123:3300/0 conn(0x7f524c0cecd0 0x7f524c06b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 --2- 172.21.15.64:0/2536069535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cb970 0x7f524c06acc0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- 172.21.15.64:0/2536069535 >> 172.21.15.64:0/2536069535 conn(0x7f524c067bb0 msgr2=0x7f524c0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- 172.21.15.64:0/2536069535 shutdown_connections 2023-09-27T23:33:40.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- 172.21.15.64:0/2536069535 wait complete. 2023-09-27T23:33:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 Processor -- start 2023-09-27T23:33:40.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- start start 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 0x7f524c14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f524c158e20 0x7f524c0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f524c0c9370 0x7f524c0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c156aa0 con 0x7f524c0cecd0 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b41c0 con 0x7f524c158e20 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b35b0 con 0x7f524c0c9370 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 0x7f524c14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 0x7f524c14aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60102/0 (socket says 172.21.15.64:60102) 2023-09-27T23:33:40.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 -- 172.21.15.64:0/3321362372 learned_addr learned my addr 172.21.15.64:0/3321362372 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525107b700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f524c0c9370 0x7f524c0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f524bfff700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f524c158e20 0x7f524c0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 -- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f524c0c9370 msgr2=0x7f524c0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f524c0c9370 0x7f524c0c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 -- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f524c158e20 msgr2=0x7f524c0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f524c158e20 0x7f524c0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 -- 172.21.15.64:0/3321362372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380094f0 con 0x7f524c0cecd0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f525087a700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 0x7f524c14aca0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f524000c590 tx=0x7f524000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240011590 con 0x7f524c0cecd0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52400116f0 con 0x7f524c0cecd0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c0b3890 con 0x7f524c0cecd0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240011ab0 con 0x7f524c0cecd0 2023-09-27T23:33:40.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.699+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c0b3e50 con 0x7f524c0cecd0 2023-09-27T23:33:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.703+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c071120 con 0x7f524c0cecd0 2023-09-27T23:33:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.703+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f524001b3b0 con 0x7f524c0cecd0 2023-09-27T23:33:40.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.703+0000 7f5248ff9700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5230060b00 0x7f5230062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.703+0000 7f524bfff700 1 -- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5230060b00 msgr2=0x7f5230062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.703+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f524007ca90 con 0x7f524c0cecd0 2023-09-27T23:33:40.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.703+0000 7f524bfff700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5230060b00 0x7f5230062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:40.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.711+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f524004c750 con 0x7f524c0cecd0 2023-09-27T23:33:40.782 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.781+0000 7f20b0d4b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:33:40.838 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.837+0000 7f20b0d4b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:33:40.852 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.847+0000 7faf43c7c000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:33:40.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.871+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f524c06a7a0 con 0x7f524c0cecd0 2023-09-27T23:33:40.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.871+0000 7f5248ff9700 1 -- 172.21.15.64:0/3321362372 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5240017020 con 0x7f524c0cecd0 2023-09-27T23:33:40.876 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5230060b00 msgr2=0x7f5230062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5230060b00 0x7f5230062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 msgr2=0x7f524c14aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 0x7f524c14aca0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f524000c590 tx=0x7f524000c950 comp rx=0 tx=0).stop 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 shutdown_connections 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5230060b00 0x7f5230062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f524c0c9370 0x7f524c0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f524c158e20 0x7f524c0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 --2- 172.21.15.64:0/3321362372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f524c0cecd0 0x7f524c14aca0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:40.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 >> 172.21.15.64:0/3321362372 conn(0x7f524c067bb0 msgr2=0x7f524c14caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:40.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 shutdown_connections 2023-09-27T23:33:40.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.875+0000 7f5252ade700 1 -- 172.21.15.64:0/3321362372 wait complete. 2023-09-27T23:33:40.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:33:40.896 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:40.897+0000 7f20b0d4b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:33:40.910 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.907+0000 7faf43c7c000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:33:40.974 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:40.971+0000 7faf43c7c000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:33:40.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.995+0000 7f87f750c700 1 Processor -- start 2023-09-27T23:33:41.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.995+0000 7f87f750c700 1 -- start start 2023-09-27T23:33:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f013f7e0 0x7f87f01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87f013fbb0 0x7f87f00af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87f00cba00 0x7f87f00cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f0157950 con 0x7f87f013fbb0 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f00cc3c0 con 0x7f87f00cba00 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00af780 con 0x7f87f013f7e0 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87f013fbb0 0x7f87f00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87f013fbb0 0x7f87f00af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60112/0 (socket says 172.21.15.64:60112) 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 -- 172.21.15.64:0/3183863374 learned_addr learned my addr 172.21.15.64:0/3183863374 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f52a8700 1 --2- 172.21.15.64:0/3183863374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f013f7e0 0x7f87f01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f5aa9700 1 --2- 172.21.15.64:0/3183863374 >> v2:172.21.15.123:3300/0 conn(0x7f87f00cba00 0x7f87f00cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 -- 172.21.15.64:0/3183863374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f013f7e0 msgr2=0x7f87f01564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 --2- 172.21.15.64:0/3183863374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f013f7e0 0x7f87f01564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 -- 172.21.15.64:0/3183863374 >> v2:172.21.15.123:3300/0 conn(0x7f87f00cba00 msgr2=0x7f87f00cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 --2- 172.21.15.64:0/3183863374 >> v2:172.21.15.123:3300/0 conn(0x7f87f00cba00 0x7f87f00cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 -- 172.21.15.64:0/3183863374 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0156af0 con 0x7f87f013fbb0 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f5aa9700 1 --2- 172.21.15.64:0/3183863374 >> v2:172.21.15.123:3300/0 conn(0x7f87f00cba00 0x7f87f00cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f4aa7700 1 --2- 172.21.15.64:0/3183863374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f00af240 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f87f00704f0 tx=0x7f87e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a993ed2bb75e4d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87e7fff700 1 -- 172.21.15.64:0/3183863374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0009e70 con 0x7f87f013fbb0 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87e7fff700 1 -- 172.21.15.64:0/3183863374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e000b8f0 con 0x7f87f013fbb0 2023-09-27T23:33:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87e7fff700 1 -- 172.21.15.64:0/3183863374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e000bc90 con 0x7f87f013fbb0 2023-09-27T23:33:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- 172.21.15.64:0/3183863374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 msgr2=0x7f87f00af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- 172.21.15.64:0/3183863374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f00af240 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f87f00704f0 tx=0x7f87e0009250 comp rx=0 tx=0).stop 2023-09-27T23:33:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- 172.21.15.64:0/3183863374 shutdown_connections 2023-09-27T23:33:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- 172.21.15.64:0/3183863374 >> v2:172.21.15.123:3300/0 conn(0x7f87f00cba00 0x7f87f00cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- 172.21.15.64:0/3183863374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f00af240 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 --2- 172.21.15.64:0/3183863374 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f013f7e0 0x7f87f01564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- 172.21.15.64:0/3183863374 >> 172.21.15.64:0/3183863374 conn(0x7f87f00cd8f0 msgr2=0x7f87f00c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- 172.21.15.64:0/3183863374 shutdown_connections 2023-09-27T23:33:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:40.999+0000 7f87f750c700 1 -- 172.21.15.64:0/3183863374 wait complete. 2023-09-27T23:33:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 Processor -- start 2023-09-27T23:33:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 -- start start 2023-09-27T23:33:41.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f00cba00 0x7f87f014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f013f7e0 0x7f87f01350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f0156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0135bc0 con 0x7f87f013fbb0 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0134020 con 0x7f87f013f7e0 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0134300 con 0x7f87f00cba00 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f0156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f0156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60120/0 (socket says 172.21.15.64:60120) 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 -- 172.21.15.64:0/1624517798 learned_addr learned my addr 172.21.15.64:0/1624517798 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f52a8700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f00cba00 0x7f87f014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f4aa7700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f013f7e0 0x7f87f01350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 -- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f00cba00 msgr2=0x7f87f014f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f00cba00 0x7f87f014f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 -- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f013f7e0 msgr2=0x7f87f01350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f013f7e0 0x7f87f01350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 -- 172.21.15.64:0/1624517798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00094f0 con 0x7f87f013fbb0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f5aa9700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f0156070 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d180 tx=0x7f87e8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e800dcc0 con 0x7f87f013fbb0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e800f040 con 0x7f87f013fbb0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f01345e0 con 0x7f87f013fbb0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8015690 con 0x7f87f013fbb0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.003+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f0134ba0 con 0x7f87f013fbb0 2023-09-27T23:33:41.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.007+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f0134d90 con 0x7f87f013fbb0 2023-09-27T23:33:41.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.007+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f87e801f440 con 0x7f87f013fbb0 2023-09-27T23:33:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.007+0000 7f87e57fa700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f87d4060b00 0x7f87d4062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.007+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f87e807deb0 con 0x7f87f013fbb0 2023-09-27T23:33:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.007+0000 7f87f52a8700 1 -- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f87d4060b00 msgr2=0x7f87d4062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:41.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.007+0000 7f87f52a8700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f87d4060b00 0x7f87d4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:41.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.015+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f87e804dae0 con 0x7f87f013fbb0 2023-09-27T23:33:41.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.171+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87f0071120 con 0x7f87f013fbb0 2023-09-27T23:33:41.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87e57fa700 1 -- 172.21.15.64:0/1624517798 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v144) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87e801a020 con 0x7f87f013fbb0 2023-09-27T23:33:41.177 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:33:41.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f87d4060b00 msgr2=0x7f87d4062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f87d4060b00 0x7f87d4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 msgr2=0x7f87f0156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f0156070 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d180 tx=0x7f87e8009a50 comp rx=0 tx=0).stop 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 shutdown_connections 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f87d4060b00 0x7f87d4062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87f013fbb0 0x7f87f0156070 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f013f7e0 0x7f87f01350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 --2- 172.21.15.64:0/1624517798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87f00cba00 0x7f87f014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 >> 172.21.15.64:0/1624517798 conn(0x7f87f00cd8f0 msgr2=0x7f87f00c1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 shutdown_connections 2023-09-27T23:33:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.175+0000 7f87f750c700 1 -- 172.21.15.64:0/1624517798 wait complete. 2023-09-27T23:33:41.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:33:41.193 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.193+0000 7f20b0d4b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:33:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.295+0000 7fa5112fe700 1 Processor -- start 2023-09-27T23:33:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.295+0000 7fa5112fe700 1 -- start start 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.295+0000 7fa5112fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c0ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa5112fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa50c155e60 0x7fa50c0ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa5112fe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0c68a0 con 0x7fa50c155a90 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa5112fe700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c0c6e60 con 0x7fa50c155e60 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa5112fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0ae4d0 con 0x7fa50c06db70 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa50c155e60 0x7fa50c0ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa50c155e60 0x7fa50c0ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60128/0 (socket says 172.21.15.64:60128) 2023-09-27T23:33:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 -- 172.21.15.64:0/1522126310 learned_addr learned my addr 172.21.15.64:0/1522126310 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa503fff700 1 --2- 172.21.15.64:0/1522126310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c0ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 -- 172.21.15.64:0/1522126310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 msgr2=0x7fa50c0ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 --2- 172.21.15.64:0/1522126310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c0ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 -- 172.21.15.64:0/1522126310 >> v1:172.21.15.123:6789/0 conn(0x7fa50c155a90 legacy=0x7fa50c0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 -- 172.21.15.64:0/1522126310 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0ae7b0 con 0x7fa50c155e60 2023-09-27T23:33:41.302 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:41.299+0000 7faf43c7c000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:33:41.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa510afd700 1 --2- 172.21.15.64:0/1522126310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155e60 0x7fa50c0ade70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d1a0 tx=0x7fa50400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5e62bdcbd0d628 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa50b7fe700 1 -- 172.21.15.64:0/1522126310 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504010070 con 0x7fa50c155e60 2023-09-27T23:33:41.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa50b7fe700 1 -- 172.21.15.64:0/1522126310 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400de10 con 0x7fa50c155e60 2023-09-27T23:33:41.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.299+0000 7fa50b7fe700 1 -- 172.21.15.64:0/1522126310 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400b320 con 0x7fa50c155e60 2023-09-27T23:33:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- 172.21.15.64:0/1522126310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155e60 msgr2=0x7fa50c0ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 --2- 172.21.15.64:0/1522126310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155e60 0x7fa50c0ade70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d1a0 tx=0x7fa50400ba50 comp rx=0 tx=0).stop 2023-09-27T23:33:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- 172.21.15.64:0/1522126310 shutdown_connections 2023-09-27T23:33:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 --2- 172.21.15.64:0/1522126310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155e60 0x7fa50c0ade70 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 --2- 172.21.15.64:0/1522126310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c0ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- 172.21.15.64:0/1522126310 >> 172.21.15.64:0/1522126310 conn(0x7fa50c0c8cd0 msgr2=0x7fa50c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- 172.21.15.64:0/1522126310 shutdown_connections 2023-09-27T23:33:41.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- 172.21.15.64:0/1522126310 wait complete. 2023-09-27T23:33:41.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 Processor -- start 2023-09-27T23:33:41.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- start start 2023-09-27T23:33:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 0x7fa50c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa50c155e60 0x7fa50c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0b0890 con 0x7fa50c155a90 2023-09-27T23:33:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c153000 con 0x7fa50c155e60 2023-09-27T23:33:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa5112fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0b0240 con 0x7fa50c06db70 2023-09-27T23:33:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa503fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 0x7fa50c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa503fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 0x7fa50c0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60140/0 (socket says 172.21.15.64:60140) 2023-09-27T23:33:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.303+0000 7fa503fff700 1 -- 172.21.15.64:0/4240154203 learned_addr learned my addr 172.21.15.64:0/4240154203 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa510afd700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa50c155e60 0x7fa50c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa50bfff700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa503fff700 1 -- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 msgr2=0x7fa50c1509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa503fff700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c1509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa503fff700 1 -- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa50c155e60 msgr2=0x7fa50c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa503fff700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa50c155e60 0x7fa50c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa503fff700 1 -- 172.21.15.64:0/4240154203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50400bcf0 con 0x7fa50c155a90 2023-09-27T23:33:41.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa503fff700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 0x7fa50c0cb330 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400ec50 tx=0x7fa4f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400c850 con 0x7fa50c155a90 2023-09-27T23:33:41.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f400c9b0 con 0x7fa50c155a90 2023-09-27T23:33:41.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c0b0520 con 0x7fa50c155a90 2023-09-27T23:33:41.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400cd70 con 0x7fa50c155a90 2023-09-27T23:33:41.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c1510a0 con 0x7fa50c155a90 2023-09-27T23:33:41.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f00052b0 con 0x7fa50c155a90 2023-09-27T23:33:41.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fa4f4013680 con 0x7fa50c155a90 2023-09-27T23:33:41.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa508ff9700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa4ec060b00 0x7fa4ec062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa4f407d5a0 con 0x7fa50c155a90 2023-09-27T23:33:41.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa50bfff700 1 -- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa4ec060b00 msgr2=0x7fa4ec062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:41.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.307+0000 7fa50bfff700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa4ec060b00 0x7fa4ec062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:41.314 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.313+0000 7f20b0d4b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:33:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.315+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa4f404d300 con 0x7fa50c155a90 2023-09-27T23:33:41.438 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:41.434+0000 7faf43c7c000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:33:41.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.478+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4f0005ae0 con 0x7fa50c155a90 2023-09-27T23:33:41.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.478+0000 7fa508ff9700 1 -- 172.21.15.64:0/4240154203 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4f4050920 con 0x7fa50c155a90 2023-09-27T23:33:41.484 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:33:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa4ec060b00 msgr2=0x7fa4ec062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:41.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa4ec060b00 0x7fa4ec062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 msgr2=0x7fa50c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 0x7fa50c0cb330 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400ec50 tx=0x7fa4f40091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 shutdown_connections 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fa4ec060b00 0x7fa4ec062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa50c155e60 0x7fa50c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa50c155a90 0x7fa50c0cb330 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 --2- 172.21.15.64:0/4240154203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa50c06db70 0x7fa50c1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 >> 172.21.15.64:0/4240154203 conn(0x7fa50c0c8cd0 msgr2=0x7fa50c0c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 shutdown_connections 2023-09-27T23:33:41.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.482+0000 7fa5112fe700 1 -- 172.21.15.64:0/4240154203 wait complete. 2023-09-27T23:33:41.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:33:41.595 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.593+0000 7f20b0d4b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:33:41.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1359e700 1 Processor -- start 2023-09-27T23:33:41.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1359e700 1 -- start start 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1406b730 con 0x7f1d140ab6d0 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d140ac390 con 0x7f1d140b1440 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140ac670 con 0x7f1d1406ae20 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d11d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d140ab6d0 0x7f1d140abb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37500/0 (socket says 172.21.15.64:37500) 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d11d9b700 1 -- 172.21.15.64:0/1375368508 learned_addr learned my addr 172.21.15.64:0/1375368508 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1259c700 1 --2- 172.21.15.64:0/1375368508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265175986 0 0) 0x7f1d1406b730 con 0x7f1d140ab6d0 2023-09-27T23:33:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d00003570 con 0x7f1d140ab6d0 2023-09-27T23:33:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272137215 0 0) 0x7f1d140ac390 con 0x7f1d140b1440 2023-09-27T23:33:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d1406b730 con 0x7f1d140b1440 2023-09-27T23:33:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1259c700 1 -- 172.21.15.64:0/1375368508 >> v1:172.21.15.123:6789/0 conn(0x7f1d140b1440 legacy=0x7f1d140b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2463190575 0 0) 0x7f1d00003570 con 0x7f1d140ab6d0 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1259c700 1 -- 172.21.15.64:0/1375368508 >> v1:172.21.15.64:6789/0 conn(0x7f1d140ab6d0 legacy=0x7f1d140abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1259c700 1 -- 172.21.15.64:0/1375368508 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d140ac950 con 0x7f1d1406ae20 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.610+0000 7f1d1259c700 1 --2- 172.21.15.64:0/1375368508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1406b1f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009e90 tx=0x7f1d04009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d1236bc796966e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400e070 con 0x7f1d1406ae20 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0400b870 con 0x7f1d1406ae20 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1159a700 1 -- 172.21.15.64:0/1375368508 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc001470 con 0x7f1d1406ae20 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- 172.21.15.64:0/1375368508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 msgr2=0x7f1d1406b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1375368508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1406b1f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009e90 tx=0x7f1d04009780 comp rx=0 tx=0).stop 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- 172.21.15.64:0/1375368508 shutdown_connections 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1375368508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1406b1f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- 172.21.15.64:0/1375368508 >> 172.21.15.64:0/1375368508 conn(0x7f1d140c8cd0 msgr2=0x7f1d140c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- 172.21.15.64:0/1375368508 shutdown_connections 2023-09-27T23:33:41.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- 172.21.15.64:0/1375368508 wait complete. 2023-09-27T23:33:41.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 Processor -- start 2023-09-27T23:33:41.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- start start 2023-09-27T23:33:41.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d140ab6d0 0x7f1d14149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 0x7f1d140cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1415a0e0 con 0x7f1d140b1440 2023-09-27T23:33:41.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1415a3c0 con 0x7f1d140ab6d0 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.614+0000 7f1d1359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1415a6a0 con 0x7f1d1406ae20 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 0x7f1d140cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d1259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 0x7f1d140cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60150/0 (socket says 172.21.15.64:60150) 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 -- 172.21.15.64:0/1769084658 learned_addr learned my addr 172.21.15.64:0/1769084658 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d11d9b700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d140ab6d0 0x7f1d14149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 -- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 msgr2=0x7f1d1415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1415b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 -- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d140ab6d0 msgr2=0x7f1d14149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d140ab6d0 0x7f1d14149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 -- 172.21.15.64:0/1769084658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04009a20 con 0x7f1d140b1440 2023-09-27T23:33:41.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d12d9d700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 0x7f1d140cb3a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00c590 tx=0x7f1d0c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c016070 con 0x7f1d140b1440 2023-09-27T23:33:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c011590 con 0x7f1d140b1440 2023-09-27T23:33:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c011970 con 0x7f1d140b1440 2023-09-27T23:33:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1415a980 con 0x7f1d140b1440 2023-09-27T23:33:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14149d70 con 0x7f1d140b1440 2023-09-27T23:33:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1415ae80 con 0x7f1d140b1440 2023-09-27T23:33:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f1d0c012420 con 0x7f1d140b1440 2023-09-27T23:33:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d0a7fc700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf4060bd0 0x7f1cf4063080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d1259c700 1 -- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf4060bd0 msgr2=0x7f1cf4063080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f1d0c07cdb0 con 0x7f1d140b1440 2023-09-27T23:33:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.618+0000 7f1d1259c700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf4060bd0 0x7f1cf4063080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.626+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1d0c04cb90 con 0x7f1d140b1440 2023-09-27T23:33:41.759 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.757+0000 7f20b0d4b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:41.768 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:41.762+0000 7faf43c7c000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:33:41.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.782+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d1404be40 con 0x7f1d140b1440 2023-09-27T23:33:41.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.782+0000 7f1d0a7fc700 1 -- 172.21.15.64:0/1769084658 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d0c0501b0 con 0x7f1d140b1440 2023-09-27T23:33:41.788 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:33:41.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf4060bd0 msgr2=0x7f1cf4063080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf4060bd0 0x7f1cf4063080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 msgr2=0x7f1d140cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 0x7f1d140cb3a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00c590 tx=0x7f1d0c007d80 comp rx=0 tx=0).stop 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 shutdown_connections 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cf4060bd0 0x7f1cf4063080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1440 0x7f1d140cb3a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d140ab6d0 0x7f1d14149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 --2- 172.21.15.64:0/1769084658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d1406ae20 0x7f1d1415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 >> 172.21.15.64:0/1769084658 conn(0x7f1d140c8cd0 msgr2=0x7f1d140b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 shutdown_connections 2023-09-27T23:33:41.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.786+0000 7f1d1359e700 1 -- 172.21.15.64:0/1769084658 wait complete. 2023-09-27T23:33:41.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:41.822 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.821+0000 7f20b0d4b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:33:41.874 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.873+0000 7f20b0d4b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:33:41.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.898+0000 7f9b3f35b700 1 Processor -- start 2023-09-27T23:33:41.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.898+0000 7f9b3f35b700 1 -- start start 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9b381533e0 0x7f9b3815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b380683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b380c98e0 con 0x7f9b38153010 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3806c980 con 0x7f9b381533e0 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3815d200 con 0x7f9b3815cde0 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9b381533e0 0x7f9b3815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9b381533e0 0x7f9b3815c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60160/0 (socket says 172.21.15.64:60160) 2023-09-27T23:33:41.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 -- 172.21.15.64:0/1983781419 learned_addr learned my addr 172.21.15.64:0/1983781419 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3eb5a700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b380683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 -- 172.21.15.64:0/1983781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 msgr2=0x7f9b380683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b380683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 -- 172.21.15.64:0/1983781419 >> v1:172.21.15.123:6789/0 conn(0x7f9b38153010 legacy=0x7f9b380c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 -- 172.21.15.64:0/1983781419 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380c9040 con 0x7f9b381533e0 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3eb5a700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b380683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3db58700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b381533e0 0x7f9b3815c8a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0099b0 tx=0x7f9b2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4207834f6e5eac29 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3d357700 1 -- 172.21.15.64:0/1983781419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00b860 con 0x7f9b381533e0 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3d357700 1 -- 172.21.15.64:0/1983781419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c00b9c0 con 0x7f9b381533e0 2023-09-27T23:33:41.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3d357700 1 -- 172.21.15.64:0/1983781419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00bd60 con 0x7f9b381533e0 2023-09-27T23:33:41.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- 172.21.15.64:0/1983781419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b381533e0 msgr2=0x7f9b3815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b381533e0 0x7f9b3815c8a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0099b0 tx=0x7f9b2c009250 comp rx=0 tx=0).stop 2023-09-27T23:33:41.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- 172.21.15.64:0/1983781419 shutdown_connections 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b380683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/1983781419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b381533e0 0x7f9b3815c8a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- 172.21.15.64:0/1983781419 >> 172.21.15.64:0/1983781419 conn(0x7f9b380cdbd0 msgr2=0x7f9b380cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- 172.21.15.64:0/1983781419 shutdown_connections 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- 172.21.15.64:0/1983781419 wait complete. 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 Processor -- start 2023-09-27T23:33:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.902+0000 7f9b3f35b700 1 -- start start 2023-09-27T23:33:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 0x7f9b3816c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b381533e0 0x7f9b380b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b381599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3816d880 con 0x7f9b38153010 2023-09-27T23:33:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3816d2c0 con 0x7f9b381533e0 2023-09-27T23:33:41.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3816d5a0 con 0x7f9b3815cde0 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 0x7f9b3816c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 0x7f9b3816c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60164/0 (socket says 172.21.15.64:60164) 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 -- 172.21.15.64:0/2150059005 learned_addr learned my addr 172.21.15.64:0/2150059005 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3db58700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b381533e0 0x7f9b380b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3eb5a700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b381599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 -- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 msgr2=0x7f9b381599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b381599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 -- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b381533e0 msgr2=0x7f9b380b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b381533e0 0x7f9b380b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 -- 172.21.15.64:0/2150059005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0094f0 con 0x7f9b38153010 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3db58700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b381533e0 0x7f9b380b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3e359700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 0x7f9b3816c990 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c590 tx=0x7f9b2800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28011590 con 0x7f9b38153010 2023-09-27T23:33:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b280116f0 con 0x7f9b38153010 2023-09-27T23:33:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b381591b0 con 0x7f9b38153010 2023-09-27T23:33:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28011ab0 con 0x7f9b38153010 2023-09-27T23:33:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3816ced0 con 0x7f9b38153010 2023-09-27T23:33:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3816d060 con 0x7f9b38153010 2023-09-27T23:33:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.906+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f9b2801b3b0 con 0x7f9b38153010 2023-09-27T23:33:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.910+0000 7f9b367fc700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9b24060b00 0x7f9b24062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.910+0000 7f9b3db58700 1 -- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9b24060b00 msgr2=0x7f9b24062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.910+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f9b2807c9f0 con 0x7f9b38153010 2023-09-27T23:33:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.910+0000 7f9b3db58700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9b24060b00 0x7f9b24062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:41.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:41.914+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9b2804c750 con 0x7f9b38153010 2023-09-27T23:33:41.930 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:41.929+0000 7f20b0d4b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:33:41.944 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:41.938+0000 7faf43c7c000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:33:42.013 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.010+0000 7faf43c7c000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:33:42.067 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.062+0000 7faf43c7c000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:33:42.101 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:42.101+0000 7f20b0d4b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:42.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.106+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b381596b0 con 0x7f9b38153010 2023-09-27T23:33:42.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.106+0000 7f9b367fc700 1 -- 172.21.15.64:0/2150059005 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f9b28017020 con 0x7f9b38153010 2023-09-27T23:33:42.112 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:42.112 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:19.775714+0000","modified":"2023-09-27T23:33:22.797158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2286000796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2286000796},{"type":"v1","addr":"172.21.15.64:6835","nonce":2286000796}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-09-27T23:33:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9b24060b00 msgr2=0x7f9b24062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9b24060b00 0x7f9b24062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 msgr2=0x7f9b3816c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 0x7f9b3816c990 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c590 tx=0x7f9b2800c950 comp rx=0 tx=0).stop 2023-09-27T23:33:42.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 shutdown_connections 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9b24060b00 0x7f9b24062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3815cde0 0x7f9b381599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b381533e0 0x7f9b380b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 --2- 172.21.15.64:0/2150059005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b38153010 0x7f9b3816c990 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 >> 172.21.15.64:0/2150059005 conn(0x7f9b380cdbd0 msgr2=0x7f9b380c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 shutdown_connections 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.110+0000 7f9b3f35b700 1 -- 172.21.15.64:0/2150059005 wait complete. 2023-09-27T23:33:42.115 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-09-27T23:33:42.126 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.122+0000 7faf43c7c000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:33:42.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:42.156 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:42.157+0000 7f20b0d4b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:33:42.218 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:42.217+0000 7f20b0d4b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:33:42.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f33019f6700 1 Processor -- start 2023-09-27T23:33:42.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f33019f6700 1 -- start start 2023-09-27T23:33:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f33019f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32fc158f70 0x7f32fc067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f33019f6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc068c70 con 0x7f32fc069100 2023-09-27T23:33:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f33019f6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc068060 con 0x7f32fc159340 2023-09-27T23:33:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f33019f6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32fc068340 con 0x7f32fc158f70 2023-09-27T23:33:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32fc158f70 0x7f32fc067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32fc158f70 0x7f32fc067980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60178/0 (socket says 172.21.15.64:60178) 2023-09-27T23:33:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32fa7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f32fc069100 0x7f32fc0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37502/0 (socket says 172.21.15.64:37502) 2023-09-27T23:33:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 -- 172.21.15.64:0/3608126924 learned_addr learned my addr 172.21.15.64:0/3608126924 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 -- 172.21.15.64:0/3608126924 >> v1:172.21.15.123:6789/0 conn(0x7f32fc159340 legacy=0x7f32fc0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 -- 172.21.15.64:0/3608126924 >> v1:172.21.15.64:6789/0 conn(0x7f32fc069100 legacy=0x7f32fc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 -- 172.21.15.64:0/3608126924 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc068620 con 0x7f32fc158f70 2023-09-27T23:33:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32faffd700 1 --2- 172.21.15.64:0/3608126924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc158f70 0x7f32fc067980 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0099e0 tx=0x7f32ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d028096ca86e078 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32f9ffb700 1 -- 172.21.15.64:0/3608126924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00b820 con 0x7f32fc158f70 2023-09-27T23:33:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32f9ffb700 1 -- 172.21.15.64:0/3608126924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec00b980 con 0x7f32fc158f70 2023-09-27T23:33:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.230+0000 7f32f9ffb700 1 -- 172.21.15.64:0/3608126924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00bd20 con 0x7f32fc158f70 2023-09-27T23:33:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- 172.21.15.64:0/3608126924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc158f70 msgr2=0x7f32fc067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 --2- 172.21.15.64:0/3608126924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc158f70 0x7f32fc067980 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0099e0 tx=0x7f32ec009250 comp rx=0 tx=0).stop 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- 172.21.15.64:0/3608126924 shutdown_connections 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 --2- 172.21.15.64:0/3608126924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc158f70 0x7f32fc067980 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- 172.21.15.64:0/3608126924 >> 172.21.15.64:0/3608126924 conn(0x7f32fc0ccc40 msgr2=0x7f32fc0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- 172.21.15.64:0/3608126924 shutdown_connections 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- 172.21.15.64:0/3608126924 wait complete. 2023-09-27T23:33:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 Processor -- start 2023-09-27T23:33:42.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- start start 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32fc069100 0x7f32fc14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32fc158f70 0x7f32fc15def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 0x7f32fc15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc16aad0 con 0x7f32fc159340 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0b6180 con 0x7f32fc158f70 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f33019f6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc15da60 con 0x7f32fc069100 2023-09-27T23:33:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f32fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 0x7f32fc15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f32fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 0x7f32fc15bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60180/0 (socket says 172.21.15.64:60180) 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f32fb7fe700 1 -- 172.21.15.64:0/3522639644 learned_addr learned my addr 172.21.15.64:0/3522639644 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f32faffd700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32fc069100 0x7f32fc14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.234+0000 7f32fa7fc700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32fc158f70 0x7f32fc15def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32fb7fe700 1 -- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32fc069100 msgr2=0x7f32fc14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32fb7fe700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32fc069100 0x7f32fc14d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32fb7fe700 1 -- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32fc158f70 msgr2=0x7f32fc15def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32fb7fe700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32fc158f70 0x7f32fc15def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32fb7fe700 1 -- 172.21.15.64:0/3522639644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0094f0 con 0x7f32fc159340 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32fb7fe700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 0x7f32fc15bab0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c840 tx=0x7f32f4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4012630 con 0x7f32fc159340 2023-09-27T23:33:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f40127b0 con 0x7f32fc159340 2023-09-27T23:33:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc14dd90 con 0x7f32fc159340 2023-09-27T23:33:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4012b70 con 0x7f32fc159340 2023-09-27T23:33:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc15d410 con 0x7f32fc159340 2023-09-27T23:33:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc15dbd0 con 0x7f32fc159340 2023-09-27T23:33:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f32f4003030 con 0x7f32fc159340 2023-09-27T23:33:42.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32e2ffd700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f32e8060b00 0x7f32e8062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32faffd700 1 -- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f32e8060b00 msgr2=0x7f32e8062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:42.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.238+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f32f407d260 con 0x7f32fc159340 2023-09-27T23:33:42.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.242+0000 7f32faffd700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f32e8060b00 0x7f32e8062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:42.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.246+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f32f404cfc0 con 0x7f32fc159340 2023-09-27T23:33:42.307 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.302+0000 7faf43c7c000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:33:42.365 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.362+0000 7faf43c7c000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:33:42.366 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:42.365+0000 7f20b0d4b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:33:42.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.406+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32fc071120 con 0x7f32fc159340 2023-09-27T23:33:42.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f32e2ffd700 1 -- 172.21.15.64:0/3522639644 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17583 (secure 0 0 0) 0x7f32f40505e0 con 0x7f32fc159340 2023-09-27T23:33:42.413 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:42.414 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":144,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:36.654857+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:18.367224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:18.752273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6833/33228":"2023-09-28T23:33:36.654780+0000","172.21.15.123:6832/33228":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/1464378569":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3916275130":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3298374072":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/2311414465":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/588895895":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3890296918":"2023-09-28T23:33:36.654780+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f32e8060b00 msgr2=0x7f32e8062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f32e8060b00 0x7f32e8062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 msgr2=0x7f32fc15bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 0x7f32fc15bab0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c840 tx=0x7f32f4001650 comp rx=0 tx=0).stop 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 shutdown_connections 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f32e8060b00 0x7f32e8062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32fc159340 0x7f32fc15bab0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32fc158f70 0x7f32fc15def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 --2- 172.21.15.64:0/3522639644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32fc069100 0x7f32fc14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 >> 172.21.15.64:0/3522639644 conn(0x7f32fc0ccc40 msgr2=0x7f32fc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:42.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 shutdown_connections 2023-09-27T23:33:42.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.410+0000 7f33019f6700 1 -- 172.21.15.64:0/3522639644 wait complete. 2023-09-27T23:33:42.426 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:33:42.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:42.427 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:42.425+0000 7f20b0d4b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:33:42.431 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.426+0000 7faf43c7c000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:33:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.522+0000 7fec16b8c700 1 Processor -- start 2023-09-27T23:33:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.522+0000 7fec16b8c700 1 -- start start 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec100b0810 0x7fec100b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7fec10158d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec100af530 con 0x7fec100af9c0 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec10159660 con 0x7fec100b0810 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec10159940 con 0x7fec1006be10 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec1638b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fec100af9c0 0x7fec10154b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37508/0 (socket says 172.21.15.64:37508) 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec1638b700 1 -- 172.21.15.64:0/3572530266 learned_addr learned my addr 172.21.15.64:0/3572530266 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15389700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7fec10158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 --2- 172.21.15.64:0/3572530266 >> v2:172.21.15.123:3300/0 conn(0x7fec100b0810 0x7fec100b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956692503 0 0) 0x7fec100af530 con 0x7fec100af9c0 2023-09-27T23:33:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febfc003570 con 0x7fec100af9c0 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2412922837 0 0) 0x7febfc003570 con 0x7fec100af9c0 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 msgr2=0x7fec10158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7fec10158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 >> v2:172.21.15.123:3300/0 conn(0x7fec100b0810 msgr2=0x7fec100b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 --2- 172.21.15.64:0/3572530266 >> v2:172.21.15.123:3300/0 conn(0x7fec100b0810 0x7fec100b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec10159c20 con 0x7fec100af9c0 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fec080017f0 con 0x7fec100af9c0 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 >> v1:172.21.15.64:6789/0 conn(0x7fec100af9c0 legacy=0x7fec10154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7febfc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febfc003fe0 0x7febfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febfc004d50 0x7febfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec10159660 con 0x7febfc003fe0 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec10159940 con 0x7febfc004d50 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc00ba90 con 0x7fec1006be10 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15389700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febfc003fe0 0x7febfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7febfc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec1638b700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febfc004d50 0x7febfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 -- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febfc004d50 msgr2=0x7febfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febfc004d50 0x7febfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 -- 172.21.15.64:0/3572530266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febfc003fe0 msgr2=0x7febfc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febfc003fe0 0x7febfc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 -- 172.21.15.64:0/3572530266 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc003570 con 0x7fec1006be10 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15389700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febfc003fe0 0x7febfc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:42.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec15b8a700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7febfc003a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009c00 tx=0x7fec0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b683ca2db290c3b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00e6b0 con 0x7fec1006be10 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00e810 con 0x7fec1006be10 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec14b88700 1 -- 172.21.15.64:0/3572530266 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00ebd0 con 0x7fec1006be10 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 msgr2=0x7febfc003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 --2- 172.21.15.64:0/3572530266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec1006be10 0x7febfc003a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009c00 tx=0x7fec0c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec1638b700 1 -- 172.21.15.64:0/3572530266 reap_dead start 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- 172.21.15.64:0/3572530266 shutdown_connections 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- 172.21.15.64:0/3572530266 >> 172.21.15.64:0/3572530266 conn(0x7fec10067bb0 msgr2=0x7fec100add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- 172.21.15.64:0/3572530266 shutdown_connections 2023-09-27T23:33:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.526+0000 7fec16b8c700 1 -- 172.21.15.64:0/3572530266 wait complete. 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 Processor -- start 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 -- start start 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 0x7fec101576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec10157c30 0x7fec1014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec10156030 0x7fec1014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec10155ba0 con 0x7fec1006be10 2023-09-27T23:33:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec101552d0 con 0x7fec10156030 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1014fc70 con 0x7fec10157c30 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 0x7fec101576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15389700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec10157c30 0x7fec1014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 0x7fec101576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60186/0 (socket says 172.21.15.64:60186) 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 -- 172.21.15.64:0/2392217865 learned_addr learned my addr 172.21.15.64:0/2392217865 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec1638b700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec10156030 0x7fec1014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 -- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec10157c30 msgr2=0x7fec1014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec10157c30 0x7fec1014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 -- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec10156030 msgr2=0x7fec1014d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec10156030 0x7fec1014d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 -- 172.21.15.64:0/2392217865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec10155550 con 0x7fec1006be10 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec15b8a700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 0x7fec101576f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7febfc00bc20 tx=0x7fec0c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c022070 con 0x7fec1006be10 2023-09-27T23:33:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00e980 con 0x7fec1006be10 2023-09-27T23:33:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec10155830 con 0x7fec1006be10 2023-09-27T23:33:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c027470 con 0x7fec1006be10 2023-09-27T23:33:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.530+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1014f6f0 con 0x7fec1006be10 2023-09-27T23:33:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.534+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1014f9d0 con 0x7fec1006be10 2023-09-27T23:33:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.534+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fec0c01c450 con 0x7fec1006be10 2023-09-27T23:33:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.534+0000 7fec05ffb700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 0x7febf4062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.534+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fec0c07ef10 con 0x7fec1006be10 2023-09-27T23:33:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.534+0000 7fec15389700 1 -- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 msgr2=0x7febf4062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.534+0000 7fec15389700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 0x7febf4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.542+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fec0c0527c0 con 0x7fec1006be10 2023-09-27T23:33:42.592 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.586+0000 7faf43c7c000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:33:42.657 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:42.654+0000 7faf43c7c000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:33:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.738+0000 7fec15389700 1 -- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 msgr2=0x7febf4062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.738+0000 7fec15389700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 0x7febf4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:33:42.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.738+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec1004be40 con 0x7fec1006be10 2023-09-27T23:33:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.738+0000 7fec05ffb700 1 -- 172.21.15.64:0/2392217865 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fec0c02c090 con 0x7fec1006be10 2023-09-27T23:33:42.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:42.744 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:19.775714+0000","modified":"2023-09-27T23:33:22.797158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2286000796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2286000796},{"type":"v1","addr":"172.21.15.64:6835","nonce":2286000796}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 msgr2=0x7febf4062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 0x7febf4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 msgr2=0x7fec101576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 0x7fec101576f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7febfc00bc20 tx=0x7fec0c01a000 comp rx=0 tx=0).stop 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 shutdown_connections 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7febf4060b00 0x7febf4062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec10156030 0x7fec1014d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec10157c30 0x7fec1014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 --2- 172.21.15.64:0/2392217865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec1006be10 0x7fec101576f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 >> 172.21.15.64:0/2392217865 conn(0x7fec10067bb0 msgr2=0x7fec1006ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:42.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 shutdown_connections 2023-09-27T23:33:42.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.742+0000 7fec16b8c700 1 -- 172.21.15.64:0/2392217865 wait complete. 2023-09-27T23:33:42.747 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-09-27T23:33:42.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.858+0000 7f055305d700 1 Processor -- start 2023-09-27T23:33:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.858+0000 7f055305d700 1 -- start start 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f0544054600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0544057470 con 0x7f05440dff90 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0544057750 con 0x7f05440dfbc0 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440075b0 con 0x7f05440e6550 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055185a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05440dff90 0x7f05440e5e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37518/0 (socket says 172.21.15.64:37518) 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055185a700 1 -- 172.21.15.64:0/2795317719 learned_addr learned my addr 172.21.15.64:0/2795317719 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055285c700 1 --2- 172.21.15.64:0/2795317719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f0544054600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f0551059700 1 -- 172.21.15.64:0/2795317719 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264263557 0 0) 0x7f0544057470 con 0x7f05440dff90 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f0551059700 1 -- 172.21.15.64:0/2795317719 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0538003570 con 0x7f05440dff90 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055285c700 1 -- 172.21.15.64:0/2795317719 >> v1:172.21.15.123:6789/0 conn(0x7f05440dfbc0 legacy=0x7f0544006dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055285c700 1 -- 172.21.15.64:0/2795317719 >> v1:172.21.15.64:6789/0 conn(0x7f05440dff90 legacy=0x7f05440e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055285c700 1 -- 172.21.15.64:0/2795317719 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544007890 con 0x7f05440e6550 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055285c700 1 --2- 172.21.15.64:0/2795317719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f0544054600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f054c071030 tx=0x7f054c066cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d73c2331fd89861 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f0551059700 1 -- 172.21.15.64:0/2795317719 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c04be40 con 0x7f05440e6550 2023-09-27T23:33:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f0551059700 1 -- 172.21.15.64:0/2795317719 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c0008d0 con 0x7f05440e6550 2023-09-27T23:33:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f0551059700 1 -- 172.21.15.64:0/2795317719 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c070460 con 0x7f05440e6550 2023-09-27T23:33:42.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- 172.21.15.64:0/2795317719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 msgr2=0x7f0544054600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 --2- 172.21.15.64:0/2795317719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f0544054600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f054c071030 tx=0x7f054c066cd0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- 172.21.15.64:0/2795317719 shutdown_connections 2023-09-27T23:33:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 --2- 172.21.15.64:0/2795317719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f0544054600 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- 172.21.15.64:0/2795317719 >> 172.21.15.64:0/2795317719 conn(0x7f054400e820 msgr2=0x7f054400ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- 172.21.15.64:0/2795317719 shutdown_connections 2023-09-27T23:33:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.862+0000 7f055305d700 1 -- 172.21.15.64:0/2795317719 wait complete. 2023-09-27T23:33:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 Processor -- start 2023-09-27T23:33:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 -- start start 2023-09-27T23:33:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 0x7f054403ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440dff90 0x7f05440d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f05440e1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440e2cd0 con 0x7f05440dfbc0 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440e20c0 con 0x7f05440dff90 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440e23a0 con 0x7f05440e6550 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 0x7f054403ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 0x7f054403ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60192/0 (socket says 172.21.15.64:60192) 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 -- 172.21.15.64:0/2510082884 learned_addr learned my addr 172.21.15.64:0/2510082884 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055285c700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f05440e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055185a700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440dff90 0x7f05440d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 -- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 msgr2=0x7f05440e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f05440e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 -- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440dff90 msgr2=0x7f05440d6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440dff90 0x7f05440d6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 -- 172.21.15.64:0/2510082884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440e2680 con 0x7f05440dfbc0 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055205b700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 0x7f054403ceb0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f054800c590 tx=0x7f0548007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548011590 con 0x7f05440dfbc0 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0548011730 con 0x7f05440dfbc0 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05440e2960 con 0x7f05440dfbc0 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548011af0 con 0x7f05440dfbc0 2023-09-27T23:33:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.866+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05440e1d60 con 0x7f05440dfbc0 2023-09-27T23:33:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.870+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 62) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f0548012420 con 0x7f05440dfbc0 2023-09-27T23:33:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.870+0000 7f05427fc700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0530060b00 0x7f0530062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.870+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f054807c6a0 con 0x7f05440dfbc0 2023-09-27T23:33:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.870+0000 7f055185a700 1 -- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0530060b00 msgr2=0x7f0530062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:33:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.870+0000 7f055185a700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0530060b00 0x7f0530062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:33:42.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.874+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0544005950 con 0x7f05440dfbc0 2023-09-27T23:33:42.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:42.878+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0548017080 con 0x7f05440dfbc0 2023-09-27T23:33:43.018 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.017+0000 7f20b0d4b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:33:43.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.050+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 63) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f054804c880 con 0x7f05440dfbc0 2023-09-27T23:33:43.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.050+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0530060b00 msgr2=0x7f0530062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:33:43.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.050+0000 7f05427fc700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0530060b00 0x7f0530062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05440e5260 con 0x7f05440dfbc0 2023-09-27T23:33:43.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f05427fc700 1 -- 172.21.15.64:0/2510082884 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f0548038260 con 0x7f05440dfbc0 2023-09-27T23:33:43.097 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:43.097 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":54,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:19.775714+0000","modified":"2023-09-27T23:33:22.797158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/2286000796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2286000796},{"type":"v1","addr":"172.21.15.64:6835","nonce":2286000796}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-09-27T23:33:43.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 msgr2=0x7f054403ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 0x7f054403ceb0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f054800c590 tx=0x7f0548007f30 comp rx=0 tx=0).stop 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 shutdown_connections 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0530060b00 0x7f0530062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440e6550 0x7f05440e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440dff90 0x7f05440d6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 --2- 172.21.15.64:0/2510082884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05440dfbc0 0x7f054403ceb0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 >> 172.21.15.64:0/2510082884 conn(0x7f054400e820 msgr2=0x7f0544053990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 shutdown_connections 2023-09-27T23:33:43.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.094+0000 7f055305d700 1 -- 172.21.15.64:0/2510082884 wait complete. 2023-09-27T23:33:43.101 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 54 2023-09-27T23:33:43.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:43.114 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207e790700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.114 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207e790700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.114 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207e790700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.114 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207e790700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.114 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207e790700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.116 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207b78a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.116 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207b78a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.116 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207b78a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.116 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207b78a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.116 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:33:43.113+0000 7f207b78a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:33:43.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a9a278700 1 Processor -- start 2023-09-27T23:33:43.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a9a278700 1 -- start start 2023-09-27T23:33:43.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a9a278700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a940b0be0 0x7f5a94154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a9a278700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940af530 con 0x7f5a940b0810 2023-09-27T23:33:43.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a9a278700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940b00d0 con 0x7f5a9406be10 2023-09-27T23:33:43.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a9a278700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a94159470 con 0x7f5a940b0be0 2023-09-27T23:33:43.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a99276700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5a940b0810 0x7f5a94158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37528/0 (socket says 172.21.15.64:37528) 2023-09-27T23:33:43.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a99276700 1 -- 172.21.15.64:0/2620349478 learned_addr learned my addr 172.21.15.64:0/2620349478 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:43.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.206+0000 7f5a99a77700 1 --2- 172.21.15.64:0/2620349478 >> v2:172.21.15.123:3300/0 conn(0x7f5a940b0be0 0x7f5a94154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178415213 0 0) 0x7f5a940af530 con 0x7f5a940b0810 2023-09-27T23:33:43.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a80003570 con 0x7f5a940b0810 2023-09-27T23:33:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663892612 0 0) 0x7f5a80003570 con 0x7f5a940b0810 2023-09-27T23:33:43.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 >> v2:172.21.15.123:3300/0 conn(0x7f5a940b0be0 msgr2=0x7f5a94154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 --2- 172.21.15.64:0/2620349478 >> v2:172.21.15.123:3300/0 conn(0x7f5a940b0be0 0x7f5a94154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 >> v1:172.21.15.123:6789/0 conn(0x7f5a9406be10 legacy=0x7f5a940af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a94159660 con 0x7f5a940b0810 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f5a90001470 con 0x7f5a940b0810 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 >> v1:172.21.15.64:6789/0 conn(0x7f5a940b0810 legacy=0x7f5a94158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a80003a60 0x7f5a80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a80004810 0x7f5a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a800051c0 0x7f5a8000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000c0a0 con 0x7f5a800051c0 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000c210 con 0x7f5a80004810 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000c3d0 con 0x7f5a80003a60 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a800051c0 0x7f5a8000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a98a75700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a80004810 0x7f5a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99276700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a80003a60 0x7f5a80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 -- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a80003a60 msgr2=0x7f5a80003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a80003a60 0x7f5a80003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 -- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a80004810 msgr2=0x7f5a80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a80004810 0x7f5a80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 -- 172.21.15.64:0/2620349478 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80003570 con 0x7f5a800051c0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a98a75700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a80004810 0x7f5a80004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 -- 172.21.15.64:0/2620349478 reap_dead start 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a99a77700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a800051c0 0x7f5a8000ba60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00a2c0 tx=0x7f5a8c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660b16fd2bfa0ec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00acc0 con 0x7f5a800051c0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c010040 con 0x7f5a800051c0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a8bfff700 1 -- 172.21.15.64:0/2620349478 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c009de0 con 0x7f5a800051c0 2023-09-27T23:33:43.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 -- 172.21.15.64:0/2620349478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a800051c0 msgr2=0x7f5a8000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a800051c0 0x7f5a8000ba60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00a2c0 tx=0x7f5a8c00a890 comp rx=0 tx=0).stop 2023-09-27T23:33:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 -- 172.21.15.64:0/2620349478 shutdown_connections 2023-09-27T23:33:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 --2- 172.21.15.64:0/2620349478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a800051c0 0x7f5a8000ba60 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 -- 172.21.15.64:0/2620349478 >> 172.21.15.64:0/2620349478 conn(0x7f5a94067bb0 msgr2=0x7f5a940add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 -- 172.21.15.64:0/2620349478 shutdown_connections 2023-09-27T23:33:43.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 -- 172.21.15.64:0/2620349478 wait complete. 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.210+0000 7f5a9a278700 1 Processor -- start 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- start start 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a800051c0 0x7f5a94157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a94154e40 0x7f5a9416cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 0x7f5a941692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94157f60 con 0x7f5a9416d050 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940c5650 con 0x7f5a94154e40 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940c5930 con 0x7f5a800051c0 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 0x7f5a941692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 0x7f5a941692e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60206/0 (socket says 172.21.15.64:60206) 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 -- 172.21.15.64:0/1165943078 learned_addr learned my addr 172.21.15.64:0/1165943078 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99276700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a800051c0 0x7f5a94157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a98a75700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a94154e40 0x7f5a9416cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 -- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a800051c0 msgr2=0x7f5a94157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a800051c0 0x7f5a94157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 -- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a94154e40 msgr2=0x7f5a9416cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a94154e40 0x7f5a9416cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 -- 172.21.15.64:0/1165943078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940c5c10 con 0x7f5a9416d050 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a98a75700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a94154e40 0x7f5a9416cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a99a77700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 0x7f5a941692e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c008340 tx=0x7f5a8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00acc0 con 0x7f5a9416d050 2023-09-27T23:33:43.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c020690 con 0x7f5a9416d050 2023-09-27T23:33:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a940c5ef0 con 0x7f5a9416d050 2023-09-27T23:33:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c020a50 con 0x7f5a9416d050 2023-09-27T23:33:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a94169820 con 0x7f5a9416d050 2023-09-27T23:33:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a940c60e0 con 0x7f5a9416d050 2023-09-27T23:33:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.214+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 63) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f5a8c01c040 con 0x7f5a9416d050 2023-09-27T23:33:43.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.218+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f5a8c00ae40 con 0x7f5a9416d050 2023-09-27T23:33:43.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.222+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5a8c04f2b0 con 0x7f5a9416d050 2023-09-27T23:33:43.293 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:33:43.290+0000 7faf43c7c000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:33:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.382+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a9404be40 con 0x7f5a9416d050 2023-09-27T23:33:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.382+0000 7f5a897fa700 1 -- 172.21.15.64:0/1165943078 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+18023 (secure 0 0 0) 0x7f5a8c018020 con 0x7f5a9416d050 2023-09-27T23:33:43.387 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:43.388 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":145,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:43.037251+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:18.367224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:18.752273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2086885373":"2023-09-28T23:33:43.037172+0000","172.21.15.123:0/2348296862":"2023-09-28T23:33:43.037172+0000","172.21.15.123:0/3951983725":"2023-09-28T23:33:43.037172+0000","172.21.15.123:0/1513523299":"2023-09-28T23:33:43.037172+0000","172.21.15.123:0/1281898802":"2023-09-28T23:33:43.037172+0000","172.21.15.123:0/2153925401":"2023-09-28T23:33:43.037172+0000","172.21.15.123:6833/33228":"2023-09-28T23:33:36.654780+0000","172.21.15.123:6832/33228":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/1464378569":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3916275130":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/846007340":"2023-09-28T23:33:43.037172+0000","172.21.15.123:0/3298374072":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/2311414465":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/588895895":"2023-09-28T23:33:36.654780+0000","172.21.15.123:0/3890296918":"2023-09-28T23:33:36.654780+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 msgr2=0x7f5a941692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 0x7f5a941692e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c008340 tx=0x7f5a8c017000 comp rx=0 tx=0).stop 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 shutdown_connections 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a9416d050 0x7f5a941692e0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a94154e40 0x7f5a9416cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 --2- 172.21.15.64:0/1165943078 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a800051c0 0x7f5a94157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 >> 172.21.15.64:0/1165943078 conn(0x7f5a94067bb0 msgr2=0x7f5a940c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 shutdown_connections 2023-09-27T23:33:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.386+0000 7f5a9a278700 1 -- 172.21.15.64:0/1165943078 wait complete. 2023-09-27T23:33:43.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:33:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 Processor -- start 2023-09-27T23:33:43.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 -- start start 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f920806a580 0x7f920806a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f920806d210 0x7f920806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920806c7c0 con 0x7f92080b1410 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920806cd80 con 0x7f920806d210 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f920cdc8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920806af10 con 0x7f920806a580 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f9207fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f920806d210 0x7f920806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f9207fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f920806d210 0x7f920806d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60214/0 (socket says 172.21.15.64:60214) 2023-09-27T23:33:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f9207fff700 1 -- 172.21.15.64:0/118456002 learned_addr learned my addr 172.21.15.64:0/118456002 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.510+0000 7f91fffff700 1 --2- 172.21.15.64:0/118456002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f920806a580 0x7f920806a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9207fff700 1 -- 172.21.15.64:0/118456002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f920806a580 msgr2=0x7f920806a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9207fff700 1 --2- 172.21.15.64:0/118456002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f920806a580 0x7f920806a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9207fff700 1 -- 172.21.15.64:0/118456002 >> v1:172.21.15.123:6789/0 conn(0x7f92080b1410 legacy=0x7f92080ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9207fff700 1 -- 172.21.15.64:0/118456002 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920806b8f0 con 0x7f920806d210 2023-09-27T23:33:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9207fff700 1 --2- 172.21.15.64:0/118456002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806d210 0x7f920806d650 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f20 tx=0x7f9200009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0e0f7ae3ffcb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9206ffd700 1 -- 172.21.15.64:0/118456002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000e070 con 0x7f920806d210 2023-09-27T23:33:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9206ffd700 1 -- 172.21.15.64:0/118456002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920000b9d0 con 0x7f920806d210 2023-09-27T23:33:43.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f9206ffd700 1 -- 172.21.15.64:0/118456002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000092e0 con 0x7f920806d210 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 -- 172.21.15.64:0/118456002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806d210 msgr2=0x7f920806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 --2- 172.21.15.64:0/118456002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806d210 0x7f920806d650 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f20 tx=0x7f9200009840 comp rx=0 tx=0).stop 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 -- 172.21.15.64:0/118456002 shutdown_connections 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 --2- 172.21.15.64:0/118456002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806d210 0x7f920806d650 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 --2- 172.21.15.64:0/118456002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f920806a580 0x7f920806a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 -- 172.21.15.64:0/118456002 >> 172.21.15.64:0/118456002 conn(0x7f9208067bb0 msgr2=0x7f92080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 -- 172.21.15.64:0/118456002 shutdown_connections 2023-09-27T23:33:43.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.514+0000 7f920cdc8700 1 -- 172.21.15.64:0/118456002 wait complete. 2023-09-27T23:33:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 Processor -- start 2023-09-27T23:33:43.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 -- start start 2023-09-27T23:33:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 0x7f9208159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f920806d210 0x7f92080c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92080b1410 0x7f92080ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:43.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080c3770 con 0x7f920806a580 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920806e120 con 0x7f920806d210 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92080c2e80 con 0x7f92080b1410 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 0x7f9208159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 0x7f9208159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60230/0 (socket says 172.21.15.64:60230) 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 -- 172.21.15.64:0/1576846378 learned_addr learned my addr 172.21.15.64:0/1576846378 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f91fffff700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f920806d210 0x7f92080c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f9207fff700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92080b1410 0x7f92080ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 -- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92080b1410 msgr2=0x7f92080ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92080b1410 0x7f92080ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 -- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f920806d210 msgr2=0x7f92080c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f920806d210 0x7f92080c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:43.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 -- 172.21.15.64:0/1576846378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200009ae0 con 0x7f920806a580 2023-09-27T23:33:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f92077fe700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 0x7f9208159480 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c590 tx=0x7f91f8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8011590 con 0x7f920806a580 2023-09-27T23:33:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f8011730 con 0x7f920806a580 2023-09-27T23:33:43.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92080c3120 con 0x7f920806a580 2023-09-27T23:33:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8011af0 con 0x7f920806a580 2023-09-27T23:33:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.518+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9208154c20 con 0x7f920806a580 2023-09-27T23:33:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.522+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920804be40 con 0x7f920806a580 2023-09-27T23:33:43.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.522+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 63) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f91f8012420 con 0x7f920806a580 2023-09-27T23:33:43.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.522+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f91f807bf20 con 0x7f920806a580 2023-09-27T23:33:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.530+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f91f8049bb0 con 0x7f920806a580 2023-09-27T23:33:43.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:43.726+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f92080cc340 con 0x7f920806a580 2023-09-27T23:33:44.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.054+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 64) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f91f8055080 con 0x7f920806a580 2023-09-27T23:33:44.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.054+0000 7f91ff7fe700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91e8066d70 0x7f91e8069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.062+0000 7f91fffff700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91e8066d70 0x7f91e8069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.066+0000 7f91fffff700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91e8066d70 0x7f91e8069150 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9208156020 tx=0x7f91f00091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:44.705 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:33:44.702+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:33:44.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.714+0000 7f91ff7fe700 1 -- 172.21.15.64:0/1576846378 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f91f806a1e0 con 0x7f920806a580 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91e8066d70 msgr2=0x7f91e8069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91e8066d70 0x7f91e8069150 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9208156020 tx=0x7f91f00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 msgr2=0x7f9208159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 0x7f9208159480 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c590 tx=0x7f91f8007f30 comp rx=0 tx=0).stop 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 shutdown_connections 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f91e8066d70 0x7f91e8069150 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92080b1410 0x7f92080ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f920806d210 0x7f92080c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 --2- 172.21.15.64:0/1576846378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920806a580 0x7f9208159480 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 >> 172.21.15.64:0/1576846378 conn(0x7f9208067bb0 msgr2=0x7f92080aa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 shutdown_connections 2023-09-27T23:33:44.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.718+0000 7f920cdc8700 1 -- 172.21.15.64:0/1576846378 wait complete. 2023-09-27T23:33:44.725 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:33:44.725 INFO:tasks.ceph.mds.c.smithi064.stderr: -2800> 2023-09-27T23:33:25.707+0000 7ff44206f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:33:44.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:33:44.754 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:33:44.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.846+0000 7f8fae0b7700 1 Processor -- start 2023-09-27T23:33:44.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.846+0000 7f8fae0b7700 1 -- start start 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fa8158e20 0x7f8fa80ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa80cb970 0x7f8fa80ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8fa80ca2c0 0x7f8fa806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa80cbdf0 con 0x7f8fa8158e20 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa80cc0d0 con 0x7f8fa80ca2c0 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80cc3b0 con 0x7f8fa80cb970 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fa8158e20 0x7f8fa80ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fa8158e20 0x7f8fa80ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54142/0 (socket says 172.21.15.64:54142) 2023-09-27T23:33:44.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 -- 172.21.15.64:0/729034043 learned_addr learned my addr 172.21.15.64:0/729034043 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:44.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa7fff700 1 --2- 172.21.15.64:0/729034043 >> v2:172.21.15.123:3300/0 conn(0x7f8fa80ca2c0 0x7f8fa806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa6ffd700 1 --2- 172.21.15.64:0/729034043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa80cb970 0x7f8fa80ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 -- 172.21.15.64:0/729034043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa80cb970 msgr2=0x7f8fa80ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 --2- 172.21.15.64:0/729034043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa80cb970 0x7f8fa80ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 -- 172.21.15.64:0/729034043 >> v2:172.21.15.123:3300/0 conn(0x7f8fa80ca2c0 msgr2=0x7f8fa806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 --2- 172.21.15.64:0/729034043 >> v2:172.21.15.123:3300/0 conn(0x7f8fa80ca2c0 0x7f8fa806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 -- 172.21.15.64:0/729034043 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa80cc690 con 0x7f8fa8158e20 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa77fe700 1 --2- 172.21.15.64:0/729034043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80ce5c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80cfd10 tx=0x7f8f98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad614ac6e77f532 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa67fc700 1 -- 172.21.15.64:0/729034043 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800b860 con 0x7f8fa8158e20 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa67fc700 1 -- 172.21.15.64:0/729034043 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9800b9c0 con 0x7f8fa8158e20 2023-09-27T23:33:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fa67fc700 1 -- 172.21.15.64:0/729034043 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800bd60 con 0x7f8fa8158e20 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- 172.21.15.64:0/729034043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 msgr2=0x7f8fa80ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/729034043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80ce5c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80cfd10 tx=0x7f8f98009250 comp rx=0 tx=0).stop 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- 172.21.15.64:0/729034043 shutdown_connections 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/729034043 >> v2:172.21.15.123:3300/0 conn(0x7f8fa80ca2c0 0x7f8fa806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/729034043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa80cb970 0x7f8fa80ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/729034043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80ce5c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- 172.21.15.64:0/729034043 >> 172.21.15.64:0/729034043 conn(0x7f8fa8067bb0 msgr2=0x7f8fa80d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- 172.21.15.64:0/729034043 shutdown_connections 2023-09-27T23:33:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.850+0000 7f8fae0b7700 1 -- 172.21.15.64:0/729034043 wait complete. 2023-09-27T23:33:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 Processor -- start 2023-09-27T23:33:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 -- start start 2023-09-27T23:33:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fa80ca2c0 0x7f8fa80b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa814eb40 0x7f8fa814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80b5f00 con 0x7f8fa8158e20 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80b43a0 con 0x7f8fa80ca2c0 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80b4e60 con 0x7f8fa814eb40 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54144/0 (socket says 172.21.15.64:54144) 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 -- 172.21.15.64:0/373954426 learned_addr learned my addr 172.21.15.64:0/373954426 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa77fe700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fa80ca2c0 0x7f8fa80b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa7fff700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa814eb40 0x7f8fa814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 -- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa814eb40 msgr2=0x7f8fa814dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa814eb40 0x7f8fa814dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 -- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fa80ca2c0 msgr2=0x7f8fa80b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fa80ca2c0 0x7f8fa80b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 -- 172.21.15.64:0/373954426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980094f0 con 0x7f8fa8158e20 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fa6ffd700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80b3e60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000d8c0 tx=0x7f8f90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f900125f0 con 0x7f8fa8158e20 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90012750 con 0x7f8fa8158e20 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa80b5140 con 0x7f8fa8158e20 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90012af0 con 0x7f8fa8158e20 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.854+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa80b5700 con 0x7f8fa8158e20 2023-09-27T23:33:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.858+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa8071120 con 0x7f8fa8158e20 2023-09-27T23:33:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.858+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 64) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f8f9001e460 con 0x7f8fa8158e20 2023-09-27T23:33:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.858+0000 7f8f9f7fe700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f880607a0 0x7f8f88062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.858+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 7109+0+0 (secure 0 0 0) 0x7f8f9007d4b0 con 0x7f8fa8158e20 2023-09-27T23:33:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.858+0000 7f8fa77fe700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f880607a0 0x7f8f88062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.862+0000 7f8fa77fe700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f880607a0 0x7f8f88062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80cfd10 tx=0x7f8f98017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:44.866+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8f9004d5c0 con 0x7f8fa8158e20 2023-09-27T23:33:45.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.058+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8fa814fe60 con 0x7f8fa8158e20 2023-09-27T23:33:45.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.718+0000 7f8f9f7fe700 1 -- 172.21.15.64:0/373954426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f90050be0 con 0x7f8fa8158e20 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f880607a0 msgr2=0x7f8f88062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f880607a0 0x7f8f88062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80cfd10 tx=0x7f8f98017000 comp rx=0 tx=0).stop 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 msgr2=0x7f8fa80b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80b3e60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000d8c0 tx=0x7f8f90008040 comp rx=0 tx=0).stop 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 shutdown_connections 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8f880607a0 0x7f8f88062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fa814eb40 0x7f8fa814dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8158e20 0x7f8fa80b3e60 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 --2- 172.21.15.64:0/373954426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fa80ca2c0 0x7f8fa80b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 >> 172.21.15.64:0/373954426 conn(0x7f8fa8067bb0 msgr2=0x7f8fa80ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.722+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 shutdown_connections 2023-09-27T23:33:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.726+0000 7f8fae0b7700 1 -- 172.21.15.64:0/373954426 wait complete. 2023-09-27T23:33:45.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:33:45.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.910+0000 7f4cfccc1700 1 Processor -- start 2023-09-27T23:33:45.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.910+0000 7f4cfccc1700 1 -- start start 2023-09-27T23:33:45.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:45.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4cf8068a10 0x7f4cf80c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:45.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf806d4d0 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf8153870 con 0x7f4cf8068a10 2023-09-27T23:33:45.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8159630 con 0x7f4cf815a0a0 2023-09-27T23:33:45.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4cf8068a10 0x7f4cf80c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:45.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4cf8068a10 0x7f4cf80c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54150/0 (socket says 172.21.15.64:54150) 2023-09-27T23:33:45.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 -- 172.21.15.64:0/3557906677 learned_addr learned my addr 172.21.15.64:0/3557906677 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:45.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf77fe700 1 --2- 172.21.15.64:0/3557906677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:45.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 -- 172.21.15.64:0/3557906677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 msgr2=0x7f4cf815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:45.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 --2- 172.21.15.64:0/3557906677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf67fc700 1 -- 172.21.15.64:0/3557906677 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645909458 0 0) 0x7f4cf806d4d0 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 -- 172.21.15.64:0/3557906677 >> v1:172.21.15.123:6789/0 conn(0x7f4cf80c6cf0 legacy=0x7f4cf8068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:45.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 -- 172.21.15.64:0/3557906677 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80c7860 con 0x7f4cf8068a10 2023-09-27T23:33:45.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf7fff700 1 --2- 172.21.15.64:0/3557906677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf8068a10 0x7f4cf80c7160 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009f20 tx=0x7f4cf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26b16541f64d89b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf67fc700 1 -- 172.21.15.64:0/3557906677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000e070 con 0x7f4cf8068a10 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf67fc700 1 -- 172.21.15.64:0/3557906677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf000b9d0 con 0x7f4cf8068a10 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cf67fc700 1 -- 172.21.15.64:0/3557906677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00092e0 con 0x7f4cf8068a10 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- 172.21.15.64:0/3557906677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf8068a10 msgr2=0x7f4cf80c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/3557906677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf8068a10 0x7f4cf80c7160 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009f20 tx=0x7f4cf0009840 comp rx=0 tx=0).stop 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- 172.21.15.64:0/3557906677 shutdown_connections 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/3557906677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf8068a10 0x7f4cf80c7160 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/3557906677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- 172.21.15.64:0/3557906677 >> 172.21.15.64:0/3557906677 conn(0x7f4cf80c8c50 msgr2=0x7f4cf80c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- 172.21.15.64:0/3557906677 shutdown_connections 2023-09-27T23:33:45.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.914+0000 7f4cfccc1700 1 -- 172.21.15.64:0/3557906677 wait complete. 2023-09-27T23:33:45.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 Processor -- start 2023-09-27T23:33:45.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 -- start start 2023-09-27T23:33:45.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf8068a10 0x7f4cf80b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:45.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 0x7f4cf8149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:45.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf80af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:45.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80cbf70 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf814e0a0 con 0x7f4cf8068a10 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80b1690 con 0x7f4cf815a0a0 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 0x7f4cf8149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 0x7f4cf8149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54152/0 (socket says 172.21.15.64:54152) 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 -- 172.21.15.64:0/2130354740 learned_addr learned my addr 172.21.15.64:0/2130354740 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf7fff700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf80af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:45.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf77fe700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf8068a10 0x7f4cf80b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:45.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 -- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 msgr2=0x7f4cf80af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:45.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf80af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 -- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf8068a10 msgr2=0x7f4cf80b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:45.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf8068a10 0x7f4cf80b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:45.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 -- 172.21.15.64:0/2130354740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf0009ae0 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cf6ffd700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 0x7f4cf8149e60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c810 tx=0x7f4ce0007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011590 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0011710 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf80abc50 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011ad0 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.918+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf80b18e0 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.922+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf804be40 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.922+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 64) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f4ce001b3b0 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.922+0000 7f4cef7fe700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4cdc0607d0 0x7f4cdc062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:45.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.922+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 7109+0+0 (secure 0 0 0) 0x7f4ce007c460 con 0x7f4cf80c6cf0 2023-09-27T23:33:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.922+0000 7f4cf77fe700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4cdc0607d0 0x7f4cdc062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:45.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.926+0000 7f4cf77fe700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4cdc0607d0 0x7f4cdc062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009db0 tx=0x7f4ce80091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:45.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:45.930+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4ce004c570 con 0x7f4cf80c6cf0 2023-09-27T23:33:46.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:46.066+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4ce004c940 con 0x7f4cf80c6cf0 2023-09-27T23:33:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:46.098+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4cf80aba30 con 0x7f4cf80c6cf0 2023-09-27T23:33:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.074+0000 7f4cef7fe700 1 -- 172.21.15.64:0/2130354740 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4ce004fd70 con 0x7f4cf80c6cf0 2023-09-27T23:33:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4cdc0607d0 msgr2=0x7f4cdc062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4cdc0607d0 0x7f4cdc062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009db0 tx=0x7f4ce80091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 msgr2=0x7f4cf8149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 0x7f4cf8149e60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c810 tx=0x7f4ce0007da0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 shutdown_connections 2023-09-27T23:33:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f4cdc0607d0 0x7f4cdc062c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4cf815a0a0 0x7f4cf80af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cf80c6cf0 0x7f4cf8149e60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 --2- 172.21.15.64:0/2130354740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf8068a10 0x7f4cf80b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 >> 172.21.15.64:0/2130354740 conn(0x7f4cf80c8c50 msgr2=0x7f4cf80c1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 shutdown_connections 2023-09-27T23:33:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.082+0000 7f4cfccc1700 1 -- 172.21.15.64:0/2130354740 wait complete. 2023-09-27T23:33:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:33:47.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:33:47.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.202+0000 7f057727d700 1 Processor -- start 2023-09-27T23:33:47.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.202+0000 7f057727d700 1 -- start start 2023-09-27T23:33:47.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057727d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f05701583f0 0x7f057006b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057727d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0570158f00 con 0x7f05701587c0 2023-09-27T23:33:47.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057727d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05701591e0 con 0x7f057006ac90 2023-09-27T23:33:47.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057727d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05701594c0 con 0x7f05701583f0 2023-09-27T23:33:47.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575a7a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05701587c0 0x7f057006a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37168/0 (socket says 172.21.15.64:37168) 2023-09-27T23:33:47.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575a7a700 1 -- 172.21.15.64:0/2686500472 learned_addr learned my addr 172.21.15.64:0/2686500472 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 --2- 172.21.15.64:0/2686500472 >> v2:172.21.15.123:3300/0 conn(0x7f05701583f0 0x7f057006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541455033 0 0) 0x7f0570158f00 con 0x7f05701587c0 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f055c003570 con 0x7f05701587c0 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558010314 0 0) 0x7f05701591e0 con 0x7f057006ac90 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0570158f00 con 0x7f057006ac90 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947341238 0 0) 0x7f055c003570 con 0x7f05701587c0 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 >> v2:172.21.15.123:3300/0 conn(0x7f05701583f0 msgr2=0x7f057006b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 --2- 172.21.15.64:0/2686500472 >> v2:172.21.15.123:3300/0 conn(0x7f05701583f0 0x7f057006b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 >> v1:172.21.15.123:6789/0 conn(0x7f057006ac90 legacy=0x7f05700aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05701597a0 con 0x7f05701587c0 2023-09-27T23:33:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f0560001470 con 0x7f05701587c0 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 >> v1:172.21.15.64:6789/0 conn(0x7f05701587c0 legacy=0x7f057006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f055c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f055c004810 0x7f055c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c0051c0 0x7f055c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05701594c0 con 0x7f055c003bf0 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c00c160 con 0x7f055c0051c0 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f055c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c00c320 con 0x7f055c004810 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575a7a700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f055c004810 0x7f055c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0576a7c700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c0051c0 0x7f055c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 -- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f055c004810 msgr2=0x7f055c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f055c004810 0x7f055c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 -- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c0051c0 msgr2=0x7f055c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c0051c0 0x7f055c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0576a7c700 1 -- 172.21.15.64:0/2686500472 reap_dead start 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 -- 172.21.15.64:0/2686500472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c003570 con 0x7f055c003bf0 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f057627b700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f055c003fe0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f056c006f00 tx=0x7f056c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e6298711cadcc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00a760 con 0x7f055c003bf0 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c00fa30 con 0x7f055c003bf0 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.206+0000 7f0575279700 1 -- 172.21.15.64:0/2686500472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00fbc0 con 0x7f055c003bf0 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- 172.21.15.64:0/2686500472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 msgr2=0x7f055c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f055c003fe0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f056c006f00 tx=0x7f056c00a330 comp rx=0 tx=0).stop 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- 172.21.15.64:0/2686500472 shutdown_connections 2023-09-27T23:33:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 --2- 172.21.15.64:0/2686500472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f055c003fe0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- 172.21.15.64:0/2686500472 >> 172.21.15.64:0/2686500472 conn(0x7f0570067bb0 msgr2=0x7f05700c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:47.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- 172.21.15.64:0/2686500472 shutdown_connections 2023-09-27T23:33:47.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- 172.21.15.64:0/2686500472 wait complete. 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 Processor -- start 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- start start 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f057015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f057015a490 0x7f057013e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0570152fc0 0x7f05701533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057015d910 con 0x7f055c003bf0 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057014cb80 con 0x7f057015a490 2023-09-27T23:33:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057727d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0570153930 con 0x7f0570152fc0 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.210+0000 7f057627b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f057015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f057015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54168/0 (socket says 172.21.15.64:54168) 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 -- 172.21.15.64:0/3030121554 learned_addr learned my addr 172.21.15.64:0/3030121554 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f0575a7a700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f057015a490 0x7f057013e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f0576a7c700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0570152fc0 0x7f05701533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 -- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0570152fc0 msgr2=0x7f05701533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0570152fc0 0x7f05701533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 -- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f057015a490 msgr2=0x7f057013e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f057015a490 0x7f057013e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 -- 172.21.15.64:0/3030121554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057015d070 con 0x7f055c003bf0 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f0575a7a700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f057015a490 0x7f057013e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:47.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057627b700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f057015c710 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c130 tx=0x7f056c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c012460 con 0x7f055c003bf0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c0125c0 con 0x7f055c003bf0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057015d350 con 0x7f055c003bf0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c0129a0 con 0x7f055c003bf0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057015cc50 con 0x7f055c003bf0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.214+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057004be40 con 0x7f055c003bf0 2023-09-27T23:33:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.218+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f056c01a3e0 con 0x7f055c003bf0 2023-09-27T23:33:47.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.218+0000 7f05667fc700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0554060a70 0x7f0554062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:47.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.218+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6671+0+0 (secure 0 0 0) 0x7f056c016070 con 0x7f055c003bf0 2023-09-27T23:33:47.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.218+0000 7f0575a7a700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0554060a70 0x7f0554062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:47.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.218+0000 7f0575a7a700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0554060a70 0x7f0554062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0560001200 tx=0x7f0560009040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.226+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f056c04ea50 con 0x7f055c003bf0 2023-09-27T23:33:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:47.386+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0570159a80 con 0x7f055c003bf0 2023-09-27T23:33:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.074+0000 7f05667fc700 1 -- 172.21.15.64:0/3030121554 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f056c029020 con 0x7f055c003bf0 2023-09-27T23:33:48.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.082+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0554060a70 msgr2=0x7f0554062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.082+0000 7f057727d700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0554060a70 0x7f0554062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0560001200 tx=0x7f0560009040 comp rx=0 tx=0).stop 2023-09-27T23:33:48.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.082+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 msgr2=0x7f057015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.082+0000 7f057727d700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f057015c710 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c130 tx=0x7f056c01d000 comp rx=0 tx=0).stop 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 shutdown_connections 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0554060a70 0x7f0554062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0570152fc0 0x7f05701533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f057015a490 0x7f057013e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 --2- 172.21.15.64:0/3030121554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f055c003bf0 0x7f057015c710 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 >> 172.21.15.64:0/3030121554 conn(0x7f0570067bb0 msgr2=0x7f05701554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 shutdown_connections 2023-09-27T23:33:48.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.086+0000 7f057727d700 1 -- 172.21.15.64:0/3030121554 wait complete. 2023-09-27T23:33:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:33:48.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:33:48.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.210+0000 7f9466d22700 1 Processor -- start 2023-09-27T23:33:48.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.210+0000 7f9466d22700 1 -- start start 2023-09-27T23:33:48.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466d22700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f946006ae20 0x7f946006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466d22700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946006b730 con 0x7f94600b1440 2023-09-27T23:33:48.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466d22700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600ac390 con 0x7f94600ab6d0 2023-09-27T23:33:48.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466d22700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600ac670 con 0x7f946006ae20 2023-09-27T23:33:48.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f94600b1440 0x7f94600b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37184/0 (socket says 172.21.15.64:37184) 2023-09-27T23:33:48.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 -- 172.21.15.64:0/1353522828 learned_addr learned my addr 172.21.15.64:0/1353522828 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:48.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9465d20700 1 --2- 172.21.15.64:0/1353522828 >> v2:172.21.15.123:3300/0 conn(0x7f946006ae20 0x7f946006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175987089 0 0) 0x7f946006b730 con 0x7f94600b1440 2023-09-27T23:33:48.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c003570 con 0x7f94600b1440 2023-09-27T23:33:48.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668877934 0 0) 0x7f944c003570 con 0x7f94600b1440 2023-09-27T23:33:48.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 >> v2:172.21.15.123:3300/0 conn(0x7f946006ae20 msgr2=0x7f946006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:48.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 --2- 172.21.15.64:0/1353522828 >> v2:172.21.15.123:3300/0 conn(0x7f946006ae20 0x7f946006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 >> v1:172.21.15.123:6789/0 conn(0x7f94600ab6d0 legacy=0x7f94600abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600ac950 con 0x7f94600b1440 2023-09-27T23:33:48.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f94580017f0 con 0x7f94600b1440 2023-09-27T23:33:48.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 >> v1:172.21.15.64:6789/0 conn(0x7f94600b1440 legacy=0x7f94600b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f944c004810 0x7f944c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f944c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600ac670 con 0x7f944c0051c0 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c00c160 con 0x7f944c004810 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c00c320 con 0x7f944c003a60 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f944c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9465d20700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f946551f700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f944c004810 0x7f944c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 -- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f944c003a60 msgr2=0x7f944c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 -- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f944c004810 msgr2=0x7f944c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f944c004810 0x7f944c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 -- 172.21.15.64:0/1353522828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c003570 con 0x7f944c0051c0 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 -- 172.21.15.64:0/1353522828 reap_dead start 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9466521700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f944c00ba60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f945800a930 tx=0x7f94580021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbf9868da633c46 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580070d0 con 0x7f944c0051c0 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458007230 con 0x7f944c0051c0 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.214+0000 7f9464d1e700 1 -- 172.21.15.64:0/1353522828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580075d0 con 0x7f944c0051c0 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 -- 172.21.15.64:0/1353522828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 msgr2=0x7f944c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f944c00ba60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f945800a930 tx=0x7f94580021d0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 -- 172.21.15.64:0/1353522828 shutdown_connections 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 --2- 172.21.15.64:0/1353522828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f944c00ba60 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 -- 172.21.15.64:0/1353522828 >> 172.21.15.64:0/1353522828 conn(0x7f94600c8cd0 msgr2=0x7f94600c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:48.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 -- 172.21.15.64:0/1353522828 shutdown_connections 2023-09-27T23:33:48.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 -- 172.21.15.64:0/1353522828 wait complete. 2023-09-27T23:33:48.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 Processor -- start 2023-09-27T23:33:48.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.218+0000 7f9466d22700 1 -- start start 2023-09-27T23:33:48.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f94600c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600c5ad0 0x7f946016a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946016aa00 0x7f946015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600c42e0 con 0x7f944c0051c0 2023-09-27T23:33:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946016ae60 con 0x7f946016aa00 2023-09-27T23:33:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600c5100 con 0x7f94600c5ad0 2023-09-27T23:33:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f94600c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f94600c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54176/0 (socket says 172.21.15.64:54176) 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 -- 172.21.15.64:0/2084772902 learned_addr learned my addr 172.21.15.64:0/2084772902 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f946551f700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600c5ad0 0x7f946016a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466521700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946016aa00 0x7f946015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 -- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600c5ad0 msgr2=0x7f946016a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600c5ad0 0x7f946016a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 -- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946016aa00 msgr2=0x7f946015a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946016aa00 0x7f946015a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 -- 172.21.15.64:0/2084772902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600c44f0 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9465d20700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f94600c5590 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f945c006ac0 tx=0x7f945c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c011ec0 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00f8e0 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600c47d0 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00fc60 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94600c4d90 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.222+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94600ad040 con 0x7f944c0051c0 2023-09-27T23:33:48.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.226+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f945c014650 con 0x7f944c0051c0 2023-09-27T23:33:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.226+0000 7f9455ffb700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9444060a70 0x7f9444062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.226+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f945c018030 con 0x7f944c0051c0 2023-09-27T23:33:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.226+0000 7f946551f700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9444060a70 0x7f9444062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.226+0000 7f946551f700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9444060a70 0x7f9444062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9460157120 tx=0x7f9450007d30 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.234+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f94600ad040 con 0x7f944c0051c0 2023-09-27T23:33:48.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.394+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f94600ad040 con 0x7f944c0051c0 2023-09-27T23:33:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.402+0000 7f9455ffb700 1 -- 172.21.15.64:0/2084772902 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v49) v1 ==== 115+0+0 (secure 0 0 0) 0x7f94600ad040 con 0x7f944c0051c0 2023-09-27T23:33:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9444060a70 msgr2=0x7f9444062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9444060a70 0x7f9444062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9460157120 tx=0x7f9450007d30 comp rx=0 tx=0).stop 2023-09-27T23:33:48.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 msgr2=0x7f94600c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f94600c5590 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f945c006ac0 tx=0x7f945c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 shutdown_connections 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9444060a70 0x7f9444062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946016aa00 0x7f946015a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600c5ad0 0x7f946016a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 --2- 172.21.15.64:0/2084772902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f944c0051c0 0x7f94600c5590 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 >> 172.21.15.64:0/2084772902 conn(0x7f94600c8cd0 msgr2=0x7f946006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:48.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 shutdown_connections 2023-09-27T23:33:48.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.406+0000 7f9466d22700 1 -- 172.21.15.64:0/2084772902 wait complete. 2023-09-27T23:33:48.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:33:48.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda77fff700 1 Processor -- start 2023-09-27T23:33:48.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda77fff700 1 -- start start 2023-09-27T23:33:48.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda77fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda78155a90 0x7fda780ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda77fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda780c68a0 con 0x7fda78155e60 2023-09-27T23:33:48.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda77fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda780c6e60 con 0x7fda7806db70 2023-09-27T23:33:48.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda77fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda780ae4d0 con 0x7fda78155a90 2023-09-27T23:33:48.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.534+0000 7fda76ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda78155a90 0x7fda780ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda777fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fda78155e60 0x7fda780c72f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37194/0 (socket says 172.21.15.64:37194) 2023-09-27T23:33:48.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda777fe700 1 -- 172.21.15.64:0/2641687118 learned_addr learned my addr 172.21.15.64:0/2641687118 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda76ffd700 1 -- 172.21.15.64:0/2641687118 >> v1:172.21.15.123:6789/0 conn(0x7fda7806db70 legacy=0x7fda780ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda76ffd700 1 -- 172.21.15.64:0/2641687118 >> v1:172.21.15.64:6789/0 conn(0x7fda78155e60 legacy=0x7fda780c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda76ffd700 1 -- 172.21.15.64:0/2641687118 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780ae7b0 con 0x7fda78155a90 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda76ffd700 1 --2- 172.21.15.64:0/2641687118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda78155a90 0x7fda780ad220 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fda68009e60 tx=0x7fda68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d45bed186e9dc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda75ffb700 1 -- 172.21.15.64:0/2641687118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6800e070 con 0x7fda78155a90 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda75ffb700 1 -- 172.21.15.64:0/2641687118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6800bc30 con 0x7fda78155a90 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda75ffb700 1 -- 172.21.15.64:0/2641687118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68012430 con 0x7fda78155a90 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 -- 172.21.15.64:0/2641687118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda78155a90 msgr2=0x7fda780ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 --2- 172.21.15.64:0/2641687118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda78155a90 0x7fda780ad220 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fda68009e60 tx=0x7fda68009780 comp rx=0 tx=0).stop 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 -- 172.21.15.64:0/2641687118 shutdown_connections 2023-09-27T23:33:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 --2- 172.21.15.64:0/2641687118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda78155a90 0x7fda780ad220 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 -- 172.21.15.64:0/2641687118 >> 172.21.15.64:0/2641687118 conn(0x7fda780c8cd0 msgr2=0x7fda780c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:48.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 -- 172.21.15.64:0/2641687118 shutdown_connections 2023-09-27T23:33:48.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.538+0000 7fda77fff700 1 -- 172.21.15.64:0/2641687118 wait complete. 2023-09-27T23:33:48.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 Processor -- start 2023-09-27T23:33:48.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 -- start start 2023-09-27T23:33:48.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 0x7fda780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda78155a90 0x7fda781539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda78155e60 0x7fda78157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda781663e0 con 0x7fda7806db70 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda781666c0 con 0x7fda78155a90 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda781669a0 con 0x7fda78155e60 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 0x7fda780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 0x7fda780a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54204/0 (socket says 172.21.15.64:54204) 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 -- 172.21.15.64:0/909777402 learned_addr learned my addr 172.21.15.64:0/909777402 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:48.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda767fc700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda78155a90 0x7fda781539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda777fe700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda78155e60 0x7fda78157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 -- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda78155e60 msgr2=0x7fda78157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda78155e60 0x7fda78157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 -- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda78155a90 msgr2=0x7fda781539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda78155a90 0x7fda781539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 -- 172.21.15.64:0/909777402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68009a20 con 0x7fda7806db70 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda767fc700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda78155a90 0x7fda781539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda76ffd700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 0x7fda780a5170 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fda680097c0 tx=0x7fda68019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6801d630 con 0x7fda7806db70 2023-09-27T23:33:48.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6801d7b0 con 0x7fda7806db70 2023-09-27T23:33:48.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda78166c80 con 0x7fda7806db70 2023-09-27T23:33:48.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6801db70 con 0x7fda7806db70 2023-09-27T23:33:48.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.542+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda78166090 con 0x7fda7806db70 2023-09-27T23:33:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.546+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7804be40 con 0x7fda7806db70 2023-09-27T23:33:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.546+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fda68024070 con 0x7fda7806db70 2023-09-27T23:33:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.546+0000 7fda6effd700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fda58060a70 0x7fda58062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.546+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fda6807f9b0 con 0x7fda7806db70 2023-09-27T23:33:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.546+0000 7fda767fc700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fda58060a70 0x7fda58062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.546+0000 7fda767fc700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fda58060a70 0x7fda58062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda60007fd0 tx=0x7fda60009040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.554+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fda6804f9b0 con 0x7fda7806db70 2023-09-27T23:33:48.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.714+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fda78166520 con 0x7fda7806db70 2023-09-27T23:33:48.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.722+0000 7fda6effd700 1 -- 172.21.15.64:0/909777402 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v50) v1 ==== 120+0+0 (secure 0 0 0) 0x7fda78166520 con 0x7fda7806db70 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fda58060a70 msgr2=0x7fda58062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fda58060a70 0x7fda58062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda60007fd0 tx=0x7fda60009040 comp rx=0 tx=0).stop 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 msgr2=0x7fda780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 0x7fda780a5170 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fda680097c0 tx=0x7fda68019000 comp rx=0 tx=0).stop 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 shutdown_connections 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fda58060a70 0x7fda58062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda78155e60 0x7fda78157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda78155a90 0x7fda781539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 --2- 172.21.15.64:0/909777402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7806db70 0x7fda780a5170 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 >> 172.21.15.64:0/909777402 conn(0x7fda780c8cd0 msgr2=0x7fda780b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 shutdown_connections 2023-09-27T23:33:48.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.726+0000 7fda77fff700 1 -- 172.21.15.64:0/909777402 wait complete. 2023-09-27T23:33:48.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:33:48.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f484510a700 1 Processor -- start 2023-09-27T23:33:48.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f484510a700 1 -- start start 2023-09-27T23:33:48.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f484510a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48400cc110 0x7f48401564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f484510a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4840156af0 con 0x7f484013f7e0 2023-09-27T23:33:48.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f484510a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4840156dd0 con 0x7f484013fbb0 2023-09-27T23:33:48.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f484510a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4840157950 con 0x7f48400cc110 2023-09-27T23:33:48.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.846+0000 7f483f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48400cc110 0x7f48401564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48400cc110 0x7f48401564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54210/0 (socket says 172.21.15.64:54210) 2023-09-27T23:33:48.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483f59e700 1 -- 172.21.15.64:0/184666891 learned_addr learned my addr 172.21.15.64:0/184666891 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:48.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483ed9d700 1 --1- 172.21.15.64:0/184666891 >> v1:172.21.15.64:6789/0 conn(0x7f484013f7e0 0x7f48400af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37210/0 (socket says 172.21.15.64:37210) 2023-09-27T23:33:48.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483f59e700 1 -- 172.21.15.64:0/184666891 >> v1:172.21.15.123:6789/0 conn(0x7f484013fbb0 legacy=0x7f48400cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483dd9b700 1 -- 172.21.15.64:0/184666891 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91578745 0 0) 0x7f4840156af0 con 0x7f484013f7e0 2023-09-27T23:33:48.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483f59e700 1 -- 172.21.15.64:0/184666891 >> v1:172.21.15.64:6789/0 conn(0x7f484013f7e0 legacy=0x7f48400af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483f59e700 1 -- 172.21.15.64:0/184666891 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48401570b0 con 0x7f48400cc110 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483f59e700 1 --2- 172.21.15.64:0/184666891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48400cc110 0x7f48401564a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4838009f20 tx=0x7f4838009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0a9aa606345989 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483dd9b700 1 -- 172.21.15.64:0/184666891 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800e070 con 0x7f48400cc110 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483dd9b700 1 -- 172.21.15.64:0/184666891 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483800bb10 con 0x7f48400cc110 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f483dd9b700 1 -- 172.21.15.64:0/184666891 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380092e0 con 0x7f48400cc110 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 -- 172.21.15.64:0/184666891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48400cc110 msgr2=0x7f48401564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 --2- 172.21.15.64:0/184666891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48400cc110 0x7f48401564a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4838009f20 tx=0x7f4838009840 comp rx=0 tx=0).stop 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 -- 172.21.15.64:0/184666891 shutdown_connections 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 --2- 172.21.15.64:0/184666891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48400cc110 0x7f48401564a0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 -- 172.21.15.64:0/184666891 >> 172.21.15.64:0/184666891 conn(0x7f48400cd8f0 msgr2=0x7f48400c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 -- 172.21.15.64:0/184666891 shutdown_connections 2023-09-27T23:33:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 -- 172.21.15.64:0/184666891 wait complete. 2023-09-27T23:33:48.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.850+0000 7f484510a700 1 Processor -- start 2023-09-27T23:33:48.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 -- start start 2023-09-27T23:33:48.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48400cc110 0x7f48400b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 0x7f4840156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484013fbb0 0x7f48400692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400b2990 con 0x7f484013f7e0 2023-09-27T23:33:48.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400b0d20 con 0x7f484013fbb0 2023-09-27T23:33:48.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400b1000 con 0x7f48400cc110 2023-09-27T23:33:48.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 0x7f4840156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 0x7f4840156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54222/0 (socket says 172.21.15.64:54222) 2023-09-27T23:33:48.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 -- 172.21.15.64:0/1969682237 learned_addr learned my addr 172.21.15.64:0/1969682237 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:48.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483ed9d700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48400cc110 0x7f48400b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483f59e700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484013fbb0 0x7f48400692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 -- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48400cc110 msgr2=0x7f48400b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48400cc110 0x7f48400b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 -- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484013fbb0 msgr2=0x7f48400692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484013fbb0 0x7f48400692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 -- 172.21.15.64:0/1969682237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838009ae0 con 0x7f484013f7e0 2023-09-27T23:33:48.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483f59e700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484013fbb0 0x7f48400692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:48.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f483e59c700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 0x7f4840156190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f483400c4e0 tx=0x7f483400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834011590 con 0x7f484013f7e0 2023-09-27T23:33:48.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4834011710 con 0x7f484013f7e0 2023-09-27T23:33:48.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48400b12e0 con 0x7f484013f7e0 2023-09-27T23:33:48.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834011ad0 con 0x7f484013f7e0 2023-09-27T23:33:48.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.854+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48400b18a0 con 0x7f484013f7e0 2023-09-27T23:33:48.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.858+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840071120 con 0x7f484013f7e0 2023-09-27T23:33:48.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.858+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f483401b3b0 con 0x7f484013f7e0 2023-09-27T23:33:48.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.858+0000 7f4826ffd700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f482c060aa0 0x7f482c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.858+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f483407c4f0 con 0x7f484013f7e0 2023-09-27T23:33:48.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.858+0000 7f483ed9d700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f482c060aa0 0x7f482c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:48.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.858+0000 7f483ed9d700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f482c060aa0 0x7f482c062f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4830007fd0 tx=0x7f4830007d30 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:48.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:48.866+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f483404c590 con 0x7f484013f7e0 2023-09-27T23:33:49.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.030+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4840157390 con 0x7f484013f7e0 2023-09-27T23:33:49.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f4826ffd700 1 -- 172.21.15.64:0/1969682237 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f483404fbb0 con 0x7f484013f7e0 2023-09-27T23:33:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f482c060aa0 msgr2=0x7f482c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f482c060aa0 0x7f482c062f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4830007fd0 tx=0x7f4830007d30 comp rx=0 tx=0).stop 2023-09-27T23:33:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 msgr2=0x7f4840156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 0x7f4840156190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f483400c4e0 tx=0x7f483400c7f0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 shutdown_connections 2023-09-27T23:33:49.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f482c060aa0 0x7f482c062f50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484013fbb0 0x7f48400692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f484013f7e0 0x7f4840156190 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 --2- 172.21.15.64:0/1969682237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48400cc110 0x7f48400b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 >> 172.21.15.64:0/1969682237 conn(0x7f48400cd8f0 msgr2=0x7f4840158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 shutdown_connections 2023-09-27T23:33:49.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.038+0000 7f484510a700 1 -- 172.21.15.64:0/1969682237 wait complete. 2023-09-27T23:33:49.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-09-27T23:33:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.154+0000 7f5a5a7fe700 1 Processor -- start 2023-09-27T23:33:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.154+0000 7f5a5a7fe700 1 -- start start 2023-09-27T23:33:49.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a540c6980 0x7f5a5406a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:49.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a540ac1b0 con 0x7f5a54068670 2023-09-27T23:33:49.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5406af00 con 0x7f5a5406aac0 2023-09-27T23:33:49.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a540c6f50 con 0x7f5a540c6980 2023-09-27T23:33:49.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a597fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5a54068670 0x7f5a5406b730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37212/0 (socket says 172.21.15.64:37212) 2023-09-27T23:33:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a597fc700 1 -- 172.21.15.64:0/4272449071 learned_addr learned my addr 172.21.15.64:0/4272449071 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a58ffb700 1 --2- 172.21.15.64:0/4272449071 >> v2:172.21.15.123:3300/0 conn(0x7f5a540c6980 0x7f5a5406a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a47fff700 1 -- 172.21.15.64:0/4272449071 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629574891 0 0) 0x7f5a540ac1b0 con 0x7f5a54068670 2023-09-27T23:33:49.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a47fff700 1 -- 172.21.15.64:0/4272449071 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a40003570 con 0x7f5a54068670 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a58ffb700 1 -- 172.21.15.64:0/4272449071 >> v1:172.21.15.123:6789/0 conn(0x7f5a5406aac0 legacy=0x7f5a540ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a58ffb700 1 -- 172.21.15.64:0/4272449071 >> v1:172.21.15.64:6789/0 conn(0x7f5a54068670 legacy=0x7f5a5406b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a47fff700 1 -- 172.21.15.64:0/4272449071 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1200655558 0 0) 0x7f5a40003570 con 0x7f5a54068670 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a58ffb700 1 -- 172.21.15.64:0/4272449071 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a540c7230 con 0x7f5a540c6980 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a58ffb700 1 --2- 172.21.15.64:0/4272449071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a540c6980 0x7f5a5406a580 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5a480098e0 tx=0x7f5a480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e01e00ee238554 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a47fff700 1 -- 172.21.15.64:0/4272449071 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800b6d0 con 0x7f5a540c6980 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a47fff700 1 -- 172.21.15.64:0/4272449071 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4800b830 con 0x7f5a540c6980 2023-09-27T23:33:49.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a47fff700 1 -- 172.21.15.64:0/4272449071 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800bbd0 con 0x7f5a540c6980 2023-09-27T23:33:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/4272449071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a540c6980 msgr2=0x7f5a5406a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/4272449071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a540c6980 0x7f5a5406a580 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5a480098e0 tx=0x7f5a480091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/4272449071 shutdown_connections 2023-09-27T23:33:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/4272449071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a540c6980 0x7f5a5406a580 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/4272449071 >> 172.21.15.64:0/4272449071 conn(0x7f5a54067bb0 msgr2=0x7f5a540cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/4272449071 shutdown_connections 2023-09-27T23:33:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.158+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/4272449071 wait complete. 2023-09-27T23:33:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 Processor -- start 2023-09-27T23:33:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 -- start start 2023-09-27T23:33:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a54068670 0x7f5a54146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5406aac0 0x7f5a540b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 0x7f5a540af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a540affe0 con 0x7f5a540c6980 2023-09-27T23:33:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a54146850 con 0x7f5a54068670 2023-09-27T23:33:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a540afd00 con 0x7f5a5406aac0 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 0x7f5a540af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 0x7f5a540af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54230/0 (socket says 172.21.15.64:54230) 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 -- 172.21.15.64:0/3265776231 learned_addr learned my addr 172.21.15.64:0/3265776231 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a58ffb700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5406aac0 0x7f5a540b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a597fc700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a54068670 0x7f5a54146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 -- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5406aac0 msgr2=0x7f5a540b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5406aac0 0x7f5a540b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 -- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a54068670 msgr2=0x7f5a54146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a54068670 0x7f5a54146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 -- 172.21.15.64:0/3265776231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48009420 con 0x7f5a540c6980 2023-09-27T23:33:49.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a58ffb700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5406aac0 0x7f5a540b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a59ffd700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 0x7f5a540af4b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c910 tx=0x7f5a4c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00a7a0 con 0x7f5a540c6980 2023-09-27T23:33:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c00a900 con 0x7f5a540c6980 2023-09-27T23:33:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00acc0 con 0x7f5a540c6980 2023-09-27T23:33:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a54146200 con 0x7f5a540c6980 2023-09-27T23:33:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.162+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a540af9f0 con 0x7f5a540c6980 2023-09-27T23:33:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.166+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a541469e0 con 0x7f5a540c6980 2023-09-27T23:33:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.166+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5a4c012630 con 0x7f5a540c6980 2023-09-27T23:33:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.166+0000 7f5a457fa700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a38060a70 0x7f5a38062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.166+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f5a4c07d370 con 0x7f5a540c6980 2023-09-27T23:33:49.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.166+0000 7f5a597fc700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a38060a70 0x7f5a38062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:49.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.170+0000 7f5a597fc700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a38060a70 0x7f5a38062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a540aeab0 tx=0x7f5a50005950 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:49.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.174+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5a4c050ea0 con 0x7f5a540c6980 2023-09-27T23:33:49.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:49.394+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f5a5404be40 con 0x7f5a540c6980 2023-09-27T23:33:50.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.090+0000 7f5a457fa700 1 -- 172.21.15.64:0/3265776231 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v763) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5a4c050a30 con 0x7f5a540c6980 2023-09-27T23:33:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a38060a70 msgr2=0x7f5a38062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a38060a70 0x7f5a38062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a540aeab0 tx=0x7f5a50005950 comp rx=0 tx=0).stop 2023-09-27T23:33:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 msgr2=0x7f5a540af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 0x7f5a540af4b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c910 tx=0x7f5a4c001710 comp rx=0 tx=0).stop 2023-09-27T23:33:50.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 shutdown_connections 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5a38060a70 0x7f5a38062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a540c6980 0x7f5a540af4b0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a5406aac0 0x7f5a540b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 --2- 172.21.15.64:0/3265776231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a54068670 0x7f5a54146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 >> 172.21.15.64:0/3265776231 conn(0x7f5a54067bb0 msgr2=0x7f5a5414b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 shutdown_connections 2023-09-27T23:33:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.094+0000 7f5a5a7fe700 1 -- 172.21.15.64:0/3265776231 wait complete. 2023-09-27T23:33:50.117 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-09-27T23:33:50.117 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-09-27T23:33:50.117 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:33:50.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-09-27T23:33:50.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c91789700 1 Processor -- start 2023-09-27T23:33:50.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c91789700 1 -- start start 2023-09-27T23:33:50.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c91789700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c8c1591f0 0x7f8c8c06a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:50.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c91789700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c06b730 con 0x7f8c8c158e20 2023-09-27T23:33:50.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c91789700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c06ab20 con 0x7f8c8c0ce810 2023-09-27T23:33:50.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c91789700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c06ae00 con 0x7f8c8c1591f0 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c8c1591f0 0x7f8c8c06a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8c8c158e20 0x7f8c8c0ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37228/0 (socket says 172.21.15.64:37228) 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8bfff700 1 -- 172.21.15.64:0/778227218 learned_addr learned my addr 172.21.15.64:0/778227218 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8affd700 1 -- 172.21.15.64:0/778227218 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573997381 0 0) 0x7f8c8c06b730 con 0x7f8c8c158e20 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8affd700 1 -- 172.21.15.64:0/778227218 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c78003570 con 0x7f8c8c158e20 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8b7fe700 1 -- 172.21.15.64:0/778227218 >> v1:172.21.15.123:6789/0 conn(0x7f8c8c0ce810 legacy=0x7f8c8c0ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8b7fe700 1 -- 172.21.15.64:0/778227218 >> v1:172.21.15.64:6789/0 conn(0x7f8c8c158e20 legacy=0x7f8c8c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.222+0000 7f8c8b7fe700 1 -- 172.21.15.64:0/778227218 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c06b0e0 con 0x7f8c8c1591f0 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c8b7fe700 1 --2- 172.21.15.64:0/778227218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c1591f0 0x7f8c8c06a540 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009e60 tx=0x7f8c7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c25906ddbcb968 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c8affd700 1 -- 172.21.15.64:0/778227218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00e070 con 0x7f8c8c1591f0 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c8affd700 1 -- 172.21.15.64:0/778227218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c00bb60 con 0x7f8c8c1591f0 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c8affd700 1 -- 172.21.15.64:0/778227218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c012430 con 0x7f8c8c1591f0 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 -- 172.21.15.64:0/778227218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c1591f0 msgr2=0x7f8c8c06a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 --2- 172.21.15.64:0/778227218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c1591f0 0x7f8c8c06a540 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009e60 tx=0x7f8c7c009780 comp rx=0 tx=0).stop 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 -- 172.21.15.64:0/778227218 shutdown_connections 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 --2- 172.21.15.64:0/778227218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c1591f0 0x7f8c8c06a540 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 -- 172.21.15.64:0/778227218 >> 172.21.15.64:0/778227218 conn(0x7f8c8c067bb0 msgr2=0x7f8c8c0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 -- 172.21.15.64:0/778227218 shutdown_connections 2023-09-27T23:33:50.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 -- 172.21.15.64:0/778227218 wait complete. 2023-09-27T23:33:50.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 Processor -- start 2023-09-27T23:33:50.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.226+0000 7f8c91789700 1 -- start start 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c0ce810 0x7f8c8c15c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 0x7f8c8c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c1591f0 0x7f8c8c0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c98c0 con 0x7f8c8c158e20 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c9310 con 0x7f8c8c0ce810 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c95e0 con 0x7f8c8c1591f0 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c0ce810 0x7f8c8c15c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:50.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 0x7f8c8c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c0ce810 0x7f8c8c15c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:38710/0 (socket says 172.21.15.64:38710) 2023-09-27T23:33:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8bfff700 1 -- 172.21.15.64:0/2615447909 learned_addr learned my addr 172.21.15.64:0/2615447909 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 0x7f8c8c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54240/0 (socket says 172.21.15.64:54240) 2023-09-27T23:33:50.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c90f88700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c1591f0 0x7f8c8c0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 -- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c1591f0 msgr2=0x7f8c8c0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c1591f0 0x7f8c8c0c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 -- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c0ce810 msgr2=0x7f8c8c15c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c0ce810 0x7f8c8c15c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 -- 172.21.15.64:0/2615447909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c009a20 con 0x7f8c8c158e20 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c8b7fe700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 0x7f8c8c0c8dd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c016000 tx=0x7f8c7c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:50.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c012cb0 con 0x7f8c8c158e20 2023-09-27T23:33:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c012e10 con 0x7f8c8c158e20 2023-09-27T23:33:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c159720 con 0x7f8c8c158e20 2023-09-27T23:33:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0276c0 con 0x7f8c8c158e20 2023-09-27T23:33:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c159ce0 con 0x7f8c8c158e20 2023-09-27T23:33:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.230+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c159ed0 con 0x7f8c8c158e20 2023-09-27T23:33:50.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.234+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8c7c024070 con 0x7f8c8c158e20 2023-09-27T23:33:50.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.234+0000 7f8c83fff700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c6c060a70 0x7f8c6c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:50.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.234+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f8c7c07f580 con 0x7f8c8c158e20 2023-09-27T23:33:50.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.234+0000 7f8c8bfff700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c6c060a70 0x7f8c6c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:50.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.234+0000 7f8c8bfff700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c6c060a70 0x7f8c6c062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8c74001ad0 tx=0x7f8c74009040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:50.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.238+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8c7c04f620 con 0x7f8c8c158e20 2023-09-27T23:33:50.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:33:50.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:33:50.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:50.458+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f8c8c04be40 con 0x7f8c8c158e20 2023-09-27T23:33:51.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c83fff700 1 -- 172.21.15.64:0/2615447909 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v764) v1 ==== 168+0+0 (secure 0 0 0) 0x7f8c7c052c40 con 0x7f8c8c158e20 2023-09-27T23:33:51.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c6c060a70 msgr2=0x7f8c6c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c6c060a70 0x7f8c6c062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8c74001ad0 tx=0x7f8c74009040 comp rx=0 tx=0).stop 2023-09-27T23:33:51.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 msgr2=0x7f8c8c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 0x7f8c8c0c8dd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c016000 tx=0x7f8c7c019000 comp rx=0 tx=0).stop 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 shutdown_connections 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8c6c060a70 0x7f8c6c062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c1591f0 0x7f8c8c0c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.102+0000 7f8c91789700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c8c158e20 0x7f8c8c0c8dd0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.106+0000 7f8c91789700 1 --2- 172.21.15.64:0/2615447909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c0ce810 0x7f8c8c15c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.106+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 >> 172.21.15.64:0/2615447909 conn(0x7f8c8c067bb0 msgr2=0x7f8c8c0c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.106+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 shutdown_connections 2023-09-27T23:33:51.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.106+0000 7f8c91789700 1 -- 172.21.15.64:0/2615447909 wait complete. 2023-09-27T23:33:51.126 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:33:51.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.238+0000 7f2a32693700 1 Processor -- start 2023-09-27T23:33:51.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.238+0000 7f2a32693700 1 -- start start 2023-09-27T23:33:51.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a32693700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2a2c0acf30 0x7f2a2c1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a32693700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c156780 con 0x7f2a2c06d1d0 2023-09-27T23:33:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a32693700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c158f90 con 0x7f2a2c0ad300 2023-09-27T23:33:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a32693700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c1532a0 con 0x7f2a2c0acf30 2023-09-27T23:33:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31e92700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2a2c06d1d0 0x7f2a2c0ac1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37230/0 (socket says 172.21.15.64:37230) 2023-09-27T23:33:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31e92700 1 -- 172.21.15.64:0/1599260743 learned_addr learned my addr 172.21.15.64:0/1599260743 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:51.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31691700 1 --2- 172.21.15.64:0/1599260743 >> v2:172.21.15.123:3300/0 conn(0x7f2a2c0acf30 0x7f2a2c1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a23fff700 1 -- 172.21.15.64:0/1599260743 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394716177 0 0) 0x7f2a2c156780 con 0x7f2a2c06d1d0 2023-09-27T23:33:51.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a23fff700 1 -- 172.21.15.64:0/1599260743 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a18003570 con 0x7f2a2c06d1d0 2023-09-27T23:33:51.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31691700 1 -- 172.21.15.64:0/1599260743 >> v1:172.21.15.123:6789/0 conn(0x7f2a2c0ad300 legacy=0x7f2a2c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a23fff700 1 -- 172.21.15.64:0/1599260743 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2182569674 0 0) 0x7f2a18003570 con 0x7f2a2c06d1d0 2023-09-27T23:33:51.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31691700 1 -- 172.21.15.64:0/1599260743 >> v1:172.21.15.64:6789/0 conn(0x7f2a2c06d1d0 legacy=0x7f2a2c0ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31691700 1 -- 172.21.15.64:0/1599260743 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c06d630 con 0x7f2a2c0acf30 2023-09-27T23:33:51.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a31691700 1 --2- 172.21.15.64:0/1599260743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c0acf30 0x7f2a2c1557d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009860 tx=0x7f2a280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8140911966ec0c59 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a23fff700 1 -- 172.21.15.64:0/1599260743 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240017f0 con 0x7f2a2c0acf30 2023-09-27T23:33:51.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a23fff700 1 -- 172.21.15.64:0/1599260743 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28011420 con 0x7f2a2c0acf30 2023-09-27T23:33:51.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.242+0000 7f2a23fff700 1 -- 172.21.15.64:0/1599260743 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280117a0 con 0x7f2a2c0acf30 2023-09-27T23:33:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 -- 172.21.15.64:0/1599260743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c0acf30 msgr2=0x7f2a2c1557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 --2- 172.21.15.64:0/1599260743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c0acf30 0x7f2a2c1557d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009860 tx=0x7f2a280091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 -- 172.21.15.64:0/1599260743 shutdown_connections 2023-09-27T23:33:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 --2- 172.21.15.64:0/1599260743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c0acf30 0x7f2a2c1557d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 -- 172.21.15.64:0/1599260743 >> 172.21.15.64:0/1599260743 conn(0x7f2a2c067b50 msgr2=0x7f2a2c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:51.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 -- 172.21.15.64:0/1599260743 shutdown_connections 2023-09-27T23:33:51.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 -- 172.21.15.64:0/1599260743 wait complete. 2023-09-27T23:33:51.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 Processor -- start 2023-09-27T23:33:51.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.246+0000 7f2a32693700 1 -- start start 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c06d1d0 0x7f2a2c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a2c0acf30 0x7f2a2c149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 0x7f2a2c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0afe90 con 0x7f2a2c0ad300 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0cb0b0 con 0x7f2a2c06d1d0 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c14da30 con 0x7f2a2c0acf30 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 0x7f2a2c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 0x7f2a2c0c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54256/0 (socket says 172.21.15.64:54256) 2023-09-27T23:33:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 -- 172.21.15.64:0/3124054630 learned_addr learned my addr 172.21.15.64:0/3124054630 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31691700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c06d1d0 0x7f2a2c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a30e90700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a2c0acf30 0x7f2a2c149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 -- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a2c0acf30 msgr2=0x7f2a2c149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a2c0acf30 0x7f2a2c149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 -- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c06d1d0 msgr2=0x7f2a2c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c06d1d0 0x7f2a2c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 -- 172.21.15.64:0/3124054630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28009420 con 0x7f2a2c0ad300 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a31e92700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 0x7f2a2c0c5fd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2a24001920 tx=0x7f2a24007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24004dc0 con 0x7f2a2c0ad300 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400a660 con 0x7f2a2c0ad300 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400aa20 con 0x7f2a2c0ad300 2023-09-27T23:33:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c14dd10 con 0x7f2a2c0ad300 2023-09-27T23:33:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.250+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c14e2d0 con 0x7f2a2c0ad300 2023-09-27T23:33:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.254+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c0b0000 con 0x7f2a2c0ad300 2023-09-27T23:33:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.254+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2a24007150 con 0x7f2a2c0ad300 2023-09-27T23:33:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.254+0000 7f2a217fa700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a10060a70 0x7f2a10062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.254+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f2a2407c810 con 0x7f2a2c0ad300 2023-09-27T23:33:51.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.254+0000 7f2a31691700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a10060a70 0x7f2a10062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.258+0000 7f2a31691700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a10060a70 0x7f2a10062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a280030c0 tx=0x7f2a28017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.262+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2a2404c890 con 0x7f2a2c0ad300 2023-09-27T23:33:51.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.430+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2a2c04be40 con 0x7f2a2c0ad300 2023-09-27T23:33:51.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a217fa700 1 -- 172.21.15.64:0/3124054630 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a240180c0 con 0x7f2a2c0ad300 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a10060a70 msgr2=0x7f2a10062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a10060a70 0x7f2a10062f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a280030c0 tx=0x7f2a28017000 comp rx=0 tx=0).stop 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 msgr2=0x7f2a2c0c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 0x7f2a2c0c5fd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2a24001920 tx=0x7f2a24007d20 comp rx=0 tx=0).stop 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 shutdown_connections 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2a10060a70 0x7f2a10062f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c0ad300 0x7f2a2c0c5fd0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a2c0acf30 0x7f2a2c149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 --2- 172.21.15.64:0/3124054630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c06d1d0 0x7f2a2c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 >> 172.21.15.64:0/3124054630 conn(0x7f2a2c067b50 msgr2=0x7f2a2c150130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 shutdown_connections 2023-09-27T23:33:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.438+0000 7f2a32693700 1 -- 172.21.15.64:0/3124054630 wait complete. 2023-09-27T23:33:51.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:33:51.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:51.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:33:51.498 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:33:51.498 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:33:51.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:51.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 Processor -- start 2023-09-27T23:33:51.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 -- start start 2023-09-27T23:33:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26500cad10 0x7f26500cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f265006b750 0x7f265006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265006b2c0 con 0x7f265006df20 2023-09-27T23:33:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265006a6b0 con 0x7f26500cad10 2023-09-27T23:33:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265006a990 con 0x7f265006b750 2023-09-27T23:33:51.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26500cad10 0x7f26500cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26500cad10 0x7f26500cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54272/0 (socket says 172.21.15.64:54272) 2023-09-27T23:33:51.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 -- 172.21.15.64:0/770117607 learned_addr learned my addr 172.21.15.64:0/770117607 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:51.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f2647fff700 1 --2- 172.21.15.64:0/770117607 >> v2:172.21.15.123:3300/0 conn(0x7f265006b750 0x7f265006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 -- 172.21.15.64:0/770117607 >> v2:172.21.15.123:3300/0 conn(0x7f265006b750 msgr2=0x7f265006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 --2- 172.21.15.64:0/770117607 >> v2:172.21.15.123:3300/0 conn(0x7f265006b750 0x7f265006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 -- 172.21.15.64:0/770117607 >> v1:172.21.15.123:6789/0 conn(0x7f265006df20 legacy=0x7f265015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 -- 172.21.15.64:0/770117607 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265006ac70 con 0x7f26500cad10 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.606+0000 7f264effd700 1 --2- 172.21.15.64:0/770117607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26500cad10 0x7f26500cb0e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f26500696f0 tx=0x7f2640009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85bbfe75dbc0bc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264e7fc700 1 -- 172.21.15.64:0/770117607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264000b860 con 0x7f26500cad10 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264e7fc700 1 -- 172.21.15.64:0/770117607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264000b9c0 con 0x7f26500cad10 2023-09-27T23:33:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264e7fc700 1 -- 172.21.15.64:0/770117607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264000bd60 con 0x7f26500cad10 2023-09-27T23:33:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 -- 172.21.15.64:0/770117607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26500cad10 msgr2=0x7f26500cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 --2- 172.21.15.64:0/770117607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26500cad10 0x7f26500cb0e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f26500696f0 tx=0x7f2640009250 comp rx=0 tx=0).stop 2023-09-27T23:33:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 -- 172.21.15.64:0/770117607 shutdown_connections 2023-09-27T23:33:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 --2- 172.21.15.64:0/770117607 >> v2:172.21.15.123:3300/0 conn(0x7f265006b750 0x7f265006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 --2- 172.21.15.64:0/770117607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26500cad10 0x7f26500cb0e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 -- 172.21.15.64:0/770117607 >> 172.21.15.64:0/770117607 conn(0x7f2650067bb0 msgr2=0x7f26500cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:51.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 -- 172.21.15.64:0/770117607 shutdown_connections 2023-09-27T23:33:51.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 -- 172.21.15.64:0/770117607 wait complete. 2023-09-27T23:33:51.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 Processor -- start 2023-09-27T23:33:51.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.610+0000 7f264ffff700 1 -- start start 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f265006b750 0x7f265014dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 0x7f2650157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26500cad10 0x7f26501584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2650159710 con 0x7f265006df20 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265015d670 con 0x7f26500cad10 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265014e000 con 0x7f265006b750 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 0x7f2650157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 0x7f2650157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54280/0 (socket says 172.21.15.64:54280) 2023-09-27T23:33:51.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 -- 172.21.15.64:0/2644031580 learned_addr learned my addr 172.21.15.64:0/2644031580 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264f7fe700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26500cad10 0x7f26501584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264effd700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f265006b750 0x7f265014dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 -- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f265006b750 msgr2=0x7f265014dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f265006b750 0x7f265014dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 -- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26500cad10 msgr2=0x7f26501584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26500cad10 0x7f26501584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 -- 172.21.15.64:0/2644031580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26400094f0 con 0x7f265006df20 2023-09-27T23:33:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2647fff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 0x7f2650157f90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f263800d8f0 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26380125f0 con 0x7f265006df20 2023-09-27T23:33:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2638012750 con 0x7f265006df20 2023-09-27T23:33:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2650158b00 con 0x7f265006df20 2023-09-27T23:33:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638012af0 con 0x7f265006df20 2023-09-27T23:33:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26501590c0 con 0x7f265006df20 2023-09-27T23:33:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.614+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26501598c0 con 0x7f265006df20 2023-09-27T23:33:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.618+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f263801e460 con 0x7f265006df20 2023-09-27T23:33:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.618+0000 7f2646ffd700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2630060a70 0x7f2630062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.618+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f263807d610 con 0x7f265006df20 2023-09-27T23:33:51.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.618+0000 7f264effd700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2630060a70 0x7f2630062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.618+0000 7f264effd700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2630060a70 0x7f2630062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f26500696f0 tx=0x7f2640017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.626+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f26501598c0 con 0x7f265006df20 2023-09-27T23:33:51.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.822+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26501598c0 con 0x7f265006df20 2023-09-27T23:33:51.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.822+0000 7f2646ffd700 1 -- 172.21.15.64:0/2644031580 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f26501598c0 con 0x7f265006df20 2023-09-27T23:33:51.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:51.828 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2328267940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2328267940},{"type":"v1","addr":"172.21.15.123:6836","nonce":2328267940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1812601333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1812601333},{"type":"v1","addr":"172.21.15.123:6837","nonce":1812601333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3854466596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3854466596},{"type":"v1","addr":"172.21.15.64:6833","nonce":3854466596}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":9096,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3540822091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3540822091},{"type":"v1","addr":"172.21.15.64:6835","nonce":3540822091}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[]} 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2630060a70 msgr2=0x7f2630062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2630060a70 0x7f2630062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f26500696f0 tx=0x7f2640017000 comp rx=0 tx=0).stop 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 msgr2=0x7f2650157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 0x7f2650157f90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f263800d8f0 tx=0x7f2638008040 comp rx=0 tx=0).stop 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 shutdown_connections 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f2630060a70 0x7f2630062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26500cad10 0x7f26501584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265006df20 0x7f2650157f90 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 --2- 172.21.15.64:0/2644031580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f265006b750 0x7f265014dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 >> 172.21.15.64:0/2644031580 conn(0x7f2650067bb0 msgr2=0x7f26500ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:51.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 shutdown_connections 2023-09-27T23:33:51.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.826+0000 7f264ffff700 1 -- 172.21.15.64:0/2644031580 wait complete. 2023-09-27T23:33:51.832 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 56 2023-09-27T23:33:51.847 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:33:51.847 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:33:51.847 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:33:51.847 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:51.848 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:33:51.848 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:33:51.848 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:33:51.848 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:51.848 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:33:51.848 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:33:51.848 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:33:51.848 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:51.849 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:33:51.849 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:33:51.849 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:33:51.849 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:33:51.858 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:33:51.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:33:51.860 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:33:51.860 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:51.861 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:33:51.861 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:33:51.863 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:33:51.863 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:51.864 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:33:51.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:33:51.865 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:33:51.865 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:33:51.867 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:33:51.867 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:51.867 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:33:51.868 DEBUG:teuthology.parallel:result is None 2023-09-27T23:33:51.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:33:51.930 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:33:51.939 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:33:51.945 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:33:51.954 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:33:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 Processor -- start 2023-09-27T23:33:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- start start 2023-09-27T23:33:51.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f541c0c7d90 0x7f541c0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c06dc00 con 0x7f541c13f880 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c06e090 con 0x7f541c06c7c0 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541c06cff0 con 0x7f541c0c7d90 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5421dc4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f541c13f880 0x7f541c0ae9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37244/0 (socket says 172.21.15.64:37244) 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5421dc4700 1 -- 172.21.15.64:0/2203903142 learned_addr learned my addr 172.21.15.64:0/2203903142 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f54215c3700 1 --2- 172.21.15.64:0/2203903142 >> v2:172.21.15.123:3300/0 conn(0x7f541c0c7d90 0x7f541c0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f540ffff700 1 -- 172.21.15.64:0/2203903142 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501507074 0 0) 0x7f541c06dc00 con 0x7f541c13f880 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f540ffff700 1 -- 172.21.15.64:0/2203903142 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5408003570 con 0x7f541c13f880 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f54215c3700 1 -- 172.21.15.64:0/2203903142 >> v1:172.21.15.123:6789/0 conn(0x7f541c06c7c0 legacy=0x7f541c13f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f54215c3700 1 -- 172.21.15.64:0/2203903142 >> v1:172.21.15.64:6789/0 conn(0x7f541c13f880 legacy=0x7f541c0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f54215c3700 1 -- 172.21.15.64:0/2203903142 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c06d2d0 con 0x7f541c0c7d90 2023-09-27T23:33:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f54215c3700 1 --2- 172.21.15.64:0/2203903142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c0c7d90 0x7f541c0c8160 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f54180098e0 tx=0x7f54180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4c1bfd6c542296 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f540ffff700 1 -- 172.21.15.64:0/2203903142 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800b640 con 0x7f541c0c7d90 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f540ffff700 1 -- 172.21.15.64:0/2203903142 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800b7a0 con 0x7f541c0c7d90 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f540ffff700 1 -- 172.21.15.64:0/2203903142 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800bb40 con 0x7f541c0c7d90 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- 172.21.15.64:0/2203903142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c0c7d90 msgr2=0x7f541c0c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 --2- 172.21.15.64:0/2203903142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c0c7d90 0x7f541c0c8160 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f54180098e0 tx=0x7f54180091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- 172.21.15.64:0/2203903142 shutdown_connections 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 --2- 172.21.15.64:0/2203903142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c0c7d90 0x7f541c0c8160 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- 172.21.15.64:0/2203903142 >> 172.21.15.64:0/2203903142 conn(0x7f541c067b60 msgr2=0x7f541c0ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- 172.21.15.64:0/2203903142 shutdown_connections 2023-09-27T23:33:51.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.978+0000 7f5423827700 1 -- 172.21.15.64:0/2203903142 wait complete. 2023-09-27T23:33:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 Processor -- start 2023-09-27T23:33:51.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 -- start start 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c06c7c0 0x7f541c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f541c0c7d90 0x7f541c13b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f541c13f880 0x7f541c0c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c7660 con 0x7f541c0c7d90 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c6a50 con 0x7f541c06c7c0 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c6d30 con 0x7f541c13f880 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5420dc2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f541c0c7d90 0x7f541c13b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c06c7c0 0x7f541c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5420dc2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f541c0c7d90 0x7f541c13b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54316/0 (socket says 172.21.15.64:54316) 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5420dc2700 1 -- 172.21.15.64:0/4056678191 learned_addr learned my addr 172.21.15.64:0/4056678191 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5421dc4700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f541c13f880 0x7f541c0c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 -- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f541c13f880 msgr2=0x7f541c0c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f541c13f880 0x7f541c0c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 -- 172.21.15.64:0/4056678191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f541c0c7d90 msgr2=0x7f541c13b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f541c0c7d90 0x7f541c13b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 -- 172.21.15.64:0/4056678191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009420 con 0x7f541c06c7c0 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f54215c3700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c06c7c0 0x7f541c0c6470 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5418016000 tx=0x7f5418017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418000d10 con 0x7f541c06c7c0 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800bb20 con 0x7f541c06c7c0 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c0c7010 con 0x7f541c06c7c0 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011670 con 0x7f541c06c7c0 2023-09-27T23:33:51.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.982+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c0c4420 con 0x7f541c06c7c0 2023-09-27T23:33:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.986+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c071120 con 0x7f541c06c7c0 2023-09-27T23:33:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.986+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f54180119e0 con 0x7f541c06c7c0 2023-09-27T23:33:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.986+0000 7f540d7fa700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5400060a40 0x7f5400062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.986+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f541807e5d0 con 0x7f541c06c7c0 2023-09-27T23:33:51.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.986+0000 7f5420dc2700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5400060a40 0x7f5400062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:51.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.986+0000 7f5420dc2700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5400060a40 0x7f5400062ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f54100078a0 tx=0x7f5410007830 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:51.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:51.994+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f541804e670 con 0x7f541c06c7c0 2023-09-27T23:33:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:52.170+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f541c13c400 con 0x7f541c06c7c0 2023-09-27T23:33:53.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.122+0000 7f540d7fa700 1 -- 172.21.15.64:0/4056678191 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v153) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5418051c90 con 0x7f541c06c7c0 2023-09-27T23:33:53.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5400060a40 msgr2=0x7f5400062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5400060a40 0x7f5400062ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f54100078a0 tx=0x7f5410007830 comp rx=0 tx=0).stop 2023-09-27T23:33:53.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c06c7c0 msgr2=0x7f541c0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c06c7c0 0x7f541c0c6470 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5418016000 tx=0x7f5418017000 comp rx=0 tx=0).stop 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 shutdown_connections 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f5400060a40 0x7f5400062ef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f541c13f880 0x7f541c0c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f541c0c7d90 0x7f541c13b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 --2- 172.21.15.64:0/4056678191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c06c7c0 0x7f541c0c6470 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 >> 172.21.15.64:0/4056678191 conn(0x7f541c067b60 msgr2=0x7f541c158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 shutdown_connections 2023-09-27T23:33:53.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.126+0000 7f5423827700 1 -- 172.21.15.64:0/4056678191 wait complete. 2023-09-27T23:33:53.132 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:33:53.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:33:53.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 Processor -- start 2023-09-27T23:33:53.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 -- start start 2023-09-27T23:33:53.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a54158e70 0x7f6a540afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a540cc470 0x7f6a540cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a540ca5a0 con 0x7f6a5406a540 2023-09-27T23:33:53.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5406dac0 con 0x7f6a540cc470 2023-09-27T23:33:53.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540b0260 con 0x7f6a54158e70 2023-09-27T23:33:53.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a51d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6a5406a540 0x7f6a5406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37258/0 (socket says 172.21.15.64:37258) 2023-09-27T23:33:53.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a51d9b700 1 -- 172.21.15.64:0/3635604256 learned_addr learned my addr 172.21.15.64:0/3635604256 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:53.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a52d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a540cc470 0x7f6a540cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5259c700 1 --2- 172.21.15.64:0/3635604256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a54158e70 0x7f6a540afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a5159a700 1 -- 172.21.15.64:0/3635604256 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844645326 0 0) 0x7f6a540ca5a0 con 0x7f6a5406a540 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a52d9d700 1 -- 172.21.15.64:0/3635604256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a54158e70 msgr2=0x7f6a540afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a52d9d700 1 --2- 172.21.15.64:0/3635604256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a54158e70 0x7f6a540afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a52d9d700 1 -- 172.21.15.64:0/3635604256 >> v1:172.21.15.64:6789/0 conn(0x7f6a5406a540 legacy=0x7f6a5406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a52d9d700 1 -- 172.21.15.64:0/3635604256 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5406d220 con 0x7f6a540cc470 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.250+0000 7f6a52d9d700 1 --2- 172.21.15.64:0/3635604256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a540cc8b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0098e0 tx=0x7f6a4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468b0f2c0cf735ad server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5159a700 1 -- 172.21.15.64:0/3635604256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00b640 con 0x7f6a540cc470 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5159a700 1 -- 172.21.15.64:0/3635604256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00b7a0 con 0x7f6a540cc470 2023-09-27T23:33:53.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5159a700 1 -- 172.21.15.64:0/3635604256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00bb40 con 0x7f6a540cc470 2023-09-27T23:33:53.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 -- 172.21.15.64:0/3635604256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 msgr2=0x7f6a540cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 --2- 172.21.15.64:0/3635604256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a540cc8b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0098e0 tx=0x7f6a4c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 -- 172.21.15.64:0/3635604256 shutdown_connections 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 --2- 172.21.15.64:0/3635604256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a540cc8b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 --2- 172.21.15.64:0/3635604256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a54158e70 0x7f6a540afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 -- 172.21.15.64:0/3635604256 >> 172.21.15.64:0/3635604256 conn(0x7f6a540670e0 msgr2=0x7f6a540674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 -- 172.21.15.64:0/3635604256 shutdown_connections 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 -- 172.21.15.64:0/3635604256 wait complete. 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 Processor -- start 2023-09-27T23:33:53.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.254+0000 7f6a5359e700 1 -- start start 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a5406a540 0x7f6a540b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a54157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a54158e70 0x7f6a5416c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540c3ca0 con 0x7f6a540cc470 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540c3f80 con 0x7f6a54158e70 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540c4260 con 0x7f6a5406a540 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a54157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a54157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54324/0 (socket says 172.21.15.64:54324) 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 -- 172.21.15.64:0/4116415892 learned_addr learned my addr 172.21.15.64:0/4116415892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5259c700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a5406a540 0x7f6a540b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a52d9d700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a54158e70 0x7f6a5416c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 -- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a5406a540 msgr2=0x7f6a540b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a5406a540 0x7f6a540b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 -- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a54158e70 msgr2=0x7f6a5416c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a54158e70 0x7f6a5416c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 -- 172.21.15.64:0/4116415892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c009420 con 0x7f6a540cc470 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a52d9d700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a54158e70 0x7f6a5416c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:53.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a51d9b700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a54157ba0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c5b0 tx=0x7f6a3c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c005b30 con 0x7f6a540cc470 2023-09-27T23:33:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c005c90 con 0x7f6a540cc470 2023-09-27T23:33:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a540c4540 con 0x7f6a540cc470 2023-09-27T23:33:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c007300 con 0x7f6a540cc470 2023-09-27T23:33:53.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.258+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a540c38c0 con 0x7f6a540cc470 2023-09-27T23:33:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.262+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a540b38b0 con 0x7f6a540cc470 2023-09-27T23:33:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.262+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6a3c01b3b0 con 0x7f6a540cc470 2023-09-27T23:33:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.262+0000 7f6a4a7fc700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6a34060a70 0x7f6a34062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.262+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6a3c07c2c0 con 0x7f6a540cc470 2023-09-27T23:33:53.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.262+0000 7f6a5259c700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6a34060a70 0x7f6a34062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.262+0000 7f6a5259c700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6a34060a70 0x7f6a34062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009fd0 tx=0x7f6a4400b040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.270+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6a540b38b0 con 0x7f6a540cc470 2023-09-27T23:33:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.482+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6a540b38b0 con 0x7f6a540cc470 2023-09-27T23:33:53.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.482+0000 7f6a4a7fc700 1 -- 172.21.15.64:0/4116415892 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6a540b38b0 con 0x7f6a540cc470 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:53.487 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:33:53.489 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.490 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:33:53.492 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.493 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:33:53.495 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:33:53.496 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:33:53.500 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:33:53.501 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:33:53.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:33:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:33:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:53.503 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:33:53.503 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:33:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6a34060a70 msgr2=0x7f6a34062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6a34060a70 0x7f6a34062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009fd0 tx=0x7f6a4400b040 comp rx=0 tx=0).stop 2023-09-27T23:33:53.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 msgr2=0x7f6a54157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a54157ba0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c5b0 tx=0x7f6a3c001070 comp rx=0 tx=0).stop 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 shutdown_connections 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6a34060a70 0x7f6a34062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a54158e70 0x7f6a5416c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a540cc470 0x7f6a54157ba0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 --2- 172.21.15.64:0/4116415892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a5406a540 0x7f6a540b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 >> 172.21.15.64:0/4116415892 conn(0x7f6a540670e0 msgr2=0x7f6a5406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 shutdown_connections 2023-09-27T23:33:53.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.486+0000 7f6a5359e700 1 -- 172.21.15.64:0/4116415892 wait complete. 2023-09-27T23:33:53.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:53.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 Processor -- start 2023-09-27T23:33:53.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- start start 2023-09-27T23:33:53.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc15d320 0x7f1edc15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1edc0c15b0 0x7f1edc0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0c7190 con 0x7f1edc06d690 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1edc0c1b50 con 0x7f1edc0c15b0 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc0c1e30 con 0x7f1edc15d320 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1edc0c15b0 0x7f1edc0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1edc0c15b0 0x7f1edc0aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54330/0 (socket says 172.21.15.64:54330) 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2318173720 learned_addr learned my addr 172.21.15.64:0/2318173720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1edad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1edc06d690 0x7f1edc0680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37264/0 (socket says 172.21.15.64:37264) 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1eda59c700 1 --2- 172.21.15.64:0/2318173720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc15d320 0x7f1edc15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2318173720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc15d320 msgr2=0x7f1edc15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 --2- 172.21.15.64:0/2318173720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc15d320 0x7f1edc15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2318173720 >> v1:172.21.15.64:6789/0 conn(0x7f1edc06d690 legacy=0x7f1edc0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2318173720 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0c6e20 con 0x7f1edc0c15b0 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed9d9b700 1 --2- 172.21.15.64:0/2318173720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0aeba0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009f20 tx=0x7f1ed4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4939e983cd3d1c38 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed959a700 1 -- 172.21.15.64:0/2318173720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400e070 con 0x7f1edc0c15b0 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed959a700 1 -- 172.21.15.64:0/2318173720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed400b9d0 con 0x7f1edc0c15b0 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ed959a700 1 -- 172.21.15.64:0/2318173720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40092e0 con 0x7f1edc0c15b0 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2318173720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 msgr2=0x7f1edc0aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2318173720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0aeba0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009f20 tx=0x7f1ed4009840 comp rx=0 tx=0).stop 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2318173720 shutdown_connections 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2318173720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0aeba0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2318173720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc15d320 0x7f1edc15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2318173720 >> 172.21.15.64:0/2318173720 conn(0x7f1edc0c8850 msgr2=0x7f1edc0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:53.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2318173720 shutdown_connections 2023-09-27T23:33:53.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.606+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2318173720 wait complete. 2023-09-27T23:33:53.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 Processor -- start 2023-09-27T23:33:53.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 -- start start 2023-09-27T23:33:53.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc06d690 0x7f1edc0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1edc15d320 0x7f1edc155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc1559d0 con 0x7f1edc0c15b0 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc155c20 con 0x7f1edc15d320 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ee0e07700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc155f00 con 0x7f1edc06d690 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54342/0 (socket says 172.21.15.64:54342) 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2203713861 learned_addr learned my addr 172.21.15.64:0/2203713861 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1eda59c700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc06d690 0x7f1edc0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1edad9d700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1edc15d320 0x7f1edc155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc06d690 msgr2=0x7f1edc0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc06d690 0x7f1edc0c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1edc15d320 msgr2=0x7f1edc155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1edc15d320 0x7f1edc155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 -- 172.21.15.64:0/2203713861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed4009ae0 con 0x7f1edc0c15b0 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1eda59c700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc06d690 0x7f1edc0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.610+0000 7f1ed9d9b700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0ca4b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400b610 tx=0x7f1ed4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40089a0 con 0x7f1edc0c15b0 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4008b00 con 0x7f1edc0c15b0 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4008e80 con 0x7f1edc0c15b0 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc0c4a40 con 0x7f1edc0c15b0 2023-09-27T23:33:53.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1edc0c5000 con 0x7f1edc0c15b0 2023-09-27T23:33:53.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1edc0c4880 con 0x7f1edc0c15b0 2023-09-27T23:33:53.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1ed4003040 con 0x7f1edc0c15b0 2023-09-27T23:33:53.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ed27fc700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ebc060a70 0x7f1ebc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.614+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1ed407e670 con 0x7f1edc0c15b0 2023-09-27T23:33:53.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.618+0000 7f1eda59c700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ebc060a70 0x7f1ebc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.618+0000 7f1eda59c700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ebc060a70 0x7f1ebc062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0c40e0 tx=0x7f1ec4008040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.622+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1ed404e990 con 0x7f1edc0c15b0 2023-09-27T23:33:53.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.786+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1edc071120 con 0x7f1edc0c15b0 2023-09-27T23:33:53.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.786+0000 7f1ed27fc700 1 -- 172.21.15.64:0/2203713861 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+13865 (secure 0 0 0) 0x7f1ed4051fb0 con 0x7f1edc0c15b0 2023-09-27T23:33:53.792 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:53.792 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":153,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:53.102691+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ebc060a70 msgr2=0x7f1ebc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ebc060a70 0x7f1ebc062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0c40e0 tx=0x7f1ec4008040 comp rx=0 tx=0).stop 2023-09-27T23:33:53.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 msgr2=0x7f1edc0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0ca4b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400b610 tx=0x7f1ed4018000 comp rx=0 tx=0).stop 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 shutdown_connections 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1ebc060a70 0x7f1ebc062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1edc15d320 0x7f1edc155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1edc0c15b0 0x7f1edc0ca4b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 --2- 172.21.15.64:0/2203713861 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1edc06d690 0x7f1edc0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 >> 172.21.15.64:0/2203713861 conn(0x7f1edc0c8850 msgr2=0x7f1edc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 shutdown_connections 2023-09-27T23:33:53.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.790+0000 7f1ee0e07700 1 -- 172.21.15.64:0/2203713861 wait complete. 2023-09-27T23:33:53.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:33:53.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:33:53.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 Processor -- start 2023-09-27T23:33:53.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 -- start start 2023-09-27T23:33:53.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f848c142a50 0x7f848c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 0x7f848c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c0cc7e0 con 0x7f848c142680 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c0cbbf0 con 0x7f848c142a50 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84930d2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0cbeb0 con 0x7f848c0ad650 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84928d1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 0x7f848c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84918cf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f848c142a50 0x7f848c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84928d1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 0x7f848c06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:58292/0 (socket says 172.21.15.64:58292) 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84928d1700 1 -- 172.21.15.64:0/1605476680 learned_addr learned my addr 172.21.15.64:0/1605476680 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:53.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84918cf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f848c142a50 0x7f848c06d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:38808/0 (socket says 172.21.15.64:38808) 2023-09-27T23:33:53.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84910ce700 1 -- 172.21.15.64:0/1605476680 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119533019 0 0) 0x7f848c0cc7e0 con 0x7f848c142680 2023-09-27T23:33:53.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84910ce700 1 -- 172.21.15.64:0/1605476680 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8478003570 con 0x7f848c142680 2023-09-27T23:33:53.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84918cf700 1 -- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 msgr2=0x7f848c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:53.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84918cf700 1 --2- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 0x7f848c06d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84918cf700 1 -- 172.21.15.64:0/1605476680 >> v1:172.21.15.123:6789/0 conn(0x7f848c142680 legacy=0x7f848c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84918cf700 1 -- 172.21.15.64:0/1605476680 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c0cc190 con 0x7f848c142a50 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.922+0000 7f84928d1700 1 --2- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 0x7f848c06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84918cf700 1 --2- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142a50 0x7f848c06d050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f84800098e0 tx=0x7f84800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71382c527924ab2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84910ce700 1 -- 172.21.15.64:0/1605476680 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011630 con 0x7f848c142a50 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84910ce700 1 -- 172.21.15.64:0/1605476680 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480011790 con 0x7f848c142a50 2023-09-27T23:33:53.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84910ce700 1 -- 172.21.15.64:0/1605476680 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011b10 con 0x7f848c142a50 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 -- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142a50 msgr2=0x7f848c06d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 --2- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142a50 0x7f848c06d050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f84800098e0 tx=0x7f84800091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 -- 172.21.15.64:0/1605476680 shutdown_connections 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 --2- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c0ad650 0x7f848c06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 --2- 172.21.15.64:0/1605476680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142a50 0x7f848c06d050 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 -- 172.21.15.64:0/1605476680 >> 172.21.15.64:0/1605476680 conn(0x7f848c0ccb30 msgr2=0x7f848c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 -- 172.21.15.64:0/1605476680 shutdown_connections 2023-09-27T23:33:53.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.926+0000 7f84930d2700 1 -- 172.21.15.64:0/1605476680 wait complete. 2023-09-27T23:33:53.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 Processor -- start 2023-09-27T23:33:53.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 -- start start 2023-09-27T23:33:53.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 0x7f848c0aa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142680 0x7f848c16c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c142a50 0x7f848c16cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0c4430 con 0x7f848c0ad650 2023-09-27T23:33:53.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0aac30 con 0x7f848c142680 2023-09-27T23:33:53.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84930d2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0aaf10 con 0x7f848c142a50 2023-09-27T23:33:53.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 0x7f848c0aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 0x7f848c0aa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54356/0 (socket says 172.21.15.64:54356) 2023-09-27T23:33:53.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 -- 172.21.15.64:0/1051274320 learned_addr learned my addr 172.21.15.64:0/1051274320 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:53.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84928d1700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c142a50 0x7f848c16cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84918cf700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142680 0x7f848c16c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 -- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c142a50 msgr2=0x7f848c16cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c142a50 0x7f848c16cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 -- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142680 msgr2=0x7f848c16c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142680 0x7f848c16c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 -- 172.21.15.64:0/1051274320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8484009ae0 con 0x7f848c0ad650 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f84920d0700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 0x7f848c0aa390 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f848800c790 tx=0x7f8488007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.930+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488010510 con 0x7f848c0ad650 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488010670 con 0x7f848c0ad650 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8480009420 con 0x7f848c0ad650 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488010a10 con 0x7f848c0ad650 2023-09-27T23:33:53.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c0c4680 con 0x7f848c0ad650 2023-09-27T23:33:53.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c0c4870 con 0x7f848c0ad650 2023-09-27T23:33:53.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f848801b3b0 con 0x7f848c0ad650 2023-09-27T23:33:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f847e7fc700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8470060a70 0x7f8470062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.934+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f848807c680 con 0x7f848c0ad650 2023-09-27T23:33:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.938+0000 7f84918cf700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8470060a70 0x7f8470062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:53.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.938+0000 7f84918cf700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8470060a70 0x7f8470062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84800091e0 tx=0x7f8480017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:53.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:53.942+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f848804ca00 con 0x7f848c0ad650 2023-09-27T23:33:54.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f848c04be40 con 0x7f848c0ad650 2023-09-27T23:33:54.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f847e7fc700 1 -- 172.21.15.64:0/1051274320 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8488017020 con 0x7f848c0ad650 2023-09-27T23:33:54.106 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:33:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8470060a70 msgr2=0x7f8470062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8470060a70 0x7f8470062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84800091e0 tx=0x7f8480017000 comp rx=0 tx=0).stop 2023-09-27T23:33:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 msgr2=0x7f848c0aa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 0x7f848c0aa390 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f848800c790 tx=0x7f8488007e10 comp rx=0 tx=0).stop 2023-09-27T23:33:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 shutdown_connections 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8470060a70 0x7f8470062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f848c142a50 0x7f848c16cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f848c142680 0x7f848c16c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 --2- 172.21.15.64:0/1051274320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f848c0ad650 0x7f848c0aa390 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 >> 172.21.15.64:0/1051274320 conn(0x7f848c0ccb30 msgr2=0x7f848c0adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 shutdown_connections 2023-09-27T23:33:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.102+0000 7f84930d2700 1 -- 172.21.15.64:0/1051274320 wait complete. 2023-09-27T23:33:54.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:33:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a3dd8700 1 Processor -- start 2023-09-27T23:33:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a3dd8700 1 -- start start 2023-09-27T23:33:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a3dd8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd19c1514a0 0x7fd19c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a3dd8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0cbee0 con 0x7fd19c06b7a0 2023-09-27T23:33:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a3dd8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c158530 con 0x7fd19c0cae60 2023-09-27T23:33:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a3dd8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c0cc370 con 0x7fd19c1514a0 2023-09-27T23:33:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd19c1514a0 0x7fd19c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd19c1514a0 0x7fd19c157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54366/0 (socket says 172.21.15.64:54366) 2023-09-27T23:33:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 -- 172.21.15.64:0/63158933 learned_addr learned my addr 172.21.15.64:0/63158933 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a2375700 1 --1- 172.21.15.64:0/63158933 >> v1:172.21.15.64:6789/0 conn(0x7fd19c06b7a0 0x7fd19c158740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37268/0 (socket says 172.21.15.64:37268) 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 -- 172.21.15.64:0/63158933 >> v1:172.21.15.123:6789/0 conn(0x7fd19c0cae60 legacy=0x7fd19c06b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 -- 172.21.15.64:0/63158933 >> v1:172.21.15.64:6789/0 conn(0x7fd19c06b7a0 legacy=0x7fd19c158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a0b72700 1 -- 172.21.15.64:0/63158933 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834472523 0 0) 0x7fd19c0cbee0 con 0x7fd19c06b7a0 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 -- 172.21.15.64:0/63158933 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0cb2d0 con 0x7fd19c1514a0 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a1b74700 1 --2- 172.21.15.64:0/63158933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c157ff0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd1980099b0 tx=0x7fd198009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1ff6427c7c906 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a0b72700 1 -- 172.21.15.64:0/63158933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800b860 con 0x7fd19c1514a0 2023-09-27T23:33:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a0b72700 1 -- 172.21.15.64:0/63158933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800b9c0 con 0x7fd19c1514a0 2023-09-27T23:33:54.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.230+0000 7fd1a0b72700 1 -- 172.21.15.64:0/63158933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800bd60 con 0x7fd19c1514a0 2023-09-27T23:33:54.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/63158933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 msgr2=0x7fd19c157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/63158933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c157ff0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd1980099b0 tx=0x7fd198009250 comp rx=0 tx=0).stop 2023-09-27T23:33:54.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/63158933 shutdown_connections 2023-09-27T23:33:54.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/63158933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c157ff0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/63158933 >> 172.21.15.64:0/63158933 conn(0x7fd19c0cc580 msgr2=0x7fd19c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:54.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/63158933 shutdown_connections 2023-09-27T23:33:54.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/63158933 wait complete. 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 Processor -- start 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- start start 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c06b7a0 0x7fd19c13e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0cae60 0x7fd19c155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0c21c0 con 0x7fd19c1514a0 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c06dbf0 con 0x7fd19c0cae60 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a3dd8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c06ce30 con 0x7fd19c06b7a0 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c1561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54382/0 (socket says 172.21.15.64:54382) 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 -- 172.21.15.64:0/3023117689 learned_addr learned my addr 172.21.15.64:0/3023117689 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a1b74700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c06b7a0 0x7fd19c13e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a1373700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0cae60 0x7fd19c155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 -- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c06b7a0 msgr2=0x7fd19c13e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c06b7a0 0x7fd19c13e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 -- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0cae60 msgr2=0x7fd19c155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0cae60 0x7fd19c155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 -- 172.21.15.64:0/3023117689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980094f0 con 0x7fd19c1514a0 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a1373700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0cae60 0x7fd19c155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a1b74700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c06b7a0 0x7fd19c13e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.234+0000 7fd1a2375700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c1561c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd19400cd80 tx=0x7fd194001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194005df0 con 0x7fd19c1514a0 2023-09-27T23:33:54.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400a900 con 0x7fd19c1514a0 2023-09-27T23:33:54.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c06d110 con 0x7fd19c1514a0 2023-09-27T23:33:54.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400acc0 con 0x7fd19c1514a0 2023-09-27T23:33:54.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c06d6d0 con 0x7fd19c1514a0 2023-09-27T23:33:54.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c16a900 con 0x7fd19c1514a0 2023-09-27T23:33:54.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd194007300 con 0x7fd19c1514a0 2023-09-27T23:33:54.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd191ffb700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd180060a70 0x7fd180062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.238+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd19407cfa0 con 0x7fd19c1514a0 2023-09-27T23:33:54.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.242+0000 7fd1a1b74700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd180060a70 0x7fd180062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.242+0000 7fd1a1b74700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd180060a70 0x7fd180062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd1980099b0 tx=0x7fd198017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.246+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd19404d3a0 con 0x7fd19c1514a0 2023-09-27T23:33:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.406+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd19c06d8f0 con 0x7fd19c1514a0 2023-09-27T23:33:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.406+0000 7fd191ffb700 1 -- 172.21.15.64:0/3023117689 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v153) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd1940509c0 con 0x7fd19c1514a0 2023-09-27T23:33:54.411 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:33:54.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd180060a70 msgr2=0x7fd180062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd180060a70 0x7fd180062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd1980099b0 tx=0x7fd198017000 comp rx=0 tx=0).stop 2023-09-27T23:33:54.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 msgr2=0x7fd19c1561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c1561c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd19400cd80 tx=0x7fd194001ae0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 shutdown_connections 2023-09-27T23:33:54.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fd180060a70 0x7fd180062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd19c1514a0 0x7fd19c1561c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0cae60 0x7fd19c155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 --2- 172.21.15.64:0/3023117689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c06b7a0 0x7fd19c13e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 >> 172.21.15.64:0/3023117689 conn(0x7fd19c0cc580 msgr2=0x7fd19c06b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:54.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 shutdown_connections 2023-09-27T23:33:54.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.410+0000 7fd1a3dd8700 1 -- 172.21.15.64:0/3023117689 wait complete. 2023-09-27T23:33:54.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:33:54.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 Processor -- start 2023-09-27T23:33:54.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- start start 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1db40ce100 0x7f1db40ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db406a540 con 0x7f1db41590b0 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db406b3a0 con 0x7f1db4158ce0 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db415d990 con 0x7f1db40ce100 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db8c58700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1db41590b0 0x7f1db406d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37284/0 (socket says 172.21.15.64:37284) 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db8c58700 1 -- 172.21.15.64:0/2264946893 learned_addr learned my addr 172.21.15.64:0/2264946893 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:54.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db9c5a700 1 --2- 172.21.15.64:0/2264946893 >> v2:172.21.15.64:3300/0 conn(0x7f1db40ce100 0x7f1db40ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1da7fff700 1 -- 172.21.15.64:0/2264946893 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514758250 0 0) 0x7f1db406a540 con 0x7f1db41590b0 2023-09-27T23:33:54.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1da7fff700 1 -- 172.21.15.64:0/2264946893 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da0003570 con 0x7f1db41590b0 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db9c5a700 1 -- 172.21.15.64:0/2264946893 >> v1:172.21.15.123:6789/0 conn(0x7f1db4158ce0 legacy=0x7f1db406b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db9c5a700 1 -- 172.21.15.64:0/2264946893 >> v1:172.21.15.64:6789/0 conn(0x7f1db41590b0 legacy=0x7f1db406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1da7fff700 1 -- 172.21.15.64:0/2264946893 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198064076 0 0) 0x7f1da0003570 con 0x7f1db41590b0 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db9c5a700 1 -- 172.21.15.64:0/2264946893 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40cea40 con 0x7f1db40ce100 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1db9c5a700 1 --2- 172.21.15.64:0/2264946893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db40ce100 0x7f1db40ce500 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1db00098e0 tx=0x7f1db00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8773879949d382 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1da7fff700 1 -- 172.21.15.64:0/2264946893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000b640 con 0x7f1db40ce100 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1da7fff700 1 -- 172.21.15.64:0/2264946893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db000b7a0 con 0x7f1db40ce100 2023-09-27T23:33:54.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1da7fff700 1 -- 172.21.15.64:0/2264946893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000bb20 con 0x7f1db40ce100 2023-09-27T23:33:54.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- 172.21.15.64:0/2264946893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db40ce100 msgr2=0x7f1db40ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 --2- 172.21.15.64:0/2264946893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db40ce100 0x7f1db40ce500 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1db00098e0 tx=0x7f1db00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- 172.21.15.64:0/2264946893 shutdown_connections 2023-09-27T23:33:54.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 --2- 172.21.15.64:0/2264946893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db40ce100 0x7f1db40ce500 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- 172.21.15.64:0/2264946893 >> 172.21.15.64:0/2264946893 conn(0x7f1db4067b70 msgr2=0x7f1db40cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:54.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- 172.21.15.64:0/2264946893 shutdown_connections 2023-09-27T23:33:54.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.530+0000 7f1dba45b700 1 -- 172.21.15.64:0/2264946893 wait complete. 2023-09-27T23:33:54.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 Processor -- start 2023-09-27T23:33:54.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 -- start start 2023-09-27T23:33:54.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 0x7f1db41595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db4158ce0 0x7f1db4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db41590b0 0x7f1db4150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4151bf0 con 0x7f1db4158ce0 2023-09-27T23:33:54.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40c7d90 con 0x7f1db41590b0 2023-09-27T23:33:54.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1dba45b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40c8070 con 0x7f1db40ce100 2023-09-27T23:33:54.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db8c58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db4158ce0 0x7f1db4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 0x7f1db41595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 0x7f1db41595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:58316/0 (socket says 172.21.15.64:58316) 2023-09-27T23:33:54.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 -- 172.21.15.64:0/3233733747 learned_addr learned my addr 172.21.15.64:0/3233733747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:54.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9c5a700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db41590b0 0x7f1db4150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 -- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db41590b0 msgr2=0x7f1db4150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db41590b0 0x7f1db4150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 -- 172.21.15.64:0/3233733747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db4158ce0 msgr2=0x7f1db4159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db4158ce0 0x7f1db4159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db9459700 1 -- 172.21.15.64:0/3233733747 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db0009420 con 0x7f1db40ce100 2023-09-27T23:33:54.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.534+0000 7f1db8c58700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db4158ce0 0x7f1db4159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1db9459700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 0x7f1db41595d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00cb10 tx=0x7f1dac0017b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00ce20 con 0x7f1db40ce100 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac00a7e0 con 0x7f1db40ce100 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db40c8350 con 0x7f1db40ce100 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00aba0 con 0x7f1db40ce100 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db414fe20 con 0x7f1db40ce100 2023-09-27T23:33:54.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db404be40 con 0x7f1db40ce100 2023-09-27T23:33:54.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1dac003030 con 0x7f1db40ce100 2023-09-27T23:33:54.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1da57fa700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d98060a70 0x7f1d98062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1dac07cd40 con 0x7f1db40ce100 2023-09-27T23:33:54.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.538+0000 7f1db8c58700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d98060a70 0x7f1d98062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.542+0000 7f1db8c58700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d98060a70 0x7f1d98062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1db415c740 tx=0x7f1da8007d10 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.550+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1dac04d0c0 con 0x7f1db40ce100 2023-09-27T23:33:54.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.718+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1db406a790 con 0x7f1db40ce100 2023-09-27T23:33:54.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.718+0000 7f1da57fa700 1 -- 172.21.15.64:0/3233733747 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v153) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1dac0506e0 con 0x7f1db40ce100 2023-09-27T23:33:54.722 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:33:54.723 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d98060a70 msgr2=0x7f1d98062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d98060a70 0x7f1d98062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1db415c740 tx=0x7f1da8007d10 comp rx=0 tx=0).stop 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 msgr2=0x7f1db41595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 0x7f1db41595d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00cb10 tx=0x7f1dac0017b0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 shutdown_connections 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1d98060a70 0x7f1d98062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1db41590b0 0x7f1db4150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1db4158ce0 0x7f1db4159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 --2- 172.21.15.64:0/3233733747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1db40ce100 0x7f1db41595d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 >> 172.21.15.64:0/3233733747 conn(0x7f1db4067b70 msgr2=0x7f1db4153110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:54.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 shutdown_connections 2023-09-27T23:33:54.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.722+0000 7f1dba45b700 1 -- 172.21.15.64:0/3233733747 wait complete. 2023-09-27T23:33:54.743 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:33:54.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:33:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.846+0000 7f6725952700 1 Processor -- start 2023-09-27T23:33:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.846+0000 7f6725952700 1 -- start start 2023-09-27T23:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67200cb130 0x7f67200cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f672006aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f672015c220 0x7f672015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672006b030 con 0x7f672015c220 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672015cbe0 con 0x7f67200cb130 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200b0ac0 con 0x7f672006a6a0 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f672015c220 0x7f672015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f672015c220 0x7f672015c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36004/0 (socket says 172.21.15.64:36004) 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 -- 172.21.15.64:0/851962217 learned_addr learned my addr 172.21.15.64:0/851962217 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671e7fc700 1 --2- 172.21.15.64:0/851962217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f672006aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671effd700 1 --2- 172.21.15.64:0/851962217 >> v2:172.21.15.123:3300/0 conn(0x7f67200cb130 0x7f67200cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 -- 172.21.15.64:0/851962217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 msgr2=0x7f672006aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 --2- 172.21.15.64:0/851962217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f672006aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 -- 172.21.15.64:0/851962217 >> v2:172.21.15.123:3300/0 conn(0x7f67200cb130 msgr2=0x7f67200cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 --2- 172.21.15.64:0/851962217 >> v2:172.21.15.123:3300/0 conn(0x7f67200cb130 0x7f67200cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 -- 172.21.15.64:0/851962217 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672006b210 con 0x7f672015c220 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671f7fe700 1 --2- 172.21.15.64:0/851962217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f672015c220 0x7f672015c6a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f672006b400 tx=0x7f6710009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc6d294053d6895 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671dffb700 1 -- 172.21.15.64:0/851962217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000e070 con 0x7f672015c220 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671dffb700 1 -- 172.21.15.64:0/851962217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000b900 con 0x7f672015c220 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f671dffb700 1 -- 172.21.15.64:0/851962217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000bcc0 con 0x7f672015c220 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- 172.21.15.64:0/851962217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f672015c220 msgr2=0x7f672015c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- 172.21.15.64:0/851962217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f672015c220 0x7f672015c6a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f672006b400 tx=0x7f6710009840 comp rx=0 tx=0).stop 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- 172.21.15.64:0/851962217 shutdown_connections 2023-09-27T23:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- 172.21.15.64:0/851962217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f672015c220 0x7f672015c6a0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- 172.21.15.64:0/851962217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f672006aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 --2- 172.21.15.64:0/851962217 >> v2:172.21.15.123:3300/0 conn(0x7f67200cb130 0x7f67200cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- 172.21.15.64:0/851962217 >> 172.21.15.64:0/851962217 conn(0x7f6720067bb0 msgr2=0x7f67200cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- 172.21.15.64:0/851962217 shutdown_connections 2023-09-27T23:33:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.850+0000 7f6725952700 1 -- 172.21.15.64:0/851962217 wait complete. 2023-09-27T23:33:54.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 Processor -- start 2023-09-27T23:33:54.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 -- start start 2023-09-27T23:33:54.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f6720151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 0x7f672015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f672015c220 0x7f672015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200b5cf0 con 0x7f67200cb130 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720152310 con 0x7f672015c220 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f6725952700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672015d930 con 0x7f672006a6a0 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f671e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 0x7f672015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f671e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 0x7f672015a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36016/0 (socket says 172.21.15.64:36016) 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.854+0000 7f671e7fc700 1 -- 172.21.15.64:0/1971876602 learned_addr learned my addr 172.21.15.64:0/1971876602 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:54.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671effd700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f6720151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671f7fe700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f672015c220 0x7f672015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671e7fc700 1 -- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 msgr2=0x7f6720151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671e7fc700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f6720151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671e7fc700 1 -- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f672015c220 msgr2=0x7f672015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671e7fc700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f672015c220 0x7f672015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671e7fc700 1 -- 172.21.15.64:0/1971876602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009ae0 con 0x7f67200cb130 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f671e7fc700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 0x7f672015a7f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f670800ec80 tx=0x7f67080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670800c850 con 0x7f67200cb130 2023-09-27T23:33:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670800c9d0 con 0x7f67200cb130 2023-09-27T23:33:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670800cd90 con 0x7f67200cb130 2023-09-27T23:33:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672015dc10 con 0x7f67200cb130 2023-09-27T23:33:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672015e1d0 con 0x7f67200cb130 2023-09-27T23:33:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.858+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672006a0b0 con 0x7f67200cb130 2023-09-27T23:33:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.862+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6708013680 con 0x7f67200cb130 2023-09-27T23:33:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.862+0000 7f6716ffd700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6700060a70 0x7f6700062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.862+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f670807d050 con 0x7f67200cb130 2023-09-27T23:33:54.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.862+0000 7f671effd700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6700060a70 0x7f6700062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:54.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.862+0000 7f671effd700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6700060a70 0x7f6700062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6718009db0 tx=0x7f67180091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:54.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:54.870+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f672006a0b0 con 0x7f67200cb130 2023-09-27T23:33:55.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.030+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f672006a0b0 con 0x7f67200cb130 2023-09-27T23:33:55.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.138+0000 7f6716ffd700 1 -- 172.21.15.64:0/1971876602 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v154) v1 ==== 138+0+0 (secure 0 0 0) 0x7f672006a0b0 con 0x7f67200cb130 2023-09-27T23:33:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.142+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6700060a70 msgr2=0x7f6700062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.142+0000 7f6725952700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6700060a70 0x7f6700062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6718009db0 tx=0x7f67180091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.142+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 msgr2=0x7f672015a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:55.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.142+0000 7f6725952700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 0x7f672015a7f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f670800ec80 tx=0x7f67080091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 shutdown_connections 2023-09-27T23:33:55.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f6700060a70 0x7f6700062f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f672015c220 0x7f672015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67200cb130 0x7f672015a7f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 --2- 172.21.15.64:0/1971876602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f672006a6a0 0x7f6720151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 >> 172.21.15.64:0/1971876602 conn(0x7f6720067bb0 msgr2=0x7f67200cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:55.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 shutdown_connections 2023-09-27T23:33:55.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.146+0000 7f6725952700 1 -- 172.21.15.64:0/1971876602 wait complete. 2023-09-27T23:33:55.153 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:33:55.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:33:55.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 Processor -- start 2023-09-27T23:33:55.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 -- start start 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7bc0ce2d0 0x7ff7bc0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0cc040 0x7ff7bc159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc0cbbb0 con 0x7ff7bc0caa10 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc06dc90 con 0x7ff7bc0ce2d0 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c1adc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0cafa0 con 0x7ff7bc0cc040 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7bbfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff7bc0caa10 0x7ff7bc158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34814/0 (socket says 172.21.15.64:34814) 2023-09-27T23:33:55.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7bbfff700 1 -- 172.21.15.64:0/3360663688 learned_addr learned my addr 172.21.15.64:0/3360663688 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:55.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c0ada700 1 --2- 172.21.15.64:0/3360663688 >> v2:172.21.15.123:3300/0 conn(0x7ff7bc0ce2d0 0x7ff7bc0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:55.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7c12db700 1 --2- 172.21.15.64:0/3360663688 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0cc040 0x7ff7bc159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.270+0000 7ff7bb7fe700 1 -- 172.21.15.64:0/3360663688 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053206931 0 0) 0x7ff7bc0cbbb0 con 0x7ff7bc0caa10 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7bb7fe700 1 -- 172.21.15.64:0/3360663688 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a8003570 con 0x7ff7bc0caa10 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7bb7fe700 1 -- 172.21.15.64:0/3360663688 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703232646 0 0) 0x7ff7a8003570 con 0x7ff7bc0caa10 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c12db700 1 -- 172.21.15.64:0/3360663688 >> v2:172.21.15.123:3300/0 conn(0x7ff7bc0ce2d0 msgr2=0x7ff7bc0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c12db700 1 --2- 172.21.15.64:0/3360663688 >> v2:172.21.15.123:3300/0 conn(0x7ff7bc0ce2d0 0x7ff7bc0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c12db700 1 -- 172.21.15.64:0/3360663688 >> v1:172.21.15.64:6789/0 conn(0x7ff7bc0caa10 legacy=0x7ff7bc158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c12db700 1 -- 172.21.15.64:0/3360663688 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0cb280 con 0x7ff7bc0cc040 2023-09-27T23:33:55.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c12db700 1 --2- 172.21.15.64:0/3360663688 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0cc040 0x7ff7bc159010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e60 tx=0x7ff7b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed4175c3144e094 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7bb7fe700 1 -- 172.21.15.64:0/3360663688 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400e070 con 0x7ff7bc0cc040 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7bb7fe700 1 -- 172.21.15.64:0/3360663688 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b400b7a0 con 0x7ff7bc0cc040 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7bb7fe700 1 -- 172.21.15.64:0/3360663688 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4001470 con 0x7ff7bc0cc040 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 -- 172.21.15.64:0/3360663688 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0cc040 msgr2=0x7ff7bc159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/3360663688 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0cc040 0x7ff7bc159010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e60 tx=0x7ff7b4009780 comp rx=0 tx=0).stop 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 -- 172.21.15.64:0/3360663688 shutdown_connections 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/3360663688 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0cc040 0x7ff7bc159010 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/3360663688 >> v2:172.21.15.123:3300/0 conn(0x7ff7bc0ce2d0 0x7ff7bc0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 -- 172.21.15.64:0/3360663688 >> 172.21.15.64:0/3360663688 conn(0x7ff7bc067bb0 msgr2=0x7ff7bc0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 -- 172.21.15.64:0/3360663688 shutdown_connections 2023-09-27T23:33:55.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.274+0000 7ff7c1adc700 1 -- 172.21.15.64:0/3360663688 wait complete. 2023-09-27T23:33:55.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 Processor -- start 2023-09-27T23:33:55.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 -- start start 2023-09-27T23:33:55.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0caa10 0x7ff7bc154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:55.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 0x7ff7bc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:55.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc0ce2d0 0x7ff7bc1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc152fe0 con 0x7ff7bc0cc040 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc155230 con 0x7ff7bc0ce2d0 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c1adc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0c6f40 con 0x7ff7bc0caa10 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 0x7ff7bc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c0ada700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0caa10 0x7ff7bc154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c0ada700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0caa10 0x7ff7bc154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:55810/0 (socket says 172.21.15.64:55810) 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 0x7ff7bc152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36032/0 (socket says 172.21.15.64:36032) 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 -- 172.21.15.64:0/1059939214 learned_addr learned my addr 172.21.15.64:0/1059939214 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:55.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c12db700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc0ce2d0 0x7ff7bc1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:55.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 -- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0caa10 msgr2=0x7ff7bc154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:55.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0caa10 0x7ff7bc154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 -- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc0ce2d0 msgr2=0x7ff7bc1517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc0ce2d0 0x7ff7bc1517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7bbfff700 1 -- 172.21.15.64:0/1059939214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4009a20 con 0x7ff7bc0cc040 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.278+0000 7ff7c12db700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc0ce2d0 0x7ff7bc1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7bbfff700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 0x7ff7bc152aa0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400cb90 tx=0x7ff7a4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007e90 con 0x7ff7bc0cc040 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a4007150 con 0x7ff7bc0cc040 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc1500a0 con 0x7ff7bc0cc040 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007530 con 0x7ff7bc0cc040 2023-09-27T23:33:55.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc154c70 con 0x7ff7bc0cc040 2023-09-27T23:33:55.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.282+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc04be40 con 0x7ff7bc0cc040 2023-09-27T23:33:55.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.286+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff7a4004a40 con 0x7ff7bc0cc040 2023-09-27T23:33:55.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.286+0000 7ff7b8ff9700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff79c060a70 0x7ff79c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:55.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.286+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5896+0+0 (secure 0 0 0) 0x7ff7a407c9e0 con 0x7ff7bc0cc040 2023-09-27T23:33:55.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.286+0000 7ff7c0ada700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff79c060a70 0x7ff79c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:55.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.286+0000 7ff7c0ada700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff79c060a70 0x7ff79c062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009d10 tx=0x7ff7ac0091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:55.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.298+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff7a404cd50 con 0x7ff7bc0cc040 2023-09-27T23:33:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:55.458+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff7bc0cb560 con 0x7ff7bc0cc040 2023-09-27T23:33:56.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.146+0000 7ff7b8ff9700 1 -- 172.21.15.64:0/1059939214 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v155) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff7a4018020 con 0x7ff7bc0cc040 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff79c060a70 msgr2=0x7ff79c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff79c060a70 0x7ff79c062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009d10 tx=0x7ff7ac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 msgr2=0x7ff7bc152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 0x7ff7bc152aa0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400cb90 tx=0x7ff7a4001ad0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 shutdown_connections 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7ff79c060a70 0x7ff79c062f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7bc0ce2d0 0x7ff7bc1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc0cc040 0x7ff7bc152aa0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 --2- 172.21.15.64:0/1059939214 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7bc0caa10 0x7ff7bc154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 >> 172.21.15.64:0/1059939214 conn(0x7ff7bc067bb0 msgr2=0x7ff7bc062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:56.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.166+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 shutdown_connections 2023-09-27T23:33:56.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.170+0000 7ff7c1adc700 1 -- 172.21.15.64:0/1059939214 wait complete. 2023-09-27T23:33:56.173 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:33:56.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1759e700 1 Processor -- start 2023-09-27T23:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1759e700 1 -- start start 2023-09-27T23:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a18141e70 0x7f4a180c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a18159b80 con 0x7f4a180c7a80 2023-09-27T23:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180c8220 con 0x7f4a180c53f0 2023-09-27T23:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180c7f40 con 0x7f4a18141e70 2023-09-27T23:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a0ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4a180c7a80 0x7f4a180c4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34820/0 (socket says 172.21.15.64:34820) 2023-09-27T23:33:56.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a0ffff700 1 -- 172.21.15.64:0/1595940913 learned_addr learned my addr 172.21.15.64:0/1595940913 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:56.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a1659c700 1 --2- 172.21.15.64:0/1595940913 >> v2:172.21.15.123:3300/0 conn(0x7f4a18141e70 0x7f4a180c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795270806 0 0) 0x7f4a18159b80 con 0x7f4a180c7a80 2023-09-27T23:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a04003570 con 0x7f4a180c7a80 2023-09-27T23:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117435728 0 0) 0x7f4a04003570 con 0x7f4a180c7a80 2023-09-27T23:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 >> v2:172.21.15.123:3300/0 conn(0x7f4a18141e70 msgr2=0x7f4a180c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 --2- 172.21.15.64:0/1595940913 >> v2:172.21.15.123:3300/0 conn(0x7f4a18141e70 0x7f4a180c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 >> v1:172.21.15.123:6789/0 conn(0x7f4a180c53f0 legacy=0x7f4a1815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18159530 con 0x7f4a180c7a80 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.286+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f4a00001470 con 0x7f4a180c7a80 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 >> v1:172.21.15.64:6789/0 conn(0x7f4a180c7a80 legacy=0x7f4a180c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a04003a60 0x7f4a04003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a04004810 0x7f4a04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a040051c0 0x7f4a0400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c7f40 con 0x7f4a040051c0 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400c0e0 con 0x7f4a04003a60 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400c2a0 con 0x7f4a04004810 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a040051c0 0x7f4a0400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1659c700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a04003a60 0x7f4a04003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a0ffff700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a04004810 0x7f4a04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 -- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a04004810 msgr2=0x7f4a04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a04004810 0x7f4a04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 -- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a04003a60 msgr2=0x7f4a04003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a04003a60 0x7f4a04003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 -- 172.21.15.64:0/1595940913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04003570 con 0x7f4a040051c0 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 -- 172.21.15.64:0/1595940913 reap_dead start 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a16d9d700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a040051c0 0x7f4a0400ba60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4a100020a0 tx=0x7f4a100016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c25d1a01d4e510 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a100019e0 con 0x7f4a040051c0 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1000c8b0 con 0x7f4a040051c0 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a15d9b700 1 -- 172.21.15.64:0/1595940913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000cc50 con 0x7f4a040051c0 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 -- 172.21.15.64:0/1595940913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a040051c0 msgr2=0x7f4a0400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a040051c0 0x7f4a0400ba60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4a100020a0 tx=0x7f4a100016d0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 -- 172.21.15.64:0/1595940913 shutdown_connections 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 --2- 172.21.15.64:0/1595940913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a040051c0 0x7f4a0400ba60 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 -- 172.21.15.64:0/1595940913 >> 172.21.15.64:0/1595940913 conn(0x7f4a18067b50 msgr2=0x7f4a180cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 -- 172.21.15.64:0/1595940913 shutdown_connections 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 -- 172.21.15.64:0/1595940913 wait complete. 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 Processor -- start 2023-09-27T23:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.290+0000 7f4a1759e700 1 -- start start 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a040051c0 0x7f4a1815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 0x7f4a1816b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181697e0 0x7f4a1815c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1813f670 con 0x7f4a18169cf0 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1813f950 con 0x7f4a181697e0 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1813fc30 con 0x7f4a040051c0 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 0x7f4a1816b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 0x7f4a1816b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36048/0 (socket says 172.21.15.64:36048) 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 -- 172.21.15.64:0/414734889 learned_addr learned my addr 172.21.15.64:0/414734889 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1659c700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a040051c0 0x7f4a1815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a16d9d700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181697e0 0x7f4a1815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 -- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a040051c0 msgr2=0x7f4a1815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a040051c0 0x7f4a1815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 -- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181697e0 msgr2=0x7f4a1815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181697e0 0x7f4a1815c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 -- 172.21.15.64:0/414734889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1813ff10 con 0x7f4a18169cf0 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0ffff700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 0x7f4a1816b7c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e780 tx=0x7f4a00000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00010490 con 0x7f4a18169cf0 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00010040 con 0x7f4a18169cf0 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a181401f0 con 0x7f4a18169cf0 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00010860 con 0x7f4a18169cf0 2023-09-27T23:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.294+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1816bd00 con 0x7f4a18169cf0 2023-09-27T23:33:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.298+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18140410 con 0x7f4a18169cf0 2023-09-27T23:33:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.298+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4a00010c10 con 0x7f4a18169cf0 2023-09-27T23:33:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.298+0000 7f4a0e7fc700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49f8060a70 0x7f49f8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.298+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f4a0007c450 con 0x7f4a18169cf0 2023-09-27T23:33:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.298+0000 7f4a1659c700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49f8060a70 0x7f49f8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.298+0000 7f4a1659c700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49f8060a70 0x7f49f8062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009170 tx=0x7f4a08008040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.306+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4a0004c3d0 con 0x7f4a18169cf0 2023-09-27T23:33:56.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.510+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4a1804be40 con 0x7f4a18169cf0 2023-09-27T23:33:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.526+0000 7f4a0e7fc700 1 -- 172.21.15.64:0/414734889 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4a000190c0 con 0x7f4a18169cf0 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49f8060a70 msgr2=0x7f49f8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49f8060a70 0x7f49f8062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009170 tx=0x7f4a08008040 comp rx=0 tx=0).stop 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 msgr2=0x7f4a1816b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 0x7f4a1816b7c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e780 tx=0x7f4a00000f60 comp rx=0 tx=0).stop 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 shutdown_connections 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f49f8060a70 0x7f49f8062f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181697e0 0x7f4a1815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a18169cf0 0x7f4a1816b7c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 --2- 172.21.15.64:0/414734889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a040051c0 0x7f4a1815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 >> 172.21.15.64:0/414734889 conn(0x7f4a18067b50 msgr2=0x7f4a1806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 shutdown_connections 2023-09-27T23:33:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.534+0000 7f4a1759e700 1 -- 172.21.15.64:0/414734889 wait complete. 2023-09-27T23:33:56.544 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 20 and data pool 21 2023-09-27T23:33:56.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:56.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 Processor -- start 2023-09-27T23:33:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 -- start start 2023-09-27T23:33:56.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a780ad000 0x7f7a780c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a7815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780add10 con 0x7f7a780acc30 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a780ad450 con 0x7f7a780ad000 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7d01c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780cc920 con 0x7f7a780c8b60 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a7815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a7815a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:55824/0 (socket says 172.21.15.64:55824) 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 -- 172.21.15.64:0/3113385207 learned_addr learned my addr 172.21.15.64:0/3113385207 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:56.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a777fe700 1 --2- 172.21.15.64:0/3113385207 >> v2:172.21.15.123:3300/0 conn(0x7f7a780ad000 0x7f7a780c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 -- 172.21.15.64:0/3113385207 >> v2:172.21.15.123:3300/0 conn(0x7f7a780ad000 msgr2=0x7f7a780c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 --2- 172.21.15.64:0/3113385207 >> v2:172.21.15.123:3300/0 conn(0x7f7a780ad000 0x7f7a780c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 -- 172.21.15.64:0/3113385207 >> v1:172.21.15.123:6789/0 conn(0x7f7a780acc30 legacy=0x7f7a780ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.650+0000 7f7a7c81b700 1 -- 172.21.15.64:0/3113385207 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780c9290 con 0x7f7a780c8b60 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7c81b700 1 --2- 172.21.15.64:0/3113385207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a7815a6f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7a680099b0 tx=0x7f7a68009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc05c905cfc8bbe server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a76ffd700 1 -- 172.21.15.64:0/3113385207 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68009ea0 con 0x7f7a780c8b60 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a76ffd700 1 -- 172.21.15.64:0/3113385207 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800b8f0 con 0x7f7a780c8b60 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a76ffd700 1 -- 172.21.15.64:0/3113385207 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800bc90 con 0x7f7a780c8b60 2023-09-27T23:33:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- 172.21.15.64:0/3113385207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 msgr2=0x7f7a7815a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/3113385207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a7815a6f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7a680099b0 tx=0x7f7a68009250 comp rx=0 tx=0).stop 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- 172.21.15.64:0/3113385207 shutdown_connections 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/3113385207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a7815a6f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/3113385207 >> v2:172.21.15.123:3300/0 conn(0x7f7a780ad000 0x7f7a780c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- 172.21.15.64:0/3113385207 >> 172.21.15.64:0/3113385207 conn(0x7f7a780cdbd0 msgr2=0x7f7a780cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- 172.21.15.64:0/3113385207 shutdown_connections 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- 172.21.15.64:0/3113385207 wait complete. 2023-09-27T23:33:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 Processor -- start 2023-09-27T23:33:56.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- start start 2023-09-27T23:33:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a780acc30 0x7f7a780b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a780ad000 0x7f7a780c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a781452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78157c60 con 0x7f7a780ad000 2023-09-27T23:33:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78158ac0 con 0x7f7a780acc30 2023-09-27T23:33:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.654+0000 7f7a7d01c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78157eb0 con 0x7f7a780c8b60 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a780ad000 0x7f7a780c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7c81b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a781452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7c81b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a781452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:55838/0 (socket says 172.21.15.64:55838) 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7c81b700 1 -- 172.21.15.64:0/1874463328 learned_addr learned my addr 172.21.15.64:0/1874463328 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a77fff700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a780acc30 0x7f7a780b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 -- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 msgr2=0x7f7a781452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a781452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 -- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a780acc30 msgr2=0x7f7a780b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a780acc30 0x7f7a780b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 -- 172.21.15.64:0/1874463328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680094f0 con 0x7f7a780ad000 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7c81b700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a781452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a777fe700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a780ad000 0x7f7a780c41c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00ea80 tx=0x7f7a6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00eec0 con 0x7f7a780ad000 2023-09-27T23:33:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c00c880 con 0x7f7a780ad000 2023-09-27T23:33:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a78158190 con 0x7f7a780ad000 2023-09-27T23:33:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00cc00 con 0x7f7a780ad000 2023-09-27T23:33:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78158750 con 0x7f7a780ad000 2023-09-27T23:33:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.658+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a78158c50 con 0x7f7a780ad000 2023-09-27T23:33:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.662+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7a6c013470 con 0x7f7a780ad000 2023-09-27T23:33:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.662+0000 7f7a5ffff700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a64060a70 0x7f7a64062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.662+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7a6c07d370 con 0x7f7a780ad000 2023-09-27T23:33:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.662+0000 7f7a77fff700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a64060a70 0x7f7a64062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.662+0000 7f7a77fff700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a64060a70 0x7f7a64062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7a70007fd0 tx=0x7f7a70001650 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.670+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a6c050d90 con 0x7f7a780ad000 2023-09-27T23:33:56.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.830+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a7804be40 con 0x7f7a780ad000 2023-09-27T23:33:56.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a5ffff700 1 -- 172.21.15.64:0/1874463328 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f7a6c050920 con 0x7f7a780ad000 2023-09-27T23:33:56.837 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:56.837 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":156,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:56.513967+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:55.078420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:55.503601+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:56.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a64060a70 msgr2=0x7f7a64062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a64060a70 0x7f7a64062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7a70007fd0 tx=0x7f7a70001650 comp rx=0 tx=0).stop 2023-09-27T23:33:56.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a780ad000 msgr2=0x7f7a780c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a780ad000 0x7f7a780c41c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00ea80 tx=0x7f7a6c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 shutdown_connections 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7a64060a70 0x7f7a64062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a780c8b60 0x7f7a781452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a780ad000 0x7f7a780c41c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 --2- 172.21.15.64:0/1874463328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a780acc30 0x7f7a780b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 >> 172.21.15.64:0/1874463328 conn(0x7f7a780cdbd0 msgr2=0x7f7a780c2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 shutdown_connections 2023-09-27T23:33:56.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.834+0000 7f7a7d01c700 1 -- 172.21.15.64:0/1874463328 wait complete. 2023-09-27T23:33:56.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:56.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d97d34700 1 Processor -- start 2023-09-27T23:33:56.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d97d34700 1 -- start start 2023-09-27T23:33:56.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d97d34700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d9015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d97d34700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900c7190 con 0x7f9d900c15b0 2023-09-27T23:33:56.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d97d34700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900c1b50 con 0x7f9d900aeba0 2023-09-27T23:33:56.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d97d34700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900c1e30 con 0x7f9d9015d320 2023-09-27T23:33:56.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d952cf700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9d900c15b0 0x7f9d9006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34830/0 (socket says 172.21.15.64:34830) 2023-09-27T23:33:56.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d952cf700 1 -- 172.21.15.64:0/178019842 learned_addr learned my addr 172.21.15.64:0/178019842 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d9015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531420771 0 0) 0x7f9d900c7190 con 0x7f9d900c15b0 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7c003570 con 0x7f9d900c15b0 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479818658 0 0) 0x7f9d900c1b50 con 0x7f9d900aeba0 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d900c7190 con 0x7f9d900aeba0 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275467758 0 0) 0x7f9d7c003570 con 0x7f9d900c15b0 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 msgr2=0x7f9d9015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d9015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 >> v1:172.21.15.123:6789/0 conn(0x7f9d900aeba0 legacy=0x7f9d900680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900c6e20 con 0x7f9d900c15b0 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f9d84001470 con 0x7f9d900c15b0 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 >> v1:172.21.15.64:6789/0 conn(0x7f9d900c15b0 legacy=0x7f9d9006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d7c003c20 0x7f9d7c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d7c004d50 0x7f9d7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900c1e30 con 0x7f9d7c004d50 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c00bbf0 con 0x7f9d7c003c20 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c00bd60 con 0x7f9d9015d320 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.954+0000 7f9d962d1700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d7c004d50 0x7f9d7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d7c003c20 0x7f9d7c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d952cf700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 -- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 msgr2=0x7f9d7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d7c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 -- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d7c003c20 msgr2=0x7f9d7c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d7c003c20 0x7f9d7c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 -- 172.21.15.64:0/178019842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c003570 con 0x7f9d7c004d50 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d952cf700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d9015d320 0x7f9d7c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d7c003c20 0x7f9d7c004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d7c004d50 0x7f9d7c0051c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9d880020f0 tx=0x7f9d880017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af497840c3c8c6c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88001ab0 con 0x7f9d7c004d50 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8800c8b0 con 0x7f9d7c004d50 2023-09-27T23:33:56.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d94ace700 1 -- 172.21.15.64:0/178019842 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800cc70 con 0x7f9d7c004d50 2023-09-27T23:33:56.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d97d34700 1 -- 172.21.15.64:0/178019842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d7c004d50 msgr2=0x7f9d7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d97d34700 1 --2- 172.21.15.64:0/178019842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d7c004d50 0x7f9d7c0051c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9d880020f0 tx=0x7f9d880017a0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d962d1700 1 -- 172.21.15.64:0/178019842 reap_dead start 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d97d34700 1 -- 172.21.15.64:0/178019842 shutdown_connections 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d97d34700 1 -- 172.21.15.64:0/178019842 >> 172.21.15.64:0/178019842 conn(0x7f9d900c8850 msgr2=0x7f9d900c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d97d34700 1 -- 172.21.15.64:0/178019842 shutdown_connections 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.958+0000 7f9d97d34700 1 -- 172.21.15.64:0/178019842 wait complete. 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 Processor -- start 2023-09-27T23:33:56.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 -- start start 2023-09-27T23:33:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d7c004d50 0x7f9d901581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 0x7f9d900cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d901609d0 0x7f9d90160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b14e0 con 0x7f9d900cae00 2023-09-27T23:33:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900cb790 con 0x7f9d901609d0 2023-09-27T23:33:56.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d952cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 0x7f9d900cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d97d34700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b0860 con 0x7f9d7c004d50 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d952cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 0x7f9d900cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36074/0 (socket says 172.21.15.64:36074) 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d952cf700 1 -- 172.21.15.64:0/1075710572 learned_addr learned my addr 172.21.15.64:0/1075710572 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d7c004d50 0x7f9d901581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.962+0000 7f9d962d1700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d901609d0 0x7f9d90160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d952cf700 1 -- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d7c004d50 msgr2=0x7f9d901581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d952cf700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d7c004d50 0x7f9d901581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d952cf700 1 -- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d901609d0 msgr2=0x7f9d90160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d952cf700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d901609d0 0x7f9d90160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d952cf700 1 -- 172.21.15.64:0/1075710572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900b0b40 con 0x7f9d900cae00 2023-09-27T23:33:56.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d962d1700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d901609d0 0x7f9d90160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d7c004d50 0x7f9d901581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d952cf700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 0x7f9d900cb250 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400e7b0 tx=0x7f9d84000cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84012500 con 0x7f9d900cae00 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84010040 con 0x7f9d900cae00 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900b0e20 con 0x7f9d900cae00 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84012890 con 0x7f9d900cae00 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d901586e0 con 0x7f9d900cae00 2023-09-27T23:33:56.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d90071120 con 0x7f9d900cae00 2023-09-27T23:33:56.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.966+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9d84012c40 con 0x7f9d900cae00 2023-09-27T23:33:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.970+0000 7f9d81ffb700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d74060aa0 0x7f9d74062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.970+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9d8407c570 con 0x7f9d900cae00 2023-09-27T23:33:56.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.970+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d74060aa0 0x7f9d74062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:56.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.970+0000 7f9d95ad0700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d74060aa0 0x7f9d74062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d50 tx=0x7f9d8c008040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:56.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:56.978+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9d8404abb0 con 0x7f9d900cae00 2023-09-27T23:33:57.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.142+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d900642a0 con 0x7f9d900cae00 2023-09-27T23:33:57.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.146+0000 7f9d81ffb700 1 -- 172.21.15.64:0/1075710572 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f9d84019090 con 0x7f9d900cae00 2023-09-27T23:33:57.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:57.150 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":156,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:56.513967+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:55.078420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:55.503601+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.146+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d74060aa0 msgr2=0x7f9d74062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.146+0000 7f9d97d34700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d74060aa0 0x7f9d74062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d50 tx=0x7f9d8c008040 comp rx=0 tx=0).stop 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 msgr2=0x7f9d900cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 0x7f9d900cb250 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400e7b0 tx=0x7f9d84000cc0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 shutdown_connections 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9d74060aa0 0x7f9d74062f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d901609d0 0x7f9d90160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d900cae00 0x7f9d900cb250 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 --2- 172.21.15.64:0/1075710572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9d7c004d50 0x7f9d901581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 >> 172.21.15.64:0/1075710572 conn(0x7f9d900c8850 msgr2=0x7f9d901638d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 shutdown_connections 2023-09-27T23:33:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.150+0000 7f9d97d34700 1 -- 172.21.15.64:0/1075710572 wait complete. 2023-09-27T23:33:57.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:33:57.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.270+0000 7f24cffff700 1 Processor -- start 2023-09-27T23:33:57.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.270+0000 7f24cffff700 1 -- start start 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24d0155520 0x7f24d00c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00ae6c0 0x7f24d006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d00ad020 con 0x7f24d00c7670 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d00c81e0 con 0x7f24d0155520 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00aca10 con 0x7f24d00ae6c0 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ceffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24d0155520 0x7f24d00c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cf7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f24d00c7670 0x7f24d0159100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34838/0 (socket says 172.21.15.64:34838) 2023-09-27T23:33:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24cf7fe700 1 -- 172.21.15.64:0/2428698392 learned_addr learned my addr 172.21.15.64:0/2428698392 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:57.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24c7fff700 1 --2- 172.21.15.64:0/2428698392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00ae6c0 0x7f24d006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ce7fc700 1 -- 172.21.15.64:0/2428698392 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082478009 0 0) 0x7f24d00ad020 con 0x7f24d00c7670 2023-09-27T23:33:57.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ce7fc700 1 -- 172.21.15.64:0/2428698392 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24bc004680 con 0x7f24d00c7670 2023-09-27T23:33:57.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ceffd700 1 -- 172.21.15.64:0/2428698392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00ae6c0 msgr2=0x7f24d006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ceffd700 1 --2- 172.21.15.64:0/2428698392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00ae6c0 0x7f24d006ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ceffd700 1 -- 172.21.15.64:0/2428698392 >> v1:172.21.15.64:6789/0 conn(0x7f24d00c7670 legacy=0x7f24d0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ceffd700 1 -- 172.21.15.64:0/2428698392 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00c7c20 con 0x7f24d0155520 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ceffd700 1 --2- 172.21.15.64:0/2428698392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d00c7130 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f24c00098e0 tx=0x7f24c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e861b147a9f895ab server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ce7fc700 1 -- 172.21.15.64:0/2428698392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0011630 con 0x7f24d0155520 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ce7fc700 1 -- 172.21.15.64:0/2428698392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0011790 con 0x7f24d0155520 2023-09-27T23:33:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.274+0000 7f24ce7fc700 1 -- 172.21.15.64:0/2428698392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0011b30 con 0x7f24d0155520 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 -- 172.21.15.64:0/2428698392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 msgr2=0x7f24d00c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 --2- 172.21.15.64:0/2428698392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d00c7130 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f24c00098e0 tx=0x7f24c00091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 -- 172.21.15.64:0/2428698392 shutdown_connections 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 --2- 172.21.15.64:0/2428698392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00ae6c0 0x7f24d006ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 --2- 172.21.15.64:0/2428698392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d00c7130 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 -- 172.21.15.64:0/2428698392 >> 172.21.15.64:0/2428698392 conn(0x7f24d00c8cd0 msgr2=0x7f24d00c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 -- 172.21.15.64:0/2428698392 shutdown_connections 2023-09-27T23:33:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 -- 172.21.15.64:0/2428698392 wait complete. 2023-09-27T23:33:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 Processor -- start 2023-09-27T23:33:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.278+0000 7f24cffff700 1 -- start start 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d00ae6c0 0x7f24d0167de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00c7670 0x7f24d0165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d0157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00afc20 con 0x7f24d0155520 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b0760 con 0x7f24d00ae6c0 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00afe30 con 0x7f24d00c7670 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d0157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d0157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36098/0 (socket says 172.21.15.64:36098) 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 -- 172.21.15.64:0/3104543859 learned_addr learned my addr 172.21.15.64:0/3104543859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00c7670 0x7f24d0165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24ceffd700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d00ae6c0 0x7f24d0167de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 -- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00c7670 msgr2=0x7f24d0165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00c7670 0x7f24d0165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 -- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d00ae6c0 msgr2=0x7f24d0167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d00ae6c0 0x7f24d0167de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 -- 172.21.15.64:0/3104543859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0009420 con 0x7f24d0155520 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24c7fff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00c7670 0x7f24d0165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cf7fe700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d0157bc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f24c8001460 tx=0x7f24c8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800a890 con 0x7f24d0155520 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c800a9f0 con 0x7f24d0155520 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800ad70 con 0x7f24d0155520 2023-09-27T23:33:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d00b0110 con 0x7f24d0155520 2023-09-27T23:33:57.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0166430 con 0x7f24d0155520 2023-09-27T23:33:57.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.282+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d01666b0 con 0x7f24d0155520 2023-09-27T23:33:57.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.286+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f24c8007300 con 0x7f24d0155520 2023-09-27T23:33:57.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.286+0000 7f24c6ffd700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24b0060a70 0x7f24b0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.286+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f24c807dfe0 con 0x7f24d0155520 2023-09-27T23:33:57.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.286+0000 7f24ceffd700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24b0060a70 0x7f24b0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.286+0000 7f24ceffd700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24b0060a70 0x7f24b0062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24c00091e0 tx=0x7f24c0017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:57.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.290+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f24c804e040 con 0x7f24d0155520 2023-09-27T23:33:57.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.486+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f24d00b08d0 con 0x7f24d0155520 2023-09-27T23:33:57.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.522+0000 7f24c6ffd700 1 -- 172.21.15.64:0/3104543859 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7f24c8051660 con 0x7f24d0155520 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24b0060a70 msgr2=0x7f24b0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24b0060a70 0x7f24b0062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24c00091e0 tx=0x7f24c0017000 comp rx=0 tx=0).stop 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 msgr2=0x7f24d0157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d0157bc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f24c8001460 tx=0x7f24c8001ae0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 shutdown_connections 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f24b0060a70 0x7f24b0062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d0155520 0x7f24d0157bc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d00c7670 0x7f24d0165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 --2- 172.21.15.64:0/3104543859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d00ae6c0 0x7f24d0167de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 >> 172.21.15.64:0/3104543859 conn(0x7f24d00c8cd0 msgr2=0x7f24d006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 shutdown_connections 2023-09-27T23:33:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.526+0000 7f24cffff700 1 -- 172.21.15.64:0/3104543859 wait complete. 2023-09-27T23:33:57.545 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:33:57.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:33:57.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.650+0000 7f7429695700 1 Processor -- start 2023-09-27T23:33:57.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.650+0000 7f7429695700 1 -- start start 2023-09-27T23:33:57.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f74240ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f742406db70 0x7f74240ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74240c68a0 con 0x7f7424155e60 2023-09-27T23:33:57.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240c6e60 con 0x7f742406db70 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74240ae4d0 con 0x7f7424155a90 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f74237fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f742406db70 0x7f74240ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7428e94700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7424155e60 0x7f74240adca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34846/0 (socket says 172.21.15.64:34846) 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7428e94700 1 -- 172.21.15.64:0/1815990644 learned_addr learned my addr 172.21.15.64:0/1815990644 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7423fff700 1 --2- 172.21.15.64:0/1815990644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f74240ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7422ffd700 1 -- 172.21.15.64:0/1815990644 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310882981 0 0) 0x7f74240c68a0 con 0x7f7424155e60 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7422ffd700 1 -- 172.21.15.64:0/1815990644 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f740c003570 con 0x7f7424155e60 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f74237fe700 1 -- 172.21.15.64:0/1815990644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 msgr2=0x7f74240ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f74237fe700 1 --2- 172.21.15.64:0/1815990644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f74240ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f74237fe700 1 -- 172.21.15.64:0/1815990644 >> v1:172.21.15.64:6789/0 conn(0x7f7424155e60 legacy=0x7f74240adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f74237fe700 1 -- 172.21.15.64:0/1815990644 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240ae7b0 con 0x7f742406db70 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f74237fe700 1 --2- 172.21.15.64:0/1815990644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f742406db70 0x7f74240ad760 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f74180098e0 tx=0x7f74180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2061f2697bc513 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7422ffd700 1 -- 172.21.15.64:0/1815990644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418011630 con 0x7f742406db70 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7422ffd700 1 -- 172.21.15.64:0/1815990644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7418011790 con 0x7f742406db70 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7422ffd700 1 -- 172.21.15.64:0/1815990644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418011b30 con 0x7f742406db70 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 -- 172.21.15.64:0/1815990644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f742406db70 msgr2=0x7f74240ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.654+0000 7f7429695700 1 --2- 172.21.15.64:0/1815990644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f742406db70 0x7f74240ad760 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f74180098e0 tx=0x7f74180091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 -- 172.21.15.64:0/1815990644 shutdown_connections 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 --2- 172.21.15.64:0/1815990644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f742406db70 0x7f74240ad760 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 --2- 172.21.15.64:0/1815990644 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f74240ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 -- 172.21.15.64:0/1815990644 >> 172.21.15.64:0/1815990644 conn(0x7f74240c8cd0 msgr2=0x7f74240c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:57.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 -- 172.21.15.64:0/1815990644 shutdown_connections 2023-09-27T23:33:57.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 -- 172.21.15.64:0/1815990644 wait complete. 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 Processor -- start 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.658+0000 7f7429695700 1 -- start start 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f742406db70 0x7f74241499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f742414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 0x7f742414a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240c4c10 con 0x7f7424155e60 2023-09-27T23:33:57.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240c4600 con 0x7f742406db70 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742414c700 con 0x7f7424155a90 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 0x7f742414a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 0x7f742414a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36116/0 (socket says 172.21.15.64:36116) 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 -- 172.21.15.64:0/2997822579 learned_addr learned my addr 172.21.15.64:0/2997822579 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7423fff700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f742406db70 0x7f74241499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f74237fe700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f742414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 -- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 msgr2=0x7f742414be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f742414be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 -- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f742406db70 msgr2=0x7f74241499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f742406db70 0x7f74241499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 -- 172.21.15.64:0/2997822579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009420 con 0x7f7424155e60 2023-09-27T23:33:57.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f74237fe700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f742414be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:33:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7428e94700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 0x7f742414a2d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f741c001460 tx=0x7f741c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00a8a0 con 0x7f7424155e60 2023-09-27T23:33:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c00aa00 con 0x7f7424155e60 2023-09-27T23:33:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00ad80 con 0x7f7424155e60 2023-09-27T23:33:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74240c48a0 con 0x7f7424155e60 2023-09-27T23:33:57.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7424149f30 con 0x7f7424155e60 2023-09-27T23:33:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74240c4d80 con 0x7f7424155e60 2023-09-27T23:33:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.662+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f741c007300 con 0x7f7424155e60 2023-09-27T23:33:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.666+0000 7f740bfff700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7410060a70 0x7f7410062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.666+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f741c07d590 con 0x7f7424155e60 2023-09-27T23:33:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.666+0000 7f7423fff700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7410060a70 0x7f7410062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:57.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.666+0000 7f7423fff700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7410060a70 0x7f7410062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7414009db0 tx=0x7f74140091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:57.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.670+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f741c04d5b0 con 0x7f7424155e60 2023-09-27T23:33:57.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:57.866+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f742404be40 con 0x7f7424155e60 2023-09-27T23:33:58.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.534+0000 7f740bfff700 1 -- 172.21.15.64:0/2997822579 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7f741c050bd0 con 0x7f7424155e60 2023-09-27T23:33:58.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7410060a70 msgr2=0x7f7410062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7410060a70 0x7f7410062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7414009db0 tx=0x7f74140091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 msgr2=0x7f742414a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 0x7f742414a2d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f741c001460 tx=0x7f741c001a40 comp rx=0 tx=0).stop 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 shutdown_connections 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7410060a70 0x7f7410062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155e60 0x7f742414a2d0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155a90 0x7f742414be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 --2- 172.21.15.64:0/2997822579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f742406db70 0x7f74241499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 >> 172.21.15.64:0/2997822579 conn(0x7f74240c8cd0 msgr2=0x7f742414f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 shutdown_connections 2023-09-27T23:33:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.542+0000 7f7429695700 1 -- 172.21.15.64:0/2997822579 wait complete. 2023-09-27T23:33:58.562 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:33:58.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:33:58.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 Processor -- start 2023-09-27T23:33:58.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 -- start start 2023-09-27T23:33:58.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea94158f70 0x7fea94067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:58.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94069100 0x7fea940cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:58.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea94068c70 con 0x7fea94159340 2023-09-27T23:33:58.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea94068060 con 0x7fea94158f70 2023-09-27T23:33:58.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea9b9b4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea94068340 con 0x7fea94069100 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea99f51700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fea94159340 0x7fea940cb3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34860/0 (socket says 172.21.15.64:34860) 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea99f51700 1 -- 172.21.15.64:0/783715142 learned_addr learned my addr 172.21.15.64:0/783715142 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea98f4f700 1 --2- 172.21.15.64:0/783715142 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94069100 0x7fea940cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea99750700 1 --2- 172.21.15.64:0/783715142 >> v2:172.21.15.123:3300/0 conn(0x7fea94158f70 0x7fea94067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea98f4f700 1 -- 172.21.15.64:0/783715142 >> v2:172.21.15.123:3300/0 conn(0x7fea94158f70 msgr2=0x7fea94067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea98f4f700 1 --2- 172.21.15.64:0/783715142 >> v2:172.21.15.123:3300/0 conn(0x7fea94158f70 0x7fea94067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea98f4f700 1 -- 172.21.15.64:0/783715142 >> v1:172.21.15.64:6789/0 conn(0x7fea94159340 legacy=0x7fea940cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea8bfff700 1 -- 172.21.15.64:0/783715142 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895131528 0 0) 0x7fea94068c70 con 0x7fea94159340 2023-09-27T23:33:58.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea98f4f700 1 -- 172.21.15.64:0/783715142 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94068620 con 0x7fea94069100 2023-09-27T23:33:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea98f4f700 1 --2- 172.21.15.64:0/783715142 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94069100 0x7fea940cf7e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009e60 tx=0x7fea8c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa61800924e9200 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea8bfff700 1 -- 172.21.15.64:0/783715142 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00e070 con 0x7fea94069100 2023-09-27T23:33:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea8bfff700 1 -- 172.21.15.64:0/783715142 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c00bb60 con 0x7fea94069100 2023-09-27T23:33:58.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.678+0000 7fea8bfff700 1 -- 172.21.15.64:0/783715142 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c012430 con 0x7fea94069100 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 -- 172.21.15.64:0/783715142 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94069100 msgr2=0x7fea940cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/783715142 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94069100 0x7fea940cf7e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009e60 tx=0x7fea8c009780 comp rx=0 tx=0).stop 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 -- 172.21.15.64:0/783715142 shutdown_connections 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/783715142 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94069100 0x7fea940cf7e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/783715142 >> v2:172.21.15.123:3300/0 conn(0x7fea94158f70 0x7fea94067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 -- 172.21.15.64:0/783715142 >> 172.21.15.64:0/783715142 conn(0x7fea940ccc40 msgr2=0x7fea940cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:58.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 -- 172.21.15.64:0/783715142 shutdown_connections 2023-09-27T23:33:58.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 -- 172.21.15.64:0/783715142 wait complete. 2023-09-27T23:33:58.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 Processor -- start 2023-09-27T23:33:58.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.682+0000 7fea9b9b4700 1 -- start start 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea94069100 0x7fea940b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94158f70 0x7fea9415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 0x7fea9415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940b5400 con 0x7fea94159340 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940b47f0 con 0x7fea94069100 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea940b4ad0 con 0x7fea94158f70 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 0x7fea9415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 0x7fea9415dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36128/0 (socket says 172.21.15.64:36128) 2023-09-27T23:33:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea98f4f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94158f70 0x7fea9415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:58.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99750700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea94069100 0x7fea940b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:58.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 -- 172.21.15.64:0/3852056399 learned_addr learned my addr 172.21.15.64:0/3852056399 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 -- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94158f70 msgr2=0x7fea9415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94158f70 0x7fea9415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 -- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea94069100 msgr2=0x7fea940b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea94069100 0x7fea940b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 -- 172.21.15.64:0/3852056399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c009a20 con 0x7fea94159340 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea98f4f700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94158f70 0x7fea9415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea99f51700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 0x7fea9415dc20 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c530 tx=0x7fea84007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:58.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84005b30 con 0x7fea94159340 2023-09-27T23:33:58.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84005c90 con 0x7fea94159340 2023-09-27T23:33:58.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea940b4db0 con 0x7fea94159340 2023-09-27T23:33:58.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84007320 con 0x7fea94159340 2023-09-27T23:33:58.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9414dee0 con 0x7fea94159340 2023-09-27T23:33:58.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.686+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea940b5570 con 0x7fea94159340 2023-09-27T23:33:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.690+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fea8401b3b0 con 0x7fea94159340 2023-09-27T23:33:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.690+0000 7fea897fa700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fea80060a70 0x7fea80062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.690+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fea8407c600 con 0x7fea94159340 2023-09-27T23:33:58.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.690+0000 7fea99750700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fea80060a70 0x7fea80062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:58.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.690+0000 7fea99750700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fea80060a70 0x7fea80062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea90009db0 tx=0x7fea900091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:58.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.698+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fea8404c590 con 0x7fea94159340 2023-09-27T23:33:58.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.898+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea94071120 con 0x7fea94159340 2023-09-27T23:33:58.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.898+0000 7fea897fa700 1 -- 172.21.15.64:0/3852056399 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fea8404fbb0 con 0x7fea94159340 2023-09-27T23:33:58.904 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:58.904 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:56.514725+0000","modified":"2023-09-27T23:33:58.531554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3161207024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3161207024},{"type":"v1","addr":"172.21.15.64:6835","nonce":3161207024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fea80060a70 msgr2=0x7fea80062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fea80060a70 0x7fea80062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea90009db0 tx=0x7fea900091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 msgr2=0x7fea9415dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 0x7fea9415dc20 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c530 tx=0x7fea84007f90 comp rx=0 tx=0).stop 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 shutdown_connections 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fea80060a70 0x7fea80062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea94159340 0x7fea9415dc20 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea94158f70 0x7fea9415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 --2- 172.21.15.64:0/3852056399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea94069100 0x7fea940b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 >> 172.21.15.64:0/3852056399 conn(0x7fea940ccc40 msgr2=0x7fea940c5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 shutdown_connections 2023-09-27T23:33:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:58.902+0000 7fea9b9b4700 1 -- 172.21.15.64:0/3852056399 wait complete. 2023-09-27T23:33:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 60 2023-09-27T23:33:58.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:33:59.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 Processor -- start 2023-09-27T23:33:59.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- start start 2023-09-27T23:33:59.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f06780c15b0 0x7f067815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780c7190 con 0x7f06780aeba0 2023-09-27T23:33:59.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780c1b50 con 0x7f067815d320 2023-09-27T23:33:59.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780aefd0 con 0x7f06780c15b0 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067d71f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f06780aeba0 0x7f06780680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34864/0 (socket says 172.21.15.64:34864) 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067d71f700 1 -- 172.21.15.64:0/3938714188 learned_addr learned my addr 172.21.15.64:0/3938714188 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f0677fff700 1 --2- 172.21.15.64:0/3938714188 >> v2:172.21.15.123:3300/0 conn(0x7f06780c15b0 0x7f067815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f06777fe700 1 -- 172.21.15.64:0/3938714188 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990241093 0 0) 0x7f06780c7190 con 0x7f06780aeba0 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f06777fe700 1 -- 172.21.15.64:0/3938714188 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0664003570 con 0x7f06780aeba0 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f0677fff700 1 -- 172.21.15.64:0/3938714188 >> v1:172.21.15.123:6789/0 conn(0x7f067815d320 legacy=0x7f067806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f0677fff700 1 -- 172.21.15.64:0/3938714188 >> v1:172.21.15.64:6789/0 conn(0x7f06780aeba0 legacy=0x7f06780680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f0677fff700 1 -- 172.21.15.64:0/3938714188 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780c1e30 con 0x7f06780c15b0 2023-09-27T23:33:59.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f0677fff700 1 --2- 172.21.15.64:0/3938714188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06780c15b0 0x7f067815d6f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0660009e60 tx=0x7f0660009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5c03199232a711 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f06777fe700 1 -- 172.21.15.64:0/3938714188 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000e070 con 0x7f06780c15b0 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f06777fe700 1 -- 172.21.15.64:0/3938714188 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000bb20 con 0x7f06780c15b0 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f06777fe700 1 -- 172.21.15.64:0/3938714188 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000bea0 con 0x7f06780c15b0 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- 172.21.15.64:0/3938714188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06780c15b0 msgr2=0x7f067815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 --2- 172.21.15.64:0/3938714188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06780c15b0 0x7f067815d6f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0660009e60 tx=0x7f0660009780 comp rx=0 tx=0).stop 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- 172.21.15.64:0/3938714188 shutdown_connections 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 --2- 172.21.15.64:0/3938714188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06780c15b0 0x7f067815d6f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.026+0000 7f067f182700 1 -- 172.21.15.64:0/3938714188 >> 172.21.15.64:0/3938714188 conn(0x7f06780c8850 msgr2=0x7f06780c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 -- 172.21.15.64:0/3938714188 shutdown_connections 2023-09-27T23:33:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 -- 172.21.15.64:0/3938714188 wait complete. 2023-09-27T23:33:59.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 Processor -- start 2023-09-27T23:33:59.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 -- start start 2023-09-27T23:33:59.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06780aeba0 0x7f067815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06780c15b0 0x7f06780c3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067815d320 0x7f06780c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067815e100 con 0x7f06780aeba0 2023-09-27T23:33:59.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067815e3c0 con 0x7f067815d320 2023-09-27T23:33:59.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067f182700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067815e6a0 con 0x7f06780c15b0 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067cf1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06780aeba0 0x7f067815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f0677fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06780c15b0 0x7f06780c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067cf1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06780aeba0 0x7f067815ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36136/0 (socket says 172.21.15.64:36136) 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067cf1e700 1 -- 172.21.15.64:0/3798150005 learned_addr learned my addr 172.21.15.64:0/3798150005 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f067d71f700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067815d320 0x7f06780c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f0677fff700 1 -- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067815d320 msgr2=0x7f06780c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f0677fff700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067815d320 0x7f06780c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f0677fff700 1 -- 172.21.15.64:0/3798150005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06780aeba0 msgr2=0x7f067815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f0677fff700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06780aeba0 0x7f067815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.030+0000 7f0677fff700 1 -- 172.21.15.64:0/3798150005 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660009a20 con 0x7f06780c15b0 2023-09-27T23:33:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0677fff700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06780c15b0 0x7f06780c3730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0660015000 tx=0x7f0660018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660012d00 con 0x7f06780c15b0 2023-09-27T23:33:59.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067815e980 con 0x7f06780c15b0 2023-09-27T23:33:59.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0660012e80 con 0x7f06780c15b0 2023-09-27T23:33:59.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06600276c0 con 0x7f06780c15b0 2023-09-27T23:33:59.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067815b1d0 con 0x7f06780c15b0 2023-09-27T23:33:59.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067815e500 con 0x7f06780c15b0 2023-09-27T23:33:59.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0660024070 con 0x7f06780c15b0 2023-09-27T23:33:59.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0674ff9700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0658060aa0 0x7f0658062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.034+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f066007fa30 con 0x7f06780c15b0 2023-09-27T23:33:59.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.038+0000 7f067cf1e700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0658060aa0 0x7f0658062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.038+0000 7f067cf1e700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0658060aa0 0x7f0658062f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06700078a0 tx=0x7f0670007830 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.046+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f066004fa60 con 0x7f06780c15b0 2023-09-27T23:33:59.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.214+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f067815e500 con 0x7f06780c15b0 2023-09-27T23:33:59.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.214+0000 7f0674ff9700 1 -- 172.21.15.64:0/3798150005 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+17085 (secure 0 0 0) 0x7f067815e500 con 0x7f06780c15b0 2023-09-27T23:33:59.219 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:59.220 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":157,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:33:57.518833+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:55.078420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:55.503601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0658060aa0 msgr2=0x7f0658062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0658060aa0 0x7f0658062f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06700078a0 tx=0x7f0670007830 comp rx=0 tx=0).stop 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06780c15b0 msgr2=0x7f06780c3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06780c15b0 0x7f06780c3730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0660015000 tx=0x7f0660018000 comp rx=0 tx=0).stop 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 shutdown_connections 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0658060aa0 0x7f0658062f50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067815d320 0x7f06780c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06780c15b0 0x7f06780c3730 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 --2- 172.21.15.64:0/3798150005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06780aeba0 0x7f067815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 >> 172.21.15.64:0/3798150005 conn(0x7f06780c8850 msgr2=0x7f06780c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:59.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 shutdown_connections 2023-09-27T23:33:59.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.218+0000 7f067f182700 1 -- 172.21.15.64:0/3798150005 wait complete. 2023-09-27T23:33:59.234 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:33:59.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.334+0000 7f908359e700 1 Processor -- start 2023-09-27T23:33:59.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.334+0000 7f908359e700 1 -- start start 2023-09-27T23:33:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9084158e70 0x7f90840afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90840ca5a0 con 0x7f90840cc470 2023-09-27T23:33:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908406dac0 con 0x7f908406a540 2023-09-27T23:33:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90840b0260 con 0x7f9084158e70 2023-09-27T23:33:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f9082d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90840cc470 0x7f90840cc890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34880/0 (socket says 172.21.15.64:34880) 2023-09-27T23:33:59.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f9082d9d700 1 -- 172.21.15.64:0/3434057464 learned_addr learned my addr 172.21.15.64:0/3434057464 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:59.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908259c700 1 --2- 172.21.15.64:0/3434057464 >> v2:172.21.15.123:3300/0 conn(0x7f9084158e70 0x7f90840afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406529828 0 0) 0x7f90840ca5a0 con 0x7f90840cc470 2023-09-27T23:33:59.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9070003570 con 0x7f90840cc470 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840721268 0 0) 0x7f908406dac0 con 0x7f908406a540 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90840ca5a0 con 0x7f908406a540 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993148747 0 0) 0x7f9070003570 con 0x7f90840cc470 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908259c700 1 -- 172.21.15.64:0/3434057464 >> v1:172.21.15.123:6789/0 conn(0x7f908406a540 legacy=0x7f908406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908259c700 1 -- 172.21.15.64:0/3434057464 >> v1:172.21.15.64:6789/0 conn(0x7f90840cc470 legacy=0x7f90840cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908259c700 1 -- 172.21.15.64:0/3434057464 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908406d220 con 0x7f9084158e70 2023-09-27T23:33:59.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908259c700 1 --2- 172.21.15.64:0/3434057464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084158e70 0x7f90840afd20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f906c0098e0 tx=0x7f906c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba49f9deb44223bd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0017f0 con 0x7f9084158e70 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c011420 con 0x7f9084158e70 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908159a700 1 -- 172.21.15.64:0/3434057464 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c0117c0 con 0x7f9084158e70 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- 172.21.15.64:0/3434057464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084158e70 msgr2=0x7f90840afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 --2- 172.21.15.64:0/3434057464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084158e70 0x7f90840afd20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f906c0098e0 tx=0x7f906c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- 172.21.15.64:0/3434057464 shutdown_connections 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 --2- 172.21.15.64:0/3434057464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084158e70 0x7f90840afd20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- 172.21.15.64:0/3434057464 >> 172.21.15.64:0/3434057464 conn(0x7f90840670e0 msgr2=0x7f90840674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- 172.21.15.64:0/3434057464 shutdown_connections 2023-09-27T23:33:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.338+0000 7f908359e700 1 -- 172.21.15.64:0/3434057464 wait complete. 2023-09-27T23:33:59.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 Processor -- start 2023-09-27T23:33:59.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 -- start start 2023-09-27T23:33:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f908406a540 0x7f90841563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90840cc470 0x7f90840b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 0x7f90840b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9084156900 con 0x7f9084158e70 2023-09-27T23:33:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90840b4190 con 0x7f90840cc470 2023-09-27T23:33:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90840b4fd0 con 0x7f908406a540 2023-09-27T23:33:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 0x7f90840b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 0x7f90840b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36140/0 (socket says 172.21.15.64:36140) 2023-09-27T23:33:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 -- 172.21.15.64:0/2647292138 learned_addr learned my addr 172.21.15.64:0/2647292138 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f908259c700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f908406a540 0x7f90841563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9081d9b700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90840cc470 0x7f90840b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 -- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f908406a540 msgr2=0x7f90841563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f908406a540 0x7f90841563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 -- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90840cc470 msgr2=0x7f90840b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90840cc470 0x7f90840b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 -- 172.21.15.64:0/2647292138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c009420 con 0x7f9084158e70 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.342+0000 7f9082d9d700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 0x7f90840b03a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f907c00ccb0 tx=0x7f907c007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c004dc0 con 0x7f9084158e70 2023-09-27T23:33:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c00a620 con 0x7f9084158e70 2023-09-27T23:33:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90840b43c0 con 0x7f9084158e70 2023-09-27T23:33:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00a9e0 con 0x7f9084158e70 2023-09-27T23:33:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90840b4980 con 0x7f9084158e70 2023-09-27T23:33:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90840b5160 con 0x7f9084158e70 2023-09-27T23:33:59.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.346+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f907c00ad50 con 0x7f9084158e70 2023-09-27T23:33:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.350+0000 7f907a7fc700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9064060aa0 0x7f9064062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.350+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f907c07ca30 con 0x7f9084158e70 2023-09-27T23:33:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.350+0000 7f908259c700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9064060aa0 0x7f9064062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.350+0000 7f908259c700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9064060aa0 0x7f9064062f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f906c0093f0 tx=0x7f906c016000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.354+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f907c050570 con 0x7f9084158e70 2023-09-27T23:33:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.510+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f908404be40 con 0x7f9064060aa0 2023-09-27T23:33:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.530+0000 7f907a7fc700 1 -- 172.21.15.64:0/2647292138 <== mgr.8979 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294973 (secure 0 0 0) 0x7f908404be40 con 0x7f9064060aa0 2023-09-27T23:33:59.540 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9064060aa0 msgr2=0x7f9064062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9064060aa0 0x7f9064062f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f906c0093f0 tx=0x7f906c016000 comp rx=0 tx=0).stop 2023-09-27T23:33:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 msgr2=0x7f90840b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 0x7f90840b03a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f907c00ccb0 tx=0x7f907c007d20 comp rx=0 tx=0).stop 2023-09-27T23:33:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 shutdown_connections 2023-09-27T23:33:59.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9064060aa0 0x7f9064062f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9084158e70 0x7f90840b03a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90840cc470 0x7f90840b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 --2- 172.21.15.64:0/2647292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f908406a540 0x7f90841563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 >> 172.21.15.64:0/2647292138 conn(0x7f90840670e0 msgr2=0x7f9084158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 shutdown_connections 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.538+0000 7f908359e700 1 -- 172.21.15.64:0/2647292138 wait complete. 2023-09-27T23:33:59.545 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:59.560 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:59.069760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238436,"kb_used_data":16036,"kb_used_omap":0,"kb_used_meta":222336,"kb_avail":754736284,"statfs":{"total":773094113280,"available":772849954816,"internally_reserved":0,"allocated":16420864,"data_stored":9123611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":227672064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001269"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557251+0000","last_change":"2023-09-27T23:33:56.552460+0000","last_active":"2023-09-27T23:33:56.557251+0000","last_peered":"2023-09-27T23:33:56.557251+0000","last_clean":"2023-09-27T23:33:56.557251+0000","last_became_active":"2023-09-27T23:33:56.552129+0000","last_became_peered":"2023-09-27T23:33:56.552129+0000","last_unstale":"2023-09-27T23:33:56.557251+0000","last_undegraded":"2023-09-27T23:33:56.557251+0000","last_fullsized":"2023-09-27T23:33:56.557251+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:21.122858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552313+0000","last_change":"2023-09-27T23:33:56.550009+0000","last_active":"2023-09-27T23:33:56.552313+0000","last_peered":"2023-09-27T23:33:56.552313+0000","last_clean":"2023-09-27T23:33:56.552313+0000","last_became_active":"2023-09-27T23:33:56.549761+0000","last_became_peered":"2023-09-27T23:33:56.549761+0000","last_unstale":"2023-09-27T23:33:56.552313+0000","last_undegraded":"2023-09-27T23:33:56.552313+0000","last_fullsized":"2023-09-27T23:33:56.552313+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:39.341931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550385+0000","last_change":"2023-09-27T23:33:56.543868+0000","last_active":"2023-09-27T23:33:56.550385+0000","last_peered":"2023-09-27T23:33:56.550385+0000","last_clean":"2023-09-27T23:33:56.550385+0000","last_became_active":"2023-09-27T23:33:56.543562+0000","last_became_peered":"2023-09-27T23:33:56.543562+0000","last_unstale":"2023-09-27T23:33:56.550385+0000","last_undegraded":"2023-09-27T23:33:56.550385+0000","last_fullsized":"2023-09-27T23:33:56.550385+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:30:55.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552498+0000","last_change":"2023-09-27T23:33:56.547885+0000","last_active":"2023-09-27T23:33:56.552498+0000","last_peered":"2023-09-27T23:33:56.552498+0000","last_clean":"2023-09-27T23:33:56.552498+0000","last_became_active":"2023-09-27T23:33:56.547546+0000","last_became_peered":"2023-09-27T23:33:56.547546+0000","last_unstale":"2023-09-27T23:33:56.552498+0000","last_undegraded":"2023-09-27T23:33:56.552498+0000","last_fullsized":"2023-09-27T23:33:56.552498+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:54.697434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569173+0000","last_change":"2023-09-27T23:33:56.565920+0000","last_active":"2023-09-27T23:33:56.569173+0000","last_peered":"2023-09-27T23:33:56.569173+0000","last_clean":"2023-09-27T23:33:56.569173+0000","last_became_active":"2023-09-27T23:33:56.565071+0000","last_became_peered":"2023-09-27T23:33:56.565071+0000","last_unstale":"2023-09-27T23:33:56.569173+0000","last_undegraded":"2023-09-27T23:33:56.569173+0000","last_fullsized":"2023-09-27T23:33:56.569173+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:02:25.734384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569208+0000","last_change":"2023-09-27T23:33:56.566463+0000","last_active":"2023-09-27T23:33:56.569208+0000","last_peered":"2023-09-27T23:33:56.569208+0000","last_clean":"2023-09-27T23:33:56.569208+0000","last_became_active":"2023-09-27T23:33:56.566181+0000","last_became_peered":"2023-09-27T23:33:56.566181+0000","last_unstale":"2023-09-27T23:33:56.569208+0000","last_undegraded":"2023-09-27T23:33:56.569208+0000","last_fullsized":"2023-09-27T23:33:56.569208+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:11:49.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550358+0000","last_change":"2023-09-27T23:33:56.542275+0000","last_active":"2023-09-27T23:33:56.550358+0000","last_peered":"2023-09-27T23:33:56.550358+0000","last_clean":"2023-09-27T23:33:56.550358+0000","last_became_active":"2023-09-27T23:33:56.542017+0000","last_became_peered":"2023-09-27T23:33:56.542017+0000","last_unstale":"2023-09-27T23:33:56.550358+0000","last_undegraded":"2023-09-27T23:33:56.550358+0000","last_fullsized":"2023-09-27T23:33:56.550358+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:24.846142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561352+0000","last_change":"2023-09-27T23:33:56.556873+0000","last_active":"2023-09-27T23:33:56.561352+0000","last_peered":"2023-09-27T23:33:56.561352+0000","last_clean":"2023-09-27T23:33:56.561352+0000","last_became_active":"2023-09-27T23:33:56.556561+0000","last_became_peered":"2023-09-27T23:33:56.556561+0000","last_unstale":"2023-09-27T23:33:56.561352+0000","last_undegraded":"2023-09-27T23:33:56.561352+0000","last_fullsized":"2023-09-27T23:33:56.561352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:01.494681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557128+0000","last_change":"2023-09-27T23:33:56.552372+0000","last_active":"2023-09-27T23:33:56.557128+0000","last_peered":"2023-09-27T23:33:56.557128+0000","last_clean":"2023-09-27T23:33:56.557128+0000","last_became_active":"2023-09-27T23:33:56.551956+0000","last_became_peered":"2023-09-27T23:33:56.551956+0000","last_unstale":"2023-09-27T23:33:56.557128+0000","last_undegraded":"2023-09-27T23:33:56.557128+0000","last_fullsized":"2023-09-27T23:33:56.557128+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:12.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539969+0000","last_change":"2023-09-27T23:33:56.171325+0000","last_active":"2023-09-27T23:33:56.539969+0000","last_peered":"2023-09-27T23:33:56.539969+0000","last_clean":"2023-09-27T23:33:56.539969+0000","last_became_active":"2023-09-27T23:33:56.170836+0000","last_became_peered":"2023-09-27T23:33:56.170836+0000","last_unstale":"2023-09-27T23:33:56.539969+0000","last_undegraded":"2023-09-27T23:33:56.539969+0000","last_fullsized":"2023-09-27T23:33:56.539969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:56:07.324327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539936+0000","last_change":"2023-09-27T23:33:56.165693+0000","last_active":"2023-09-27T23:33:56.539936+0000","last_peered":"2023-09-27T23:33:56.539936+0000","last_clean":"2023-09-27T23:33:56.539936+0000","last_became_active":"2023-09-27T23:33:56.165301+0000","last_became_peered":"2023-09-27T23:33:56.165301+0000","last_unstale":"2023-09-27T23:33:56.539936+0000","last_undegraded":"2023-09-27T23:33:56.539936+0000","last_fullsized":"2023-09-27T23:33:56.539936+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:17.631606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561289+0000","last_change":"2023-09-27T23:33:56.556576+0000","last_active":"2023-09-27T23:33:56.561289+0000","last_peered":"2023-09-27T23:33:56.561289+0000","last_clean":"2023-09-27T23:33:56.561289+0000","last_became_active":"2023-09-27T23:33:56.555849+0000","last_became_peered":"2023-09-27T23:33:56.555849+0000","last_unstale":"2023-09-27T23:33:56.561289+0000","last_undegraded":"2023-09-27T23:33:56.561289+0000","last_fullsized":"2023-09-27T23:33:56.561289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:55.099301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539954+0000","last_change":"2023-09-27T23:33:56.164762+0000","last_active":"2023-09-27T23:33:56.539954+0000","last_peered":"2023-09-27T23:33:56.539954+0000","last_clean":"2023-09-27T23:33:56.539954+0000","last_became_active":"2023-09-27T23:33:56.164480+0000","last_became_peered":"2023-09-27T23:33:56.164480+0000","last_unstale":"2023-09-27T23:33:56.539954+0000","last_undegraded":"2023-09-27T23:33:56.539954+0000","last_fullsized":"2023-09-27T23:33:56.539954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:27.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'271","reported_seq":824,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928872+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:52.928872+0000","last_peered":"2023-09-27T23:33:52.928872+0000","last_clean":"2023-09-27T23:33:52.928872+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:52.928872+0000","last_undegraded":"2023-09-27T23:33:52.928872+0000","last_fullsized":"2023-09-27T23:33:52.928872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":271,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928736+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:52.928736+0000","last_peered":"2023-09-27T23:33:52.928736+0000","last_clean":"2023-09-27T23:33:52.928736+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:52.928736+0000","last_undegraded":"2023-09-27T23:33:52.928736+0000","last_fullsized":"2023-09-27T23:33:52.928736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.971677+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:51.971677+0000","last_peered":"2023-09-27T23:33:51.971677+0000","last_clean":"2023-09-27T23:33:51.971677+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:51.971677+0000","last_undegraded":"2023-09-27T23:33:51.971677+0000","last_fullsized":"2023-09-27T23:33:51.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540062+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:56.540062+0000","last_peered":"2023-09-27T23:33:56.540062+0000","last_clean":"2023-09-27T23:33:56.540062+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:56.540062+0000","last_undegraded":"2023-09-27T23:33:56.540062+0000","last_fullsized":"2023-09-27T23:33:56.540062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928702+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:52.928702+0000","last_peered":"2023-09-27T23:33:52.928702+0000","last_clean":"2023-09-27T23:33:52.928702+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:52.928702+0000","last_undegraded":"2023-09-27T23:33:52.928702+0000","last_fullsized":"2023-09-27T23:33:52.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540304+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:56.540304+0000","last_peered":"2023-09-27T23:33:56.540304+0000","last_clean":"2023-09-27T23:33:56.540304+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:56.540304+0000","last_undegraded":"2023-09-27T23:33:56.540304+0000","last_fullsized":"2023-09-27T23:33:56.540304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538204+0000","last_change":"2023-09-27T23:33:56.170155+0000","last_active":"2023-09-27T23:33:56.538204+0000","last_peered":"2023-09-27T23:33:56.538204+0000","last_clean":"2023-09-27T23:33:56.538204+0000","last_became_active":"2023-09-27T23:33:56.169711+0000","last_became_peered":"2023-09-27T23:33:56.169711+0000","last_unstale":"2023-09-27T23:33:56.538204+0000","last_undegraded":"2023-09-27T23:33:56.538204+0000","last_fullsized":"2023-09-27T23:33:56.538204+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:24:27.214773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.927845+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:52.927845+0000","last_peered":"2023-09-27T23:33:52.927845+0000","last_clean":"2023-09-27T23:33:52.927845+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:52.927845+0000","last_undegraded":"2023-09-27T23:33:52.927845+0000","last_fullsized":"2023-09-27T23:33:52.927845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509683,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":1.006},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.69999999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.125}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.879}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.155}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.214},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":1.097}]}]},{"osd":6,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28424,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94343416,"statfs":{"total":96636764160,"available":96607657984,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":0.997},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.087},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.149}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.022}]}]},{"osd":5,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38088,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":36224,"kb_avail":94333752,"statfs":{"total":96636764160,"available":96597762048,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37093376},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":1.498}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":1.1659999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":1.546},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.149},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.3129999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.431},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.2170000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":24060,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":94347780,"statfs":{"total":96636764160,"available":96612126720,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":1.133},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":1.1819999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.90300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.749},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.87},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.2370000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.0169999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509682,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":27264,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27918336},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":1.161},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.036}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":1.321},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":1.5620000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.6000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.762}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.887},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.5029999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22908,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94348932,"statfs":{"total":96636764160,"available":96613306368,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.86499999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.93899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.03},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":1.0660000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":1.179},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.272}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542390,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38332,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94333508,"statfs":{"total":96636764160,"available":96597512192,"internally_reserved":0,"allocated":1953792,"data_stored":1019257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0349999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.099},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.165},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.133}]}]},{"osd":0,"up_from":15,"seq":64424509684,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":26880,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27525120},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":1.194},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.2869999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.831}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.391}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:59.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:33:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe6850e8700 1 Processor -- start 2023-09-27T23:33:59.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe6850e8700 1 -- start start 2023-09-27T23:33:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe6850e8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe6800b0810 0x7fe6800b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe6850e8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6800af530 con 0x7fe68006be10 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe6850e8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe680159660 con 0x7fe6800af9c0 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe6850e8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe680159470 con 0x7fe6800b0810 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe6800b0810 0x7fe6800b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe68006be10 0x7fe680158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34890/0 (socket says 172.21.15.64:34890) 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe6800b0810 0x7fe6800b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36146/0 (socket says 172.21.15.64:36146) 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67f7fe700 1 -- 172.21.15.64:0/1693182635 learned_addr learned my addr 172.21.15.64:0/1693182635 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:33:59.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67ffff700 1 -- 172.21.15.64:0/1693182635 >> v1:172.21.15.123:6789/0 conn(0x7fe6800af9c0 legacy=0x7fe680154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67ffff700 1 -- 172.21.15.64:0/1693182635 >> v1:172.21.15.64:6789/0 conn(0x7fe68006be10 legacy=0x7fe680158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.674+0000 7fe67ffff700 1 -- 172.21.15.64:0/1693182635 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680159940 con 0x7fe6800b0810 2023-09-27T23:33:59.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe67ffff700 1 --2- 172.21.15.64:0/1693182635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800b0810 0x7fe6800b0be0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098e0 tx=0x7fe6700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba855bf828b1c0f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe67effd700 1 -- 172.21.15.64:0/1693182635 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670009e80 con 0x7fe6800b0810 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe67effd700 1 -- 172.21.15.64:0/1693182635 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67000b8f0 con 0x7fe6800b0810 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe67effd700 1 -- 172.21.15.64:0/1693182635 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67000bc70 con 0x7fe6800b0810 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 -- 172.21.15.64:0/1693182635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800b0810 msgr2=0x7fe6800b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 --2- 172.21.15.64:0/1693182635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800b0810 0x7fe6800b0be0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098e0 tx=0x7fe6700091a0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 -- 172.21.15.64:0/1693182635 shutdown_connections 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 --2- 172.21.15.64:0/1693182635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800b0810 0x7fe6800b0be0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 -- 172.21.15.64:0/1693182635 >> 172.21.15.64:0/1693182635 conn(0x7fe680067bb0 msgr2=0x7fe6800add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 -- 172.21.15.64:0/1693182635 shutdown_connections 2023-09-27T23:33:59.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.678+0000 7fe6850e8700 1 -- 172.21.15.64:0/1693182635 wait complete. 2023-09-27T23:33:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 Processor -- start 2023-09-27T23:33:59.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 -- start start 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006be10 0x7fe68014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 0x7fe680156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6800b0810 0x7fe680168700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801579b0 con 0x7fe6800af9c0 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801576d0 con 0x7fe6800b0810 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe680154a90 con 0x7fe68006be10 2023-09-27T23:33:59.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 0x7fe680156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 0x7fe680156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36162/0 (socket says 172.21.15.64:36162) 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 -- 172.21.15.64:0/2025076436 learned_addr learned my addr 172.21.15.64:0/2025076436 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67ffff700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006be10 0x7fe68014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6848e7700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6800b0810 0x7fe680168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 -- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006be10 msgr2=0x7fe68014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006be10 0x7fe68014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 -- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6800b0810 msgr2=0x7fe680168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6800b0810 0x7fe680168700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 -- 172.21.15.64:0/2025076436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670009420 con 0x7fe6800af9c0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe67f7fe700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 0x7fe680156cc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe674010c70 tx=0x7fe674001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674010e20 con 0x7fe6800af9c0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67400eb70 con 0x7fe6800af9c0 2023-09-27T23:33:59.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe680154d70 con 0x7fe6800af9c0 2023-09-27T23:33:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.682+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674016400 con 0x7fe6800af9c0 2023-09-27T23:33:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.686+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe680157200 con 0x7fe6800af9c0 2023-09-27T23:33:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.686+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6801548f0 con 0x7fe6800af9c0 2023-09-27T23:33:59.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.686+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe67400bd50 con 0x7fe6800af9c0 2023-09-27T23:33:59.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.686+0000 7fe667fff700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe66c060a70 0x7fe66c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:33:59.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.686+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe674081240 con 0x7fe6800af9c0 2023-09-27T23:33:59.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.686+0000 7fe67ffff700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe66c060a70 0x7fe66c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:33:59.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.690+0000 7fe67ffff700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe66c060a70 0x7fe66c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6700093f0 tx=0x7fe670017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:33:59.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.694+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe6801548f0 con 0x7fe6800af9c0 2023-09-27T23:33:59.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.854+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6801548f0 con 0x7fe66c060a70 2023-09-27T23:33:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.874+0000 7fe667fff700 1 -- 172.21.15.64:0/2025076436 <== mgr.8979 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294973 (secure 0 0 0) 0x7fe6801548f0 con 0x7fe66c060a70 2023-09-27T23:33:59.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:33:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe66c060a70 msgr2=0x7fe66c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe66c060a70 0x7fe66c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6700093f0 tx=0x7fe670017000 comp rx=0 tx=0).stop 2023-09-27T23:33:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 msgr2=0x7fe680156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 0x7fe680156cc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe674010c70 tx=0x7fe674001690 comp rx=0 tx=0).stop 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 shutdown_connections 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fe66c060a70 0x7fe66c062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6800b0810 0x7fe680168700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6800af9c0 0x7fe680156cc0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 --2- 172.21.15.64:0/2025076436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe68006be10 0x7fe68014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 >> 172.21.15.64:0/2025076436 conn(0x7fe680067bb0 msgr2=0x7fe68006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 shutdown_connections 2023-09-27T23:33:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:33:59.882+0000 7fe6850e8700 1 -- 172.21.15.64:0/2025076436 wait complete. 2023-09-27T23:33:59.887 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:33:59.899 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:59.069760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238436,"kb_used_data":16036,"kb_used_omap":0,"kb_used_meta":222336,"kb_avail":754736284,"statfs":{"total":773094113280,"available":772849954816,"internally_reserved":0,"allocated":16420864,"data_stored":9123611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":227672064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001269"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557251+0000","last_change":"2023-09-27T23:33:56.552460+0000","last_active":"2023-09-27T23:33:56.557251+0000","last_peered":"2023-09-27T23:33:56.557251+0000","last_clean":"2023-09-27T23:33:56.557251+0000","last_became_active":"2023-09-27T23:33:56.552129+0000","last_became_peered":"2023-09-27T23:33:56.552129+0000","last_unstale":"2023-09-27T23:33:56.557251+0000","last_undegraded":"2023-09-27T23:33:56.557251+0000","last_fullsized":"2023-09-27T23:33:56.557251+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:21.122858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552313+0000","last_change":"2023-09-27T23:33:56.550009+0000","last_active":"2023-09-27T23:33:56.552313+0000","last_peered":"2023-09-27T23:33:56.552313+0000","last_clean":"2023-09-27T23:33:56.552313+0000","last_became_active":"2023-09-27T23:33:56.549761+0000","last_became_peered":"2023-09-27T23:33:56.549761+0000","last_unstale":"2023-09-27T23:33:56.552313+0000","last_undegraded":"2023-09-27T23:33:56.552313+0000","last_fullsized":"2023-09-27T23:33:56.552313+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:39.341931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550385+0000","last_change":"2023-09-27T23:33:56.543868+0000","last_active":"2023-09-27T23:33:56.550385+0000","last_peered":"2023-09-27T23:33:56.550385+0000","last_clean":"2023-09-27T23:33:56.550385+0000","last_became_active":"2023-09-27T23:33:56.543562+0000","last_became_peered":"2023-09-27T23:33:56.543562+0000","last_unstale":"2023-09-27T23:33:56.550385+0000","last_undegraded":"2023-09-27T23:33:56.550385+0000","last_fullsized":"2023-09-27T23:33:56.550385+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:30:55.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552498+0000","last_change":"2023-09-27T23:33:56.547885+0000","last_active":"2023-09-27T23:33:56.552498+0000","last_peered":"2023-09-27T23:33:56.552498+0000","last_clean":"2023-09-27T23:33:56.552498+0000","last_became_active":"2023-09-27T23:33:56.547546+0000","last_became_peered":"2023-09-27T23:33:56.547546+0000","last_unstale":"2023-09-27T23:33:56.552498+0000","last_undegraded":"2023-09-27T23:33:56.552498+0000","last_fullsized":"2023-09-27T23:33:56.552498+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:54.697434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569173+0000","last_change":"2023-09-27T23:33:56.565920+0000","last_active":"2023-09-27T23:33:56.569173+0000","last_peered":"2023-09-27T23:33:56.569173+0000","last_clean":"2023-09-27T23:33:56.569173+0000","last_became_active":"2023-09-27T23:33:56.565071+0000","last_became_peered":"2023-09-27T23:33:56.565071+0000","last_unstale":"2023-09-27T23:33:56.569173+0000","last_undegraded":"2023-09-27T23:33:56.569173+0000","last_fullsized":"2023-09-27T23:33:56.569173+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:02:25.734384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569208+0000","last_change":"2023-09-27T23:33:56.566463+0000","last_active":"2023-09-27T23:33:56.569208+0000","last_peered":"2023-09-27T23:33:56.569208+0000","last_clean":"2023-09-27T23:33:56.569208+0000","last_became_active":"2023-09-27T23:33:56.566181+0000","last_became_peered":"2023-09-27T23:33:56.566181+0000","last_unstale":"2023-09-27T23:33:56.569208+0000","last_undegraded":"2023-09-27T23:33:56.569208+0000","last_fullsized":"2023-09-27T23:33:56.569208+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:11:49.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550358+0000","last_change":"2023-09-27T23:33:56.542275+0000","last_active":"2023-09-27T23:33:56.550358+0000","last_peered":"2023-09-27T23:33:56.550358+0000","last_clean":"2023-09-27T23:33:56.550358+0000","last_became_active":"2023-09-27T23:33:56.542017+0000","last_became_peered":"2023-09-27T23:33:56.542017+0000","last_unstale":"2023-09-27T23:33:56.550358+0000","last_undegraded":"2023-09-27T23:33:56.550358+0000","last_fullsized":"2023-09-27T23:33:56.550358+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:24.846142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561352+0000","last_change":"2023-09-27T23:33:56.556873+0000","last_active":"2023-09-27T23:33:56.561352+0000","last_peered":"2023-09-27T23:33:56.561352+0000","last_clean":"2023-09-27T23:33:56.561352+0000","last_became_active":"2023-09-27T23:33:56.556561+0000","last_became_peered":"2023-09-27T23:33:56.556561+0000","last_unstale":"2023-09-27T23:33:56.561352+0000","last_undegraded":"2023-09-27T23:33:56.561352+0000","last_fullsized":"2023-09-27T23:33:56.561352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:01.494681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557128+0000","last_change":"2023-09-27T23:33:56.552372+0000","last_active":"2023-09-27T23:33:56.557128+0000","last_peered":"2023-09-27T23:33:56.557128+0000","last_clean":"2023-09-27T23:33:56.557128+0000","last_became_active":"2023-09-27T23:33:56.551956+0000","last_became_peered":"2023-09-27T23:33:56.551956+0000","last_unstale":"2023-09-27T23:33:56.557128+0000","last_undegraded":"2023-09-27T23:33:56.557128+0000","last_fullsized":"2023-09-27T23:33:56.557128+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:12.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539969+0000","last_change":"2023-09-27T23:33:56.171325+0000","last_active":"2023-09-27T23:33:56.539969+0000","last_peered":"2023-09-27T23:33:56.539969+0000","last_clean":"2023-09-27T23:33:56.539969+0000","last_became_active":"2023-09-27T23:33:56.170836+0000","last_became_peered":"2023-09-27T23:33:56.170836+0000","last_unstale":"2023-09-27T23:33:56.539969+0000","last_undegraded":"2023-09-27T23:33:56.539969+0000","last_fullsized":"2023-09-27T23:33:56.539969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:56:07.324327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539936+0000","last_change":"2023-09-27T23:33:56.165693+0000","last_active":"2023-09-27T23:33:56.539936+0000","last_peered":"2023-09-27T23:33:56.539936+0000","last_clean":"2023-09-27T23:33:56.539936+0000","last_became_active":"2023-09-27T23:33:56.165301+0000","last_became_peered":"2023-09-27T23:33:56.165301+0000","last_unstale":"2023-09-27T23:33:56.539936+0000","last_undegraded":"2023-09-27T23:33:56.539936+0000","last_fullsized":"2023-09-27T23:33:56.539936+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:17.631606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561289+0000","last_change":"2023-09-27T23:33:56.556576+0000","last_active":"2023-09-27T23:33:56.561289+0000","last_peered":"2023-09-27T23:33:56.561289+0000","last_clean":"2023-09-27T23:33:56.561289+0000","last_became_active":"2023-09-27T23:33:56.555849+0000","last_became_peered":"2023-09-27T23:33:56.555849+0000","last_unstale":"2023-09-27T23:33:56.561289+0000","last_undegraded":"2023-09-27T23:33:56.561289+0000","last_fullsized":"2023-09-27T23:33:56.561289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:55.099301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539954+0000","last_change":"2023-09-27T23:33:56.164762+0000","last_active":"2023-09-27T23:33:56.539954+0000","last_peered":"2023-09-27T23:33:56.539954+0000","last_clean":"2023-09-27T23:33:56.539954+0000","last_became_active":"2023-09-27T23:33:56.164480+0000","last_became_peered":"2023-09-27T23:33:56.164480+0000","last_unstale":"2023-09-27T23:33:56.539954+0000","last_undegraded":"2023-09-27T23:33:56.539954+0000","last_fullsized":"2023-09-27T23:33:56.539954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:27.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'271","reported_seq":824,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928872+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:52.928872+0000","last_peered":"2023-09-27T23:33:52.928872+0000","last_clean":"2023-09-27T23:33:52.928872+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:52.928872+0000","last_undegraded":"2023-09-27T23:33:52.928872+0000","last_fullsized":"2023-09-27T23:33:52.928872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":271,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928736+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:52.928736+0000","last_peered":"2023-09-27T23:33:52.928736+0000","last_clean":"2023-09-27T23:33:52.928736+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:52.928736+0000","last_undegraded":"2023-09-27T23:33:52.928736+0000","last_fullsized":"2023-09-27T23:33:52.928736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.971677+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:51.971677+0000","last_peered":"2023-09-27T23:33:51.971677+0000","last_clean":"2023-09-27T23:33:51.971677+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:51.971677+0000","last_undegraded":"2023-09-27T23:33:51.971677+0000","last_fullsized":"2023-09-27T23:33:51.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540062+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:56.540062+0000","last_peered":"2023-09-27T23:33:56.540062+0000","last_clean":"2023-09-27T23:33:56.540062+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:56.540062+0000","last_undegraded":"2023-09-27T23:33:56.540062+0000","last_fullsized":"2023-09-27T23:33:56.540062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928702+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:52.928702+0000","last_peered":"2023-09-27T23:33:52.928702+0000","last_clean":"2023-09-27T23:33:52.928702+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:52.928702+0000","last_undegraded":"2023-09-27T23:33:52.928702+0000","last_fullsized":"2023-09-27T23:33:52.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540304+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:56.540304+0000","last_peered":"2023-09-27T23:33:56.540304+0000","last_clean":"2023-09-27T23:33:56.540304+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:56.540304+0000","last_undegraded":"2023-09-27T23:33:56.540304+0000","last_fullsized":"2023-09-27T23:33:56.540304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538204+0000","last_change":"2023-09-27T23:33:56.170155+0000","last_active":"2023-09-27T23:33:56.538204+0000","last_peered":"2023-09-27T23:33:56.538204+0000","last_clean":"2023-09-27T23:33:56.538204+0000","last_became_active":"2023-09-27T23:33:56.169711+0000","last_became_peered":"2023-09-27T23:33:56.169711+0000","last_unstale":"2023-09-27T23:33:56.538204+0000","last_undegraded":"2023-09-27T23:33:56.538204+0000","last_fullsized":"2023-09-27T23:33:56.538204+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:24:27.214773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.927845+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:52.927845+0000","last_peered":"2023-09-27T23:33:52.927845+0000","last_clean":"2023-09-27T23:33:52.927845+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:52.927845+0000","last_undegraded":"2023-09-27T23:33:52.927845+0000","last_fullsized":"2023-09-27T23:33:52.927845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509683,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":1.006},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.69999999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.125}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.879}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.155}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.214},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":1.097}]}]},{"osd":6,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28424,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94343416,"statfs":{"total":96636764160,"available":96607657984,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":0.997},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.087},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.149}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.022}]}]},{"osd":5,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38088,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":36224,"kb_avail":94333752,"statfs":{"total":96636764160,"available":96597762048,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37093376},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":1.498}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":1.1659999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":1.546},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.149},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.3129999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.431},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.2170000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":24060,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":94347780,"statfs":{"total":96636764160,"available":96612126720,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":1.133},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":1.1819999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.90300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.749},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.87},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.2370000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.0169999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509682,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":27264,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27918336},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":1.161},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.036}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":1.321},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":1.5620000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.6000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.762}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.887},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.5029999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22908,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94348932,"statfs":{"total":96636764160,"available":96613306368,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.86499999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.93899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.03},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":1.0660000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":1.179},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.272}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542390,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38332,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94333508,"statfs":{"total":96636764160,"available":96597512192,"internally_reserved":0,"allocated":1953792,"data_stored":1019257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0349999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.099},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.165},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.133}]}]},{"osd":0,"up_from":15,"seq":64424509684,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":26880,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27525120},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":1.194},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.2869999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.831}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.391}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:33:59.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:34:00.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f359e700 1 Processor -- start 2023-09-27T23:34:00.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f359e700 1 -- start start 2023-09-27T23:34:00.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66f415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f406d990 con 0x7f66f40c7f00 2023-09-27T23:34:00.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f415a570 con 0x7f66f406de20 2023-09-27T23:34:00.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f406cd80 con 0x7f66f4155960 2023-09-27T23:34:00.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f2d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f66f40c7f00 0x7f66f40aca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34904/0 (socket says 172.21.15.64:34904) 2023-09-27T23:34:00.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f2d9d700 1 -- 172.21.15.64:0/187539365 learned_addr learned my addr 172.21.15.64:0/187539365 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:00.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456472765 0 0) 0x7f66f406d990 con 0x7f66f40c7f00 2023-09-27T23:34:00.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f259c700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66f415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc003570 con 0x7f66f40c7f00 2023-09-27T23:34:00.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786869840 0 0) 0x7f66dc003570 con 0x7f66f40c7f00 2023-09-27T23:34:00.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 msgr2=0x7f66f415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66f415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 >> v1:172.21.15.123:6789/0 conn(0x7f66f406de20 legacy=0x7f66f40c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f406d060 con 0x7f66f40c7f00 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f66ec0017f0 con 0x7f66f40c7f00 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f259c700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66f415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 >> v1:172.21.15.64:6789/0 conn(0x7f66f40c7f00 legacy=0x7f66f40aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66dc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66dc004080 0x7f66dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66dc004d50 0x7f66dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f406cd80 con 0x7f66dc004080 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00bb70 con 0x7f66dc004d50 2023-09-27T23:34:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc00bd30 con 0x7f66f4155960 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f1d9b700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66dc004080 0x7f66dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f259c700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66dc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.010+0000 7f66f2d9d700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66dc004d50 0x7f66dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f1d9b700 1 -- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 msgr2=0x7f66dc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f1d9b700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4155960 0x7f66dc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f1d9b700 1 -- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66dc004d50 msgr2=0x7f66dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f1d9b700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66dc004d50 0x7f66dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f1d9b700 1 -- 172.21.15.64:0/187539365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc003570 con 0x7f66dc004080 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f2d9d700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66dc004d50 0x7f66dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f1d9b700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66dc004080 0x7f66dc004810 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f66e40019a0 tx=0x7f66e4007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43c5cbc7cea0a0e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400c5d0 con 0x7f66dc004080 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e400c750 con 0x7f66dc004080 2023-09-27T23:34:00.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f159a700 1 -- 172.21.15.64:0/187539365 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400cb10 con 0x7f66dc004080 2023-09-27T23:34:00.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f359e700 1 -- 172.21.15.64:0/187539365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66dc004080 msgr2=0x7f66dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f359e700 1 --2- 172.21.15.64:0/187539365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66dc004080 0x7f66dc004810 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f66e40019a0 tx=0x7f66e4007d10 comp rx=0 tx=0).stop 2023-09-27T23:34:00.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f2d9d700 1 -- 172.21.15.64:0/187539365 reap_dead start 2023-09-27T23:34:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f359e700 1 -- 172.21.15.64:0/187539365 shutdown_connections 2023-09-27T23:34:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f359e700 1 -- 172.21.15.64:0/187539365 >> 172.21.15.64:0/187539365 conn(0x7f66f4067bb0 msgr2=0x7f66f40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f359e700 1 -- 172.21.15.64:0/187539365 shutdown_connections 2023-09-27T23:34:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.014+0000 7f66f359e700 1 -- 172.21.15.64:0/187539365 wait complete. 2023-09-27T23:34:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 Processor -- start 2023-09-27T23:34:00.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 -- start start 2023-09-27T23:34:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4140580 0x7f66f4150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f4151350 0x7f66f4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 0x7f66f40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4152aa0 con 0x7f66f4153cc0 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41517c0 con 0x7f66f4151350 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4154160 con 0x7f66f4140580 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 0x7f66f40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 0x7f66f40aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36182/0 (socket says 172.21.15.64:36182) 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 -- 172.21.15.64:0/149057035 learned_addr learned my addr 172.21.15.64:0/149057035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4140580 0x7f66f4150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f1d9b700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f4151350 0x7f66f4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 -- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4140580 msgr2=0x7f66f4150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4140580 0x7f66f4150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 -- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f4151350 msgr2=0x7f66f4153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f4151350 0x7f66f4153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 -- 172.21.15.64:0/149057035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4151e90 con 0x7f66f4153cc0 2023-09-27T23:34:00.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f2d9d700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 0x7f66f40aec50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f66f40c8300 tx=0x7f66ec004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec006c30 con 0x7f66f4153cc0 2023-09-27T23:34:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec006d90 con 0x7f66f4153cc0 2023-09-27T23:34:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f4152170 con 0x7f66f4153cc0 2023-09-27T23:34:00.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0136a0 con 0x7f66f4153cc0 2023-09-27T23:34:00.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.018+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4152730 con 0x7f66f4153cc0 2023-09-27T23:34:00.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.022+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f404be40 con 0x7f66f4153cc0 2023-09-27T23:34:00.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.022+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f66ec013a90 con 0x7f66f4153cc0 2023-09-27T23:34:00.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.022+0000 7f66e27fc700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f66d4060a70 0x7f66d4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.022+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f66ec07f520 con 0x7f66f4153cc0 2023-09-27T23:34:00.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.022+0000 7f66f259c700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f66d4060a70 0x7f66d4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.022+0000 7f66f259c700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f66d4060a70 0x7f66d4062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66e8006fd0 tx=0x7f66e8008040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.030+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f66ec04f550 con 0x7f66f4153cc0 2023-09-27T23:34:00.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.186+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66f4152be0 con 0x7f66d4060a70 2023-09-27T23:34:00.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.202+0000 7f66e27fc700 1 -- 172.21.15.64:0/149057035 <== mgr.8979 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294973 (secure 0 0 0) 0x7f66f4152be0 con 0x7f66d4060a70 2023-09-27T23:34:00.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:34:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f66d4060a70 msgr2=0x7f66d4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f66d4060a70 0x7f66d4062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66e8006fd0 tx=0x7f66e8008040 comp rx=0 tx=0).stop 2023-09-27T23:34:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 msgr2=0x7f66f40aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 0x7f66f40aec50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f66f40c8300 tx=0x7f66ec004bb0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 shutdown_connections 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f66d4060a70 0x7f66d4062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4153cc0 0x7f66f40aec50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f4151350 0x7f66f4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 --2- 172.21.15.64:0/149057035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66f4140580 0x7f66f4150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 >> 172.21.15.64:0/149057035 conn(0x7f66f4067bb0 msgr2=0x7f66f4159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 shutdown_connections 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.206+0000 7f66f359e700 1 -- 172.21.15.64:0/149057035 wait complete. 2023-09-27T23:34:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:34:00.223 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:59.069760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238436,"kb_used_data":16036,"kb_used_omap":0,"kb_used_meta":222336,"kb_avail":754736284,"statfs":{"total":773094113280,"available":772849954816,"internally_reserved":0,"allocated":16420864,"data_stored":9123611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":227672064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001269"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557251+0000","last_change":"2023-09-27T23:33:56.552460+0000","last_active":"2023-09-27T23:33:56.557251+0000","last_peered":"2023-09-27T23:33:56.557251+0000","last_clean":"2023-09-27T23:33:56.557251+0000","last_became_active":"2023-09-27T23:33:56.552129+0000","last_became_peered":"2023-09-27T23:33:56.552129+0000","last_unstale":"2023-09-27T23:33:56.557251+0000","last_undegraded":"2023-09-27T23:33:56.557251+0000","last_fullsized":"2023-09-27T23:33:56.557251+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:21.122858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552313+0000","last_change":"2023-09-27T23:33:56.550009+0000","last_active":"2023-09-27T23:33:56.552313+0000","last_peered":"2023-09-27T23:33:56.552313+0000","last_clean":"2023-09-27T23:33:56.552313+0000","last_became_active":"2023-09-27T23:33:56.549761+0000","last_became_peered":"2023-09-27T23:33:56.549761+0000","last_unstale":"2023-09-27T23:33:56.552313+0000","last_undegraded":"2023-09-27T23:33:56.552313+0000","last_fullsized":"2023-09-27T23:33:56.552313+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:39.341931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550385+0000","last_change":"2023-09-27T23:33:56.543868+0000","last_active":"2023-09-27T23:33:56.550385+0000","last_peered":"2023-09-27T23:33:56.550385+0000","last_clean":"2023-09-27T23:33:56.550385+0000","last_became_active":"2023-09-27T23:33:56.543562+0000","last_became_peered":"2023-09-27T23:33:56.543562+0000","last_unstale":"2023-09-27T23:33:56.550385+0000","last_undegraded":"2023-09-27T23:33:56.550385+0000","last_fullsized":"2023-09-27T23:33:56.550385+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:30:55.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552498+0000","last_change":"2023-09-27T23:33:56.547885+0000","last_active":"2023-09-27T23:33:56.552498+0000","last_peered":"2023-09-27T23:33:56.552498+0000","last_clean":"2023-09-27T23:33:56.552498+0000","last_became_active":"2023-09-27T23:33:56.547546+0000","last_became_peered":"2023-09-27T23:33:56.547546+0000","last_unstale":"2023-09-27T23:33:56.552498+0000","last_undegraded":"2023-09-27T23:33:56.552498+0000","last_fullsized":"2023-09-27T23:33:56.552498+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:54.697434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569173+0000","last_change":"2023-09-27T23:33:56.565920+0000","last_active":"2023-09-27T23:33:56.569173+0000","last_peered":"2023-09-27T23:33:56.569173+0000","last_clean":"2023-09-27T23:33:56.569173+0000","last_became_active":"2023-09-27T23:33:56.565071+0000","last_became_peered":"2023-09-27T23:33:56.565071+0000","last_unstale":"2023-09-27T23:33:56.569173+0000","last_undegraded":"2023-09-27T23:33:56.569173+0000","last_fullsized":"2023-09-27T23:33:56.569173+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:02:25.734384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569208+0000","last_change":"2023-09-27T23:33:56.566463+0000","last_active":"2023-09-27T23:33:56.569208+0000","last_peered":"2023-09-27T23:33:56.569208+0000","last_clean":"2023-09-27T23:33:56.569208+0000","last_became_active":"2023-09-27T23:33:56.566181+0000","last_became_peered":"2023-09-27T23:33:56.566181+0000","last_unstale":"2023-09-27T23:33:56.569208+0000","last_undegraded":"2023-09-27T23:33:56.569208+0000","last_fullsized":"2023-09-27T23:33:56.569208+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:11:49.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550358+0000","last_change":"2023-09-27T23:33:56.542275+0000","last_active":"2023-09-27T23:33:56.550358+0000","last_peered":"2023-09-27T23:33:56.550358+0000","last_clean":"2023-09-27T23:33:56.550358+0000","last_became_active":"2023-09-27T23:33:56.542017+0000","last_became_peered":"2023-09-27T23:33:56.542017+0000","last_unstale":"2023-09-27T23:33:56.550358+0000","last_undegraded":"2023-09-27T23:33:56.550358+0000","last_fullsized":"2023-09-27T23:33:56.550358+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:24.846142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561352+0000","last_change":"2023-09-27T23:33:56.556873+0000","last_active":"2023-09-27T23:33:56.561352+0000","last_peered":"2023-09-27T23:33:56.561352+0000","last_clean":"2023-09-27T23:33:56.561352+0000","last_became_active":"2023-09-27T23:33:56.556561+0000","last_became_peered":"2023-09-27T23:33:56.556561+0000","last_unstale":"2023-09-27T23:33:56.561352+0000","last_undegraded":"2023-09-27T23:33:56.561352+0000","last_fullsized":"2023-09-27T23:33:56.561352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:01.494681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557128+0000","last_change":"2023-09-27T23:33:56.552372+0000","last_active":"2023-09-27T23:33:56.557128+0000","last_peered":"2023-09-27T23:33:56.557128+0000","last_clean":"2023-09-27T23:33:56.557128+0000","last_became_active":"2023-09-27T23:33:56.551956+0000","last_became_peered":"2023-09-27T23:33:56.551956+0000","last_unstale":"2023-09-27T23:33:56.557128+0000","last_undegraded":"2023-09-27T23:33:56.557128+0000","last_fullsized":"2023-09-27T23:33:56.557128+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:12.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539969+0000","last_change":"2023-09-27T23:33:56.171325+0000","last_active":"2023-09-27T23:33:56.539969+0000","last_peered":"2023-09-27T23:33:56.539969+0000","last_clean":"2023-09-27T23:33:56.539969+0000","last_became_active":"2023-09-27T23:33:56.170836+0000","last_became_peered":"2023-09-27T23:33:56.170836+0000","last_unstale":"2023-09-27T23:33:56.539969+0000","last_undegraded":"2023-09-27T23:33:56.539969+0000","last_fullsized":"2023-09-27T23:33:56.539969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:56:07.324327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539936+0000","last_change":"2023-09-27T23:33:56.165693+0000","last_active":"2023-09-27T23:33:56.539936+0000","last_peered":"2023-09-27T23:33:56.539936+0000","last_clean":"2023-09-27T23:33:56.539936+0000","last_became_active":"2023-09-27T23:33:56.165301+0000","last_became_peered":"2023-09-27T23:33:56.165301+0000","last_unstale":"2023-09-27T23:33:56.539936+0000","last_undegraded":"2023-09-27T23:33:56.539936+0000","last_fullsized":"2023-09-27T23:33:56.539936+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:17.631606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561289+0000","last_change":"2023-09-27T23:33:56.556576+0000","last_active":"2023-09-27T23:33:56.561289+0000","last_peered":"2023-09-27T23:33:56.561289+0000","last_clean":"2023-09-27T23:33:56.561289+0000","last_became_active":"2023-09-27T23:33:56.555849+0000","last_became_peered":"2023-09-27T23:33:56.555849+0000","last_unstale":"2023-09-27T23:33:56.561289+0000","last_undegraded":"2023-09-27T23:33:56.561289+0000","last_fullsized":"2023-09-27T23:33:56.561289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:55.099301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539954+0000","last_change":"2023-09-27T23:33:56.164762+0000","last_active":"2023-09-27T23:33:56.539954+0000","last_peered":"2023-09-27T23:33:56.539954+0000","last_clean":"2023-09-27T23:33:56.539954+0000","last_became_active":"2023-09-27T23:33:56.164480+0000","last_became_peered":"2023-09-27T23:33:56.164480+0000","last_unstale":"2023-09-27T23:33:56.539954+0000","last_undegraded":"2023-09-27T23:33:56.539954+0000","last_fullsized":"2023-09-27T23:33:56.539954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:27.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'271","reported_seq":824,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928872+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:52.928872+0000","last_peered":"2023-09-27T23:33:52.928872+0000","last_clean":"2023-09-27T23:33:52.928872+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:52.928872+0000","last_undegraded":"2023-09-27T23:33:52.928872+0000","last_fullsized":"2023-09-27T23:33:52.928872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":271,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928736+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:52.928736+0000","last_peered":"2023-09-27T23:33:52.928736+0000","last_clean":"2023-09-27T23:33:52.928736+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:52.928736+0000","last_undegraded":"2023-09-27T23:33:52.928736+0000","last_fullsized":"2023-09-27T23:33:52.928736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.971677+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:51.971677+0000","last_peered":"2023-09-27T23:33:51.971677+0000","last_clean":"2023-09-27T23:33:51.971677+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:51.971677+0000","last_undegraded":"2023-09-27T23:33:51.971677+0000","last_fullsized":"2023-09-27T23:33:51.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540062+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:56.540062+0000","last_peered":"2023-09-27T23:33:56.540062+0000","last_clean":"2023-09-27T23:33:56.540062+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:56.540062+0000","last_undegraded":"2023-09-27T23:33:56.540062+0000","last_fullsized":"2023-09-27T23:33:56.540062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928702+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:52.928702+0000","last_peered":"2023-09-27T23:33:52.928702+0000","last_clean":"2023-09-27T23:33:52.928702+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:52.928702+0000","last_undegraded":"2023-09-27T23:33:52.928702+0000","last_fullsized":"2023-09-27T23:33:52.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540304+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:56.540304+0000","last_peered":"2023-09-27T23:33:56.540304+0000","last_clean":"2023-09-27T23:33:56.540304+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:56.540304+0000","last_undegraded":"2023-09-27T23:33:56.540304+0000","last_fullsized":"2023-09-27T23:33:56.540304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538204+0000","last_change":"2023-09-27T23:33:56.170155+0000","last_active":"2023-09-27T23:33:56.538204+0000","last_peered":"2023-09-27T23:33:56.538204+0000","last_clean":"2023-09-27T23:33:56.538204+0000","last_became_active":"2023-09-27T23:33:56.169711+0000","last_became_peered":"2023-09-27T23:33:56.169711+0000","last_unstale":"2023-09-27T23:33:56.538204+0000","last_undegraded":"2023-09-27T23:33:56.538204+0000","last_fullsized":"2023-09-27T23:33:56.538204+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:24:27.214773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.927845+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:52.927845+0000","last_peered":"2023-09-27T23:33:52.927845+0000","last_clean":"2023-09-27T23:33:52.927845+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:52.927845+0000","last_undegraded":"2023-09-27T23:33:52.927845+0000","last_fullsized":"2023-09-27T23:33:52.927845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509683,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":1.006},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.69999999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.125}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.879}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.155}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.214},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":1.097}]}]},{"osd":6,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28424,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94343416,"statfs":{"total":96636764160,"available":96607657984,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":0.997},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.087},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.149}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.022}]}]},{"osd":5,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38088,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":36224,"kb_avail":94333752,"statfs":{"total":96636764160,"available":96597762048,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37093376},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":1.498}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":1.1659999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":1.546},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.149},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.3129999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.431},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.2170000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":24060,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":94347780,"statfs":{"total":96636764160,"available":96612126720,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":1.133},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":1.1819999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.90300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.749},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.87},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.2370000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.0169999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509682,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":27264,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27918336},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":1.161},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.036}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":1.321},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":1.5620000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.6000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.762}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.887},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.5029999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22908,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94348932,"statfs":{"total":96636764160,"available":96613306368,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.86499999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.93899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.03},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":1.0660000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":1.179},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.272}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542390,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38332,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94333508,"statfs":{"total":96636764160,"available":96597512192,"internally_reserved":0,"allocated":1953792,"data_stored":1019257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0349999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.099},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.165},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.133}]}]},{"osd":0,"up_from":15,"seq":64424509684,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":26880,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27525120},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":1.194},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.2869999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.831}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.391}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:34:00.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2023-09-27T23:34:00.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2023-09-27T23:34:00.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2023-09-27T23:34:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2023-09-27T23:34:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2023-09-27T23:34:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1d is not active+clean 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2023-09-27T23:34:00.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1c is not active+clean 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1a is not active+clean 2023-09-27T23:34:00.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.18 is not active+clean 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2023-09-27T23:34:00.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.f is not active+clean 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2023-09-27T23:34:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2023-09-27T23:34:00.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2023-09-27T23:34:00.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2023-09-27T23:34:00.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2023-09-27T23:34:00.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:55.121414+0000', 'last_change': '2023-09-27T23:33:55.121414+0000', 'last_active': '2023-09-27T23:33:55.121414+0000', 'last_peered': '2023-09-27T23:33:55.121414+0000', 'last_clean': '2023-09-27T23:33:55.121414+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:55.121414+0000', 'last_undegraded': '2023-09-27T23:33:55.121414+0000', 'last_fullsized': '2023-09-27T23:33:55.121414+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:55.121414+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2023-09-27T23:34:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:33:56.133650+0000', 'last_change': '2023-09-27T23:33:56.133650+0000', 'last_active': '2023-09-27T23:33:56.133650+0000', 'last_peered': '2023-09-27T23:33:56.133650+0000', 'last_clean': '2023-09-27T23:33:56.133650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:33:56.133650+0000', 'last_undegraded': '2023-09-27T23:33:56.133650+0000', 'last_fullsized': '2023-09-27T23:33:56.133650+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'last_clean_scrub_stamp': '2023-09-27T23:33:56.133650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:34:00.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:34:00.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 Processor -- start 2023-09-27T23:34:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- start start 2023-09-27T23:34:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f8c0b1410 0x7f1f8c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f8c06aa00 0x7f1f8c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c06c7c0 con 0x7f1f8c06a580 2023-09-27T23:34:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c06cd80 con 0x7f1f8c0b1410 2023-09-27T23:34:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c06b8f0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f8c0b1410 0x7f1f8c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f8c0b1410 0x7f1f8c0ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36186/0 (socket says 172.21.15.64:36186) 2023-09-27T23:34:00.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 -- 172.21.15.64:0/2396075100 learned_addr learned my addr 172.21.15.64:0/2396075100 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:00.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f919bf700 1 --2- 172.21.15.64:0/2396075100 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c06aa00 0x7f1f8c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 -- 172.21.15.64:0/2396075100 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c06aa00 msgr2=0x7f1f8c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 --2- 172.21.15.64:0/2396075100 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c06aa00 0x7f1f8c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 -- 172.21.15.64:0/2396075100 >> v1:172.21.15.64:6789/0 conn(0x7f1f8c06a580 legacy=0x7f1f8c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 -- 172.21.15.64:0/2396075100 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0ac1e0 con 0x7f1f8c0b1410 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f919bf700 1 --2- 172.21.15.64:0/2396075100 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c06aa00 0x7f1f8c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f911be700 1 --2- 172.21.15.64:0/2396075100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c0b1410 0x7f1f8c0ab440 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c15b900 tx=0x7f1f88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b502a62cadcf53a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f83fff700 1 -- 172.21.15.64:0/2396075100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800e070 con 0x7f1f8c0b1410 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f83fff700 1 -- 172.21.15.64:0/2396075100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800ba50 con 0x7f1f8c0b1410 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f83fff700 1 -- 172.21.15.64:0/2396075100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880092e0 con 0x7f1f8c0b1410 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- 172.21.15.64:0/2396075100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c0b1410 msgr2=0x7f1f8c0ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 --2- 172.21.15.64:0/2396075100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c0b1410 0x7f1f8c0ab440 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c15b900 tx=0x7f1f88009840 comp rx=0 tx=0).stop 2023-09-27T23:34:00.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- 172.21.15.64:0/2396075100 shutdown_connections 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 --2- 172.21.15.64:0/2396075100 >> v2:172.21.15.123:3300/0 conn(0x7f1f8c06aa00 0x7f1f8c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 --2- 172.21.15.64:0/2396075100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c0b1410 0x7f1f8c0ab440 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- 172.21.15.64:0/2396075100 >> 172.21.15.64:0/2396075100 conn(0x7f1f8c067bb0 msgr2=0x7f1f8c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- 172.21.15.64:0/2396075100 shutdown_connections 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.350+0000 7f1f921c0700 1 -- 172.21.15.64:0/2396075100 wait complete. 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 Processor -- start 2023-09-27T23:34:00.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 -- start start 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c06a580 0x7f1f8c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 0x7f1f8c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b1410 0x7f1f8c14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c16a140 con 0x7f1f8c06aa00 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c158b00 con 0x7f1f8c06a580 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f921c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c169e60 con 0x7f1f8c0b1410 2023-09-27T23:34:00.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 0x7f1f8c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 0x7f1f8c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36194/0 (socket says 172.21.15.64:36194) 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 -- 172.21.15.64:0/3066146700 learned_addr learned my addr 172.21.15.64:0/3066146700 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f911be700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c06a580 0x7f1f8c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f919bf700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b1410 0x7f1f8c14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 -- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b1410 msgr2=0x7f1f8c14a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b1410 0x7f1f8c14a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 -- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c06a580 msgr2=0x7f1f8c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c06a580 0x7f1f8c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 -- 172.21.15.64:0/3066146700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88009ae0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f911be700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c06a580 0x7f1f8c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f919bf700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b1410 0x7f1f8c14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:34:00.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f909bd700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 0x7f1f8c0add40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c590 tx=0x7f1f7c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.354+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c011590 con 0x7f1f8c06aa00 2023-09-27T23:34:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0116f0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c1584b0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c011ab0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c0ae280 con 0x7f1f8c06aa00 2023-09-27T23:34:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c1582f0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1f7c01b3b0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f817fa700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f70060aa0 0x7f1f70062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1f7c07c6f0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.358+0000 7f1f911be700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f70060aa0 0x7f1f70062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.362+0000 7f1f911be700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f70060aa0 0x7f1f70062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009880 tx=0x7f1f88018000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.366+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1f7c04c720 con 0x7f1f8c06aa00 2023-09-27T23:34:00.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.586+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1f8c0ae470 con 0x7f1f8c06aa00 2023-09-27T23:34:00.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.586+0000 7f1f817fa700 1 -- 172.21.15.64:0/3066146700 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3224 (secure 0 0 0) 0x7f1f7c017cb0 con 0x7f1f8c06aa00 2023-09-27T23:34:00.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:34:00.592 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1239,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":157,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":82},{"state_name":"active+clean","count":23}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":244158464,"bytes_avail":772849954816,"bytes_total":773094113280,"unknown_pgs_ratio":0.78095239400863647},"fsmap":{"epoch":60,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9183}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-09-27T23:33:45.065237+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f70060aa0 msgr2=0x7f1f70062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f70060aa0 0x7f1f70062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009880 tx=0x7f1f88018000 comp rx=0 tx=0).stop 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 msgr2=0x7f1f8c0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 0x7f1f8c0add40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c590 tx=0x7f1f7c00c950 comp rx=0 tx=0).stop 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 shutdown_connections 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1f70060aa0 0x7f1f70062f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f8c0b1410 0x7f1f8c14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c06aa00 0x7f1f8c0add40 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 --2- 172.21.15.64:0/3066146700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c06a580 0x7f1f8c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 >> 172.21.15.64:0/3066146700 conn(0x7f1f8c067bb0 msgr2=0x7f1f8c0b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 shutdown_connections 2023-09-27T23:34:00.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.590+0000 7f1f921c0700 1 -- 172.21.15.64:0/3066146700 wait complete. 2023-09-27T23:34:00.612 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:34:00.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:34:00.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 Processor -- start 2023-09-27T23:34:00.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 -- start start 2023-09-27T23:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0714154ff0 0x7f071406a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07140c6980 0x7f071406aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07140ac1b0 con 0x7f0714154c20 2023-09-27T23:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07140c6f50 con 0x7f0714154ff0 2023-09-27T23:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a951700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07140c7230 con 0x7f07140c6980 2023-09-27T23:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071914e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0714154ff0 0x7f071406a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071994f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0714154c20 0x7f071406b730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34908/0 (socket says 172.21.15.64:34908) 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071994f700 1 -- 172.21.15.64:0/1279673426 learned_addr learned my addr 172.21.15.64:0/1279673426 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071a150700 1 --2- 172.21.15.64:0/1279673426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07140c6980 0x7f071406aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071914e700 1 -- 172.21.15.64:0/1279673426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07140c6980 msgr2=0x7f071406aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071914e700 1 --2- 172.21.15.64:0/1279673426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07140c6980 0x7f071406aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071914e700 1 -- 172.21.15.64:0/1279673426 >> v1:172.21.15.64:6789/0 conn(0x7f0714154c20 legacy=0x7f071406b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071914e700 1 -- 172.21.15.64:0/1279673426 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07140c7510 con 0x7f0714154ff0 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071914e700 1 --2- 172.21.15.64:0/1279673426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714154ff0 0x7f071406a580 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f07100098e0 tx=0x7f07100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1533fe5e3c323a38 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071894d700 1 -- 172.21.15.64:0/1279673426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710009e80 con 0x7f0714154ff0 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071894d700 1 -- 172.21.15.64:0/1279673426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071000b8f0 con 0x7f0714154ff0 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.714+0000 7f071894d700 1 -- 172.21.15.64:0/1279673426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000bc70 con 0x7f0714154ff0 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 -- 172.21.15.64:0/1279673426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714154ff0 msgr2=0x7f071406a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 --2- 172.21.15.64:0/1279673426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714154ff0 0x7f071406a580 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f07100098e0 tx=0x7f07100091a0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 -- 172.21.15.64:0/1279673426 shutdown_connections 2023-09-27T23:34:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 --2- 172.21.15.64:0/1279673426 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07140c6980 0x7f071406aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 --2- 172.21.15.64:0/1279673426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0714154ff0 0x7f071406a580 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 -- 172.21.15.64:0/1279673426 >> 172.21.15.64:0/1279673426 conn(0x7f0714067bb0 msgr2=0x7f07140cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 -- 172.21.15.64:0/1279673426 shutdown_connections 2023-09-27T23:34:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 -- 172.21.15.64:0/1279673426 wait complete. 2023-09-27T23:34:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 Processor -- start 2023-09-27T23:34:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.718+0000 7f071a951700 1 -- start start 2023-09-27T23:34:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 0x7f07140afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0714154c20 0x7f071415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0714154ff0 0x7f0714145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714147740 con 0x7f07140c6980 2023-09-27T23:34:00.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071415ebe0 con 0x7f0714154c20 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0714145a70 con 0x7f0714154ff0 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 0x7f07140afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 0x7f07140afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36216/0 (socket says 172.21.15.64:36216) 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 -- 172.21.15.64:0/2197030784 learned_addr learned my addr 172.21.15.64:0/2197030784 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071914e700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0714154c20 0x7f071415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a150700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0714154ff0 0x7f0714145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 -- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0714154ff0 msgr2=0x7f0714145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0714154ff0 0x7f0714145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 -- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0714154c20 msgr2=0x7f071415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0714154c20 0x7f071415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 -- 172.21.15.64:0/2197030784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710009420 con 0x7f07140c6980 2023-09-27T23:34:00.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071914e700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0714154c20 0x7f071415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071994f700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 0x7f07140afb60 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f070400c7f0 tx=0x7f0704007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704005b30 con 0x7f07140c6980 2023-09-27T23:34:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0704005c90 con 0x7f07140c6980 2023-09-27T23:34:00.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07140b0240 con 0x7f07140c6980 2023-09-27T23:34:00.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704007320 con 0x7f07140c6980 2023-09-27T23:34:00.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.722+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07140b0800 con 0x7f07140c6980 2023-09-27T23:34:00.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.726+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071404be40 con 0x7f07140c6980 2023-09-27T23:34:00.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.726+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0704007690 con 0x7f07140c6980 2023-09-27T23:34:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.726+0000 7f0709ffb700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f06f8060a70 0x7f06f8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.726+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f070407c690 con 0x7f07140c6980 2023-09-27T23:34:00.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.726+0000 7f071914e700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f06f8060a70 0x7f06f8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:00.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.726+0000 7f071914e700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f06f8060a70 0x7f06f8062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f07100091e0 tx=0x7f0710017000 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:00.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.734+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f070404c6c0 con 0x7f07140c6980 2023-09-27T23:34:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.890+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f071414b950 con 0x7f06f8060a70 2023-09-27T23:34:00.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.910+0000 7f0709ffb700 1 -- 172.21.15.64:0/2197030784 <== mgr.8979 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294973 (secure 0 0 0) 0x7f071414b950 con 0x7f06f8060a70 2023-09-27T23:34:00.916 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:34:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f06f8060a70 msgr2=0x7f06f8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f06f8060a70 0x7f06f8062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f07100091e0 tx=0x7f0710017000 comp rx=0 tx=0).stop 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 msgr2=0x7f07140afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 0x7f07140afb60 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f070400c7f0 tx=0x7f0704007f30 comp rx=0 tx=0).stop 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 shutdown_connections 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f06f8060a70 0x7f06f8062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0714154ff0 0x7f0714145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0714154c20 0x7f071415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 --2- 172.21.15.64:0/2197030784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07140c6980 0x7f07140afb60 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 >> 172.21.15.64:0/2197030784 conn(0x7f0714067bb0 msgr2=0x7f071414c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 shutdown_connections 2023-09-27T23:34:00.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:00.914+0000 7f071a951700 1 -- 172.21.15.64:0/2197030784 wait complete. 2023-09-27T23:34:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:34:00.931 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:33:59.069760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":46,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238436,"kb_used_data":16036,"kb_used_omap":0,"kb_used_meta":222336,"kb_avail":754736284,"statfs":{"total":773094113280,"available":772849954816,"internally_reserved":0,"allocated":16420864,"data_stored":9123611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":227672064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001269"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557251+0000","last_change":"2023-09-27T23:33:56.552460+0000","last_active":"2023-09-27T23:33:56.557251+0000","last_peered":"2023-09-27T23:33:56.557251+0000","last_clean":"2023-09-27T23:33:56.557251+0000","last_became_active":"2023-09-27T23:33:56.552129+0000","last_became_peered":"2023-09-27T23:33:56.552129+0000","last_unstale":"2023-09-27T23:33:56.557251+0000","last_undegraded":"2023-09-27T23:33:56.557251+0000","last_fullsized":"2023-09-27T23:33:56.557251+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:21.122858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552313+0000","last_change":"2023-09-27T23:33:56.550009+0000","last_active":"2023-09-27T23:33:56.552313+0000","last_peered":"2023-09-27T23:33:56.552313+0000","last_clean":"2023-09-27T23:33:56.552313+0000","last_became_active":"2023-09-27T23:33:56.549761+0000","last_became_peered":"2023-09-27T23:33:56.549761+0000","last_unstale":"2023-09-27T23:33:56.552313+0000","last_undegraded":"2023-09-27T23:33:56.552313+0000","last_fullsized":"2023-09-27T23:33:56.552313+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:39.341931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550385+0000","last_change":"2023-09-27T23:33:56.543868+0000","last_active":"2023-09-27T23:33:56.550385+0000","last_peered":"2023-09-27T23:33:56.550385+0000","last_clean":"2023-09-27T23:33:56.550385+0000","last_became_active":"2023-09-27T23:33:56.543562+0000","last_became_peered":"2023-09-27T23:33:56.543562+0000","last_unstale":"2023-09-27T23:33:56.550385+0000","last_undegraded":"2023-09-27T23:33:56.550385+0000","last_fullsized":"2023-09-27T23:33:56.550385+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:30:55.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552498+0000","last_change":"2023-09-27T23:33:56.547885+0000","last_active":"2023-09-27T23:33:56.552498+0000","last_peered":"2023-09-27T23:33:56.552498+0000","last_clean":"2023-09-27T23:33:56.552498+0000","last_became_active":"2023-09-27T23:33:56.547546+0000","last_became_peered":"2023-09-27T23:33:56.547546+0000","last_unstale":"2023-09-27T23:33:56.552498+0000","last_undegraded":"2023-09-27T23:33:56.552498+0000","last_fullsized":"2023-09-27T23:33:56.552498+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:54.697434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569173+0000","last_change":"2023-09-27T23:33:56.565920+0000","last_active":"2023-09-27T23:33:56.569173+0000","last_peered":"2023-09-27T23:33:56.569173+0000","last_clean":"2023-09-27T23:33:56.569173+0000","last_became_active":"2023-09-27T23:33:56.565071+0000","last_became_peered":"2023-09-27T23:33:56.565071+0000","last_unstale":"2023-09-27T23:33:56.569173+0000","last_undegraded":"2023-09-27T23:33:56.569173+0000","last_fullsized":"2023-09-27T23:33:56.569173+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:02:25.734384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569208+0000","last_change":"2023-09-27T23:33:56.566463+0000","last_active":"2023-09-27T23:33:56.569208+0000","last_peered":"2023-09-27T23:33:56.569208+0000","last_clean":"2023-09-27T23:33:56.569208+0000","last_became_active":"2023-09-27T23:33:56.566181+0000","last_became_peered":"2023-09-27T23:33:56.566181+0000","last_unstale":"2023-09-27T23:33:56.569208+0000","last_undegraded":"2023-09-27T23:33:56.569208+0000","last_fullsized":"2023-09-27T23:33:56.569208+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:11:49.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550358+0000","last_change":"2023-09-27T23:33:56.542275+0000","last_active":"2023-09-27T23:33:56.550358+0000","last_peered":"2023-09-27T23:33:56.550358+0000","last_clean":"2023-09-27T23:33:56.550358+0000","last_became_active":"2023-09-27T23:33:56.542017+0000","last_became_peered":"2023-09-27T23:33:56.542017+0000","last_unstale":"2023-09-27T23:33:56.550358+0000","last_undegraded":"2023-09-27T23:33:56.550358+0000","last_fullsized":"2023-09-27T23:33:56.550358+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:24.846142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561352+0000","last_change":"2023-09-27T23:33:56.556873+0000","last_active":"2023-09-27T23:33:56.561352+0000","last_peered":"2023-09-27T23:33:56.561352+0000","last_clean":"2023-09-27T23:33:56.561352+0000","last_became_active":"2023-09-27T23:33:56.556561+0000","last_became_peered":"2023-09-27T23:33:56.556561+0000","last_unstale":"2023-09-27T23:33:56.561352+0000","last_undegraded":"2023-09-27T23:33:56.561352+0000","last_fullsized":"2023-09-27T23:33:56.561352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:01.494681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557128+0000","last_change":"2023-09-27T23:33:56.552372+0000","last_active":"2023-09-27T23:33:56.557128+0000","last_peered":"2023-09-27T23:33:56.557128+0000","last_clean":"2023-09-27T23:33:56.557128+0000","last_became_active":"2023-09-27T23:33:56.551956+0000","last_became_peered":"2023-09-27T23:33:56.551956+0000","last_unstale":"2023-09-27T23:33:56.557128+0000","last_undegraded":"2023-09-27T23:33:56.557128+0000","last_fullsized":"2023-09-27T23:33:56.557128+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:12.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539969+0000","last_change":"2023-09-27T23:33:56.171325+0000","last_active":"2023-09-27T23:33:56.539969+0000","last_peered":"2023-09-27T23:33:56.539969+0000","last_clean":"2023-09-27T23:33:56.539969+0000","last_became_active":"2023-09-27T23:33:56.170836+0000","last_became_peered":"2023-09-27T23:33:56.170836+0000","last_unstale":"2023-09-27T23:33:56.539969+0000","last_undegraded":"2023-09-27T23:33:56.539969+0000","last_fullsized":"2023-09-27T23:33:56.539969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:56:07.324327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539936+0000","last_change":"2023-09-27T23:33:56.165693+0000","last_active":"2023-09-27T23:33:56.539936+0000","last_peered":"2023-09-27T23:33:56.539936+0000","last_clean":"2023-09-27T23:33:56.539936+0000","last_became_active":"2023-09-27T23:33:56.165301+0000","last_became_peered":"2023-09-27T23:33:56.165301+0000","last_unstale":"2023-09-27T23:33:56.539936+0000","last_undegraded":"2023-09-27T23:33:56.539936+0000","last_fullsized":"2023-09-27T23:33:56.539936+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:17.631606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561289+0000","last_change":"2023-09-27T23:33:56.556576+0000","last_active":"2023-09-27T23:33:56.561289+0000","last_peered":"2023-09-27T23:33:56.561289+0000","last_clean":"2023-09-27T23:33:56.561289+0000","last_became_active":"2023-09-27T23:33:56.555849+0000","last_became_peered":"2023-09-27T23:33:56.555849+0000","last_unstale":"2023-09-27T23:33:56.561289+0000","last_undegraded":"2023-09-27T23:33:56.561289+0000","last_fullsized":"2023-09-27T23:33:56.561289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:55.099301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539954+0000","last_change":"2023-09-27T23:33:56.164762+0000","last_active":"2023-09-27T23:33:56.539954+0000","last_peered":"2023-09-27T23:33:56.539954+0000","last_clean":"2023-09-27T23:33:56.539954+0000","last_became_active":"2023-09-27T23:33:56.164480+0000","last_became_peered":"2023-09-27T23:33:56.164480+0000","last_unstale":"2023-09-27T23:33:56.539954+0000","last_undegraded":"2023-09-27T23:33:56.539954+0000","last_fullsized":"2023-09-27T23:33:56.539954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:27.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'271","reported_seq":824,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928872+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:52.928872+0000","last_peered":"2023-09-27T23:33:52.928872+0000","last_clean":"2023-09-27T23:33:52.928872+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:52.928872+0000","last_undegraded":"2023-09-27T23:33:52.928872+0000","last_fullsized":"2023-09-27T23:33:52.928872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":271,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928736+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:52.928736+0000","last_peered":"2023-09-27T23:33:52.928736+0000","last_clean":"2023-09-27T23:33:52.928736+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:52.928736+0000","last_undegraded":"2023-09-27T23:33:52.928736+0000","last_fullsized":"2023-09-27T23:33:52.928736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.971677+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:51.971677+0000","last_peered":"2023-09-27T23:33:51.971677+0000","last_clean":"2023-09-27T23:33:51.971677+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:51.971677+0000","last_undegraded":"2023-09-27T23:33:51.971677+0000","last_fullsized":"2023-09-27T23:33:51.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:51.975281+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:51.975281+0000","last_peered":"2023-09-27T23:33:51.975281+0000","last_clean":"2023-09-27T23:33:51.975281+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:51.975281+0000","last_undegraded":"2023-09-27T23:33:51.975281+0000","last_fullsized":"2023-09-27T23:33:51.975281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540062+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:56.540062+0000","last_peered":"2023-09-27T23:33:56.540062+0000","last_clean":"2023-09-27T23:33:56.540062+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:56.540062+0000","last_undegraded":"2023-09-27T23:33:56.540062+0000","last_fullsized":"2023-09-27T23:33:56.540062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.928702+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:52.928702+0000","last_peered":"2023-09-27T23:33:52.928702+0000","last_clean":"2023-09-27T23:33:52.928702+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:52.928702+0000","last_undegraded":"2023-09-27T23:33:52.928702+0000","last_fullsized":"2023-09-27T23:33:52.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:55.121414+0000","last_change":"2023-09-27T23:33:55.121414+0000","last_active":"2023-09-27T23:33:55.121414+0000","last_peered":"2023-09-27T23:33:55.121414+0000","last_clean":"2023-09-27T23:33:55.121414+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:55.121414+0000","last_undegraded":"2023-09-27T23:33:55.121414+0000","last_fullsized":"2023-09-27T23:33:55.121414+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540304+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:56.540304+0000","last_peered":"2023-09-27T23:33:56.540304+0000","last_clean":"2023-09-27T23:33:56.540304+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:56.540304+0000","last_undegraded":"2023-09-27T23:33:56.540304+0000","last_fullsized":"2023-09-27T23:33:56.540304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:33:56.133650+0000","last_change":"2023-09-27T23:33:56.133650+0000","last_active":"2023-09-27T23:33:56.133650+0000","last_peered":"2023-09-27T23:33:56.133650+0000","last_clean":"2023-09-27T23:33:56.133650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:33:56.133650+0000","last_undegraded":"2023-09-27T23:33:56.133650+0000","last_fullsized":"2023-09-27T23:33:56.133650+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538204+0000","last_change":"2023-09-27T23:33:56.170155+0000","last_active":"2023-09-27T23:33:56.538204+0000","last_peered":"2023-09-27T23:33:56.538204+0000","last_clean":"2023-09-27T23:33:56.538204+0000","last_became_active":"2023-09-27T23:33:56.169711+0000","last_became_peered":"2023-09-27T23:33:56.169711+0000","last_unstale":"2023-09-27T23:33:56.538204+0000","last_undegraded":"2023-09-27T23:33:56.538204+0000","last_fullsized":"2023-09-27T23:33:56.538204+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:24:27.214773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-09-27T23:33:52.927845+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:52.927845+0000","last_peered":"2023-09-27T23:33:52.927845+0000","last_clean":"2023-09-27T23:33:52.927845+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:52.927845+0000","last_undegraded":"2023-09-27T23:33:52.927845+0000","last_fullsized":"2023-09-27T23:33:52.927845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509683,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":1.006},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.69999999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":1.125}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.879}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.155}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.214},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":1.097}]}]},{"osd":6,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28424,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94343416,"statfs":{"total":96636764160,"available":96607657984,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":0.997},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":0.77700000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":1.087},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.149}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.022}]}]},{"osd":5,"up_from":15,"seq":64424509682,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38088,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":36224,"kb_avail":94333752,"statfs":{"total":96636764160,"available":96597762048,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37093376},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":1.498}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":1.1659999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":1.546},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.149},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.3129999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.431},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.2170000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":24060,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":94347780,"statfs":{"total":96636764160,"available":96612126720,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":1.133},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":1.1819999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":0.90300000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.749},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.87},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.2370000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.0169999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509682,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":27264,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":1900544,"data_stored":990892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27918336},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":1.161},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.036}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":1.321},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":1.5620000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.6000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.762}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.887},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.5029999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22908,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":94348932,"statfs":{"total":96636764160,"available":96613306368,"internally_reserved":0,"allocated":1888256,"data_stored":984579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.86499999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.93899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.03},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":1.0660000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":1.179},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.272}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":1.365}]}]},{"osd":1,"up_from":14,"seq":60129542390,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38332,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94333508,"statfs":{"total":96636764160,"available":96597512192,"internally_reserved":0,"allocated":1953792,"data_stored":1019257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0349999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.099},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.165},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.133}]}]},{"osd":0,"up_from":15,"seq":64424509684,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29324,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":26880,"kb_avail":94342516,"statfs":{"total":96636764160,"available":96606736384,"internally_reserved":0,"allocated":2494464,"data_stored":1581260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27525120},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":1.194},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.2869999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.831}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.391}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:34:02.455 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-09-27T23:34:02.450+0000 7f555da08700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:34:03.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:34:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.054+0000 7f742968c700 1 Processor -- start 2023-09-27T23:34:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.054+0000 7f742968c700 1 -- start start 2023-09-27T23:34:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155990 0x7f74240c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74240b1580 0x7f742406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742406cf00 con 0x7f74240b0670 2023-09-27T23:34:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240ae7e0 con 0x7f74240b1580 2023-09-27T23:34:04.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74240cbc50 con 0x7f7424155990 2023-09-27T23:34:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74240b1580 0x7f742406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7423fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155990 0x7f74240c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f741bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f74240b0670 0x7f74240b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:57194/0 (socket says 172.21.15.64:57194) 2023-09-27T23:34:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f741bfff700 1 -- 172.21.15.64:0/2018907381 learned_addr learned my addr 172.21.15.64:0/2018907381 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74240b1580 0x7f742406d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:55940/0 (socket says 172.21.15.64:55940) 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f74237fe700 1 -- 172.21.15.64:0/2018907381 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776579969 0 0) 0x7f742406cf00 con 0x7f74240b0670 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f74237fe700 1 -- 172.21.15.64:0/2018907381 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7410003570 con 0x7f74240b0670 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 -- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155990 msgr2=0x7f74240c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 --2- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155990 0x7f74240c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 -- 172.21.15.64:0/2018907381 >> v1:172.21.15.123:6789/0 conn(0x7f74240b0670 legacy=0x7f74240b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 -- 172.21.15.64:0/2018907381 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240b01d0 con 0x7f74240b1580 2023-09-27T23:34:04.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7423fff700 1 --2- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155990 0x7f74240c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f7428e8b700 1 --2- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f742406d390 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f741c009e60 tx=0x7f741c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd6371f9422c57d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f74237fe700 1 -- 172.21.15.64:0/2018907381 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00e070 con 0x7f74240b1580 2023-09-27T23:34:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f74237fe700 1 -- 172.21.15.64:0/2018907381 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c00bab0 con 0x7f74240b1580 2023-09-27T23:34:04.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f74237fe700 1 -- 172.21.15.64:0/2018907381 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00be70 con 0x7f74240b1580 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 msgr2=0x7f742406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 --2- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f742406d390 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f741c009e60 tx=0x7f741c009780 comp rx=0 tx=0).stop 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- 172.21.15.64:0/2018907381 shutdown_connections 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 --2- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f742406d390 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 --2- 172.21.15.64:0/2018907381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7424155990 0x7f74240c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- 172.21.15.64:0/2018907381 >> 172.21.15.64:0/2018907381 conn(0x7f7424067bb0 msgr2=0x7f74240add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.058+0000 7f742968c700 1 -- 172.21.15.64:0/2018907381 shutdown_connections 2023-09-27T23:34:04.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.062+0000 7f742968c700 1 -- 172.21.15.64:0/2018907381 wait complete. 2023-09-27T23:34:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.062+0000 7f742968c700 1 Processor -- start 2023-09-27T23:34:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.062+0000 7f742968c700 1 -- start start 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74240b0670 0x7f74240c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f74240c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 0x7f7424141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240c78f0 con 0x7f7424155990 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7424142500 con 0x7f74240b1580 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74241421f0 con 0x7f74240b0670 2023-09-27T23:34:04.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 0x7f7424141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7423fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74240b0670 0x7f74240c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 0x7f7424141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36226/0 (socket says 172.21.15.64:36226) 2023-09-27T23:34:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 -- 172.21.15.64:0/3941963378 learned_addr learned my addr 172.21.15.64:0/3941963378 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:04.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f741bfff700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f74240c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 -- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74240b0670 msgr2=0x7f74240c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74240b0670 0x7f74240c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 -- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 msgr2=0x7f74240c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f74240c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 -- 172.21.15.64:0/3941963378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7414009420 con 0x7f7424155990 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7428e8b700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 0x7f7424141cb0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f741c013000 tx=0x7f741c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c012e50 con 0x7f7424155990 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c027470 con 0x7f7424155990 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c009a20 con 0x7f7424155990 2023-09-27T23:34:04.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c027830 con 0x7f7424155990 2023-09-27T23:34:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74240c7e00 con 0x7f7424155990 2023-09-27T23:34:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.066+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f741c024070 con 0x7f7424155990 2023-09-27T23:34:04.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.070+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742404be40 con 0x7f7424155990 2023-09-27T23:34:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.070+0000 7f7420ff9700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7404060a70 0x7f7404062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.070+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f741c07f700 con 0x7f7424155990 2023-09-27T23:34:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.070+0000 7f7423fff700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7404060a70 0x7f7404062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.070+0000 7f7423fff700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7404060a70 0x7f7404062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7414009f40 tx=0x7f74140091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.074+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f741c04f730 con 0x7f7424155990 2023-09-27T23:34:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.238+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74240ccc90 con 0x7f7404060a70 2023-09-27T23:34:04.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.254+0000 7f7420ff9700 1 -- 172.21.15.64:0/3941963378 <== mgr.8979 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308069 (secure 0 0 0) 0x7f74240ccc90 con 0x7f7404060a70 2023-09-27T23:34:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7404060a70 msgr2=0x7f7404062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7404060a70 0x7f7404062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7414009f40 tx=0x7f74140091a0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 msgr2=0x7f7424141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 0x7f7424141cb0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f741c013000 tx=0x7f741c019000 comp rx=0 tx=0).stop 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 shutdown_connections 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7404060a70 0x7f7404062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7424155990 0x7f7424141cb0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74240b1580 0x7f74240c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 --2- 172.21.15.64:0/3941963378 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74240b0670 0x7f74240c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 >> 172.21.15.64:0/3941963378 conn(0x7f7424067bb0 msgr2=0x7f74240ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 shutdown_connections 2023-09-27T23:34:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.262+0000 7f742968c700 1 -- 172.21.15.64:0/3941963378 wait complete. 2023-09-27T23:34:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:34:04.281 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:34:03.071647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239972,"kb_used_data":16420,"kb_used_omap":0,"kb_used_meta":223488,"kb_avail":754734748,"statfs":{"total":773094113280,"available":772848381952,"internally_reserved":0,"allocated":16814080,"data_stored":9314089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":228851712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.927788"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557251+0000","last_change":"2023-09-27T23:33:56.552460+0000","last_active":"2023-09-27T23:33:56.557251+0000","last_peered":"2023-09-27T23:33:56.557251+0000","last_clean":"2023-09-27T23:33:56.557251+0000","last_became_active":"2023-09-27T23:33:56.552129+0000","last_became_peered":"2023-09-27T23:33:56.552129+0000","last_unstale":"2023-09-27T23:33:56.557251+0000","last_undegraded":"2023-09-27T23:33:56.557251+0000","last_fullsized":"2023-09-27T23:33:56.557251+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:49:21.122858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.553051+0000","last_change":"2023-09-27T23:33:56.548771+0000","last_active":"2023-09-27T23:33:56.553051+0000","last_peered":"2023-09-27T23:33:56.553051+0000","last_clean":"2023-09-27T23:33:56.553051+0000","last_became_active":"2023-09-27T23:33:56.548413+0000","last_became_peered":"2023-09-27T23:33:56.548413+0000","last_unstale":"2023-09-27T23:33:56.553051+0000","last_undegraded":"2023-09-27T23:33:56.553051+0000","last_fullsized":"2023-09-27T23:33:56.553051+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:32:55.078605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.554265+0000","last_change":"2023-09-27T23:33:56.547643+0000","last_active":"2023-09-27T23:33:56.554265+0000","last_peered":"2023-09-27T23:33:56.554265+0000","last_clean":"2023-09-27T23:33:56.554265+0000","last_became_active":"2023-09-27T23:33:56.547338+0000","last_became_peered":"2023-09-27T23:33:56.547338+0000","last_unstale":"2023-09-27T23:33:56.554265+0000","last_undegraded":"2023-09-27T23:33:56.554265+0000","last_fullsized":"2023-09-27T23:33:56.554265+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:29.111813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564577+0000","last_change":"2023-09-27T23:33:56.559953+0000","last_active":"2023-09-27T23:33:56.564577+0000","last_peered":"2023-09-27T23:33:56.564577+0000","last_clean":"2023-09-27T23:33:56.564577+0000","last_became_active":"2023-09-27T23:33:56.558182+0000","last_became_peered":"2023-09-27T23:33:56.558182+0000","last_unstale":"2023-09-27T23:33:56.564577+0000","last_undegraded":"2023-09-27T23:33:56.564577+0000","last_fullsized":"2023-09-27T23:33:56.564577+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:25:28.471825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.549476+0000","last_change":"2023-09-27T23:33:56.541753+0000","last_active":"2023-09-27T23:33:56.549476+0000","last_peered":"2023-09-27T23:33:56.549476+0000","last_clean":"2023-09-27T23:33:56.549476+0000","last_became_active":"2023-09-27T23:33:56.541387+0000","last_became_peered":"2023-09-27T23:33:56.541387+0000","last_unstale":"2023-09-27T23:33:56.549476+0000","last_undegraded":"2023-09-27T23:33:56.549476+0000","last_fullsized":"2023-09-27T23:33:56.549476+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:32:50.691196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552313+0000","last_change":"2023-09-27T23:33:56.550009+0000","last_active":"2023-09-27T23:33:56.552313+0000","last_peered":"2023-09-27T23:33:56.552313+0000","last_clean":"2023-09-27T23:33:56.552313+0000","last_became_active":"2023-09-27T23:33:56.549761+0000","last_became_peered":"2023-09-27T23:33:56.549761+0000","last_unstale":"2023-09-27T23:33:56.552313+0000","last_undegraded":"2023-09-27T23:33:56.552313+0000","last_fullsized":"2023-09-27T23:33:56.552313+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:55:39.341931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.560872+0000","last_change":"2023-09-27T23:33:56.556097+0000","last_active":"2023-09-27T23:33:56.560872+0000","last_peered":"2023-09-27T23:33:56.560872+0000","last_clean":"2023-09-27T23:33:56.560872+0000","last_became_active":"2023-09-27T23:33:56.555781+0000","last_became_peered":"2023-09-27T23:33:56.555781+0000","last_unstale":"2023-09-27T23:33:56.560872+0000","last_undegraded":"2023-09-27T23:33:56.560872+0000","last_fullsized":"2023-09-27T23:33:56.560872+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:59:51.656216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550385+0000","last_change":"2023-09-27T23:33:56.543868+0000","last_active":"2023-09-27T23:33:56.550385+0000","last_peered":"2023-09-27T23:33:56.550385+0000","last_clean":"2023-09-27T23:33:56.550385+0000","last_became_active":"2023-09-27T23:33:56.543562+0000","last_became_peered":"2023-09-27T23:33:56.543562+0000","last_unstale":"2023-09-27T23:33:56.550385+0000","last_undegraded":"2023-09-27T23:33:56.550385+0000","last_fullsized":"2023-09-27T23:33:56.550385+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:30:55.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.551014+0000","last_change":"2023-09-27T23:33:56.546963+0000","last_active":"2023-09-27T23:33:56.551014+0000","last_peered":"2023-09-27T23:33:56.551014+0000","last_clean":"2023-09-27T23:33:56.551014+0000","last_became_active":"2023-09-27T23:33:56.546648+0000","last_became_peered":"2023-09-27T23:33:56.546648+0000","last_unstale":"2023-09-27T23:33:56.551014+0000","last_undegraded":"2023-09-27T23:33:56.551014+0000","last_fullsized":"2023-09-27T23:33:56.551014+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:16:20.462488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.560195+0000","last_change":"2023-09-27T23:33:56.558441+0000","last_active":"2023-09-27T23:33:56.560195+0000","last_peered":"2023-09-27T23:33:56.560195+0000","last_clean":"2023-09-27T23:33:56.560195+0000","last_became_active":"2023-09-27T23:33:56.558174+0000","last_became_peered":"2023-09-27T23:33:56.558174+0000","last_unstale":"2023-09-27T23:33:56.560195+0000","last_undegraded":"2023-09-27T23:33:56.560195+0000","last_fullsized":"2023-09-27T23:33:56.560195+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:23:41.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552498+0000","last_change":"2023-09-27T23:33:56.547885+0000","last_active":"2023-09-27T23:33:56.552498+0000","last_peered":"2023-09-27T23:33:56.552498+0000","last_clean":"2023-09-27T23:33:56.552498+0000","last_became_active":"2023-09-27T23:33:56.547546+0000","last_became_peered":"2023-09-27T23:33:56.547546+0000","last_unstale":"2023-09-27T23:33:56.552498+0000","last_undegraded":"2023-09-27T23:33:56.552498+0000","last_fullsized":"2023-09-27T23:33:56.552498+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:52:54.697434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556031+0000","last_change":"2023-09-27T23:33:56.549061+0000","last_active":"2023-09-27T23:33:56.556031+0000","last_peered":"2023-09-27T23:33:56.556031+0000","last_clean":"2023-09-27T23:33:56.556031+0000","last_became_active":"2023-09-27T23:33:56.547530+0000","last_became_peered":"2023-09-27T23:33:56.547530+0000","last_unstale":"2023-09-27T23:33:56.556031+0000","last_undegraded":"2023-09-27T23:33:56.556031+0000","last_fullsized":"2023-09-27T23:33:56.556031+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:38:07.887609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.549514+0000","last_change":"2023-09-27T23:33:56.543885+0000","last_active":"2023-09-27T23:33:56.549514+0000","last_peered":"2023-09-27T23:33:56.549514+0000","last_clean":"2023-09-27T23:33:56.549514+0000","last_became_active":"2023-09-27T23:33:56.543543+0000","last_became_peered":"2023-09-27T23:33:56.543543+0000","last_unstale":"2023-09-27T23:33:56.549514+0000","last_undegraded":"2023-09-27T23:33:56.549514+0000","last_fullsized":"2023-09-27T23:33:56.549514+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:29:49.687888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564325+0000","last_change":"2023-09-27T23:33:56.558433+0000","last_active":"2023-09-27T23:33:56.564325+0000","last_peered":"2023-09-27T23:33:56.564325+0000","last_clean":"2023-09-27T23:33:56.564325+0000","last_became_active":"2023-09-27T23:33:56.558058+0000","last_became_peered":"2023-09-27T23:33:56.558058+0000","last_unstale":"2023-09-27T23:33:56.564325+0000","last_undegraded":"2023-09-27T23:33:56.564325+0000","last_fullsized":"2023-09-27T23:33:56.564325+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:53:48.564613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556089+0000","last_change":"2023-09-27T23:33:56.549354+0000","last_active":"2023-09-27T23:33:56.556089+0000","last_peered":"2023-09-27T23:33:56.556089+0000","last_clean":"2023-09-27T23:33:56.556089+0000","last_became_active":"2023-09-27T23:33:56.549018+0000","last_became_peered":"2023-09-27T23:33:56.549018+0000","last_unstale":"2023-09-27T23:33:56.556089+0000","last_undegraded":"2023-09-27T23:33:56.556089+0000","last_fullsized":"2023-09-27T23:33:56.556089+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:30:11.192179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.568412+0000","last_change":"2023-09-27T23:33:56.561724+0000","last_active":"2023-09-27T23:33:56.568412+0000","last_peered":"2023-09-27T23:33:56.568412+0000","last_clean":"2023-09-27T23:33:56.568412+0000","last_became_active":"2023-09-27T23:33:56.560645+0000","last_became_peered":"2023-09-27T23:33:56.560645+0000","last_unstale":"2023-09-27T23:33:56.568412+0000","last_undegraded":"2023-09-27T23:33:56.568412+0000","last_fullsized":"2023-09-27T23:33:56.568412+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:30:11.192179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556231+0000","last_change":"2023-09-27T23:33:56.553081+0000","last_active":"2023-09-27T23:33:56.556231+0000","last_peered":"2023-09-27T23:33:56.556231+0000","last_clean":"2023-09-27T23:33:56.556231+0000","last_became_active":"2023-09-27T23:33:56.552768+0000","last_became_peered":"2023-09-27T23:33:56.552768+0000","last_unstale":"2023-09-27T23:33:56.556231+0000","last_undegraded":"2023-09-27T23:33:56.556231+0000","last_fullsized":"2023-09-27T23:33:56.556231+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:41:44.407055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.554323+0000","last_change":"2023-09-27T23:33:56.547658+0000","last_active":"2023-09-27T23:33:56.554323+0000","last_peered":"2023-09-27T23:33:56.554323+0000","last_clean":"2023-09-27T23:33:56.554323+0000","last_became_active":"2023-09-27T23:33:56.547410+0000","last_became_peered":"2023-09-27T23:33:56.547410+0000","last_unstale":"2023-09-27T23:33:56.554323+0000","last_undegraded":"2023-09-27T23:33:56.554323+0000","last_fullsized":"2023-09-27T23:33:56.554323+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:23:41.288128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569173+0000","last_change":"2023-09-27T23:33:56.565920+0000","last_active":"2023-09-27T23:33:56.569173+0000","last_peered":"2023-09-27T23:33:56.569173+0000","last_clean":"2023-09-27T23:33:56.569173+0000","last_became_active":"2023-09-27T23:33:56.565071+0000","last_became_peered":"2023-09-27T23:33:56.565071+0000","last_unstale":"2023-09-27T23:33:56.569173+0000","last_undegraded":"2023-09-27T23:33:56.569173+0000","last_fullsized":"2023-09-27T23:33:56.569173+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:02:25.734384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.565191+0000","last_change":"2023-09-27T23:33:56.558593+0000","last_active":"2023-09-27T23:33:56.565191+0000","last_peered":"2023-09-27T23:33:56.565191+0000","last_clean":"2023-09-27T23:33:56.565191+0000","last_became_active":"2023-09-27T23:33:56.558309+0000","last_became_peered":"2023-09-27T23:33:56.558309+0000","last_unstale":"2023-09-27T23:33:56.565191+0000","last_undegraded":"2023-09-27T23:33:56.565191+0000","last_fullsized":"2023-09-27T23:33:56.565191+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:06.242895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561042+0000","last_change":"2023-09-27T23:33:56.559023+0000","last_active":"2023-09-27T23:33:56.561042+0000","last_peered":"2023-09-27T23:33:56.561042+0000","last_clean":"2023-09-27T23:33:56.561042+0000","last_became_active":"2023-09-27T23:33:56.558667+0000","last_became_peered":"2023-09-27T23:33:56.558667+0000","last_unstale":"2023-09-27T23:33:56.561042+0000","last_undegraded":"2023-09-27T23:33:56.561042+0000","last_fullsized":"2023-09-27T23:33:56.561042+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:23:34.554266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564455+0000","last_change":"2023-09-27T23:33:56.559389+0000","last_active":"2023-09-27T23:33:56.564455+0000","last_peered":"2023-09-27T23:33:56.564455+0000","last_clean":"2023-09-27T23:33:56.564455+0000","last_became_active":"2023-09-27T23:33:56.558387+0000","last_became_peered":"2023-09-27T23:33:56.558387+0000","last_unstale":"2023-09-27T23:33:56.564455+0000","last_undegraded":"2023-09-27T23:33:56.564455+0000","last_fullsized":"2023-09-27T23:33:56.564455+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:10:33.764490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564315+0000","last_change":"2023-09-27T23:33:56.558339+0000","last_active":"2023-09-27T23:33:56.564315+0000","last_peered":"2023-09-27T23:33:56.564315+0000","last_clean":"2023-09-27T23:33:56.564315+0000","last_became_active":"2023-09-27T23:33:56.558057+0000","last_became_peered":"2023-09-27T23:33:56.558057+0000","last_unstale":"2023-09-27T23:33:56.564315+0000","last_undegraded":"2023-09-27T23:33:56.564315+0000","last_fullsized":"2023-09-27T23:33:56.564315+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:57:17.335290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561071+0000","last_change":"2023-09-27T23:33:56.558961+0000","last_active":"2023-09-27T23:33:56.561071+0000","last_peered":"2023-09-27T23:33:56.561071+0000","last_clean":"2023-09-27T23:33:56.561071+0000","last_became_active":"2023-09-27T23:33:56.558311+0000","last_became_peered":"2023-09-27T23:33:56.558311+0000","last_unstale":"2023-09-27T23:33:56.561071+0000","last_undegraded":"2023-09-27T23:33:56.561071+0000","last_fullsized":"2023-09-27T23:33:56.561071+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:29.111813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564832+0000","last_change":"2023-09-27T23:33:56.544782+0000","last_active":"2023-09-27T23:33:56.564832+0000","last_peered":"2023-09-27T23:33:56.564832+0000","last_clean":"2023-09-27T23:33:56.564832+0000","last_became_active":"2023-09-27T23:33:56.544508+0000","last_became_peered":"2023-09-27T23:33:56.544508+0000","last_unstale":"2023-09-27T23:33:56.564832+0000","last_undegraded":"2023-09-27T23:33:56.564832+0000","last_fullsized":"2023-09-27T23:33:56.564832+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:29:18.395043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569208+0000","last_change":"2023-09-27T23:33:56.566463+0000","last_active":"2023-09-27T23:33:56.569208+0000","last_peered":"2023-09-27T23:33:56.569208+0000","last_clean":"2023-09-27T23:33:56.569208+0000","last_became_active":"2023-09-27T23:33:56.566181+0000","last_became_peered":"2023-09-27T23:33:56.566181+0000","last_unstale":"2023-09-27T23:33:56.569208+0000","last_undegraded":"2023-09-27T23:33:56.569208+0000","last_fullsized":"2023-09-27T23:33:56.569208+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:11:49.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.565271+0000","last_change":"2023-09-27T23:33:56.558500+0000","last_active":"2023-09-27T23:33:56.565271+0000","last_peered":"2023-09-27T23:33:56.565271+0000","last_clean":"2023-09-27T23:33:56.565271+0000","last_became_active":"2023-09-27T23:33:56.558145+0000","last_became_peered":"2023-09-27T23:33:56.558145+0000","last_unstale":"2023-09-27T23:33:56.565271+0000","last_undegraded":"2023-09-27T23:33:56.565271+0000","last_fullsized":"2023-09-27T23:33:56.565271+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:50:30.749787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550819+0000","last_change":"2023-09-27T23:33:56.545166+0000","last_active":"2023-09-27T23:33:56.550819+0000","last_peered":"2023-09-27T23:33:56.550819+0000","last_clean":"2023-09-27T23:33:56.550819+0000","last_became_active":"2023-09-27T23:33:56.544591+0000","last_became_peered":"2023-09-27T23:33:56.544591+0000","last_unstale":"2023-09-27T23:33:56.550819+0000","last_undegraded":"2023-09-27T23:33:56.550819+0000","last_fullsized":"2023-09-27T23:33:56.550819+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:56.078159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550358+0000","last_change":"2023-09-27T23:33:56.542275+0000","last_active":"2023-09-27T23:33:56.550358+0000","last_peered":"2023-09-27T23:33:56.550358+0000","last_clean":"2023-09-27T23:33:56.550358+0000","last_became_active":"2023-09-27T23:33:56.542017+0000","last_became_peered":"2023-09-27T23:33:56.542017+0000","last_unstale":"2023-09-27T23:33:56.550358+0000","last_undegraded":"2023-09-27T23:33:56.550358+0000","last_fullsized":"2023-09-27T23:33:56.550358+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:24.846142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564264+0000","last_change":"2023-09-27T23:33:56.558245+0000","last_active":"2023-09-27T23:33:56.564264+0000","last_peered":"2023-09-27T23:33:56.564264+0000","last_clean":"2023-09-27T23:33:56.564264+0000","last_became_active":"2023-09-27T23:33:56.557710+0000","last_became_peered":"2023-09-27T23:33:56.557710+0000","last_unstale":"2023-09-27T23:33:56.564264+0000","last_undegraded":"2023-09-27T23:33:56.564264+0000","last_fullsized":"2023-09-27T23:33:56.564264+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:11:50.622521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.552980+0000","last_change":"2023-09-27T23:33:56.547922+0000","last_active":"2023-09-27T23:33:56.552980+0000","last_peered":"2023-09-27T23:33:56.552980+0000","last_clean":"2023-09-27T23:33:56.552980+0000","last_became_active":"2023-09-27T23:33:56.547601+0000","last_became_peered":"2023-09-27T23:33:56.547601+0000","last_unstale":"2023-09-27T23:33:56.552980+0000","last_undegraded":"2023-09-27T23:33:56.552980+0000","last_fullsized":"2023-09-27T23:33:56.552980+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:41.210573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.568433+0000","last_change":"2023-09-27T23:33:56.562052+0000","last_active":"2023-09-27T23:33:56.568433+0000","last_peered":"2023-09-27T23:33:56.568433+0000","last_clean":"2023-09-27T23:33:56.568433+0000","last_became_active":"2023-09-27T23:33:56.561106+0000","last_became_peered":"2023-09-27T23:33:56.561106+0000","last_unstale":"2023-09-27T23:33:56.568433+0000","last_undegraded":"2023-09-27T23:33:56.568433+0000","last_fullsized":"2023-09-27T23:33:56.568433+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:37.524210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556166+0000","last_change":"2023-09-27T23:33:56.550959+0000","last_active":"2023-09-27T23:33:56.556166+0000","last_peered":"2023-09-27T23:33:56.556166+0000","last_clean":"2023-09-27T23:33:56.556166+0000","last_became_active":"2023-09-27T23:33:56.550674+0000","last_became_peered":"2023-09-27T23:33:56.550674+0000","last_unstale":"2023-09-27T23:33:56.556166+0000","last_undegraded":"2023-09-27T23:33:56.556166+0000","last_fullsized":"2023-09-27T23:33:56.556166+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:59.590003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"156'4","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.554141+0000","last_change":"2023-09-27T23:33:56.174348+0000","last_active":"2023-09-27T23:33:56.554141+0000","last_peered":"2023-09-27T23:33:56.554141+0000","last_clean":"2023-09-27T23:33:56.554141+0000","last_became_active":"2023-09-27T23:33:56.173884+0000","last_became_peered":"2023-09-27T23:33:56.173884+0000","last_unstale":"2023-09-27T23:33:56.554141+0000","last_undegraded":"2023-09-27T23:33:56.554141+0000","last_fullsized":"2023-09-27T23:33:56.554141+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:23.026871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.553079+0000","last_change":"2023-09-27T23:33:56.548762+0000","last_active":"2023-09-27T23:33:56.553079+0000","last_peered":"2023-09-27T23:33:56.553079+0000","last_clean":"2023-09-27T23:33:56.553079+0000","last_became_active":"2023-09-27T23:33:56.548452+0000","last_became_peered":"2023-09-27T23:33:56.548452+0000","last_unstale":"2023-09-27T23:33:56.553079+0000","last_undegraded":"2023-09-27T23:33:56.553079+0000","last_fullsized":"2023-09-27T23:33:56.553079+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:37:51.319798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.551852+0000","last_change":"2023-09-27T23:33:56.172416+0000","last_active":"2023-09-27T23:33:56.551852+0000","last_peered":"2023-09-27T23:33:56.551852+0000","last_clean":"2023-09-27T23:33:56.551852+0000","last_became_active":"2023-09-27T23:33:56.172038+0000","last_became_peered":"2023-09-27T23:33:56.172038+0000","last_unstale":"2023-09-27T23:33:56.551852+0000","last_undegraded":"2023-09-27T23:33:56.551852+0000","last_fullsized":"2023-09-27T23:33:56.551852+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:41:43.394820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561352+0000","last_change":"2023-09-27T23:33:56.556873+0000","last_active":"2023-09-27T23:33:56.561352+0000","last_peered":"2023-09-27T23:33:56.561352+0000","last_clean":"2023-09-27T23:33:56.561352+0000","last_became_active":"2023-09-27T23:33:56.556561+0000","last_became_peered":"2023-09-27T23:33:56.556561+0000","last_unstale":"2023-09-27T23:33:56.561352+0000","last_undegraded":"2023-09-27T23:33:56.561352+0000","last_fullsized":"2023-09-27T23:33:56.561352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:17:01.494681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557628+0000","last_change":"2023-09-27T23:33:56.170110+0000","last_active":"2023-09-27T23:33:56.557628+0000","last_peered":"2023-09-27T23:33:56.557628+0000","last_clean":"2023-09-27T23:33:56.557628+0000","last_became_active":"2023-09-27T23:33:56.169807+0000","last_became_peered":"2023-09-27T23:33:56.169807+0000","last_unstale":"2023-09-27T23:33:56.557628+0000","last_undegraded":"2023-09-27T23:33:56.557628+0000","last_fullsized":"2023-09-27T23:33:56.557628+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:19:22.894462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556118+0000","last_change":"2023-09-27T23:33:56.549475+0000","last_active":"2023-09-27T23:33:56.556118+0000","last_peered":"2023-09-27T23:33:56.556118+0000","last_clean":"2023-09-27T23:33:56.556118+0000","last_became_active":"2023-09-27T23:33:56.549149+0000","last_became_peered":"2023-09-27T23:33:56.549149+0000","last_unstale":"2023-09-27T23:33:56.556118+0000","last_undegraded":"2023-09-27T23:33:56.556118+0000","last_fullsized":"2023-09-27T23:33:56.556118+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:56:03.508822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.562091+0000","last_change":"2023-09-27T23:33:56.168107+0000","last_active":"2023-09-27T23:33:56.562091+0000","last_peered":"2023-09-27T23:33:56.562091+0000","last_clean":"2023-09-27T23:33:56.562091+0000","last_became_active":"2023-09-27T23:33:56.167799+0000","last_became_peered":"2023-09-27T23:33:56.167799+0000","last_unstale":"2023-09-27T23:33:56.562091+0000","last_undegraded":"2023-09-27T23:33:56.562091+0000","last_fullsized":"2023-09-27T23:33:56.562091+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:03.211541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.545619+0000","last_change":"2023-09-27T23:33:56.543000+0000","last_active":"2023-09-27T23:33:56.545619+0000","last_peered":"2023-09-27T23:33:56.545619+0000","last_clean":"2023-09-27T23:33:56.545619+0000","last_became_active":"2023-09-27T23:33:56.542671+0000","last_became_peered":"2023-09-27T23:33:56.542671+0000","last_unstale":"2023-09-27T23:33:56.545619+0000","last_undegraded":"2023-09-27T23:33:56.545619+0000","last_fullsized":"2023-09-27T23:33:56.545619+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:27:35.575350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.546426+0000","last_change":"2023-09-27T23:33:56.166615+0000","last_active":"2023-09-27T23:33:56.546426+0000","last_peered":"2023-09-27T23:33:56.546426+0000","last_clean":"2023-09-27T23:33:56.546426+0000","last_became_active":"2023-09-27T23:33:56.166302+0000","last_became_peered":"2023-09-27T23:33:56.166302+0000","last_unstale":"2023-09-27T23:33:56.546426+0000","last_undegraded":"2023-09-27T23:33:56.546426+0000","last_fullsized":"2023-09-27T23:33:56.546426+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:29:48.675652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564194+0000","last_change":"2023-09-27T23:33:56.557589+0000","last_active":"2023-09-27T23:33:56.564194+0000","last_peered":"2023-09-27T23:33:56.564194+0000","last_clean":"2023-09-27T23:33:56.564194+0000","last_became_active":"2023-09-27T23:33:56.557157+0000","last_became_peered":"2023-09-27T23:33:56.557157+0000","last_unstale":"2023-09-27T23:33:56.564194+0000","last_undegraded":"2023-09-27T23:33:56.564194+0000","last_fullsized":"2023-09-27T23:33:56.564194+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:18:51.913079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.541373+0000","last_change":"2023-09-27T23:33:56.169216+0000","last_active":"2023-09-27T23:33:56.541373+0000","last_peered":"2023-09-27T23:33:56.541373+0000","last_clean":"2023-09-27T23:33:56.541373+0000","last_became_active":"2023-09-27T23:33:56.168882+0000","last_became_peered":"2023-09-27T23:33:56.168882+0000","last_unstale":"2023-09-27T23:33:56.541373+0000","last_undegraded":"2023-09-27T23:33:56.541373+0000","last_fullsized":"2023-09-27T23:33:56.541373+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:57:49.477351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.553039+0000","last_change":"2023-09-27T23:33:56.548672+0000","last_active":"2023-09-27T23:33:56.553039+0000","last_peered":"2023-09-27T23:33:56.553039+0000","last_clean":"2023-09-27T23:33:56.553039+0000","last_became_active":"2023-09-27T23:33:56.548352+0000","last_became_peered":"2023-09-27T23:33:56.548352+0000","last_unstale":"2023-09-27T23:33:56.553039+0000","last_undegraded":"2023-09-27T23:33:56.553039+0000","last_fullsized":"2023-09-27T23:33:56.553039+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:33:18.272506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.523458+0000","last_change":"2023-09-27T23:33:56.158318+0000","last_active":"2023-09-27T23:33:56.523458+0000","last_peered":"2023-09-27T23:33:56.523458+0000","last_clean":"2023-09-27T23:33:56.523458+0000","last_became_active":"2023-09-27T23:33:56.157954+0000","last_became_peered":"2023-09-27T23:33:56.157954+0000","last_unstale":"2023-09-27T23:33:56.523458+0000","last_undegraded":"2023-09-27T23:33:56.523458+0000","last_fullsized":"2023-09-27T23:33:56.523458+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:35.265697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.553669+0000","last_change":"2023-09-27T23:33:56.549133+0000","last_active":"2023-09-27T23:33:56.553669+0000","last_peered":"2023-09-27T23:33:56.553669+0000","last_clean":"2023-09-27T23:33:56.553669+0000","last_became_active":"2023-09-27T23:33:56.548821+0000","last_became_peered":"2023-09-27T23:33:56.548821+0000","last_unstale":"2023-09-27T23:33:56.553669+0000","last_undegraded":"2023-09-27T23:33:56.553669+0000","last_fullsized":"2023-09-27T23:33:56.553669+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:35:35.296858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540799+0000","last_change":"2023-09-27T23:33:56.169678+0000","last_active":"2023-09-27T23:33:56.540799+0000","last_peered":"2023-09-27T23:33:56.540799+0000","last_clean":"2023-09-27T23:33:56.540799+0000","last_became_active":"2023-09-27T23:33:56.168847+0000","last_became_peered":"2023-09-27T23:33:56.168847+0000","last_unstale":"2023-09-27T23:33:56.540799+0000","last_undegraded":"2023-09-27T23:33:56.540799+0000","last_fullsized":"2023-09-27T23:33:56.540799+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:45:27.524793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556145+0000","last_change":"2023-09-27T23:33:56.550338+0000","last_active":"2023-09-27T23:33:56.556145+0000","last_peered":"2023-09-27T23:33:56.556145+0000","last_clean":"2023-09-27T23:33:56.556145+0000","last_became_active":"2023-09-27T23:33:56.550022+0000","last_became_peered":"2023-09-27T23:33:56.550022+0000","last_unstale":"2023-09-27T23:33:56.556145+0000","last_undegraded":"2023-09-27T23:33:56.556145+0000","last_fullsized":"2023-09-27T23:33:56.556145+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:21:14.885170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.537321+0000","last_change":"2023-09-27T23:33:56.163285+0000","last_active":"2023-09-27T23:33:56.537321+0000","last_peered":"2023-09-27T23:33:56.537321+0000","last_clean":"2023-09-27T23:33:56.537321+0000","last_became_active":"2023-09-27T23:33:56.163030+0000","last_became_peered":"2023-09-27T23:33:56.163030+0000","last_unstale":"2023-09-27T23:33:56.537321+0000","last_undegraded":"2023-09-27T23:33:56.537321+0000","last_fullsized":"2023-09-27T23:33:56.537321+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:46.540226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.560686+0000","last_change":"2023-09-27T23:33:56.542586+0000","last_active":"2023-09-27T23:33:56.560686+0000","last_peered":"2023-09-27T23:33:56.560686+0000","last_clean":"2023-09-27T23:33:56.560686+0000","last_became_active":"2023-09-27T23:33:56.542312+0000","last_became_peered":"2023-09-27T23:33:56.542312+0000","last_unstale":"2023-09-27T23:33:56.560686+0000","last_undegraded":"2023-09-27T23:33:56.560686+0000","last_fullsized":"2023-09-27T23:33:56.560686+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:22:01.768873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.548114+0000","last_change":"2023-09-27T23:33:56.169380+0000","last_active":"2023-09-27T23:33:56.548114+0000","last_peered":"2023-09-27T23:33:56.548114+0000","last_clean":"2023-09-27T23:33:56.548114+0000","last_became_active":"2023-09-27T23:33:56.168974+0000","last_became_peered":"2023-09-27T23:33:56.168974+0000","last_unstale":"2023-09-27T23:33:56.548114+0000","last_undegraded":"2023-09-27T23:33:56.548114+0000","last_fullsized":"2023-09-27T23:33:56.548114+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:23:40.275892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.562190+0000","last_change":"2023-09-27T23:33:56.559855+0000","last_active":"2023-09-27T23:33:56.562190+0000","last_peered":"2023-09-27T23:33:56.562190+0000","last_clean":"2023-09-27T23:33:56.562190+0000","last_became_active":"2023-09-27T23:33:56.559505+0000","last_became_peered":"2023-09-27T23:33:56.559505+0000","last_unstale":"2023-09-27T23:33:56.562190+0000","last_undegraded":"2023-09-27T23:33:56.562190+0000","last_fullsized":"2023-09-27T23:33:56.562190+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:23:34.554266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540878+0000","last_change":"2023-09-27T23:33:56.166552+0000","last_active":"2023-09-27T23:33:56.540878+0000","last_peered":"2023-09-27T23:33:56.540878+0000","last_clean":"2023-09-27T23:33:56.540878+0000","last_became_active":"2023-09-27T23:33:56.166251+0000","last_became_peered":"2023-09-27T23:33:56.166251+0000","last_unstale":"2023-09-27T23:33:56.540878+0000","last_undegraded":"2023-09-27T23:33:56.540878+0000","last_fullsized":"2023-09-27T23:33:56.540878+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:26:12.876958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.557128+0000","last_change":"2023-09-27T23:33:56.552372+0000","last_active":"2023-09-27T23:33:56.557128+0000","last_peered":"2023-09-27T23:33:56.557128+0000","last_clean":"2023-09-27T23:33:56.557128+0000","last_became_active":"2023-09-27T23:33:56.551956+0000","last_became_peered":"2023-09-27T23:33:56.551956+0000","last_unstale":"2023-09-27T23:33:56.557128+0000","last_undegraded":"2023-09-27T23:33:56.557128+0000","last_fullsized":"2023-09-27T23:33:56.557128+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:14:12.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.537947+0000","last_change":"2023-09-27T23:33:56.169821+0000","last_active":"2023-09-27T23:33:56.537947+0000","last_peered":"2023-09-27T23:33:56.537947+0000","last_clean":"2023-09-27T23:33:56.537947+0000","last_became_active":"2023-09-27T23:33:56.169543+0000","last_became_peered":"2023-09-27T23:33:56.169543+0000","last_unstale":"2023-09-27T23:33:56.537947+0000","last_undegraded":"2023-09-27T23:33:56.537947+0000","last_fullsized":"2023-09-27T23:33:56.537947+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:56:02.496586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564198+0000","last_change":"2023-09-27T23:33:56.557463+0000","last_active":"2023-09-27T23:33:56.564198+0000","last_peered":"2023-09-27T23:33:56.564198+0000","last_clean":"2023-09-27T23:33:56.564198+0000","last_became_active":"2023-09-27T23:33:56.557136+0000","last_became_peered":"2023-09-27T23:33:56.557136+0000","last_unstale":"2023-09-27T23:33:56.564198+0000","last_undegraded":"2023-09-27T23:33:56.564198+0000","last_fullsized":"2023-09-27T23:33:56.564198+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:28:34.924368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.549370+0000","last_change":"2023-09-27T23:33:56.167253+0000","last_active":"2023-09-27T23:33:56.549370+0000","last_peered":"2023-09-27T23:33:56.549370+0000","last_clean":"2023-09-27T23:33:56.549370+0000","last_became_active":"2023-09-27T23:33:56.166943+0000","last_became_peered":"2023-09-27T23:33:56.166943+0000","last_unstale":"2023-09-27T23:33:56.549370+0000","last_undegraded":"2023-09-27T23:33:56.549370+0000","last_fullsized":"2023-09-27T23:33:56.549370+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:37:50.307562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564665+0000","last_change":"2023-09-27T23:33:56.557667+0000","last_active":"2023-09-27T23:33:56.564665+0000","last_peered":"2023-09-27T23:33:56.564665+0000","last_clean":"2023-09-27T23:33:56.564665+0000","last_became_active":"2023-09-27T23:33:56.557372+0000","last_became_peered":"2023-09-27T23:33:56.557372+0000","last_unstale":"2023-09-27T23:33:56.564665+0000","last_undegraded":"2023-09-27T23:33:56.564665+0000","last_fullsized":"2023-09-27T23:33:56.564665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:30:18.934917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.533701+0000","last_change":"2023-09-27T23:33:56.173481+0000","last_active":"2023-09-27T23:33:56.533701+0000","last_peered":"2023-09-27T23:33:56.533701+0000","last_clean":"2023-09-27T23:33:56.533701+0000","last_became_active":"2023-09-27T23:33:56.173076+0000","last_became_peered":"2023-09-27T23:33:56.173076+0000","last_unstale":"2023-09-27T23:33:56.533701+0000","last_undegraded":"2023-09-27T23:33:56.533701+0000","last_fullsized":"2023-09-27T23:33:56.533701+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:02:36.511974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564216+0000","last_change":"2023-09-27T23:33:56.557757+0000","last_active":"2023-09-27T23:33:56.564216+0000","last_peered":"2023-09-27T23:33:56.564216+0000","last_clean":"2023-09-27T23:33:56.564216+0000","last_became_active":"2023-09-27T23:33:56.557182+0000","last_became_peered":"2023-09-27T23:33:56.557182+0000","last_unstale":"2023-09-27T23:33:56.564216+0000","last_undegraded":"2023-09-27T23:33:56.564216+0000","last_fullsized":"2023-09-27T23:33:56.564216+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:24:31.871021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.533476+0000","last_change":"2023-09-27T23:33:56.173368+0000","last_active":"2023-09-27T23:33:56.533476+0000","last_peered":"2023-09-27T23:33:56.533476+0000","last_clean":"2023-09-27T23:33:56.533476+0000","last_became_active":"2023-09-27T23:33:56.173051+0000","last_became_peered":"2023-09-27T23:33:56.173051+0000","last_unstale":"2023-09-27T23:33:56.533476+0000","last_undegraded":"2023-09-27T23:33:56.533476+0000","last_fullsized":"2023-09-27T23:33:56.533476+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:17:49.376189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.560892+0000","last_change":"2023-09-27T23:33:56.556156+0000","last_active":"2023-09-27T23:33:56.560892+0000","last_peered":"2023-09-27T23:33:56.560892+0000","last_clean":"2023-09-27T23:33:56.560892+0000","last_became_active":"2023-09-27T23:33:56.555819+0000","last_became_peered":"2023-09-27T23:33:56.555819+0000","last_unstale":"2023-09-27T23:33:56.560892+0000","last_undegraded":"2023-09-27T23:33:56.560892+0000","last_fullsized":"2023-09-27T23:33:56.560892+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:49:36.810674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.537594+0000","last_change":"2023-09-27T23:33:56.169503+0000","last_active":"2023-09-27T23:33:56.537594+0000","last_peered":"2023-09-27T23:33:56.537594+0000","last_clean":"2023-09-27T23:33:56.537594+0000","last_became_active":"2023-09-27T23:33:56.169177+0000","last_became_peered":"2023-09-27T23:33:56.169177+0000","last_unstale":"2023-09-27T23:33:56.537594+0000","last_undegraded":"2023-09-27T23:33:56.537594+0000","last_fullsized":"2023-09-27T23:33:56.537594+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:50:29.737551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.553689+0000","last_change":"2023-09-27T23:33:56.551097+0000","last_active":"2023-09-27T23:33:56.553689+0000","last_peered":"2023-09-27T23:33:56.553689+0000","last_clean":"2023-09-27T23:33:56.553689+0000","last_became_active":"2023-09-27T23:33:56.550806+0000","last_became_peered":"2023-09-27T23:33:56.550806+0000","last_unstale":"2023-09-27T23:33:56.553689+0000","last_undegraded":"2023-09-27T23:33:56.553689+0000","last_fullsized":"2023-09-27T23:33:56.553689+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:26:36.277933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550842+0000","last_change":"2023-09-27T23:33:56.173547+0000","last_active":"2023-09-27T23:33:56.550842+0000","last_peered":"2023-09-27T23:33:56.550842+0000","last_clean":"2023-09-27T23:33:56.550842+0000","last_became_active":"2023-09-27T23:33:56.173211+0000","last_became_peered":"2023-09-27T23:33:56.173211+0000","last_unstale":"2023-09-27T23:33:56.550842+0000","last_undegraded":"2023-09-27T23:33:56.550842+0000","last_fullsized":"2023-09-27T23:33:56.550842+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:30:10.761632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561176+0000","last_change":"2023-09-27T23:33:56.558181+0000","last_active":"2023-09-27T23:33:56.561176+0000","last_peered":"2023-09-27T23:33:56.561176+0000","last_clean":"2023-09-27T23:33:56.561176+0000","last_became_active":"2023-09-27T23:33:56.557863+0000","last_became_peered":"2023-09-27T23:33:56.557863+0000","last_unstale":"2023-09-27T23:33:56.561176+0000","last_undegraded":"2023-09-27T23:33:56.561176+0000","last_fullsized":"2023-09-27T23:33:56.561176+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:43.653085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539969+0000","last_change":"2023-09-27T23:33:56.171325+0000","last_active":"2023-09-27T23:33:56.539969+0000","last_peered":"2023-09-27T23:33:56.539969+0000","last_clean":"2023-09-27T23:33:56.539969+0000","last_became_active":"2023-09-27T23:33:56.170836+0000","last_became_peered":"2023-09-27T23:33:56.170836+0000","last_unstale":"2023-09-27T23:33:56.539969+0000","last_undegraded":"2023-09-27T23:33:56.539969+0000","last_fullsized":"2023-09-27T23:33:56.539969+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:56:07.324327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564621+0000","last_change":"2023-09-27T23:33:56.560226+0000","last_active":"2023-09-27T23:33:56.564621+0000","last_peered":"2023-09-27T23:33:56.564621+0000","last_clean":"2023-09-27T23:33:56.564621+0000","last_became_active":"2023-09-27T23:33:56.559984+0000","last_became_peered":"2023-09-27T23:33:56.559984+0000","last_unstale":"2023-09-27T23:33:56.564621+0000","last_undegraded":"2023-09-27T23:33:56.564621+0000","last_fullsized":"2023-09-27T23:33:56.564621+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:20:58.201114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539936+0000","last_change":"2023-09-27T23:33:56.165693+0000","last_active":"2023-09-27T23:33:56.539936+0000","last_peered":"2023-09-27T23:33:56.539936+0000","last_clean":"2023-09-27T23:33:56.539936+0000","last_became_active":"2023-09-27T23:33:56.165301+0000","last_became_peered":"2023-09-27T23:33:56.165301+0000","last_unstale":"2023-09-27T23:33:56.539936+0000","last_undegraded":"2023-09-27T23:33:56.539936+0000","last_fullsized":"2023-09-27T23:33:56.539936+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:17.631606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561289+0000","last_change":"2023-09-27T23:33:56.556576+0000","last_active":"2023-09-27T23:33:56.561289+0000","last_peered":"2023-09-27T23:33:56.561289+0000","last_clean":"2023-09-27T23:33:56.561289+0000","last_became_active":"2023-09-27T23:33:56.555849+0000","last_became_peered":"2023-09-27T23:33:56.555849+0000","last_unstale":"2023-09-27T23:33:56.561289+0000","last_undegraded":"2023-09-27T23:33:56.561289+0000","last_fullsized":"2023-09-27T23:33:56.561289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:57:55.099301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539954+0000","last_change":"2023-09-27T23:33:56.164762+0000","last_active":"2023-09-27T23:33:56.539954+0000","last_peered":"2023-09-27T23:33:56.539954+0000","last_clean":"2023-09-27T23:33:56.539954+0000","last_became_active":"2023-09-27T23:33:56.164480+0000","last_became_peered":"2023-09-27T23:33:56.164480+0000","last_unstale":"2023-09-27T23:33:56.539954+0000","last_undegraded":"2023-09-27T23:33:56.539954+0000","last_fullsized":"2023-09-27T23:33:56.539954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:12:27.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.565155+0000","last_change":"2023-09-27T23:33:56.558597+0000","last_active":"2023-09-27T23:33:56.565155+0000","last_peered":"2023-09-27T23:33:56.565155+0000","last_clean":"2023-09-27T23:33:56.565155+0000","last_became_active":"2023-09-27T23:33:56.558215+0000","last_became_peered":"2023-09-27T23:33:56.558215+0000","last_unstale":"2023-09-27T23:33:56.565155+0000","last_undegraded":"2023-09-27T23:33:56.565155+0000","last_fullsized":"2023-09-27T23:33:56.565155+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:46:41.341499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.535582+0000","last_change":"2023-09-27T23:33:56.166719+0000","last_active":"2023-09-27T23:33:56.535582+0000","last_peered":"2023-09-27T23:33:56.535582+0000","last_clean":"2023-09-27T23:33:56.535582+0000","last_became_active":"2023-09-27T23:33:56.166434+0000","last_became_peered":"2023-09-27T23:33:56.166434+0000","last_unstale":"2023-09-27T23:33:56.535582+0000","last_undegraded":"2023-09-27T23:33:56.535582+0000","last_fullsized":"2023-09-27T23:33:56.535582+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:45:27.524793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.549497+0000","last_change":"2023-09-27T23:33:56.542551+0000","last_active":"2023-09-27T23:33:56.549497+0000","last_peered":"2023-09-27T23:33:56.549497+0000","last_clean":"2023-09-27T23:33:56.549497+0000","last_became_active":"2023-09-27T23:33:56.542253+0000","last_became_peered":"2023-09-27T23:33:56.542253+0000","last_unstale":"2023-09-27T23:33:56.549497+0000","last_undegraded":"2023-09-27T23:33:56.549497+0000","last_fullsized":"2023-09-27T23:33:56.549497+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:26:13.889194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.533803+0000","last_change":"2023-09-27T23:33:56.174169+0000","last_active":"2023-09-27T23:33:56.533803+0000","last_peered":"2023-09-27T23:33:56.533803+0000","last_clean":"2023-09-27T23:33:56.533803+0000","last_became_active":"2023-09-27T23:33:56.173757+0000","last_became_peered":"2023-09-27T23:33:56.173757+0000","last_unstale":"2023-09-27T23:33:56.533803+0000","last_undegraded":"2023-09-27T23:33:56.533803+0000","last_fullsized":"2023-09-27T23:33:56.533803+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:21:48.910451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.565917+0000","last_change":"2023-09-27T23:33:56.565123+0000","last_active":"2023-09-27T23:33:56.565917+0000","last_peered":"2023-09-27T23:33:56.565917+0000","last_clean":"2023-09-27T23:33:56.565917+0000","last_became_active":"2023-09-27T23:33:56.564857+0000","last_became_peered":"2023-09-27T23:33:56.564857+0000","last_unstale":"2023-09-27T23:33:56.565917+0000","last_undegraded":"2023-09-27T23:33:56.565917+0000","last_fullsized":"2023-09-27T23:33:56.565917+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:43.653085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.537765+0000","last_change":"2023-09-27T23:33:56.172687+0000","last_active":"2023-09-27T23:33:56.537765+0000","last_peered":"2023-09-27T23:33:56.537765+0000","last_clean":"2023-09-27T23:33:56.537765+0000","last_became_active":"2023-09-27T23:33:56.172286+0000","last_became_peered":"2023-09-27T23:33:56.172286+0000","last_unstale":"2023-09-27T23:33:56.537765+0000","last_undegraded":"2023-09-27T23:33:56.537765+0000","last_fullsized":"2023-09-27T23:33:56.537765+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:22:00.756637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.549234+0000","last_change":"2023-09-27T23:33:56.541495+0000","last_active":"2023-09-27T23:33:56.549234+0000","last_peered":"2023-09-27T23:33:56.549234+0000","last_clean":"2023-09-27T23:33:56.549234+0000","last_became_active":"2023-09-27T23:33:56.541112+0000","last_became_peered":"2023-09-27T23:33:56.541112+0000","last_unstale":"2023-09-27T23:33:56.549234+0000","last_undegraded":"2023-09-27T23:33:56.549234+0000","last_fullsized":"2023-09-27T23:33:56.549234+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:52:19.029631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.536952+0000","last_change":"2023-09-27T23:33:56.171687+0000","last_active":"2023-09-27T23:33:56.536952+0000","last_peered":"2023-09-27T23:33:56.536952+0000","last_clean":"2023-09-27T23:33:56.536952+0000","last_became_active":"2023-09-27T23:33:56.171400+0000","last_became_peered":"2023-09-27T23:33:56.171400+0000","last_unstale":"2023-09-27T23:33:56.536952+0000","last_undegraded":"2023-09-27T23:33:56.536952+0000","last_fullsized":"2023-09-27T23:33:56.536952+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:27:39.987812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.564743+0000","last_change":"2023-09-27T23:33:56.558647+0000","last_active":"2023-09-27T23:33:56.564743+0000","last_peered":"2023-09-27T23:33:56.564743+0000","last_clean":"2023-09-27T23:33:56.564743+0000","last_became_active":"2023-09-27T23:33:56.558337+0000","last_became_peered":"2023-09-27T23:33:56.558337+0000","last_unstale":"2023-09-27T23:33:56.564743+0000","last_undegraded":"2023-09-27T23:33:56.564743+0000","last_fullsized":"2023-09-27T23:33:56.564743+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:16:43.597595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"156'3","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550129+0000","last_change":"2023-09-27T23:33:56.170215+0000","last_active":"2023-09-27T23:33:56.550129+0000","last_peered":"2023-09-27T23:33:56.550129+0000","last_clean":"2023-09-27T23:33:56.550129+0000","last_became_active":"2023-09-27T23:33:56.169879+0000","last_became_peered":"2023-09-27T23:33:56.169879+0000","last_unstale":"2023-09-27T23:33:56.550129+0000","last_undegraded":"2023-09-27T23:33:56.550129+0000","last_fullsized":"2023-09-27T23:33:56.550129+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:05.230659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"145'271","reported_seq":828,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.534207+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:33:56.534207+0000","last_peered":"2023-09-27T23:33:56.534207+0000","last_clean":"2023-09-27T23:33:56.534207+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:33:56.534207+0000","last_undegraded":"2023-09-27T23:33:56.534207+0000","last_fullsized":"2023-09-27T23:33:56.534207+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":271,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":142,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.531187+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:33:56.531187+0000","last_peered":"2023-09-27T23:33:56.531187+0000","last_clean":"2023-09-27T23:33:56.531187+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:33:56.531187+0000","last_undegraded":"2023-09-27T23:33:56.531187+0000","last_fullsized":"2023-09-27T23:33:56.531187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.556215+0000","last_change":"2023-09-27T23:33:56.552748+0000","last_active":"2023-09-27T23:33:56.556215+0000","last_peered":"2023-09-27T23:33:56.556215+0000","last_clean":"2023-09-27T23:33:56.556215+0000","last_became_active":"2023-09-27T23:33:56.550829+0000","last_became_peered":"2023-09-27T23:33:56.550829+0000","last_unstale":"2023-09-27T23:33:56.556215+0000","last_undegraded":"2023-09-27T23:33:56.556215+0000","last_fullsized":"2023-09-27T23:33:56.556215+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:43:39.144938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.539660+0000","last_change":"2023-09-27T23:33:56.173935+0000","last_active":"2023-09-27T23:33:56.539660+0000","last_peered":"2023-09-27T23:33:56.539660+0000","last_clean":"2023-09-27T23:33:56.539660+0000","last_became_active":"2023-09-27T23:33:56.173593+0000","last_became_peered":"2023-09-27T23:33:56.173593+0000","last_unstale":"2023-09-27T23:33:56.539660+0000","last_undegraded":"2023-09-27T23:33:56.539660+0000","last_fullsized":"2023-09-27T23:33:56.539660+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:01:23.274708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.534099+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:33:56.534099+0000","last_peered":"2023-09-27T23:33:56.534099+0000","last_clean":"2023-09-27T23:33:56.534099+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:33:56.534099+0000","last_undegraded":"2023-09-27T23:33:56.534099+0000","last_fullsized":"2023-09-27T23:33:56.534099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.548929+0000","last_change":"2023-09-27T23:33:56.539552+0000","last_active":"2023-09-27T23:33:56.548929+0000","last_peered":"2023-09-27T23:33:56.548929+0000","last_clean":"2023-09-27T23:33:56.548929+0000","last_became_active":"2023-09-27T23:33:56.539254+0000","last_became_peered":"2023-09-27T23:33:56.539254+0000","last_unstale":"2023-09-27T23:33:56.548929+0000","last_undegraded":"2023-09-27T23:33:56.548929+0000","last_fullsized":"2023-09-27T23:33:56.548929+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:53:18.611305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.537259+0000","last_change":"2023-09-27T23:33:56.170680+0000","last_active":"2023-09-27T23:33:56.537259+0000","last_peered":"2023-09-27T23:33:56.537259+0000","last_clean":"2023-09-27T23:33:56.537259+0000","last_became_active":"2023-09-27T23:33:56.170350+0000","last_became_peered":"2023-09-27T23:33:56.170350+0000","last_unstale":"2023-09-27T23:33:56.537259+0000","last_undegraded":"2023-09-27T23:33:56.537259+0000","last_fullsized":"2023-09-27T23:33:56.537259+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:28.099577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.523144+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:33:56.523144+0000","last_peered":"2023-09-27T23:33:56.523144+0000","last_clean":"2023-09-27T23:33:56.523144+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:33:56.523144+0000","last_undegraded":"2023-09-27T23:33:56.523144+0000","last_fullsized":"2023-09-27T23:33:56.523144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.560468+0000","last_change":"2023-09-27T23:33:56.536368+0000","last_active":"2023-09-27T23:33:56.560468+0000","last_peered":"2023-09-27T23:33:56.560468+0000","last_clean":"2023-09-27T23:33:56.560468+0000","last_became_active":"2023-09-27T23:33:56.536076+0000","last_became_peered":"2023-09-27T23:33:56.536076+0000","last_unstale":"2023-09-27T23:33:56.560468+0000","last_undegraded":"2023-09-27T23:33:56.560468+0000","last_fullsized":"2023-09-27T23:33:56.560468+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:05:24.039107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"156'6","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.569531+0000","last_change":"2023-09-27T23:33:56.161329+0000","last_active":"2023-09-27T23:33:56.569531+0000","last_peered":"2023-09-27T23:33:56.569531+0000","last_clean":"2023-09-27T23:33:56.569531+0000","last_became_active":"2023-09-27T23:33:56.161081+0000","last_became_peered":"2023-09-27T23:33:56.161081+0000","last_unstale":"2023-09-27T23:33:56.569531+0000","last_undegraded":"2023-09-27T23:33:56.569531+0000","last_fullsized":"2023-09-27T23:33:56.569531+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:32:54.066369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.529477+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:33:56.529477+0000","last_peered":"2023-09-27T23:33:56.529477+0000","last_clean":"2023-09-27T23:33:56.529477+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:33:56.529477+0000","last_undegraded":"2023-09-27T23:33:56.529477+0000","last_fullsized":"2023-09-27T23:33:56.529477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.558830+0000","last_change":"2023-09-27T23:33:56.551450+0000","last_active":"2023-09-27T23:33:56.558830+0000","last_peered":"2023-09-27T23:33:56.558830+0000","last_clean":"2023-09-27T23:33:56.558830+0000","last_became_active":"2023-09-27T23:33:56.551147+0000","last_became_peered":"2023-09-27T23:33:56.551147+0000","last_unstale":"2023-09-27T23:33:56.558830+0000","last_undegraded":"2023-09-27T23:33:56.558830+0000","last_fullsized":"2023-09-27T23:33:56.558830+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:29:33.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538085+0000","last_change":"2023-09-27T23:33:56.160457+0000","last_active":"2023-09-27T23:33:56.538085+0000","last_peered":"2023-09-27T23:33:56.538085+0000","last_clean":"2023-09-27T23:33:56.538085+0000","last_became_active":"2023-09-27T23:33:56.160115+0000","last_became_peered":"2023-09-27T23:33:56.160115+0000","last_unstale":"2023-09-27T23:33:56.538085+0000","last_undegraded":"2023-09-27T23:33:56.538085+0000","last_fullsized":"2023-09-27T23:33:56.538085+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:29:17.382807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540062+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:33:56.540062+0000","last_peered":"2023-09-27T23:33:56.540062+0000","last_clean":"2023-09-27T23:33:56.540062+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:33:56.540062+0000","last_undegraded":"2023-09-27T23:33:56.540062+0000","last_fullsized":"2023-09-27T23:33:56.540062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.561182+0000","last_change":"2023-09-27T23:33:56.558694+0000","last_active":"2023-09-27T23:33:56.561182+0000","last_peered":"2023-09-27T23:33:56.561182+0000","last_clean":"2023-09-27T23:33:56.561182+0000","last_became_active":"2023-09-27T23:33:56.558404+0000","last_became_peered":"2023-09-27T23:33:56.558404+0000","last_unstale":"2023-09-27T23:33:56.561182+0000","last_undegraded":"2023-09-27T23:33:56.561182+0000","last_fullsized":"2023-09-27T23:33:56.561182+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:56:03.508822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.553497+0000","last_change":"2023-09-27T23:33:56.170446+0000","last_active":"2023-09-27T23:33:56.553497+0000","last_peered":"2023-09-27T23:33:56.553497+0000","last_clean":"2023-09-27T23:33:56.553497+0000","last_became_active":"2023-09-27T23:33:56.170182+0000","last_became_peered":"2023-09-27T23:33:56.170182+0000","last_unstale":"2023-09-27T23:33:56.553497+0000","last_undegraded":"2023-09-27T23:33:56.553497+0000","last_fullsized":"2023-09-27T23:33:56.553497+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:30:10.179943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.534481+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:33:56.534481+0000","last_peered":"2023-09-27T23:33:56.534481+0000","last_clean":"2023-09-27T23:33:56.534481+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:33:56.534481+0000","last_undegraded":"2023-09-27T23:33:56.534481+0000","last_fullsized":"2023-09-27T23:33:56.534481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.568706+0000","last_change":"2023-09-27T23:33:56.568032+0000","last_active":"2023-09-27T23:33:56.568706+0000","last_peered":"2023-09-27T23:33:56.568706+0000","last_clean":"2023-09-27T23:33:56.568706+0000","last_became_active":"2023-09-27T23:33:56.567722+0000","last_became_peered":"2023-09-27T23:33:56.567722+0000","last_unstale":"2023-09-27T23:33:56.568706+0000","last_undegraded":"2023-09-27T23:33:56.568706+0000","last_fullsized":"2023-09-27T23:33:56.568706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:59.590003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"156'8","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.565297+0000","last_change":"2023-09-27T23:33:56.160127+0000","last_active":"2023-09-27T23:33:56.565297+0000","last_peered":"2023-09-27T23:33:56.565297+0000","last_clean":"2023-09-27T23:33:56.565297+0000","last_became_active":"2023-09-27T23:33:56.159551+0000","last_became_peered":"2023-09-27T23:33:56.159551+0000","last_unstale":"2023-09-27T23:33:56.565297+0000","last_undegraded":"2023-09-27T23:33:56.565297+0000","last_fullsized":"2023-09-27T23:33:56.565297+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:32:36.819518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.540304+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:33:56.540304+0000","last_peered":"2023-09-27T23:33:56.540304+0000","last_clean":"2023-09-27T23:33:56.540304+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:33:56.540304+0000","last_undegraded":"2023-09-27T23:33:56.540304+0000","last_fullsized":"2023-09-27T23:33:56.540304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.550425+0000","last_change":"2023-09-27T23:33:56.544582+0000","last_active":"2023-09-27T23:33:56.550425+0000","last_peered":"2023-09-27T23:33:56.550425+0000","last_clean":"2023-09-27T23:33:56.550425+0000","last_became_active":"2023-09-27T23:33:56.544299+0000","last_became_peered":"2023-09-27T23:33:56.544299+0000","last_unstale":"2023-09-27T23:33:56.550425+0000","last_undegraded":"2023-09-27T23:33:56.550425+0000","last_fullsized":"2023-09-27T23:33:56.550425+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:56.133650+0000","last_clean_scrub_stamp":"2023-09-27T23:33:56.133650+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:32:37.831754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538204+0000","last_change":"2023-09-27T23:33:56.170155+0000","last_active":"2023-09-27T23:33:56.538204+0000","last_peered":"2023-09-27T23:33:56.538204+0000","last_clean":"2023-09-27T23:33:56.538204+0000","last_became_active":"2023-09-27T23:33:56.169711+0000","last_became_peered":"2023-09-27T23:33:56.169711+0000","last_unstale":"2023-09-27T23:33:56.538204+0000","last_undegraded":"2023-09-27T23:33:56.538204+0000","last_fullsized":"2023-09-27T23:33:56.538204+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:33:55.121414+0000","last_clean_scrub_stamp":"2023-09-27T23:33:55.121414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:24:27.214773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":126,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-09-27T23:33:56.538279+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:33:56.538279+0000","last_peered":"2023-09-27T23:33:56.538279+0000","last_clean":"2023-09-27T23:33:56.538279+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:33:56.538279+0000","last_undegraded":"2023-09-27T23:33:56.538279+0000","last_fullsized":"2023-09-27T23:33:56.538279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509684,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28420,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":25920,"kb_avail":94343420,"statfs":{"total":96636764160,"available":96607662080,"internally_reserved":0,"allocated":2551808,"data_stored":1609695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":26542080},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.875,"15min":0.90000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.214,"5min":1.3169999999999999,"15min":1.7949999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92800000000000005,"15min":0.93000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.143,"5min":1.5489999999999999,"15min":1.712},"last":0.85699999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4610000000000001,"15min":1.5389999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.28299999999999997},"max":{"1min":1.248,"5min":1.337,"15min":1.7589999999999999},"last":0.98599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.91200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.307},"max":{"1min":1.167,"5min":1.3129999999999999,"15min":2.681},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.99099999999999999,"15min":0.98199999999999998},"min":{"1min":0.75700000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2889999999999999,"5min":1.405,"15min":1.718},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.93899999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.421},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"max":{"1min":1.2430000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0780000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.41099999999999998},"max":{"1min":1.2949999999999999,"5min":1.3240000000000001,"15min":2.6230000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.98699999999999999},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.48099999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.8879999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0429999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.57499999999999996},"max":{"1min":1.2649999999999999,"5min":1.79,"15min":1.79},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1180000000000001,"15min":1.1100000000000001},"min":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.54100000000000004},"max":{"1min":1.3759999999999999,"5min":1.7190000000000001,"15min":2.7280000000000002},"last":1.056}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.159,"15min":1.1419999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.504,"5min":1.839,"15min":1.839},"last":1.101},{"interface":"front","average":{"1min":1.137,"5min":1.113,"15min":1.131},"min":{"1min":0.73699999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.639,"15min":2.548},"last":1.0269999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28676,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":26752,"kb_avail":94343164,"statfs":{"total":96636764160,"available":96607399936,"internally_reserved":0,"allocated":1961984,"data_stored":1018849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27394048},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.34300000000000003},"max":{"1min":1.069,"5min":1.224,"15min":1.3779999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.004,"15min":1.014},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.41799999999999998},"max":{"1min":1.1830000000000001,"5min":1.321,"15min":1.5149999999999999},"last":0.92300000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86099999999999999,"15min":0.88},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":1.248,"15min":1.323},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":1.371,"5min":1.403,"15min":1.403},"last":1.264}]},{"osd":2,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.872,"15min":0.872},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36799999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.353},"last":0.87},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90900000000000003,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.129,"5min":1.3660000000000001,"15min":1.3740000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":1.3069999999999999,"5min":1.51,"15min":1.734},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.72799999999999998,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.3149999999999999,"5min":1.4279999999999999,"15min":1.4970000000000001},"last":0.98599999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89900000000000002,"15min":0.90600000000000003},"min":{"1min":0.48899999999999999,"5min":0.438,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.3280000000000001,"15min":1.544},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98199999999999998,"15min":1.016},"min":{"1min":0.72399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1659999999999999,"5min":1.5980000000000001,"15min":1.6040000000000001},"last":1.05}]},{"osd":5,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97999999999999998,"15min":0.97499999999999998},"min":{"1min":0.63700000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.5820000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1200000000000001,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.69099999999999995,"15min":0.63500000000000001},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.49},"last":1.129}]},{"osd":7,"last update":"Wed Sep 27 23:33:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.109,"15min":1.1319999999999999},"min":{"1min":0.754,"5min":0.55300000000000005,"15min":0.51700000000000002},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":1.7390000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1299999999999999,"15min":1.143},"min":{"1min":0.68799999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.6619999999999999},"last":1.155}]}]},{"osd":5,"up_from":15,"seq":64424509683,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38268,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":36352,"kb_avail":94333572,"statfs":{"total":96636764160,"available":96597577728,"internally_reserved":0,"allocated":1953792,"data_stored":1018725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37224448},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1240000000000001,"5min":1.323,"15min":1.323},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":0.90100000000000002},"min":{"1min":0.55800000000000005,"5min":0.51200000000000001,"15min":0.437},"max":{"1min":1.161,"5min":1.27,"15min":1.3440000000000001},"last":0.90800000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.53500000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1499999999999999,"5min":1.2729999999999999,"15min":1.3109999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.68500000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.24,"5min":1.242,"15min":1.242},"last":0.752}]},{"osd":2,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.66000000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.474,"15min":1.474},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3089999999999999,"5min":1.383,"15min":1.4730000000000001},"last":0.95199999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"max":{"1min":1.2170000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.70499999999999996,"5min":0.61399999999999999,"15min":0.56799999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.3819999999999999},"last":1.101}]},{"osd":4,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89200000000000002},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.27100000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.026,"5min":1.024,"15min":1.02},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.53500000000000003},"max":{"1min":1.327,"5min":1.613,"15min":1.613},"last":1.0620000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.018,"15min":1.012},"min":{"1min":0.64900000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.5580000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.84999999999999998,"5min":0.58999999999999997,"15min":0.54800000000000004},"max":{"1min":1.4410000000000001,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.99399999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:32:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.52700000000000002},"max":{"1min":1.4099999999999999,"5min":1.665,"15min":1.6679999999999999},"last":1.169},{"interface":"front","average":{"1min":1.242,"5min":1.2050000000000001,"15min":1.1919999999999999},"min":{"1min":0.998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.633},"last":1.25}]}]},{"osd":4,"up_from":15,"seq":64424509684,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24312,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":22400,"kb_avail":94347528,"statfs":{"total":96636764160,"available":96611868672,"internally_reserved":0,"allocated":1949696,"data_stored":1018703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22937600},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.89300000000000002},"min":{"1min":0.59999999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.151,"5min":1.27,"15min":1.3080000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.073,"5min":1.232,"15min":1.45},"last":1.216}]},{"osd":1,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.88600000000000001},"min":{"1min":0.64500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2549999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.92600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1559999999999999,"5min":1.573,"15min":1.573},"last":1.5029999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2390000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97899999999999998,"15min":0.96399999999999997},"min":{"1min":0.77000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.169,"5min":1.331,"15min":1.407},"last":1.431}]},{"osd":3,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.92100000000000004,"15min":0.92300000000000004},"min":{"1min":0.68300000000000005,"5min":0.61099999999999999,"15min":0.45500000000000002},"max":{"1min":1.2450000000000001,"5min":1.272,"15min":1.2829999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.034,"5min":1.0289999999999999,"15min":1.026},"min":{"1min":0.66100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3779999999999999,"5min":1.4850000000000001,"15min":1.5129999999999999},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93700000000000006,"15min":0.91600000000000004},"min":{"1min":0.55100000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.202,"5min":1.609,"15min":1.609},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2430000000000001,"5min":1.5269999999999999,"15min":1.5489999999999999},"last":1.3300000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.038,"15min":1.04},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3180000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.1519999999999999,"15min":1.1479999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.44800000000000001},"max":{"1min":1.276,"5min":1.536,"15min":1.536},"last":1.3480000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.091,"15min":1.095},"min":{"1min":0.82499999999999996,"5min":0.64000000000000001,"15min":0.59799999999999998},"max":{"1min":1.474,"5min":1.4770000000000001,"15min":1.627},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.85799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.651,"5min":1.7929999999999999,"15min":1.859},"last":1.0960000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509683,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":27392,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":1941504,"data_stored":1011832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28049408},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.218},"max":{"1min":0.96899999999999997,"5min":1.2410000000000001,"15min":1.3240000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75900000000000001},"min":{"1min":0.40799999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93100000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.64300000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.014,"5min":1.0740000000000001,"15min":1.2869999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82699999999999996,"15min":0.85999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.145,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.86199999999999999},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.95699999999999996},"min":{"1min":0.66900000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.288,"5min":1.613,"15min":1.613},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0640000000000001,"15min":1.075},"min":{"1min":0.67100000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.3109999999999999,"5min":1.605,"15min":1.605},"last":0.92200000000000004}]},{"osd":5,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1200000000000001,"15min":1.1519999999999999},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.099,"15min":1.1100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5409999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.099},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.075}]},{"osd":7,"last update":"Wed Sep 27 23:33:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.198,"15min":1.23},"min":{"1min":0.79300000000000004,"5min":0.59699999999999998,"15min":0.54800000000000004},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.04},{"interface":"front","average":{"1min":1.143,"5min":1.1519999999999999,"15min":1.1879999999999999},"min":{"1min":0.63400000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.405,"5min":1.837,"15min":1.837},"last":1.1799999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509684,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23164,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":94348676,"statfs":{"total":96636764160,"available":96613044224,"internally_reserved":0,"allocated":1953792,"data_stored":1013348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1699999999999999,"5min":1.222,"15min":1.222},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.52400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.903},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78200000000000003,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78900000000000003,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.28199999999999997},"max":{"1min":1.0569999999999999,"5min":1.228,"15min":1.391},"last":0.89800000000000002}]},{"osd":3,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.97599999999999998,"5min":1.49,"15min":1.9710000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":0.93100000000000005},"min":{"1min":0.66400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.26,"5min":1.399,"15min":1.444},"last":0.84999999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.014,"15min":1.032},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.40600000000000003},"max":{"1min":1.4139999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.056,"15min":1.054},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.635,"15min":1.7390000000000001},"last":1.0549999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0169999999999999,"15min":1.028},"min":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.59599999999999997},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":2.0459999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.135,"15min":1.1379999999999999},"min":{"1min":0.77900000000000003,"5min":0.68200000000000005,"15min":0.53600000000000003},"max":{"1min":1.635,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.123}]},{"osd":6,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.157},"min":{"1min":0.79700000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4830000000000001,"5min":1.722,"15min":1.9450000000000001},"last":1.204},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.1519999999999999},"min":{"1min":0.88,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.363,"5min":1.623,"15min":2.0129999999999999},"last":1.248}]},{"osd":7,"last update":"Wed Sep 27 23:33:05 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2010000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.69699999999999995,"15min":0.54000000000000004},"max":{"1min":1.573,"5min":1.8109999999999999,"15min":1.9510000000000001},"last":1.329},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.71799999999999997,"15min":0.625},"max":{"1min":1.351,"5min":1.556,"15min":2.073},"last":1.0900000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542390,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38332,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94333508,"statfs":{"total":96636764160,"available":96597512192,"internally_reserved":0,"allocated":1953792,"data_stored":1019257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.52400000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.048,"5min":1.123,"15min":1.3560000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1100000000000001,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95199999999999996,"5min":1.2490000000000001,"15min":1.321},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84499999999999997,"15min":0.876},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.41899999999999998},"max":{"1min":1.196,"5min":1.3819999999999999,"15min":1.399},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.208,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":1.2090000000000001,"5min":1.696,"15min":1.696},"last":0.92800000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.006,"15min":0.97499999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63100000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.774},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.78400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.72499999999999998},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.0349999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.206,"15min":1.222},"min":{"1min":0.80400000000000005,"5min":0.70999999999999996,"15min":0.61599999999999999},"max":{"1min":1.78,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.099},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1699999999999999,"15min":1.1890000000000001},"min":{"1min":0.81599999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed Sep 27 23:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1890000000000001,"15min":1.222},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8109999999999999,"5min":1.877,"15min":1.877},"last":1.165},{"interface":"front","average":{"1min":1.212,"5min":1.1899999999999999,"15min":1.206},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.78700000000000003},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.133}]}]},{"osd":0,"up_from":15,"seq":64424509685,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29504,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":27008,"kb_avail":94342336,"statfs":{"total":96636764160,"available":96606552064,"internally_reserved":0,"allocated":2547712,"data_stored":1603680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27656192},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.55700000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0129999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.501},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.58399999999999996,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1299999999999999,"5min":1.3,"15min":1.3},"last":1.236}]},{"osd":2,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.58999999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0720000000000001,"5min":1.1619999999999999,"15min":1.2969999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.58399999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.175,"5min":1.228,"15min":1.228},"last":0.90700000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.58299999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.004,"5min":1.117,"15min":1.2769999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.68300000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.27,"5min":1.3160000000000001,"15min":1.325},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.01,"15min":1.0069999999999999},"min":{"1min":0.85399999999999998,"5min":0.55300000000000005,"15min":0.51900000000000002},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.171,"5min":1.024,"15min":1.036},"min":{"1min":0.84199999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.1220000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.091,"15min":1.093},"min":{"1min":0.84699999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.78},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.1479999999999999,"15min":1.1339999999999999},"min":{"1min":0.80700000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.6739999999999999,"5min":4.6779999999999999,"15min":4.6779999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.1379999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.206,"15min":1.216},"min":{"1min":1.0589999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.8300000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.24,"15min":1.236},"min":{"1min":0.996,"5min":0.78500000000000003,"15min":0.72899999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.8109999999999999},"last":1.573},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2230000000000001,"15min":1.226},"min":{"1min":1.02,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.752,"5min":1.798,"15min":1.798},"last":1.1559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:34:04.288 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:34:04.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:34:04.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.402+0000 7f7aed6ea700 1 Processor -- start 2023-09-27T23:34:04.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.402+0000 7f7aed6ea700 1 -- start start 2023-09-27T23:34:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7aed6ea700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ae8157fc0 0x7f7ae8158480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7aed6ea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7aed6ea700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae80c7820 con 0x7f7ae81547a0 2023-09-27T23:34:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7aed6ea700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae80ade40 con 0x7f7ae8157fc0 2023-09-27T23:34:04.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7aed6ea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae806da10 con 0x7f7ae806d050 2023-09-27T23:34:04.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ae8157fc0 0x7f7ae8158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ae8157fc0 0x7f7ae8158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36234/0 (socket says 172.21.15.64:36234) 2023-09-27T23:34:04.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 -- 172.21.15.64:0/1004450997 learned_addr learned my addr 172.21.15.64:0/1004450997 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:04.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7aecee9700 1 --2- 172.21.15.64:0/1004450997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 -- 172.21.15.64:0/1004450997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 msgr2=0x7f7ae806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 --2- 172.21.15.64:0/1004450997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 -- 172.21.15.64:0/1004450997 >> v1:172.21.15.123:6789/0 conn(0x7f7ae81547a0 legacy=0x7f7ae80ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 -- 172.21.15.64:0/1004450997 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae80c7210 con 0x7f7ae8157fc0 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7ae77fe700 1 -- 172.21.15.64:0/1004450997 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205382998 0 0) 0x7f7ae80c7820 con 0x7f7ae81547a0 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7adf7fe700 1 --2- 172.21.15.64:0/1004450997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8158480 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00099e0 tx=0x7f7ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2e0f671535dfd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7ae77fe700 1 -- 172.21.15.64:0/1004450997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000b820 con 0x7f7ae8157fc0 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7ae77fe700 1 -- 172.21.15.64:0/1004450997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad000b980 con 0x7f7ae8157fc0 2023-09-27T23:34:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.406+0000 7f7ae77fe700 1 -- 172.21.15.64:0/1004450997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000bd20 con 0x7f7ae8157fc0 2023-09-27T23:34:04.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 -- 172.21.15.64:0/1004450997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae8157fc0 msgr2=0x7f7ae8158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/1004450997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8158480 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00099e0 tx=0x7f7ad0009250 comp rx=0 tx=0).stop 2023-09-27T23:34:04.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 -- 172.21.15.64:0/1004450997 shutdown_connections 2023-09-27T23:34:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/1004450997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/1004450997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8158480 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 -- 172.21.15.64:0/1004450997 >> 172.21.15.64:0/1004450997 conn(0x7f7ae80ccb30 msgr2=0x7f7ae80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 -- 172.21.15.64:0/1004450997 shutdown_connections 2023-09-27T23:34:04.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 -- 172.21.15.64:0/1004450997 wait complete. 2023-09-27T23:34:04.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 Processor -- start 2023-09-27T23:34:04.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.410+0000 7f7aed6ea700 1 -- start start 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae81669f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 0x7f7ae80ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80ad380 con 0x7f7ae81547a0 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8167990 con 0x7f7ae8157fc0 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8154d30 con 0x7f7ae806d050 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 0x7f7ae80ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 0x7f7ae80ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36250/0 (socket says 172.21.15.64:36250) 2023-09-27T23:34:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 -- 172.21.15.64:0/3663814786 learned_addr learned my addr 172.21.15.64:0/3663814786 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7ae7fff700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae81669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aecee9700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 -- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 msgr2=0x7f7ae81669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae81669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 -- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae8157fc0 msgr2=0x7f7ae8151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 -- 172.21.15.64:0/3663814786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00094f0 con 0x7f7ae81547a0 2023-09-27T23:34:04.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7adf7fe700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 0x7f7ae80ace40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f7ae806b210 tx=0x7f7ad0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0011e20 con 0x7f7ae81547a0 2023-09-27T23:34:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad001a450 con 0x7f7ae81547a0 2023-09-27T23:34:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae81670f0 con 0x7f7ae81547a0 2023-09-27T23:34:04.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad001a810 con 0x7f7ae81547a0 2023-09-27T23:34:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.414+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae81676b0 con 0x7f7ae81547a0 2023-09-27T23:34:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.418+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae804be40 con 0x7f7ae81547a0 2023-09-27T23:34:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.418+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7ad001ab80 con 0x7f7ae81547a0 2023-09-27T23:34:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.418+0000 7f7ae4ff9700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7ac8060aa0 0x7f7ac8062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.418+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7ad007ebd0 con 0x7f7ae81547a0 2023-09-27T23:34:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.418+0000 7f7ae7fff700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7ac8060aa0 0x7f7ac8062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.418+0000 7f7ae7fff700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7ac8060aa0 0x7f7ac8062f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8007fd0 tx=0x7f7ad8007d30 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.426+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7ad004eb60 con 0x7f7ae81547a0 2023-09-27T23:34:04.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.650+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7ae80ae030 con 0x7f7ae81547a0 2023-09-27T23:34:04.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7ae4ff9700 1 -- 172.21.15.64:0/3663814786 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7ad00150e0 con 0x7f7ae81547a0 2023-09-27T23:34:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7ac8060aa0 msgr2=0x7f7ac8062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7ac8060aa0 0x7f7ac8062f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8007fd0 tx=0x7f7ad8007d30 comp rx=0 tx=0).stop 2023-09-27T23:34:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 msgr2=0x7f7ae80ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 0x7f7ae80ace40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f7ae806b210 tx=0x7f7ad0017000 comp rx=0 tx=0).stop 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 shutdown_connections 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f7ac8060aa0 0x7f7ac8062f50 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8007fd0 tx=0x7f7ad8007d30 comp rx=0 tx=0).stop 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae8157fc0 0x7f7ae8151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ae81547a0 0x7f7ae80ace40 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 --2- 172.21.15.64:0/3663814786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae806d050 0x7f7ae81669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 >> 172.21.15.64:0/3663814786 conn(0x7f7ae80ccb30 msgr2=0x7f7ae80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 shutdown_connections 2023-09-27T23:34:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.662+0000 7f7aed6ea700 1 -- 172.21.15.64:0/3663814786 wait complete. 2023-09-27T23:34:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:34:04.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:34:04.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e66be9700 1 Processor -- start 2023-09-27T23:34:04.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e66be9700 1 -- start start 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e66be9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e6015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e66be9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6006d990 con 0x7f9e6006de20 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e66be9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6015a570 con 0x7f9e600c7f00 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e66be9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6006cd80 con 0x7f9e60155960 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e653e6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9e6006de20 0x7f9e600c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49296/0 (socket says 172.21.15.64:49296) 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.782+0000 7f9e653e6700 1 -- 172.21.15.64:0/1471873371 learned_addr learned my addr 172.21.15.64:0/1471873371 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:04.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e65be7700 1 --2- 172.21.15.64:0/1471873371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e6015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316860257 0 0) 0x7f9e6006d990 con 0x7f9e6006de20 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e48003570 con 0x7f9e6006de20 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120021306 0 0) 0x7f9e6015a570 con 0x7f9e600c7f00 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e6006d990 con 0x7f9e600c7f00 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91552794 0 0) 0x7f9e48003570 con 0x7f9e6006de20 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e65be7700 1 -- 172.21.15.64:0/1471873371 >> v1:172.21.15.123:6789/0 conn(0x7f9e600c7f00 legacy=0x7f9e600aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e65be7700 1 -- 172.21.15.64:0/1471873371 >> v1:172.21.15.64:6789/0 conn(0x7f9e6006de20 legacy=0x7f9e600c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e65be7700 1 -- 172.21.15.64:0/1471873371 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6006d060 con 0x7f9e60155960 2023-09-27T23:34:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e65be7700 1 --2- 172.21.15.64:0/1471873371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e6015a030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009910 tx=0x7f9e500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7c5244da0c8999 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54001470 con 0x7f9e60155960 2023-09-27T23:34:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50011420 con 0x7f9e60155960 2023-09-27T23:34:04.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e64be5700 1 -- 172.21.15.64:0/1471873371 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500117c0 con 0x7f9e60155960 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 -- 172.21.15.64:0/1471873371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 msgr2=0x7f9e6015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 --2- 172.21.15.64:0/1471873371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e6015a030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009910 tx=0x7f9e500091a0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 -- 172.21.15.64:0/1471873371 shutdown_connections 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 --2- 172.21.15.64:0/1471873371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e6015a030 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 -- 172.21.15.64:0/1471873371 >> 172.21.15.64:0/1471873371 conn(0x7f9e60067bb0 msgr2=0x7f9e600cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 -- 172.21.15.64:0/1471873371 shutdown_connections 2023-09-27T23:34:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 -- 172.21.15.64:0/1471873371 wait complete. 2023-09-27T23:34:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.786+0000 7f9e66be9700 1 Processor -- start 2023-09-27T23:34:04.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 -- start start 2023-09-27T23:34:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e6006de20 0x7f9e600aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e600c7f00 0x7f9e6014dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e60169670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600aec30 con 0x7f9e600c7f00 2023-09-27T23:34:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600ae8e0 con 0x7f9e6006de20 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e60169ec0 con 0x7f9e60155960 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e653e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e600c7f00 0x7f9e6014dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e6006de20 0x7f9e600aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e653e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e600c7f00 0x7f9e6014dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45904/0 (socket says 172.21.15.64:45904) 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e653e6700 1 -- 172.21.15.64:0/2509745788 learned_addr learned my addr 172.21.15.64:0/2509745788 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e663e8700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e60169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 -- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 msgr2=0x7f9e60169670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e60169670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 -- 172.21.15.64:0/2509745788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e600c7f00 msgr2=0x7f9e6014dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e600c7f00 0x7f9e6014dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 -- 172.21.15.64:0/2509745788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50009420 con 0x7f9e6006de20 2023-09-27T23:34:04.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e65be7700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e6006de20 0x7f9e600aff90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9e50015000 tx=0x7f9e50016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001cb30 con 0x7f9e6006de20 2023-09-27T23:34:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5001ccd0 con 0x7f9e6006de20 2023-09-27T23:34:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001a6a0 con 0x7f9e6006de20 2023-09-27T23:34:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.790+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6016a1a0 con 0x7f9e6006de20 2023-09-27T23:34:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.794+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e60169bb0 con 0x7f9e6006de20 2023-09-27T23:34:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.794+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6004be40 con 0x7f9e6006de20 2023-09-27T23:34:04.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.794+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9e50044ab0 con 0x7f9e6006de20 2023-09-27T23:34:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.794+0000 7f9e5dffb700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e4c060a70 0x7f9e4c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.794+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9e5007ee00 con 0x7f9e6006de20 2023-09-27T23:34:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.794+0000 7f9e653e6700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e4c060a70 0x7f9e4c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.798+0000 7f9e653e6700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e4c060a70 0x7f9e4c062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e600ab510 tx=0x7f9e54000bc0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:04.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:04.806+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9e5004ee30 con 0x7f9e6006de20 2023-09-27T23:34:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.002+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e6006d340 con 0x7f9e6006de20 2023-09-27T23:34:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e5dffb700 1 -- 172.21.15.64:0/2509745788 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9e50052450 con 0x7f9e6006de20 2023-09-27T23:34:05.009 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:34:05.009 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:56.514725+0000","modified":"2023-09-27T23:33:58.531554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3161207024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3161207024},{"type":"v1","addr":"172.21.15.64:6835","nonce":3161207024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-09-27T23:34:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e4c060a70 msgr2=0x7f9e4c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e4c060a70 0x7f9e4c062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e600ab510 tx=0x7f9e54000bc0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e6006de20 msgr2=0x7f9e600aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e6006de20 0x7f9e600aff90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9e50015000 tx=0x7f9e50016000 comp rx=0 tx=0).stop 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 shutdown_connections 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9e4c060a70 0x7f9e4c062f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60155960 0x7f9e60169670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e600c7f00 0x7f9e6014dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 --2- 172.21.15.64:0/2509745788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e6006de20 0x7f9e600aff90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 >> 172.21.15.64:0/2509745788 conn(0x7f9e60067bb0 msgr2=0x7f9e600ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 shutdown_connections 2023-09-27T23:34:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.006+0000 7f9e66be9700 1 -- 172.21.15.64:0/2509745788 wait complete. 2023-09-27T23:34:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 60 2023-09-27T23:34:05.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:33:56.514725+0000', 'modified': '2023-09-27T23:33:58.531554+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9183}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9183': {'gid': 9183, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/3161207024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 3161207024}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 3161207024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:34:05.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:34:05.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-27T23:34:05.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 Processor -- start 2023-09-27T23:34:05.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 -- start start 2023-09-27T23:34:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca1c158e70 0x7fca1c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fca1c06a540 0x7fca1c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c0cc470 0x7fca1c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c0ca5a0 con 0x7fca1c06a540 2023-09-27T23:34:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c06dac0 con 0x7fca1c158e70 2023-09-27T23:34:05.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca21fd4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b0260 con 0x7fca1c0cc470 2023-09-27T23:34:05.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca1bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fca1c06a540 0x7fca1c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca217d3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c0cc470 0x7fca1c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca1bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fca1c06a540 0x7fca1c06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45914/0 (socket says 172.21.15.64:45914) 2023-09-27T23:34:05.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca1bfff700 1 -- 172.21.15.64:0/3235912966 learned_addr learned my addr 172.21.15.64:0/3235912966 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca20fd2700 1 --2- 172.21.15.64:0/3235912966 >> v2:172.21.15.123:3300/0 conn(0x7fca1c158e70 0x7fca1c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca217d3700 1 -- 172.21.15.64:0/3235912966 >> v2:172.21.15.123:3300/0 conn(0x7fca1c158e70 msgr2=0x7fca1c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca217d3700 1 --2- 172.21.15.64:0/3235912966 >> v2:172.21.15.123:3300/0 conn(0x7fca1c158e70 0x7fca1c0afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca217d3700 1 -- 172.21.15.64:0/3235912966 >> v2:172.21.15.64:3300/0 conn(0x7fca1c06a540 msgr2=0x7fca1c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca217d3700 1 --2- 172.21.15.64:0/3235912966 >> v2:172.21.15.64:3300/0 conn(0x7fca1c06a540 0x7fca1c06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca217d3700 1 -- 172.21.15.64:0/3235912966 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c06d220 con 0x7fca1c0cc470 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.542+0000 7fca20fd2700 1 --2- 172.21.15.64:0/3235912966 >> v2:172.21.15.123:3300/0 conn(0x7fca1c158e70 0x7fca1c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:05.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca217d3700 1 --2- 172.21.15.64:0/3235912966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c0cc470 0x7fca1c0cc8a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca1c06a210 tx=0x7fca14009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45281a1182efcb18 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca1b7fe700 1 -- 172.21.15.64:0/3235912966 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1400e070 con 0x7fca1c0cc470 2023-09-27T23:34:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca1b7fe700 1 -- 172.21.15.64:0/3235912966 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1400bab0 con 0x7fca1c0cc470 2023-09-27T23:34:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca1b7fe700 1 -- 172.21.15.64:0/3235912966 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1400be50 con 0x7fca1c0cc470 2023-09-27T23:34:05.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 -- 172.21.15.64:0/3235912966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c0cc470 msgr2=0x7fca1c0cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 --2- 172.21.15.64:0/3235912966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c0cc470 0x7fca1c0cc8a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca1c06a210 tx=0x7fca14009780 comp rx=0 tx=0).stop 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 -- 172.21.15.64:0/3235912966 shutdown_connections 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 --2- 172.21.15.64:0/3235912966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c0cc470 0x7fca1c0cc8a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 --2- 172.21.15.64:0/3235912966 >> v2:172.21.15.64:3300/0 conn(0x7fca1c06a540 0x7fca1c06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 --2- 172.21.15.64:0/3235912966 >> v2:172.21.15.123:3300/0 conn(0x7fca1c158e70 0x7fca1c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 -- 172.21.15.64:0/3235912966 >> 172.21.15.64:0/3235912966 conn(0x7fca1c0670e0 msgr2=0x7fca1c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 -- 172.21.15.64:0/3235912966 shutdown_connections 2023-09-27T23:34:05.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 -- 172.21.15.64:0/3235912966 wait complete. 2023-09-27T23:34:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 Processor -- start 2023-09-27T23:34:05.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.546+0000 7fca21fd4700 1 -- start start 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 0x7fca1c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c0cc470 0x7fca1c158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c158e70 0x7fca1c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c15b470 con 0x7fca1c06a540 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b4680 con 0x7fca1c0cc470 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b3ab0 con 0x7fca1c158e70 2023-09-27T23:34:05.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca20fd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 0x7fca1c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca20fd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 0x7fca1c0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45922/0 (socket says 172.21.15.64:45922) 2023-09-27T23:34:05.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca217d3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c158e70 0x7fca1c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca20fd2700 1 -- 172.21.15.64:0/2678114647 learned_addr learned my addr 172.21.15.64:0/2678114647 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c0cc470 0x7fca1c158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 -- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c158e70 msgr2=0x7fca1c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c158e70 0x7fca1c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 -- 172.21.15.64:0/2678114647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 msgr2=0x7fca1c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 0x7fca1c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 -- 172.21.15.64:0/2678114647 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca10009420 con 0x7fca1c0cc470 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca20fd2700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 0x7fca1c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:05.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca1bfff700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c0cc470 0x7fca1c158120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fca0c006d40 tx=0x7fca0c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca18ff9700 1 -- 172.21.15.64:0/2678114647 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c013680 con 0x7fca1c0cc470 2023-09-27T23:34:05.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca18ff9700 1 -- 172.21.15.64:0/2678114647 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c0137e0 con 0x7fca1c0cc470 2023-09-27T23:34:05.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca14009a20 con 0x7fca1c0cc470 2023-09-27T23:34:05.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca18ff9700 1 -- 172.21.15.64:0/2678114647 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c013b60 con 0x7fca1c0cc470 2023-09-27T23:34:05.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.550+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c0b40b0 con 0x7fca1c0cc470 2023-09-27T23:34:05.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.554+0000 7fca18ff9700 1 -- 172.21.15.64:0/2678114647 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fca0c00c3e0 con 0x7fca1c0cc470 2023-09-27T23:34:05.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.558+0000 7fca18ff9700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca08060a70 0x7fca08062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.558+0000 7fca18ff9700 1 -- 172.21.15.64:0/2678114647 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fca0c07d1a0 con 0x7fca1c0cc470 2023-09-27T23:34:05.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.558+0000 7fca20fd2700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca08060a70 0x7fca08062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.558+0000 7fca20fd2700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca08060a70 0x7fca08062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca10011640 tx=0x7fca100091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.558+0000 7fca254df740 1 Processor -- start 2023-09-27T23:34:05.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.558+0000 7fca254df740 1 -- start start 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd88e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd89390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xcb6f30 con 0xb0b710 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xcb2ed0 con 0xb56790 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xc790b0 con 0xb0bae0 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd89390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd89390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45926/0 (socket says 172.21.15.64:45926) 2023-09-27T23:34:05.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca20fd2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd88e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 -- 172.21.15.64:0/864974117 learned_addr learned my addr 172.21.15.64:0/864974117 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca217d3700 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 -- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 msgr2=0xd88e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd88e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 -- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 msgr2=0xd898d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd898d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 -- 172.21.15.64:0/864974117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xb579e0 con 0xb0b710 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca1bfff700 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd89390 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fca0c04d390 tx=0x7fca0c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5158828dcfb52e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca01ffb700 1 -- 172.21.15.64:0/864974117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c066070 con 0xb0b710 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca01ffb700 1 -- 172.21.15.64:0/864974117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c050c60 con 0xb0b710 2023-09-27T23:34:05.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca01ffb700 1 -- 172.21.15.64:0/864974117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c054120 con 0xb0b710 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- 172.21.15.64:0/864974117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 msgr2=0xd89390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd89390 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fca0c04d390 tx=0x7fca0c006f90 comp rx=0 tx=0).stop 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- 172.21.15.64:0/864974117 shutdown_connections 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd89390 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 --2- 172.21.15.64:0/864974117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd88e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- 172.21.15.64:0/864974117 >> 172.21.15.64:0/864974117 conn(0xb0beb0 msgr2=0xd45880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- 172.21.15.64:0/864974117 shutdown_connections 2023-09-27T23:34:05.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.562+0000 7fca254df740 1 -- 172.21.15.64:0/864974117 wait complete. 2023-09-27T23:34:05.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 -- start start 2023-09-27T23:34:05.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 Processor -- start 2023-09-27T23:34:05.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb0bae0 0xd6ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 0xd6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xc78d50 con 0xb56790 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xc51970 con 0xb0bae0 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xcb9290 con 0xb0b710 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca254df740 10 client.0 _ll_register_callbacks cb 0xcfcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 0xd6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 0xd6f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45938/0 (socket says 172.21.15.64:45938) 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb0bae0 0xd6ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 -- 172.21.15.64:0/2638388040 learned_addr learned my addr 172.21.15.64:0/2638388040 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca20fd2700 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 -- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 msgr2=0xd45740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd45740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 -- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb0bae0 msgr2=0xd6ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb0bae0 0xd6ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 -- 172.21.15.64:0/2638388040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c0b40b0 con 0xb56790 2023-09-27T23:34:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fca217d3700 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 0xd6f140 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fca140128f0 tx=0x7fca14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f202f8defa39d7d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.566+0000 7fc9faffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 1 -- 172.21.15.64:0/2638388040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14022070 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 1 -- 172.21.15.64:0/2638388040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fca14012db0 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc51cc0 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 1 -- 172.21.15.64:0/2638388040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca14027610 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 10 client.9351 fetch_fsmap learned FSMap version 60 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0xc81370 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 1 -- 172.21.15.64:0/2638388040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc51cc0 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 1 -- 172.21.15.64:0/2638388040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 60) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca14027770 con 0xb56790 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 10 client.9351 fetch_fsmap finished waiting for FSMap version 60 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 10 client.9351 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:34:05.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 20 client.9351 populate_metadata read hostname 'smithi064' 2023-09-27T23:34:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xd956b0 0xd97a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 4 client.9351 mds_command: new command op to 9183 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:34:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 --> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc553e0 con 0xd956b0 2023-09-27T23:34:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca20fd2700 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xd956b0 0xd97a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 10 client.9351 ms_handle_connect on v2:172.21.15.64:6834/3161207024 2023-09-27T23:34:05.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca20fd2700 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xd956b0 0xd97a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74714ee4a4217b3c server_cookie=d580d67d2503629f in_seq=0 out_seq=0 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 1 -- 172.21.15.64:0/2638388040 <== mds.0 v2:172.21.15.64:6834/3161207024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0xc553e0 con 0xd956b0 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 10 client.9351 handle_command_reply: tid=0 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 client.9351 shutdown 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 msgr2=0xd6f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 0xd6f140 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fca140128f0 tx=0x7fca14019000 comp rx=0 tx=0).stop 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 shutdown_connections 2023-09-27T23:34:05.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xd956b0 0xd97a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb56790 0xd6f140 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb0bae0 0xd6ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 --2- 172.21.15.64:0/2638388040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd45740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 >> 172.21.15.64:0/2638388040 conn(0xb0beb0 msgr2=0xd3ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fc9faffd700 0 client.9351 ms_handle_reset on v2:172.21.15.64:6834/3161207024 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 shutdown_connections 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 1 -- 172.21.15.64:0/2638388040 wait complete. 2023-09-27T23:34:05.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.570+0000 7fca254df740 20 client.9351 trim_cache size 0 max 16384 2023-09-27T23:34:05.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 Processor -- start 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 -- start start 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xcda3b0 con 0xb0b710 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xb847c0 con 0xb56790 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.590+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xc64790 con 0xb0bae0 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca20fd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca20fd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45948/0 (socket says 172.21.15.64:45948) 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca20fd2700 1 -- 172.21.15.64:0/3282453607 learned_addr learned my addr 172.21.15.64:0/3282453607 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca217d3700 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 -- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 msgr2=0xd98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 -- 172.21.15.64:0/3282453607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 msgr2=0xd976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 -- 172.21.15.64:0/3282453607 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0cf150 con 0xb0bae0 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca20fd2700 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca1bfff700 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd97c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fca0c050e20 tx=0x7fca0c064000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc143599355eed6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca027fc700 1 -- 172.21.15.64:0/3282453607 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c066040 con 0xb0bae0 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca027fc700 1 -- 172.21.15.64:0/3282453607 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c04d1d0 con 0xb0bae0 2023-09-27T23:34:05.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca027fc700 1 -- 172.21.15.64:0/3282453607 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c057e70 con 0xb0bae0 2023-09-27T23:34:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 -- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 msgr2=0xd97c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd97c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fca0c050e20 tx=0x7fca0c064000 comp rx=0 tx=0).stop 2023-09-27T23:34:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 -- 172.21.15.64:0/3282453607 shutdown_connections 2023-09-27T23:34:05.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0bae0 0xd97c00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 --2- 172.21.15.64:0/3282453607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0b710 0xd976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 -- 172.21.15.64:0/3282453607 >> 172.21.15.64:0/3282453607 conn(0xb0beb0 msgr2=0xd3e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 -- 172.21.15.64:0/3282453607 shutdown_connections 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.594+0000 7fca254df740 1 -- 172.21.15.64:0/3282453607 wait complete. 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 -- start start 2023-09-27T23:34:05.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 Processor -- start 2023-09-27T23:34:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd6a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 0xd6aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd6afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0xc5c510 con 0xb0bae0 2023-09-27T23:34:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0xbdfda0 con 0xb56790 2023-09-27T23:34:05.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0xcb7810 con 0xb0b710 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 0xd6aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 0xd6aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45954/0 (socket says 172.21.15.64:45954) 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 -- 172.21.15.64:0/39184536 learned_addr learned my addr 172.21.15.64:0/39184536 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca20fd2700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd6a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca254df740 10 client.0 _ll_register_callbacks cb 0xbcf0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca217d3700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd6afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 -- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 msgr2=0xd6a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd6a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 -- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 msgr2=0xd6afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd6afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca1bfff700 1 -- 172.21.15.64:0/39184536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca10014870 con 0xb0bae0 2023-09-27T23:34:05.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.598+0000 7fca20fd2700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd6a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:05.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca1bfff700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 0xd6aa80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fca0c050b90 tx=0x7fca0c06a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b785f538994ad0f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 1 -- 172.21.15.64:0/39184536 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c066040 con 0xb0bae0 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 1 -- 172.21.15.64:0/39184536 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fca0c057e70 con 0xb0bae0 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca0c050c60 con 0xb0bae0 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 1 -- 172.21.15.64:0/39184536 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca0c04d5a0 con 0xb0bae0 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 10 client.9357 fetch_fsmap learned FSMap version 60 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0xb579e0 con 0xb0bae0 2023-09-27T23:34:05.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 1 -- 172.21.15.64:0/39184536 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c071080 con 0xb0bae0 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 1 -- 172.21.15.64:0/39184536 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 60) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca0c051130 con 0xb0bae0 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 10 client.9357 fetch_fsmap finished waiting for FSMap version 60 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 10 client.9357 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 20 client.9357 populate_metadata read hostname 'smithi064' 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xde8320 0xdea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 4 client.9357 mds_command: new command op to 9183 tid=0[{"prefix": "status"}] 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 --> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] -- command(tid 0: {"prefix": "status"}) v1 -- 0xc87410 con 0xde8320 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca20fd2700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xde8320 0xdea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca20fd2700 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xde8320 0xdea700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6522e3ea16b84fa0 server_cookie=4d653b60aca6e161 in_seq=0 out_seq=0 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 10 client.9357 ms_handle_connect on v2:172.21.15.64:6834/3161207024 2023-09-27T23:34:05.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 1 -- 172.21.15.64:0/39184536 <== mds.0 v2:172.21.15.64:6834/3161207024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0xc87410 con 0xde8320 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 10 client.9357 handle_command_reply: tid=0 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 client.9357 shutdown 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 msgr2=0xd6aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 0xd6aa80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fca0c050b90 tx=0x7fca0c06a000 comp rx=0 tx=0).stop 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 shutdown_connections 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0xb56790 0xd6afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:6834/3161207024,v1:172.21.15.64:6835/3161207024] conn(0xde8320 0xdea700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0xb0bae0 0xd6aa80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fc9faffd700 0 client.9357 ms_handle_reset on v2:172.21.15.64:6834/3161207024 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 --2- 172.21.15.64:0/39184536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0xb0b710 0xd6a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 >> 172.21.15.64:0/39184536 conn(0xb0beb0 msgr2=0xd6dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 shutdown_connections 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 1 -- 172.21.15.64:0/39184536 wait complete. 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.602+0000 7fca254df740 20 client.9357 trim_cache size 0 max 16384 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 9183, 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:34:05.611 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:34:05.612 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.0876357260000002, 2023-09-27T23:34:05.612 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 60, 2023-09-27T23:34:05.612 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 157, 2023-09-27T23:34:05.612 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 157, 2023-09-27T23:34:05.612 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 13.649696185 2023-09-27T23:34:05.612 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca08060a70 msgr2=0x7fca08062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca08060a70 0x7fca08062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca10011640 tx=0x7fca100091a0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c0cc470 msgr2=0x7fca1c158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c0cc470 0x7fca1c158120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fca0c006d40 tx=0x7fca0c009250 comp rx=0 tx=0).stop 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 shutdown_connections 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fca08060a70 0x7fca08062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca1c158e70 0x7fca1c0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c0cc470 0x7fca1c158120 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 --2- 172.21.15.64:0/2678114647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca1c06a540 0x7fca1c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 >> 172.21.15.64:0/2678114647 conn(0x7fca1c0670e0 msgr2=0x7fca1c0674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 shutdown_connections 2023-09-27T23:34:05.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.606+0000 7fca21fd4700 1 -- 172.21.15.64:0/2678114647 wait complete. 2023-09-27T23:34:05.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:34:05.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.730+0000 7f847359e700 1 Processor -- start 2023-09-27T23:34:05.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.730+0000 7f847359e700 1 -- start start 2023-09-27T23:34:05.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.730+0000 7f847359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f84740c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.730+0000 7f847359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84740ccb00 con 0x7f84740c8360 2023-09-27T23:34:05.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.730+0000 7f847359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84740cbef0 con 0x7f847415add0 2023-09-27T23:34:05.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.730+0000 7f847359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84740cc1d0 con 0x7f84740acdd0 2023-09-27T23:34:05.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f84740c8360 0x7f847415a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49308/0 (socket says 172.21.15.64:49308) 2023-09-27T23:34:05.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 -- 172.21.15.64:0/3118338377 learned_addr learned my addr 172.21.15.64:0/3118338377 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847259c700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f84740c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928194065 0 0) 0x7f84740ccb00 con 0x7f84740c8360 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8460003570 con 0x7f84740c8360 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463319193 0 0) 0x7f8460003570 con 0x7f84740c8360 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 msgr2=0x7f84740c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f84740c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 >> v1:172.21.15.123:6789/0 conn(0x7f847415add0 legacy=0x7f84740ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740cc4b0 con 0x7f84740c8360 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f845c001470 con 0x7f84740c8360 2023-09-27T23:34:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847259c700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f84740c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 >> v1:172.21.15.64:6789/0 conn(0x7f84740c8360 legacy=0x7f847415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8460003a90 0x7f8460003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8460004810 0x7f8460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f8460005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84740cc1d0 con 0x7f8460004810 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846000bbf0 con 0x7f8460003a90 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846000bdb0 con 0x7f84740acdd0 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8460004810 0x7f8460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847259c700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8460003a90 0x7f8460003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8472d9d700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f8460005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 -- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 msgr2=0x7f8460005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84740acdd0 0x7f8460005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 -- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8460003a90 msgr2=0x7f8460003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8460003a90 0x7f8460003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 -- 172.21.15.64:0/3118338377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460003570 con 0x7f8460004810 2023-09-27T23:34:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8469d9b700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8460004810 0x7f8460004c80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f845c00c3c0 tx=0x7f845c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7878d5a4a842f484 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00cc90 con 0x7f8460004810 2023-09-27T23:34:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c00cdf0 con 0x7f8460004810 2023-09-27T23:34:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8471d9b700 1 -- 172.21.15.64:0/3118338377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c005b50 con 0x7f8460004810 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847359e700 1 -- 172.21.15.64:0/3118338377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8460004810 msgr2=0x7f8460004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847359e700 1 --2- 172.21.15.64:0/3118338377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8460004810 0x7f8460004c80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f845c00c3c0 tx=0x7f845c001e50 comp rx=0 tx=0).stop 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f8472d9d700 1 -- 172.21.15.64:0/3118338377 reap_dead start 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847359e700 1 -- 172.21.15.64:0/3118338377 shutdown_connections 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847359e700 1 -- 172.21.15.64:0/3118338377 >> 172.21.15.64:0/3118338377 conn(0x7f84740cdbd0 msgr2=0x7f84740cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847359e700 1 -- 172.21.15.64:0/3118338377 shutdown_connections 2023-09-27T23:34:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.734+0000 7f847359e700 1 -- 172.21.15.64:0/3118338377 wait complete. 2023-09-27T23:34:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.738+0000 7f847359e700 1 Processor -- start 2023-09-27T23:34:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.738+0000 7f847359e700 1 -- start start 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8460004810 0x7f8474167470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 0x7f84740b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f847416a010 0x7f847416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84740b2270 con 0x7f8474157150 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474158ce0 con 0x7f847416a010 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84740b19d0 con 0x7f8460004810 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 0x7f84740b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 0x7f84740b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45982/0 (socket says 172.21.15.64:45982) 2023-09-27T23:34:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 -- 172.21.15.64:0/2145246252 learned_addr learned my addr 172.21.15.64:0/2145246252 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847259c700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8460004810 0x7f8474167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8472d9d700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f847416a010 0x7f847416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 -- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8460004810 msgr2=0x7f8474167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8460004810 0x7f8474167470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 -- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f847416a010 msgr2=0x7f847416a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f847416a010 0x7f847416a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 -- 172.21.15.64:0/2145246252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740b1cb0 con 0x7f8474157150 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f8469d9b700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 0x7f84740b1180 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f845c007cd0 tx=0x7f845c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0070d0 con 0x7f8474157150 2023-09-27T23:34:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c007230 con 0x7f8474157150 2023-09-27T23:34:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84740b1f90 con 0x7f8474157150 2023-09-27T23:34:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0075f0 con 0x7f8474157150 2023-09-27T23:34:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847406d790 con 0x7f8474157150 2023-09-27T23:34:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.742+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84740cccb0 con 0x7f8474157150 2023-09-27T23:34:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.746+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f845c019030 con 0x7f8474157150 2023-09-27T23:34:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.746+0000 7f846affd700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8454060a70 0x7f8454062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.746+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f845c07e6a0 con 0x7f8474157150 2023-09-27T23:34:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.746+0000 7f847259c700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8454060a70 0x7f8454062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:05.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.746+0000 7f847259c700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8454060a70 0x7f8454062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8464006fd0 tx=0x7f8464008040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:05.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.754+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f84740cccb0 con 0x7f8474157150 2023-09-27T23:34:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84740c9c80 con 0x7f8474157150 2023-09-27T23:34:05.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f846affd700 1 -- 172.21.15.64:0/2145246252 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84740c9c80 con 0x7f8474157150 2023-09-27T23:34:05.957 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:34:05.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8454060a70 msgr2=0x7f8454062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8454060a70 0x7f8454062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8464006fd0 tx=0x7f8464008040 comp rx=0 tx=0).stop 2023-09-27T23:34:05.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 msgr2=0x7f84740b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:05.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 0x7f84740b1180 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f845c007cd0 tx=0x7f845c015000 comp rx=0 tx=0).stop 2023-09-27T23:34:05.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 shutdown_connections 2023-09-27T23:34:05.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f8454060a70 0x7f8454062f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f847416a010 0x7f847416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8474157150 0x7f84740b1180 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 --2- 172.21.15.64:0/2145246252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8460004810 0x7f8474167470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 >> 172.21.15.64:0/2145246252 conn(0x7f84740cdbd0 msgr2=0x7f84740c41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 shutdown_connections 2023-09-27T23:34:05.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:05.954+0000 7f847359e700 1 -- 172.21.15.64:0/2145246252 wait complete. 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:34:05.972 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:34:05.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:34:05.973 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41338sec preferred_lft 41338sec 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:34:05.982 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:34:05.983 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:34:05.984 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:34:05.984 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:34:05.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:34:05.984 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:34:05.998 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:34:06.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:34:06.000 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:34:06.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:34:06.009 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:34:06.009 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:34:06.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:34:06.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:34:06.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:34:06.009 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:34:06.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:34:06.095 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:34:06.102 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:34:06.102 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:34:06.102 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:34:06.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:34:06.102 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:34:06.115 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:34:06.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:34:06.115 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:34:06.121 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:34:06.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:34:06.122 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:34:06.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:34:06.184 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:34:06.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:34:06.200 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:34:06.201 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:34:06.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:06.201 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:34:06.207 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:34:06.207 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:34:06.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:34:06.225 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:34:06.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:34:06.284 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:34:06.285 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:34:06.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:06.286 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:34:06.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:34:06.290+0000 7f026aa64380 -1 init, newargv = 0x5643057ccdf0 newargc=15 2023-09-27T23:34:06.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[58501]: starting ceph client 2023-09-27T23:34:06.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[58501]: starting fuse 2023-09-27T23:34:07.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:34:07.409 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:34:07.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:34:07.468 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:34:07.468 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:34:07.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:07.469 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:34:07.476 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:34:07.476 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:34:07.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:34:07.476 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:34:07.477 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:34:07.478 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:34:07.562 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.58501.asok 2023-09-27T23:34:07.745 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58501.asok 2023-09-27T23:34:07.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:07.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58501.asok status 2023-09-27T23:34:07.839 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "58501", 2023-09-27T23:34:07.840 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 9369, 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 9369 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:34:07.841 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1007850438 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:34:07.842 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1007850438 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.9369 192.168.0.1:0/1007850438", 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/1007850438", 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 60, 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 157, 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:34:07.843 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:34:07.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:07.849 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:34:07.849 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:34:07.855 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:34:07.855 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:34:07.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:07.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:34:07.866 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:34:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.974+0000 7f0d37e92700 1 Processor -- start 2023-09-27T23:34:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d37e92700 1 -- start start 2023-09-27T23:34:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d37e92700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d37e92700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300af8c0 con 0x7f0d300abb20 2023-09-27T23:34:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d37e92700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d30159ad0 con 0x7f0d30155450 2023-09-27T23:34:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d37e92700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30159db0 con 0x7f0d30155820 2023-09-27T23:34:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3542d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0d300abb20 0x7f0d300c98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49312/0 (socket says 172.21.15.64:49312) 2023-09-27T23:34:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3542d700 1 -- 172.21.15.64:0/1154090981 learned_addr learned my addr 172.21.15.64:0/1154090981 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3642f700 1 --2- 172.21.15.64:0/1154090981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d34c2c700 1 -- 172.21.15.64:0/1154090981 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669805160 0 0) 0x7f0d300af8c0 con 0x7f0d300abb20 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d34c2c700 1 -- 172.21.15.64:0/1154090981 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d1c003570 con 0x7f0d300abb20 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3642f700 1 -- 172.21.15.64:0/1154090981 >> v1:172.21.15.123:6789/0 conn(0x7f0d30155450 legacy=0x7f0d30159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3642f700 1 -- 172.21.15.64:0/1154090981 >> v1:172.21.15.64:6789/0 conn(0x7f0d300abb20 legacy=0x7f0d300c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d34c2c700 1 -- 172.21.15.64:0/1154090981 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202271268 0 0) 0x7f0d30159ad0 con 0x7f0d30155450 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3642f700 1 -- 172.21.15.64:0/1154090981 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3015a090 con 0x7f0d30155820 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d3642f700 1 --2- 172.21.15.64:0/1154090981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006b310 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009e90 tx=0x7f0d28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c742a044b148be2e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d34c2c700 1 -- 172.21.15.64:0/1154090981 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800e070 con 0x7f0d30155820 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d34c2c700 1 -- 172.21.15.64:0/1154090981 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800bc30 con 0x7f0d30155820 2023-09-27T23:34:07.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.978+0000 7f0d34c2c700 1 -- 172.21.15.64:0/1154090981 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28012430 con 0x7f0d30155820 2023-09-27T23:34:07.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- 172.21.15.64:0/1154090981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 msgr2=0x7f0d3006b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:07.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 --2- 172.21.15.64:0/1154090981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006b310 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009e90 tx=0x7f0d28009780 comp rx=0 tx=0).stop 2023-09-27T23:34:07.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- 172.21.15.64:0/1154090981 shutdown_connections 2023-09-27T23:34:07.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 --2- 172.21.15.64:0/1154090981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006b310 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:07.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- 172.21.15.64:0/1154090981 >> 172.21.15.64:0/1154090981 conn(0x7f0d30067540 msgr2=0x7f0d30067910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:07.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- 172.21.15.64:0/1154090981 shutdown_connections 2023-09-27T23:34:07.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- 172.21.15.64:0/1154090981 wait complete. 2023-09-27T23:34:07.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 Processor -- start 2023-09-27T23:34:07.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- start start 2023-09-27T23:34:07.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 0x7f0d30167a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155450 0x7f0d30164c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30167fd0 con 0x7f0d300abb20 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30169360 con 0x7f0d30155450 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.982+0000 7f0d37e92700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30166ec0 con 0x7f0d30155820 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 0x7f0d30167a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 0x7f0d30167a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45994/0 (socket says 172.21.15.64:45994) 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 -- 172.21.15.64:0/293997045 learned_addr learned my addr 172.21.15.64:0/293997045 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d3642f700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:07.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d3542d700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155450 0x7f0d30164c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:07.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 -- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 msgr2=0x7f0d3006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 -- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155450 msgr2=0x7f0d30164c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155450 0x7f0d30164c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 -- 172.21.15.64:0/293997045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009a20 con 0x7f0d300abb20 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d3642f700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d35c2e700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 0x7f0d30167a90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c7e0 tx=0x7f0d2c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0105c0 con 0x7f0d300abb20 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c010740 con 0x7f0d300abb20 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d30168750 con 0x7f0d300abb20 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c010b00 con 0x7f0d300abb20 2023-09-27T23:34:07.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d30168d10 con 0x7f0d300abb20 2023-09-27T23:34:07.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.986+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d30071120 con 0x7f0d300abb20 2023-09-27T23:34:07.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.990+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0d2c01b3b0 con 0x7f0d300abb20 2023-09-27T23:34:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.990+0000 7f0d25ffb700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d14060a70 0x7f0d14062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.990+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0d2c07cb70 con 0x7f0d300abb20 2023-09-27T23:34:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.990+0000 7f0d3542d700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d14060a70 0x7f0d14062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.990+0000 7f0d3542d700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d14060a70 0x7f0d14062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d30164c00 tx=0x7f0d20009040 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:07.998+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0d2c04cba0 con 0x7f0d300abb20 2023-09-27T23:34:08.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.186+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d300aef50 con 0x7f0d14060a70 2023-09-27T23:34:08.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.186+0000 7f0d25ffb700 1 -- 172.21.15.64:0/293997045 <== mgr.8979 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0d300aef50 con 0x7f0d14060a70 2023-09-27T23:34:08.192 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:34:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:34:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:34:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:34:08.192 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:34:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d14060a70 msgr2=0x7f0d14062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d14060a70 0x7f0d14062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d30164c00 tx=0x7f0d20009040 comp rx=0 tx=0).stop 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 msgr2=0x7f0d30167a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 0x7f0d30167a90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c7e0 tx=0x7f0d2c007df0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 shutdown_connections 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f0d14060a70 0x7f0d14062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d30155820 0x7f0d3006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155450 0x7f0d30164c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 --2- 172.21.15.64:0/293997045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d300abb20 0x7f0d30167a90 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 >> 172.21.15.64:0/293997045 conn(0x7f0d30067540 msgr2=0x7f0d30156b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 shutdown_connections 2023-09-27T23:34:08.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.190+0000 7f0d37e92700 1 -- 172.21.15.64:0/293997045 wait complete. 2023-09-27T23:34:08.206 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:34:08.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 Processor -- start 2023-09-27T23:34:08.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 -- start start 2023-09-27T23:34:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcf4413f2d0 0x7fcf440ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf440c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf440adc60 con 0x7fcf4413f6a0 2023-09-27T23:34:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4406d7d0 con 0x7fcf4413f2d0 2023-09-27T23:34:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.310+0000 7fcf43fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440c8140 con 0x7fcf440c7780 2023-09-27T23:34:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf440c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf440c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:38532/0 (socket says 172.21.15.64:38532) 2023-09-27T23:34:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 -- 172.21.15.64:0/4055400931 learned_addr learned my addr 172.21.15.64:0/4055400931 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf42ffd700 1 --2- 172.21.15.64:0/4055400931 >> v2:172.21.15.123:3300/0 conn(0x7fcf4413f2d0 0x7fcf440ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 -- 172.21.15.64:0/4055400931 >> v2:172.21.15.123:3300/0 conn(0x7fcf4413f2d0 msgr2=0x7fcf440ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 --2- 172.21.15.64:0/4055400931 >> v2:172.21.15.123:3300/0 conn(0x7fcf4413f2d0 0x7fcf440ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 -- 172.21.15.64:0/4055400931 >> v1:172.21.15.123:6789/0 conn(0x7fcf4413f6a0 legacy=0x7fcf4415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 -- 172.21.15.64:0/4055400931 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440ad050 con 0x7fcf440c7780 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf437fe700 1 --2- 172.21.15.64:0/4055400931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf440c7c00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099b0 tx=0x7fcf3c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f581cfc464e4a5a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf41ffb700 1 -- 172.21.15.64:0/4055400931 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c009ea0 con 0x7fcf440c7780 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf41ffb700 1 -- 172.21.15.64:0/4055400931 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00b860 con 0x7fcf440c7780 2023-09-27T23:34:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf41ffb700 1 -- 172.21.15.64:0/4055400931 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00bc00 con 0x7fcf440c7780 2023-09-27T23:34:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 -- 172.21.15.64:0/4055400931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 msgr2=0x7fcf440c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 --2- 172.21.15.64:0/4055400931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf440c7c00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099b0 tx=0x7fcf3c009250 comp rx=0 tx=0).stop 2023-09-27T23:34:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 -- 172.21.15.64:0/4055400931 shutdown_connections 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 --2- 172.21.15.64:0/4055400931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf440c7c00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 --2- 172.21.15.64:0/4055400931 >> v2:172.21.15.123:3300/0 conn(0x7fcf4413f2d0 0x7fcf440ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 -- 172.21.15.64:0/4055400931 >> 172.21.15.64:0/4055400931 conn(0x7fcf44067b50 msgr2=0x7fcf440cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 -- 172.21.15.64:0/4055400931 shutdown_connections 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.314+0000 7fcf43fff700 1 -- 172.21.15.64:0/4055400931 wait complete. 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 Processor -- start 2023-09-27T23:34:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 -- start start 2023-09-27T23:34:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf44135f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 0x7fcf440c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4413f6a0 0x7fcf441554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4413c990 con 0x7fcf4413f2d0 2023-09-27T23:34:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4413bd80 con 0x7fcf4413f6a0 2023-09-27T23:34:08.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4413c060 con 0x7fcf440c7780 2023-09-27T23:34:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 0x7fcf440c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 0x7fcf440c0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46010/0 (socket says 172.21.15.64:46010) 2023-09-27T23:34:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 -- 172.21.15.64:0/443640642 learned_addr learned my addr 172.21.15.64:0/443640642 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf42ffd700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf44135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf437fe700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4413f6a0 0x7fcf441554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 -- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 msgr2=0x7fcf44135f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf44135f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 -- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4413f6a0 msgr2=0x7fcf441554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4413f6a0 0x7fcf441554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 -- 172.21.15.64:0/443640642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0094f0 con 0x7fcf4413f2d0 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf437fe700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4413f6a0 0x7fcf441554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf427fc700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 0x7fcf440c0ae0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c910 tx=0x7fcf340016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34012630 con 0x7fcf4413f2d0 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf340127b0 con 0x7fcf4413f2d0 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4413c340 con 0x7fcf4413f2d0 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34012b70 con 0x7fcf4413f2d0 2023-09-27T23:34:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.318+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf44136480 con 0x7fcf4413f2d0 2023-09-27T23:34:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.322+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 65) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fcf34003030 con 0x7fcf4413f2d0 2023-09-27T23:34:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.322+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4404be40 con 0x7fcf4413f2d0 2023-09-27T23:34:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.322+0000 7fcf2affd700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf30060a70 0x7fcf30062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.322+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fcf3407cfe0 con 0x7fcf4413f2d0 2023-09-27T23:34:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.322+0000 7fcf42ffd700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf30060a70 0x7fcf30062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.322+0000 7fcf42ffd700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf30060a70 0x7fcf30062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009db0 tx=0x7fcf380091a0 comp rx=0 tx=0).ready entity=mgr.8979 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:08.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.330+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcf3404d010 con 0x7fcf4413f2d0 2023-09-27T23:34:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.530+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fcf4406ba60 con 0x7fcf4413f2d0 2023-09-27T23:34:08.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.686+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 66) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fcf3404d3e0 con 0x7fcf4413f2d0 2023-09-27T23:34:08.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.690+0000 7fcf2affd700 1 -- 172.21.15.64:0/443640642 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v66) v1 ==== 92+0+0 (secure 0 0 0) 0x7fcf34050cd0 con 0x7fcf4413f2d0 2023-09-27T23:34:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf30060a70 msgr2=0x7fcf30062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf30060a70 0x7fcf30062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009db0 tx=0x7fcf380091a0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 msgr2=0x7fcf440c0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 0x7fcf440c0ae0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c910 tx=0x7fcf340016d0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 shutdown_connections 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fcf30060a70 0x7fcf30062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4413f6a0 0x7fcf441554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf4413f2d0 0x7fcf440c0ae0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 --2- 172.21.15.64:0/443640642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf440c7780 0x7fcf44135f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 >> 172.21.15.64:0/443640642 conn(0x7fcf44067b50 msgr2=0x7fcf440c5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 shutdown_connections 2023-09-27T23:34:08.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.694+0000 7fcf43fff700 1 -- 172.21.15.64:0/443640642 wait complete. 2023-09-27T23:34:08.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:34:08.812 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:08.810+0000 7ff4e61c8000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:34:08.813 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:08.812+0000 7fbcb7da6000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:34:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b8db4700 1 Processor -- start 2023-09-27T23:34:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b8db4700 1 -- start start 2023-09-27T23:34:08.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b8db4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79b4150200 0x7f79b40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b8db4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40af250 con 0x7f79b40ae770 2023-09-27T23:34:08.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b8db4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40ca8a0 con 0x7f79b40ace30 2023-09-27T23:34:08.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b8db4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b40af530 con 0x7f79b4150200 2023-09-27T23:34:08.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b2ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f79b40ae770 0x7f79b40ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49344/0 (socket says 172.21.15.64:49344) 2023-09-27T23:34:08.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b2ffd700 1 -- 172.21.15.64:0/1691764284 learned_addr learned my addr 172.21.15.64:0/1691764284 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:08.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b37fe700 1 --2- 172.21.15.64:0/1691764284 >> v2:172.21.15.123:3300/0 conn(0x7f79b4150200 0x7f79b40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b27fc700 1 -- 172.21.15.64:0/1691764284 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048649660 0 0) 0x7f79b40af250 con 0x7f79b40ae770 2023-09-27T23:34:08.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b27fc700 1 -- 172.21.15.64:0/1691764284 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003570 con 0x7f79b40ae770 2023-09-27T23:34:08.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b37fe700 1 -- 172.21.15.64:0/1691764284 >> v1:172.21.15.123:6789/0 conn(0x7f79b40ace30 legacy=0x7f79b40cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b37fe700 1 -- 172.21.15.64:0/1691764284 >> v1:172.21.15.64:6789/0 conn(0x7f79b40ae770 legacy=0x7f79b40ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b37fe700 1 -- 172.21.15.64:0/1691764284 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b40cab80 con 0x7f79b4150200 2023-09-27T23:34:08.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.818+0000 7f79b37fe700 1 --2- 172.21.15.64:0/1691764284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b4150200 0x7f79b40c7170 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f79a40098e0 tx=0x7f79a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70c7c5dcd43dcb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:08.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b27fc700 1 -- 172.21.15.64:0/1691764284 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400b640 con 0x7f79b4150200 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b27fc700 1 -- 172.21.15.64:0/1691764284 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400b7a0 con 0x7f79b4150200 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b27fc700 1 -- 172.21.15.64:0/1691764284 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400bb40 con 0x7f79b4150200 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- 172.21.15.64:0/1691764284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b4150200 msgr2=0x7f79b40c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 --2- 172.21.15.64:0/1691764284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b4150200 0x7f79b40c7170 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f79a40098e0 tx=0x7f79a40091a0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- 172.21.15.64:0/1691764284 shutdown_connections 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 --2- 172.21.15.64:0/1691764284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b4150200 0x7f79b40c7170 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- 172.21.15.64:0/1691764284 >> 172.21.15.64:0/1691764284 conn(0x7f79b40ccb30 msgr2=0x7f79b40ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- 172.21.15.64:0/1691764284 shutdown_connections 2023-09-27T23:34:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- 172.21.15.64:0/1691764284 wait complete. 2023-09-27T23:34:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 Processor -- start 2023-09-27T23:34:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- start start 2023-09-27T23:34:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b40ace30 0x7f79b414cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b40ae770 0x7f79b40b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79b4150200 0x7f79b406d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b416d890 con 0x7f79b40ae770 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b416cc80 con 0x7f79b40ace30 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b8db4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b416cf60 con 0x7f79b4150200 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b40ae770 0x7f79b40b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b40ace30 0x7f79b414cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b40ae770 0x7f79b40b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46036/0 (socket says 172.21.15.64:46036) 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b2ffd700 1 -- 172.21.15.64:0/3437332003 learned_addr learned my addr 172.21.15.64:0/3437332003 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b3fff700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79b4150200 0x7f79b406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.822+0000 7f79b37fe700 1 -- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79b4150200 msgr2=0x7f79b406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b37fe700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79b4150200 0x7f79b406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b37fe700 1 -- 172.21.15.64:0/3437332003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b40ae770 msgr2=0x7f79b40b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b37fe700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b40ae770 0x7f79b40b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b37fe700 1 -- 172.21.15.64:0/3437332003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4009420 con 0x7f79b40ace30 2023-09-27T23:34:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b37fe700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b40ace30 0x7f79b414cbe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f79a4016000 tx=0x7f79a4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4000d10 con 0x7f79b40ace30 2023-09-27T23:34:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400bb20 con 0x7f79b40ace30 2023-09-27T23:34:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b416d240 con 0x7f79b40ace30 2023-09-27T23:34:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4011650 con 0x7f79b40ace30 2023-09-27T23:34:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b414d120 con 0x7f79b40ace30 2023-09-27T23:34:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 66) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f79a40119c0 con 0x7f79b40ace30 2023-09-27T23:34:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.826+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b416d0a0 con 0x7f79b40ace30 2023-09-27T23:34:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.830+0000 7f799b7fe700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79a0060ac0 0x7f79a0062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:08.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.830+0000 7f79b2ffd700 1 -- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79a0060ac0 msgr2=0x7f79a0062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:34:08.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.830+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f79a407e5f0 con 0x7f79b40ace30 2023-09-27T23:34:08.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.830+0000 7f79b2ffd700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79a0060ac0 0x7f79a0062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:34:08.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.838+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f79b416d0a0 con 0x7f79b40ace30 2023-09-27T23:34:09.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:08.998+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f79b416d0a0 con 0x7f79b40ace30 2023-09-27T23:34:09.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.010+0000 7f799b7fe700 1 -- 172.21.15.64:0/3437332003 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f79b416d0a0 con 0x7f79b40ace30 2023-09-27T23:34:09.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79a0060ac0 msgr2=0x7f79a0062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:34:09.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79a0060ac0 0x7f79a0062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b40ace30 msgr2=0x7f79b414cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b40ace30 0x7f79b414cbe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f79a4016000 tx=0x7f79a4017000 comp rx=0 tx=0).stop 2023-09-27T23:34:09.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 shutdown_connections 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f79a0060ac0 0x7f79a0062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79b4150200 0x7f79b406d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79b40ae770 0x7f79b40b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 --2- 172.21.15.64:0/3437332003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b40ace30 0x7f79b414cbe0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 >> 172.21.15.64:0/3437332003 conn(0x7f79b40ccb30 msgr2=0x7f79b40c2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 shutdown_connections 2023-09-27T23:34:09.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.014+0000 7f79b8db4700 1 -- 172.21.15.64:0/3437332003 wait complete. 2023-09-27T23:34:09.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:34:09.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfed00700 1 Processor -- start 2023-09-27T23:34:09.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfed00700 1 -- start start 2023-09-27T23:34:09.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfed00700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cf8068c50 0x7f1cf80b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfed00700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf80b15e0 con 0x7f1cf80b1f00 2023-09-27T23:34:09.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfed00700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf80b3c40 con 0x7f1cf80b03f0 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfed00700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf815a5d0 con 0x7f1cf8068c50 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfd29d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1cf80b1f00 0x7f1cf80670e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49348/0 (socket says 172.21.15.64:49348) 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfd29d700 1 -- 172.21.15.64:0/1729395668 learned_addr learned my addr 172.21.15.64:0/1729395668 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cfca9c700 1 --2- 172.21.15.64:0/1729395668 >> v2:172.21.15.123:3300/0 conn(0x7f1cf8068c50 0x7f1cf80b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.138+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444487582 0 0) 0x7f1cf80b15e0 con 0x7f1cf80b1f00 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce8003570 con 0x7f1cf80b1f00 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3920808791 0 0) 0x7f1ce8003570 con 0x7f1cf80b1f00 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 >> v2:172.21.15.123:3300/0 conn(0x7f1cf8068c50 msgr2=0x7f1cf80b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 --2- 172.21.15.64:0/1729395668 >> v2:172.21.15.123:3300/0 conn(0x7f1cf8068c50 0x7f1cf80b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 >> v1:172.21.15.123:6789/0 conn(0x7f1cf80b03f0 legacy=0x7f1cf80b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8159810 con 0x7f1cf80b1f00 2023-09-27T23:34:09.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f1cf0001810 con 0x7f1cf80b1f00 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 >> v1:172.21.15.64:6789/0 conn(0x7f1cf80b1f00 legacy=0x7f1cf80670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce8003a60 0x7f1ce8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce8004810 0x7f1ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce80051c0 0x7f1ce800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf815a5d0 con 0x7f1ce8004810 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce800c0d0 con 0x7f1ce8003a60 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce800c290 con 0x7f1ce80051c0 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce8004810 0x7f1ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfca9c700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce8003a60 0x7f1ce8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfd29d700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce80051c0 0x7f1ce800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 -- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce80051c0 msgr2=0x7f1ce800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce80051c0 0x7f1ce800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 -- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce8003a60 msgr2=0x7f1ce8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:09.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce8003a60 0x7f1ce8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 -- 172.21.15.64:0/1729395668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8003570 con 0x7f1ce8004810 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfd29d700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce80051c0 0x7f1ce800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfd29d700 1 -- 172.21.15.64:0/1729395668 reap_dead start 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce8004810 0x7f1ce8004c80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40018b0 tx=0x7f1ce4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6b4236cfe5b3dd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4012420 con 0x7f1ce8004810 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40125a0 con 0x7f1ce8004810 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cf77fe700 1 -- 172.21.15.64:0/1729395668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4012960 con 0x7f1ce8004810 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 -- 172.21.15.64:0/1729395668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce8004810 msgr2=0x7f1ce8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce8004810 0x7f1ce8004c80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40018b0 tx=0x7f1ce4007d30 comp rx=0 tx=0).stop 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 -- 172.21.15.64:0/1729395668 shutdown_connections 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 --2- 172.21.15.64:0/1729395668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce8004810 0x7f1ce8004c80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 -- 172.21.15.64:0/1729395668 >> 172.21.15.64:0/1729395668 conn(0x7f1cf806dce0 msgr2=0x7f1cf80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 -- 172.21.15.64:0/1729395668 shutdown_connections 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.142+0000 7f1cfed00700 1 -- 172.21.15.64:0/1729395668 wait complete. 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 Processor -- start 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 -- start start 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce8004810 0x7f1cf80acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 0x7f1cf8169620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cf80c8b60 0x7f1cf80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf816ac30 con 0x7f1cf81691b0 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80c9900 con 0x7f1cf80c8b60 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf816a5e0 con 0x7f1ce8004810 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 0x7f1cf8169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfd29d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cf80c8b60 0x7f1cf80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 0x7f1cf8169620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46058/0 (socket says 172.21.15.64:46058) 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 -- 172.21.15.64:0/3554125708 learned_addr learned my addr 172.21.15.64:0/3554125708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:09.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfca9c700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce8004810 0x7f1cf80acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 -- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce8004810 msgr2=0x7f1cf80acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce8004810 0x7f1cf80acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 -- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cf80c8b60 msgr2=0x7f1cf80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cf80c8b60 0x7f1cf80c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 -- 172.21.15.64:0/3554125708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf816a8c0 con 0x7f1cf81691b0 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfd29d700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cf80c8b60 0x7f1cf80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf7fff700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 0x7f1cf8169620 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c3f0 tx=0x7f1ce4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4001200 con 0x7f1cf81691b0 2023-09-27T23:34:09.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4012710 con 0x7f1cf81691b0 2023-09-27T23:34:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf8169b60 con 0x7f1cf81691b0 2023-09-27T23:34:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce401f410 con 0x7f1cf81691b0 2023-09-27T23:34:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.146+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf80c94f0 con 0x7f1cf81691b0 2023-09-27T23:34:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.150+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf80cd930 con 0x7f1cf81691b0 2023-09-27T23:34:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.150+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 66) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f1ce401f7a0 con 0x7f1cf81691b0 2023-09-27T23:34:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.150+0000 7f1cf4ff9700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cdc060af0 0x7f1cdc062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.150+0000 7f1cfca9c700 1 -- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cdc060af0 msgr2=0x7f1cdc062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:34:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.150+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1ce407edf0 con 0x7f1cf81691b0 2023-09-27T23:34:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.150+0000 7f1cfca9c700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cdc060af0 0x7f1cdc062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:34:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.158+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1cf80cd930 con 0x7f1cf81691b0 2023-09-27T23:34:09.235 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.232+0000 7fbcb7da6000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:34:09.264 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.258+0000 7ff4e61c8000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:34:09.283 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.280+0000 7fbcb7da6000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:34:09.317 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.314+0000 7ff4e61c8000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:34:09.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.314+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f1cf80cd930 con 0x7f1cf81691b0 2023-09-27T23:34:09.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cf4ff9700 1 -- 172.21.15.64:0/3554125708 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1cf80cd930 con 0x7f1cf81691b0 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cdc060af0 msgr2=0x7f1cdc062fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cdc060af0 0x7f1cdc062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 msgr2=0x7f1cf8169620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 0x7f1cf8169620 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c3f0 tx=0x7f1ce4015000 comp rx=0 tx=0).stop 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 shutdown_connections 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f1cdc060af0 0x7f1cdc062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cf80c8b60 0x7f1cf80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf81691b0 0x7f1cf8169620 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 --2- 172.21.15.64:0/3554125708 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ce8004810 0x7f1cf80acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 >> 172.21.15.64:0/3554125708 conn(0x7f1cf806dce0 msgr2=0x7f1cf80cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:09.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 shutdown_connections 2023-09-27T23:34:09.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.322+0000 7f1cfed00700 1 -- 172.21.15.64:0/3554125708 wait complete. 2023-09-27T23:34:09.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:09.339 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:34:09.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:34:09.353 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-09-27T23:34:09.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-09-27T23:34:09.390 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.388+0000 7fbcb7da6000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:34:09.431 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.426+0000 7ff4e61c8000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:34:09.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.522+0000 7fefaa911700 1 Processor -- start 2023-09-27T23:34:09.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.522+0000 7fefaa911700 1 -- start start 2023-09-27T23:34:09.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fefa41558d0 0x7fefa41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefa40abe10 0x7fefa40ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa40ac2b0 con 0x7fefa40adc80 2023-09-27T23:34:09.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa4156fa0 con 0x7fefa40abe10 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa40abc60 con 0x7fefa41558d0 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefa40abe10 0x7fefa40ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fefa40abe10 0x7fefa40ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46068/0 (socket says 172.21.15.64:46068) 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 -- 172.21.15.64:0/2484244638 learned_addr learned my addr 172.21.15.64:0/2484244638 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa990f700 1 --2- 172.21.15.64:0/2484244638 >> v2:172.21.15.123:3300/0 conn(0x7fefa41558d0 0x7fefa41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 -- 172.21.15.64:0/2484244638 >> v2:172.21.15.123:3300/0 conn(0x7fefa41558d0 msgr2=0x7fefa41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 --2- 172.21.15.64:0/2484244638 >> v2:172.21.15.123:3300/0 conn(0x7fefa41558d0 0x7fefa41569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 -- 172.21.15.64:0/2484244638 >> v1:172.21.15.123:6789/0 conn(0x7fefa40adc80 legacy=0x7fefa40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 -- 172.21.15.64:0/2484244638 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40cbfb0 con 0x7fefa40abe10 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa990f700 1 --2- 172.21.15.64:0/2484244638 >> v2:172.21.15.123:3300/0 conn(0x7fefa41558d0 0x7fefa41569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa910e700 1 --2- 172.21.15.64:0/2484244638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40abe10 0x7fefa40ad740 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fefa40abdc0 tx=0x7fef9400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1dac6303cb3038 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa890d700 1 -- 172.21.15.64:0/2484244638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400dc50 con 0x7fefa40abe10 2023-09-27T23:34:09.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa890d700 1 -- 172.21.15.64:0/2484244638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400ddb0 con 0x7fefa40abe10 2023-09-27T23:34:09.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefa890d700 1 -- 172.21.15.64:0/2484244638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef940196d0 con 0x7fefa40abe10 2023-09-27T23:34:09.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- 172.21.15.64:0/2484244638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40abe10 msgr2=0x7fefa40ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 --2- 172.21.15.64:0/2484244638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40abe10 0x7fefa40ad740 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fefa40abdc0 tx=0x7fef9400b4f0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- 172.21.15.64:0/2484244638 shutdown_connections 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 --2- 172.21.15.64:0/2484244638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa40abe10 0x7fefa40ad740 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 --2- 172.21.15.64:0/2484244638 >> v2:172.21.15.123:3300/0 conn(0x7fefa41558d0 0x7fefa41569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- 172.21.15.64:0/2484244638 >> 172.21.15.64:0/2484244638 conn(0x7fefa406b5b0 msgr2=0x7fefa406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- 172.21.15.64:0/2484244638 shutdown_connections 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.526+0000 7fefaa911700 1 -- 172.21.15.64:0/2484244638 wait complete. 2023-09-27T23:34:09.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 Processor -- start 2023-09-27T23:34:09.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 -- start start 2023-09-27T23:34:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40abe10 0x7fefa41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40adc80 0x7fefa40c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 0x7fefa4157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa41650e0 con 0x7fefa41558d0 2023-09-27T23:34:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40c73d0 con 0x7fefa40abe10 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa41652e0 con 0x7fefa40adc80 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 0x7fefa4157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 0x7fefa4157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46072/0 (socket says 172.21.15.64:46072) 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 -- 172.21.15.64:0/1829110224 learned_addr learned my addr 172.21.15.64:0/1829110224 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40abe10 0x7fefa41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefa910e700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40adc80 0x7fefa40c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40adc80 msgr2=0x7fefa40c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40adc80 0x7fefa40c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40abe10 msgr2=0x7fefa41541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40abe10 0x7fefa41541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 -- 172.21.15.64:0/1829110224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9400b790 con 0x7fefa41558d0 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa110700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 0x7fefa4157d30 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c8e0 tx=0x7fef9c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c012630 con 0x7fefa41558d0 2023-09-27T23:34:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c012790 con 0x7fefa41558d0 2023-09-27T23:34:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa40c6b30 con 0x7fefa41558d0 2023-09-27T23:34:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.530+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c012b50 con 0x7fefa41558d0 2023-09-27T23:34:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa40c70f0 con 0x7fefa41558d0 2023-09-27T23:34:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 66) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fef9c00a3b0 con 0x7fefa41558d0 2023-09-27T23:34:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa404be40 con 0x7fefa41558d0 2023-09-27T23:34:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fef99ffb700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fef9c07d500 con 0x7fefa41558d0 2023-09-27T23:34:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fefa990f700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 msgr2=0x7fef88062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:34:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.534+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:34:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.542+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fef9c04d530 con 0x7fefa41558d0 2023-09-27T23:34:09.584 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.584+0000 7fbcb7da6000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:34:09.632 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.632+0000 7fbcb7da6000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:34:09.655 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.650+0000 7ff4e61c8000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:34:09.707 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.702+0000 7ff4e61c8000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:34:09.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.718+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fefa40c3820 con 0x7fef88060ac0 2023-09-27T23:34:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.738+0000 7fefa990f700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 msgr2=0x7fef88062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:34:09.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:09.738+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:34:09.786 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.784+0000 7fbcb7da6000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:34:09.842 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.840+0000 7fbcb7da6000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:34:09.877 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.874+0000 7ff4e61c8000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:34:09.900 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:09.900+0000 7fbcb7da6000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:34:09.940 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:09.934+0000 7ff4e61c8000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:34:10.004 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:10.002+0000 7ff4e61c8000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:34:10.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:10.142+0000 7fefa990f700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 msgr2=0x7fef88062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:34:10.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:10.142+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:34:10.195 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.192+0000 7fbcb7da6000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:34:10.307 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.304+0000 7fbcb7da6000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:34:10.341 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:10.338+0000 7ff4e61c8000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:34:10.459 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:10.454+0000 7ff4e61c8000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:34:10.587 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.584+0000 7fbcb7da6000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:34:10.748 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.748+0000 7fbcb7da6000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:34:10.751 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:10.746+0000 7ff4e61c8000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:34:10.813 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.812+0000 7fbcb7da6000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:34:10.864 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.864+0000 7fbcb7da6000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:34:10.918 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:10.916+0000 7fbcb7da6000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:34:10.920 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:10.914+0000 7ff4e61c8000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:34:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:10.946+0000 7fefa990f700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 msgr2=0x7fef88062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/33228 2023-09-27T23:34:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:10.946+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:34:10.987 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:10.982+0000 7ff4e61c8000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:34:11.041 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.038+0000 7ff4e61c8000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:34:11.091 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:11.088+0000 7fbcb7da6000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:34:11.099 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.094+0000 7ff4e61c8000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:34:11.145 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:11.144+0000 7fbcb7da6000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:34:11.208 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:11.208+0000 7fbcb7da6000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:34:11.276 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.274+0000 7ff4e61c8000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:34:11.333 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.330+0000 7ff4e61c8000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:34:11.358 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:11.356+0000 7fbcb7da6000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:34:11.400 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.394+0000 7ff4e61c8000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:34:11.421 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:11.420+0000 7fbcb7da6000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:34:11.557 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.554+0000 7ff4e61c8000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:34:11.620 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:11.614+0000 7ff4e61c8000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:34:12.010 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.008+0000 7fbcb7da6000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:34:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:12.042+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 67) v1 ==== 79987+0+0 (secure 0 0 0) 0x7fef9c04d900 con 0x7fefa41558d0 2023-09-27T23:34:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:12.042+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 msgr2=0x7fef88062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:34:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:12.042+0000 7fef99ffb700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:12.202 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc7af56700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.202 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc7af56700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.202 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc7af56700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.202 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc7af56700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.202 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc7af56700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.202 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc78751700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.203 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc78751700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.203 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc78751700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.203 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc78751700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.203 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:34:12.200+0000 7fbc78751700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:34:12.256 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:34:12.254+0000 7ff4e61c8000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:34:13.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.046+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 68) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fef9c07b740 con 0x7fefa41558d0 2023-09-27T23:34:13.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.050+0000 7fef99ffb700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.050+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fefa40c3820 con 0x7fef88060ac0 2023-09-27T23:34:13.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.050+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.054+0000 7fefa990f700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefa0006f00 tx=0x7fefa0008040 comp rx=0 tx=0).ready entity=mgr.8057 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:13.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.178+0000 7fef99ffb700 1 -- 172.21.15.64:0/1829110224 <== mgr.8057 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fefa40c3820 con 0x7fef88060ac0 2023-09-27T23:34:13.180 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-09-27T23:34:13.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.178+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 msgr2=0x7fef88062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.178+0000 7fefaa911700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fefa0006f00 tx=0x7fefa0008040 comp rx=0 tx=0).stop 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.178+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 msgr2=0x7fefa4157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.178+0000 7fefaa911700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 0x7fefa4157d30 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c8e0 tx=0x7fef9c001710 comp rx=0 tx=0).stop 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 shutdown_connections 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7fef88060ac0 0x7fef88062ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41558d0 0x7fefa4157d30 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fefa40adc80 0x7fefa40c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 --2- 172.21.15.64:0/1829110224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefa40abe10 0x7fefa41541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 >> 172.21.15.64:0/1829110224 conn(0x7fefa406b5b0 msgr2=0x7fefa4154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 shutdown_connections 2023-09-27T23:34:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.182+0000 7fefaa911700 1 -- 172.21.15.64:0/1829110224 wait complete. 2023-09-27T23:34:13.202 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-09-27T23:34:13.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-09-27T23:34:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 Processor -- start 2023-09-27T23:34:13.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 -- start start 2023-09-27T23:34:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f928815a0a0 0x7f928815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92880c6cf0 0x7f92880c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9288068300 0x7f9288068780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9288153870 con 0x7f9288068300 2023-09-27T23:34:13.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9288068cc0 con 0x7f928815a0a0 2023-09-27T23:34:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f928cd6b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928806d4d0 con 0x7f92880c6cf0 2023-09-27T23:34:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9288068300 0x7f9288068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9288068300 0x7f9288068780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46082/0 (socket says 172.21.15.64:46082) 2023-09-27T23:34:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 -- 172.21.15.64:0/1363886063 learned_addr learned my addr 172.21.15.64:0/1363886063 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f92877fe700 1 --2- 172.21.15.64:0/1363886063 >> v2:172.21.15.123:3300/0 conn(0x7f928815a0a0 0x7f928815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f927effd700 1 --2- 172.21.15.64:0/1363886063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92880c6cf0 0x7f92880c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 -- 172.21.15.64:0/1363886063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92880c6cf0 msgr2=0x7f92880c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 --2- 172.21.15.64:0/1363886063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92880c6cf0 0x7f92880c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 -- 172.21.15.64:0/1363886063 >> v2:172.21.15.123:3300/0 conn(0x7f928815a0a0 msgr2=0x7f928815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 --2- 172.21.15.64:0/1363886063 >> v2:172.21.15.123:3300/0 conn(0x7f928815a0a0 0x7f928815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.322+0000 7f9287fff700 1 -- 172.21.15.64:0/1363886063 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288159630 con 0x7f9288068300 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f9287fff700 1 --2- 172.21.15.64:0/1363886063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288068300 0x7f9288068780 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f928806aad0 tx=0x7f9280009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cceb1c318b7d2cf server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f9286ffd700 1 -- 172.21.15.64:0/1363886063 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000e070 con 0x7f9288068300 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f9286ffd700 1 -- 172.21.15.64:0/1363886063 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928000b930 con 0x7f9288068300 2023-09-27T23:34:13.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f9286ffd700 1 -- 172.21.15.64:0/1363886063 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92800092e0 con 0x7f9288068300 2023-09-27T23:34:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 -- 172.21.15.64:0/1363886063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288068300 msgr2=0x7f9288068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 --2- 172.21.15.64:0/1363886063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288068300 0x7f9288068780 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f928806aad0 tx=0x7f9280009840 comp rx=0 tx=0).stop 2023-09-27T23:34:13.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 -- 172.21.15.64:0/1363886063 shutdown_connections 2023-09-27T23:34:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 --2- 172.21.15.64:0/1363886063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9288068300 0x7f9288068780 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 --2- 172.21.15.64:0/1363886063 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92880c6cf0 0x7f92880c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 --2- 172.21.15.64:0/1363886063 >> v2:172.21.15.123:3300/0 conn(0x7f928815a0a0 0x7f928815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 -- 172.21.15.64:0/1363886063 >> 172.21.15.64:0/1363886063 conn(0x7f92880c8c50 msgr2=0x7f92880c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 -- 172.21.15.64:0/1363886063 shutdown_connections 2023-09-27T23:34:13.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 -- 172.21.15.64:0/1363886063 wait complete. 2023-09-27T23:34:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 Processor -- start 2023-09-27T23:34:13.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.326+0000 7f928cd6b700 1 -- start start 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9288068300 0x7f92880abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92880c6cf0 0x7f928815e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 0x7f92880ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928815e940 con 0x7f928815a0a0 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880afb00 con 0x7f92880c6cf0 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928815eb30 con 0x7f9288068300 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 0x7f92880ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 0x7f92880ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46094/0 (socket says 172.21.15.64:46094) 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 -- 172.21.15.64:0/627932340 learned_addr learned my addr 172.21.15.64:0/627932340 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f927effd700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92880c6cf0 0x7f928815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f92877fe700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9288068300 0x7f92880abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 -- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9288068300 msgr2=0x7f92880abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9288068300 0x7f92880abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 -- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92880c6cf0 msgr2=0x7f928815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92880c6cf0 0x7f928815e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 -- 172.21.15.64:0/627932340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9280009ae0 con 0x7f928815a0a0 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f92877fe700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9288068300 0x7f92880abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:34:13.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f9287fff700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 0x7f92880ad5b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f9288064070 tx=0x7f9280018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92800032f0 con 0x7f928815a0a0 2023-09-27T23:34:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9280008b80 con 0x7f928815a0a0 2023-09-27T23:34:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928815ee10 con 0x7f928815a0a0 2023-09-27T23:34:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928001c410 con 0x7f928815a0a0 2023-09-27T23:34:13.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.330+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928815f3d0 con 0x7f928815a0a0 2023-09-27T23:34:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.334+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92880afc70 con 0x7f928815a0a0 2023-09-27T23:34:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.334+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 68) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f9280003040 con 0x7f928815a0a0 2023-09-27T23:34:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.334+0000 7f927ffff700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9268060820 0x7f9268062cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:34:13.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.334+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f928007e5c0 con 0x7f928815a0a0 2023-09-27T23:34:13.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.334+0000 7f92877fe700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9268060820 0x7f9268062cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:34:13.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.334+0000 7f92877fe700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9268060820 0x7f9268062cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9278009d80 tx=0x7f9278008040 comp rx=0 tx=0).ready entity=mgr.8057 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:34:13.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.342+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f92880afc70 con 0x7f928815a0a0 2023-09-27T23:34:13.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.522+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f92880afc70 con 0x7f9268060820 2023-09-27T23:34:13.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f927ffff700 1 -- 172.21.15.64:0/627932340 <== mgr.8057 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f92880afc70 con 0x7f9268060820 2023-09-27T23:34:13.636 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9268060820 msgr2=0x7f9268062cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9268060820 0x7f9268062cd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9278009d80 tx=0x7f9278008040 comp rx=0 tx=0).stop 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 msgr2=0x7f92880ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 0x7f92880ad5b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f9288064070 tx=0x7f9280018000 comp rx=0 tx=0).stop 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 shutdown_connections 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f9268060820 0x7f9268062cd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f928815a0a0 0x7f92880ad5b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92880c6cf0 0x7f928815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 --2- 172.21.15.64:0/627932340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9288068300 0x7f92880abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:34:13.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 >> 172.21.15.64:0/627932340 conn(0x7f92880c8c50 msgr2=0x7f9288152860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:34:13.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 shutdown_connections 2023-09-27T23:34:13.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:34:13.634+0000 7f928cd6b700 1 -- 172.21.15.64:0/627932340 wait complete. 2023-09-27T23:34:13.655 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-09-27T23:34:20.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:34:20.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:34:50.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:34:50.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:35:00.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:35:00.655+0000 7fbc6a735700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:35:00.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:35:00.655+0000 7fbc6a735700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:35:00.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:35:00.655+0000 7fbc6a735700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:35:00.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:35:00.655+0000 7fbc6a735700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:35:00.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:35:00.655+0000 7fbc6a735700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:35:20.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:35:20.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:35:50.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:35:50.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:36:20.583 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:36:20.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:36:50.659 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:36:50.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:37:20.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:37:20.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:37:50.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:37:50.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:38:20.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:38:20.877 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:38:50.958 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:38:50.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:39:21.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:39:21.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:39:51.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:39:51.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:40:21.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:40:21.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:40:51.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:40:51.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:41:21.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:41:21.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:41:51.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:41:51.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:42:21.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:42:21.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:42:51.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:42:51.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:43:21.566 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:43:21.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:43:51.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:43:51.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:44:21.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:44:21.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:44:51.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:44:52.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:45:15.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:45:15.656 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-09-27T23:45:15.676 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_35_00_UTC 2023-09-27T23:45:15.676 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_36_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_37_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_38_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_39_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_40_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_41_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_42_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_43_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_44_00_UTC 2023-09-27T23:45:15.677 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_45_00_UTC 2023-09-27T23:45:15.678 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_35_00_UTC', 'scheduled-2023-09-27-23_36_00_UTC', 'scheduled-2023-09-27-23_37_00_UTC', 'scheduled-2023-09-27-23_38_00_UTC', 'scheduled-2023-09-27-23_39_00_UTC', 'scheduled-2023-09-27-23_40_00_UTC', 'scheduled-2023-09-27-23_41_00_UTC', 'scheduled-2023-09-27-23_42_00_UTC', 'scheduled-2023-09-27-23_43_00_UTC', 'scheduled-2023-09-27-23_44_00_UTC', 'scheduled-2023-09-27-23_45_00_UTC'] 2023-09-27T23:45:15.678 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-09-27T23:45:15.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 Processor -- start 2023-09-27T23:45:15.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 -- start start 2023-09-27T23:45:15.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94380ae720 0x7f94380acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f943806ced0 0x7f9438159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380c85d0 con 0x7f9438158ed0 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380aeb70 con 0x7f94380ae720 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943e78b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380ae290 con 0x7f943806ced0 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94380ae720 0x7f94380acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f94380ae720 0x7f94380acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42470/0 (socket says 172.21.15.64:42470) 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943d789700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9438158ed0 0x7f943806c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:41610/0 (socket says 172.21.15.64:41610) 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 -- 172.21.15.64:0/3165124403 learned_addr learned my addr 172.21.15.64:0/3165124403 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:45:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943df8a700 1 --2- 172.21.15.64:0/3165124403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f943806ced0 0x7f9438159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 -- 172.21.15.64:0/3165124403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f943806ced0 msgr2=0x7f9438159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 --2- 172.21.15.64:0/3165124403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f943806ced0 0x7f9438159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f942ffff700 1 -- 172.21.15.64:0/3165124403 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804768305 0 0) 0x7f94380c85d0 con 0x7f9438158ed0 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 -- 172.21.15.64:0/3165124403 >> v1:172.21.15.123:6789/0 conn(0x7f9438158ed0 legacy=0x7f943806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.797+0000 7f943cf88700 1 -- 172.21.15.64:0/3165124403 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380ad430 con 0x7f94380ae720 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943cf88700 1 --2- 172.21.15.64:0/3165124403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380acef0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f94280099b0 tx=0x7f9428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d2cfd15d4ab791 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f942ffff700 1 -- 172.21.15.64:0/3165124403 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428009ea0 con 0x7f94380ae720 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f942ffff700 1 -- 172.21.15.64:0/3165124403 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942800b860 con 0x7f94380ae720 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f942ffff700 1 -- 172.21.15.64:0/3165124403 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942800bc00 con 0x7f94380ae720 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 -- 172.21.15.64:0/3165124403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 msgr2=0x7f94380acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 --2- 172.21.15.64:0/3165124403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380acef0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f94280099b0 tx=0x7f9428009250 comp rx=0 tx=0).stop 2023-09-27T23:45:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 -- 172.21.15.64:0/3165124403 shutdown_connections 2023-09-27T23:45:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 --2- 172.21.15.64:0/3165124403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f943806ced0 0x7f9438159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 --2- 172.21.15.64:0/3165124403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380acef0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 -- 172.21.15.64:0/3165124403 >> 172.21.15.64:0/3165124403 conn(0x7f94380c8c00 msgr2=0x7f94380c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:45:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 -- 172.21.15.64:0/3165124403 shutdown_connections 2023-09-27T23:45:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 -- 172.21.15.64:0/3165124403 wait complete. 2023-09-27T23:45:15.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 Processor -- start 2023-09-27T23:45:15.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.801+0000 7f943e78b700 1 -- start start 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943806ced0 0x7f94380c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9438158ed0 0x7f94380c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438165ef0 con 0x7f94380ae720 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c1170 con 0x7f943806ced0 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c1450 con 0x7f9438158ed0 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42480/0 (socket says 172.21.15.64:42480) 2023-09-27T23:45:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 -- 172.21.15.64:0/3779837451 learned_addr learned my addr 172.21.15.64:0/3779837451 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:45:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943d789700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943806ced0 0x7f94380c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943df8a700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9438158ed0 0x7f94380c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 -- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9438158ed0 msgr2=0x7f94380c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9438158ed0 0x7f94380c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 -- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943806ced0 msgr2=0x7f94380c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943806ced0 0x7f94380c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 -- 172.21.15.64:0/3779837451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280094f0 con 0x7f94380ae720 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943cf88700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380c0af0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f943806a8c0 tx=0x7f9428017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428000d10 con 0x7f94380ae720 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942800bbe0 con 0x7f94380ae720 2023-09-27T23:45:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9438166220 con 0x7f94380ae720 2023-09-27T23:45:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011690 con 0x7f94380ae720 2023-09-27T23:45:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.805+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94381667e0 con 0x7f94380ae720 2023-09-27T23:45:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.809+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380b2120 con 0x7f94380ae720 2023-09-27T23:45:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.809+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 71) v1 ==== 80390+0+0 (secure 0 0 0) 0x7f9428011ac0 con 0x7f94380ae720 2023-09-27T23:45:15.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.809+0000 7f942d7fa700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f941c060af0 0x7f941c062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:15.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.809+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6751+0+0 (secure 0 0 0) 0x7f942807eb90 con 0x7f94380ae720 2023-09-27T23:45:15.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.809+0000 7f943d789700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f941c060af0 0x7f941c062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:15.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.809+0000 7f943d789700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f941c060af0 0x7f941c062fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f94380c82c0 tx=0x7f9434007ef0 comp rx=0 tx=0).ready entity=mgr.8057 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:45:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:15.817+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f94380b2120 con 0x7f94380ae720 2023-09-27T23:45:16.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.009+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 --> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9438068360 con 0x7f941c060af0 2023-09-27T23:45:16.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.025+0000 7f942d7fa700 1 -- 172.21.15.64:0/3779837451 <== mgr.8057 v2:172.21.15.123:6832/33228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9438068360 con 0x7f941c060af0 2023-09-27T23:45:16.030 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:45:16.030 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:34:13", "first": "2023-09-27T23:35:00", "last": "2023-09-27T23:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-09-27T23:45:16.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f941c060af0 msgr2=0x7f941c062fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f941c060af0 0x7f941c062fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f94380c82c0 tx=0x7f9434007ef0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 msgr2=0x7f94380c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380c0af0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f943806a8c0 tx=0x7f9428017000 comp rx=0 tx=0).stop 2023-09-27T23:45:16.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 shutdown_connections 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f941c060af0 0x7f941c062fa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9438158ed0 0x7f94380c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94380ae720 0x7f94380c0af0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 --2- 172.21.15.64:0/3779837451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f943806ced0 0x7f94380c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 >> 172.21.15.64:0/3779837451 conn(0x7f94380c8c00 msgr2=0x7f94380c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 shutdown_connections 2023-09-27T23:45:16.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.029+0000 7f943e78b700 1 -- 172.21.15.64:0/3779837451 wait complete. 2023-09-27T23:45:16.046 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:34:13", "first": "2023-09-27T23:35:00", "last": "2023-09-27T23:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-09-27T23:45:16.047 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:34:13', 'first': '2023-09-27T23:35:00', 'last': '2023-09-27T23:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-09-27T23:45:16.047 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-27T23:45:16.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 Processor -- start 2023-09-27T23:45:16.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- start start 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3664140710 0x7f36640c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3664140ae0 0x7f36640680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366406e150 con 0x7f36640c75e0 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366406d540 con 0x7f3664140ae0 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366406d820 con 0x7f3664140710 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3664140ae0 0x7f36640680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:16.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3664140ae0 0x7f36640680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42492/0 (socket says 172.21.15.64:42492) 2023-09-27T23:45:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 -- 172.21.15.64:0/666227439 learned_addr learned my addr 172.21.15.64:0/666227439 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:45:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f366259c700 1 --2- 172.21.15.64:0/666227439 >> v2:172.21.15.123:3300/0 conn(0x7f3664140710 0x7f36640c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 -- 172.21.15.64:0/666227439 >> v2:172.21.15.123:3300/0 conn(0x7f3664140710 msgr2=0x7f36640c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:45:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 --2- 172.21.15.64:0/666227439 >> v2:172.21.15.123:3300/0 conn(0x7f3664140710 0x7f36640c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 -- 172.21.15.64:0/666227439 >> v1:172.21.15.123:6789/0 conn(0x7f36640c75e0 legacy=0x7f366406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 -- 172.21.15.64:0/666227439 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366406db00 con 0x7f3664140ae0 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3661d9b700 1 --2- 172.21.15.64:0/666227439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3664140ae0 0x7f36640680f0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3664068c30 tx=0x7f364c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac22b6910577b6ac server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f366159a700 1 -- 172.21.15.64:0/666227439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00b820 con 0x7f3664140ae0 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f366159a700 1 -- 172.21.15.64:0/666227439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c00b980 con 0x7f3664140ae0 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f366159a700 1 -- 172.21.15.64:0/666227439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00bd20 con 0x7f3664140ae0 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- 172.21.15.64:0/666227439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3664140ae0 msgr2=0x7f36640680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 --2- 172.21.15.64:0/666227439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3664140ae0 0x7f36640680f0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3664068c30 tx=0x7f364c009250 comp rx=0 tx=0).stop 2023-09-27T23:45:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- 172.21.15.64:0/666227439 shutdown_connections 2023-09-27T23:45:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 --2- 172.21.15.64:0/666227439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3664140ae0 0x7f36640680f0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 --2- 172.21.15.64:0/666227439 >> v2:172.21.15.123:3300/0 conn(0x7f3664140710 0x7f36640c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- 172.21.15.64:0/666227439 >> 172.21.15.64:0/666227439 conn(0x7f36640c8850 msgr2=0x7f36640c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:45:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- 172.21.15.64:0/666227439 shutdown_connections 2023-09-27T23:45:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.157+0000 7f3668dfc700 1 -- 172.21.15.64:0/666227439 wait complete. 2023-09-27T23:45:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 Processor -- start 2023-09-27T23:45:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 -- start start 2023-09-27T23:45:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 0x7f3664168350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3664140710 0x7f366416a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3664140ae0 0x7f36641598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640c4960 con 0x7f36640c75e0 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664157270 con 0x7f3664140710 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3668dfc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3664167e10 con 0x7f3664140ae0 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f366259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 0x7f3664168350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f366259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 0x7f3664168350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42494/0 (socket says 172.21.15.64:42494) 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f366259c700 1 -- 172.21.15.64:0/310770150 learned_addr learned my addr 172.21.15.64:0/310770150 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.161+0000 7f3661d9b700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3664140710 0x7f366416a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:16.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f3662d9d700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3664140ae0 0x7f36641598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:16.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f366259c700 1 -- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3664140ae0 msgr2=0x7f36641598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f366259c700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3664140ae0 0x7f36641598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f366259c700 1 -- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3664140710 msgr2=0x7f366416a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f366259c700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3664140710 0x7f366416a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f366259c700 1 -- 172.21.15.64:0/310770150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c0094f0 con 0x7f36640c75e0 2023-09-27T23:45:16.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f3662d9d700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3664140ae0 0x7f36641598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f366259c700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 0x7f3664168350 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f365400d890 tx=0x7f3654008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540125f0 con 0x7f36640c75e0 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654012750 con 0x7f36640c75e0 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36641680f0 con 0x7f36640c75e0 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654012ad0 con 0x7f36640c75e0 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640c50b0 con 0x7f36640c75e0 2023-09-27T23:45:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664071120 con 0x7f36640c75e0 2023-09-27T23:45:16.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.165+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 71) v1 ==== 80390+0+0 (secure 0 0 0) 0x7f365401e460 con 0x7f36640c75e0 2023-09-27T23:45:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.169+0000 7f365a7fc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3644060ac0 0x7f3644062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:45:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.169+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6751+0+0 (secure 0 0 0) 0x7f365407d9b0 con 0x7f36640c75e0 2023-09-27T23:45:16.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.169+0000 7f3661d9b700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3644060ac0 0x7f3644062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:45:16.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.169+0000 7f3661d9b700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3644060ac0 0x7f3644062f70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3664068c30 tx=0x7f364c016000 comp rx=0 tx=0).ready entity=mgr.8057 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:45:16.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.177+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f365404d870 con 0x7f36640c75e0 2023-09-27T23:45:16.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.377+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f36640642a0 con 0x7f36640c75e0 2023-09-27T23:45:16.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.385+0000 7f365a7fc700 1 -- 172.21.15.64:0/310770150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v71) v1 ==== 74+0+154867 (secure 0 0 0) 0x7f3654050e90 con 0x7f36640c75e0 2023-09-27T23:45:16.392 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3644060ac0 msgr2=0x7f3644062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3644060ac0 0x7f3644062f70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3664068c30 tx=0x7f364c016000 comp rx=0 tx=0).stop 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 msgr2=0x7f3664168350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 0x7f3664168350 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f365400d890 tx=0x7f3654008040 comp rx=0 tx=0).stop 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 shutdown_connections 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:6832/33228,v1:172.21.15.123:6833/33228] conn(0x7f3644060ac0 0x7f3644062f70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3664140ae0 0x7f36641598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3664140710 0x7f366416a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 --2- 172.21.15.64:0/310770150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36640c75e0 0x7f3664168350 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 >> 172.21.15.64:0/310770150 conn(0x7f36640c8850 msgr2=0x7f36640c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 shutdown_connections 2023-09-27T23:45:16.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:45:16.389+0000 7f3668dfc700 1 -- 172.21.15.64:0/310770150 wait complete. 2023-09-27T23:45:16.407 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":71,"active_gid":8057,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":33228},{"type":"v1","addr":"172.21.15.123:6833","nonce":33228}]},"active_addr":"172.21.15.123:6833/33228","active_change":"2023-09-27T23:34:12.030776+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":8054,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":158,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1316115001}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1077033381}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1602424936}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3448188613}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1684150586}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2431874217}]}]} 2023-09-27T23:45:16.411 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-09-27T23:45:16.412 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-27T23:45:16.412 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-09-27T23:45:16.412 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-27T23:45:16.412 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:45:16.443 INFO:tasks.ceph.mgr.x:Stopped 2023-09-27T23:45:16.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-27T23:45:16.445 INFO:tasks.ceph.mgr.x:Started 2023-09-27T23:45:16.604 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:16.605+0000 7f9d6d3ce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:45:17.028 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.029+0000 7f9d6d3ce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:45:17.078 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.077+0000 7f9d6d3ce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:45:17.182 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.181+0000 7f9d6d3ce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:45:17.375 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.373+0000 7f9d6d3ce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:45:17.423 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.421+0000 7f9d6d3ce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:45:17.575 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.573+0000 7f9d6d3ce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:45:17.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.629+0000 7f9d6d3ce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:45:17.689 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.689+0000 7f9d6d3ce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:45:17.987 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:17.985+0000 7f9d6d3ce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:45:18.098 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.097+0000 7f9d6d3ce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:45:18.379 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.377+0000 7f9d6d3ce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:45:18.538 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.537+0000 7f9d6d3ce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:45:18.602 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.601+0000 7f9d6d3ce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:45:18.655 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.653+0000 7f9d6d3ce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:45:18.711 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.709+0000 7f9d6d3ce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:45:18.886 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.885+0000 7f9d6d3ce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:45:18.941 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:18.941+0000 7f9d6d3ce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:45:19.002 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:19.001+0000 7f9d6d3ce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:45:19.151 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:19.149+0000 7f9d6d3ce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:45:19.210 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:19.209+0000 7f9d6d3ce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:45:19.798 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:19.797+0000 7f9d6d3ce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2fd7d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2fd7d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2fd7d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2fd7d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2fd7d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2cd77700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2cd77700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2cd77700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2cd77700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:20.067 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:45:20.065+0000 7f9d2cd77700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:45:22.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:45:22.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:45:52.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:45:52.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:46:00.062 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:46:00.060+0000 7f9d3858e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:46:00.063 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:46:00.060+0000 7f9d3858e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:46:00.063 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:46:00.060+0000 7f9d3858e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:46:00.063 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:46:00.060+0000 7f9d3858e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:46:00.063 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:46:00.060+0000 7f9d3858e700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:46:22.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:46:22.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:46:52.530 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:46:52.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:47:22.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:47:22.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:47:52.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:47:52.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:48:22.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:48:22.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:48:52.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:48:52.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:49:22.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:49:22.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:49:52.921 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:49:52.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:50:16.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-09-27T23:50:16.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.567+0000 7fd667fff700 1 Processor -- start 2023-09-27T23:50:16.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.567+0000 7fd667fff700 1 -- start start 2023-09-27T23:50:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd668156640 0x7fd668156a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd6680c8ae0 0x7fd668157210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66806c7c0 con 0x7fd668157750 2023-09-27T23:50:16.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66806ca10 con 0x7fd6680c8ae0 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66806ccf0 con 0x7fd668156640 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd6680c8ae0 0x7fd668157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd6680c8ae0 0x7fd668157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47726/0 (socket says 172.21.15.64:47726) 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6677fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd668157750 0x7fd66806d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37102/0 (socket says 172.21.15.64:37102) 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 -- 172.21.15.64:0/1597095764 learned_addr learned my addr 172.21.15.64:0/1597095764 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd666ffd700 1 --2- 172.21.15.64:0/1597095764 >> v2:172.21.15.123:3300/0 conn(0x7fd668156640 0x7fd668156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 -- 172.21.15.64:0/1597095764 >> v2:172.21.15.123:3300/0 conn(0x7fd668156640 msgr2=0x7fd668156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 --2- 172.21.15.64:0/1597095764 >> v2:172.21.15.123:3300/0 conn(0x7fd668156640 0x7fd668156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 -- 172.21.15.64:0/1597095764 >> v1:172.21.15.64:6789/0 conn(0x7fd668157750 legacy=0x7fd66806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 -- 172.21.15.64:0/1597095764 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66806cfd0 con 0x7fd6680c8ae0 2023-09-27T23:50:16.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd665ffb700 1 -- 172.21.15.64:0/1597095764 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522496986 0 0) 0x7fd66806c7c0 con 0x7fd668157750 2023-09-27T23:50:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd666ffd700 1 --2- 172.21.15.64:0/1597095764 >> v2:172.21.15.123:3300/0 conn(0x7fd668156640 0x7fd668156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd6667fc700 1 --2- 172.21.15.64:0/1597095764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6680c8ae0 0x7fd668157210 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd668068060 tx=0x7fd658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffd8644faff8801 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:16.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd665ffb700 1 -- 172.21.15.64:0/1597095764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800b860 con 0x7fd6680c8ae0 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd665ffb700 1 -- 172.21.15.64:0/1597095764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65800b9c0 con 0x7fd6680c8ae0 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd665ffb700 1 -- 172.21.15.64:0/1597095764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800bd60 con 0x7fd6680c8ae0 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- 172.21.15.64:0/1597095764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6680c8ae0 msgr2=0x7fd668157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 --2- 172.21.15.64:0/1597095764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6680c8ae0 0x7fd668157210 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd668068060 tx=0x7fd658009250 comp rx=0 tx=0).stop 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- 172.21.15.64:0/1597095764 shutdown_connections 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 --2- 172.21.15.64:0/1597095764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6680c8ae0 0x7fd668157210 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 --2- 172.21.15.64:0/1597095764 >> v2:172.21.15.123:3300/0 conn(0x7fd668156640 0x7fd668156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- 172.21.15.64:0/1597095764 >> 172.21.15.64:0/1597095764 conn(0x7fd668067bb0 msgr2=0x7fd6680c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- 172.21.15.64:0/1597095764 shutdown_connections 2023-09-27T23:50:16.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.571+0000 7fd667fff700 1 -- 172.21.15.64:0/1597095764 wait complete. 2023-09-27T23:50:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 Processor -- start 2023-09-27T23:50:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 -- start start 2023-09-27T23:50:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6680c8ae0 0x7fd66816b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 0x7fd668154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd668157750 0x7fd668153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681534c0 con 0x7fd668156640 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668155500 con 0x7fd6680c8ae0 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd667fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681557e0 con 0x7fd668157750 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 0x7fd668154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 0x7fd668154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47734/0 (socket says 172.21.15.64:47734) 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 -- 172.21.15.64:0/3768980352 learned_addr learned my addr 172.21.15.64:0/3768980352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6677fe700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd668157750 0x7fd668153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:16.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd666ffd700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6680c8ae0 0x7fd66816b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 -- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd668157750 msgr2=0x7fd668153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd668157750 0x7fd668153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 -- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6680c8ae0 msgr2=0x7fd66816b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6680c8ae0 0x7fd66816b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 -- 172.21.15.64:0/3768980352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580094f0 con 0x7fd668156640 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd6667fc700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 0x7fd668154cb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd6580090b0 tx=0x7fd658017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658026480 con 0x7fd668156640 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.575+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65801f070 con 0x7fd668156640 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd668152e70 con 0x7fd668156640 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658026830 con 0x7fd668156640 2023-09-27T23:50:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6681551f0 con 0x7fd668156640 2023-09-27T23:50:16.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668064040 con 0x7fd668156640 2023-09-27T23:50:16.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 76) v1 ==== 80390+0+0 (secure 0 0 0) 0x7fd65801a430 con 0x7fd668156640 2023-09-27T23:50:16.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd64effd700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd654060ac0 0x7fd654062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:16.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fd658021070 con 0x7fd668156640 2023-09-27T23:50:16.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.579+0000 7fd666ffd700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd654060ac0 0x7fd654062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:16.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.583+0000 7fd666ffd700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd654060ac0 0x7fd654062f70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006f00 tx=0x7fd65c008040 comp rx=0 tx=0).ready entity=mgr.9349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:16.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.583+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd65804ef90 con 0x7fd668156640 2023-09-27T23:50:16.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.767+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd66804be40 con 0x7fd654060ac0 2023-09-27T23:50:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.895+0000 7fd64effd700 1 -- 172.21.15.64:0/3768980352 <== mgr.9349 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fd66804be40 con 0x7fd654060ac0 2023-09-27T23:50:16.899 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-09-27T23:50:16.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd654060ac0 msgr2=0x7fd654062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:16.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd654060ac0 0x7fd654062f70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006f00 tx=0x7fd65c008040 comp rx=0 tx=0).stop 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 msgr2=0x7fd668154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 0x7fd668154cb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd6580090b0 tx=0x7fd658017000 comp rx=0 tx=0).stop 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 shutdown_connections 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd654060ac0 0x7fd654062f70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd668157750 0x7fd668153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668156640 0x7fd668154cb0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 --2- 172.21.15.64:0/3768980352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6680c8ae0 0x7fd66816b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 >> 172.21.15.64:0/3768980352 conn(0x7fd668067bb0 msgr2=0x7fd6680b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 shutdown_connections 2023-09-27T23:50:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:16.899+0000 7fd667fff700 1 -- 172.21.15.64:0/3768980352 wait complete. 2023-09-27T23:50:16.919 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-09-27T23:50:16.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:16.920 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-09-27T23:50:16.940 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_35_00_UTC 2023-09-27T23:50:16.940 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_36_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_37_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_38_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_39_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_40_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_41_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_42_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_43_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_44_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_45_00_UTC 2023-09-27T23:50:16.941 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_46_00_UTC 2023-09-27T23:50:16.942 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_47_00_UTC 2023-09-27T23:50:16.942 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_48_00_UTC 2023-09-27T23:50:16.942 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_49_00_UTC 2023-09-27T23:50:16.942 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_50_00_UTC 2023-09-27T23:50:16.943 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-09-27-23_35_00_UTC', 'scheduled-2023-09-27-23_36_00_UTC', 'scheduled-2023-09-27-23_37_00_UTC', 'scheduled-2023-09-27-23_38_00_UTC', 'scheduled-2023-09-27-23_39_00_UTC', 'scheduled-2023-09-27-23_40_00_UTC', 'scheduled-2023-09-27-23_41_00_UTC', 'scheduled-2023-09-27-23_42_00_UTC', 'scheduled-2023-09-27-23_43_00_UTC', 'scheduled-2023-09-27-23_44_00_UTC', 'scheduled-2023-09-27-23_45_00_UTC', 'scheduled-2023-09-27-23_46_00_UTC', 'scheduled-2023-09-27-23_47_00_UTC', 'scheduled-2023-09-27-23_48_00_UTC', 'scheduled-2023-09-27-23_49_00_UTC', 'scheduled-2023-09-27-23_50_00_UTC'] 2023-09-27T23:50:16.943 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-09-27T23:50:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 Processor -- start 2023-09-27T23:50:17.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- start start 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd04c0c8f30 0x7fd04c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c06c7c0 con 0x7fd04c156640 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c06ca10 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c06ccf0 con 0x7fd04c0c8f30 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052804700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd04c0c8f30 0x7fd04c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052003700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd04c156640 0x7fd04c156a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37116/0 (socket says 172.21.15.64:37116) 2023-09-27T23:50:17.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052003700 1 -- 172.21.15.64:0/1900343736 learned_addr learned my addr 172.21.15.64:0/1900343736 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:17.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052804700 1 -- 172.21.15.64:0/1900343736 >> v1:172.21.15.123:6789/0 conn(0x7fd04c0c8ae0 legacy=0x7fd04c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052804700 1 -- 172.21.15.64:0/1900343736 >> v1:172.21.15.64:6789/0 conn(0x7fd04c156640 legacy=0x7fd04c156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052804700 1 -- 172.21.15.64:0/1900343736 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c06cfd0 con 0x7fd04c0c8f30 2023-09-27T23:50:17.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd051001700 1 -- 172.21.15.64:0/1900343736 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478223397 0 0) 0x7fd04c06c7c0 con 0x7fd04c156640 2023-09-27T23:50:17.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd052804700 1 --2- 172.21.15.64:0/1900343736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8f30 0x7fd04c06d7d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd044009e60 tx=0x7fd044009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901a2caa0c6e467f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:17.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd051001700 1 -- 172.21.15.64:0/1900343736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04400e070 con 0x7fd04c0c8f30 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd051001700 1 -- 172.21.15.64:0/1900343736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04400bc30 con 0x7fd04c0c8f30 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd051001700 1 -- 172.21.15.64:0/1900343736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044012430 con 0x7fd04c0c8f30 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- 172.21.15.64:0/1900343736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8f30 msgr2=0x7fd04c06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 --2- 172.21.15.64:0/1900343736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8f30 0x7fd04c06d7d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd044009e60 tx=0x7fd044009780 comp rx=0 tx=0).stop 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- 172.21.15.64:0/1900343736 shutdown_connections 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 --2- 172.21.15.64:0/1900343736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8f30 0x7fd04c06d7d0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- 172.21.15.64:0/1900343736 >> 172.21.15.64:0/1900343736 conn(0x7fd04c067bb0 msgr2=0x7fd04c0c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- 172.21.15.64:0/1900343736 shutdown_connections 2023-09-27T23:50:17.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.063+0000 7fd053005700 1 -- 172.21.15.64:0/1900343736 wait complete. 2023-09-27T23:50:17.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 Processor -- start 2023-09-27T23:50:17.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 -- start start 2023-09-27T23:50:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 0x7fd04c13c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd04c0c8f30 0x7fd04c14d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd04c156640 0x7fd04c06a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c14ec10 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c14e780 con 0x7fd04c156640 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd053005700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c14db70 con 0x7fd04c0c8f30 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd052003700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 0x7fd04c13c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd052003700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 0x7fd04c13c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47744/0 (socket says 172.21.15.64:47744) 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd052003700 1 -- 172.21.15.64:0/1276869938 learned_addr learned my addr 172.21.15.64:0/1276869938 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd051802700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd04c0c8f30 0x7fd04c14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.067+0000 7fd052804700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd04c156640 0x7fd04c06a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd052003700 1 -- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd04c0c8f30 msgr2=0x7fd04c14d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd052003700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd04c0c8f30 0x7fd04c14d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd052003700 1 -- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd04c156640 msgr2=0x7fd04c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd052003700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd04c156640 0x7fd04c06a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd052003700 1 -- 172.21.15.64:0/1276869938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd044009a20 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd052003700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 0x7fd04c13c6c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c7e0 tx=0x7fd040007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040005b30 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040005c90 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040007300 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c14de50 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c14e410 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.071+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c04be40 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.075+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 76) v1 ==== 80390+0+0 (secure 0 0 0) 0x7fd04001b3b0 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.075+0000 7fd03e7fc700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd030060ac0 0x7fd030062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.075+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fd04007c8e0 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.075+0000 7fd051802700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd030060ac0 0x7fd030062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.075+0000 7fd051802700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd030060ac0 0x7fd030062f70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd04c150140 tx=0x7fd048007ec0 comp rx=0 tx=0).ready entity=mgr.9349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:17.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.083+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd04004c690 con 0x7fd04c0c8ae0 2023-09-27T23:50:17.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.267+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd04c14e8c0 con 0x7fd030060ac0 2023-09-27T23:50:17.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.283+0000 7fd03e7fc700 1 -- 172.21.15.64:0/1276869938 <== mgr.9349 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7fd048003670 con 0x7fd030060ac0 2023-09-27T23:50:17.286 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:17.286 INFO:teuthology.orchestra.run.smithi064.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:34:13", "first": "2023-09-27T23:35:00", "last": "2023-09-27T23:50:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-09-27T23:50:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd030060ac0 msgr2=0x7fd030062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd030060ac0 0x7fd030062f70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd04c150140 tx=0x7fd048007ec0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 msgr2=0x7fd04c13c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 0x7fd04c13c6c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c7e0 tx=0x7fd040007df0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 shutdown_connections 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd030060ac0 0x7fd030062f70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd04c156640 0x7fd04c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd04c0c8f30 0x7fd04c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 --2- 172.21.15.64:0/1276869938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd04c0c8ae0 0x7fd04c13c6c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 >> 172.21.15.64:0/1276869938 conn(0x7fd04c067bb0 msgr2=0x7fd04c14b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 shutdown_connections 2023-09-27T23:50:17.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.287+0000 7fd053005700 1 -- 172.21.15.64:0/1276869938 wait complete. 2023-09-27T23:50:17.305 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-09-27T00:00:00", "created": "2023-09-27T23:34:13", "first": "2023-09-27T23:35:00", "last": "2023-09-27T23:50:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-09-27T23:50:17.305 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-09-27T00:00:00', 'created': '2023-09-27T23:34:13', 'first': '2023-09-27T23:35:00', 'last': '2023-09-27T23:50:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-09-27T23:50:17.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-09-27T23:50:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7886152700 1 Processor -- start 2023-09-27T23:50:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7886152700 1 -- start start 2023-09-27T23:50:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7886152700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f788006b350 0x7f7880158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7886152700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800c85a0 con 0x7f78800abe40 2023-09-27T23:50:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7886152700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788006d9b0 con 0x7f7880158b00 2023-09-27T23:50:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7886152700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78800c67d0 con 0x7f788006b350 2023-09-27T23:50:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f788494f700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f78800abe40 0x7f788006ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37130/0 (socket says 172.21.15.64:37130) 2023-09-27T23:50:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f788494f700 1 -- 172.21.15.64:0/175921925 learned_addr learned my addr 172.21.15.64:0/175921925 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:17.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7885951700 1 --2- 172.21.15.64:0/175921925 >> v2:172.21.15.123:3300/0 conn(0x7f788006b350 0x7f7880158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.423+0000 7f7877fff700 1 -- 172.21.15.64:0/175921925 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069977592 0 0) 0x7f78800c85a0 con 0x7f78800abe40 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7877fff700 1 -- 172.21.15.64:0/175921925 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c003570 con 0x7f78800abe40 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7885951700 1 -- 172.21.15.64:0/175921925 >> v1:172.21.15.123:6789/0 conn(0x7f7880158b00 legacy=0x7f788006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7885951700 1 -- 172.21.15.64:0/175921925 >> v1:172.21.15.64:6789/0 conn(0x7f78800abe40 legacy=0x7f788006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7885951700 1 -- 172.21.15.64:0/175921925 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800c6ab0 con 0x7f788006b350 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7885951700 1 --2- 172.21.15.64:0/175921925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788006b350 0x7f7880158ed0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7878009e60 tx=0x7f7878009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e936e169ff3a27ff server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7877fff700 1 -- 172.21.15.64:0/175921925 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787800e070 con 0x7f788006b350 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7877fff700 1 -- 172.21.15.64:0/175921925 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787800bb60 con 0x7f788006b350 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7877fff700 1 -- 172.21.15.64:0/175921925 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878012430 con 0x7f788006b350 2023-09-27T23:50:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- 172.21.15.64:0/175921925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788006b350 msgr2=0x7f7880158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 --2- 172.21.15.64:0/175921925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788006b350 0x7f7880158ed0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7878009e60 tx=0x7f7878009780 comp rx=0 tx=0).stop 2023-09-27T23:50:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- 172.21.15.64:0/175921925 shutdown_connections 2023-09-27T23:50:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 --2- 172.21.15.64:0/175921925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788006b350 0x7f7880158ed0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- 172.21.15.64:0/175921925 >> 172.21.15.64:0/175921925 conn(0x7f78800c8c00 msgr2=0x7f78800c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- 172.21.15.64:0/175921925 shutdown_connections 2023-09-27T23:50:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- 172.21.15.64:0/175921925 wait complete. 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 Processor -- start 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- start start 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788006b350 0x7f7880169720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78800abe40 0x7f78800a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 0x7f78800b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800aa730 con 0x7f788006b350 2023-09-27T23:50:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800aaa10 con 0x7f7880158b00 2023-09-27T23:50:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.427+0000 7f7886152700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800aacf0 con 0x7f78800abe40 2023-09-27T23:50:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 0x7f78800b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 0x7f78800b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:52092/0 (socket says 172.21.15.64:52092) 2023-09-27T23:50:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 -- 172.21.15.64:0/2268830952 learned_addr learned my addr 172.21.15.64:0/2268830952 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885150700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788006b350 0x7f7880169720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f788494f700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78800abe40 0x7f78800a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 -- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78800abe40 msgr2=0x7f78800a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78800abe40 0x7f78800a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 -- 172.21.15.64:0/2268830952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788006b350 msgr2=0x7f7880169720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788006b350 0x7f7880169720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 -- 172.21.15.64:0/2268830952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7878009a20 con 0x7f7880158b00 2023-09-27T23:50:17.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7885951700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 0x7f78800b1520 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f788006a910 tx=0x7f7878019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878012ca0 con 0x7f7880158b00 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878021070 con 0x7f7880158b00 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78800aafd0 con 0x7f7880158b00 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878026680 con 0x7f7880158b00 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7880169c60 con 0x7f7880158b00 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.431+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78800ab450 con 0x7f7880158b00 2023-09-27T23:50:17.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.435+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 76) v1 ==== 80390+0+0 (secure 0 0 0) 0x7f7878023030 con 0x7f7880158b00 2023-09-27T23:50:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.435+0000 7f78757fa700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7864060ac0 0x7f7864062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.435+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f787807f580 con 0x7f7880158b00 2023-09-27T23:50:17.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.435+0000 7f7885150700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7864060ac0 0x7f7864062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:17.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.435+0000 7f7885150700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7864060ac0 0x7f7864062f70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78800b14c0 tx=0x7f787c009040 comp rx=0 tx=0).ready entity=mgr.9349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:17.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.447+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f787804f330 con 0x7f7880158b00 2023-09-27T23:50:17.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.631+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f788004be40 con 0x7f7864060ac0 2023-09-27T23:50:17.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f78757fa700 1 -- 172.21.15.64:0/2268830952 <== mgr.9349 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f788004be40 con 0x7f7864060ac0 2023-09-27T23:50:17.780 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-09-27T23:50:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7864060ac0 msgr2=0x7f7864062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7864060ac0 0x7f7864062f70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78800b14c0 tx=0x7f787c009040 comp rx=0 tx=0).stop 2023-09-27T23:50:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 msgr2=0x7f78800b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 0x7f78800b1520 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f788006a910 tx=0x7f7878019000 comp rx=0 tx=0).stop 2023-09-27T23:50:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 shutdown_connections 2023-09-27T23:50:17.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7864060ac0 0x7f7864062f70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7880158b00 0x7f78800b1520 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78800abe40 0x7f78800a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 --2- 172.21.15.64:0/2268830952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788006b350 0x7f7880169720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:17.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.779+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 >> 172.21.15.64:0/2268830952 conn(0x7f78800c8c00 msgr2=0x7f78801578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:17.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.783+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 shutdown_connections 2023-09-27T23:50:17.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:17.783+0000 7f7886152700 1 -- 172.21.15.64:0/2268830952 wait complete. 2023-09-27T23:50:17.799 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-09-27T23:50:17.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:17.799 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_35_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_36_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_37_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_38_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_39_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_40_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_41_00_UTC 2023-09-27T23:50:17.816 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_42_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_43_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_44_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_45_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_46_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_47_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_48_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_49_00_UTC 2023-09-27T23:50:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_50_00_UTC 2023-09-27T23:50:17.818 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_35_00_UTC 2023-09-27T23:50:17.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:17.818 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_35_00_UTC) 2023-09-27T23:50:17.883 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_36_00_UTC 2023-09-27T23:50:17.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:17.884 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_36_00_UTC) 2023-09-27T23:50:17.902 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_37_00_UTC 2023-09-27T23:50:17.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:17.902 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_37_00_UTC) 2023-09-27T23:50:17.924 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_38_00_UTC 2023-09-27T23:50:17.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:17.925 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_38_00_UTC) 2023-09-27T23:50:17.992 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_39_00_UTC 2023-09-27T23:50:17.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:17.992 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_39_00_UTC) 2023-09-27T23:50:18.016 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_40_00_UTC 2023-09-27T23:50:18.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.017 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_40_00_UTC) 2023-09-27T23:50:18.037 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_41_00_UTC 2023-09-27T23:50:18.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.037 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_41_00_UTC) 2023-09-27T23:50:18.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_42_00_UTC 2023-09-27T23:50:18.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.060 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_42_00_UTC) 2023-09-27T23:50:18.084 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_43_00_UTC 2023-09-27T23:50:18.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.085 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_43_00_UTC) 2023-09-27T23:50:18.105 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_44_00_UTC 2023-09-27T23:50:18.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.105 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_44_00_UTC) 2023-09-27T23:50:18.126 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_45_00_UTC 2023-09-27T23:50:18.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.126 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_45_00_UTC) 2023-09-27T23:50:18.147 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_46_00_UTC 2023-09-27T23:50:18.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.148 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_46_00_UTC) 2023-09-27T23:50:18.215 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_47_00_UTC 2023-09-27T23:50:18.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.215 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_47_00_UTC) 2023-09-27T23:50:18.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_48_00_UTC 2023-09-27T23:50:18.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.239 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_48_00_UTC) 2023-09-27T23:50:18.264 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_49_00_UTC 2023-09-27T23:50:18.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.264 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_49_00_UTC) 2023-09-27T23:50:18.331 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_50_00_UTC 2023-09-27T23:50:18.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.332 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-09-27-23_50_00_UTC) 2023-09-27T23:50:18.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:18.352 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-09-27T23:50:18.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:50:18.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.479+0000 7f10bd787700 1 Processor -- start 2023-09-27T23:50:18.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.479+0000 7f10bd787700 1 -- start start 2023-09-27T23:50:18.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10b80ab870 0x7f10b80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10b80b08d0 0x7f10b8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b8156470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b815a3d0 con 0x7f10b80ab870 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80b0e40 con 0x7f10b80b08d0 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80b1120 con 0x7f10b8159fb0 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10b80ab870 0x7f10b80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10b80ab870 0x7f10b80afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47758/0 (socket says 172.21.15.64:47758) 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 -- 172.21.15.64:0/331920246 learned_addr learned my addr 172.21.15.64:0/331920246 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b67fc700 1 --2- 172.21.15.64:0/331920246 >> v2:172.21.15.123:3300/0 conn(0x7f10b80b08d0 0x7f10b8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b77fe700 1 --2- 172.21.15.64:0/331920246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 -- 172.21.15.64:0/331920246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 msgr2=0x7f10b8156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 --2- 172.21.15.64:0/331920246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b8156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 -- 172.21.15.64:0/331920246 >> v2:172.21.15.123:3300/0 conn(0x7f10b80b08d0 msgr2=0x7f10b8159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 --2- 172.21.15.64:0/331920246 >> v2:172.21.15.123:3300/0 conn(0x7f10b80b08d0 0x7f10b8159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 -- 172.21.15.64:0/331920246 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80b1400 con 0x7f10b80ab870 2023-09-27T23:50:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b67fc700 1 --2- 172.21.15.64:0/331920246 >> v2:172.21.15.123:3300/0 conn(0x7f10b80b08d0 0x7f10b8159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b6ffd700 1 --2- 172.21.15.64:0/331920246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80ab870 0x7f10b80afdb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f10b806eb50 tx=0x7f10a800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54595113964fc741 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b5ffb700 1 -- 172.21.15.64:0/331920246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800dc50 con 0x7f10b80ab870 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b5ffb700 1 -- 172.21.15.64:0/331920246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a800ddb0 con 0x7f10b80ab870 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10b5ffb700 1 -- 172.21.15.64:0/331920246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80196d0 con 0x7f10b80ab870 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- 172.21.15.64:0/331920246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80ab870 msgr2=0x7f10b80afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- 172.21.15.64:0/331920246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80ab870 0x7f10b80afdb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f10b806eb50 tx=0x7f10a800b4f0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- 172.21.15.64:0/331920246 shutdown_connections 2023-09-27T23:50:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- 172.21.15.64:0/331920246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b8156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- 172.21.15.64:0/331920246 >> v2:172.21.15.123:3300/0 conn(0x7f10b80b08d0 0x7f10b8159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 --2- 172.21.15.64:0/331920246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80ab870 0x7f10b80afdb0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- 172.21.15.64:0/331920246 >> 172.21.15.64:0/331920246 conn(0x7f10b80cc580 msgr2=0x7f10b80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- 172.21.15.64:0/331920246 shutdown_connections 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.483+0000 7f10bd787700 1 -- 172.21.15.64:0/331920246 wait complete. 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 Processor -- start 2023-09-27T23:50:18.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 -- start start 2023-09-27T23:50:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10b80ab870 0x7f10b8150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 0x7f10b8150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8158750 con 0x7f10b80b08d0 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81651c0 con 0x7f10b80ab870 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8157b40 con 0x7f10b8159fb0 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 0x7f10b8150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 0x7f10b8150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47764/0 (socket says 172.21.15.64:47764) 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 -- 172.21.15.64:0/4006842575 learned_addr learned my addr 172.21.15.64:0/4006842575 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b6ffd700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10b80ab870 0x7f10b8150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b77fe700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 -- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 msgr2=0x7f10b80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 -- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10b80ab870 msgr2=0x7f10b8150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:18.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10b80ab870 0x7f10b8150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:18.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 -- 172.21.15.64:0/4006842575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a800b790 con 0x7f10b80b08d0 2023-09-27T23:50:18.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10b67fc700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 0x7f10b8150e60 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00d860 tx=0x7f10ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:18.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0125f0 con 0x7f10b80b08d0 2023-09-27T23:50:18.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac012750 con 0x7f10b80b08d0 2023-09-27T23:50:18.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b8157e20 con 0x7f10b80b08d0 2023-09-27T23:50:18.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac012af0 con 0x7f10b80b08d0 2023-09-27T23:50:18.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.487+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b81583e0 con 0x7f10b80b08d0 2023-09-27T23:50:18.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.491+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b8071120 con 0x7f10b80b08d0 2023-09-27T23:50:18.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.491+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 76) v1 ==== 80390+0+0 (secure 0 0 0) 0x7f10ac013480 con 0x7f10b80b08d0 2023-09-27T23:50:18.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.491+0000 7f109effd700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f10a4060ac0 0x7f10a4062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:18.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.491+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f10ac07d580 con 0x7f10b80b08d0 2023-09-27T23:50:18.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.491+0000 7f10b6ffd700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f10a4060ac0 0x7f10a4062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:18.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.495+0000 7f10b6ffd700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f10a4060ac0 0x7f10a4062f70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f10b806eb50 tx=0x7f10a801b000 comp rx=0 tx=0).ready entity=mgr.9349 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:18.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.499+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f10ac04d330 con 0x7f10b80b08d0 2023-09-27T23:50:18.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:18.707+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f10b8158890 con 0x7f10b80b08d0 2023-09-27T23:50:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.039+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f10ac04d700 con 0x7f10b80b08d0 2023-09-27T23:50:19.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.039+0000 7f109effd700 1 -- 172.21.15.64:0/4006842575 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v77) v1 ==== 93+0+0 (secure 0 0 0) 0x7f10ac04d330 con 0x7f10b80b08d0 2023-09-27T23:50:19.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f10a4060ac0 msgr2=0x7f10a4062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f10a4060ac0 0x7f10a4062f70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f10b806eb50 tx=0x7f10a801b000 comp rx=0 tx=0).stop 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 msgr2=0x7f10b8150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 0x7f10b8150e60 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00d860 tx=0x7f10ac008040 comp rx=0 tx=0).stop 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 shutdown_connections 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f10a4060ac0 0x7f10a4062f70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10b8159fb0 0x7f10b80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10b80b08d0 0x7f10b8150e60 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 --2- 172.21.15.64:0/4006842575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10b80ab870 0x7f10b8150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 >> 172.21.15.64:0/4006842575 conn(0x7f10b80cc580 msgr2=0x7f10b8159890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 shutdown_connections 2023-09-27T23:50:19.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.043+0000 7f10bd787700 1 -- 172.21.15.64:0/4006842575 wait complete. 2023-09-27T23:50:19.062 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:50:19.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:50:19.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:50:19.111 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:50:19.111 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:50:19.111 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:50:19.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:19.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[58501]: fuse finished with error 0 and tester_r 0 2023-09-27T23:50:19.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:50:19.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:19.127 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:50:19.171 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:19.166+0000 7f110c74d000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:50:19.171 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:19.167+0000 7f6920182000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:50:19.299 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:50:19.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:19.300 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:50:19.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:50:19.309 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:50:19.309 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:50:19.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:19.310 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:50:19.350 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:50:19.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:50:19.351 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:50:19.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:19.351 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:50:19.360 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:50:19.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:50:19.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f9054c4d700 1 Processor -- start 2023-09-27T23:50:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f9054c4d700 1 -- start start 2023-09-27T23:50:19.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f9054c4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f905006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f9054c4d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905006b730 con 0x7f9050158e20 2023-09-27T23:50:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f9054c4d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905006ab20 con 0x7f90500ce810 2023-09-27T23:50:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f9054c4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905006ae00 con 0x7f90501591f0 2023-09-27T23:50:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f904f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9050158e20 0x7f90500ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37134/0 (socket says 172.21.15.64:37134) 2023-09-27T23:50:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f904f7fe700 1 -- 172.21.15.64:0/1966765888 learned_addr learned my addr 172.21.15.64:0/1966765888 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:19.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.471+0000 7f904effd700 1 --2- 172.21.15.64:0/1966765888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f905006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904e7fc700 1 -- 172.21.15.64:0/1966765888 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725517365 0 0) 0x7f905006b730 con 0x7f9050158e20 2023-09-27T23:50:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904e7fc700 1 -- 172.21.15.64:0/1966765888 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9038003570 con 0x7f9050158e20 2023-09-27T23:50:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904effd700 1 -- 172.21.15.64:0/1966765888 >> v1:172.21.15.123:6789/0 conn(0x7f90500ce810 legacy=0x7f90500ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904effd700 1 -- 172.21.15.64:0/1966765888 >> v1:172.21.15.64:6789/0 conn(0x7f9050158e20 legacy=0x7f90500ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904effd700 1 -- 172.21.15.64:0/1966765888 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905006b0e0 con 0x7f90501591f0 2023-09-27T23:50:19.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904effd700 1 --2- 172.21.15.64:0/1966765888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f905006a540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9044009910 tx=0x7f90440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca474b44f44757c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904e7fc700 1 -- 172.21.15.64:0/1966765888 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400b680 con 0x7f90501591f0 2023-09-27T23:50:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904e7fc700 1 -- 172.21.15.64:0/1966765888 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400b7e0 con 0x7f90501591f0 2023-09-27T23:50:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f904e7fc700 1 -- 172.21.15.64:0/1966765888 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400bb60 con 0x7f90501591f0 2023-09-27T23:50:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 -- 172.21.15.64:0/1966765888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 msgr2=0x7f905006a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 --2- 172.21.15.64:0/1966765888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f905006a540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9044009910 tx=0x7f90440091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 -- 172.21.15.64:0/1966765888 shutdown_connections 2023-09-27T23:50:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 --2- 172.21.15.64:0/1966765888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f905006a540 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 -- 172.21.15.64:0/1966765888 >> 172.21.15.64:0/1966765888 conn(0x7f9050067bb0 msgr2=0x7f90500cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 -- 172.21.15.64:0/1966765888 shutdown_connections 2023-09-27T23:50:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.475+0000 7f9054c4d700 1 -- 172.21.15.64:0/1966765888 wait complete. 2023-09-27T23:50:19.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 Processor -- start 2023-09-27T23:50:19.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 -- start start 2023-09-27T23:50:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 0x7f905016c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9050158e20 0x7f9050169170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f90500c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905016a6a0 con 0x7f90500ce810 2023-09-27T23:50:19.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050154dc0 con 0x7f9050158e20 2023-09-27T23:50:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90501697d0 con 0x7f90501591f0 2023-09-27T23:50:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 0x7f905016c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 0x7f905016c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47778/0 (socket says 172.21.15.64:47778) 2023-09-27T23:50:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 -- 172.21.15.64:0/2089995490 learned_addr learned my addr 172.21.15.64:0/2089995490 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904ffff700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f90500c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904effd700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9050158e20 0x7f9050169170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 -- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 msgr2=0x7f90500c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f90500c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 -- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9050158e20 msgr2=0x7f9050169170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9050158e20 0x7f9050169170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 -- 172.21.15.64:0/2089995490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044009420 con 0x7f90500ce810 2023-09-27T23:50:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904ffff700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f90500c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f904f7fe700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 0x7f905016c1a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f904000c7c0 tx=0x7f9040007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040005cd0 con 0x7f90500ce810 2023-09-27T23:50:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040005e30 con 0x7f90500ce810 2023-09-27T23:50:19.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9050169ab0 con 0x7f90500ce810 2023-09-27T23:50:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040007320 con 0x7f90500ce810 2023-09-27T23:50:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.479+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905016a070 con 0x7f90500ce810 2023-09-27T23:50:19.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.483+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050157760 con 0x7f90500ce810 2023-09-27T23:50:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.483+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f904001b3b0 con 0x7f90500ce810 2023-09-27T23:50:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.483+0000 7f90377fe700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 0x7f903c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.483+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f904007cad0 con 0x7f90500ce810 2023-09-27T23:50:19.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.483+0000 7f904effd700 1 -- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 msgr2=0x7f903c062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:19.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.483+0000 7f904effd700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 0x7f903c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:19.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.491+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f904004c880 con 0x7f90500ce810 2023-09-27T23:50:19.594 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:19.590+0000 7f110c74d000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:50:19.617 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:19.615+0000 7f6920182000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:50:19.642 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:19.638+0000 7f110c74d000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:50:19.669 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:19.667+0000 7f6920182000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:50:19.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.683+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f905004be40 con 0x7f90500ce810 2023-09-27T23:50:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f904effd700 1 -- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 msgr2=0x7f903c062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f904effd700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 0x7f903c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:50:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f90377fe700 1 -- 172.21.15.64:0/2089995490 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9040017090 con 0x7f90500ce810 2023-09-27T23:50:19.688 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:19.688 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:56.514725+0000","modified":"2023-09-27T23:33:58.531554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3161207024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3161207024},{"type":"v1","addr":"172.21.15.64:6835","nonce":3161207024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 msgr2=0x7f903c062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 0x7f903c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 msgr2=0x7f905016c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 0x7f905016c1a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f904000c7c0 tx=0x7f9040007e00 comp rx=0 tx=0).stop 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 shutdown_connections 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f903c060a70 0x7f903c062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90501591f0 0x7f90500c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9050158e20 0x7f9050169170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 --2- 172.21.15.64:0/2089995490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90500ce810 0x7f905016c1a0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 >> 172.21.15.64:0/2089995490 conn(0x7f9050067bb0 msgr2=0x7f90500b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:19.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 shutdown_connections 2023-09-27T23:50:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.687+0000 7f9054c4d700 1 -- 172.21.15.64:0/2089995490 wait complete. 2023-09-27T23:50:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 60 2023-09-27T23:50:19.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:19.748 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:19.742+0000 7f110c74d000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:19.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 Processor -- start 2023-09-27T23:50:19.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 -- start start 2023-09-27T23:50:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6800ab910 0x7fb680159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800670e0 0x7fb6800abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6800cd430 0x7fb6800c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800ac220 con 0x7fb6800cd430 2023-09-27T23:50:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800c1910 con 0x7fb6800ab910 2023-09-27T23:50:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb68487c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb680159cf0 con 0x7fb6800670e0 2023-09-27T23:50:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6800cd430 0x7fb6800c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb6800cd430 0x7fb6800c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47780/0 (socket says 172.21.15.64:47780) 2023-09-27T23:50:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 -- 172.21.15.64:0/1751015737 learned_addr learned my addr 172.21.15.64:0/1751015737 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67effd700 1 --2- 172.21.15.64:0/1751015737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800670e0 0x7fb6800abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67f7fe700 1 --2- 172.21.15.64:0/1751015737 >> v2:172.21.15.123:3300/0 conn(0x7fb6800ab910 0x7fb680159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.804 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:19.799+0000 7f6920182000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:19.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 -- 172.21.15.64:0/1751015737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800670e0 msgr2=0x7fb6800abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 --2- 172.21.15.64:0/1751015737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800670e0 0x7fb6800abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 -- 172.21.15.64:0/1751015737 >> v2:172.21.15.123:3300/0 conn(0x7fb6800ab910 msgr2=0x7fb680159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 --2- 172.21.15.64:0/1751015737 >> v2:172.21.15.123:3300/0 conn(0x7fb6800ab910 0x7fb680159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67ffff700 1 -- 172.21.15.64:0/1751015737 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680159fd0 con 0x7fb6800cd430 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.799+0000 7fb67f7fe700 1 --2- 172.21.15.64:0/1751015737 >> v2:172.21.15.123:3300/0 conn(0x7fb6800ab910 0x7fb680159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb67ffff700 1 --2- 172.21.15.64:0/1751015737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800cd430 0x7fb6800c2f80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fb680063e30 tx=0x7fb678009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79fdba4619d7498 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb67e7fc700 1 -- 172.21.15.64:0/1751015737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800e070 con 0x7fb6800cd430 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb67e7fc700 1 -- 172.21.15.64:0/1751015737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67800ba20 con 0x7fb6800cd430 2023-09-27T23:50:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb67e7fc700 1 -- 172.21.15.64:0/1751015737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780092e0 con 0x7fb6800cd430 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 -- 172.21.15.64:0/1751015737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800cd430 msgr2=0x7fb6800c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 --2- 172.21.15.64:0/1751015737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800cd430 0x7fb6800c2f80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fb680063e30 tx=0x7fb678009840 comp rx=0 tx=0).stop 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 -- 172.21.15.64:0/1751015737 shutdown_connections 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 --2- 172.21.15.64:0/1751015737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800cd430 0x7fb6800c2f80 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 --2- 172.21.15.64:0/1751015737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800670e0 0x7fb6800abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 --2- 172.21.15.64:0/1751015737 >> v2:172.21.15.123:3300/0 conn(0x7fb6800ab910 0x7fb680159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 -- 172.21.15.64:0/1751015737 >> 172.21.15.64:0/1751015737 conn(0x7fb68006c080 msgr2=0x7fb6800ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 -- 172.21.15.64:0/1751015737 shutdown_connections 2023-09-27T23:50:19.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 -- 172.21.15.64:0/1751015737 wait complete. 2023-09-27T23:50:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 Processor -- start 2023-09-27T23:50:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.803+0000 7fb68487c700 1 -- start start 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6800670e0 0x7fb6800b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 0x7fb6800cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800cd430 0x7fb680151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015a7d0 con 0x7fb6800ab910 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015aab0 con 0x7fb6800670e0 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb680151740 con 0x7fb6800cd430 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 0x7fb6800cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 0x7fb6800cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47784/0 (socket says 172.21.15.64:47784) 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 -- 172.21.15.64:0/743956969 learned_addr learned my addr 172.21.15.64:0/743956969 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67ffff700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800cd430 0x7fb680151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67f7fe700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6800670e0 0x7fb6800b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 -- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800cd430 msgr2=0x7fb680151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800cd430 0x7fb680151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 -- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6800670e0 msgr2=0x7fb6800b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6800670e0 0x7fb6800b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 -- 172.21.15.64:0/743956969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009ae0 con 0x7fb6800ab910 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67f7fe700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6800670e0 0x7fb6800b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb67effd700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 0x7fb6800cb900 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb67400e960 tx=0x7fb6740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674013680 con 0x7fb6800ab910 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6740137e0 con 0x7fb6800ab910 2023-09-27T23:50:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800b1f40 con 0x7fb6800ab910 2023-09-27T23:50:19.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674013b80 con 0x7fb6800ab910 2023-09-27T23:50:19.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6800b2500 con 0x7fb6800ab910 2023-09-27T23:50:19.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.807+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68015ac40 con 0x7fb6800ab910 2023-09-27T23:50:19.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.811+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fb67400c490 con 0x7fb6800ab910 2023-09-27T23:50:19.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.811+0000 7fb6677fe700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb66c060a70 0x7fb66c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:19.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.811+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fb67407dc90 con 0x7fb6800ab910 2023-09-27T23:50:19.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.811+0000 7fb67f7fe700 1 -- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb66c060a70 msgr2=0x7fb66c062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:19.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.811+0000 7fb67f7fe700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb66c060a70 0x7fb66c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.815+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb67404dac0 con 0x7fb6800ab910 2023-09-27T23:50:19.941 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:19.938+0000 7f110c74d000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:50:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.975+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb68004be40 con 0x7fb6800ab910 2023-09-27T23:50:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb6677fe700 1 -- 172.21.15.64:0/743956969 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+18011 (secure 0 0 0) 0x7fb674017090 con 0x7fb6800ab910 2023-09-27T23:50:19.981 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:19.981 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":161,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:45:19.819278+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:55.078420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:55.503601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2a","mappings":[{"from":4,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3448188613":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1602424936":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1684150586":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1316115001":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6833/33228":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/1077033381":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6832/33228":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/1459316426":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2445385630":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2621537395":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/4021019210":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/181603911":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2431874217":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/2124381486":"2023-09-28T23:34:12.030420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:19.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb66c060a70 msgr2=0x7fb66c062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:19.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb66c060a70 0x7fb66c062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 msgr2=0x7fb6800cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:19.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 0x7fb6800cb900 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb67400e960 tx=0x7fb6740091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 shutdown_connections 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb66c060a70 0x7fb66c062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6800cd430 0x7fb680151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6800ab910 0x7fb6800cb900 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 --2- 172.21.15.64:0/743956969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6800670e0 0x7fb6800b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 >> 172.21.15.64:0/743956969 conn(0x7fb68006c080 msgr2=0x7fb6800c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 shutdown_connections 2023-09-27T23:50:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:19.979+0000 7fb68487c700 1 -- 172.21.15.64:0/743956969 wait complete. 2023-09-27T23:50:19.990 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:19.986+0000 7f110c74d000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:19.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:33:55.078420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:33:55.503601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:50:19.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:50:20.014 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.011+0000 7f6920182000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:50:20.070 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.067+0000 7f6920182000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 Processor -- start 2023-09-27T23:50:20.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- start start 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4e9c0ce2d0 0x7f4e9c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4e9c0cc040 0x7f4e9c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0cbbb0 con 0x7f4e9c0caa10 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c06dc90 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c0cafa0 con 0x7f4e9c0cc040 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea251b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4e9c0ce2d0 0x7f4e9c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1d1a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e9c0caa10 0x7f4e9c158900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37144/0 (socket says 172.21.15.64:37144) 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1d1a700 1 -- 172.21.15.64:0/3113664401 learned_addr learned my addr 172.21.15.64:0/3113664401 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea2d1c700 1 --2- 172.21.15.64:0/3113664401 >> v2:172.21.15.123:3300/0 conn(0x7f4e9c0cc040 0x7f4e9c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1519700 1 -- 172.21.15.64:0/3113664401 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456807424 0 0) 0x7f4e9c0cbbb0 con 0x7f4e9c0caa10 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1519700 1 -- 172.21.15.64:0/3113664401 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e88003570 con 0x7f4e9c0caa10 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea251b700 1 -- 172.21.15.64:0/3113664401 >> v2:172.21.15.123:3300/0 conn(0x7f4e9c0cc040 msgr2=0x7f4e9c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea251b700 1 --2- 172.21.15.64:0/3113664401 >> v2:172.21.15.123:3300/0 conn(0x7f4e9c0cc040 0x7f4e9c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea251b700 1 -- 172.21.15.64:0/3113664401 >> v1:172.21.15.64:6789/0 conn(0x7f4e9c0caa10 legacy=0x7f4e9c158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea251b700 1 -- 172.21.15.64:0/3113664401 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0cb280 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea251b700 1 --2- 172.21.15.64:0/3113664401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c0ce6a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ca350 tx=0x7f4e980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44b340139799dc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1519700 1 -- 172.21.15.64:0/3113664401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98011630 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1519700 1 -- 172.21.15.64:0/3113664401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e98011790 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea1519700 1 -- 172.21.15.64:0/3113664401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98011b30 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- 172.21.15.64:0/3113664401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 msgr2=0x7f4e9c0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 --2- 172.21.15.64:0/3113664401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c0ce6a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ca350 tx=0x7f4e980091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- 172.21.15.64:0/3113664401 shutdown_connections 2023-09-27T23:50:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 --2- 172.21.15.64:0/3113664401 >> v2:172.21.15.123:3300/0 conn(0x7f4e9c0cc040 0x7f4e9c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 --2- 172.21.15.64:0/3113664401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c0ce6a0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.099+0000 7f4ea351d700 1 -- 172.21.15.64:0/3113664401 >> 172.21.15.64:0/3113664401 conn(0x7f4e9c067bb0 msgr2=0x7f4e9c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- 172.21.15.64:0/3113664401 shutdown_connections 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- 172.21.15.64:0/3113664401 wait complete. 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 Processor -- start 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- start start 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e9c0caa10 0x7f4e9c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e9c0cc040 0x7f4e9c1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0c7c00 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0b55c0 con 0x7f4e9c0caa10 2023-09-27T23:50:20.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0b4a40 con 0x7f4e9c0cc040 2023-09-27T23:50:20.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c1586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47802/0 (socket says 172.21.15.64:47802) 2023-09-27T23:50:20.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 -- 172.21.15.64:0/1413031730 learned_addr learned my addr 172.21.15.64:0/1413031730 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:20.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea1d1a700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e9c0cc040 0x7f4e9c1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea251b700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e9c0caa10 0x7f4e9c0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 -- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e9c0cc040 msgr2=0x7f4e9c1419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e9c0cc040 0x7f4e9c1419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 -- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e9c0caa10 msgr2=0x7f4e9c0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e9c0caa10 0x7f4e9c0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 -- 172.21.15.64:0/1413031730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e98009420 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea2d1c700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c1586d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009fa0 tx=0x7f4e94009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:20.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400ebc0 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400ed20 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0b4d20 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94014670 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.103+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c0b52e0 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.107+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f4e940149e0 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.107+0000 7f4e927fc700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4e80060a70 0x7f4e80062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.107+0000 7f4ea251b700 1 -- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4e80060a70 msgr2=0x7f4e80062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:20.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.107+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f4e9407da10 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.107+0000 7f4ea251b700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4e80060a70 0x7f4e80062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:20.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.111+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c15a5d0 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.115+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4e9404d7c0 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.143 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:20.138+0000 7f110c74d000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:50:20.200 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:20.194+0000 7f110c74d000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:50:20.239 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.235+0000 7f6920182000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:50:20.260 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:20.254+0000 7f110c74d000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:50:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.279+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e9c04be40 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.279+0000 7f4e927fc700 1 -- 172.21.15.64:0/1413031730 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e94050de0 con 0x7f4e9c0ce2d0 2023-09-27T23:50:20.283 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:50:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4e80060a70 msgr2=0x7f4e80062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4e80060a70 0x7f4e80062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 msgr2=0x7f4e9c1586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c1586d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009fa0 tx=0x7f4e94009900 comp rx=0 tx=0).stop 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 shutdown_connections 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4e80060a70 0x7f4e80062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c0ce2d0 0x7f4e9c1586d0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4e9c0cc040 0x7f4e9c1419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 --2- 172.21.15.64:0/1413031730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e9c0caa10 0x7f4e9c0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 >> 172.21.15.64:0/1413031730 conn(0x7f4e9c067bb0 msgr2=0x7f4e9c0c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 shutdown_connections 2023-09-27T23:50:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.283+0000 7f4ea351d700 1 -- 172.21.15.64:0/1413031730 wait complete. 2023-09-27T23:50:20.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:50:20.299 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.295+0000 7f6920182000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:50:20.362 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.359+0000 7f6920182000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:50:20.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.403+0000 7fd3965f5700 1 Processor -- start 2023-09-27T23:50:20.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.403+0000 7fd3965f5700 1 -- start start 2023-09-27T23:50:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd390156eb0 0x7fd3900ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd39015aa20 0x7fd3900acd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd3900cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3900ac8f0 con 0x7fd39015aa20 2023-09-27T23:50:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39015ae50 con 0x7fd390156eb0 2023-09-27T23:50:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39015b120 con 0x7fd3900ad2c0 2023-09-27T23:50:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd39015aa20 0x7fd3900acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd39015aa20 0x7fd3900acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47818/0 (socket says 172.21.15.64:47818) 2023-09-27T23:50:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 -- 172.21.15.64:0/2936669916 learned_addr learned my addr 172.21.15.64:0/2936669916 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:20.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd394b92700 1 --2- 172.21.15.64:0/2936669916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd3900cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38ffff700 1 --2- 172.21.15.64:0/2936669916 >> v2:172.21.15.123:3300/0 conn(0x7fd390156eb0 0x7fd3900ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 -- 172.21.15.64:0/2936669916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 msgr2=0x7fd3900cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2936669916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd3900cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 -- 172.21.15.64:0/2936669916 >> v2:172.21.15.123:3300/0 conn(0x7fd390156eb0 msgr2=0x7fd3900ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2936669916 >> v2:172.21.15.123:3300/0 conn(0x7fd390156eb0 0x7fd3900ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 -- 172.21.15.64:0/2936669916 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39015b400 con 0x7fd39015aa20 2023-09-27T23:50:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2936669916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd3900acd80 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd39006f430 tx=0x7fd384009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38fc04ff6fa5318 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38effd700 1 -- 172.21.15.64:0/2936669916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400b640 con 0x7fd39015aa20 2023-09-27T23:50:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38effd700 1 -- 172.21.15.64:0/2936669916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400b7a0 con 0x7fd39015aa20 2023-09-27T23:50:20.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd38effd700 1 -- 172.21.15.64:0/2936669916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400bb60 con 0x7fd39015aa20 2023-09-27T23:50:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- 172.21.15.64:0/2936669916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 msgr2=0x7fd3900acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- 172.21.15.64:0/2936669916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd3900acd80 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd39006f430 tx=0x7fd384009250 comp rx=0 tx=0).stop 2023-09-27T23:50:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- 172.21.15.64:0/2936669916 shutdown_connections 2023-09-27T23:50:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- 172.21.15.64:0/2936669916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd3900cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- 172.21.15.64:0/2936669916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd3900acd80 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 --2- 172.21.15.64:0/2936669916 >> v2:172.21.15.123:3300/0 conn(0x7fd390156eb0 0x7fd3900ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- 172.21.15.64:0/2936669916 >> 172.21.15.64:0/2936669916 conn(0x7fd3900cd8f0 msgr2=0x7fd3900c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- 172.21.15.64:0/2936669916 shutdown_connections 2023-09-27T23:50:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.407+0000 7fd3965f5700 1 -- 172.21.15.64:0/2936669916 wait complete. 2023-09-27T23:50:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 Processor -- start 2023-09-27T23:50:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 -- start start 2023-09-27T23:50:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd390154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd390156eb0 0x7fd39013f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd390157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390168760 con 0x7fd39015aa20 2023-09-27T23:50:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390154580 con 0x7fd390156eb0 2023-09-27T23:50:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd3965f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39013fd90 con 0x7fd3900ad2c0 2023-09-27T23:50:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd390154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd390154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:51938/0 (socket says 172.21.15.64:51938) 2023-09-27T23:50:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 -- 172.21.15.64:0/1654429803 learned_addr learned my addr 172.21.15.64:0/1654429803 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd394b92700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd390157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd390156eb0 0x7fd39013f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 -- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd390156eb0 msgr2=0x7fd39013f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd390156eb0 0x7fd39013f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 -- 172.21.15.64:0/1654429803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 msgr2=0x7fd390157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd390157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 -- 172.21.15.64:0/1654429803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840094f0 con 0x7fd3900ad2c0 2023-09-27T23:50:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd38ffff700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd390154040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd38000e930 tx=0x7fd3800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.411+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380013680 con 0x7fd3900ad2c0 2023-09-27T23:50:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3800137e0 con 0x7fd3900ad2c0 2023-09-27T23:50:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd390167b50 con 0x7fd3900ad2c0 2023-09-27T23:50:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380013b80 con 0x7fd3900ad2c0 2023-09-27T23:50:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd390168110 con 0x7fd3900ad2c0 2023-09-27T23:50:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd390071120 con 0x7fd3900ad2c0 2023-09-27T23:50:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fd38000c470 con 0x7fd3900ad2c0 2023-09-27T23:50:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd377fff700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd37c060aa0 0x7fd37c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd38f7fe700 1 -- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd37c060aa0 msgr2=0x7fd37c062f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fd38007d520 con 0x7fd3900ad2c0 2023-09-27T23:50:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.415+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd37c060aa0 0x7fd37c062f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:20.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.423+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd38004d230 con 0x7fd3900ad2c0 2023-09-27T23:50:20.557 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:20.554+0000 7f110c74d000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:50:20.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.587+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd39006a240 con 0x7fd3900ad2c0 2023-09-27T23:50:20.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd377fff700 1 -- 172.21.15.64:0/1654429803 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v161) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd380050850 con 0x7fd3900ad2c0 2023-09-27T23:50:20.592 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd37c060aa0 msgr2=0x7fd37c062f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd37c060aa0 0x7fd37c062f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 msgr2=0x7fd390154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd390154040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd38000e930 tx=0x7fd3800091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 shutdown_connections 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd37c060aa0 0x7fd37c062f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd39015aa20 0x7fd390157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd390156eb0 0x7fd39013f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 --2- 172.21.15.64:0/1654429803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3900ad2c0 0x7fd390154040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 >> 172.21.15.64:0/1654429803 conn(0x7fd3900cd8f0 msgr2=0x7fd3900c2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 shutdown_connections 2023-09-27T23:50:20.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.591+0000 7fd3965f5700 1 -- 172.21.15.64:0/1654429803 wait complete. 2023-09-27T23:50:20.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:50:20.667 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:20.662+0000 7f110c74d000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:50:20.678 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.675+0000 7f6920182000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:50:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.711+0000 7fb4990a8700 1 Processor -- start 2023-09-27T23:50:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.711+0000 7fb4990a8700 1 -- start start 2023-09-27T23:50:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb49415a180 0x7fb494160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4940b0e40 con 0x7fb4940ab870 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4940b1120 con 0x7fb4940b08d0 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4940b1400 con 0x7fb49415a180 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb492d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb4940ab870 0x7fb4940afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37158/0 (socket says 172.21.15.64:37158) 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb492d9d700 1 -- 172.21.15.64:0/964883688 learned_addr learned my addr 172.21.15.64:0/964883688 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb49359e700 1 --2- 172.21.15.64:0/964883688 >> v2:172.21.15.64:3300/0 conn(0x7fb49415a180 0x7fb494160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb491d9b700 1 -- 172.21.15.64:0/964883688 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683100068 0 0) 0x7fb4940b0e40 con 0x7fb4940ab870 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb49359e700 1 -- 172.21.15.64:0/964883688 >> v1:172.21.15.123:6789/0 conn(0x7fb4940b08d0 legacy=0x7fb494159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb49359e700 1 -- 172.21.15.64:0/964883688 >> v1:172.21.15.64:6789/0 conn(0x7fb4940ab870 legacy=0x7fb4940afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb49359e700 1 -- 172.21.15.64:0/964883688 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940b16e0 con 0x7fb49415a180 2023-09-27T23:50:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb49359e700 1 --2- 172.21.15.64:0/964883688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49415a180 0x7fb494160e10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009e60 tx=0x7fb48c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5aecc8117307b8d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb491d9b700 1 -- 172.21.15.64:0/964883688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00e070 con 0x7fb49415a180 2023-09-27T23:50:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb491d9b700 1 -- 172.21.15.64:0/964883688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c00bc30 con 0x7fb49415a180 2023-09-27T23:50:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb491d9b700 1 -- 172.21.15.64:0/964883688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c012430 con 0x7fb49415a180 2023-09-27T23:50:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- 172.21.15.64:0/964883688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49415a180 msgr2=0x7fb494160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 --2- 172.21.15.64:0/964883688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49415a180 0x7fb494160e10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009e60 tx=0x7fb48c009780 comp rx=0 tx=0).stop 2023-09-27T23:50:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- 172.21.15.64:0/964883688 shutdown_connections 2023-09-27T23:50:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 --2- 172.21.15.64:0/964883688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49415a180 0x7fb494160e10 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- 172.21.15.64:0/964883688 >> 172.21.15.64:0/964883688 conn(0x7fb4940cc580 msgr2=0x7fb4940cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- 172.21.15.64:0/964883688 shutdown_connections 2023-09-27T23:50:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.715+0000 7fb4990a8700 1 -- 172.21.15.64:0/964883688 wait complete. 2023-09-27T23:50:20.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 Processor -- start 2023-09-27T23:50:20.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 -- start start 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4940ab870 0x7fb4940ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 0x7fb494151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49415a180 0x7fb49406b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415db00 con 0x7fb4940b08d0 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415cef0 con 0x7fb49415a180 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb4990a8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415d1d0 con 0x7fb4940ab870 2023-09-27T23:50:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb49259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 0x7fb494151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb492d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4940ab870 0x7fb4940ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb49259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 0x7fb494151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47832/0 (socket says 172.21.15.64:47832) 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb49259c700 1 -- 172.21.15.64:0/2521028352 learned_addr learned my addr 172.21.15.64:0/2521028352 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.719+0000 7fb49359e700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49415a180 0x7fb49406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb49259c700 1 -- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4940ab870 msgr2=0x7fb4940ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb49259c700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4940ab870 0x7fb4940ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb49259c700 1 -- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49415a180 msgr2=0x7fb49406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb49259c700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49415a180 0x7fb49406b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb49259c700 1 -- 172.21.15.64:0/2521028352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c009a20 con 0x7fb4940b08d0 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb492d9d700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4940ab870 0x7fb4940ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb49259c700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 0x7fb494151f30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c590 tx=0x7fb48800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488011590 con 0x7fb4940b08d0 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb488011710 con 0x7fb4940b08d0 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49415d4b0 con 0x7fb4940b08d0 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488011ad0 con 0x7fb4940b08d0 2023-09-27T23:50:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4940addb0 con 0x7fb4940b08d0 2023-09-27T23:50:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49415dc70 con 0x7fb4940b08d0 2023-09-27T23:50:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fb48801b3b0 con 0x7fb4940b08d0 2023-09-27T23:50:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb47affd700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb480060a70 0x7fb480062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fb48807c890 con 0x7fb4940b08d0 2023-09-27T23:50:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb492d9d700 1 -- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb480060a70 msgr2=0x7fb480062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.723+0000 7fb492d9d700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb480060a70 0x7fb480062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:20.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.731+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb48804c640 con 0x7fb4940b08d0 2023-09-27T23:50:20.792 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:20.791+0000 7f6920182000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:50:20.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.899+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb49415d930 con 0x7fb4940b08d0 2023-09-27T23:50:20.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.899+0000 7fb47affd700 1 -- 172.21.15.64:0/2521028352 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v161) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4880170a0 con 0x7fb4940b08d0 2023-09-27T23:50:20.902 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb480060a70 msgr2=0x7fb480062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb480060a70 0x7fb480062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 msgr2=0x7fb494151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 0x7fb494151f30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c590 tx=0x7fb48800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 shutdown_connections 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb480060a70 0x7fb480062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49415a180 0x7fb49406b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4940b08d0 0x7fb494151f30 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 --2- 172.21.15.64:0/2521028352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4940ab870 0x7fb4940ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 >> 172.21.15.64:0/2521028352 conn(0x7fb4940cc580 msgr2=0x7fb494155620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 shutdown_connections 2023-09-27T23:50:20.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:20.903+0000 7fb4990a8700 1 -- 172.21.15.64:0/2521028352 wait complete. 2023-09-27T23:50:20.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:50:20.948 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:20.946+0000 7f110c74d000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:50:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 Processor -- start 2023-09-27T23:50:21.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 -- start start 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d0d0 0x7f5764158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f57640cae40 0x7f576406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640cbf30 con 0x7f576413d4a0 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57641585b0 con 0x7f57640cae40 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f576a563700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cc3c0 con 0x7f576413d0d0 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f5768b00700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f576413d4a0 0x7f576415abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37168/0 (socket says 172.21.15.64:37168) 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f5768b00700 1 -- 172.21.15.64:0/4154846347 learned_addr learned my addr 172.21.15.64:0/4154846347 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f5763fff700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d0d0 0x7f5764158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.027+0000 7f57637fe700 1 --2- 172.21.15.64:0/4154846347 >> v2:172.21.15.123:3300/0 conn(0x7f57640cae40 0x7f576406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f5762ffd700 1 -- 172.21.15.64:0/4154846347 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941520518 0 0) 0x7f57640cbf30 con 0x7f576413d4a0 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f5762ffd700 1 -- 172.21.15.64:0/4154846347 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c003570 con 0x7f576413d4a0 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f57637fe700 1 -- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d0d0 msgr2=0x7f5764158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f57637fe700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d0d0 0x7f5764158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f57637fe700 1 -- 172.21.15.64:0/4154846347 >> v1:172.21.15.64:6789/0 conn(0x7f576413d4a0 legacy=0x7f576415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f57637fe700 1 -- 172.21.15.64:0/4154846347 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640cb320 con 0x7f57640cae40 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f5763fff700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d0d0 0x7f5764158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f57637fe700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57640cae40 0x7f576406b0f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f57580098e0 tx=0x7f57580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3adcb32a737b4976 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f5762ffd700 1 -- 172.21.15.64:0/4154846347 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758011630 con 0x7f57640cae40 2023-09-27T23:50:21.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f5762ffd700 1 -- 172.21.15.64:0/4154846347 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758011790 con 0x7f57640cae40 2023-09-27T23:50:21.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f5762ffd700 1 -- 172.21.15.64:0/4154846347 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758011b30 con 0x7f57640cae40 2023-09-27T23:50:21.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 -- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57640cae40 msgr2=0x7f576406b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57640cae40 0x7f576406b0f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f57580098e0 tx=0x7f57580091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 -- 172.21.15.64:0/4154846347 shutdown_connections 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57640cae40 0x7f576406b0f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 --2- 172.21.15.64:0/4154846347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d0d0 0x7f5764158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 -- 172.21.15.64:0/4154846347 >> 172.21.15.64:0/4154846347 conn(0x7f5764068b40 msgr2=0x7f5764068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 -- 172.21.15.64:0/4154846347 shutdown_connections 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 -- 172.21.15.64:0/4154846347 wait complete. 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 Processor -- start 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.031+0000 7f576a563700 1 -- start start 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 0x7f57640ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f576413d0d0 0x7f576413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 0x7f57640ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5764167e80 con 0x7f57640cae40 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640adbc0 con 0x7f576413d0d0 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5764167270 con 0x7f576413d4a0 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5763fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 0x7f57640ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 0x7f57640ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5763fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 0x7f57640ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47836/0 (socket says 172.21.15.64:47836) 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 0x7f57640ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:51954/0 (socket says 172.21.15.64:51954) 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5763fff700 1 -- 172.21.15.64:0/873069301 learned_addr learned my addr 172.21.15.64:0/873069301 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f57637fe700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f576413d0d0 0x7f576413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 -- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f576413d0d0 msgr2=0x7f576413ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f576413d0d0 0x7f576413ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 -- 172.21.15.64:0/873069301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 msgr2=0x7f57640ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 0x7f57640ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 -- 172.21.15.64:0/873069301 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540093c0 con 0x7f576413d4a0 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5763fff700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 0x7f57640ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f5768b00700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 0x7f57640ac460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f575c00cb90 tx=0x7f575c001980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00a7e0 con 0x7f576413d4a0 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5758009420 con 0x7f576413d4a0 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00a940 con 0x7f576413d4a0 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00acc0 con 0x7f576413d4a0 2023-09-27T23:50:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5764167830 con 0x7f576413d4a0 2023-09-27T23:50:21.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.035+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57440052b0 con 0x7f576413d4a0 2023-09-27T23:50:21.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.039+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f575c007300 con 0x7f576413d4a0 2023-09-27T23:50:21.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.039+0000 7f574bfff700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5750060aa0 0x7f5750062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.039+0000 7f5763fff700 1 -- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5750060aa0 msgr2=0x7f5750062f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:21.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.039+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f575c07cf60 con 0x7f576413d4a0 2023-09-27T23:50:21.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.039+0000 7f5763fff700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5750060aa0 0x7f5750062f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:21.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.051+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f575c049bb0 con 0x7f576413d4a0 2023-09-27T23:50:21.098 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.095+0000 7f6920182000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:50:21.110 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.106+0000 7f110c74d000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:21.175 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.170+0000 7f110c74d000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:50:21.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.211+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5744005ae0 con 0x7f576413d4a0 2023-09-27T23:50:21.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.211+0000 7f574bfff700 1 -- 172.21.15.64:0/873069301 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v161) v1 ==== 100+0+11 (secure 0 0 0) 0x7f575c0170a0 con 0x7f576413d4a0 2023-09-27T23:50:21.214 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:50:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5750060aa0 msgr2=0x7f5750062f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5750060aa0 0x7f5750062f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 msgr2=0x7f57640ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 0x7f57640ac460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f575c00cb90 tx=0x7f575c001980 comp rx=0 tx=0).stop 2023-09-27T23:50:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 shutdown_connections 2023-09-27T23:50:21.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5750060aa0 0x7f5750062f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f576413d4a0 0x7f57640ac460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f576413d0d0 0x7f576413ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 --2- 172.21.15.64:0/873069301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57640cae40 0x7f57640ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 >> 172.21.15.64:0/873069301 conn(0x7f5764068b40 msgr2=0x7f576415a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 shutdown_connections 2023-09-27T23:50:21.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.215+0000 7f576a563700 1 -- 172.21.15.64:0/873069301 wait complete. 2023-09-27T23:50:21.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:21.233 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.230+0000 7f110c74d000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:50:21.269 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.267+0000 7f6920182000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:21.292 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.290+0000 7f110c74d000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:50:21.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.327+0000 7f18335c7700 1 Processor -- start 2023-09-27T23:50:21.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.327+0000 7f18335c7700 1 -- start start 2023-09-27T23:50:21.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f182c0b0110 0x7f182c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0ccfd0 0x7f182c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0ccb40 con 0x7f182c153290 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0cbf30 con 0x7f182c0b0110 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0cc210 con 0x7f182c0ccfd0 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f182c0b0110 0x7f182c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f182c0b0110 0x7f182c0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47844/0 (socket says 172.21.15.64:47844) 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 -- 172.21.15.64:0/2603597037 learned_addr learned my addr 172.21.15.64:0/2603597037 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1830b62700 1 --2- 172.21.15.64:0/2603597037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0ccfd0 0x7f182c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 -- 172.21.15.64:0/2603597037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0ccfd0 msgr2=0x7f182c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 --2- 172.21.15.64:0/2603597037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0ccfd0 0x7f182c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 -- 172.21.15.64:0/2603597037 >> v1:172.21.15.64:6789/0 conn(0x7f182c153290 legacy=0x7f182c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 -- 172.21.15.64:0/2603597037 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0cc4f0 con 0x7f182c0b0110 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1831363700 1 --2- 172.21.15.64:0/2603597037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0b0110 0x7f182c0b04e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f18280099b0 tx=0x7f1828009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32d01f3b304df74 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1823fff700 1 -- 172.21.15.64:0/2603597037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182800b860 con 0x7f182c0b0110 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1823fff700 1 -- 172.21.15.64:0/2603597037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182800b9c0 con 0x7f182c0b0110 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f1823fff700 1 -- 172.21.15.64:0/2603597037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182800bd60 con 0x7f182c0b0110 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- 172.21.15.64:0/2603597037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0b0110 msgr2=0x7f182c0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 --2- 172.21.15.64:0/2603597037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0b0110 0x7f182c0b04e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f18280099b0 tx=0x7f1828009250 comp rx=0 tx=0).stop 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- 172.21.15.64:0/2603597037 shutdown_connections 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 --2- 172.21.15.64:0/2603597037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c0ccfd0 0x7f182c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 --2- 172.21.15.64:0/2603597037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0b0110 0x7f182c0b04e0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- 172.21.15.64:0/2603597037 >> 172.21.15.64:0/2603597037 conn(0x7f182c06c7c0 msgr2=0x7f182c06cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- 172.21.15.64:0/2603597037 shutdown_connections 2023-09-27T23:50:21.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.331+0000 7f18335c7700 1 -- 172.21.15.64:0/2603597037 wait complete. 2023-09-27T23:50:21.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 Processor -- start 2023-09-27T23:50:21.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- start start 2023-09-27T23:50:21.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b0110 0x7f182c169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 0x7f182c151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c153290 0x7f182c0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c158b60 con 0x7f182c0ccfd0 2023-09-27T23:50:21.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c06aef0 con 0x7f182c0b0110 2023-09-27T23:50:21.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c158db0 con 0x7f182c153290 2023-09-27T23:50:21.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 0x7f182c151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 0x7f182c151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47858/0 (socket says 172.21.15.64:47858) 2023-09-27T23:50:21.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 -- 172.21.15.64:0/3448709235 learned_addr learned my addr 172.21.15.64:0/3448709235 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:21.342 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f6920182000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:50:21.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1831b64700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c153290 0x7f182c0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1831363700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b0110 0x7f182c169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 -- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c153290 msgr2=0x7f182c0c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c153290 0x7f182c0c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 -- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b0110 msgr2=0x7f182c169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b0110 0x7f182c169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 -- 172.21.15.64:0/3448709235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280094f0 con 0x7f182c0ccfd0 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f1830b62700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 0x7f182c151f60 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f181c00d890 tx=0x7f181c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00b600 con 0x7f182c0ccfd0 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00b760 con 0x7f182c0ccfd0 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c159090 con 0x7f182c0ccfd0 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00bae0 con 0x7f182c0ccfd0 2023-09-27T23:50:21.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c169670 con 0x7f182c0ccfd0 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.335+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c071120 con 0x7f182c0ccfd0 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.339+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f181c01e460 con 0x7f182c0ccfd0 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.339+0000 7f18217fa700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 0x7f1810062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.339+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f181c07dad0 con 0x7f182c0ccfd0 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.339+0000 7f1831363700 1 -- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 msgr2=0x7f1810062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.339+0000 7f1831363700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 0x7f1810062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:21.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.343+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f181c04d880 con 0x7f182c0ccfd0 2023-09-27T23:50:21.392 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.391+0000 7f6920182000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:50:21.449 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.447+0000 7f6920182000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:50:21.486 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.482+0000 7f110c74d000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:21.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.539+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f182c0c8700 con 0x7f182c0ccfd0 2023-09-27T23:50:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.539+0000 7f18217fa700 1 -- 172.21.15.64:0/3448709235 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f181c050ea0 con 0x7f182c0ccfd0 2023-09-27T23:50:21.542 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:21.542 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:56.514725+0000","modified":"2023-09-27T23:33:58.531554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3161207024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3161207024},{"type":"v1","addr":"172.21.15.64:6835","nonce":3161207024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-09-27T23:50:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f1831363700 1 -- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 msgr2=0x7f1810062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:21.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f1831363700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 0x7f1810062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:50:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 msgr2=0x7f1810062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 0x7f1810062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 msgr2=0x7f182c151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 0x7f182c151f60 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f181c00d890 tx=0x7f181c008040 comp rx=0 tx=0).stop 2023-09-27T23:50:21.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 shutdown_connections 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1810060a70 0x7f1810062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f182c153290 0x7f182c0c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f182c0ccfd0 0x7f182c151f60 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 --2- 172.21.15.64:0/3448709235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f182c0b0110 0x7f182c169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 >> 172.21.15.64:0/3448709235 conn(0x7f182c06c7c0 msgr2=0x7f182c15a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 shutdown_connections 2023-09-27T23:50:21.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.543+0000 7f18335c7700 1 -- 172.21.15.64:0/3448709235 wait complete. 2023-09-27T23:50:21.547 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.542+0000 7f110c74d000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:50:21.547 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 60 2023-09-27T23:50:21.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:21.614 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.610+0000 7f110c74d000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:50:21.629 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.627+0000 7f6920182000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:21.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.659+0000 7f3b7ce40700 1 Processor -- start 2023-09-27T23:50:21.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.659+0000 7f3b7ce40700 1 -- start start 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3b780b2310 0x7f3b7815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b780c8e80 0x7f3b78067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b780670e0 con 0x7f3b780b26e0 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7815a700 con 0x7f3b780c8e80 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b780676a0 con 0x7f3b780b2310 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b77fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3b780c8e80 0x7f3b78067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b76ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3b780b26e0 0x7f3b780c8770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37184/0 (socket says 172.21.15.64:37184) 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b76ffd700 1 -- 172.21.15.64:0/1284700215 learned_addr learned my addr 172.21.15.64:0/1284700215 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b777fe700 1 --2- 172.21.15.64:0/1284700215 >> v2:172.21.15.123:3300/0 conn(0x7f3b780b2310 0x7f3b7815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b77fff700 1 -- 172.21.15.64:0/1284700215 >> v2:172.21.15.123:3300/0 conn(0x7f3b780b2310 msgr2=0x7f3b7815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b77fff700 1 --2- 172.21.15.64:0/1284700215 >> v2:172.21.15.123:3300/0 conn(0x7f3b780b2310 0x7f3b7815a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b77fff700 1 -- 172.21.15.64:0/1284700215 >> v1:172.21.15.64:6789/0 conn(0x7f3b780b26e0 legacy=0x7f3b780c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b767fc700 1 -- 172.21.15.64:0/1284700215 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981018671 0 0) 0x7f3b780670e0 con 0x7f3b780b26e0 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b77fff700 1 -- 172.21.15.64:0/1284700215 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780c3510 con 0x7f3b780c8e80 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b77fff700 1 --2- 172.21.15.64:0/1284700215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780c8e80 0x7f3b78067b30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3b78064490 tx=0x7f3b70009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1741265b0fa050d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b767fc700 1 -- 172.21.15.64:0/1284700215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000e070 con 0x7f3b780c8e80 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b767fc700 1 -- 172.21.15.64:0/1284700215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7000bc30 con 0x7f3b780c8e80 2023-09-27T23:50:21.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b767fc700 1 -- 172.21.15.64:0/1284700215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70012430 con 0x7f3b780c8e80 2023-09-27T23:50:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1284700215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780c8e80 msgr2=0x7f3b78067b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1284700215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780c8e80 0x7f3b78067b30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3b78064490 tx=0x7f3b70009780 comp rx=0 tx=0).stop 2023-09-27T23:50:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1284700215 shutdown_connections 2023-09-27T23:50:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1284700215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780c8e80 0x7f3b78067b30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1284700215 >> v2:172.21.15.123:3300/0 conn(0x7f3b780b2310 0x7f3b7815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1284700215 >> 172.21.15.64:0/1284700215 conn(0x7f3b7806c080 msgr2=0x7f3b780ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1284700215 shutdown_connections 2023-09-27T23:50:21.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.663+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1284700215 wait complete. 2023-09-27T23:50:21.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 Processor -- start 2023-09-27T23:50:21.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 -- start start 2023-09-27T23:50:21.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b780b2310 0x7f3b78153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 0x7f3b78154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b780c8e80 0x7f3b78154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78168b50 con 0x7f3b780b26e0 2023-09-27T23:50:21.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78169b20 con 0x7f3b780c8e80 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78155110 con 0x7f3b780b2310 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 0x7f3b78154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 0x7f3b78154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47880/0 (socket says 172.21.15.64:47880) 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 -- 172.21.15.64:0/1218578685 learned_addr learned my addr 172.21.15.64:0/1218578685 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b777fe700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b780b2310 0x7f3b78153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b77fff700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b780c8e80 0x7f3b78154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 -- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b780b2310 msgr2=0x7f3b78153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b780b2310 0x7f3b78153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 -- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b780c8e80 msgr2=0x7f3b78154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b780c8e80 0x7f3b78154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 -- 172.21.15.64:0/1218578685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70009a20 con 0x7f3b780b26e0 2023-09-27T23:50:21.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b76ffd700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 0x7f3b78154570 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00c810 tx=0x7f3b6c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c005c00 con 0x7f3b780b26e0 2023-09-27T23:50:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c005d60 con 0x7f3b780b26e0 2023-09-27T23:50:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b78168d60 con 0x7f3b780b26e0 2023-09-27T23:50:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c007300 con 0x7f3b780b26e0 2023-09-27T23:50:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.667+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b78169600 con 0x7f3b780b26e0 2023-09-27T23:50:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.671+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b78169850 con 0x7f3b780b26e0 2023-09-27T23:50:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.671+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f3b6c01b3b0 con 0x7f3b780b26e0 2023-09-27T23:50:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.671+0000 7f3b5f7fe700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3b64060a70 0x7f3b64062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.671+0000 7f3b777fe700 1 -- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3b64060a70 msgr2=0x7f3b64062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.671+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f3b6c07c890 con 0x7f3b780b26e0 2023-09-27T23:50:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.671+0000 7f3b777fe700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3b64060a70 0x7f3b64062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:21.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.679+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3b6c04c640 con 0x7f3b780b26e0 2023-09-27T23:50:21.688 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.687+0000 7f6920182000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:50:21.753 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.751+0000 7f6920182000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:50:21.779 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.774+0000 7f110c74d000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:50:21.844 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:21.842+0000 7f110c74d000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:50:21.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.843+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b78168f50 con 0x7f3b780b26e0 2023-09-27T23:50:21.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.847+0000 7f3b5f7fe700 1 -- 172.21.15.64:0/1218578685 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+18011 (secure 0 0 0) 0x7f3b6c04fc60 con 0x7f3b780b26e0 2023-09-27T23:50:21.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:21.850 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":161,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:45:19.819278+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:55.078420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:55.503601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2a","mappings":[{"from":4,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3448188613":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1602424936":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1684150586":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1316115001":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6833/33228":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/1077033381":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6832/33228":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/1459316426":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2445385630":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2621537395":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/4021019210":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/181603911":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2431874217":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/2124381486":"2023-09-28T23:34:12.030420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:21.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3b64060a70 msgr2=0x7f3b64062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:21.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3b64060a70 0x7f3b64062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 msgr2=0x7f3b78154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 0x7f3b78154570 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00c810 tx=0x7f3b6c007db0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 shutdown_connections 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3b64060a70 0x7f3b64062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b780c8e80 0x7f3b78154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b780b26e0 0x7f3b78154570 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 --2- 172.21.15.64:0/1218578685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b780b2310 0x7f3b78153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 >> 172.21.15.64:0/1218578685 conn(0x7f3b7806c080 msgr2=0x7f3b780ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 shutdown_connections 2023-09-27T23:50:21.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.851+0000 7f3b7ce40700 1 -- 172.21.15.64:0/1218578685 wait complete. 2023-09-27T23:50:21.869 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:50:21.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:21.911 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.907+0000 7f6920182000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:50:21.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 Processor -- start 2023-09-27T23:50:21.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 -- start start 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbba406e1d0 0x7fbba4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbba406ad70 0x7fbba406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba414f4d0 0x7fbba414f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba406b750 con 0x7fbba406e1d0 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba415a260 con 0x7fbba406ad70 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba88ad700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40afad0 con 0x7fbba414f4d0 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbba406e1d0 0x7fbba4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbba406e1d0 0x7fbba4159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47894/0 (socket says 172.21.15.64:47894) 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 -- 172.21.15.64:0/1241417779 learned_addr learned my addr 172.21.15.64:0/1241417779 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba359e700 1 --2- 172.21.15.64:0/1241417779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba414f4d0 0x7fbba414f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba259c700 1 --2- 172.21.15.64:0/1241417779 >> v2:172.21.15.123:3300/0 conn(0x7fbba406ad70 0x7fbba406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 -- 172.21.15.64:0/1241417779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba414f4d0 msgr2=0x7fbba414f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 --2- 172.21.15.64:0/1241417779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba414f4d0 0x7fbba414f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 -- 172.21.15.64:0/1241417779 >> v2:172.21.15.123:3300/0 conn(0x7fbba406ad70 msgr2=0x7fbba406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 --2- 172.21.15.64:0/1241417779 >> v2:172.21.15.123:3300/0 conn(0x7fbba406ad70 0x7fbba406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 -- 172.21.15.64:0/1241417779 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40afdb0 con 0x7fbba406e1d0 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba259c700 1 --2- 172.21.15.64:0/1241417779 >> v2:172.21.15.123:3300/0 conn(0x7fbba406ad70 0x7fbba406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba2d9d700 1 --2- 172.21.15.64:0/1241417779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406e1d0 0x7fbba4159d20 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fbba40af460 tx=0x7fbb9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae66d11506be937 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.959+0000 7fbba1d9b700 1 -- 172.21.15.64:0/1241417779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00e070 con 0x7fbba406e1d0 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba1d9b700 1 -- 172.21.15.64:0/1241417779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c00ba20 con 0x7fbba406e1d0 2023-09-27T23:50:21.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba1d9b700 1 -- 172.21.15.64:0/1241417779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0092e0 con 0x7fbba406e1d0 2023-09-27T23:50:21.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- 172.21.15.64:0/1241417779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406e1d0 msgr2=0x7fbba4159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- 172.21.15.64:0/1241417779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406e1d0 0x7fbba4159d20 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fbba40af460 tx=0x7fbb9c009840 comp rx=0 tx=0).stop 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- 172.21.15.64:0/1241417779 shutdown_connections 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- 172.21.15.64:0/1241417779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba414f4d0 0x7fbba414f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- 172.21.15.64:0/1241417779 >> v2:172.21.15.123:3300/0 conn(0x7fbba406ad70 0x7fbba406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- 172.21.15.64:0/1241417779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406e1d0 0x7fbba4159d20 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- 172.21.15.64:0/1241417779 >> 172.21.15.64:0/1241417779 conn(0x7fbba4067bb0 msgr2=0x7fbba40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- 172.21.15.64:0/1241417779 shutdown_connections 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- 172.21.15.64:0/1241417779 wait complete. 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 Processor -- start 2023-09-27T23:50:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- start start 2023-09-27T23:50:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 0x7fbba4148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba406e1d0 0x7fbba414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbba414f4d0 0x7fbba415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba414cb20 con 0x7fbba406ad70 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4148d20 con 0x7fbba414f4d0 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.963+0000 7fbba88ad700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4149000 con 0x7fbba406e1d0 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 0x7fbba4148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 0x7fbba4148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47902/0 (socket says 172.21.15.64:47902) 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 -- 172.21.15.64:0/3464152237 learned_addr learned my addr 172.21.15.64:0/3464152237 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba259c700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba406e1d0 0x7fbba414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba359e700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbba414f4d0 0x7fbba415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 -- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba406e1d0 msgr2=0x7fbba414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba406e1d0 0x7fbba414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 -- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbba414f4d0 msgr2=0x7fbba415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbba414f4d0 0x7fbba415ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 -- 172.21.15.64:0/3464152237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c009ae0 con 0x7fbba406ad70 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba359e700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbba414f4d0 0x7fbba415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba2d9d700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 0x7fbba4148690 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbba4069fb0 tx=0x7fbb9c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c008b80 con 0x7fbba406ad70 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c008d00 con 0x7fbba406ad70 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba41492e0 con 0x7fbba406ad70 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0266b0 con 0x7fbba406ad70 2023-09-27T23:50:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba41498a0 con 0x7fbba406ad70 2023-09-27T23:50:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7fbb9c003040 con 0x7fbba406ad70 2023-09-27T23:50:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba4168ce0 con 0x7fbba406ad70 2023-09-27T23:50:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbb9affd700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 0x7fbb84062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba259c700 1 -- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 msgr2=0x7fbb84062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fbb9c07f460 con 0x7fbba406ad70 2023-09-27T23:50:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.967+0000 7fbba259c700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 0x7fbb84062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:21.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:21.975+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbb9c04f060 con 0x7fbba406ad70 2023-09-27T23:50:21.978 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:21.975+0000 7f6920182000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:50:22.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.167+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbba404be40 con 0x7fbba406ad70 2023-09-27T23:50:22.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.167+0000 7fbb9affd700 1 -- 172.21.15.64:0/3464152237 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fbb9c052680 con 0x7fbba406ad70 2023-09-27T23:50:22.171 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:22.171 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:56.514725+0000","modified":"2023-09-27T23:33:58.531554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3161207024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3161207024},{"type":"v1","addr":"172.21.15.64:6835","nonce":3161207024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-09-27T23:50:22.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba259c700 1 -- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 msgr2=0x7fbb84062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:22.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba259c700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 0x7fbb84062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 msgr2=0x7fbb84062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 0x7fbb84062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 msgr2=0x7fbba4148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 0x7fbba4148690 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbba4069fb0 tx=0x7fbb9c018000 comp rx=0 tx=0).stop 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 shutdown_connections 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb84060a70 0x7fbb84062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbba414f4d0 0x7fbba415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbba406e1d0 0x7fbba414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 --2- 172.21.15.64:0/3464152237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba406ad70 0x7fbba4148690 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 >> 172.21.15.64:0/3464152237 conn(0x7fbba4067bb0 msgr2=0x7fbba406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:22.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 shutdown_connections 2023-09-27T23:50:22.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.171+0000 7fbba88ad700 1 -- 172.21.15.64:0/3464152237 wait complete. 2023-09-27T23:50:22.175 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 60 2023-09-27T23:50:22.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 Processor -- start 2023-09-27T23:50:22.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- start start 2023-09-27T23:50:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f124c156c90 0x7f124c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0cd5a0 con 0x7f124c157060 2023-09-27T23:50:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0cc990 con 0x7f124c0cda30 2023-09-27T23:50:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c157470 con 0x7f124c156c90 2023-09-27T23:50:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252348700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f124c156c90 0x7f124c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252b49700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f124c157060 0x7f124c15ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37192/0 (socket says 172.21.15.64:37192) 2023-09-27T23:50:22.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252b49700 1 -- 172.21.15.64:0/1781886527 learned_addr learned my addr 172.21.15.64:0/1781886527 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252348700 1 -- 172.21.15.64:0/1781886527 >> v1:172.21.15.123:6789/0 conn(0x7f124c0cda30 legacy=0x7f124c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252348700 1 -- 172.21.15.64:0/1781886527 >> v1:172.21.15.64:6789/0 conn(0x7f124c157060 legacy=0x7f124c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252348700 1 -- 172.21.15.64:0/1781886527 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0ccc70 con 0x7f124c156c90 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1252348700 1 --2- 172.21.15.64:0/1781886527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c0cc3b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f12480098e0 tx=0x7f12480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23640b3e6cad24e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1251346700 1 -- 172.21.15.64:0/1781886527 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248009e80 con 0x7f124c156c90 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1251346700 1 -- 172.21.15.64:0/1781886527 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124800b760 con 0x7f124c156c90 2023-09-27T23:50:22.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f1251346700 1 -- 172.21.15.64:0/1781886527 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124800bb00 con 0x7f124c156c90 2023-09-27T23:50:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- 172.21.15.64:0/1781886527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 msgr2=0x7f124c0cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 --2- 172.21.15.64:0/1781886527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c0cc3b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f12480098e0 tx=0x7f12480091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- 172.21.15.64:0/1781886527 shutdown_connections 2023-09-27T23:50:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 --2- 172.21.15.64:0/1781886527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c0cc3b0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- 172.21.15.64:0/1781886527 >> 172.21.15.64:0/1781886527 conn(0x7f124c06c7c0 msgr2=0x7f124c06cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- 172.21.15.64:0/1781886527 shutdown_connections 2023-09-27T23:50:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.287+0000 7f12545ac700 1 -- 172.21.15.64:0/1781886527 wait complete. 2023-09-27T23:50:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 Processor -- start 2023-09-27T23:50:22.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 -- start start 2023-09-27T23:50:22.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0cda30 0x7f124c16d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c157060 0x7f124c0b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c1595c0 con 0x7f124c156c90 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c1589b0 con 0x7f124c157060 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c158c90 con 0x7f124c0cda30 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47912/0 (socket says 172.21.15.64:47912) 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 -- 172.21.15.64:0/3898094801 learned_addr learned my addr 172.21.15.64:0/3898094801 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1252b49700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c157060 0x7f124c0b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1252348700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0cda30 0x7f124c16d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 -- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0cda30 msgr2=0x7f124c16d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0cda30 0x7f124c16d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 -- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c157060 msgr2=0x7f124c0b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c157060 0x7f124c0b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 -- 172.21.15.64:0/3898094801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1248009420 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f1251b47700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c157f80 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c5d0 tx=0x7f123c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c011590 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c011710 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c158f70 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c011ad0 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.291+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c16d090 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.295+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c071120 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.295+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 77) v1 ==== 80373+0+0 (secure 0 0 0) 0x7f123c01c460 con 0x7f124c156c90 2023-09-27T23:50:22.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.295+0000 7f12427fc700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 0x7f1230062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.295+0000 7f1252348700 1 -- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 msgr2=0x7f1230062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.295+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f123c07d040 con 0x7f124c156c90 2023-09-27T23:50:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.295+0000 7f1252348700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 0x7f1230062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:22.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.299+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f123c04cdf0 con 0x7f124c156c90 2023-09-27T23:50:22.466 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.462+0000 7f110c74d000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:50:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.499+0000 7f1252348700 1 -- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 msgr2=0x7f1230062f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.499+0000 7f1252348700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 0x7f1230062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:50:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.507+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f124c0ccf50 con 0x7f124c156c90 2023-09-27T23:50:22.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.507+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 78) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f123c04d1c0 con 0x7f124c156c90 2023-09-27T23:50:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.511+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 msgr2=0x7f1230062f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:22.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.511+0000 7f12427fc700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 0x7f1230062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.519+0000 7f12427fc700 1 -- 172.21.15.64:0/3898094801 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f123c068b80 con 0x7f124c156c90 2023-09-27T23:50:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:22.524 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:33:56.514725+0000","modified":"2023-09-27T23:33:58.531554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3161207024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3161207024},{"type":"v1","addr":"172.21.15.64:6835","nonce":3161207024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-09-27T23:50:22.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 msgr2=0x7f124c157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c157f80 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c5d0 tx=0x7f123c007f60 comp rx=0 tx=0).stop 2023-09-27T23:50:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 shutdown_connections 2023-09-27T23:50:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1230060a70 0x7f1230062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c157060 0x7f124c0b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f124c156c90 0x7f124c157f80 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 --2- 172.21.15.64:0/3898094801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0cda30 0x7f124c16d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 >> 172.21.15.64:0/3898094801 conn(0x7f124c06c7c0 msgr2=0x7f124c06cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 shutdown_connections 2023-09-27T23:50:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.535+0000 7f12545ac700 1 -- 172.21.15.64:0/3898094801 wait complete. 2023-09-27T23:50:22.551 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 60 2023-09-27T23:50:22.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:22.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.626+0000 7f10d5188700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.626+0000 7f10d5188700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.626+0000 7f10d5188700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.626+0000 7f10d5188700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.631 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.626+0000 7f10d5188700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.642 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.638+0000 7f10da192700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.642 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.638+0000 7f10da192700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.643 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.638+0000 7f10da192700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.643 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.638+0000 7f10da192700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.643 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:22.638+0000 7f10da192700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:22.700 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:22.699+0000 7f6920182000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:50:22.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.735+0000 7ffa46cba700 1 Processor -- start 2023-09-27T23:50:22.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.735+0000 7ffa46cba700 1 -- start start 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400c97d0 0x7ffa400b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffa400b13b0 0x7ffa4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400c9c20 con 0x7ffa40155990 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa400ae7e0 con 0x7ffa400b13b0 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400cbc50 con 0x7ffa400c97d0 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa45cb8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa40155990 0x7ffa400b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37202/0 (socket says 172.21.15.64:37202) 2023-09-27T23:50:22.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa45cb8700 1 -- 172.21.15.64:0/3368664120 learned_addr learned my addr 172.21.15.64:0/3368664120 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa464b9700 1 --2- 172.21.15.64:0/3368664120 >> v2:172.21.15.64:3300/0 conn(0x7ffa400b13b0 0x7ffa4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa454b7700 1 --2- 172.21.15.64:0/3368664120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400c97d0 0x7ffa400b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa44cb6700 1 -- 172.21.15.64:0/3368664120 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223054067 0 0) 0x7ffa400c9c20 con 0x7ffa40155990 2023-09-27T23:50:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa44cb6700 1 -- 172.21.15.64:0/3368664120 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa2c003570 con 0x7ffa40155990 2023-09-27T23:50:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa454b7700 1 -- 172.21.15.64:0/3368664120 >> v2:172.21.15.64:3300/0 conn(0x7ffa400b13b0 msgr2=0x7ffa4006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:22.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa454b7700 1 --2- 172.21.15.64:0/3368664120 >> v2:172.21.15.64:3300/0 conn(0x7ffa400b13b0 0x7ffa4006d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa454b7700 1 -- 172.21.15.64:0/3368664120 >> v1:172.21.15.64:6789/0 conn(0x7ffa40155990 legacy=0x7ffa400b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa454b7700 1 -- 172.21.15.64:0/3368664120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4006cf00 con 0x7ffa400c97d0 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa464b9700 1 --2- 172.21.15.64:0/3368664120 >> v2:172.21.15.64:3300/0 conn(0x7ffa400b13b0 0x7ffa4006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa454b7700 1 --2- 172.21.15.64:0/3368664120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400c97d0 0x7ffa400b0e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffa300098e0 tx=0x7ffa300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fa3069971958c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa44cb6700 1 -- 172.21.15.64:0/3368664120 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011630 con 0x7ffa400c97d0 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa44cb6700 1 -- 172.21.15.64:0/3368664120 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa30011790 con 0x7ffa400c97d0 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa44cb6700 1 -- 172.21.15.64:0/3368664120 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011b10 con 0x7ffa400c97d0 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- 172.21.15.64:0/3368664120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400c97d0 msgr2=0x7ffa400b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 --2- 172.21.15.64:0/3368664120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400c97d0 0x7ffa400b0e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffa300098e0 tx=0x7ffa300091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- 172.21.15.64:0/3368664120 shutdown_connections 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 --2- 172.21.15.64:0/3368664120 >> v2:172.21.15.64:3300/0 conn(0x7ffa400b13b0 0x7ffa4006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 --2- 172.21.15.64:0/3368664120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa400c97d0 0x7ffa400b0e70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- 172.21.15.64:0/3368664120 >> 172.21.15.64:0/3368664120 conn(0x7ffa40067bb0 msgr2=0x7ffa400add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:22.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- 172.21.15.64:0/3368664120 shutdown_connections 2023-09-27T23:50:22.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- 172.21.15.64:0/3368664120 wait complete. 2023-09-27T23:50:22.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 Processor -- start 2023-09-27T23:50:22.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.739+0000 7ffa46cba700 1 -- start start 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 0x7ffa40150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400c97d0 0x7ffa4014ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa40155990 0x7ffa4014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400c65f0 con 0x7ffa400b13b0 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400c6160 con 0x7ffa400c97d0 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400c5550 con 0x7ffa40155990 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 0x7ffa40150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 0x7ffa40150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47924/0 (socket says 172.21.15.64:47924) 2023-09-27T23:50:22.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 -- 172.21.15.64:0/961620705 learned_addr learned my addr 172.21.15.64:0/961620705 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa464b9700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa40155990 0x7ffa4014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa454b7700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400c97d0 0x7ffa4014ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 -- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa40155990 msgr2=0x7ffa4014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa40155990 0x7ffa4014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 -- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400c97d0 msgr2=0x7ffa4014ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400c97d0 0x7ffa4014ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 -- 172.21.15.64:0/961620705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38009a20 con 0x7ffa400b13b0 2023-09-27T23:50:22.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa45cb8700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 0x7ffa40150ee0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0011e0 tx=0x7ffa3c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c005b30 con 0x7ffa400b13b0 2023-09-27T23:50:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c005c90 con 0x7ffa400b13b0 2023-09-27T23:50:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa30009420 con 0x7ffa400b13b0 2023-09-27T23:50:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c007320 con 0x7ffa400b13b0 2023-09-27T23:50:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.743+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa400c5b10 con 0x7ffa400b13b0 2023-09-27T23:50:22.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.747+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4004be40 con 0x7ffa400b13b0 2023-09-27T23:50:22.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.747+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 78) v1 ==== 79970+0+0 (secure 0 0 0) 0x7ffa3c007690 con 0x7ffa400b13b0 2023-09-27T23:50:22.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.747+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 7409+0+0 (secure 0 0 0) 0x7ffa3c07c240 con 0x7ffa400b13b0 2023-09-27T23:50:22.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.751+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ffa3c04c3f0 con 0x7ffa400b13b0 2023-09-27T23:50:22.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.915+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa400afef0 con 0x7ffa400b13b0 2023-09-27T23:50:22.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.915+0000 7ffa35ffb700 1 -- 172.21.15.64:0/961620705 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+18624 (secure 0 0 0) 0x7ffa3c017090 con 0x7ffa400b13b0 2023-09-27T23:50:22.919 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:22.919 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":162,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:50:22.487224+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:33:55.078420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-09-27T23:33:55.503601+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":162,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2a","mappings":[{"from":4,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/4101010910":"2023-09-28T23:50:22.487141+0000","172.21.15.123:6833/43188":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/1086060816":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/3448188613":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1602424936":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6832/43188":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/1684150586":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/1316115001":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6833/33228":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/1077033381":"2023-09-28T23:45:19.819177+0000","172.21.15.123:6832/33228":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/1459316426":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/527922338":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/2280107059":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/2445385630":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/2621537395":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/3210548590":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/4021019210":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/181603911":"2023-09-28T23:34:12.030420+0000","172.21.15.123:0/3759085274":"2023-09-28T23:50:22.487141+0000","172.21.15.123:0/2431874217":"2023-09-28T23:45:19.819177+0000","172.21.15.123:0/2124381486":"2023-09-28T23:34:12.030420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:22.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 msgr2=0x7ffa40150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:22.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 0x7ffa40150ee0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0011e0 tx=0x7ffa3c007df0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 shutdown_connections 2023-09-27T23:50:22.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa40155990 0x7ffa4014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa400c97d0 0x7ffa4014ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 --2- 172.21.15.64:0/961620705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa400b13b0 0x7ffa40150ee0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:22.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 >> 172.21.15.64:0/961620705 conn(0x7ffa40067bb0 msgr2=0x7ffa400af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:22.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 shutdown_connections 2023-09-27T23:50:22.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:22.919+0000 7ffa46cba700 1 -- 172.21.15.64:0/961620705 wait complete. 2023-09-27T23:50:22.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:50:23.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:50:23.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:50:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 Processor -- start 2023-09-27T23:50:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 -- start start 2023-09-27T23:50:23.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc81c0af380 0x7fc81c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:23.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc81c0b0610 0x7fc81c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:23.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c06b3f0 con 0x7fc81c155b50 2023-09-27T23:50:23.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c0b0f90 con 0x7fc81c0b0610 2023-09-27T23:50:23.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc823b20700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c06a7e0 con 0x7fc81c0af380 2023-09-27T23:50:23.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc8220bd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc81c0b0610 0x7fc81c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:23.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc8220bd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc81c0b0610 0x7fc81c0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47928/0 (socket says 172.21.15.64:47928) 2023-09-27T23:50:23.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc8220bd700 1 -- 172.21.15.64:0/1137144775 learned_addr learned my addr 172.21.15.64:0/1137144775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:23.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.043+0000 7fc8210bb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc81c0af380 0x7fc81c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:23.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8220bd700 1 -- 172.21.15.64:0/1137144775 >> v2:172.21.15.123:3300/0 conn(0x7fc81c0af380 msgr2=0x7fc81c0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:23.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8220bd700 1 --2- 172.21.15.64:0/1137144775 >> v2:172.21.15.123:3300/0 conn(0x7fc81c0af380 0x7fc81c0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:23.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8220bd700 1 -- 172.21.15.64:0/1137144775 >> v1:172.21.15.123:6789/0 conn(0x7fc81c155b50 legacy=0x7fc81c06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:23.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8220bd700 1 -- 172.21.15.64:0/1137144775 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c06aac0 con 0x7fc81c0b0610 2023-09-27T23:50:23.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8220bd700 1 --2- 172.21.15.64:0/1137144775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0b0610 0x7fc81c0b0a50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc81c06a6f0 tx=0x7fc814009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53362c59c65c3bf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8208ba700 1 -- 172.21.15.64:0/1137144775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81400e070 con 0x7fc81c0b0610 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8208ba700 1 -- 172.21.15.64:0/1137144775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81400ba20 con 0x7fc81c0b0610 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc8208ba700 1 -- 172.21.15.64:0/1137144775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140092e0 con 0x7fc81c0b0610 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 -- 172.21.15.64:0/1137144775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0b0610 msgr2=0x7fc81c0b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 --2- 172.21.15.64:0/1137144775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0b0610 0x7fc81c0b0a50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc81c06a6f0 tx=0x7fc814009840 comp rx=0 tx=0).stop 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 -- 172.21.15.64:0/1137144775 shutdown_connections 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 --2- 172.21.15.64:0/1137144775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0b0610 0x7fc81c0b0a50 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 --2- 172.21.15.64:0/1137144775 >> v2:172.21.15.123:3300/0 conn(0x7fc81c0af380 0x7fc81c0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 -- 172.21.15.64:0/1137144775 >> 172.21.15.64:0/1137144775 conn(0x7fc81c067bb0 msgr2=0x7fc81c0ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 -- 172.21.15.64:0/1137144775 shutdown_connections 2023-09-27T23:50:23.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 -- 172.21.15.64:0/1137144775 wait complete. 2023-09-27T23:50:23.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 Processor -- start 2023-09-27T23:50:23.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.047+0000 7fc823b20700 1 -- start start 2023-09-27T23:50:23.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 0x7fc81c13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:23.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc81c0b0610 0x7fc81c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:23.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc81c155b50 0x7fc81c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:23.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0c3370 con 0x7fc81c0af380 2023-09-27T23:50:23.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0c8170 con 0x7fc81c155b50 2023-09-27T23:50:23.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0c2a40 con 0x7fc81c0b0610 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 0x7fc81c13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 0x7fc81c13ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47930/0 (socket says 172.21.15.64:47930) 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 -- 172.21.15.64:0/3259448651 learned_addr learned my addr 172.21.15.64:0/3259448651 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8210bb700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc81c0b0610 0x7fc81c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8220bd700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc81c155b50 0x7fc81c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 -- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc81c0b0610 msgr2=0x7fc81c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc81c0b0610 0x7fc81c1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 -- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc81c155b50 msgr2=0x7fc81c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc81c155b50 0x7fc81c154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 -- 172.21.15.64:0/3259448651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814009ae0 con 0x7fc81c0af380 2023-09-27T23:50:23.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc8218bc700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 0x7fc81c13ed00 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc81800e9d0 tx=0x7fc81800c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:23.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180106e0 con 0x7fc81c0af380 2023-09-27T23:50:23.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818010860 con 0x7fc81c0af380 2023-09-27T23:50:23.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c0c2d20 con 0x7fc81c0af380 2023-09-27T23:50:23.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818010c20 con 0x7fc81c0af380 2023-09-27T23:50:23.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c13f240 con 0x7fc81c0af380 2023-09-27T23:50:23.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.051+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c06dda0 con 0x7fc81c0af380 2023-09-27T23:50:23.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.055+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 78) v1 ==== 79970+0+0 (secure 0 0 0) 0x7fc818017030 con 0x7fc81c0af380 2023-09-27T23:50:23.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.055+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 7409+0+0 (secure 0 0 0) 0x7fc818015070 con 0x7fc81c0af380 2023-09-27T23:50:23.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.063+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc81c06dda0 con 0x7fc81c0af380 2023-09-27T23:50:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.247+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc81c06dda0 con 0x7fc81c0af380 2023-09-27T23:50:23.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.503+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 79) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fc81804ec80 con 0x7fc81c0af380 2023-09-27T23:50:23.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.503+0000 7fc811ffb700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc800067100 0x7fc8000694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:23.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.507+0000 7fc8210bb700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc800067100 0x7fc8000694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:23.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:23.511+0000 7fc8210bb700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc800067100 0x7fc8000694e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc81c145df0 tx=0x7fc80c0091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:24.067 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:50:24.063+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:50:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.079+0000 7fc811ffb700 1 -- 172.21.15.64:0/3259448651 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc81806b560 con 0x7fc81c0af380 2023-09-27T23:50:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc800067100 msgr2=0x7fc8000694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc800067100 0x7fc8000694e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc81c145df0 tx=0x7fc80c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 msgr2=0x7fc81c13ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 0x7fc81c13ed00 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc81800e9d0 tx=0x7fc81800c300 comp rx=0 tx=0).stop 2023-09-27T23:50:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 shutdown_connections 2023-09-27T23:50:24.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc800067100 0x7fc8000694e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc81c155b50 0x7fc81c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc81c0b0610 0x7fc81c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 --2- 172.21.15.64:0/3259448651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc81c0af380 0x7fc81c13ed00 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 >> 172.21.15.64:0/3259448651 conn(0x7fc81c067bb0 msgr2=0x7fc81c0c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 shutdown_connections 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.087+0000 7fc823b20700 1 -- 172.21.15.64:0/3259448651 wait complete. 2023-09-27T23:50:24.090 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:50:24.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:50:24.131 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:50:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 Processor -- start 2023-09-27T23:50:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 -- start start 2023-09-27T23:50:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c28158e20 0x7f5c280ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 0x7f5c280ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c280ca2c0 0x7f5c2806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280cbdf0 con 0x7f5c280cb970 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280cc0d0 con 0x7f5c280ca2c0 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2f8ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280cc3b0 con 0x7f5c28158e20 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2ce55700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 0x7f5c280ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2d656700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c28158e20 0x7f5c280ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2ce55700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 0x7f5c280ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47952/0 (socket says 172.21.15.64:47952) 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2ce55700 1 -- 172.21.15.64:0/3619591681 learned_addr learned my addr 172.21.15.64:0/3619591681 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.215+0000 7f5c2de57700 1 --2- 172.21.15.64:0/3619591681 >> v2:172.21.15.123:3300/0 conn(0x7f5c280ca2c0 0x7f5c2806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2d656700 1 -- 172.21.15.64:0/3619591681 >> v2:172.21.15.123:3300/0 conn(0x7f5c280ca2c0 msgr2=0x7f5c2806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2d656700 1 --2- 172.21.15.64:0/3619591681 >> v2:172.21.15.123:3300/0 conn(0x7f5c280ca2c0 0x7f5c2806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2d656700 1 -- 172.21.15.64:0/3619591681 >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 msgr2=0x7f5c280ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2d656700 1 --2- 172.21.15.64:0/3619591681 >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 0x7f5c280ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2d656700 1 -- 172.21.15.64:0/3619591681 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280cc690 con 0x7f5c28158e20 2023-09-27T23:50:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2ce55700 1 --2- 172.21.15.64:0/3619591681 >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 0x7f5c280ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2d656700 1 --2- 172.21.15.64:0/3619591681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c28158e20 0x7f5c280ce5c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cfd10 tx=0x7f5c240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe510644f9c89b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c1ffff700 1 -- 172.21.15.64:0/3619591681 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24011630 con 0x7f5c28158e20 2023-09-27T23:50:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c1ffff700 1 -- 172.21.15.64:0/3619591681 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c240117b0 con 0x7f5c28158e20 2023-09-27T23:50:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c1ffff700 1 -- 172.21.15.64:0/3619591681 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24011b70 con 0x7f5c28158e20 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/3619591681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c28158e20 msgr2=0x7f5c280ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/3619591681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c28158e20 0x7f5c280ce5c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cfd10 tx=0x7f5c240091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/3619591681 shutdown_connections 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/3619591681 >> v2:172.21.15.123:3300/0 conn(0x7f5c280ca2c0 0x7f5c2806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/3619591681 >> v2:172.21.15.64:3300/0 conn(0x7f5c280cb970 0x7f5c280ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/3619591681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c28158e20 0x7f5c280ce5c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/3619591681 >> 172.21.15.64:0/3619591681 conn(0x7f5c28067bb0 msgr2=0x7f5c280d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/3619591681 shutdown_connections 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.219+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/3619591681 wait complete. 2023-09-27T23:50:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 Processor -- start 2023-09-27T23:50:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 -- start start 2023-09-27T23:50:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c280ca2c0 0x7f5c280c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 0x7f5c281512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c281551a0 0x7f5c28148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28151830 con 0x7f5c28158e20 2023-09-27T23:50:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28154d10 con 0x7f5c281551a0 2023-09-27T23:50:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28151a40 con 0x7f5c280ca2c0 2023-09-27T23:50:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 0x7f5c281512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 0x7f5c281512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47962/0 (socket says 172.21.15.64:47962) 2023-09-27T23:50:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 -- 172.21.15.64:0/2248908612 learned_addr learned my addr 172.21.15.64:0/2248908612 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2d656700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c280ca2c0 0x7f5c280c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2de57700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c281551a0 0x7f5c28148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 -- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c280ca2c0 msgr2=0x7f5c280c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c280ca2c0 0x7f5c280c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 -- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c281551a0 msgr2=0x7f5c28148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c281551a0 0x7f5c28148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 -- 172.21.15.64:0/2248908612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180094f0 con 0x7f5c28158e20 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2d656700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c280ca2c0 0x7f5c280c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:24.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2ce55700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 0x7f5c281512f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800ec50 tx=0x7f5c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:24.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800c850 con 0x7f5c28158e20 2023-09-27T23:50:24.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1800c9b0 con 0x7f5c28158e20 2023-09-27T23:50:24.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c24009420 con 0x7f5c28158e20 2023-09-27T23:50:24.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800cd50 con 0x7f5c28158e20 2023-09-27T23:50:24.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.223+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c281546c0 con 0x7f5c28158e20 2023-09-27T23:50:24.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.227+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c28154ea0 con 0x7f5c28158e20 2023-09-27T23:50:24.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.227+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 79) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f5c18013680 con 0x7f5c28158e20 2023-09-27T23:50:24.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.227+0000 7f5c1d7fa700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5c0c0607a0 0x7f5c0c062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:24.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.227+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 7467+0+0 (secure 0 0 0) 0x7f5c1807d6e0 con 0x7f5c28158e20 2023-09-27T23:50:24.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.227+0000 7f5c2d656700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5c0c0607a0 0x7f5c0c062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:24.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.227+0000 7f5c2d656700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5c0c0607a0 0x7f5c0c062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cfd10 tx=0x7f5c24017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:24.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.235+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5c1804d690 con 0x7f5c28158e20 2023-09-27T23:50:24.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:24.423+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5c28071120 con 0x7f5c28158e20 2023-09-27T23:50:25.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.083+0000 7f5c1d7fa700 1 -- 172.21.15.64:0/2248908612 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5c18050cb0 con 0x7f5c28158e20 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5c0c0607a0 msgr2=0x7f5c0c062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5c0c0607a0 0x7f5c0c062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5c280cfd10 tx=0x7f5c24017000 comp rx=0 tx=0).stop 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 msgr2=0x7f5c281512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 0x7f5c281512f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800ec50 tx=0x7f5c18009250 comp rx=0 tx=0).stop 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 shutdown_connections 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5c0c0607a0 0x7f5c0c062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c281551a0 0x7f5c28148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158e20 0x7f5c281512f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 --2- 172.21.15.64:0/2248908612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c280ca2c0 0x7f5c280c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.087+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 >> 172.21.15.64:0/2248908612 conn(0x7f5c28067bb0 msgr2=0x7f5c280c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:25.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.091+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 shutdown_connections 2023-09-27T23:50:25.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.091+0000 7f5c2f8ba700 1 -- 172.21.15.64:0/2248908612 wait complete. 2023-09-27T23:50:25.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:50:25.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.267+0000 7fe0c5cf8700 1 Processor -- start 2023-09-27T23:50:25.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.267+0000 7fe0c5cf8700 1 -- start start 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c0155a40 0x7fe0c00ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0c00ae920 0x7fe0c0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c00ad3c0 0x7fe0c0159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00ae490 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00ad880 con 0x7fe0c00ae920 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00adb60 con 0x7fe0c0155a40 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c00ad3c0 0x7fe0c0159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c00ad3c0 0x7fe0c0159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55734/0 (socket says 172.21.15.64:55734) 2023-09-27T23:50:25.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 -- 172.21.15.64:0/4229611377 learned_addr learned my addr 172.21.15.64:0/4229611377 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:25.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0bffff700 1 --2- 172.21.15.64:0/4229611377 >> v2:172.21.15.123:3300/0 conn(0x7fe0c00ae920 0x7fe0c0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c4cf6700 1 --2- 172.21.15.64:0/4229611377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c0155a40 0x7fe0c00ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 -- 172.21.15.64:0/4229611377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c0155a40 msgr2=0x7fe0c00ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 --2- 172.21.15.64:0/4229611377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c0155a40 0x7fe0c00ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 -- 172.21.15.64:0/4229611377 >> v2:172.21.15.123:3300/0 conn(0x7fe0c00ae920 msgr2=0x7fe0c0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 --2- 172.21.15.64:0/4229611377 >> v2:172.21.15.123:3300/0 conn(0x7fe0c00ae920 0x7fe0c0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 -- 172.21.15.64:0/4229611377 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00ade40 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c54f7700 1 --2- 172.21.15.64:0/4229611377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0159b60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00cc0f0 tx=0x7fe0b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4ee647acca973 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:25.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0bf7fe700 1 -- 172.21.15.64:0/4229611377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800e070 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0bf7fe700 1 -- 172.21.15.64:0/4229611377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800b970 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0bf7fe700 1 -- 172.21.15.64:0/4229611377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80092e0 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/4229611377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 msgr2=0x7fe0c0159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/4229611377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0159b60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00cc0f0 tx=0x7fe0b8009840 comp rx=0 tx=0).stop 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/4229611377 shutdown_connections 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/4229611377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0159b60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.271+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/4229611377 >> v2:172.21.15.123:3300/0 conn(0x7fe0c00ae920 0x7fe0c0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.275+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/4229611377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c0155a40 0x7fe0c00ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.275+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/4229611377 >> 172.21.15.64:0/4229611377 conn(0x7fe0c00c8c00 msgr2=0x7fe0c00c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:25.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.275+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/4229611377 shutdown_connections 2023-09-27T23:50:25.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.275+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/4229611377 wait complete. 2023-09-27T23:50:25.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.275+0000 7fe0c5cf8700 1 Processor -- start 2023-09-27T23:50:25.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.275+0000 7fe0c5cf8700 1 -- start start 2023-09-27T23:50:25.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c00ae920 0x7fe0c00c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c0155a40 0x7fe0c01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c7ef0 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c0de0 con 0x7fe0c0155a40 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c04b0 con 0x7fe0c00ae920 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55736/0 (socket says 172.21.15.64:55736) 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 -- 172.21.15.64:0/2647891524 learned_addr learned my addr 172.21.15.64:0/2647891524 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0bffff700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c00ae920 0x7fe0c00c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c54f7700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c0155a40 0x7fe0c01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 -- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c00ae920 msgr2=0x7fe0c00c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:25.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c00ae920 0x7fe0c00c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 -- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c0155a40 msgr2=0x7fe0c01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:25.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c0155a40 0x7fe0c01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:25.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 -- 172.21.15.64:0/2647891524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8009ae0 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c4cf6700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0150fd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000e960 tx=0x7fe0b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0013680 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b00137e0 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c00c0790 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0013b80 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.279+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00c01a0 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.283+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c004be40 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.283+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 79) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fe0b00123d0 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.283+0000 7fe0bcff9700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe0a00607a0 0x7fe0a0062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:25.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.283+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 7467+0+0 (secure 0 0 0) 0x7fe0b007d260 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.283+0000 7fe0bffff700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe0a00607a0 0x7fe0a0062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:25.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.287+0000 7fe0bffff700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe0a00607a0 0x7fe0a0062c50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0167e90 tx=0x7fe0a80091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:25.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.291+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe0b004d140 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.447+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0c00ae120 con 0x7fe0c00ad3c0 2023-09-27T23:50:25.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:25.515+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe0b004d510 con 0x7fe0c00ad3c0 2023-09-27T23:50:26.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.091+0000 7fe0bcff9700 1 -- 172.21.15.64:0/2647891524 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe0b0050bd0 con 0x7fe0c00ad3c0 2023-09-27T23:50:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe0a00607a0 msgr2=0x7fe0a0062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe0a00607a0 0x7fe0a0062c50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0167e90 tx=0x7fe0a80091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 msgr2=0x7fe0c0150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0150fd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000e960 tx=0x7fe0b00091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 shutdown_connections 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe0a00607a0 0x7fe0a0062c50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0c0155a40 0x7fe0c01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0c00ae920 0x7fe0c00c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 --2- 172.21.15.64:0/2647891524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ad3c0 0x7fe0c0150fd0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 >> 172.21.15.64:0/2647891524 conn(0x7fe0c00c8c00 msgr2=0x7fe0c00670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 shutdown_connections 2023-09-27T23:50:26.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.099+0000 7fe0c5cf8700 1 -- 172.21.15.64:0/2647891524 wait complete. 2023-09-27T23:50:26.103 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:50:26.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:50:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.223+0000 7efceac1f700 1 Processor -- start 2023-09-27T23:50:26.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.223+0000 7efceac1f700 1 -- start start 2023-09-27T23:50:26.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efce4156640 0x7efce4156a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce4157210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efce4157750 0x7efce406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce406c7c0 con 0x7efce4157750 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce406ca10 con 0x7efce4156640 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce406ccf0 con 0x7efce40c8ae0 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efce4157750 0x7efce406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efce4157750 0x7efce406d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55746/0 (socket says 172.21.15.64:55746) 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 -- 172.21.15.64:0/1423953943 learned_addr learned my addr 172.21.15.64:0/1423953943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efce941c700 1 --2- 172.21.15.64:0/1423953943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efce9c1d700 1 --2- 172.21.15.64:0/1423953943 >> v2:172.21.15.123:3300/0 conn(0x7efce4156640 0x7efce4156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 -- 172.21.15.64:0/1423953943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 msgr2=0x7efce4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 --2- 172.21.15.64:0/1423953943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce4157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 -- 172.21.15.64:0/1423953943 >> v2:172.21.15.123:3300/0 conn(0x7efce4156640 msgr2=0x7efce4156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:26.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 --2- 172.21.15.64:0/1423953943 >> v2:172.21.15.123:3300/0 conn(0x7efce4156640 0x7efce4156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 -- 172.21.15.64:0/1423953943 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce406cfd0 con 0x7efce4157750 2023-09-27T23:50:26.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efcea41e700 1 --2- 172.21.15.64:0/1423953943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce406d7d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7efce4068060 tx=0x7efcdc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859696a32c9f48b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:26.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efce8c1b700 1 -- 172.21.15.64:0/1423953943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00e070 con 0x7efce4157750 2023-09-27T23:50:26.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efce8c1b700 1 -- 172.21.15.64:0/1423953943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc00ba20 con 0x7efce4157750 2023-09-27T23:50:26.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efce8c1b700 1 -- 172.21.15.64:0/1423953943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0092e0 con 0x7efce4157750 2023-09-27T23:50:26.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- 172.21.15.64:0/1423953943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 msgr2=0x7efce406d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- 172.21.15.64:0/1423953943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce406d7d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7efce4068060 tx=0x7efcdc009840 comp rx=0 tx=0).stop 2023-09-27T23:50:26.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- 172.21.15.64:0/1423953943 shutdown_connections 2023-09-27T23:50:26.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- 172.21.15.64:0/1423953943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce406d7d0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- 172.21.15.64:0/1423953943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 --2- 172.21.15.64:0/1423953943 >> v2:172.21.15.123:3300/0 conn(0x7efce4156640 0x7efce4156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- 172.21.15.64:0/1423953943 >> 172.21.15.64:0/1423953943 conn(0x7efce4067bb0 msgr2=0x7efce40c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:26.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- 172.21.15.64:0/1423953943 shutdown_connections 2023-09-27T23:50:26.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.227+0000 7efceac1f700 1 -- 172.21.15.64:0/1423953943 wait complete. 2023-09-27T23:50:26.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 Processor -- start 2023-09-27T23:50:26.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 -- start start 2023-09-27T23:50:26.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce41425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efce4156640 0x7efce414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce4146c80 con 0x7efce4157750 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce414d550 con 0x7efce4156640 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.231+0000 7efceac1f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce41466c0 con 0x7efce40c8ae0 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efce9c1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce41425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efce941c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efce4156640 0x7efce414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efce9c1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce41425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:52878/0 (socket says 172.21.15.64:52878) 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efce9c1d700 1 -- 172.21.15.64:0/1189353480 learned_addr learned my addr 172.21.15.64:0/1189353480 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:26.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce414c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55762/0 (socket says 172.21.15.64:55762) 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 -- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 msgr2=0x7efce41425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce41425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 -- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efce4156640 msgr2=0x7efce414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efce4156640 0x7efce414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 -- 172.21.15.64:0/1189353480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc009ae0 con 0x7efce4157750 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efce9c1d700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce41425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:26.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcea41e700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce414c340 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7efce414d110 tx=0x7efcdc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0032f0 con 0x7efce4157750 2023-09-27T23:50:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc008e50 con 0x7efce4157750 2023-09-27T23:50:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce41469a0 con 0x7efce4157750 2023-09-27T23:50:26.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0266b0 con 0x7efce4157750 2023-09-27T23:50:26.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce4143160 con 0x7efce4157750 2023-09-27T23:50:26.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.235+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce4158d50 con 0x7efce4157750 2023-09-27T23:50:26.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.239+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7efcdc003040 con 0x7efce4157750 2023-09-27T23:50:26.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.239+0000 7efcd9ffb700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efcc8060a70 0x7efcc8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.239+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 7029+0+0 (secure 0 0 0) 0x7efcdc07f570 con 0x7efce4157750 2023-09-27T23:50:26.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.239+0000 7efce9c1d700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efcc8060a70 0x7efcc8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.239+0000 7efce9c1d700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efcc8060a70 0x7efcc8062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efce0006fd0 tx=0x7efce0008040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:26.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.247+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efce4158d50 con 0x7efce4157750 2023-09-27T23:50:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.423+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efce4158d50 con 0x7efce4157750 2023-09-27T23:50:26.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.719+0000 7efcd9ffb700 1 -- 172.21.15.64:0/1189353480 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) v1 ==== 169+0+0 (secure 0 0 0) 0x7efce4158d50 con 0x7efce4157750 2023-09-27T23:50:26.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efcc8060a70 msgr2=0x7efcc8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efcc8060a70 0x7efcc8062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efce0006fd0 tx=0x7efce0008040 comp rx=0 tx=0).stop 2023-09-27T23:50:26.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 msgr2=0x7efce414c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce414c340 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7efce414d110 tx=0x7efcdc018000 comp rx=0 tx=0).stop 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 shutdown_connections 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efcc8060a70 0x7efcc8062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce4157750 0x7efce414c340 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efce4156640 0x7efce414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 --2- 172.21.15.64:0/1189353480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efce40c8ae0 0x7efce41425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 >> 172.21.15.64:0/1189353480 conn(0x7efce4067bb0 msgr2=0x7efce406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:26.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 shutdown_connections 2023-09-27T23:50:26.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.727+0000 7efceac1f700 1 -- 172.21.15.64:0/1189353480 wait complete. 2023-09-27T23:50:26.731 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:50:26.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:50:26.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 Processor -- start 2023-09-27T23:50:26.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 -- start start 2023-09-27T23:50:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f80e015a010 0x7f80e00c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80e0142690 0x7f80e00c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e0159b80 con 0x7f80e0141e70 2023-09-27T23:50:26.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e00c8220 con 0x7f80e0142690 2023-09-27T23:50:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80e49b2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e00c7f40 con 0x7f80e015a010 2023-09-27T23:50:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80e0142690 0x7f80e00c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80e0142690 0x7f80e00c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55774/0 (socket says 172.21.15.64:55774) 2023-09-27T23:50:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 -- 172.21.15.64:0/1996125770 learned_addr learned my addr 172.21.15.64:0/1996125770 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:26.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80de59c700 1 --2- 172.21.15.64:0/1996125770 >> v2:172.21.15.123:3300/0 conn(0x7f80e015a010 0x7f80e00c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 -- 172.21.15.64:0/1996125770 >> v2:172.21.15.123:3300/0 conn(0x7f80e015a010 msgr2=0x7f80e00c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 --2- 172.21.15.64:0/1996125770 >> v2:172.21.15.123:3300/0 conn(0x7f80e015a010 0x7f80e00c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 -- 172.21.15.64:0/1996125770 >> v1:172.21.15.123:6789/0 conn(0x7f80e0141e70 legacy=0x7f80e00c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 -- 172.21.15.64:0/1996125770 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0159530 con 0x7f80e0142690 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.851+0000 7f80df59e700 1 --2- 172.21.15.64:0/1996125770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0142690 0x7f80e00c5220 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f80e006e830 tx=0x7f80d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ef3a48b5fc496f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80ddd9b700 1 -- 172.21.15.64:0/1996125770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d800e070 con 0x7f80e0142690 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80ddd9b700 1 -- 172.21.15.64:0/1996125770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d800ba20 con 0x7f80e0142690 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80ddd9b700 1 -- 172.21.15.64:0/1996125770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80092e0 con 0x7f80e0142690 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- 172.21.15.64:0/1996125770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0142690 msgr2=0x7f80e00c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 --2- 172.21.15.64:0/1996125770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0142690 0x7f80e00c5220 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f80e006e830 tx=0x7f80d8009840 comp rx=0 tx=0).stop 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- 172.21.15.64:0/1996125770 shutdown_connections 2023-09-27T23:50:26.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 --2- 172.21.15.64:0/1996125770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0142690 0x7f80e00c5220 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 --2- 172.21.15.64:0/1996125770 >> v2:172.21.15.123:3300/0 conn(0x7f80e015a010 0x7f80e00c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- 172.21.15.64:0/1996125770 >> 172.21.15.64:0/1996125770 conn(0x7f80e0067b50 msgr2=0x7f80e00cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- 172.21.15.64:0/1996125770 shutdown_connections 2023-09-27T23:50:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- 172.21.15.64:0/1996125770 wait complete. 2023-09-27T23:50:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 Processor -- start 2023-09-27T23:50:26.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- start start 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 0x7f80e006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80e0142690 0x7f80e01402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80e015a010 0x7f80e01407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e013bf40 con 0x7f80e0141e70 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e0140e40 con 0x7f80e015a010 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80e49b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e0141120 con 0x7f80e0142690 2023-09-27T23:50:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.855+0000 7f80ded9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 0x7f80e006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 0x7f80e006d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55786/0 (socket says 172.21.15.64:55786) 2023-09-27T23:50:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 -- 172.21.15.64:0/2204618674 learned_addr learned my addr 172.21.15.64:0/2204618674 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80df59e700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80e015a010 0x7f80e01407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80de59c700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80e0142690 0x7f80e01402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 -- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80e0142690 msgr2=0x7f80e01402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80e0142690 0x7f80e01402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 -- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80e015a010 msgr2=0x7f80e01407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80e015a010 0x7f80e01407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 -- 172.21.15.64:0/2204618674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8009ae0 con 0x7f80e0141e70 2023-09-27T23:50:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80ded9d700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 0x7f80e006d5c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c4e0 tx=0x7f80d000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:26.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0011590 con 0x7f80e0141e70 2023-09-27T23:50:26.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0011710 con 0x7f80e0141e70 2023-09-27T23:50:26.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e0141400 con 0x7f80e0141e70 2023-09-27T23:50:26.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0011ad0 con 0x7f80e0141e70 2023-09-27T23:50:26.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e006db00 con 0x7f80e0141e70 2023-09-27T23:50:26.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e006dcf0 con 0x7f80e0141e70 2023-09-27T23:50:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f80d001b3b0 con 0x7f80e0141e70 2023-09-27T23:50:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80d6ffd700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80c0060a70 0x7f80c0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.859+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80d007c660 con 0x7f80e0141e70 2023-09-27T23:50:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.863+0000 7f80de59c700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80c0060a70 0x7f80c0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.863+0000 7f80de59c700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80c0060a70 0x7f80c0062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f80e013bcf0 tx=0x7f80c8008040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:26.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:26.871+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f80d004c5c0 con 0x7f80e0141e70 2023-09-27T23:50:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.031+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f80e013c080 con 0x7f80e0141e70 2023-09-27T23:50:27.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.035+0000 7f80d6ffd700 1 -- 172.21.15.64:0/2204618674 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v55) v1 ==== 115+0+0 (secure 0 0 0) 0x7f80d00170b0 con 0x7f80e0141e70 2023-09-27T23:50:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80c0060a70 msgr2=0x7f80c0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80c0060a70 0x7f80c0062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f80e013bcf0 tx=0x7f80c8008040 comp rx=0 tx=0).stop 2023-09-27T23:50:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 msgr2=0x7f80e006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 0x7f80e006d5c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c4e0 tx=0x7f80d000c7f0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 shutdown_connections 2023-09-27T23:50:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80c0060a70 0x7f80c0062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80e015a010 0x7f80e01407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80e0142690 0x7f80e01402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 --2- 172.21.15.64:0/2204618674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e0141e70 0x7f80e006d5c0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 >> 172.21.15.64:0/2204618674 conn(0x7f80e0067b50 msgr2=0x7f80e006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 shutdown_connections 2023-09-27T23:50:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.039+0000 7f80e49b2700 1 -- 172.21.15.64:0/2204618674 wait complete. 2023-09-27T23:50:27.055 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:50:27.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.167+0000 7f031cd4e700 1 Processor -- start 2023-09-27T23:50:27.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.167+0000 7f031cd4e700 1 -- start start 2023-09-27T23:50:27.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f031cd4e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 0x7f03180caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f031cd4e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0318159770 0x7f031806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f031cd4e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031806c850 con 0x7f031806aeb0 2023-09-27T23:50:27.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f031cd4e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031806b470 con 0x7f0318159770 2023-09-27T23:50:27.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f031cd4e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031806b750 con 0x7f0318158c90 2023-09-27T23:50:27.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f0316ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f031806aeb0 0x7f0318159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52892/0 (socket says 172.21.15.64:52892) 2023-09-27T23:50:27.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f0316ffd700 1 -- 172.21.15.64:0/3882071395 learned_addr learned my addr 172.21.15.64:0/3882071395 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:27.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 0x7f03180caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f0317fff700 1 --2- 172.21.15.64:0/3882071395 >> v2:172.21.15.123:3300/0 conn(0x7f0318159770 0x7f031806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065676753 0 0) 0x7f031806c850 con 0x7f031806aeb0 2023-09-27T23:50:27.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0304003570 con 0x7f031806aeb0 2023-09-27T23:50:27.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423257102 0 0) 0x7f0304003570 con 0x7f031806aeb0 2023-09-27T23:50:27.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 msgr2=0x7f03180caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 0x7f03180caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 >> v2:172.21.15.123:3300/0 conn(0x7f0318159770 msgr2=0x7f031806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 --2- 172.21.15.64:0/3882071395 >> v2:172.21.15.123:3300/0 conn(0x7f0318159770 0x7f031806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031806ba30 con 0x7f031806aeb0 2023-09-27T23:50:27.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f0300001470 con 0x7f031806aeb0 2023-09-27T23:50:27.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 >> v1:172.21.15.64:6789/0 conn(0x7f031806aeb0 legacy=0x7f0318159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0304003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 0x7f0304004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0304004d50 0x7f03040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031806b470 con 0x7f0304003a60 2023-09-27T23:50:27.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031806b750 con 0x7f0304004d50 2023-09-27T23:50:27.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030400ba60 con 0x7f0318158c90 2023-09-27T23:50:27.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0304003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f0316ffd700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 0x7f0304004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f0317fff700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0304004d50 0x7f03040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 -- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 msgr2=0x7f0304004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0318158c90 0x7f0304004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 -- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0304004d50 msgr2=0x7f03040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0304004d50 0x7f03040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 -- 172.21.15.64:0/3882071395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0304003570 con 0x7f0304003a60 2023-09-27T23:50:27.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f0317fff700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0304004d50 0x7f03040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03177fe700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0304003e70 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0308009c00 tx=0x7f03080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73825d911991fda4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800e6b0 con 0x7f0304003a60 2023-09-27T23:50:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030800e810 con 0x7f0304003a60 2023-09-27T23:50:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.171+0000 7f03167fc700 1 -- 172.21.15.64:0/3882071395 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800ebd0 con 0x7f0304003a60 2023-09-27T23:50:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 -- 172.21.15.64:0/3882071395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 msgr2=0x7f0304003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 --2- 172.21.15.64:0/3882071395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0304003e70 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0308009c00 tx=0x7f03080091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f0317fff700 1 -- 172.21.15.64:0/3882071395 reap_dead start 2023-09-27T23:50:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 -- 172.21.15.64:0/3882071395 shutdown_connections 2023-09-27T23:50:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 -- 172.21.15.64:0/3882071395 >> 172.21.15.64:0/3882071395 conn(0x7f0318067bb0 msgr2=0x7f03180cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 -- 172.21.15.64:0/3882071395 shutdown_connections 2023-09-27T23:50:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 -- 172.21.15.64:0/3882071395 wait complete. 2023-09-27T23:50:27.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 Processor -- start 2023-09-27T23:50:27.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.175+0000 7f031cd4e700 1 -- start start 2023-09-27T23:50:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0318150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0318169960 0x7f0318169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03181577e0 0x7f0318157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03181581c0 con 0x7f0304003a60 2023-09-27T23:50:27.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318155c70 con 0x7f0318169960 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03181550a0 con 0x7f03181577e0 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0318150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0318150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55796/0 (socket says 172.21.15.64:55796) 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 -- 172.21.15.64:0/620232067 learned_addr learned my addr 172.21.15.64:0/620232067 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f0316ffd700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0318169960 0x7f0318169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f0317fff700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03181577e0 0x7f0318157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 -- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03181577e0 msgr2=0x7f0318157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03181577e0 0x7f0318157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 -- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0318169960 msgr2=0x7f0318169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0318169960 0x7f0318169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 -- 172.21.15.64:0/620232067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318155340 con 0x7f0304003a60 2023-09-27T23:50:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f03177fe700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0318150c10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f0304004730 tx=0x7f030801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308022070 con 0x7f0304003a60 2023-09-27T23:50:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030800e980 con 0x7f0304003a60 2023-09-27T23:50:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0318155620 con 0x7f0304003a60 2023-09-27T23:50:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308027470 con 0x7f0304003a60 2023-09-27T23:50:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0318156100 con 0x7f0304003a60 2023-09-27T23:50:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f03080278a0 con 0x7f0304003a60 2023-09-27T23:50:27.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.179+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031804be40 con 0x7f0304003a60 2023-09-27T23:50:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.183+0000 7f030f7fe700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f02f8060aa0 0x7f02f8062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.183+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f030807ebc0 con 0x7f0304003a60 2023-09-27T23:50:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.183+0000 7f0316ffd700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f02f8060aa0 0x7f02f8062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.183+0000 7f0316ffd700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f02f8060aa0 0x7f02f8062f50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0300000e10 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.187+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f030804ec10 con 0x7f0304003a60 2023-09-27T23:50:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.347+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f031806bd10 con 0x7f0304003a60 2023-09-27T23:50:27.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.355+0000 7f030f7fe700 1 -- 172.21.15.64:0/620232067 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v56) v1 ==== 120+0+0 (secure 0 0 0) 0x7f030802c0b0 con 0x7f0304003a60 2023-09-27T23:50:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f02f8060aa0 msgr2=0x7f02f8062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f02f8060aa0 0x7f02f8062f50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0300000e10 tx=0x7f0300009040 comp rx=0 tx=0).stop 2023-09-27T23:50:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 msgr2=0x7f0318150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0318150c10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f0304004730 tx=0x7f030801a000 comp rx=0 tx=0).stop 2023-09-27T23:50:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 shutdown_connections 2023-09-27T23:50:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f02f8060aa0 0x7f02f8062f50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03181577e0 0x7f0318157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0318169960 0x7f0318169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 --2- 172.21.15.64:0/620232067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0304003a60 0x7f0318150c10 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 >> 172.21.15.64:0/620232067 conn(0x7f0318067bb0 msgr2=0x7f03180afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:27.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 shutdown_connections 2023-09-27T23:50:27.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.359+0000 7f031cd4e700 1 -- 172.21.15.64:0/620232067 wait complete. 2023-09-27T23:50:27.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:50:27.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.491+0000 7f11af929700 1 Processor -- start 2023-09-27T23:50:27.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.491+0000 7f11af929700 1 -- start start 2023-09-27T23:50:27.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11af929700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f11a80cecd0 0x7f11a806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11af929700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a80cbdf0 con 0x7f11a80cb970 2023-09-27T23:50:27.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11af929700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a80cc0d0 con 0x7f11a8158e20 2023-09-27T23:50:27.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11af929700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a80cc3b0 con 0x7f11a80cecd0 2023-09-27T23:50:27.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11acec4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f11a80cb970 0x7f11a806acc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52898/0 (socket says 172.21.15.64:52898) 2023-09-27T23:50:27.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11acec4700 1 -- 172.21.15.64:0/741309508 learned_addr learned my addr 172.21.15.64:0/741309508 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:27.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11adec6700 1 --2- 172.21.15.64:0/741309508 >> v2:172.21.15.123:3300/0 conn(0x7f11a80cecd0 0x7f11a806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506430337 0 0) 0x7f11a80cbdf0 con 0x7f11a80cb970 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1194003570 con 0x7f11a80cb970 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257491268 0 0) 0x7f1194003570 con 0x7f11a80cb970 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 >> v2:172.21.15.123:3300/0 conn(0x7f11a80cecd0 msgr2=0x7f11a806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 --2- 172.21.15.64:0/741309508 >> v2:172.21.15.123:3300/0 conn(0x7f11a80cecd0 0x7f11a806b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 >> v1:172.21.15.123:6789/0 conn(0x7f11a8158e20 legacy=0x7f11a80ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80cc690 con 0x7f11a80cb970 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f11a4001470 con 0x7f11a80cb970 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 >> v1:172.21.15.64:6789/0 conn(0x7f11a80cb970 legacy=0x7f11a806acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1194003a60 0x7f1194003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194004810 0x7f1194004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11940051c0 0x7f119400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80cc3b0 con 0x7f1194003a60 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119400c160 con 0x7f1194004810 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119400c320 con 0x7f11940051c0 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11adec6700 1 --2- 172.21.15.64:0/741309508 >> v2:172.21.15.123:3300/0 conn(0x7f11a80cecd0 0x7f11a806b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1194003a60 0x7f1194003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11acec4700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194004810 0x7f1194004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11adec6700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11940051c0 0x7f119400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 -- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11940051c0 msgr2=0x7f119400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f11940051c0 0x7f119400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 -- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194004810 msgr2=0x7f1194004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194004810 0x7f1194004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11adec6700 1 -- 172.21.15.64:0/741309508 reap_dead start 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 -- 172.21.15.64:0/741309508 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194003570 con 0x7f1194003a60 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.495+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1194003a60 0x7f1194003e70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f11980018b0 tx=0x7f1198007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10f800d9d04af36 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198012420 con 0x7f1194003a60 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198012580 con 0x7f1194003a60 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f119ffff700 1 -- 172.21.15.64:0/741309508 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198012920 con 0x7f1194003a60 2023-09-27T23:50:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 -- 172.21.15.64:0/741309508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1194003a60 msgr2=0x7f1194003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1194003a60 0x7f1194003e70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f11980018b0 tx=0x7f1198007d30 comp rx=0 tx=0).stop 2023-09-27T23:50:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 -- 172.21.15.64:0/741309508 shutdown_connections 2023-09-27T23:50:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 --2- 172.21.15.64:0/741309508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1194003a60 0x7f1194003e70 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 -- 172.21.15.64:0/741309508 >> 172.21.15.64:0/741309508 conn(0x7f11a8067bb0 msgr2=0x7f11a80d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 -- 172.21.15.64:0/741309508 shutdown_connections 2023-09-27T23:50:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 -- 172.21.15.64:0/741309508 wait complete. 2023-09-27T23:50:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 Processor -- start 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.499+0000 7f11af929700 1 -- start start 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1194003a60 0x7f11a813d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 0x7f11a815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11a80cffb0 0x7f11a80d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a815bfd0 con 0x7f11a815a900 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80d0960 con 0x7f11a80cffb0 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a815b3c0 con 0x7f1194003a60 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 0x7f11a815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 0x7f11a815ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55818/0 (socket says 172.21.15.64:55818) 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 -- 172.21.15.64:0/3995511334 learned_addr learned my addr 172.21.15.64:0/3995511334 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1194003a60 0x7f11a813d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11adec6700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11a80cffb0 0x7f11a80d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 -- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1194003a60 msgr2=0x7f11a813d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1194003a60 0x7f11a813d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 -- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11a80cffb0 msgr2=0x7f11a80d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11a80cffb0 0x7f11a80d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 -- 172.21.15.64:0/3995511334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a815b6a0 con 0x7f11a815a900 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11adec6700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11a80cffb0 0x7f11a80d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11acec4700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 0x7f11a815ad70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f11a4007390 tx=0x7f11a4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4018030 con 0x7f11a815a900 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a400f8d0 con 0x7f11a815a900 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a815b980 con 0x7f11a815a900 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a400fc70 con 0x7f11a815a900 2023-09-27T23:50:27.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a814d350 con 0x7f11a815a900 2023-09-27T23:50:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.503+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a8071120 con 0x7f11a815a900 2023-09-27T23:50:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.507+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f11a401b070 con 0x7f11a815a900 2023-09-27T23:50:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.507+0000 7f119d7fa700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f118c060a70 0x7f118c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.507+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11a4019070 con 0x7f11a815a900 2023-09-27T23:50:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.507+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f118c060a70 0x7f118c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.507+0000 7f11ad6c5700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f118c060a70 0x7f118c062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11a8150500 tx=0x7f1198015000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.511+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f11a404d020 con 0x7f11a815a900 2023-09-27T23:50:27.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.671+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f11a806a7a0 con 0x7f11a815a900 2023-09-27T23:50:27.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f119d7fa700 1 -- 172.21.15.64:0/3995511334 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11a4050640 con 0x7f11a815a900 2023-09-27T23:50:27.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f118c060a70 msgr2=0x7f118c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f118c060a70 0x7f118c062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f11a8150500 tx=0x7f1198015000 comp rx=0 tx=0).stop 2023-09-27T23:50:27.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 msgr2=0x7f11a815ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 0x7f11a815ad70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f11a4007390 tx=0x7f11a4001e50 comp rx=0 tx=0).stop 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 shutdown_connections 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f118c060a70 0x7f118c062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11a80cffb0 0x7f11a80d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a815a900 0x7f11a815ad70 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 --2- 172.21.15.64:0/3995511334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1194003a60 0x7f11a813d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.679+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 >> 172.21.15.64:0/3995511334 conn(0x7f11a8067bb0 msgr2=0x7f11a806abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.683+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 shutdown_connections 2023-09-27T23:50:27.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.683+0000 7f11af929700 1 -- 172.21.15.64:0/3995511334 wait complete. 2023-09-27T23:50:27.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-09-27T23:50:27.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.799+0000 7f5e49901700 1 Processor -- start 2023-09-27T23:50:27.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.799+0000 7f5e49901700 1 -- start start 2023-09-27T23:50:27.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e44158e70 0x7f5e440afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440cc470 0x7f5e440cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440ca5a0 con 0x7f5e4406a540 2023-09-27T23:50:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4406dac0 con 0x7f5e44158e70 2023-09-27T23:50:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b0260 con 0x7f5e440cc470 2023-09-27T23:50:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e488ff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e44158e70 0x7f5e440afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e43fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5e4406a540 0x7f5e4406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:49744/0 (socket says 172.21.15.64:49744) 2023-09-27T23:50:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e43fff700 1 -- 172.21.15.64:0/2213496709 learned_addr learned my addr 172.21.15.64:0/2213496709 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:27.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49100700 1 --2- 172.21.15.64:0/2213496709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440cc470 0x7f5e440cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e437fe700 1 -- 172.21.15.64:0/2213496709 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261719270 0 0) 0x7f5e440ca5a0 con 0x7f5e4406a540 2023-09-27T23:50:27.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e437fe700 1 -- 172.21.15.64:0/2213496709 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e30003570 con 0x7f5e4406a540 2023-09-27T23:50:27.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49100700 1 -- 172.21.15.64:0/2213496709 >> v2:172.21.15.64:3300/0 conn(0x7f5e44158e70 msgr2=0x7f5e440afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:27.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49100700 1 --2- 172.21.15.64:0/2213496709 >> v2:172.21.15.64:3300/0 conn(0x7f5e44158e70 0x7f5e440afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49100700 1 -- 172.21.15.64:0/2213496709 >> v1:172.21.15.123:6789/0 conn(0x7f5e4406a540 legacy=0x7f5e4406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49100700 1 -- 172.21.15.64:0/2213496709 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4406d220 con 0x7f5e440cc470 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e488ff700 1 --2- 172.21.15.64:0/2213496709 >> v2:172.21.15.64:3300/0 conn(0x7f5e44158e70 0x7f5e440afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49100700 1 --2- 172.21.15.64:0/2213496709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440cc470 0x7f5e440cc8d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009e60 tx=0x7f5e3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110aa2665e2db843 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e437fe700 1 -- 172.21.15.64:0/2213496709 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00e070 con 0x7f5e440cc470 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e437fe700 1 -- 172.21.15.64:0/2213496709 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c00bab0 con 0x7f5e440cc470 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e437fe700 1 -- 172.21.15.64:0/2213496709 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00be70 con 0x7f5e440cc470 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- 172.21.15.64:0/2213496709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440cc470 msgr2=0x7f5e440cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 --2- 172.21.15.64:0/2213496709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440cc470 0x7f5e440cc8d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009e60 tx=0x7f5e3c009780 comp rx=0 tx=0).stop 2023-09-27T23:50:27.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- 172.21.15.64:0/2213496709 shutdown_connections 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 --2- 172.21.15.64:0/2213496709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e440cc470 0x7f5e440cc8d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 --2- 172.21.15.64:0/2213496709 >> v2:172.21.15.64:3300/0 conn(0x7f5e44158e70 0x7f5e440afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- 172.21.15.64:0/2213496709 >> 172.21.15.64:0/2213496709 conn(0x7f5e440670e0 msgr2=0x7f5e440674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- 172.21.15.64:0/2213496709 shutdown_connections 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.803+0000 7f5e49901700 1 -- 172.21.15.64:0/2213496709 wait complete. 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 Processor -- start 2023-09-27T23:50:27.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 -- start start 2023-09-27T23:50:27.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e4406a540 0x7f5e4415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e440cc470 0x7f5e4416c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 0x7f5e440c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4415a310 con 0x7f5e44158e70 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e44159690 con 0x7f5e440cc470 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e44159970 con 0x7f5e4406a540 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 0x7f5e440c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 0x7f5e440c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55848/0 (socket says 172.21.15.64:55848) 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 -- 172.21.15.64:0/2202887073 learned_addr learned my addr 172.21.15.64:0/2202887073 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e43fff700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e440cc470 0x7f5e4416c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e488ff700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e4406a540 0x7f5e4415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 -- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e4406a540 msgr2=0x7f5e4415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e4406a540 0x7f5e4415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 -- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e440cc470 msgr2=0x7f5e4416c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e440cc470 0x7f5e4416c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 -- 172.21.15.64:0/2202887073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009420 con 0x7f5e44158e70 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49100700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 0x7f5e440c5ad0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5e4406a4f0 tx=0x7f5e3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c012e10 con 0x7f5e44158e70 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c027470 con 0x7f5e44158e70 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c027830 con 0x7f5e44158e70 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.807+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c009a20 con 0x7f5e44158e70 2023-09-27T23:50:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.811+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e44159d00 con 0x7f5e44158e70 2023-09-27T23:50:27.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.811+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4404be40 con 0x7f5e44158e70 2023-09-27T23:50:27.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.811+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5e3c024070 con 0x7f5e44158e70 2023-09-27T23:50:27.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.811+0000 7f5e40ff9700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5e24060a70 0x7f5e24062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:27.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.811+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e3c07fbf0 con 0x7f5e44158e70 2023-09-27T23:50:27.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.811+0000 7f5e488ff700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5e24060a70 0x7f5e24062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:27.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.815+0000 7f5e488ff700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5e24060a70 0x7f5e24062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009f40 tx=0x7f5e2c0091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:27.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:27.819+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5e3c04fab0 con 0x7f5e44158e70 2023-09-27T23:50:28.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.047+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f5e4406bb10 con 0x7f5e44158e70 2023-09-27T23:50:28.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.099+0000 7f5e40ff9700 1 -- 172.21.15.64:0/2202887073 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v1285) v1 ==== 165+0+0 (secure 0 0 0) 0x7f5e3c02c020 con 0x7f5e44158e70 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5e24060a70 msgr2=0x7f5e24062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5e24060a70 0x7f5e24062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009f40 tx=0x7f5e2c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 msgr2=0x7f5e440c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 0x7f5e440c5ad0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5e4406a4f0 tx=0x7f5e3c019000 comp rx=0 tx=0).stop 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 shutdown_connections 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5e24060a70 0x7f5e24062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e44158e70 0x7f5e440c5ad0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e440cc470 0x7f5e4416c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 --2- 172.21.15.64:0/2202887073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e4406a540 0x7f5e4415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 >> 172.21.15.64:0/2202887073 conn(0x7f5e440670e0 msgr2=0x7f5e440cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 shutdown_connections 2023-09-27T23:50:28.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.103+0000 7f5e49901700 1 -- 172.21.15.64:0/2202887073 wait complete. 2023-09-27T23:50:28.125 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-09-27T23:50:28.125 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-09-27T23:50:28.125 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-09-27T23:50:28.126 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-09-27T23:50:28.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 Processor -- start 2023-09-27T23:50:28.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 -- start start 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80f815d320 0x7f80f815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f80f80c15b0 0x7f80f80aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f80c7190 con 0x7f80f806d690 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f80c1b50 con 0x7f80f815d320 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.239+0000 7f80ffc0f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f80c1e30 con 0x7f80f80c15b0 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80f815d320 0x7f80f815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f80f815d320 0x7f80f815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55852/0 (socket says 172.21.15.64:55852) 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 -- 172.21.15.64:0/4134787509 learned_addr learned my addr 172.21.15.64:0/4134787509 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:28.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd1aa700 1 --2- 172.21.15.64:0/4134787509 >> v2:172.21.15.123:3300/0 conn(0x7f80f80c15b0 0x7f80f80aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 -- 172.21.15.64:0/4134787509 >> v2:172.21.15.123:3300/0 conn(0x7f80f80c15b0 msgr2=0x7f80f80aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 --2- 172.21.15.64:0/4134787509 >> v2:172.21.15.123:3300/0 conn(0x7f80f80c15b0 0x7f80f80aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 -- 172.21.15.64:0/4134787509 >> v1:172.21.15.64:6789/0 conn(0x7f80f806d690 legacy=0x7f80f80680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 -- 172.21.15.64:0/4134787509 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f80c6e20 con 0x7f80f815d320 2023-09-27T23:50:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fd9ab700 1 --2- 172.21.15.64:0/4134787509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f815d320 0x7f80f815d6f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f80f806f530 tx=0x7f80e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bd10e1524ba541 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fc9a9700 1 -- 172.21.15.64:0/4134787509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e800b860 con 0x7f80f815d320 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fc9a9700 1 -- 172.21.15.64:0/4134787509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e800b9c0 con 0x7f80f815d320 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80fc9a9700 1 -- 172.21.15.64:0/4134787509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e800bd60 con 0x7f80f815d320 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 -- 172.21.15.64:0/4134787509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f815d320 msgr2=0x7f80f815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/4134787509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f815d320 0x7f80f815d6f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f80f806f530 tx=0x7f80e8009250 comp rx=0 tx=0).stop 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 -- 172.21.15.64:0/4134787509 shutdown_connections 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/4134787509 >> v2:172.21.15.123:3300/0 conn(0x7f80f80c15b0 0x7f80f80aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/4134787509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f815d320 0x7f80f815d6f0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 -- 172.21.15.64:0/4134787509 >> 172.21.15.64:0/4134787509 conn(0x7f80f80c8850 msgr2=0x7f80f80c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 -- 172.21.15.64:0/4134787509 shutdown_connections 2023-09-27T23:50:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.243+0000 7f80ffc0f700 1 -- 172.21.15.64:0/4134787509 wait complete. 2023-09-27T23:50:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 Processor -- start 2023-09-27T23:50:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 -- start start 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80f806d690 0x7f80f8154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 0x7f80f814f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f815d320 0x7f80f8163330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80c49d0 con 0x7f80f80c15b0 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80af390 con 0x7f80f815d320 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80ffc0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f80af670 con 0x7f80f806d690 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 0x7f80f814f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 0x7f80f814f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55856/0 (socket says 172.21.15.64:55856) 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 -- 172.21.15.64:0/3509600983 learned_addr learned my addr 172.21.15.64:0/3509600983 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fe1ac700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f815d320 0x7f80f8163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd9ab700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80f806d690 0x7f80f8154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 -- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80f806d690 msgr2=0x7f80f8154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80f806d690 0x7f80f8154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 -- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f815d320 msgr2=0x7f80f8163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f815d320 0x7f80f8163330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.247+0000 7f80fd1aa700 1 -- 172.21.15.64:0/3509600983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e80094f0 con 0x7f80f80c15b0 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80fd1aa700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 0x7f80f814f3d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e190 tx=0x7f80f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f400ecd0 con 0x7f80f80c15b0 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f400ee30 con 0x7f80f80c15b0 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f80af950 con 0x7f80f80c15b0 2023-09-27T23:50:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4014670 con 0x7f80f80c15b0 2023-09-27T23:50:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f80aff10 con 0x7f80f80c15b0 2023-09-27T23:50:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f8071120 con 0x7f80f80c15b0 2023-09-27T23:50:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.251+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f80f4014a20 con 0x7f80f80c15b0 2023-09-27T23:50:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.255+0000 7f80edffb700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80dc060a70 0x7f80dc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.255+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80f407d8a0 con 0x7f80f80c15b0 2023-09-27T23:50:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.255+0000 7f80fd9ab700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80dc060a70 0x7f80dc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.255+0000 7f80fd9ab700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80dc060a70 0x7f80dc062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f80f806f530 tx=0x7f80e8017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:28.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.263+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f80f404d800 con 0x7f80f80c15b0 2023-09-27T23:50:28.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:28.495+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f80f80642a0 con 0x7f80f80c15b0 2023-09-27T23:50:29.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.115+0000 7f80edffb700 1 -- 172.21.15.64:0/3509600983 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v1286) v1 ==== 166+0+0 (secure 0 0 0) 0x7f80f4009a60 con 0x7f80f80c15b0 2023-09-27T23:50:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80dc060a70 msgr2=0x7f80dc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80dc060a70 0x7f80dc062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f80f806f530 tx=0x7f80e8017000 comp rx=0 tx=0).stop 2023-09-27T23:50:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 msgr2=0x7f80f814f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 0x7f80f814f3d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e190 tx=0x7f80f4009920 comp rx=0 tx=0).stop 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 shutdown_connections 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f80dc060a70 0x7f80dc062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f815d320 0x7f80f8163330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f80c15b0 0x7f80f814f3d0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 --2- 172.21.15.64:0/3509600983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f80f806d690 0x7f80f8154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 >> 172.21.15.64:0/3509600983 conn(0x7f80f80c8850 msgr2=0x7f80f815fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 shutdown_connections 2023-09-27T23:50:29.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.119+0000 7f80ffc0f700 1 -- 172.21.15.64:0/3509600983 wait complete. 2023-09-27T23:50:29.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:50:29.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 Processor -- start 2023-09-27T23:50:29.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 -- start start 2023-09-27T23:50:29.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effc80cb970 0x7effc806acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc80cecd0 0x7effc806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc80cbdf0 con 0x7effc8158e20 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80cc0d0 con 0x7effc80cb970 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effd0163700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc80cc3b0 con 0x7effc80cecd0 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effcd6fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effc80cb970 0x7effc806acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.259+0000 7effcd6fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effc80cb970 0x7effc806acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55862/0 (socket says 172.21.15.64:55862) 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcd6fe700 1 -- 172.21.15.64:0/4294126797 learned_addr learned my addr 172.21.15.64:0/4294126797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcdeff700 1 --1- 172.21.15.64:0/4294126797 >> v1:172.21.15.64:6789/0 conn(0x7effc8158e20 0x7effc80ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52920/0 (socket says 172.21.15.64:52920) 2023-09-27T23:50:29.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effce700700 1 --2- 172.21.15.64:0/4294126797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc80cecd0 0x7effc806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcd6fe700 1 -- 172.21.15.64:0/4294126797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc80cecd0 msgr2=0x7effc806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:29.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcd6fe700 1 --2- 172.21.15.64:0/4294126797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc80cecd0 0x7effc806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcd6fe700 1 -- 172.21.15.64:0/4294126797 >> v1:172.21.15.64:6789/0 conn(0x7effc8158e20 legacy=0x7effc80ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effccefd700 1 -- 172.21.15.64:0/4294126797 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500419618 0 0) 0x7effc80cbdf0 con 0x7effc8158e20 2023-09-27T23:50:29.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcd6fe700 1 -- 172.21.15.64:0/4294126797 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80cc690 con 0x7effc80cb970 2023-09-27T23:50:29.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effcd6fe700 1 --2- 172.21.15.64:0/4294126797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80cb970 0x7effc806acc0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7effc40099e0 tx=0x7effc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1fb20b2f46ce4e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effccefd700 1 -- 172.21.15.64:0/4294126797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400b820 con 0x7effc80cb970 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effccefd700 1 -- 172.21.15.64:0/4294126797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400b980 con 0x7effc80cb970 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effccefd700 1 -- 172.21.15.64:0/4294126797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400bd20 con 0x7effc80cb970 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 -- 172.21.15.64:0/4294126797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80cb970 msgr2=0x7effc806acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 --2- 172.21.15.64:0/4294126797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80cb970 0x7effc806acc0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7effc40099e0 tx=0x7effc4009250 comp rx=0 tx=0).stop 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 -- 172.21.15.64:0/4294126797 shutdown_connections 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 --2- 172.21.15.64:0/4294126797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc80cecd0 0x7effc806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 --2- 172.21.15.64:0/4294126797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80cb970 0x7effc806acc0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 -- 172.21.15.64:0/4294126797 >> 172.21.15.64:0/4294126797 conn(0x7effc8067bb0 msgr2=0x7effc80d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 -- 172.21.15.64:0/4294126797 shutdown_connections 2023-09-27T23:50:29.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 -- 172.21.15.64:0/4294126797 wait complete. 2023-09-27T23:50:29.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.263+0000 7effd0163700 1 Processor -- start 2023-09-27T23:50:29.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 -- start start 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effc80cecd0 0x7effc814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc8158e20 0x7effc814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 0x7effc814eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc814fb00 con 0x7effc80b5b00 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc814e560 con 0x7effc80cecd0 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc814db90 con 0x7effc8158e20 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 0x7effc814eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 0x7effc814eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55872/0 (socket says 172.21.15.64:55872) 2023-09-27T23:50:29.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 -- 172.21.15.64:0/4184733666 learned_addr learned my addr 172.21.15.64:0/4184733666 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effcd6fe700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc8158e20 0x7effc814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effcdeff700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effc80cecd0 0x7effc814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 -- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc8158e20 msgr2=0x7effc814bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc8158e20 0x7effc814bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 -- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effc80cecd0 msgr2=0x7effc814e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effc80cecd0 0x7effc814e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 -- 172.21.15.64:0/4184733666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40094f0 con 0x7effc80b5b00 2023-09-27T23:50:29.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effce700700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 0x7effc814eef0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f20 tx=0x7effc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:29.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000cb70 con 0x7effc80b5b00 2023-09-27T23:50:29.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc000f040 con 0x7effc80b5b00 2023-09-27T23:50:29.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc814f820 con 0x7effc80b5b00 2023-09-27T23:50:29.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc0014480 con 0x7effc80b5b00 2023-09-27T23:50:29.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.267+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc814d820 con 0x7effc80b5b00 2023-09-27T23:50:29.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.271+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc8071120 con 0x7effc80b5b00 2023-09-27T23:50:29.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.271+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7effc0014830 con 0x7effc80b5b00 2023-09-27T23:50:29.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.271+0000 7effb9ffb700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effac060aa0 0x7effac062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.271+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7effc007df00 con 0x7effc80b5b00 2023-09-27T23:50:29.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.271+0000 7effcdeff700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effac060aa0 0x7effac062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.275+0000 7effcdeff700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effac060aa0 0x7effac062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7effbc007180 tx=0x7effbc007ef0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:29.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.279+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7effc004dd50 con 0x7effc80b5b00 2023-09-27T23:50:29.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.439+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7effc80cca20 con 0x7effc80b5b00 2023-09-27T23:50:29.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effb9ffb700 1 -- 172.21.15.64:0/4184733666 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7effc0018020 con 0x7effc80b5b00 2023-09-27T23:50:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effac060aa0 msgr2=0x7effac062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effac060aa0 0x7effac062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7effbc007180 tx=0x7effbc007ef0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 msgr2=0x7effc814eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 0x7effc814eef0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f20 tx=0x7effc000b040 comp rx=0 tx=0).stop 2023-09-27T23:50:29.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 shutdown_connections 2023-09-27T23:50:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effac060aa0 0x7effac062f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effc80b5b00 0x7effc814eef0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effc8158e20 0x7effc814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 --2- 172.21.15.64:0/4184733666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effc80cecd0 0x7effc814e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 >> 172.21.15.64:0/4184733666 conn(0x7effc8067bb0 msgr2=0x7effc81563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 shutdown_connections 2023-09-27T23:50:29.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.447+0000 7effd0163700 1 -- 172.21.15.64:0/4184733666 wait complete. 2023-09-27T23:50:29.465 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:50:29.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:50:29.487 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:50:29.509 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:50:29.510 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:50:29.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.643+0000 7f3ff0072700 1 Processor -- start 2023-09-27T23:50:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.643+0000 7f3ff0072700 1 -- start start 2023-09-27T23:50:29.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8069100 0x7f3fe8067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe8068c70 con 0x7f3fe8158f70 2023-09-27T23:50:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe8068060 con 0x7f3fe8159340 2023-09-27T23:50:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8068340 con 0x7f3fe8069100 2023-09-27T23:50:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fede0e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3fe8158f70 0x7f3fe80cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52926/0 (socket says 172.21.15.64:52926) 2023-09-27T23:50:29.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fede0e700 1 -- 172.21.15.64:0/322768413 learned_addr learned my addr 172.21.15.64:0/322768413 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:29.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fed60d700 1 --2- 172.21.15.64:0/322768413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8069100 0x7f3fe8067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795118302 0 0) 0x7f3fe8068c70 con 0x7f3fe8158f70 2023-09-27T23:50:29.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4003570 con 0x7f3fe8158f70 2023-09-27T23:50:29.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275532845 0 0) 0x7f3fe8068060 con 0x7f3fe8159340 2023-09-27T23:50:29.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe8068c70 con 0x7f3fe8159340 2023-09-27T23:50:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fed60d700 1 -- 172.21.15.64:0/322768413 >> v1:172.21.15.123:6789/0 conn(0x7f3fe8159340 legacy=0x7f3fe80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fed60d700 1 -- 172.21.15.64:0/322768413 >> v1:172.21.15.64:6789/0 conn(0x7f3fe8158f70 legacy=0x7f3fe80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fed60d700 1 -- 172.21.15.64:0/322768413 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8068620 con 0x7f3fe8069100 2023-09-27T23:50:29.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fed60d700 1 --2- 172.21.15.64:0/322768413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8069100 0x7f3fe8067980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009910 tx=0x7f3fe40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb6e0214b92378 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:29.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011630 con 0x7f3fe8069100 2023-09-27T23:50:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe4011790 con 0x7f3fe8069100 2023-09-27T23:50:29.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3fece0c700 1 -- 172.21.15.64:0/322768413 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011b30 con 0x7f3fe8069100 2023-09-27T23:50:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- 172.21.15.64:0/322768413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8069100 msgr2=0x7f3fe8067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 --2- 172.21.15.64:0/322768413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8069100 0x7f3fe8067980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009910 tx=0x7f3fe40091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- 172.21.15.64:0/322768413 shutdown_connections 2023-09-27T23:50:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 --2- 172.21.15.64:0/322768413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8069100 0x7f3fe8067980 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- 172.21.15.64:0/322768413 >> 172.21.15.64:0/322768413 conn(0x7f3fe80ccc40 msgr2=0x7f3fe80cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- 172.21.15.64:0/322768413 shutdown_connections 2023-09-27T23:50:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.647+0000 7f3ff0072700 1 -- 172.21.15.64:0/322768413 wait complete. 2023-09-27T23:50:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 Processor -- start 2023-09-27T23:50:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 -- start start 2023-09-27T23:50:29.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe8069100 0x7f3fe815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8158f70 0x7f3fe8154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 0x7f3fe80c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80b09e0 con 0x7f3fe8159340 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80c9cc0 con 0x7f3fe8069100 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3ff0072700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80b1260 con 0x7f3fe8158f70 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 0x7f3fe80c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 0x7f3fe80c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55880/0 (socket says 172.21.15.64:55880) 2023-09-27T23:50:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 -- 172.21.15.64:0/928633319 learned_addr learned my addr 172.21.15.64:0/928633319 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fed60d700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8158f70 0x7f3fe8154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fede0e700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe8069100 0x7f3fe815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 -- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8158f70 msgr2=0x7f3fe8154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8158f70 0x7f3fe8154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 -- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe8069100 msgr2=0x7f3fe815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe8069100 0x7f3fe815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fee60f700 1 -- 172.21.15.64:0/928633319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4009420 con 0x7f3fe8159340 2023-09-27T23:50:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.651+0000 7f3fed60d700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8158f70 0x7f3fe8154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3fee60f700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 0x7f3fe80c9020 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c860 tx=0x7f3fe0002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0002030 con 0x7f3fe8159340 2023-09-27T23:50:29.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00021b0 con 0x7f3fe8159340 2023-09-27T23:50:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe8154680 con 0x7f3fe8159340 2023-09-27T23:50:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0012630 con 0x7f3fe8159340 2023-09-27T23:50:29.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80c9a90 con 0x7f3fe8159340 2023-09-27T23:50:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3fe0012a20 con 0x7f3fe8159340 2023-09-27T23:50:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe80b13d0 con 0x7f3fe8159340 2023-09-27T23:50:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.655+0000 7f3fddffb700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3fcc060a70 0x7f3fcc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:29.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.659+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3fe007d340 con 0x7f3fe8159340 2023-09-27T23:50:29.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.659+0000 7f3fede0e700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3fcc060a70 0x7f3fcc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:29.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.659+0000 7f3fede0e700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3fcc060a70 0x7f3fcc062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80058d0 tx=0x7f3fd8009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:29.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.663+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3fe004d200 con 0x7f3fe8159340 2023-09-27T23:50:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.863+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe8071120 con 0x7f3fe8159340 2023-09-27T23:50:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3fddffb700 1 -- 172.21.15.64:0/928633319 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f3fe0050820 con 0x7f3fe8159340 2023-09-27T23:50:29.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:29.869 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/596838736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":596838736},{"type":"v1","addr":"172.21.15.123:6835","nonce":596838736}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1265273364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1265273364},{"type":"v1","addr":"172.21.15.64:6833","nonce":1265273364}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2728052361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2728052361},{"type":"v1","addr":"172.21.15.123:6837","nonce":2728052361}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":9690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3534430604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3534430604},{"type":"v1","addr":"172.21.15.64:6835","nonce":3534430604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[]} 2023-09-27T23:50:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3fcc060a70 msgr2=0x7f3fcc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3fcc060a70 0x7f3fcc062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80058d0 tx=0x7f3fd8009040 comp rx=0 tx=0).stop 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 msgr2=0x7f3fe80c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 0x7f3fe80c9020 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c860 tx=0x7f3fe0002700 comp rx=0 tx=0).stop 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 shutdown_connections 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3fcc060a70 0x7f3fcc062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8159340 0x7f3fe80c9020 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3fe8158f70 0x7f3fe8154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 --2- 172.21.15.64:0/928633319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe8069100 0x7f3fe815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 >> 172.21.15.64:0/928633319 conn(0x7f3fe80ccc40 msgr2=0x7f3fe80c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 shutdown_connections 2023-09-27T23:50:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:29.867+0000 7f3ff0072700 1 -- 172.21.15.64:0/928633319 wait complete. 2023-09-27T23:50:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 62 2023-09-27T23:50:29.884 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:50:29.884 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:50:29.884 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:50:29.884 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:50:29.885 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:50:29.885 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:50:29.885 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:50:29.885 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:50:29.885 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:50:29.885 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:50:29.885 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:50:29.886 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:50:29.886 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:50:29.886 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:50:29.886 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:50:29.887 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:50:29.899 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:50:29.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:50:29.901 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:50:29.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:50:29.903 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:50:29.904 DEBUG:teuthology.parallel:result is None 2023-09-27T23:50:29.904 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:50:29.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:50:29.905 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:50:29.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:50:29.907 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:50:29.907 DEBUG:teuthology.parallel:result is None 2023-09-27T23:50:29.908 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:50:29.908 DEBUG:teuthology.parallel:result is None 2023-09-27T23:50:29.909 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:50:29.909 DEBUG:teuthology.parallel:result is None 2023-09-27T23:50:29.909 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:50:29.972 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:50:29.978 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:50:29.980 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:50:30.001 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:50:30.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.027+0000 7f1fa7cf2700 1 Processor -- start 2023-09-27T23:50:30.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.027+0000 7f1fa7cf2700 1 -- start start 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa7cf2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 0x7f1fa0163af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa7cf2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa006bc30 con 0x7f1fa006d370 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa7cf2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa006b020 con 0x7f1fa0158e50 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa7cf2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006b300 con 0x7f1fa006ced0 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1fa006d370 0x7f1fa006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52938/0 (socket says 172.21.15.64:52938) 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 -- 172.21.15.64:0/2942548651 learned_addr learned my addr 172.21.15.64:0/2942548651 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa628f700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 0x7f1fa0163af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616490765 0 0) 0x7f1fa006bc30 con 0x7f1fa006d370 2023-09-27T23:50:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f8c003570 con 0x7f1fa006d370 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1018185003 0 0) 0x7f1f8c003570 con 0x7f1fa006d370 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 msgr2=0x7f1fa0163af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 0x7f1fa0163af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 >> v1:172.21.15.123:6789/0 conn(0x7f1fa0158e50 legacy=0x7f1fa006a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa006b5e0 con 0x7f1fa006d370 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f1f90001470 con 0x7f1fa006d370 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 >> v1:172.21.15.64:6789/0 conn(0x7f1fa006d370 legacy=0x7f1fa006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c003a90 0x7f1f8c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c004810 0x7f1f8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 0x7f1f8c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006b300 con 0x7f1f8c004810 2023-09-27T23:50:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c00bb60 con 0x7f1f8c003a90 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c00bd20 con 0x7f1fa006ced0 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c004810 0x7f1f8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa5a8e700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c003a90 0x7f1f8c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa628f700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 0x7f1f8c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 -- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 msgr2=0x7f1f8c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa006ced0 0x7f1f8c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 -- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c003a90 msgr2=0x7f1f8c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c003a90 0x7f1f8c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 -- 172.21.15.64:0/2942548651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c003570 con 0x7f1f8c004810 2023-09-27T23:50:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa5a8e700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f8c003a90 0x7f1f8c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa528d700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c004810 0x7f1f8c004c80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1fa006d7b0 tx=0x7f1fa006bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc49fb72c922851 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000cc30 con 0x7f1f8c004810 2023-09-27T23:50:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9000cd90 con 0x7f1f8c004810 2023-09-27T23:50:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.031+0000 7f1fa4a8c700 1 -- 172.21.15.64:0/2942548651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90005b70 con 0x7f1f8c004810 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/2942548651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c004810 msgr2=0x7f1f8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/2942548651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f8c004810 0x7f1f8c004c80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1fa006d7b0 tx=0x7f1fa006bba0 comp rx=0 tx=0).stop 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa628f700 1 -- 172.21.15.64:0/2942548651 reap_dead start 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/2942548651 shutdown_connections 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/2942548651 >> 172.21.15.64:0/2942548651 conn(0x7f1fa0067bb0 msgr2=0x7f1fa00ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/2942548651 shutdown_connections 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/2942548651 wait complete. 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 Processor -- start 2023-09-27T23:50:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- start start 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa00cad90 0x7f1fa014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 0x7f1fa014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa00acd10 0x7f1fa00ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00ad6d0 con 0x7f1fa00cb160 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00c8580 con 0x7f1fa00acd10 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa7cf2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00c9100 con 0x7f1fa00cad90 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 0x7f1fa014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 0x7f1fa014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55898/0 (socket says 172.21.15.64:55898) 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 -- 172.21.15.64:0/3057437381 learned_addr learned my addr 172.21.15.64:0/3057437381 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa628f700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa00acd10 0x7f1fa00ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa5a8e700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa00cad90 0x7f1fa014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 -- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa00cad90 msgr2=0x7f1fa014e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:30.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa00cad90 0x7f1fa014e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:30.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 -- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa00acd10 msgr2=0x7f1fa00ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:30.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa00acd10 0x7f1fa00ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:30.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 -- 172.21.15.64:0/3057437381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa014c2e0 con 0x7f1fa00cb160 2023-09-27T23:50:30.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.035+0000 7f1fa528d700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 0x7f1fa014bda0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1f90011000 tx=0x7f1f90015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.039+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90022070 con 0x7f1fa00cb160 2023-09-27T23:50:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.039+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f900070d0 con 0x7f1fa00cb160 2023-09-27T23:50:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.039+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000cc30 con 0x7f1fa00cb160 2023-09-27T23:50:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.039+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa00c87d0 con 0x7f1fa00cb160 2023-09-27T23:50:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.039+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa00c8d90 con 0x7f1fa00cb160 2023-09-27T23:50:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.039+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa0071120 con 0x7f1fa00cb160 2023-09-27T23:50:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.043+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1f90007680 con 0x7f1fa00cb160 2023-09-27T23:50:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.043+0000 7f1f95ffb700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f84060aa0 0x7f1f84062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.043+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f1f9007e2b0 con 0x7f1fa00cb160 2023-09-27T23:50:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.043+0000 7f1fa5a8e700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f84060aa0 0x7f1f84062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.043+0000 7f1fa5a8e700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f84060aa0 0x7f1f84062f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c007fd0 tx=0x7f1f9c009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.051+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1f9004e230 con 0x7f1fa00cb160 2023-09-27T23:50:30.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:30.223+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1fa00c9240 con 0x7f1fa00cb160 2023-09-27T23:50:31.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.127+0000 7f1f95ffb700 1 -- 172.21.15.64:0/3057437381 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v170) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f9001c320 con 0x7f1fa00cb160 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f84060aa0 msgr2=0x7f1f84062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f84060aa0 0x7f1f84062f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c007fd0 tx=0x7f1f9c009040 comp rx=0 tx=0).stop 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 msgr2=0x7f1fa014bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 0x7f1fa014bda0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1f90011000 tx=0x7f1f90015000 comp rx=0 tx=0).stop 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 shutdown_connections 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f84060aa0 0x7f1f84062f50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa00acd10 0x7f1fa00ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa00cb160 0x7f1fa014bda0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 --2- 172.21.15.64:0/3057437381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa00cad90 0x7f1fa014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 >> 172.21.15.64:0/3057437381 conn(0x7f1fa0067bb0 msgr2=0x7f1fa00b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 shutdown_connections 2023-09-27T23:50:31.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.131+0000 7f1fa7cf2700 1 -- 172.21.15.64:0/3057437381 wait complete. 2023-09-27T23:50:31.136 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:50:31.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:50:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 Processor -- start 2023-09-27T23:50:31.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 -- start start 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6b8158e70 0x7fb6b80afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b806a540 0x7fb6b806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b80ca5a0 con 0x7fb6b80cc470 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b806dac0 con 0x7fb6b8158e70 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bf3d0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b0260 con 0x7fb6b806a540 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6b8158e70 0x7fb6b80afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bdbcd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b806a540 0x7fb6b806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb6b8158e70 0x7fb6b80afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:45086/0 (socket says 172.21.15.64:45086) 2023-09-27T23:50:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 -- 172.21.15.64:0/2758927023 learned_addr learned my addr 172.21.15.64:0/2758927023 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bebcf700 1 --1- 172.21.15.64:0/2758927023 >> v1:172.21.15.123:6789/0 conn(0x7fb6b80cc470 0x7fb6b80cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:49772/0 (socket says 172.21.15.64:49772) 2023-09-27T23:50:31.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bdbcd700 1 --2- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b806a540 0x7fb6b806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:53056/0 (socket says 172.21.15.64:53056) 2023-09-27T23:50:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 -- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b806a540 msgr2=0x7fb6b806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 --2- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b806a540 0x7fb6b806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 -- 172.21.15.64:0/2758927023 >> v1:172.21.15.123:6789/0 conn(0x7fb6b80cc470 legacy=0x7fb6b80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 -- 172.21.15.64:0/2758927023 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b806d220 con 0x7fb6b8158e70 2023-09-27T23:50:31.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6be3ce700 1 --2- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b80afd20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009f20 tx=0x7fb6a8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1956458c794082 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bd3cc700 1 -- 172.21.15.64:0/2758927023 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800e070 con 0x7fb6b8158e70 2023-09-27T23:50:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bd3cc700 1 -- 172.21.15.64:0/2758927023 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a800b9d0 con 0x7fb6b8158e70 2023-09-27T23:50:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.255+0000 7fb6bd3cc700 1 -- 172.21.15.64:0/2758927023 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80092e0 con 0x7fb6b8158e70 2023-09-27T23:50:31.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 msgr2=0x7fb6b80afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b80afd20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009f20 tx=0x7fb6a8009840 comp rx=0 tx=0).stop 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/2758927023 shutdown_connections 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b806a540 0x7fb6b806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/2758927023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b80afd20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/2758927023 >> 172.21.15.64:0/2758927023 conn(0x7fb6b80670e0 msgr2=0x7fb6b80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/2758927023 shutdown_connections 2023-09-27T23:50:31.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/2758927023 wait complete. 2023-09-27T23:50:31.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 Processor -- start 2023-09-27T23:50:31.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- start start 2023-09-27T23:50:31.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b806a540 0x7fb6b80c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 0x7fb6b8154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b81546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80c6630 con 0x7fb6b806a540 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8154bf0 con 0x7fb6b8158e70 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bf3d0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80c47b0 con 0x7fb6b80cc470 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6be3ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b806a540 0x7fb6b80c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 0x7fb6b8154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 0x7fb6b8154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:53066/0 (socket says 172.21.15.64:53066) 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 -- 172.21.15.64:0/1846154933 learned_addr learned my addr 172.21.15.64:0/1846154933 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6be3ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b806a540 0x7fb6b80c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55900/0 (socket says 172.21.15.64:55900) 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bebcf700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b81546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 -- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 msgr2=0x7fb6b81546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b81546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 -- 172.21.15.64:0/1846154933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b806a540 msgr2=0x7fb6b80c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b806a540 0x7fb6b80c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 -- 172.21.15.64:0/1846154933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8009ae0 con 0x7fb6b80cc470 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bebcf700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b81546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.259+0000 7fb6bdbcd700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 0x7fb6b8154170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000ec80 tx=0x7fb6a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000c850 con 0x7fb6b80cc470 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a000c9b0 con 0x7fb6b80cc470 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b80c38c0 con 0x7fb6b80cc470 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000cd50 con 0x7fb6b80cc470 2023-09-27T23:50:31.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b80c3e80 con 0x7fb6b80cc470 2023-09-27T23:50:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80c4940 con 0x7fb6b80cc470 2023-09-27T23:50:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb6a0005a00 con 0x7fb6b80cc470 2023-09-27T23:50:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6b67fc700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb698060aa0 0x7fb698062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fb6a007d280 con 0x7fb6b80cc470 2023-09-27T23:50:31.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.263+0000 7fb6be3ce700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb698060aa0 0x7fb698062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.267+0000 7fb6be3ce700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb698060aa0 0x7fb698062f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800bd40 tx=0x7fb6a8018000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.271+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb6a004d600 con 0x7fb6b80cc470 2023-09-27T23:50:31.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.491+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb6b804be40 con 0x7fb6b80cc470 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6b67fc700 1 -- 172.21.15.64:0/1846154933 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb6a0017090 con 0x7fb6b80cc470 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:50:31.496 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:50:31.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:50:31.499 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.500 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:50:31.501 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.502 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:50:31.504 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:50:31.505 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.507 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:50:31.508 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:50:31.509 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:50:31.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:31.511 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:50:31.512 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:50:31.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb698060aa0 msgr2=0x7fb698062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb698060aa0 0x7fb698062f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800bd40 tx=0x7fb6a8018000 comp rx=0 tx=0).stop 2023-09-27T23:50:31.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 msgr2=0x7fb6b8154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 0x7fb6b8154170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000ec80 tx=0x7fb6a00091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 shutdown_connections 2023-09-27T23:50:31.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb698060aa0 0x7fb698062f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8158e70 0x7fb6b81546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6b80cc470 0x7fb6b8154170 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 --2- 172.21.15.64:0/1846154933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6b806a540 0x7fb6b80c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 >> 172.21.15.64:0/1846154933 conn(0x7fb6b80670e0 msgr2=0x7fb6b8156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:31.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 shutdown_connections 2023-09-27T23:50:31.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.495+0000 7fb6bf3d0700 1 -- 172.21.15.64:0/1846154933 wait complete. 2023-09-27T23:50:31.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:31.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.619+0000 7fcd3bfff700 1 Processor -- start 2023-09-27T23:50:31.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- start start 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcd3c155960 0x7fcd3c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd3c0c7f00 0x7fcd3c0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c06d990 con 0x7fcd3c06de20 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c15a570 con 0x7fcd3c0c7f00 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c06cd80 con 0x7fcd3c155960 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd3c0c7f00 0x7fcd3c0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd3c0c7f00 0x7fcd3c0aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55914/0 (socket says 172.21.15.64:55914) 2023-09-27T23:50:31.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/974589195 learned_addr learned my addr 172.21.15.64:0/974589195 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:31.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3affd700 1 --2- 172.21.15.64:0/974589195 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c155960 0x7fcd3c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/974589195 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c155960 msgr2=0x7fcd3c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/974589195 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c155960 0x7fcd3c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/974589195 >> v1:172.21.15.123:6789/0 conn(0x7fcd3c06de20 legacy=0x7fcd3c0c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3a7fc700 1 -- 172.21.15.64:0/974589195 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310480659 0 0) 0x7fcd3c06d990 con 0x7fcd3c06de20 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/974589195 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c06d060 con 0x7fcd3c0c7f00 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/974589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0c7f00 0x7fcd3c0aca10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c064070 tx=0x7fcd34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd9cca94ab35886 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3a7fc700 1 -- 172.21.15.64:0/974589195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400e070 con 0x7fcd3c0c7f00 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3a7fc700 1 -- 172.21.15.64:0/974589195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400ba20 con 0x7fcd3c0c7f00 2023-09-27T23:50:31.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3a7fc700 1 -- 172.21.15.64:0/974589195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340092e0 con 0x7fcd3c0c7f00 2023-09-27T23:50:31.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- 172.21.15.64:0/974589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0c7f00 msgr2=0x7fcd3c0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/974589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0c7f00 0x7fcd3c0aca10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c064070 tx=0x7fcd34009840 comp rx=0 tx=0).stop 2023-09-27T23:50:31.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- 172.21.15.64:0/974589195 shutdown_connections 2023-09-27T23:50:31.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/974589195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0c7f00 0x7fcd3c0aca10 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/974589195 >> v2:172.21.15.123:3300/0 conn(0x7fcd3c155960 0x7fcd3c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- 172.21.15.64:0/974589195 >> 172.21.15.64:0/974589195 conn(0x7fcd3c067bb0 msgr2=0x7fcd3c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:31.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- 172.21.15.64:0/974589195 shutdown_connections 2023-09-27T23:50:31.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.623+0000 7fcd3bfff700 1 -- 172.21.15.64:0/974589195 wait complete. 2023-09-27T23:50:31.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 Processor -- start 2023-09-27T23:50:31.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 -- start start 2023-09-27T23:50:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 0x7fcd3c169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c0c7f00 0x7fcd3c0a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c155960 0x7fcd3c06a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c16a500 con 0x7fcd3c06de20 2023-09-27T23:50:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c146680 con 0x7fcd3c155960 2023-09-27T23:50:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0aa730 con 0x7fcd3c0c7f00 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 0x7fcd3c169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 0x7fcd3c169fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55928/0 (socket says 172.21.15.64:55928) 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 -- 172.21.15.64:0/275026523 learned_addr learned my addr 172.21.15.64:0/275026523 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd327fc700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c0c7f00 0x7fcd3c0a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c155960 0x7fcd3c06a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 -- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c0c7f00 msgr2=0x7fcd3c0a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c0c7f00 0x7fcd3c0a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 -- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c155960 msgr2=0x7fcd3c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c155960 0x7fcd3c06a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 -- 172.21.15.64:0/275026523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34009ae0 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.627+0000 7fcd3affd700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 0x7fcd3c169fc0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00e930 tx=0x7fcd2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c013680 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0137e0 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c0aaa10 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c013b80 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c0aafd0 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fcd2c00c460 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c04be40 con 0x7fcd3c06de20 2023-09-27T23:50:31.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd337fe700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcd1c060a70 0x7fcd1c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fcd2c07cf60 con 0x7fcd3c06de20 2023-09-27T23:50:31.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd327fc700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcd1c060a70 0x7fcd1c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.631+0000 7fcd327fc700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcd1c060a70 0x7fcd1c062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c149bc0 tx=0x7fcd24007d30 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.639+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcd2c04d2e0 con 0x7fcd3c06de20 2023-09-27T23:50:31.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.807+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd3c06d340 con 0x7fcd3c06de20 2023-09-27T23:50:31.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.807+0000 7fcd337fe700 1 -- 172.21.15.64:0/275026523 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+13865 (secure 0 0 0) 0x7fcd2c050900 con 0x7fcd3c06de20 2023-09-27T23:50:31.810 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:31.810 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":170,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:50:31.113697+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:31.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcd1c060a70 msgr2=0x7fcd1c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcd1c060a70 0x7fcd1c062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c149bc0 tx=0x7fcd24007d30 comp rx=0 tx=0).stop 2023-09-27T23:50:31.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 msgr2=0x7fcd3c169fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 0x7fcd3c169fc0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00e930 tx=0x7fcd2c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 shutdown_connections 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcd1c060a70 0x7fcd1c062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c155960 0x7fcd3c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd3c0c7f00 0x7fcd3c0a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/275026523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c06de20 0x7fcd3c169fc0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 >> 172.21.15.64:0/275026523 conn(0x7fcd3c067bb0 msgr2=0x7fcd3c0af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 shutdown_connections 2023-09-27T23:50:31.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.811+0000 7fcd3bfff700 1 -- 172.21.15.64:0/275026523 wait complete. 2023-09-27T23:50:31.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:50:31.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:50:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 Processor -- start 2023-09-27T23:50:31.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 -- start start 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff40806aeb0 0x7ff408159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff408159a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40806c850 con 0x7ff408158c90 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4080cb650 con 0x7ff40806aeb0 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff407fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40806b470 con 0x7ff4081595a0 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff40806aeb0 0x7ff408159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff40806aeb0 0x7ff408159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55942/0 (socket says 172.21.15.64:55942) 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 -- 172.21.15.64:0/646893550 learned_addr learned my addr 172.21.15.64:0/646893550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:31.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4077fe700 1 --2- 172.21.15.64:0/646893550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff408159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 -- 172.21.15.64:0/646893550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 msgr2=0x7ff408159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 --2- 172.21.15.64:0/646893550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff408159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 -- 172.21.15.64:0/646893550 >> v1:172.21.15.123:6789/0 conn(0x7ff408158c90 legacy=0x7ff4080caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 -- 172.21.15.64:0/646893550 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40806b750 con 0x7ff40806aeb0 2023-09-27T23:50:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.935+0000 7ff4067fc700 1 --2- 172.21.15.64:0/646893550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff408159060 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0099b0 tx=0x7ff3fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af65b17bbd3a830 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff405ffb700 1 -- 172.21.15.64:0/646893550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc009ea0 con 0x7ff40806aeb0 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff405ffb700 1 -- 172.21.15.64:0/646893550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc00b860 con 0x7ff40806aeb0 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff405ffb700 1 -- 172.21.15.64:0/646893550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00bc00 con 0x7ff40806aeb0 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 -- 172.21.15.64:0/646893550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 msgr2=0x7ff408159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 --2- 172.21.15.64:0/646893550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff408159060 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0099b0 tx=0x7ff3fc009250 comp rx=0 tx=0).stop 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 -- 172.21.15.64:0/646893550 shutdown_connections 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 --2- 172.21.15.64:0/646893550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff408159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 --2- 172.21.15.64:0/646893550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff408159060 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 -- 172.21.15.64:0/646893550 >> 172.21.15.64:0/646893550 conn(0x7ff408067bb0 msgr2=0x7ff4080cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 -- 172.21.15.64:0/646893550 shutdown_connections 2023-09-27T23:50:31.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 -- 172.21.15.64:0/646893550 wait complete. 2023-09-27T23:50:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 Processor -- start 2023-09-27T23:50:31.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.939+0000 7ff407fff700 1 -- start start 2023-09-27T23:50:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff4081488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff408158c90 0x7ff4081575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff4080c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408148e10 con 0x7ff40806aeb0 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080c5a50 con 0x7ff408158c90 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080c4e80 con 0x7ff4081595a0 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff406ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff4081488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff406ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff4081488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55948/0 (socket says 172.21.15.64:55948) 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff406ffd700 1 -- 172.21.15.64:0/1492854033 learned_addr learned my addr 172.21.15.64:0/1492854033 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff4080c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4067fc700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff408158c90 0x7ff4081575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 -- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff408158c90 msgr2=0x7ff4081575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff408158c90 0x7ff4081575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 -- 172.21.15.64:0/1492854033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 msgr2=0x7ff4081488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff4081488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 -- 172.21.15.64:0/1492854033 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc0094f0 con 0x7ff4081595a0 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff4077fe700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff4080c66b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff40000ecc0 tx=0x7ff40000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000c8c0 con 0x7ff4081595a0 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40000f040 con 0x7ff4081595a0 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000cc90 con 0x7ff4081595a0 2023-09-27T23:50:31.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4080c5120 con 0x7ff4081595a0 2023-09-27T23:50:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.943+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4080c56e0 con 0x7ff4081595a0 2023-09-27T23:50:31.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.947+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40804be40 con 0x7ff4081595a0 2023-09-27T23:50:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.947+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff400014690 con 0x7ff4081595a0 2023-09-27T23:50:31.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.947+0000 7ff3eeffd700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff3f4060a70 0x7ff3f4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.947+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5526+0+0 (secure 0 0 0) 0x7ff40007da90 con 0x7ff4081595a0 2023-09-27T23:50:31.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.947+0000 7ff406ffd700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff3f4060a70 0x7ff3f4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:31.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.947+0000 7ff406ffd700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff3f4060a70 0x7ff3f4062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff408157350 tx=0x7ff3f8009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:31.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:31.955+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff40004dc60 con 0x7ff4081595a0 2023-09-27T23:50:32.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.119+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4080c5b90 con 0x7ff4081595a0 2023-09-27T23:50:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.119+0000 7ff3eeffd700 1 -- 172.21.15.64:0/1492854033 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff400018020 con 0x7ff4081595a0 2023-09-27T23:50:32.124 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:50:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff3f4060a70 msgr2=0x7ff3f4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff3f4060a70 0x7ff3f4062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff408157350 tx=0x7ff3f8009040 comp rx=0 tx=0).stop 2023-09-27T23:50:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 msgr2=0x7ff4080c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff4080c66b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff40000ecc0 tx=0x7ff40000b040 comp rx=0 tx=0).stop 2023-09-27T23:50:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 shutdown_connections 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff3f4060a70 0x7ff3f4062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4081595a0 0x7ff4080c66b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff408158c90 0x7ff4081575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 --2- 172.21.15.64:0/1492854033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff40806aeb0 0x7ff4081488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 >> 172.21.15.64:0/1492854033 conn(0x7ff408067bb0 msgr2=0x7ff408062ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 shutdown_connections 2023-09-27T23:50:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.123+0000 7ff407fff700 1 -- 172.21.15.64:0/1492854033 wait complete. 2023-09-27T23:50:32.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:50:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.243+0000 7f907e41f700 1 Processor -- start 2023-09-27T23:50:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.243+0000 7f907e41f700 1 -- start start 2023-09-27T23:50:32.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f90780acf30 0x7f90781557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9078156780 con 0x7f90780ad300 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90781532a0 con 0x7f90780acf30 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9078158f90 con 0x7f907806d1d0 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907d41d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f90780acf30 0x7f90781557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:53094/0 (socket says 172.21.15.64:53094) 2023-09-27T23:50:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 -- 172.21.15.64:0/1461924136 learned_addr learned my addr 172.21.15.64:0/1461924136 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:32.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907cc1c700 1 --1- 172.21.15.64:0/1461924136 >> v1:172.21.15.123:6789/0 conn(0x7f90780ad300 0x7f9078159420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:49798/0 (socket says 172.21.15.64:49798) 2023-09-27T23:50:32.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 -- 172.21.15.64:0/1461924136 >> v2:172.21.15.123:3300/0 conn(0x7f90780acf30 msgr2=0x7f90781557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:32.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 --2- 172.21.15.64:0/1461924136 >> v2:172.21.15.123:3300/0 conn(0x7f90780acf30 0x7f90781557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 -- 172.21.15.64:0/1461924136 >> v1:172.21.15.123:6789/0 conn(0x7f90780ad300 legacy=0x7f9078159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 -- 172.21.15.64:0/1461924136 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907806d650 con 0x7f907806d1d0 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907dc1e700 1 --2- 172.21.15.64:0/1461924136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780ac1e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f90700099b0 tx=0x7f9070009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47cb8611667105cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f9077fff700 1 -- 172.21.15.64:0/1461924136 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070009ea0 con 0x7f907806d1d0 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f9077fff700 1 -- 172.21.15.64:0/1461924136 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907000b860 con 0x7f907806d1d0 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f9077fff700 1 -- 172.21.15.64:0/1461924136 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907000bc00 con 0x7f907806d1d0 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- 172.21.15.64:0/1461924136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 msgr2=0x7f90780ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 --2- 172.21.15.64:0/1461924136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780ac1e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f90700099b0 tx=0x7f9070009250 comp rx=0 tx=0).stop 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- 172.21.15.64:0/1461924136 shutdown_connections 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 --2- 172.21.15.64:0/1461924136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780ac1e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 --2- 172.21.15.64:0/1461924136 >> v2:172.21.15.123:3300/0 conn(0x7f90780acf30 0x7f90781557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- 172.21.15.64:0/1461924136 >> 172.21.15.64:0/1461924136 conn(0x7f9078067b50 msgr2=0x7f90780cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:32.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- 172.21.15.64:0/1461924136 shutdown_connections 2023-09-27T23:50:32.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.247+0000 7f907e41f700 1 -- 172.21.15.64:0/1461924136 wait complete. 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 Processor -- start 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 -- start start 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90780acf30 0x7f90781437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ad300 0x7f9078157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078147790 con 0x7f90780acf30 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b0dc0 con 0x7f90780ad300 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b10a0 con 0x7f907806d1d0 2023-09-27T23:50:32.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907cc1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90780acf30 0x7f90781437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ad300 0x7f9078157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907d41d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907cc1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90780acf30 0x7f90781437c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55952/0 (socket says 172.21.15.64:55952) 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907cc1c700 1 -- 172.21.15.64:0/2782634007 learned_addr learned my addr 172.21.15.64:0/2782634007 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 -- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 msgr2=0x7f90780aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 -- 172.21.15.64:0/2782634007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90780acf30 msgr2=0x7f90781437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90780acf30 0x7f90781437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 -- 172.21.15.64:0/2782634007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700094f0 con 0x7f90780ad300 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907dc1e700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ad300 0x7f9078157df0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f907806e830 tx=0x7f9070017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907000b9d0 con 0x7f90780ad300 2023-09-27T23:50:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907000bb50 con 0x7f90780ad300 2023-09-27T23:50:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070025670 con 0x7f90780ad300 2023-09-27T23:50:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90780b1380 con 0x7f90780ad300 2023-09-27T23:50:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.251+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90780af390 con 0x7f90780ad300 2023-09-27T23:50:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.255+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90781479b0 con 0x7f90780ad300 2023-09-27T23:50:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.255+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9070023070 con 0x7f90780ad300 2023-09-27T23:50:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.255+0000 7f90757fa700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9058060a70 0x7f9058062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.255+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f907007de40 con 0x7f90780ad300 2023-09-27T23:50:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.255+0000 7f907d41d700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9058060a70 0x7f9058062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.255+0000 7f907d41d700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9058060a70 0x7f9058062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9060009d10 tx=0x7f90600091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.263+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f907004e1c0 con 0x7f90780ad300 2023-09-27T23:50:32.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.427+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90781479b0 con 0x7f90780ad300 2023-09-27T23:50:32.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.427+0000 7f90757fa700 1 -- 172.21.15.64:0/2782634007 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v170) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90781479b0 con 0x7f90780ad300 2023-09-27T23:50:32.430 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9058060a70 msgr2=0x7f9058062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9058060a70 0x7f9058062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9060009d10 tx=0x7f90600091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ad300 msgr2=0x7f9078157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ad300 0x7f9078157df0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f907806e830 tx=0x7f9070017000 comp rx=0 tx=0).stop 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 shutdown_connections 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9058060a70 0x7f9058062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90780ad300 0x7f9078157df0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90780acf30 0x7f90781437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 --2- 172.21.15.64:0/2782634007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f907806d1d0 0x7f90780aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 >> 172.21.15.64:0/2782634007 conn(0x7f9078067b50 msgr2=0x7f90780c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 shutdown_connections 2023-09-27T23:50:32.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.431+0000 7f907e41f700 1 -- 172.21.15.64:0/2782634007 wait complete. 2023-09-27T23:50:32.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:50:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 Processor -- start 2023-09-27T23:50:32.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 -- start start 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480cbdb0 0x7fd64806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd6480c2d30 0x7fd6481605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6480c92c0 con 0x7fd64806d1f0 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6480c8450 con 0x7fd6480c2d30 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64fa0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480c86b0 con 0x7fd6480cbdb0 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64d7ab700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd64806d1f0 0x7fd6480c7d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52948/0 (socket says 172.21.15.64:52948) 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64d7ab700 1 -- 172.21.15.64:0/3943038516 learned_addr learned my addr 172.21.15.64:0/3943038516 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64dfac700 1 --2- 172.21.15.64:0/3943038516 >> v2:172.21.15.64:3300/0 conn(0x7fd6480c2d30 0x7fd6481605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64cfaa700 1 --2- 172.21.15.64:0/3943038516 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480cbdb0 0x7fd64806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd647fff700 1 -- 172.21.15.64:0/3943038516 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534647981 0 0) 0x7fd6480c92c0 con 0x7fd64806d1f0 2023-09-27T23:50:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd647fff700 1 -- 172.21.15.64:0/3943038516 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd634003570 con 0x7fd64806d1f0 2023-09-27T23:50:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64dfac700 1 -- 172.21.15.64:0/3943038516 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480cbdb0 msgr2=0x7fd64806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64dfac700 1 --2- 172.21.15.64:0/3943038516 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480cbdb0 0x7fd64806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64dfac700 1 -- 172.21.15.64:0/3943038516 >> v1:172.21.15.64:6789/0 conn(0x7fd64806d1f0 legacy=0x7fd6480c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64dfac700 1 -- 172.21.15.64:0/3943038516 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480c8990 con 0x7fd6480c2d30 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd64dfac700 1 --2- 172.21.15.64:0/3943038516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480c2d30 0x7fd6481605e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098e0 tx=0x7fd6380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30970166d15ed1ea server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd647fff700 1 -- 172.21.15.64:0/3943038516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638011630 con 0x7fd6480c2d30 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd647fff700 1 -- 172.21.15.64:0/3943038516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd638011790 con 0x7fd6480c2d30 2023-09-27T23:50:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.551+0000 7fd647fff700 1 -- 172.21.15.64:0/3943038516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638011b30 con 0x7fd6480c2d30 2023-09-27T23:50:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3943038516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480c2d30 msgr2=0x7fd6481605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3943038516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480c2d30 0x7fd6481605e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098e0 tx=0x7fd6380091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3943038516 shutdown_connections 2023-09-27T23:50:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3943038516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480c2d30 0x7fd6481605e0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3943038516 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480cbdb0 0x7fd64806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3943038516 >> 172.21.15.64:0/3943038516 conn(0x7fd6480cd9c0 msgr2=0x7fd6480c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3943038516 shutdown_connections 2023-09-27T23:50:32.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3943038516 wait complete. 2023-09-27T23:50:32.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 Processor -- start 2023-09-27T23:50:32.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- start start 2023-09-27T23:50:32.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64806d1f0 0x7fd64815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480c2d30 0x7fd64815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 0x7fd648159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64815da60 con 0x7fd6480cbdb0 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480af3d0 con 0x7fd64806d1f0 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64fa0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64815f650 con 0x7fd6480c2d30 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64dfac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 0x7fd648159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64dfac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 0x7fd648159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55974/0 (socket says 172.21.15.64:55974) 2023-09-27T23:50:32.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.555+0000 7fd64dfac700 1 -- 172.21.15.64:0/3813874587 learned_addr learned my addr 172.21.15.64:0/3813874587 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64cfaa700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480c2d30 0x7fd64815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64d7ab700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64806d1f0 0x7fd64815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64dfac700 1 -- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480c2d30 msgr2=0x7fd64815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64dfac700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480c2d30 0x7fd64815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64dfac700 1 -- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64806d1f0 msgr2=0x7fd64815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64dfac700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64806d1f0 0x7fd64815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64dfac700 1 -- 172.21.15.64:0/3813874587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638009420 con 0x7fd6480cbdb0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64dfac700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 0x7fd648159fa0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd64806cfa0 tx=0x7fd638017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638011b10 con 0x7fd6480cbdb0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6380233f0 con 0x7fd6480cbdb0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6480af0f0 con 0x7fd6480cbdb0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638023770 con 0x7fd6480cbdb0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64815f2e0 con 0x7fd6480cbdb0 2023-09-27T23:50:32.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64815ee60 con 0x7fd6480cbdb0 2023-09-27T23:50:32.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd638023b20 con 0x7fd6480cbdb0 2023-09-27T23:50:32.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.559+0000 7fd6457fa700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd628060aa0 0x7fd628062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.563+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd63807e110 con 0x7fd6480cbdb0 2023-09-27T23:50:32.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.563+0000 7fd64d7ab700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd628060aa0 0x7fd628062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.563+0000 7fd64d7ab700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd628060aa0 0x7fd628062f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd640005c50 tx=0x7fd640009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.567+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd64815ee60 con 0x7fd6480cbdb0 2023-09-27T23:50:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.731+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd64815ee60 con 0x7fd6480cbdb0 2023-09-27T23:50:32.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd6457fa700 1 -- 172.21.15.64:0/3813874587 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v170) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd638015090 con 0x7fd6480cbdb0 2023-09-27T23:50:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:32.737 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd628060aa0 msgr2=0x7fd628062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd628060aa0 0x7fd628062f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd640005c50 tx=0x7fd640009040 comp rx=0 tx=0).stop 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 msgr2=0x7fd648159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 0x7fd648159fa0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd64806cfa0 tx=0x7fd638017000 comp rx=0 tx=0).stop 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 shutdown_connections 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd628060aa0 0x7fd628062f50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6480cbdb0 0x7fd648159fa0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6480c2d30 0x7fd64815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 --2- 172.21.15.64:0/3813874587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64806d1f0 0x7fd64815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 >> 172.21.15.64:0/3813874587 conn(0x7fd6480cd9c0 msgr2=0x7fd648067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 shutdown_connections 2023-09-27T23:50:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.735+0000 7fd64fa0f700 1 -- 172.21.15.64:0/3813874587 wait complete. 2023-09-27T23:50:32.751 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:50:32.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:50:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 Processor -- start 2023-09-27T23:50:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 -- start start 2023-09-27T23:50:32.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b8c155590 0x7f5b8c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c0ac9d0 0x7f5b8c0ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8c069900 con 0x7f5b8c1551c0 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8c1559f0 con 0x7f5b8c155590 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b9087f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0c7340 con 0x7f5b8c0ac9d0 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b83fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b8c155590 0x7f5b8c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b8a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5b8c1551c0 0x7f5b8c0690d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52952/0 (socket says 172.21.15.64:52952) 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b8a59c700 1 -- 172.21.15.64:0/3235855591 learned_addr learned my addr 172.21.15.64:0/3235855591 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b8ad9d700 1 --2- 172.21.15.64:0/3235855591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c0ac9d0 0x7f5b8c0ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b83fff700 1 -- 172.21.15.64:0/3235855591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c0ac9d0 msgr2=0x7f5b8c0ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b83fff700 1 --2- 172.21.15.64:0/3235855591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c0ac9d0 0x7f5b8c0ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b83fff700 1 -- 172.21.15.64:0/3235855591 >> v1:172.21.15.64:6789/0 conn(0x7f5b8c1551c0 legacy=0x7f5b8c0690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b83fff700 1 -- 172.21.15.64:0/3235855591 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0c6a10 con 0x7f5b8c155590 2023-09-27T23:50:32.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b83fff700 1 --2- 172.21.15.64:0/3235855591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c155590 0x7f5b8c159b30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5b740098e0 tx=0x7f5b740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90048880732b95ea server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b89d9b700 1 -- 172.21.15.64:0/3235855591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400f6b0 con 0x7f5b8c155590 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b89d9b700 1 -- 172.21.15.64:0/3235855591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400f810 con 0x7f5b8c155590 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.867+0000 7f5b89d9b700 1 -- 172.21.15.64:0/3235855591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400fbb0 con 0x7f5b8c155590 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- 172.21.15.64:0/3235855591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c155590 msgr2=0x7f5b8c159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 --2- 172.21.15.64:0/3235855591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c155590 0x7f5b8c159b30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5b740098e0 tx=0x7f5b740091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- 172.21.15.64:0/3235855591 shutdown_connections 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 --2- 172.21.15.64:0/3235855591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c0ac9d0 0x7f5b8c0ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 --2- 172.21.15.64:0/3235855591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c155590 0x7f5b8c159b30 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- 172.21.15.64:0/3235855591 >> 172.21.15.64:0/3235855591 conn(0x7f5b8c0c8db0 msgr2=0x7f5b8c0c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:32.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- 172.21.15.64:0/3235855591 shutdown_connections 2023-09-27T23:50:32.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- 172.21.15.64:0/3235855591 wait complete. 2023-09-27T23:50:32.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 Processor -- start 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- start start 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8c0ac9d0 0x7f5b8c168a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 0x7f5b8c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c155590 0x7f5b8c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0b1fe0 con 0x7f5b8c1551c0 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0b1360 con 0x7f5b8c0ac9d0 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b9087f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c0b1640 con 0x7f5b8c155590 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 0x7f5b8c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 0x7f5b8c0ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55994/0 (socket says 172.21.15.64:55994) 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 -- 172.21.15.64:0/45854219 learned_addr learned my addr 172.21.15.64:0/45854219 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b8a59c700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8c0ac9d0 0x7f5b8c168a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b8ad9d700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c155590 0x7f5b8c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 -- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c155590 msgr2=0x7f5b8c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c155590 0x7f5b8c157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 -- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8c0ac9d0 msgr2=0x7f5b8c168a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8c0ac9d0 0x7f5b8c168a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b83fff700 1 -- 172.21.15.64:0/45854219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b74009420 con 0x7f5b8c1551c0 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.871+0000 7f5b8ad9d700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c155590 0x7f5b8c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:32.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b83fff700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 0x7f5b8c0ca920 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f5b740093f0 tx=0x7f5b74019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7401acd0 con 0x7f5b8c1551c0 2023-09-27T23:50:32.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7401ae30 con 0x7f5b8c1551c0 2023-09-27T23:50:32.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8c0b1920 con 0x7f5b8c1551c0 2023-09-27T23:50:32.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740106c0 con 0x7f5b8c1551c0 2023-09-27T23:50:32.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8c0a8c80 con 0x7f5b8c1551c0 2023-09-27T23:50:32.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8c071120 con 0x7f5b8c1551c0 2023-09-27T23:50:32.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5b740183e0 con 0x7f5b8c1551c0 2023-09-27T23:50:32.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b827fc700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5b6c060aa0 0x7f5b6c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:32.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.875+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f5b74014070 con 0x7f5b8c1551c0 2023-09-27T23:50:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.879+0000 7f5b8a59c700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5b6c060aa0 0x7f5b6c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.879+0000 7f5b8a59c700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5b6c060aa0 0x7f5b6c062f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c007130 tx=0x7f5b7c007eb0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:32.883+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5b7404eab0 con 0x7f5b8c1551c0 2023-09-27T23:50:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.047+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5b8c0c6cf0 con 0x7f5b8c1551c0 2023-09-27T23:50:33.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.147+0000 7f5b827fc700 1 -- 172.21.15.64:0/45854219 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v171) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5b740520d0 con 0x7f5b8c1551c0 2023-09-27T23:50:33.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5b6c060aa0 msgr2=0x7f5b6c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:33.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5b6c060aa0 0x7f5b6c062f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c007130 tx=0x7f5b7c007eb0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 msgr2=0x7f5b8c0ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 0x7f5b8c0ca920 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f5b740093f0 tx=0x7f5b74019000 comp rx=0 tx=0).stop 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 shutdown_connections 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5b6c060aa0 0x7f5b6c062f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8c155590 0x7f5b8c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b8c1551c0 0x7f5b8c0ca920 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 --2- 172.21.15.64:0/45854219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8c0ac9d0 0x7f5b8c168a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.151+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 >> 172.21.15.64:0/45854219 conn(0x7f5b8c0c8db0 msgr2=0x7f5b8c0adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:33.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.155+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 shutdown_connections 2023-09-27T23:50:33.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.155+0000 7f5b9087f700 1 -- 172.21.15.64:0/45854219 wait complete. 2023-09-27T23:50:33.159 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:50:33.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:50:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.275+0000 7f2e225ba700 1 Processor -- start 2023-09-27T23:50:33.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.275+0000 7f2e225ba700 1 -- start start 2023-09-27T23:50:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e225ba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2e1c155a90 0x7f2e1c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e225ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2e1c06db70 0x7f2e1c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e225ba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c0c68a0 con 0x7f2e1c155e60 2023-09-27T23:50:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e225ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c6e60 con 0x7f2e1c06db70 2023-09-27T23:50:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e225ba700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c0ae4d0 con 0x7f2e1c155a90 2023-09-27T23:50:33.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2e1c06db70 0x7f2e1c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:33.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e21db9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2e1c155e60 0x7f2e1c0adca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52954/0 (socket says 172.21.15.64:52954) 2023-09-27T23:50:33.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e21db9700 1 -- 172.21.15.64:0/3044726423 learned_addr learned my addr 172.21.15.64:0/3044726423 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:33.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2e1c06db70 0x7f2e1c0ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56006/0 (socket says 172.21.15.64:56006) 2023-09-27T23:50:33.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e215b8700 1 --2- 172.21.15.64:0/3044726423 >> v2:172.21.15.123:3300/0 conn(0x7f2e1c155a90 0x7f2e1c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 -- 172.21.15.64:0/3044726423 >> v2:172.21.15.123:3300/0 conn(0x7f2e1c155a90 msgr2=0x7f2e1c0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 --2- 172.21.15.64:0/3044726423 >> v2:172.21.15.123:3300/0 conn(0x7f2e1c155a90 0x7f2e1c0ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 -- 172.21.15.64:0/3044726423 >> v1:172.21.15.64:6789/0 conn(0x7f2e1c155e60 legacy=0x7f2e1c0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e13fff700 1 -- 172.21.15.64:0/3044726423 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323386142 0 0) 0x7f2e1c0c68a0 con 0x7f2e1c155e60 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 -- 172.21.15.64:0/3044726423 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0ae7b0 con 0x7f2e1c06db70 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e20db7700 1 --2- 172.21.15.64:0/3044726423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c06db70 0x7f2e1c0ad760 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c06a8d0 tx=0x7f2e0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46f6b5f4c2a0071 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e13fff700 1 -- 172.21.15.64:0/3044726423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c011630 con 0x7f2e1c06db70 2023-09-27T23:50:33.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e13fff700 1 -- 172.21.15.64:0/3044726423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c011790 con 0x7f2e1c06db70 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.279+0000 7f2e13fff700 1 -- 172.21.15.64:0/3044726423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c011b30 con 0x7f2e1c06db70 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- 172.21.15.64:0/3044726423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c06db70 msgr2=0x7f2e1c0ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 --2- 172.21.15.64:0/3044726423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c06db70 0x7f2e1c0ad760 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c06a8d0 tx=0x7f2e0c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- 172.21.15.64:0/3044726423 shutdown_connections 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 --2- 172.21.15.64:0/3044726423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c06db70 0x7f2e1c0ad760 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 --2- 172.21.15.64:0/3044726423 >> v2:172.21.15.123:3300/0 conn(0x7f2e1c155a90 0x7f2e1c0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- 172.21.15.64:0/3044726423 >> 172.21.15.64:0/3044726423 conn(0x7f2e1c0c8cd0 msgr2=0x7f2e1c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:33.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- 172.21.15.64:0/3044726423 shutdown_connections 2023-09-27T23:50:33.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- 172.21.15.64:0/3044726423 wait complete. 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 Processor -- start 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- start start 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e1c06db70 0x7f2e1c14b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 0x7f2e1c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e1c155e60 0x7f2e1c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0b0c30 con 0x7f2e1c155a90 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0b0050 con 0x7f2e1c06db70 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e225ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0afbc0 con 0x7f2e1c155e60 2023-09-27T23:50:33.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 0x7f2e1c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 0x7f2e1c159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56012/0 (socket says 172.21.15.64:56012) 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 -- 172.21.15.64:0/672192660 learned_addr learned my addr 172.21.15.64:0/672192660 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e215b8700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e1c06db70 0x7f2e1c14b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e21db9700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e1c155e60 0x7f2e1c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 -- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e1c155e60 msgr2=0x7f2e1c0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e1c155e60 0x7f2e1c0b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 -- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e1c06db70 msgr2=0x7f2e1c14b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e1c06db70 0x7f2e1c14b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:33.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e20db7700 1 -- 172.21.15.64:0/672192660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009420 con 0x7f2e1c155a90 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.283+0000 7f2e215b8700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e1c06db70 0x7f2e1c14b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e20db7700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 0x7f2e1c159010 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009f50 tx=0x7f2e0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c000d10 con 0x7f2e1c155a90 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c011900 con 0x7f2e1c155a90 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c0af290 con 0x7f2e1c155a90 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c025420 con 0x7f2e1c155a90 2023-09-27T23:50:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c0af850 con 0x7f2e1c155a90 2023-09-27T23:50:33.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c0afd50 con 0x7f2e1c155a90 2023-09-27T23:50:33.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.287+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2e0c023070 con 0x7f2e1c155a90 2023-09-27T23:50:33.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.291+0000 7f2e117fa700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e00060b40 0x7f2e00062ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:33.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.291+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f2e0c07e5c0 con 0x7f2e1c155a90 2023-09-27T23:50:33.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.291+0000 7f2e215b8700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e00060b40 0x7f2e00062ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:33.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.291+0000 7f2e215b8700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e00060b40 0x7f2e00062ff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e18011480 tx=0x7f2e180091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:33.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.295+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2e0c04e830 con 0x7f2e1c155a90 2023-09-27T23:50:33.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:33.451+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2e1c04be40 con 0x7f2e1c155a90 2023-09-27T23:50:34.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.151+0000 7f2e117fa700 1 -- 172.21.15.64:0/672192660 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v172) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2e0c051e50 con 0x7f2e1c155a90 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e00060b40 msgr2=0x7f2e00062ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e00060b40 0x7f2e00062ff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e18011480 tx=0x7f2e180091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 msgr2=0x7f2e1c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 0x7f2e1c159010 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009f50 tx=0x7f2e0c017000 comp rx=0 tx=0).stop 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 shutdown_connections 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e00060b40 0x7f2e00062ff0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2e1c155e60 0x7f2e1c0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e1c155a90 0x7f2e1c159010 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 --2- 172.21.15.64:0/672192660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e1c06db70 0x7f2e1c14b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 >> 172.21.15.64:0/672192660 conn(0x7f2e1c0c8cd0 msgr2=0x7f2e1c0ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 shutdown_connections 2023-09-27T23:50:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.175+0000 7f2e225ba700 1 -- 172.21.15.64:0/672192660 wait complete. 2023-09-27T23:50:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:50:34.193 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:50:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 Processor -- start 2023-09-27T23:50:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 -- start start 2023-09-27T23:50:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 0x7fc73c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc73c06b530 0x7fc73c0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c0ad020 con 0x7fc73c0ae6c0 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c0c81e0 con 0x7fc73c06b530 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc7408f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0aca10 con 0x7fc73c155520 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 0x7fc73c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc73c06b530 0x7fc73c0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 0x7fc73c0c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:53174/0 (socket says 172.21.15.64:53174) 2023-09-27T23:50:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 -- 172.21.15.64:0/3756472044 learned_addr learned my addr 172.21.15.64:0/3756472044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 -- 172.21.15.64:0/3756472044 >> v2:172.21.15.123:3300/0 conn(0x7fc73c06b530 msgr2=0x7fc73c0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 --2- 172.21.15.64:0/3756472044 >> v2:172.21.15.123:3300/0 conn(0x7fc73c06b530 0x7fc73c0ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 -- 172.21.15.64:0/3756472044 >> v1:172.21.15.123:6789/0 conn(0x7fc73c0ae6c0 legacy=0x7fc73c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73ad9d700 1 -- 172.21.15.64:0/3756472044 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c0c7c20 con 0x7fc73c155520 2023-09-27T23:50:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.299+0000 7fc73b59e700 1 --2- 172.21.15.64:0/3756472044 >> v2:172.21.15.123:3300/0 conn(0x7fc73c06b530 0x7fc73c0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc73ad9d700 1 --2- 172.21.15.64:0/3756472044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 0x7fc73c0c7130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0099b0 tx=0x7fc72c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321e582103df133f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc73a59c700 1 -- 172.21.15.64:0/3756472044 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c009ea0 con 0x7fc73c155520 2023-09-27T23:50:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc73a59c700 1 -- 172.21.15.64:0/3756472044 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00b8f0 con 0x7fc73c155520 2023-09-27T23:50:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc73a59c700 1 -- 172.21.15.64:0/3756472044 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00bc90 con 0x7fc73c155520 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 -- 172.21.15.64:0/3756472044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 msgr2=0x7fc73c0c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3756472044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 0x7fc73c0c7130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0099b0 tx=0x7fc72c009250 comp rx=0 tx=0).stop 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 -- 172.21.15.64:0/3756472044 shutdown_connections 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3756472044 >> v2:172.21.15.123:3300/0 conn(0x7fc73c06b530 0x7fc73c0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3756472044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c155520 0x7fc73c0c7130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 -- 172.21.15.64:0/3756472044 >> 172.21.15.64:0/3756472044 conn(0x7fc73c0c8cd0 msgr2=0x7fc73c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 -- 172.21.15.64:0/3756472044 shutdown_connections 2023-09-27T23:50:34.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.303+0000 7fc7408f7700 1 -- 172.21.15.64:0/3756472044 wait complete. 2023-09-27T23:50:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 Processor -- start 2023-09-27T23:50:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 -- start start 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c06b530 0x7fc73c169660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c0ae6c0 0x7fc73c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 0x7fc73c152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0ab110 con 0x7fc73c155520 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c169ba0 con 0x7fc73c0ae6c0 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0c56c0 con 0x7fc73c06b530 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 0x7fc73c152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 0x7fc73c152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56028/0 (socket says 172.21.15.64:56028) 2023-09-27T23:50:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc733fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c0ae6c0 0x7fc73c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 -- 172.21.15.64:0/3986672542 learned_addr learned my addr 172.21.15.64:0/3986672542 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73ad9d700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c06b530 0x7fc73c169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 -- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c06b530 msgr2=0x7fc73c169660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c06b530 0x7fc73c169660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 -- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c0ae6c0 msgr2=0x7fc73c0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c0ae6c0 0x7fc73c0ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 -- 172.21.15.64:0/3986672542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73400ab30 con 0x7fc73c155520 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc73b59e700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 0x7fc73c152d10 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc734009e90 tx=0x7fc73400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734010cc0 con 0x7fc73c155520 2023-09-27T23:50:34.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc734010e40 con 0x7fc73c155520 2023-09-27T23:50:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c0094f0 con 0x7fc73c155520 2023-09-27T23:50:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340136b0 con 0x7fc73c155520 2023-09-27T23:50:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.307+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c155c70 con 0x7fc73c155520 2023-09-27T23:50:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.311+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73c04be40 con 0x7fc73c155520 2023-09-27T23:50:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.311+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc734013a20 con 0x7fc73c155520 2023-09-27T23:50:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.311+0000 7fc732ffd700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc71c060a70 0x7fc71c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.311+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fc7340805b0 con 0x7fc73c155520 2023-09-27T23:50:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.311+0000 7fc73ad9d700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc71c060a70 0x7fc71c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.315+0000 7fc73ad9d700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc71c060a70 0x7fc71c062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0099b0 tx=0x7fc72c017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.315+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc734050510 con 0x7fc73c155520 2023-09-27T23:50:34.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.515+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc73c0c9bf0 con 0x7fc73c155520 2023-09-27T23:50:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.531+0000 7fc732ffd700 1 -- 172.21.15.64:0/3986672542 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc7340190c0 con 0x7fc73c155520 2023-09-27T23:50:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.555+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc71c060a70 msgr2=0x7fc71c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.555+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc71c060a70 0x7fc71c062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0099b0 tx=0x7fc72c017000 comp rx=0 tx=0).stop 2023-09-27T23:50:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.555+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 msgr2=0x7fc73c152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.555+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 0x7fc73c152d10 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc734009e90 tx=0x7fc73400c040 comp rx=0 tx=0).stop 2023-09-27T23:50:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 shutdown_connections 2023-09-27T23:50:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc71c060a70 0x7fc71c062f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73c155520 0x7fc73c152d10 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c0ae6c0 0x7fc73c0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 --2- 172.21.15.64:0/3986672542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc73c06b530 0x7fc73c169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 >> 172.21.15.64:0/3986672542 conn(0x7fc73c0c8cd0 msgr2=0x7fc73c0c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:34.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 shutdown_connections 2023-09-27T23:50:34.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.559+0000 7fc7408f7700 1 -- 172.21.15.64:0/3986672542 wait complete. 2023-09-27T23:50:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 22 and data pool 23 2023-09-27T23:50:34.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 Processor -- start 2023-09-27T23:50:34.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 -- start start 2023-09-27T23:50:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f719406d280 0x7f719406d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7194068160 0x7f71940685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71940ad070 con 0x7f71940670e0 2023-09-27T23:50:34.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71940ac780 con 0x7f7194068160 2023-09-27T23:50:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f719a6ba700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7194068e90 con 0x7f719406d280 2023-09-27T23:50:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f7198eb7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7194068160 0x7f71940685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f7198eb7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7194068160 0x7f71940685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54414/0 (socket says 172.21.15.64:54414) 2023-09-27T23:50:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f7198eb7700 1 -- 172.21.15.64:0/2509573410 learned_addr learned my addr 172.21.15.64:0/2509573410 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:34.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.671+0000 7f71996b8700 1 --2- 172.21.15.64:0/2509573410 >> v2:172.21.15.123:3300/0 conn(0x7f719406d280 0x7f719406d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f7198eb7700 1 -- 172.21.15.64:0/2509573410 >> v2:172.21.15.123:3300/0 conn(0x7f719406d280 msgr2=0x7f719406d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f7198eb7700 1 --2- 172.21.15.64:0/2509573410 >> v2:172.21.15.123:3300/0 conn(0x7f719406d280 0x7f719406d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f7198eb7700 1 -- 172.21.15.64:0/2509573410 >> v1:172.21.15.123:6789/0 conn(0x7f71940670e0 legacy=0x7f7194067540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f7198eb7700 1 -- 172.21.15.64:0/2509573410 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940aca20 con 0x7f7194068160 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f7198eb7700 1 --2- 172.21.15.64:0/2509573410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7194068160 0x7f71940685c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7194064b20 tx=0x7f7184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c23114c53feed31 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f718bfff700 1 -- 172.21.15.64:0/2509573410 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184009e70 con 0x7f7194068160 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f718bfff700 1 -- 172.21.15.64:0/2509573410 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718400b8f0 con 0x7f7194068160 2023-09-27T23:50:34.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f718bfff700 1 -- 172.21.15.64:0/2509573410 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718400bc90 con 0x7f7194068160 2023-09-27T23:50:34.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 -- 172.21.15.64:0/2509573410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7194068160 msgr2=0x7f71940685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 --2- 172.21.15.64:0/2509573410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7194068160 0x7f71940685c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7194064b20 tx=0x7f7184009250 comp rx=0 tx=0).stop 2023-09-27T23:50:34.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 -- 172.21.15.64:0/2509573410 shutdown_connections 2023-09-27T23:50:34.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 --2- 172.21.15.64:0/2509573410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7194068160 0x7f71940685c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 --2- 172.21.15.64:0/2509573410 >> v2:172.21.15.123:3300/0 conn(0x7f719406d280 0x7f719406d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 -- 172.21.15.64:0/2509573410 >> 172.21.15.64:0/2509573410 conn(0x7f719406c080 msgr2=0x7f71940ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 -- 172.21.15.64:0/2509573410 shutdown_connections 2023-09-27T23:50:34.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.675+0000 7f719a6ba700 1 -- 172.21.15.64:0/2509573410 wait complete. 2023-09-27T23:50:34.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 Processor -- start 2023-09-27T23:50:34.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 -- start start 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71940670e0 0x7f71940c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7194068160 0x7f7194151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 0x7f7194158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71941547d0 con 0x7f719406d280 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71940c2780 con 0x7f7194068160 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719416c850 con 0x7f71940670e0 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 0x7f7194158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 0x7f7194158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54422/0 (socket says 172.21.15.64:54422) 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 -- 172.21.15.64:0/1076976557 learned_addr learned my addr 172.21.15.64:0/1076976557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f71996b8700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71940670e0 0x7f71940c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7198eb7700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7194068160 0x7f7194151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 -- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71940670e0 msgr2=0x7f71940c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71940670e0 0x7f71940c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 -- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7194068160 msgr2=0x7f7194151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7194068160 0x7f7194151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 -- 172.21.15.64:0/1076976557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71840094f0 con 0x7f719406d280 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f7199eb9700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 0x7f7194158bc0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c8e0 tx=0x7f718c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c012630 con 0x7f719406d280 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c012790 con 0x7f719406d280 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719416cb30 con 0x7f719406d280 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c012b50 con 0x7f719406d280 2023-09-27T23:50:34.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.679+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719416d0f0 con 0x7f719406d280 2023-09-27T23:50:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.683+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f718c01a3a0 con 0x7f719406d280 2023-09-27T23:50:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.683+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719415b980 con 0x7f719406d280 2023-09-27T23:50:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.683+0000 7f71897fa700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7178060a70 0x7f7178062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.683+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f718c07d320 con 0x7f719406d280 2023-09-27T23:50:34.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.683+0000 7f71996b8700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7178060a70 0x7f7178062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.687+0000 7f71996b8700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7178060a70 0x7f7178062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7190009d10 tx=0x7f71900091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.691+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f719415b980 con 0x7f719406d280 2023-09-27T23:50:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.851+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f719415b980 con 0x7f719406d280 2023-09-27T23:50:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.855+0000 7f71897fa700 1 -- 172.21.15.64:0/1076976557 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f719415b980 con 0x7f719406d280 2023-09-27T23:50:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:34.858 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":173,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:50:34.519566+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:50:33.091150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-09-27T23:50:33.495902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7178060a70 msgr2=0x7f7178062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7178060a70 0x7f7178062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7190009d10 tx=0x7f71900091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 msgr2=0x7f7194158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 0x7f7194158bc0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c8e0 tx=0x7f718c001710 comp rx=0 tx=0).stop 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 shutdown_connections 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7178060a70 0x7f7178062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f719406d280 0x7f7194158bc0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7194068160 0x7f7194151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 --2- 172.21.15.64:0/1076976557 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71940670e0 0x7f71940c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 >> 172.21.15.64:0/1076976557 conn(0x7f719406c080 msgr2=0x7f71941525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 shutdown_connections 2023-09-27T23:50:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.859+0000 7f719a6ba700 1 -- 172.21.15.64:0/1076976557 wait complete. 2023-09-27T23:50:34.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:34.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.979+0000 7fcb8ffff700 1 Processor -- start 2023-09-27T23:50:34.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.979+0000 7fcb8ffff700 1 -- start start 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb90158df0 0x7fcb9006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb9006d590 0x7fcb901591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb90159f90 con 0x7fcb90159700 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9006da10 con 0x7fcb90158df0 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9015a270 con 0x7fcb9006d590 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb90158df0 0x7fcb9006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb90158df0 0x7fcb9006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54428/0 (socket says 172.21.15.64:54428) 2023-09-27T23:50:34.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8e7fc700 1 --2- 172.21.15.64:0/2831078356 >> v2:172.21.15.123:3300/0 conn(0x7fcb9006d590 0x7fcb901591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 -- 172.21.15.64:0/2831078356 learned_addr learned my addr 172.21.15.64:0/2831078356 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 -- 172.21.15.64:0/2831078356 >> v2:172.21.15.123:3300/0 conn(0x7fcb9006d590 msgr2=0x7fcb901591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 --2- 172.21.15.64:0/2831078356 >> v2:172.21.15.123:3300/0 conn(0x7fcb9006d590 0x7fcb901591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 -- 172.21.15.64:0/2831078356 >> v1:172.21.15.123:6789/0 conn(0x7fcb90159700 legacy=0x7fcb9015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 -- 172.21.15.64:0/2831078356 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900c7930 con 0x7fcb90158df0 2023-09-27T23:50:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8effd700 1 --2- 172.21.15.64:0/2831078356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb90158df0 0x7fcb9006d050 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb9006b3e0 tx=0x7fcb78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3059f17c00c5bd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8dffb700 1 -- 172.21.15.64:0/2831078356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78009e70 con 0x7fcb90158df0 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8dffb700 1 -- 172.21.15.64:0/2831078356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7800b860 con 0x7fcb90158df0 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8dffb700 1 -- 172.21.15.64:0/2831078356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800bc20 con 0x7fcb90158df0 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- 172.21.15.64:0/2831078356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb90158df0 msgr2=0x7fcb9006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/2831078356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb90158df0 0x7fcb9006d050 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb9006b3e0 tx=0x7fcb78009250 comp rx=0 tx=0).stop 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- 172.21.15.64:0/2831078356 shutdown_connections 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/2831078356 >> v2:172.21.15.123:3300/0 conn(0x7fcb9006d590 0x7fcb901591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/2831078356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb90158df0 0x7fcb9006d050 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- 172.21.15.64:0/2831078356 >> 172.21.15.64:0/2831078356 conn(0x7fcb9006c080 msgr2=0x7fcb900ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- 172.21.15.64:0/2831078356 shutdown_connections 2023-09-27T23:50:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.983+0000 7fcb8ffff700 1 -- 172.21.15.64:0/2831078356 wait complete. 2023-09-27T23:50:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 Processor -- start 2023-09-27T23:50:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 -- start start 2023-09-27T23:50:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 0x7fcb900a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb90158df0 0x7fcb90169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb90159700 0x7fcb90157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9016a1a0 con 0x7fcb9006d590 2023-09-27T23:50:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90157540 con 0x7fcb90158df0 2023-09-27T23:50:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900a9a40 con 0x7fcb90159700 2023-09-27T23:50:34.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 0x7fcb900a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 0x7fcb900a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54434/0 (socket says 172.21.15.64:54434) 2023-09-27T23:50:34.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 -- 172.21.15.64:0/71095532 learned_addr learned my addr 172.21.15.64:0/71095532 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:34.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8e7fc700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb90158df0 0x7fcb90169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8f7fe700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb90159700 0x7fcb90157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 -- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb90159700 msgr2=0x7fcb90157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb90159700 0x7fcb90157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 -- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb90158df0 msgr2=0x7fcb90169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:34.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb90158df0 0x7fcb90169c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 -- 172.21.15.64:0/71095532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780094f0 con 0x7fcb9006d590 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb8effd700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 0x7fcb900a8850 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcb900ca600 tx=0x7fcb78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801f070 con 0x7fcb9006d590 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.987+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78002f30 con 0x7fcb9006d590 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb900a8e30 con 0x7fcb9006d590 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78000bc0 con 0x7fcb9006d590 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb900a93f0 con 0x7fcb9006d590 2023-09-27T23:50:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9004be40 con 0x7fcb9006d590 2023-09-27T23:50:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fcb78023070 con 0x7fcb9006d590 2023-09-27T23:50:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb86ffd700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb70060aa0 0x7fcb70062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fcb7807e350 con 0x7fcb9006d590 2023-09-27T23:50:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.991+0000 7fcb8e7fc700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb70060aa0 0x7fcb70062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:34.995+0000 7fcb8e7fc700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb70060aa0 0x7fcb70062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009cf0 tx=0x7fcb800091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.003+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcb7804e380 con 0x7fcb9006d590 2023-09-27T23:50:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.167+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb900cd1f0 con 0x7fcb9006d590 2023-09-27T23:50:35.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.171+0000 7fcb86ffd700 1 -- 172.21.15.64:0/71095532 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17098 (secure 0 0 0) 0x7fcb7800bbe0 con 0x7fcb9006d590 2023-09-27T23:50:35.175 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:35.176 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":173,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:50:34.519566+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:50:33.091150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-09-27T23:50:33.495902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:35.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb70060aa0 msgr2=0x7fcb70062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb70060aa0 0x7fcb70062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009cf0 tx=0x7fcb800091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 msgr2=0x7fcb900a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 0x7fcb900a8850 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcb900ca600 tx=0x7fcb78017000 comp rx=0 tx=0).stop 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 shutdown_connections 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb70060aa0 0x7fcb70062f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb90159700 0x7fcb90157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb90158df0 0x7fcb90169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 --2- 172.21.15.64:0/71095532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb9006d590 0x7fcb900a8850 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 >> 172.21.15.64:0/71095532 conn(0x7fcb9006c080 msgr2=0x7fcb900ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:35.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 shutdown_connections 2023-09-27T23:50:35.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.175+0000 7fcb8ffff700 1 -- 172.21.15.64:0/71095532 wait complete. 2023-09-27T23:50:35.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:50:35.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.303+0000 7f96b359e700 1 Processor -- start 2023-09-27T23:50:35.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.303+0000 7f96b359e700 1 -- start start 2023-09-27T23:50:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96b413f6a0 0x7f96b40ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b40c7780 0x7f96b40c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b40adc60 con 0x7f96b413f2d0 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b406d7d0 con 0x7f96b413f6a0 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40c8140 con 0x7f96b40c7780 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96b413f6a0 0x7f96b40ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96b413f6a0 0x7f96b40ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54438/0 (socket says 172.21.15.64:54438) 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b259c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f96b413f2d0 0x7f96b415a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40940/0 (socket says 172.21.15.64:40940) 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 -- 172.21.15.64:0/2278051690 learned_addr learned my addr 172.21.15.64:0/2278051690 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b2d9d700 1 --2- 172.21.15.64:0/2278051690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b40c7780 0x7f96b40c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 -- 172.21.15.64:0/2278051690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b40c7780 msgr2=0x7f96b40c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 --2- 172.21.15.64:0/2278051690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b40c7780 0x7f96b40c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b159a700 1 -- 172.21.15.64:0/2278051690 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155306883 0 0) 0x7f96b40adc60 con 0x7f96b413f2d0 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 -- 172.21.15.64:0/2278051690 >> v1:172.21.15.64:6789/0 conn(0x7f96b413f2d0 legacy=0x7f96b415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 -- 172.21.15.64:0/2278051690 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b40ad050 con 0x7f96b413f6a0 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b1d9b700 1 --2- 172.21.15.64:0/2278051690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40ac9b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f969c0099e0 tx=0x7f969c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961d5ce7eb09f896 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b159a700 1 -- 172.21.15.64:0/2278051690 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00b820 con 0x7f96b413f6a0 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b159a700 1 -- 172.21.15.64:0/2278051690 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c00b980 con 0x7f96b413f6a0 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b159a700 1 -- 172.21.15.64:0/2278051690 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00bd20 con 0x7f96b413f6a0 2023-09-27T23:50:35.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- 172.21.15.64:0/2278051690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 msgr2=0x7f96b40ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 --2- 172.21.15.64:0/2278051690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40ac9b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f969c0099e0 tx=0x7f969c009250 comp rx=0 tx=0).stop 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- 172.21.15.64:0/2278051690 shutdown_connections 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 --2- 172.21.15.64:0/2278051690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b40c7780 0x7f96b40c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 --2- 172.21.15.64:0/2278051690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40ac9b0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- 172.21.15.64:0/2278051690 >> 172.21.15.64:0/2278051690 conn(0x7f96b4067b50 msgr2=0x7f96b40cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- 172.21.15.64:0/2278051690 shutdown_connections 2023-09-27T23:50:35.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- 172.21.15.64:0/2278051690 wait complete. 2023-09-27T23:50:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 Processor -- start 2023-09-27T23:50:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.307+0000 7f96b359e700 1 -- start start 2023-09-27T23:50:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b40c7780 0x7f96b416a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b413f2d0 0x7f96b41555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40af960 con 0x7f96b413f6a0 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40aea70 con 0x7f96b40c7780 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40aed50 con 0x7f96b413f2d0 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54440/0 (socket says 172.21.15.64:54440) 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 -- 172.21.15.64:0/2002687392 learned_addr learned my addr 172.21.15.64:0/2002687392 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b259c700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b40c7780 0x7f96b416a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b1d9b700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b413f2d0 0x7f96b41555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 -- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b413f2d0 msgr2=0x7f96b41555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b413f2d0 0x7f96b41555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 -- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b40c7780 msgr2=0x7f96b416a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b40c7780 0x7f96b416a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 -- 172.21.15.64:0/2002687392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0094f0 con 0x7f96b413f6a0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b259c700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b40c7780 0x7f96b416a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b2d9d700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40a4e20 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009df0 tx=0x7f96ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00cb50 con 0x7f96b413f6a0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac00f040 con 0x7f96b413f6a0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac014480 con 0x7f96b413f6a0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b40af030 con 0x7f96b413f6a0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b40af5f0 con 0x7f96b413f6a0 2023-09-27T23:50:35.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.311+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b404be40 con 0x7f96b413f6a0 2023-09-27T23:50:35.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.315+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f96ac014830 con 0x7f96b413f6a0 2023-09-27T23:50:35.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.315+0000 7f96aa7fc700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9698060a70 0x7f9698062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.315+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f96ac07de90 con 0x7f96b413f6a0 2023-09-27T23:50:35.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.315+0000 7f96b259c700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9698060a70 0x7f9698062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.315+0000 7f96b259c700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9698060a70 0x7f9698062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96a40011e0 tx=0x7f96a4007c90 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.323+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f96ac04ddb0 con 0x7f96b413f6a0 2023-09-27T23:50:35.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.519+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f96b406ba60 con 0x7f96b413f6a0 2023-09-27T23:50:35.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.531+0000 7f96aa7fc700 1 -- 172.21.15.64:0/2002687392 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f96ac0180b0 con 0x7f96b413f6a0 2023-09-27T23:50:35.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9698060a70 msgr2=0x7f9698062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9698060a70 0x7f9698062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96a40011e0 tx=0x7f96a4007c90 comp rx=0 tx=0).stop 2023-09-27T23:50:35.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 msgr2=0x7f96b40a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40a4e20 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009df0 tx=0x7f96ac00b040 comp rx=0 tx=0).stop 2023-09-27T23:50:35.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 shutdown_connections 2023-09-27T23:50:35.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9698060a70 0x7f9698062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b413f6a0 0x7f96b40a4e20 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96b413f2d0 0x7f96b41555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 --2- 172.21.15.64:0/2002687392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b40c7780 0x7f96b416a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 >> 172.21.15.64:0/2002687392 conn(0x7f96b4067b50 msgr2=0x7f96b40ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 shutdown_connections 2023-09-27T23:50:35.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.535+0000 7f96b359e700 1 -- 172.21.15.64:0/2002687392 wait complete. 2023-09-27T23:50:35.553 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:50:35.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:50:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.663+0000 7f15a447c700 1 Processor -- start 2023-09-27T23:50:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.663+0000 7f15a447c700 1 -- start start 2023-09-27T23:50:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c158ed0 0x7f159c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f159c1592a0 0x7f159c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c0ca280 con 0x7f159c15d190 2023-09-27T23:50:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c0cca20 con 0x7f159c158ed0 2023-09-27T23:50:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c15d7c0 con 0x7f159c1592a0 2023-09-27T23:50:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c158ed0 0x7f159c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c158ed0 0x7f159c0cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54446/0 (socket says 172.21.15.64:54446) 2023-09-27T23:50:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 -- 172.21.15.64:0/3738594922 learned_addr learned my addr 172.21.15.64:0/3738594922 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:35.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a1a17700 1 --2- 172.21.15.64:0/3738594922 >> v2:172.21.15.123:3300/0 conn(0x7f159c1592a0 0x7f159c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 -- 172.21.15.64:0/3738594922 >> v2:172.21.15.123:3300/0 conn(0x7f159c1592a0 msgr2=0x7f159c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 --2- 172.21.15.64:0/3738594922 >> v2:172.21.15.123:3300/0 conn(0x7f159c1592a0 0x7f159c15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 -- 172.21.15.64:0/3738594922 >> v1:172.21.15.123:6789/0 conn(0x7f159c15d190 legacy=0x7f159c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a1216700 1 -- 172.21.15.64:0/3738594922 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60485647 0 0) 0x7f159c0ca280 con 0x7f159c15d190 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 -- 172.21.15.64:0/3738594922 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c15daa0 con 0x7f159c158ed0 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a1a17700 1 --2- 172.21.15.64:0/3738594922 >> v2:172.21.15.123:3300/0 conn(0x7f159c1592a0 0x7f159c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a2218700 1 --2- 172.21.15.64:0/3738594922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c158ed0 0x7f159c0cc4e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f159c06b6b0 tx=0x7f1590009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bd428cb5c5061 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a1216700 1 -- 172.21.15.64:0/3738594922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590009e70 con 0x7f159c158ed0 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a1216700 1 -- 172.21.15.64:0/3738594922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159000b860 con 0x7f159c158ed0 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a1216700 1 -- 172.21.15.64:0/3738594922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159000bc00 con 0x7f159c158ed0 2023-09-27T23:50:35.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- 172.21.15.64:0/3738594922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c158ed0 msgr2=0x7f159c0cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 --2- 172.21.15.64:0/3738594922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c158ed0 0x7f159c0cc4e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f159c06b6b0 tx=0x7f1590009250 comp rx=0 tx=0).stop 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- 172.21.15.64:0/3738594922 shutdown_connections 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 --2- 172.21.15.64:0/3738594922 >> v2:172.21.15.123:3300/0 conn(0x7f159c1592a0 0x7f159c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 --2- 172.21.15.64:0/3738594922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c158ed0 0x7f159c0cc4e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- 172.21.15.64:0/3738594922 >> 172.21.15.64:0/3738594922 conn(0x7f159c0cd8f0 msgr2=0x7f159c0c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- 172.21.15.64:0/3738594922 shutdown_connections 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 -- 172.21.15.64:0/3738594922 wait complete. 2023-09-27T23:50:35.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.667+0000 7f15a447c700 1 Processor -- start 2023-09-27T23:50:35.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 -- start start 2023-09-27T23:50:35.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c158ed0 0x7f159c169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c1592a0 0x7f159c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 0x7f159c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c156d20 con 0x7f159c15d190 2023-09-27T23:50:35.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c156890 con 0x7f159c158ed0 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c155a00 con 0x7f159c1592a0 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 0x7f159c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 0x7f159c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54454/0 (socket says 172.21.15.64:54454) 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 -- 172.21.15.64:0/1433947016 learned_addr learned my addr 172.21.15.64:0/1433947016 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2218700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c158ed0 0x7f159c169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a1a17700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c1592a0 0x7f159c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 -- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c1592a0 msgr2=0x7f159c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c1592a0 0x7f159c1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 -- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c158ed0 msgr2=0x7f159c169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c158ed0 0x7f159c169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:35.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 -- 172.21.15.64:0/1433947016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009420 con 0x7f159c15d190 2023-09-27T23:50:35.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a2a19700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 0x7f159c15c910 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f159400c8f0 tx=0x7f1594001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594012630 con 0x7f159c15d190 2023-09-27T23:50:35.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15940127b0 con 0x7f159c15d190 2023-09-27T23:50:35.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15900094f0 con 0x7f159c15d190 2023-09-27T23:50:35.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594012b70 con 0x7f159c15d190 2023-09-27T23:50:35.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.671+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c155f60 con 0x7f159c15d190 2023-09-27T23:50:35.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.675+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c156a20 con 0x7f159c15d190 2023-09-27T23:50:35.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.675+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1594003030 con 0x7f159c15d190 2023-09-27T23:50:35.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.675+0000 7f158e7fc700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1580060a70 0x7f1580062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:35.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.675+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f159407d090 con 0x7f159c15d190 2023-09-27T23:50:35.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.675+0000 7f15a2218700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1580060a70 0x7f1580062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:35.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.675+0000 7f15a2218700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1580060a70 0x7f1580062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f159c06b6b0 tx=0x7f1590017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:35.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.683+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f159404d020 con 0x7f159c15d190 2023-09-27T23:50:35.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:35.875+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f159c071120 con 0x7f159c15d190 2023-09-27T23:50:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.543+0000 7f158e7fc700 1 -- 172.21.15.64:0/1433947016 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1594050640 con 0x7f159c15d190 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1580060a70 msgr2=0x7f1580062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1580060a70 0x7f1580062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f159c06b6b0 tx=0x7f1590017000 comp rx=0 tx=0).stop 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 msgr2=0x7f159c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 0x7f159c15c910 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f159400c8f0 tx=0x7f1594001670 comp rx=0 tx=0).stop 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 shutdown_connections 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1580060a70 0x7f1580062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c15d190 0x7f159c15c910 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f159c1592a0 0x7f159c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 --2- 172.21.15.64:0/1433947016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f159c158ed0 0x7f159c169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 >> 172.21.15.64:0/1433947016 conn(0x7f159c0cd8f0 msgr2=0x7f159c0ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 shutdown_connections 2023-09-27T23:50:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.547+0000 7f15a447c700 1 -- 172.21.15.64:0/1433947016 wait complete. 2023-09-27T23:50:36.565 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:50:36.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 Processor -- start 2023-09-27T23:50:36.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 -- start start 2023-09-27T23:50:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16801591f0 0x7f16800ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168006b730 con 0x7f168015ca80 2023-09-27T23:50:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168006ab20 con 0x7f16801591f0 2023-09-27T23:50:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168736e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168006ae00 con 0x7f1680158e20 2023-09-27T23:50:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16801591f0 0x7f16800ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16801591f0 0x7f16800ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54470/0 (socket says 172.21.15.64:54470) 2023-09-27T23:50:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 -- 172.21.15.64:0/2546478531 learned_addr learned my addr 172.21.15.64:0/2546478531 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f168636c700 1 --2- 172.21.15.64:0/2546478531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 -- 172.21.15.64:0/2546478531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 msgr2=0x7f168006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 --2- 172.21.15.64:0/2546478531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 -- 172.21.15.64:0/2546478531 >> v1:172.21.15.123:6789/0 conn(0x7f168015ca80 legacy=0x7f168006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.675+0000 7f1685b6b700 1 -- 172.21.15.64:0/2546478531 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168006b0e0 con 0x7f16801591f0 2023-09-27T23:50:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f1685b6b700 1 --2- 172.21.15.64:0/2546478531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f16800ce100 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f16700099b0 tx=0x7f1670009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2027518a6a17d42b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168536a700 1 -- 172.21.15.64:0/2546478531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670009ea0 con 0x7f16801591f0 2023-09-27T23:50:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168536a700 1 -- 172.21.15.64:0/2546478531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167000b860 con 0x7f16801591f0 2023-09-27T23:50:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168536a700 1 -- 172.21.15.64:0/2546478531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000bc00 con 0x7f16801591f0 2023-09-27T23:50:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 -- 172.21.15.64:0/2546478531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 msgr2=0x7f16800ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 --2- 172.21.15.64:0/2546478531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f16800ce100 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f16700099b0 tx=0x7f1670009250 comp rx=0 tx=0).stop 2023-09-27T23:50:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 -- 172.21.15.64:0/2546478531 shutdown_connections 2023-09-27T23:50:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 --2- 172.21.15.64:0/2546478531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f16800ce100 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 --2- 172.21.15.64:0/2546478531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 -- 172.21.15.64:0/2546478531 >> 172.21.15.64:0/2546478531 conn(0x7f1680067bb0 msgr2=0x7f16800cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 -- 172.21.15.64:0/2546478531 shutdown_connections 2023-09-27T23:50:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 -- 172.21.15.64:0/2546478531 wait complete. 2023-09-27T23:50:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 Processor -- start 2023-09-27T23:50:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.679+0000 7f168736e700 1 -- start start 2023-09-27T23:50:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f1680157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f1680159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168015ca80 0x7f16800c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168015a450 con 0x7f16801591f0 2023-09-27T23:50:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168015a730 con 0x7f168015ca80 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168015aa10 con 0x7f1680158e20 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f1680159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f1680159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54484/0 (socket says 172.21.15.64:54484) 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 -- 172.21.15.64:0/741650606 learned_addr learned my addr 172.21.15.64:0/741650606 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1686b6d700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168015ca80 0x7f16800c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168636c700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f1680157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 -- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 msgr2=0x7f1680157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f1680157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 -- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168015ca80 msgr2=0x7f16800c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168015ca80 0x7f16800c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 -- 172.21.15.64:0/741650606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700094f0 con 0x7f16801591f0 2023-09-27T23:50:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168636c700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f1680157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:36.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f1685b6b700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f1680159990 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f168006e810 tx=0x7f1670017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:36.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000bbe0 con 0x7f16801591f0 2023-09-27T23:50:36.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670011420 con 0x7f16801591f0 2023-09-27T23:50:36.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168015acf0 con 0x7f16801591f0 2023-09-27T23:50:36.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700117c0 con 0x7f16801591f0 2023-09-27T23:50:36.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1680169f50 con 0x7f16801591f0 2023-09-27T23:50:36.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.683+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168004be40 con 0x7f16801591f0 2023-09-27T23:50:36.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.687+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1670011bf0 con 0x7f16801591f0 2023-09-27T23:50:36.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.687+0000 7f16767fc700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1664060a70 0x7f1664062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:36.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.687+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f167007f1d0 con 0x7f16801591f0 2023-09-27T23:50:36.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.687+0000 7f168636c700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1664060a70 0x7f1664062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:36.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.687+0000 7f168636c700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1664060a70 0x7f1664062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f168015b6b0 tx=0x7f167c008040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:36.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.695+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1670052d10 con 0x7f16801591f0 2023-09-27T23:50:36.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f168006b3c0 con 0x7f16801591f0 2023-09-27T23:50:36.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f16767fc700 1 -- 172.21.15.64:0/741650606 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f16700528a0 con 0x7f16801591f0 2023-09-27T23:50:36.897 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:36.898 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:50:34.520359+0000","modified":"2023-09-27T23:50:36.536424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/660311009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":660311009},{"type":"v1","addr":"172.21.15.64:6835","nonce":660311009}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1664060a70 msgr2=0x7f1664062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1664060a70 0x7f1664062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f168015b6b0 tx=0x7f167c008040 comp rx=0 tx=0).stop 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 msgr2=0x7f1680159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f1680159990 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f168006e810 tx=0x7f1670017000 comp rx=0 tx=0).stop 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 shutdown_connections 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1664060a70 0x7f1664062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f168015ca80 0x7f16800c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16801591f0 0x7f1680159990 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 --2- 172.21.15.64:0/741650606 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1680158e20 0x7f1680157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:36.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 >> 172.21.15.64:0/741650606 conn(0x7f1680067bb0 msgr2=0x7f16800b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:36.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.895+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 shutdown_connections 2023-09-27T23:50:36.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:36.899+0000 7f168736e700 1 -- 172.21.15.64:0/741650606 wait complete. 2023-09-27T23:50:36.901 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 66 2023-09-27T23:50:36.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:50:37.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 Processor -- start 2023-09-27T23:50:37.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 -- start start 2023-09-27T23:50:37.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe62406a580 0x7fe62406a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe62415a860 0x7fe62415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62406c7c0 con 0x7fe62415a860 2023-09-27T23:50:37.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62406cd80 con 0x7fe62406a580 2023-09-27T23:50:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe629d3b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62406af10 con 0x7fe6240b1410 2023-09-27T23:50:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe62953a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe62415a860 0x7fe62415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe62953a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe62415a860 0x7fe62415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54488/0 (socket says 172.21.15.64:54488) 2023-09-27T23:50:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe62953a700 1 -- 172.21.15.64:0/3566149674 learned_addr learned my addr 172.21.15.64:0/3566149674 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe628d39700 1 --2- 172.21.15.64:0/3566149674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.015+0000 7fe623fff700 1 --2- 172.21.15.64:0/3566149674 >> v2:172.21.15.123:3300/0 conn(0x7fe62406a580 0x7fe62406a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe62953a700 1 -- 172.21.15.64:0/3566149674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 msgr2=0x7fe6240ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe62953a700 1 --2- 172.21.15.64:0/3566149674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe62953a700 1 -- 172.21.15.64:0/3566149674 >> v2:172.21.15.123:3300/0 conn(0x7fe62406a580 msgr2=0x7fe62406a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe62953a700 1 --2- 172.21.15.64:0/3566149674 >> v2:172.21.15.123:3300/0 conn(0x7fe62406a580 0x7fe62406a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe62953a700 1 -- 172.21.15.64:0/3566149674 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62406b8f0 con 0x7fe62415a860 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe623fff700 1 --2- 172.21.15.64:0/3566149674 >> v2:172.21.15.123:3300/0 conn(0x7fe62406a580 0x7fe62406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe62953a700 1 --2- 172.21.15.64:0/3566149674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe62415aca0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe624064070 tx=0x7fe614009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5909ebcbd0c0530 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe6237fe700 1 -- 172.21.15.64:0/3566149674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614009e70 con 0x7fe62415a860 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe6237fe700 1 -- 172.21.15.64:0/3566149674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61400b860 con 0x7fe62415a860 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe6237fe700 1 -- 172.21.15.64:0/3566149674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400bc00 con 0x7fe62415a860 2023-09-27T23:50:37.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 -- 172.21.15.64:0/3566149674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 msgr2=0x7fe62415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 --2- 172.21.15.64:0/3566149674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe62415aca0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe624064070 tx=0x7fe614009250 comp rx=0 tx=0).stop 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 -- 172.21.15.64:0/3566149674 shutdown_connections 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 --2- 172.21.15.64:0/3566149674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe62415aca0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 --2- 172.21.15.64:0/3566149674 >> v2:172.21.15.123:3300/0 conn(0x7fe62406a580 0x7fe62406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 --2- 172.21.15.64:0/3566149674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 -- 172.21.15.64:0/3566149674 >> 172.21.15.64:0/3566149674 conn(0x7fe624067bb0 msgr2=0x7fe6240cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 -- 172.21.15.64:0/3566149674 shutdown_connections 2023-09-27T23:50:37.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 -- 172.21.15.64:0/3566149674 wait complete. 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 Processor -- start 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.019+0000 7fe629d3b700 1 -- start start 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62406a580 0x7fe6240afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe624154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6241679b0 con 0x7fe62415a860 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624166da0 con 0x7fe62406a580 2023-09-27T23:50:37.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe624167080 con 0x7fe6240b1410 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe624154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe624154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54496/0 (socket says 172.21.15.64:54496) 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 -- 172.21.15.64:0/4047686230 learned_addr learned my addr 172.21.15.64:0/4047686230 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe628d39700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62406a580 0x7fe6240afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe623fff700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 -- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 msgr2=0x7fe6240c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 -- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62406a580 msgr2=0x7fe6240afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62406a580 0x7fe6240afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 -- 172.21.15.64:0/4047686230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140094f0 con 0x7fe62415a860 2023-09-27T23:50:37.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe628d39700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62406a580 0x7fe6240afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:37.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe62953a700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe624154fd0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe6140090b0 tx=0x7fe614017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400bb20 con 0x7fe62415a860 2023-09-27T23:50:37.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe614025420 con 0x7fe62415a860 2023-09-27T23:50:37.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe624167360 con 0x7fe62415a860 2023-09-27T23:50:37.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140257e0 con 0x7fe62415a860 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe624166a40 con 0x7fe62415a860 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62404be40 con 0x7fe62415a860 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.023+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe614023070 con 0x7fe62415a860 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.027+0000 7fe620ff9700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe610060aa0 0x7fe610062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.027+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe61407eb50 con 0x7fe62415a860 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.027+0000 7fe628d39700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe610060aa0 0x7fe610062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.027+0000 7fe628d39700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe610060aa0 0x7fe610062f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6240cb650 tx=0x7fe61c00b040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.031+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe61404eb80 con 0x7fe62415a860 2023-09-27T23:50:37.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.191+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6240ab220 con 0x7fe62415a860 2023-09-27T23:50:37.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.195+0000 7fe620ff9700 1 -- 172.21.15.64:0/4047686230 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+17085 (secure 0 0 0) 0x7fe6140521a0 con 0x7fe62415a860 2023-09-27T23:50:37.197 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:37.198 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":174,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:50:35.524509+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:50:33.091150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-09-27T23:50:33.495902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.195+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe610060aa0 msgr2=0x7fe610062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.195+0000 7fe629d3b700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe610060aa0 0x7fe610062f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6240cb650 tx=0x7fe61c00b040 comp rx=0 tx=0).stop 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.195+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 msgr2=0x7fe624154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe624154fd0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe6140090b0 tx=0x7fe614017000 comp rx=0 tx=0).stop 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 shutdown_connections 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe610060aa0 0x7fe610062f50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe62415a860 0x7fe624154fd0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6240b1410 0x7fe6240c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 --2- 172.21.15.64:0/4047686230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62406a580 0x7fe6240afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 >> 172.21.15.64:0/4047686230 conn(0x7fe624067bb0 msgr2=0x7fe6240cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 shutdown_connections 2023-09-27T23:50:37.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.199+0000 7fe629d3b700 1 -- 172.21.15.64:0/4047686230 wait complete. 2023-09-27T23:50:37.212 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:50:37.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:50:37.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47f59e700 1 Processor -- start 2023-09-27T23:50:37.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47f59e700 1 -- start start 2023-09-27T23:50:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc48006a540 0x7fc48006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800ca5a0 con 0x7fc4800cc470 2023-09-27T23:50:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48006dac0 con 0x7fc480158e70 2023-09-27T23:50:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48006d220 con 0x7fc48006a540 2023-09-27T23:50:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc4800cc470 0x7fc4800cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40952/0 (socket says 172.21.15.64:40952) 2023-09-27T23:50:37.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47ed9d700 1 -- 172.21.15.64:0/4051841973 learned_addr learned my addr 172.21.15.64:0/4051841973 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:37.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47dd9b700 1 --2- 172.21.15.64:0/4051841973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc48006a540 0x7fc48006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47e59c700 1 --1- 172.21.15.64:0/4051841973 >> v1:172.21.15.123:6789/0 conn(0x7fc480158e70 0x7fc4800afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:47966/0 (socket says 172.21.15.64:47966) 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47d59a700 1 -- 172.21.15.64:0/4051841973 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456921983 0 0) 0x7fc48006dac0 con 0x7fc480158e70 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47d59a700 1 -- 172.21.15.64:0/4051841973 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468003570 con 0x7fc480158e70 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47d59a700 1 -- 172.21.15.64:0/4051841973 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615581620 0 0) 0x7fc4800ca5a0 con 0x7fc4800cc470 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47dd9b700 1 -- 172.21.15.64:0/4051841973 >> v1:172.21.15.123:6789/0 conn(0x7fc480158e70 legacy=0x7fc4800afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47dd9b700 1 -- 172.21.15.64:0/4051841973 >> v1:172.21.15.64:6789/0 conn(0x7fc4800cc470 legacy=0x7fc4800cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47dd9b700 1 -- 172.21.15.64:0/4051841973 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48006d500 con 0x7fc48006a540 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.315+0000 7fc47dd9b700 1 --2- 172.21.15.64:0/4051841973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc48006a540 0x7fc48006c9d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc474009910 tx=0x7fc4740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ebd7b427b485 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47d59a700 1 -- 172.21.15.64:0/4051841973 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400b640 con 0x7fc48006a540 2023-09-27T23:50:37.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47d59a700 1 -- 172.21.15.64:0/4051841973 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47400b7a0 con 0x7fc48006a540 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47d59a700 1 -- 172.21.15.64:0/4051841973 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400bb20 con 0x7fc48006a540 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 -- 172.21.15.64:0/4051841973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc48006a540 msgr2=0x7fc48006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 --2- 172.21.15.64:0/4051841973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc48006a540 0x7fc48006c9d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc474009910 tx=0x7fc4740091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 -- 172.21.15.64:0/4051841973 shutdown_connections 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 --2- 172.21.15.64:0/4051841973 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc48006a540 0x7fc48006c9d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 -- 172.21.15.64:0/4051841973 >> 172.21.15.64:0/4051841973 conn(0x7fc4800670e0 msgr2=0x7fc4800674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 -- 172.21.15.64:0/4051841973 shutdown_connections 2023-09-27T23:50:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 -- 172.21.15.64:0/4051841973 wait complete. 2023-09-27T23:50:37.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 Processor -- start 2023-09-27T23:50:37.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.319+0000 7fc47f59e700 1 -- start start 2023-09-27T23:50:37.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 0x7fc4801574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4800cc470 0x7fc4800c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480158e70 0x7fc4800b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800c8c00 con 0x7fc48006a540 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800c8ee0 con 0x7fc480158e70 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800c91c0 con 0x7fc4800cc470 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 0x7fc4801574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 0x7fc4801574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54504/0 (socket says 172.21.15.64:54504) 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 -- 172.21.15.64:0/2131756123 learned_addr learned my addr 172.21.15.64:0/2131756123 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47dd9b700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4800cc470 0x7fc4800c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47ed9d700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480158e70 0x7fc4800b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 -- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4800cc470 msgr2=0x7fc4800c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4800cc470 0x7fc4800c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 -- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480158e70 msgr2=0x7fc4800b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480158e70 0x7fc4800b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 -- 172.21.15.64:0/2131756123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474009420 con 0x7fc48006a540 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47e59c700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 0x7fc4801574e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc47000cd80 tx=0x7fc470001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470005df0 con 0x7fc48006a540 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000a960 con 0x7fc48006a540 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4800c94a0 con 0x7fc48006a540 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48015a360 con 0x7fc48006a540 2023-09-27T23:50:37.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000ace0 con 0x7fc48006a540 2023-09-27T23:50:37.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.323+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48004be40 con 0x7fc48006a540 2023-09-27T23:50:37.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.327+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc470007300 con 0x7fc48006a540 2023-09-27T23:50:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.327+0000 7fc46e7fc700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc460060a70 0x7fc460062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.327+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc47007d1b0 con 0x7fc48006a540 2023-09-27T23:50:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.327+0000 7fc47dd9b700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc460060a70 0x7fc460062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.327+0000 7fc47dd9b700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc460060a70 0x7fc460062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc474009860 tx=0x7fc474016000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.335+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc47004d1e0 con 0x7fc48006a540 2023-09-27T23:50:37.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.495+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4800c9630 con 0x7fc460060a70 2023-09-27T23:50:37.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.515+0000 7fc46e7fc700 1 -- 172.21.15.64:0/2131756123 <== mgr.8210 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297089 (secure 0 0 0) 0x7fc4800c9630 con 0x7fc460060a70 2023-09-27T23:50:37.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:37.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc460060a70 msgr2=0x7fc460062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc460060a70 0x7fc460062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc474009860 tx=0x7fc474016000 comp rx=0 tx=0).stop 2023-09-27T23:50:37.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 msgr2=0x7fc4801574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 0x7fc4801574e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc47000cd80 tx=0x7fc470001aa0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 shutdown_connections 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc460060a70 0x7fc460062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480158e70 0x7fc4800b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4800cc470 0x7fc4800c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 --2- 172.21.15.64:0/2131756123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc48006a540 0x7fc4801574e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 >> 172.21.15.64:0/2131756123 conn(0x7fc4800670e0 msgr2=0x7fc4800cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 shutdown_connections 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.519+0000 7fc47f59e700 1 -- 172.21.15.64:0/2131756123 wait complete. 2023-09-27T23:50:37.524 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:50:37.535 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-09-27T23:50:36.536365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264768,"kb_used_data":17728,"kb_used_omap":0,"kb_used_meta":246976,"kb_avail":754709952,"statfs":{"total":773094113280,"available":772822990848,"internally_reserved":0,"allocated":18153472,"data_stored":10089756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":252903424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002339"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.159677+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:37.863006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157624+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151466+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:39.785598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.155020+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:39.171495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149401+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:25:45.048987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.152131+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:40.565795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157802+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:10.178409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158342+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149396+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:58.865116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.150826+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:19:43.857264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158475+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158606+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:13.108314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.154259+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:57.422152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.159790+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:50.280393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.152989+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:47:00.473077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158488+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146975+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149588+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:08:16.153188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.156804+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146864+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:59.225226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157540+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:56:50.893048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151058+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:12:04.021162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.156787+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:00.466792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158038+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:02.947172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157610+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:28:23.297242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"162'304","reported_seq":957,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164795+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:50:32.164795+0000","last_peered":"2023-09-27T23:50:32.164795+0000","last_clean":"2023-09-27T23:50:32.164795+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:50:32.164795+0000","last_undegraded":"2023-09-27T23:50:32.164795+0000","last_fullsized":"2023-09-27T23:50:32.164795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":304,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.132236+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:50:31.132236+0000","last_peered":"2023-09-27T23:50:31.132236+0000","last_clean":"2023-09-27T23:50:31.132236+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:50:31.132236+0000","last_undegraded":"2023-09-27T23:50:31.132236+0000","last_fullsized":"2023-09-27T23:50:31.132236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158167+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:13.856739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164771+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:50:32.164771+0000","last_peered":"2023-09-27T23:50:32.164771+0000","last_clean":"2023-09-27T23:50:32.164771+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:50:32.164771+0000","last_undegraded":"2023-09-27T23:50:32.164771+0000","last_fullsized":"2023-09-27T23:50:32.164771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.150966+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:56:14.072256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.162428+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:50:32.162428+0000","last_peered":"2023-09-27T23:50:32.162428+0000","last_clean":"2023-09-27T23:50:32.162428+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:50:32.162428+0000","last_undegraded":"2023-09-27T23:50:32.162428+0000","last_fullsized":"2023-09-27T23:50:32.162428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151362+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.132223+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:50:31.132223+0000","last_peered":"2023-09-27T23:50:31.132223+0000","last_clean":"2023-09-27T23:50:31.132223+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:50:31.132223+0000","last_undegraded":"2023-09-27T23:50:31.132223+0000","last_fullsized":"2023-09-27T23:50:31.132223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.160602+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:34.510803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.130948+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:50:31.130948+0000","last_peered":"2023-09-27T23:50:31.130948+0000","last_clean":"2023-09-27T23:50:31.130948+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:50:31.130948+0000","last_undegraded":"2023-09-27T23:50:31.130948+0000","last_fullsized":"2023-09-27T23:50:31.130948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.153811+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:06:48.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164785+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:50:32.164785+0000","last_peered":"2023-09-27T23:50:32.164785+0000","last_clean":"2023-09-27T23:50:32.164785+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:50:32.164785+0000","last_undegraded":"2023-09-27T23:50:32.164785+0000","last_fullsized":"2023-09-27T23:50:32.164785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146863+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.131143+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:50:31.131143+0000","last_peered":"2023-09-27T23:50:31.131143+0000","last_clean":"2023-09-27T23:50:31.131143+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:50:31.131143+0000","last_undegraded":"2023-09-27T23:50:31.131143+0000","last_fullsized":"2023-09-27T23:50:31.131143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.154600+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:47:05.296212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164067+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:50:32.164067+0000","last_peered":"2023-09-27T23:50:32.164067+0000","last_clean":"2023-09-27T23:50:32.164067+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:50:32.164067+0000","last_undegraded":"2023-09-27T23:50:32.164067+0000","last_fullsized":"2023-09-27T23:50:32.164067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35500,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":32832,"kb_avail":94336340,"statfs":{"total":96636764160,"available":96600412160,"internally_reserved":0,"allocated":2723840,"data_stored":1708854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33619968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89200000000000002,"15min":0.89500000000000002},"min":{"1min":0.73399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1899999999999999,"5min":1.2809999999999999,"15min":1.3440000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.73999999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2529999999999999,"5min":1.3520000000000001,"15min":1.403},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.90500000000000003},"min":{"1min":0.65100000000000002,"5min":0.61099999999999999,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":0.96299999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.63600000000000001},"max":{"1min":1.1919999999999999,"5min":1.2629999999999999,"15min":1.4530000000000001},"last":1.208}]},{"osd":2,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.69999999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":0.96699999999999997},"min":{"1min":0.68100000000000005,"5min":0.63600000000000001,"15min":0.628},"max":{"1min":1.341,"5min":1.341,"15min":1.3460000000000001},"last":1.032}]},{"osd":3,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95199999999999996,"15min":0.94199999999999995},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.66900000000000004},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0900000000000001,"15min":1.0800000000000001},"min":{"1min":0.87,"5min":0.755,"15min":0.71899999999999997},"max":{"1min":1.2470000000000001,"5min":1.3240000000000001,"15min":1.3879999999999999},"last":1.115}]},{"osd":4,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.109,"5min":1.3819999999999999,"15min":1.411},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.02,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67900000000000005,"15min":0.622},"max":{"1min":1.343,"5min":1.488,"15min":1.488},"last":1.008}]},{"osd":5,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.65600000000000003},"max":{"1min":1.2949999999999999,"5min":1.484,"15min":1.484},"last":1.181},{"interface":"front","average":{"1min":1.133,"5min":1.117,"15min":1.1180000000000001},"min":{"1min":0.79400000000000004,"5min":0.75,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.3169999999999999,"5min":1.425,"15min":1.538},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.155,"15min":1.149},"min":{"1min":0.93500000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.409,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.1439999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2129920,"data_stored":1118486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.99399999999999999,"15min":0.97099999999999997},"min":{"1min":0.65900000000000003,"5min":0.60599999999999998,"15min":0.51800000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0529999999999999,"15min":1.0600000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.69399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.66800000000000004,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.57099999999999995},"max":{"1min":1.226,"5min":1.53,"15min":1.53},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61399999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.63200000000000001},"max":{"1min":1.2709999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.0840000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.74099999999999999,"5min":0.65700000000000003,"15min":0.63100000000000001},"max":{"1min":1.53,"5min":1.583,"15min":1.583},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.169,"5min":1.325,"15min":1.325},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.878,"15min":0.873},"min":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.41599999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0129999999999999,"15min":1},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3700000000000001,"5min":1.633,"15min":1.633},"last":0.78000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.95099999999999996},"min":{"1min":0.71099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.4430000000000001},"last":1.016},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.149,"15min":1.141},"min":{"1min":0.81299999999999994,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.143}]},{"osd":7,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.1180000000000001},"min":{"1min":0.755,"5min":0.627,"15min":0.627},"max":{"1min":1.623,"5min":1.679,"15min":1.7210000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.423,"5min":1.488,"15min":1.542},"last":1.1160000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39068,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36992,"kb_avail":94332772,"statfs":{"total":96636764160,"available":96596758528,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37879808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86699999999999999,"15min":0.86199999999999999},"min":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.51900000000000002},"max":{"1min":1.248,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.47899999999999998},"max":{"1min":1.2250000000000001,"5min":1.292,"15min":1.5640000000000001},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.302,"5min":1.425,"15min":1.538},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.194,"5min":1.3819999999999999,"15min":1.397},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.629},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.4690000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.626},"max":{"1min":1.153,"5min":1.3049999999999999,"15min":1.4470000000000001},"last":0.80200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.3240000000000001,"5min":1.373,"15min":1.611},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0349999999999999,"15min":1.0249999999999999},"min":{"1min":0.84599999999999997,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.228,"5min":1.419,"15min":1.4370000000000001},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.88},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.3129999999999999,"5min":1.484,"15min":1.484},"last":0.998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.58699999999999997},"max":{"1min":1.45,"5min":1.45,"15min":1.4510000000000001},"last":1.1000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.026},"min":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.242,"5min":1.496,"15min":1.5069999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.099,"15min":1.0980000000000001},"min":{"1min":0.84899999999999998,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":1.262,"5min":1.546,"15min":1.5840000000000001},"last":1.123}]},{"osd":7,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1619999999999999,"15min":1.1459999999999999},"min":{"1min":0.94099999999999995,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7709999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":0.98099999999999998,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.2390000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25372,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":23296,"kb_avail":94346468,"statfs":{"total":96636764160,"available":96610783232,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23855104},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.92500000000000004,"15min":0.90700000000000003},"min":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.45000000000000001},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88800000000000001,"15min":0.89500000000000002},"min":{"1min":0.63200000000000001,"5min":0.624,"15min":0.41599999999999998},"max":{"1min":1.893,"5min":1.893,"15min":2.2770000000000001},"last":1.024}]},{"osd":1,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.90200000000000002},"min":{"1min":0.75900000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.53500000000000003},"max":{"1min":1.758,"5min":1.758,"15min":2.1110000000000002},"last":0.89800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.98299999999999998,"15min":0.96799999999999997},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.966,"5min":1.966,"15min":2.056},"last":1.1990000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.998,"5min":1.998,"15min":2.032},"last":1.143},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.0880000000000001,"15min":1.054},"min":{"1min":0.82899999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.82899999999999996}]},{"osd":5,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93700000000000006,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.61399999999999999,"15min":0.45500000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.9930000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0600000000000001,"15min":1.0509999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.694,"5min":1.873,"15min":1.873},"last":1.117},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.1759999999999999,"15min":1.1679999999999999},"min":{"1min":0.96199999999999997,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1800000000000002},"last":1.262}]},{"osd":7,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.105,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1779999999999999,"15min":1.177},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1799999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30044,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341796,"statfs":{"total":96636764160,"available":96605999104,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83999999999999997,"5min":1.1639999999999999,"15min":1.3560000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.746},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81599999999999995,"5min":1.2629999999999999,"15min":1.377},"last":0.73499999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1799999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.69299999999999995,"5min":0.63,"15min":0.58999999999999997},"max":{"1min":1.2509999999999999,"5min":1.272,"15min":1.2849999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.876,"15min":0.873},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"max":{"1min":1.1599999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94099999999999995},"min":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.63800000000000001},"max":{"1min":1.175,"5min":1.3420000000000001,"15min":1.383},"last":1.1699999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.998,"15min":0.98399999999999999},"min":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.621},"max":{"1min":1.0600000000000001,"5min":1.599,"15min":1.599},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.86399999999999999,"5min":0.72099999999999997,"15min":0.68700000000000006},"max":{"1min":1.3069999999999999,"5min":1.587,"15min":1.587},"last":0.873}]},{"osd":5,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.123,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.484,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.181},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.72699999999999998},"max":{"1min":1.6499999999999999,"5min":1.659,"15min":1.659},"last":1.4530000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.92900000000000005,"5min":0.878,"15min":0.77800000000000002},"max":{"1min":1.288,"5min":1.5249999999999999,"15min":1.659},"last":1.079},{"interface":"front","average":{"1min":1.222,"5min":1.258,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.90500000000000003,"15min":0.82599999999999996},"max":{"1min":1.413,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.135}]},{"osd":7,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2629999999999999,"15min":1.25},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5629999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.94999999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.409,"5min":1.657,"15min":1.657},"last":0.97599999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29980,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":27904,"kb_avail":94341860,"statfs":{"total":96636764160,"available":96606064640,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28573696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.873,"5min":1.252,"15min":1.5329999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.55300000000000005,"15min":0.53700000000000003},"max":{"1min":1.2250000000000001,"5min":1.345,"15min":1.7969999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1899999999999999,"5min":1.242,"15min":2.0760000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.50800000000000001,"15min":0.495},"max":{"1min":1.103,"5min":1.1890000000000001,"15min":1.397},"last":1.383}]},{"osd":3,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.54900000000000004},"max":{"1min":0.90400000000000003,"5min":1.3149999999999999,"15min":1.667},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93600000000000005,"15min":0.95299999999999996},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2090000000000001,"5min":1.4359999999999999,"15min":1.7},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0900000000000001,"15min":1.085},"min":{"1min":0.86799999999999999,"5min":0.78600000000000003,"15min":0.73799999999999999},"max":{"1min":1.407,"5min":1.5349999999999999,"15min":1.972},"last":1.296},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0600000000000001},"min":{"1min":0.89400000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3819999999999999,"5min":1.655,"15min":1.8009999999999999},"last":1.179}]},{"osd":5,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.83999999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5209999999999999,"5min":1.5589999999999999,"15min":1.772},"last":1.456},{"interface":"front","average":{"1min":1.143,"5min":1.171,"15min":1.1819999999999999},"min":{"1min":0.84699999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5860000000000001,"5min":1.653,"15min":1.859},"last":0.99299999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1659999999999999,"15min":1.1599999999999999},"min":{"1min":0.92600000000000005,"5min":0.82299999999999995,"15min":0.80600000000000005},"max":{"1min":1.5529999999999999,"5min":1.6000000000000001,"15min":1.8999999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":1.0049999999999999,"5min":0.79800000000000004,"15min":0.76200000000000001},"max":{"1min":1.55,"5min":1.6439999999999999,"15min":1.736},"last":1.1180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.234,"15min":1.23},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.86599999999999999},"max":{"1min":1.548,"5min":1.615,"15min":2.0059999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.238},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.591,"5min":1.8129999999999999,"15min":1.8340000000000001},"last":1.335}]}]},{"osd":1,"up_from":14,"seq":60129542587,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44124,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":42048,"kb_avail":94327716,"statfs":{"total":96636764160,"available":96591581184,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43057152},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1819999999999999,"15min":1.2070000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.73999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.095,"5min":1.177,"15min":1.361},"last":0.73199999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.88,"5min":1.2509999999999999,"15min":1.383},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2350000000000001,"5min":1.3169999999999999,"15min":1.427},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.64600000000000002,"5min":0.626,"15min":0.58899999999999997},"max":{"1min":1.1890000000000001,"5min":1.3080000000000001,"15min":1.3720000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.83599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.454},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.98699999999999999,"15min":0.98999999999999999},"min":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.325,"15min":1.5229999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.196,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.71999999999999997},"max":{"1min":1.4299999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.091,"15min":1.0980000000000001},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.621},"max":{"1min":1.399,"5min":1.5449999999999999,"15min":1.583},"last":0.877},{"interface":"front","average":{"1min":1.077,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.747},"max":{"1min":1.3,"5min":1.4710000000000001,"15min":1.575},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2450000000000001,"15min":1.234},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.76800000000000002},"max":{"1min":1.5529999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.72299999999999998},"max":{"1min":1.4970000000000001,"5min":1.7789999999999999,"15min":1.804},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.94499999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.514,"5min":1.7130000000000001,"15min":1.853},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.95699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4139999999999999,"5min":1.585,"15min":1.827},"last":1.204}]}]},{"osd":0,"up_from":15,"seq":64424509882,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31136,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340704,"statfs":{"total":96636764160,"available":96604880896,"internally_reserved":0,"allocated":2711552,"data_stored":1702376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.85499999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.47499999999999998},"max":{"1min":1.131,"5min":1.1779999999999999,"15min":1.306},"last":0.76900000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.502},"max":{"1min":0.98299999999999998,"5min":1.236,"15min":1.5960000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.101,"5min":1.288,"15min":1.863},"last":0.94499999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.875},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.78600000000000003,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.92500000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3,"5min":1.4099999999999999,"15min":1.7110000000000001},"last":1.573},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.70699999999999996},"max":{"1min":1.516,"5min":1.5249999999999999,"15min":1.651},"last":1.4810000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.111,"15min":1.111},"min":{"1min":0.90700000000000003,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.407,"5min":1.4259999999999999,"15min":1.512},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.165,"15min":1.1779999999999999},"min":{"1min":0.86399999999999999,"5min":0.83399999999999996,"15min":0.79800000000000004},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.847},"last":1.444}]},{"osd":6,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1870000000000001,"15min":1.1830000000000001},"min":{"1min":0.90700000000000003,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.895},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2330000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.84999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5720000000000001,"15min":2.0510000000000002},"last":1.1930000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.286,"15min":1.2789999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.84799999999999998},"max":{"1min":1.554,"5min":1.6100000000000001,"15min":2.0030000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.85599999999999998},"max":{"1min":1.4790000000000001,"5min":1.806,"15min":2.004},"last":1.5229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:50:37.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:50:37.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 Processor -- start 2023-09-27T23:50:37.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 -- start start 2023-09-27T23:50:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d30155960 0x7f0d3015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0d3006de20 0x7f0d300c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006c7c0 0x7f0d300c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3006d990 con 0x7f0d30155960 2023-09-27T23:50:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3015a570 con 0x7f0d3006de20 2023-09-27T23:50:37.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d34efe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3006cd80 con 0x7f0d3006c7c0 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d30155960 0x7f0d3015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d30155960 0x7f0d3015a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54508/0 (socket says 172.21.15.64:54508) 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/2927975028 learned_addr learned my addr 172.21.15.64:0/2927975028 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2effd700 1 --2- 172.21.15.64:0/2927975028 >> v2:172.21.15.123:3300/0 conn(0x7f0d3006de20 0x7f0d300c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2ffff700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006c7c0 0x7f0d300c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/2927975028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006c7c0 msgr2=0x7f0d300c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006c7c0 0x7f0d300c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/2927975028 >> v2:172.21.15.123:3300/0 conn(0x7f0d3006de20 msgr2=0x7f0d300c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:37.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 --2- 172.21.15.64:0/2927975028 >> v2:172.21.15.123:3300/0 conn(0x7f0d3006de20 0x7f0d300c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/2927975028 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3006d060 con 0x7f0d30155960 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2ffff700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006c7c0 0x7f0d300c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.651+0000 7f0d2effd700 1 --2- 172.21.15.64:0/2927975028 >> v2:172.21.15.123:3300/0 conn(0x7f0d3006de20 0x7f0d300c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d2f7fe700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d30155960 0x7f0d3015a030 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0d3006bf60 tx=0x7f0d20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fedd1642ce6c005 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/2927975028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000b860 con 0x7f0d30155960 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/2927975028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2000b9c0 con 0x7f0d30155960 2023-09-27T23:50:37.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d2e7fc700 1 -- 172.21.15.64:0/2927975028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000bd60 con 0x7f0d30155960 2023-09-27T23:50:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 -- 172.21.15.64:0/2927975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d30155960 msgr2=0x7f0d3015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d30155960 0x7f0d3015a030 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0d3006bf60 tx=0x7f0d20009250 comp rx=0 tx=0).stop 2023-09-27T23:50:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 -- 172.21.15.64:0/2927975028 shutdown_connections 2023-09-27T23:50:37.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006c7c0 0x7f0d300c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 --2- 172.21.15.64:0/2927975028 >> v2:172.21.15.123:3300/0 conn(0x7f0d3006de20 0x7f0d300c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 --2- 172.21.15.64:0/2927975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d30155960 0x7f0d3015a030 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 -- 172.21.15.64:0/2927975028 >> 172.21.15.64:0/2927975028 conn(0x7f0d30067bb0 msgr2=0x7f0d300cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 -- 172.21.15.64:0/2927975028 shutdown_connections 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 -- 172.21.15.64:0/2927975028 wait complete. 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 Processor -- start 2023-09-27T23:50:37.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.655+0000 7f0d34efe700 1 -- start start 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 0x7f0d300b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006de20 0x7f0d30154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155960 0x7f0d300b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300b0c00 con 0x7f0d3006c7c0 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301551a0 con 0x7f0d30155960 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30156140 con 0x7f0d3006de20 2023-09-27T23:50:37.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 0x7f0d300b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 0x7f0d300b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54522/0 (socket says 172.21.15.64:54522) 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/1658265457 learned_addr learned my addr 172.21.15.64:0/1658265457 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2ffff700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155960 0x7f0d300b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2effd700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006de20 0x7f0d30154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006de20 msgr2=0x7f0d30154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006de20 0x7f0d30154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155960 msgr2=0x7f0d300b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155960 0x7f0d300b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 -- 172.21.15.64:0/1658265457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200094f0 con 0x7f0d3006c7c0 2023-09-27T23:50:37.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d2f7fe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 0x7f0d300b0040 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009290 tx=0x7f0d20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20026480 con 0x7f0d3006c7c0 2023-09-27T23:50:37.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2001f070 con 0x7f0d3006c7c0 2023-09-27T23:50:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d30154be0 con 0x7f0d3006c7c0 2023-09-27T23:50:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20026830 con 0x7f0d3006c7c0 2023-09-27T23:50:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.659+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d30155dd0 con 0x7f0d3006c7c0 2023-09-27T23:50:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.663+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3004be40 con 0x7f0d3006c7c0 2023-09-27T23:50:37.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.663+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0d2001a430 con 0x7f0d3006c7c0 2023-09-27T23:50:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.663+0000 7f0d177fe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0d1c060a70 0x7f0d1c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.663+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0d20021070 con 0x7f0d3006c7c0 2023-09-27T23:50:37.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.663+0000 7f0d2effd700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0d1c060a70 0x7f0d1c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.663+0000 7f0d2effd700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0d1c060a70 0x7f0d1c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0d3016a6a0 tx=0x7f0d24008040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.671+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0d20082030 con 0x7f0d3006c7c0 2023-09-27T23:50:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.831+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d3006d340 con 0x7f0d1c060a70 2023-09-27T23:50:37.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.847+0000 7f0d177fe700 1 -- 172.21.15.64:0/1658265457 <== mgr.8210 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297089 (secure 0 0 0) 0x7f0d3006d340 con 0x7f0d1c060a70 2023-09-27T23:50:37.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0d1c060a70 msgr2=0x7f0d1c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0d1c060a70 0x7f0d1c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0d3016a6a0 tx=0x7f0d24008040 comp rx=0 tx=0).stop 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 msgr2=0x7f0d300b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 0x7f0d300b0040 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009290 tx=0x7f0d20017000 comp rx=0 tx=0).stop 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 shutdown_connections 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0d1c060a70 0x7f0d1c062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d30155960 0x7f0d300b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d3006de20 0x7f0d30154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 --2- 172.21.15.64:0/1658265457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3006c7c0 0x7f0d300b0040 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 >> 172.21.15.64:0/1658265457 conn(0x7f0d30067bb0 msgr2=0x7f0d300c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 shutdown_connections 2023-09-27T23:50:37.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.855+0000 7f0d34efe700 1 -- 172.21.15.64:0/1658265457 wait complete. 2023-09-27T23:50:37.858 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:50:37.870 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-09-27T23:50:36.536365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264768,"kb_used_data":17728,"kb_used_omap":0,"kb_used_meta":246976,"kb_avail":754709952,"statfs":{"total":773094113280,"available":772822990848,"internally_reserved":0,"allocated":18153472,"data_stored":10089756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":252903424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002339"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.159677+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:37.863006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157624+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151466+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:39.785598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.155020+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:39.171495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149401+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:25:45.048987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.152131+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:40.565795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157802+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:10.178409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158342+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149396+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:58.865116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.150826+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:19:43.857264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158475+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158606+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:13.108314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.154259+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:57.422152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.159790+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:50.280393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.152989+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:47:00.473077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158488+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146975+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149588+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:08:16.153188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.156804+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146864+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:59.225226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157540+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:56:50.893048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151058+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:12:04.021162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.156787+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:00.466792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158038+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:02.947172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157610+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:28:23.297242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"162'304","reported_seq":957,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164795+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:50:32.164795+0000","last_peered":"2023-09-27T23:50:32.164795+0000","last_clean":"2023-09-27T23:50:32.164795+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:50:32.164795+0000","last_undegraded":"2023-09-27T23:50:32.164795+0000","last_fullsized":"2023-09-27T23:50:32.164795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":304,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.132236+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:50:31.132236+0000","last_peered":"2023-09-27T23:50:31.132236+0000","last_clean":"2023-09-27T23:50:31.132236+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:50:31.132236+0000","last_undegraded":"2023-09-27T23:50:31.132236+0000","last_fullsized":"2023-09-27T23:50:31.132236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158167+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:13.856739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164771+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:50:32.164771+0000","last_peered":"2023-09-27T23:50:32.164771+0000","last_clean":"2023-09-27T23:50:32.164771+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:50:32.164771+0000","last_undegraded":"2023-09-27T23:50:32.164771+0000","last_fullsized":"2023-09-27T23:50:32.164771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.150966+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:56:14.072256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.162428+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:50:32.162428+0000","last_peered":"2023-09-27T23:50:32.162428+0000","last_clean":"2023-09-27T23:50:32.162428+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:50:32.162428+0000","last_undegraded":"2023-09-27T23:50:32.162428+0000","last_fullsized":"2023-09-27T23:50:32.162428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151362+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.132223+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:50:31.132223+0000","last_peered":"2023-09-27T23:50:31.132223+0000","last_clean":"2023-09-27T23:50:31.132223+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:50:31.132223+0000","last_undegraded":"2023-09-27T23:50:31.132223+0000","last_fullsized":"2023-09-27T23:50:31.132223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.160602+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:34.510803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.130948+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:50:31.130948+0000","last_peered":"2023-09-27T23:50:31.130948+0000","last_clean":"2023-09-27T23:50:31.130948+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:50:31.130948+0000","last_undegraded":"2023-09-27T23:50:31.130948+0000","last_fullsized":"2023-09-27T23:50:31.130948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.153811+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:06:48.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164785+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:50:32.164785+0000","last_peered":"2023-09-27T23:50:32.164785+0000","last_clean":"2023-09-27T23:50:32.164785+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:50:32.164785+0000","last_undegraded":"2023-09-27T23:50:32.164785+0000","last_fullsized":"2023-09-27T23:50:32.164785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146863+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.131143+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:50:31.131143+0000","last_peered":"2023-09-27T23:50:31.131143+0000","last_clean":"2023-09-27T23:50:31.131143+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:50:31.131143+0000","last_undegraded":"2023-09-27T23:50:31.131143+0000","last_fullsized":"2023-09-27T23:50:31.131143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.154600+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:47:05.296212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164067+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:50:32.164067+0000","last_peered":"2023-09-27T23:50:32.164067+0000","last_clean":"2023-09-27T23:50:32.164067+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:50:32.164067+0000","last_undegraded":"2023-09-27T23:50:32.164067+0000","last_fullsized":"2023-09-27T23:50:32.164067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35500,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":32832,"kb_avail":94336340,"statfs":{"total":96636764160,"available":96600412160,"internally_reserved":0,"allocated":2723840,"data_stored":1708854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33619968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89200000000000002,"15min":0.89500000000000002},"min":{"1min":0.73399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1899999999999999,"5min":1.2809999999999999,"15min":1.3440000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.73999999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2529999999999999,"5min":1.3520000000000001,"15min":1.403},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.90500000000000003},"min":{"1min":0.65100000000000002,"5min":0.61099999999999999,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":0.96299999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.63600000000000001},"max":{"1min":1.1919999999999999,"5min":1.2629999999999999,"15min":1.4530000000000001},"last":1.208}]},{"osd":2,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.69999999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":0.96699999999999997},"min":{"1min":0.68100000000000005,"5min":0.63600000000000001,"15min":0.628},"max":{"1min":1.341,"5min":1.341,"15min":1.3460000000000001},"last":1.032}]},{"osd":3,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95199999999999996,"15min":0.94199999999999995},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.66900000000000004},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0900000000000001,"15min":1.0800000000000001},"min":{"1min":0.87,"5min":0.755,"15min":0.71899999999999997},"max":{"1min":1.2470000000000001,"5min":1.3240000000000001,"15min":1.3879999999999999},"last":1.115}]},{"osd":4,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.109,"5min":1.3819999999999999,"15min":1.411},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.02,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67900000000000005,"15min":0.622},"max":{"1min":1.343,"5min":1.488,"15min":1.488},"last":1.008}]},{"osd":5,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.65600000000000003},"max":{"1min":1.2949999999999999,"5min":1.484,"15min":1.484},"last":1.181},{"interface":"front","average":{"1min":1.133,"5min":1.117,"15min":1.1180000000000001},"min":{"1min":0.79400000000000004,"5min":0.75,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.3169999999999999,"5min":1.425,"15min":1.538},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.155,"15min":1.149},"min":{"1min":0.93500000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.409,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.1439999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2129920,"data_stored":1118486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.99399999999999999,"15min":0.97099999999999997},"min":{"1min":0.65900000000000003,"5min":0.60599999999999998,"15min":0.51800000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0529999999999999,"15min":1.0600000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.69399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.66800000000000004,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.57099999999999995},"max":{"1min":1.226,"5min":1.53,"15min":1.53},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61399999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.63200000000000001},"max":{"1min":1.2709999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.0840000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.74099999999999999,"5min":0.65700000000000003,"15min":0.63100000000000001},"max":{"1min":1.53,"5min":1.583,"15min":1.583},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.169,"5min":1.325,"15min":1.325},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.878,"15min":0.873},"min":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.41599999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0129999999999999,"15min":1},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3700000000000001,"5min":1.633,"15min":1.633},"last":0.78000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.95099999999999996},"min":{"1min":0.71099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.4430000000000001},"last":1.016},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.149,"15min":1.141},"min":{"1min":0.81299999999999994,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.143}]},{"osd":7,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.1180000000000001},"min":{"1min":0.755,"5min":0.627,"15min":0.627},"max":{"1min":1.623,"5min":1.679,"15min":1.7210000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.423,"5min":1.488,"15min":1.542},"last":1.1160000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39068,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36992,"kb_avail":94332772,"statfs":{"total":96636764160,"available":96596758528,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37879808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86699999999999999,"15min":0.86199999999999999},"min":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.51900000000000002},"max":{"1min":1.248,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.47899999999999998},"max":{"1min":1.2250000000000001,"5min":1.292,"15min":1.5640000000000001},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.302,"5min":1.425,"15min":1.538},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.194,"5min":1.3819999999999999,"15min":1.397},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.629},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.4690000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.626},"max":{"1min":1.153,"5min":1.3049999999999999,"15min":1.4470000000000001},"last":0.80200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.3240000000000001,"5min":1.373,"15min":1.611},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0349999999999999,"15min":1.0249999999999999},"min":{"1min":0.84599999999999997,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.228,"5min":1.419,"15min":1.4370000000000001},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.88},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.3129999999999999,"5min":1.484,"15min":1.484},"last":0.998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.58699999999999997},"max":{"1min":1.45,"5min":1.45,"15min":1.4510000000000001},"last":1.1000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.026},"min":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.242,"5min":1.496,"15min":1.5069999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.099,"15min":1.0980000000000001},"min":{"1min":0.84899999999999998,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":1.262,"5min":1.546,"15min":1.5840000000000001},"last":1.123}]},{"osd":7,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1619999999999999,"15min":1.1459999999999999},"min":{"1min":0.94099999999999995,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7709999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":0.98099999999999998,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.2390000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25372,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":23296,"kb_avail":94346468,"statfs":{"total":96636764160,"available":96610783232,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23855104},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.92500000000000004,"15min":0.90700000000000003},"min":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.45000000000000001},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88800000000000001,"15min":0.89500000000000002},"min":{"1min":0.63200000000000001,"5min":0.624,"15min":0.41599999999999998},"max":{"1min":1.893,"5min":1.893,"15min":2.2770000000000001},"last":1.024}]},{"osd":1,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.90200000000000002},"min":{"1min":0.75900000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.53500000000000003},"max":{"1min":1.758,"5min":1.758,"15min":2.1110000000000002},"last":0.89800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.98299999999999998,"15min":0.96799999999999997},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.966,"5min":1.966,"15min":2.056},"last":1.1990000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.998,"5min":1.998,"15min":2.032},"last":1.143},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.0880000000000001,"15min":1.054},"min":{"1min":0.82899999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.82899999999999996}]},{"osd":5,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93700000000000006,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.61399999999999999,"15min":0.45500000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.9930000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0600000000000001,"15min":1.0509999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.694,"5min":1.873,"15min":1.873},"last":1.117},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.1759999999999999,"15min":1.1679999999999999},"min":{"1min":0.96199999999999997,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1800000000000002},"last":1.262}]},{"osd":7,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.105,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1779999999999999,"15min":1.177},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1799999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30044,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341796,"statfs":{"total":96636764160,"available":96605999104,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83999999999999997,"5min":1.1639999999999999,"15min":1.3560000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.746},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81599999999999995,"5min":1.2629999999999999,"15min":1.377},"last":0.73499999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1799999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.69299999999999995,"5min":0.63,"15min":0.58999999999999997},"max":{"1min":1.2509999999999999,"5min":1.272,"15min":1.2849999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.876,"15min":0.873},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"max":{"1min":1.1599999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94099999999999995},"min":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.63800000000000001},"max":{"1min":1.175,"5min":1.3420000000000001,"15min":1.383},"last":1.1699999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.998,"15min":0.98399999999999999},"min":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.621},"max":{"1min":1.0600000000000001,"5min":1.599,"15min":1.599},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.86399999999999999,"5min":0.72099999999999997,"15min":0.68700000000000006},"max":{"1min":1.3069999999999999,"5min":1.587,"15min":1.587},"last":0.873}]},{"osd":5,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.123,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.484,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.181},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.72699999999999998},"max":{"1min":1.6499999999999999,"5min":1.659,"15min":1.659},"last":1.4530000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.92900000000000005,"5min":0.878,"15min":0.77800000000000002},"max":{"1min":1.288,"5min":1.5249999999999999,"15min":1.659},"last":1.079},{"interface":"front","average":{"1min":1.222,"5min":1.258,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.90500000000000003,"15min":0.82599999999999996},"max":{"1min":1.413,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.135}]},{"osd":7,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2629999999999999,"15min":1.25},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5629999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.94999999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.409,"5min":1.657,"15min":1.657},"last":0.97599999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29980,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":27904,"kb_avail":94341860,"statfs":{"total":96636764160,"available":96606064640,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28573696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.873,"5min":1.252,"15min":1.5329999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.55300000000000005,"15min":0.53700000000000003},"max":{"1min":1.2250000000000001,"5min":1.345,"15min":1.7969999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1899999999999999,"5min":1.242,"15min":2.0760000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.50800000000000001,"15min":0.495},"max":{"1min":1.103,"5min":1.1890000000000001,"15min":1.397},"last":1.383}]},{"osd":3,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.54900000000000004},"max":{"1min":0.90400000000000003,"5min":1.3149999999999999,"15min":1.667},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93600000000000005,"15min":0.95299999999999996},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2090000000000001,"5min":1.4359999999999999,"15min":1.7},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0900000000000001,"15min":1.085},"min":{"1min":0.86799999999999999,"5min":0.78600000000000003,"15min":0.73799999999999999},"max":{"1min":1.407,"5min":1.5349999999999999,"15min":1.972},"last":1.296},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0600000000000001},"min":{"1min":0.89400000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3819999999999999,"5min":1.655,"15min":1.8009999999999999},"last":1.179}]},{"osd":5,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.83999999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5209999999999999,"5min":1.5589999999999999,"15min":1.772},"last":1.456},{"interface":"front","average":{"1min":1.143,"5min":1.171,"15min":1.1819999999999999},"min":{"1min":0.84699999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5860000000000001,"5min":1.653,"15min":1.859},"last":0.99299999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1659999999999999,"15min":1.1599999999999999},"min":{"1min":0.92600000000000005,"5min":0.82299999999999995,"15min":0.80600000000000005},"max":{"1min":1.5529999999999999,"5min":1.6000000000000001,"15min":1.8999999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":1.0049999999999999,"5min":0.79800000000000004,"15min":0.76200000000000001},"max":{"1min":1.55,"5min":1.6439999999999999,"15min":1.736},"last":1.1180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.234,"15min":1.23},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.86599999999999999},"max":{"1min":1.548,"5min":1.615,"15min":2.0059999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.238},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.591,"5min":1.8129999999999999,"15min":1.8340000000000001},"last":1.335}]}]},{"osd":1,"up_from":14,"seq":60129542587,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44124,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":42048,"kb_avail":94327716,"statfs":{"total":96636764160,"available":96591581184,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43057152},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1819999999999999,"15min":1.2070000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.73999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.095,"5min":1.177,"15min":1.361},"last":0.73199999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.88,"5min":1.2509999999999999,"15min":1.383},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2350000000000001,"5min":1.3169999999999999,"15min":1.427},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.64600000000000002,"5min":0.626,"15min":0.58899999999999997},"max":{"1min":1.1890000000000001,"5min":1.3080000000000001,"15min":1.3720000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.83599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.454},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.98699999999999999,"15min":0.98999999999999999},"min":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.325,"15min":1.5229999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.196,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.71999999999999997},"max":{"1min":1.4299999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.091,"15min":1.0980000000000001},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.621},"max":{"1min":1.399,"5min":1.5449999999999999,"15min":1.583},"last":0.877},{"interface":"front","average":{"1min":1.077,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.747},"max":{"1min":1.3,"5min":1.4710000000000001,"15min":1.575},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2450000000000001,"15min":1.234},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.76800000000000002},"max":{"1min":1.5529999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.72299999999999998},"max":{"1min":1.4970000000000001,"5min":1.7789999999999999,"15min":1.804},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.94499999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.514,"5min":1.7130000000000001,"15min":1.853},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.95699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4139999999999999,"5min":1.585,"15min":1.827},"last":1.204}]}]},{"osd":0,"up_from":15,"seq":64424509882,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31136,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340704,"statfs":{"total":96636764160,"available":96604880896,"internally_reserved":0,"allocated":2711552,"data_stored":1702376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.85499999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.47499999999999998},"max":{"1min":1.131,"5min":1.1779999999999999,"15min":1.306},"last":0.76900000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.502},"max":{"1min":0.98299999999999998,"5min":1.236,"15min":1.5960000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.101,"5min":1.288,"15min":1.863},"last":0.94499999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.875},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.78600000000000003,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.92500000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3,"5min":1.4099999999999999,"15min":1.7110000000000001},"last":1.573},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.70699999999999996},"max":{"1min":1.516,"5min":1.5249999999999999,"15min":1.651},"last":1.4810000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.111,"15min":1.111},"min":{"1min":0.90700000000000003,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.407,"5min":1.4259999999999999,"15min":1.512},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.165,"15min":1.1779999999999999},"min":{"1min":0.86399999999999999,"5min":0.83399999999999996,"15min":0.79800000000000004},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.847},"last":1.444}]},{"osd":6,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1870000000000001,"15min":1.1830000000000001},"min":{"1min":0.90700000000000003,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.895},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2330000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.84999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5720000000000001,"15min":2.0510000000000002},"last":1.1930000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.286,"15min":1.2789999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.84799999999999998},"max":{"1min":1.554,"5min":1.6100000000000001,"15min":2.0030000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.85599999999999998},"max":{"1min":1.4790000000000001,"5min":1.806,"15min":2.004},"last":1.5229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:50:37.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:50:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 Processor -- start 2023-09-27T23:50:37.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 -- start start 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83d8155960 0x7f83d815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 0x7f83d80aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d806d990 con 0x7f83d806de20 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d815a570 con 0x7f83d80c7f00 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83df3ec700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d806cd80 con 0x7f83d8155960 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83debeb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 0x7f83d80aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83de3ea700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83d8155960 0x7f83d815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83debeb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 0x7f83d80aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54526/0 (socket says 172.21.15.64:54526) 2023-09-27T23:50:37.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83debeb700 1 -- 172.21.15.64:0/70812594 learned_addr learned my addr 172.21.15.64:0/70812594 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83de3ea700 1 -- 172.21.15.64:0/70812594 >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 msgr2=0x7f83d80aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83de3ea700 1 --2- 172.21.15.64:0/70812594 >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 0x7f83d80aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83de3ea700 1 -- 172.21.15.64:0/70812594 >> v1:172.21.15.123:6789/0 conn(0x7f83d806de20 legacy=0x7f83d80c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83de3ea700 1 -- 172.21.15.64:0/70812594 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d806d060 con 0x7f83d8155960 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.987+0000 7f83debeb700 1 --2- 172.21.15.64:0/70812594 >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 0x7f83d80aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83de3ea700 1 --2- 172.21.15.64:0/70812594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d8155960 0x7f83d815a030 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f83d8064070 tx=0x7f83d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68f99e859447d703 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83dd3e8700 1 -- 172.21.15.64:0/70812594 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4011630 con 0x7f83d8155960 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83dd3e8700 1 -- 172.21.15.64:0/70812594 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d4011790 con 0x7f83d8155960 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83dd3e8700 1 -- 172.21.15.64:0/70812594 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4011b30 con 0x7f83d8155960 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 -- 172.21.15.64:0/70812594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d8155960 msgr2=0x7f83d815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 --2- 172.21.15.64:0/70812594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d8155960 0x7f83d815a030 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f83d8064070 tx=0x7f83d40091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 -- 172.21.15.64:0/70812594 shutdown_connections 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 --2- 172.21.15.64:0/70812594 >> v2:172.21.15.64:3300/0 conn(0x7f83d80c7f00 0x7f83d80aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 --2- 172.21.15.64:0/70812594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d8155960 0x7f83d815a030 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 -- 172.21.15.64:0/70812594 >> 172.21.15.64:0/70812594 conn(0x7f83d8067bb0 msgr2=0x7f83d80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 -- 172.21.15.64:0/70812594 shutdown_connections 2023-09-27T23:50:37.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 -- 172.21.15.64:0/70812594 wait complete. 2023-09-27T23:50:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 Processor -- start 2023-09-27T23:50:37.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.991+0000 7f83df3ec700 1 -- start start 2023-09-27T23:50:37.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83d806de20 0x7f83d8153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d80c7f00 0x7f83d81574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 0x7f83d814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d81523f0 con 0x7f83d8155960 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8157bb0 con 0x7f83d80c7f00 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8157e90 con 0x7f83d806de20 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 0x7f83d814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 0x7f83d814e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54542/0 (socket says 172.21.15.64:54542) 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 -- 172.21.15.64:0/2467954266 learned_addr learned my addr 172.21.15.64:0/2467954266 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83de3ea700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83d806de20 0x7f83d8153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83ddbe9700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d80c7f00 0x7f83d81574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 -- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83d806de20 msgr2=0x7f83d8153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83d806de20 0x7f83d8153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 -- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d80c7f00 msgr2=0x7f83d81574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d80c7f00 0x7f83d81574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 -- 172.21.15.64:0/2467954266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0009ae0 con 0x7f83d8155960 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83debeb700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 0x7f83d814e4a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009fd0 tx=0x7f83d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0018070 con 0x7f83d8155960 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d000f040 con 0x7f83d8155960 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d4009420 con 0x7f83d8155960 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000cc80 con 0x7f83d8155960 2023-09-27T23:50:37.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d8152080 con 0x7f83d8155960 2023-09-27T23:50:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d81525a0 con 0x7f83d8155960 2023-09-27T23:50:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.995+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f83d0003020 con 0x7f83d8155960 2023-09-27T23:50:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.999+0000 7f83ce7fc700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f83bc060aa0 0x7f83bc062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.999+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f83d007d730 con 0x7f83d8155960 2023-09-27T23:50:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.999+0000 7f83de3ea700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f83bc060aa0 0x7f83bc062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:37.999+0000 7f83de3ea700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f83bc060aa0 0x7f83bc062f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83d40093f0 tx=0x7f83d4017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.003+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f83d81525a0 con 0x7f83d8155960 2023-09-27T23:50:38.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.167+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83d804be40 con 0x7f83bc060aa0 2023-09-27T23:50:38.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.183+0000 7f83ce7fc700 1 -- 172.21.15.64:0/2467954266 <== mgr.8210 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297089 (secure 0 0 0) 0x7f83d804be40 con 0x7f83bc060aa0 2023-09-27T23:50:38.189 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f83bc060aa0 msgr2=0x7f83bc062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f83bc060aa0 0x7f83bc062f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83d40093f0 tx=0x7f83d4017000 comp rx=0 tx=0).stop 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 msgr2=0x7f83d814e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 0x7f83d814e4a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009fd0 tx=0x7f83d000b040 comp rx=0 tx=0).stop 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 shutdown_connections 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f83bc060aa0 0x7f83bc062f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83d8155960 0x7f83d814e4a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d80c7f00 0x7f83d81574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 --2- 172.21.15.64:0/2467954266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83d806de20 0x7f83d8153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 >> 172.21.15.64:0/2467954266 conn(0x7f83d8067bb0 msgr2=0x7f83d81513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:38.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 shutdown_connections 2023-09-27T23:50:38.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.187+0000 7f83df3ec700 1 -- 172.21.15.64:0/2467954266 wait complete. 2023-09-27T23:50:38.193 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:50:38.210 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-09-27T23:50:36.536365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264768,"kb_used_data":17728,"kb_used_omap":0,"kb_used_meta":246976,"kb_avail":754709952,"statfs":{"total":773094113280,"available":772822990848,"internally_reserved":0,"allocated":18153472,"data_stored":10089756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":252903424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002339"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.159677+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:37.863006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157624+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151466+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:15:39.785598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.155020+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:02:39.171495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149401+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:25:45.048987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.152131+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:23:40.565795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157802+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:10.178409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158342+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149396+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:59:58.865116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.150826+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:19:43.857264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158475+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158606+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:13.108314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.154259+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:57.422152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.159790+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:50.280393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.152989+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:47:00.473077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158488+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146975+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.149588+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:08:16.153188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.156804+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146864+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:57:59.225226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157540+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:56:50.893048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151058+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:12:04.021162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.156787+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:00.466792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158038+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:48:02.947172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.157610+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:28:23.297242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"162'304","reported_seq":957,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164795+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:50:32.164795+0000","last_peered":"2023-09-27T23:50:32.164795+0000","last_clean":"2023-09-27T23:50:32.164795+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:50:32.164795+0000","last_undegraded":"2023-09-27T23:50:32.164795+0000","last_fullsized":"2023-09-27T23:50:32.164795+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":304,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.132236+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:50:31.132236+0000","last_peered":"2023-09-27T23:50:31.132236+0000","last_clean":"2023-09-27T23:50:31.132236+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:50:31.132236+0000","last_undegraded":"2023-09-27T23:50:31.132236+0000","last_fullsized":"2023-09-27T23:50:31.132236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.158167+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:40:13.856739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164771+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:50:32.164771+0000","last_peered":"2023-09-27T23:50:32.164771+0000","last_clean":"2023-09-27T23:50:32.164771+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:50:32.164771+0000","last_undegraded":"2023-09-27T23:50:32.164771+0000","last_fullsized":"2023-09-27T23:50:32.164771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.150966+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:56:14.072256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.162428+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:50:32.162428+0000","last_peered":"2023-09-27T23:50:32.162428+0000","last_clean":"2023-09-27T23:50:32.162428+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:50:32.162428+0000","last_undegraded":"2023-09-27T23:50:32.162428+0000","last_fullsized":"2023-09-27T23:50:32.162428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.151362+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.132223+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:50:31.132223+0000","last_peered":"2023-09-27T23:50:31.132223+0000","last_clean":"2023-09-27T23:50:31.132223+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:50:31.132223+0000","last_undegraded":"2023-09-27T23:50:31.132223+0000","last_fullsized":"2023-09-27T23:50:31.132223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.160602+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:18:34.510803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":149,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.130948+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:50:31.130948+0000","last_peered":"2023-09-27T23:50:31.130948+0000","last_clean":"2023-09-27T23:50:31.130948+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:50:31.130948+0000","last_undegraded":"2023-09-27T23:50:31.130948+0000","last_fullsized":"2023-09-27T23:50:31.130948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.153811+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:06:48.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164785+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:50:32.164785+0000","last_peered":"2023-09-27T23:50:32.164785+0000","last_clean":"2023-09-27T23:50:32.164785+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:50:32.164785+0000","last_undegraded":"2023-09-27T23:50:32.164785+0000","last_fullsized":"2023-09-27T23:50:32.164785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.146863+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:31.131143+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:50:31.131143+0000","last_peered":"2023-09-27T23:50:31.131143+0000","last_clean":"2023-09-27T23:50:31.131143+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:50:31.131143+0000","last_undegraded":"2023-09-27T23:50:31.131143+0000","last_fullsized":"2023-09-27T23:50:31.131143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:50:34.140545+0000","last_change":"2023-09-27T23:50:34.140545+0000","last_active":"2023-09-27T23:50:34.140545+0000","last_peered":"2023-09-27T23:50:34.140545+0000","last_clean":"2023-09-27T23:50:34.140545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:34.140545+0000","last_undegraded":"2023-09-27T23:50:34.140545+0000","last_fullsized":"2023-09-27T23:50:34.140545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-09-27T23:50:33.128807+0000","last_change":"2023-09-27T23:50:33.154600+0000","last_active":"2023-09-27T23:50:33.128807+0000","last_peered":"2023-09-27T23:50:33.128807+0000","last_clean":"2023-09-27T23:50:33.128807+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:50:33.128807+0000","last_undegraded":"2023-09-27T23:50:33.128807+0000","last_fullsized":"2023-09-27T23:50:33.128807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:47:05.296212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-09-27T23:50:32.164067+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:50:32.164067+0000","last_peered":"2023-09-27T23:50:32.164067+0000","last_clean":"2023-09-27T23:50:32.164067+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:50:32.164067+0000","last_undegraded":"2023-09-27T23:50:32.164067+0000","last_fullsized":"2023-09-27T23:50:32.164067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35500,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":32832,"kb_avail":94336340,"statfs":{"total":96636764160,"available":96600412160,"internally_reserved":0,"allocated":2723840,"data_stored":1708854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33619968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89200000000000002,"15min":0.89500000000000002},"min":{"1min":0.73399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1899999999999999,"5min":1.2809999999999999,"15min":1.3440000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.93799999999999994,"15min":0.92100000000000004},"min":{"1min":0.73999999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2529999999999999,"5min":1.3520000000000001,"15min":1.403},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.92700000000000005,"15min":0.90500000000000003},"min":{"1min":0.65100000000000002,"5min":0.61099999999999999,"15min":0.57199999999999995},"max":{"1min":1.1879999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":0.96299999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.63600000000000001},"max":{"1min":1.1919999999999999,"5min":1.2629999999999999,"15min":1.4530000000000001},"last":1.208}]},{"osd":2,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.69999999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":0.96699999999999997},"min":{"1min":0.68100000000000005,"5min":0.63600000000000001,"15min":0.628},"max":{"1min":1.341,"5min":1.341,"15min":1.3460000000000001},"last":1.032}]},{"osd":3,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95199999999999996,"15min":0.94199999999999995},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.66900000000000004},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0900000000000001,"15min":1.0800000000000001},"min":{"1min":0.87,"5min":0.755,"15min":0.71899999999999997},"max":{"1min":1.2470000000000001,"5min":1.3240000000000001,"15min":1.3879999999999999},"last":1.115}]},{"osd":4,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.109,"5min":1.3819999999999999,"15min":1.411},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.02,"15min":1.006},"min":{"1min":0.78600000000000003,"5min":0.67900000000000005,"15min":0.622},"max":{"1min":1.343,"5min":1.488,"15min":1.488},"last":1.008}]},{"osd":5,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.65600000000000003},"max":{"1min":1.2949999999999999,"5min":1.484,"15min":1.484},"last":1.181},{"interface":"front","average":{"1min":1.133,"5min":1.117,"15min":1.1180000000000001},"min":{"1min":0.79400000000000004,"5min":0.75,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed Sep 27 23:49:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.3169999999999999,"5min":1.425,"15min":1.538},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.155,"15min":1.149},"min":{"1min":0.93500000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.409,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.1439999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2129920,"data_stored":1118486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.99399999999999999,"15min":0.97099999999999997},"min":{"1min":0.65900000000000003,"5min":0.60599999999999998,"15min":0.51800000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0529999999999999,"15min":1.0600000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.69399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.66800000000000004,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.57099999999999995},"max":{"1min":1.226,"5min":1.53,"15min":1.53},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61399999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.63200000000000001},"max":{"1min":1.2709999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.0840000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.74099999999999999,"5min":0.65700000000000003,"15min":0.63100000000000001},"max":{"1min":1.53,"5min":1.583,"15min":1.583},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.169,"5min":1.325,"15min":1.325},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.878,"15min":0.873},"min":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.41599999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0129999999999999,"15min":1},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3700000000000001,"5min":1.633,"15min":1.633},"last":0.78000000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.95099999999999996},"min":{"1min":0.71099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.4430000000000001},"last":1.016},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.149,"15min":1.141},"min":{"1min":0.81299999999999994,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.143}]},{"osd":7,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.1180000000000001},"min":{"1min":0.755,"5min":0.627,"15min":0.627},"max":{"1min":1.623,"5min":1.679,"15min":1.7210000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.423,"5min":1.488,"15min":1.542},"last":1.1160000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39068,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":36992,"kb_avail":94332772,"statfs":{"total":96636764160,"available":96596758528,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37879808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86699999999999999,"15min":0.86199999999999999},"min":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.51900000000000002},"max":{"1min":1.248,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.47899999999999998},"max":{"1min":1.2250000000000001,"5min":1.292,"15min":1.5640000000000001},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.302,"5min":1.425,"15min":1.538},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.194,"5min":1.3819999999999999,"15min":1.397},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.629},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.4690000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.626},"max":{"1min":1.153,"5min":1.3049999999999999,"15min":1.4470000000000001},"last":0.80200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.3240000000000001,"5min":1.373,"15min":1.611},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0349999999999999,"15min":1.0249999999999999},"min":{"1min":0.84599999999999997,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.228,"5min":1.419,"15min":1.4370000000000001},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.88},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.3129999999999999,"5min":1.484,"15min":1.484},"last":0.998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.58699999999999997},"max":{"1min":1.45,"5min":1.45,"15min":1.4510000000000001},"last":1.1000000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.026},"min":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.242,"5min":1.496,"15min":1.5069999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.099,"15min":1.0980000000000001},"min":{"1min":0.84899999999999998,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":1.262,"5min":1.546,"15min":1.5840000000000001},"last":1.123}]},{"osd":7,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1619999999999999,"15min":1.1459999999999999},"min":{"1min":0.94099999999999995,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7709999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":0.98099999999999998,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.2390000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25372,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":23296,"kb_avail":94346468,"statfs":{"total":96636764160,"available":96610783232,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23855104},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.92500000000000004,"15min":0.90700000000000003},"min":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.45000000000000001},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88800000000000001,"15min":0.89500000000000002},"min":{"1min":0.63200000000000001,"5min":0.624,"15min":0.41599999999999998},"max":{"1min":1.893,"5min":1.893,"15min":2.2770000000000001},"last":1.024}]},{"osd":1,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.90200000000000002},"min":{"1min":0.75900000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.53500000000000003},"max":{"1min":1.758,"5min":1.758,"15min":2.1110000000000002},"last":0.89800000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.98299999999999998,"15min":0.96799999999999997},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.966,"5min":1.966,"15min":2.056},"last":1.1990000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.998,"5min":1.998,"15min":2.032},"last":1.143},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.0880000000000001,"15min":1.054},"min":{"1min":0.82899999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.82899999999999996}]},{"osd":5,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93700000000000006,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.61399999999999999,"15min":0.45500000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.9930000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0600000000000001,"15min":1.0509999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.694,"5min":1.873,"15min":1.873},"last":1.117},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.1759999999999999,"15min":1.1679999999999999},"min":{"1min":0.96199999999999997,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1800000000000002},"last":1.262}]},{"osd":7,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.105,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1779999999999999,"15min":1.177},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1799999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30044,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341796,"statfs":{"total":96636764160,"available":96605999104,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83999999999999997,"5min":1.1639999999999999,"15min":1.3560000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.746},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81599999999999995,"5min":1.2629999999999999,"15min":1.377},"last":0.73499999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1799999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.69299999999999995,"5min":0.63,"15min":0.58999999999999997},"max":{"1min":1.2509999999999999,"5min":1.272,"15min":1.2849999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.876,"15min":0.873},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"max":{"1min":1.1599999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94099999999999995},"min":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.63800000000000001},"max":{"1min":1.175,"5min":1.3420000000000001,"15min":1.383},"last":1.1699999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.998,"15min":0.98399999999999999},"min":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.621},"max":{"1min":1.0600000000000001,"5min":1.599,"15min":1.599},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.86399999999999999,"5min":0.72099999999999997,"15min":0.68700000000000006},"max":{"1min":1.3069999999999999,"5min":1.587,"15min":1.587},"last":0.873}]},{"osd":5,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.123,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.484,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.181},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.72699999999999998},"max":{"1min":1.6499999999999999,"5min":1.659,"15min":1.659},"last":1.4530000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.92900000000000005,"5min":0.878,"15min":0.77800000000000002},"max":{"1min":1.288,"5min":1.5249999999999999,"15min":1.659},"last":1.079},{"interface":"front","average":{"1min":1.222,"5min":1.258,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.90500000000000003,"15min":0.82599999999999996},"max":{"1min":1.413,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.135}]},{"osd":7,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2629999999999999,"15min":1.25},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5629999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.94999999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.409,"5min":1.657,"15min":1.657},"last":0.97599999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29980,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":27904,"kb_avail":94341860,"statfs":{"total":96636764160,"available":96606064640,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28573696},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.873,"5min":1.252,"15min":1.5329999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.55300000000000005,"15min":0.53700000000000003},"max":{"1min":1.2250000000000001,"5min":1.345,"15min":1.7969999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1899999999999999,"5min":1.242,"15min":2.0760000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.50800000000000001,"15min":0.495},"max":{"1min":1.103,"5min":1.1890000000000001,"15min":1.397},"last":1.383}]},{"osd":3,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.54900000000000004},"max":{"1min":0.90400000000000003,"5min":1.3149999999999999,"15min":1.667},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93600000000000005,"15min":0.95299999999999996},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2090000000000001,"5min":1.4359999999999999,"15min":1.7},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0900000000000001,"15min":1.085},"min":{"1min":0.86799999999999999,"5min":0.78600000000000003,"15min":0.73799999999999999},"max":{"1min":1.407,"5min":1.5349999999999999,"15min":1.972},"last":1.296},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0600000000000001},"min":{"1min":0.89400000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3819999999999999,"5min":1.655,"15min":1.8009999999999999},"last":1.179}]},{"osd":5,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.83999999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5209999999999999,"5min":1.5589999999999999,"15min":1.772},"last":1.456},{"interface":"front","average":{"1min":1.143,"5min":1.171,"15min":1.1819999999999999},"min":{"1min":0.84699999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5860000000000001,"5min":1.653,"15min":1.859},"last":0.99299999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1659999999999999,"15min":1.1599999999999999},"min":{"1min":0.92600000000000005,"5min":0.82299999999999995,"15min":0.80600000000000005},"max":{"1min":1.5529999999999999,"5min":1.6000000000000001,"15min":1.8999999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":1.0049999999999999,"5min":0.79800000000000004,"15min":0.76200000000000001},"max":{"1min":1.55,"5min":1.6439999999999999,"15min":1.736},"last":1.1180000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:34 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.234,"15min":1.23},"min":{"1min":0.97799999999999998,"5min":0.92500000000000004,"15min":0.86599999999999999},"max":{"1min":1.548,"5min":1.615,"15min":2.0059999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.238},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.591,"5min":1.8129999999999999,"15min":1.8340000000000001},"last":1.335}]}]},{"osd":1,"up_from":14,"seq":60129542587,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44124,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":42048,"kb_avail":94327716,"statfs":{"total":96636764160,"available":96591581184,"internally_reserved":0,"allocated":2117632,"data_stored":1112008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43057152},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1819999999999999,"15min":1.2070000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.73999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.095,"5min":1.177,"15min":1.361},"last":0.73199999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.88,"5min":1.2509999999999999,"15min":1.383},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2350000000000001,"5min":1.3169999999999999,"15min":1.427},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.64600000000000002,"5min":0.626,"15min":0.58899999999999997},"max":{"1min":1.1890000000000001,"5min":1.3080000000000001,"15min":1.3720000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.83599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.454},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.98699999999999999,"15min":0.98999999999999999},"min":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.325,"15min":1.5229999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.196,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.71999999999999997},"max":{"1min":1.4299999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.091,"15min":1.0980000000000001},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.621},"max":{"1min":1.399,"5min":1.5449999999999999,"15min":1.583},"last":0.877},{"interface":"front","average":{"1min":1.077,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.747},"max":{"1min":1.3,"5min":1.4710000000000001,"15min":1.575},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2450000000000001,"15min":1.234},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.76800000000000002},"max":{"1min":1.5529999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.72299999999999998},"max":{"1min":1.4970000000000001,"5min":1.7789999999999999,"15min":1.804},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.94499999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.514,"5min":1.7130000000000001,"15min":1.853},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.95699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4139999999999999,"5min":1.585,"15min":1.827},"last":1.204}]}]},{"osd":0,"up_from":15,"seq":64424509882,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31136,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340704,"statfs":{"total":96636764160,"available":96604880896,"internally_reserved":0,"allocated":2711552,"data_stored":1702376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.85499999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.47499999999999998},"max":{"1min":1.131,"5min":1.1779999999999999,"15min":1.306},"last":0.76900000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.502},"max":{"1min":0.98299999999999998,"5min":1.236,"15min":1.5960000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.101,"5min":1.288,"15min":1.863},"last":0.94499999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.875},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.78600000000000003,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.92500000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3,"5min":1.4099999999999999,"15min":1.7110000000000001},"last":1.573},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.70699999999999996},"max":{"1min":1.516,"5min":1.5249999999999999,"15min":1.651},"last":1.4810000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.111,"15min":1.111},"min":{"1min":0.90700000000000003,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.407,"5min":1.4259999999999999,"15min":1.512},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.165,"15min":1.1779999999999999},"min":{"1min":0.86399999999999999,"5min":0.83399999999999996,"15min":0.79800000000000004},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.847},"last":1.444}]},{"osd":6,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1870000000000001,"15min":1.1830000000000001},"min":{"1min":0.90700000000000003,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.895},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2330000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.84999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5720000000000001,"15min":2.0510000000000002},"last":1.1930000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.286,"15min":1.2789999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.84799999999999998},"max":{"1min":1.554,"5min":1.6100000000000001,"15min":2.0030000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.85599999999999998},"max":{"1min":1.4790000000000001,"5min":1.806,"15min":2.004},"last":1.5229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:50:38.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3f is not active+clean 2023-09-27T23:50:38.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3e is not active+clean 2023-09-27T23:50:38.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3d is not active+clean 2023-09-27T23:50:38.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3c is not active+clean 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3b is not active+clean 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3a is not active+clean 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.39 is not active+clean 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.38 is not active+clean 2023-09-27T23:50:38.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.37 is not active+clean 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.36 is not active+clean 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.35 is not active+clean 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.34 is not active+clean 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.33 is not active+clean 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.32 is not active+clean 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.31 is not active+clean 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.30 is not active+clean 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2f is not active+clean 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2e is not active+clean 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2d is not active+clean 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2c is not active+clean 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2b is not active+clean 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2a is not active+clean 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.29 is not active+clean 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.28 is not active+clean 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.27 is not active+clean 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.26 is not active+clean 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.25 is not active+clean 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.24 is not active+clean 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.23 is not active+clean 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.22 is not active+clean 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.21 is not active+clean 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.20 is not active+clean 2023-09-27T23:50:38.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1e is not active+clean 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.159677+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T01:36:37.863006+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1f is not active+clean 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.157624+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1c is not active+clean 2023-09-27T23:50:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.151466+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T03:15:39.785598+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1d is not active+clean 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.155020+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T09:02:39.171495+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1a is not active+clean 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2023-09-27T23:50:38.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.149401+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T06:25:45.048987+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1b is not active+clean 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.152131+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T06:23:40.565795+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.18 is not active+clean 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.157802+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T10:39:10.178409+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.19 is not active+clean 2023-09-27T23:50:38.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.158342+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T09:19:06.929184+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.e is not active+clean 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.149396+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T00:59:58.865116+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.f is not active+clean 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2023-09-27T23:50:38.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.150826+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T11:19:43.857264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.c is not active+clean 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.158475+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.d is not active+clean 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.158606+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T00:02:13.108314+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-27T23:50:38.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.a is not active+clean 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.154259+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T10:04:57.422152+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.b is not active+clean 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.159790+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T08:15:50.280393+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.8 is not active+clean 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.152989+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T07:47:00.473077+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.9 is not active+clean 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.158488+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.6 is not active+clean 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.146975+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T00:17:58.304153+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-27T23:50:38.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.7 is not active+clean 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.149588+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T11:08:16.153188+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.4 is not active+clean 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.156804+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.5 is not active+clean 2023-09-27T23:50:38.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.146864+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T02:57:59.225226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2 is not active+clean 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.157540+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T02:56:50.893048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3 is not active+clean 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2023-09-27T23:50:38.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.151058+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T00:12:04.021162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.0 is not active+clean 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.156787+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T10:51:00.466792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1 is not active+clean 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.158038+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T00:48:02.947172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.16 is not active+clean 2023-09-27T23:50:38.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.157610+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T07:28:23.297242+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.15 is not active+clean 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.158167+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T05:40:13.856739+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.14 is not active+clean 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2023-09-27T23:50:38.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.150966+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T08:56:14.072256+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.17 is not active+clean 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.151362+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T03:43:04.166279+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.11 is not active+clean 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.160602+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T08:18:34.510803+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-27T23:50:38.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.10 is not active+clean 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.153811+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T04:06:48.834774+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.13 is not active+clean 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.146863+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T07:21:04.692439+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.12 is not active+clean 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:50:34.140545+0000', 'last_change': '2023-09-27T23:50:34.140545+0000', 'last_active': '2023-09-27T23:50:34.140545+0000', 'last_peered': '2023-09-27T23:50:34.140545+0000', 'last_clean': '2023-09-27T23:50:34.140545+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:34.140545+0000', 'last_undegraded': '2023-09-27T23:50:34.140545+0000', 'last_fullsized': '2023-09-27T23:50:34.140545+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:34.140545+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:50:38.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2023-09-27T23:50:38.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-09-27T23:50:33.128807+0000', 'last_change': '2023-09-27T23:50:33.154600+0000', 'last_active': '2023-09-27T23:50:33.128807+0000', 'last_peered': '2023-09-27T23:50:33.128807+0000', 'last_clean': '2023-09-27T23:50:33.128807+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:50:33.128807+0000', 'last_undegraded': '2023-09-27T23:50:33.128807+0000', 'last_fullsized': '2023-09-27T23:50:33.128807+0000', 'mapping_epoch': 171, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 171, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'last_clean_scrub_stamp': '2023-09-27T23:50:33.128807+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-09-29T03:47:05.296212+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-27T23:50:38.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:50:38.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 Processor -- start 2023-09-27T23:50:38.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- start start 2023-09-27T23:50:38.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc65015dc80 0x7fc6501681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6500c7ca0 con 0x7fc65006d760 2023-09-27T23:50:38.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65015e0b0 con 0x7fc65015d1a0 2023-09-27T23:50:38.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65015e380 con 0x7fc65015dc80 2023-09-27T23:50:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc647fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc65006d760 0x7fc65015d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40966/0 (socket says 172.21.15.64:40966) 2023-09-27T23:50:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc647fff700 1 -- 172.21.15.64:0/1619948164 learned_addr learned my addr 172.21.15.64:0/1619948164 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:38.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6555c4700 1 --2- 172.21.15.64:0/1619948164 >> v2:172.21.15.64:3300/0 conn(0x7fc65015dc80 0x7fc6501681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6477fe700 1 -- 172.21.15.64:0/1619948164 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528247417 0 0) 0x7fc6500c7ca0 con 0x7fc65006d760 2023-09-27T23:50:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6477fe700 1 -- 172.21.15.64:0/1619948164 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc638003570 con 0x7fc65006d760 2023-09-27T23:50:38.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6477fe700 1 -- 172.21.15.64:0/1619948164 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 656649604 0 0) 0x7fc638003570 con 0x7fc65006d760 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6555c4700 1 -- 172.21.15.64:0/1619948164 >> v1:172.21.15.123:6789/0 conn(0x7fc65015d1a0 legacy=0x7fc65006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6555c4700 1 -- 172.21.15.64:0/1619948164 >> v1:172.21.15.64:6789/0 conn(0x7fc65006d760 legacy=0x7fc65015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6555c4700 1 -- 172.21.15.64:0/1619948164 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65015e660 con 0x7fc65015dc80 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6555c4700 1 --2- 172.21.15.64:0/1619948164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc6501681f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fc648009e60 tx=0x7fc648009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54339c723ffb2d91 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6477fe700 1 -- 172.21.15.64:0/1619948164 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800e070 con 0x7fc65015dc80 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6477fe700 1 -- 172.21.15.64:0/1619948164 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64800b850 con 0x7fc65015dc80 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc6477fe700 1 -- 172.21.15.64:0/1619948164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640001470 con 0x7fc65015dc80 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- 172.21.15.64:0/1619948164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 msgr2=0x7fc6501681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 --2- 172.21.15.64:0/1619948164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc6501681f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fc648009e60 tx=0x7fc648009780 comp rx=0 tx=0).stop 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- 172.21.15.64:0/1619948164 shutdown_connections 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 --2- 172.21.15.64:0/1619948164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc6501681f0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- 172.21.15.64:0/1619948164 >> 172.21.15.64:0/1619948164 conn(0x7fc65006c080 msgr2=0x7fc6500ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.346+0000 7fc655dc5700 1 -- 172.21.15.64:0/1619948164 shutdown_connections 2023-09-27T23:50:38.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- 172.21.15.64:0/1619948164 wait complete. 2023-09-27T23:50:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 Processor -- start 2023-09-27T23:50:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- start start 2023-09-27T23:50:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65006d760 0x7fc6501496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc65015d1a0 0x7fc650149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc650160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6501605c0 con 0x7fc65015dc80 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500a4690 con 0x7fc65006d760 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015f9b0 con 0x7fc65015d1a0 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc650160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc650160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54556/0 (socket says 172.21.15.64:54556) 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 -- 172.21.15.64:0/482659909 learned_addr learned my addr 172.21.15.64:0/482659909 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc647fff700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc65015d1a0 0x7fc650149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc654dc3700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65006d760 0x7fc6501496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 -- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc65015d1a0 msgr2=0x7fc650149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc65015d1a0 0x7fc650149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 -- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65006d760 msgr2=0x7fc6501496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65006d760 0x7fc6501496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 -- 172.21.15.64:0/482659909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648009a20 con 0x7fc65015dc80 2023-09-27T23:50:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc654dc3700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65006d760 0x7fc6501496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:50:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc6555c4700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc650160a50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc64800b6c0 tx=0x7fc648017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648012ca0 con 0x7fc65015dc80 2023-09-27T23:50:38.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc648012e20 con 0x7fc65015dc80 2023-09-27T23:50:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65015fc90 con 0x7fc65015dc80 2023-09-27T23:50:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64801d6c0 con 0x7fc65015dc80 2023-09-27T23:50:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.350+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc650160250 con 0x7fc65015dc80 2023-09-27T23:50:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.354+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc650160730 con 0x7fc65015dc80 2023-09-27T23:50:38.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.354+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc64801dab0 con 0x7fc65015dc80 2023-09-27T23:50:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.354+0000 7fc644ff9700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc63c060aa0 0x7fc63c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.354+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc64807ef30 con 0x7fc65015dc80 2023-09-27T23:50:38.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.354+0000 7fc654dc3700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc63c060aa0 0x7fc63c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.354+0000 7fc654dc3700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc63c060aa0 0x7fc63c062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc64c001840 tx=0x7fc64c009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.362+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc64804ee50 con 0x7fc65015dc80 2023-09-27T23:50:38.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.582+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc65004be40 con 0x7fc65015dc80 2023-09-27T23:50:38.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc644ff9700 1 -- 172.21.15.64:0/482659909 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3216 (secure 0 0 0) 0x7fc648016ca0 con 0x7fc65015dc80 2023-09-27T23:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:38.588 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":2237,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":174,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":272801792,"bytes_avail":772821311488,"bytes_total":773094113280,"write_bytes_sec":2468,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":66,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9783}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2023-09-27T23:50:24.531635+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:50:38.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc63c060aa0 msgr2=0x7fc63c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc63c060aa0 0x7fc63c062f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc64c001840 tx=0x7fc64c009040 comp rx=0 tx=0).stop 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 msgr2=0x7fc650160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc650160a50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc64800b6c0 tx=0x7fc648017000 comp rx=0 tx=0).stop 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 shutdown_connections 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc63c060aa0 0x7fc63c062f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc65015dc80 0x7fc650160a50 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc65015d1a0 0x7fc650149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 --2- 172.21.15.64:0/482659909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65006d760 0x7fc6501496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 >> 172.21.15.64:0/482659909 conn(0x7fc65006c080 msgr2=0x7fc6500cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:38.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 shutdown_connections 2023-09-27T23:50:38.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.586+0000 7fc655dc5700 1 -- 172.21.15.64:0/482659909 wait complete. 2023-09-27T23:50:38.603 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:50:38.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:50:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.702+0000 7f3bdad4b700 1 Processor -- start 2023-09-27T23:50:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.702+0000 7f3bdad4b700 1 -- start start 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.702+0000 7f3bdad4b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3bd4158e70 0x7f3bd40afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd40ca5a0 con 0x7f3bd40cc470 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd406dac0 con 0x7f3bd406a540 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd40b0260 con 0x7f3bd4158e70 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd9d49700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3bd4158e70 0x7f3bd40afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bda54a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3bd40cc470 0x7f3bd40cc890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40978/0 (socket says 172.21.15.64:40978) 2023-09-27T23:50:38.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bda54a700 1 -- 172.21.15.64:0/2302762139 learned_addr learned my addr 172.21.15.64:0/2302762139 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd8d47700 1 -- 172.21.15.64:0/2302762139 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383126243 0 0) 0x7f3bd40ca5a0 con 0x7f3bd40cc470 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd8d47700 1 -- 172.21.15.64:0/2302762139 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc0003570 con 0x7f3bd40cc470 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd9d49700 1 -- 172.21.15.64:0/2302762139 >> v1:172.21.15.123:6789/0 conn(0x7f3bd406a540 legacy=0x7f3bd406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd8d47700 1 -- 172.21.15.64:0/2302762139 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1768694573 0 0) 0x7f3bc0003570 con 0x7f3bd40cc470 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd9d49700 1 -- 172.21.15.64:0/2302762139 >> v1:172.21.15.64:6789/0 conn(0x7f3bd40cc470 legacy=0x7f3bd40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd9d49700 1 -- 172.21.15.64:0/2302762139 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd406d220 con 0x7f3bd4158e70 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd9d49700 1 --2- 172.21.15.64:0/2302762139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd4158e70 0x7f3bd40afd20 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098e0 tx=0x7f3bd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faf4a181b96b935 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd8d47700 1 -- 172.21.15.64:0/2302762139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0017f0 con 0x7f3bd4158e70 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd8d47700 1 -- 172.21.15.64:0/2302762139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd0011420 con 0x7f3bd4158e70 2023-09-27T23:50:38.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bd8d47700 1 -- 172.21.15.64:0/2302762139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00117c0 con 0x7f3bd4158e70 2023-09-27T23:50:38.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- 172.21.15.64:0/2302762139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd4158e70 msgr2=0x7f3bd40afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/2302762139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd4158e70 0x7f3bd40afd20 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098e0 tx=0x7f3bd00091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- 172.21.15.64:0/2302762139 shutdown_connections 2023-09-27T23:50:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/2302762139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd4158e70 0x7f3bd40afd20 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- 172.21.15.64:0/2302762139 >> 172.21.15.64:0/2302762139 conn(0x7f3bd40670e0 msgr2=0x7f3bd40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- 172.21.15.64:0/2302762139 shutdown_connections 2023-09-27T23:50:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.706+0000 7f3bdad4b700 1 -- 172.21.15.64:0/2302762139 wait complete. 2023-09-27T23:50:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.710+0000 7f3bdad4b700 1 Processor -- start 2023-09-27T23:50:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.710+0000 7f3bdad4b700 1 -- start start 2023-09-27T23:50:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bd406a540 0x7f3bd4168f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 0x7f3bd40b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bd4158e70 0x7f3bd4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40b3b80 con 0x7f3bd40cc470 2023-09-27T23:50:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40b4db0 con 0x7f3bd406a540 2023-09-27T23:50:38.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40b4480 con 0x7f3bd4158e70 2023-09-27T23:50:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 0x7f3bd40b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 0x7f3bd40b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54562/0 (socket says 172.21.15.64:54562) 2023-09-27T23:50:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 -- 172.21.15.64:0/3184711720 learned_addr learned my addr 172.21.15.64:0/3184711720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9d49700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bd406a540 0x7f3bd4168f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bda54a700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bd4158e70 0x7f3bd4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 -- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bd4158e70 msgr2=0x7f3bd4159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bd4158e70 0x7f3bd4159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 -- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bd406a540 msgr2=0x7f3bd4168f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bd406a540 0x7f3bd4168f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 -- 172.21.15.64:0/3184711720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0009420 con 0x7f3bd40cc470 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9d49700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bd406a540 0x7f3bd4168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bd9548700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 0x7f3bd40b5240 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c810 tx=0x7f3bc4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4012630 con 0x7f3bd40cc470 2023-09-27T23:50:38.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc40127b0 con 0x7f3bd40cc470 2023-09-27T23:50:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd40b4760 con 0x7f3bd40cc470 2023-09-27T23:50:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4012b70 con 0x7f3bd40cc470 2023-09-27T23:50:38.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd4169480 con 0x7f3bd40cc470 2023-09-27T23:50:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.714+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd40b4f60 con 0x7f3bd40cc470 2023-09-27T23:50:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.718+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3bc401b400 con 0x7f3bd40cc470 2023-09-27T23:50:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.718+0000 7f3bc9ffb700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3bb8060a70 0x7f3bb8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.718+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3bc407ceb0 con 0x7f3bd40cc470 2023-09-27T23:50:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.718+0000 7f3bd9d49700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3bb8060a70 0x7f3bb8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:38.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.718+0000 7f3bd9d49700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3bb8060a70 0x7f3bb8062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00030c0 tx=0x7f3bd0017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:38.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.726+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3bd40b4f60 con 0x7f3bd40cc470 2023-09-27T23:50:38.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.886+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bd40b4f20 con 0x7f3bb8060a70 2023-09-27T23:50:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.902+0000 7f3bc9ffb700 1 -- 172.21.15.64:0/3184711720 <== mgr.8210 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308150 (secure 0 0 0) 0x7f3bd40b4f20 con 0x7f3bb8060a70 2023-09-27T23:50:38.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3bb8060a70 msgr2=0x7f3bb8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3bb8060a70 0x7f3bb8062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00030c0 tx=0x7f3bd0017000 comp rx=0 tx=0).stop 2023-09-27T23:50:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 msgr2=0x7f3bd40b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 0x7f3bd40b5240 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c810 tx=0x7f3bc4007f30 comp rx=0 tx=0).stop 2023-09-27T23:50:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 shutdown_connections 2023-09-27T23:50:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3bb8060a70 0x7f3bb8062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bd4158e70 0x7f3bd4159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bd40cc470 0x7f3bd40b5240 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 --2- 172.21.15.64:0/3184711720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bd406a540 0x7f3bd4168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 >> 172.21.15.64:0/3184711720 conn(0x7f3bd40670e0 msgr2=0x7f3bd406a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 shutdown_connections 2023-09-27T23:50:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:38.910+0000 7f3bdad4b700 1 -- 172.21.15.64:0/3184711720 wait complete. 2023-09-27T23:50:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:50:38.929 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:50:38.536994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":543,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266408,"kb_used_data":18152,"kb_used_omap":0,"kb_used_meta":248192,"kb_avail":754708312,"statfs":{"total":773094113280,"available":772821311488,"internally_reserved":0,"allocated":18587648,"data_stored":10303900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254148608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.393349"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567756+0000","last_change":"2023-09-27T23:50:34.559306+0000","last_active":"2023-09-27T23:50:34.567756+0000","last_peered":"2023-09-27T23:50:34.567756+0000","last_clean":"2023-09-27T23:50:34.567756+0000","last_became_active":"2023-09-27T23:50:34.557020+0000","last_became_peered":"2023-09-27T23:50:34.557020+0000","last_unstale":"2023-09-27T23:50:34.567756+0000","last_undegraded":"2023-09-27T23:50:34.567756+0000","last_fullsized":"2023-09-27T23:50:34.567756+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:01:54.141241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568612+0000","last_change":"2023-09-27T23:50:34.564749+0000","last_active":"2023-09-27T23:50:34.568612+0000","last_peered":"2023-09-27T23:50:34.568612+0000","last_clean":"2023-09-27T23:50:34.568612+0000","last_became_active":"2023-09-27T23:50:34.563936+0000","last_became_peered":"2023-09-27T23:50:34.563936+0000","last_unstale":"2023-09-27T23:50:34.568612+0000","last_undegraded":"2023-09-27T23:50:34.568612+0000","last_fullsized":"2023-09-27T23:50:34.568612+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:21:30.130546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565266+0000","last_change":"2023-09-27T23:50:34.559269+0000","last_active":"2023-09-27T23:50:34.565266+0000","last_peered":"2023-09-27T23:50:34.565266+0000","last_clean":"2023-09-27T23:50:34.565266+0000","last_became_active":"2023-09-27T23:50:34.558969+0000","last_became_peered":"2023-09-27T23:50:34.558969+0000","last_unstale":"2023-09-27T23:50:34.565266+0000","last_undegraded":"2023-09-27T23:50:34.565266+0000","last_fullsized":"2023-09-27T23:50:34.565266+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:42:30.339684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.555397+0000","last_change":"2023-09-27T23:50:34.548611+0000","last_active":"2023-09-27T23:50:34.555397+0000","last_peered":"2023-09-27T23:50:34.555397+0000","last_clean":"2023-09-27T23:50:34.555397+0000","last_became_active":"2023-09-27T23:50:34.548157+0000","last_became_peered":"2023-09-27T23:50:34.548157+0000","last_unstale":"2023-09-27T23:50:34.555397+0000","last_undegraded":"2023-09-27T23:50:34.555397+0000","last_fullsized":"2023-09-27T23:50:34.555397+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:50:01.524610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566787+0000","last_change":"2023-09-27T23:50:34.561451+0000","last_active":"2023-09-27T23:50:34.566787+0000","last_peered":"2023-09-27T23:50:34.566787+0000","last_clean":"2023-09-27T23:50:34.566787+0000","last_became_active":"2023-09-27T23:50:34.560978+0000","last_became_peered":"2023-09-27T23:50:34.560978+0000","last_unstale":"2023-09-27T23:50:34.566787+0000","last_undegraded":"2023-09-27T23:50:34.566787+0000","last_fullsized":"2023-09-27T23:50:34.566787+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:00.236964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557051+0000","last_change":"2023-09-27T23:50:34.549806+0000","last_active":"2023-09-27T23:50:34.557051+0000","last_peered":"2023-09-27T23:50:34.557051+0000","last_clean":"2023-09-27T23:50:34.557051+0000","last_became_active":"2023-09-27T23:50:34.549518+0000","last_became_peered":"2023-09-27T23:50:34.549518+0000","last_unstale":"2023-09-27T23:50:34.557051+0000","last_undegraded":"2023-09-27T23:50:34.557051+0000","last_fullsized":"2023-09-27T23:50:34.557051+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:14.120052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567301+0000","last_change":"2023-09-27T23:50:34.561892+0000","last_active":"2023-09-27T23:50:34.567301+0000","last_peered":"2023-09-27T23:50:34.567301+0000","last_clean":"2023-09-27T23:50:34.567301+0000","last_became_active":"2023-09-27T23:50:34.561632+0000","last_became_peered":"2023-09-27T23:50:34.561632+0000","last_unstale":"2023-09-27T23:50:34.567301+0000","last_undegraded":"2023-09-27T23:50:34.567301+0000","last_fullsized":"2023-09-27T23:50:34.567301+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:11:15.187943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568630+0000","last_change":"2023-09-27T23:50:34.564850+0000","last_active":"2023-09-27T23:50:34.568630+0000","last_peered":"2023-09-27T23:50:34.568630+0000","last_clean":"2023-09-27T23:50:34.568630+0000","last_became_active":"2023-09-27T23:50:34.563909+0000","last_became_peered":"2023-09-27T23:50:34.563909+0000","last_unstale":"2023-09-27T23:50:34.568630+0000","last_undegraded":"2023-09-27T23:50:34.568630+0000","last_fullsized":"2023-09-27T23:50:34.568630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:14.120052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568601+0000","last_change":"2023-09-27T23:50:34.564661+0000","last_active":"2023-09-27T23:50:34.568601+0000","last_peered":"2023-09-27T23:50:34.568601+0000","last_clean":"2023-09-27T23:50:34.568601+0000","last_became_active":"2023-09-27T23:50:34.564204+0000","last_became_peered":"2023-09-27T23:50:34.564204+0000","last_unstale":"2023-09-27T23:50:34.568601+0000","last_undegraded":"2023-09-27T23:50:34.568601+0000","last_fullsized":"2023-09-27T23:50:34.568601+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:51.292130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566560+0000","last_change":"2023-09-27T23:50:34.561035+0000","last_active":"2023-09-27T23:50:34.566560+0000","last_peered":"2023-09-27T23:50:34.566560+0000","last_clean":"2023-09-27T23:50:34.566560+0000","last_became_active":"2023-09-27T23:50:34.560706+0000","last_became_peered":"2023-09-27T23:50:34.560706+0000","last_unstale":"2023-09-27T23:50:34.566560+0000","last_undegraded":"2023-09-27T23:50:34.566560+0000","last_fullsized":"2023-09-27T23:50:34.566560+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:59.315891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566641+0000","last_change":"2023-09-27T23:50:34.561307+0000","last_active":"2023-09-27T23:50:34.566641+0000","last_peered":"2023-09-27T23:50:34.566641+0000","last_clean":"2023-09-27T23:50:34.566641+0000","last_became_active":"2023-09-27T23:50:34.560994+0000","last_became_peered":"2023-09-27T23:50:34.560994+0000","last_unstale":"2023-09-27T23:50:34.566641+0000","last_undegraded":"2023-09-27T23:50:34.566641+0000","last_fullsized":"2023-09-27T23:50:34.566641+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:05.704176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566725+0000","last_change":"2023-09-27T23:50:34.561839+0000","last_active":"2023-09-27T23:50:34.566725+0000","last_peered":"2023-09-27T23:50:34.566725+0000","last_clean":"2023-09-27T23:50:34.566725+0000","last_became_active":"2023-09-27T23:50:34.561533+0000","last_became_peered":"2023-09-27T23:50:34.561533+0000","last_unstale":"2023-09-27T23:50:34.566725+0000","last_undegraded":"2023-09-27T23:50:34.566725+0000","last_fullsized":"2023-09-27T23:50:34.566725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:38.874744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562389+0000","last_change":"2023-09-27T23:50:34.554162+0000","last_active":"2023-09-27T23:50:34.562389+0000","last_peered":"2023-09-27T23:50:34.562389+0000","last_clean":"2023-09-27T23:50:34.562389+0000","last_became_active":"2023-09-27T23:50:34.553738+0000","last_became_peered":"2023-09-27T23:50:34.553738+0000","last_unstale":"2023-09-27T23:50:34.562389+0000","last_undegraded":"2023-09-27T23:50:34.562389+0000","last_fullsized":"2023-09-27T23:50:34.562389+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:23.930966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562452+0000","last_change":"2023-09-27T23:50:34.554787+0000","last_active":"2023-09-27T23:50:34.562452+0000","last_peered":"2023-09-27T23:50:34.562452+0000","last_clean":"2023-09-27T23:50:34.562452+0000","last_became_active":"2023-09-27T23:50:34.554419+0000","last_became_peered":"2023-09-27T23:50:34.554419+0000","last_unstale":"2023-09-27T23:50:34.562452+0000","last_undegraded":"2023-09-27T23:50:34.562452+0000","last_fullsized":"2023-09-27T23:50:34.562452+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:46:29.884638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567128+0000","last_change":"2023-09-27T23:50:34.556680+0000","last_active":"2023-09-27T23:50:34.567128+0000","last_peered":"2023-09-27T23:50:34.567128+0000","last_clean":"2023-09-27T23:50:34.567128+0000","last_became_active":"2023-09-27T23:50:34.556372+0000","last_became_peered":"2023-09-27T23:50:34.556372+0000","last_unstale":"2023-09-27T23:50:34.567128+0000","last_undegraded":"2023-09-27T23:50:34.567128+0000","last_fullsized":"2023-09-27T23:50:34.567128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:33.637539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567347+0000","last_change":"2023-09-27T23:50:34.562541+0000","last_active":"2023-09-27T23:50:34.567347+0000","last_peered":"2023-09-27T23:50:34.567347+0000","last_clean":"2023-09-27T23:50:34.567347+0000","last_became_active":"2023-09-27T23:50:34.562186+0000","last_became_peered":"2023-09-27T23:50:34.562186+0000","last_unstale":"2023-09-27T23:50:34.567347+0000","last_undegraded":"2023-09-27T23:50:34.567347+0000","last_fullsized":"2023-09-27T23:50:34.567347+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:07.395190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557012+0000","last_change":"2023-09-27T23:50:34.548939+0000","last_active":"2023-09-27T23:50:34.557012+0000","last_peered":"2023-09-27T23:50:34.557012+0000","last_clean":"2023-09-27T23:50:34.557012+0000","last_became_active":"2023-09-27T23:50:34.548670+0000","last_became_peered":"2023-09-27T23:50:34.548670+0000","last_unstale":"2023-09-27T23:50:34.557012+0000","last_undegraded":"2023-09-27T23:50:34.557012+0000","last_fullsized":"2023-09-27T23:50:34.557012+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:38.874744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.555929+0000","last_change":"2023-09-27T23:50:34.553100+0000","last_active":"2023-09-27T23:50:34.555929+0000","last_peered":"2023-09-27T23:50:34.555929+0000","last_clean":"2023-09-27T23:50:34.555929+0000","last_became_active":"2023-09-27T23:50:34.552586+0000","last_became_peered":"2023-09-27T23:50:34.552586+0000","last_unstale":"2023-09-27T23:50:34.555929+0000","last_undegraded":"2023-09-27T23:50:34.555929+0000","last_fullsized":"2023-09-27T23:50:34.555929+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:22.875022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564471+0000","last_change":"2023-09-27T23:50:34.558021+0000","last_active":"2023-09-27T23:50:34.564471+0000","last_peered":"2023-09-27T23:50:34.564471+0000","last_clean":"2023-09-27T23:50:34.564471+0000","last_became_active":"2023-09-27T23:50:34.557537+0000","last_became_peered":"2023-09-27T23:50:34.557537+0000","last_unstale":"2023-09-27T23:50:34.564471+0000","last_undegraded":"2023-09-27T23:50:34.564471+0000","last_fullsized":"2023-09-27T23:50:34.564471+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:41:46.194689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562674+0000","last_change":"2023-09-27T23:50:34.554619+0000","last_active":"2023-09-27T23:50:34.562674+0000","last_peered":"2023-09-27T23:50:34.562674+0000","last_clean":"2023-09-27T23:50:34.562674+0000","last_became_active":"2023-09-27T23:50:34.554325+0000","last_became_peered":"2023-09-27T23:50:34.554325+0000","last_unstale":"2023-09-27T23:50:34.562674+0000","last_undegraded":"2023-09-27T23:50:34.562674+0000","last_fullsized":"2023-09-27T23:50:34.562674+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:31:08.484234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.559518+0000","last_change":"2023-09-27T23:50:34.553184+0000","last_active":"2023-09-27T23:50:34.559518+0000","last_peered":"2023-09-27T23:50:34.559518+0000","last_clean":"2023-09-27T23:50:34.559518+0000","last_became_active":"2023-09-27T23:50:34.552913+0000","last_became_peered":"2023-09-27T23:50:34.552913+0000","last_unstale":"2023-09-27T23:50:34.559518+0000","last_undegraded":"2023-09-27T23:50:34.559518+0000","last_fullsized":"2023-09-27T23:50:34.559518+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:51.292130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566621+0000","last_change":"2023-09-27T23:50:34.561306+0000","last_active":"2023-09-27T23:50:34.566621+0000","last_peered":"2023-09-27T23:50:34.566621+0000","last_clean":"2023-09-27T23:50:34.566621+0000","last_became_active":"2023-09-27T23:50:34.560926+0000","last_became_peered":"2023-09-27T23:50:34.560926+0000","last_unstale":"2023-09-27T23:50:34.566621+0000","last_undegraded":"2023-09-27T23:50:34.566621+0000","last_fullsized":"2023-09-27T23:50:34.566621+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:01:33.992859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565802+0000","last_change":"2023-09-27T23:50:34.560354+0000","last_active":"2023-09-27T23:50:34.565802+0000","last_peered":"2023-09-27T23:50:34.565802+0000","last_clean":"2023-09-27T23:50:34.565802+0000","last_became_active":"2023-09-27T23:50:34.560081+0000","last_became_peered":"2023-09-27T23:50:34.560081+0000","last_unstale":"2023-09-27T23:50:34.565802+0000","last_undegraded":"2023-09-27T23:50:34.565802+0000","last_fullsized":"2023-09-27T23:50:34.565802+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:59.315891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566821+0000","last_change":"2023-09-27T23:50:34.564305+0000","last_active":"2023-09-27T23:50:34.566821+0000","last_peered":"2023-09-27T23:50:34.566821+0000","last_clean":"2023-09-27T23:50:34.566821+0000","last_became_active":"2023-09-27T23:50:34.561802+0000","last_became_peered":"2023-09-27T23:50:34.561802+0000","last_unstale":"2023-09-27T23:50:34.566821+0000","last_undegraded":"2023-09-27T23:50:34.566821+0000","last_fullsized":"2023-09-27T23:50:34.566821+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:18:54.071865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565255+0000","last_change":"2023-09-27T23:50:34.559133+0000","last_active":"2023-09-27T23:50:34.565255+0000","last_peered":"2023-09-27T23:50:34.565255+0000","last_clean":"2023-09-27T23:50:34.565255+0000","last_became_active":"2023-09-27T23:50:34.558896+0000","last_became_peered":"2023-09-27T23:50:34.558896+0000","last_unstale":"2023-09-27T23:50:34.565255+0000","last_undegraded":"2023-09-27T23:50:34.565255+0000","last_fullsized":"2023-09-27T23:50:34.565255+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:24:03.170328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565171+0000","last_change":"2023-09-27T23:50:34.554988+0000","last_active":"2023-09-27T23:50:34.565171+0000","last_peered":"2023-09-27T23:50:34.565171+0000","last_clean":"2023-09-27T23:50:34.565171+0000","last_became_active":"2023-09-27T23:50:34.554709+0000","last_became_peered":"2023-09-27T23:50:34.554709+0000","last_unstale":"2023-09-27T23:50:34.565171+0000","last_undegraded":"2023-09-27T23:50:34.565171+0000","last_fullsized":"2023-09-27T23:50:34.565171+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:57:07.898196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562510+0000","last_change":"2023-09-27T23:50:34.555078+0000","last_active":"2023-09-27T23:50:34.562510+0000","last_peered":"2023-09-27T23:50:34.562510+0000","last_clean":"2023-09-27T23:50:34.562510+0000","last_became_active":"2023-09-27T23:50:34.554055+0000","last_became_peered":"2023-09-27T23:50:34.554055+0000","last_unstale":"2023-09-27T23:50:34.562510+0000","last_undegraded":"2023-09-27T23:50:34.562510+0000","last_fullsized":"2023-09-27T23:50:34.562510+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:12.959974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562592+0000","last_change":"2023-09-27T23:50:34.554232+0000","last_active":"2023-09-27T23:50:34.562592+0000","last_peered":"2023-09-27T23:50:34.562592+0000","last_clean":"2023-09-27T23:50:34.562592+0000","last_became_active":"2023-09-27T23:50:34.553835+0000","last_became_peered":"2023-09-27T23:50:34.553835+0000","last_unstale":"2023-09-27T23:50:34.562592+0000","last_undegraded":"2023-09-27T23:50:34.562592+0000","last_fullsized":"2023-09-27T23:50:34.562592+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:33:01.154675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565302+0000","last_change":"2023-09-27T23:50:34.557901+0000","last_active":"2023-09-27T23:50:34.565302+0000","last_peered":"2023-09-27T23:50:34.565302+0000","last_clean":"2023-09-27T23:50:34.565302+0000","last_became_active":"2023-09-27T23:50:34.557646+0000","last_became_peered":"2023-09-27T23:50:34.557646+0000","last_unstale":"2023-09-27T23:50:34.565302+0000","last_undegraded":"2023-09-27T23:50:34.565302+0000","last_fullsized":"2023-09-27T23:50:34.565302+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:34:12.347546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565903+0000","last_change":"2023-09-27T23:50:34.561125+0000","last_active":"2023-09-27T23:50:34.565903+0000","last_peered":"2023-09-27T23:50:34.565903+0000","last_clean":"2023-09-27T23:50:34.565903+0000","last_became_active":"2023-09-27T23:50:34.560816+0000","last_became_peered":"2023-09-27T23:50:34.560816+0000","last_unstale":"2023-09-27T23:50:34.565903+0000","last_undegraded":"2023-09-27T23:50:34.565903+0000","last_fullsized":"2023-09-27T23:50:34.565903+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:21:43.479240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.563597+0000","last_change":"2023-09-27T23:50:34.553110+0000","last_active":"2023-09-27T23:50:34.563597+0000","last_peered":"2023-09-27T23:50:34.563597+0000","last_clean":"2023-09-27T23:50:34.563597+0000","last_became_active":"2023-09-27T23:50:34.552708+0000","last_became_peered":"2023-09-27T23:50:34.552708+0000","last_unstale":"2023-09-27T23:50:34.563597+0000","last_undegraded":"2023-09-27T23:50:34.563597+0000","last_fullsized":"2023-09-27T23:50:34.563597+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:11:15.187943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.558417+0000","last_change":"2023-09-27T23:50:34.557307+0000","last_active":"2023-09-27T23:50:34.558417+0000","last_peered":"2023-09-27T23:50:34.558417+0000","last_clean":"2023-09-27T23:50:34.558417+0000","last_became_active":"2023-09-27T23:50:34.556999+0000","last_became_peered":"2023-09-27T23:50:34.556999+0000","last_unstale":"2023-09-27T23:50:34.558417+0000","last_undegraded":"2023-09-27T23:50:34.558417+0000","last_fullsized":"2023-09-27T23:50:34.558417+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:39:44.668021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565195+0000","last_change":"2023-09-27T23:50:34.555118+0000","last_active":"2023-09-27T23:50:34.565195+0000","last_peered":"2023-09-27T23:50:34.565195+0000","last_clean":"2023-09-27T23:50:34.565195+0000","last_became_active":"2023-09-27T23:50:34.554853+0000","last_became_peered":"2023-09-27T23:50:34.554853+0000","last_unstale":"2023-09-27T23:50:34.565195+0000","last_undegraded":"2023-09-27T23:50:34.565195+0000","last_fullsized":"2023-09-27T23:50:34.565195+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:29:59.791766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"173'4","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562882+0000","last_change":"2023-09-27T23:50:34.185949+0000","last_active":"2023-09-27T23:50:34.562882+0000","last_peered":"2023-09-27T23:50:34.562882+0000","last_clean":"2023-09-27T23:50:34.562882+0000","last_became_active":"2023-09-27T23:50:34.185633+0000","last_became_peered":"2023-09-27T23:50:34.185633+0000","last_unstale":"2023-09-27T23:50:34.562882+0000","last_undegraded":"2023-09-27T23:50:34.562882+0000","last_fullsized":"2023-09-27T23:50:34.562882+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:01:32.981121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565799+0000","last_change":"2023-09-27T23:50:34.559072+0000","last_active":"2023-09-27T23:50:34.565799+0000","last_peered":"2023-09-27T23:50:34.565799+0000","last_clean":"2023-09-27T23:50:34.565799+0000","last_became_active":"2023-09-27T23:50:34.558745+0000","last_became_peered":"2023-09-27T23:50:34.558745+0000","last_unstale":"2023-09-27T23:50:34.565799+0000","last_undegraded":"2023-09-27T23:50:34.565799+0000","last_fullsized":"2023-09-27T23:50:34.565799+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:32:35.695537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566688+0000","last_change":"2023-09-27T23:50:34.171899+0000","last_active":"2023-09-27T23:50:34.566688+0000","last_peered":"2023-09-27T23:50:34.566688+0000","last_clean":"2023-09-27T23:50:34.566688+0000","last_became_active":"2023-09-27T23:50:34.171630+0000","last_became_peered":"2023-09-27T23:50:34.171630+0000","last_unstale":"2023-09-27T23:50:34.566688+0000","last_undegraded":"2023-09-27T23:50:34.566688+0000","last_fullsized":"2023-09-27T23:50:34.566688+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:39.245022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.556951+0000","last_change":"2023-09-27T23:50:34.547941+0000","last_active":"2023-09-27T23:50:34.556951+0000","last_peered":"2023-09-27T23:50:34.556951+0000","last_clean":"2023-09-27T23:50:34.556951+0000","last_became_active":"2023-09-27T23:50:34.547658+0000","last_became_peered":"2023-09-27T23:50:34.547658+0000","last_unstale":"2023-09-27T23:50:34.556951+0000","last_undegraded":"2023-09-27T23:50:34.556951+0000","last_fullsized":"2023-09-27T23:50:34.556951+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:34.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.560146+0000","last_change":"2023-09-27T23:50:34.172268+0000","last_active":"2023-09-27T23:50:34.560146+0000","last_peered":"2023-09-27T23:50:34.560146+0000","last_clean":"2023-09-27T23:50:34.560146+0000","last_became_active":"2023-09-27T23:50:34.171691+0000","last_became_peered":"2023-09-27T23:50:34.171691+0000","last_unstale":"2023-09-27T23:50:34.560146+0000","last_undegraded":"2023-09-27T23:50:34.560146+0000","last_fullsized":"2023-09-27T23:50:34.560146+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:54:32.568632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562617+0000","last_change":"2023-09-27T23:50:34.556244+0000","last_active":"2023-09-27T23:50:34.562617+0000","last_peered":"2023-09-27T23:50:34.562617+0000","last_clean":"2023-09-27T23:50:34.562617+0000","last_became_active":"2023-09-27T23:50:34.555302+0000","last_became_peered":"2023-09-27T23:50:34.555302+0000","last_unstale":"2023-09-27T23:50:34.562617+0000","last_undegraded":"2023-09-27T23:50:34.562617+0000","last_fullsized":"2023-09-27T23:50:34.562617+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:07:49.146420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566343+0000","last_change":"2023-09-27T23:50:34.174024+0000","last_active":"2023-09-27T23:50:34.566343+0000","last_peered":"2023-09-27T23:50:34.566343+0000","last_clean":"2023-09-27T23:50:34.566343+0000","last_became_active":"2023-09-27T23:50:34.173678+0000","last_became_peered":"2023-09-27T23:50:34.173678+0000","last_unstale":"2023-09-27T23:50:34.566343+0000","last_undegraded":"2023-09-27T23:50:34.566343+0000","last_fullsized":"2023-09-27T23:50:34.566343+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:17.655398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.561417+0000","last_change":"2023-09-27T23:50:34.548919+0000","last_active":"2023-09-27T23:50:34.561417+0000","last_peered":"2023-09-27T23:50:34.561417+0000","last_clean":"2023-09-27T23:50:34.561417+0000","last_became_active":"2023-09-27T23:50:34.548568+0000","last_became_peered":"2023-09-27T23:50:34.548568+0000","last_unstale":"2023-09-27T23:50:34.561417+0000","last_undegraded":"2023-09-27T23:50:34.561417+0000","last_fullsized":"2023-09-27T23:50:34.561417+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:32:35.695537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.554327+0000","last_change":"2023-09-27T23:50:34.184176+0000","last_active":"2023-09-27T23:50:34.554327+0000","last_peered":"2023-09-27T23:50:34.554327+0000","last_clean":"2023-09-27T23:50:34.554327+0000","last_became_active":"2023-09-27T23:50:34.183738+0000","last_became_peered":"2023-09-27T23:50:34.183738+0000","last_unstale":"2023-09-27T23:50:34.554327+0000","last_undegraded":"2023-09-27T23:50:34.554327+0000","last_fullsized":"2023-09-27T23:50:34.554327+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:56:51.376598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.558743+0000","last_change":"2023-09-27T23:50:34.551680+0000","last_active":"2023-09-27T23:50:34.558743+0000","last_peered":"2023-09-27T23:50:34.558743+0000","last_clean":"2023-09-27T23:50:34.558743+0000","last_became_active":"2023-09-27T23:50:34.551461+0000","last_became_peered":"2023-09-27T23:50:34.551461+0000","last_unstale":"2023-09-27T23:50:34.558743+0000","last_undegraded":"2023-09-27T23:50:34.558743+0000","last_fullsized":"2023-09-27T23:50:34.558743+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:41:16.104461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.544066+0000","last_change":"2023-09-27T23:50:34.172167+0000","last_active":"2023-09-27T23:50:34.544066+0000","last_peered":"2023-09-27T23:50:34.544066+0000","last_clean":"2023-09-27T23:50:34.544066+0000","last_became_active":"2023-09-27T23:50:34.171847+0000","last_became_peered":"2023-09-27T23:50:34.171847+0000","last_unstale":"2023-09-27T23:50:34.544066+0000","last_undegraded":"2023-09-27T23:50:34.544066+0000","last_fullsized":"2023-09-27T23:50:34.544066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:38.120967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562007+0000","last_change":"2023-09-27T23:50:34.552662+0000","last_active":"2023-09-27T23:50:34.562007+0000","last_peered":"2023-09-27T23:50:34.562007+0000","last_clean":"2023-09-27T23:50:34.562007+0000","last_became_active":"2023-09-27T23:50:34.552380+0000","last_became_peered":"2023-09-27T23:50:34.552380+0000","last_unstale":"2023-09-27T23:50:34.562007+0000","last_undegraded":"2023-09-27T23:50:34.562007+0000","last_fullsized":"2023-09-27T23:50:34.562007+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:53.905465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.546907+0000","last_change":"2023-09-27T23:50:34.170798+0000","last_active":"2023-09-27T23:50:34.546907+0000","last_peered":"2023-09-27T23:50:34.546907+0000","last_clean":"2023-09-27T23:50:34.546907+0000","last_became_active":"2023-09-27T23:50:34.170528+0000","last_became_peered":"2023-09-27T23:50:34.170528+0000","last_unstale":"2023-09-27T23:50:34.546907+0000","last_undegraded":"2023-09-27T23:50:34.546907+0000","last_fullsized":"2023-09-27T23:50:34.546907+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:13.637097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557727+0000","last_change":"2023-09-27T23:50:34.550712+0000","last_active":"2023-09-27T23:50:34.557727+0000","last_peered":"2023-09-27T23:50:34.557727+0000","last_clean":"2023-09-27T23:50:34.557727+0000","last_became_active":"2023-09-27T23:50:34.550415+0000","last_became_peered":"2023-09-27T23:50:34.550415+0000","last_unstale":"2023-09-27T23:50:34.557727+0000","last_undegraded":"2023-09-27T23:50:34.557727+0000","last_fullsized":"2023-09-27T23:50:34.557727+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:46:29.884638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.544516+0000","last_change":"2023-09-27T23:50:34.171801+0000","last_active":"2023-09-27T23:50:34.544516+0000","last_peered":"2023-09-27T23:50:34.544516+0000","last_clean":"2023-09-27T23:50:34.544516+0000","last_became_active":"2023-09-27T23:50:34.171494+0000","last_became_peered":"2023-09-27T23:50:34.171494+0000","last_unstale":"2023-09-27T23:50:34.544516+0000","last_undegraded":"2023-09-27T23:50:34.544516+0000","last_fullsized":"2023-09-27T23:50:34.544516+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:28:23.297242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562895+0000","last_change":"2023-09-27T23:50:34.557380+0000","last_active":"2023-09-27T23:50:34.562895+0000","last_peered":"2023-09-27T23:50:34.562895+0000","last_clean":"2023-09-27T23:50:34.562895+0000","last_became_active":"2023-09-27T23:50:34.557154+0000","last_became_peered":"2023-09-27T23:50:34.557154+0000","last_unstale":"2023-09-27T23:50:34.562895+0000","last_undegraded":"2023-09-27T23:50:34.562895+0000","last_fullsized":"2023-09-27T23:50:34.562895+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:43:59.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.561545+0000","last_change":"2023-09-27T23:50:34.183896+0000","last_active":"2023-09-27T23:50:34.561545+0000","last_peered":"2023-09-27T23:50:34.561545+0000","last_clean":"2023-09-27T23:50:34.561545+0000","last_became_active":"2023-09-27T23:50:34.183535+0000","last_became_peered":"2023-09-27T23:50:34.183535+0000","last_unstale":"2023-09-27T23:50:34.561545+0000","last_undegraded":"2023-09-27T23:50:34.561545+0000","last_fullsized":"2023-09-27T23:50:34.561545+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:29:52.638295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.554015+0000","last_change":"2023-09-27T23:50:34.551237+0000","last_active":"2023-09-27T23:50:34.554015+0000","last_peered":"2023-09-27T23:50:34.554015+0000","last_clean":"2023-09-27T23:50:34.554015+0000","last_became_active":"2023-09-27T23:50:34.550972+0000","last_became_peered":"2023-09-27T23:50:34.550972+0000","last_unstale":"2023-09-27T23:50:34.554015+0000","last_undegraded":"2023-09-27T23:50:34.554015+0000","last_fullsized":"2023-09-27T23:50:34.554015+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:27:11.741782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.554454+0000","last_change":"2023-09-27T23:50:34.184001+0000","last_active":"2023-09-27T23:50:34.554454+0000","last_peered":"2023-09-27T23:50:34.554454+0000","last_clean":"2023-09-27T23:50:34.554454+0000","last_became_active":"2023-09-27T23:50:34.183650+0000","last_became_peered":"2023-09-27T23:50:34.183650+0000","last_unstale":"2023-09-27T23:50:34.554454+0000","last_undegraded":"2023-09-27T23:50:34.554454+0000","last_fullsized":"2023-09-27T23:50:34.554454+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:07:59.145229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562567+0000","last_change":"2023-09-27T23:50:34.554086+0000","last_active":"2023-09-27T23:50:34.562567+0000","last_peered":"2023-09-27T23:50:34.562567+0000","last_clean":"2023-09-27T23:50:34.562567+0000","last_became_active":"2023-09-27T23:50:34.552464+0000","last_became_peered":"2023-09-27T23:50:34.552464+0000","last_unstale":"2023-09-27T23:50:34.562567+0000","last_undegraded":"2023-09-27T23:50:34.562567+0000","last_fullsized":"2023-09-27T23:50:34.562567+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:43:43.152499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.559124+0000","last_change":"2023-09-27T23:50:34.178593+0000","last_active":"2023-09-27T23:50:34.559124+0000","last_peered":"2023-09-27T23:50:34.559124+0000","last_clean":"2023-09-27T23:50:34.559124+0000","last_became_active":"2023-09-27T23:50:34.178320+0000","last_became_peered":"2023-09-27T23:50:34.178320+0000","last_unstale":"2023-09-27T23:50:34.559124+0000","last_undegraded":"2023-09-27T23:50:34.559124+0000","last_fullsized":"2023-09-27T23:50:34.559124+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:46.192596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567284+0000","last_change":"2023-09-27T23:50:34.561852+0000","last_active":"2023-09-27T23:50:34.567284+0000","last_peered":"2023-09-27T23:50:34.567284+0000","last_clean":"2023-09-27T23:50:34.567284+0000","last_became_active":"2023-09-27T23:50:34.561567+0000","last_became_peered":"2023-09-27T23:50:34.561567+0000","last_unstale":"2023-09-27T23:50:34.567284+0000","last_undegraded":"2023-09-27T23:50:34.567284+0000","last_fullsized":"2023-09-27T23:50:34.567284+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:02:04.216295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.544712+0000","last_change":"2023-09-27T23:50:34.173943+0000","last_active":"2023-09-27T23:50:34.544712+0000","last_peered":"2023-09-27T23:50:34.544712+0000","last_clean":"2023-09-27T23:50:34.544712+0000","last_became_active":"2023-09-27T23:50:34.173672+0000","last_became_peered":"2023-09-27T23:50:34.173672+0000","last_unstale":"2023-09-27T23:50:34.544712+0000","last_undegraded":"2023-09-27T23:50:34.544712+0000","last_fullsized":"2023-09-27T23:50:34.544712+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:21:29.118808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562331+0000","last_change":"2023-09-27T23:50:34.550713+0000","last_active":"2023-09-27T23:50:34.562331+0000","last_peered":"2023-09-27T23:50:34.562331+0000","last_clean":"2023-09-27T23:50:34.562331+0000","last_became_active":"2023-09-27T23:50:34.550350+0000","last_became_peered":"2023-09-27T23:50:34.550350+0000","last_unstale":"2023-09-27T23:50:34.562331+0000","last_undegraded":"2023-09-27T23:50:34.562331+0000","last_fullsized":"2023-09-27T23:50:34.562331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.190147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565130+0000","last_change":"2023-09-27T23:50:34.176646+0000","last_active":"2023-09-27T23:50:34.565130+0000","last_peered":"2023-09-27T23:50:34.565130+0000","last_clean":"2023-09-27T23:50:34.565130+0000","last_became_active":"2023-09-27T23:50:34.175097+0000","last_became_peered":"2023-09-27T23:50:34.175097+0000","last_unstale":"2023-09-27T23:50:34.565130+0000","last_undegraded":"2023-09-27T23:50:34.565130+0000","last_fullsized":"2023-09-27T23:50:34.565130+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:20:31.754486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564856+0000","last_change":"2023-09-27T23:50:34.553653+0000","last_active":"2023-09-27T23:50:34.564856+0000","last_peered":"2023-09-27T23:50:34.564856+0000","last_clean":"2023-09-27T23:50:34.564856+0000","last_became_active":"2023-09-27T23:50:34.553141+0000","last_became_peered":"2023-09-27T23:50:34.553141+0000","last_unstale":"2023-09-27T23:50:34.564856+0000","last_undegraded":"2023-09-27T23:50:34.564856+0000","last_fullsized":"2023-09-27T23:50:34.564856+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:33:00.188060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542595+0000","last_change":"2023-09-27T23:50:34.170080+0000","last_active":"2023-09-27T23:50:34.542595+0000","last_peered":"2023-09-27T23:50:34.542595+0000","last_clean":"2023-09-27T23:50:34.542595+0000","last_became_active":"2023-09-27T23:50:34.169718+0000","last_became_peered":"2023-09-27T23:50:34.169718+0000","last_unstale":"2023-09-27T23:50:34.542595+0000","last_undegraded":"2023-09-27T23:50:34.542595+0000","last_fullsized":"2023-09-27T23:50:34.542595+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:18:53.060128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.569732+0000","last_change":"2023-09-27T23:50:34.567762+0000","last_active":"2023-09-27T23:50:34.569732+0000","last_peered":"2023-09-27T23:50:34.569732+0000","last_clean":"2023-09-27T23:50:34.569732+0000","last_became_active":"2023-09-27T23:50:34.567433+0000","last_became_peered":"2023-09-27T23:50:34.567433+0000","last_unstale":"2023-09-27T23:50:34.569732+0000","last_undegraded":"2023-09-27T23:50:34.569732+0000","last_fullsized":"2023-09-27T23:50:34.569732+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:29:13.632950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.536248+0000","last_change":"2023-09-27T23:50:34.182887+0000","last_active":"2023-09-27T23:50:34.536248+0000","last_peered":"2023-09-27T23:50:34.536248+0000","last_clean":"2023-09-27T23:50:34.536248+0000","last_became_active":"2023-09-27T23:50:34.182514+0000","last_became_peered":"2023-09-27T23:50:34.182514+0000","last_unstale":"2023-09-27T23:50:34.536248+0000","last_undegraded":"2023-09-27T23:50:34.536248+0000","last_fullsized":"2023-09-27T23:50:34.536248+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:16:45.217748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565822+0000","last_change":"2023-09-27T23:50:34.560465+0000","last_active":"2023-09-27T23:50:34.565822+0000","last_peered":"2023-09-27T23:50:34.565822+0000","last_clean":"2023-09-27T23:50:34.565822+0000","last_became_active":"2023-09-27T23:50:34.560146+0000","last_became_peered":"2023-09-27T23:50:34.560146+0000","last_unstale":"2023-09-27T23:50:34.565822+0000","last_undegraded":"2023-09-27T23:50:34.565822+0000","last_fullsized":"2023-09-27T23:50:34.565822+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:00.236964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542213+0000","last_change":"2023-09-27T23:50:34.169977+0000","last_active":"2023-09-27T23:50:34.542213+0000","last_peered":"2023-09-27T23:50:34.542213+0000","last_clean":"2023-09-27T23:50:34.542213+0000","last_became_active":"2023-09-27T23:50:34.169371+0000","last_became_peered":"2023-09-27T23:50:34.169371+0000","last_unstale":"2023-09-27T23:50:34.542213+0000","last_undegraded":"2023-09-27T23:50:34.542213+0000","last_fullsized":"2023-09-27T23:50:34.542213+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:11:56.481823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.552038+0000","last_change":"2023-09-27T23:50:34.548580+0000","last_active":"2023-09-27T23:50:34.552038+0000","last_peered":"2023-09-27T23:50:34.552038+0000","last_clean":"2023-09-27T23:50:34.552038+0000","last_became_active":"2023-09-27T23:50:34.548257+0000","last_became_peered":"2023-09-27T23:50:34.548257+0000","last_unstale":"2023-09-27T23:50:34.552038+0000","last_undegraded":"2023-09-27T23:50:34.552038+0000","last_fullsized":"2023-09-27T23:50:34.552038+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:01:54.141241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.560217+0000","last_change":"2023-09-27T23:50:34.173122+0000","last_active":"2023-09-27T23:50:34.560217+0000","last_peered":"2023-09-27T23:50:34.560217+0000","last_clean":"2023-09-27T23:50:34.560217+0000","last_became_active":"2023-09-27T23:50:34.172766+0000","last_became_peered":"2023-09-27T23:50:34.172766+0000","last_unstale":"2023-09-27T23:50:34.560217+0000","last_undegraded":"2023-09-27T23:50:34.560217+0000","last_fullsized":"2023-09-27T23:50:34.560217+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:50.280393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567332+0000","last_change":"2023-09-27T23:50:34.561897+0000","last_active":"2023-09-27T23:50:34.567332+0000","last_peered":"2023-09-27T23:50:34.567332+0000","last_clean":"2023-09-27T23:50:34.567332+0000","last_became_active":"2023-09-27T23:50:34.561596+0000","last_became_peered":"2023-09-27T23:50:34.561596+0000","last_unstale":"2023-09-27T23:50:34.567332+0000","last_undegraded":"2023-09-27T23:50:34.567332+0000","last_fullsized":"2023-09-27T23:50:34.567332+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:21.898058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.539051+0000","last_change":"2023-09-27T23:50:34.183999+0000","last_active":"2023-09-27T23:50:34.539051+0000","last_peered":"2023-09-27T23:50:34.539051+0000","last_clean":"2023-09-27T23:50:34.539051+0000","last_became_active":"2023-09-27T23:50:34.183618+0000","last_became_peered":"2023-09-27T23:50:34.183618+0000","last_unstale":"2023-09-27T23:50:34.539051+0000","last_undegraded":"2023-09-27T23:50:34.539051+0000","last_fullsized":"2023-09-27T23:50:34.539051+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:54:47.011915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557303+0000","last_change":"2023-09-27T23:50:34.550011+0000","last_active":"2023-09-27T23:50:34.557303+0000","last_peered":"2023-09-27T23:50:34.557303+0000","last_clean":"2023-09-27T23:50:34.557303+0000","last_became_active":"2023-09-27T23:50:34.549815+0000","last_became_peered":"2023-09-27T23:50:34.549815+0000","last_unstale":"2023-09-27T23:50:34.557303+0000","last_undegraded":"2023-09-27T23:50:34.557303+0000","last_fullsized":"2023-09-27T23:50:34.557303+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:05.704176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.541359+0000","last_change":"2023-09-27T23:50:34.173955+0000","last_active":"2023-09-27T23:50:34.541359+0000","last_peered":"2023-09-27T23:50:34.541359+0000","last_clean":"2023-09-27T23:50:34.541359+0000","last_became_active":"2023-09-27T23:50:34.173650+0000","last_became_peered":"2023-09-27T23:50:34.173650+0000","last_unstale":"2023-09-27T23:50:34.541359+0000","last_undegraded":"2023-09-27T23:50:34.541359+0000","last_fullsized":"2023-09-27T23:50:34.541359+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:22.919228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.549090+0000","last_change":"2023-09-27T23:50:34.545479+0000","last_active":"2023-09-27T23:50:34.549090+0000","last_peered":"2023-09-27T23:50:34.549090+0000","last_clean":"2023-09-27T23:50:34.549090+0000","last_became_active":"2023-09-27T23:50:34.545208+0000","last_became_peered":"2023-09-27T23:50:34.545208+0000","last_unstale":"2023-09-27T23:50:34.549090+0000","last_undegraded":"2023-09-27T23:50:34.549090+0000","last_fullsized":"2023-09-27T23:50:34.549090+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:27.462843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.539125+0000","last_change":"2023-09-27T23:50:34.182805+0000","last_active":"2023-09-27T23:50:34.539125+0000","last_peered":"2023-09-27T23:50:34.539125+0000","last_clean":"2023-09-27T23:50:34.539125+0000","last_became_active":"2023-09-27T23:50:34.182453+0000","last_became_peered":"2023-09-27T23:50:34.182453+0000","last_unstale":"2023-09-27T23:50:34.539125+0000","last_undegraded":"2023-09-27T23:50:34.539125+0000","last_fullsized":"2023-09-27T23:50:34.539125+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:13.108314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.555923+0000","last_change":"2023-09-27T23:50:34.552924+0000","last_active":"2023-09-27T23:50:34.555923+0000","last_peered":"2023-09-27T23:50:34.555923+0000","last_clean":"2023-09-27T23:50:34.555923+0000","last_became_active":"2023-09-27T23:50:34.552657+0000","last_became_peered":"2023-09-27T23:50:34.552657+0000","last_unstale":"2023-09-27T23:50:34.555923+0000","last_undegraded":"2023-09-27T23:50:34.555923+0000","last_fullsized":"2023-09-27T23:50:34.555923+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:38:24.228982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.547450+0000","last_change":"2023-09-27T23:50:34.173743+0000","last_active":"2023-09-27T23:50:34.547450+0000","last_peered":"2023-09-27T23:50:34.547450+0000","last_clean":"2023-09-27T23:50:34.547450+0000","last_became_active":"2023-09-27T23:50:34.173439+0000","last_became_peered":"2023-09-27T23:50:34.173439+0000","last_unstale":"2023-09-27T23:50:34.547450+0000","last_undegraded":"2023-09-27T23:50:34.547450+0000","last_fullsized":"2023-09-27T23:50:34.547450+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:35:49.393660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.563657+0000","last_change":"2023-09-27T23:50:34.552914+0000","last_active":"2023-09-27T23:50:34.563657+0000","last_peered":"2023-09-27T23:50:34.563657+0000","last_clean":"2023-09-27T23:50:34.563657+0000","last_became_active":"2023-09-27T23:50:34.552670+0000","last_became_peered":"2023-09-27T23:50:34.552670+0000","last_unstale":"2023-09-27T23:50:34.563657+0000","last_undegraded":"2023-09-27T23:50:34.563657+0000","last_fullsized":"2023-09-27T23:50:34.563657+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:07.395190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.535326+0000","last_change":"2023-09-27T23:50:34.167715+0000","last_active":"2023-09-27T23:50:34.535326+0000","last_peered":"2023-09-27T23:50:34.535326+0000","last_clean":"2023-09-27T23:50:34.535326+0000","last_became_active":"2023-09-27T23:50:34.167114+0000","last_became_peered":"2023-09-27T23:50:34.167114+0000","last_unstale":"2023-09-27T23:50:34.535326+0000","last_undegraded":"2023-09-27T23:50:34.535326+0000","last_fullsized":"2023-09-27T23:50:34.535326+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:59:40.239632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564722+0000","last_change":"2023-09-27T23:50:34.552691+0000","last_active":"2023-09-27T23:50:34.564722+0000","last_peered":"2023-09-27T23:50:34.564722+0000","last_clean":"2023-09-27T23:50:34.564722+0000","last_became_active":"2023-09-27T23:50:34.552424+0000","last_became_peered":"2023-09-27T23:50:34.552424+0000","last_unstale":"2023-09-27T23:50:34.564722+0000","last_undegraded":"2023-09-27T23:50:34.564722+0000","last_fullsized":"2023-09-27T23:50:34.564722+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:21:43.479240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.547020+0000","last_change":"2023-09-27T23:50:34.170730+0000","last_active":"2023-09-27T23:50:34.547020+0000","last_peered":"2023-09-27T23:50:34.547020+0000","last_clean":"2023-09-27T23:50:34.547020+0000","last_became_active":"2023-09-27T23:50:34.170447+0000","last_became_peered":"2023-09-27T23:50:34.170447+0000","last_unstale":"2023-09-27T23:50:34.547020+0000","last_undegraded":"2023-09-27T23:50:34.547020+0000","last_fullsized":"2023-09-27T23:50:34.547020+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:07:48.134682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565337+0000","last_change":"2023-09-27T23:50:34.560112+0000","last_active":"2023-09-27T23:50:34.565337+0000","last_peered":"2023-09-27T23:50:34.565337+0000","last_clean":"2023-09-27T23:50:34.565337+0000","last_became_active":"2023-09-27T23:50:34.559812+0000","last_became_peered":"2023-09-27T23:50:34.559812+0000","last_unstale":"2023-09-27T23:50:34.565337+0000","last_undegraded":"2023-09-27T23:50:34.565337+0000","last_fullsized":"2023-09-27T23:50:34.565337+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:14.648835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.549412+0000","last_change":"2023-09-27T23:50:34.174716+0000","last_active":"2023-09-27T23:50:34.549412+0000","last_peered":"2023-09-27T23:50:34.549412+0000","last_clean":"2023-09-27T23:50:34.549412+0000","last_became_active":"2023-09-27T23:50:34.174450+0000","last_became_peered":"2023-09-27T23:50:34.174450+0000","last_unstale":"2023-09-27T23:50:34.549412+0000","last_undegraded":"2023-09-27T23:50:34.549412+0000","last_fullsized":"2023-09-27T23:50:34.549412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:32:56.183670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.552060+0000","last_change":"2023-09-27T23:50:34.548838+0000","last_active":"2023-09-27T23:50:34.552060+0000","last_peered":"2023-09-27T23:50:34.552060+0000","last_clean":"2023-09-27T23:50:34.552060+0000","last_became_active":"2023-09-27T23:50:34.548290+0000","last_became_peered":"2023-09-27T23:50:34.548290+0000","last_unstale":"2023-09-27T23:50:34.552060+0000","last_undegraded":"2023-09-27T23:50:34.552060+0000","last_fullsized":"2023-09-27T23:50:34.552060+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:57:49.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.548651+0000","last_change":"2023-09-27T23:50:34.169552+0000","last_active":"2023-09-27T23:50:34.548651+0000","last_peered":"2023-09-27T23:50:34.548651+0000","last_clean":"2023-09-27T23:50:34.548651+0000","last_became_active":"2023-09-27T23:50:34.169229+0000","last_became_peered":"2023-09-27T23:50:34.169229+0000","last_unstale":"2023-09-27T23:50:34.548651+0000","last_undegraded":"2023-09-27T23:50:34.548651+0000","last_fullsized":"2023-09-27T23:50:34.548651+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:07:39.032105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"162'304","reported_seq":960,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542422+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:50:34.542422+0000","last_peered":"2023-09-27T23:50:34.542422+0000","last_clean":"2023-09-27T23:50:34.542422+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:50:34.542422+0000","last_undegraded":"2023-09-27T23:50:34.542422+0000","last_fullsized":"2023-09-27T23:50:34.542422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":304,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.538792+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:50:34.538792+0000","last_peered":"2023-09-27T23:50:34.538792+0000","last_clean":"2023-09-27T23:50:34.538792+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:50:34.538792+0000","last_undegraded":"2023-09-27T23:50:34.538792+0000","last_fullsized":"2023-09-27T23:50:34.538792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.563638+0000","last_change":"2023-09-27T23:50:34.555347+0000","last_active":"2023-09-27T23:50:34.563638+0000","last_peered":"2023-09-27T23:50:34.563638+0000","last_clean":"2023-09-27T23:50:34.563638+0000","last_became_active":"2023-09-27T23:50:34.555039+0000","last_became_peered":"2023-09-27T23:50:34.555039+0000","last_unstale":"2023-09-27T23:50:34.563638+0000","last_undegraded":"2023-09-27T23:50:34.563638+0000","last_fullsized":"2023-09-27T23:50:34.563638+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:02:04.216295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"173'6","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.570300+0000","last_change":"2023-09-27T23:50:34.172631+0000","last_active":"2023-09-27T23:50:34.570300+0000","last_peered":"2023-09-27T23:50:34.570300+0000","last_clean":"2023-09-27T23:50:34.570300+0000","last_became_active":"2023-09-27T23:50:34.172362+0000","last_became_peered":"2023-09-27T23:50:34.172362+0000","last_unstale":"2023-09-27T23:50:34.570300+0000","last_undegraded":"2023-09-27T23:50:34.570300+0000","last_fullsized":"2023-09-27T23:50:34.570300+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:18:25.363148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542322+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:50:34.542322+0000","last_peered":"2023-09-27T23:50:34.542322+0000","last_clean":"2023-09-27T23:50:34.542322+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:50:34.542322+0000","last_undegraded":"2023-09-27T23:50:34.542322+0000","last_fullsized":"2023-09-27T23:50:34.542322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557740+0000","last_change":"2023-09-27T23:50:34.555682+0000","last_active":"2023-09-27T23:50:34.557740+0000","last_peered":"2023-09-27T23:50:34.557740+0000","last_clean":"2023-09-27T23:50:34.557740+0000","last_became_active":"2023-09-27T23:50:34.555243+0000","last_became_peered":"2023-09-27T23:50:34.555243+0000","last_unstale":"2023-09-27T23:50:34.557740+0000","last_undegraded":"2023-09-27T23:50:34.557740+0000","last_fullsized":"2023-09-27T23:50:34.557740+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:33.637539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"173'3","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562306+0000","last_change":"2023-09-27T23:50:34.164815+0000","last_active":"2023-09-27T23:50:34.562306+0000","last_peered":"2023-09-27T23:50:34.562306+0000","last_clean":"2023-09-27T23:50:34.562306+0000","last_became_active":"2023-09-27T23:50:34.164485+0000","last_became_peered":"2023-09-27T23:50:34.164485+0000","last_unstale":"2023-09-27T23:50:34.562306+0000","last_undegraded":"2023-09-27T23:50:34.562306+0000","last_fullsized":"2023-09-27T23:50:34.562306+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:10.178409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.530465+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:50:34.530465+0000","last_peered":"2023-09-27T23:50:34.530465+0000","last_clean":"2023-09-27T23:50:34.530465+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:50:34.530465+0000","last_undegraded":"2023-09-27T23:50:34.530465+0000","last_fullsized":"2023-09-27T23:50:34.530465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.552089+0000","last_change":"2023-09-27T23:50:34.547700+0000","last_active":"2023-09-27T23:50:34.552089+0000","last_peered":"2023-09-27T23:50:34.552089+0000","last_clean":"2023-09-27T23:50:34.552089+0000","last_became_active":"2023-09-27T23:50:34.547437+0000","last_became_peered":"2023-09-27T23:50:34.547437+0000","last_unstale":"2023-09-27T23:50:34.552089+0000","last_undegraded":"2023-09-27T23:50:34.552089+0000","last_fullsized":"2023-09-27T23:50:34.552089+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:29:13.632950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.530828+0000","last_change":"2023-09-27T23:50:34.183336+0000","last_active":"2023-09-27T23:50:34.530828+0000","last_peered":"2023-09-27T23:50:34.530828+0000","last_clean":"2023-09-27T23:50:34.530828+0000","last_became_active":"2023-09-27T23:50:34.182941+0000","last_became_peered":"2023-09-27T23:50:34.182941+0000","last_unstale":"2023-09-27T23:50:34.530828+0000","last_undegraded":"2023-09-27T23:50:34.530828+0000","last_fullsized":"2023-09-27T23:50:34.530828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:00.466792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.538946+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:50:34.538946+0000","last_peered":"2023-09-27T23:50:34.538946+0000","last_clean":"2023-09-27T23:50:34.538946+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:50:34.538946+0000","last_undegraded":"2023-09-27T23:50:34.538946+0000","last_fullsized":"2023-09-27T23:50:34.538946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562397+0000","last_change":"2023-09-27T23:50:34.550653+0000","last_active":"2023-09-27T23:50:34.562397+0000","last_peered":"2023-09-27T23:50:34.562397+0000","last_clean":"2023-09-27T23:50:34.562397+0000","last_became_active":"2023-09-27T23:50:34.550290+0000","last_became_peered":"2023-09-27T23:50:34.550290+0000","last_unstale":"2023-09-27T23:50:34.562397+0000","last_undegraded":"2023-09-27T23:50:34.562397+0000","last_fullsized":"2023-09-27T23:50:34.562397+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:07:18.419504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"173'8","reported_seq":18,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568003+0000","last_change":"2023-09-27T23:50:34.176694+0000","last_active":"2023-09-27T23:50:34.568003+0000","last_peered":"2023-09-27T23:50:34.568003+0000","last_clean":"2023-09-27T23:50:34.568003+0000","last_became_active":"2023-09-27T23:50:34.176430+0000","last_became_peered":"2023-09-27T23:50:34.176430+0000","last_unstale":"2023-09-27T23:50:34.568003+0000","last_undegraded":"2023-09-27T23:50:34.568003+0000","last_fullsized":"2023-09-27T23:50:34.568003+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:29.373167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.539964+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:50:34.539964+0000","last_peered":"2023-09-27T23:50:34.539964+0000","last_clean":"2023-09-27T23:50:34.539964+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:50:34.539964+0000","last_undegraded":"2023-09-27T23:50:34.539964+0000","last_fullsized":"2023-09-27T23:50:34.539964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562598+0000","last_change":"2023-09-27T23:50:34.553645+0000","last_active":"2023-09-27T23:50:34.562598+0000","last_peered":"2023-09-27T23:50:34.562598+0000","last_clean":"2023-09-27T23:50:34.562598+0000","last_became_active":"2023-09-27T23:50:34.553380+0000","last_became_peered":"2023-09-27T23:50:34.553380+0000","last_unstale":"2023-09-27T23:50:34.562598+0000","last_undegraded":"2023-09-27T23:50:34.562598+0000","last_fullsized":"2023-09-27T23:50:34.562598+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:53:07.240729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.536137+0000","last_change":"2023-09-27T23:50:34.172169+0000","last_active":"2023-09-27T23:50:34.536137+0000","last_peered":"2023-09-27T23:50:34.536137+0000","last_clean":"2023-09-27T23:50:34.536137+0000","last_became_active":"2023-09-27T23:50:34.171760+0000","last_became_peered":"2023-09-27T23:50:34.171760+0000","last_unstale":"2023-09-27T23:50:34.536137+0000","last_undegraded":"2023-09-27T23:50:34.536137+0000","last_fullsized":"2023-09-27T23:50:34.536137+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:28:53.536449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542926+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:50:34.542926+0000","last_peered":"2023-09-27T23:50:34.542926+0000","last_clean":"2023-09-27T23:50:34.542926+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:50:34.542926+0000","last_undegraded":"2023-09-27T23:50:34.542926+0000","last_fullsized":"2023-09-27T23:50:34.542926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564875+0000","last_change":"2023-09-27T23:50:34.554195+0000","last_active":"2023-09-27T23:50:34.564875+0000","last_peered":"2023-09-27T23:50:34.564875+0000","last_clean":"2023-09-27T23:50:34.564875+0000","last_became_active":"2023-09-27T23:50:34.553917+0000","last_became_peered":"2023-09-27T23:50:34.553917+0000","last_unstale":"2023-09-27T23:50:34.564875+0000","last_undegraded":"2023-09-27T23:50:34.564875+0000","last_fullsized":"2023-09-27T23:50:34.564875+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:22:35.188618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.538483+0000","last_change":"2023-09-27T23:50:34.173869+0000","last_active":"2023-09-27T23:50:34.538483+0000","last_peered":"2023-09-27T23:50:34.538483+0000","last_clean":"2023-09-27T23:50:34.538483+0000","last_became_active":"2023-09-27T23:50:34.173576+0000","last_became_peered":"2023-09-27T23:50:34.173576+0000","last_unstale":"2023-09-27T23:50:34.538483+0000","last_undegraded":"2023-09-27T23:50:34.538483+0000","last_fullsized":"2023-09-27T23:50:34.538483+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:37.863006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.541006+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:50:34.541006+0000","last_peered":"2023-09-27T23:50:34.541006+0000","last_clean":"2023-09-27T23:50:34.541006+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:50:34.541006+0000","last_undegraded":"2023-09-27T23:50:34.541006+0000","last_fullsized":"2023-09-27T23:50:34.541006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565485+0000","last_change":"2023-09-27T23:50:34.560491+0000","last_active":"2023-09-27T23:50:34.565485+0000","last_peered":"2023-09-27T23:50:34.565485+0000","last_clean":"2023-09-27T23:50:34.565485+0000","last_became_active":"2023-09-27T23:50:34.560184+0000","last_became_peered":"2023-09-27T23:50:34.560184+0000","last_unstale":"2023-09-27T23:50:34.565485+0000","last_undegraded":"2023-09-27T23:50:34.565485+0000","last_fullsized":"2023-09-27T23:50:34.565485+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:56:51.904785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.558616+0000","last_change":"2023-09-27T23:50:34.170530+0000","last_active":"2023-09-27T23:50:34.558616+0000","last_peered":"2023-09-27T23:50:34.558616+0000","last_clean":"2023-09-27T23:50:34.558616+0000","last_became_active":"2023-09-27T23:50:34.170207+0000","last_became_peered":"2023-09-27T23:50:34.170207+0000","last_unstale":"2023-09-27T23:50:34.558616+0000","last_undegraded":"2023-09-27T23:50:34.558616+0000","last_fullsized":"2023-09-27T23:50:34.558616+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:41:22.504977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.540860+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:50:34.540860+0000","last_peered":"2023-09-27T23:50:34.540860+0000","last_clean":"2023-09-27T23:50:34.540860+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:50:34.540860+0000","last_undegraded":"2023-09-27T23:50:34.540860+0000","last_fullsized":"2023-09-27T23:50:34.540860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35688,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":32960,"kb_avail":94336152,"statfs":{"total":96636764160,"available":96600219648,"internally_reserved":0,"allocated":2785280,"data_stored":1732437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33751040},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.90300000000000002,"15min":0.90000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.873}]},{"osd":1,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.90000000000000002},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57199999999999995},"max":{"1min":1.3,"5min":1.333,"15min":1.3360000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.97299999999999998,"15min":0.97599999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.1019999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91600000000000004,"15min":0.93000000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1799999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.98799999999999999,"15min":0.96599999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.228,"5min":1.341,"15min":1.341},"last":1.0920000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93700000000000006},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2589999999999999,"5min":1.359,"15min":1.359},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0820000000000001,"15min":1.079},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.369,"5min":1.369,"15min":1.3879999999999999},"last":1.1499999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0369999999999999,"15min":1.0089999999999999},"min":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.622},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":1.169}]},{"osd":5,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0700000000000001,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.4530000000000001,"5min":1.484,"15min":1.484},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3620000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.119}]},{"osd":6,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1479999999999999,"15min":1.145},"min":{"1min":0.85499999999999998,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.3280000000000001,"5min":1.425,"15min":1.448},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.153,"15min":1.145},"min":{"1min":0.85699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.48,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.381}]}]},{"osd":6,"up_from":15,"seq":64424509881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29712,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":27584,"kb_avail":94342128,"statfs":{"total":96636764160,"available":96606339072,"internally_reserved":0,"allocated":2170880,"data_stored":1139841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28246016},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.99399999999999999,"15min":0.97099999999999997},"min":{"1min":0.65900000000000003,"5min":0.60599999999999998,"15min":0.51800000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.071},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0529999999999999,"15min":1.0600000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.69399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.749}]},{"osd":1,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.66800000000000004,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.57099999999999995},"max":{"1min":1.226,"5min":1.53,"15min":1.53},"last":1.1599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61399999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.63200000000000001},"max":{"1min":1.2709999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.74099999999999999,"5min":0.65700000000000003,"15min":0.63100000000000001},"max":{"1min":1.53,"5min":1.583,"15min":1.583},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.169,"5min":1.325,"15min":1.325},"last":0.97499999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.878,"15min":0.873},"min":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.41599999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0129999999999999,"15min":1},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3700000000000001,"5min":1.633,"15min":1.633},"last":1.0409999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.95099999999999996},"min":{"1min":0.71099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.4430000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.149,"15min":1.141},"min":{"1min":0.81299999999999994,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.1910000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.1180000000000001},"min":{"1min":0.755,"5min":0.627,"15min":0.627},"max":{"1min":1.623,"5min":1.679,"15min":1.7210000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.423,"5min":1.488,"15min":1.542},"last":1.242}]}]},{"osd":5,"up_from":15,"seq":64424509881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39316,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":37184,"kb_avail":94332524,"statfs":{"total":96636764160,"available":96596504576,"internally_reserved":0,"allocated":2174976,"data_stored":1140395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38076416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86699999999999999,"15min":0.86199999999999999},"min":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.51900000000000002},"max":{"1min":1.248,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":10.058999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.47899999999999998},"max":{"1min":1.2250000000000001,"5min":1.292,"15min":1.5640000000000001},"last":13.757999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.302,"5min":1.425,"15min":1.538},"last":4.1970000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.194,"5min":1.3819999999999999,"15min":1.397},"last":2.8650000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.629},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.4690000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.626},"max":{"1min":1.153,"5min":1.3049999999999999,"15min":1.4470000000000001},"last":4.0999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.3240000000000001,"5min":1.373,"15min":1.611},"last":3.7450000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0349999999999999,"15min":1.0249999999999999},"min":{"1min":0.84599999999999997,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.228,"5min":1.419,"15min":1.4370000000000001},"last":4.2350000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.88},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.3129999999999999,"5min":1.484,"15min":1.484},"last":2.0270000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.58699999999999997},"max":{"1min":1.45,"5min":1.45,"15min":1.4510000000000001},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.026},"min":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.242,"5min":1.496,"15min":1.5069999999999999},"last":4.157},{"interface":"front","average":{"1min":1.089,"5min":1.099,"15min":1.0980000000000001},"min":{"1min":0.84899999999999998,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":1.262,"5min":1.546,"15min":1.5840000000000001},"last":2.1150000000000002}]},{"osd":7,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1619999999999999,"15min":1.1459999999999999},"min":{"1min":0.94099999999999995,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7709999999999999},"last":6.3710000000000004},{"interface":"front","average":{"1min":1.194,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":0.98099999999999998,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":14.119}]}]},{"osd":4,"up_from":15,"seq":64424509882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25628,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":23488,"kb_avail":94346212,"statfs":{"total":96636764160,"available":96610521088,"internally_reserved":0,"allocated":2183168,"data_stored":1141425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24051712},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.92500000000000004,"15min":0.90700000000000003},"min":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.45000000000000001},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88800000000000001,"15min":0.89500000000000002},"min":{"1min":0.63200000000000001,"5min":0.624,"15min":0.41599999999999998},"max":{"1min":1.893,"5min":1.893,"15min":2.2770000000000001},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.90200000000000002},"min":{"1min":0.75900000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.53500000000000003},"max":{"1min":1.758,"5min":1.758,"15min":2.1110000000000002},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.98299999999999998,"15min":0.96799999999999997},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.966,"5min":1.966,"15min":2.056},"last":0.96999999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.998,"5min":1.998,"15min":2.032},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.0880000000000001,"15min":1.054},"min":{"1min":0.82899999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.0129999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93700000000000006,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.61399999999999999,"15min":0.45500000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.9930000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":0.751}]},{"osd":6,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0600000000000001,"15min":1.0509999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.694,"5min":1.873,"15min":1.873},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.1759999999999999,"15min":1.1679999999999999},"min":{"1min":0.96199999999999997,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1800000000000002},"last":1.081}]},{"osd":7,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.105,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.002},{"interface":"front","average":{"1min":1.177,"5min":1.1779999999999999,"15min":1.177},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1180000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30224,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":28096,"kb_avail":94341616,"statfs":{"total":96636764160,"available":96605814784,"internally_reserved":0,"allocated":2170880,"data_stored":1139853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28770304},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83999999999999997,"5min":1.1639999999999999,"15min":1.3560000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.746},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81599999999999995,"5min":1.2629999999999999,"15min":1.377},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1799999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.69299999999999995,"5min":0.63,"15min":0.58999999999999997},"max":{"1min":1.2509999999999999,"5min":1.272,"15min":1.2849999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.876,"15min":0.873},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"max":{"1min":1.1599999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94099999999999995},"min":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.63800000000000001},"max":{"1min":1.175,"5min":1.3420000000000001,"15min":1.383},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.998,"15min":0.98399999999999999},"min":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.621},"max":{"1min":1.0600000000000001,"5min":1.599,"15min":1.599},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.86399999999999999,"5min":0.72099999999999997,"15min":0.68700000000000006},"max":{"1min":1.3069999999999999,"5min":1.587,"15min":1.587},"last":0.91300000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.123,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.484,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.181},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.72699999999999998},"max":{"1min":1.6499999999999999,"5min":1.659,"15min":1.659},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.92900000000000005,"5min":0.878,"15min":0.77800000000000002},"max":{"1min":1.288,"5min":1.5249999999999999,"15min":1.659},"last":0.873},{"interface":"front","average":{"1min":1.222,"5min":1.258,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.90500000000000003,"15min":0.82599999999999996},"max":{"1min":1.413,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.002}]},{"osd":7,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2629999999999999,"15min":1.25},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5629999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.94999999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.409,"5min":1.657,"15min":1.657},"last":1.0509999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30160,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":28032,"kb_avail":94341680,"statfs":{"total":96636764160,"available":96605880320,"internally_reserved":0,"allocated":2170880,"data_stored":1139909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28704768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1879999999999999,"5min":1.252,"15min":1.5329999999999999},"last":1.081},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97299999999999998,"5min":1.345,"15min":1.7969999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.073,"5min":1.242,"15min":2.0760000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84299999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.754,"15min":0.76900000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.2350000000000001,"15min":1.667},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.91100000000000003,"15min":0.94499999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.159,"5min":1.262,"15min":1.7},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0820000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.972},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.361,"5min":1.645,"15min":1.8009999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0429999999999999,"15min":1.0489999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.772},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.151,"15min":1.173},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.351,"5min":1.653,"15min":1.859},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.145,"15min":1.151},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.8999999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.175,"15min":1.1759999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6259999999999999,"5min":1.6439999999999999,"15min":1.736},"last":0.95399999999999996}]},{"osd":7,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.86599999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":2.0059999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2190000000000001,"15min":1.2310000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5669999999999999,"5min":1.8129999999999999,"15min":1.8340000000000001},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44368,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":42240,"kb_avail":94327472,"statfs":{"total":96636764160,"available":96591331328,"internally_reserved":0,"allocated":2170880,"data_stored":1139853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43253760},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1819999999999999,"15min":1.2070000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.73999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.095,"5min":1.177,"15min":1.361},"last":0.73199999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.88,"5min":1.2509999999999999,"15min":1.383},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2350000000000001,"5min":1.3169999999999999,"15min":1.427},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.64600000000000002,"5min":0.626,"15min":0.58899999999999997},"max":{"1min":1.1890000000000001,"5min":1.3080000000000001,"15min":1.3720000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.83599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.454},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.98699999999999999,"15min":0.98999999999999999},"min":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.325,"15min":1.5229999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.196,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.71999999999999997},"max":{"1min":1.4299999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.091,"15min":1.0980000000000001},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.621},"max":{"1min":1.399,"5min":1.5449999999999999,"15min":1.583},"last":0.877},{"interface":"front","average":{"1min":1.077,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.747},"max":{"1min":1.3,"5min":1.4710000000000001,"15min":1.575},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2450000000000001,"15min":1.234},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.76800000000000002},"max":{"1min":1.5529999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.72299999999999998},"max":{"1min":1.4970000000000001,"5min":1.7789999999999999,"15min":1.804},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.94499999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.514,"5min":1.7130000000000001,"15min":1.853},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.95699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4139999999999999,"5min":1.585,"15min":1.827},"last":1.204}]}]},{"osd":0,"up_from":15,"seq":64424509883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31312,"kb_used_data":2696,"kb_used_omap":0,"kb_used_meta":28608,"kb_avail":94340528,"statfs":{"total":96636764160,"available":96604700672,"internally_reserved":0,"allocated":2760704,"data_stored":1730187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29294592},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.85499999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.47499999999999998},"max":{"1min":1.131,"5min":1.1779999999999999,"15min":1.306},"last":0.84399999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.502},"max":{"1min":0.98299999999999998,"5min":1.236,"15min":1.5960000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.101,"5min":1.288,"15min":1.863},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.78600000000000003,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.92500000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3,"5min":1.4099999999999999,"15min":1.7110000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.70699999999999996},"max":{"1min":1.516,"5min":1.5249999999999999,"15min":1.651},"last":0.98899999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.111,"15min":1.111},"min":{"1min":0.90700000000000003,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.407,"5min":1.4259999999999999,"15min":1.512},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.165,"15min":1.1779999999999999},"min":{"1min":0.86399999999999999,"5min":0.83399999999999996,"15min":0.79800000000000004},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.847},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1870000000000001,"15min":1.1830000000000001},"min":{"1min":0.90700000000000003,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.895},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2330000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.84999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5720000000000001,"15min":2.0510000000000002},"last":1.3}]},{"osd":7,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.286,"15min":1.2789999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.84799999999999998},"max":{"1min":1.554,"5min":1.6100000000000001,"15min":2.0030000000000001},"last":1.21},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.85599999999999998},"max":{"1min":1.4790000000000001,"5min":1.806,"15min":2.004},"last":1.0469999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:50:40.464 INFO:tasks.ceph.mds.c.smithi064.stderr:2023-09-27T23:50:40.462+0000 7f35cce75700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:50:41.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:50:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.046+0000 7f7642b75700 1 Processor -- start 2023-09-27T23:50:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.046+0000 7f7642b75700 1 -- start start 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.046+0000 7f7642b75700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f763c068800 0x7f763c0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.046+0000 7f7642b75700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c06c9f0 con 0x7f763c0c2980 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.046+0000 7f7642b75700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c0c7f20 con 0x7f763c13f890 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.046+0000 7f7642b75700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0c1c50 con 0x7f763c068800 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f763c0c2980 0x7f763c0680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40980/0 (socket says 172.21.15.64:40980) 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763bfff700 1 -- 172.21.15.64:0/1557006949 learned_addr learned my addr 172.21.15.64:0/1557006949 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7641112700 1 --2- 172.21.15.64:0/1557006949 >> v2:172.21.15.123:3300/0 conn(0x7f763c068800 0x7f763c0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763b7fe700 1 -- 172.21.15.64:0/1557006949 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714067666 0 0) 0x7f763c06c9f0 con 0x7f763c0c2980 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763b7fe700 1 -- 172.21.15.64:0/1557006949 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f762c003570 con 0x7f763c0c2980 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7641112700 1 -- 172.21.15.64:0/1557006949 >> v1:172.21.15.123:6789/0 conn(0x7f763c13f890 legacy=0x7f763c0c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7641112700 1 -- 172.21.15.64:0/1557006949 >> v1:172.21.15.64:6789/0 conn(0x7f763c0c2980 legacy=0x7f763c0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7641112700 1 -- 172.21.15.64:0/1557006949 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0c8190 con 0x7f763c068800 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7641112700 1 --2- 172.21.15.64:0/1557006949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c068800 0x7f763c0aeba0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7634009e60 tx=0x7f7634009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2eb9a986839d05d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763b7fe700 1 -- 172.21.15.64:0/1557006949 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400e070 con 0x7f763c068800 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763b7fe700 1 -- 172.21.15.64:0/1557006949 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763400bb60 con 0x7f763c068800 2023-09-27T23:50:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f763b7fe700 1 -- 172.21.15.64:0/1557006949 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634012430 con 0x7f763c068800 2023-09-27T23:50:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 -- 172.21.15.64:0/1557006949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c068800 msgr2=0x7f763c0aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 --2- 172.21.15.64:0/1557006949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c068800 0x7f763c0aeba0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7634009e60 tx=0x7f7634009780 comp rx=0 tx=0).stop 2023-09-27T23:50:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 -- 172.21.15.64:0/1557006949 shutdown_connections 2023-09-27T23:50:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 --2- 172.21.15.64:0/1557006949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c068800 0x7f763c0aeba0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 -- 172.21.15.64:0/1557006949 >> 172.21.15.64:0/1557006949 conn(0x7f763c0c8850 msgr2=0x7f763c0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 -- 172.21.15.64:0/1557006949 shutdown_connections 2023-09-27T23:50:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.050+0000 7f7642b75700 1 -- 172.21.15.64:0/1557006949 wait complete. 2023-09-27T23:50:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 Processor -- start 2023-09-27T23:50:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 -- start start 2023-09-27T23:50:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 0x7f763c0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763c0c2980 0x7f763c0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c13f890 0x7f763c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c15b2d0 con 0x7f763c068800 2023-09-27T23:50:42.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0b1a80 con 0x7f763c13f890 2023-09-27T23:50:42.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7642b75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c167e40 con 0x7f763c0c2980 2023-09-27T23:50:42.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 0x7f763c0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 0x7f763c0b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54578/0 (socket says 172.21.15.64:54578) 2023-09-27T23:50:42.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f763bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763c0c2980 0x7f763c0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 -- 172.21.15.64:0/2201108122 learned_addr learned my addr 172.21.15.64:0/2201108122 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7641112700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c13f890 0x7f763c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 -- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763c0c2980 msgr2=0x7f763c0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763c0c2980 0x7f763c0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 -- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c13f890 msgr2=0x7f763c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c13f890 0x7f763c1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.054+0000 7f7640911700 1 -- 172.21.15.64:0/2201108122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7634009a20 con 0x7f763c068800 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7640911700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 0x7f763c0b0b00 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f763000c5c0 tx=0x7f7630007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630011590 con 0x7f763c068800 2023-09-27T23:50:42.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7630011730 con 0x7f763c068800 2023-09-27T23:50:42.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c0c2e90 con 0x7f763c068800 2023-09-27T23:50:42.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630011af0 con 0x7f763c068800 2023-09-27T23:50:42.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c167510 con 0x7f763c068800 2023-09-27T23:50:42.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c071120 con 0x7f763c068800 2023-09-27T23:50:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.058+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7630012420 con 0x7f763c068800 2023-09-27T23:50:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.062+0000 7f7638ff9700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7620060a70 0x7f7620062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.062+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f763007c320 con 0x7f763c068800 2023-09-27T23:50:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.062+0000 7f763bfff700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7620060a70 0x7f7620062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.062+0000 7f763bfff700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7620060a70 0x7f7620062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f763c0c7380 tx=0x7f7628009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.066+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7630049bb0 con 0x7f763c068800 2023-09-27T23:50:42.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.226+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f763c167f80 con 0x7f7620060a70 2023-09-27T23:50:42.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.246+0000 7f7638ff9700 1 -- 172.21.15.64:0/2201108122 <== mgr.8210 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308150 (secure 0 0 0) 0x7f762800f680 con 0x7f7620060a70 2023-09-27T23:50:42.255 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7620060a70 msgr2=0x7f7620062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7620060a70 0x7f7620062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f763c0c7380 tx=0x7f7628009040 comp rx=0 tx=0).stop 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 msgr2=0x7f763c0b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 0x7f763c0b0b00 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f763000c5c0 tx=0x7f7630007f30 comp rx=0 tx=0).stop 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 shutdown_connections 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7620060a70 0x7f7620062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f763c13f890 0x7f763c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f763c0c2980 0x7f763c0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 --2- 172.21.15.64:0/2201108122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763c068800 0x7f763c0b0b00 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 >> 172.21.15.64:0/2201108122 conn(0x7f763c0c8850 msgr2=0x7f763c155540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:42.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 shutdown_connections 2023-09-27T23:50:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.254+0000 7f7642b75700 1 -- 172.21.15.64:0/2201108122 wait complete. 2023-09-27T23:50:42.259 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:50:42.272 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-27T23:50:40.537922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":543,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266408,"kb_used_data":18152,"kb_used_omap":0,"kb_used_meta":248192,"kb_avail":754708312,"statfs":{"total":773094113280,"available":772821311488,"internally_reserved":0,"allocated":18587648,"data_stored":10303900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254148608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.388811"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567756+0000","last_change":"2023-09-27T23:50:34.559306+0000","last_active":"2023-09-27T23:50:34.567756+0000","last_peered":"2023-09-27T23:50:34.567756+0000","last_clean":"2023-09-27T23:50:34.567756+0000","last_became_active":"2023-09-27T23:50:34.557020+0000","last_became_peered":"2023-09-27T23:50:34.557020+0000","last_unstale":"2023-09-27T23:50:34.567756+0000","last_undegraded":"2023-09-27T23:50:34.567756+0000","last_fullsized":"2023-09-27T23:50:34.567756+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:01:54.141241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568612+0000","last_change":"2023-09-27T23:50:34.564749+0000","last_active":"2023-09-27T23:50:34.568612+0000","last_peered":"2023-09-27T23:50:34.568612+0000","last_clean":"2023-09-27T23:50:34.568612+0000","last_became_active":"2023-09-27T23:50:34.563936+0000","last_became_peered":"2023-09-27T23:50:34.563936+0000","last_unstale":"2023-09-27T23:50:34.568612+0000","last_undegraded":"2023-09-27T23:50:34.568612+0000","last_fullsized":"2023-09-27T23:50:34.568612+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:21:30.130546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565266+0000","last_change":"2023-09-27T23:50:34.559269+0000","last_active":"2023-09-27T23:50:34.565266+0000","last_peered":"2023-09-27T23:50:34.565266+0000","last_clean":"2023-09-27T23:50:34.565266+0000","last_became_active":"2023-09-27T23:50:34.558969+0000","last_became_peered":"2023-09-27T23:50:34.558969+0000","last_unstale":"2023-09-27T23:50:34.565266+0000","last_undegraded":"2023-09-27T23:50:34.565266+0000","last_fullsized":"2023-09-27T23:50:34.565266+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:42:30.339684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.555397+0000","last_change":"2023-09-27T23:50:34.548611+0000","last_active":"2023-09-27T23:50:34.555397+0000","last_peered":"2023-09-27T23:50:34.555397+0000","last_clean":"2023-09-27T23:50:34.555397+0000","last_became_active":"2023-09-27T23:50:34.548157+0000","last_became_peered":"2023-09-27T23:50:34.548157+0000","last_unstale":"2023-09-27T23:50:34.555397+0000","last_undegraded":"2023-09-27T23:50:34.555397+0000","last_fullsized":"2023-09-27T23:50:34.555397+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:50:01.524610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566787+0000","last_change":"2023-09-27T23:50:34.561451+0000","last_active":"2023-09-27T23:50:34.566787+0000","last_peered":"2023-09-27T23:50:34.566787+0000","last_clean":"2023-09-27T23:50:34.566787+0000","last_became_active":"2023-09-27T23:50:34.560978+0000","last_became_peered":"2023-09-27T23:50:34.560978+0000","last_unstale":"2023-09-27T23:50:34.566787+0000","last_undegraded":"2023-09-27T23:50:34.566787+0000","last_fullsized":"2023-09-27T23:50:34.566787+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:00.236964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557051+0000","last_change":"2023-09-27T23:50:34.549806+0000","last_active":"2023-09-27T23:50:34.557051+0000","last_peered":"2023-09-27T23:50:34.557051+0000","last_clean":"2023-09-27T23:50:34.557051+0000","last_became_active":"2023-09-27T23:50:34.549518+0000","last_became_peered":"2023-09-27T23:50:34.549518+0000","last_unstale":"2023-09-27T23:50:34.557051+0000","last_undegraded":"2023-09-27T23:50:34.557051+0000","last_fullsized":"2023-09-27T23:50:34.557051+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:14.120052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567301+0000","last_change":"2023-09-27T23:50:34.561892+0000","last_active":"2023-09-27T23:50:34.567301+0000","last_peered":"2023-09-27T23:50:34.567301+0000","last_clean":"2023-09-27T23:50:34.567301+0000","last_became_active":"2023-09-27T23:50:34.561632+0000","last_became_peered":"2023-09-27T23:50:34.561632+0000","last_unstale":"2023-09-27T23:50:34.567301+0000","last_undegraded":"2023-09-27T23:50:34.567301+0000","last_fullsized":"2023-09-27T23:50:34.567301+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:11:15.187943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568630+0000","last_change":"2023-09-27T23:50:34.564850+0000","last_active":"2023-09-27T23:50:34.568630+0000","last_peered":"2023-09-27T23:50:34.568630+0000","last_clean":"2023-09-27T23:50:34.568630+0000","last_became_active":"2023-09-27T23:50:34.563909+0000","last_became_peered":"2023-09-27T23:50:34.563909+0000","last_unstale":"2023-09-27T23:50:34.568630+0000","last_undegraded":"2023-09-27T23:50:34.568630+0000","last_fullsized":"2023-09-27T23:50:34.568630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:14.120052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568601+0000","last_change":"2023-09-27T23:50:34.564661+0000","last_active":"2023-09-27T23:50:34.568601+0000","last_peered":"2023-09-27T23:50:34.568601+0000","last_clean":"2023-09-27T23:50:34.568601+0000","last_became_active":"2023-09-27T23:50:34.564204+0000","last_became_peered":"2023-09-27T23:50:34.564204+0000","last_unstale":"2023-09-27T23:50:34.568601+0000","last_undegraded":"2023-09-27T23:50:34.568601+0000","last_fullsized":"2023-09-27T23:50:34.568601+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:51.292130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566560+0000","last_change":"2023-09-27T23:50:34.561035+0000","last_active":"2023-09-27T23:50:34.566560+0000","last_peered":"2023-09-27T23:50:34.566560+0000","last_clean":"2023-09-27T23:50:34.566560+0000","last_became_active":"2023-09-27T23:50:34.560706+0000","last_became_peered":"2023-09-27T23:50:34.560706+0000","last_unstale":"2023-09-27T23:50:34.566560+0000","last_undegraded":"2023-09-27T23:50:34.566560+0000","last_fullsized":"2023-09-27T23:50:34.566560+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:59.315891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566641+0000","last_change":"2023-09-27T23:50:34.561307+0000","last_active":"2023-09-27T23:50:34.566641+0000","last_peered":"2023-09-27T23:50:34.566641+0000","last_clean":"2023-09-27T23:50:34.566641+0000","last_became_active":"2023-09-27T23:50:34.560994+0000","last_became_peered":"2023-09-27T23:50:34.560994+0000","last_unstale":"2023-09-27T23:50:34.566641+0000","last_undegraded":"2023-09-27T23:50:34.566641+0000","last_fullsized":"2023-09-27T23:50:34.566641+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:05.704176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566725+0000","last_change":"2023-09-27T23:50:34.561839+0000","last_active":"2023-09-27T23:50:34.566725+0000","last_peered":"2023-09-27T23:50:34.566725+0000","last_clean":"2023-09-27T23:50:34.566725+0000","last_became_active":"2023-09-27T23:50:34.561533+0000","last_became_peered":"2023-09-27T23:50:34.561533+0000","last_unstale":"2023-09-27T23:50:34.566725+0000","last_undegraded":"2023-09-27T23:50:34.566725+0000","last_fullsized":"2023-09-27T23:50:34.566725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:38.874744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562389+0000","last_change":"2023-09-27T23:50:34.554162+0000","last_active":"2023-09-27T23:50:34.562389+0000","last_peered":"2023-09-27T23:50:34.562389+0000","last_clean":"2023-09-27T23:50:34.562389+0000","last_became_active":"2023-09-27T23:50:34.553738+0000","last_became_peered":"2023-09-27T23:50:34.553738+0000","last_unstale":"2023-09-27T23:50:34.562389+0000","last_undegraded":"2023-09-27T23:50:34.562389+0000","last_fullsized":"2023-09-27T23:50:34.562389+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:23.930966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562452+0000","last_change":"2023-09-27T23:50:34.554787+0000","last_active":"2023-09-27T23:50:34.562452+0000","last_peered":"2023-09-27T23:50:34.562452+0000","last_clean":"2023-09-27T23:50:34.562452+0000","last_became_active":"2023-09-27T23:50:34.554419+0000","last_became_peered":"2023-09-27T23:50:34.554419+0000","last_unstale":"2023-09-27T23:50:34.562452+0000","last_undegraded":"2023-09-27T23:50:34.562452+0000","last_fullsized":"2023-09-27T23:50:34.562452+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:46:29.884638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567128+0000","last_change":"2023-09-27T23:50:34.556680+0000","last_active":"2023-09-27T23:50:34.567128+0000","last_peered":"2023-09-27T23:50:34.567128+0000","last_clean":"2023-09-27T23:50:34.567128+0000","last_became_active":"2023-09-27T23:50:34.556372+0000","last_became_peered":"2023-09-27T23:50:34.556372+0000","last_unstale":"2023-09-27T23:50:34.567128+0000","last_undegraded":"2023-09-27T23:50:34.567128+0000","last_fullsized":"2023-09-27T23:50:34.567128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:33.637539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567347+0000","last_change":"2023-09-27T23:50:34.562541+0000","last_active":"2023-09-27T23:50:34.567347+0000","last_peered":"2023-09-27T23:50:34.567347+0000","last_clean":"2023-09-27T23:50:34.567347+0000","last_became_active":"2023-09-27T23:50:34.562186+0000","last_became_peered":"2023-09-27T23:50:34.562186+0000","last_unstale":"2023-09-27T23:50:34.567347+0000","last_undegraded":"2023-09-27T23:50:34.567347+0000","last_fullsized":"2023-09-27T23:50:34.567347+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:07.395190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557012+0000","last_change":"2023-09-27T23:50:34.548939+0000","last_active":"2023-09-27T23:50:34.557012+0000","last_peered":"2023-09-27T23:50:34.557012+0000","last_clean":"2023-09-27T23:50:34.557012+0000","last_became_active":"2023-09-27T23:50:34.548670+0000","last_became_peered":"2023-09-27T23:50:34.548670+0000","last_unstale":"2023-09-27T23:50:34.557012+0000","last_undegraded":"2023-09-27T23:50:34.557012+0000","last_fullsized":"2023-09-27T23:50:34.557012+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:38.874744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.555929+0000","last_change":"2023-09-27T23:50:34.553100+0000","last_active":"2023-09-27T23:50:34.555929+0000","last_peered":"2023-09-27T23:50:34.555929+0000","last_clean":"2023-09-27T23:50:34.555929+0000","last_became_active":"2023-09-27T23:50:34.552586+0000","last_became_peered":"2023-09-27T23:50:34.552586+0000","last_unstale":"2023-09-27T23:50:34.555929+0000","last_undegraded":"2023-09-27T23:50:34.555929+0000","last_fullsized":"2023-09-27T23:50:34.555929+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:22.875022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564471+0000","last_change":"2023-09-27T23:50:34.558021+0000","last_active":"2023-09-27T23:50:34.564471+0000","last_peered":"2023-09-27T23:50:34.564471+0000","last_clean":"2023-09-27T23:50:34.564471+0000","last_became_active":"2023-09-27T23:50:34.557537+0000","last_became_peered":"2023-09-27T23:50:34.557537+0000","last_unstale":"2023-09-27T23:50:34.564471+0000","last_undegraded":"2023-09-27T23:50:34.564471+0000","last_fullsized":"2023-09-27T23:50:34.564471+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:41:46.194689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562674+0000","last_change":"2023-09-27T23:50:34.554619+0000","last_active":"2023-09-27T23:50:34.562674+0000","last_peered":"2023-09-27T23:50:34.562674+0000","last_clean":"2023-09-27T23:50:34.562674+0000","last_became_active":"2023-09-27T23:50:34.554325+0000","last_became_peered":"2023-09-27T23:50:34.554325+0000","last_unstale":"2023-09-27T23:50:34.562674+0000","last_undegraded":"2023-09-27T23:50:34.562674+0000","last_fullsized":"2023-09-27T23:50:34.562674+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:31:08.484234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.559518+0000","last_change":"2023-09-27T23:50:34.553184+0000","last_active":"2023-09-27T23:50:34.559518+0000","last_peered":"2023-09-27T23:50:34.559518+0000","last_clean":"2023-09-27T23:50:34.559518+0000","last_became_active":"2023-09-27T23:50:34.552913+0000","last_became_peered":"2023-09-27T23:50:34.552913+0000","last_unstale":"2023-09-27T23:50:34.559518+0000","last_undegraded":"2023-09-27T23:50:34.559518+0000","last_fullsized":"2023-09-27T23:50:34.559518+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:51.292130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566621+0000","last_change":"2023-09-27T23:50:34.561306+0000","last_active":"2023-09-27T23:50:34.566621+0000","last_peered":"2023-09-27T23:50:34.566621+0000","last_clean":"2023-09-27T23:50:34.566621+0000","last_became_active":"2023-09-27T23:50:34.560926+0000","last_became_peered":"2023-09-27T23:50:34.560926+0000","last_unstale":"2023-09-27T23:50:34.566621+0000","last_undegraded":"2023-09-27T23:50:34.566621+0000","last_fullsized":"2023-09-27T23:50:34.566621+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:01:33.992859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565802+0000","last_change":"2023-09-27T23:50:34.560354+0000","last_active":"2023-09-27T23:50:34.565802+0000","last_peered":"2023-09-27T23:50:34.565802+0000","last_clean":"2023-09-27T23:50:34.565802+0000","last_became_active":"2023-09-27T23:50:34.560081+0000","last_became_peered":"2023-09-27T23:50:34.560081+0000","last_unstale":"2023-09-27T23:50:34.565802+0000","last_undegraded":"2023-09-27T23:50:34.565802+0000","last_fullsized":"2023-09-27T23:50:34.565802+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:17:59.315891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566821+0000","last_change":"2023-09-27T23:50:34.564305+0000","last_active":"2023-09-27T23:50:34.566821+0000","last_peered":"2023-09-27T23:50:34.566821+0000","last_clean":"2023-09-27T23:50:34.566821+0000","last_became_active":"2023-09-27T23:50:34.561802+0000","last_became_peered":"2023-09-27T23:50:34.561802+0000","last_unstale":"2023-09-27T23:50:34.566821+0000","last_undegraded":"2023-09-27T23:50:34.566821+0000","last_fullsized":"2023-09-27T23:50:34.566821+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:18:54.071865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565255+0000","last_change":"2023-09-27T23:50:34.559133+0000","last_active":"2023-09-27T23:50:34.565255+0000","last_peered":"2023-09-27T23:50:34.565255+0000","last_clean":"2023-09-27T23:50:34.565255+0000","last_became_active":"2023-09-27T23:50:34.558896+0000","last_became_peered":"2023-09-27T23:50:34.558896+0000","last_unstale":"2023-09-27T23:50:34.565255+0000","last_undegraded":"2023-09-27T23:50:34.565255+0000","last_fullsized":"2023-09-27T23:50:34.565255+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:24:03.170328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565171+0000","last_change":"2023-09-27T23:50:34.554988+0000","last_active":"2023-09-27T23:50:34.565171+0000","last_peered":"2023-09-27T23:50:34.565171+0000","last_clean":"2023-09-27T23:50:34.565171+0000","last_became_active":"2023-09-27T23:50:34.554709+0000","last_became_peered":"2023-09-27T23:50:34.554709+0000","last_unstale":"2023-09-27T23:50:34.565171+0000","last_undegraded":"2023-09-27T23:50:34.565171+0000","last_fullsized":"2023-09-27T23:50:34.565171+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-28T23:57:07.898196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562510+0000","last_change":"2023-09-27T23:50:34.555078+0000","last_active":"2023-09-27T23:50:34.562510+0000","last_peered":"2023-09-27T23:50:34.562510+0000","last_clean":"2023-09-27T23:50:34.562510+0000","last_became_active":"2023-09-27T23:50:34.554055+0000","last_became_peered":"2023-09-27T23:50:34.554055+0000","last_unstale":"2023-09-27T23:50:34.562510+0000","last_undegraded":"2023-09-27T23:50:34.562510+0000","last_fullsized":"2023-09-27T23:50:34.562510+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:12.959974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562592+0000","last_change":"2023-09-27T23:50:34.554232+0000","last_active":"2023-09-27T23:50:34.562592+0000","last_peered":"2023-09-27T23:50:34.562592+0000","last_clean":"2023-09-27T23:50:34.562592+0000","last_became_active":"2023-09-27T23:50:34.553835+0000","last_became_peered":"2023-09-27T23:50:34.553835+0000","last_unstale":"2023-09-27T23:50:34.562592+0000","last_undegraded":"2023-09-27T23:50:34.562592+0000","last_fullsized":"2023-09-27T23:50:34.562592+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:33:01.154675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565302+0000","last_change":"2023-09-27T23:50:34.557901+0000","last_active":"2023-09-27T23:50:34.565302+0000","last_peered":"2023-09-27T23:50:34.565302+0000","last_clean":"2023-09-27T23:50:34.565302+0000","last_became_active":"2023-09-27T23:50:34.557646+0000","last_became_peered":"2023-09-27T23:50:34.557646+0000","last_unstale":"2023-09-27T23:50:34.565302+0000","last_undegraded":"2023-09-27T23:50:34.565302+0000","last_fullsized":"2023-09-27T23:50:34.565302+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:34:12.347546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565903+0000","last_change":"2023-09-27T23:50:34.561125+0000","last_active":"2023-09-27T23:50:34.565903+0000","last_peered":"2023-09-27T23:50:34.565903+0000","last_clean":"2023-09-27T23:50:34.565903+0000","last_became_active":"2023-09-27T23:50:34.560816+0000","last_became_peered":"2023-09-27T23:50:34.560816+0000","last_unstale":"2023-09-27T23:50:34.565903+0000","last_undegraded":"2023-09-27T23:50:34.565903+0000","last_fullsized":"2023-09-27T23:50:34.565903+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:21:43.479240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.563597+0000","last_change":"2023-09-27T23:50:34.553110+0000","last_active":"2023-09-27T23:50:34.563597+0000","last_peered":"2023-09-27T23:50:34.563597+0000","last_clean":"2023-09-27T23:50:34.563597+0000","last_became_active":"2023-09-27T23:50:34.552708+0000","last_became_peered":"2023-09-27T23:50:34.552708+0000","last_unstale":"2023-09-27T23:50:34.563597+0000","last_undegraded":"2023-09-27T23:50:34.563597+0000","last_fullsized":"2023-09-27T23:50:34.563597+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:11:15.187943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.558417+0000","last_change":"2023-09-27T23:50:34.557307+0000","last_active":"2023-09-27T23:50:34.558417+0000","last_peered":"2023-09-27T23:50:34.558417+0000","last_clean":"2023-09-27T23:50:34.558417+0000","last_became_active":"2023-09-27T23:50:34.556999+0000","last_became_peered":"2023-09-27T23:50:34.556999+0000","last_unstale":"2023-09-27T23:50:34.558417+0000","last_undegraded":"2023-09-27T23:50:34.558417+0000","last_fullsized":"2023-09-27T23:50:34.558417+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:39:44.668021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565195+0000","last_change":"2023-09-27T23:50:34.555118+0000","last_active":"2023-09-27T23:50:34.565195+0000","last_peered":"2023-09-27T23:50:34.565195+0000","last_clean":"2023-09-27T23:50:34.565195+0000","last_became_active":"2023-09-27T23:50:34.554853+0000","last_became_peered":"2023-09-27T23:50:34.554853+0000","last_unstale":"2023-09-27T23:50:34.565195+0000","last_undegraded":"2023-09-27T23:50:34.565195+0000","last_fullsized":"2023-09-27T23:50:34.565195+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:29:59.791766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"173'4","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562882+0000","last_change":"2023-09-27T23:50:34.185949+0000","last_active":"2023-09-27T23:50:34.562882+0000","last_peered":"2023-09-27T23:50:34.562882+0000","last_clean":"2023-09-27T23:50:34.562882+0000","last_became_active":"2023-09-27T23:50:34.185633+0000","last_became_peered":"2023-09-27T23:50:34.185633+0000","last_unstale":"2023-09-27T23:50:34.562882+0000","last_undegraded":"2023-09-27T23:50:34.562882+0000","last_fullsized":"2023-09-27T23:50:34.562882+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:01:32.981121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565799+0000","last_change":"2023-09-27T23:50:34.559072+0000","last_active":"2023-09-27T23:50:34.565799+0000","last_peered":"2023-09-27T23:50:34.565799+0000","last_clean":"2023-09-27T23:50:34.565799+0000","last_became_active":"2023-09-27T23:50:34.558745+0000","last_became_peered":"2023-09-27T23:50:34.558745+0000","last_unstale":"2023-09-27T23:50:34.565799+0000","last_undegraded":"2023-09-27T23:50:34.565799+0000","last_fullsized":"2023-09-27T23:50:34.565799+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:32:35.695537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566688+0000","last_change":"2023-09-27T23:50:34.171899+0000","last_active":"2023-09-27T23:50:34.566688+0000","last_peered":"2023-09-27T23:50:34.566688+0000","last_clean":"2023-09-27T23:50:34.566688+0000","last_became_active":"2023-09-27T23:50:34.171630+0000","last_became_peered":"2023-09-27T23:50:34.171630+0000","last_unstale":"2023-09-27T23:50:34.566688+0000","last_undegraded":"2023-09-27T23:50:34.566688+0000","last_fullsized":"2023-09-27T23:50:34.566688+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:39.245022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.556951+0000","last_change":"2023-09-27T23:50:34.547941+0000","last_active":"2023-09-27T23:50:34.556951+0000","last_peered":"2023-09-27T23:50:34.556951+0000","last_clean":"2023-09-27T23:50:34.556951+0000","last_became_active":"2023-09-27T23:50:34.547658+0000","last_became_peered":"2023-09-27T23:50:34.547658+0000","last_unstale":"2023-09-27T23:50:34.556951+0000","last_undegraded":"2023-09-27T23:50:34.556951+0000","last_fullsized":"2023-09-27T23:50:34.556951+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:40:34.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.560146+0000","last_change":"2023-09-27T23:50:34.172268+0000","last_active":"2023-09-27T23:50:34.560146+0000","last_peered":"2023-09-27T23:50:34.560146+0000","last_clean":"2023-09-27T23:50:34.560146+0000","last_became_active":"2023-09-27T23:50:34.171691+0000","last_became_peered":"2023-09-27T23:50:34.171691+0000","last_unstale":"2023-09-27T23:50:34.560146+0000","last_undegraded":"2023-09-27T23:50:34.560146+0000","last_fullsized":"2023-09-27T23:50:34.560146+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:54:32.568632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562617+0000","last_change":"2023-09-27T23:50:34.556244+0000","last_active":"2023-09-27T23:50:34.562617+0000","last_peered":"2023-09-27T23:50:34.562617+0000","last_clean":"2023-09-27T23:50:34.562617+0000","last_became_active":"2023-09-27T23:50:34.555302+0000","last_became_peered":"2023-09-27T23:50:34.555302+0000","last_unstale":"2023-09-27T23:50:34.562617+0000","last_undegraded":"2023-09-27T23:50:34.562617+0000","last_fullsized":"2023-09-27T23:50:34.562617+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:07:49.146420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.566343+0000","last_change":"2023-09-27T23:50:34.174024+0000","last_active":"2023-09-27T23:50:34.566343+0000","last_peered":"2023-09-27T23:50:34.566343+0000","last_clean":"2023-09-27T23:50:34.566343+0000","last_became_active":"2023-09-27T23:50:34.173678+0000","last_became_peered":"2023-09-27T23:50:34.173678+0000","last_unstale":"2023-09-27T23:50:34.566343+0000","last_undegraded":"2023-09-27T23:50:34.566343+0000","last_fullsized":"2023-09-27T23:50:34.566343+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:17.655398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.561417+0000","last_change":"2023-09-27T23:50:34.548919+0000","last_active":"2023-09-27T23:50:34.561417+0000","last_peered":"2023-09-27T23:50:34.561417+0000","last_clean":"2023-09-27T23:50:34.561417+0000","last_became_active":"2023-09-27T23:50:34.548568+0000","last_became_peered":"2023-09-27T23:50:34.548568+0000","last_unstale":"2023-09-27T23:50:34.561417+0000","last_undegraded":"2023-09-27T23:50:34.561417+0000","last_fullsized":"2023-09-27T23:50:34.561417+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:32:35.695537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.554327+0000","last_change":"2023-09-27T23:50:34.184176+0000","last_active":"2023-09-27T23:50:34.554327+0000","last_peered":"2023-09-27T23:50:34.554327+0000","last_clean":"2023-09-27T23:50:34.554327+0000","last_became_active":"2023-09-27T23:50:34.183738+0000","last_became_peered":"2023-09-27T23:50:34.183738+0000","last_unstale":"2023-09-27T23:50:34.554327+0000","last_undegraded":"2023-09-27T23:50:34.554327+0000","last_fullsized":"2023-09-27T23:50:34.554327+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:56:51.376598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.558743+0000","last_change":"2023-09-27T23:50:34.551680+0000","last_active":"2023-09-27T23:50:34.558743+0000","last_peered":"2023-09-27T23:50:34.558743+0000","last_clean":"2023-09-27T23:50:34.558743+0000","last_became_active":"2023-09-27T23:50:34.551461+0000","last_became_peered":"2023-09-27T23:50:34.551461+0000","last_unstale":"2023-09-27T23:50:34.558743+0000","last_undegraded":"2023-09-27T23:50:34.558743+0000","last_fullsized":"2023-09-27T23:50:34.558743+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:41:16.104461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.544066+0000","last_change":"2023-09-27T23:50:34.172167+0000","last_active":"2023-09-27T23:50:34.544066+0000","last_peered":"2023-09-27T23:50:34.544066+0000","last_clean":"2023-09-27T23:50:34.544066+0000","last_became_active":"2023-09-27T23:50:34.171847+0000","last_became_peered":"2023-09-27T23:50:34.171847+0000","last_unstale":"2023-09-27T23:50:34.544066+0000","last_undegraded":"2023-09-27T23:50:34.544066+0000","last_fullsized":"2023-09-27T23:50:34.544066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:03:38.120967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562007+0000","last_change":"2023-09-27T23:50:34.552662+0000","last_active":"2023-09-27T23:50:34.562007+0000","last_peered":"2023-09-27T23:50:34.562007+0000","last_clean":"2023-09-27T23:50:34.562007+0000","last_became_active":"2023-09-27T23:50:34.552380+0000","last_became_peered":"2023-09-27T23:50:34.552380+0000","last_unstale":"2023-09-27T23:50:34.562007+0000","last_undegraded":"2023-09-27T23:50:34.562007+0000","last_fullsized":"2023-09-27T23:50:34.562007+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:53.905465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.546907+0000","last_change":"2023-09-27T23:50:34.170798+0000","last_active":"2023-09-27T23:50:34.546907+0000","last_peered":"2023-09-27T23:50:34.546907+0000","last_clean":"2023-09-27T23:50:34.546907+0000","last_became_active":"2023-09-27T23:50:34.170528+0000","last_became_peered":"2023-09-27T23:50:34.170528+0000","last_unstale":"2023-09-27T23:50:34.546907+0000","last_undegraded":"2023-09-27T23:50:34.546907+0000","last_fullsized":"2023-09-27T23:50:34.546907+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:13.637097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557727+0000","last_change":"2023-09-27T23:50:34.550712+0000","last_active":"2023-09-27T23:50:34.557727+0000","last_peered":"2023-09-27T23:50:34.557727+0000","last_clean":"2023-09-27T23:50:34.557727+0000","last_became_active":"2023-09-27T23:50:34.550415+0000","last_became_peered":"2023-09-27T23:50:34.550415+0000","last_unstale":"2023-09-27T23:50:34.557727+0000","last_undegraded":"2023-09-27T23:50:34.557727+0000","last_fullsized":"2023-09-27T23:50:34.557727+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:46:29.884638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.544516+0000","last_change":"2023-09-27T23:50:34.171801+0000","last_active":"2023-09-27T23:50:34.544516+0000","last_peered":"2023-09-27T23:50:34.544516+0000","last_clean":"2023-09-27T23:50:34.544516+0000","last_became_active":"2023-09-27T23:50:34.171494+0000","last_became_peered":"2023-09-27T23:50:34.171494+0000","last_unstale":"2023-09-27T23:50:34.544516+0000","last_undegraded":"2023-09-27T23:50:34.544516+0000","last_fullsized":"2023-09-27T23:50:34.544516+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:28:23.297242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562895+0000","last_change":"2023-09-27T23:50:34.557380+0000","last_active":"2023-09-27T23:50:34.562895+0000","last_peered":"2023-09-27T23:50:34.562895+0000","last_clean":"2023-09-27T23:50:34.562895+0000","last_became_active":"2023-09-27T23:50:34.557154+0000","last_became_peered":"2023-09-27T23:50:34.557154+0000","last_unstale":"2023-09-27T23:50:34.562895+0000","last_undegraded":"2023-09-27T23:50:34.562895+0000","last_fullsized":"2023-09-27T23:50:34.562895+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:43:59.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.561545+0000","last_change":"2023-09-27T23:50:34.183896+0000","last_active":"2023-09-27T23:50:34.561545+0000","last_peered":"2023-09-27T23:50:34.561545+0000","last_clean":"2023-09-27T23:50:34.561545+0000","last_became_active":"2023-09-27T23:50:34.183535+0000","last_became_peered":"2023-09-27T23:50:34.183535+0000","last_unstale":"2023-09-27T23:50:34.561545+0000","last_undegraded":"2023-09-27T23:50:34.561545+0000","last_fullsized":"2023-09-27T23:50:34.561545+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:29:52.638295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.554015+0000","last_change":"2023-09-27T23:50:34.551237+0000","last_active":"2023-09-27T23:50:34.554015+0000","last_peered":"2023-09-27T23:50:34.554015+0000","last_clean":"2023-09-27T23:50:34.554015+0000","last_became_active":"2023-09-27T23:50:34.550972+0000","last_became_peered":"2023-09-27T23:50:34.550972+0000","last_unstale":"2023-09-27T23:50:34.554015+0000","last_undegraded":"2023-09-27T23:50:34.554015+0000","last_fullsized":"2023-09-27T23:50:34.554015+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:27:11.741782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.554454+0000","last_change":"2023-09-27T23:50:34.184001+0000","last_active":"2023-09-27T23:50:34.554454+0000","last_peered":"2023-09-27T23:50:34.554454+0000","last_clean":"2023-09-27T23:50:34.554454+0000","last_became_active":"2023-09-27T23:50:34.183650+0000","last_became_peered":"2023-09-27T23:50:34.183650+0000","last_unstale":"2023-09-27T23:50:34.554454+0000","last_undegraded":"2023-09-27T23:50:34.554454+0000","last_fullsized":"2023-09-27T23:50:34.554454+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:07:59.145229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562567+0000","last_change":"2023-09-27T23:50:34.554086+0000","last_active":"2023-09-27T23:50:34.562567+0000","last_peered":"2023-09-27T23:50:34.562567+0000","last_clean":"2023-09-27T23:50:34.562567+0000","last_became_active":"2023-09-27T23:50:34.552464+0000","last_became_peered":"2023-09-27T23:50:34.552464+0000","last_unstale":"2023-09-27T23:50:34.562567+0000","last_undegraded":"2023-09-27T23:50:34.562567+0000","last_fullsized":"2023-09-27T23:50:34.562567+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:43:43.152499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.559124+0000","last_change":"2023-09-27T23:50:34.178593+0000","last_active":"2023-09-27T23:50:34.559124+0000","last_peered":"2023-09-27T23:50:34.559124+0000","last_clean":"2023-09-27T23:50:34.559124+0000","last_became_active":"2023-09-27T23:50:34.178320+0000","last_became_peered":"2023-09-27T23:50:34.178320+0000","last_unstale":"2023-09-27T23:50:34.559124+0000","last_undegraded":"2023-09-27T23:50:34.559124+0000","last_fullsized":"2023-09-27T23:50:34.559124+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:56:46.192596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567284+0000","last_change":"2023-09-27T23:50:34.561852+0000","last_active":"2023-09-27T23:50:34.567284+0000","last_peered":"2023-09-27T23:50:34.567284+0000","last_clean":"2023-09-27T23:50:34.567284+0000","last_became_active":"2023-09-27T23:50:34.561567+0000","last_became_peered":"2023-09-27T23:50:34.561567+0000","last_unstale":"2023-09-27T23:50:34.567284+0000","last_undegraded":"2023-09-27T23:50:34.567284+0000","last_fullsized":"2023-09-27T23:50:34.567284+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:02:04.216295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.544712+0000","last_change":"2023-09-27T23:50:34.173943+0000","last_active":"2023-09-27T23:50:34.544712+0000","last_peered":"2023-09-27T23:50:34.544712+0000","last_clean":"2023-09-27T23:50:34.544712+0000","last_became_active":"2023-09-27T23:50:34.173672+0000","last_became_peered":"2023-09-27T23:50:34.173672+0000","last_unstale":"2023-09-27T23:50:34.544712+0000","last_undegraded":"2023-09-27T23:50:34.544712+0000","last_fullsized":"2023-09-27T23:50:34.544712+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:21:29.118808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562331+0000","last_change":"2023-09-27T23:50:34.550713+0000","last_active":"2023-09-27T23:50:34.562331+0000","last_peered":"2023-09-27T23:50:34.562331+0000","last_clean":"2023-09-27T23:50:34.562331+0000","last_became_active":"2023-09-27T23:50:34.550350+0000","last_became_peered":"2023-09-27T23:50:34.550350+0000","last_unstale":"2023-09-27T23:50:34.562331+0000","last_undegraded":"2023-09-27T23:50:34.562331+0000","last_fullsized":"2023-09-27T23:50:34.562331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:11.190147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565130+0000","last_change":"2023-09-27T23:50:34.176646+0000","last_active":"2023-09-27T23:50:34.565130+0000","last_peered":"2023-09-27T23:50:34.565130+0000","last_clean":"2023-09-27T23:50:34.565130+0000","last_became_active":"2023-09-27T23:50:34.175097+0000","last_became_peered":"2023-09-27T23:50:34.175097+0000","last_unstale":"2023-09-27T23:50:34.565130+0000","last_undegraded":"2023-09-27T23:50:34.565130+0000","last_fullsized":"2023-09-27T23:50:34.565130+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:20:31.754486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564856+0000","last_change":"2023-09-27T23:50:34.553653+0000","last_active":"2023-09-27T23:50:34.564856+0000","last_peered":"2023-09-27T23:50:34.564856+0000","last_clean":"2023-09-27T23:50:34.564856+0000","last_became_active":"2023-09-27T23:50:34.553141+0000","last_became_peered":"2023-09-27T23:50:34.553141+0000","last_unstale":"2023-09-27T23:50:34.564856+0000","last_undegraded":"2023-09-27T23:50:34.564856+0000","last_fullsized":"2023-09-27T23:50:34.564856+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:33:00.188060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542595+0000","last_change":"2023-09-27T23:50:34.170080+0000","last_active":"2023-09-27T23:50:34.542595+0000","last_peered":"2023-09-27T23:50:34.542595+0000","last_clean":"2023-09-27T23:50:34.542595+0000","last_became_active":"2023-09-27T23:50:34.169718+0000","last_became_peered":"2023-09-27T23:50:34.169718+0000","last_unstale":"2023-09-27T23:50:34.542595+0000","last_undegraded":"2023-09-27T23:50:34.542595+0000","last_fullsized":"2023-09-27T23:50:34.542595+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:18:53.060128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.569732+0000","last_change":"2023-09-27T23:50:34.567762+0000","last_active":"2023-09-27T23:50:34.569732+0000","last_peered":"2023-09-27T23:50:34.569732+0000","last_clean":"2023-09-27T23:50:34.569732+0000","last_became_active":"2023-09-27T23:50:34.567433+0000","last_became_peered":"2023-09-27T23:50:34.567433+0000","last_unstale":"2023-09-27T23:50:34.569732+0000","last_undegraded":"2023-09-27T23:50:34.569732+0000","last_fullsized":"2023-09-27T23:50:34.569732+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:29:13.632950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.536248+0000","last_change":"2023-09-27T23:50:34.182887+0000","last_active":"2023-09-27T23:50:34.536248+0000","last_peered":"2023-09-27T23:50:34.536248+0000","last_clean":"2023-09-27T23:50:34.536248+0000","last_became_active":"2023-09-27T23:50:34.182514+0000","last_became_peered":"2023-09-27T23:50:34.182514+0000","last_unstale":"2023-09-27T23:50:34.536248+0000","last_undegraded":"2023-09-27T23:50:34.536248+0000","last_fullsized":"2023-09-27T23:50:34.536248+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:16:45.217748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565822+0000","last_change":"2023-09-27T23:50:34.560465+0000","last_active":"2023-09-27T23:50:34.565822+0000","last_peered":"2023-09-27T23:50:34.565822+0000","last_clean":"2023-09-27T23:50:34.565822+0000","last_became_active":"2023-09-27T23:50:34.560146+0000","last_became_peered":"2023-09-27T23:50:34.560146+0000","last_unstale":"2023-09-27T23:50:34.565822+0000","last_undegraded":"2023-09-27T23:50:34.565822+0000","last_fullsized":"2023-09-27T23:50:34.565822+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:58:00.236964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542213+0000","last_change":"2023-09-27T23:50:34.169977+0000","last_active":"2023-09-27T23:50:34.542213+0000","last_peered":"2023-09-27T23:50:34.542213+0000","last_clean":"2023-09-27T23:50:34.542213+0000","last_became_active":"2023-09-27T23:50:34.169371+0000","last_became_peered":"2023-09-27T23:50:34.169371+0000","last_unstale":"2023-09-27T23:50:34.542213+0000","last_undegraded":"2023-09-27T23:50:34.542213+0000","last_fullsized":"2023-09-27T23:50:34.542213+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:11:56.481823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.552038+0000","last_change":"2023-09-27T23:50:34.548580+0000","last_active":"2023-09-27T23:50:34.552038+0000","last_peered":"2023-09-27T23:50:34.552038+0000","last_clean":"2023-09-27T23:50:34.552038+0000","last_became_active":"2023-09-27T23:50:34.548257+0000","last_became_peered":"2023-09-27T23:50:34.548257+0000","last_unstale":"2023-09-27T23:50:34.552038+0000","last_undegraded":"2023-09-27T23:50:34.552038+0000","last_fullsized":"2023-09-27T23:50:34.552038+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:01:54.141241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.560217+0000","last_change":"2023-09-27T23:50:34.173122+0000","last_active":"2023-09-27T23:50:34.560217+0000","last_peered":"2023-09-27T23:50:34.560217+0000","last_clean":"2023-09-27T23:50:34.560217+0000","last_became_active":"2023-09-27T23:50:34.172766+0000","last_became_peered":"2023-09-27T23:50:34.172766+0000","last_unstale":"2023-09-27T23:50:34.560217+0000","last_undegraded":"2023-09-27T23:50:34.560217+0000","last_fullsized":"2023-09-27T23:50:34.560217+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:15:50.280393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.567332+0000","last_change":"2023-09-27T23:50:34.561897+0000","last_active":"2023-09-27T23:50:34.567332+0000","last_peered":"2023-09-27T23:50:34.567332+0000","last_clean":"2023-09-27T23:50:34.567332+0000","last_became_active":"2023-09-27T23:50:34.561596+0000","last_became_peered":"2023-09-27T23:50:34.561596+0000","last_unstale":"2023-09-27T23:50:34.567332+0000","last_undegraded":"2023-09-27T23:50:34.567332+0000","last_fullsized":"2023-09-27T23:50:34.567332+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:04:21.898058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.539051+0000","last_change":"2023-09-27T23:50:34.183999+0000","last_active":"2023-09-27T23:50:34.539051+0000","last_peered":"2023-09-27T23:50:34.539051+0000","last_clean":"2023-09-27T23:50:34.539051+0000","last_became_active":"2023-09-27T23:50:34.183618+0000","last_became_peered":"2023-09-27T23:50:34.183618+0000","last_unstale":"2023-09-27T23:50:34.539051+0000","last_undegraded":"2023-09-27T23:50:34.539051+0000","last_fullsized":"2023-09-27T23:50:34.539051+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:54:47.011915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557303+0000","last_change":"2023-09-27T23:50:34.550011+0000","last_active":"2023-09-27T23:50:34.557303+0000","last_peered":"2023-09-27T23:50:34.557303+0000","last_clean":"2023-09-27T23:50:34.557303+0000","last_became_active":"2023-09-27T23:50:34.549815+0000","last_became_peered":"2023-09-27T23:50:34.549815+0000","last_unstale":"2023-09-27T23:50:34.557303+0000","last_undegraded":"2023-09-27T23:50:34.557303+0000","last_fullsized":"2023-09-27T23:50:34.557303+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:21:05.704176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.541359+0000","last_change":"2023-09-27T23:50:34.173955+0000","last_active":"2023-09-27T23:50:34.541359+0000","last_peered":"2023-09-27T23:50:34.541359+0000","last_clean":"2023-09-27T23:50:34.541359+0000","last_became_active":"2023-09-27T23:50:34.173650+0000","last_became_peered":"2023-09-27T23:50:34.173650+0000","last_unstale":"2023-09-27T23:50:34.541359+0000","last_undegraded":"2023-09-27T23:50:34.541359+0000","last_fullsized":"2023-09-27T23:50:34.541359+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:59:22.919228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.549090+0000","last_change":"2023-09-27T23:50:34.545479+0000","last_active":"2023-09-27T23:50:34.549090+0000","last_peered":"2023-09-27T23:50:34.549090+0000","last_clean":"2023-09-27T23:50:34.549090+0000","last_became_active":"2023-09-27T23:50:34.545208+0000","last_became_peered":"2023-09-27T23:50:34.545208+0000","last_unstale":"2023-09-27T23:50:34.549090+0000","last_undegraded":"2023-09-27T23:50:34.549090+0000","last_fullsized":"2023-09-27T23:50:34.549090+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:10:27.462843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.539125+0000","last_change":"2023-09-27T23:50:34.182805+0000","last_active":"2023-09-27T23:50:34.539125+0000","last_peered":"2023-09-27T23:50:34.539125+0000","last_clean":"2023-09-27T23:50:34.539125+0000","last_became_active":"2023-09-27T23:50:34.182453+0000","last_became_peered":"2023-09-27T23:50:34.182453+0000","last_unstale":"2023-09-27T23:50:34.539125+0000","last_undegraded":"2023-09-27T23:50:34.539125+0000","last_fullsized":"2023-09-27T23:50:34.539125+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:13.108314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.555923+0000","last_change":"2023-09-27T23:50:34.552924+0000","last_active":"2023-09-27T23:50:34.555923+0000","last_peered":"2023-09-27T23:50:34.555923+0000","last_clean":"2023-09-27T23:50:34.555923+0000","last_became_active":"2023-09-27T23:50:34.552657+0000","last_became_peered":"2023-09-27T23:50:34.552657+0000","last_unstale":"2023-09-27T23:50:34.555923+0000","last_undegraded":"2023-09-27T23:50:34.555923+0000","last_fullsized":"2023-09-27T23:50:34.555923+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:38:24.228982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.547450+0000","last_change":"2023-09-27T23:50:34.173743+0000","last_active":"2023-09-27T23:50:34.547450+0000","last_peered":"2023-09-27T23:50:34.547450+0000","last_clean":"2023-09-27T23:50:34.547450+0000","last_became_active":"2023-09-27T23:50:34.173439+0000","last_became_peered":"2023-09-27T23:50:34.173439+0000","last_unstale":"2023-09-27T23:50:34.547450+0000","last_undegraded":"2023-09-27T23:50:34.547450+0000","last_fullsized":"2023-09-27T23:50:34.547450+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:35:49.393660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.563657+0000","last_change":"2023-09-27T23:50:34.552914+0000","last_active":"2023-09-27T23:50:34.563657+0000","last_peered":"2023-09-27T23:50:34.563657+0000","last_clean":"2023-09-27T23:50:34.563657+0000","last_became_active":"2023-09-27T23:50:34.552670+0000","last_became_peered":"2023-09-27T23:50:34.552670+0000","last_unstale":"2023-09-27T23:50:34.563657+0000","last_undegraded":"2023-09-27T23:50:34.563657+0000","last_fullsized":"2023-09-27T23:50:34.563657+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:22:07.395190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.535326+0000","last_change":"2023-09-27T23:50:34.167715+0000","last_active":"2023-09-27T23:50:34.535326+0000","last_peered":"2023-09-27T23:50:34.535326+0000","last_clean":"2023-09-27T23:50:34.535326+0000","last_became_active":"2023-09-27T23:50:34.167114+0000","last_became_peered":"2023-09-27T23:50:34.167114+0000","last_unstale":"2023-09-27T23:50:34.535326+0000","last_undegraded":"2023-09-27T23:50:34.535326+0000","last_fullsized":"2023-09-27T23:50:34.535326+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:59:40.239632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564722+0000","last_change":"2023-09-27T23:50:34.552691+0000","last_active":"2023-09-27T23:50:34.564722+0000","last_peered":"2023-09-27T23:50:34.564722+0000","last_clean":"2023-09-27T23:50:34.564722+0000","last_became_active":"2023-09-27T23:50:34.552424+0000","last_became_peered":"2023-09-27T23:50:34.552424+0000","last_unstale":"2023-09-27T23:50:34.564722+0000","last_undegraded":"2023-09-27T23:50:34.564722+0000","last_fullsized":"2023-09-27T23:50:34.564722+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:21:43.479240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.547020+0000","last_change":"2023-09-27T23:50:34.170730+0000","last_active":"2023-09-27T23:50:34.547020+0000","last_peered":"2023-09-27T23:50:34.547020+0000","last_clean":"2023-09-27T23:50:34.547020+0000","last_became_active":"2023-09-27T23:50:34.170447+0000","last_became_peered":"2023-09-27T23:50:34.170447+0000","last_unstale":"2023-09-27T23:50:34.547020+0000","last_undegraded":"2023-09-27T23:50:34.547020+0000","last_fullsized":"2023-09-27T23:50:34.547020+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:07:48.134682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565337+0000","last_change":"2023-09-27T23:50:34.560112+0000","last_active":"2023-09-27T23:50:34.565337+0000","last_peered":"2023-09-27T23:50:34.565337+0000","last_clean":"2023-09-27T23:50:34.565337+0000","last_became_active":"2023-09-27T23:50:34.559812+0000","last_became_peered":"2023-09-27T23:50:34.559812+0000","last_unstale":"2023-09-27T23:50:34.565337+0000","last_undegraded":"2023-09-27T23:50:34.565337+0000","last_fullsized":"2023-09-27T23:50:34.565337+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:23:14.648835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.549412+0000","last_change":"2023-09-27T23:50:34.174716+0000","last_active":"2023-09-27T23:50:34.549412+0000","last_peered":"2023-09-27T23:50:34.549412+0000","last_clean":"2023-09-27T23:50:34.549412+0000","last_became_active":"2023-09-27T23:50:34.174450+0000","last_became_peered":"2023-09-27T23:50:34.174450+0000","last_unstale":"2023-09-27T23:50:34.549412+0000","last_undegraded":"2023-09-27T23:50:34.549412+0000","last_fullsized":"2023-09-27T23:50:34.549412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:32:56.183670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.552060+0000","last_change":"2023-09-27T23:50:34.548838+0000","last_active":"2023-09-27T23:50:34.552060+0000","last_peered":"2023-09-27T23:50:34.552060+0000","last_clean":"2023-09-27T23:50:34.552060+0000","last_became_active":"2023-09-27T23:50:34.548290+0000","last_became_peered":"2023-09-27T23:50:34.548290+0000","last_unstale":"2023-09-27T23:50:34.552060+0000","last_undegraded":"2023-09-27T23:50:34.552060+0000","last_fullsized":"2023-09-27T23:50:34.552060+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:57:49.885313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.548651+0000","last_change":"2023-09-27T23:50:34.169552+0000","last_active":"2023-09-27T23:50:34.548651+0000","last_peered":"2023-09-27T23:50:34.548651+0000","last_clean":"2023-09-27T23:50:34.548651+0000","last_became_active":"2023-09-27T23:50:34.169229+0000","last_became_peered":"2023-09-27T23:50:34.169229+0000","last_unstale":"2023-09-27T23:50:34.548651+0000","last_undegraded":"2023-09-27T23:50:34.548651+0000","last_fullsized":"2023-09-27T23:50:34.548651+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:07:39.032105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"162'304","reported_seq":960,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542422+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:50:34.542422+0000","last_peered":"2023-09-27T23:50:34.542422+0000","last_clean":"2023-09-27T23:50:34.542422+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:50:34.542422+0000","last_undegraded":"2023-09-27T23:50:34.542422+0000","last_fullsized":"2023-09-27T23:50:34.542422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":304,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.538792+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:50:34.538792+0000","last_peered":"2023-09-27T23:50:34.538792+0000","last_clean":"2023-09-27T23:50:34.538792+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:50:34.538792+0000","last_undegraded":"2023-09-27T23:50:34.538792+0000","last_fullsized":"2023-09-27T23:50:34.538792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.563638+0000","last_change":"2023-09-27T23:50:34.555347+0000","last_active":"2023-09-27T23:50:34.563638+0000","last_peered":"2023-09-27T23:50:34.563638+0000","last_clean":"2023-09-27T23:50:34.563638+0000","last_became_active":"2023-09-27T23:50:34.555039+0000","last_became_peered":"2023-09-27T23:50:34.555039+0000","last_unstale":"2023-09-27T23:50:34.563638+0000","last_undegraded":"2023-09-27T23:50:34.563638+0000","last_fullsized":"2023-09-27T23:50:34.563638+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:02:04.216295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"173'6","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.570300+0000","last_change":"2023-09-27T23:50:34.172631+0000","last_active":"2023-09-27T23:50:34.570300+0000","last_peered":"2023-09-27T23:50:34.570300+0000","last_clean":"2023-09-27T23:50:34.570300+0000","last_became_active":"2023-09-27T23:50:34.172362+0000","last_became_peered":"2023-09-27T23:50:34.172362+0000","last_unstale":"2023-09-27T23:50:34.570300+0000","last_undegraded":"2023-09-27T23:50:34.570300+0000","last_fullsized":"2023-09-27T23:50:34.570300+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:18:25.363148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542322+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:50:34.542322+0000","last_peered":"2023-09-27T23:50:34.542322+0000","last_clean":"2023-09-27T23:50:34.542322+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:50:34.542322+0000","last_undegraded":"2023-09-27T23:50:34.542322+0000","last_fullsized":"2023-09-27T23:50:34.542322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.557740+0000","last_change":"2023-09-27T23:50:34.555682+0000","last_active":"2023-09-27T23:50:34.557740+0000","last_peered":"2023-09-27T23:50:34.557740+0000","last_clean":"2023-09-27T23:50:34.557740+0000","last_became_active":"2023-09-27T23:50:34.555243+0000","last_became_peered":"2023-09-27T23:50:34.555243+0000","last_unstale":"2023-09-27T23:50:34.557740+0000","last_undegraded":"2023-09-27T23:50:34.557740+0000","last_fullsized":"2023-09-27T23:50:34.557740+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:15:33.637539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"173'3","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562306+0000","last_change":"2023-09-27T23:50:34.164815+0000","last_active":"2023-09-27T23:50:34.562306+0000","last_peered":"2023-09-27T23:50:34.562306+0000","last_clean":"2023-09-27T23:50:34.562306+0000","last_became_active":"2023-09-27T23:50:34.164485+0000","last_became_peered":"2023-09-27T23:50:34.164485+0000","last_unstale":"2023-09-27T23:50:34.562306+0000","last_undegraded":"2023-09-27T23:50:34.562306+0000","last_fullsized":"2023-09-27T23:50:34.562306+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:39:10.178409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.530465+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:50:34.530465+0000","last_peered":"2023-09-27T23:50:34.530465+0000","last_clean":"2023-09-27T23:50:34.530465+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:50:34.530465+0000","last_undegraded":"2023-09-27T23:50:34.530465+0000","last_fullsized":"2023-09-27T23:50:34.530465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.552089+0000","last_change":"2023-09-27T23:50:34.547700+0000","last_active":"2023-09-27T23:50:34.552089+0000","last_peered":"2023-09-27T23:50:34.552089+0000","last_clean":"2023-09-27T23:50:34.552089+0000","last_became_active":"2023-09-27T23:50:34.547437+0000","last_became_peered":"2023-09-27T23:50:34.547437+0000","last_unstale":"2023-09-27T23:50:34.552089+0000","last_undegraded":"2023-09-27T23:50:34.552089+0000","last_fullsized":"2023-09-27T23:50:34.552089+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:29:13.632950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.530828+0000","last_change":"2023-09-27T23:50:34.183336+0000","last_active":"2023-09-27T23:50:34.530828+0000","last_peered":"2023-09-27T23:50:34.530828+0000","last_clean":"2023-09-27T23:50:34.530828+0000","last_became_active":"2023-09-27T23:50:34.182941+0000","last_became_peered":"2023-09-27T23:50:34.182941+0000","last_unstale":"2023-09-27T23:50:34.530828+0000","last_undegraded":"2023-09-27T23:50:34.530828+0000","last_fullsized":"2023-09-27T23:50:34.530828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:51:00.466792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.538946+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:50:34.538946+0000","last_peered":"2023-09-27T23:50:34.538946+0000","last_clean":"2023-09-27T23:50:34.538946+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:50:34.538946+0000","last_undegraded":"2023-09-27T23:50:34.538946+0000","last_fullsized":"2023-09-27T23:50:34.538946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562397+0000","last_change":"2023-09-27T23:50:34.550653+0000","last_active":"2023-09-27T23:50:34.562397+0000","last_peered":"2023-09-27T23:50:34.562397+0000","last_clean":"2023-09-27T23:50:34.562397+0000","last_became_active":"2023-09-27T23:50:34.550290+0000","last_became_peered":"2023-09-27T23:50:34.550290+0000","last_unstale":"2023-09-27T23:50:34.562397+0000","last_undegraded":"2023-09-27T23:50:34.562397+0000","last_fullsized":"2023-09-27T23:50:34.562397+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:07:18.419504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"173'8","reported_seq":18,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.568003+0000","last_change":"2023-09-27T23:50:34.176694+0000","last_active":"2023-09-27T23:50:34.568003+0000","last_peered":"2023-09-27T23:50:34.568003+0000","last_clean":"2023-09-27T23:50:34.568003+0000","last_became_active":"2023-09-27T23:50:34.176430+0000","last_became_peered":"2023-09-27T23:50:34.176430+0000","last_unstale":"2023-09-27T23:50:34.568003+0000","last_undegraded":"2023-09-27T23:50:34.568003+0000","last_fullsized":"2023-09-27T23:50:34.568003+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:47:29.373167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.539964+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:50:34.539964+0000","last_peered":"2023-09-27T23:50:34.539964+0000","last_clean":"2023-09-27T23:50:34.539964+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:50:34.539964+0000","last_undegraded":"2023-09-27T23:50:34.539964+0000","last_fullsized":"2023-09-27T23:50:34.539964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.562598+0000","last_change":"2023-09-27T23:50:34.553645+0000","last_active":"2023-09-27T23:50:34.562598+0000","last_peered":"2023-09-27T23:50:34.562598+0000","last_clean":"2023-09-27T23:50:34.562598+0000","last_became_active":"2023-09-27T23:50:34.553380+0000","last_became_peered":"2023-09-27T23:50:34.553380+0000","last_unstale":"2023-09-27T23:50:34.562598+0000","last_undegraded":"2023-09-27T23:50:34.562598+0000","last_fullsized":"2023-09-27T23:50:34.562598+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:53:07.240729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.536137+0000","last_change":"2023-09-27T23:50:34.172169+0000","last_active":"2023-09-27T23:50:34.536137+0000","last_peered":"2023-09-27T23:50:34.536137+0000","last_clean":"2023-09-27T23:50:34.536137+0000","last_became_active":"2023-09-27T23:50:34.171760+0000","last_became_peered":"2023-09-27T23:50:34.171760+0000","last_unstale":"2023-09-27T23:50:34.536137+0000","last_undegraded":"2023-09-27T23:50:34.536137+0000","last_fullsized":"2023-09-27T23:50:34.536137+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:28:53.536449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.542926+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:50:34.542926+0000","last_peered":"2023-09-27T23:50:34.542926+0000","last_clean":"2023-09-27T23:50:34.542926+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:50:34.542926+0000","last_undegraded":"2023-09-27T23:50:34.542926+0000","last_fullsized":"2023-09-27T23:50:34.542926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.564875+0000","last_change":"2023-09-27T23:50:34.554195+0000","last_active":"2023-09-27T23:50:34.564875+0000","last_peered":"2023-09-27T23:50:34.564875+0000","last_clean":"2023-09-27T23:50:34.564875+0000","last_became_active":"2023-09-27T23:50:34.553917+0000","last_became_peered":"2023-09-27T23:50:34.553917+0000","last_unstale":"2023-09-27T23:50:34.564875+0000","last_undegraded":"2023-09-27T23:50:34.564875+0000","last_fullsized":"2023-09-27T23:50:34.564875+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:22:35.188618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.538483+0000","last_change":"2023-09-27T23:50:34.173869+0000","last_active":"2023-09-27T23:50:34.538483+0000","last_peered":"2023-09-27T23:50:34.538483+0000","last_clean":"2023-09-27T23:50:34.538483+0000","last_became_active":"2023-09-27T23:50:34.173576+0000","last_became_peered":"2023-09-27T23:50:34.173576+0000","last_unstale":"2023-09-27T23:50:34.538483+0000","last_undegraded":"2023-09-27T23:50:34.538483+0000","last_fullsized":"2023-09-27T23:50:34.538483+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:36:37.863006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.541006+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:50:34.541006+0000","last_peered":"2023-09-27T23:50:34.541006+0000","last_clean":"2023-09-27T23:50:34.541006+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:50:34.541006+0000","last_undegraded":"2023-09-27T23:50:34.541006+0000","last_fullsized":"2023-09-27T23:50:34.541006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.565485+0000","last_change":"2023-09-27T23:50:34.560491+0000","last_active":"2023-09-27T23:50:34.565485+0000","last_peered":"2023-09-27T23:50:34.565485+0000","last_clean":"2023-09-27T23:50:34.565485+0000","last_became_active":"2023-09-27T23:50:34.560184+0000","last_became_peered":"2023-09-27T23:50:34.560184+0000","last_unstale":"2023-09-27T23:50:34.565485+0000","last_undegraded":"2023-09-27T23:50:34.565485+0000","last_fullsized":"2023-09-27T23:50:34.565485+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:34.140545+0000","last_clean_scrub_stamp":"2023-09-27T23:50:34.140545+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:56:51.904785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"173'2","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.558616+0000","last_change":"2023-09-27T23:50:34.170530+0000","last_active":"2023-09-27T23:50:34.558616+0000","last_peered":"2023-09-27T23:50:34.558616+0000","last_clean":"2023-09-27T23:50:34.558616+0000","last_became_active":"2023-09-27T23:50:34.170207+0000","last_became_peered":"2023-09-27T23:50:34.170207+0000","last_unstale":"2023-09-27T23:50:34.558616+0000","last_undegraded":"2023-09-27T23:50:34.558616+0000","last_fullsized":"2023-09-27T23:50:34.558616+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:50:33.128807+0000","last_clean_scrub_stamp":"2023-09-27T23:50:33.128807+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:41:22.504977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-09-27T23:50:34.540860+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:50:34.540860+0000","last_peered":"2023-09-27T23:50:34.540860+0000","last_clean":"2023-09-27T23:50:34.540860+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:50:34.540860+0000","last_undegraded":"2023-09-27T23:50:34.540860+0000","last_fullsized":"2023-09-27T23:50:34.540860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35688,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":32960,"kb_avail":94336152,"statfs":{"total":96636764160,"available":96600219648,"internally_reserved":0,"allocated":2785280,"data_stored":1732437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":33751040},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.90300000000000002,"15min":0.90000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.873}]},{"osd":1,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.90000000000000002},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57199999999999995},"max":{"1min":1.3,"5min":1.333,"15min":1.3360000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.97299999999999998,"15min":0.97599999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.1019999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.91600000000000004,"15min":0.93000000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1799999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.98799999999999999,"15min":0.96599999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.228,"5min":1.341,"15min":1.341},"last":1.0920000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.93600000000000005,"15min":0.93700000000000006},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2589999999999999,"5min":1.359,"15min":1.359},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0820000000000001,"15min":1.079},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.369,"5min":1.369,"15min":1.3879999999999999},"last":1.1499999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0369999999999999,"15min":1.0089999999999999},"min":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.622},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":1.169}]},{"osd":5,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0700000000000001,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.4530000000000001,"5min":1.484,"15min":1.484},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3620000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.119}]},{"osd":6,"last update":"Wed Sep 27 23:50:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1479999999999999,"15min":1.145},"min":{"1min":0.85499999999999998,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.3280000000000001,"5min":1.425,"15min":1.448},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.153,"15min":1.145},"min":{"1min":0.85699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.48,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.381}]}]},{"osd":6,"up_from":15,"seq":64424509881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29712,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":27584,"kb_avail":94342128,"statfs":{"total":96636764160,"available":96606339072,"internally_reserved":0,"allocated":2170880,"data_stored":1139841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28246016},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.99399999999999999,"15min":0.97099999999999997},"min":{"1min":0.65900000000000003,"5min":0.60599999999999998,"15min":0.51800000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.071},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0529999999999999,"15min":1.0600000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.69399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.749}]},{"osd":1,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.66800000000000004,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.57099999999999995},"max":{"1min":1.226,"5min":1.53,"15min":1.53},"last":1.1599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90900000000000003,"15min":0.90300000000000002},"min":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61399999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.63200000000000001},"max":{"1min":1.2709999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.74099999999999999,"5min":0.65700000000000003,"15min":0.63100000000000001},"max":{"1min":1.53,"5min":1.583,"15min":1.583},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.169,"5min":1.325,"15min":1.325},"last":0.97499999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.878,"15min":0.873},"min":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.41599999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0129999999999999,"15min":1},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3700000000000001,"5min":1.633,"15min":1.633},"last":1.0409999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.95099999999999996},"min":{"1min":0.71099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.4430000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.149,"15min":1.141},"min":{"1min":0.81299999999999994,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.1910000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:49:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.1180000000000001},"min":{"1min":0.755,"5min":0.627,"15min":0.627},"max":{"1min":1.623,"5min":1.679,"15min":1.7210000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.423,"5min":1.488,"15min":1.542},"last":1.242}]}]},{"osd":5,"up_from":15,"seq":64424509881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39316,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":37184,"kb_avail":94332524,"statfs":{"total":96636764160,"available":96596504576,"internally_reserved":0,"allocated":2174976,"data_stored":1140395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38076416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.86699999999999999,"15min":0.86199999999999999},"min":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.51900000000000002},"max":{"1min":1.248,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":10.058999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.90900000000000003,"15min":0.92100000000000004},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.47899999999999998},"max":{"1min":1.2250000000000001,"5min":1.292,"15min":1.5640000000000001},"last":13.757999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.302,"5min":1.425,"15min":1.538},"last":4.1970000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90500000000000003,"15min":0.90700000000000003},"min":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.194,"5min":1.3819999999999999,"15min":1.397},"last":2.8650000000000002}]},{"osd":2,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"min":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.629},"max":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.4690000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.626},"max":{"1min":1.153,"5min":1.3049999999999999,"15min":1.4470000000000001},"last":4.0999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.3240000000000001,"5min":1.373,"15min":1.611},"last":3.7450000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0349999999999999,"15min":1.0249999999999999},"min":{"1min":0.84599999999999997,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.228,"5min":1.419,"15min":1.4370000000000001},"last":4.2350000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.88},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.3129999999999999,"5min":1.484,"15min":1.484},"last":2.0270000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.58699999999999997},"max":{"1min":1.45,"5min":1.45,"15min":1.4510000000000001},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.026},"min":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.242,"5min":1.496,"15min":1.5069999999999999},"last":4.157},{"interface":"front","average":{"1min":1.089,"5min":1.099,"15min":1.0980000000000001},"min":{"1min":0.84899999999999998,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":1.262,"5min":1.546,"15min":1.5840000000000001},"last":2.1150000000000002}]},{"osd":7,"last update":"Wed Sep 27 23:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1619999999999999,"15min":1.1459999999999999},"min":{"1min":0.94099999999999995,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7709999999999999},"last":6.3710000000000004},{"interface":"front","average":{"1min":1.194,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":0.98099999999999998,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":14.119}]}]},{"osd":4,"up_from":15,"seq":64424509882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25628,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":23488,"kb_avail":94346212,"statfs":{"total":96636764160,"available":96610521088,"internally_reserved":0,"allocated":2183168,"data_stored":1141425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24051712},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.92500000000000004,"15min":0.90700000000000003},"min":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.45000000000000001},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88800000000000001,"15min":0.89500000000000002},"min":{"1min":0.63200000000000001,"5min":0.624,"15min":0.41599999999999998},"max":{"1min":1.893,"5min":1.893,"15min":2.2770000000000001},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.90200000000000002},"min":{"1min":0.75900000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.97499999999999998,"15min":0.95899999999999996},"min":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.53500000000000003},"max":{"1min":1.758,"5min":1.758,"15min":2.1110000000000002},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.73899999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.98299999999999998,"15min":0.96799999999999997},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.966,"5min":1.966,"15min":2.056},"last":0.96999999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.998,"5min":1.998,"15min":2.032},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.0880000000000001,"15min":1.054},"min":{"1min":0.82899999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.0129999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93700000000000006,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.61399999999999999,"15min":0.45500000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.9930000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":0.751}]},{"osd":6,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0600000000000001,"15min":1.0509999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.694,"5min":1.873,"15min":1.873},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.1759999999999999,"15min":1.1679999999999999},"min":{"1min":0.96199999999999997,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1800000000000002},"last":1.081}]},{"osd":7,"last update":"Wed Sep 27 23:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.105,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.002},{"interface":"front","average":{"1min":1.177,"5min":1.1779999999999999,"15min":1.177},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1180000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30224,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":28096,"kb_avail":94341616,"statfs":{"total":96636764160,"available":96605814784,"internally_reserved":0,"allocated":2170880,"data_stored":1139853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28770304},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83999999999999997,"5min":1.1639999999999999,"15min":1.3560000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.747,"15min":0.746},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.81599999999999995,"5min":1.2629999999999999,"15min":1.377},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1799999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.69299999999999995,"5min":0.63,"15min":0.58999999999999997},"max":{"1min":1.2509999999999999,"5min":1.272,"15min":1.2849999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.876,"15min":0.873},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58099999999999996},"max":{"1min":1.1599999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94099999999999995},"min":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.63800000000000001},"max":{"1min":1.175,"5min":1.3420000000000001,"15min":1.383},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.998,"15min":0.98399999999999999},"min":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.621},"max":{"1min":1.0600000000000001,"5min":1.599,"15min":1.599},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.86399999999999999,"5min":0.72099999999999997,"15min":0.68700000000000006},"max":{"1min":1.3069999999999999,"5min":1.587,"15min":1.587},"last":0.91300000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.123,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.484,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.181},"min":{"1min":0.93300000000000005,"5min":0.88600000000000001,"15min":0.72699999999999998},"max":{"1min":1.6499999999999999,"5min":1.659,"15min":1.659},"last":1.0660000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.92900000000000005,"5min":0.878,"15min":0.77800000000000002},"max":{"1min":1.288,"5min":1.5249999999999999,"15min":1.659},"last":0.873},{"interface":"front","average":{"1min":1.222,"5min":1.258,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.90500000000000003,"15min":0.82599999999999996},"max":{"1min":1.413,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.002}]},{"osd":7,"last update":"Wed Sep 27 23:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2629999999999999,"15min":1.25},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5629999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.94999999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.409,"5min":1.657,"15min":1.657},"last":1.0509999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30160,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":28032,"kb_avail":94341680,"statfs":{"total":96636764160,"available":96605880320,"internally_reserved":0,"allocated":2170880,"data_stored":1139909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28704768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1879999999999999,"5min":1.252,"15min":1.5329999999999999},"last":1.081},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97299999999999998,"5min":1.345,"15min":1.7969999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.073,"5min":1.242,"15min":2.0760000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84299999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.754,"15min":0.76900000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.2350000000000001,"15min":1.667},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.91100000000000003,"15min":0.94499999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.159,"5min":1.262,"15min":1.7},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0820000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.972},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.0700000000000001,"15min":1.0620000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.361,"5min":1.645,"15min":1.8009999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0429999999999999,"15min":1.0489999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.772},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.151,"15min":1.173},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.351,"5min":1.653,"15min":1.859},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.145,"15min":1.151},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.8999999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.175,"15min":1.1759999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6259999999999999,"5min":1.6439999999999999,"15min":1.736},"last":0.95399999999999996}]},{"osd":7,"last update":"Wed Sep 27 23:50:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2250000000000001,"15min":1.2270000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.86599999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":2.0059999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2190000000000001,"15min":1.2310000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5669999999999999,"5min":1.8129999999999999,"15min":1.8340000000000001},"last":0.875}]}]},{"osd":1,"up_from":14,"seq":60129542588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44368,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":42240,"kb_avail":94327472,"statfs":{"total":96636764160,"available":96591331328,"internally_reserved":0,"allocated":2170880,"data_stored":1139853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43253760},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1819999999999999,"15min":1.2070000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.73999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.095,"5min":1.177,"15min":1.361},"last":0.73199999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.88,"5min":1.2509999999999999,"15min":1.383},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2350000000000001,"5min":1.3169999999999999,"15min":1.427},"last":0.68999999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94399999999999995,"15min":0.94499999999999995},"min":{"1min":0.64600000000000002,"5min":0.626,"15min":0.58899999999999997},"max":{"1min":1.1890000000000001,"5min":1.3080000000000001,"15min":1.3720000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.83599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.454},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.98699999999999999,"15min":0.98999999999999999},"min":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.58599999999999997},"max":{"1min":1.127,"5min":1.325,"15min":1.5229999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.196,"5min":1.1379999999999999,"15min":1.1419999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.71999999999999997},"max":{"1min":1.4299999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.091,"15min":1.0980000000000001},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.621},"max":{"1min":1.399,"5min":1.5449999999999999,"15min":1.583},"last":0.877},{"interface":"front","average":{"1min":1.077,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.747},"max":{"1min":1.3,"5min":1.4710000000000001,"15min":1.575},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2450000000000001,"15min":1.234},"min":{"1min":0.97999999999999998,"5min":0.84899999999999998,"15min":0.76800000000000002},"max":{"1min":1.5529999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.72299999999999998},"max":{"1min":1.4970000000000001,"5min":1.7789999999999999,"15min":1.804},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 27 23:49:51 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.94499999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.514,"5min":1.7130000000000001,"15min":1.853},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.95699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4139999999999999,"5min":1.585,"15min":1.827},"last":1.204}]}]},{"osd":0,"up_from":15,"seq":64424509883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31312,"kb_used_data":2696,"kb_used_omap":0,"kb_used_meta":28608,"kb_avail":94340528,"statfs":{"total":96636764160,"available":96604700672,"internally_reserved":0,"allocated":2760704,"data_stored":1730187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29294592},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.85499999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.47499999999999998},"max":{"1min":1.131,"5min":1.1779999999999999,"15min":1.306},"last":0.84399999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.502},"max":{"1min":0.98299999999999998,"5min":1.236,"15min":1.5960000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.101,"5min":1.288,"15min":1.863},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.52700000000000002},"max":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.78600000000000003,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.044,"15min":1.0529999999999999},"min":{"1min":0.92500000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3,"5min":1.4099999999999999,"15min":1.7110000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.056,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.70699999999999996},"max":{"1min":1.516,"5min":1.5249999999999999,"15min":1.651},"last":0.98899999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.111,"15min":1.111},"min":{"1min":0.90700000000000003,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.407,"5min":1.4259999999999999,"15min":1.512},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.165,"15min":1.1779999999999999},"min":{"1min":0.86399999999999999,"5min":0.83399999999999996,"15min":0.79800000000000004},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.847},"last":1.083}]},{"osd":6,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1870000000000001,"15min":1.1830000000000001},"min":{"1min":0.90700000000000003,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.895},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2330000000000001},"min":{"1min":0.997,"5min":0.997,"15min":0.84999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5720000000000001,"15min":2.0510000000000002},"last":1.3}]},{"osd":7,"last update":"Wed Sep 27 23:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.286,"15min":1.2789999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.84799999999999998},"max":{"1min":1.554,"5min":1.6100000000000001,"15min":2.0030000000000001},"last":1.21},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.85599999999999998},"max":{"1min":1.4790000000000001,"5min":1.806,"15min":2.004},"last":1.0469999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:50:42.280 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:50:42.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:50:42.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.430+0000 7fc04ffff700 1 Processor -- start 2023-09-27T23:50:42.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.430+0000 7fc04ffff700 1 -- start start 2023-09-27T23:50:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.430+0000 7fc04ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc050155520 0x7fc0500c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc05006b530 0x7fc0500ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0500ad020 con 0x7fc0500ae6c0 2023-09-27T23:50:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0500c81e0 con 0x7fc050155520 2023-09-27T23:50:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0500aca10 con 0x7fc05006b530 2023-09-27T23:50:42.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc050155520 0x7fc0500c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc050155520 0x7fc0500c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54590/0 (socket says 172.21.15.64:54590) 2023-09-27T23:50:42.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04e7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc0500ae6c0 0x7fc05006ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40986/0 (socket says 172.21.15.64:40986) 2023-09-27T23:50:42.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 -- 172.21.15.64:0/2588878770 learned_addr learned my addr 172.21.15.64:0/2588878770 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:42.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04f7fe700 1 --2- 172.21.15.64:0/2588878770 >> v2:172.21.15.123:3300/0 conn(0x7fc05006b530 0x7fc0500ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 -- 172.21.15.64:0/2588878770 >> v2:172.21.15.123:3300/0 conn(0x7fc05006b530 msgr2=0x7fc0500ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 --2- 172.21.15.64:0/2588878770 >> v2:172.21.15.123:3300/0 conn(0x7fc05006b530 0x7fc0500ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 -- 172.21.15.64:0/2588878770 >> v1:172.21.15.64:6789/0 conn(0x7fc0500ae6c0 legacy=0x7fc05006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04dffb700 1 -- 172.21.15.64:0/2588878770 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965794328 0 0) 0x7fc0500ad020 con 0x7fc0500ae6c0 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 -- 172.21.15.64:0/2588878770 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0500c7c20 con 0x7fc050155520 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04effd700 1 --2- 172.21.15.64:0/2588878770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc0500c7130 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc0500c6830 tx=0x7fc040009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f562d33a80d1c04 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04dffb700 1 -- 172.21.15.64:0/2588878770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040009e70 con 0x7fc050155520 2023-09-27T23:50:42.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04dffb700 1 -- 172.21.15.64:0/2588878770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04000b8f0 con 0x7fc050155520 2023-09-27T23:50:42.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04dffb700 1 -- 172.21.15.64:0/2588878770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04000bc90 con 0x7fc050155520 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- 172.21.15.64:0/2588878770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 msgr2=0x7fc0500c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 --2- 172.21.15.64:0/2588878770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc0500c7130 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc0500c6830 tx=0x7fc040009250 comp rx=0 tx=0).stop 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- 172.21.15.64:0/2588878770 shutdown_connections 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 --2- 172.21.15.64:0/2588878770 >> v2:172.21.15.123:3300/0 conn(0x7fc05006b530 0x7fc0500ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 --2- 172.21.15.64:0/2588878770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc0500c7130 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- 172.21.15.64:0/2588878770 >> 172.21.15.64:0/2588878770 conn(0x7fc0500c8cd0 msgr2=0x7fc0500c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- 172.21.15.64:0/2588878770 shutdown_connections 2023-09-27T23:50:42.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.434+0000 7fc04ffff700 1 -- 172.21.15.64:0/2588878770 wait complete. 2023-09-27T23:50:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 Processor -- start 2023-09-27T23:50:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 -- start start 2023-09-27T23:50:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc05006b530 0x7fc0501542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0500ae6c0 0x7fc05015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc05015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc050166ab0 con 0x7fc050155520 2023-09-27T23:50:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501568c0 con 0x7fc0500ae6c0 2023-09-27T23:50:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc050156ba0 con 0x7fc05006b530 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc05015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc05015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54598/0 (socket says 172.21.15.64:54598) 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 -- 172.21.15.64:0/1328263050 learned_addr learned my addr 172.21.15.64:0/1328263050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04e7fc700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0500ae6c0 0x7fc05015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04effd700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc05006b530 0x7fc0501542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 -- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc05006b530 msgr2=0x7fc0501542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc05006b530 0x7fc0501542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 -- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0500ae6c0 msgr2=0x7fc05015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0500ae6c0 0x7fc05015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04f7fe700 1 -- 172.21.15.64:0/1328263050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400094f0 con 0x7fc050155520 2023-09-27T23:50:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.438+0000 7fc04e7fc700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0500ae6c0 0x7fc05015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc04f7fe700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc05015adf0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc048009ef0 tx=0x7fc04800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800cc50 con 0x7fc050155520 2023-09-27T23:50:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800f040 con 0x7fc050155520 2023-09-27T23:50:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc050156e80 con 0x7fc050155520 2023-09-27T23:50:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048014480 con 0x7fc050155520 2023-09-27T23:50:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc050154820 con 0x7fc050155520 2023-09-27T23:50:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc050166c20 con 0x7fc050155520 2023-09-27T23:50:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fc0480147f0 con 0x7fc050155520 2023-09-27T23:50:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc036ffd700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc03c060a70 0x7fc03c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.442+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc04807dfb0 con 0x7fc050155520 2023-09-27T23:50:42.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.446+0000 7fc04effd700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc03c060a70 0x7fc03c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.446+0000 7fc04effd700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc03c060a70 0x7fc03c062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0500c6830 tx=0x7fc040017000 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.450+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc0480adbb0 con 0x7fc050155520 2023-09-27T23:50:42.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.674+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc050157380 con 0x7fc050155520 2023-09-27T23:50:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc036ffd700 1 -- 172.21.15.64:0/1328263050 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc0480180e0 con 0x7fc050155520 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc03c060a70 msgr2=0x7fc03c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc03c060a70 0x7fc03c062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0500c6830 tx=0x7fc040017000 comp rx=0 tx=0).stop 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 msgr2=0x7fc05015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc05015adf0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc048009ef0 tx=0x7fc04800b040 comp rx=0 tx=0).stop 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 shutdown_connections 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fc03c060a70 0x7fc03c062f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc050155520 0x7fc05015adf0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0500ae6c0 0x7fc05015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 --2- 172.21.15.64:0/1328263050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc05006b530 0x7fc0501542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 >> 172.21.15.64:0/1328263050 conn(0x7fc0500c8cd0 msgr2=0x7fc0500b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 shutdown_connections 2023-09-27T23:50:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.686+0000 7fc04ffff700 1 -- 172.21.15.64:0/1328263050 wait complete. 2023-09-27T23:50:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:50:42.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:50:42.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 Processor -- start 2023-09-27T23:50:42.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 -- start start 2023-09-27T23:50:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ae4155990 0x7f2ae40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ae40b0670 0x7f2ae40b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae40b13b0 0x7f2ae406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae40ae7e0 con 0x7f2ae40b0670 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae40cbc50 con 0x7f2ae4155990 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40b1810 con 0x7f2ae40b13b0 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ae40b0670 0x7f2ae40b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ae40b0670 0x7f2ae40b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54602/0 (socket says 172.21.15.64:54602) 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae40b13b0 0x7f2ae406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 -- 172.21.15.64:0/1819890798 learned_addr learned my addr 172.21.15.64:0/1819890798 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ae4155990 0x7f2ae40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 -- 172.21.15.64:0/1819890798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae40b13b0 msgr2=0x7f2ae406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 --2- 172.21.15.64:0/1819890798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae40b13b0 0x7f2ae406d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 -- 172.21.15.64:0/1819890798 >> v2:172.21.15.123:3300/0 conn(0x7f2ae4155990 msgr2=0x7f2ae40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 --2- 172.21.15.64:0/1819890798 >> v2:172.21.15.123:3300/0 conn(0x7f2ae4155990 0x7f2ae40c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.810+0000 7f2ae27fc700 1 -- 172.21.15.64:0/1819890798 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae406cf00 con 0x7f2ae40b0670 2023-09-27T23:50:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae27fc700 1 --2- 172.21.15.64:0/1819890798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b0670 0x7f2ae40b0e70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2ae406f0c0 tx=0x7f2ad8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2925eebf4e236db7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae1ffb700 1 -- 172.21.15.64:0/1819890798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800b640 con 0x7f2ae40b0670 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae1ffb700 1 -- 172.21.15.64:0/1819890798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad800b7a0 con 0x7f2ae40b0670 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae1ffb700 1 -- 172.21.15.64:0/1819890798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800bb60 con 0x7f2ae40b0670 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 -- 172.21.15.64:0/1819890798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b0670 msgr2=0x7f2ae40b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/1819890798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b0670 0x7f2ae40b0e70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2ae406f0c0 tx=0x7f2ad8009250 comp rx=0 tx=0).stop 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 -- 172.21.15.64:0/1819890798 shutdown_connections 2023-09-27T23:50:42.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/1819890798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae40b13b0 0x7f2ae406d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/1819890798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b0670 0x7f2ae40b0e70 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/1819890798 >> v2:172.21.15.123:3300/0 conn(0x7f2ae4155990 0x7f2ae40c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 -- 172.21.15.64:0/1819890798 >> 172.21.15.64:0/1819890798 conn(0x7f2ae4067bb0 msgr2=0x7f2ae40add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 -- 172.21.15.64:0/1819890798 shutdown_connections 2023-09-27T23:50:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 -- 172.21.15.64:0/1819890798 wait complete. 2023-09-27T23:50:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 Processor -- start 2023-09-27T23:50:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.814+0000 7f2ae3fff700 1 -- start start 2023-09-27T23:50:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae40b0670 0x7f2ae414ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 0x7f2ae4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae4155990 0x7f2ae40abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41584f0 con 0x7f2ae40b13b0 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41578e0 con 0x7f2ae40b0670 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae4157bc0 con 0x7f2ae4155990 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 0x7f2ae4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 0x7f2ae4156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54604/0 (socket says 172.21.15.64:54604) 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 -- 172.21.15.64:0/2730215120 learned_addr learned my addr 172.21.15.64:0/2730215120 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae37fe700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae4155990 0x7f2ae40abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae2ffd700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae40b0670 0x7f2ae414ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 -- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae4155990 msgr2=0x7f2ae40abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae4155990 0x7f2ae40abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 -- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae40b0670 msgr2=0x7f2ae414ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae40b0670 0x7f2ae414ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 -- 172.21.15.64:0/2730215120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80094f0 con 0x7f2ae40b13b0 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae27fc700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 0x7f2ae4156f70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4069fb0 tx=0x7f2ad8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800b910 con 0x7f2ae40b13b0 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad800ba70 con 0x7f2ae40b13b0 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8011630 con 0x7f2ae40b13b0 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae4157ea0 con 0x7f2ae40b13b0 2023-09-27T23:50:42.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae414d230 con 0x7f2ae40b13b0 2023-09-27T23:50:42.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.818+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae404be40 con 0x7f2ae40b13b0 2023-09-27T23:50:42.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.822+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2ad80119a0 con 0x7f2ae40b13b0 2023-09-27T23:50:42.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.822+0000 7f2acaffd700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ad0060a70 0x7f2ad0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:42.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.822+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2ad807e400 con 0x7f2ae40b13b0 2023-09-27T23:50:42.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.822+0000 7f2ae2ffd700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ad0060a70 0x7f2ad0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:42.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.822+0000 7f2ae2ffd700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ad0060a70 0x7f2ad0062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009d10 tx=0x7f2ad40091a0 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:42.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:42.826+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2ad804e4c0 con 0x7f2ae40b13b0 2023-09-27T23:50:43.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.022+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ae40adf30 con 0x7f2ae40b13b0 2023-09-27T23:50:43.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.022+0000 7f2acaffd700 1 -- 172.21.15.64:0/2730215120 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f2ad8051ae0 con 0x7f2ae40b13b0 2023-09-27T23:50:43.027 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:43.027 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:50:34.520359+0000","modified":"2023-09-27T23:50:36.536424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/660311009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":660311009},{"type":"v1","addr":"172.21.15.64:6835","nonce":660311009}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ad0060a70 msgr2=0x7f2ad0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ad0060a70 0x7f2ad0062f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009d10 tx=0x7f2ad40091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 msgr2=0x7f2ae4156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 0x7f2ae4156f70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4069fb0 tx=0x7f2ad8017000 comp rx=0 tx=0).stop 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 shutdown_connections 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ad0060a70 0x7f2ad0062f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ae4155990 0x7f2ae40abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae40b13b0 0x7f2ae4156f70 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 --2- 172.21.15.64:0/2730215120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ae40b0670 0x7f2ae414ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 >> 172.21.15.64:0/2730215120 conn(0x7f2ae4067bb0 msgr2=0x7f2ae40ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 shutdown_connections 2023-09-27T23:50:43.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.026+0000 7f2ae3fff700 1 -- 172.21.15.64:0/2730215120 wait complete. 2023-09-27T23:50:43.031 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 66 2023-09-27T23:50:43.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:50:34.520359+0000', 'modified': '2023-09-27T23:50:36.536424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9783}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9783': {'gid': 9783, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/660311009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 660311009}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 660311009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:50:43.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:50:43.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-27T23:50:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 Processor -- start 2023-09-27T23:50:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 -- start start 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ed0139d50 0x7f8ed00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ed00b27c0 0x7f8ed015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed00cc430 con 0x7f8ed013a120 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed00cce00 con 0x7f8ed00b27c0 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed7b2d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed015bad0 con 0x7f8ed0139d50 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed50c8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8ed013a120 0x7f8ed00b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40996/0 (socket says 172.21.15.64:40996) 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed60ca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ed00b27c0 0x7f8ed015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed50c8700 1 -- 172.21.15.64:0/2790867173 learned_addr learned my addr 172.21.15.64:0/2790867173 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/2790867173 >> v2:172.21.15.123:3300/0 conn(0x7f8ed0139d50 0x7f8ed00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed60ca700 1 -- 172.21.15.64:0/2790867173 >> v2:172.21.15.123:3300/0 conn(0x7f8ed0139d50 msgr2=0x7f8ed00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/2790867173 >> v2:172.21.15.123:3300/0 conn(0x7f8ed0139d50 0x7f8ed00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed60ca700 1 -- 172.21.15.64:0/2790867173 >> v1:172.21.15.64:6789/0 conn(0x7f8ed013a120 legacy=0x7f8ed00b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed60ca700 1 -- 172.21.15.64:0/2790867173 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00cbb00 con 0x7f8ed00b27c0 2023-09-27T23:50:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/2790867173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed015ad80 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f8ed006afb0 tx=0x7f8ec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a6f8bb3faf60d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed48c7700 1 -- 172.21.15.64:0/2790867173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011630 con 0x7f8ed00b27c0 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed48c7700 1 -- 172.21.15.64:0/2790867173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8011790 con 0x7f8ed00b27c0 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.562+0000 7f8ed48c7700 1 -- 172.21.15.64:0/2790867173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011b30 con 0x7f8ed00b27c0 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/2790867173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 msgr2=0x7f8ed015ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/2790867173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed015ad80 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f8ed006afb0 tx=0x7f8ec80091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/2790867173 shutdown_connections 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/2790867173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed015ad80 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/2790867173 >> v2:172.21.15.123:3300/0 conn(0x7f8ed0139d50 0x7f8ed00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/2790867173 >> 172.21.15.64:0/2790867173 conn(0x7f8ed0069310 msgr2=0x7f8ed00696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/2790867173 shutdown_connections 2023-09-27T23:50:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/2790867173 wait complete. 2023-09-27T23:50:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 Processor -- start 2023-09-27T23:50:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.566+0000 7f8ed7b2d700 1 -- start start 2023-09-27T23:50:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed013ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed0139d50 0x7f8ed01575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ed013a120 0x7f8ed0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed016d6a0 con 0x7f8ed00b27c0 2023-09-27T23:50:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed016cde0 con 0x7f8ed0139d50 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed016d050 con 0x7f8ed013a120 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed013ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed013ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54610/0 (socket says 172.21.15.64:54610) 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 -- 172.21.15.64:0/1985508853 learned_addr learned my addr 172.21.15.64:0/1985508853 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ed013a120 0x7f8ed0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed0139d50 0x7f8ed01575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 -- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ed013a120 msgr2=0x7f8ed0157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ed013a120 0x7f8ed0157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 -- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed0139d50 msgr2=0x7f8ed01575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed0139d50 0x7f8ed01575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 -- 172.21.15.64:0/1985508853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8009420 con 0x7f8ed00b27c0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ed013a120 0x7f8ed0157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed013ce20 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800ecb0 tx=0x7f8eb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ecdffb700 1 -- 172.21.15.64:0/1985508853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800c9d0 con 0x7f8ed00b27c0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ecdffb700 1 -- 172.21.15.64:0/1985508853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb800cb30 con 0x7f8ed00b27c0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ecdffb700 1 -- 172.21.15.64:0/1985508853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800ced0 con 0x7f8ed00b27c0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed016d330 con 0x7f8ed00b27c0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.570+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed013d360 con 0x7f8ed00b27c0 2023-09-27T23:50:43.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.574+0000 7f8ecdffb700 1 -- 172.21.15.64:0/1985508853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8eb8013680 con 0x7f8ed00b27c0 2023-09-27T23:50:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.574+0000 7f8ecdffb700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8eb0060a70 0x7f8eb0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.574+0000 7f8ecdffb700 1 -- 172.21.15.64:0/1985508853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8eb807d4c0 con 0x7f8ed00b27c0 2023-09-27T23:50:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.574+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8eb0060a70 0x7f8eb0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.574+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8eb0060a70 0x7f8eb0062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00af200 tx=0x7f8ec0009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 Processor -- start 2023-09-27T23:50:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 -- start start 2023-09-27T23:50:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x259ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x259d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 0x259d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x24caf30 con 0x236a790 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x24c6ed0 con 0x231f710 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x248d0b0 con 0x231fae0 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 0x259d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed50c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x259d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 0x259d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54624/0 (socket says 172.21.15.64:54624) 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed58c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x259ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 -- 172.21.15.64:0/3581706305 learned_addr learned my addr 172.21.15.64:0/3581706305 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 -- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 msgr2=0x259ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x259ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 -- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 msgr2=0x259d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x259d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 -- 172.21.15.64:0/3581706305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x236b9e0 con 0x236a790 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x259d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 0x259d8d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80091e0 tx=0x7f8ec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f64e64675026ecc server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ec6ffd700 1 -- 172.21.15.64:0/3581706305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8020070 con 0x236a790 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.578+0000 7f8ec6ffd700 1 -- 172.21.15.64:0/3581706305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8011900 con 0x236a790 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ec6ffd700 1 -- 172.21.15.64:0/3581706305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011630 con 0x236a790 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3581706305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 msgr2=0x259d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 0x259d8d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80091e0 tx=0x7f8ec8017000 comp rx=0 tx=0).stop 2023-09-27T23:50:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3581706305 shutdown_connections 2023-09-27T23:50:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x236a790 0x259d8d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x259d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3581706305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x259ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3581706305 >> 172.21.15.64:0/3581706305 conn(0x231feb0 msgr2=0x2559880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3581706305 shutdown_connections 2023-09-27T23:50:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3581706305 wait complete. 2023-09-27T23:50:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 -- start start 2023-09-27T23:50:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.582+0000 7f8ed9dd6740 1 Processor -- start 2023-09-27T23:50:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 0x2559740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x2582c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x2583140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x248cd50 con 0x231f710 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x2465970 con 0x236a790 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x2470510 con 0x231fae0 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 10 client.0 _ll_register_callbacks cb 0x2510b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 0x2559740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 0x2559740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54628/0 (socket says 172.21.15.64:54628) 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 -- 172.21.15.64:0/290711864 learned_addr learned my addr 172.21.15.64:0/290711864 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x2582c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x2583140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 -- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 msgr2=0x2582c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x2582c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 -- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 msgr2=0x2583140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x2583140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 -- 172.21.15.64:0/290711864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec000f040 con 0x231f710 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x2582c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x2583140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 0x2559740 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8050eb0 tx=0x7f8eb807d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c10868e91c8a67 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 1 -- 172.21.15.64:0/290711864 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb804f630 con 0x231f710 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 1 -- 172.21.15.64:0/290711864 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8eb807d2c0 con 0x231f710 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2465cc0 con 0x231f710 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 1 -- 172.21.15.64:0/290711864 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb8051660 con 0x231f710 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 10 client.9948 fetch_fsmap learned FSMap version 66 2023-09-27T23:50:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x7f8ed00cbb00 con 0x231f710 2023-09-27T23:50:43.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 1 -- 172.21.15.64:0/290711864 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb807b020 con 0x231f710 2023-09-27T23:50:43.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 1 -- 172.21.15.64:0/290711864 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8eb807b190 con 0x231f710 2023-09-27T23:50:43.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 10 client.9948 fetch_fsmap finished waiting for FSMap version 66 2023-09-27T23:50:43.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 10 client.9948 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 20 client.9948 populate_metadata read hostname 'smithi064' 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x25a96b0 0x25aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 4 client.9948 mds_command: new command op to 9783 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 --> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x24693e0 con 0x25a96b0 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x25a96b0 0x25aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x25a96b0 0x25aba90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e83700ab4672df server_cookie=6dabfa2634b979b9 in_seq=0 out_seq=0 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.586+0000 7f8eaffff700 10 client.9948 ms_handle_connect on v2:172.21.15.64:6834/660311009 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8eaffff700 1 -- 172.21.15.64:0/290711864 <== mds.0 v2:172.21.15.64:6834/660311009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x24693e0 con 0x25a96b0 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8eaffff700 10 client.9948 handle_command_reply: tid=0 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 client.9948 shutdown 2023-09-27T23:50:43.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 msgr2=0x2559740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 0x2559740 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8050eb0 tx=0x7f8eb807d040 comp rx=0 tx=0).stop 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 shutdown_connections 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x25a96b0 0x25aba90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x2583140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231fae0 0x2582c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/290711864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231f710 0x2559740 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8eaffff700 0 client.9948 ms_handle_reset on v2:172.21.15.64:6834/660311009 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 >> 172.21.15.64:0/290711864 conn(0x231feb0 msgr2=0x2551ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 shutdown_connections 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/290711864 wait complete. 2023-09-27T23:50:43.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.590+0000 7f8ed9dd6740 20 client.9948 trim_cache size 0 max 16384 2023-09-27T23:50:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 Processor -- start 2023-09-27T23:50:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 -- start start 2023-09-27T23:50:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231f710 0x25f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x25f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x25f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x23987c0 con 0x231fae0 2023-09-27T23:50:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x24cbef0 con 0x236a790 2023-09-27T23:50:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x249b610 con 0x231f710 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x25f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed60ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x25f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x25f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54638/0 (socket says 172.21.15.64:54638) 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 -- 172.21.15.64:0/3437450834 learned_addr learned my addr 172.21.15.64:0/3437450834 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231f710 0x25f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 -- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231f710 msgr2=0x25f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231f710 0x25f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 -- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 msgr2=0x25f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x25f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 -- 172.21.15.64:0/3437450834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00b2210 con 0x231fae0 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231f710 0x25f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x25f25e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009010 tx=0x7f8ec0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f87c3843663df3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ec77fe700 1 -- 172.21.15.64:0/3437450834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00250e0 con 0x231fae0 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ec77fe700 1 -- 172.21.15.64:0/3437450834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0007700 con 0x231fae0 2023-09-27T23:50:43.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.606+0000 7f8ec77fe700 1 -- 172.21.15.64:0/3437450834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0007a80 con 0x231fae0 2023-09-27T23:50:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3437450834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 msgr2=0x25f25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x25f25e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009010 tx=0x7f8ec0015040 comp rx=0 tx=0).stop 2023-09-27T23:50:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3437450834 shutdown_connections 2023-09-27T23:50:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x236a790 0x25f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x25f25e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/3437450834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x231f710 0x25f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3437450834 >> 172.21.15.64:0/3437450834 conn(0x231feb0 msgr2=0x2552760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3437450834 shutdown_connections 2023-09-27T23:50:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/3437450834 wait complete. 2023-09-27T23:50:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 -- start start 2023-09-27T23:50:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.610+0000 7f8ed9dd6740 1 Processor -- start 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x25ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x257c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x236a790 0x257cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x249c010 con 0x231fae0 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x24c2b50 con 0x231f710 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x249a040 con 0x236a790 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 10 client.0 _ll_register_callbacks cb 0x23e30e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:50:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x257c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x257c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54646/0 (socket says 172.21.15.64:54646) 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 -- 172.21.15.64:0/1904585667 learned_addr learned my addr 172.21.15.64:0/1904585667 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed60ca700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x236a790 0x257cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x25ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 -- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x236a790 msgr2=0x257cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x236a790 0x257cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 -- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 msgr2=0x25ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x25ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 -- 172.21.15.64:0/1904585667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed00cc430 con 0x231fae0 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed50c8700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x257c6b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x24c7d20 tx=0x7f8ec0017030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f586aa0ae551f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:50:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 1 -- 172.21.15.64:0/1904585667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0007070 con 0x231fae0 2023-09-27T23:50:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 1 -- 172.21.15.64:0/1904585667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8ec0038d70 con 0x231fae0 2023-09-27T23:50:43.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ec8011900 con 0x231fae0 2023-09-27T23:50:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 1 -- 172.21.15.64:0/1904585667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec0025bd0 con 0x231fae0 2023-09-27T23:50:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 10 client.9954 fetch_fsmap learned FSMap version 66 2023-09-27T23:50:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x2495370 con 0x231fae0 2023-09-27T23:50:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 1 -- 172.21.15.64:0/1904585667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8011900 con 0x231fae0 2023-09-27T23:50:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 1 -- 172.21.15.64:0/1904585667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8ec0013420 con 0x231fae0 2023-09-27T23:50:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 10 client.9954 fetch_fsmap finished waiting for FSMap version 66 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 10 client.9954 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 20 client.9954 populate_metadata read hostname 'smithi064' 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x257d130 0x25f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 4 client.9954 mds_command: new command op to 9783 tid=0[{"prefix": "status"}] 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 --> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] -- command(tid 0: {"prefix": "status"}) v1 -- 0x249b410 con 0x257d130 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x257d130 0x25f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8eaffff700 10 client.9954 ms_handle_connect on v2:172.21.15.64:6834/660311009 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.614+0000 7f8ed58c9700 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x257d130 0x25f63d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d32c8a3ed5c584 server_cookie=d8ac1de535108395 in_seq=0 out_seq=0 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8eaffff700 1 -- 172.21.15.64:0/1904585667 <== mds.0 v2:172.21.15.64:6834/660311009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x249b410 con 0x257d130 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8eaffff700 10 client.9954 handle_command_reply: tid=0 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 client.9954 shutdown 2023-09-27T23:50:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 msgr2=0x257c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x257c6b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x24c7d20 tx=0x7f8ec0017030 comp rx=0 tx=0).stop 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 shutdown_connections 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:6834/660311009,v1:172.21.15.64:6835/660311009] conn(0x257d130 0x25f63d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x236a790 0x257cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8eaffff700 0 client.9954 ms_handle_reset on v2:172.21.15.64:6834/660311009 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x231fae0 0x257c6b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 --2- 172.21.15.64:0/1904585667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x231f710 0x25ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 >> 172.21.15.64:0/1904585667 conn(0x231feb0 msgr2=0x25f1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 shutdown_connections 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 1 -- 172.21.15.64:0/1904585667 wait complete. 2023-09-27T23:50:43.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed9dd6740 20 client.9954 trim_cache size 0 max 16384 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 9783, 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.0953833690000003, 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 66, 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 174, 2023-09-27T23:50:43.624 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 174, 2023-09-27T23:50:43.625 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 13.616582639000001 2023-09-27T23:50:43.625 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8eb0060a70 msgr2=0x7f8eb0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8eb0060a70 0x7f8eb0062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00af200 tx=0x7f8ec0009040 comp rx=0 tx=0).stop 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 msgr2=0x7f8ed013ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed013ce20 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800ecb0 tx=0x7f8eb80091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 shutdown_connections 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8eb0060a70 0x7f8eb0062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ed013a120 0x7f8ed0157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed0139d50 0x7f8ed01575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 --2- 172.21.15.64:0/1985508853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ed00b27c0 0x7f8ed013ce20 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 >> 172.21.15.64:0/1985508853 conn(0x7f8ed0069310 msgr2=0x7f8ed0136ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 shutdown_connections 2023-09-27T23:50:43.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.618+0000 7f8ed7b2d700 1 -- 172.21.15.64:0/1985508853 wait complete. 2023-09-27T23:50:43.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:50:43.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb30fe700 1 Processor -- start 2023-09-27T23:50:43.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb30fe700 1 -- start start 2023-09-27T23:50:43.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb30fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1fac159340 0x7f1fac067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb30fe700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac068c70 con 0x7f1fac158f70 2023-09-27T23:50:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb30fe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac068060 con 0x7f1fac069100 2023-09-27T23:50:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb30fe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac068340 con 0x7f1fac159340 2023-09-27T23:50:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb0e9a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1fac158f70 0x7f1fac0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41006/0 (socket says 172.21.15.64:41006) 2023-09-27T23:50:43.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb0e9a700 1 -- 172.21.15.64:0/4094002176 learned_addr learned my addr 172.21.15.64:0/4094002176 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb169b700 1 --2- 172.21.15.64:0/4094002176 >> v2:172.21.15.123:3300/0 conn(0x7f1fac159340 0x7f1fac067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557549702 0 0) 0x7f1fac068c70 con 0x7f1fac158f70 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f8c003570 con 0x7f1fac158f70 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075132388 0 0) 0x7f1fac068060 con 0x7f1fac069100 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac068c70 con 0x7f1fac069100 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb169b700 1 -- 172.21.15.64:0/4094002176 >> v1:172.21.15.123:6789/0 conn(0x7f1fac069100 legacy=0x7f1fac0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb169b700 1 -- 172.21.15.64:0/4094002176 >> v1:172.21.15.64:6789/0 conn(0x7f1fac158f70 legacy=0x7f1fac0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb169b700 1 -- 172.21.15.64:0/4094002176 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac068620 con 0x7f1fac159340 2023-09-27T23:50:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.750+0000 7f1fb169b700 1 --2- 172.21.15.64:0/4094002176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac159340 0x7f1fac067980 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009e90 tx=0x7f1fa4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=113c74806b17c1fa server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400e070 con 0x7f1fac159340 2023-09-27T23:50:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa400bb90 con 0x7f1fac159340 2023-09-27T23:50:43.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fab7fe700 1 -- 172.21.15.64:0/4094002176 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4012430 con 0x7f1fac159340 2023-09-27T23:50:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4094002176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac159340 msgr2=0x7f1fac067980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4094002176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac159340 0x7f1fac067980 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009e90 tx=0x7f1fa4009780 comp rx=0 tx=0).stop 2023-09-27T23:50:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4094002176 shutdown_connections 2023-09-27T23:50:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4094002176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac159340 0x7f1fac067980 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4094002176 >> 172.21.15.64:0/4094002176 conn(0x7f1fac0ccc40 msgr2=0x7f1fac0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4094002176 shutdown_connections 2023-09-27T23:50:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4094002176 wait complete. 2023-09-27T23:50:43.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 Processor -- start 2023-09-27T23:50:43.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.754+0000 7f1fb30fe700 1 -- start start 2023-09-27T23:50:43.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac069100 0x7f1fac16c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac158f70 0x7f1fac1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 0x7f1fac14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac151820 con 0x7f1fac159340 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac16ccd0 con 0x7f1fac069100 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac16cfb0 con 0x7f1fac158f70 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 0x7f1fac14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 0x7f1fac14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54658/0 (socket says 172.21.15.64:54658) 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 -- 172.21.15.64:0/4143714225 learned_addr learned my addr 172.21.15.64:0/4143714225 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fabfff700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac158f70 0x7f1fac1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb0e9a700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac069100 0x7f1fac16c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 -- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac158f70 msgr2=0x7f1fac1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac158f70 0x7f1fac1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 -- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac069100 msgr2=0x7f1fac16c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac069100 0x7f1fac16c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 -- 172.21.15.64:0/4143714225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4009a20 con 0x7f1fac159340 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb169b700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 0x7f1fac14cfd0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4013000 tx=0x7f1fa4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa401b420 con 0x7f1fac159340 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa401b5a0 con 0x7f1fac159340 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac16d290 con 0x7f1fac159340 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa401b960 con 0x7f1fac159340 2023-09-27T23:50:43.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac14d510 con 0x7f1fac159340 2023-09-27T23:50:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.758+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac0c87a0 con 0x7f1fac159340 2023-09-27T23:50:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.762+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1fa401bcd0 con 0x7f1fac159340 2023-09-27T23:50:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.762+0000 7f1fa8ff9700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f90060a70 0x7f1f90062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.762+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1fa407eda0 con 0x7f1fac159340 2023-09-27T23:50:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.762+0000 7f1fb0e9a700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f90060a70 0x7f1f90062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.762+0000 7f1fb0e9a700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f90060a70 0x7f1f90062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c007fd0 tx=0x7f1f9c009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.770+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1fac0c87a0 con 0x7f1fac159340 2023-09-27T23:50:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.970+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1fac0c87a0 con 0x7f1fac159340 2023-09-27T23:50:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.970+0000 7f1fa8ff9700 1 -- 172.21.15.64:0/4143714225 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1fac0c87a0 con 0x7f1fac159340 2023-09-27T23:50:43.975 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f90060a70 msgr2=0x7f1f90062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f90060a70 0x7f1f90062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c007fd0 tx=0x7f1f9c009040 comp rx=0 tx=0).stop 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 msgr2=0x7f1fac14cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 0x7f1fac14cfd0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4013000 tx=0x7f1fa4019000 comp rx=0 tx=0).stop 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 shutdown_connections 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f90060a70 0x7f1f90062f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fac159340 0x7f1fac14cfd0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fac158f70 0x7f1fac1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 --2- 172.21.15.64:0/4143714225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fac069100 0x7f1fac16c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 >> 172.21.15.64:0/4143714225 conn(0x7f1fac0ccc40 msgr2=0x7f1fac0b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 shutdown_connections 2023-09-27T23:50:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:43.974+0000 7f1fb30fe700 1 -- 172.21.15.64:0/4143714225 wait complete. 2023-09-27T23:50:43.989 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:50:43.989 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:50:43.989 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:50:43.990 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:50:43.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:43.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:50:43.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:50:43.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:50:43.990 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:50:43.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:43.990 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:50:43.998 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 40340sec preferred_lft 40340sec 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:50:43.999 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:50:44.000 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:44.001 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:50:44.001 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:50:44.001 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:50:44.001 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:50:44.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:44.001 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:50:44.015 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:50:44.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:44.017 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:50:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:50:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:50:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:50:44.068 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:50:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:50:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:50:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:50:44.069 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:50:44.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:44.069 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:50:44.069 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:50:44.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:50:44.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:50:44.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:50:44.070 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:50:44.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:44.110 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:50:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:50:44.116 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:50:44.116 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:50:44.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:50:44.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:50:44.129 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:50:44.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:50:44.129 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:44.135 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:50:44.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:50:44.135 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:44.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:50:44.199 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:50:44.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:50:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:50:44.210 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:50:44.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:44.210 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:50:44.215 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:50:44.215 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:50:44.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:50:44.279 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:50:44.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:50:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:50:44.288 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:50:44.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:44.289 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:50:44.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:50:44.342+0000 7fdbd19b6380 -1 init, newargv = 0x5621cb708dd0 newargc=15 2023-09-27T23:50:44.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[60532]: starting ceph client 2023-09-27T23:50:44.354 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[60532]: starting fuse 2023-09-27T23:50:45.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:50:45.459 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:50:45.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:50:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:50:45.520 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:50:45.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:45.521 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:50:45.528 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:50:45.528 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:50:45.528 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:50:45.528 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:50:45.528 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:50:45.529 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:50:45.530 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:50:45.531 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:50:45.531 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:50:45.616 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.60532.asok 2023-09-27T23:50:45.799 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60532.asok 2023-09-27T23:50:45.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:45.799 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60532.asok status 2023-09-27T23:50:45.936 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:50:45.936 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:50:45.936 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:50:45.936 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:50:45.936 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "60532", 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 9966, 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 9966 2023-09-27T23:50:45.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1644862408 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1644862408 2023-09-27T23:50:45.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.9966 192.168.0.1:0/1644862408", 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/1644862408", 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 66, 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 174, 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:50:45.939 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:50:45.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:45.946 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:50:45.946 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:45.954 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:50:45.954 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:45.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:45.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:50:45.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:50:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.078+0000 7f666db72700 1 Processor -- start 2023-09-27T23:50:46.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.078+0000 7f666db72700 1 -- start start 2023-09-27T23:50:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f666806a580 0x7f666806aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666806c7c0 con 0x7f66680b1410 2023-09-27T23:50:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666806cd80 con 0x7f666806d210 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666806af40 con 0x7f666806a580 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f666806a580 0x7f666806aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666cb70700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f66680b1410 0x7f66680ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33726/0 (socket says 172.21.15.64:33726) 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666cb70700 1 -- 172.21.15.64:0/2707408905 learned_addr learned my addr 172.21.15.64:0/2707408905 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665ffff700 1 -- 172.21.15.64:0/2707408905 >> v1:172.21.15.123:6789/0 conn(0x7f666806d210 legacy=0x7f666806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665ffff700 1 -- 172.21.15.64:0/2707408905 >> v1:172.21.15.64:6789/0 conn(0x7f66680b1410 legacy=0x7f66680ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665ffff700 1 -- 172.21.15.64:0/2707408905 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666806b8f0 con 0x7f666806a580 2023-09-27T23:50:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665ffff700 1 --2- 172.21.15.64:0/2707408905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806a580 0x7f666806aa00 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f66580098e0 tx=0x7f66580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1040215d839ccfd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665f7fe700 1 -- 172.21.15.64:0/2707408905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658009e80 con 0x7f666806a580 2023-09-27T23:50:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665f7fe700 1 -- 172.21.15.64:0/2707408905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665800b860 con 0x7f666806a580 2023-09-27T23:50:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f665f7fe700 1 -- 172.21.15.64:0/2707408905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665800bbe0 con 0x7f666806a580 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- 172.21.15.64:0/2707408905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806a580 msgr2=0x7f666806aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 --2- 172.21.15.64:0/2707408905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806a580 0x7f666806aa00 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f66580098e0 tx=0x7f66580091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- 172.21.15.64:0/2707408905 shutdown_connections 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 --2- 172.21.15.64:0/2707408905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806a580 0x7f666806aa00 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- 172.21.15.64:0/2707408905 >> 172.21.15.64:0/2707408905 conn(0x7f6668067bb0 msgr2=0x7f66680cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- 172.21.15.64:0/2707408905 shutdown_connections 2023-09-27T23:50:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.082+0000 7f666db72700 1 -- 172.21.15.64:0/2707408905 wait complete. 2023-09-27T23:50:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.086+0000 7f666db72700 1 Processor -- start 2023-09-27T23:50:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.086+0000 7f666db72700 1 -- start start 2023-09-27T23:50:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f666806a580 0x7f6668146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 0x7f66680c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66680b1410 0x7f66680c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668146980 con 0x7f666806d210 2023-09-27T23:50:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66680c2890 con 0x7f666806a580 2023-09-27T23:50:46.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66680c2b70 con 0x7f66680b1410 2023-09-27T23:50:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 0x7f66680c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666cb70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f666806a580 0x7f6668146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 0x7f66680c2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51186/0 (socket says 172.21.15.64:51186) 2023-09-27T23:50:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 -- 172.21.15.64:0/902316550 learned_addr learned my addr 172.21.15.64:0/902316550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:46.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666d371700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66680b1410 0x7f66680c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 -- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66680b1410 msgr2=0x7f66680c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66680b1410 0x7f66680c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 -- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f666806a580 msgr2=0x7f6668146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f666806a580 0x7f6668146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 -- 172.21.15.64:0/902316550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658009420 con 0x7f666806d210 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666d371700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66680b1410 0x7f66680c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:46.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665ffff700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 0x7f66680c2040 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6668064040 tx=0x7f6658017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665800bbe0 con 0x7f666806d210 2023-09-27T23:50:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66680c2e50 con 0x7f666806d210 2023-09-27T23:50:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658011420 con 0x7f666806d210 2023-09-27T23:50:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658011800 con 0x7f666806d210 2023-09-27T23:50:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66680c3410 con 0x7f666806d210 2023-09-27T23:50:46.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66680c3600 con 0x7f666806d210 2023-09-27T23:50:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.090+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6658011bf0 con 0x7f666806d210 2023-09-27T23:50:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.094+0000 7f665cff9700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6654060a70 0x7f6654062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.094+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f665807ecb0 con 0x7f666806d210 2023-09-27T23:50:46.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.094+0000 7f666cb70700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6654060a70 0x7f6654062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.094+0000 7f666cb70700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6654060a70 0x7f6654062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6664001950 tx=0x7f6664009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.102+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f665804ece0 con 0x7f666806d210 2023-09-27T23:50:46.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.282+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f666804be40 con 0x7f6654060a70 2023-09-27T23:50:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.286+0000 7f665cff9700 1 -- 172.21.15.64:0/902316550 <== mgr.8210 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f666804be40 con 0x7f6654060a70 2023-09-27T23:50:46.290 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:50:46.290 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:50:46.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:50:46.290 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:50:46.290 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:50:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.286+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6654060a70 msgr2=0x7f6654062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.286+0000 7f666db72700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6654060a70 0x7f6654062f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6664001950 tx=0x7f6664009040 comp rx=0 tx=0).stop 2023-09-27T23:50:46.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.286+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 msgr2=0x7f66680c2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.286+0000 7f666db72700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 0x7f66680c2040 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6668064040 tx=0x7f6658017000 comp rx=0 tx=0).stop 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 shutdown_connections 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6654060a70 0x7f6654062f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66680b1410 0x7f66680c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666806d210 0x7f66680c2040 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 --2- 172.21.15.64:0/902316550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f666806a580 0x7f6668146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 >> 172.21.15.64:0/902316550 conn(0x7f6668067bb0 msgr2=0x7f6668156210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 shutdown_connections 2023-09-27T23:50:46.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.290+0000 7f666db72700 1 -- 172.21.15.64:0/902316550 wait complete. 2023-09-27T23:50:46.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:50:46.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.410+0000 7f1a9ec2f700 1 Processor -- start 2023-09-27T23:50:46.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9ec2f700 1 -- start start 2023-09-27T23:50:46.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9ec2f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a9806baa0 0x7f1a9815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9ec2f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 0x7f1a980adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9ec2f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9815ae30 con 0x7f1a98154780 2023-09-27T23:50:46.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9ec2f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a980abf70 con 0x7f1a9806baa0 2023-09-27T23:50:46.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9ec2f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98154b50 con 0x7f1a980adab0 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 0x7f1a980adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9d42c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a9806baa0 0x7f1a9815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 0x7f1a980adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:46960/0 (socket says 172.21.15.64:46960) 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 -- 172.21.15.64:0/820838319 learned_addr learned my addr 172.21.15.64:0/820838319 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 -- 172.21.15.64:0/820838319 >> v2:172.21.15.123:3300/0 conn(0x7f1a9806baa0 msgr2=0x7f1a9815b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 --2- 172.21.15.64:0/820838319 >> v2:172.21.15.123:3300/0 conn(0x7f1a9806baa0 0x7f1a9815b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 -- 172.21.15.64:0/820838319 >> v1:172.21.15.123:6789/0 conn(0x7f1a98154780 legacy=0x7f1a98157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 -- 172.21.15.64:0/820838319 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a980ac210 con 0x7f1a980adab0 2023-09-27T23:50:46.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9e42e700 1 --2- 172.21.15.64:0/820838319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 0x7f1a980adef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009f20 tx=0x7f1a90009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9936cac3539b0fb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9cc2b700 1 -- 172.21.15.64:0/820838319 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9000e070 con 0x7f1a980adab0 2023-09-27T23:50:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9cc2b700 1 -- 172.21.15.64:0/820838319 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9000b9d0 con 0x7f1a980adab0 2023-09-27T23:50:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.414+0000 7f1a9cc2b700 1 -- 172.21.15.64:0/820838319 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a900092e0 con 0x7f1a980adab0 2023-09-27T23:50:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/820838319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 msgr2=0x7f1a980adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/820838319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 0x7f1a980adef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009f20 tx=0x7f1a90009840 comp rx=0 tx=0).stop 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/820838319 shutdown_connections 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/820838319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a980adab0 0x7f1a980adef0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/820838319 >> v2:172.21.15.123:3300/0 conn(0x7f1a9806baa0 0x7f1a9815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/820838319 >> 172.21.15.64:0/820838319 conn(0x7f1a98067b50 msgr2=0x7f1a980cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/820838319 shutdown_connections 2023-09-27T23:50:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/820838319 wait complete. 2023-09-27T23:50:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 Processor -- start 2023-09-27T23:50:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.418+0000 7f1a9ec2f700 1 -- start start 2023-09-27T23:50:46.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a9806baa0 0x7f1a981453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 0x7f1a980af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a98154780 0x7f1a980afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98158b00 con 0x7f1a9806baa0 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a981587b0 con 0x7f1a980adab0 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980ca7f0 con 0x7f1a98154780 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9dc2d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a9806baa0 0x7f1a981453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 0x7f1a980af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 0x7f1a980af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:46670/0 (socket says 172.21.15.64:46670) 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 -- 172.21.15.64:0/448367127 learned_addr learned my addr 172.21.15.64:0/448367127 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9e42e700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a98154780 0x7f1a980afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 -- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a98154780 msgr2=0x7f1a980afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a98154780 0x7f1a980afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 -- 172.21.15.64:0/448367127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a9806baa0 msgr2=0x7f1a981453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a9806baa0 0x7f1a981453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 -- 172.21.15.64:0/448367127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90009ae0 con 0x7f1a980adab0 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9dc2d700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a9806baa0 0x7f1a981453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9d42c700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 0x7f1a980af600 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800e900 tx=0x7f1a880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88013680 con 0x7f1a980adab0 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a880137e0 con 0x7f1a980adab0 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a980caad0 con 0x7f1a980adab0 2023-09-27T23:50:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88013ba0 con 0x7f1a980adab0 2023-09-27T23:50:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.422+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a98145930 con 0x7f1a980adab0 2023-09-27T23:50:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.426+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a98158cb0 con 0x7f1a980adab0 2023-09-27T23:50:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.426+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 80) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1a880123d0 con 0x7f1a980adab0 2023-09-27T23:50:46.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.426+0000 7f1a8dffb700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1a7c060a70 0x7f1a7c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.426+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1a8807d190 con 0x7f1a980adab0 2023-09-27T23:50:46.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.426+0000 7f1a9dc2d700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1a7c060a70 0x7f1a7c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.430+0000 7f1a9dc2d700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1a7c060a70 0x7f1a7c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a94001d20 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.8210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.438+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1a98158cb0 con 0x7f1a980adab0 2023-09-27T23:50:46.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.642+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1a98158940 con 0x7f1a980adab0 2023-09-27T23:50:46.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.766+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 81) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f1a880560c0 con 0x7f1a980adab0 2023-09-27T23:50:46.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a8dffb700 1 -- 172.21.15.64:0/448367127 <== mon.1 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v81) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1a88050c50 con 0x7f1a980adab0 2023-09-27T23:50:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1a7c060a70 msgr2=0x7f1a7c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1a7c060a70 0x7f1a7c062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a94001d20 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 msgr2=0x7f1a980af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 0x7f1a980af600 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800e900 tx=0x7f1a880091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 shutdown_connections 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1a7c060a70 0x7f1a7c062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a98154780 0x7f1a980afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a980adab0 0x7f1a980af600 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 --2- 172.21.15.64:0/448367127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a9806baa0 0x7f1a981453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 >> 172.21.15.64:0/448367127 conn(0x7f1a98067b50 msgr2=0x7f1a981533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 shutdown_connections 2023-09-27T23:50:46.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.770+0000 7f1a9ec2f700 1 -- 172.21.15.64:0/448367127 wait complete. 2023-09-27T23:50:46.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:50:46.884 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:46.885+0000 7f22821bb000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:50:46.885 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:46.882+0000 7ff890edd000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:50:46.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda80a7700 1 Processor -- start 2023-09-27T23:50:46.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda80a7700 1 -- start start 2023-09-27T23:50:46.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda80a7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efda00cc110 0x7efda01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda80a7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda0156af0 con 0x7efda013fbb0 2023-09-27T23:50:46.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda80a7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda0156dd0 con 0x7efda013f7e0 2023-09-27T23:50:46.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda80a7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda0157950 con 0x7efda00cc110 2023-09-27T23:50:46.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda6644700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efda00cc110 0x7efda01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda5642700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efda013fbb0 0x7efda00cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33754/0 (socket says 172.21.15.64:33754) 2023-09-27T23:50:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda5642700 1 -- 172.21.15.64:0/181694059 learned_addr learned my addr 172.21.15.64:0/181694059 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda6644700 1 -- 172.21.15.64:0/181694059 >> v1:172.21.15.123:6789/0 conn(0x7efda013f7e0 legacy=0x7efda00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda6644700 1 -- 172.21.15.64:0/181694059 >> v1:172.21.15.64:6789/0 conn(0x7efda013fbb0 legacy=0x7efda00cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda6644700 1 -- 172.21.15.64:0/181694059 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda01570b0 con 0x7efda00cc110 2023-09-27T23:50:46.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.910+0000 7efda6644700 1 --2- 172.21.15.64:0/181694059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda01564a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e60 tx=0x7efd98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d4b17fa651f296 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda4e41700 1 -- 172.21.15.64:0/181694059 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9800e070 con 0x7efda00cc110 2023-09-27T23:50:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda4e41700 1 -- 172.21.15.64:0/181694059 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9800bcc0 con 0x7efda00cc110 2023-09-27T23:50:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda4e41700 1 -- 172.21.15.64:0/181694059 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98012680 con 0x7efda00cc110 2023-09-27T23:50:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- 172.21.15.64:0/181694059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 msgr2=0x7efda01564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 --2- 172.21.15.64:0/181694059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda01564a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e60 tx=0x7efd98009780 comp rx=0 tx=0).stop 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- 172.21.15.64:0/181694059 shutdown_connections 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 --2- 172.21.15.64:0/181694059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda01564a0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- 172.21.15.64:0/181694059 >> 172.21.15.64:0/181694059 conn(0x7efda00cd8f0 msgr2=0x7efda00c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- 172.21.15.64:0/181694059 shutdown_connections 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- 172.21.15.64:0/181694059 wait complete. 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 Processor -- start 2023-09-27T23:50:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- start start 2023-09-27T23:50:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda0155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda013f7e0 0x7efda00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efda013fbb0 0x7efda0067a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda013a590 con 0x7efda00cc110 2023-09-27T23:50:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0139fd0 con 0x7efda013f7e0 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda80a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda013a2b0 con 0x7efda013fbb0 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda5e43700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda0155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda5e43700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda0155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51236/0 (socket says 172.21.15.64:51236) 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda5e43700 1 -- 172.21.15.64:0/1031667756 learned_addr learned my addr 172.21.15.64:0/1031667756 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda5642700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda013f7e0 0x7efda00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda6644700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efda013fbb0 0x7efda0067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda5e43700 1 -- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efda013fbb0 msgr2=0x7efda0067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.914+0000 7efda5e43700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efda013fbb0 0x7efda0067a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda5e43700 1 -- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda013f7e0 msgr2=0x7efda00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda5e43700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda013f7e0 0x7efda00c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda5e43700 1 -- 172.21.15.64:0/1031667756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009a20 con 0x7efda00cc110 2023-09-27T23:50:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda6644700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efda013fbb0 0x7efda0067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:50:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda5e43700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda0155a20 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c590 tx=0x7efd9c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c011590 con 0x7efda00cc110 2023-09-27T23:50:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c0116f0 con 0x7efda00cc110 2023-09-27T23:50:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c011ab0 con 0x7efda00cc110 2023-09-27T23:50:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda00c8d90 con 0x7efda00cc110 2023-09-27T23:50:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda0155f60 con 0x7efda00cc110 2023-09-27T23:50:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 81) v1 ==== 80378+0+0 (secure 0 0 0) 0x7efd9c01b3b0 con 0x7efda00cc110 2023-09-27T23:50:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efd95ffb700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd84060af0 0x7efd84062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd9c07c770 con 0x7efda00cc110 2023-09-27T23:50:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda5642700 1 -- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd84060af0 msgr2=0x7efd84062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.918+0000 7efda5642700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd84060af0 0x7efd84062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:46.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.922+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda00c9290 con 0x7efda00cc110 2023-09-27T23:50:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:46.930+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efd9c017080 con 0x7efda00cc110 2023-09-27T23:50:47.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.090+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7efda0071120 con 0x7efda00cc110 2023-09-27T23:50:47.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.098+0000 7efd95ffb700 1 -- 172.21.15.64:0/1031667756 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7efd9c04c7a0 con 0x7efda00cc110 2023-09-27T23:50:47.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd84060af0 msgr2=0x7efd84062fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd84060af0 0x7efd84062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 msgr2=0x7efda0155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda0155a20 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c590 tx=0x7efd9c00c950 comp rx=0 tx=0).stop 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 shutdown_connections 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd84060af0 0x7efd84062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efda013fbb0 0x7efda0067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efda013f7e0 0x7efda00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 --2- 172.21.15.64:0/1031667756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda00cc110 0x7efda0155a20 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 >> 172.21.15.64:0/1031667756 conn(0x7efda00cd8f0 msgr2=0x7efda0158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 shutdown_connections 2023-09-27T23:50:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.102+0000 7efda80a7700 1 -- 172.21.15.64:0/1031667756 wait complete. 2023-09-27T23:50:47.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:50:47.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.226+0000 7fe24489f700 1 Processor -- start 2023-09-27T23:50:47.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- start start 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe24006dbb0 0x7fe2400b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400ac390 con 0x7fe2400ab6d0 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400ac670 con 0x7fe24006ae20 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400ac950 con 0x7fe24006dbb0 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe2400ab6d0 0x7fe2400abb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33770/0 (socket says 172.21.15.64:33770) 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23effd700 1 -- 172.21.15.64:0/1021672419 learned_addr learned my addr 172.21.15.64:0/1021672419 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:47.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23ffff700 1 --2- 172.21.15.64:0/1021672419 >> v2:172.21.15.64:3300/0 conn(0x7fe24006dbb0 0x7fe2400b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23e7fc700 1 -- 172.21.15.64:0/1021672419 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376384071 0 0) 0x7fe2400ac390 con 0x7fe2400ab6d0 2023-09-27T23:50:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23e7fc700 1 -- 172.21.15.64:0/1021672419 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe228003570 con 0x7fe2400ab6d0 2023-09-27T23:50:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23ffff700 1 -- 172.21.15.64:0/1021672419 >> v1:172.21.15.123:6789/0 conn(0x7fe24006ae20 legacy=0x7fe24006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23ffff700 1 -- 172.21.15.64:0/1021672419 >> v1:172.21.15.64:6789/0 conn(0x7fe2400ab6d0 legacy=0x7fe2400abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23ffff700 1 -- 172.21.15.64:0/1021672419 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400acc30 con 0x7fe24006dbb0 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23ffff700 1 --2- 172.21.15.64:0/1021672419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe24006dbb0 0x7fe2400b1440 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fe238009e60 tx=0x7fe238009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f963b34c1d6fb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23e7fc700 1 -- 172.21.15.64:0/1021672419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800e070 con 0x7fe24006dbb0 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23e7fc700 1 -- 172.21.15.64:0/1021672419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800bb60 con 0x7fe24006dbb0 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe23e7fc700 1 -- 172.21.15.64:0/1021672419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012430 con 0x7fe24006dbb0 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- 172.21.15.64:0/1021672419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe24006dbb0 msgr2=0x7fe2400b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 --2- 172.21.15.64:0/1021672419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe24006dbb0 0x7fe2400b1440 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fe238009e60 tx=0x7fe238009780 comp rx=0 tx=0).stop 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- 172.21.15.64:0/1021672419 shutdown_connections 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 --2- 172.21.15.64:0/1021672419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe24006dbb0 0x7fe2400b1440 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- 172.21.15.64:0/1021672419 >> 172.21.15.64:0/1021672419 conn(0x7fe2400c8cd0 msgr2=0x7fe2400c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- 172.21.15.64:0/1021672419 shutdown_connections 2023-09-27T23:50:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.230+0000 7fe24489f700 1 -- 172.21.15.64:0/1021672419 wait complete. 2023-09-27T23:50:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 Processor -- start 2023-09-27T23:50:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 -- start start 2023-09-27T23:50:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe24006ae20 0x7fe240155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe24006dbb0 0x7fe2400b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 0x7fe2400b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240168f70 con 0x7fe2400ab6d0 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240169250 con 0x7fe24006dbb0 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe240169530 con 0x7fe24006ae20 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 0x7fe2400b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 0x7fe2400b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51252/0 (socket says 172.21.15.64:51252) 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 -- 172.21.15.64:0/149159976 learned_addr learned my addr 172.21.15.64:0/149159976 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23effd700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe24006dbb0 0x7fe2400b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23f7fe700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe24006ae20 0x7fe240155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 -- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe24006ae20 msgr2=0x7fe240155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe24006ae20 0x7fe240155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 -- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe24006dbb0 msgr2=0x7fe2400b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe24006dbb0 0x7fe2400b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 -- 172.21.15.64:0/149159976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238009a20 con 0x7fe2400ab6d0 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23effd700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe24006dbb0 0x7fe2400b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe23ffff700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 0x7fe2400b0f40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe2400680c0 tx=0x7fe238019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238027470 con 0x7fe2400ab6d0 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2380275f0 con 0x7fe2400ab6d0 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2380279b0 con 0x7fe2400ab6d0 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.234+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe240169810 con 0x7fe2400ab6d0 2023-09-27T23:50:47.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.238+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe240155970 con 0x7fe2400ab6d0 2023-09-27T23:50:47.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.238+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240169d10 con 0x7fe2400ab6d0 2023-09-27T23:50:47.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.238+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 81) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fe238024070 con 0x7fe2400ab6d0 2023-09-27T23:50:47.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.238+0000 7fe2277fe700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe228060ea0 0x7fe228063350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.238+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe23807fa60 con 0x7fe2400ab6d0 2023-09-27T23:50:47.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.246+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe23804fa90 con 0x7fe2400ab6d0 2023-09-27T23:50:47.311 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.309+0000 7f22821bb000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:50:47.348 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:47.346+0000 7ff890edd000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:50:47.358 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.357+0000 7f22821bb000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:50:47.401 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:47.398+0000 7ff890edd000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:50:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.406+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe24004be40 con 0x7fe2400ab6d0 2023-09-27T23:50:47.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.418+0000 7fe2277fe700 1 -- 172.21.15.64:0/149159976 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe2380530b0 con 0x7fe2400ab6d0 2023-09-27T23:50:47.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe228060ea0 msgr2=0x7fe228063350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-27T23:50:47.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe228060ea0 0x7fe228063350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 msgr2=0x7fe2400b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 0x7fe2400b0f40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe2400680c0 tx=0x7fe238019000 comp rx=0 tx=0).stop 2023-09-27T23:50:47.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 shutdown_connections 2023-09-27T23:50:47.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe228060ea0 0x7fe228063350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2400ab6d0 0x7fe2400b0f40 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe24006dbb0 0x7fe2400b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 --2- 172.21.15.64:0/149159976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe24006ae20 0x7fe240155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 >> 172.21.15.64:0/149159976 conn(0x7fe2400c8cd0 msgr2=0x7fe2400cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:47.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 shutdown_connections 2023-09-27T23:50:47.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.422+0000 7fe24489f700 1 -- 172.21.15.64:0/149159976 wait complete. 2023-09-27T23:50:47.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:47.436 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:50:47.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-09-27T23:50:47.465 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.465+0000 7f22821bb000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:47.521 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:47.518+0000 7ff890edd000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 Processor -- start 2023-09-27T23:50:47.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 -- start start 2023-09-27T23:50:47.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b6c158b00 0x7f0b6c158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b6c06bb70 0x7f0b6c06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c06d9b0 con 0x7f0b6c0abe40 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c85a0 con 0x7f0b6c158b00 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b716e2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c67d0 con 0x7f0b6c06bb70 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b6c158b00 0x7f0b6c158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b6c158b00 0x7f0b6c158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51256/0 (socket says 172.21.15.64:51256) 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2925221133 learned_addr learned my addr 172.21.15.64:0/2925221133 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b70ee1700 1 --2- 172.21.15.64:0/2925221133 >> v2:172.21.15.123:3300/0 conn(0x7f0b6c06bb70 0x7f0b6c06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2925221133 >> v2:172.21.15.123:3300/0 conn(0x7f0b6c06bb70 msgr2=0x7f0b6c06ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:47.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2925221133 >> v2:172.21.15.123:3300/0 conn(0x7f0b6c06bb70 0x7f0b6c06ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2925221133 >> v1:172.21.15.123:6789/0 conn(0x7f0b6c0abe40 legacy=0x7f0b6c06d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2925221133 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0c6ab0 con 0x7f0b6c158b00 2023-09-27T23:50:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.566+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2925221133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c158ed0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0c3130 tx=0x7f0b5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5ef94c4b08f6b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b6affd700 1 -- 172.21.15.64:0/2925221133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c009e70 con 0x7f0b6c158b00 2023-09-27T23:50:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b6affd700 1 -- 172.21.15.64:0/2925221133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c00b860 con 0x7f0b6c158b00 2023-09-27T23:50:47.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b6affd700 1 -- 172.21.15.64:0/2925221133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00bc00 con 0x7f0b6c158b00 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 -- 172.21.15.64:0/2925221133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 msgr2=0x7f0b6c158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2925221133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c158ed0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0c3130 tx=0x7f0b5c009250 comp rx=0 tx=0).stop 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 -- 172.21.15.64:0/2925221133 shutdown_connections 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2925221133 >> v2:172.21.15.123:3300/0 conn(0x7f0b6c06bb70 0x7f0b6c06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2925221133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c158ed0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 -- 172.21.15.64:0/2925221133 >> 172.21.15.64:0/2925221133 conn(0x7f0b6c0c8c00 msgr2=0x7f0b6c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 -- 172.21.15.64:0/2925221133 shutdown_connections 2023-09-27T23:50:47.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 -- 172.21.15.64:0/2925221133 wait complete. 2023-09-27T23:50:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 Processor -- start 2023-09-27T23:50:47.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.570+0000 7f0b716e2700 1 -- start start 2023-09-27T23:50:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b6c06bb70 0x7f0b6c0c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b6c0abe40 0x7f0b6c0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c24b0 con 0x7f0b6c158b00 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c1600 con 0x7f0b6c0abe40 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c154030 con 0x7f0b6c06bb70 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c1461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51266/0 (socket says 172.21.15.64:51266) 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 -- 172.21.15.64:0/2916508962 learned_addr learned my addr 172.21.15.64:0/2916508962 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b6c06bb70 0x7f0b6c0c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b6b7fe700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b6c0abe40 0x7f0b6c0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:47.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b6c06bb70 msgr2=0x7f0b6c0c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b6c06bb70 0x7f0b6c0c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b6c0abe40 msgr2=0x7f0b6c0b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:47.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b6c0abe40 0x7f0b6c0b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 -- 172.21.15.64:0/2916508962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0094f0 con 0x7f0b6c158b00 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b70ee1700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c1461c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009ec0 tx=0x7f0b6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6400cb70 con 0x7f0b6c158b00 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6400f040 con 0x7f0b6c158b00 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c0c1830 con 0x7f0b6c158b00 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64014480 con 0x7f0b6c158b00 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c0c20d0 con 0x7f0b6c158b00 2023-09-27T23:50:47.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.574+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c0c25f0 con 0x7f0b6c158b00 2023-09-27T23:50:47.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.578+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 81) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f0b64014830 con 0x7f0b6c158b00 2023-09-27T23:50:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.578+0000 7f0b53fff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.578+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 msgr2=0x7f0b58062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.578+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0b6407ddf0 con 0x7f0b6c158b00 2023-09-27T23:50:47.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.578+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:50:47.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.586+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0b6c0c25f0 con 0x7f0b6c158b00 2023-09-27T23:50:47.659 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.657+0000 7f22821bb000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:50:47.708 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.705+0000 7f22821bb000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:47.741 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:47.738+0000 7ff890edd000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:50:47.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.774+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b6c0c1c80 con 0x7f0b58060af0 2023-09-27T23:50:47.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.782+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 msgr2=0x7f0b58062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:47.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:47.782+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:50:47.795 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:47.790+0000 7ff890edd000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:47.860 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.857+0000 7f22821bb000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:50:47.915 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.913+0000 7f22821bb000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:50:47.966 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:47.962+0000 7ff890edd000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:50:47.974 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:47.973+0000 7f22821bb000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:50:48.030 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:48.026+0000 7ff890edd000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:50:48.095 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:48.090+0000 7ff890edd000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:50:48.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:48.186+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 msgr2=0x7f0b58062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:48.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:48.186+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:50:48.268 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.265+0000 7f22821bb000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:50:48.378 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.377+0000 7f22821bb000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:50:48.433 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:48.430+0000 7ff890edd000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:50:48.558 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:48.554+0000 7ff890edd000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:50:48.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.657+0000 7f22821bb000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:50:48.819 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.817+0000 7f22821bb000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:48.875 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:48.870+0000 7ff890edd000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:50:48.883 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.881+0000 7f22821bb000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:50:48.937 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.937+0000 7f22821bb000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:50:48.991 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:48.989+0000 7f22821bb000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:50:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:48.990+0000 7f0b6bfff700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 msgr2=0x7f0b58062fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:50:48.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:48.990+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:50:49.045 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.042+0000 7ff890edd000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:50:49.113 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.110+0000 7ff890edd000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:50:49.164 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:49.165+0000 7f22821bb000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:49.168 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.166+0000 7ff890edd000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:50:49.217 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:49.217+0000 7f22821bb000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:50:49.224 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.222+0000 7ff890edd000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:50:49.280 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:49.281+0000 7f22821bb000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:50:49.403 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.398+0000 7ff890edd000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:50:49.428 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:49.429+0000 7f22821bb000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:50:49.460 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.458+0000 7ff890edd000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:50:49.487 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:49.485+0000 7f22821bb000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:50:49.526 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.522+0000 7ff890edd000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:50:49.682 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.678+0000 7ff890edd000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:50:49.746 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:49.742+0000 7ff890edd000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:50:50.077 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.077+0000 7f22821bb000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:50:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:50.110+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 82) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f0b6407c060 con 0x7f0b6c158b00 2023-09-27T23:50:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:50.110+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 msgr2=0x7f0b58062fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:50:50.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:50.110+0000 7f0b53fff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f2242b66700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f2242b66700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f2242b66700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f2242b66700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f2242b66700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f224536b700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f224536b700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.253 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f224536b700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.254 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f224536b700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.254 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:50:50.253+0000 7f224536b700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:50:50.377 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:50:50.374+0000 7ff890edd000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:50:51.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.118+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 83) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f0b64024300 con 0x7f0b6c158b00 2023-09-27T23:50:51.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.118+0000 7f0b53fff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.118+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b6c0c1c80 con 0x7f0b58060af0 2023-09-27T23:50:51.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.118+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.122+0000 7f0b6bfff700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0c3130 tx=0x7f0b5c017000 comp rx=0 tx=0).ready entity=mgr.8525 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.222+0000 7f0b53fff700 1 -- 172.21.15.64:0/2916508962 <== mgr.8525 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b6c0c1c80 con 0x7f0b58060af0 2023-09-27T23:50:51.224 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path snap_test_dir1 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 msgr2=0x7f0b58062fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0c3130 tx=0x7f0b5c017000 comp rx=0 tx=0).stop 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 msgr2=0x7f0b6c1461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c1461c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009ec0 tx=0x7f0b6400b040 comp rx=0 tx=0).stop 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 shutdown_connections 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0b58060af0 0x7f0b58062fa0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b6c158b00 0x7f0b6c1461c0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b6c0abe40 0x7f0b6c0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 --2- 172.21.15.64:0/2916508962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b6c06bb70 0x7f0b6c0c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 >> 172.21.15.64:0/2916508962 conn(0x7f0b6c0c8c00 msgr2=0x7f0b6c067310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:51.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 shutdown_connections 2023-09-27T23:50:51.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.226+0000 7f0b716e2700 1 -- 172.21.15.64:0/2916508962 wait complete. 2023-09-27T23:50:51.247 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-09-27T23:50:51.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-09-27T23:50:51.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.370+0000 7f52becb0700 1 Processor -- start 2023-09-27T23:50:51.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.370+0000 7f52becb0700 1 -- start start 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.370+0000 7f52becb0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b80ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52b80acc60 0x7f52b80cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b80ad0c0 con 0x7f52b8150200 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b80af530 con 0x7f52b80acc60 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80c75e0 con 0x7f52b80c7170 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bdcae700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f52b8150200 0x7f52b80ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:33786/0 (socket says 172.21.15.64:33786) 2023-09-27T23:50:51.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bdcae700 1 -- 172.21.15.64:0/821767865 learned_addr learned my addr 172.21.15.64:0/821767865 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:50:51.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bd4ad700 1 --2- 172.21.15.64:0/821767865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b80ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52be4af700 1 --2- 172.21.15.64:0/821767865 >> v2:172.21.15.123:3300/0 conn(0x7f52b80acc60 0x7f52b80cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bccac700 1 -- 172.21.15.64:0/821767865 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090068197 0 0) 0x7f52b4001110 con 0x7f52b8150200 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bccac700 1 -- 172.21.15.64:0/821767865 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a4003570 con 0x7f52b8150200 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bd4ad700 1 -- 172.21.15.64:0/821767865 >> v2:172.21.15.123:3300/0 conn(0x7f52b80acc60 msgr2=0x7f52b80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bd4ad700 1 --2- 172.21.15.64:0/821767865 >> v2:172.21.15.123:3300/0 conn(0x7f52b80acc60 0x7f52b80cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bd4ad700 1 -- 172.21.15.64:0/821767865 >> v1:172.21.15.64:6789/0 conn(0x7f52b8150200 legacy=0x7f52b80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bd4ad700 1 -- 172.21.15.64:0/821767865 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80af250 con 0x7f52b80c7170 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bd4ad700 1 --2- 172.21.15.64:0/821767865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b80ac720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0098e0 tx=0x7f52ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d59762cc3579cf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bccac700 1 -- 172.21.15.64:0/821767865 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00f6b0 con 0x7f52b80c7170 2023-09-27T23:50:51.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bccac700 1 -- 172.21.15.64:0/821767865 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00f810 con 0x7f52b80c7170 2023-09-27T23:50:51.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52bccac700 1 -- 172.21.15.64:0/821767865 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00fbb0 con 0x7f52b80c7170 2023-09-27T23:50:51.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- 172.21.15.64:0/821767865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 msgr2=0x7f52b80ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 --2- 172.21.15.64:0/821767865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b80ac720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0098e0 tx=0x7f52ac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- 172.21.15.64:0/821767865 shutdown_connections 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 --2- 172.21.15.64:0/821767865 >> v2:172.21.15.123:3300/0 conn(0x7f52b80acc60 0x7f52b80cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 --2- 172.21.15.64:0/821767865 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b80ac720 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- 172.21.15.64:0/821767865 >> 172.21.15.64:0/821767865 conn(0x7f52b80ccb30 msgr2=0x7f52b80ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- 172.21.15.64:0/821767865 shutdown_connections 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.374+0000 7f52becb0700 1 -- 172.21.15.64:0/821767865 wait complete. 2023-09-27T23:50:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 Processor -- start 2023-09-27T23:50:51.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 -- start start 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 0x7f52b80a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b8168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8150200 0x7f52b80c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80aadb0 con 0x7f52b80acc60 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80aa1a0 con 0x7f52b8150200 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52becb0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80aa480 con 0x7f52b80c7170 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 0x7f52b80a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 0x7f52b80a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51282/0 (socket says 172.21.15.64:51282) 2023-09-27T23:50:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 -- 172.21.15.64:0/4062263664 learned_addr learned my addr 172.21.15.64:0/4062263664 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bd4ad700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b8168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52be4af700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8150200 0x7f52b80c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 -- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 msgr2=0x7f52b8168dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b8168dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 -- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8150200 msgr2=0x7f52b80c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8150200 0x7f52b80c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52bdcae700 1 -- 172.21.15.64:0/4062263664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009420 con 0x7f52b80acc60 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.378+0000 7f52be4af700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8150200 0x7f52b80c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:51.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52bdcae700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 0x7f52b80a9260 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c7c0 tx=0x7f52b4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4005b30 con 0x7f52b80acc60 2023-09-27T23:50:51.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4005cb0 con 0x7f52b80acc60 2023-09-27T23:50:51.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b80aa760 con 0x7f52b80acc60 2023-09-27T23:50:51.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4007320 con 0x7f52b80acc60 2023-09-27T23:50:51.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b80a97a0 con 0x7f52b80acc60 2023-09-27T23:50:51.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b804be40 con 0x7f52b80acc60 2023-09-27T23:50:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.382+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 83) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f52b401b3b0 con 0x7f52b80acc60 2023-09-27T23:50:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.386+0000 7f52a9ffb700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f529c0607f0 0x7f529c062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.386+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f52b407c260 con 0x7f52b80acc60 2023-09-27T23:50:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.386+0000 7f52bd4ad700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f529c0607f0 0x7f529c062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.386+0000 7f52bd4ad700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f529c0607f0 0x7f529c062ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0093f0 tx=0x7f52ac019000 comp rx=0 tx=0).ready entity=mgr.8525 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.390+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f52b404c420 con 0x7f52b80acc60 2023-09-27T23:50:51.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.566+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f52b80aaef0 con 0x7f529c0607f0 2023-09-27T23:50:51.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.670+0000 7f52a9ffb700 1 -- 172.21.15.64:0/4062263664 <== mgr.8525 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f52ac000bc0 con 0x7f529c0607f0 2023-09-27T23:50:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:Retention added to path snap_test_dir1 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f529c0607f0 msgr2=0x7f529c062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f529c0607f0 0x7f529c062ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0093f0 tx=0x7f52ac019000 comp rx=0 tx=0).stop 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 msgr2=0x7f52b80a9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 0x7f52b80a9260 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c7c0 tx=0x7f52b4007c70 comp rx=0 tx=0).stop 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 shutdown_connections 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f529c0607f0 0x7f529c062ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8150200 0x7f52b80c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b80c7170 0x7f52b8168dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 --2- 172.21.15.64:0/4062263664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b80acc60 0x7f52b80a9260 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 >> 172.21.15.64:0/4062263664 conn(0x7f52b80ccb30 msgr2=0x7f52b80c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 shutdown_connections 2023-09-27T23:50:51.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.674+0000 7f52becb0700 1 -- 172.21.15.64:0/4062263664 wait complete. 2023-09-27T23:50:51.696 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-09-27T23:50:51.696 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-09-27-23_51_00 in ~9s... 2023-09-27T23:50:51.696 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-09-27T23:50:51.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-09-27T23:50:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 Processor -- start 2023-09-27T23:50:51.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 -- start start 2023-09-27T23:50:51.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3a18158da0 0x7f3a1806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a1806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a180caca0 0x7f3a1806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a180ca290 con 0x7f3a180caca0 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1815d720 con 0x7f3a18158da0 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1d3e9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180ca4a0 con 0x7f3a1806df10 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a180caca0 0x7f3a1806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a180caca0 0x7f3a1806a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51286/0 (socket says 172.21.15.64:51286) 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 -- 172.21.15.64:0/2643961136 learned_addr learned my addr 172.21.15.64:0/2643961136 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a177fe700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a1806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a17fff700 1 --2- 172.21.15.64:0/2643961136 >> v2:172.21.15.123:3300/0 conn(0x7f3a18158da0 0x7f3a1806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 -- 172.21.15.64:0/2643961136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 msgr2=0x7f3a1806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a1806ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 -- 172.21.15.64:0/2643961136 >> v2:172.21.15.123:3300/0 conn(0x7f3a18158da0 msgr2=0x7f3a1806e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 --2- 172.21.15.64:0/2643961136 >> v2:172.21.15.123:3300/0 conn(0x7f3a18158da0 0x7f3a1806e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 -- 172.21.15.64:0/2643961136 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180ca780 con 0x7f3a180caca0 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a177fe700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a1806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a17fff700 1 --2- 172.21.15.64:0/2643961136 >> v2:172.21.15.123:3300/0 conn(0x7f3a18158da0 0x7f3a1806e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a1cbe8700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a1806a7d0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f3a1806a310 tx=0x7f3a10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6633242bb683cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a16ffd700 1 -- 172.21.15.64:0/2643961136 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000e070 con 0x7f3a180caca0 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a16ffd700 1 -- 172.21.15.64:0/2643961136 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1000ba50 con 0x7f3a180caca0 2023-09-27T23:50:51.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.814+0000 7f3a16ffd700 1 -- 172.21.15.64:0/2643961136 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100092e0 con 0x7f3a180caca0 2023-09-27T23:50:51.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/2643961136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 msgr2=0x7f3a1806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a1806a7d0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f3a1806a310 tx=0x7f3a10009840 comp rx=0 tx=0).stop 2023-09-27T23:50:51.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/2643961136 shutdown_connections 2023-09-27T23:50:51.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a1806a7d0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/2643961136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a1806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/2643961136 >> v2:172.21.15.123:3300/0 conn(0x7f3a18158da0 0x7f3a1806e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/2643961136 >> 172.21.15.64:0/2643961136 conn(0x7f3a180cc760 msgr2=0x7f3a180ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:51.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/2643961136 shutdown_connections 2023-09-27T23:50:51.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/2643961136 wait complete. 2023-09-27T23:50:51.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 Processor -- start 2023-09-27T23:50:51.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- start start 2023-09-27T23:50:51.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a180b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a180cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a18158da0 0x7f3a180cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180b2e60 con 0x7f3a180caca0 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180b3140 con 0x7f3a18158da0 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a1d3e9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180b3420 con 0x7f3a1806df10 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a180cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a180cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51300/0 (socket says 172.21.15.64:51300) 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.818+0000 7f3a177fe700 1 -- 172.21.15.64:0/3804503726 learned_addr learned my addr 172.21.15.64:0/3804503726 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a1cbe8700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a18158da0 0x7f3a180cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a17fff700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a180b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a177fe700 1 -- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 msgr2=0x7f3a180b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a177fe700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a180b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a177fe700 1 -- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a18158da0 msgr2=0x7f3a180cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:51.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a177fe700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a18158da0 0x7f3a180cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a177fe700 1 -- 172.21.15.64:0/3804503726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009ae0 con 0x7f3a180caca0 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a1cbe8700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a18158da0 0x7f3a180cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a177fe700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a180cefc0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00e8d0 tx=0x7f3a0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00ee70 con 0x7f3a180caca0 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00c5e0 con 0x7f3a180caca0 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a180b3700 con 0x7f3a180caca0 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00c980 con 0x7f3a180caca0 2023-09-27T23:50:51.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a180b2aa0 con 0x7f3a180caca0 2023-09-27T23:50:51.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 83) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f3a0c013410 con 0x7f3a180caca0 2023-09-27T23:50:51.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1804be40 con 0x7f3a180caca0 2023-09-27T23:50:51.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f39fffff700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3a040607f0 0x7f3a04062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:50:51.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f3a0c07cca0 con 0x7f3a180caca0 2023-09-27T23:50:51.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.822+0000 7f3a17fff700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3a040607f0 0x7f3a04062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:50:51.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.826+0000 7f3a17fff700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3a040607f0 0x7f3a04062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006fd0 tx=0x7f3a08008040 comp rx=0 tx=0).ready entity=mgr.8525 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:50:51.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:51.830+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3a0c04cf30 con 0x7f3a180caca0 2023-09-27T23:50:52.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.014+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a180c8600 con 0x7f3a040607f0 2023-09-27T23:50:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.030+0000 7f39fffff700 1 -- 172.21.15.64:0/3804503726 <== mgr.8525 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f3a0800f710 con 0x7f3a040607f0 2023-09-27T23:50:52.033 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:50:52.033 INFO:teuthology.orchestra.run.smithi064.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-09-27T23:50:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.030+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3a040607f0 msgr2=0x7f3a04062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3a040607f0 0x7f3a04062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006fd0 tx=0x7f3a08008040 comp rx=0 tx=0).stop 2023-09-27T23:50:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 msgr2=0x7f3a180cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:50:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a180cefc0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00e8d0 tx=0x7f3a0c008040 comp rx=0 tx=0).stop 2023-09-27T23:50:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 shutdown_connections 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3a040607f0 0x7f3a04062ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a18158da0 0x7f3a180cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a180caca0 0x7f3a180cefc0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 --2- 172.21.15.64:0/3804503726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a1806df10 0x7f3a180b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 >> 172.21.15.64:0/3804503726 conn(0x7f3a180cc760 msgr2=0x7f3a180af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 shutdown_connections 2023-09-27T23:50:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:50:52.034+0000 7f3a1d3e9700 1 -- 172.21.15.64:0/3804503726 wait complete. 2023-09-27T23:50:52.049 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-09-27T23:50:52.050 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-09-27T23:50:52.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:52.050 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:52.064 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:52.064 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:52.064 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:53.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:53.065 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:53.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:50:53.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:50:53.085 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:53.085 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:53.085 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:54.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:54.086 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:54.104 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:54.104 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:54.104 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:55.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:55.328 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:55.351 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:55.351 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:55.351 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:56.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:56.353 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:56.372 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:56.372 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:56.372 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:57.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:57.374 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:57.394 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:57.394 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:57.394 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:58.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:58.395 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:58.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:58.414 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:58.414 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:50:59.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:50:59.415 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:50:59.433 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:50:59.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:50:59.433 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:00.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:00.434 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:00.452 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-09-27T23:51:00.452 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:00.452 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:00.693 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:51:00.693+0000 7f2234b4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:51:00.694 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:51:00.693+0000 7f2234b4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:51:00.694 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:51:00.693+0000 7f2234b4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:51:00.694 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:51:00.693+0000 7f2234b4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:51:00.694 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:51:00.693+0000 7f2234b4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:51:01.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:01.453 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:01.470 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:01.471 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:01.471 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:01.471 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:01.472 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-09-27-23_51_00_UTC'] 2023-09-27T23:51:01.472 DEBUG:tasks.cephfs.test_snap_schedules:exec=1695858651.696445, now = 1695858661.4721582, timo = 9 2023-09-27T23:51:02.472 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-09-27-23_51_00 in ~69s... 2023-09-27T23:51:02.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:02.472 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:02.489 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:02.490 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:02.490 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:02.491 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:03.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:03.492 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:03.509 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:03.510 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:03.511 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:03.511 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:04.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:04.512 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:04.529 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:04.530 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:04.530 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:04.530 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:05.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:05.531 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:05.549 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:05.550 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:05.550 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:05.550 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:06.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:06.552 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:06.565 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:06.566 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:06.566 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:06.566 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:07.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:07.567 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:07.583 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:07.584 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:07.584 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:07.584 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:08.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:08.585 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:08.603 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:08.604 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:08.604 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:08.604 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:09.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:09.605 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:09.622 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:09.622 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:09.623 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:09.623 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:10.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:10.624 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:10.641 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:10.642 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:10.642 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:10.642 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:11.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:11.643 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:11.659 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:11.660 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:11.660 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:11.660 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:12.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:12.662 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:12.679 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:12.680 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:12.680 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:12.680 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:13.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:13.682 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:13.698 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:13.699 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:13.699 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:13.699 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:14.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:14.700 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:14.717 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:14.718 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:14.718 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:14.719 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:15.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:15.719 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:15.736 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:15.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:15.737 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:15.737 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:16.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:16.738 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:16.753 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:16.754 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:16.754 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:16.754 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:17.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:17.756 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:17.774 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:17.775 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:17.775 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:17.775 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:18.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:18.777 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:18.794 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:18.795 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:18.795 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:18.795 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:19.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:19.796 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:19.813 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:19.814 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:19.814 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:19.815 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:20.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:20.815 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:20.832 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:20.833 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:20.833 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:20.833 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:21.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:21.834 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:21.851 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:21.852 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:21.852 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:21.852 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:22.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:22.853 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:22.870 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:22.871 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:22.871 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:22.871 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:23.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:51:23.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:51:23.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:23.873 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:23.890 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:23.891 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:23.891 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:23.891 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:24.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:24.894 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:24.912 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:24.913 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:24.913 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:24.913 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:25.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:25.914 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:25.931 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:25.932 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:25.933 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:25.933 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:26.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:26.934 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:26.951 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:26.952 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:26.952 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:26.953 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:27.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:27.953 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:27.971 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:27.972 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:27.972 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:27.972 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:28.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:28.973 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:28.990 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:28.991 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:28.991 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:28.991 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:29.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:29.992 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:30.009 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:30.010 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:30.010 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:30.010 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:31.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:31.011 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:31.028 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:31.029 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:31.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:31.030 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:32.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:32.031 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:32.048 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:32.049 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:32.049 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:32.049 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:33.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:33.051 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:33.068 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:33.069 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:33.069 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:33.069 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:34.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:34.070 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:34.087 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:34.088 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:34.088 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:34.089 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:35.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:35.090 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:35.108 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:35.109 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:35.109 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:35.109 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:36.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:36.110 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:36.126 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:36.127 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:36.127 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:36.127 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:37.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:37.129 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:37.145 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:37.146 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:37.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:37.146 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:38.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:38.147 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:38.165 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:38.165 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:38.166 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:38.166 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:39.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:39.166 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:39.184 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:39.185 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:39.185 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:39.185 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:40.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:40.186 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:40.202 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:40.203 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:40.203 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:40.203 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:41.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:41.204 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:41.220 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:41.221 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:41.221 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:41.221 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:42.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:42.221 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:42.237 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:42.238 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:42.238 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:42.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:43.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:43.239 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:43.256 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:43.257 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:43.257 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:43.257 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:44.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:44.258 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:44.275 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:44.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:44.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:44.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:45.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:45.277 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:45.294 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:45.295 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:45.295 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:45.295 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:46.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:46.296 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:46.312 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:46.313 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:46.314 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:46.314 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:47.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:47.315 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:47.333 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:47.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:47.334 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:47.334 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:48.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:48.335 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:48.353 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:48.354 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:48.354 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:48.354 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:49.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:49.355 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:49.374 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:49.375 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:49.375 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:49.375 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:50.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:50.375 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:50.394 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:50.394 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:50.395 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:50.395 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:51.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:51.396 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:51.413 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:51.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:51.414 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:51.414 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:52.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:52.415 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:52.432 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:52.433 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:52.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:52.433 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:53.193 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:51:53.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:51:53.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:53.433 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:53.450 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:53.451 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:53.451 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:53.451 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:54.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:54.452 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:54.469 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:54.470 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:54.470 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:54.470 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:55.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:55.471 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:55.489 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:55.489 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:55.490 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:55.490 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:56.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:56.491 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:56.508 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:56.509 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:56.509 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:56.510 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:57.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:57.511 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:57.528 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:57.529 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:57.529 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:57.529 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:58.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:58.531 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:58.548 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:58.549 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:58.549 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:58.549 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:51:59.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:51:59.551 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:51:59.568 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:51:59.569 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:51:59.569 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:51:59.569 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:52:00.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:00.570 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:52:00.587 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_51_00_UTC 2023-09-27T23:52:00.589 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:52:00.589 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-09-27T23:52:00.589 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-09-27T23:52:01.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:01.590 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:52:01.607 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_52_00_UTC 2023-09-27T23:52:01.608 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-09-27-23_52_00_UTC'} 2023-09-27T23:52:01.608 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-09-27-23_52_00_UTC'} 2023-09-27T23:52:01.608 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-09-27-23_51_00_UTC'} 2023-09-27T23:52:01.608 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-09-27-23_51_00_UTC'] 2023-09-27T23:52:01.608 DEBUG:tasks.cephfs.test_snap_schedules:exec=1695858651.696445, now = 1695858721.608774, timo = 69 2023-09-27T23:52:02.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-09-27T23:52:02.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.729+0000 7f7a6f7f8700 1 Processor -- start 2023-09-27T23:52:02.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6f7f8700 1 -- start start 2023-09-27T23:52:02.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6f7f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 0x7f7a680cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6f7f8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b0ac0 con 0x7f7a6815c930 2023-09-27T23:52:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6f7f8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6806b210 con 0x7f7a6806a6a0 2023-09-27T23:52:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6f7f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680ca810 con 0x7f7a680cb130 2023-09-27T23:52:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6dd95700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7a6815c930 0x7f7a680b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58920/0 (socket says 172.21.15.64:58920) 2023-09-27T23:52:02.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6dd95700 1 -- 172.21.15.64:0/162337569 learned_addr learned my addr 172.21.15.64:0/162337569 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:02.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6d594700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 0x7f7a680cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196391210 0 0) 0x7f7a680b0ac0 con 0x7f7a6815c930 2023-09-27T23:52:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a54003570 con 0x7f7a6815c930 2023-09-27T23:52:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367903707 0 0) 0x7f7a6806b210 con 0x7f7a6806a6a0 2023-09-27T23:52:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a680b0ac0 con 0x7f7a6806a6a0 2023-09-27T23:52:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791111884 0 0) 0x7f7a54003570 con 0x7f7a6815c930 2023-09-27T23:52:02.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 msgr2=0x7f7a680cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 0x7f7a680cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 >> v1:172.21.15.123:6789/0 conn(0x7f7a6806a6a0 legacy=0x7f7a6815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680b0190 con 0x7f7a6815c930 2023-09-27T23:52:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f7a600017f0 con 0x7f7a6815c930 2023-09-27T23:52:02.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 >> v1:172.21.15.64:6789/0 conn(0x7f7a6815c930 legacy=0x7f7a680b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:02.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54003c20 0x7f7a54004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54004810 0x7f7a54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 0x7f7a540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680ca810 con 0x7f7a54004810 2023-09-27T23:52:02.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5400bb40 con 0x7f7a54003c20 2023-09-27T23:52:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5400bd00 con 0x7f7a680cb130 2023-09-27T23:52:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6cd93700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54004810 0x7f7a54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6d594700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54003c20 0x7f7a54004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6dd95700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 0x7f7a540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6cd93700 1 -- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 msgr2=0x7f7a540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:02.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6cd93700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a680cb130 0x7f7a540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6cd93700 1 -- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54003c20 msgr2=0x7f7a54004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6cd93700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54003c20 0x7f7a54004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.733+0000 7f7a6cd93700 1 -- 172.21.15.64:0/162337569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54003570 con 0x7f7a54004810 2023-09-27T23:52:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6cd93700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54004810 0x7f7a54004c80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f7a58001a10 tx=0x7f7a58007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f4113f69623a0a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800c5d0 con 0x7f7a54004810 2023-09-27T23:52:02.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5800c730 con 0x7f7a54004810 2023-09-27T23:52:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a5ffff700 1 -- 172.21.15.64:0/162337569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800cad0 con 0x7f7a54004810 2023-09-27T23:52:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/162337569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54004810 msgr2=0x7f7a54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/162337569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54004810 0x7f7a54004c80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f7a58001a10 tx=0x7f7a58007d30 comp rx=0 tx=0).stop 2023-09-27T23:52:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6dd95700 1 -- 172.21.15.64:0/162337569 reap_dead start 2023-09-27T23:52:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/162337569 shutdown_connections 2023-09-27T23:52:02.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/162337569 >> 172.21.15.64:0/162337569 conn(0x7f7a68067bb0 msgr2=0x7f7a680cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/162337569 shutdown_connections 2023-09-27T23:52:02.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/162337569 wait complete. 2023-09-27T23:52:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 Processor -- start 2023-09-27T23:52:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.737+0000 7f7a6f7f8700 1 -- start start 2023-09-27T23:52:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a54004810 0x7f7a681698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a68169e10 0x7f7a680c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 0x7f7a680c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680c9c30 con 0x7f7a680c97e0 2023-09-27T23:52:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680c8e10 con 0x7f7a68169e10 2023-09-27T23:52:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680c8200 con 0x7f7a54004810 2023-09-27T23:52:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 0x7f7a680c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 0x7f7a680c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39478/0 (socket says 172.21.15.64:39478) 2023-09-27T23:52:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 -- 172.21.15.64:0/3222301825 learned_addr learned my addr 172.21.15.64:0/3222301825 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6d594700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a54004810 0x7f7a681698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6cd93700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a68169e10 0x7f7a680c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 -- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a54004810 msgr2=0x7f7a681698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a54004810 0x7f7a681698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 -- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a68169e10 msgr2=0x7f7a680c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a68169e10 0x7f7a680c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 -- 172.21.15.64:0/3222301825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680c84e0 con 0x7f7a680c97e0 2023-09-27T23:52:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6dd95700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 0x7f7a680c74d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7a6815cd00 tx=0x7f7a60001480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000ec90 con 0x7f7a680c97e0 2023-09-27T23:52:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6000edf0 con 0x7f7a680c97e0 2023-09-27T23:52:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680c87c0 con 0x7f7a680c97e0 2023-09-27T23:52:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60013670 con 0x7f7a680c97e0 2023-09-27T23:52:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68157d80 con 0x7f7a680c97e0 2023-09-27T23:52:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.741+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6816a1e0 con 0x7f7a680c97e0 2023-09-27T23:52:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.745+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 85) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f7a600139e0 con 0x7f7a680c97e0 2023-09-27T23:52:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.745+0000 7f7a5d7fa700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a4c060c10 0x7f7a4c0630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.745+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f7a6007db40 con 0x7f7a680c97e0 2023-09-27T23:52:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.745+0000 7f7a6d594700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a4c060c10 0x7f7a4c0630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.745+0000 7f7a6d594700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a4c060c10 0x7f7a4c0630c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006dc0 tx=0x7f7a64008040 comp rx=0 tx=0).ready entity=mgr.8525 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:02.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.749+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7a6004d890 con 0x7f7a680c97e0 2023-09-27T23:52:02.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:02.933+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a68157fa0 con 0x7f7a4c060c10 2023-09-27T23:52:03.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.053+0000 7f7a5d7fa700 1 -- 172.21.15.64:0/3222301825 <== mgr.8525 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7a68157fa0 con 0x7f7a4c060c10 2023-09-27T23:52:03.057 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path snap_test_dir1 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a4c060c10 msgr2=0x7f7a4c0630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a4c060c10 0x7f7a4c0630c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006dc0 tx=0x7f7a64008040 comp rx=0 tx=0).stop 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 msgr2=0x7f7a680c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 0x7f7a680c74d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7a6815cd00 tx=0x7f7a60001480 comp rx=0 tx=0).stop 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 shutdown_connections 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a4c060c10 0x7f7a4c0630c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680c97e0 0x7f7a680c74d0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a68169e10 0x7f7a680c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 --2- 172.21.15.64:0/3222301825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a54004810 0x7f7a681698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 >> 172.21.15.64:0/3222301825 conn(0x7f7a68067bb0 msgr2=0x7f7a680cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 shutdown_connections 2023-09-27T23:52:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.057+0000 7f7a6f7f8700 1 -- 172.21.15.64:0/3222301825 wait complete. 2023-09-27T23:52:03.080 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-09-27T23:52:03.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:03.080 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:52:03.097 INFO:teuthology.orchestra.run.smithi064.stdout:scheduled-2023-09-27-23_52_00_UTC 2023-09-27T23:52:03.098 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-09-27-23_52_00_UTC 2023-09-27T23:52:03.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:03.098 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-09-27-23_52_00_UTC) 2023-09-27T23:52:03.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:03.121 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-09-27T23:52:03.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:52:03.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 Processor -- start 2023-09-27T23:52:03.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 -- start start 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4598158fd0 0x7f459806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45980cc610 0x7f45980c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459806d750 con 0x7f45980cc1e0 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45980cbaf0 con 0x7f4598158fd0 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459e9d6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459806e120 con 0x7f45980cc610 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459d9d4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4598158fd0 0x7f459806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459d9d4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4598158fd0 0x7f459806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39490/0 (socket says 172.21.15.64:39490) 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.253+0000 7f459d9d4700 1 -- 172.21.15.64:0/3997721373 learned_addr learned my addr 172.21.15.64:0/3997721373 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e1d5700 1 --2- 172.21.15.64:0/3997721373 >> v2:172.21.15.123:3300/0 conn(0x7f45980cc610 0x7f45980c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:03.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459d9d4700 1 -- 172.21.15.64:0/3997721373 >> v2:172.21.15.123:3300/0 conn(0x7f45980cc610 msgr2=0x7f45980c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459d9d4700 1 --2- 172.21.15.64:0/3997721373 >> v2:172.21.15.123:3300/0 conn(0x7f45980cc610 0x7f45980c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459d9d4700 1 -- 172.21.15.64:0/3997721373 >> v1:172.21.15.64:6789/0 conn(0x7f45980cc1e0 legacy=0x7f45980c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459d9d4700 1 -- 172.21.15.64:0/3997721373 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459806cb40 con 0x7f4598158fd0 2023-09-27T23:52:03.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459d9d4700 1 --2- 172.21.15.64:0/3997721373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4598158fd0 0x7f459806dbe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f45980b0780 tx=0x7f4594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344a5e00ab9a2694 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:03.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459c9d2700 1 -- 172.21.15.64:0/3997721373 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400b860 con 0x7f4598158fd0 2023-09-27T23:52:03.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459c9d2700 1 -- 172.21.15.64:0/3997721373 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400b9c0 con 0x7f4598158fd0 2023-09-27T23:52:03.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459c9d2700 1 -- 172.21.15.64:0/3997721373 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400bd60 con 0x7f4598158fd0 2023-09-27T23:52:03.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 -- 172.21.15.64:0/3997721373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4598158fd0 msgr2=0x7f459806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3997721373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4598158fd0 0x7f459806dbe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f45980b0780 tx=0x7f4594009250 comp rx=0 tx=0).stop 2023-09-27T23:52:03.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 -- 172.21.15.64:0/3997721373 shutdown_connections 2023-09-27T23:52:03.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3997721373 >> v2:172.21.15.123:3300/0 conn(0x7f45980cc610 0x7f45980c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3997721373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4598158fd0 0x7f459806dbe0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 -- 172.21.15.64:0/3997721373 >> 172.21.15.64:0/3997721373 conn(0x7f45980cdc00 msgr2=0x7f45980ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:03.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 -- 172.21.15.64:0/3997721373 shutdown_connections 2023-09-27T23:52:03.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.257+0000 7f459e9d6700 1 -- 172.21.15.64:0/3997721373 wait complete. 2023-09-27T23:52:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 Processor -- start 2023-09-27T23:52:03.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 -- start start 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45980cc1e0 0x7f4598169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 0x7f4598154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4598158fd0 0x7f45980caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45980ca5c0 con 0x7f45980cc610 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45980cb490 con 0x7f4598158fd0 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459e9d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45980ca810 con 0x7f45980cc1e0 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459d1d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 0x7f4598154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459d9d4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45980cc1e0 0x7f4598169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459d1d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 0x7f4598154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39504/0 (socket says 172.21.15.64:39504) 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.261+0000 7f459d1d3700 1 -- 172.21.15.64:0/3822471649 learned_addr learned my addr 172.21.15.64:0/3822471649 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:03.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459e1d5700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4598158fd0 0x7f45980caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459d1d3700 1 -- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45980cc1e0 msgr2=0x7f4598169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459d1d3700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45980cc1e0 0x7f4598169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459d1d3700 1 -- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4598158fd0 msgr2=0x7f45980caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459d1d3700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4598158fd0 0x7f45980caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459d1d3700 1 -- 172.21.15.64:0/3822471649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940094f0 con 0x7f45980cc610 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459d1d3700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 0x7f4598154880 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f458800c590 tx=0x7f458800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588011590 con 0x7f45980cc610 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4588011710 con 0x7f45980cc610 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45980caaf0 con 0x7f45980cc610 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588011ad0 con 0x7f45980cc610 2023-09-27T23:52:03.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4598169980 con 0x7f45980cc610 2023-09-27T23:52:03.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.265+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4598169ba0 con 0x7f45980cc610 2023-09-27T23:52:03.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.269+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 85) v1 ==== 80492+0+0 (secure 0 0 0) 0x7f458801b3b0 con 0x7f45980cc610 2023-09-27T23:52:03.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.269+0000 7f458dffb700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f457c060be0 0x7f457c063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:03.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.269+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f458807c980 con 0x7f45980cc610 2023-09-27T23:52:03.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.269+0000 7f459d9d4700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f457c060be0 0x7f457c063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:03.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.269+0000 7f459d9d4700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f457c060be0 0x7f457c063090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45980b0780 tx=0x7f4594017000 comp rx=0 tx=0).ready entity=mgr.8525 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.273+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f458804c630 con 0x7f45980cc610 2023-09-27T23:52:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.477+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f459804be40 con 0x7f45980cc610 2023-09-27T23:52:03.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.785+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f458804ca00 con 0x7f45980cc610 2023-09-27T23:52:03.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.785+0000 7f458dffb700 1 -- 172.21.15.64:0/3822471649 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v86) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4588077220 con 0x7f45980cc610 2023-09-27T23:52:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f457c060be0 msgr2=0x7f457c063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f457c060be0 0x7f457c063090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45980b0780 tx=0x7f4594017000 comp rx=0 tx=0).stop 2023-09-27T23:52:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 msgr2=0x7f4598154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 0x7f4598154880 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f458800c590 tx=0x7f458800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 shutdown_connections 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f457c060be0 0x7f457c063090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4598158fd0 0x7f45980caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45980cc610 0x7f4598154880 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 --2- 172.21.15.64:0/3822471649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45980cc1e0 0x7f4598169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 >> 172.21.15.64:0/3822471649 conn(0x7f45980cdc00 msgr2=0x7f4598159430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 shutdown_connections 2023-09-27T23:52:03.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:03.793+0000 7f459e9d6700 1 -- 172.21.15.64:0/3822471649 wait complete. 2023-09-27T23:52:03.813 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:52:03.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:52:03.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:52:03.852 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:52:03.852 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:52:03.852 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:52:03.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:03.867 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[60532]: fuse finished with error 0 and tester_r 0 2023-09-27T23:52:03.868 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:52:03.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:03.868 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:52:03.921 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:03.919+0000 7f3c36e32000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:52:03.922 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:03.917+0000 7fe30ea74000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:52:03.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:52:03.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:03.987 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:52:03.994 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:52:03.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:52:03.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:52:03.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:03.995 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:52:04.042 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:52:04.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:52:04.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:52:04.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:04.044 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:52:04.051 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:52:04.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:52:04.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.153+0000 7face4b32700 1 Processor -- start 2023-09-27T23:52:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.153+0000 7face4b32700 1 -- start start 2023-09-27T23:52:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face0159c10 0x7face0159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7face015a690 0x7face00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7face013f430 0x7face013f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00cc6d0 con 0x7face013f430 2023-09-27T23:52:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00c22e0 con 0x7face015a690 2023-09-27T23:52:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00cc8e0 con 0x7face0159c10 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7face013f430 0x7face013f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7face013f430 0x7face013f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39522/0 (socket says 172.21.15.64:39522) 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 -- 172.21.15.64:0/1248948273 learned_addr learned my addr 172.21.15.64:0/1248948273 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facddd9b700 1 --2- 172.21.15.64:0/1248948273 >> v2:172.21.15.123:3300/0 conn(0x7face015a690 0x7face00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facde59c700 1 --2- 172.21.15.64:0/1248948273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face0159c10 0x7face0159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 -- 172.21.15.64:0/1248948273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face0159c10 msgr2=0x7face0159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 --2- 172.21.15.64:0/1248948273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face0159c10 0x7face0159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 -- 172.21.15.64:0/1248948273 >> v2:172.21.15.123:3300/0 conn(0x7face015a690 msgr2=0x7face00ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 --2- 172.21.15.64:0/1248948273 >> v2:172.21.15.123:3300/0 conn(0x7face015a690 0x7face00ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 -- 172.21.15.64:0/1248948273 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00ccbc0 con 0x7face013f430 2023-09-27T23:52:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facded9d700 1 --2- 172.21.15.64:0/1248948273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face013f430 0x7face013f880 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7face006b3c0 tx=0x7facd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac052cb25f87b9b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facdd59a700 1 -- 172.21.15.64:0/1248948273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e070 con 0x7face013f430 2023-09-27T23:52:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facdd59a700 1 -- 172.21.15.64:0/1248948273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800b970 con 0x7face013f430 2023-09-27T23:52:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7facdd59a700 1 -- 172.21.15.64:0/1248948273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd80092e0 con 0x7face013f430 2023-09-27T23:52:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- 172.21.15.64:0/1248948273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face013f430 msgr2=0x7face013f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- 172.21.15.64:0/1248948273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face013f430 0x7face013f880 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7face006b3c0 tx=0x7facd8009840 comp rx=0 tx=0).stop 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- 172.21.15.64:0/1248948273 shutdown_connections 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- 172.21.15.64:0/1248948273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face013f430 0x7face013f880 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- 172.21.15.64:0/1248948273 >> v2:172.21.15.123:3300/0 conn(0x7face015a690 0x7face00ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 --2- 172.21.15.64:0/1248948273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face0159c10 0x7face0159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- 172.21.15.64:0/1248948273 >> 172.21.15.64:0/1248948273 conn(0x7face00cd9c0 msgr2=0x7face00c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- 172.21.15.64:0/1248948273 shutdown_connections 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.157+0000 7face4b32700 1 -- 172.21.15.64:0/1248948273 wait complete. 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 Processor -- start 2023-09-27T23:52:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 -- start start 2023-09-27T23:52:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face013f430 0x7face015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 0x7face01550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face015a690 0x7face013e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face0156700 con 0x7face0159c10 2023-09-27T23:52:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face015b450 con 0x7face015a690 2023-09-27T23:52:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face015b730 con 0x7face013f430 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 0x7face01550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 0x7face01550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39534/0 (socket says 172.21.15.64:39534) 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 -- 172.21.15.64:0/1533055672 learned_addr learned my addr 172.21.15.64:0/1533055672 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facde59c700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face013f430 0x7face015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facded9d700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face015a690 0x7face013e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 -- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face013f430 msgr2=0x7face015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face013f430 0x7face015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 -- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face015a690 msgr2=0x7face013e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face015a690 0x7face013e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 -- 172.21.15.64:0/1533055672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8009ae0 con 0x7face0159c10 2023-09-27T23:52:04.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facddd9b700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 0x7face01550e0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7facd400e930 tx=0x7facd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:04.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4013680 con 0x7face0159c10 2023-09-27T23:52:04.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd40137e0 con 0x7face0159c10 2023-09-27T23:52:04.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face015ba10 con 0x7face0159c10 2023-09-27T23:52:04.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4013b80 con 0x7face0159c10 2023-09-27T23:52:04.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.161+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face0155620 con 0x7face0159c10 2023-09-27T23:52:04.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.165+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face01564d0 con 0x7face0159c10 2023-09-27T23:52:04.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.165+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7facd400c3e0 con 0x7face0159c10 2023-09-27T23:52:04.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.165+0000 7facce7fc700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 0x7facc0063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.165+0000 7facde59c700 1 -- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 msgr2=0x7facc0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:04.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.165+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7facd407d510 con 0x7face0159c10 2023-09-27T23:52:04.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.165+0000 7facde59c700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 0x7facc0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:04.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.173+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7facd404d260 con 0x7face0159c10 2023-09-27T23:52:04.347 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.343+0000 7f3c36e32000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:52:04.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.365+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7face015bc00 con 0x7face0159c10 2023-09-27T23:52:04.370 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:04.365+0000 7fe30ea74000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:52:04.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.365+0000 7facce7fc700 1 -- 172.21.15.64:0/1533055672 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4604 (secure 0 0 0) 0x7facd4050880 con 0x7face0159c10 2023-09-27T23:52:04.371 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:04.372 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:50:34.520359+0000","modified":"2023-09-27T23:50:36.536424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/660311009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":660311009},{"type":"v1","addr":"172.21.15.64:6835","nonce":660311009}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-09-27T23:52:04.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7facde59c700 1 -- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 msgr2=0x7facc0063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:04.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7facde59c700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 0x7facc0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:52:04.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 msgr2=0x7facc0063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:04.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 0x7facc0063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 msgr2=0x7face01550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 0x7face01550e0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7facd400e930 tx=0x7facd40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 shutdown_connections 2023-09-27T23:52:04.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7facc0060b90 0x7facc0063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face015a690 0x7face013e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face0159c10 0x7face01550e0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 --2- 172.21.15.64:0/1533055672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face013f430 0x7face015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 >> 172.21.15.64:0/1533055672 conn(0x7face00cd9c0 msgr2=0x7face0155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:04.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 shutdown_connections 2023-09-27T23:52:04.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.369+0000 7face4b32700 1 -- 172.21.15.64:0/1533055672 wait complete. 2023-09-27T23:52:04.376 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 66 2023-09-27T23:52:04.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:04.395 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.391+0000 7f3c36e32000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:52:04.424 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:04.421+0000 7fe30ea74000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:52:04.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.481+0000 7f2b5777e700 1 Processor -- start 2023-09-27T23:52:04.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.481+0000 7f2b5777e700 1 -- start start 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b50155b30 0x7f2b500c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b500af470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5006c860 con 0x7f2b50155f00 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5006cb00 con 0x7f2b50155b30 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5006cde0 con 0x7f2b500ca3c0 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5551a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2b50155b30 0x7f2b500c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b55d1b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2b50155f00 0x7f2b5006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:58942/0 (socket says 172.21.15.64:58942) 2023-09-27T23:52:04.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b55d1b700 1 -- 172.21.15.64:0/3260612751 learned_addr learned my addr 172.21.15.64:0/3260612751 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b54d19700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b500af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b47fff700 1 -- 172.21.15.64:0/3260612751 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721223635 0 0) 0x7f2b5006c860 con 0x7f2b50155f00 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b47fff700 1 -- 172.21.15.64:0/3260612751 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c003570 con 0x7f2b50155f00 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5551a700 1 -- 172.21.15.64:0/3260612751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 msgr2=0x7f2b500af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5551a700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b500af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5551a700 1 -- 172.21.15.64:0/3260612751 >> v1:172.21.15.64:6789/0 conn(0x7f2b50155f00 legacy=0x7f2b5006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5551a700 1 -- 172.21.15.64:0/3260612751 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5006d0c0 con 0x7f2b50155b30 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b54d19700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b500af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:04.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5551a700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155b30 0x7f2b500c9e80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0098e0 tx=0x7f2b4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c5f8e60d9b01e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:04.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b47fff700 1 -- 172.21.15.64:0/3260612751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00b640 con 0x7f2b50155b30 2023-09-27T23:52:04.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b47fff700 1 -- 172.21.15.64:0/3260612751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c00b7a0 con 0x7f2b50155b30 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b47fff700 1 -- 172.21.15.64:0/3260612751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00bb20 con 0x7f2b50155b30 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- 172.21.15.64:0/3260612751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155b30 msgr2=0x7f2b500c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155b30 0x7f2b500c9e80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0098e0 tx=0x7f2b4c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- 172.21.15.64:0/3260612751 shutdown_connections 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b500af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 --2- 172.21.15.64:0/3260612751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155b30 0x7f2b500c9e80 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- 172.21.15.64:0/3260612751 >> 172.21.15.64:0/3260612751 conn(0x7f2b50067410 msgr2=0x7f2b500677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- 172.21.15.64:0/3260612751 shutdown_connections 2023-09-27T23:52:04.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.485+0000 7f2b5777e700 1 -- 172.21.15.64:0/3260612751 wait complete. 2023-09-27T23:52:04.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 Processor -- start 2023-09-27T23:52:04.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 -- start start 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b501468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b50155b30 0x7f2b5006b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 0x7f2b5006bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500ad300 con 0x7f2b50155f00 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50166180 con 0x7f2b50155b30 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5777e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50165570 con 0x7f2b500ca3c0 2023-09-27T23:52:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 0x7f2b5006bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 0x7f2b5006bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39548/0 (socket says 172.21.15.64:39548) 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 -- 172.21.15.64:0/2159880400 learned_addr learned my addr 172.21.15.64:0/2159880400 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b5551a700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b501468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b54d19700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b50155b30 0x7f2b5006b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 -- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 msgr2=0x7f2b501468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b501468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 -- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b50155b30 msgr2=0x7f2b5006b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b50155b30 0x7f2b5006b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.489+0000 7f2b55d1b700 1 -- 172.21.15.64:0/2159880400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c009420 con 0x7f2b50155f00 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b55d1b700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 0x7f2b5006bc80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2b48001460 tx=0x7f2b48001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800a870 con 0x7f2b50155f00 2023-09-27T23:52:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800a9d0 con 0x7f2b50155f00 2023-09-27T23:52:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b50165850 con 0x7f2b50155f00 2023-09-27T23:52:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800ad50 con 0x7f2b50155f00 2023-09-27T23:52:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b50165e10 con 0x7f2b50155f00 2023-09-27T23:52:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b50149150 con 0x7f2b50155f00 2023-09-27T23:52:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f2b48007300 con 0x7f2b50155f00 2023-09-27T23:52:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.493+0000 7f2b457fa700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2b34060b90 0x7f2b34063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.497+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f2b4807d700 con 0x7f2b50155f00 2023-09-27T23:52:04.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.497+0000 7f2b5551a700 1 -- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2b34060b90 msgr2=0x7f2b34063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:04.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.497+0000 7f2b5551a700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2b34060b90 0x7f2b34063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:04.502 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.499+0000 7f3c36e32000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.501+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2b50149150 con 0x7f2b50155f00 2023-09-27T23:52:04.538 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:04.533+0000 7fe30ea74000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.661+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b50149150 con 0x7f2b50155f00 2023-09-27T23:52:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.661+0000 7f2b457fa700 1 -- 172.21.15.64:0/2159880400 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+17583 (secure 0 0 0) 0x7f2b50149150 con 0x7f2b50155f00 2023-09-27T23:52:04.668 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:04.668 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":175,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:50:50.099407+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:50:33.091150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-09-27T23:50:33.495902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2589357507":"2023-09-28T23:50:50.099294+0000","172.21.15.123:6832/43188":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3659445879":"2023-09-28T23:50:50.099294+0000","172.21.15.123:6833/43188":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3288865613":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/1822031684":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/883856417":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/1530058000":"2023-09-28T23:50:50.099294+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2b34060b90 msgr2=0x7f2b34063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2b34060b90 0x7f2b34063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 msgr2=0x7f2b5006bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 0x7f2b5006bc80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2b48001460 tx=0x7f2b48001a40 comp rx=0 tx=0).stop 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 shutdown_connections 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2b34060b90 0x7f2b34063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b50155f00 0x7f2b5006bc80 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b50155b30 0x7f2b5006b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 --2- 172.21.15.64:0/2159880400 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2b500ca3c0 0x7f2b501468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 >> 172.21.15.64:0/2159880400 conn(0x7f2b50067410 msgr2=0x7f2b50148740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 shutdown_connections 2023-09-27T23:52:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.665+0000 7f2b5777e700 1 -- 172.21.15.64:0/2159880400 wait complete. 2023-09-27T23:52:04.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:50:33.091150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '173', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:50:33.495902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:52:04.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:52:04.696 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.695+0000 7f3c36e32000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:52:04.744 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.743+0000 7f3c36e32000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:04.745 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:04.741+0000 7fe30ea74000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:52:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 Processor -- start 2023-09-27T23:52:04.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- start start 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59080ace30 0x7f59080cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080c7610 con 0x7f5908150200 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080af250 con 0x7f59080c7170 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080af530 con 0x7f59080ace30 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590dec7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5908150200 0x7f59080ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34020/0 (socket says 172.21.15.64:34020) 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590e6c8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59080ace30 0x7f59080cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590dec7700 1 -- 172.21.15.64:0/475969679 learned_addr learned my addr 172.21.15.64:0/475969679 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590e6c8700 1 -- 172.21.15.64:0/475969679 >> v1:172.21.15.123:6789/0 conn(0x7f59080c7170 legacy=0x7f59080ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590e6c8700 1 -- 172.21.15.64:0/475969679 >> v1:172.21.15.64:6789/0 conn(0x7f5908150200 legacy=0x7f59080ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590cec5700 1 -- 172.21.15.64:0/475969679 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160798261 0 0) 0x7f59080c7610 con 0x7f5908150200 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590e6c8700 1 -- 172.21.15.64:0/475969679 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080ca8a0 con 0x7f59080ace30 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590e6c8700 1 --2- 172.21.15.64:0/475969679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080ace30 0x7f59080cb380 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5900009e60 tx=0x7f5900009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b890deca7be4d08b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590cec5700 1 -- 172.21.15.64:0/475969679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590000e070 con 0x7f59080ace30 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590cec5700 1 -- 172.21.15.64:0/475969679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590000bc30 con 0x7f59080ace30 2023-09-27T23:52:04.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590cec5700 1 -- 172.21.15.64:0/475969679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900012430 con 0x7f59080ace30 2023-09-27T23:52:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- 172.21.15.64:0/475969679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080ace30 msgr2=0x7f59080cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 --2- 172.21.15.64:0/475969679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080ace30 0x7f59080cb380 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5900009e60 tx=0x7f5900009780 comp rx=0 tx=0).stop 2023-09-27T23:52:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- 172.21.15.64:0/475969679 shutdown_connections 2023-09-27T23:52:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 --2- 172.21.15.64:0/475969679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080ace30 0x7f59080cb380 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- 172.21.15.64:0/475969679 >> 172.21.15.64:0/475969679 conn(0x7f59080ccb30 msgr2=0x7f59080ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:04.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- 172.21.15.64:0/475969679 shutdown_connections 2023-09-27T23:52:04.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.789+0000 7f590eec9700 1 -- 172.21.15.64:0/475969679 wait complete. 2023-09-27T23:52:04.797 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:04.793+0000 7fe30ea74000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 Processor -- start 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 -- start start 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59080ace30 0x7f5908069bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 0x7f59080c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908150200 0x7f59080c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080b0de0 con 0x7f59080c7170 2023-09-27T23:52:04.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080c49d0 con 0x7f5908150200 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080c4cb0 con 0x7f59080ace30 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 0x7f59080c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590dec7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59080ace30 0x7f5908069bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 0x7f59080c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49188/0 (socket says 172.21.15.64:49188) 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 -- 172.21.15.64:0/3251388292 learned_addr learned my addr 172.21.15.64:0/3251388292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590e6c8700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908150200 0x7f59080c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 -- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59080ace30 msgr2=0x7f5908069bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59080ace30 0x7f5908069bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 -- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908150200 msgr2=0x7f59080c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908150200 0x7f59080c4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 -- 172.21.15.64:0/3251388292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900009a20 con 0x7f59080c7170 2023-09-27T23:52:04.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590d6c6700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 0x7f59080c3df0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f58f800c590 tx=0x7f58f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8011590 con 0x7f59080c7170 2023-09-27T23:52:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8011710 con 0x7f59080c7170 2023-09-27T23:52:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59080c4f90 con 0x7f59080c7170 2023-09-27T23:52:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8011ad0 con 0x7f59080c7170 2023-09-27T23:52:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.793+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59080c5270 con 0x7f59080c7170 2023-09-27T23:52:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.797+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590804be40 con 0x7f59080c7170 2023-09-27T23:52:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.797+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f58f801b3b0 con 0x7f59080c7170 2023-09-27T23:52:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.797+0000 7f58fdffb700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58ec060b90 0x7f58ec063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.797+0000 7f590dec7700 1 -- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58ec060b90 msgr2=0x7f58ec063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.797+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f58f807c900 con 0x7f59080c7170 2023-09-27T23:52:04.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.797+0000 7f590dec7700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58ec060b90 0x7f58ec063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:04.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.801+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f58f804c650 con 0x7f59080c7170 2023-09-27T23:52:04.895 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.891+0000 7f3c36e32000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:52:04.952 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:04.947+0000 7f3c36e32000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:52:04.954 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:04.949+0000 7fe30ea74000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:52:04.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.961+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59080ad290 con 0x7f59080c7170 2023-09-27T23:52:04.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f58fdffb700 1 -- 172.21.15.64:0/3251388292 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58f8017020 con 0x7f59080c7170 2023-09-27T23:52:04.968 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58ec060b90 msgr2=0x7f58ec063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58ec060b90 0x7f58ec063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 msgr2=0x7f59080c3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 0x7f59080c3df0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f58f800c590 tx=0x7f58f800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 shutdown_connections 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58ec060b90 0x7f58ec063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908150200 0x7f59080c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59080c7170 0x7f59080c3df0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 --2- 172.21.15.64:0/3251388292 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59080ace30 0x7f5908069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:04.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 >> 172.21.15.64:0/3251388292 conn(0x7f59080ccb30 msgr2=0x7f590815bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 shutdown_connections 2023-09-27T23:52:04.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:04.965+0000 7f590eec9700 1 -- 172.21.15.64:0/3251388292 wait complete. 2023-09-27T23:52:04.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:52:05.011 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:05.007+0000 7f3c36e32000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:52:05.015 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:05.009+0000 7fe30ea74000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:52:05.078 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:05.073+0000 7fe30ea74000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:52:05.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 Processor -- start 2023-09-27T23:52:05.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 -- start start 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b690 0x7f98940afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98940b0110 0x7f9894158d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989406c7c0 con 0x7f989406b2c0 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989415a2c0 con 0x7f98940b0110 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9899f01700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989406cd80 con 0x7f989406b690 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9893fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98940b0110 0x7f9894158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f98937fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f989406b2c0 0x7f989406d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34032/0 (socket says 172.21.15.64:34032) 2023-09-27T23:52:05.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f98937fe700 1 -- 172.21.15.64:0/504747164 learned_addr learned my addr 172.21.15.64:0/504747164 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f988bfff700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b690 0x7f98940afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9893fff700 1 -- 172.21.15.64:0/504747164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b690 msgr2=0x7f98940afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9893fff700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b690 0x7f98940afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9893fff700 1 -- 172.21.15.64:0/504747164 >> v1:172.21.15.64:6789/0 conn(0x7f989406b2c0 legacy=0x7f989406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9892ffd700 1 -- 172.21.15.64:0/504747164 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512131198 0 0) 0x7f989406c7c0 con 0x7f989406b2c0 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9893fff700 1 -- 172.21.15.64:0/504747164 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940b0940 con 0x7f98940b0110 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f988bfff700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b690 0x7f98940afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9893fff700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940b0110 0x7f9894158d70 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f988c009e60 tx=0x7f988c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad1f095ceccbaef server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9892ffd700 1 -- 172.21.15.64:0/504747164 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00e070 con 0x7f98940b0110 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9892ffd700 1 -- 172.21.15.64:0/504747164 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00bc30 con 0x7f98940b0110 2023-09-27T23:52:05.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.085+0000 7f9892ffd700 1 -- 172.21.15.64:0/504747164 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c012430 con 0x7f98940b0110 2023-09-27T23:52:05.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- 172.21.15.64:0/504747164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940b0110 msgr2=0x7f9894158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940b0110 0x7f9894158d70 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f988c009e60 tx=0x7f988c009780 comp rx=0 tx=0).stop 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- 172.21.15.64:0/504747164 shutdown_connections 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940b0110 0x7f9894158d70 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 --2- 172.21.15.64:0/504747164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b690 0x7f98940afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- 172.21.15.64:0/504747164 >> 172.21.15.64:0/504747164 conn(0x7f98940cc580 msgr2=0x7f98940cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- 172.21.15.64:0/504747164 shutdown_connections 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- 172.21.15.64:0/504747164 wait complete. 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 Processor -- start 2023-09-27T23:52:05.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- start start 2023-09-27T23:52:05.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b2c0 0x7f98940a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 0x7f98940aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98940b0110 0x7f98940c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940a8d90 con 0x7f989406b690 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940ab330 con 0x7f98940b0110 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9899f01700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98940ab610 con 0x7f989406b2c0 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 0x7f98940aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 0x7f98940aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49204/0 (socket says 172.21.15.64:49204) 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 -- 172.21.15.64:0/2083142106 learned_addr learned my addr 172.21.15.64:0/2083142106 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f9893fff700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98940b0110 0x7f98940c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f98937fe700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b2c0 0x7f98940a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 -- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b2c0 msgr2=0x7f98940a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b2c0 0x7f98940a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 -- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98940b0110 msgr2=0x7f98940c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98940b0110 0x7f98940c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f988bfff700 1 -- 172.21.15.64:0/2083142106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009a20 con 0x7f989406b690 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.089+0000 7f98937fe700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b2c0 0x7f98940a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988bfff700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 0x7f98940aaa00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e8d0 tx=0x7f987c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c013410 con 0x7f989406b690 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c013570 con 0x7f989406b690 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98940ab8f0 con 0x7f989406b690 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c013910 con 0x7f989406b690 2023-09-27T23:52:05.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98940c16a0 con 0x7f989406b690 2023-09-27T23:52:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f987c013d00 con 0x7f989406b690 2023-09-27T23:52:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9894071120 con 0x7f989406b690 2023-09-27T23:52:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988b7fe700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9874060b90 0x7f9874063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f98937fe700 1 -- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9874060b90 msgr2=0x7f9874063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f987c07d370 con 0x7f989406b690 2023-09-27T23:52:05.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.093+0000 7f98937fe700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9874060b90 0x7f9874063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:05.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.101+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f987c04cf10 con 0x7f989406b690 2023-09-27T23:52:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.261+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98940b0c20 con 0x7f989406b690 2023-09-27T23:52:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.261+0000 7f988b7fe700 1 -- 172.21.15.64:0/2083142106 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v175) v1 ==== 92+0+10 (secure 0 0 0) 0x7f987c050530 con 0x7f989406b690 2023-09-27T23:52:05.267 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9874060b90 msgr2=0x7f9874063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9874060b90 0x7f9874063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 msgr2=0x7f98940aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 0x7f98940aaa00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e8d0 tx=0x7f987c008040 comp rx=0 tx=0).stop 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 shutdown_connections 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9874060b90 0x7f9874063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98940b0110 0x7f98940c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989406b690 0x7f98940aaa00 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 --2- 172.21.15.64:0/2083142106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989406b2c0 0x7f98940a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 >> 172.21.15.64:0/2083142106 conn(0x7f98940cc580 msgr2=0x7f9894158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 shutdown_connections 2023-09-27T23:52:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.265+0000 7f9899f01700 1 -- 172.21.15.64:0/2083142106 wait complete. 2023-09-27T23:52:05.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:52:05.307 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:05.303+0000 7f3c36e32000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:52:05.392 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:05.389+0000 7fe30ea74000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:52:05.417 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:05.415+0000 7f3c36e32000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:52:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa2b105700 1 Processor -- start 2023-09-27T23:52:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa2b105700 1 -- start start 2023-09-27T23:52:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa2b105700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faa2413f7e0 0x7faa241564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa2b105700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa24156af0 con 0x7faa2413fbb0 2023-09-27T23:52:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa2b105700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa24156dd0 con 0x7faa240cba00 2023-09-27T23:52:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa2b105700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa24157950 con 0x7faa2413f7e0 2023-09-27T23:52:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7faa2413fbb0 0x7faa240af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34044/0 (socket says 172.21.15.64:34044) 2023-09-27T23:52:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1bfff700 1 -- 172.21.15.64:0/338776739 learned_addr learned my addr 172.21.15.64:0/338776739 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa28ea1700 1 --2- 172.21.15.64:0/338776739 >> v2:172.21.15.123:3300/0 conn(0x7faa2413f7e0 0x7faa241564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827478187 0 0) 0x7faa24156af0 con 0x7faa2413fbb0 2023-09-27T23:52:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0c003570 con 0x7faa2413fbb0 2023-09-27T23:52:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557227390 0 0) 0x7faa0c003570 con 0x7faa2413fbb0 2023-09-27T23:52:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 >> v2:172.21.15.123:3300/0 conn(0x7faa2413f7e0 msgr2=0x7faa241564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 --2- 172.21.15.64:0/338776739 >> v2:172.21.15.123:3300/0 conn(0x7faa2413f7e0 0x7faa241564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 >> v1:172.21.15.123:6789/0 conn(0x7faa240cba00 legacy=0x7faa24157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa241570b0 con 0x7faa2413fbb0 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7faa14001470 con 0x7faa2413fbb0 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 >> v1:172.21.15.64:6789/0 conn(0x7faa2413fbb0 legacy=0x7faa240af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.433+0000 7faa1b7fe700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa0c003a60 0x7faa0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa24157950 con 0x7faa0c004810 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c00e2e0 con 0x7faa0c0051c0 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c00e4a0 con 0x7faa0c003a60 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa28ea1700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa0c003a60 0x7faa0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa296a2700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 -- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa0c003a60 msgr2=0x7faa0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa0c003a60 0x7faa0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 -- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0c0051c0 msgr2=0x7faa0c00dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 -- 172.21.15.64:0/338776739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c003570 con 0x7faa0c004810 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa296a2700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa296a2700 1 -- 172.21.15.64:0/338776739 reap_dead start 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1bfff700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c3c0 tx=0x7faa14001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d37c2415aaae81 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400cc90 con 0x7faa0c004810 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1400cdf0 con 0x7faa0c004810 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa1b7fe700 1 -- 172.21.15.64:0/338776739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14005b50 con 0x7faa0c004810 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 -- 172.21.15.64:0/338776739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 msgr2=0x7faa0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c3c0 tx=0x7faa14001e50 comp rx=0 tx=0).stop 2023-09-27T23:52:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 -- 172.21.15.64:0/338776739 shutdown_connections 2023-09-27T23:52:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 --2- 172.21.15.64:0/338776739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 -- 172.21.15.64:0/338776739 >> 172.21.15.64:0/338776739 conn(0x7faa240cd8f0 msgr2=0x7faa240c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 -- 172.21.15.64:0/338776739 shutdown_connections 2023-09-27T23:52:05.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 -- 172.21.15.64:0/338776739 wait complete. 2023-09-27T23:52:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 Processor -- start 2023-09-27T23:52:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.437+0000 7faa2b105700 1 -- start start 2023-09-27T23:52:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa2413b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa2413bba0 0x7faa2413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa240b1bd0 0x7faa240b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa241663c0 con 0x7faa0c004810 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa240b2570 con 0x7faa2413bba0 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa24166660 con 0x7faa240b1bd0 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa2413b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa2413b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49230/0 (socket says 172.21.15.64:49230) 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 -- 172.21.15.64:0/3152988015 learned_addr learned my addr 172.21.15.64:0/3152988015 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa296a2700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa240b1bd0 0x7faa240b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa1bfff700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa2413bba0 0x7faa2413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 -- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa240b1bd0 msgr2=0x7faa240b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa240b1bd0 0x7faa240b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 -- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa2413bba0 msgr2=0x7faa2413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa2413bba0 0x7faa2413bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 -- 172.21.15.64:0/3152988015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24069190 con 0x7faa0c004810 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa28ea1700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa2413b390 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7faa20011d40 tx=0x7faa200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000fa60 con 0x7faa0c004810 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2000fbc0 con 0x7faa0c004810 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2413c640 con 0x7faa0c004810 2023-09-27T23:52:05.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20014440 con 0x7faa0c004810 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2413cc00 con 0x7faa0c004810 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.441+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa24071120 con 0x7faa0c004810 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.445+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7faa20014a80 con 0x7faa0c004810 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.445+0000 7faa18ff9700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7faa10060b90 0x7faa10063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.445+0000 7faa1bfff700 1 -- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7faa10060b90 msgr2=0x7faa10063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.445+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7faa20018030 con 0x7faa0c004810 2023-09-27T23:52:05.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.445+0000 7faa1bfff700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7faa10060b90 0x7faa10063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:05.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.449+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7faa2004df80 con 0x7faa0c004810 2023-09-27T23:52:05.509 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:05.505+0000 7fe30ea74000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:52:05.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.605+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa2413ce20 con 0x7faa0c004810 2023-09-27T23:52:05.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.605+0000 7faa18ff9700 1 -- 172.21.15.64:0/3152988015 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa200515a0 con 0x7faa0c004810 2023-09-27T23:52:05.611 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7faa10060b90 msgr2=0x7faa10063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7faa10060b90 0x7faa10063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 msgr2=0x7faa2413b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa2413b390 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7faa20011d40 tx=0x7faa200091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 shutdown_connections 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7faa10060b90 0x7faa10063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faa240b1bd0 0x7faa240b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa2413bba0 0x7faa2413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 --2- 172.21.15.64:0/3152988015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa0c004810 0x7faa2413b390 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 >> 172.21.15.64:0/3152988015 conn(0x7faa240cd8f0 msgr2=0x7faa24155860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 shutdown_connections 2023-09-27T23:52:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.609+0000 7faa2b105700 1 -- 172.21.15.64:0/3152988015 wait complete. 2023-09-27T23:52:05.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:52:05.696 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:05.695+0000 7f3c36e32000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:52:05.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.729+0000 7feea814a700 1 Processor -- start 2023-09-27T23:52:05.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.729+0000 7feea814a700 1 -- start start 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feea01514a0 0x7feea0157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea00cae60 0x7feea006b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feea006b5d0 0x7feea015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea006ba50 con 0x7feea006b5d0 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea00cbee0 con 0x7feea01514a0 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea0158530 con 0x7feea00cae60 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feea006b5d0 0x7feea015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feea006b5d0 0x7feea015ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49232/0 (socket says 172.21.15.64:49232) 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 -- 172.21.15.64:0/1518211910 learned_addr learned my addr 172.21.15.64:0/1518211910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea56e5700 1 --2- 172.21.15.64:0/1518211910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea00cae60 0x7feea006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea5ee6700 1 --2- 172.21.15.64:0/1518211910 >> v2:172.21.15.123:3300/0 conn(0x7feea01514a0 0x7feea0157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 -- 172.21.15.64:0/1518211910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea00cae60 msgr2=0x7feea006b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 --2- 172.21.15.64:0/1518211910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea00cae60 0x7feea006b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 -- 172.21.15.64:0/1518211910 >> v2:172.21.15.123:3300/0 conn(0x7feea01514a0 msgr2=0x7feea0157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 --2- 172.21.15.64:0/1518211910 >> v2:172.21.15.123:3300/0 conn(0x7feea01514a0 0x7feea0157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 -- 172.21.15.64:0/1518211910 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00cc370 con 0x7feea006b5d0 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea66e7700 1 --2- 172.21.15.64:0/1518211910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea006b5d0 0x7feea015ab70 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7feea006ae90 tx=0x7fee98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d11e4d7af7c754 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:05.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea4ee4700 1 -- 172.21.15.64:0/1518211910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9800e070 con 0x7feea006b5d0 2023-09-27T23:52:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea4ee4700 1 -- 172.21.15.64:0/1518211910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9800b970 con 0x7feea006b5d0 2023-09-27T23:52:05.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea4ee4700 1 -- 172.21.15.64:0/1518211910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980092e0 con 0x7feea006b5d0 2023-09-27T23:52:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 -- 172.21.15.64:0/1518211910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea006b5d0 msgr2=0x7feea015ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 --2- 172.21.15.64:0/1518211910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea006b5d0 0x7feea015ab70 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7feea006ae90 tx=0x7fee98009840 comp rx=0 tx=0).stop 2023-09-27T23:52:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 -- 172.21.15.64:0/1518211910 shutdown_connections 2023-09-27T23:52:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 --2- 172.21.15.64:0/1518211910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea006b5d0 0x7feea015ab70 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.733+0000 7feea814a700 1 --2- 172.21.15.64:0/1518211910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea00cae60 0x7feea006b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 --2- 172.21.15.64:0/1518211910 >> v2:172.21.15.123:3300/0 conn(0x7feea01514a0 0x7feea0157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- 172.21.15.64:0/1518211910 >> 172.21.15.64:0/1518211910 conn(0x7feea00cc580 msgr2=0x7feea00cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- 172.21.15.64:0/1518211910 shutdown_connections 2023-09-27T23:52:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- 172.21.15.64:0/1518211910 wait complete. 2023-09-27T23:52:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 Processor -- start 2023-09-27T23:52:05.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- start start 2023-09-27T23:52:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea006b5d0 0x7feea0137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea00cae60 0x7feea013f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 0x7feea013fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea006c7c0 con 0x7feea01514a0 2023-09-27T23:52:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea0138440 con 0x7feea00cae60 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea814a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea006ca10 con 0x7feea006b5d0 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 0x7feea013fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 0x7feea013fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49248/0 (socket says 172.21.15.64:49248) 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 -- 172.21.15.64:0/1846478530 learned_addr learned my addr 172.21.15.64:0/1846478530 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea56e5700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea00cae60 0x7feea013f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea5ee6700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea006b5d0 0x7feea0137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 -- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea006b5d0 msgr2=0x7feea0137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea006b5d0 0x7feea0137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 -- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea00cae60 msgr2=0x7feea013f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea00cae60 0x7feea013f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 -- 172.21.15.64:0/1846478530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98009ae0 con 0x7feea01514a0 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea56e5700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea00cae60 0x7feea013f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea5ee6700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea006b5d0 0x7feea0137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.737+0000 7feea66e7700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 0x7feea013fac0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7feea0064900 tx=0x7fee98018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98008b80 con 0x7feea01514a0 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98008ce0 con 0x7feea01514a0 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea006ccf0 con 0x7feea01514a0 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980266b0 con 0x7feea01514a0 2023-09-27T23:52:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea006d2b0 con 0x7feea01514a0 2023-09-27T23:52:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea006d500 con 0x7feea01514a0 2023-09-27T23:52:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fee98003040 con 0x7feea01514a0 2023-09-27T23:52:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7fee95ffb700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fee84060b90 0x7fee84063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7feea5ee6700 1 -- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fee84060b90 msgr2=0x7fee84063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fee9807f5d0 con 0x7feea01514a0 2023-09-27T23:52:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.741+0000 7feea5ee6700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fee84060b90 0x7fee84063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:05.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.749+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fee9804f170 con 0x7feea01514a0 2023-09-27T23:52:05.810 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:05.805+0000 7fe30ea74000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:52:05.884 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:05.879+0000 7f3c36e32000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:05.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.921+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feea0071120 con 0x7feea01514a0 2023-09-27T23:52:05.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7fee95ffb700 1 -- 172.21.15.64:0/1846478530 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee98052790 con 0x7feea01514a0 2023-09-27T23:52:05.928 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:52:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fee84060b90 msgr2=0x7fee84063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:05.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fee84060b90 0x7fee84063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 msgr2=0x7feea013fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 0x7feea013fac0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7feea0064900 tx=0x7fee98018000 comp rx=0 tx=0).stop 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 shutdown_connections 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fee84060b90 0x7fee84063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feea01514a0 0x7feea013fac0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea00cae60 0x7feea013f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 --2- 172.21.15.64:0/1846478530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feea006b5d0 0x7feea0137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 >> 172.21.15.64:0/1846478530 conn(0x7feea00cc580 msgr2=0x7feea00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 shutdown_connections 2023-09-27T23:52:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:05.925+0000 7feea814a700 1 -- 172.21.15.64:0/1846478530 wait complete. 2023-09-27T23:52:05.941 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:05.951 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:05.947+0000 7f3c36e32000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:52:05.990 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:05.985+0000 7fe30ea74000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:06.009 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.007+0000 7f3c36e32000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:52:06.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c73fb700 1 Processor -- start 2023-09-27T23:52:06.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c73fb700 1 -- start start 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c73fb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9c006de30 0x7fb9c00cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c73fb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c006d9a0 con 0x7fb9c00cc5e0 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c73fb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00cce80 con 0x7fb9c00c8ad0 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c73fb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c00ad250 con 0x7fb9c006de30 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb9c00cc5e0 0x7fb9c0162da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34046/0 (socket says 172.21.15.64:34046) 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 -- 172.21.15.64:0/1828800568 learned_addr learned my addr 172.21.15.64:0/1828800568 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c4996700 1 --2- 172.21.15.64:0/1828800568 >> v2:172.21.15.123:3300/0 conn(0x7fb9c006de30 0x7fb9c00cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783291113 0 0) 0x7fb9c006d9a0 con 0x7fb9c00cc5e0 2023-09-27T23:52:06.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a8003570 con 0x7fb9c00cc5e0 2023-09-27T23:52:06.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165106217 0 0) 0x7fb9c00cce80 con 0x7fb9c00c8ad0 2023-09-27T23:52:06.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c006d9a0 con 0x7fb9c00c8ad0 2023-09-27T23:52:06.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143570835 0 0) 0x7fb9a8003570 con 0x7fb9c00cc5e0 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 >> v2:172.21.15.123:3300/0 conn(0x7fb9c006de30 msgr2=0x7fb9c00cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 --2- 172.21.15.64:0/1828800568 >> v2:172.21.15.123:3300/0 conn(0x7fb9c006de30 0x7fb9c00cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 >> v1:172.21.15.123:6789/0 conn(0x7fb9c00c8ad0 legacy=0x7fb9c00c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00ccba0 con 0x7fb9c00cc5e0 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fb9b80017f0 con 0x7fb9c00cc5e0 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c4996700 1 --2- 172.21.15.64:0/1828800568 >> v2:172.21.15.123:3300/0 conn(0x7fb9c006de30 0x7fb9c00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 >> v1:172.21.15.64:6789/0 conn(0x7fb9c00cc5e0 legacy=0x7fb9c0162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a8003bf0 0x7fb9a8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9a8004810 0x7fb9a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00ad250 con 0x7fb9a80051c0 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a800c0b0 con 0x7fb9a8003bf0 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a800c270 con 0x7fb9a8004810 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5197700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a8003bf0 0x7fb9a8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c4996700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9a8004810 0x7fb9a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 -- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9a8004810 msgr2=0x7fb9a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9a8004810 0x7fb9a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 -- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a8003bf0 msgr2=0x7fb9a8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a8003bf0 0x7fb9a8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 -- 172.21.15.64:0/1828800568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8003570 con 0x7fb9a80051c0 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5197700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9a8003bf0 0x7fb9a8003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 -- 172.21.15.64:0/1828800568 reap_dead start 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.033+0000 7fb9c5998700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9a800ba60 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800a8f0 tx=0x7fb9b8002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4774a4f45821da6c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80070d0 con 0x7fb9a80051c0 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8007250 con 0x7fb9a80051c0 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9bffff700 1 -- 172.21.15.64:0/1828800568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8007610 con 0x7fb9a80051c0 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 -- 172.21.15.64:0/1828800568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 msgr2=0x7fb9a800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9a800ba60 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800a8f0 tx=0x7fb9b8002190 comp rx=0 tx=0).stop 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 -- 172.21.15.64:0/1828800568 shutdown_connections 2023-09-27T23:52:06.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/1828800568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9a800ba60 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 -- 172.21.15.64:0/1828800568 >> 172.21.15.64:0/1828800568 conn(0x7fb9c00ce000 msgr2=0x7fb9c00ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 -- 172.21.15.64:0/1828800568 shutdown_connections 2023-09-27T23:52:06.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 -- 172.21.15.64:0/1828800568 wait complete. 2023-09-27T23:52:06.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 Processor -- start 2023-09-27T23:52:06.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.037+0000 7fb9c73fb700 1 -- start start 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9c0169300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9c0155160 0x7fb9c00b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c01555b0 0x7fb9c00c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00aed80 con 0x7fb9a80051c0 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0169840 con 0x7fb9c01555b0 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00ae170 con 0x7fb9c0155160 2023-09-27T23:52:06.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9c0169300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9c0169300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49272/0 (socket says 172.21.15.64:49272) 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 -- 172.21.15.64:0/4086423301 learned_addr learned my addr 172.21.15.64:0/4086423301 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c4996700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9c0155160 0x7fb9c00b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5998700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c01555b0 0x7fb9c00c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 -- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9c0155160 msgr2=0x7fb9c00b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9c0155160 0x7fb9c00b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 -- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c01555b0 msgr2=0x7fb9c00c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c01555b0 0x7fb9c00c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 -- 172.21.15.64:0/4086423301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00ae450 con 0x7fb9a80051c0 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c5197700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9c0169300 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e7b0 tx=0x7fb9b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00026a0 con 0x7fb9a80051c0 2023-09-27T23:52:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0010040 con 0x7fb9a80051c0 2023-09-27T23:52:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0002a10 con 0x7fb9a80051c0 2023-09-27T23:52:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c00ae730 con 0x7fb9a80051c0 2023-09-27T23:52:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c0155b60 con 0x7fb9a80051c0 2023-09-27T23:52:06.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.041+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0071120 con 0x7fb9a80051c0 2023-09-27T23:52:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.045+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7fb9b00143f0 con 0x7fb9a80051c0 2023-09-27T23:52:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.045+0000 7fb9bd7fa700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 0x7fb9ac063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.045+0000 7fb9c4996700 1 -- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 msgr2=0x7fb9ac063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.045+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6778+0+0 (secure 0 0 0) 0x7fb9b007cfd0 con 0x7fb9a80051c0 2023-09-27T23:52:06.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.045+0000 7fb9c4996700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 0x7fb9ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:06.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.049+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb9b004bbb0 con 0x7fb9a80051c0 2023-09-27T23:52:06.061 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.057+0000 7fe30ea74000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:52:06.068 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.067+0000 7f3c36e32000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:52:06.113 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.109+0000 7fe30ea74000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:52:06.170 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.165+0000 7fe30ea74000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:52:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.245+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9c00aebb0 con 0x7fb9a80051c0 2023-09-27T23:52:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.245+0000 7fb9bd7fa700 1 -- 172.21.15.64:0/4086423301 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fb9b0019020 con 0x7fb9a80051c0 2023-09-27T23:52:06.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:06.250 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:50:34.520359+0000","modified":"2023-09-27T23:50:36.536424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/660311009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":660311009},{"type":"v1","addr":"172.21.15.64:6835","nonce":660311009}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-09-27T23:52:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c4996700 1 -- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 msgr2=0x7fb9ac063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:06.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c4996700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 0x7fb9ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:52:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 msgr2=0x7fb9ac063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 0x7fb9ac063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 msgr2=0x7fb9c0169300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9c0169300 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e7b0 tx=0x7fb9b0009040 comp rx=0 tx=0).stop 2023-09-27T23:52:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 shutdown_connections 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb9ac060b90 0x7fb9ac063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c01555b0 0x7fb9c00c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9c0155160 0x7fb9c00b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 --2- 172.21.15.64:0/4086423301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a80051c0 0x7fb9c0169300 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 >> 172.21.15.64:0/4086423301 conn(0x7fb9c00ce000 msgr2=0x7fb9c00ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 shutdown_connections 2023-09-27T23:52:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.249+0000 7fb9c73fb700 1 -- 172.21.15.64:0/4086423301 wait complete. 2023-09-27T23:52:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 66 2023-09-27T23:52:06.260 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.259+0000 7f3c36e32000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:06.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:06.320 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.319+0000 7f3c36e32000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:52:06.350 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.345+0000 7fe30ea74000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 Processor -- start 2023-09-27T23:52:06.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 -- start start 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f21700c8ad0 0x7f21700cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21700cc5e0 0x7f2170162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217006d9a0 con 0x7f217006de30 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21700cce80 con 0x7f21700cc5e0 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21700ad250 con 0x7f21700c8ad0 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21700cc5e0 0x7f2170162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21700cc5e0 0x7f2170162da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49278/0 (socket says 172.21.15.64:49278) 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 -- 172.21.15.64:0/2662753375 learned_addr learned my addr 172.21.15.64:0/2662753375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216ed9d700 1 --2- 172.21.15.64:0/2662753375 >> v2:172.21.15.123:3300/0 conn(0x7f21700c8ad0 0x7f21700cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 -- 172.21.15.64:0/2662753375 >> v2:172.21.15.123:3300/0 conn(0x7f21700c8ad0 msgr2=0x7f21700cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 --2- 172.21.15.64:0/2662753375 >> v2:172.21.15.123:3300/0 conn(0x7f21700c8ad0 0x7f21700cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 -- 172.21.15.64:0/2662753375 >> v1:172.21.15.123:6789/0 conn(0x7f217006de30 legacy=0x7f21700c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 -- 172.21.15.64:0/2662753375 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700ccba0 con 0x7f21700cc5e0 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216ed9d700 1 --2- 172.21.15.64:0/2662753375 >> v2:172.21.15.123:3300/0 conn(0x7f21700c8ad0 0x7f21700cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216f59e700 1 --2- 172.21.15.64:0/2662753375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cc5e0 0x7f2170162da0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f21700cbb80 tx=0x7f2168009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ee5450400ba619 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216dd9b700 1 -- 172.21.15.64:0/2662753375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216800e070 con 0x7f21700cc5e0 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216dd9b700 1 -- 172.21.15.64:0/2662753375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216800ba50 con 0x7f21700cc5e0 2023-09-27T23:52:06.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f216dd9b700 1 -- 172.21.15.64:0/2662753375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680092e0 con 0x7f21700cc5e0 2023-09-27T23:52:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.365+0000 7f21750f9700 1 -- 172.21.15.64:0/2662753375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cc5e0 msgr2=0x7f2170162da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 --2- 172.21.15.64:0/2662753375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cc5e0 0x7f2170162da0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f21700cbb80 tx=0x7f2168009840 comp rx=0 tx=0).stop 2023-09-27T23:52:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- 172.21.15.64:0/2662753375 shutdown_connections 2023-09-27T23:52:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 --2- 172.21.15.64:0/2662753375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21700cc5e0 0x7f2170162da0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 --2- 172.21.15.64:0/2662753375 >> v2:172.21.15.123:3300/0 conn(0x7f21700c8ad0 0x7f21700cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- 172.21.15.64:0/2662753375 >> 172.21.15.64:0/2662753375 conn(0x7f21700ce000 msgr2=0x7f21700ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- 172.21.15.64:0/2662753375 shutdown_connections 2023-09-27T23:52:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- 172.21.15.64:0/2662753375 wait complete. 2023-09-27T23:52:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 Processor -- start 2023-09-27T23:52:06.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- start start 2023-09-27T23:52:06.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 0x7f21701443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21700c8ad0 0x7f21700afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cc5e0 0x7f217016c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700b2830 con 0x7f217006de30 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700b1cb0 con 0x7f21700c8ad0 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f21750f9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700b1f90 con 0x7f21700cc5e0 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f216ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 0x7f21701443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f216ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 0x7f21701443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49290/0 (socket says 172.21.15.64:49290) 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f216ed9d700 1 -- 172.21.15.64:0/1236888677 learned_addr learned my addr 172.21.15.64:0/1236888677 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f216e59c700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21700c8ad0 0x7f21700afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.369+0000 7f216f59e700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cc5e0 0x7f217016c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216ed9d700 1 -- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cc5e0 msgr2=0x7f217016c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216ed9d700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cc5e0 0x7f217016c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216ed9d700 1 -- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21700c8ad0 msgr2=0x7f21700afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216ed9d700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21700c8ad0 0x7f21700afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216ed9d700 1 -- 172.21.15.64:0/1236888677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009ae0 con 0x7f217006de30 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216f59e700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cc5e0 0x7f217016c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216ed9d700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 0x7f21701443e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f216000e8d0 tx=0x7f2160008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160013410 con 0x7f217006de30 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160013570 con 0x7f217006de30 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21700b2270 con 0x7f217006de30 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160013910 con 0x7f217006de30 2023-09-27T23:52:06.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2170144920 con 0x7f217006de30 2023-09-27T23:52:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2170071120 con 0x7f217006de30 2023-09-27T23:52:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f2160013d40 con 0x7f217006de30 2023-09-27T23:52:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f2156ffd700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f215c060c60 0x7f215c063110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216e59c700 1 -- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f215c060c60 msgr2=0x7f215c063110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6778+0+0 (secure 0 0 0) 0x7f216007d250 con 0x7f217006de30 2023-09-27T23:52:06.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.373+0000 7f216e59c700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f215c060c60 0x7f215c063110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:06.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.381+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f216004cee0 con 0x7f217006de30 2023-09-27T23:52:06.388 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.387+0000 7f3c36e32000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:52:06.408 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.405+0000 7fe30ea74000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:52:06.474 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.469+0000 7fe30ea74000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:52:06.550 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.547+0000 7f3c36e32000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:52:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.549+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21700690d0 con 0x7f217006de30 2023-09-27T23:52:06.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f2156ffd700 1 -- 172.21.15.64:0/1236888677 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+17694 (secure 0 0 0) 0x7f2160050500 con 0x7f217006de30 2023-09-27T23:52:06.557 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:06.557 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":176,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:05.782858+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:50:33.091150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-09-27T23:50:33.495902+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2589357507":"2023-09-28T23:50:50.099294+0000","172.21.15.123:6832/43188":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3659445879":"2023-09-28T23:50:50.099294+0000","172.21.15.123:6833/43188":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3288865613":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/1822031684":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/883856417":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/1530058000":"2023-09-28T23:50:50.099294+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:06.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f215c060c60 msgr2=0x7f215c063110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f215c060c60 0x7f215c063110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 msgr2=0x7f21701443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 0x7f21701443e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f216000e8d0 tx=0x7f2160008040 comp rx=0 tx=0).stop 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 shutdown_connections 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f215c060c60 0x7f215c063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21700cc5e0 0x7f217016c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21700c8ad0 0x7f21700afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 --2- 172.21.15.64:0/1236888677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f217006de30 0x7f21701443e0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 >> 172.21.15.64:0/1236888677 conn(0x7f21700ce000 msgr2=0x7f2170158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 shutdown_connections 2023-09-27T23:52:06.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.553+0000 7f21750f9700 1 -- 172.21.15.64:0/1236888677 wait complete. 2023-09-27T23:52:06.571 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:52:06.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:06.616 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:06.615+0000 7f3c36e32000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:52:06.634 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.629+0000 7fe30ea74000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:52:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.665+0000 7f296ffff700 1 Processor -- start 2023-09-27T23:52:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.665+0000 7f296ffff700 1 -- start start 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.665+0000 7f296ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700c3690 0x7f2970159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.665+0000 7f296ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29700c1910 con 0x7f29700670e0 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.665+0000 7f296ffff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2970159cf0 con 0x7f29700ab910 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.665+0000 7f296ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2970159fd0 con 0x7f29700c3690 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296e7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f29700670e0 0x7f29700c2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34048/0 (socket says 172.21.15.64:34048) 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296e7fc700 1 -- 172.21.15.64:0/3973538427 learned_addr learned my addr 172.21.15.64:0/3973538427 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:06.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296f7fe700 1 --2- 172.21.15.64:0/3973538427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700c3690 0x7f2970159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296dffb700 1 -- 172.21.15.64:0/3973538427 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227471103 0 0) 0x7f29700c1910 con 0x7f29700670e0 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296dffb700 1 -- 172.21.15.64:0/3973538427 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2958003570 con 0x7f29700670e0 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296f7fe700 1 -- 172.21.15.64:0/3973538427 >> v1:172.21.15.123:6789/0 conn(0x7f29700ab910 legacy=0x7f29700cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296f7fe700 1 -- 172.21.15.64:0/3973538427 >> v1:172.21.15.64:6789/0 conn(0x7f29700670e0 legacy=0x7f29700c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296dffb700 1 -- 172.21.15.64:0/3973538427 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107842299 0 0) 0x7f2958003570 con 0x7f29700670e0 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296f7fe700 1 -- 172.21.15.64:0/3973538427 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297015a2b0 con 0x7f29700c3690 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296f7fe700 1 --2- 172.21.15.64:0/3973538427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700c3690 0x7f2970159710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2968009e90 tx=0x7f2968009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78731689b608155 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296dffb700 1 -- 172.21.15.64:0/3973538427 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800e070 con 0x7f29700c3690 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296dffb700 1 -- 172.21.15.64:0/3973538427 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800b920 con 0x7f29700c3690 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296dffb700 1 -- 172.21.15.64:0/3973538427 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964001470 con 0x7f29700c3690 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 -- 172.21.15.64:0/3973538427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700c3690 msgr2=0x7f2970159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 --2- 172.21.15.64:0/3973538427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700c3690 0x7f2970159710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2968009e90 tx=0x7f2968009780 comp rx=0 tx=0).stop 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 -- 172.21.15.64:0/3973538427 shutdown_connections 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 --2- 172.21.15.64:0/3973538427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700c3690 0x7f2970159710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 -- 172.21.15.64:0/3973538427 >> 172.21.15.64:0/3973538427 conn(0x7f297006c080 msgr2=0x7f29700ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 -- 172.21.15.64:0/3973538427 shutdown_connections 2023-09-27T23:52:06.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 -- 172.21.15.64:0/3973538427 wait complete. 2023-09-27T23:52:06.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 Processor -- start 2023-09-27T23:52:06.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.669+0000 7f296ffff700 1 -- start start 2023-09-27T23:52:06.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700670e0 0x7f297014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29700ab910 0x7f29701453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 0x7f29700c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297014c880 con 0x7f29700c3690 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297014d6e0 con 0x7f29700ab910 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700af880 con 0x7f29700670e0 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 0x7f29700c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700670e0 0x7f297014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700670e0 0x7f297014df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:42230/0 (socket says 172.21.15.64:42230) 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296effd700 1 -- 172.21.15.64:0/519151034 learned_addr learned my addr 172.21.15.64:0/519151034 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 0x7f29700c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49296/0 (socket says 172.21.15.64:49296) 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296e7fc700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29700ab910 0x7f29701453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 -- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700670e0 msgr2=0x7f297014df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700670e0 0x7f297014df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 -- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29700ab910 msgr2=0x7f29701453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29700ab910 0x7f29701453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 -- 172.21.15.64:0/519151034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968009a20 con 0x7f29700c3690 2023-09-27T23:52:06.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296f7fe700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 0x7f29700c8ec0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2968014000 tx=0x7f2968017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968012c40 con 0x7f29700c3690 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2968012de0 con 0x7f29700c3690 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297014cad0 con 0x7f29700c3690 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680266b0 con 0x7f29700c3690 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297014d090 con 0x7f29700c3690 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.673+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297004be40 con 0x7f29700c3690 2023-09-27T23:52:06.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.677+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f296801d440 con 0x7f29700c3690 2023-09-27T23:52:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.677+0000 7f2956ffd700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f295c060bc0 0x7f295c063070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.677+0000 7f296effd700 1 -- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f295c060bc0 msgr2=0x7f295c063070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.677+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6778+0+0 (secure 0 0 0) 0x7f296807f8e0 con 0x7f29700c3690 2023-09-27T23:52:06.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.677+0000 7f296effd700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f295c060bc0 0x7f295c063070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:06.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.681+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f296804f4d0 con 0x7f29700c3690 2023-09-27T23:52:06.696 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:06.693+0000 7fe30ea74000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:52:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2970068cc0 con 0x7f29700c3690 2023-09-27T23:52:06.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f2956ffd700 1 -- 172.21.15.64:0/519151034 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f2968016090 con 0x7f29700c3690 2023-09-27T23:52:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:06.878 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:50:34.520359+0000","modified":"2023-09-27T23:50:36.536424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/660311009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":660311009},{"type":"v1","addr":"172.21.15.64:6835","nonce":660311009}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f295c060bc0 msgr2=0x7f295c063070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f295c060bc0 0x7f295c063070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 msgr2=0x7f29700c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 0x7f29700c8ec0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2968014000 tx=0x7f2968017000 comp rx=0 tx=0).stop 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 shutdown_connections 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f295c060bc0 0x7f295c063070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29700c3690 0x7f29700c8ec0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.873+0000 7f296ffff700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29700ab910 0x7f29701453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.877+0000 7f296ffff700 1 --2- 172.21.15.64:0/519151034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29700670e0 0x7f297014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.877+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 >> 172.21.15.64:0/519151034 conn(0x7f297006c080 msgr2=0x7f29700c2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.877+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 shutdown_connections 2023-09-27T23:52:06.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.877+0000 7f296ffff700 1 -- 172.21.15.64:0/519151034 wait complete. 2023-09-27T23:52:06.881 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 66 2023-09-27T23:52:06.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 Processor -- start 2023-09-27T23:52:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 -- start start 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0006b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b000c97c0 0x7f4b0006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b000cb030 con 0x7f4b0015a5e0 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b001599a0 con 0x7f4b000c97c0 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b07916700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000cb290 con 0x7f4b0015a210 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b05eb3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b000c97c0 0x7f4b0006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b04eb1700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4b0015a5e0 0x7f4b000c90b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34060/0 (socket says 172.21.15.64:34060) 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b04eb1700 1 -- 172.21.15.64:0/224893881 learned_addr learned my addr 172.21.15.64:0/224893881 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b056b2700 1 --2- 172.21.15.64:0/224893881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b05eb3700 1 -- 172.21.15.64:0/224893881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 msgr2=0x7f4b0006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:06.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b05eb3700 1 --2- 172.21.15.64:0/224893881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0006b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b05eb3700 1 -- 172.21.15.64:0/224893881 >> v1:172.21.15.64:6789/0 conn(0x7f4b0015a5e0 legacy=0x7f4b000c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b05eb3700 1 -- 172.21.15.64:0/224893881 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000cb570 con 0x7f4b000c97c0 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4af7fff700 1 -- 172.21.15.64:0/224893881 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498786411 0 0) 0x7f4b000cb030 con 0x7f4b0015a5e0 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4b05eb3700 1 --2- 172.21.15.64:0/224893881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b0006b7c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0098e0 tx=0x7f4afc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fa5d8ef041aa07 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4af7fff700 1 -- 172.21.15.64:0/224893881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00b640 con 0x7f4b000c97c0 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4af7fff700 1 -- 172.21.15.64:0/224893881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc00b7a0 con 0x7f4b000c97c0 2023-09-27T23:52:06.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.989+0000 7f4af7fff700 1 -- 172.21.15.64:0/224893881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00bb40 con 0x7f4b000c97c0 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- 172.21.15.64:0/224893881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 msgr2=0x7f4b0006b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 --2- 172.21.15.64:0/224893881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b0006b7c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0098e0 tx=0x7f4afc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- 172.21.15.64:0/224893881 shutdown_connections 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 --2- 172.21.15.64:0/224893881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b0006b7c0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 --2- 172.21.15.64:0/224893881 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0006b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- 172.21.15.64:0/224893881 >> 172.21.15.64:0/224893881 conn(0x7f4b00067410 msgr2=0x7f4b000677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- 172.21.15.64:0/224893881 shutdown_connections 2023-09-27T23:52:06.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- 172.21.15.64:0/224893881 wait complete. 2023-09-27T23:52:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 Processor -- start 2023-09-27T23:52:06.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- start start 2023-09-27T23:52:06.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b000a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0016ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b0015a5e0 0x7f4b00143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b001572b0 con 0x7f4b000c97c0 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00156920 con 0x7f4b0015a5e0 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b07916700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000a4980 con 0x7f4b0015a210 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b000a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b000a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49326/0 (socket says 172.21.15.64:49326) 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 -- 172.21.15.64:0/3938961648 learned_addr learned my addr 172.21.15.64:0/3938961648 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b04eb1700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0016ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b05eb3700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b0015a5e0 0x7f4b00143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 -- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 msgr2=0x7f4b0016ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0016ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 -- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b0015a5e0 msgr2=0x7f4b00143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b0015a5e0 0x7f4b00143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.993+0000 7f4b056b2700 1 -- 172.21.15.64:0/3938961648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc009420 con 0x7f4b000c97c0 2023-09-27T23:52:07.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4b056b2700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b000a4440 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009f40 tx=0x7f4af800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:07.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800cb70 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af800f040 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b00156640 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8014480 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b00154a60 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00071120 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 86) v1 ==== 80475+0+0 (secure 0 0 0) 0x7f4af8014830 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4af57fa700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4ae4060b90 0x7f4ae4063040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6778+0+0 (secure 0 0 0) 0x7f4af807e240 con 0x7f4b000c97c0 2023-09-27T23:52:07.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4b04eb1700 1 -- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4ae4060b90 msgr2=0x7f4ae4063040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:07.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:06.997+0000 7f4b04eb1700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4ae4060b90 0x7f4ae4063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:07.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.005+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4af804de50 con 0x7f4b000c97c0 2023-09-27T23:52:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.193+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b0006be20 con 0x7f4b000c97c0 2023-09-27T23:52:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4af57fa700 1 -- 172.21.15.64:0/3938961648 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f4af8051470 con 0x7f4b000c97c0 2023-09-27T23:52:07.201 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:07.201 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:50:34.520359+0000","modified":"2023-09-27T23:50:36.536424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/660311009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":660311009},{"type":"v1","addr":"172.21.15.64:6835","nonce":660311009}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-09-27T23:52:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4ae4060b90 msgr2=0x7f4ae4063040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4ae4060b90 0x7f4ae4063040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 msgr2=0x7f4b000a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b000a4440 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009f40 tx=0x7f4af800b040 comp rx=0 tx=0).stop 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 shutdown_connections 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4ae4060b90 0x7f4ae4063040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b0015a5e0 0x7f4b00143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b0015a210 0x7f4b0016ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 --2- 172.21.15.64:0/3938961648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b000c97c0 0x7f4b000a4440 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 >> 172.21.15.64:0/3938961648 conn(0x7f4b00067410 msgr2=0x7f4b0014f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 shutdown_connections 2023-09-27T23:52:07.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.197+0000 7f4b07916700 1 -- 172.21.15.64:0/3938961648 wait complete. 2023-09-27T23:52:07.204 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 66 2023-09-27T23:52:07.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:07.230 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.227+0000 7f3c36e32000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:52:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 Processor -- start 2023-09-27T23:52:07.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 -- start start 2023-09-27T23:52:07.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f90f8155350 0x7f90f8159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f90f80ab960 0x7f90f806b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f80b0080 con 0x7f90f8155720 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f815a010 con 0x7f90f8155350 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f910071d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f806e740 con 0x7f90f80ab960 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f90f8155350 0x7f90f8159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f90f8155350 0x7f90f8159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49330/0 (socket says 172.21.15.64:49330) 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 -- 172.21.15.64:0/2089361025 learned_addr learned my addr 172.21.15.64:0/2089361025 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fdcb8700 1 --2- 172.21.15.64:0/2089361025 >> v2:172.21.15.123:3300/0 conn(0x7f90f80ab960 0x7f90f806b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 -- 172.21.15.64:0/2089361025 >> v2:172.21.15.123:3300/0 conn(0x7f90f80ab960 msgr2=0x7f90f806b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:07.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 --2- 172.21.15.64:0/2089361025 >> v2:172.21.15.123:3300/0 conn(0x7f90f80ab960 0x7f90f806b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 -- 172.21.15.64:0/2089361025 >> v1:172.21.15.123:6789/0 conn(0x7f90f8155720 legacy=0x7f90f806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 -- 172.21.15.64:0/2089361025 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f80af470 con 0x7f90f8155350 2023-09-27T23:52:07.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fe4b9700 1 --2- 172.21.15.64:0/2089361025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155350 0x7f90f8159ad0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f90f8070490 tx=0x7f90f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100e139bd81ad137 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:07.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fd4b7700 1 -- 172.21.15.64:0/2089361025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4009e70 con 0x7f90f8155350 2023-09-27T23:52:07.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fd4b7700 1 -- 172.21.15.64:0/2089361025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f400b860 con 0x7f90f8155350 2023-09-27T23:52:07.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.317+0000 7f90fd4b7700 1 -- 172.21.15.64:0/2089361025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f400bc00 con 0x7f90f8155350 2023-09-27T23:52:07.326 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.319+0000 7f3c01871700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.326 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.319+0000 7f3c01871700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.326 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.319+0000 7f3c01871700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.326 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.319+0000 7f3c01871700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.319+0000 7f3c01871700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.323+0000 7f3c04877700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.323+0000 7f3c04877700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.323+0000 7f3c04877700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.323+0000 7f3c04877700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.327 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:07.323+0000 7f3c04877700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 -- 172.21.15.64:0/2089361025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155350 msgr2=0x7f90f8159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 --2- 172.21.15.64:0/2089361025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155350 0x7f90f8159ad0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f90f8070490 tx=0x7f90f4009250 comp rx=0 tx=0).stop 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 -- 172.21.15.64:0/2089361025 shutdown_connections 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 --2- 172.21.15.64:0/2089361025 >> v2:172.21.15.123:3300/0 conn(0x7f90f80ab960 0x7f90f806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 --2- 172.21.15.64:0/2089361025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155350 0x7f90f8159ad0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 -- 172.21.15.64:0/2089361025 >> 172.21.15.64:0/2089361025 conn(0x7f90f8067410 msgr2=0x7f90f80677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 -- 172.21.15.64:0/2089361025 shutdown_connections 2023-09-27T23:52:07.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 -- 172.21.15.64:0/2089361025 wait complete. 2023-09-27T23:52:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 Processor -- start 2023-09-27T23:52:07.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.321+0000 7f910071d700 1 -- start start 2023-09-27T23:52:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f910071d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90f80ab960 0x7f90f80cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f910071d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90f8155350 0x7f90f81472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f910071d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 0x7f90f8147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f910071d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80cb280 con 0x7f90f8155720 2023-09-27T23:52:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f910071d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80c9f40 con 0x7f90f80ab960 2023-09-27T23:52:07.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f910071d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80ca1e0 con 0x7f90f8155350 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 0x7f90f8147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fe4b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90f80ab960 0x7f90f80cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 0x7f90f8147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49336/0 (socket says 172.21.15.64:49336) 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 -- 172.21.15.64:0/451087340 learned_addr learned my addr 172.21.15.64:0/451087340 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fe4b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90f80ab960 0x7f90f80cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:50628/0 (socket says 172.21.15.64:50628) 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fdcb8700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90f8155350 0x7f90f81472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 -- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90f8155350 msgr2=0x7f90f81472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90f8155350 0x7f90f81472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 -- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90f80ab960 msgr2=0x7f90f80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90f80ab960 0x7f90f80cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 -- 172.21.15.64:0/451087340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40094f0 con 0x7f90f8155720 2023-09-27T23:52:07.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.325+0000 7f90fecba700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 0x7f90f8147830 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f90f000ed50 tx=0x7f90f000c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:07.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000ee50 con 0x7f90f8155720 2023-09-27T23:52:07.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f0001e00 con 0x7f90f8155720 2023-09-27T23:52:07.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f80ca4c0 con 0x7f90f8155720 2023-09-27T23:52:07.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f80caa80 con 0x7f90f8155720 2023-09-27T23:52:07.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0011470 con 0x7f90f8155720 2023-09-27T23:52:07.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 87) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f90f0003030 con 0x7f90f8155720 2023-09-27T23:52:07.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e00052b0 con 0x7f90f8155720 2023-09-27T23:52:07.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.329+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7094+0+0 (secure 0 0 0) 0x7f90f0015070 con 0x7f90f8155720 2023-09-27T23:52:07.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.337+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f90f004f7d0 con 0x7f90f8155720 2023-09-27T23:52:07.360 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:07.357+0000 7fe30ea74000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:52:07.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.493+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e0005560 con 0x7f90f8155720 2023-09-27T23:52:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f90ee7fc700 1 -- 172.21.15.64:0/451087340 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18152 (secure 0 0 0) 0x7f90f0052df0 con 0x7f90f8155720 2023-09-27T23:52:07.501 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:07.501 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":177,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:07.251183+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:50:33.091150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-09-27T23:50:33.495902+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3312795330":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/1560334267":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/1480322982":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/2539930007":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/750042396":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/4189138962":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/2589357507":"2023-09-28T23:50:50.099294+0000","172.21.15.123:6832/43188":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3659445879":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/614088824":"2023-09-28T23:52:07.251116+0000","172.21.15.123:6833/43188":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3288865613":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/3933455277":"2023-09-28T23:52:07.251116+0000","172.21.15.123:0/1822031684":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/883856417":"2023-09-28T23:50:50.099294+0000","172.21.15.123:0/1530058000":"2023-09-28T23:50:50.099294+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:07.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 msgr2=0x7f90f8147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 0x7f90f8147830 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f90f000ed50 tx=0x7f90f000c5d0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 shutdown_connections 2023-09-27T23:52:07.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90f8155720 0x7f90f8147830 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90f8155350 0x7f90f81472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 --2- 172.21.15.64:0/451087340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90f80ab960 0x7f90f80cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 >> 172.21.15.64:0/451087340 conn(0x7f90f8067410 msgr2=0x7f90f8152fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:07.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 shutdown_connections 2023-09-27T23:52:07.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.497+0000 7f910071d700 1 -- 172.21.15.64:0/451087340 wait complete. 2023-09-27T23:52:07.515 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:52:07.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 Processor -- start 2023-09-27T23:52:07.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 -- start start 2023-09-27T23:52:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62d815a8d0 0x7f62d815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62d8158880 0x7f62d8158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d815b1e0 con 0x7f62d81515a0 2023-09-27T23:52:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80cb400 con 0x7f62d8158880 2023-09-27T23:52:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62df951700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80cb6e0 con 0x7f62d815a8d0 2023-09-27T23:52:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62d8158880 0x7f62d8158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62dd6ed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62d815a8d0 0x7f62d815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62d8158880 0x7f62d8158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49346/0 (socket says 172.21.15.64:49346) 2023-09-27T23:52:07.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 -- 172.21.15.64:0/3832034035 learned_addr learned my addr 172.21.15.64:0/3832034035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 -- 172.21.15.64:0/3832034035 >> v2:172.21.15.123:3300/0 conn(0x7f62d815a8d0 msgr2=0x7f62d815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:07.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 --2- 172.21.15.64:0/3832034035 >> v2:172.21.15.123:3300/0 conn(0x7f62d815a8d0 0x7f62d815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 -- 172.21.15.64:0/3832034035 >> v1:172.21.15.123:6789/0 conn(0x7f62d81515a0 legacy=0x7f62d80cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 -- 172.21.15.64:0/3832034035 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80cb9c0 con 0x7f62d8158880 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62ddeee700 1 --2- 172.21.15.64:0/3832034035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8158880 0x7f62d8158ce0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f62d806f1a0 tx=0x7f62d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb32ba49b21c48 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62cbfff700 1 -- 172.21.15.64:0/3832034035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0009e70 con 0x7f62d8158880 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62cbfff700 1 -- 172.21.15.64:0/3832034035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000b860 con 0x7f62d8158880 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.621+0000 7f62cbfff700 1 -- 172.21.15.64:0/3832034035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000bc00 con 0x7f62d8158880 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- 172.21.15.64:0/3832034035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8158880 msgr2=0x7f62d8158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 --2- 172.21.15.64:0/3832034035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8158880 0x7f62d8158ce0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f62d806f1a0 tx=0x7f62d0009250 comp rx=0 tx=0).stop 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- 172.21.15.64:0/3832034035 shutdown_connections 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 --2- 172.21.15.64:0/3832034035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8158880 0x7f62d8158ce0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 --2- 172.21.15.64:0/3832034035 >> v2:172.21.15.123:3300/0 conn(0x7f62d815a8d0 0x7f62d815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- 172.21.15.64:0/3832034035 >> 172.21.15.64:0/3832034035 conn(0x7f62d80686f0 msgr2=0x7f62d8068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- 172.21.15.64:0/3832034035 shutdown_connections 2023-09-27T23:52:07.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- 172.21.15.64:0/3832034035 wait complete. 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 Processor -- start 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- start start 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62d8158880 0x7f62d80ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d815a8d0 0x7f62d81671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 0x7f62d8153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b1b90 con 0x7f62d8167720 2023-09-27T23:52:07.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b0b50 con 0x7f62d815a8d0 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62df951700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b0df0 con 0x7f62d8158880 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 0x7f62d8153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 0x7f62d8153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49352/0 (socket says 172.21.15.64:49352) 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 -- 172.21.15.64:0/2887138014 learned_addr learned my addr 172.21.15.64:0/2887138014 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62dd6ed700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62d8158880 0x7f62d80ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62dceec700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d815a8d0 0x7f62d81671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 -- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62d8158880 msgr2=0x7f62d80ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62d8158880 0x7f62d80ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 -- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d815a8d0 msgr2=0x7f62d81671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d815a8d0 0x7f62d81671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 -- 172.21.15.64:0/2887138014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00094f0 con 0x7f62d8167720 2023-09-27T23:52:07.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62dd6ed700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62d8158880 0x7f62d80ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.625+0000 7f62ddeee700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 0x7f62d8153de0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009290 tx=0x7f62d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000baa0 con 0x7f62d8167720 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0025420 con 0x7f62d8167720 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d80b10d0 con 0x7f62d8167720 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000bc00 con 0x7f62d8167720 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80b1690 con 0x7f62d8167720 2023-09-27T23:52:07.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8071120 con 0x7f62d8167720 2023-09-27T23:52:07.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 87) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f62d0023070 con 0x7f62d8167720 2023-09-27T23:52:07.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.629+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7094+0+0 (secure 0 0 0) 0x7f62d007e4c0 con 0x7f62d8167720 2023-09-27T23:52:07.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.641+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f62d004e6f0 con 0x7f62d8167720 2023-09-27T23:52:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:07.833+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f62d8155b20 con 0x7f62d8167720 2023-09-27T23:52:08.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.265+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 88) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f62d004eac0 con 0x7f62d8167720 2023-09-27T23:52:08.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.265+0000 7f62c97fa700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f62bc066e20 0x7f62bc069200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.269+0000 7f62dd6ed700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f62bc066e20 0x7f62bc069200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.273+0000 7f62dd6ed700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f62bc066e20 0x7f62bc069200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62d40076c0 tx=0x7f62d400b040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:08.801 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:52:08.797+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:52:08.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.809+0000 7f62c97fa700 1 -- 172.21.15.64:0/2887138014 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7f62d0054370 con 0x7f62d8167720 2023-09-27T23:52:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f62bc066e20 msgr2=0x7f62bc069200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f62bc066e20 0x7f62bc069200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62d40076c0 tx=0x7f62d400b040 comp rx=0 tx=0).stop 2023-09-27T23:52:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 msgr2=0x7f62d8153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 0x7f62d8153de0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009290 tx=0x7f62d0017000 comp rx=0 tx=0).stop 2023-09-27T23:52:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 shutdown_connections 2023-09-27T23:52:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f62bc066e20 0x7f62bc069200 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d8167720 0x7f62d8153de0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d815a8d0 0x7f62d81671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 --2- 172.21.15.64:0/2887138014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62d8158880 0x7f62d80ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 >> 172.21.15.64:0/2887138014 conn(0x7f62d80686f0 msgr2=0x7f62d80c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 shutdown_connections 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.813+0000 7f62df951700 1 -- 172.21.15.64:0/2887138014 wait complete. 2023-09-27T23:52:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:52:08.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:52:08.872 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:52:08.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.945+0000 7efdd0167700 1 Processor -- start 2023-09-27T23:52:08.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.945+0000 7efdd0167700 1 -- start start 2023-09-27T23:52:08.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdd0167700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efdc81554d0 0x7efdc8159a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdd0167700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdd0167700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc806db20 con 0x7efdc806b5a0 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdd0167700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc80cc310 con 0x7efdc81554d0 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdd0167700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc806ba20 con 0x7efdc806e530 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdce704700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efdc806b5a0 0x7efdc80b0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34062/0 (socket says 172.21.15.64:34062) 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdce704700 1 -- 172.21.15.64:0/3204068459 learned_addr learned my addr 172.21.15.64:0/3204068459 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcd702700 1 --2- 172.21.15.64:0/3204068459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcdf03700 1 --2- 172.21.15.64:0/3204068459 >> v2:172.21.15.123:3300/0 conn(0x7efdc81554d0 0x7efdc8159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdccf01700 1 -- 172.21.15.64:0/3204068459 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467937369 0 0) 0x7efdc806db20 con 0x7efdc806b5a0 2023-09-27T23:52:08.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdccf01700 1 -- 172.21.15.64:0/3204068459 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdb4004680 con 0x7efdc806b5a0 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcd702700 1 -- 172.21.15.64:0/3204068459 >> v2:172.21.15.123:3300/0 conn(0x7efdc81554d0 msgr2=0x7efdc8159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcd702700 1 --2- 172.21.15.64:0/3204068459 >> v2:172.21.15.123:3300/0 conn(0x7efdc81554d0 0x7efdc8159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcd702700 1 -- 172.21.15.64:0/3204068459 >> v1:172.21.15.64:6789/0 conn(0x7efdc806b5a0 legacy=0x7efdc80b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcd702700 1 -- 172.21.15.64:0/3204068459 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc806dfb0 con 0x7efdc806e530 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdcd702700 1 --2- 172.21.15.64:0/3204068459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc806b060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efdb80098e0 tx=0x7efdb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f7174fb65d1072 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.949+0000 7efdccf01700 1 -- 172.21.15.64:0/3204068459 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800f6b0 con 0x7efdc806e530 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdccf01700 1 -- 172.21.15.64:0/3204068459 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb800f810 con 0x7efdc806e530 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdccf01700 1 -- 172.21.15.64:0/3204068459 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800fbb0 con 0x7efdc806e530 2023-09-27T23:52:08.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 -- 172.21.15.64:0/3204068459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 msgr2=0x7efdc806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 --2- 172.21.15.64:0/3204068459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc806b060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efdb80098e0 tx=0x7efdb80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 -- 172.21.15.64:0/3204068459 shutdown_connections 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 --2- 172.21.15.64:0/3204068459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc806b060 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 --2- 172.21.15.64:0/3204068459 >> v2:172.21.15.123:3300/0 conn(0x7efdc81554d0 0x7efdc8159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 -- 172.21.15.64:0/3204068459 >> 172.21.15.64:0/3204068459 conn(0x7efdc80cc580 msgr2=0x7efdc80cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 -- 172.21.15.64:0/3204068459 shutdown_connections 2023-09-27T23:52:08.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 -- 172.21.15.64:0/3204068459 wait complete. 2023-09-27T23:52:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 Processor -- start 2023-09-27T23:52:08.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.953+0000 7efdd0167700 1 -- start start 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 0x7efdc80c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc8153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc81554d0 0x7efdc81575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc81528f0 con 0x7efdc806b5a0 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8153710 con 0x7efdc81554d0 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8152b00 con 0x7efdc806e530 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 0x7efdc80c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 0x7efdc80c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49372/0 (socket says 172.21.15.64:49372) 2023-09-27T23:52:08.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 -- 172.21.15.64:0/2165525424 learned_addr learned my addr 172.21.15.64:0/2165525424 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:08.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdce704700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc81554d0 0x7efdc81575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcd702700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc8153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 -- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 msgr2=0x7efdc8153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc8153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 -- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc81554d0 msgr2=0x7efdc81575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc81554d0 0x7efdc81575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 -- 172.21.15.64:0/2165525424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8009420 con 0x7efdc806b5a0 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdcdf03700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 0x7efdc80c7be0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7efdc400ec80 tx=0x7efdc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400c850 con 0x7efdc806b5a0 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc8152de0 con 0x7efdc806b5a0 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc400c9b0 con 0x7efdc806b5a0 2023-09-27T23:52:08.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc81533a0 con 0x7efdc806b5a0 2023-09-27T23:52:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.957+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400cd30 con 0x7efdc806b5a0 2023-09-27T23:52:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.961+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc80c81e0 con 0x7efdc806b5a0 2023-09-27T23:52:08.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.961+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 88) v1 ==== 80097+0+0 (secure 0 0 0) 0x7efdc4005a00 con 0x7efdc806b5a0 2023-09-27T23:52:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.961+0000 7efdbdffb700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efdac0607a0 0x7efdac062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.961+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7180+0+0 (secure 0 0 0) 0x7efdc407cfb0 con 0x7efdc806b5a0 2023-09-27T23:52:08.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.961+0000 7efdcd702700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efdac0607a0 0x7efdac062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.961+0000 7efdcd702700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efdac0607a0 0x7efdac062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efdb80091e0 tx=0x7efdb8019000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:08.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:08.965+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efdc404d080 con 0x7efdc806b5a0 2023-09-27T23:52:09.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.165+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efdc8071120 con 0x7efdc806b5a0 2023-09-27T23:52:09.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.817+0000 7efdbdffb700 1 -- 172.21.15.64:0/2165525424 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7efdc4005cb0 con 0x7efdc806b5a0 2023-09-27T23:52:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efdac0607a0 msgr2=0x7efdac062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efdac0607a0 0x7efdac062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efdb80091e0 tx=0x7efdb8019000 comp rx=0 tx=0).stop 2023-09-27T23:52:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 msgr2=0x7efdc80c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 0x7efdc80c7be0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7efdc400ec80 tx=0x7efdc40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 shutdown_connections 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efdac0607a0 0x7efdac062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc81554d0 0x7efdc81575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdc806e530 0x7efdc8153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 --2- 172.21.15.64:0/2165525424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdc806b5a0 0x7efdc80c7be0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 >> 172.21.15.64:0/2165525424 conn(0x7efdc80cc580 msgr2=0x7efdc80ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 shutdown_connections 2023-09-27T23:52:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.821+0000 7efdd0167700 1 -- 172.21.15.64:0/2165525424 wait complete. 2023-09-27T23:52:09.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:52:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.993+0000 7fbb3d11a700 1 Processor -- start 2023-09-27T23:52:09.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.993+0000 7fbb3d11a700 1 -- start start 2023-09-27T23:52:10.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3d11a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbb380cb130 0x7fbb380cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3d11a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380b0ac0 con 0x7fbb3815c930 2023-09-27T23:52:10.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3d11a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3806b210 con 0x7fbb3806a6a0 2023-09-27T23:52:10.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3d11a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb380ca810 con 0x7fbb380cb130 2023-09-27T23:52:10.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbb3815c930 0x7fbb380b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34068/0 (socket says 172.21.15.64:34068) 2023-09-27T23:52:10.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 -- 172.21.15.64:0/2756952616 learned_addr learned my addr 172.21.15.64:0/2756952616 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb36d9d700 1 --2- 172.21.15.64:0/2756952616 >> v2:172.21.15.123:3300/0 conn(0x7fbb380cb130 0x7fbb380cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578132315 0 0) 0x7fbb380b0ac0 con 0x7fbb3815c930 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb24003570 con 0x7fbb3815c930 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14727519 0 0) 0x7fbb3806b210 con 0x7fbb3806a6a0 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb380b0ac0 con 0x7fbb3806a6a0 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077613550 0 0) 0x7fbb24003570 con 0x7fbb3815c930 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 >> v2:172.21.15.123:3300/0 conn(0x7fbb380cb130 msgr2=0x7fbb380cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 --2- 172.21.15.64:0/2756952616 >> v2:172.21.15.123:3300/0 conn(0x7fbb380cb130 0x7fbb380cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 >> v1:172.21.15.123:6789/0 conn(0x7fbb3806a6a0 legacy=0x7fbb3815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380b0190 con 0x7fbb3815c930 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fbb300017f0 con 0x7fbb3815c930 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 >> v1:172.21.15.64:6789/0 conn(0x7fbb3815c930 legacy=0x7fbb380b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb24003bf0 0x7fbb24003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24004810 0x7fbb24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb240051c0 0x7fbb2400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380ca810 con 0x7fbb240051c0 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2400c0b0 con 0x7fbb24004810 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2400c270 con 0x7fbb24003bf0 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb240051c0 0x7fbb2400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3659c700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24004810 0x7fbb24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb36d9d700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb24003bf0 0x7fbb24003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 -- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb24003bf0 msgr2=0x7fbb24003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb24003bf0 0x7fbb24003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 -- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24004810 msgr2=0x7fbb24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb24004810 0x7fbb24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 -- 172.21.15.64:0/2756952616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb24003570 con 0x7fbb240051c0 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 -- 172.21.15.64:0/2756952616 reap_dead start 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb3759e700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb240051c0 0x7fbb2400ba60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000a8a0 tx=0x7fbb30002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e1f0d4db9019e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:09.997+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300070d0 con 0x7fbb240051c0 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30007230 con 0x7fbb240051c0 2023-09-27T23:52:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb35d9b700 1 -- 172.21.15.64:0/2756952616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300075f0 con 0x7fbb240051c0 2023-09-27T23:52:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 -- 172.21.15.64:0/2756952616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb240051c0 msgr2=0x7fbb2400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb240051c0 0x7fbb2400ba60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000a8a0 tx=0x7fbb30002190 comp rx=0 tx=0).stop 2023-09-27T23:52:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 -- 172.21.15.64:0/2756952616 shutdown_connections 2023-09-27T23:52:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/2756952616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb240051c0 0x7fbb2400ba60 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 -- 172.21.15.64:0/2756952616 >> 172.21.15.64:0/2756952616 conn(0x7fbb38067bb0 msgr2=0x7fbb380cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 -- 172.21.15.64:0/2756952616 shutdown_connections 2023-09-27T23:52:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.001+0000 7fbb3d11a700 1 -- 172.21.15.64:0/2756952616 wait complete. 2023-09-27T23:52:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.005+0000 7fbb3d11a700 1 Processor -- start 2023-09-27T23:52:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.005+0000 7fbb3d11a700 1 -- start start 2023-09-27T23:52:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240051c0 0x7fbb38159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 0x7fbb3815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb3815ea80 0x7fbb3815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3815ada0 con 0x7fbb38159ca0 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3815f460 con 0x7fbb240051c0 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3815a190 con 0x7fbb3815ea80 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 0x7fbb3815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 0x7fbb3815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49398/0 (socket says 172.21.15.64:49398) 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 -- 172.21.15.64:0/1956495992 learned_addr learned my addr 172.21.15.64:0/1956495992 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb36d9d700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240051c0 0x7fbb38159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3759e700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb3815ea80 0x7fbb3815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 -- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb3815ea80 msgr2=0x7fbb3815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb3815ea80 0x7fbb3815ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 -- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240051c0 msgr2=0x7fbb38159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240051c0 0x7fbb38159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 -- 172.21.15.64:0/1956495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3815a470 con 0x7fbb38159ca0 2023-09-27T23:52:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb36d9d700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240051c0 0x7fbb38159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3659c700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 0x7fbb3815e540 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbb20002b80 tx=0x7fbb20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:10.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000fa00 con 0x7fbb38159ca0 2023-09-27T23:52:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000fb60 con 0x7fbb38159ca0 2023-09-27T23:52:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3815a750 con 0x7fbb38159ca0 2023-09-27T23:52:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20006820 con 0x7fbb38159ca0 2023-09-27T23:52:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.009+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb381400e0 con 0x7fbb38159ca0 2023-09-27T23:52:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.013+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3815af50 con 0x7fbb38159ca0 2023-09-27T23:52:10.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.013+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 88) v1 ==== 80097+0+0 (secure 0 0 0) 0x7fbb200133f0 con 0x7fbb38159ca0 2023-09-27T23:52:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.013+0000 7fbb2effd700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb180607d0 0x7fbb18062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.013+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7180+0+0 (secure 0 0 0) 0x7fbb2007d2b0 con 0x7fbb38159ca0 2023-09-27T23:52:10.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.013+0000 7fbb36d9d700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb180607d0 0x7fbb18062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.013+0000 7fbb36d9d700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb180607d0 0x7fbb18062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006c80 tx=0x7fbb28008040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:10.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.021+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbb3815af50 con 0x7fbb38159ca0 2023-09-27T23:52:10.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.185+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb38071120 con 0x7fbb38159ca0 2023-09-27T23:52:10.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.277+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fbb2004d6c0 con 0x7fbb38159ca0 2023-09-27T23:52:10.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.821+0000 7fbb2effd700 1 -- 172.21.15.64:0/1956495992 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v179) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbb2004d2f0 con 0x7fbb38159ca0 2023-09-27T23:52:10.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb180607d0 msgr2=0x7fbb18062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb180607d0 0x7fbb18062c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006c80 tx=0x7fbb28008040 comp rx=0 tx=0).stop 2023-09-27T23:52:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 msgr2=0x7fbb3815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 0x7fbb3815e540 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbb20002b80 tx=0x7fbb20007d30 comp rx=0 tx=0).stop 2023-09-27T23:52:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 shutdown_connections 2023-09-27T23:52:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbb180607d0 0x7fbb18062c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbb3815ea80 0x7fbb3815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb38159ca0 0x7fbb3815e540 secure :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbb20002b80 tx=0x7fbb20007d30 comp rx=0 tx=0).stop 2023-09-27T23:52:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 --2- 172.21.15.64:0/1956495992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb240051c0 0x7fbb38159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.825+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 >> 172.21.15.64:0/1956495992 conn(0x7fbb38067bb0 msgr2=0x7fbb380cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.829+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 shutdown_connections 2023-09-27T23:52:10.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.829+0000 7fbb3d11a700 1 -- 172.21.15.64:0/1956495992 wait complete. 2023-09-27T23:52:10.835 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:52:10.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:52:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.957+0000 7fb581c3a700 1 Processor -- start 2023-09-27T23:52:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.957+0000 7fb581c3a700 1 -- start start 2023-09-27T23:52:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb57c0acd20 0x7fb57c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c0adf30 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c0cd6f0 con 0x7fb57c0acd20 2023-09-27T23:52:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0cd9d0 con 0x7fb57c0af3d0 2023-09-27T23:52:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57affd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb57c0ad0f0 0x7fb57c0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34074/0 (socket says 172.21.15.64:34074) 2023-09-27T23:52:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57affd700 1 -- 172.21.15.64:0/1641990191 learned_addr learned my addr 172.21.15.64:0/1641990191 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57b7fe700 1 --2- 172.21.15.64:0/1641990191 >> v2:172.21.15.64:3300/0 conn(0x7fb57c0acd20 0x7fb57c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57bfff700 1 --2- 172.21.15.64:0/1641990191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57a7fc700 1 -- 172.21.15.64:0/1641990191 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392810875 0 0) 0x7fb57c0adf30 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57a7fc700 1 -- 172.21.15.64:0/1641990191 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb568003570 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57bfff700 1 -- 172.21.15.64:0/1641990191 >> v2:172.21.15.64:3300/0 conn(0x7fb57c0acd20 msgr2=0x7fb57c0cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57bfff700 1 --2- 172.21.15.64:0/1641990191 >> v2:172.21.15.64:3300/0 conn(0x7fb57c0acd20 0x7fb57c0cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57bfff700 1 -- 172.21.15.64:0/1641990191 >> v1:172.21.15.64:6789/0 conn(0x7fb57c0ad0f0 legacy=0x7fb57c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57bfff700 1 -- 172.21.15.64:0/1641990191 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0adbc0 con 0x7fb57c0af3d0 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57b7fe700 1 --2- 172.21.15.64:0/1641990191 >> v2:172.21.15.64:3300/0 conn(0x7fb57c0acd20 0x7fb57c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57bfff700 1 --2- 172.21.15.64:0/1641990191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c06c7c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb574009e60 tx=0x7fb574009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64fab50f7b4c09c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57a7fc700 1 -- 172.21.15.64:0/1641990191 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400e070 con 0x7fb57c0af3d0 2023-09-27T23:52:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57a7fc700 1 -- 172.21.15.64:0/1641990191 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400bab0 con 0x7fb57c0af3d0 2023-09-27T23:52:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb57a7fc700 1 -- 172.21.15.64:0/1641990191 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400be70 con 0x7fb57c0af3d0 2023-09-27T23:52:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- 172.21.15.64:0/1641990191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 msgr2=0x7fb57c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 --2- 172.21.15.64:0/1641990191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c06c7c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb574009e60 tx=0x7fb574009780 comp rx=0 tx=0).stop 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- 172.21.15.64:0/1641990191 shutdown_connections 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 --2- 172.21.15.64:0/1641990191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c06c7c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 --2- 172.21.15.64:0/1641990191 >> v2:172.21.15.64:3300/0 conn(0x7fb57c0acd20 0x7fb57c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- 172.21.15.64:0/1641990191 >> 172.21.15.64:0/1641990191 conn(0x7fb57c0cdce0 msgr2=0x7fb57c0ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- 172.21.15.64:0/1641990191 shutdown_connections 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.961+0000 7fb581c3a700 1 -- 172.21.15.64:0/1641990191 wait complete. 2023-09-27T23:52:10.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 Processor -- start 2023-09-27T23:52:10.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 -- start start 2023-09-27T23:52:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0acd20 0x7fb57c068d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 0x7fb57c0c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c067c90 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c067f70 con 0x7fb57c0acd20 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb581c3a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c068250 con 0x7fb57c0af3d0 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb57affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 0x7fb57c0c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb57affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 0x7fb57c0c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49408/0 (socket says 172.21.15.64:49408) 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb57affd700 1 -- 172.21.15.64:0/3866390773 learned_addr learned my addr 172.21.15.64:0/3866390773 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb57bfff700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.965+0000 7fb57b7fe700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0acd20 0x7fb57c068d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57affd700 1 -- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 msgr2=0x7fb57c147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57affd700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57affd700 1 -- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0acd20 msgr2=0x7fb57c068d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57affd700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0acd20 0x7fb57c068d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57affd700 1 -- 172.21.15.64:0/3866390773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c00d8d0 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57b7fe700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0acd20 0x7fb57c068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb57affd700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 0x7fb57c0c3ce0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c7c0 tx=0x7fb564007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564005c00 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564005d60 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb574009a20 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564007300 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c068810 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57c068a60 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb56401b3b0 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.969+0000 7fb5737fe700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb55c060a70 0x7fb55c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.973+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6742+0+0 (secure 0 0 0) 0x7fb56407c8a0 con 0x7fb57c0ad0f0 2023-09-27T23:52:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.973+0000 7fb57b7fe700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb55c060a70 0x7fb55c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:10.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.973+0000 7fb57b7fe700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb55c060a70 0x7fb55c062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00d440 tx=0x7fb56c00f040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:10.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:10.981+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb56404c760 con 0x7fb57c0ad0f0 2023-09-27T23:52:11.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.137+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb57c071120 con 0x7fb57c0ad0f0 2023-09-27T23:52:11.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.757+0000 7fb5737fe700 1 -- 172.21.15.64:0/3866390773 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v180) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb5640170d0 con 0x7fb57c0ad0f0 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb55c060a70 msgr2=0x7fb55c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb55c060a70 0x7fb55c062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00d440 tx=0x7fb56c00f040 comp rx=0 tx=0).stop 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 msgr2=0x7fb57c0c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 0x7fb57c0c3ce0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c7c0 tx=0x7fb564007dd0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 shutdown_connections 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb55c060a70 0x7fb55c062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0af3d0 0x7fb57c147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb57c0ad0f0 0x7fb57c0c3ce0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 --2- 172.21.15.64:0/3866390773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0acd20 0x7fb57c068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 >> 172.21.15.64:0/3866390773 conn(0x7fb57c0cdce0 msgr2=0x7fb57c0af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:11.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 shutdown_connections 2023-09-27T23:52:11.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.761+0000 7fb581c3a700 1 -- 172.21.15.64:0/3866390773 wait complete. 2023-09-27T23:52:11.769 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:52:11.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:52:11.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.885+0000 7f39e8d9b700 1 Processor -- start 2023-09-27T23:52:11.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.885+0000 7f39e8d9b700 1 -- start start 2023-09-27T23:52:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4158e20 0x7f39e40ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39e40cb970 0x7f39e40ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e40cbdf0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e40cc0d0 con 0x7f39e40cb970 2023-09-27T23:52:11.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e40cc3b0 con 0x7f39e4158e20 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39e40cb970 0x7f39e40ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39e40cb970 0x7f39e40ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49422/0 (socket says 172.21.15.64:49422) 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 -- 172.21.15.64:0/1905463316 learned_addr learned my addr 172.21.15.64:0/1905463316 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e259c700 1 --2- 172.21.15.64:0/1905463316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4158e20 0x7f39e40ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e159a700 1 -- 172.21.15.64:0/1905463316 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994977585 0 0) 0x7f39e40cbdf0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 -- 172.21.15.64:0/1905463316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4158e20 msgr2=0x7f39e40ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 --2- 172.21.15.64:0/1905463316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4158e20 0x7f39e40ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 -- 172.21.15.64:0/1905463316 >> v1:172.21.15.123:6789/0 conn(0x7f39e40ca2c0 legacy=0x7f39e406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 -- 172.21.15.64:0/1905463316 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40cc690 con 0x7f39e40cb970 2023-09-27T23:52:11.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e1d9b700 1 --2- 172.21.15.64:0/1905463316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40cb970 0x7f39e40ceb00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0099b0 tx=0x7f39cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636b5fdb2f187487 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e159a700 1 -- 172.21.15.64:0/1905463316 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc009ea0 con 0x7f39e40cb970 2023-09-27T23:52:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e159a700 1 -- 172.21.15.64:0/1905463316 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc00b860 con 0x7f39e40cb970 2023-09-27T23:52:11.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e159a700 1 -- 172.21.15.64:0/1905463316 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00bc00 con 0x7f39e40cb970 2023-09-27T23:52:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1905463316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40cb970 msgr2=0x7f39e40ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1905463316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40cb970 0x7f39e40ceb00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0099b0 tx=0x7f39cc009250 comp rx=0 tx=0).stop 2023-09-27T23:52:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1905463316 shutdown_connections 2023-09-27T23:52:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1905463316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40cb970 0x7f39e40ceb00 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1905463316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4158e20 0x7f39e40ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1905463316 >> 172.21.15.64:0/1905463316 conn(0x7f39e4067bb0 msgr2=0x7f39e40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1905463316 shutdown_connections 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1905463316 wait complete. 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 Processor -- start 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.889+0000 7f39e8d9b700 1 -- start start 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 0x7f39e406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e4158e20 0x7f39e406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4149ca0 0x7f39e4150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:11.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41537b0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e416d130 con 0x7f39e4158e20 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e416c560 con 0x7f39e4149ca0 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 0x7f39e406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 0x7f39e406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49434/0 (socket says 172.21.15.64:49434) 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 -- 172.21.15.64:0/1181503514 learned_addr learned my addr 172.21.15.64:0/1181503514 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e2d9d700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4149ca0 0x7f39e4150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e1d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e4158e20 0x7f39e406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 -- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4149ca0 msgr2=0x7f39e4150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4149ca0 0x7f39e4150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 -- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e4158e20 msgr2=0x7f39e406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e4158e20 0x7f39e406cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 -- 172.21.15.64:0/1181503514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc0094f0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e2d9d700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4149ca0 0x7f39e4150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:11.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e259c700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 0x7f39e406c7c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f39d400e930 tx=0x7f39d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4013680 con 0x7f39e40ca2c0 2023-09-27T23:52:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d40137e0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e416c800 con 0x7f39e40ca2c0 2023-09-27T23:52:11.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4013b80 con 0x7f39e40ca2c0 2023-09-27T23:52:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e416cdc0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.893+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e4071120 con 0x7f39e40ca2c0 2023-09-27T23:52:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.897+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f39d400c3e0 con 0x7f39e40ca2c0 2023-09-27T23:52:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.897+0000 7f39da7fc700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f39c4060a70 0x7f39c4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.897+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f39d407d190 con 0x7f39e40ca2c0 2023-09-27T23:52:11.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.897+0000 7f39e1d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f39c4060a70 0x7f39c4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:11.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.897+0000 7f39e1d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f39c4060a70 0x7f39c4062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0099b0 tx=0x7f39cc017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:11.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:11.905+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f39d404d1f0 con 0x7f39e40ca2c0 2023-09-27T23:52:12.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.065+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f39e414a120 con 0x7f39e40ca2c0 2023-09-27T23:52:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.073+0000 7f39da7fc700 1 -- 172.21.15.64:0/1181503514 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v61) v1 ==== 115+0+0 (secure 0 0 0) 0x7f39d4050810 con 0x7f39e40ca2c0 2023-09-27T23:52:12.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f39c4060a70 msgr2=0x7f39c4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f39c4060a70 0x7f39c4062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0099b0 tx=0x7f39cc017000 comp rx=0 tx=0).stop 2023-09-27T23:52:12.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 msgr2=0x7f39e406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 0x7f39e406c7c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f39d400e930 tx=0x7f39d40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 shutdown_connections 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f39c4060a70 0x7f39c4062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39e4149ca0 0x7f39e4150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e4158e20 0x7f39e406cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 --2- 172.21.15.64:0/1181503514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e40ca2c0 0x7f39e406c7c0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 >> 172.21.15.64:0/1181503514 conn(0x7f39e4067bb0 msgr2=0x7f39e40c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 shutdown_connections 2023-09-27T23:52:12.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.077+0000 7f39e8d9b700 1 -- 172.21.15.64:0/1181503514 wait complete. 2023-09-27T23:52:12.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:52:12.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 Processor -- start 2023-09-27T23:52:12.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 -- start start 2023-09-27T23:52:12.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac1514a0 0x7fd1ac157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd1ac06b7a0 0x7fd1ac158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0cbee0 con 0x7fd1ac0cae60 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac158530 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b4522700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cc370 con 0x7fd1ac1514a0 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b1abd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd1ac0cae60 0x7fd1ac06b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:34088/0 (socket says 172.21.15.64:34088) 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b1abd700 1 -- 172.21.15.64:0/2875451204 learned_addr learned my addr 172.21.15.64:0/2875451204 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b2abf700 1 --2- 172.21.15.64:0/2875451204 >> v2:172.21.15.123:3300/0 conn(0x7fd1ac06b7a0 0x7fd1ac158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b22be700 1 --2- 172.21.15.64:0/2875451204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac1514a0 0x7fd1ac157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b12bc700 1 -- 172.21.15.64:0/2875451204 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407045047 0 0) 0x7fd1ac0cbee0 con 0x7fd1ac0cae60 2023-09-27T23:52:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b12bc700 1 -- 172.21.15.64:0/2875451204 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd198003570 con 0x7fd1ac0cae60 2023-09-27T23:52:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b22be700 1 -- 172.21.15.64:0/2875451204 >> v2:172.21.15.123:3300/0 conn(0x7fd1ac06b7a0 msgr2=0x7fd1ac158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b22be700 1 --2- 172.21.15.64:0/2875451204 >> v2:172.21.15.123:3300/0 conn(0x7fd1ac06b7a0 0x7fd1ac158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b22be700 1 -- 172.21.15.64:0/2875451204 >> v1:172.21.15.64:6789/0 conn(0x7fd1ac0cae60 legacy=0x7fd1ac06b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b12bc700 1 -- 172.21.15.64:0/2875451204 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2202167114 0 0) 0x7fd198003570 con 0x7fd1ac0cae60 2023-09-27T23:52:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b22be700 1 -- 172.21.15.64:0/2875451204 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0cb2d0 con 0x7fd1ac1514a0 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b2abf700 1 --2- 172.21.15.64:0/2875451204 >> v2:172.21.15.123:3300/0 conn(0x7fd1ac06b7a0 0x7fd1ac158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.205+0000 7fd1b22be700 1 --2- 172.21.15.64:0/2875451204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac1514a0 0x7fd1ac157ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098e0 tx=0x7fd1a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd85b65b751b8e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b12bc700 1 -- 172.21.15.64:0/2875451204 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c001470 con 0x7fd1ac1514a0 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b12bc700 1 -- 172.21.15.64:0/2875451204 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a8011420 con 0x7fd1ac1514a0 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b12bc700 1 -- 172.21.15.64:0/2875451204 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80117c0 con 0x7fd1ac1514a0 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 -- 172.21.15.64:0/2875451204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac1514a0 msgr2=0x7fd1ac157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 --2- 172.21.15.64:0/2875451204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac1514a0 0x7fd1ac157ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098e0 tx=0x7fd1a80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 -- 172.21.15.64:0/2875451204 shutdown_connections 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 --2- 172.21.15.64:0/2875451204 >> v2:172.21.15.123:3300/0 conn(0x7fd1ac06b7a0 0x7fd1ac158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 --2- 172.21.15.64:0/2875451204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac1514a0 0x7fd1ac157ff0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 -- 172.21.15.64:0/2875451204 >> 172.21.15.64:0/2875451204 conn(0x7fd1ac0cc580 msgr2=0x7fd1ac0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 -- 172.21.15.64:0/2875451204 shutdown_connections 2023-09-27T23:52:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 -- 172.21.15.64:0/2875451204 wait complete. 2023-09-27T23:52:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 Processor -- start 2023-09-27T23:52:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.209+0000 7fd1b4522700 1 -- start start 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 0x7fd1ac13e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac0cae60 0x7fd1ac13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac1514a0 0x7fd1ac158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15aa60 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac13bcc0 con 0x7fd1ac1514a0 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac159de0 con 0x7fd1ac0cae60 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 0x7fd1ac13e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 0x7fd1ac13e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49448/0 (socket says 172.21.15.64:49448) 2023-09-27T23:52:12.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 -- 172.21.15.64:0/258635172 learned_addr learned my addr 172.21.15.64:0/258635172 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b1abd700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac0cae60 0x7fd1ac13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b2abf700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac1514a0 0x7fd1ac158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 -- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac0cae60 msgr2=0x7fd1ac13e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac0cae60 0x7fd1ac13e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 -- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac1514a0 msgr2=0x7fd1ac158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac1514a0 0x7fd1ac158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 -- 172.21.15.64:0/258635172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a4009a20 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b22be700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 0x7fd1ac13e020 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0cc4b0 tx=0x7fd1a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a801cc50 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a801cdb0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a8009420 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a801a6a0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.213+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac15a3a0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.217+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac15aba0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.217+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd1a801aad0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.217+0000 7fd1a27fc700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd190060a70 0x7fd190062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.217+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd1a807ea50 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.217+0000 7fd1b1abd700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd190060a70 0x7fd190062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.217+0000 7fd1b1abd700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd190060a70 0x7fd190062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0ab8b0 tx=0x7fd19c001520 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.225+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd1ac15aba0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.385+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd1ac071120 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.397+0000 7fd1a27fc700 1 -- 172.21.15.64:0/258635172 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v62) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd1a80290b0 con 0x7fd1ac06b7a0 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd190060a70 msgr2=0x7fd190062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd190060a70 0x7fd190062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0ab8b0 tx=0x7fd19c001520 comp rx=0 tx=0).stop 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 msgr2=0x7fd1ac13e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 0x7fd1ac13e020 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0cc4b0 tx=0x7fd1a8016000 comp rx=0 tx=0).stop 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 shutdown_connections 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd190060a70 0x7fd190062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac1514a0 0x7fd1ac158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac0cae60 0x7fd1ac13e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 --2- 172.21.15.64:0/258635172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac06b7a0 0x7fd1ac13e020 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 >> 172.21.15.64:0/258635172 conn(0x7fd1ac0cc580 msgr2=0x7fd1ac0abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:12.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 shutdown_connections 2023-09-27T23:52:12.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.401+0000 7fd1b4522700 1 -- 172.21.15.64:0/258635172 wait complete. 2023-09-27T23:52:12.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:52:12.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 Processor -- start 2023-09-27T23:52:12.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 -- start start 2023-09-27T23:52:12.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f642c0ce2d0 0x7f642c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c0cbbb0 con 0x7f642c0cc040 2023-09-27T23:52:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c15d960 con 0x7f642c0ce2d0 2023-09-27T23:52:12.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6432544700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c06dc90 con 0x7f642c0caa10 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6431542700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f642c0ce2d0 0x7f642c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c15d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:42370/0 (socket says 172.21.15.64:42370) 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 -- 172.21.15.64:0/713779924 learned_addr learned my addr 172.21.15.64:0/713779924 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 -- 172.21.15.64:0/713779924 >> v2:172.21.15.123:3300/0 conn(0x7f642c0ce2d0 msgr2=0x7f642c0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 --2- 172.21.15.64:0/713779924 >> v2:172.21.15.123:3300/0 conn(0x7f642c0ce2d0 0x7f642c0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 -- 172.21.15.64:0/713779924 >> v1:172.21.15.123:6789/0 conn(0x7f642c0cc040 legacy=0x7f642c158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 -- 172.21.15.64:0/713779924 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0cafa0 con 0x7f642c0caa10 2023-09-27T23:52:12.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6430d41700 1 --2- 172.21.15.64:0/713779924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c15d420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f641c0099b0 tx=0x7f641c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4e22ce086930bd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6423fff700 1 -- 172.21.15.64:0/713779924 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c009ea0 con 0x7f642c0caa10 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6423fff700 1 -- 172.21.15.64:0/713779924 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c00b860 con 0x7f642c0caa10 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.529+0000 7f6423fff700 1 -- 172.21.15.64:0/713779924 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c00bc00 con 0x7f642c0caa10 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- 172.21.15.64:0/713779924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 msgr2=0x7f642c15d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 --2- 172.21.15.64:0/713779924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c15d420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f641c0099b0 tx=0x7f641c009250 comp rx=0 tx=0).stop 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- 172.21.15.64:0/713779924 shutdown_connections 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 --2- 172.21.15.64:0/713779924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c15d420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 --2- 172.21.15.64:0/713779924 >> v2:172.21.15.123:3300/0 conn(0x7f642c0ce2d0 0x7f642c0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- 172.21.15.64:0/713779924 >> 172.21.15.64:0/713779924 conn(0x7f642c067bb0 msgr2=0x7f642c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- 172.21.15.64:0/713779924 shutdown_connections 2023-09-27T23:52:12.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- 172.21.15.64:0/713779924 wait complete. 2023-09-27T23:52:12.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 Processor -- start 2023-09-27T23:52:12.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- start start 2023-09-27T23:52:12.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c1468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 0x7f642c16d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f642c0ce2d0 0x7f642c1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0c9980 con 0x7f642c0cc040 2023-09-27T23:52:12.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c146e20 con 0x7f642c0ce2d0 2023-09-27T23:52:12.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.533+0000 7f6432544700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c16d9f0 con 0x7f642c0caa10 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 0x7f642c16d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 0x7f642c16d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49458/0 (socket says 172.21.15.64:49458) 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 -- 172.21.15.64:0/1920522782 learned_addr learned my addr 172.21.15.64:0/1920522782 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6431d43700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f642c0ce2d0 0x7f642c1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6431542700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c1468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 -- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 msgr2=0x7f642c1468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c1468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 -- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f642c0ce2d0 msgr2=0x7f642c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f642c0ce2d0 0x7f642c1562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 -- 172.21.15.64:0/1920522782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c0094f0 con 0x7f642c0cc040 2023-09-27T23:52:12.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6430d41700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 0x7f642c16d4b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f641c006fd0 tx=0x7f641c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c00bbe0 con 0x7f642c0cc040 2023-09-27T23:52:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c000c90 con 0x7f642c0cc040 2023-09-27T23:52:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c0c93c0 con 0x7f642c0cc040 2023-09-27T23:52:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c011650 con 0x7f642c0cc040 2023-09-27T23:52:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c158870 con 0x7f642c0cc040 2023-09-27T23:52:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c1586b0 con 0x7f642c0cc040 2023-09-27T23:52:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.537+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f641c011a40 con 0x7f642c0cc040 2023-09-27T23:52:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.541+0000 7f64217fa700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6410060a70 0x7f6410062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.541+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f641c07e510 con 0x7f642c0cc040 2023-09-27T23:52:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.541+0000 7f6431542700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6410060a70 0x7f6410062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.541+0000 7f6431542700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6410060a70 0x7f6410062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6428009d10 tx=0x7f64280091a0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.549+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f641c04e5f0 con 0x7f642c0cc040 2023-09-27T23:52:12.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.713+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f642c04be40 con 0x7f642c0cc040 2023-09-27T23:52:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.725+0000 7f64217fa700 1 -- 172.21.15.64:0/1920522782 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f641c051c10 con 0x7f642c0cc040 2023-09-27T23:52:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6410060a70 msgr2=0x7f6410062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6410060a70 0x7f6410062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6428009d10 tx=0x7f64280091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 msgr2=0x7f642c16d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 0x7f642c16d4b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f641c006fd0 tx=0x7f641c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 shutdown_connections 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6410060a70 0x7f6410062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f642c0ce2d0 0x7f642c1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642c0cc040 0x7f642c16d4b0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 --2- 172.21.15.64:0/1920522782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f642c0caa10 0x7f642c1468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 >> 172.21.15.64:0/1920522782 conn(0x7f642c067bb0 msgr2=0x7f642c0c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 shutdown_connections 2023-09-27T23:52:12.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.729+0000 7f6432544700 1 -- 172.21.15.64:0/1920522782 wait complete. 2023-09-27T23:52:12.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-09-27T23:52:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 Processor -- start 2023-09-27T23:52:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 -- start start 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c406b990 0x7f37c40c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37c415a5e0 0x7f37c40c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40cb030 con 0x7f37c415a210 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c41599a0 con 0x7f37c415a5e0 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c9b87700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40cb290 con 0x7f37c406b990 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37c415a5e0 0x7f37c40c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f37c415a5e0 0x7f37c40c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49474/0 (socket says 172.21.15.64:49474) 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c3fff700 1 -- 172.21.15.64:0/3938351492 learned_addr learned my addr 172.21.15.64:0/3938351492 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c2ffd700 1 --2- 172.21.15.64:0/3938351492 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c406b990 0x7f37c40c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c3fff700 1 -- 172.21.15.64:0/3938351492 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c406b990 msgr2=0x7f37c40c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.845+0000 7f37c3fff700 1 --2- 172.21.15.64:0/3938351492 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c406b990 0x7f37c40c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c3fff700 1 -- 172.21.15.64:0/3938351492 >> v1:172.21.15.123:6789/0 conn(0x7f37c415a210 legacy=0x7f37c406b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c3fff700 1 -- 172.21.15.64:0/3938351492 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40cb570 con 0x7f37c415a5e0 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c3fff700 1 --2- 172.21.15.64:0/3938351492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a5e0 0x7f37c40c95f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f20 tx=0x7f37bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c8f5c0ebdc3998 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c27fc700 1 -- 172.21.15.64:0/3938351492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00e070 con 0x7f37c415a5e0 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c27fc700 1 -- 172.21.15.64:0/3938351492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00ba80 con 0x7f37c415a5e0 2023-09-27T23:52:12.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c27fc700 1 -- 172.21.15.64:0/3938351492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0092e0 con 0x7f37c415a5e0 2023-09-27T23:52:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/3938351492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a5e0 msgr2=0x7f37c40c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/3938351492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a5e0 0x7f37c40c95f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f20 tx=0x7f37bc009840 comp rx=0 tx=0).stop 2023-09-27T23:52:12.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/3938351492 shutdown_connections 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/3938351492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a5e0 0x7f37c40c95f0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/3938351492 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c406b990 0x7f37c40c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/3938351492 >> 172.21.15.64:0/3938351492 conn(0x7f37c4067410 msgr2=0x7f37c40677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/3938351492 shutdown_connections 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/3938351492 wait complete. 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 Processor -- start 2023-09-27T23:52:12.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- start start 2023-09-27T23:52:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c406b990 0x7f37c415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 0x7f37c40b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c415a5e0 0x7f37c414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4152c20 con 0x7f37c415a210 2023-09-27T23:52:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4152f00 con 0x7f37c406b990 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c9b87700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41531e0 con 0x7f37c415a5e0 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 0x7f37c40b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 0x7f37c40b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49490/0 (socket says 172.21.15.64:49490) 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 -- 172.21.15.64:0/1494369443 learned_addr learned my addr 172.21.15.64:0/1494369443 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c37fe700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c406b990 0x7f37c415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c3fff700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c415a5e0 0x7f37c414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 -- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c415a5e0 msgr2=0x7f37c414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c415a5e0 0x7f37c414b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 -- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c406b990 msgr2=0x7f37c415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c406b990 0x7f37c415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c2ffd700 1 -- 172.21.15.64:0/1494369443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc009ae0 con 0x7f37c415a210 2023-09-27T23:52:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.849+0000 7f37c37fe700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c406b990 0x7f37c415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37c2ffd700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 0x7f37c40b0fa0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d890 tx=0x7f37b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800b6c0 con 0x7f37c415a210 2023-09-27T23:52:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800b840 con 0x7f37c415a210 2023-09-27T23:52:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c41534c0 con 0x7f37c415a210 2023-09-27T23:52:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800bc00 con 0x7f37c415a210 2023-09-27T23:52:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c4152840 con 0x7f37c415a210 2023-09-27T23:52:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f37b801e460 con 0x7f37c415a210 2023-09-27T23:52:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c414d970 con 0x7f37c415a210 2023-09-27T23:52:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37ab7fe700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f37b0060a70 0x7f37b0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f37b807d8b0 con 0x7f37c415a210 2023-09-27T23:52:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37c37fe700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f37b0060a70 0x7f37b0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.853+0000 7f37c37fe700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f37b0060a70 0x7f37b0062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f37b4007fd0 tx=0x7f37b4009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:12.857+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f37c414d970 con 0x7f37c415a210 2023-09-27T23:52:13.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.073+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f37c414d970 con 0x7f37c415a210 2023-09-27T23:52:13.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.845+0000 7f37ab7fe700 1 -- 172.21.15.64:0/1494369443 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v1352) v1 ==== 163+0+0 (secure 0 0 0) 0x7f37c414d970 con 0x7f37c415a210 2023-09-27T23:52:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f37b0060a70 msgr2=0x7f37b0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f37b0060a70 0x7f37b0062f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f37b4007fd0 tx=0x7f37b4009040 comp rx=0 tx=0).stop 2023-09-27T23:52:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 msgr2=0x7f37c40b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 0x7f37c40b0fa0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d890 tx=0x7f37b8008040 comp rx=0 tx=0).stop 2023-09-27T23:52:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 shutdown_connections 2023-09-27T23:52:13.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f37b0060a70 0x7f37b0062f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37c415a5e0 0x7f37c414b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37c415a210 0x7f37c40b0fa0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 --2- 172.21.15.64:0/1494369443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c406b990 0x7f37c415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 >> 172.21.15.64:0/1494369443 conn(0x7f37c4067410 msgr2=0x7f37c4156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 shutdown_connections 2023-09-27T23:52:13.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.849+0000 7f37c9b87700 1 -- 172.21.15.64:0/1494369443 wait complete. 2023-09-27T23:52:13.873 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-09-27T23:52:13.873 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-09-27T23:52:13.874 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-09-27T23:52:13.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-09-27T23:52:13.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.985+0000 7f528adc1700 1 Processor -- start 2023-09-27T23:52:13.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.985+0000 7f528adc1700 1 -- start start 2023-09-27T23:52:13.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f5284159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52840670e0 0x7f52840abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52840c1910 con 0x7f52840cd430 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52840ac220 con 0x7f52840670e0 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5284159cf0 con 0x7f52840ab910 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f5284159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f5284159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:42400/0 (socket says 172.21.15.64:42400) 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 -- 172.21.15.64:0/1481403473 learned_addr learned my addr 172.21.15.64:0/1481403473 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:13.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f52895be700 1 --2- 172.21.15.64:0/1481403473 >> v2:172.21.15.123:3300/0 conn(0x7f52840670e0 0x7f52840abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 -- 172.21.15.64:0/1481403473 >> v2:172.21.15.123:3300/0 conn(0x7f52840670e0 msgr2=0x7f52840abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:13.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 --2- 172.21.15.64:0/1481403473 >> v2:172.21.15.123:3300/0 conn(0x7f52840670e0 0x7f52840abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 -- 172.21.15.64:0/1481403473 >> v1:172.21.15.123:6789/0 conn(0x7f52840cd430 legacy=0x7f52840c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 -- 172.21.15.64:0/1481403473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284159fd0 con 0x7f52840ab910 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5288dbd700 1 -- 172.21.15.64:0/1481403473 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959183733 0 0) 0x7f52840c1910 con 0x7f52840cd430 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5289dbf700 1 --2- 172.21.15.64:0/1481403473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f5284159710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5280009f20 tx=0x7f5280009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537e1e5eac007f0e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5288dbd700 1 -- 172.21.15.64:0/1481403473 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528000e070 con 0x7f52840ab910 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5288dbd700 1 -- 172.21.15.64:0/1481403473 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528000b9d0 con 0x7f52840ab910 2023-09-27T23:52:13.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f5288dbd700 1 -- 172.21.15.64:0/1481403473 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800092e0 con 0x7f52840ab910 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 -- 172.21.15.64:0/1481403473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 msgr2=0x7f5284159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 --2- 172.21.15.64:0/1481403473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f5284159710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5280009f20 tx=0x7f5280009840 comp rx=0 tx=0).stop 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 -- 172.21.15.64:0/1481403473 shutdown_connections 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 --2- 172.21.15.64:0/1481403473 >> v2:172.21.15.123:3300/0 conn(0x7f52840670e0 0x7f52840abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 --2- 172.21.15.64:0/1481403473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f5284159710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.989+0000 7f528adc1700 1 -- 172.21.15.64:0/1481403473 >> 172.21.15.64:0/1481403473 conn(0x7f528406c080 msgr2=0x7f52840ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 -- 172.21.15.64:0/1481403473 shutdown_connections 2023-09-27T23:52:13.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 -- 172.21.15.64:0/1481403473 wait complete. 2023-09-27T23:52:13.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 Processor -- start 2023-09-27T23:52:13.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 -- start start 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52840670e0 0x7f528416cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f52840aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 0x7f5284147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528416d330 con 0x7f52840cd430 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52840ab290 con 0x7f52840670e0 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.993+0000 7f528adc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5284169880 con 0x7f52840ab910 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 0x7f5284147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f5289dbf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52840670e0 0x7f528416cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:14.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 0x7f5284147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49502/0 (socket says 172.21.15.64:49502) 2023-09-27T23:52:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 -- 172.21.15.64:0/2015481475 learned_addr learned my addr 172.21.15.64:0/2015481475 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:14.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f52895be700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f52840aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 -- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 msgr2=0x7f52840aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f52840aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 -- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52840670e0 msgr2=0x7f528416cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52840670e0 0x7f528416cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 -- 172.21.15.64:0/2015481475 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5280009ae0 con 0x7f52840cd430 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528a5c0700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 0x7f5284147500 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c600 tx=0x7f527c007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c011590 con 0x7f52840cd430 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c0116f0 con 0x7f52840cd430 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5284169d10 con 0x7f52840cd430 2023-09-27T23:52:14.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c011ab0 con 0x7f52840cd430 2023-09-27T23:52:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5284168f50 con 0x7f52840cd430 2023-09-27T23:52:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:13.997+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528404be40 con 0x7f52840cd430 2023-09-27T23:52:14.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.001+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f527c01b3b0 con 0x7f52840cd430 2023-09-27T23:52:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.001+0000 7f5279ffb700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5268060a70 0x7f5268062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:14.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.001+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f527c07c590 con 0x7f52840cd430 2023-09-27T23:52:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.001+0000 7f5289dbf700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5268060a70 0x7f5268062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:14.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.001+0000 7f5289dbf700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5268060a70 0x7f5268062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5280009880 tx=0x7f5280018000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:14.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.009+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f527c04c5f0 con 0x7f52840cd430 2023-09-27T23:52:14.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.233+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f5284068cc0 con 0x7f52840cd430 2023-09-27T23:52:14.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.849+0000 7f5279ffb700 1 -- 172.21.15.64:0/2015481475 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v1353) v1 ==== 158+0+0 (secure 0 0 0) 0x7f527c0170e0 con 0x7f52840cd430 2023-09-27T23:52:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5268060a70 msgr2=0x7f5268062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5268060a70 0x7f5268062f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5280009880 tx=0x7f5280018000 comp rx=0 tx=0).stop 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 msgr2=0x7f5284147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 0x7f5284147500 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c600 tx=0x7f527c007d70 comp rx=0 tx=0).stop 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 shutdown_connections 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5268060a70 0x7f5268062f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52840cd430 0x7f5284147500 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52840ab910 0x7f52840aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 --2- 172.21.15.64:0/2015481475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52840670e0 0x7f528416cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 >> 172.21.15.64:0/2015481475 conn(0x7f528406c080 msgr2=0x7f52840c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 shutdown_connections 2023-09-27T23:52:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.853+0000 7f528adc1700 1 -- 172.21.15.64:0/2015481475 wait complete. 2023-09-27T23:52:14.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-27T23:52:14.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.985+0000 7f15ac9fd700 1 Processor -- start 2023-09-27T23:52:14.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.985+0000 7f15ac9fd700 1 -- start start 2023-09-27T23:52:14.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a8158fd0 0x7f15a806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:14.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f15a80cc1e0 0x7f15a80c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:14.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a806d750 con 0x7f15a80cc610 2023-09-27T23:52:14.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80cbaf0 con 0x7f15a8158fd0 2023-09-27T23:52:14.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a806e120 con 0x7f15a80cc1e0 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a6d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15a8158fd0 0x7f15a806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a759e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f15a80cc610 0x7f15a80c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55838/0 (socket says 172.21.15.64:55838) 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a759e700 1 -- 172.21.15.64:0/4174471192 learned_addr learned my addr 172.21.15.64:0/4174471192 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a659c700 1 --2- 172.21.15.64:0/4174471192 >> v2:172.21.15.123:3300/0 conn(0x7f15a80cc1e0 0x7f15a80c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a5d9b700 1 -- 172.21.15.64:0/4174471192 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509793813 0 0) 0x7f15a806d750 con 0x7f15a80cc610 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a5d9b700 1 -- 172.21.15.64:0/4174471192 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1590003570 con 0x7f15a80cc610 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a6d9d700 1 -- 172.21.15.64:0/4174471192 >> v2:172.21.15.123:3300/0 conn(0x7f15a80cc1e0 msgr2=0x7f15a80c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a6d9d700 1 --2- 172.21.15.64:0/4174471192 >> v2:172.21.15.123:3300/0 conn(0x7f15a80cc1e0 0x7f15a80c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a6d9d700 1 -- 172.21.15.64:0/4174471192 >> v1:172.21.15.64:6789/0 conn(0x7f15a80cc610 legacy=0x7f15a80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:14.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a6d9d700 1 -- 172.21.15.64:0/4174471192 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a806cb40 con 0x7f15a8158fd0 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a6d9d700 1 --2- 172.21.15.64:0/4174471192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a806dbe0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f15a806b390 tx=0x7f15980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a513363a8c770be server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a5d9b700 1 -- 172.21.15.64:0/4174471192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011630 con 0x7f15a8158fd0 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a5d9b700 1 -- 172.21.15.64:0/4174471192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15980117b0 con 0x7f15a8158fd0 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15a5d9b700 1 -- 172.21.15.64:0/4174471192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011b70 con 0x7f15a8158fd0 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- 172.21.15.64:0/4174471192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 msgr2=0x7f15a806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/4174471192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a806dbe0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f15a806b390 tx=0x7f15980091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- 172.21.15.64:0/4174471192 shutdown_connections 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/4174471192 >> v2:172.21.15.123:3300/0 conn(0x7f15a80cc1e0 0x7f15a80c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/4174471192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a806dbe0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- 172.21.15.64:0/4174471192 >> 172.21.15.64:0/4174471192 conn(0x7f15a80cdc00 msgr2=0x7f15a80ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- 172.21.15.64:0/4174471192 shutdown_connections 2023-09-27T23:52:14.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.989+0000 7f15ac9fd700 1 -- 172.21.15.64:0/4174471192 wait complete. 2023-09-27T23:52:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.993+0000 7f15ac9fd700 1 Processor -- start 2023-09-27T23:52:14.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.993+0000 7f15ac9fd700 1 -- start start 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15a80cc1e0 0x7f15a815da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15a80cc610 0x7f15a80cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a815df90 con 0x7f15a8158fd0 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a815c9b0 con 0x7f15a80cc1e0 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a815cc90 con 0x7f15a80cc610 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37550/0 (socket says 172.21.15.64:37550) 2023-09-27T23:52:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 -- 172.21.15.64:0/1628283194 learned_addr learned my addr 172.21.15.64:0/1628283194 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a659c700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15a80cc610 0x7f15a80cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a6d9d700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15a80cc1e0 0x7f15a815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 -- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15a80cc610 msgr2=0x7f15a80cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15a80cc610 0x7f15a80cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 -- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15a80cc1e0 msgr2=0x7f15a815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15a80cc1e0 0x7f15a815da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 -- 172.21.15.64:0/1628283194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009420 con 0x7f15a8158fd0 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a659c700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15a80cc610 0x7f15a80cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a6d9d700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15a80cc1e0 0x7f15a815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15a759e700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a8156010 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f15a0001460 tx=0x7f15a0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000ce80 con 0x7f15a8158fd0 2023-09-27T23:52:15.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000ab70 con 0x7f15a8158fd0 2023-09-27T23:52:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a815cf70 con 0x7f15a8158fd0 2023-09-27T23:52:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00070f0 con 0x7f15a8158fd0 2023-09-27T23:52:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a815d530 con 0x7f15a8158fd0 2023-09-27T23:52:15.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:14.997+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a804be40 con 0x7f15a8158fd0 2023-09-27T23:52:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.001+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f15a0007520 con 0x7f15a8158fd0 2023-09-27T23:52:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.001+0000 7f158effd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1594060aa0 0x7f1594062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.001+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f15a007d520 con 0x7f15a8158fd0 2023-09-27T23:52:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.001+0000 7f15a6d9d700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1594060aa0 0x7f1594062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.001+0000 7f15a6d9d700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1594060aa0 0x7f1594062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f15a806b390 tx=0x7f1598017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.009+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f15a004d600 con 0x7f15a8158fd0 2023-09-27T23:52:15.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.169+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f15a806ce20 con 0x7f15a8158fd0 2023-09-27T23:52:15.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f158effd700 1 -- 172.21.15.64:0/1628283194 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f15a0050c20 con 0x7f15a8158fd0 2023-09-27T23:52:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1594060aa0 msgr2=0x7f1594062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1594060aa0 0x7f1594062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f15a806b390 tx=0x7f1598017000 comp rx=0 tx=0).stop 2023-09-27T23:52:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 msgr2=0x7f15a8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a8156010 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f15a0001460 tx=0x7f15a0001ae0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 shutdown_connections 2023-09-27T23:52:15.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1594060aa0 0x7f1594062f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15a8158fd0 0x7f15a8156010 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f15a80cc610 0x7f15a80cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 --2- 172.21.15.64:0/1628283194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15a80cc1e0 0x7f15a815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 >> 172.21.15.64:0/1628283194 conn(0x7f15a80cdc00 msgr2=0x7f15a8157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:15.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 shutdown_connections 2023-09-27T23:52:15.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.177+0000 7f15ac9fd700 1 -- 172.21.15.64:0/1628283194 wait complete. 2023-09-27T23:52:15.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:52:15.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:52:15.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:52:15.235 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:52:15.235 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-27T23:52:15.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 Processor -- start 2023-09-27T23:52:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 -- start start 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb8c06ad70 0x7feb8c06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c14f4d0 0x7feb8c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8c0afad0 con 0x7feb8c06e1d0 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c06b750 con 0x7feb8c06ad70 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91966700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0afdb0 con 0x7feb8c14f4d0 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb90964700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7feb8c06e1d0 0x7feb8c159d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55844/0 (socket says 172.21.15.64:55844) 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb90964700 1 -- 172.21.15.64:0/710797333 learned_addr learned my addr 172.21.15.64:0/710797333 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:15.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8bfff700 1 --2- 172.21.15.64:0/710797333 >> v2:172.21.15.64:3300/0 conn(0x7feb8c06ad70 0x7feb8c06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb91165700 1 --2- 172.21.15.64:0/710797333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c14f4d0 0x7feb8c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8b7fe700 1 -- 172.21.15.64:0/710797333 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093769304 0 0) 0x7feb8c0afad0 con 0x7feb8c06e1d0 2023-09-27T23:52:15.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8b7fe700 1 -- 172.21.15.64:0/710797333 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb7c003570 con 0x7feb8c06e1d0 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8bfff700 1 -- 172.21.15.64:0/710797333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c14f4d0 msgr2=0x7feb8c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8bfff700 1 --2- 172.21.15.64:0/710797333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c14f4d0 0x7feb8c14f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8bfff700 1 -- 172.21.15.64:0/710797333 >> v1:172.21.15.64:6789/0 conn(0x7feb8c06e1d0 legacy=0x7feb8c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8bfff700 1 -- 172.21.15.64:0/710797333 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c0b0090 con 0x7feb8c06ad70 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8bfff700 1 --2- 172.21.15.64:0/710797333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06ad70 0x7feb8c06b210 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7feb780098e0 tx=0x7feb780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ffcebbdabdb386 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8b7fe700 1 -- 172.21.15.64:0/710797333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78011630 con 0x7feb8c06ad70 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8b7fe700 1 -- 172.21.15.64:0/710797333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb78011790 con 0x7feb8c06ad70 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.341+0000 7feb8b7fe700 1 -- 172.21.15.64:0/710797333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78011b10 con 0x7feb8c06ad70 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 -- 172.21.15.64:0/710797333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06ad70 msgr2=0x7feb8c06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 --2- 172.21.15.64:0/710797333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06ad70 0x7feb8c06b210 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7feb780098e0 tx=0x7feb780091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 -- 172.21.15.64:0/710797333 shutdown_connections 2023-09-27T23:52:15.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 --2- 172.21.15.64:0/710797333 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c14f4d0 0x7feb8c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 --2- 172.21.15.64:0/710797333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06ad70 0x7feb8c06b210 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 -- 172.21.15.64:0/710797333 >> 172.21.15.64:0/710797333 conn(0x7feb8c067bb0 msgr2=0x7feb8c0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 -- 172.21.15.64:0/710797333 shutdown_connections 2023-09-27T23:52:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 -- 172.21.15.64:0/710797333 wait complete. 2023-09-27T23:52:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 Processor -- start 2023-09-27T23:52:15.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.345+0000 7feb91966700 1 -- start start 2023-09-27T23:52:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c06ad70 0x7feb8c154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 0x7feb8c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb8c14f4d0 0x7feb8c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1525a0 con 0x7feb8c06e1d0 2023-09-27T23:52:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c13acd0 con 0x7feb8c14f4d0 2023-09-27T23:52:15.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c151990 con 0x7feb8c06ad70 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 0x7feb8c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 0x7feb8c150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37554/0 (socket says 172.21.15.64:37554) 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 -- 172.21.15.64:0/560025304 learned_addr learned my addr 172.21.15.64:0/560025304 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb90964700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c06ad70 0x7feb8c154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91165700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb8c14f4d0 0x7feb8c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 -- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c06ad70 msgr2=0x7feb8c154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c06ad70 0x7feb8c154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 -- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb8c14f4d0 msgr2=0x7feb8c152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb8c14f4d0 0x7feb8c152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 -- 172.21.15.64:0/560025304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb78009420 con 0x7feb8c06e1d0 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91165700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb8c14f4d0 0x7feb8c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:15.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb8bfff700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 0x7feb8c150f90 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7feb8c069fb0 tx=0x7feb78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78011900 con 0x7feb8c06e1d0 2023-09-27T23:52:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb78011a80 con 0x7feb8c06e1d0 2023-09-27T23:52:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c151c70 con 0x7feb8c06e1d0 2023-09-27T23:52:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7801a680 con 0x7feb8c06e1d0 2023-09-27T23:52:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c152230 con 0x7feb8c06e1d0 2023-09-27T23:52:15.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c04be40 con 0x7feb8c06e1d0 2023-09-27T23:52:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.349+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7feb7801aa70 con 0x7feb8c06e1d0 2023-09-27T23:52:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.353+0000 7feb88ff9700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7feb70060aa0 0x7feb70062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.353+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feb7807ec80 con 0x7feb8c06e1d0 2023-09-27T23:52:15.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.353+0000 7feb90964700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7feb70060aa0 0x7feb70062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.353+0000 7feb90964700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7feb70060aa0 0x7feb70062f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb80007fd0 tx=0x7feb80009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.361+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7feb7804ece0 con 0x7feb8c06e1d0 2023-09-27T23:52:15.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.557+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb8c06bb80 con 0x7feb8c06e1d0 2023-09-27T23:52:15.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.557+0000 7feb88ff9700 1 -- 172.21.15.64:0/560025304 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3533 (secure 0 0 0) 0x7feb78015020 con 0x7feb8c06e1d0 2023-09-27T23:52:15.562 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:15.562 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3549874384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3549874384},{"type":"v1","addr":"172.21.15.123:6836","nonce":3549874384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2119394931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2119394931},{"type":"v1","addr":"172.21.15.64:6833","nonce":2119394931}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":9777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1073145936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1073145936},{"type":"v1","addr":"172.21.15.123:6837","nonce":1073145936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/876586310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":876586310},{"type":"v1","addr":"172.21.15.64:6835","nonce":876586310}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[]} 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7feb70060aa0 msgr2=0x7feb70062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7feb70060aa0 0x7feb70062f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb80007fd0 tx=0x7feb80009040 comp rx=0 tx=0).stop 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 msgr2=0x7feb8c150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 0x7feb8c150f90 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7feb8c069fb0 tx=0x7feb78017000 comp rx=0 tx=0).stop 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 shutdown_connections 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7feb70060aa0 0x7feb70062f50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb8c14f4d0 0x7feb8c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb8c06e1d0 0x7feb8c150f90 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 --2- 172.21.15.64:0/560025304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb8c06ad70 0x7feb8c154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 >> 172.21.15.64:0/560025304 conn(0x7feb8c067bb0 msgr2=0x7feb8c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 shutdown_connections 2023-09-27T23:52:15.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.561+0000 7feb91966700 1 -- 172.21.15.64:0/560025304 wait complete. 2023-09-27T23:52:15.566 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 68 2023-09-27T23:52:15.577 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-27T23:52:15.577 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-27T23:52:15.577 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:52:15.577 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:52:15.577 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-27T23:52:15.577 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-27T23:52:15.578 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:52:15.578 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:52:15.578 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-27T23:52:15.578 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-27T23:52:15.578 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:52:15.578 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:52:15.578 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-27T23:52:15.579 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-27T23:52:15.579 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:52:15.579 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:52:15.588 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:52:15.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-27T23:52:15.591 INFO:tasks.ceph.mds.a:Started 2023-09-27T23:52:15.591 DEBUG:teuthology.parallel:result is None 2023-09-27T23:52:15.591 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:52:15.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-27T23:52:15.594 INFO:tasks.ceph.mds.c:Started 2023-09-27T23:52:15.594 DEBUG:teuthology.parallel:result is None 2023-09-27T23:52:15.595 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:52:15.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-27T23:52:15.596 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:52:15.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-27T23:52:15.597 INFO:tasks.ceph.mds.b:Started 2023-09-27T23:52:15.597 DEBUG:teuthology.parallel:result is None 2023-09-27T23:52:15.598 INFO:tasks.ceph.mds.d:Started 2023-09-27T23:52:15.599 DEBUG:teuthology.parallel:result is None 2023-09-27T23:52:15.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-27T23:52:15.657 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-09-27T23:52:15.666 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-09-27T23:52:15.674 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:52:15.675 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-09-27T23:52:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.693+0000 7efe7ce6b700 1 Processor -- start 2023-09-27T23:52:15.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.693+0000 7efe7ce6b700 1 -- start start 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7ce6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe78155b50 0x7efe7806b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7ce6b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 0x7efe780af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7ce6b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7806b3f0 con 0x7efe780b0610 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7ce6b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7806a7e0 con 0x7efe780af380 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7ce6b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7806aac0 con 0x7efe78155b50 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe75d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 0x7efe780af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe78155b50 0x7efe7806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe75d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 0x7efe780af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37588/0 (socket says 172.21.15.64:37588) 2023-09-27T23:52:15.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe76d9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efe780b0610 0x7efe780c9220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:44276/0 (socket says 172.21.15.64:44276) 2023-09-27T23:52:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe75d9b700 1 -- 172.21.15.64:0/2347824689 learned_addr learned my addr 172.21.15.64:0/2347824689 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7659c700 1 -- 172.21.15.64:0/2347824689 >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 msgr2=0x7efe780af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7659c700 1 --2- 172.21.15.64:0/2347824689 >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 0x7efe780af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7659c700 1 -- 172.21.15.64:0/2347824689 >> v1:172.21.15.123:6789/0 conn(0x7efe780b0610 legacy=0x7efe780c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7659c700 1 -- 172.21.15.64:0/2347824689 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7806ada0 con 0x7efe78155b50 2023-09-27T23:52:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe75d9b700 1 --2- 172.21.15.64:0/2347824689 >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 0x7efe780af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:15.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.697+0000 7efe7659c700 1 --2- 172.21.15.64:0/2347824689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe78155b50 0x7efe7806b880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe680098e0 tx=0x7efe680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e29b03f2d737a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.705+0000 7efe7559a700 1 -- 172.21.15.64:0/2347824689 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800b640 con 0x7efe78155b50 2023-09-27T23:52:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.705+0000 7efe7559a700 1 -- 172.21.15.64:0/2347824689 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6800b7a0 con 0x7efe78155b50 2023-09-27T23:52:15.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.705+0000 7efe7559a700 1 -- 172.21.15.64:0/2347824689 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800bb40 con 0x7efe78155b50 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 -- 172.21.15.64:0/2347824689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe78155b50 msgr2=0x7efe7806b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/2347824689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe78155b50 0x7efe7806b880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe680098e0 tx=0x7efe680091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 -- 172.21.15.64:0/2347824689 shutdown_connections 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/2347824689 >> v2:172.21.15.64:3300/0 conn(0x7efe780af380 0x7efe780af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/2347824689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe78155b50 0x7efe7806b880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 -- 172.21.15.64:0/2347824689 >> 172.21.15.64:0/2347824689 conn(0x7efe78067bb0 msgr2=0x7efe780ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 -- 172.21.15.64:0/2347824689 shutdown_connections 2023-09-27T23:52:15.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 -- 172.21.15.64:0/2347824689 wait complete. 2023-09-27T23:52:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.709+0000 7efe7ce6b700 1 Processor -- start 2023-09-27T23:52:15.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 -- start start 2023-09-27T23:52:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe780af380 0x7efe780c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe780b0610 0x7efe78156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 0x7efe780c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7814f730 con 0x7efe78155b50 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe780c1d90 con 0x7efe780b0610 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe780c3640 con 0x7efe780af380 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 0x7efe780c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe780b0610 0x7efe78156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe780af380 0x7efe780c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 0x7efe780c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37604/0 (socket says 172.21.15.64:37604) 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe780b0610 0x7efe78156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:35698/0 (socket says 172.21.15.64:35698) 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe780af380 0x7efe780c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:42958/0 (socket says 172.21.15.64:42958) 2023-09-27T23:52:15.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 -- 172.21.15.64:0/4214338065 learned_addr learned my addr 172.21.15.64:0/4214338065 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 -- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe780af380 msgr2=0x7efe780c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe780af380 0x7efe780c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 -- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe780b0610 msgr2=0x7efe78156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe780b0610 0x7efe78156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 -- 172.21.15.64:0/4214338065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c00b790 con 0x7efe78155b50 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe76d9d700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 0x7efe780c4400 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7efe7000cd00 tx=0x7efe70001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000ce60 con 0x7efe78155b50 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70005df0 con 0x7efe78155b50 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe68009420 con 0x7efe78155b50 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000ab00 con 0x7efe78155b50 2023-09-27T23:52:15.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.713+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe780c3c00 con 0x7efe78155b50 2023-09-27T23:52:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.717+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78071120 con 0x7efe78155b50 2023-09-27T23:52:15.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.717+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7efe700070f0 con 0x7efe78155b50 2023-09-27T23:52:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.717+0000 7efe667fc700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efe58060a70 0x7efe58062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.717+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6300+0+0 (secure 0 0 0) 0x7efe7007d220 con 0x7efe78155b50 2023-09-27T23:52:15.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.717+0000 7efe7659c700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efe58060a70 0x7efe58062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:15.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.717+0000 7efe7659c700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efe58060a70 0x7efe58062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe680093f0 tx=0x7efe68017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:15.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.725+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efe7004d200 con 0x7efe78155b50 2023-09-27T23:52:15.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-09-27T23:52:15.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-09-27T23:52:15.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:15.897+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efe7806b080 con 0x7efe78155b50 2023-09-27T23:52:16.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.860+0000 7efe667fc700 1 -- 172.21.15.64:0/4214338065 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v185) v1 ==== 97+0+0 (secure 0 0 0) 0x7efe70050820 con 0x7efe78155b50 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efe58060a70 msgr2=0x7efe58062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efe58060a70 0x7efe58062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe680093f0 tx=0x7efe68017000 comp rx=0 tx=0).stop 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 msgr2=0x7efe780c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 0x7efe780c4400 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7efe7000cd00 tx=0x7efe70001b10 comp rx=0 tx=0).stop 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 shutdown_connections 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efe58060a70 0x7efe58062f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe78155b50 0x7efe780c4400 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe780b0610 0x7efe78156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 --2- 172.21.15.64:0/4214338065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe780af380 0x7efe780c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 >> 172.21.15.64:0/4214338065 conn(0x7efe78067bb0 msgr2=0x7efe78152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 shutdown_connections 2023-09-27T23:52:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:16.868+0000 7efe7ce6b700 1 -- 172.21.15.64:0/4214338065 wait complete. 2023-09-27T23:52:16.874 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-09-27T23:52:16.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-27T23:52:17.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.000+0000 7f046cb47700 1 Processor -- start 2023-09-27T23:52:17.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.000+0000 7f046cb47700 1 -- start start 2023-09-27T23:52:17.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046cb47700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0468068800 0x7f0468161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046cb47700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046806e150 con 0x7f0468140ae0 2023-09-27T23:52:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046cb47700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046806d540 con 0x7f0468140710 2023-09-27T23:52:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046cb47700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046806d820 con 0x7f0468068800 2023-09-27T23:52:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0465d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0468140ae0 0x7f04680680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55870/0 (socket says 172.21.15.64:55870) 2023-09-27T23:52:17.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0465d9b700 1 -- 172.21.15.64:0/1265819691 learned_addr learned my addr 172.21.15.64:0/1265819691 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:17.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0466d9d700 1 --2- 172.21.15.64:0/1265819691 >> v2:172.21.15.123:3300/0 conn(0x7f0468068800 0x7f0468161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791429207 0 0) 0x7f046806e150 con 0x7f0468140ae0 2023-09-27T23:52:17.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0450003570 con 0x7f0468140ae0 2023-09-27T23:52:17.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939167905 0 0) 0x7f046806d540 con 0x7f0468140710 2023-09-27T23:52:17.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f046806e150 con 0x7f0468140710 2023-09-27T23:52:17.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1952763543 0 0) 0x7f0450003570 con 0x7f0468140ae0 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 >> v2:172.21.15.123:3300/0 conn(0x7f0468068800 msgr2=0x7f0468161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 --2- 172.21.15.64:0/1265819691 >> v2:172.21.15.123:3300/0 conn(0x7f0468068800 0x7f0468161cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 >> v1:172.21.15.123:6789/0 conn(0x7f0468140710 legacy=0x7f04680c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046806db00 con 0x7f0468140ae0 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f0458001470 con 0x7f0468140ae0 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 >> v1:172.21.15.64:6789/0 conn(0x7f0468140ae0 legacy=0x7f04680680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0450003c40 0x7f0450004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0450004810 0x7f0450004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450005210 0x7f045000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046806d820 con 0x7f0450003c40 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045000c130 con 0x7f0450004810 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045000c2c0 con 0x7f0450005210 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0450003c40 0x7f0450004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0466d9d700 1 --2- 172.21.15.64:0/1265819691 >> v2:172.21.15.123:3300/0 conn(0x7f0468068800 0x7f0468161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0465d9b700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0450004810 0x7f0450004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0466d9d700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450005210 0x7f045000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 -- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450005210 msgr2=0x7f045000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450005210 0x7f045000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 -- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0450004810 msgr2=0x7f0450004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:17.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0450004810 0x7f0450004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 -- 172.21.15.64:0/1265819691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450003570 con 0x7f0450003c40 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0465d9b700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0450004810 0x7f0450004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0466d9d700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450005210 0x7f045000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f0466d9d700 1 -- 172.21.15.64:0/1265819691 reap_dead start 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046659c700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0450003c40 0x7f0450004030 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f045c001a50 tx=0x7f045c007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b33dbd546a45c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c0013b0 con 0x7f0450003c40 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c001e90 con 0x7f0450003c40 2023-09-27T23:52:17.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.004+0000 7f046559a700 1 -- 172.21.15.64:0/1265819691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c00c7a0 con 0x7f0450003c40 2023-09-27T23:52:17.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 -- 172.21.15.64:0/1265819691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0450003c40 msgr2=0x7f0450004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0450003c40 0x7f0450004030 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f045c001a50 tx=0x7f045c007cf0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 -- 172.21.15.64:0/1265819691 shutdown_connections 2023-09-27T23:52:17.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 --2- 172.21.15.64:0/1265819691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0450003c40 0x7f0450004030 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 -- 172.21.15.64:0/1265819691 >> 172.21.15.64:0/1265819691 conn(0x7f04680c8850 msgr2=0x7f04680c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:17.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 -- 172.21.15.64:0/1265819691 shutdown_connections 2023-09-27T23:52:17.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.008+0000 7f046cb47700 1 -- 172.21.15.64:0/1265819691 wait complete. 2023-09-27T23:52:17.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 Processor -- start 2023-09-27T23:52:17.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 -- start start 2023-09-27T23:52:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450003c40 0x7f04680c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04680c33d0 0x7f046815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 0x7f046815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04680c3840 con 0x7f046815c7a0 2023-09-27T23:52:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468158cf0 con 0x7f04680c33d0 2023-09-27T23:52:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0468158120 con 0x7f0450003c40 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 0x7f046815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0465d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04680c33d0 0x7f046815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 0x7f046815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37620/0 (socket says 172.21.15.64:37620) 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 -- 172.21.15.64:0/2511772365 learned_addr learned my addr 172.21.15.64:0/2511772365 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046659c700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450003c40 0x7f04680c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 -- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450003c40 msgr2=0x7f04680c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450003c40 0x7f04680c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 -- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04680c33d0 msgr2=0x7f046815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04680c33d0 0x7f046815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 -- 172.21.15.64:0/2511772365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04681583c0 con 0x7f046815c7a0 2023-09-27T23:52:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f0466d9d700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 0x7f046815cc10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0468068c00 tx=0x7f046000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000fc20 con 0x7f046815c7a0 2023-09-27T23:52:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046000fd80 con 0x7f046815c7a0 2023-09-27T23:52:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04681586a0 con 0x7f046815c7a0 2023-09-27T23:52:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460016650 con 0x7f046815c7a0 2023-09-27T23:52:17.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.012+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046815d150 con 0x7f046815c7a0 2023-09-27T23:52:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.016+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0468158e60 con 0x7f046815c7a0 2023-09-27T23:52:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.016+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f0460016a00 con 0x7f046815c7a0 2023-09-27T23:52:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.016+0000 7f04567fc700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0448060aa0 0x7f0448062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.016+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f046007da10 con 0x7f046815c7a0 2023-09-27T23:52:17.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.016+0000 7f046659c700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0448060aa0 0x7f0448062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.016+0000 7f046659c700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0448060aa0 0x7f0448062f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f045c0019f0 tx=0x7f045c015000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.024+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f046004dc80 con 0x7f046815c7a0 2023-09-27T23:52:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.244+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0468071120 con 0x7f046815c7a0 2023-09-27T23:52:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.244+0000 7f04567fc700 1 -- 172.21.15.64:0/2511772365 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0460016cb0 con 0x7f046815c7a0 2023-09-27T23:52:17.250 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrA5GIBAA4NcmcGWbpzZUsNkHYrDQEQ==", 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl6LfYJhAA2rhtxwwzBly3VbIhgZ6DHw==", 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlqSZ9IxAATcir+R+UuRJSeAuJEYROvA==", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlDL34KRAAVWT+eMgXCyjNggHWyVqA9g==", 2023-09-27T23:52:17.253 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDzthRlOE21HBAAtn4dkhKivTjjoUXlfrzLLg==", 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.254 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD2thRljs8YLxAAuwzTLMFlE6VFKaSAvLV0LQ==", 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD5thRlKE2ZMBAAtq1l5D8zxhfd50IdvMS3VA==", 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQD8thRlsw6aNhAAvguHOPLNQ153dlT0ZqyqMA==", 2023-09-27T23:52:17.256 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQACtxRl09UlHBAAJRZqrlOl4jRM9icD8dy2GQ==", 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.257 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAFtxRlKBLmLRAAZaZoV1hMsr8qTG4uuoB0/w==", 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.258 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQAItxRlcrFkMxAAPXd5Y2k3rN/mruZEh2B03w==", 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQALtxRlWxU+ORAA708+8rxTmg1Iv2qk41lYlQ==", 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.259 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRloGIWLRAA2LNgFhp1nfuTwB8XJTiBKA==", 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl2fZPNRAATiMPBqK9xRvTOk4TluNALg==", 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-09-27T23:52:17.261 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlfph2MhAAodYjpAC/gz3JbPypuOKmiw==", 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl3qvKOhAAp2yqKaV+PbAEN8bL91Gkkw==", 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.262 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlrjF4EBAAHMOYxPnx1FM1QR/l/XrvtA==", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-09-27T23:52:17.263 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRl/PBIHRAAgT0aX69+ihaYAE4VPHChdw==", 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDwthRlNiKNGhAAA5M6FkrHvza9Zo2MHAWkhQ==", 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-27T23:52:17.265 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0448060aa0 msgr2=0x7f0448062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0448060aa0 0x7f0448062f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f045c0019f0 tx=0x7f045c015000 comp rx=0 tx=0).stop 2023-09-27T23:52:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 msgr2=0x7f046815cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 0x7f046815cc10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0468068c00 tx=0x7f046000b040 comp rx=0 tx=0).stop 2023-09-27T23:52:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 shutdown_connections 2023-09-27T23:52:17.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0448060aa0 0x7f0448062f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046815c7a0 0x7f046815cc10 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04680c33d0 0x7f046815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 --2- 172.21.15.64:0/2511772365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0450003c40 0x7f04680c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 >> 172.21.15.64:0/2511772365 conn(0x7f04680c8850 msgr2=0x7f04680caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 shutdown_connections 2023-09-27T23:52:17.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.248+0000 7f046cb47700 1 -- 172.21.15.64:0/2511772365 wait complete. 2023-09-27T23:52:17.268 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 Processor -- start 2023-09-27T23:52:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 -- start start 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ea40b1410 0x7f1ea40ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea406aa00 0x7f1ea415af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea406c7c0 con 0x7f1ea406a580 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea406cd80 con 0x7f1ea40b1410 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea982f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406b8f0 con 0x7f1ea406aa00 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea882d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ea40b1410 0x7f1ea40ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea882d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ea40b1410 0x7f1ea40ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37634/0 (socket says 172.21.15.64:37634) 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea882d700 1 -- 172.21.15.64:0/4058217807 learned_addr learned my addr 172.21.15.64:0/4058217807 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.372+0000 7f1ea902e700 1 --2- 172.21.15.64:0/4058217807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea406aa00 0x7f1ea415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea882d700 1 -- 172.21.15.64:0/4058217807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea406aa00 msgr2=0x7f1ea415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:17.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea882d700 1 --2- 172.21.15.64:0/4058217807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea406aa00 0x7f1ea415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea882d700 1 -- 172.21.15.64:0/4058217807 >> v1:172.21.15.64:6789/0 conn(0x7f1ea406a580 legacy=0x7f1ea415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea882d700 1 -- 172.21.15.64:0/4058217807 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40ac1e0 con 0x7f1ea40b1410 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea882d700 1 --2- 172.21.15.64:0/4058217807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea40b1410 0x7f1ea40ab440 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1e980099b0 tx=0x7f1e98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fc4711e94c8172 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea37fe700 1 -- 172.21.15.64:0/4058217807 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800b860 con 0x7f1ea40b1410 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea37fe700 1 -- 172.21.15.64:0/4058217807 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9800b9c0 con 0x7f1ea40b1410 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea37fe700 1 -- 172.21.15.64:0/4058217807 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800bd60 con 0x7f1ea40b1410 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 -- 172.21.15.64:0/4058217807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea40b1410 msgr2=0x7f1ea40ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 --2- 172.21.15.64:0/4058217807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea40b1410 0x7f1ea40ab440 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1e980099b0 tx=0x7f1e98009250 comp rx=0 tx=0).stop 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 -- 172.21.15.64:0/4058217807 shutdown_connections 2023-09-27T23:52:17.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 --2- 172.21.15.64:0/4058217807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea406aa00 0x7f1ea415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 --2- 172.21.15.64:0/4058217807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea40b1410 0x7f1ea40ab440 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 -- 172.21.15.64:0/4058217807 >> 172.21.15.64:0/4058217807 conn(0x7f1ea4067bb0 msgr2=0x7f1ea40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:17.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 -- 172.21.15.64:0/4058217807 shutdown_connections 2023-09-27T23:52:17.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 -- 172.21.15.64:0/4058217807 wait complete. 2023-09-27T23:52:17.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 Processor -- start 2023-09-27T23:52:17.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.376+0000 7f1ea982f700 1 -- start start 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 0x7f1ea414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ea406aa00 0x7f1ea40c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea40b1410 0x7f1ea4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40c2ea0 con 0x7f1ea406a580 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40c3180 con 0x7f1ea406aa00 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40c3460 con 0x7f1ea40b1410 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 0x7f1ea414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 0x7f1ea414a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37642/0 (socket says 172.21.15.64:37642) 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 -- 172.21.15.64:0/706608461 learned_addr learned my addr 172.21.15.64:0/706608461 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea902e700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea40b1410 0x7f1ea4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea3fff700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ea406aa00 0x7f1ea40c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 -- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea40b1410 msgr2=0x7f1ea4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea40b1410 0x7f1ea4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 -- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ea406aa00 msgr2=0x7f1ea40c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ea406aa00 0x7f1ea40c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 -- 172.21.15.64:0/706608461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980094f0 con 0x7f1ea406a580 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea882d700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 0x7f1ea414a350 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009fd0 tx=0x7f1e98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98011e40 con 0x7f1ea406a580 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e98024440 con 0x7f1ea406a580 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea40c3740 con 0x7f1ea406a580 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980247e0 con 0x7f1ea406a580 2023-09-27T23:52:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea4154850 con 0x7f1ea406a580 2023-09-27T23:52:17.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.380+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea404be40 con 0x7f1ea406a580 2023-09-27T23:52:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.384+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1e9801a420 con 0x7f1ea406a580 2023-09-27T23:52:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.384+0000 7f1ea0ff9700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e88060a70 0x7f1e88062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.384+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1e9807ed90 con 0x7f1ea406a580 2023-09-27T23:52:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.384+0000 7f1ea3fff700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e88060a70 0x7f1e88062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.384+0000 7f1ea3fff700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e88060a70 0x7f1e88062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ea416a480 tx=0x7f1e90007ea0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.392+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1e9804f070 con 0x7f1ea406a580 2023-09-27T23:52:17.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.556+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ea40ab220 con 0x7f1ea406a580 2023-09-27T23:52:17.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.556+0000 7f1ea0ff9700 1 -- 172.21.15.64:0/706608461 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+13865 (secure 0 0 0) 0x7f1e98052690 con 0x7f1ea406a580 2023-09-27T23:52:17.563 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:17.563 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":185,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:16.850684+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e88060a70 msgr2=0x7f1e88062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e88060a70 0x7f1e88062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ea416a480 tx=0x7f1e90007ea0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 msgr2=0x7f1ea414a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 0x7f1ea414a350 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009fd0 tx=0x7f1e98017000 comp rx=0 tx=0).stop 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 shutdown_connections 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e88060a70 0x7f1e88062f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ea40b1410 0x7f1ea4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ea406aa00 0x7f1ea40c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 --2- 172.21.15.64:0/706608461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ea406a580 0x7f1ea414a350 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 >> 172.21.15.64:0/706608461 conn(0x7f1ea4067bb0 msgr2=0x7f1ea4155470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:17.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 shutdown_connections 2023-09-27T23:52:17.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.560+0000 7f1ea982f700 1 -- 172.21.15.64:0/706608461 wait complete. 2023-09-27T23:52:17.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-27T23:52:17.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:52:17.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.684+0000 7eff5f59e700 1 Processor -- start 2023-09-27T23:52:17.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.684+0000 7eff5f59e700 1 -- start start 2023-09-27T23:52:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff600c8bf0 0x7eff600ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff6006d390 0x7eff6006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff600ae110 con 0x7eff600ace10 2023-09-27T23:52:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff60157110 con 0x7eff6006d390 2023-09-27T23:52:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6006e0c0 con 0x7eff600c8bf0 2023-09-27T23:52:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff6006d390 0x7eff6006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff6006d390 0x7eff6006d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37646/0 (socket says 172.21.15.64:37646) 2023-09-27T23:52:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1070493000 learned_addr learned my addr 172.21.15.64:0/1070493000 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5dd9b700 1 --2- 172.21.15.64:0/1070493000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff600c8bf0 0x7eff600ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1070493000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff600c8bf0 msgr2=0x7eff600ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 --2- 172.21.15.64:0/1070493000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff600c8bf0 0x7eff600ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1070493000 >> v1:172.21.15.64:6789/0 conn(0x7eff600ace10 legacy=0x7eff600c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1070493000 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600ad7e0 con 0x7eff6006d390 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5ed9d700 1 --2- 172.21.15.64:0/1070493000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff6006d390 0x7eff6006d7f0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7eff58009f20 tx=0x7eff58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8b9abcce8b3cb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5d59a700 1 -- 172.21.15.64:0/1070493000 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800e070 con 0x7eff6006d390 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5d59a700 1 -- 172.21.15.64:0/1070493000 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5800b9d0 con 0x7eff6006d390 2023-09-27T23:52:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5d59a700 1 -- 172.21.15.64:0/1070493000 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff580092e0 con 0x7eff6006d390 2023-09-27T23:52:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- 172.21.15.64:0/1070493000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff6006d390 msgr2=0x7eff6006d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1070493000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff6006d390 0x7eff6006d7f0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7eff58009f20 tx=0x7eff58009840 comp rx=0 tx=0).stop 2023-09-27T23:52:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- 172.21.15.64:0/1070493000 shutdown_connections 2023-09-27T23:52:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1070493000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff6006d390 0x7eff6006d7f0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1070493000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff600c8bf0 0x7eff600ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- 172.21.15.64:0/1070493000 >> 172.21.15.64:0/1070493000 conn(0x7eff600cdc00 msgr2=0x7eff600ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- 172.21.15.64:0/1070493000 shutdown_connections 2023-09-27T23:52:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.688+0000 7eff5f59e700 1 -- 172.21.15.64:0/1070493000 wait complete. 2023-09-27T23:52:17.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.692+0000 7eff5f59e700 1 Processor -- start 2023-09-27T23:52:17.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.692+0000 7eff5f59e700 1 -- start start 2023-09-27T23:52:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff6006d390 0x7eff600c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600ace10 0x7eff6015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 0x7eff600ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff60167d50 con 0x7eff600c8bf0 2023-09-27T23:52:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff60166e90 con 0x7eff600ace10 2023-09-27T23:52:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff60167140 con 0x7eff6006d390 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 0x7eff600ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 0x7eff600ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37662/0 (socket says 172.21.15.64:37662) 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1372018232 learned_addr learned my addr 172.21.15.64:0/1372018232 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5e59c700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff6006d390 0x7eff600c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5dd9b700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600ace10 0x7eff6015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff6006d390 msgr2=0x7eff600c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff6006d390 0x7eff600c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600ace10 msgr2=0x7eff6015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600ace10 0x7eff6015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 -- 172.21.15.64:0/1372018232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009ae0 con 0x7eff600c8bf0 2023-09-27T23:52:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5ed9d700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 0x7eff600ae630 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b610 tx=0x7eff58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58008ad0 con 0x7eff600c8bf0 2023-09-27T23:52:17.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58008c30 con 0x7eff600c8bf0 2023-09-27T23:52:17.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff60167420 con 0x7eff600c8bf0 2023-09-27T23:52:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58026460 con 0x7eff600c8bf0 2023-09-27T23:52:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff601679e0 con 0x7eff600c8bf0 2023-09-27T23:52:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.696+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6004be40 con 0x7eff600c8bf0 2023-09-27T23:52:17.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.700+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7eff58003040 con 0x7eff600c8bf0 2023-09-27T23:52:17.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.700+0000 7eff567fc700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff40060a70 0x7eff40062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.700+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5526+0+0 (secure 0 0 0) 0x7eff5807ee10 con 0x7eff600c8bf0 2023-09-27T23:52:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.700+0000 7eff5e59c700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff40060a70 0x7eff40062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:17.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.700+0000 7eff5e59c700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff40060a70 0x7eff40062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff50003a80 tx=0x7eff50007ec0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:17.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.708+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff58082030 con 0x7eff600c8bf0 2023-09-27T23:52:17.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.868+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff600adac0 con 0x7eff600c8bf0 2023-09-27T23:52:17.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.872+0000 7eff567fc700 1 -- 172.21.15.64:0/1372018232 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff5804efe0 con 0x7eff600c8bf0 2023-09-27T23:52:17.877 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.872+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff40060a70 msgr2=0x7eff40062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.872+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff40060a70 0x7eff40062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff50003a80 tx=0x7eff50007ec0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.872+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 msgr2=0x7eff600ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.872+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 0x7eff600ae630 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b610 tx=0x7eff58018000 comp rx=0 tx=0).stop 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 shutdown_connections 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff40060a70 0x7eff40062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff600c8bf0 0x7eff600ae630 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff600ace10 0x7eff6015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 --2- 172.21.15.64:0/1372018232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff6006d390 0x7eff600c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:17.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 >> 172.21.15.64:0/1372018232 conn(0x7eff600cdc00 msgr2=0x7eff60152c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:17.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 shutdown_connections 2023-09-27T23:52:17.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:17.876+0000 7eff5f59e700 1 -- 172.21.15.64:0/1372018232 wait complete. 2023-09-27T23:52:17.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:52:18.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 Processor -- start 2023-09-27T23:52:18.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 -- start start 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effe4158e20 0x7effe40ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40cb970 0x7effe40ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effe40ca2c0 0x7effe406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe40cbdf0 con 0x7effe40ca2c0 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe40cc0d0 con 0x7effe4158e20 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effeaa4f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40cc3b0 con 0x7effe40cb970 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effe8fec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effe40ca2c0 0x7effe406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effe8fec700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effe40ca2c0 0x7effe406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37672/0 (socket says 172.21.15.64:37672) 2023-09-27T23:52:18.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effe8fec700 1 -- 172.21.15.64:0/4134468880 learned_addr learned my addr 172.21.15.64:0/4134468880 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effe37fe700 1 --2- 172.21.15.64:0/4134468880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40cb970 0x7effe40ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.000+0000 7effe3fff700 1 --2- 172.21.15.64:0/4134468880 >> v2:172.21.15.123:3300/0 conn(0x7effe4158e20 0x7effe40ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe8fec700 1 -- 172.21.15.64:0/4134468880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40cb970 msgr2=0x7effe40ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe8fec700 1 --2- 172.21.15.64:0/4134468880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40cb970 0x7effe40ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe8fec700 1 -- 172.21.15.64:0/4134468880 >> v2:172.21.15.123:3300/0 conn(0x7effe4158e20 msgr2=0x7effe40ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe8fec700 1 --2- 172.21.15.64:0/4134468880 >> v2:172.21.15.123:3300/0 conn(0x7effe4158e20 0x7effe40ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe8fec700 1 -- 172.21.15.64:0/4134468880 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40cc690 con 0x7effe40ca2c0 2023-09-27T23:52:18.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe8fec700 1 --2- 172.21.15.64:0/4134468880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe406c7c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7effe40cdc10 tx=0x7effdc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c043498f021ac60a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe2ffd700 1 -- 172.21.15.64:0/4134468880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc00e070 con 0x7effe40ca2c0 2023-09-27T23:52:18.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe2ffd700 1 -- 172.21.15.64:0/4134468880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc00ba50 con 0x7effe40ca2c0 2023-09-27T23:52:18.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effe2ffd700 1 -- 172.21.15.64:0/4134468880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc0092e0 con 0x7effe40ca2c0 2023-09-27T23:52:18.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 -- 172.21.15.64:0/4134468880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 msgr2=0x7effe406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 --2- 172.21.15.64:0/4134468880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe406c7c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7effe40cdc10 tx=0x7effdc009840 comp rx=0 tx=0).stop 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 -- 172.21.15.64:0/4134468880 shutdown_connections 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 --2- 172.21.15.64:0/4134468880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe406c7c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 --2- 172.21.15.64:0/4134468880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40cb970 0x7effe40ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 --2- 172.21.15.64:0/4134468880 >> v2:172.21.15.123:3300/0 conn(0x7effe4158e20 0x7effe40ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 -- 172.21.15.64:0/4134468880 >> 172.21.15.64:0/4134468880 conn(0x7effe4067bb0 msgr2=0x7effe40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 -- 172.21.15.64:0/4134468880 shutdown_connections 2023-09-27T23:52:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 -- 172.21.15.64:0/4134468880 wait complete. 2023-09-27T23:52:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 Processor -- start 2023-09-27T23:52:18.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.004+0000 7effeaa4f700 1 -- start start 2023-09-27T23:52:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe40b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe4158e20 0x7effe414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40b39c0 0x7effe415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe414d210 con 0x7effe40ca2c0 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe415dcb0 con 0x7effe4158e20 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe415d980 con 0x7effe40b39c0 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe40b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe40b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37684/0 (socket says 172.21.15.64:37684) 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 -- 172.21.15.64:0/2336576059 learned_addr learned my addr 172.21.15.64:0/2336576059 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe8fec700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40b39c0 0x7effe415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe37fe700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe4158e20 0x7effe414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 -- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40b39c0 msgr2=0x7effe415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40b39c0 0x7effe415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 -- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe4158e20 msgr2=0x7effe414ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe4158e20 0x7effe414ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 -- 172.21.15.64:0/2336576059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009ae0 con 0x7effe40ca2c0 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effe3fff700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe40b5bc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7effd800d890 tx=0x7effd8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd800b600 con 0x7effe40ca2c0 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe40b4130 con 0x7effe40ca2c0 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd800b760 con 0x7effe40ca2c0 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe40b46f0 con 0x7effe40ca2c0 2023-09-27T23:52:18.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.008+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd800bae0 con 0x7effe40ca2c0 2023-09-27T23:52:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.012+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe40b4910 con 0x7effe40ca2c0 2023-09-27T23:52:18.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.012+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7effd801e460 con 0x7effe40ca2c0 2023-09-27T23:52:18.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.012+0000 7effcbfff700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effd0060a70 0x7effd0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.012+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5526+0+0 (secure 0 0 0) 0x7effd807e080 con 0x7effe40ca2c0 2023-09-27T23:52:18.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.012+0000 7effe37fe700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effd0060a70 0x7effd0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.016+0000 7effe37fe700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effd0060a70 0x7effd0062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effe4156020 tx=0x7effd4008040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.020+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7effd804e400 con 0x7effe40ca2c0 2023-09-27T23:52:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.176+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7effe4071120 con 0x7effe40ca2c0 2023-09-27T23:52:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.176+0000 7effcbfff700 1 -- 172.21.15.64:0/2336576059 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v185) v1 ==== 92+0+10 (secure 0 0 0) 0x7effd8051a20 con 0x7effe40ca2c0 2023-09-27T23:52:18.183 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:52:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effd0060a70 msgr2=0x7effd0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effd0060a70 0x7effd0062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effe4156020 tx=0x7effd4008040 comp rx=0 tx=0).stop 2023-09-27T23:52:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 msgr2=0x7effe40b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe40b5bc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7effd800d890 tx=0x7effd8008040 comp rx=0 tx=0).stop 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 shutdown_connections 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7effd0060a70 0x7effd0062f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7effe40b39c0 0x7effe415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe4158e20 0x7effe414ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 --2- 172.21.15.64:0/2336576059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe40ca2c0 0x7effe40b5bc0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 >> 172.21.15.64:0/2336576059 conn(0x7effe4067bb0 msgr2=0x7effe40cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 shutdown_connections 2023-09-27T23:52:18.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.180+0000 7effeaa4f700 1 -- 172.21.15.64:0/2336576059 wait complete. 2023-09-27T23:52:18.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-27T23:52:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 Processor -- start 2023-09-27T23:52:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 -- start start 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd900b0810 0x7efd900b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900af9c0 0x7efd90154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd900af530 con 0x7efd9006be10 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd90159470 con 0x7efd900b0810 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd972b5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd90159660 con 0x7efd900af9c0 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd95ab2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efd9006be10 0x7efd90158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55884/0 (socket says 172.21.15.64:55884) 2023-09-27T23:52:18.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd95ab2700 1 -- 172.21.15.64:0/4176692130 learned_addr learned my addr 172.21.15.64:0/4176692130 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd962b3700 1 --2- 172.21.15.64:0/4176692130 >> v2:172.21.15.123:3300/0 conn(0x7efd900b0810 0x7efd900b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd96ab4700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900af9c0 0x7efd90154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.300+0000 7efd952b1700 1 -- 172.21.15.64:0/4176692130 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501149851 0 0) 0x7efd900af530 con 0x7efd9006be10 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd952b1700 1 -- 172.21.15.64:0/4176692130 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd7c004680 con 0x7efd9006be10 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd962b3700 1 -- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900af9c0 msgr2=0x7efd90154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd962b3700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900af9c0 0x7efd90154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd962b3700 1 -- 172.21.15.64:0/4176692130 >> v1:172.21.15.64:6789/0 conn(0x7efd9006be10 legacy=0x7efd90158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd962b3700 1 -- 172.21.15.64:0/4176692130 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd90159940 con 0x7efd900b0810 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd96ab4700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900af9c0 0x7efd90154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd962b3700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd900b0810 0x7efd900b0be0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0098e0 tx=0x7efd8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc12b5c0906eb54 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd952b1700 1 -- 172.21.15.64:0/4176692130 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c011630 con 0x7efd900b0810 2023-09-27T23:52:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd952b1700 1 -- 172.21.15.64:0/4176692130 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c011790 con 0x7efd900b0810 2023-09-27T23:52:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd952b1700 1 -- 172.21.15.64:0/4176692130 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c011b10 con 0x7efd900b0810 2023-09-27T23:52:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 -- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd900b0810 msgr2=0x7efd900b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd900b0810 0x7efd900b0be0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0098e0 tx=0x7efd8c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 -- 172.21.15.64:0/4176692130 shutdown_connections 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900af9c0 0x7efd90154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 --2- 172.21.15.64:0/4176692130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd900b0810 0x7efd900b0be0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 -- 172.21.15.64:0/4176692130 >> 172.21.15.64:0/4176692130 conn(0x7efd90067bb0 msgr2=0x7efd900add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 -- 172.21.15.64:0/4176692130 shutdown_connections 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 -- 172.21.15.64:0/4176692130 wait complete. 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 Processor -- start 2023-09-27T23:52:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 -- start start 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd9006be10 0x7efd90154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 0x7efd90150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.304+0000 7efd972b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900b0810 0x7efd90149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd972b5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900c6eb0 con 0x7efd900af9c0 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd972b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9016a1c0 con 0x7efd9006be10 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd972b5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9014c510 con 0x7efd900b0810 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 0x7efd90150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 0x7efd90150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37686/0 (socket says 172.21.15.64:37686) 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 -- 172.21.15.64:0/1188131904 learned_addr learned my addr 172.21.15.64:0/1188131904 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd962b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd9006be10 0x7efd90154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd96ab4700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900b0810 0x7efd90149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 -- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900b0810 msgr2=0x7efd90149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900b0810 0x7efd90149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 -- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd9006be10 msgr2=0x7efd90154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd9006be10 0x7efd90154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 -- 172.21.15.64:0/1188131904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88009a20 con 0x7efd900af9c0 2023-09-27T23:52:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd95ab2700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 0x7efd90150bd0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c7c0 tx=0x7efd80007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80005c00 con 0x7efd900af9c0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd80005d60 con 0x7efd900af9c0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c009420 con 0x7efd900af9c0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800072e0 con 0x7efd900af9c0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9014cad0 con 0x7efd900af9c0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.308+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9004be40 con 0x7efd900af9c0 2023-09-27T23:52:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.312+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7efd80007690 con 0x7efd900af9c0 2023-09-27T23:52:18.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.312+0000 7efd867fc700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd74060a70 0x7efd74062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.312+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5526+0+0 (secure 0 0 0) 0x7efd8007c650 con 0x7efd900af9c0 2023-09-27T23:52:18.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.312+0000 7efd962b3700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd74060a70 0x7efd74062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.312+0000 7efd962b3700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd74060a70 0x7efd74062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0091e0 tx=0x7efd8c017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.316+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efd8004cb40 con 0x7efd900af9c0 2023-09-27T23:52:18.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.480+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efd90159c20 con 0x7efd900af9c0 2023-09-27T23:52:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.480+0000 7efd867fc700 1 -- 172.21.15.64:0/1188131904 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v185) v1 ==== 84+0+126 (secure 0 0 0) 0x7efd80017020 con 0x7efd900af9c0 2023-09-27T23:52:18.486 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:18.486 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 2, 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": "rbd" 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:18.487 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:52:18.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd74060a70 msgr2=0x7efd74062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd74060a70 0x7efd74062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0091e0 tx=0x7efd8c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:18.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 msgr2=0x7efd90150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 0x7efd90150bd0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c7c0 tx=0x7efd80007d90 comp rx=0 tx=0).stop 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 shutdown_connections 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd74060a70 0x7efd74062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd900b0810 0x7efd90149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd900af9c0 0x7efd90150bd0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 --2- 172.21.15.64:0/1188131904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd9006be10 0x7efd90154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 >> 172.21.15.64:0/1188131904 conn(0x7efd90067bb0 msgr2=0x7efd900c9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 shutdown_connections 2023-09-27T23:52:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.484+0000 7efd972b5700 1 -- 172.21.15.64:0/1188131904 wait complete. 2023-09-27T23:52:18.502 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-27T23:52:18.502 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-27T23:52:18.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.620+0000 7f52b3fff700 1 Processor -- start 2023-09-27T23:52:18.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.620+0000 7f52b3fff700 1 -- start start 2023-09-27T23:52:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52b4158da0 0x7f52b406e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b415d720 con 0x7f52b40caca0 2023-09-27T23:52:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b40ca4a0 con 0x7f52b406df10 2023-09-27T23:52:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b40ca290 con 0x7f52b4158da0 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f52b40caca0 0x7f52b406aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55888/0 (socket says 172.21.15.64:55888) 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b37fe700 1 -- 172.21.15.64:0/1522105047 learned_addr learned my addr 172.21.15.64:0/1522105047 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b2ffd700 1 --2- 172.21.15.64:0/1522105047 >> v2:172.21.15.123:3300/0 conn(0x7f52b4158da0 0x7f52b406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b1ffb700 1 -- 172.21.15.64:0/1522105047 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49227950 0 0) 0x7f52b415d720 con 0x7f52b40caca0 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b1ffb700 1 -- 172.21.15.64:0/1522105047 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a0003570 con 0x7f52b40caca0 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b2ffd700 1 -- 172.21.15.64:0/1522105047 >> v1:172.21.15.123:6789/0 conn(0x7f52b406df10 legacy=0x7f52b406a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b2ffd700 1 -- 172.21.15.64:0/1522105047 >> v1:172.21.15.64:6789/0 conn(0x7f52b40caca0 legacy=0x7f52b406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b2ffd700 1 -- 172.21.15.64:0/1522105047 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40ca780 con 0x7f52b4158da0 2023-09-27T23:52:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b2ffd700 1 --2- 172.21.15.64:0/1522105047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b406e670 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098e0 tx=0x7f529c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71595b257da1407 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b1ffb700 1 -- 172.21.15.64:0/1522105047 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c00b640 con 0x7f52b4158da0 2023-09-27T23:52:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b1ffb700 1 -- 172.21.15.64:0/1522105047 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c00b7a0 con 0x7f52b4158da0 2023-09-27T23:52:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b1ffb700 1 -- 172.21.15.64:0/1522105047 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c00bb40 con 0x7f52b4158da0 2023-09-27T23:52:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- 172.21.15.64:0/1522105047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 msgr2=0x7f52b406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1522105047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b406e670 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098e0 tx=0x7f529c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- 172.21.15.64:0/1522105047 shutdown_connections 2023-09-27T23:52:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1522105047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b406e670 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- 172.21.15.64:0/1522105047 >> 172.21.15.64:0/1522105047 conn(0x7f52b40cc760 msgr2=0x7f52b40ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- 172.21.15.64:0/1522105047 shutdown_connections 2023-09-27T23:52:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.624+0000 7f52b3fff700 1 -- 172.21.15.64:0/1522105047 wait complete. 2023-09-27T23:52:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 Processor -- start 2023-09-27T23:52:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 -- start start 2023-09-27T23:52:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b406df10 0x7f52b4150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 0x7f52b40af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b40afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4153150 con 0x7f52b40caca0 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4152540 con 0x7f52b4158da0 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4152820 con 0x7f52b406df10 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 0x7f52b40af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b40afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b406df10 0x7f52b4150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 0x7f52b40af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37698/0 (socket says 172.21.15.64:37698) 2023-09-27T23:52:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 -- 172.21.15.64:0/1926438813 learned_addr learned my addr 172.21.15.64:0/1926438813 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 -- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b406df10 msgr2=0x7f52b4150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b406df10 0x7f52b4150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 -- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 msgr2=0x7f52b40afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b40afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 -- 172.21.15.64:0/1926438813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c009420 con 0x7f52b40caca0 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b27fc700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 0x7f52b40af8f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c590 tx=0x7f52a400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4011590 con 0x7f52b40caca0 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4011710 con 0x7f52b40caca0 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b4152b00 con 0x7f52b40caca0 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.628+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4011ad0 con 0x7f52b40caca0 2023-09-27T23:52:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.632+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b4150640 con 0x7f52b40caca0 2023-09-27T23:52:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.632+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b404be40 con 0x7f52b40caca0 2023-09-27T23:52:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.632+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f52a401b3b0 con 0x7f52b40caca0 2023-09-27T23:52:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.632+0000 7f52aaffd700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5294060a70 0x7f5294062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.632+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f52a407c2b0 con 0x7f52b40caca0 2023-09-27T23:52:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.632+0000 7f52b2ffd700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5294060a70 0x7f5294062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.636+0000 7f52b2ffd700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5294060a70 0x7f5294062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f529c0093f0 tx=0x7f529c017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.644+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f52a404c590 con 0x7f52b40caca0 2023-09-27T23:52:18.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.808+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f52b406f360 con 0x7f52b40caca0 2023-09-27T23:52:18.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.884+0000 7f52aaffd700 1 -- 172.21.15.64:0/1926438813 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v186) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52a40170b0 con 0x7f52b40caca0 2023-09-27T23:52:18.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5294060a70 msgr2=0x7f5294062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5294060a70 0x7f5294062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f529c0093f0 tx=0x7f529c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:18.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 msgr2=0x7f52b40af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:18.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 0x7f52b40af8f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c590 tx=0x7f52a400c8a0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 shutdown_connections 2023-09-27T23:52:18.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5294060a70 0x7f5294062f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b4158da0 0x7f52b40afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b40caca0 0x7f52b40af8f0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 --2- 172.21.15.64:0/1926438813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52b406df10 0x7f52b4150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.892+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 >> 172.21.15.64:0/1926438813 conn(0x7f52b40cc760 msgr2=0x7f52b40b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.896+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 shutdown_connections 2023-09-27T23:52:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:18.896+0000 7f52b3fff700 1 -- 172.21.15.64:0/1926438813 wait complete. 2023-09-27T23:52:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-27T23:52:18.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-27T23:52:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.024+0000 7f9162521700 1 Processor -- start 2023-09-27T23:52:19.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.024+0000 7f9162521700 1 -- start start 2023-09-27T23:52:19.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f915c06ae20 0x7f915c06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:19.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c06b730 con 0x7f915c0b1440 2023-09-27T23:52:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c0ac390 con 0x7f915c0ab6d0 2023-09-27T23:52:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915c0ac670 con 0x7f915c06ae20 2023-09-27T23:52:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9161d20700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f915c0b1440 0x7f915c0b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55896/0 (socket says 172.21.15.64:55896) 2023-09-27T23:52:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9161d20700 1 -- 172.21.15.64:0/3935067369 learned_addr learned my addr 172.21.15.64:0/3935067369 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:19.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f916151f700 1 --2- 172.21.15.64:0/3935067369 >> v2:172.21.15.123:3300/0 conn(0x7f915c06ae20 0x7f915c06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9153fff700 1 -- 172.21.15.64:0/3935067369 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88401849 0 0) 0x7f915c06b730 con 0x7f915c0b1440 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9153fff700 1 -- 172.21.15.64:0/3935067369 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9148003570 con 0x7f915c0b1440 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f916151f700 1 -- 172.21.15.64:0/3935067369 >> v1:172.21.15.123:6789/0 conn(0x7f915c0ab6d0 legacy=0x7f915c0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f916151f700 1 -- 172.21.15.64:0/3935067369 >> v1:172.21.15.64:6789/0 conn(0x7f915c0b1440 legacy=0x7f915c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f916151f700 1 -- 172.21.15.64:0/3935067369 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0ac950 con 0x7f915c06ae20 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f916151f700 1 --2- 172.21.15.64:0/3935067369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c06ae20 0x7f915c06b1f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f91580098e0 tx=0x7f91580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f444d663d59ef19a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:19.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9153fff700 1 -- 172.21.15.64:0/3935067369 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915800b640 con 0x7f915c06ae20 2023-09-27T23:52:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9153fff700 1 -- 172.21.15.64:0/3935067369 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915800b7a0 con 0x7f915c06ae20 2023-09-27T23:52:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9153fff700 1 -- 172.21.15.64:0/3935067369 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915800bb20 con 0x7f915c06ae20 2023-09-27T23:52:19.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- 172.21.15.64:0/3935067369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c06ae20 msgr2=0x7f915c06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 --2- 172.21.15.64:0/3935067369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c06ae20 0x7f915c06b1f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f91580098e0 tx=0x7f91580091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- 172.21.15.64:0/3935067369 shutdown_connections 2023-09-27T23:52:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 --2- 172.21.15.64:0/3935067369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c06ae20 0x7f915c06b1f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- 172.21.15.64:0/3935067369 >> 172.21.15.64:0/3935067369 conn(0x7f915c0c8cd0 msgr2=0x7f915c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- 172.21.15.64:0/3935067369 shutdown_connections 2023-09-27T23:52:19.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.028+0000 7f9162521700 1 -- 172.21.15.64:0/3935067369 wait complete. 2023-09-27T23:52:19.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 Processor -- start 2023-09-27T23:52:19.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 -- start start 2023-09-27T23:52:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f915c06ae20 0x7f915c0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:19.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f915c0ab6d0 0x7f915c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 0x7f915c14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0c3380 con 0x7f915c06ae20 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0c35d0 con 0x7f915c0b1440 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f9162521700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0c38b0 con 0x7f915c0ab6d0 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.032+0000 7f916151f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f915c06ae20 0x7f915c0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 0x7f915c14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 0x7f915c14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:35810/0 (socket says 172.21.15.64:35810) 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 -- 172.21.15.64:0/1319846442 learned_addr learned my addr 172.21.15.64:0/1319846442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9160d1e700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f915c0ab6d0 0x7f915c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 -- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f915c0ab6d0 msgr2=0x7f915c14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f915c0ab6d0 0x7f915c14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 -- 172.21.15.64:0/1319846442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f915c06ae20 msgr2=0x7f915c0c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f915c06ae20 0x7f915c0c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 -- 172.21.15.64:0/1319846442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009420 con 0x7f915c0b1440 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9161d20700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 0x7f915c14c670 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9154001460 tx=0x7f9154001ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400a940 con 0x7f915c0b1440 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c0c3b90 con 0x7f915c0b1440 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915400aaa0 con 0x7f915c0b1440 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c0c4150 con 0x7f915c0b1440 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400ae20 con 0x7f915c0b1440 2023-09-27T23:52:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.036+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c04be40 con 0x7f915c0b1440 2023-09-27T23:52:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.040+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9154007300 con 0x7f915c0b1440 2023-09-27T23:52:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.040+0000 7f91517fa700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9140060a70 0x7f9140062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.040+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f915407df50 con 0x7f915c0b1440 2023-09-27T23:52:19.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.040+0000 7f916151f700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9140060a70 0x7f9140062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:19.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.040+0000 7f916151f700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9140060a70 0x7f9140062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f91580093f0 tx=0x7f9158017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:19.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.048+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f915404e150 con 0x7f915c0b1440 2023-09-27T23:52:19.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.212+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f915c0acc30 con 0x7f915c0b1440 2023-09-27T23:52:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.904+0000 7f91517fa700 1 -- 172.21.15.64:0/1319846442 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v187) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9154051770 con 0x7f915c0b1440 2023-09-27T23:52:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9140060a70 msgr2=0x7f9140062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9140060a70 0x7f9140062f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f91580093f0 tx=0x7f9158017000 comp rx=0 tx=0).stop 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 msgr2=0x7f915c14c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 0x7f915c14c670 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9154001460 tx=0x7f9154001ae0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 shutdown_connections 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9140060a70 0x7f9140062f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f915c0b1440 0x7f915c14c670 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f915c0ab6d0 0x7f915c14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 --2- 172.21.15.64:0/1319846442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f915c06ae20 0x7f915c0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 >> 172.21.15.64:0/1319846442 conn(0x7f915c0c8cd0 msgr2=0x7f915c156770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 shutdown_connections 2023-09-27T23:52:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:19.928+0000 7f9162521700 1 -- 172.21.15.64:0/1319846442 wait complete. 2023-09-27T23:52:19.935 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-27T23:52:19.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-27T23:52:20.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.048+0000 7fa9dabd3700 1 Processor -- start 2023-09-27T23:52:20.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.048+0000 7fa9dabd3700 1 -- start start 2023-09-27T23:52:20.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa9d413f7e0 0x7fa9d41564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9d413fbb0 0x7fa9d40af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d40cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d4157950 con 0x7fa9d413fbb0 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40cc3c0 con 0x7fa9d413f7e0 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40af780 con 0x7fa9d40cba00 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9d413fbb0 0x7fa9d40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa9d413fbb0 0x7fa9d40af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37720/0 (socket says 172.21.15.64:37720) 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 -- 172.21.15.64:0/1271950392 learned_addr learned my addr 172.21.15.64:0/1271950392 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9d9170700 1 --2- 172.21.15.64:0/1271950392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d40cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9d896f700 1 --2- 172.21.15.64:0/1271950392 >> v2:172.21.15.123:3300/0 conn(0x7fa9d413f7e0 0x7fa9d41564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 -- 172.21.15.64:0/1271950392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 msgr2=0x7fa9d40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 --2- 172.21.15.64:0/1271950392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d40cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 -- 172.21.15.64:0/1271950392 >> v2:172.21.15.123:3300/0 conn(0x7fa9d413f7e0 msgr2=0x7fa9d41564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 --2- 172.21.15.64:0/1271950392 >> v2:172.21.15.123:3300/0 conn(0x7fa9d413f7e0 0x7fa9d41564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 -- 172.21.15.64:0/1271950392 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4156af0 con 0x7fa9d413fbb0 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9d896f700 1 --2- 172.21.15.64:0/1271950392 >> v2:172.21.15.123:3300/0 conn(0x7fa9d413f7e0 0x7fa9d41564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cbfff700 1 --2- 172.21.15.64:0/1271950392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413fbb0 0x7fa9d40af240 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40cde80 tx=0x7fa9c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10eb52c54f62574 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cb7fe700 1 -- 172.21.15.64:0/1271950392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4009e70 con 0x7fa9d413fbb0 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cb7fe700 1 -- 172.21.15.64:0/1271950392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400b8f0 con 0x7fa9d413fbb0 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9cb7fe700 1 -- 172.21.15.64:0/1271950392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400bc90 con 0x7fa9d413fbb0 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 -- 172.21.15.64:0/1271950392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413fbb0 msgr2=0x7fa9d40af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.052+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/1271950392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413fbb0 0x7fa9d40af240 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40cde80 tx=0x7fa9c4009250 comp rx=0 tx=0).stop 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- 172.21.15.64:0/1271950392 shutdown_connections 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/1271950392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/1271950392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413fbb0 0x7fa9d40af240 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/1271950392 >> v2:172.21.15.123:3300/0 conn(0x7fa9d413f7e0 0x7fa9d41564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- 172.21.15.64:0/1271950392 >> 172.21.15.64:0/1271950392 conn(0x7fa9d40cd8f0 msgr2=0x7fa9d40c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- 172.21.15.64:0/1271950392 shutdown_connections 2023-09-27T23:52:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- 172.21.15.64:0/1271950392 wait complete. 2023-09-27T23:52:20.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 Processor -- start 2023-09-27T23:52:20.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- start start 2023-09-27T23:52:20.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d4155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 0x7fa9d413b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d413fbb0 0x7fa9d4069320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4067af0 con 0x7fa9d413f7e0 2023-09-27T23:52:20.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4068950 con 0x7fa9d413fbb0 2023-09-27T23:52:20.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9dabd3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4155ba0 con 0x7fa9d40cba00 2023-09-27T23:52:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 0x7fa9d413b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 0x7fa9d413b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37736/0 (socket says 172.21.15.64:37736) 2023-09-27T23:52:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9cbfff700 1 -- 172.21.15.64:0/2592454273 learned_addr learned my addr 172.21.15.64:0/2592454273 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9d896f700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d4155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.056+0000 7fa9d9170700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d413fbb0 0x7fa9d4069320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9cbfff700 1 -- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 msgr2=0x7fa9d4155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9cbfff700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d4155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9cbfff700 1 -- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d413fbb0 msgr2=0x7fa9d4069320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9cbfff700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d413fbb0 0x7fa9d4069320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9cbfff700 1 -- 172.21.15.64:0/2592454273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c40094f0 con 0x7fa9d413f7e0 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9cbfff700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 0x7fa9d413b320 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40094c0 tx=0x7fa9c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4000d10 con 0x7fa9d413f7e0 2023-09-27T23:52:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c401f070 con 0x7fa9d413f7e0 2023-09-27T23:52:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d41565c0 con 0x7fa9d413f7e0 2023-09-27T23:52:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40266b0 con 0x7fa9d413f7e0 2023-09-27T23:52:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d4068020 con 0x7fa9d413f7e0 2023-09-27T23:52:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d4071120 con 0x7fa9d413f7e0 2023-09-27T23:52:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.060+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fa9c401a430 con 0x7fa9d413f7e0 2023-09-27T23:52:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.064+0000 7fa9c8ff9700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa9c0060a70 0x7fa9c0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.064+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fa9c4021070 con 0x7fa9d413f7e0 2023-09-27T23:52:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.064+0000 7fa9d896f700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa9c0060a70 0x7fa9c0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.064+0000 7fa9d896f700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa9c0060a70 0x7fa9c0062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006f00 tx=0x7fa9d0008040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.068+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa9c404ee00 con 0x7fa9d413f7e0 2023-09-27T23:52:20.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.260+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa9d4068a90 con 0x7fa9d413f7e0 2023-09-27T23:52:20.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.276+0000 7fa9c8ff9700 1 -- 172.21.15.64:0/2592454273 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa9d4068a90 con 0x7fa9d413f7e0 2023-09-27T23:52:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa9c0060a70 msgr2=0x7fa9c0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa9c0060a70 0x7fa9c0062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006f00 tx=0x7fa9d0008040 comp rx=0 tx=0).stop 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 msgr2=0x7fa9d413b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 0x7fa9d413b320 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40094c0 tx=0x7fa9c4017000 comp rx=0 tx=0).stop 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 shutdown_connections 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa9c0060a70 0x7fa9c0062f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d413fbb0 0x7fa9d4069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d413f7e0 0x7fa9d413b320 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 --2- 172.21.15.64:0/2592454273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9d40cba00 0x7fa9d4155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 >> 172.21.15.64:0/2592454273 conn(0x7fa9d40cd8f0 msgr2=0x7fa9d413c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 shutdown_connections 2023-09-27T23:52:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.296+0000 7fa9dabd3700 1 -- 172.21.15.64:0/2592454273 wait complete. 2023-09-27T23:52:20.305 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 24 and data pool 25 2023-09-27T23:52:20.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.408+0000 7ff81579f700 1 Processor -- start 2023-09-27T23:52:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.408+0000 7ff81579f700 1 -- start start 2023-09-27T23:52:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.408+0000 7ff81579f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff810155960 0x7ff81015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8100c7f00 0x7ff8100aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81006d990 con 0x7ff81006de20 2023-09-27T23:52:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81015a570 con 0x7ff8100c7f00 2023-09-27T23:52:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81006cd80 con 0x7ff810155960 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8100c7f00 0x7ff8100aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff80f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff81006de20 0x7ff8100c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55900/0 (socket says 172.21.15.64:55900) 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8100c7f00 0x7ff8100aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37740/0 (socket says 172.21.15.64:37740) 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff80f7fe700 1 -- 172.21.15.64:0/2303035283 learned_addr learned my addr 172.21.15.64:0/2303035283 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff80ffff700 1 --2- 172.21.15.64:0/2303035283 >> v2:172.21.15.123:3300/0 conn(0x7ff810155960 0x7ff81015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 -- 172.21.15.64:0/2303035283 >> v2:172.21.15.123:3300/0 conn(0x7ff810155960 msgr2=0x7ff81015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 --2- 172.21.15.64:0/2303035283 >> v2:172.21.15.123:3300/0 conn(0x7ff810155960 0x7ff81015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 -- 172.21.15.64:0/2303035283 >> v1:172.21.15.64:6789/0 conn(0x7ff81006de20 legacy=0x7ff8100c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 -- 172.21.15.64:0/2303035283 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81006d060 con 0x7ff8100c7f00 2023-09-27T23:52:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff814f9e700 1 --2- 172.21.15.64:0/2303035283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100aca10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7ff810064070 tx=0x7ff80800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b7d201c39a1075 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff80effd700 1 -- 172.21.15.64:0/2303035283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808010070 con 0x7ff8100c7f00 2023-09-27T23:52:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff80effd700 1 -- 172.21.15.64:0/2303035283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80800b2e0 con 0x7ff8100c7f00 2023-09-27T23:52:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff80effd700 1 -- 172.21.15.64:0/2303035283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80800dd50 con 0x7ff8100c7f00 2023-09-27T23:52:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- 172.21.15.64:0/2303035283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 msgr2=0x7ff8100aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 --2- 172.21.15.64:0/2303035283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100aca10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7ff810064070 tx=0x7ff80800b990 comp rx=0 tx=0).stop 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- 172.21.15.64:0/2303035283 shutdown_connections 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 --2- 172.21.15.64:0/2303035283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100aca10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 --2- 172.21.15.64:0/2303035283 >> v2:172.21.15.123:3300/0 conn(0x7ff810155960 0x7ff81015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- 172.21.15.64:0/2303035283 >> 172.21.15.64:0/2303035283 conn(0x7ff810067bb0 msgr2=0x7ff8100cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- 172.21.15.64:0/2303035283 shutdown_connections 2023-09-27T23:52:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.412+0000 7ff81579f700 1 -- 172.21.15.64:0/2303035283 wait complete. 2023-09-27T23:52:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 Processor -- start 2023-09-27T23:52:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 -- start start 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff81006de20 0x7ff8100c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100c1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff810155960 0x7ff8100b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8101414f0 con 0x7ff8100c7f00 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81013f8f0 con 0x7ff81006de20 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81013ece0 con 0x7ff810155960 2023-09-27T23:52:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100c1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37744/0 (socket says 172.21.15.64:37744) 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 -- 172.21.15.64:0/1034424207 learned_addr learned my addr 172.21.15.64:0/1034424207 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80ffff700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff81006de20 0x7ff8100c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff814f9e700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff810155960 0x7ff8100b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 -- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff810155960 msgr2=0x7ff8100b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff810155960 0x7ff8100b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 -- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff81006de20 msgr2=0x7ff8100c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff81006de20 0x7ff8100c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 -- 172.21.15.64:0/1034424207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80800bc30 con 0x7ff8100c7f00 2023-09-27T23:52:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff80f7fe700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100c1200 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c7c0 tx=0x7ff800007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800005b30 con 0x7ff8100c7f00 2023-09-27T23:52:20.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800005c90 con 0x7ff8100c7f00 2023-09-27T23:52:20.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800004a80 con 0x7ff8100c7f00 2023-09-27T23:52:20.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81013efc0 con 0x7ff8100c7f00 2023-09-27T23:52:20.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.416+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81013f580 con 0x7ff8100c7f00 2023-09-27T23:52:20.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.420+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8100ca6f0 con 0x7ff8100c7f00 2023-09-27T23:52:20.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.420+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff8000070d0 con 0x7ff8100c7f00 2023-09-27T23:52:20.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.424+0000 7ff7f7fff700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff7fc060a70 0x7ff7fc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.424+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff80007cf40 con 0x7ff8100c7f00 2023-09-27T23:52:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.424+0000 7ff80ffff700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff7fc060a70 0x7ff7fc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.424+0000 7ff80ffff700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff7fc060a70 0x7ff7fc062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff804009c80 tx=0x7ff8040091a0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.428+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff80004cf50 con 0x7ff8100c7f00 2023-09-27T23:52:20.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.588+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8100ca6f0 con 0x7ff8100c7f00 2023-09-27T23:52:20.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.592+0000 7ff7f7fff700 1 -- 172.21.15.64:0/1034424207 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17098 (secure 0 0 0) 0x7ff8100ca6f0 con 0x7ff8100c7f00 2023-09-27T23:52:20.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:20.598 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":188,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:20.268021+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:52:18.850130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-09-27T23:52:19.262770+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff7fc060a70 msgr2=0x7ff7fc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff7fc060a70 0x7ff7fc062f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff804009c80 tx=0x7ff8040091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 msgr2=0x7ff8100c1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100c1200 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c7c0 tx=0x7ff800007f70 comp rx=0 tx=0).stop 2023-09-27T23:52:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 shutdown_connections 2023-09-27T23:52:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff7fc060a70 0x7ff7fc062f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff810155960 0x7ff8100b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8100c7f00 0x7ff8100c1200 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 --2- 172.21.15.64:0/1034424207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff81006de20 0x7ff8100c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 >> 172.21.15.64:0/1034424207 conn(0x7ff810067bb0 msgr2=0x7ff8100aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 shutdown_connections 2023-09-27T23:52:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.596+0000 7ff81579f700 1 -- 172.21.15.64:0/1034424207 wait complete. 2023-09-27T23:52:20.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.708+0000 7f831b59e700 1 Processor -- start 2023-09-27T23:52:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.708+0000 7f831b59e700 1 -- start start 2023-09-27T23:52:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.708+0000 7f831b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f831c158e70 0x7f831c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.708+0000 7f831b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c0ca5a0 con 0x7f831c0cc470 2023-09-27T23:52:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.708+0000 7f831b59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c06dac0 con 0x7f831c06a540 2023-09-27T23:52:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.708+0000 7f831b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c0b0260 con 0x7f831c158e70 2023-09-27T23:52:20.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831ad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f831c0cc470 0x7f831c0cc890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55914/0 (socket says 172.21.15.64:55914) 2023-09-27T23:52:20.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831ad9d700 1 -- 172.21.15.64:0/803327763 learned_addr learned my addr 172.21.15.64:0/803327763 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:20.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831a59c700 1 --2- 172.21.15.64:0/803327763 >> v2:172.21.15.123:3300/0 conn(0x7f831c158e70 0x7f831c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918074719 0 0) 0x7f831c0ca5a0 con 0x7f831c0cc470 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8304003570 con 0x7f831c0cc470 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495573047 0 0) 0x7f831c06dac0 con 0x7f831c06a540 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f831c0ca5a0 con 0x7f831c06a540 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831a59c700 1 -- 172.21.15.64:0/803327763 >> v1:172.21.15.123:6789/0 conn(0x7f831c06a540 legacy=0x7f831c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831a59c700 1 -- 172.21.15.64:0/803327763 >> v1:172.21.15.64:6789/0 conn(0x7f831c0cc470 legacy=0x7f831c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831a59c700 1 -- 172.21.15.64:0/803327763 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c06d220 con 0x7f831c158e70 2023-09-27T23:52:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831a59c700 1 --2- 172.21.15.64:0/803327763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c0afd20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f83100098e0 tx=0x7f83100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120e31db7b9e101f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310011630 con 0x7f831c158e70 2023-09-27T23:52:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8310011790 con 0x7f831c158e70 2023-09-27T23:52:20.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831959a700 1 -- 172.21.15.64:0/803327763 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310011b30 con 0x7f831c158e70 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 -- 172.21.15.64:0/803327763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 msgr2=0x7f831c0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 --2- 172.21.15.64:0/803327763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c0afd20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f83100098e0 tx=0x7f83100091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 -- 172.21.15.64:0/803327763 shutdown_connections 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 --2- 172.21.15.64:0/803327763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c0afd20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 -- 172.21.15.64:0/803327763 >> 172.21.15.64:0/803327763 conn(0x7f831c0670e0 msgr2=0x7f831c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 -- 172.21.15.64:0/803327763 shutdown_connections 2023-09-27T23:52:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.712+0000 7f831b59e700 1 -- 172.21.15.64:0/803327763 wait complete. 2023-09-27T23:52:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 Processor -- start 2023-09-27T23:52:20.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 -- start start 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 0x7f831c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f831c0cc470 0x7f831c15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c3e00 con 0x7f831c06a540 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c40e0 con 0x7f831c158e70 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831b59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c43c0 con 0x7f831c0cc470 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 0x7f831c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 0x7f831c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37746/0 (socket says 172.21.15.64:37746) 2023-09-27T23:52:20.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 -- 172.21.15.64:0/1837710124 learned_addr learned my addr 172.21.15.64:0/1837710124 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:20.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831ad9d700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f8319d9b700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f831c0cc470 0x7f831c15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 -- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f831c0cc470 msgr2=0x7f831c15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f831c0cc470 0x7f831c15b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 -- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 msgr2=0x7f831c14de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.716+0000 7f831a59c700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c14de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f831a59c700 1 -- 172.21.15.64:0/1837710124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310009420 con 0x7f831c06a540 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f831ad9d700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f831a59c700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 0x7f831c1568c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f831c0cfb30 tx=0x7f8310017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831001f070 con 0x7f831c06a540 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8310000bc0 con 0x7f831c06a540 2023-09-27T23:52:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c0c46a0 con 0x7f831c06a540 2023-09-27T23:52:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310025420 con 0x7f831c06a540 2023-09-27T23:52:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c0c3ad0 con 0x7f831c06a540 2023-09-27T23:52:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c04be40 con 0x7f831c06a540 2023-09-27T23:52:20.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.720+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8310023070 con 0x7f831c06a540 2023-09-27T23:52:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.724+0000 7f830a7fc700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f82fc060a70 0x7f82fc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.724+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f831007e3d0 con 0x7f831c06a540 2023-09-27T23:52:20.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.724+0000 7f8319d9b700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f82fc060a70 0x7f82fc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:20.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.724+0000 7f8319d9b700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f82fc060a70 0x7f82fc062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f831c0c6970 tx=0x7f830c007d50 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:20.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.728+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f831004e400 con 0x7f831c06a540 2023-09-27T23:52:20.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.892+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f831c06bb10 con 0x7f831c06a540 2023-09-27T23:52:20.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.900+0000 7f830a7fc700 1 -- 172.21.15.64:0/1837710124 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17098 (secure 0 0 0) 0x7f8310051a20 con 0x7f831c06a540 2023-09-27T23:52:20.908 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:20.908 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":188,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:20.268021+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:52:18.850130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-09-27T23:52:19.262770+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f82fc060a70 msgr2=0x7f82fc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f82fc060a70 0x7f82fc062f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f831c0c6970 tx=0x7f830c007d50 comp rx=0 tx=0).stop 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 msgr2=0x7f831c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 0x7f831c1568c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f831c0cfb30 tx=0x7f8310017000 comp rx=0 tx=0).stop 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 shutdown_connections 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f82fc060a70 0x7f82fc062f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f831c158e70 0x7f831c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f831c0cc470 0x7f831c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 --2- 172.21.15.64:0/1837710124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f831c06a540 0x7f831c1568c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:20.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 >> 172.21.15.64:0/1837710124 conn(0x7f831c0670e0 msgr2=0x7f831c0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:20.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 shutdown_connections 2023-09-27T23:52:20.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:20.904+0000 7f831b59e700 1 -- 172.21.15.64:0/1837710124 wait complete. 2023-09-27T23:52:20.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-27T23:52:21.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.036+0000 7fb020d34700 1 Processor -- start 2023-09-27T23:52:21.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.036+0000 7fb020d34700 1 -- start start 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb01c0cb290 0x7fb01c158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c06bb50 0x7fb01c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c06b6c0 con 0x7fb01c158b00 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c15d880 con 0x7fb01c0cb290 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c06aab0 con 0x7fb01c06bb50 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb01c0cb290 0x7fb01c158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb01c0cb290 0x7fb01c158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37754/0 (socket says 172.21.15.64:37754) 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 -- 172.21.15.64:0/548916706 learned_addr learned my addr 172.21.15.64:0/548916706 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb01b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb01c158b00 0x7fb01c0cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55920/0 (socket says 172.21.15.64:55920) 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb01bfff700 1 --2- 172.21.15.64:0/548916706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c06bb50 0x7fb01c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 -- 172.21.15.64:0/548916706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c06bb50 msgr2=0x7fb01c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 --2- 172.21.15.64:0/548916706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c06bb50 0x7fb01c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 -- 172.21.15.64:0/548916706 >> v1:172.21.15.64:6789/0 conn(0x7fb01c158b00 legacy=0x7fb01c0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 -- 172.21.15.64:0/548916706 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c06ad90 con 0x7fb01c0cb290 2023-09-27T23:52:21.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb013fff700 1 --2- 172.21.15.64:0/548916706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c0cb290 0x7fb01c158ed0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb0040099b0 tx=0x7fb004009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fe87d6d3195efb server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:21.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb01affd700 1 -- 172.21.15.64:0/548916706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400b860 con 0x7fb01c0cb290 2023-09-27T23:52:21.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb01affd700 1 -- 172.21.15.64:0/548916706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00400b9c0 con 0x7fb01c0cb290 2023-09-27T23:52:21.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb01affd700 1 -- 172.21.15.64:0/548916706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400bd60 con 0x7fb01c0cb290 2023-09-27T23:52:21.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- 172.21.15.64:0/548916706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c0cb290 msgr2=0x7fb01c158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 --2- 172.21.15.64:0/548916706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c0cb290 0x7fb01c158ed0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb0040099b0 tx=0x7fb004009250 comp rx=0 tx=0).stop 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- 172.21.15.64:0/548916706 shutdown_connections 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 --2- 172.21.15.64:0/548916706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c06bb50 0x7fb01c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 --2- 172.21.15.64:0/548916706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c0cb290 0x7fb01c158ed0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- 172.21.15.64:0/548916706 >> 172.21.15.64:0/548916706 conn(0x7fb01c067bb0 msgr2=0x7fb01c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- 172.21.15.64:0/548916706 shutdown_connections 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.040+0000 7fb020d34700 1 -- 172.21.15.64:0/548916706 wait complete. 2023-09-27T23:52:21.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 Processor -- start 2023-09-27T23:52:21.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 -- start start 2023-09-27T23:52:21.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01c06bb50 0x7fb01c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c0cb290 0x7fb01c14d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 0x7fb01c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b2d60 con 0x7fb01c158b00 2023-09-27T23:52:21.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b3320 con 0x7fb01c06bb50 2023-09-27T23:52:21.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb020d34700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c156f80 con 0x7fb01c0cb290 2023-09-27T23:52:21.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 0x7fb01c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb01bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 0x7fb01c0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37768/0 (socket says 172.21.15.64:37768) 2023-09-27T23:52:21.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb01bfff700 1 -- 172.21.15.64:0/4116493847 learned_addr learned my addr 172.21.15.64:0/4116493847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:21.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb013fff700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c0cb290 0x7fb01c14d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb01b7fe700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01c06bb50 0x7fb01c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.044+0000 7fb01bfff700 1 -- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c0cb290 msgr2=0x7fb01c14d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb01bfff700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c0cb290 0x7fb01c14d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb01bfff700 1 -- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01c06bb50 msgr2=0x7fb01c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb01bfff700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01c06bb50 0x7fb01c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb01bfff700 1 -- 172.21.15.64:0/4116493847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040094f0 con 0x7fb01c158b00 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb01bfff700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 0x7fb01c0b0990 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fb014007f60 tx=0x7fb0140098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400eb90 con 0x7fb01c158b00 2023-09-27T23:52:21.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01400ecf0 con 0x7fb01c158b00 2023-09-27T23:52:21.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c157b90 con 0x7fb01c158b00 2023-09-27T23:52:21.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014014670 con 0x7fb01c158b00 2023-09-27T23:52:21.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c158150 con 0x7fb01c158b00 2023-09-27T23:52:21.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0b34b0 con 0x7fb01c158b00 2023-09-27T23:52:21.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.048+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb014014a60 con 0x7fb01c158b00 2023-09-27T23:52:21.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.052+0000 7fb0137fe700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb000060a70 0x7fb000062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.052+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb01407d6f0 con 0x7fb01c158b00 2023-09-27T23:52:21.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.052+0000 7fb01b7fe700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb000060a70 0x7fb000062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.052+0000 7fb01b7fe700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb000060a70 0x7fb000062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb00c004fd0 tx=0x7fb00c007eb0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:21.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.060+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb01404d720 con 0x7fb01c158b00 2023-09-27T23:52:21.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.256+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb01c04be40 con 0x7fb01c158b00 2023-09-27T23:52:21.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.276+0000 7fb0137fe700 1 -- 172.21.15.64:0/4116493847 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb014050d40 con 0x7fb01c158b00 2023-09-27T23:52:21.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb000060a70 msgr2=0x7fb000062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb000060a70 0x7fb000062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb00c004fd0 tx=0x7fb00c007eb0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 msgr2=0x7fb01c0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 0x7fb01c0b0990 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fb014007f60 tx=0x7fb0140098e0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 shutdown_connections 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb000060a70 0x7fb000062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c158b00 0x7fb01c0b0990 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb01c0cb290 0x7fb01c14d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 --2- 172.21.15.64:0/4116493847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb01c06bb50 0x7fb01c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 >> 172.21.15.64:0/4116493847 conn(0x7fb01c067bb0 msgr2=0x7fb01c155660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:21.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 shutdown_connections 2023-09-27T23:52:21.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.280+0000 7fb020d34700 1 -- 172.21.15.64:0/4116493847 wait complete. 2023-09-27T23:52:21.300 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-27T23:52:21.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-27T23:52:21.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 Processor -- start 2023-09-27T23:52:21.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 -- start start 2023-09-27T23:52:21.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 0x7f44900cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f449006d050 0x7f449006d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449006d9b0 con 0x7f4490158ed0 2023-09-27T23:52:21.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900cca20 con 0x7f449006d050 2023-09-27T23:52:21.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.400+0000 7f449702d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900ca280 con 0x7f44901592a0 2023-09-27T23:52:21.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 0x7f44900cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f44955ca700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f449006d050 0x7f449006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f4494dc9700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4490158ed0 0x7f449015cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:44316/0 (socket says 172.21.15.64:44316) 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f4494dc9700 1 -- 172.21.15.64:0/128845623 learned_addr learned my addr 172.21.15.64:0/128845623 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 0x7f44900cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.64:43132/0 (socket says 172.21.15.64:43132) 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f44955ca700 1 -- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 msgr2=0x7f44900cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f44955ca700 1 --2- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 0x7f44900cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f44955ca700 1 -- 172.21.15.64:0/128845623 >> v1:172.21.15.123:6789/0 conn(0x7f4490158ed0 legacy=0x7f449015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448f7fe700 1 -- 172.21.15.64:0/128845623 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566699488 0 0) 0x7f449006d9b0 con 0x7f4490158ed0 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f44955ca700 1 -- 172.21.15.64:0/128845623 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449015d7c0 con 0x7f449006d050 2023-09-27T23:52:21.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448ffff700 1 --2- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 0x7f44900cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:21.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f44955ca700 1 --2- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449006d050 0x7f449006d470 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e60 tx=0x7f4488009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445c1fcb66e75e65 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448f7fe700 1 -- 172.21.15.64:0/128845623 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800e070 con 0x7f449006d050 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448f7fe700 1 -- 172.21.15.64:0/128845623 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800bb60 con 0x7f449006d050 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f448f7fe700 1 -- 172.21.15.64:0/128845623 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488012430 con 0x7f449006d050 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 -- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449006d050 msgr2=0x7f449006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 --2- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449006d050 0x7f449006d470 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e60 tx=0x7f4488009780 comp rx=0 tx=0).stop 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 -- 172.21.15.64:0/128845623 shutdown_connections 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 --2- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449006d050 0x7f449006d470 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 --2- 172.21.15.64:0/128845623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44901592a0 0x7f44900cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 -- 172.21.15.64:0/128845623 >> 172.21.15.64:0/128845623 conn(0x7f44900cd8f0 msgr2=0x7f44900c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 -- 172.21.15.64:0/128845623 shutdown_connections 2023-09-27T23:52:21.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.404+0000 7f449702d700 1 -- 172.21.15.64:0/128845623 wait complete. 2023-09-27T23:52:21.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 Processor -- start 2023-09-27T23:52:21.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 -- start start 2023-09-27T23:52:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f449006d050 0x7f4490153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 0x7f44900b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44901592a0 0x7f44901574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490153bb0 con 0x7f4490158ed0 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490067330 con 0x7f44901592a0 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f449702d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900b0f90 con 0x7f449006d050 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 0x7f44900b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 0x7f44900b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37772/0 (socket says 172.21.15.64:37772) 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 -- 172.21.15.64:0/2557862474 learned_addr learned my addr 172.21.15.64:0/2557862474 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f44955ca700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44901592a0 0x7f44901574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f4494dc9700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f449006d050 0x7f4490153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 -- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f449006d050 msgr2=0x7f4490153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f449006d050 0x7f4490153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 -- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44901592a0 msgr2=0x7f44901574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:21.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44901592a0 0x7f44901574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 -- 172.21.15.64:0/2557862474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009420 con 0x7f4490158ed0 2023-09-27T23:52:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448ffff700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 0x7f44900b05d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f448000ec80 tx=0x7f44800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:21.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.408+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000c920 con 0x7f4490158ed0 2023-09-27T23:52:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448000ca80 con 0x7f4490158ed0 2023-09-27T23:52:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4488009a20 con 0x7f4490158ed0 2023-09-27T23:52:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000ce20 con 0x7f4490158ed0 2023-09-27T23:52:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44900b1550 con 0x7f4490158ed0 2023-09-27T23:52:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449006b1f0 con 0x7f4490158ed0 2023-09-27T23:52:21.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4480013680 con 0x7f4490158ed0 2023-09-27T23:52:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f448cff9700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4470060a70 0x7f4470062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.412+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f448007d5b0 con 0x7f4490158ed0 2023-09-27T23:52:21.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.416+0000 7f4494dc9700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4470060a70 0x7f4470062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:21.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.416+0000 7f4494dc9700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4470060a70 0x7f4470062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f447800c8f0 tx=0x7f4478009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:21.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.428+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f449006b1f0 con 0x7f4490158ed0 2023-09-27T23:52:21.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:21.624+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f449006db90 con 0x7f4490158ed0 2023-09-27T23:52:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.296+0000 7f448cff9700 1 -- 172.21.15.64:0/2557862474 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7f449006db90 con 0x7f4490158ed0 2023-09-27T23:52:22.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4470060a70 msgr2=0x7f4470062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4470060a70 0x7f4470062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f447800c8f0 tx=0x7f4478009040 comp rx=0 tx=0).stop 2023-09-27T23:52:22.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 msgr2=0x7f44900b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 0x7f44900b05d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f448000ec80 tx=0x7f44800091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 shutdown_connections 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4470060a70 0x7f4470062f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44901592a0 0x7f44901574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4490158ed0 0x7f44900b05d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 --2- 172.21.15.64:0/2557862474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f449006d050 0x7f4490153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 >> 172.21.15.64:0/2557862474 conn(0x7f44900cd8f0 msgr2=0x7f44900c1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 shutdown_connections 2023-09-27T23:52:22.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.300+0000 7f449702d700 1 -- 172.21.15.64:0/2557862474 wait complete. 2023-09-27T23:52:22.321 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-27T23:52:22.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:22.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.428+0000 7fe9611f7700 1 Processor -- start 2023-09-27T23:52:22.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.428+0000 7fe9611f7700 1 -- start start 2023-09-27T23:52:22.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c06d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c06c7c0 con 0x7fe95c0b1410 2023-09-27T23:52:22.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c06cd80 con 0x7fe95c06a580 2023-09-27T23:52:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c06b8f0 con 0x7fe95c06aa00 2023-09-27T23:52:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe95c0b1410 0x7fe95c0ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55926/0 (socket says 172.21.15.64:55926) 2023-09-27T23:52:22.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95bfff700 1 -- 172.21.15.64:0/2829949356 learned_addr learned my addr 172.21.15.64:0/2829949356 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9609f6700 1 --2- 172.21.15.64:0/2829949356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c06d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622759082 0 0) 0x7fe95c06c7c0 con 0x7fe95c0b1410 2023-09-27T23:52:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe944003570 con 0x7fe95c0b1410 2023-09-27T23:52:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197359356 0 0) 0x7fe95c06cd80 con 0x7fe95c06a580 2023-09-27T23:52:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe95c06c7c0 con 0x7fe95c06a580 2023-09-27T23:52:22.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9609f6700 1 -- 172.21.15.64:0/2829949356 >> v1:172.21.15.123:6789/0 conn(0x7fe95c06a580 legacy=0x7fe95c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9609f6700 1 -- 172.21.15.64:0/2829949356 >> v1:172.21.15.64:6789/0 conn(0x7fe95c0b1410 legacy=0x7fe95c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9609f6700 1 -- 172.21.15.64:0/2829949356 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0ac1e0 con 0x7fe95c06aa00 2023-09-27T23:52:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9609f6700 1 --2- 172.21.15.64:0/2829949356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c06d920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe954009e90 tx=0x7fe954009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14ad16fc8d0418c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400e070 con 0x7fe95c06aa00 2023-09-27T23:52:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400bb80 con 0x7fe95c06aa00 2023-09-27T23:52:22.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe95affd700 1 -- 172.21.15.64:0/2829949356 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954012430 con 0x7fe95c06aa00 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- 172.21.15.64:0/2829949356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 msgr2=0x7fe95c06d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 --2- 172.21.15.64:0/2829949356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c06d920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe954009e90 tx=0x7fe954009780 comp rx=0 tx=0).stop 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- 172.21.15.64:0/2829949356 shutdown_connections 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 --2- 172.21.15.64:0/2829949356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c06d920 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- 172.21.15.64:0/2829949356 >> 172.21.15.64:0/2829949356 conn(0x7fe95c067bb0 msgr2=0x7fe95c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- 172.21.15.64:0/2829949356 shutdown_connections 2023-09-27T23:52:22.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.432+0000 7fe9611f7700 1 -- 172.21.15.64:0/2829949356 wait complete. 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 Processor -- start 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 -- start start 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe95c06a580 0x7fe95c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c0c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 0x7fe95c0c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c0860 con 0x7fe95c0b1410 2023-09-27T23:52:22.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c13e0 con 0x7fe95c06a580 2023-09-27T23:52:22.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c1549d0 con 0x7fe95c06aa00 2023-09-27T23:52:22.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 0x7fe95c0c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 0x7fe95c0c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37786/0 (socket says 172.21.15.64:37786) 2023-09-27T23:52:22.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 -- 172.21.15.64:0/468835031 learned_addr learned my addr 172.21.15.64:0/468835031 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe95b7fe700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c0c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe95bfff700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe95c06a580 0x7fe95c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 -- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 msgr2=0x7fe95c0c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c0c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 -- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe95c06a580 msgr2=0x7fe95c154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe95c06a580 0x7fe95c154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 -- 172.21.15.64:0/468835031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954009a20 con 0x7fe95c0b1410 2023-09-27T23:52:22.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9609f6700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 0x7fe95c0c24c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0cc310 tx=0x7fe954019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:22.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9540263f0 con 0x7fe95c0b1410 2023-09-27T23:52:22.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe954026550 con 0x7fe95c0b1410 2023-09-27T23:52:22.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0c0ab0 con 0x7fe95c0b1410 2023-09-27T23:52:22.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954026910 con 0x7fe95c0b1410 2023-09-27T23:52:22.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.436+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0c1070 con 0x7fe95c0b1410 2023-09-27T23:52:22.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.440+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0c1550 con 0x7fe95c0b1410 2023-09-27T23:52:22.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.440+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe954026d00 con 0x7fe95c0b1410 2023-09-27T23:52:22.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.440+0000 7fe943fff700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe948060aa0 0x7fe948062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.440+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe95407fa80 con 0x7fe95c0b1410 2023-09-27T23:52:22.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.440+0000 7fe95bfff700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe948060aa0 0x7fe948062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.440+0000 7fe95bfff700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe948060aa0 0x7fe948062f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe9500058d0 tx=0x7fe950009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:22.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.448+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe9540534b0 con 0x7fe95c0b1410 2023-09-27T23:52:22.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-09-27T23:52:22.511 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-09-27T23:52:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.648+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe95c04be40 con 0x7fe95c0b1410 2023-09-27T23:52:22.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.648+0000 7fe943fff700 1 -- 172.21.15.64:0/468835031 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4610 (secure 0 0 0) 0x7fe954053040 con 0x7fe95c0b1410 2023-09-27T23:52:22.654 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:22.654 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:52:20.268986+0000","modified":"2023-09-27T23:52:22.285887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10144":{"gid":10144,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/1929792685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1929792685},{"type":"v1","addr":"172.21.15.123:6835","nonce":1929792685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-09-27T23:52:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe948060aa0 msgr2=0x7fe948062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe948060aa0 0x7fe948062f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe9500058d0 tx=0x7fe950009040 comp rx=0 tx=0).stop 2023-09-27T23:52:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 msgr2=0x7fe95c0c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 0x7fe95c0c24c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0cc310 tx=0x7fe954019000 comp rx=0 tx=0).stop 2023-09-27T23:52:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 shutdown_connections 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe948060aa0 0x7fe948062f50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe95c0b1410 0x7fe95c0c24c0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe95c06aa00 0x7fe95c0c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 --2- 172.21.15.64:0/468835031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe95c06a580 0x7fe95c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 >> 172.21.15.64:0/468835031 conn(0x7fe95c067bb0 msgr2=0x7fe95c0cac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 shutdown_connections 2023-09-27T23:52:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.652+0000 7fe9611f7700 1 -- 172.21.15.64:0/468835031 wait complete. 2023-09-27T23:52:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 72 2023-09-27T23:52:22.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1759e700 1 Processor -- start 2023-09-27T23:52:22.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1759e700 1 -- start start 2023-09-27T23:52:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c1806aa00 0x7f5c1806d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1806c7c0 con 0x7f5c180b1410 2023-09-27T23:52:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1806cd80 con 0x7f5c1806a580 2023-09-27T23:52:22.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1806b8f0 con 0x7f5c1806aa00 2023-09-27T23:52:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.768+0000 7f5c1659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5c180b1410 0x7f5c180ab440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55934/0 (socket says 172.21.15.64:55934) 2023-09-27T23:52:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1659c700 1 -- 172.21.15.64:0/3539660973 learned_addr learned my addr 172.21.15.64:0/3539660973 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c16d9d700 1 --2- 172.21.15.64:0/3539660973 >> v2:172.21.15.123:3300/0 conn(0x7f5c1806aa00 0x7f5c1806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1559a700 1 -- 172.21.15.64:0/3539660973 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665410898 0 0) 0x7f5c1806c7c0 con 0x7f5c180b1410 2023-09-27T23:52:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1559a700 1 -- 172.21.15.64:0/3539660973 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c00003570 con 0x7f5c180b1410 2023-09-27T23:52:22.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c16d9d700 1 -- 172.21.15.64:0/3539660973 >> v1:172.21.15.123:6789/0 conn(0x7f5c1806a580 legacy=0x7f5c1806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c16d9d700 1 -- 172.21.15.64:0/3539660973 >> v1:172.21.15.64:6789/0 conn(0x7f5c180b1410 legacy=0x7f5c180ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c16d9d700 1 -- 172.21.15.64:0/3539660973 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180ac1e0 con 0x7f5c1806aa00 2023-09-27T23:52:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c16d9d700 1 --2- 172.21.15.64:0/3539660973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806aa00 0x7f5c1806d920 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5c100098e0 tx=0x7f5c100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb20ec6690a4527 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1559a700 1 -- 172.21.15.64:0/3539660973 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1000b640 con 0x7f5c1806aa00 2023-09-27T23:52:22.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1559a700 1 -- 172.21.15.64:0/3539660973 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1000b7a0 con 0x7f5c1806aa00 2023-09-27T23:52:22.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1559a700 1 -- 172.21.15.64:0/3539660973 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1000bb20 con 0x7f5c1806aa00 2023-09-27T23:52:22.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 -- 172.21.15.64:0/3539660973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806aa00 msgr2=0x7f5c1806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 --2- 172.21.15.64:0/3539660973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806aa00 0x7f5c1806d920 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5c100098e0 tx=0x7f5c100091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 -- 172.21.15.64:0/3539660973 shutdown_connections 2023-09-27T23:52:22.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 --2- 172.21.15.64:0/3539660973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806aa00 0x7f5c1806d920 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 -- 172.21.15.64:0/3539660973 >> 172.21.15.64:0/3539660973 conn(0x7f5c18067bb0 msgr2=0x7f5c180cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:22.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 -- 172.21.15.64:0/3539660973 shutdown_connections 2023-09-27T23:52:22.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.772+0000 7f5c1759e700 1 -- 172.21.15.64:0/3539660973 wait complete. 2023-09-27T23:52:22.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 Processor -- start 2023-09-27T23:52:22.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 -- start start 2023-09-27T23:52:22.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806a580 0x7f5c18159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 0x7f5c18143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c180b1410 0x7f5c180c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18159700 con 0x7f5c1806aa00 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c181692a0 con 0x7f5c1806a580 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18158af0 con 0x7f5c180b1410 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 0x7f5c18143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 0x7f5c18143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37796/0 (socket says 172.21.15.64:37796) 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 -- 172.21.15.64:0/1402717572 learned_addr learned my addr 172.21.15.64:0/1402717572 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1659c700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806a580 0x7f5c18159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c16d9d700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c180b1410 0x7f5c180c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 -- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c180b1410 msgr2=0x7f5c180c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c180b1410 0x7f5c180c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 -- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806a580 msgr2=0x7f5c18159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806a580 0x7f5c18159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 -- 172.21.15.64:0/1402717572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10009420 con 0x7f5c1806aa00 2023-09-27T23:52:22.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c15d9b700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 0x7f5c18143d90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800cb40 tx=0x7f5c08001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:22.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08002060 con 0x7f5c1806aa00 2023-09-27T23:52:22.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800a7e0 con 0x7f5c1806aa00 2023-09-27T23:52:22.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c18158dd0 con 0x7f5c1806aa00 2023-09-27T23:52:22.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800aba0 con 0x7f5c1806aa00 2023-09-27T23:52:22.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.776+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c18159390 con 0x7f5c1806aa00 2023-09-27T23:52:22.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.780+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c18144330 con 0x7f5c1806aa00 2023-09-27T23:52:22.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.780+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f5c08003030 con 0x7f5c1806aa00 2023-09-27T23:52:22.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.780+0000 7f5c067fc700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5bf8060a70 0x7f5bf8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:22.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.780+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5c0807d0d0 con 0x7f5c1806aa00 2023-09-27T23:52:22.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.780+0000 7f5c1659c700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5bf8060a70 0x7f5bf8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:22.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.780+0000 7f5c1659c700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5bf8060a70 0x7f5bf8062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c007fd0 tx=0x7f5c0c009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:22.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.792+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5c0804d060 con 0x7f5c1806aa00 2023-09-27T23:52:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.952+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c1804be40 con 0x7f5c1806aa00 2023-09-27T23:52:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.952+0000 7f5c067fc700 1 -- 172.21.15.64:0/1402717572 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+17085 (secure 0 0 0) 0x7f5c08050680 con 0x7f5c1806aa00 2023-09-27T23:52:22.960 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:22.960 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":189,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:21.273084+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:52:18.850130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-09-27T23:52:19.262770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5bf8060a70 msgr2=0x7f5bf8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5bf8060a70 0x7f5bf8062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c007fd0 tx=0x7f5c0c009040 comp rx=0 tx=0).stop 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 msgr2=0x7f5c18143d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 0x7f5c18143d90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800cb40 tx=0x7f5c08001770 comp rx=0 tx=0).stop 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 shutdown_connections 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5bf8060a70 0x7f5bf8062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c180b1410 0x7f5c180c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1806aa00 0x7f5c18143d90 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 --2- 172.21.15.64:0/1402717572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c1806a580 0x7f5c18159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 >> 172.21.15.64:0/1402717572 conn(0x7f5c18067bb0 msgr2=0x7f5c180cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 shutdown_connections 2023-09-27T23:52:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:22.956+0000 7f5c1759e700 1 -- 172.21.15.64:0/1402717572 wait complete. 2023-09-27T23:52:22.975 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-27T23:52:22.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.076+0000 7f167f59e700 1 Processor -- start 2023-09-27T23:52:23.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.076+0000 7f167f59e700 1 -- start start 2023-09-27T23:52:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16800ce810 0x7f16800ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168006b730 con 0x7f16801591f0 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168006ab20 con 0x7f1680158e20 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168006ae00 con 0x7f16800ce810 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37804/0 (socket says 172.21.15.64:37804) 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 -- 172.21.15.64:0/1456287700 learned_addr learned my addr 172.21.15.64:0/1456287700 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167ed9d700 1 --2- 172.21.15.64:0/1456287700 >> v2:172.21.15.123:3300/0 conn(0x7f16800ce810 0x7f16800ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 -- 172.21.15.64:0/1456287700 >> v2:172.21.15.123:3300/0 conn(0x7f16800ce810 msgr2=0x7f16800ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 --2- 172.21.15.64:0/1456287700 >> v2:172.21.15.123:3300/0 conn(0x7f16800ce810 0x7f16800ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 -- 172.21.15.64:0/1456287700 >> v1:172.21.15.123:6789/0 conn(0x7f16801591f0 legacy=0x7f16800ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 -- 172.21.15.64:0/1456287700 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168006b0e0 con 0x7f1680158e20 2023-09-27T23:52:23.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167e59c700 1 --2- 172.21.15.64:0/1456287700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1680158e20 0x7f168006a540 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f168006dc00 tx=0x7f1678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3081c3ac6d3fb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167d59a700 1 -- 172.21.15.64:0/1456287700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678009e70 con 0x7f1680158e20 2023-09-27T23:52:23.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167d59a700 1 -- 172.21.15.64:0/1456287700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167800b860 con 0x7f1680158e20 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167d59a700 1 -- 172.21.15.64:0/1456287700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167800bc00 con 0x7f1680158e20 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- 172.21.15.64:0/1456287700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1680158e20 msgr2=0x7f168006a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 --2- 172.21.15.64:0/1456287700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1680158e20 0x7f168006a540 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f168006dc00 tx=0x7f1678009250 comp rx=0 tx=0).stop 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- 172.21.15.64:0/1456287700 shutdown_connections 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 --2- 172.21.15.64:0/1456287700 >> v2:172.21.15.123:3300/0 conn(0x7f16800ce810 0x7f16800ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 --2- 172.21.15.64:0/1456287700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1680158e20 0x7f168006a540 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- 172.21.15.64:0/1456287700 >> 172.21.15.64:0/1456287700 conn(0x7f1680067bb0 msgr2=0x7f16800cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- 172.21.15.64:0/1456287700 shutdown_connections 2023-09-27T23:52:23.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.080+0000 7f167f59e700 1 -- 172.21.15.64:0/1456287700 wait complete. 2023-09-27T23:52:23.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 Processor -- start 2023-09-27T23:52:23.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 -- start start 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 0x7f168015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1680158e20 0x7f16800c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16801591f0 0x7f16801436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c73b0 con 0x7f16800ce810 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c81b0 con 0x7f1680158e20 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16800c75a0 con 0x7f16801591f0 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 0x7f168015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 0x7f168015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37814/0 (socket says 172.21.15.64:37814) 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 -- 172.21.15.64:0/3173823289 learned_addr learned my addr 172.21.15.64:0/3173823289 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167dd9b700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1680158e20 0x7f16800c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167ed9d700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16801591f0 0x7f16801436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 -- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16801591f0 msgr2=0x7f16801436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16801591f0 0x7f16801436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 -- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1680158e20 msgr2=0x7f16800c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1680158e20 0x7f16800c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 -- 172.21.15.64:0/3173823289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780094f0 con 0x7f16800ce810 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167dd9b700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1680158e20 0x7f16800c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167e59c700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 0x7f168015bc20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f168006e810 tx=0x7f1678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678011420 con 0x7f16800ce810 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16780115a0 con 0x7f16800ce810 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16800c7880 con 0x7f16800ce810 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678011960 con 0x7f16800ce810 2023-09-27T23:52:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.084+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16800c7e40 con 0x7f16800ce810 2023-09-27T23:52:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.088+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168004be40 con 0x7f16800ce810 2023-09-27T23:52:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.088+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f16780253f0 con 0x7f16800ce810 2023-09-27T23:52:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.088+0000 7f16767fc700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1660060a70 0x7f1660062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.088+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f167807ebb0 con 0x7f16800ce810 2023-09-27T23:52:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.088+0000 7f167dd9b700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1660060a70 0x7f1660062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.088+0000 7f167dd9b700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1660060a70 0x7f1660062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1668007fd0 tx=0x7f1668009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.096+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f167804eb40 con 0x7f16800ce810 2023-09-27T23:52:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.252+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f168006b3c0 con 0x7f1660060a70 2023-09-27T23:52:23.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:52:23.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:52:23.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.272+0000 7f16767fc700 1 -- 172.21.15.64:0/3173823289 <== mgr.9940 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292741 (secure 0 0 0) 0x7f168006b3c0 con 0x7f1660060a70 2023-09-27T23:52:23.277 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1660060a70 msgr2=0x7f1660062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1660060a70 0x7f1660062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1668007fd0 tx=0x7f1668009040 comp rx=0 tx=0).stop 2023-09-27T23:52:23.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 msgr2=0x7f168015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 0x7f168015bc20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f168006e810 tx=0x7f1678017000 comp rx=0 tx=0).stop 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 shutdown_connections 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1660060a70 0x7f1660062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16801591f0 0x7f16801436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1680158e20 0x7f16800c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 --2- 172.21.15.64:0/3173823289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16800ce810 0x7f168015bc20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 >> 172.21.15.64:0/3173823289 conn(0x7f1680067bb0 msgr2=0x7f1680152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 shutdown_connections 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.276+0000 7f167f59e700 1 -- 172.21.15.64:0/3173823289 wait complete. 2023-09-27T23:52:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:23.295 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-09-27T23:52:21.288627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":275016,"kb_used_data":19208,"kb_used_omap":0,"kb_used_meta":255744,"kb_avail":754699704,"statfs":{"total":773094113280,"available":772812496896,"internally_reserved":0,"allocated":19668992,"data_stored":10911184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261881856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008429"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"177'326","reported_seq":1047,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200927+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:17.200927+0000","last_peered":"2023-09-27T23:52:17.200927+0000","last_clean":"2023-09-27T23:52:17.200927+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:17.200927+0000","last_undegraded":"2023-09-27T23:52:17.200927+0000","last_fullsized":"2023-09-27T23:52:17.200927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":326,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710096+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:15.710096+0000","last_peered":"2023-09-27T23:52:15.710096+0000","last_clean":"2023-09-27T23:52:15.710096+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:15.710096+0000","last_undegraded":"2023-09-27T23:52:15.710096+0000","last_fullsized":"2023-09-27T23:52:15.710096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200923+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:17.200923+0000","last_peered":"2023-09-27T23:52:17.200923+0000","last_clean":"2023-09-27T23:52:17.200923+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:17.200923+0000","last_undegraded":"2023-09-27T23:52:17.200923+0000","last_fullsized":"2023-09-27T23:52:17.200923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:16.220162+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:16.220162+0000","last_peered":"2023-09-27T23:52:16.220162+0000","last_clean":"2023-09-27T23:52:16.220162+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:16.220162+0000","last_undegraded":"2023-09-27T23:52:16.220162+0000","last_fullsized":"2023-09-27T23:52:16.220162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710169+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:15.710169+0000","last_peered":"2023-09-27T23:52:15.710169+0000","last_clean":"2023-09-27T23:52:15.710169+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:15.710169+0000","last_undegraded":"2023-09-27T23:52:15.710169+0000","last_fullsized":"2023-09-27T23:52:15.710169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705037+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:15.705037+0000","last_peered":"2023-09-27T23:52:15.705037+0000","last_clean":"2023-09-27T23:52:15.705037+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:15.705037+0000","last_undegraded":"2023-09-27T23:52:15.705037+0000","last_fullsized":"2023-09-27T23:52:15.705037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:17.200922+0000","last_peered":"2023-09-27T23:52:17.200922+0000","last_clean":"2023-09-27T23:52:17.200922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:17.200922+0000","last_undegraded":"2023-09-27T23:52:17.200922+0000","last_fullsized":"2023-09-27T23:52:17.200922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705120+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:15.705120+0000","last_peered":"2023-09-27T23:52:15.705120+0000","last_clean":"2023-09-27T23:52:15.705120+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:15.705120+0000","last_undegraded":"2023-09-27T23:52:15.705120+0000","last_fullsized":"2023-09-27T23:52:15.705120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.833607+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:15.833607+0000","last_peered":"2023-09-27T23:52:15.833607+0000","last_clean":"2023-09-27T23:52:15.833607+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:15.833607+0000","last_undegraded":"2023-09-27T23:52:15.833607+0000","last_fullsized":"2023-09-27T23:52:15.833607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36636,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94335204,"statfs":{"total":96636764160,"available":96599248896,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.89600000000000002},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2250000000000001,"5min":1.569,"15min":1.569},"last":1.446},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.94899999999999995,"15min":0.93400000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.466,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.248,"5min":1.333,"15min":1.3360000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96799999999999997,"15min":0.97499999999999998},"min":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2929999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.2949999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.92500000000000004},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.3069999999999999,"5min":1.341,"15min":1.341},"last":1.004}]},{"osd":3,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.70799999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.202,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0760000000000001,"15min":1.079},"min":{"1min":0.77700000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.3879999999999999},"last":1.23}]},{"osd":4,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.216,"5min":1.821,"15min":1.821},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.03,"15min":1.0109999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.226,"5min":3.02,"15min":3.02},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0569999999999999,"15min":1.0620000000000001},"min":{"1min":0.75,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":1.339},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3759999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.131,"15min":1.143},"min":{"1min":0.749,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.423,"5min":1.423,"15min":1.448},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.145,"15min":1.145},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.381,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.268}]}]},{"osd":6,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30040,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":27776,"kb_avail":94341800,"statfs":{"total":96636764160,"available":96606003200,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28442624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.97199999999999998},"min":{"1min":0.70599999999999996,"5min":0.60599999999999998,"15min":0.53400000000000003},"max":{"1min":1.214,"5min":1.792,"15min":1.792},"last":1.016},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.80400000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90200000000000002,"15min":0.90300000000000002},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67100000000000004,"5min":0.622,"15min":0.57099999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89500000000000002,"15min":0.90600000000000003},"min":{"1min":0.78100000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.147,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.018,"5min":0.98399999999999999,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.78000000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91000000000000003,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.6639999999999999,"5min":1.7,"15min":1.7},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.079,"5min":1.0289999999999999,"15min":1.0029999999999999},"min":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3089999999999999,"5min":1.3700000000000001,"15min":1.633},"last":0.876}]},{"osd":5,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.161,"15min":1.145},"min":{"1min":0.89500000000000002,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.075}]},{"osd":7,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.81100000000000005,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":1.637,"15min":1.679},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.155,"15min":1.1359999999999999},"min":{"1min":0.69199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":37632,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38535168},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.95999999999999996,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.51900000000000002},"max":{"1min":10.058999999999999,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.657,"5min":1.0609999999999999,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.47899999999999998},"max":{"1min":13.757999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.96999999999999997,"15min":0.93200000000000005},"min":{"1min":0.70699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.1970000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.95999999999999996,"15min":0.96099999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.629},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.97699999999999998,"15min":0.95899999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.999,"15min":0.98999999999999999},"min":{"1min":0.84299999999999997,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.075,"15min":1.042},"min":{"1min":0.79000000000000004,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.747}]},{"osd":4,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91200000000000003,"15min":0.88900000000000001},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.04,"5min":1.0469999999999999,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3640000000000001,"5min":1.45,"15min":1.4510000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.119,"15min":1.105},"min":{"1min":0.91900000000000004,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.026}]},{"osd":7,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.222,"15min":1.169},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":6.3710000000000004,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.363,"15min":1.2629999999999999},"min":{"1min":1.0329999999999999,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":14.119,"5min":14.119,"15min":14.119},"last":1.087}]}]},{"osd":4,"up_from":15,"seq":64424509901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30872,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28608,"kb_avail":94340968,"statfs":{"total":96636764160,"available":96605151232,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29294592},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.45000000000000001},"max":{"1min":1.1240000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.1180000000000001,"5min":1.893,"15min":2.2770000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2010000000000001,"5min":1.746,"15min":1.746},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.53500000000000003},"max":{"1min":1.2390000000000001,"5min":1.758,"15min":2.1110000000000002},"last":1.339}]},{"osd":2,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2490000000000001,"5min":1.659,"15min":1.659},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98299999999999998,"15min":0.96099999999999997},"min":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.234,"5min":1.966,"15min":2.056},"last":1.216}]},{"osd":3,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93400000000000005,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.998,"15min":2.032},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.079,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":1.2709999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.5}]},{"osd":5,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.92400000000000004},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.45500000000000002},"max":{"1min":1.25,"5min":1.556,"15min":1.9930000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.006,"15min":1.0149999999999999},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3129999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.419}]},{"osd":6,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0529999999999999,"15min":1.0509999999999999},"min":{"1min":0.81399999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.1830000000000001,"15min":1.1679999999999999},"min":{"1min":0.92100000000000004,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.099,"15min":1.097},"min":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":1.4350000000000001,"5min":2.069,"15min":2.069},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.161,"15min":1.1739999999999999},"min":{"1min":0.74399999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.329,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.8400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30616,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28352,"kb_avail":94341224,"statfs":{"total":96636764160,"available":96605413376,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29032448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.081},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.0209999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.113},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1559999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.1759999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30488,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94341352,"statfs":{"total":96636764160,"available":96605544448,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1339999999999999,"5min":1.345,"15min":1.4670000000000001},"last":1.0509999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.84599999999999997},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.82999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.57099999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.3640000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94399999999999995},"min":{"1min":0.71299999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.262,"15min":1.4359999999999999},"last":1.0920000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0600000000000001,"15min":1.0800000000000001},"min":{"1min":0.79900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.415,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.91800000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3560000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.3959999999999999,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1439999999999999,"15min":1.175},"min":{"1min":0.85199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5680000000000001,"5min":1.6459999999999999,"15min":1.8},"last":1.4399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.96599999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.466,"5min":1.5780000000000001,"15min":1.718},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.196,"15min":1.1819999999999999},"min":{"1min":0.88600000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":1.6439999999999999,"15min":1.736},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2350000000000001,"15min":1.23},"min":{"1min":1.121,"5min":0.873,"15min":0.873},"max":{"1min":1.5349999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.232},"min":{"1min":0.98399999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.607,"5min":1.6970000000000001,"15min":1.8129999999999999},"last":1.1439999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542607,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44760,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":42496,"kb_avail":94327080,"statfs":{"total":96636764160,"available":96590929920,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43515904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.98599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":1.034},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.014},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.367}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.96299999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":28864,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29556736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.113,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.47499999999999998},"max":{"1min":1.145,"5min":1.198,"15min":1.2470000000000001},"last":1.028}]},{"osd":2,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.502},"max":{"1min":1.169,"5min":1.169,"15min":1.2509999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.119,"5min":1.288,"15min":1.863},"last":0.82099999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88600000000000001,"15min":0.876},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2190000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.27,"15min":1.4390000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.042,"15min":1.052},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3160000000000001,"5min":1.573,"15min":1.7110000000000001},"last":1.365},{"interface":"front","average":{"1min":0.996,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.411,"5min":1.516,"15min":1.651},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.1100000000000001},"min":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.3380000000000001,"5min":1.444,"15min":1.512},"last":1.276},{"interface":"front","average":{"1min":1.131,"5min":1.1519999999999999,"15min":1.175},"min":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4810000000000001,"5min":1.5629999999999999,"15min":1.847},"last":1.171}]},{"osd":6,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.167,"15min":1.1870000000000001},"min":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.3500000000000001,"5min":1.6499999999999999,"15min":1.895},"last":1.329},{"interface":"front","average":{"1min":1.204,"5min":1.226,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.615,"5min":1.722,"15min":2.0510000000000002},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2589999999999999,"15min":1.2789999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.722,"5min":1.786,"15min":2.0030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4099999999999999,"5min":1.806,"15min":2.004},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:23.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:23.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.444+0000 7fd3c13e4700 1 Processor -- start 2023-09-27T23:52:23.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.444+0000 7fd3c13e4700 1 -- start start 2023-09-27T23:52:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3c13e4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3bc155350 0x7fd3bc159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3c13e4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc0b0080 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3c13e4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc06e740 con 0x7fd3bc0ab960 2023-09-27T23:52:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3c13e4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc15a010 con 0x7fd3bc155350 2023-09-27T23:52:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3bb7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd3bc06b8b0 0x7fd3bc0b0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55942/0 (socket says 172.21.15.64:55942) 2023-09-27T23:52:23.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3bb7fe700 1 -- 172.21.15.64:0/3565188205 learned_addr learned my addr 172.21.15.64:0/3565188205 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:23.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3baffd700 1 --2- 172.21.15.64:0/3565188205 >> v2:172.21.15.123:3300/0 conn(0x7fd3bc155350 0x7fd3bc159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3b9ffb700 1 -- 172.21.15.64:0/3565188205 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178420919 0 0) 0x7fd3bc0b0080 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3b9ffb700 1 -- 172.21.15.64:0/3565188205 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a4004680 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3b9ffb700 1 -- 172.21.15.64:0/3565188205 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2436821317 0 0) 0x7fd3a4004680 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3baffd700 1 -- 172.21.15.64:0/3565188205 >> v1:172.21.15.123:6789/0 conn(0x7fd3bc0ab960 legacy=0x7fd3bc06b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3baffd700 1 -- 172.21.15.64:0/3565188205 >> v1:172.21.15.64:6789/0 conn(0x7fd3bc06b8b0 legacy=0x7fd3bc0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3baffd700 1 -- 172.21.15.64:0/3565188205 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc0af470 con 0x7fd3bc155350 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3baffd700 1 --2- 172.21.15.64:0/3565188205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc159ad0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009910 tx=0x7fd3ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad00668736c2085 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3b9ffb700 1 -- 172.21.15.64:0/3565188205 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40017f0 con 0x7fd3bc155350 2023-09-27T23:52:23.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3b9ffb700 1 -- 172.21.15.64:0/3565188205 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac011420 con 0x7fd3bc155350 2023-09-27T23:52:23.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.448+0000 7fd3b9ffb700 1 -- 172.21.15.64:0/3565188205 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0117c0 con 0x7fd3bc155350 2023-09-27T23:52:23.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 -- 172.21.15.64:0/3565188205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 msgr2=0x7fd3bc159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/3565188205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc159ad0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009910 tx=0x7fd3ac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 -- 172.21.15.64:0/3565188205 shutdown_connections 2023-09-27T23:52:23.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/3565188205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc159ad0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 -- 172.21.15.64:0/3565188205 >> 172.21.15.64:0/3565188205 conn(0x7fd3bc067410 msgr2=0x7fd3bc0677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:23.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 -- 172.21.15.64:0/3565188205 shutdown_connections 2023-09-27T23:52:23.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 -- 172.21.15.64:0/3565188205 wait complete. 2023-09-27T23:52:23.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 Processor -- start 2023-09-27T23:52:23.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.452+0000 7fd3c13e4700 1 -- start start 2023-09-27T23:52:23.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 0x7fd3bc0a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3bc0ab960 0x7fd3bc0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc0a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0a9190 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc156de0 con 0x7fd3bc155350 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0a7380 con 0x7fd3bc0ab960 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 0x7fd3bc0a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 0x7fd3bc0a6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37828/0 (socket says 172.21.15.64:37828) 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 -- 172.21.15.64:0/1603059312 learned_addr learned my addr 172.21.15.64:0/1603059312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3bb7fe700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc0a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3ba7fc700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3bc0ab960 0x7fd3bc0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 -- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3bc0ab960 msgr2=0x7fd3bc0a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3bc0ab960 0x7fd3bc0a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 -- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 msgr2=0x7fd3bc0a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc0a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 -- 172.21.15.64:0/1603059312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac009420 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3ba7fc700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3bc0ab960 0x7fd3bc0a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:23.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3baffd700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 0x7fd3bc0a6c80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0030c0 tx=0x7fd3ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac01bc80 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac020070 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc0a7660 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac027480 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.456+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc156a70 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.460+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc071120 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.460+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd3ac01c450 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.460+0000 7fd3a2ffd700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd3a8060aa0 0x7fd3a8062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.460+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd3ac022070 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.460+0000 7fd3ba7fc700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd3a8060aa0 0x7fd3a8062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.460+0000 7fd3ba7fc700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd3a8060aa0 0x7fd3a8062f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007fd0 tx=0x7fd3b0009040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.468+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd3ac04eef0 con 0x7fd3bc06b8b0 2023-09-27T23:52:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.624+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3bc156f20 con 0x7fd3a8060aa0 2023-09-27T23:52:23.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.640+0000 7fd3a2ffd700 1 -- 172.21.15.64:0/1603059312 <== mgr.9940 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292741 (secure 0 0 0) 0x7fd3b0001d20 con 0x7fd3a8060aa0 2023-09-27T23:52:23.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd3a8060aa0 msgr2=0x7fd3a8062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd3a8060aa0 0x7fd3a8062f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007fd0 tx=0x7fd3b0009040 comp rx=0 tx=0).stop 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 msgr2=0x7fd3bc0a6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 0x7fd3bc0a6c80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0030c0 tx=0x7fd3ac016000 comp rx=0 tx=0).stop 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 shutdown_connections 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd3a8060aa0 0x7fd3a8062f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3bc155350 0x7fd3bc0a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3bc0ab960 0x7fd3bc0a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 --2- 172.21.15.64:0/1603059312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc06b8b0 0x7fd3bc0a6c80 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 >> 172.21.15.64:0/1603059312 conn(0x7fd3bc067410 msgr2=0x7fd3bc1540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 shutdown_connections 2023-09-27T23:52:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.644+0000 7fd3c13e4700 1 -- 172.21.15.64:0/1603059312 wait complete. 2023-09-27T23:52:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:23.666 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:52:23.281985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":275016,"kb_used_data":19208,"kb_used_omap":0,"kb_used_meta":255744,"kb_avail":754699704,"statfs":{"total":773094113280,"available":772812496896,"internally_reserved":0,"allocated":19668992,"data_stored":10911184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261881856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.390233"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"177'326","reported_seq":1047,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200927+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:17.200927+0000","last_peered":"2023-09-27T23:52:17.200927+0000","last_clean":"2023-09-27T23:52:17.200927+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:17.200927+0000","last_undegraded":"2023-09-27T23:52:17.200927+0000","last_fullsized":"2023-09-27T23:52:17.200927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":326,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710096+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:15.710096+0000","last_peered":"2023-09-27T23:52:15.710096+0000","last_clean":"2023-09-27T23:52:15.710096+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:15.710096+0000","last_undegraded":"2023-09-27T23:52:15.710096+0000","last_fullsized":"2023-09-27T23:52:15.710096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200923+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:17.200923+0000","last_peered":"2023-09-27T23:52:17.200923+0000","last_clean":"2023-09-27T23:52:17.200923+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:17.200923+0000","last_undegraded":"2023-09-27T23:52:17.200923+0000","last_fullsized":"2023-09-27T23:52:17.200923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:16.220162+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:16.220162+0000","last_peered":"2023-09-27T23:52:16.220162+0000","last_clean":"2023-09-27T23:52:16.220162+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:16.220162+0000","last_undegraded":"2023-09-27T23:52:16.220162+0000","last_fullsized":"2023-09-27T23:52:16.220162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710169+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:15.710169+0000","last_peered":"2023-09-27T23:52:15.710169+0000","last_clean":"2023-09-27T23:52:15.710169+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:15.710169+0000","last_undegraded":"2023-09-27T23:52:15.710169+0000","last_fullsized":"2023-09-27T23:52:15.710169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705037+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:15.705037+0000","last_peered":"2023-09-27T23:52:15.705037+0000","last_clean":"2023-09-27T23:52:15.705037+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:15.705037+0000","last_undegraded":"2023-09-27T23:52:15.705037+0000","last_fullsized":"2023-09-27T23:52:15.705037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:17.200922+0000","last_peered":"2023-09-27T23:52:17.200922+0000","last_clean":"2023-09-27T23:52:17.200922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:17.200922+0000","last_undegraded":"2023-09-27T23:52:17.200922+0000","last_fullsized":"2023-09-27T23:52:17.200922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705120+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:15.705120+0000","last_peered":"2023-09-27T23:52:15.705120+0000","last_clean":"2023-09-27T23:52:15.705120+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:15.705120+0000","last_undegraded":"2023-09-27T23:52:15.705120+0000","last_fullsized":"2023-09-27T23:52:15.705120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.833607+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:15.833607+0000","last_peered":"2023-09-27T23:52:15.833607+0000","last_clean":"2023-09-27T23:52:15.833607+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:15.833607+0000","last_undegraded":"2023-09-27T23:52:15.833607+0000","last_fullsized":"2023-09-27T23:52:15.833607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36636,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94335204,"statfs":{"total":96636764160,"available":96599248896,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.89600000000000002},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2250000000000001,"5min":1.569,"15min":1.569},"last":1.446},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.94899999999999995,"15min":0.93400000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.466,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.248,"5min":1.333,"15min":1.3360000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96799999999999997,"15min":0.97499999999999998},"min":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2929999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.2949999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.92500000000000004},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.3069999999999999,"5min":1.341,"15min":1.341},"last":1.004}]},{"osd":3,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.70799999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.202,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0760000000000001,"15min":1.079},"min":{"1min":0.77700000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.3879999999999999},"last":1.23}]},{"osd":4,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.216,"5min":1.821,"15min":1.821},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.03,"15min":1.0109999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.226,"5min":3.02,"15min":3.02},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0569999999999999,"15min":1.0620000000000001},"min":{"1min":0.75,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":1.339},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3759999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.131,"15min":1.143},"min":{"1min":0.749,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.423,"5min":1.423,"15min":1.448},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.145,"15min":1.145},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.381,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.268}]}]},{"osd":6,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30040,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":27776,"kb_avail":94341800,"statfs":{"total":96636764160,"available":96606003200,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28442624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.97199999999999998},"min":{"1min":0.70599999999999996,"5min":0.60599999999999998,"15min":0.53400000000000003},"max":{"1min":1.214,"5min":1.792,"15min":1.792},"last":1.016},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.80400000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90200000000000002,"15min":0.90300000000000002},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67100000000000004,"5min":0.622,"15min":0.57099999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89500000000000002,"15min":0.90600000000000003},"min":{"1min":0.78100000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.147,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.018,"5min":0.98399999999999999,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.78000000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91000000000000003,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.6639999999999999,"5min":1.7,"15min":1.7},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.079,"5min":1.0289999999999999,"15min":1.0029999999999999},"min":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3089999999999999,"5min":1.3700000000000001,"15min":1.633},"last":0.876}]},{"osd":5,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.161,"15min":1.145},"min":{"1min":0.89500000000000002,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.075}]},{"osd":7,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.81100000000000005,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":1.637,"15min":1.679},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.155,"15min":1.1359999999999999},"min":{"1min":0.69199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":37632,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38535168},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.95999999999999996,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.51900000000000002},"max":{"1min":10.058999999999999,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.657,"5min":1.0609999999999999,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.47899999999999998},"max":{"1min":13.757999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.96999999999999997,"15min":0.93200000000000005},"min":{"1min":0.70699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.1970000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.95999999999999996,"15min":0.96099999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.629},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.97699999999999998,"15min":0.95899999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.999,"15min":0.98999999999999999},"min":{"1min":0.84299999999999997,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.075,"15min":1.042},"min":{"1min":0.79000000000000004,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.747}]},{"osd":4,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91200000000000003,"15min":0.88900000000000001},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.04,"5min":1.0469999999999999,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3640000000000001,"5min":1.45,"15min":1.4510000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.119,"15min":1.105},"min":{"1min":0.91900000000000004,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.026}]},{"osd":7,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.222,"15min":1.169},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":6.3710000000000004,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.363,"15min":1.2629999999999999},"min":{"1min":1.0329999999999999,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":14.119,"5min":14.119,"15min":14.119},"last":1.087}]}]},{"osd":4,"up_from":15,"seq":64424509901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30872,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28608,"kb_avail":94340968,"statfs":{"total":96636764160,"available":96605151232,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29294592},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.45000000000000001},"max":{"1min":1.1240000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.1180000000000001,"5min":1.893,"15min":2.2770000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2010000000000001,"5min":1.746,"15min":1.746},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.53500000000000003},"max":{"1min":1.2390000000000001,"5min":1.758,"15min":2.1110000000000002},"last":1.339}]},{"osd":2,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2490000000000001,"5min":1.659,"15min":1.659},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98299999999999998,"15min":0.96099999999999997},"min":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.234,"5min":1.966,"15min":2.056},"last":1.216}]},{"osd":3,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93400000000000005,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.998,"15min":2.032},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.079,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":1.2709999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.5}]},{"osd":5,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.92400000000000004},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.45500000000000002},"max":{"1min":1.25,"5min":1.556,"15min":1.9930000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.006,"15min":1.0149999999999999},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3129999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.419}]},{"osd":6,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0529999999999999,"15min":1.0509999999999999},"min":{"1min":0.81399999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.1830000000000001,"15min":1.1679999999999999},"min":{"1min":0.92100000000000004,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.099,"15min":1.097},"min":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":1.4350000000000001,"5min":2.069,"15min":2.069},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.161,"15min":1.1739999999999999},"min":{"1min":0.74399999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.329,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.8400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30616,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28352,"kb_avail":94341224,"statfs":{"total":96636764160,"available":96605413376,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29032448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.081},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.0209999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.113},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1559999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.1759999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30488,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94341352,"statfs":{"total":96636764160,"available":96605544448,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1339999999999999,"5min":1.345,"15min":1.4670000000000001},"last":1.0509999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.84599999999999997},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.82999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.57099999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.3640000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94399999999999995},"min":{"1min":0.71299999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.262,"15min":1.4359999999999999},"last":1.0920000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0600000000000001,"15min":1.0800000000000001},"min":{"1min":0.79900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.415,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.91800000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3560000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.3959999999999999,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1439999999999999,"15min":1.175},"min":{"1min":0.85199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5680000000000001,"5min":1.6459999999999999,"15min":1.8},"last":1.4399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.96599999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.466,"5min":1.5780000000000001,"15min":1.718},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.196,"15min":1.1819999999999999},"min":{"1min":0.88600000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":1.6439999999999999,"15min":1.736},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2350000000000001,"15min":1.23},"min":{"1min":1.121,"5min":0.873,"15min":0.873},"max":{"1min":1.5349999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.232},"min":{"1min":0.98399999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.607,"5min":1.6970000000000001,"15min":1.8129999999999999},"last":1.1439999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542607,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44760,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":42496,"kb_avail":94327080,"statfs":{"total":96636764160,"available":96590929920,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43515904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.98599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":1.034},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.014},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.367}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.96299999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":28864,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29556736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.113,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.47499999999999998},"max":{"1min":1.145,"5min":1.198,"15min":1.2470000000000001},"last":1.028}]},{"osd":2,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.502},"max":{"1min":1.169,"5min":1.169,"15min":1.2509999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.119,"5min":1.288,"15min":1.863},"last":0.82099999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88600000000000001,"15min":0.876},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2190000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.27,"15min":1.4390000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.042,"15min":1.052},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3160000000000001,"5min":1.573,"15min":1.7110000000000001},"last":1.365},{"interface":"front","average":{"1min":0.996,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.411,"5min":1.516,"15min":1.651},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.1100000000000001},"min":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.3380000000000001,"5min":1.444,"15min":1.512},"last":1.276},{"interface":"front","average":{"1min":1.131,"5min":1.1519999999999999,"15min":1.175},"min":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4810000000000001,"5min":1.5629999999999999,"15min":1.847},"last":1.171}]},{"osd":6,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.167,"15min":1.1870000000000001},"min":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.3500000000000001,"5min":1.6499999999999999,"15min":1.895},"last":1.329},{"interface":"front","average":{"1min":1.204,"5min":1.226,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.615,"5min":1.722,"15min":2.0510000000000002},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2589999999999999,"15min":1.2789999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.722,"5min":1.786,"15min":2.0030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4099999999999999,"5min":1.806,"15min":2.004},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:23.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2032bf1700 1 Processor -- start 2023-09-27T23:52:23.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2032bf1700 1 -- start start 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2032bf1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f202c0b1410 0x7f202c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2032bf1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c06c7c0 con 0x7f202c06aa00 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2032bf1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c06cd80 con 0x7f202c06a580 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2032bf1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c15b680 con 0x7f202c0b1410 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2031bef700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f202c0b1410 0x7f202c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f20323f0700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f202c06aa00 0x7f202c15af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55958/0 (socket says 172.21.15.64:55958) 2023-09-27T23:52:23.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f20323f0700 1 -- 172.21.15.64:0/3208378723 learned_addr learned my addr 172.21.15.64:0/3208378723 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:23.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2031bef700 1 -- 172.21.15.64:0/3208378723 >> v1:172.21.15.123:6789/0 conn(0x7f202c06a580 legacy=0x7f202c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2031bef700 1 -- 172.21.15.64:0/3208378723 >> v1:172.21.15.64:6789/0 conn(0x7f202c06aa00 legacy=0x7f202c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2030bed700 1 -- 172.21.15.64:0/3208378723 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311468713 0 0) 0x7f202c06c7c0 con 0x7f202c06aa00 2023-09-27T23:52:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2031bef700 1 -- 172.21.15.64:0/3208378723 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c06b8f0 con 0x7f202c0b1410 2023-09-27T23:52:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2031bef700 1 --2- 172.21.15.64:0/3208378723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c0ab440 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f20240098e0 tx=0x7f20240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f006f3d770cd8b23 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2030bed700 1 -- 172.21.15.64:0/3208378723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024009e80 con 0x7f202c0b1410 2023-09-27T23:52:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2030bed700 1 -- 172.21.15.64:0/3208378723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202400b860 con 0x7f202c0b1410 2023-09-27T23:52:23.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.776+0000 7f2030bed700 1 -- 172.21.15.64:0/3208378723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202400bbe0 con 0x7f202c0b1410 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- 172.21.15.64:0/3208378723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 msgr2=0x7f202c0ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3208378723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c0ab440 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f20240098e0 tx=0x7f20240091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- 172.21.15.64:0/3208378723 shutdown_connections 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3208378723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c0ab440 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- 172.21.15.64:0/3208378723 >> 172.21.15.64:0/3208378723 conn(0x7f202c067bb0 msgr2=0x7f202c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- 172.21.15.64:0/3208378723 shutdown_connections 2023-09-27T23:52:23.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- 172.21.15.64:0/3208378723 wait complete. 2023-09-27T23:52:23.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 Processor -- start 2023-09-27T23:52:23.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- start start 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f202c06a580 0x7f202c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f202c06aa00 0x7f202c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c14cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c14da40 con 0x7f202c0b1410 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1504a0 con 0x7f202c06aa00 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2032bf1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1592f0 con 0x7f202c06a580 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c14cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20313ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f202c06aa00 0x7f202c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f2031bef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f202c06a580 0x7f202c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c14cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37854/0 (socket says 172.21.15.64:37854) 2023-09-27T23:52:23.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20313ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f202c06aa00 0x7f202c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:35948/0 (socket says 172.21.15.64:35948) 2023-09-27T23:52:23.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 -- 172.21.15.64:0/3936734279 learned_addr learned my addr 172.21.15.64:0/3936734279 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:23.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 -- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f202c06a580 msgr2=0x7f202c157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f202c06a580 0x7f202c157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 -- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f202c06aa00 msgr2=0x7f202c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f202c06aa00 0x7f202c14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.780+0000 7f20323f0700 1 -- 172.21.15.64:0/3936734279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024009420 con 0x7f202c0b1410 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f20323f0700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c14cef0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f201c0011e0 tx=0x7f201c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c005b30 con 0x7f202c0b1410 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c005c90 con 0x7f202c0b1410 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c14d430 con 0x7f202c0b1410 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c007320 con 0x7f202c0b1410 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c158710 con 0x7f202c0b1410 2023-09-27T23:52:23.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.784+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c159480 con 0x7f202c0b1410 2023-09-27T23:52:23.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.788+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f201c007690 con 0x7f202c0b1410 2023-09-27T23:52:23.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.788+0000 7f2029ffb700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f200c060a70 0x7f200c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:23.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.788+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f201c07c5a0 con 0x7f202c0b1410 2023-09-27T23:52:23.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.788+0000 7f2031bef700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f200c060a70 0x7f200c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:23.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.788+0000 7f2031bef700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f200c060a70 0x7f200c062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20240093f0 tx=0x7f2024017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:23.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.796+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f201c04c5d0 con 0x7f202c0b1410 2023-09-27T23:52:23.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.952+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f202c04be40 con 0x7f200c060a70 2023-09-27T23:52:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.968+0000 7f2029ffb700 1 -- 172.21.15.64:0/3936734279 <== mgr.9940 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292741 (secure 0 0 0) 0x7f202c04be40 con 0x7f200c060a70 2023-09-27T23:52:23.978 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f200c060a70 msgr2=0x7f200c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f200c060a70 0x7f200c062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20240093f0 tx=0x7f2024017000 comp rx=0 tx=0).stop 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 msgr2=0x7f202c14cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c14cef0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f201c0011e0 tx=0x7f201c007d30 comp rx=0 tx=0).stop 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 shutdown_connections 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f200c060a70 0x7f200c062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202c0b1410 0x7f202c14cef0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f202c06aa00 0x7f202c14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 --2- 172.21.15.64:0/3936734279 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f202c06a580 0x7f202c157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 >> 172.21.15.64:0/3936734279 conn(0x7f202c067bb0 msgr2=0x7f202c14e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 shutdown_connections 2023-09-27T23:52:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:23.976+0000 7f2032bf1700 1 -- 172.21.15.64:0/3936734279 wait complete. 2023-09-27T23:52:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:23.996 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:52:23.281985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":275016,"kb_used_data":19208,"kb_used_omap":0,"kb_used_meta":255744,"kb_avail":754699704,"statfs":{"total":773094113280,"available":772812496896,"internally_reserved":0,"allocated":19668992,"data_stored":10911184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261881856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.390233"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"177'326","reported_seq":1047,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200927+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:17.200927+0000","last_peered":"2023-09-27T23:52:17.200927+0000","last_clean":"2023-09-27T23:52:17.200927+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:17.200927+0000","last_undegraded":"2023-09-27T23:52:17.200927+0000","last_fullsized":"2023-09-27T23:52:17.200927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":326,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710096+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:15.710096+0000","last_peered":"2023-09-27T23:52:15.710096+0000","last_clean":"2023-09-27T23:52:15.710096+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:15.710096+0000","last_undegraded":"2023-09-27T23:52:15.710096+0000","last_fullsized":"2023-09-27T23:52:15.710096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200923+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:17.200923+0000","last_peered":"2023-09-27T23:52:17.200923+0000","last_clean":"2023-09-27T23:52:17.200923+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:17.200923+0000","last_undegraded":"2023-09-27T23:52:17.200923+0000","last_fullsized":"2023-09-27T23:52:17.200923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:16.220162+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:16.220162+0000","last_peered":"2023-09-27T23:52:16.220162+0000","last_clean":"2023-09-27T23:52:16.220162+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:16.220162+0000","last_undegraded":"2023-09-27T23:52:16.220162+0000","last_fullsized":"2023-09-27T23:52:16.220162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710169+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:15.710169+0000","last_peered":"2023-09-27T23:52:15.710169+0000","last_clean":"2023-09-27T23:52:15.710169+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:15.710169+0000","last_undegraded":"2023-09-27T23:52:15.710169+0000","last_fullsized":"2023-09-27T23:52:15.710169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705037+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:15.705037+0000","last_peered":"2023-09-27T23:52:15.705037+0000","last_clean":"2023-09-27T23:52:15.705037+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:15.705037+0000","last_undegraded":"2023-09-27T23:52:15.705037+0000","last_fullsized":"2023-09-27T23:52:15.705037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:17.200922+0000","last_peered":"2023-09-27T23:52:17.200922+0000","last_clean":"2023-09-27T23:52:17.200922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:17.200922+0000","last_undegraded":"2023-09-27T23:52:17.200922+0000","last_fullsized":"2023-09-27T23:52:17.200922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705120+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:15.705120+0000","last_peered":"2023-09-27T23:52:15.705120+0000","last_clean":"2023-09-27T23:52:15.705120+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:15.705120+0000","last_undegraded":"2023-09-27T23:52:15.705120+0000","last_fullsized":"2023-09-27T23:52:15.705120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.833607+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:15.833607+0000","last_peered":"2023-09-27T23:52:15.833607+0000","last_clean":"2023-09-27T23:52:15.833607+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:15.833607+0000","last_undegraded":"2023-09-27T23:52:15.833607+0000","last_fullsized":"2023-09-27T23:52:15.833607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36636,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94335204,"statfs":{"total":96636764160,"available":96599248896,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.89600000000000002},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2250000000000001,"5min":1.569,"15min":1.569},"last":1.446},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.94899999999999995,"15min":0.93400000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.466,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.248,"5min":1.333,"15min":1.3360000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96799999999999997,"15min":0.97499999999999998},"min":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2929999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.2949999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.92500000000000004},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.3069999999999999,"5min":1.341,"15min":1.341},"last":1.004}]},{"osd":3,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.70799999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.202,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0760000000000001,"15min":1.079},"min":{"1min":0.77700000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.3879999999999999},"last":1.23}]},{"osd":4,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.216,"5min":1.821,"15min":1.821},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.03,"15min":1.0109999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.226,"5min":3.02,"15min":3.02},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0569999999999999,"15min":1.0620000000000001},"min":{"1min":0.75,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":1.339},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3759999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.131,"15min":1.143},"min":{"1min":0.749,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.423,"5min":1.423,"15min":1.448},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.145,"15min":1.145},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.381,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.268}]}]},{"osd":6,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30040,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":27776,"kb_avail":94341800,"statfs":{"total":96636764160,"available":96606003200,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28442624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.97199999999999998},"min":{"1min":0.70599999999999996,"5min":0.60599999999999998,"15min":0.53400000000000003},"max":{"1min":1.214,"5min":1.792,"15min":1.792},"last":1.016},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.80400000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90200000000000002,"15min":0.90300000000000002},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67100000000000004,"5min":0.622,"15min":0.57099999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89500000000000002,"15min":0.90600000000000003},"min":{"1min":0.78100000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.147,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.018,"5min":0.98399999999999999,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.78000000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91000000000000003,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.6639999999999999,"5min":1.7,"15min":1.7},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.079,"5min":1.0289999999999999,"15min":1.0029999999999999},"min":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3089999999999999,"5min":1.3700000000000001,"15min":1.633},"last":0.876}]},{"osd":5,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.161,"15min":1.145},"min":{"1min":0.89500000000000002,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.075}]},{"osd":7,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.81100000000000005,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":1.637,"15min":1.679},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.155,"15min":1.1359999999999999},"min":{"1min":0.69199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":37632,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38535168},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.95999999999999996,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.51900000000000002},"max":{"1min":10.058999999999999,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.657,"5min":1.0609999999999999,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.47899999999999998},"max":{"1min":13.757999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.96999999999999997,"15min":0.93200000000000005},"min":{"1min":0.70699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.1970000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.95999999999999996,"15min":0.96099999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.629},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.97699999999999998,"15min":0.95899999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.999,"15min":0.98999999999999999},"min":{"1min":0.84299999999999997,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.075,"15min":1.042},"min":{"1min":0.79000000000000004,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.747}]},{"osd":4,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91200000000000003,"15min":0.88900000000000001},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.04,"5min":1.0469999999999999,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3640000000000001,"5min":1.45,"15min":1.4510000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.119,"15min":1.105},"min":{"1min":0.91900000000000004,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.026}]},{"osd":7,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.222,"15min":1.169},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":6.3710000000000004,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.363,"15min":1.2629999999999999},"min":{"1min":1.0329999999999999,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":14.119,"5min":14.119,"15min":14.119},"last":1.087}]}]},{"osd":4,"up_from":15,"seq":64424509901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30872,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28608,"kb_avail":94340968,"statfs":{"total":96636764160,"available":96605151232,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29294592},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.45000000000000001},"max":{"1min":1.1240000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.1180000000000001,"5min":1.893,"15min":2.2770000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2010000000000001,"5min":1.746,"15min":1.746},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.53500000000000003},"max":{"1min":1.2390000000000001,"5min":1.758,"15min":2.1110000000000002},"last":1.339}]},{"osd":2,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2490000000000001,"5min":1.659,"15min":1.659},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98299999999999998,"15min":0.96099999999999997},"min":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.234,"5min":1.966,"15min":2.056},"last":1.216}]},{"osd":3,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93400000000000005,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.998,"15min":2.032},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.079,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":1.2709999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.5}]},{"osd":5,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.92400000000000004},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.45500000000000002},"max":{"1min":1.25,"5min":1.556,"15min":1.9930000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.006,"15min":1.0149999999999999},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3129999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.419}]},{"osd":6,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0529999999999999,"15min":1.0509999999999999},"min":{"1min":0.81399999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.1830000000000001,"15min":1.1679999999999999},"min":{"1min":0.92100000000000004,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.099,"15min":1.097},"min":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":1.4350000000000001,"5min":2.069,"15min":2.069},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.161,"15min":1.1739999999999999},"min":{"1min":0.74399999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.329,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.8400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30616,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28352,"kb_avail":94341224,"statfs":{"total":96636764160,"available":96605413376,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29032448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.081},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.0209999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.113},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1559999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.1759999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30488,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94341352,"statfs":{"total":96636764160,"available":96605544448,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1339999999999999,"5min":1.345,"15min":1.4670000000000001},"last":1.0509999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.84599999999999997},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.82999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.57099999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.3640000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94399999999999995},"min":{"1min":0.71299999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.262,"15min":1.4359999999999999},"last":1.0920000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0600000000000001,"15min":1.0800000000000001},"min":{"1min":0.79900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.415,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.91800000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3560000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.3959999999999999,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1439999999999999,"15min":1.175},"min":{"1min":0.85199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5680000000000001,"5min":1.6459999999999999,"15min":1.8},"last":1.4399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.96599999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.466,"5min":1.5780000000000001,"15min":1.718},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.196,"15min":1.1819999999999999},"min":{"1min":0.88600000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":1.6439999999999999,"15min":1.736},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2350000000000001,"15min":1.23},"min":{"1min":1.121,"5min":0.873,"15min":0.873},"max":{"1min":1.5349999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.232},"min":{"1min":0.98399999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.607,"5min":1.6970000000000001,"15min":1.8129999999999999},"last":1.1439999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542607,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44760,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":42496,"kb_avail":94327080,"statfs":{"total":96636764160,"available":96590929920,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43515904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.98599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":1.034},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.014},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.367}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.96299999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":28864,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29556736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.113,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.47499999999999998},"max":{"1min":1.145,"5min":1.198,"15min":1.2470000000000001},"last":1.028}]},{"osd":2,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.502},"max":{"1min":1.169,"5min":1.169,"15min":1.2509999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.119,"5min":1.288,"15min":1.863},"last":0.82099999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88600000000000001,"15min":0.876},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2190000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.27,"15min":1.4390000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.042,"15min":1.052},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3160000000000001,"5min":1.573,"15min":1.7110000000000001},"last":1.365},{"interface":"front","average":{"1min":0.996,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.411,"5min":1.516,"15min":1.651},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.1100000000000001},"min":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.3380000000000001,"5min":1.444,"15min":1.512},"last":1.276},{"interface":"front","average":{"1min":1.131,"5min":1.1519999999999999,"15min":1.175},"min":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4810000000000001,"5min":1.5629999999999999,"15min":1.847},"last":1.171}]},{"osd":6,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.167,"15min":1.1870000000000001},"min":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.3500000000000001,"5min":1.6499999999999999,"15min":1.895},"last":1.329},{"interface":"front","average":{"1min":1.204,"5min":1.226,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.615,"5min":1.722,"15min":2.0510000000000002},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2589999999999999,"15min":1.2789999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.722,"5min":1.786,"15min":2.0030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4099999999999999,"5min":1.806,"15min":2.004},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:24.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3f is not active+clean 2023-09-27T23:52:24.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3e is not active+clean 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3d is not active+clean 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3c is not active+clean 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3b is not active+clean 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3a is not active+clean 2023-09-27T23:52:24.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.39 is not active+clean 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.38 is not active+clean 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.37 is not active+clean 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.36 is not active+clean 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.35 is not active+clean 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.34 is not active+clean 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.33 is not active+clean 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.32 is not active+clean 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.31 is not active+clean 2023-09-27T23:52:24.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.30 is not active+clean 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2f is not active+clean 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2e is not active+clean 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2d is not active+clean 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2c is not active+clean 2023-09-27T23:52:24.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2b is not active+clean 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2a is not active+clean 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.29 is not active+clean 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.28 is not active+clean 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.27 is not active+clean 2023-09-27T23:52:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.26 is not active+clean 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.25 is not active+clean 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.24 is not active+clean 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.23 is not active+clean 2023-09-27T23:52:24.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.22 is not active+clean 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.21 is not active+clean 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.20 is not active+clean 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.16 is not active+clean 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2023-09-27T23:52:24.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.17 is not active+clean 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.14 is not active+clean 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.15 is not active+clean 2023-09-27T23:52:24.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.12 is not active+clean 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.13 is not active+clean 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2023-09-27T23:52:24.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.10 is not active+clean 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.11 is not active+clean 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.e is not active+clean 2023-09-27T23:52:24.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.f is not active+clean 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.c is not active+clean 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2023-09-27T23:52:24.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.d is not active+clean 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.c is not active+clean 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.a is not active+clean 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.b is not active+clean 2023-09-27T23:52:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.8 is not active+clean 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.9 is not active+clean 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2023-09-27T23:52:24.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.6 is not active+clean 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.7 is not active+clean 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.4 is not active+clean 2023-09-27T23:52:24.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.5 is not active+clean 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2 is not active+clean 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2023-09-27T23:52:24.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3 is not active+clean 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.0 is not active+clean 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1 is not active+clean 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1 is not active+clean 2023-09-27T23:52:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.18 is not active+clean 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1b is not active+clean 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2023-09-27T23:52:24.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1a is not active+clean 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.19 is not active+clean 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1f is not active+clean 2023-09-27T23:52:24.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1e is not active+clean 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1d is not active+clean 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1c is not active+clean 2023-09-27T23:52:24.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1c is not active+clean 2023-09-27T23:52:24.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:19.885476+0000', 'last_change': '2023-09-27T23:52:19.885476+0000', 'last_active': '2023-09-27T23:52:19.885476+0000', 'last_peered': '2023-09-27T23:52:19.885476+0000', 'last_clean': '2023-09-27T23:52:19.885476+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:19.885476+0000', 'last_undegraded': '2023-09-27T23:52:19.885476+0000', 'last_fullsized': '2023-09-27T23:52:19.885476+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:19.885476+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2023-09-27T23:52:24.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-27T23:52:18.872640+0000', 'last_change': '2023-09-27T23:52:18.872640+0000', 'last_active': '2023-09-27T23:52:18.872640+0000', 'last_peered': '2023-09-27T23:52:18.872640+0000', 'last_clean': '2023-09-27T23:52:18.872640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-27T23:52:18.872640+0000', 'last_undegraded': '2023-09-27T23:52:18.872640+0000', 'last_fullsized': '2023-09-27T23:52:18.872640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'last_clean_scrub_stamp': '2023-09-27T23:52:18.872640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-27T23:52:24.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-27T23:52:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.124+0000 7f3834cfd700 1 Processor -- start 2023-09-27T23:52:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.124+0000 7f3834cfd700 1 -- start start 2023-09-27T23:52:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f383015d320 0x7f383015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f38300680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38300c7190 con 0x7f38300c15b0 2023-09-27T23:52:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38300aefd0 con 0x7f383015d320 2023-09-27T23:52:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38300c1b50 con 0x7f38300aeba0 2023-09-27T23:52:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382dd9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f38300c15b0 0x7f383006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55974/0 (socket says 172.21.15.64:55974) 2023-09-27T23:52:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382dd9b700 1 -- 172.21.15.64:0/120552332 learned_addr learned my addr 172.21.15.64:0/120552332 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382ed9d700 1 --2- 172.21.15.64:0/120552332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f38300680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382e59c700 1 --2- 172.21.15.64:0/120552332 >> v2:172.21.15.123:3300/0 conn(0x7f383015d320 0x7f383015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382d59a700 1 -- 172.21.15.64:0/120552332 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096996959 0 0) 0x7f38300c7190 con 0x7f38300c15b0 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382d59a700 1 -- 172.21.15.64:0/120552332 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f381c003570 con 0x7f38300c15b0 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382ed9d700 1 -- 172.21.15.64:0/120552332 >> v2:172.21.15.123:3300/0 conn(0x7f383015d320 msgr2=0x7f383015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382ed9d700 1 --2- 172.21.15.64:0/120552332 >> v2:172.21.15.123:3300/0 conn(0x7f383015d320 0x7f383015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382ed9d700 1 -- 172.21.15.64:0/120552332 >> v1:172.21.15.64:6789/0 conn(0x7f38300c15b0 legacy=0x7f383006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382ed9d700 1 -- 172.21.15.64:0/120552332 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300c1e30 con 0x7f38300aeba0 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382ed9d700 1 --2- 172.21.15.64:0/120552332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f38300680f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3828009e60 tx=0x7f3828009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8773411fa1a64816 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382d59a700 1 -- 172.21.15.64:0/120552332 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800e070 con 0x7f38300aeba0 2023-09-27T23:52:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382d59a700 1 -- 172.21.15.64:0/120552332 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382800bab0 con 0x7f38300aeba0 2023-09-27T23:52:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f382d59a700 1 -- 172.21.15.64:0/120552332 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800be30 con 0x7f38300aeba0 2023-09-27T23:52:24.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- 172.21.15.64:0/120552332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 msgr2=0x7f38300680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 --2- 172.21.15.64:0/120552332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f38300680f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3828009e60 tx=0x7f3828009780 comp rx=0 tx=0).stop 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- 172.21.15.64:0/120552332 shutdown_connections 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 --2- 172.21.15.64:0/120552332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f38300680f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 --2- 172.21.15.64:0/120552332 >> v2:172.21.15.123:3300/0 conn(0x7f383015d320 0x7f383015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- 172.21.15.64:0/120552332 >> 172.21.15.64:0/120552332 conn(0x7f38300c8850 msgr2=0x7f38300c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- 172.21.15.64:0/120552332 shutdown_connections 2023-09-27T23:52:24.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.128+0000 7f3834cfd700 1 -- 172.21.15.64:0/120552332 wait complete. 2023-09-27T23:52:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.132+0000 7f3834cfd700 1 Processor -- start 2023-09-27T23:52:24.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.132+0000 7f3834cfd700 1 -- start start 2023-09-27T23:52:24.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f383016dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38300c15b0 0x7f38301571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 0x7f383006acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383015c580 con 0x7f383015d320 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383015c860 con 0x7f38300c15b0 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383016e550 con 0x7f38300aeba0 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 0x7f383006acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38300c15b0 0x7f38301571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 0x7f383006acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37862/0 (socket says 172.21.15.64:37862) 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 -- 172.21.15.64:0/4046280111 learned_addr learned my addr 172.21.15.64:0/4046280111 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:24.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382e59c700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f383016dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 -- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 msgr2=0x7f383016dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f383016dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 -- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38300c15b0 msgr2=0x7f38301571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38300c15b0 0x7f38301571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 -- 172.21.15.64:0/4046280111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828009a20 con 0x7f383015d320 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382dd9b700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38300c15b0 0x7f38301571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f382ed9d700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 0x7f383006acb0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f383006a2b0 tx=0x7f3828019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828012c40 con 0x7f383015d320 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3828021070 con 0x7f383015d320 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383016e830 con 0x7f383015d320 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828026430 con 0x7f383015d320 2023-09-27T23:52:24.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383016e300 con 0x7f383015d320 2023-09-27T23:52:24.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.136+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383016ea50 con 0x7f383015d320 2023-09-27T23:52:24.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.140+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3828023030 con 0x7f383015d320 2023-09-27T23:52:24.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.140+0000 7f38267fc700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3810060a70 0x7f3810062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.140+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f382807f1b0 con 0x7f383015d320 2023-09-27T23:52:24.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.140+0000 7f382e59c700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3810060a70 0x7f3810062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.140+0000 7f382e59c700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3810060a70 0x7f3810062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3820009d10 tx=0x7f38200091a0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:24.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.148+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f382804f140 con 0x7f383015d320 2023-09-27T23:52:24.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.364+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3830071120 con 0x7f383015d320 2023-09-27T23:52:24.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.364+0000 7f38267fc700 1 -- 172.21.15.64:0/4046280111 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3226 (secure 0 0 0) 0x7f3828052760 con 0x7f383015d320 2023-09-27T23:52:24.371 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:24.372 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":2343,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":189,"num_osds":8,"num_up_osds":8,"osd_up_since":1695856411,"num_in_osds":8,"osd_in_since":1695856403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":281616384,"bytes_avail":772812496896,"bytes_total":773094113280,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":72,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"b","status":"up:active","gid":10144}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2023-09-27T23:52:09.277822+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-27T23:52:24.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3810060a70 msgr2=0x7f3810062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3810060a70 0x7f3810062f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3820009d10 tx=0x7f38200091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 msgr2=0x7f383006acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 0x7f383006acb0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f383006a2b0 tx=0x7f3828019000 comp rx=0 tx=0).stop 2023-09-27T23:52:24.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 shutdown_connections 2023-09-27T23:52:24.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3810060a70 0x7f3810062f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f383015d320 0x7f383006acb0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38300c15b0 0x7f38301571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 --2- 172.21.15.64:0/4046280111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38300aeba0 0x7f383016dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 >> 172.21.15.64:0/4046280111 conn(0x7f38300c8850 msgr2=0x7f38300caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 shutdown_connections 2023-09-27T23:52:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.368+0000 7f3834cfd700 1 -- 172.21.15.64:0/4046280111 wait complete. 2023-09-27T23:52:24.385 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-27T23:52:24.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:24.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.488+0000 7f2ee73b3700 1 Processor -- start 2023-09-27T23:52:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.488+0000 7f2ee73b3700 1 -- start start 2023-09-27T23:52:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ee0158e70 0x7f2ee00afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ee006a540 0x7f2ee006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee00cc470 0x7f2ee00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee00ca5a0 con 0x7f2ee0158e70 2023-09-27T23:52:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee006dac0 con 0x7f2ee006a540 2023-09-27T23:52:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00b0260 con 0x7f2ee00cc470 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ee0158e70 0x7f2ee00afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ee0158e70 0x7f2ee00afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37864/0 (socket says 172.21.15.64:37864) 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 -- 172.21.15.64:0/331318342 learned_addr learned my addr 172.21.15.64:0/331318342 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee5bb0700 1 --2- 172.21.15.64:0/331318342 >> v2:172.21.15.123:3300/0 conn(0x7f2ee006a540 0x7f2ee006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee6bb2700 1 --2- 172.21.15.64:0/331318342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee00cc470 0x7f2ee00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 -- 172.21.15.64:0/331318342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee00cc470 msgr2=0x7f2ee00cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 --2- 172.21.15.64:0/331318342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee00cc470 0x7f2ee00cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 -- 172.21.15.64:0/331318342 >> v2:172.21.15.123:3300/0 conn(0x7f2ee006a540 msgr2=0x7f2ee006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 --2- 172.21.15.64:0/331318342 >> v2:172.21.15.123:3300/0 conn(0x7f2ee006a540 0x7f2ee006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 -- 172.21.15.64:0/331318342 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee006d220 con 0x7f2ee0158e70 2023-09-27T23:52:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee5bb0700 1 --2- 172.21.15.64:0/331318342 >> v2:172.21.15.123:3300/0 conn(0x7f2ee006a540 0x7f2ee006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee63b1700 1 --2- 172.21.15.64:0/331318342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0158e70 0x7f2ee00afd20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00cfd20 tx=0x7f2edc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f544d9e3216ed1b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee53af700 1 -- 172.21.15.64:0/331318342 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00b860 con 0x7f2ee0158e70 2023-09-27T23:52:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee53af700 1 -- 172.21.15.64:0/331318342 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc00b9c0 con 0x7f2ee0158e70 2023-09-27T23:52:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee53af700 1 -- 172.21.15.64:0/331318342 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00bd80 con 0x7f2ee0158e70 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- 172.21.15.64:0/331318342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0158e70 msgr2=0x7f2ee00afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/331318342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0158e70 0x7f2ee00afd20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00cfd20 tx=0x7f2edc009250 comp rx=0 tx=0).stop 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- 172.21.15.64:0/331318342 shutdown_connections 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/331318342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee00cc470 0x7f2ee00cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/331318342 >> v2:172.21.15.123:3300/0 conn(0x7f2ee006a540 0x7f2ee006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/331318342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0158e70 0x7f2ee00afd20 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- 172.21.15.64:0/331318342 >> 172.21.15.64:0/331318342 conn(0x7f2ee00670e0 msgr2=0x7f2ee00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- 172.21.15.64:0/331318342 shutdown_connections 2023-09-27T23:52:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- 172.21.15.64:0/331318342 wait complete. 2023-09-27T23:52:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 Processor -- start 2023-09-27T23:52:24.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.492+0000 7f2ee73b3700 1 -- start start 2023-09-27T23:52:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 0x7f2ee00c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee00cc470 0x7f2ee00b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee0158e70 0x7f2ee0156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01690c0 con 0x7f2ee006a540 2023-09-27T23:52:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee015c190 con 0x7f2ee00cc470 2023-09-27T23:52:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee015b8f0 con 0x7f2ee0158e70 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 0x7f2ee00c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 0x7f2ee00c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37868/0 (socket says 172.21.15.64:37868) 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 -- 172.21.15.64:0/1515663483 learned_addr learned my addr 172.21.15.64:0/1515663483 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee5bb0700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee00cc470 0x7f2ee00b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee6bb2700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee0158e70 0x7f2ee0156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 -- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee0158e70 msgr2=0x7f2ee0156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee0158e70 0x7f2ee0156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 -- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee00cc470 msgr2=0x7f2ee00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee00cc470 0x7f2ee00b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 -- 172.21.15.64:0/1515663483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc0094f0 con 0x7f2ee006a540 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee63b1700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 0x7f2ee00c9620 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011e90 con 0x7f2ee006a540 2023-09-27T23:52:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc026450 con 0x7f2ee006a540 2023-09-27T23:52:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee015bbd0 con 0x7f2ee006a540 2023-09-27T23:52:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc026810 con 0x7f2ee006a540 2023-09-27T23:52:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.496+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee00c9b60 con 0x7f2ee006a540 2023-09-27T23:52:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.500+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee0168e90 con 0x7f2ee006a540 2023-09-27T23:52:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.500+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f2edc026bc0 con 0x7f2ee006a540 2023-09-27T23:52:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.500+0000 7f2ed67fc700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ec4060a70 0x7f2ec4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.500+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2edc07eca0 con 0x7f2ee006a540 2023-09-27T23:52:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.500+0000 7f2ee5bb0700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ec4060a70 0x7f2ec4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.500+0000 7f2ee5bb0700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ec4060a70 0x7f2ec4062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006f00 tx=0x7f2ed0008040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:24.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.504+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2edc04ecd0 con 0x7f2ee006a540 2023-09-27T23:52:24.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.668+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ee004be40 con 0x7f2ec4060a70 2023-09-27T23:52:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.688+0000 7f2ed67fc700 1 -- 172.21.15.64:0/1515663483 <== mgr.9940 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292741 (secure 0 0 0) 0x7f2ee004be40 con 0x7f2ec4060a70 2023-09-27T23:52:24.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ec4060a70 msgr2=0x7f2ec4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ec4060a70 0x7f2ec4062f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006f00 tx=0x7f2ed0008040 comp rx=0 tx=0).stop 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 msgr2=0x7f2ee00c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 0x7f2ee00c9620 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc017000 comp rx=0 tx=0).stop 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 shutdown_connections 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2ec4060a70 0x7f2ec4062f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ee0158e70 0x7f2ee0156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee00cc470 0x7f2ee00b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 --2- 172.21.15.64:0/1515663483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee006a540 0x7f2ee00c9620 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:24.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 >> 172.21.15.64:0/1515663483 conn(0x7f2ee00670e0 msgr2=0x7f2ee00cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:24.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 shutdown_connections 2023-09-27T23:52:24.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:24.692+0000 7f2ee73b3700 1 -- 172.21.15.64:0/1515663483 wait complete. 2023-09-27T23:52:24.699 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:24.712 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:52:23.281985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":275016,"kb_used_data":19208,"kb_used_omap":0,"kb_used_meta":255744,"kb_avail":754699704,"statfs":{"total":773094113280,"available":772812496896,"internally_reserved":0,"allocated":19668992,"data_stored":10911184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261881856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.390233"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"177'326","reported_seq":1047,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200927+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:17.200927+0000","last_peered":"2023-09-27T23:52:17.200927+0000","last_clean":"2023-09-27T23:52:17.200927+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:17.200927+0000","last_undegraded":"2023-09-27T23:52:17.200927+0000","last_fullsized":"2023-09-27T23:52:17.200927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":326,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710096+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:15.710096+0000","last_peered":"2023-09-27T23:52:15.710096+0000","last_clean":"2023-09-27T23:52:15.710096+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:15.710096+0000","last_undegraded":"2023-09-27T23:52:15.710096+0000","last_fullsized":"2023-09-27T23:52:15.710096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200923+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:17.200923+0000","last_peered":"2023-09-27T23:52:17.200923+0000","last_clean":"2023-09-27T23:52:17.200923+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:17.200923+0000","last_undegraded":"2023-09-27T23:52:17.200923+0000","last_fullsized":"2023-09-27T23:52:17.200923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:16.220162+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:16.220162+0000","last_peered":"2023-09-27T23:52:16.220162+0000","last_clean":"2023-09-27T23:52:16.220162+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:16.220162+0000","last_undegraded":"2023-09-27T23:52:16.220162+0000","last_fullsized":"2023-09-27T23:52:16.220162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.710169+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:15.710169+0000","last_peered":"2023-09-27T23:52:15.710169+0000","last_clean":"2023-09-27T23:52:15.710169+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:15.710169+0000","last_undegraded":"2023-09-27T23:52:15.710169+0000","last_fullsized":"2023-09-27T23:52:15.710169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705037+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:15.705037+0000","last_peered":"2023-09-27T23:52:15.705037+0000","last_clean":"2023-09-27T23:52:15.705037+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:15.705037+0000","last_undegraded":"2023-09-27T23:52:15.705037+0000","last_fullsized":"2023-09-27T23:52:15.705037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:17.200922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:17.200922+0000","last_peered":"2023-09-27T23:52:17.200922+0000","last_clean":"2023-09-27T23:52:17.200922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:17.200922+0000","last_undegraded":"2023-09-27T23:52:17.200922+0000","last_fullsized":"2023-09-27T23:52:17.200922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.705120+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:15.705120+0000","last_peered":"2023-09-27T23:52:15.705120+0000","last_clean":"2023-09-27T23:52:15.705120+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:15.705120+0000","last_undegraded":"2023-09-27T23:52:15.705120+0000","last_fullsized":"2023-09-27T23:52:15.705120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:19.885476+0000","last_change":"2023-09-27T23:52:19.885476+0000","last_active":"2023-09-27T23:52:19.885476+0000","last_peered":"2023-09-27T23:52:19.885476+0000","last_clean":"2023-09-27T23:52:19.885476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:19.885476+0000","last_undegraded":"2023-09-27T23:52:19.885476+0000","last_fullsized":"2023-09-27T23:52:19.885476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-27T23:52:18.872640+0000","last_change":"2023-09-27T23:52:18.872640+0000","last_active":"2023-09-27T23:52:18.872640+0000","last_peered":"2023-09-27T23:52:18.872640+0000","last_clean":"2023-09-27T23:52:18.872640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-27T23:52:18.872640+0000","last_undegraded":"2023-09-27T23:52:18.872640+0000","last_fullsized":"2023-09-27T23:52:18.872640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-09-27T23:52:15.833607+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:15.833607+0000","last_peered":"2023-09-27T23:52:15.833607+0000","last_clean":"2023-09-27T23:52:15.833607+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:15.833607+0000","last_undegraded":"2023-09-27T23:52:15.833607+0000","last_fullsized":"2023-09-27T23:52:15.833607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36636,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":33792,"kb_avail":94335204,"statfs":{"total":96636764160,"available":96599248896,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34603008},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.89600000000000002},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2250000000000001,"5min":1.569,"15min":1.569},"last":1.446},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.94899999999999995,"15min":0.93400000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.466,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.248,"5min":1.333,"15min":1.3360000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96799999999999997,"15min":0.97499999999999998},"min":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2929999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.2949999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.92500000000000004},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.3069999999999999,"5min":1.341,"15min":1.341},"last":1.004}]},{"osd":3,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.70799999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.202,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0760000000000001,"15min":1.079},"min":{"1min":0.77700000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.3879999999999999},"last":1.23}]},{"osd":4,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.216,"5min":1.821,"15min":1.821},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.03,"15min":1.0109999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.226,"5min":3.02,"15min":3.02},"last":1.0329999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0569999999999999,"15min":1.0620000000000001},"min":{"1min":0.75,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":1.339},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3759999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.131,"15min":1.143},"min":{"1min":0.749,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.423,"5min":1.423,"15min":1.448},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.145,"15min":1.145},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.381,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.268}]}]},{"osd":6,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30040,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":27776,"kb_avail":94341800,"statfs":{"total":96636764160,"available":96606003200,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28442624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.97199999999999998},"min":{"1min":0.70599999999999996,"5min":0.60599999999999998,"15min":0.53400000000000003},"max":{"1min":1.214,"5min":1.792,"15min":1.792},"last":1.016},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.80400000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90200000000000002,"15min":0.90300000000000002},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67100000000000004,"5min":0.622,"15min":0.57099999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89500000000000002,"15min":0.90600000000000003},"min":{"1min":0.78100000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.147,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.018,"5min":0.98399999999999999,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.78000000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91000000000000003,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.6639999999999999,"5min":1.7,"15min":1.7},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.079,"5min":1.0289999999999999,"15min":1.0029999999999999},"min":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3089999999999999,"5min":1.3700000000000001,"15min":1.633},"last":0.876}]},{"osd":5,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.161,"15min":1.145},"min":{"1min":0.89500000000000002,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.075}]},{"osd":7,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.81100000000000005,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":1.637,"15min":1.679},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.155,"15min":1.1359999999999999},"min":{"1min":0.69199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":37632,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38535168},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.95999999999999996,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.51900000000000002},"max":{"1min":10.058999999999999,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.657,"5min":1.0609999999999999,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.47899999999999998},"max":{"1min":13.757999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.96999999999999997,"15min":0.93200000000000005},"min":{"1min":0.70699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.1970000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.95999999999999996,"15min":0.96099999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.629},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.97699999999999998,"15min":0.95899999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.999,"15min":0.98999999999999999},"min":{"1min":0.84299999999999997,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.075,"15min":1.042},"min":{"1min":0.79000000000000004,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.747}]},{"osd":4,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91200000000000003,"15min":0.88900000000000001},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.04,"5min":1.0469999999999999,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3640000000000001,"5min":1.45,"15min":1.4510000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.119,"15min":1.105},"min":{"1min":0.91900000000000004,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.026}]},{"osd":7,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.222,"15min":1.169},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":6.3710000000000004,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.363,"15min":1.2629999999999999},"min":{"1min":1.0329999999999999,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":14.119,"5min":14.119,"15min":14.119},"last":1.087}]}]},{"osd":4,"up_from":15,"seq":64424509901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30872,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28608,"kb_avail":94340968,"statfs":{"total":96636764160,"available":96605151232,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29294592},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.45000000000000001},"max":{"1min":1.1240000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.1180000000000001,"5min":1.893,"15min":2.2770000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2010000000000001,"5min":1.746,"15min":1.746},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.53500000000000003},"max":{"1min":1.2390000000000001,"5min":1.758,"15min":2.1110000000000002},"last":1.339}]},{"osd":2,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2490000000000001,"5min":1.659,"15min":1.659},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98299999999999998,"15min":0.96099999999999997},"min":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.234,"5min":1.966,"15min":2.056},"last":1.216}]},{"osd":3,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93400000000000005,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.998,"15min":2.032},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.079,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":1.2709999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.5}]},{"osd":5,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.92400000000000004},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.45500000000000002},"max":{"1min":1.25,"5min":1.556,"15min":1.9930000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.006,"15min":1.0149999999999999},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3129999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.419}]},{"osd":6,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0529999999999999,"15min":1.0509999999999999},"min":{"1min":0.81399999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.1830000000000001,"15min":1.1679999999999999},"min":{"1min":0.92100000000000004,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.099,"15min":1.097},"min":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":1.4350000000000001,"5min":2.069,"15min":2.069},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.161,"15min":1.1739999999999999},"min":{"1min":0.74399999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.329,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.8400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30616,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28352,"kb_avail":94341224,"statfs":{"total":96636764160,"available":96605413376,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29032448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.081},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.0209999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.113},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1559999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.1759999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30488,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94341352,"statfs":{"total":96636764160,"available":96605544448,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1339999999999999,"5min":1.345,"15min":1.4670000000000001},"last":1.0509999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.84599999999999997},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.82999999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.57099999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.3640000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94399999999999995},"min":{"1min":0.71299999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.262,"15min":1.4359999999999999},"last":1.0920000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0600000000000001,"15min":1.0800000000000001},"min":{"1min":0.79900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.415,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.91800000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3560000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.3959999999999999,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1439999999999999,"15min":1.175},"min":{"1min":0.85199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5680000000000001,"5min":1.6459999999999999,"15min":1.8},"last":1.4399999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.96599999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.466,"5min":1.5780000000000001,"15min":1.718},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.196,"15min":1.1819999999999999},"min":{"1min":0.88600000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":1.6439999999999999,"15min":1.736},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2350000000000001,"15min":1.23},"min":{"1min":1.121,"5min":0.873,"15min":0.873},"max":{"1min":1.5349999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.232},"min":{"1min":0.98399999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.607,"5min":1.6970000000000001,"15min":1.8129999999999999},"last":1.1439999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542607,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44760,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":42496,"kb_avail":94327080,"statfs":{"total":96636764160,"available":96590929920,"internally_reserved":0,"allocated":2310144,"data_stored":1216306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43515904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.98599999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":1.034},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.014},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.94499999999999995}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.367}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.96299999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":28864,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":2904064,"data_stored":1806674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29556736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.113,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.47499999999999998},"max":{"1min":1.145,"5min":1.198,"15min":1.2470000000000001},"last":1.028}]},{"osd":2,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.502},"max":{"1min":1.169,"5min":1.169,"15min":1.2509999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.119,"5min":1.288,"15min":1.863},"last":0.82099999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88600000000000001,"15min":0.876},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2190000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.27,"15min":1.4390000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.042,"15min":1.052},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3160000000000001,"5min":1.573,"15min":1.7110000000000001},"last":1.365},{"interface":"front","average":{"1min":0.996,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.411,"5min":1.516,"15min":1.651},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.1100000000000001},"min":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.3380000000000001,"5min":1.444,"15min":1.512},"last":1.276},{"interface":"front","average":{"1min":1.131,"5min":1.1519999999999999,"15min":1.175},"min":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4810000000000001,"5min":1.5629999999999999,"15min":1.847},"last":1.171}]},{"osd":6,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.167,"15min":1.1870000000000001},"min":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.3500000000000001,"5min":1.6499999999999999,"15min":1.895},"last":1.329},{"interface":"front","average":{"1min":1.204,"5min":1.226,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.615,"5min":1.722,"15min":2.0510000000000002},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2589999999999999,"15min":1.2789999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.722,"5min":1.786,"15min":2.0030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4099999999999999,"5min":1.806,"15min":2.004},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:26.195 INFO:tasks.ceph.mds.b.smithi123.stderr:2023-09-27T23:52:26.195+0000 7f9d13a2a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:52:27.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:27.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.832+0000 7f42434a8700 1 Processor -- start 2023-09-27T23:52:27.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.832+0000 7f42434a8700 1 -- start start 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.832+0000 7f42434a8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c06be10 0x7f423c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.832+0000 7f42434a8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f423c0af9c0 0x7f423c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c0af530 con 0x7f423c0b0810 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c159470 con 0x7f423c0af9c0 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c159660 con 0x7f423c06be10 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f423c0af9c0 0x7f423c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:27.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f423c0af9c0 0x7f423c154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55892/0 (socket says 172.21.15.64:55892) 2023-09-27T23:52:27.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 -- 172.21.15.64:0/340221912 learned_addr learned my addr 172.21.15.64:0/340221912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:27.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42424a6700 1 --1- 172.21.15.64:0/340221912 >> v1:172.21.15.123:6789/0 conn(0x7f423c0b0810 0x7f423c158d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:41000/0 (socket says 172.21.15.64:41000) 2023-09-27T23:52:27.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4241ca5700 1 --2- 172.21.15.64:0/340221912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c06be10 0x7f423c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:27.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 -- 172.21.15.64:0/340221912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c06be10 msgr2=0x7f423c0b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:27.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 --2- 172.21.15.64:0/340221912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c06be10 0x7f423c0b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:27.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 -- 172.21.15.64:0/340221912 >> v1:172.21.15.123:6789/0 conn(0x7f423c0b0810 legacy=0x7f423c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:27.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42414a4700 1 -- 172.21.15.64:0/340221912 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109842449 0 0) 0x7f423c0af530 con 0x7f423c0b0810 2023-09-27T23:52:27.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 -- 172.21.15.64:0/340221912 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c159940 con 0x7f423c0af9c0 2023-09-27T23:52:27.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f4242ca7700 1 --2- 172.21.15.64:0/340221912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c154b20 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f423400c100 tx=0x7f423400a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736a07531e67559f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42414a4700 1 -- 172.21.15.64:0/340221912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400f070 con 0x7f423c0af9c0 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42414a4700 1 -- 172.21.15.64:0/340221912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400cdc0 con 0x7f423c0af9c0 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42414a4700 1 -- 172.21.15.64:0/340221912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400a320 con 0x7f423c0af9c0 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- 172.21.15.64:0/340221912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 msgr2=0x7f423c154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 --2- 172.21.15.64:0/340221912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c154b20 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f423400c100 tx=0x7f423400a950 comp rx=0 tx=0).stop 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- 172.21.15.64:0/340221912 shutdown_connections 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 --2- 172.21.15.64:0/340221912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c154b20 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 --2- 172.21.15.64:0/340221912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c06be10 0x7f423c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- 172.21.15.64:0/340221912 >> 172.21.15.64:0/340221912 conn(0x7f423c067bb0 msgr2=0x7f423c0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- 172.21.15.64:0/340221912 shutdown_connections 2023-09-27T23:52:27.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.836+0000 7f42434a8700 1 -- 172.21.15.64:0/340221912 wait complete. 2023-09-27T23:52:27.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 Processor -- start 2023-09-27T23:52:27.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 -- start start 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f423c06be10 0x7f423c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c0b0810 0x7f423c142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1696c0 con 0x7f423c0af9c0 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0c6760 con 0x7f423c06be10 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c16a640 con 0x7f423c0b0810 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:27.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c0c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55908/0 (socket says 172.21.15.64:55908) 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 -- 172.21.15.64:0/4034084591 learned_addr learned my addr 172.21.15.64:0/4034084591 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42424a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f423c06be10 0x7f423c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4242ca7700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c0b0810 0x7f423c142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 -- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c0b0810 msgr2=0x7f423c142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c0b0810 0x7f423c142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 -- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f423c06be10 msgr2=0x7f423c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f423c06be10 0x7f423c0cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 -- 172.21.15.64:0/4034084591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423400abf0 con 0x7f423c0af9c0 2023-09-27T23:52:27.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f4241ca5700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c0c6220 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f422c00ecb0 tx=0x7f422c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c00c850 con 0x7f423c0af9c0 2023-09-27T23:52:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c00c9b0 con 0x7f423c0af9c0 2023-09-27T23:52:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c16a1b0 con 0x7f423c0af9c0 2023-09-27T23:52:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c00cd50 con 0x7f423c0af9c0 2023-09-27T23:52:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.840+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c169b60 con 0x7f423c0af9c0 2023-09-27T23:52:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.844+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c04be40 con 0x7f423c0af9c0 2023-09-27T23:52:27.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.844+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f422c013680 con 0x7f423c0af9c0 2023-09-27T23:52:27.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.844+0000 7f42327fc700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4220060a70 0x7f4220062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:27.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.844+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f422c07d3a0 con 0x7f423c0af9c0 2023-09-27T23:52:27.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.844+0000 7f42424a6700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4220060a70 0x7f4220062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:27.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.844+0000 7f42424a6700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4220060a70 0x7f4220062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4238007180 tx=0x7f4238007ef0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:27.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:27.852+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f422c04d330 con 0x7f423c0af9c0 2023-09-27T23:52:28.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.016+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f423c157140 con 0x7f4220060a70 2023-09-27T23:52:28.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.036+0000 7f42327fc700 1 -- 172.21.15.64:0/4034084591 <== mgr.9940 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307803 (secure 0 0 0) 0x7f423c157140 con 0x7f4220060a70 2023-09-27T23:52:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:28.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4220060a70 msgr2=0x7f4220062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4220060a70 0x7f4220062f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4238007180 tx=0x7f4238007ef0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 msgr2=0x7f423c0c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c0c6220 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f422c00ecb0 tx=0x7f422c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 shutdown_connections 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4220060a70 0x7f4220062f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f423c0b0810 0x7f423c142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f423c0af9c0 0x7f423c0c6220 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 --2- 172.21.15.64:0/4034084591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f423c06be10 0x7f423c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 >> 172.21.15.64:0/4034084591 conn(0x7f423c067bb0 msgr2=0x7f423c157ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 shutdown_connections 2023-09-27T23:52:28.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.040+0000 7f42434a8700 1 -- 172.21.15.64:0/4034084591 wait complete. 2023-09-27T23:52:28.048 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:28.062 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-09-27T23:52:27.283554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276784,"kb_used_data":19632,"kb_used_omap":0,"kb_used_meta":257088,"kb_avail":754697936,"statfs":{"total":773094113280,"available":772810686464,"internally_reserved":0,"allocated":20103168,"data_stored":11124362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":263258112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.381379"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.323177+0000","last_change":"2023-09-27T23:52:20.319770+0000","last_active":"2023-09-27T23:52:20.323177+0000","last_peered":"2023-09-27T23:52:20.323177+0000","last_clean":"2023-09-27T23:52:20.323177+0000","last_became_active":"2023-09-27T23:52:20.319436+0000","last_became_peered":"2023-09-27T23:52:20.319436+0000","last_unstale":"2023-09-27T23:52:20.323177+0000","last_undegraded":"2023-09-27T23:52:20.323177+0000","last_fullsized":"2023-09-27T23:52:20.323177+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:49:13.078344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.299718+0000","last_change":"2023-09-27T23:52:20.299718+0000","last_active":"2023-09-27T23:52:20.299718+0000","last_peered":"2023-09-27T23:52:20.299718+0000","last_clean":"2023-09-27T23:52:20.299718+0000","last_became_active":"2023-09-27T23:52:20.299327+0000","last_became_peered":"2023-09-27T23:52:20.299327+0000","last_unstale":"2023-09-27T23:52:20.299718+0000","last_undegraded":"2023-09-27T23:52:20.299718+0000","last_fullsized":"2023-09-27T23:52:20.299718+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:25:18.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.297202+0000","last_change":"2023-09-27T23:52:20.297202+0000","last_active":"2023-09-27T23:52:20.297202+0000","last_peered":"2023-09-27T23:52:20.297202+0000","last_clean":"2023-09-27T23:52:20.297202+0000","last_became_active":"2023-09-27T23:52:20.296917+0000","last_became_peered":"2023-09-27T23:52:20.296917+0000","last_unstale":"2023-09-27T23:52:20.297202+0000","last_undegraded":"2023-09-27T23:52:20.297202+0000","last_fullsized":"2023-09-27T23:52:20.297202+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:52.500936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.307075+0000","last_change":"2023-09-27T23:52:20.307075+0000","last_active":"2023-09-27T23:52:20.307075+0000","last_peered":"2023-09-27T23:52:20.307075+0000","last_clean":"2023-09-27T23:52:20.307075+0000","last_became_active":"2023-09-27T23:52:20.306623+0000","last_became_peered":"2023-09-27T23:52:20.306623+0000","last_unstale":"2023-09-27T23:52:20.307075+0000","last_undegraded":"2023-09-27T23:52:20.307075+0000","last_fullsized":"2023-09-27T23:52:20.307075+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:51.036181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304432+0000","last_change":"2023-09-27T23:52:20.299855+0000","last_active":"2023-09-27T23:52:20.304432+0000","last_peered":"2023-09-27T23:52:20.304432+0000","last_clean":"2023-09-27T23:52:20.304432+0000","last_became_active":"2023-09-27T23:52:20.299391+0000","last_became_peered":"2023-09-27T23:52:20.299391+0000","last_unstale":"2023-09-27T23:52:20.304432+0000","last_undegraded":"2023-09-27T23:52:20.304432+0000","last_fullsized":"2023-09-27T23:52:20.304432+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:28.153330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302793+0000","last_change":"2023-09-27T23:52:20.302793+0000","last_active":"2023-09-27T23:52:20.302793+0000","last_peered":"2023-09-27T23:52:20.302793+0000","last_clean":"2023-09-27T23:52:20.302793+0000","last_became_active":"2023-09-27T23:52:20.301470+0000","last_became_peered":"2023-09-27T23:52:20.301470+0000","last_unstale":"2023-09-27T23:52:20.302793+0000","last_undegraded":"2023-09-27T23:52:20.302793+0000","last_fullsized":"2023-09-27T23:52:20.302793+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:28:34.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.320810+0000","last_change":"2023-09-27T23:52:20.315550+0000","last_active":"2023-09-27T23:52:20.320810+0000","last_peered":"2023-09-27T23:52:20.320810+0000","last_clean":"2023-09-27T23:52:20.320810+0000","last_became_active":"2023-09-27T23:52:20.315239+0000","last_became_peered":"2023-09-27T23:52:20.315239+0000","last_unstale":"2023-09-27T23:52:20.320810+0000","last_undegraded":"2023-09-27T23:52:20.320810+0000","last_fullsized":"2023-09-27T23:52:20.320810+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:43:44.379107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.313184+0000","last_change":"2023-09-27T23:52:20.310465+0000","last_active":"2023-09-27T23:52:20.313184+0000","last_peered":"2023-09-27T23:52:20.313184+0000","last_clean":"2023-09-27T23:52:20.313184+0000","last_became_active":"2023-09-27T23:52:20.309869+0000","last_became_peered":"2023-09-27T23:52:20.309869+0000","last_unstale":"2023-09-27T23:52:20.313184+0000","last_undegraded":"2023-09-27T23:52:20.313184+0000","last_fullsized":"2023-09-27T23:52:20.313184+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:28.803598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.309248+0000","last_change":"2023-09-27T23:52:20.304114+0000","last_active":"2023-09-27T23:52:20.309248+0000","last_peered":"2023-09-27T23:52:20.309248+0000","last_clean":"2023-09-27T23:52:20.309248+0000","last_became_active":"2023-09-27T23:52:20.303748+0000","last_became_peered":"2023-09-27T23:52:20.303748+0000","last_unstale":"2023-09-27T23:52:20.309248+0000","last_undegraded":"2023-09-27T23:52:20.309248+0000","last_fullsized":"2023-09-27T23:52:20.309248+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:17:11.567762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302095+0000","last_change":"2023-09-27T23:52:20.296083+0000","last_active":"2023-09-27T23:52:20.302095+0000","last_peered":"2023-09-27T23:52:20.302095+0000","last_clean":"2023-09-27T23:52:20.302095+0000","last_became_active":"2023-09-27T23:52:20.295849+0000","last_became_peered":"2023-09-27T23:52:20.295849+0000","last_unstale":"2023-09-27T23:52:20.302095+0000","last_undegraded":"2023-09-27T23:52:20.302095+0000","last_fullsized":"2023-09-27T23:52:20.302095+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:46.987460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.322179+0000","last_change":"2023-09-27T23:52:20.319494+0000","last_active":"2023-09-27T23:52:20.322179+0000","last_peered":"2023-09-27T23:52:20.322179+0000","last_clean":"2023-09-27T23:52:20.322179+0000","last_became_active":"2023-09-27T23:52:20.319205+0000","last_became_peered":"2023-09-27T23:52:20.319205+0000","last_unstale":"2023-09-27T23:52:20.322179+0000","last_undegraded":"2023-09-27T23:52:20.322179+0000","last_fullsized":"2023-09-27T23:52:20.322179+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:13:21.277852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.303943+0000","last_change":"2023-09-27T23:52:20.303943+0000","last_active":"2023-09-27T23:52:20.303943+0000","last_peered":"2023-09-27T23:52:20.303943+0000","last_clean":"2023-09-27T23:52:20.303943+0000","last_became_active":"2023-09-27T23:52:20.303266+0000","last_became_peered":"2023-09-27T23:52:20.303266+0000","last_unstale":"2023-09-27T23:52:20.303943+0000","last_undegraded":"2023-09-27T23:52:20.303943+0000","last_fullsized":"2023-09-27T23:52:20.303943+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:41:56.282379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.313866+0000","last_change":"2023-09-27T23:52:20.313866+0000","last_active":"2023-09-27T23:52:20.313866+0000","last_peered":"2023-09-27T23:52:20.313866+0000","last_clean":"2023-09-27T23:52:20.313866+0000","last_became_active":"2023-09-27T23:52:20.313574+0000","last_became_peered":"2023-09-27T23:52:20.313574+0000","last_unstale":"2023-09-27T23:52:20.313866+0000","last_undegraded":"2023-09-27T23:52:20.313866+0000","last_fullsized":"2023-09-27T23:52:20.313866+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:46:42.425649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.312726+0000","last_change":"2023-09-27T23:52:20.307264+0000","last_active":"2023-09-27T23:52:20.312726+0000","last_peered":"2023-09-27T23:52:20.312726+0000","last_clean":"2023-09-27T23:52:20.312726+0000","last_became_active":"2023-09-27T23:52:20.304597+0000","last_became_peered":"2023-09-27T23:52:20.304597+0000","last_unstale":"2023-09-27T23:52:20.312726+0000","last_undegraded":"2023-09-27T23:52:20.312726+0000","last_fullsized":"2023-09-27T23:52:20.312726+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:53:37.706666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302786+0000","last_change":"2023-09-27T23:52:20.302786+0000","last_active":"2023-09-27T23:52:20.302786+0000","last_peered":"2023-09-27T23:52:20.302786+0000","last_clean":"2023-09-27T23:52:20.302786+0000","last_became_active":"2023-09-27T23:52:20.302503+0000","last_became_peered":"2023-09-27T23:52:20.302503+0000","last_unstale":"2023-09-27T23:52:20.302786+0000","last_undegraded":"2023-09-27T23:52:20.302786+0000","last_fullsized":"2023-09-27T23:52:20.302786+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:44:59.566006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304418+0000","last_change":"2023-09-27T23:52:20.300784+0000","last_active":"2023-09-27T23:52:20.304418+0000","last_peered":"2023-09-27T23:52:20.304418+0000","last_clean":"2023-09-27T23:52:20.304418+0000","last_became_active":"2023-09-27T23:52:20.300490+0000","last_became_peered":"2023-09-27T23:52:20.300490+0000","last_unstale":"2023-09-27T23:52:20.304418+0000","last_undegraded":"2023-09-27T23:52:20.304418+0000","last_fullsized":"2023-09-27T23:52:20.304418+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:08:36.246217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304376+0000","last_change":"2023-09-27T23:52:20.299859+0000","last_active":"2023-09-27T23:52:20.304376+0000","last_peered":"2023-09-27T23:52:20.304376+0000","last_clean":"2023-09-27T23:52:20.304376+0000","last_became_active":"2023-09-27T23:52:20.299408+0000","last_became_peered":"2023-09-27T23:52:20.299408+0000","last_unstale":"2023-09-27T23:52:20.304376+0000","last_undegraded":"2023-09-27T23:52:20.304376+0000","last_fullsized":"2023-09-27T23:52:20.304376+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:11:04.716716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.323059+0000","last_change":"2023-09-27T23:52:20.317322+0000","last_active":"2023-09-27T23:52:20.323059+0000","last_peered":"2023-09-27T23:52:20.323059+0000","last_clean":"2023-09-27T23:52:20.323059+0000","last_became_active":"2023-09-27T23:52:20.317022+0000","last_became_peered":"2023-09-27T23:52:20.317022+0000","last_unstale":"2023-09-27T23:52:20.323059+0000","last_undegraded":"2023-09-27T23:52:20.323059+0000","last_fullsized":"2023-09-27T23:52:20.323059+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:39:47.670685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.301906+0000","last_change":"2023-09-27T23:52:20.295583+0000","last_active":"2023-09-27T23:52:20.301906+0000","last_peered":"2023-09-27T23:52:20.301906+0000","last_clean":"2023-09-27T23:52:20.301906+0000","last_became_active":"2023-09-27T23:52:20.295322+0000","last_became_peered":"2023-09-27T23:52:20.295322+0000","last_unstale":"2023-09-27T23:52:20.301906+0000","last_undegraded":"2023-09-27T23:52:20.301906+0000","last_fullsized":"2023-09-27T23:52:20.301906+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:43:55.951880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304035+0000","last_change":"2023-09-27T23:52:20.298721+0000","last_active":"2023-09-27T23:52:20.304035+0000","last_peered":"2023-09-27T23:52:20.304035+0000","last_clean":"2023-09-27T23:52:20.304035+0000","last_became_active":"2023-09-27T23:52:20.298437+0000","last_became_peered":"2023-09-27T23:52:20.298437+0000","last_unstale":"2023-09-27T23:52:20.304035+0000","last_undegraded":"2023-09-27T23:52:20.304035+0000","last_fullsized":"2023-09-27T23:52:20.304035+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:08:38.543159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304412+0000","last_change":"2023-09-27T23:52:20.299669+0000","last_active":"2023-09-27T23:52:20.304412+0000","last_peered":"2023-09-27T23:52:20.304412+0000","last_clean":"2023-09-27T23:52:20.304412+0000","last_became_active":"2023-09-27T23:52:20.299030+0000","last_became_peered":"2023-09-27T23:52:20.299030+0000","last_unstale":"2023-09-27T23:52:20.304412+0000","last_undegraded":"2023-09-27T23:52:20.304412+0000","last_fullsized":"2023-09-27T23:52:20.304412+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:56:48.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.312637+0000","last_change":"2023-09-27T23:52:20.301911+0000","last_active":"2023-09-27T23:52:20.312637+0000","last_peered":"2023-09-27T23:52:20.312637+0000","last_clean":"2023-09-27T23:52:20.312637+0000","last_became_active":"2023-09-27T23:52:20.298622+0000","last_became_peered":"2023-09-27T23:52:20.298622+0000","last_unstale":"2023-09-27T23:52:20.312637+0000","last_undegraded":"2023-09-27T23:52:20.312637+0000","last_fullsized":"2023-09-27T23:52:20.312637+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:28.166692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.301588+0000","last_change":"2023-09-27T23:52:20.301588+0000","last_active":"2023-09-27T23:52:20.301588+0000","last_peered":"2023-09-27T23:52:20.301588+0000","last_clean":"2023-09-27T23:52:20.301588+0000","last_became_active":"2023-09-27T23:52:20.301234+0000","last_became_peered":"2023-09-27T23:52:20.301234+0000","last_unstale":"2023-09-27T23:52:20.301588+0000","last_undegraded":"2023-09-27T23:52:20.301588+0000","last_fullsized":"2023-09-27T23:52:20.301588+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:16:36.045037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.305284+0000","last_change":"2023-09-27T23:52:20.303461+0000","last_active":"2023-09-27T23:52:20.305284+0000","last_peered":"2023-09-27T23:52:20.305284+0000","last_clean":"2023-09-27T23:52:20.305284+0000","last_became_active":"2023-09-27T23:52:20.303173+0000","last_became_peered":"2023-09-27T23:52:20.303173+0000","last_unstale":"2023-09-27T23:52:20.305284+0000","last_undegraded":"2023-09-27T23:52:20.305284+0000","last_fullsized":"2023-09-27T23:52:20.305284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:51:03.588700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304201+0000","last_change":"2023-09-27T23:52:20.304201+0000","last_active":"2023-09-27T23:52:20.304201+0000","last_peered":"2023-09-27T23:52:20.304201+0000","last_clean":"2023-09-27T23:52:20.304201+0000","last_became_active":"2023-09-27T23:52:20.303506+0000","last_became_peered":"2023-09-27T23:52:20.303506+0000","last_unstale":"2023-09-27T23:52:20.304201+0000","last_undegraded":"2023-09-27T23:52:20.304201+0000","last_fullsized":"2023-09-27T23:52:20.304201+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:24.482351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.320776+0000","last_change":"2023-09-27T23:52:20.315293+0000","last_active":"2023-09-27T23:52:20.320776+0000","last_peered":"2023-09-27T23:52:20.320776+0000","last_clean":"2023-09-27T23:52:20.320776+0000","last_became_active":"2023-09-27T23:52:20.314932+0000","last_became_peered":"2023-09-27T23:52:20.314932+0000","last_unstale":"2023-09-27T23:52:20.320776+0000","last_undegraded":"2023-09-27T23:52:20.320776+0000","last_fullsized":"2023-09-27T23:52:20.320776+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:23:54.680402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.301898+0000","last_change":"2023-09-27T23:52:20.301898+0000","last_active":"2023-09-27T23:52:20.301898+0000","last_peered":"2023-09-27T23:52:20.301898+0000","last_clean":"2023-09-27T23:52:20.301898+0000","last_became_active":"2023-09-27T23:52:20.301575+0000","last_became_peered":"2023-09-27T23:52:20.301575+0000","last_unstale":"2023-09-27T23:52:20.301898+0000","last_undegraded":"2023-09-27T23:52:20.301898+0000","last_fullsized":"2023-09-27T23:52:20.301898+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:40:35.498812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304014+0000","last_change":"2023-09-27T23:52:20.298334+0000","last_active":"2023-09-27T23:52:20.304014+0000","last_peered":"2023-09-27T23:52:20.304014+0000","last_clean":"2023-09-27T23:52:20.304014+0000","last_became_active":"2023-09-27T23:52:20.298036+0000","last_became_peered":"2023-09-27T23:52:20.298036+0000","last_unstale":"2023-09-27T23:52:20.304014+0000","last_undegraded":"2023-09-27T23:52:20.304014+0000","last_fullsized":"2023-09-27T23:52:20.304014+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:51.036181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.317846+0000","last_change":"2023-09-27T23:52:20.317846+0000","last_active":"2023-09-27T23:52:20.317846+0000","last_peered":"2023-09-27T23:52:20.317846+0000","last_clean":"2023-09-27T23:52:20.317846+0000","last_became_active":"2023-09-27T23:52:20.317521+0000","last_became_peered":"2023-09-27T23:52:20.317521+0000","last_unstale":"2023-09-27T23:52:20.317846+0000","last_undegraded":"2023-09-27T23:52:20.317846+0000","last_fullsized":"2023-09-27T23:52:20.317846+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:59:19.179321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.316257+0000","last_change":"2023-09-27T23:52:20.315209+0000","last_active":"2023-09-27T23:52:20.316257+0000","last_peered":"2023-09-27T23:52:20.316257+0000","last_clean":"2023-09-27T23:52:20.316257+0000","last_became_active":"2023-09-27T23:52:20.314912+0000","last_became_peered":"2023-09-27T23:52:20.314912+0000","last_unstale":"2023-09-27T23:52:20.316257+0000","last_undegraded":"2023-09-27T23:52:20.316257+0000","last_fullsized":"2023-09-27T23:52:20.316257+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:43:10.009695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.301831+0000","last_change":"2023-09-27T23:52:20.301831+0000","last_active":"2023-09-27T23:52:20.301831+0000","last_peered":"2023-09-27T23:52:20.301831+0000","last_clean":"2023-09-27T23:52:20.301831+0000","last_became_active":"2023-09-27T23:52:20.301563+0000","last_became_peered":"2023-09-27T23:52:20.301563+0000","last_unstale":"2023-09-27T23:52:20.301831+0000","last_undegraded":"2023-09-27T23:52:20.301831+0000","last_fullsized":"2023-09-27T23:52:20.301831+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:08:13.492091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.315520+0000","last_change":"2023-09-27T23:52:20.311397+0000","last_active":"2023-09-27T23:52:20.315520+0000","last_peered":"2023-09-27T23:52:20.315520+0000","last_clean":"2023-09-27T23:52:20.315520+0000","last_became_active":"2023-09-27T23:52:20.311097+0000","last_became_peered":"2023-09-27T23:52:20.311097+0000","last_unstale":"2023-09-27T23:52:20.315520+0000","last_undegraded":"2023-09-27T23:52:20.315520+0000","last_fullsized":"2023-09-27T23:52:20.315520+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:44:59.566006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.297940+0000","last_change":"2023-09-27T23:52:20.297940+0000","last_active":"2023-09-27T23:52:20.297940+0000","last_peered":"2023-09-27T23:52:20.297940+0000","last_clean":"2023-09-27T23:52:20.297940+0000","last_became_active":"2023-09-27T23:52:20.297653+0000","last_became_peered":"2023-09-27T23:52:20.297653+0000","last_unstale":"2023-09-27T23:52:20.297940+0000","last_undegraded":"2023-09-27T23:52:20.297940+0000","last_fullsized":"2023-09-27T23:52:20.297940+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:32:33.317732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302226+0000","last_change":"2023-09-27T23:52:19.925512+0000","last_active":"2023-09-27T23:52:20.302226+0000","last_peered":"2023-09-27T23:52:20.302226+0000","last_clean":"2023-09-27T23:52:20.302226+0000","last_became_active":"2023-09-27T23:52:19.925191+0000","last_became_peered":"2023-09-27T23:52:19.925191+0000","last_unstale":"2023-09-27T23:52:20.302226+0000","last_undegraded":"2023-09-27T23:52:20.302226+0000","last_fullsized":"2023-09-27T23:52:20.302226+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:02:57.773554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.303235+0000","last_change":"2023-09-27T23:52:20.300065+0000","last_active":"2023-09-27T23:52:20.303235+0000","last_peered":"2023-09-27T23:52:20.303235+0000","last_clean":"2023-09-27T23:52:20.303235+0000","last_became_active":"2023-09-27T23:52:20.299829+0000","last_became_peered":"2023-09-27T23:52:20.299829+0000","last_unstale":"2023-09-27T23:52:20.303235+0000","last_undegraded":"2023-09-27T23:52:20.303235+0000","last_fullsized":"2023-09-27T23:52:20.303235+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:38:18.100761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.288126+0000","last_change":"2023-09-27T23:52:19.930645+0000","last_active":"2023-09-27T23:52:20.288126+0000","last_peered":"2023-09-27T23:52:20.288126+0000","last_clean":"2023-09-27T23:52:20.288126+0000","last_became_active":"2023-09-27T23:52:19.930140+0000","last_became_peered":"2023-09-27T23:52:19.930140+0000","last_unstale":"2023-09-27T23:52:20.288126+0000","last_undegraded":"2023-09-27T23:52:20.288126+0000","last_fullsized":"2023-09-27T23:52:20.288126+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:12:00.388045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.318541+0000","last_change":"2023-09-27T23:52:20.314119+0000","last_active":"2023-09-27T23:52:20.318541+0000","last_peered":"2023-09-27T23:52:20.318541+0000","last_clean":"2023-09-27T23:52:20.318541+0000","last_became_active":"2023-09-27T23:52:20.313825+0000","last_became_peered":"2023-09-27T23:52:20.313825+0000","last_unstale":"2023-09-27T23:52:20.318541+0000","last_undegraded":"2023-09-27T23:52:20.318541+0000","last_fullsized":"2023-09-27T23:52:20.318541+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:40:54.514135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"188'3","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.298896+0000","last_change":"2023-09-27T23:52:19.931467+0000","last_active":"2023-09-27T23:52:20.298896+0000","last_peered":"2023-09-27T23:52:20.298896+0000","last_clean":"2023-09-27T23:52:20.298896+0000","last_became_active":"2023-09-27T23:52:19.931102+0000","last_became_peered":"2023-09-27T23:52:19.931102+0000","last_unstale":"2023-09-27T23:52:20.298896+0000","last_undegraded":"2023-09-27T23:52:20.298896+0000","last_fullsized":"2023-09-27T23:52:20.298896+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:07:13.171579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.308231+0000","last_change":"2023-09-27T23:52:20.298876+0000","last_active":"2023-09-27T23:52:20.308231+0000","last_peered":"2023-09-27T23:52:20.308231+0000","last_clean":"2023-09-27T23:52:20.308231+0000","last_became_active":"2023-09-27T23:52:20.298107+0000","last_became_peered":"2023-09-27T23:52:20.298107+0000","last_unstale":"2023-09-27T23:52:20.308231+0000","last_undegraded":"2023-09-27T23:52:20.308231+0000","last_fullsized":"2023-09-27T23:52:20.308231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:14.745804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"188'6","reported_seq":15,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.309760+0000","last_change":"2023-09-27T23:52:19.922635+0000","last_active":"2023-09-27T23:52:20.309760+0000","last_peered":"2023-09-27T23:52:20.309760+0000","last_clean":"2023-09-27T23:52:20.309760+0000","last_became_active":"2023-09-27T23:52:19.922308+0000","last_became_peered":"2023-09-27T23:52:19.922308+0000","last_unstale":"2023-09-27T23:52:20.309760+0000","last_undegraded":"2023-09-27T23:52:20.309760+0000","last_fullsized":"2023-09-27T23:52:20.309760+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:07:47.715772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.301127+0000","last_change":"2023-09-27T23:52:20.301127+0000","last_active":"2023-09-27T23:52:20.301127+0000","last_peered":"2023-09-27T23:52:20.301127+0000","last_clean":"2023-09-27T23:52:20.301127+0000","last_became_active":"2023-09-27T23:52:20.298887+0000","last_became_peered":"2023-09-27T23:52:20.298887+0000","last_unstale":"2023-09-27T23:52:20.301127+0000","last_undegraded":"2023-09-27T23:52:20.301127+0000","last_fullsized":"2023-09-27T23:52:20.301127+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:31:55.005913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"188'2","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.321521+0000","last_change":"2023-09-27T23:52:19.921777+0000","last_active":"2023-09-27T23:52:20.321521+0000","last_peered":"2023-09-27T23:52:20.321521+0000","last_clean":"2023-09-27T23:52:20.321521+0000","last_became_active":"2023-09-27T23:52:19.921305+0000","last_became_peered":"2023-09-27T23:52:19.921305+0000","last_unstale":"2023-09-27T23:52:20.321521+0000","last_undegraded":"2023-09-27T23:52:20.321521+0000","last_fullsized":"2023-09-27T23:52:20.321521+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:07:14.396964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.320853+0000","last_change":"2023-09-27T23:52:20.316240+0000","last_active":"2023-09-27T23:52:20.320853+0000","last_peered":"2023-09-27T23:52:20.320853+0000","last_clean":"2023-09-27T23:52:20.320853+0000","last_became_active":"2023-09-27T23:52:20.315961+0000","last_became_peered":"2023-09-27T23:52:20.315961+0000","last_unstale":"2023-09-27T23:52:20.320853+0000","last_undegraded":"2023-09-27T23:52:20.320853+0000","last_fullsized":"2023-09-27T23:52:20.320853+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:16:28.084985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.283480+0000","last_change":"2023-09-27T23:52:19.932073+0000","last_active":"2023-09-27T23:52:20.283480+0000","last_peered":"2023-09-27T23:52:20.283480+0000","last_clean":"2023-09-27T23:52:20.283480+0000","last_became_active":"2023-09-27T23:52:19.931733+0000","last_became_peered":"2023-09-27T23:52:19.931733+0000","last_unstale":"2023-09-27T23:52:20.283480+0000","last_undegraded":"2023-09-27T23:52:20.283480+0000","last_fullsized":"2023-09-27T23:52:20.283480+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:43.148538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.307677+0000","last_change":"2023-09-27T23:52:20.300494+0000","last_active":"2023-09-27T23:52:20.307677+0000","last_peered":"2023-09-27T23:52:20.307677+0000","last_clean":"2023-09-27T23:52:20.307677+0000","last_became_active":"2023-09-27T23:52:20.300162+0000","last_became_peered":"2023-09-27T23:52:20.300162+0000","last_unstale":"2023-09-27T23:52:20.307677+0000","last_undegraded":"2023-09-27T23:52:20.307677+0000","last_fullsized":"2023-09-27T23:52:20.307677+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:27:49.778010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.285687+0000","last_change":"2023-09-27T23:52:19.923539+0000","last_active":"2023-09-27T23:52:20.285687+0000","last_peered":"2023-09-27T23:52:20.285687+0000","last_clean":"2023-09-27T23:52:20.285687+0000","last_became_active":"2023-09-27T23:52:19.923229+0000","last_became_peered":"2023-09-27T23:52:19.923229+0000","last_unstale":"2023-09-27T23:52:20.285687+0000","last_undegraded":"2023-09-27T23:52:20.285687+0000","last_fullsized":"2023-09-27T23:52:20.285687+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:22.106678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.315069+0000","last_change":"2023-09-27T23:52:20.308037+0000","last_active":"2023-09-27T23:52:20.315069+0000","last_peered":"2023-09-27T23:52:20.315069+0000","last_clean":"2023-09-27T23:52:20.315069+0000","last_became_active":"2023-09-27T23:52:20.307444+0000","last_became_peered":"2023-09-27T23:52:20.307444+0000","last_unstale":"2023-09-27T23:52:20.315069+0000","last_undegraded":"2023-09-27T23:52:20.315069+0000","last_fullsized":"2023-09-27T23:52:20.315069+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:29:11.263279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"188'8","reported_seq":18,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.323116+0000","last_change":"2023-09-27T23:52:19.921630+0000","last_active":"2023-09-27T23:52:20.323116+0000","last_peered":"2023-09-27T23:52:20.323116+0000","last_clean":"2023-09-27T23:52:20.323116+0000","last_became_active":"2023-09-27T23:52:19.921197+0000","last_became_peered":"2023-09-27T23:52:19.921197+0000","last_unstale":"2023-09-27T23:52:20.323116+0000","last_undegraded":"2023-09-27T23:52:20.323116+0000","last_fullsized":"2023-09-27T23:52:20.323116+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:27.140494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.313147+0000","last_change":"2023-09-27T23:52:20.307674+0000","last_active":"2023-09-27T23:52:20.313147+0000","last_peered":"2023-09-27T23:52:20.313147+0000","last_clean":"2023-09-27T23:52:20.313147+0000","last_became_active":"2023-09-27T23:52:20.307372+0000","last_became_peered":"2023-09-27T23:52:20.307372+0000","last_unstale":"2023-09-27T23:52:20.313147+0000","last_undegraded":"2023-09-27T23:52:20.313147+0000","last_fullsized":"2023-09-27T23:52:20.313147+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:52:25.571632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"188'1","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.299171+0000","last_change":"2023-09-27T23:52:19.930030+0000","last_active":"2023-09-27T23:52:20.299171+0000","last_peered":"2023-09-27T23:52:20.299171+0000","last_clean":"2023-09-27T23:52:20.299171+0000","last_became_active":"2023-09-27T23:52:19.929723+0000","last_became_peered":"2023-09-27T23:52:19.929723+0000","last_unstale":"2023-09-27T23:52:20.299171+0000","last_undegraded":"2023-09-27T23:52:20.299171+0000","last_fullsized":"2023-09-27T23:52:20.299171+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:05:09.881355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.307213+0000","last_change":"2023-09-27T23:52:20.301782+0000","last_active":"2023-09-27T23:52:20.307213+0000","last_peered":"2023-09-27T23:52:20.307213+0000","last_clean":"2023-09-27T23:52:20.307213+0000","last_became_active":"2023-09-27T23:52:20.301383+0000","last_became_peered":"2023-09-27T23:52:20.301383+0000","last_unstale":"2023-09-27T23:52:20.307213+0000","last_undegraded":"2023-09-27T23:52:20.307213+0000","last_fullsized":"2023-09-27T23:52:20.307213+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:05:10.894191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"188'2","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304545+0000","last_change":"2023-09-27T23:52:19.921340+0000","last_active":"2023-09-27T23:52:20.304545+0000","last_peered":"2023-09-27T23:52:20.304545+0000","last_clean":"2023-09-27T23:52:20.304545+0000","last_became_active":"2023-09-27T23:52:19.921051+0000","last_became_peered":"2023-09-27T23:52:19.921051+0000","last_unstale":"2023-09-27T23:52:20.304545+0000","last_undegraded":"2023-09-27T23:52:20.304545+0000","last_fullsized":"2023-09-27T23:52:20.304545+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:07:14.396964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.312273+0000","last_change":"2023-09-27T23:52:20.307116+0000","last_active":"2023-09-27T23:52:20.312273+0000","last_peered":"2023-09-27T23:52:20.312273+0000","last_clean":"2023-09-27T23:52:20.312273+0000","last_became_active":"2023-09-27T23:52:20.304129+0000","last_became_peered":"2023-09-27T23:52:20.304129+0000","last_unstale":"2023-09-27T23:52:20.312273+0000","last_undegraded":"2023-09-27T23:52:20.312273+0000","last_fullsized":"2023-09-27T23:52:20.312273+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:39:14.995012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"188'1","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.318856+0000","last_change":"2023-09-27T23:52:19.921885+0000","last_active":"2023-09-27T23:52:20.318856+0000","last_peered":"2023-09-27T23:52:20.318856+0000","last_clean":"2023-09-27T23:52:20.318856+0000","last_became_active":"2023-09-27T23:52:19.921579+0000","last_became_peered":"2023-09-27T23:52:19.921579+0000","last_unstale":"2023-09-27T23:52:20.318856+0000","last_undegraded":"2023-09-27T23:52:20.318856+0000","last_fullsized":"2023-09-27T23:52:20.318856+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:08:37.530323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.307138+0000","last_change":"2023-09-27T23:52:20.307138+0000","last_active":"2023-09-27T23:52:20.307138+0000","last_peered":"2023-09-27T23:52:20.307138+0000","last_clean":"2023-09-27T23:52:20.307138+0000","last_became_active":"2023-09-27T23:52:20.306594+0000","last_became_peered":"2023-09-27T23:52:20.306594+0000","last_unstale":"2023-09-27T23:52:20.307138+0000","last_undegraded":"2023-09-27T23:52:20.307138+0000","last_fullsized":"2023-09-27T23:52:20.307138+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:55:28.153330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.303822+0000","last_change":"2023-09-27T23:52:19.926747+0000","last_active":"2023-09-27T23:52:20.303822+0000","last_peered":"2023-09-27T23:52:20.303822+0000","last_clean":"2023-09-27T23:52:20.303822+0000","last_became_active":"2023-09-27T23:52:19.926415+0000","last_became_peered":"2023-09-27T23:52:19.926415+0000","last_unstale":"2023-09-27T23:52:20.303822+0000","last_undegraded":"2023-09-27T23:52:20.303822+0000","last_fullsized":"2023-09-27T23:52:20.303822+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:18:43.148538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302019+0000","last_change":"2023-09-27T23:52:20.290505+0000","last_active":"2023-09-27T23:52:20.302019+0000","last_peered":"2023-09-27T23:52:20.302019+0000","last_clean":"2023-09-27T23:52:20.302019+0000","last_became_active":"2023-09-27T23:52:20.290196+0000","last_became_peered":"2023-09-27T23:52:20.290196+0000","last_unstale":"2023-09-27T23:52:20.302019+0000","last_undegraded":"2023-09-27T23:52:20.302019+0000","last_fullsized":"2023-09-27T23:52:20.302019+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:03:03.284236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"188'2","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302702+0000","last_change":"2023-09-27T23:52:19.927278+0000","last_active":"2023-09-27T23:52:20.302702+0000","last_peered":"2023-09-27T23:52:20.302702+0000","last_clean":"2023-09-27T23:52:20.302702+0000","last_became_active":"2023-09-27T23:52:19.926850+0000","last_became_peered":"2023-09-27T23:52:19.926850+0000","last_unstale":"2023-09-27T23:52:20.302702+0000","last_undegraded":"2023-09-27T23:52:20.302702+0000","last_fullsized":"2023-09-27T23:52:20.302702+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:23:11.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.318440+0000","last_change":"2023-09-27T23:52:20.311467+0000","last_active":"2023-09-27T23:52:20.318440+0000","last_peered":"2023-09-27T23:52:20.318440+0000","last_clean":"2023-09-27T23:52:20.318440+0000","last_became_active":"2023-09-27T23:52:20.311162+0000","last_became_peered":"2023-09-27T23:52:20.311162+0000","last_unstale":"2023-09-27T23:52:20.318440+0000","last_undegraded":"2023-09-27T23:52:20.318440+0000","last_fullsized":"2023-09-27T23:52:20.318440+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:43:02.886695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.288077+0000","last_change":"2023-09-27T23:52:19.926154+0000","last_active":"2023-09-27T23:52:20.288077+0000","last_peered":"2023-09-27T23:52:20.288077+0000","last_clean":"2023-09-27T23:52:20.288077+0000","last_became_active":"2023-09-27T23:52:19.925861+0000","last_became_peered":"2023-09-27T23:52:19.925861+0000","last_unstale":"2023-09-27T23:52:20.288077+0000","last_undegraded":"2023-09-27T23:52:20.288077+0000","last_fullsized":"2023-09-27T23:52:20.288077+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:13:38.494678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.320460+0000","last_change":"2023-09-27T23:52:20.320460+0000","last_active":"2023-09-27T23:52:20.320460+0000","last_peered":"2023-09-27T23:52:20.320460+0000","last_clean":"2023-09-27T23:52:20.320460+0000","last_became_active":"2023-09-27T23:52:20.320175+0000","last_became_peered":"2023-09-27T23:52:20.320175+0000","last_unstale":"2023-09-27T23:52:20.320460+0000","last_undegraded":"2023-09-27T23:52:20.320460+0000","last_fullsized":"2023-09-27T23:52:20.320460+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:56:48.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-09-27T23:52:19.936880+0000","last_change":"2023-09-27T23:52:19.930773+0000","last_active":"2023-09-27T23:52:19.936880+0000","last_peered":"2023-09-27T23:52:19.936880+0000","last_clean":"2023-09-27T23:52:19.936880+0000","last_became_active":"2023-09-27T23:52:19.930300+0000","last_became_peered":"2023-09-27T23:52:19.930300+0000","last_unstale":"2023-09-27T23:52:19.936880+0000","last_undegraded":"2023-09-27T23:52:19.936880+0000","last_fullsized":"2023-09-27T23:52:19.936880+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:34:43.061557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.298459+0000","last_change":"2023-09-27T23:52:20.293910+0000","last_active":"2023-09-27T23:52:20.298459+0000","last_peered":"2023-09-27T23:52:20.298459+0000","last_clean":"2023-09-27T23:52:20.298459+0000","last_became_active":"2023-09-27T23:52:20.293618+0000","last_became_peered":"2023-09-27T23:52:20.293618+0000","last_unstale":"2023-09-27T23:52:20.298459+0000","last_undegraded":"2023-09-27T23:52:20.298459+0000","last_fullsized":"2023-09-27T23:52:20.298459+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:25:18.530979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.292303+0000","last_change":"2023-09-27T23:52:19.930537+0000","last_active":"2023-09-27T23:52:20.292303+0000","last_peered":"2023-09-27T23:52:20.292303+0000","last_clean":"2023-09-27T23:52:20.292303+0000","last_became_active":"2023-09-27T23:52:19.930308+0000","last_became_peered":"2023-09-27T23:52:19.930308+0000","last_unstale":"2023-09-27T23:52:20.292303+0000","last_undegraded":"2023-09-27T23:52:20.292303+0000","last_fullsized":"2023-09-27T23:52:20.292303+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:46:15.201681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.296193+0000","last_change":"2023-09-27T23:52:20.293373+0000","last_active":"2023-09-27T23:52:20.296193+0000","last_peered":"2023-09-27T23:52:20.296193+0000","last_clean":"2023-09-27T23:52:20.296193+0000","last_became_active":"2023-09-27T23:52:20.293113+0000","last_became_peered":"2023-09-27T23:52:20.293113+0000","last_unstale":"2023-09-27T23:52:20.296193+0000","last_undegraded":"2023-09-27T23:52:20.296193+0000","last_fullsized":"2023-09-27T23:52:20.296193+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:49:24.482351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"188'2","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.321388+0000","last_change":"2023-09-27T23:52:19.933015+0000","last_active":"2023-09-27T23:52:20.321388+0000","last_peered":"2023-09-27T23:52:20.321388+0000","last_clean":"2023-09-27T23:52:20.321388+0000","last_became_active":"2023-09-27T23:52:19.932670+0000","last_became_peered":"2023-09-27T23:52:19.932670+0000","last_unstale":"2023-09-27T23:52:20.321388+0000","last_undegraded":"2023-09-27T23:52:20.321388+0000","last_fullsized":"2023-09-27T23:52:20.321388+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:11:03.703879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.318587+0000","last_change":"2023-09-27T23:52:20.316119+0000","last_active":"2023-09-27T23:52:20.318587+0000","last_peered":"2023-09-27T23:52:20.318587+0000","last_clean":"2023-09-27T23:52:20.318587+0000","last_became_active":"2023-09-27T23:52:20.315794+0000","last_became_peered":"2023-09-27T23:52:20.315794+0000","last_unstale":"2023-09-27T23:52:20.318587+0000","last_undegraded":"2023-09-27T23:52:20.318587+0000","last_fullsized":"2023-09-27T23:52:20.318587+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:13:21.277852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.283928+0000","last_change":"2023-09-27T23:52:19.921938+0000","last_active":"2023-09-27T23:52:20.283928+0000","last_peered":"2023-09-27T23:52:20.283928+0000","last_clean":"2023-09-27T23:52:20.283928+0000","last_became_active":"2023-09-27T23:52:19.920353+0000","last_became_peered":"2023-09-27T23:52:19.920353+0000","last_unstale":"2023-09-27T23:52:20.283928+0000","last_undegraded":"2023-09-27T23:52:20.283928+0000","last_fullsized":"2023-09-27T23:52:20.283928+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:27:53.484848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.313233+0000","last_change":"2023-09-27T23:52:20.311164+0000","last_active":"2023-09-27T23:52:20.313233+0000","last_peered":"2023-09-27T23:52:20.313233+0000","last_clean":"2023-09-27T23:52:20.313233+0000","last_became_active":"2023-09-27T23:52:20.310875+0000","last_became_peered":"2023-09-27T23:52:20.310875+0000","last_unstale":"2023-09-27T23:52:20.313233+0000","last_undegraded":"2023-09-27T23:52:20.313233+0000","last_fullsized":"2023-09-27T23:52:20.313233+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:06:54.071790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.284087+0000","last_change":"2023-09-27T23:52:19.922580+0000","last_active":"2023-09-27T23:52:20.284087+0000","last_peered":"2023-09-27T23:52:20.284087+0000","last_clean":"2023-09-27T23:52:20.284087+0000","last_became_active":"2023-09-27T23:52:19.922275+0000","last_became_peered":"2023-09-27T23:52:19.922275+0000","last_unstale":"2023-09-27T23:52:20.284087+0000","last_undegraded":"2023-09-27T23:52:20.284087+0000","last_fullsized":"2023-09-27T23:52:20.284087+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:07:26.195100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304472+0000","last_change":"2023-09-27T23:52:20.302510+0000","last_active":"2023-09-27T23:52:20.304472+0000","last_peered":"2023-09-27T23:52:20.304472+0000","last_clean":"2023-09-27T23:52:20.304472+0000","last_became_active":"2023-09-27T23:52:20.302254+0000","last_became_peered":"2023-09-27T23:52:20.302254+0000","last_unstale":"2023-09-27T23:52:20.304472+0000","last_undegraded":"2023-09-27T23:52:20.304472+0000","last_fullsized":"2023-09-27T23:52:20.304472+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:11:44.867529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.292592+0000","last_change":"2023-09-27T23:52:19.922744+0000","last_active":"2023-09-27T23:52:20.292592+0000","last_peered":"2023-09-27T23:52:20.292592+0000","last_clean":"2023-09-27T23:52:20.292592+0000","last_became_active":"2023-09-27T23:52:19.922448+0000","last_became_peered":"2023-09-27T23:52:19.922448+0000","last_unstale":"2023-09-27T23:52:20.292592+0000","last_undegraded":"2023-09-27T23:52:20.292592+0000","last_fullsized":"2023-09-27T23:52:20.292592+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:30.196936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.323028+0000","last_change":"2023-09-27T23:52:20.317143+0000","last_active":"2023-09-27T23:52:20.323028+0000","last_peered":"2023-09-27T23:52:20.323028+0000","last_clean":"2023-09-27T23:52:20.323028+0000","last_became_active":"2023-09-27T23:52:20.316843+0000","last_became_peered":"2023-09-27T23:52:20.316843+0000","last_unstale":"2023-09-27T23:52:20.323028+0000","last_undegraded":"2023-09-27T23:52:20.323028+0000","last_fullsized":"2023-09-27T23:52:20.323028+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:24:54.085264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.294646+0000","last_change":"2023-09-27T23:52:19.924875+0000","last_active":"2023-09-27T23:52:20.294646+0000","last_peered":"2023-09-27T23:52:20.294646+0000","last_clean":"2023-09-27T23:52:20.294646+0000","last_became_active":"2023-09-27T23:52:19.924590+0000","last_became_peered":"2023-09-27T23:52:19.924590+0000","last_unstale":"2023-09-27T23:52:20.294646+0000","last_undegraded":"2023-09-27T23:52:20.294646+0000","last_fullsized":"2023-09-27T23:52:20.294646+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:10:09.800070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.323018+0000","last_change":"2023-09-27T23:52:20.318546+0000","last_active":"2023-09-27T23:52:20.323018+0000","last_peered":"2023-09-27T23:52:20.323018+0000","last_clean":"2023-09-27T23:52:20.323018+0000","last_became_active":"2023-09-27T23:52:20.316995+0000","last_became_peered":"2023-09-27T23:52:20.316995+0000","last_unstale":"2023-09-27T23:52:20.323018+0000","last_undegraded":"2023-09-27T23:52:20.323018+0000","last_fullsized":"2023-09-27T23:52:20.323018+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:31:24.179256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.291837+0000","last_change":"2023-09-27T23:52:19.924526+0000","last_active":"2023-09-27T23:52:20.291837+0000","last_peered":"2023-09-27T23:52:20.291837+0000","last_clean":"2023-09-27T23:52:20.291837+0000","last_became_active":"2023-09-27T23:52:19.924220+0000","last_became_peered":"2023-09-27T23:52:19.924220+0000","last_unstale":"2023-09-27T23:52:20.291837+0000","last_undegraded":"2023-09-27T23:52:20.291837+0000","last_fullsized":"2023-09-27T23:52:20.291837+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T01:35:38.884820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":8,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.312836+0000","last_change":"2023-09-27T23:52:20.312836+0000","last_active":"2023-09-27T23:52:20.312836+0000","last_peered":"2023-09-27T23:52:20.312836+0000","last_clean":"2023-09-27T23:52:20.312836+0000","last_became_active":"2023-09-27T23:52:20.312549+0000","last_became_peered":"2023-09-27T23:52:20.312549+0000","last_unstale":"2023-09-27T23:52:20.312836+0000","last_undegraded":"2023-09-27T23:52:20.312836+0000","last_fullsized":"2023-09-27T23:52:20.312836+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:11:04.716716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-09-27T23:52:19.931735+0000","last_change":"2023-09-27T23:52:19.930414+0000","last_active":"2023-09-27T23:52:19.931735+0000","last_peered":"2023-09-27T23:52:19.931735+0000","last_clean":"2023-09-27T23:52:19.931735+0000","last_became_active":"2023-09-27T23:52:19.930178+0000","last_became_peered":"2023-09-27T23:52:19.930178+0000","last_unstale":"2023-09-27T23:52:19.931735+0000","last_undegraded":"2023-09-27T23:52:19.931735+0000","last_fullsized":"2023-09-27T23:52:19.931735+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:13:38.494678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304082+0000","last_change":"2023-09-27T23:52:20.299543+0000","last_active":"2023-09-27T23:52:20.304082+0000","last_peered":"2023-09-27T23:52:20.304082+0000","last_clean":"2023-09-27T23:52:20.304082+0000","last_became_active":"2023-09-27T23:52:20.299231+0000","last_became_peered":"2023-09-27T23:52:20.299231+0000","last_unstale":"2023-09-27T23:52:20.304082+0000","last_undegraded":"2023-09-27T23:52:20.304082+0000","last_fullsized":"2023-09-27T23:52:20.304082+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:07:15.409800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-09-27T23:52:19.929182+0000","last_change":"2023-09-27T23:52:19.927183+0000","last_active":"2023-09-27T23:52:19.929182+0000","last_peered":"2023-09-27T23:52:19.929182+0000","last_clean":"2023-09-27T23:52:19.929182+0000","last_became_active":"2023-09-27T23:52:19.926855+0000","last_became_peered":"2023-09-27T23:52:19.926855+0000","last_unstale":"2023-09-27T23:52:19.929182+0000","last_undegraded":"2023-09-27T23:52:19.929182+0000","last_fullsized":"2023-09-27T23:52:19.929182+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:31:15.111852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.313053+0000","last_change":"2023-09-27T23:52:20.306980+0000","last_active":"2023-09-27T23:52:20.313053+0000","last_peered":"2023-09-27T23:52:20.313053+0000","last_clean":"2023-09-27T23:52:20.313053+0000","last_became_active":"2023-09-27T23:52:20.306361+0000","last_became_peered":"2023-09-27T23:52:20.306361+0000","last_unstale":"2023-09-27T23:52:20.313053+0000","last_undegraded":"2023-09-27T23:52:20.313053+0000","last_fullsized":"2023-09-27T23:52:20.313053+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:06:59.757946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.298246+0000","last_change":"2023-09-27T23:52:19.917733+0000","last_active":"2023-09-27T23:52:20.298246+0000","last_peered":"2023-09-27T23:52:20.298246+0000","last_clean":"2023-09-27T23:52:20.298246+0000","last_became_active":"2023-09-27T23:52:19.917383+0000","last_became_peered":"2023-09-27T23:52:19.917383+0000","last_unstale":"2023-09-27T23:52:20.298246+0000","last_undegraded":"2023-09-27T23:52:20.298246+0000","last_fullsized":"2023-09-27T23:52:20.298246+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:23:30.196936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"177'326","reported_seq":1051,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.317348+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:20.317348+0000","last_peered":"2023-09-27T23:52:20.317348+0000","last_clean":"2023-09-27T23:52:20.317348+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:20.317348+0000","last_undegraded":"2023-09-27T23:52:20.317348+0000","last_fullsized":"2023-09-27T23:52:20.317348+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":326,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":174,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.294756+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:20.294756+0000","last_peered":"2023-09-27T23:52:20.294756+0000","last_clean":"2023-09-27T23:52:20.294756+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:20.294756+0000","last_undegraded":"2023-09-27T23:52:20.294756+0000","last_fullsized":"2023-09-27T23:52:20.294756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.300884+0000","last_change":"2023-09-27T23:52:20.299023+0000","last_active":"2023-09-27T23:52:20.300884+0000","last_peered":"2023-09-27T23:52:20.300884+0000","last_clean":"2023-09-27T23:52:20.300884+0000","last_became_active":"2023-09-27T23:52:20.298727+0000","last_became_peered":"2023-09-27T23:52:20.298727+0000","last_unstale":"2023-09-27T23:52:20.300884+0000","last_undegraded":"2023-09-27T23:52:20.300884+0000","last_fullsized":"2023-09-27T23:52:20.300884+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:53:49.181346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"188'1","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.310681+0000","last_change":"2023-09-27T23:52:19.919693+0000","last_active":"2023-09-27T23:52:20.310681+0000","last_peered":"2023-09-27T23:52:20.310681+0000","last_clean":"2023-09-27T23:52:20.310681+0000","last_became_active":"2023-09-27T23:52:19.919312+0000","last_became_peered":"2023-09-27T23:52:19.919312+0000","last_unstale":"2023-09-27T23:52:20.310681+0000","last_undegraded":"2023-09-27T23:52:20.310681+0000","last_fullsized":"2023-09-27T23:52:20.310681+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T10:33:10.666347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.316588+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:20.316588+0000","last_peered":"2023-09-27T23:52:20.316588+0000","last_clean":"2023-09-27T23:52:20.316588+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:20.316588+0000","last_undegraded":"2023-09-27T23:52:20.316588+0000","last_fullsized":"2023-09-27T23:52:20.316588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.312660+0000","last_change":"2023-09-27T23:52:20.298883+0000","last_active":"2023-09-27T23:52:20.312660+0000","last_peered":"2023-09-27T23:52:20.312660+0000","last_clean":"2023-09-27T23:52:20.312660+0000","last_became_active":"2023-09-27T23:52:20.298246+0000","last_became_peered":"2023-09-27T23:52:20.298246+0000","last_unstale":"2023-09-27T23:52:20.312660+0000","last_undegraded":"2023-09-27T23:52:20.312660+0000","last_fullsized":"2023-09-27T23:52:20.312660+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:56:37.063490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"188'1","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.298776+0000","last_change":"2023-09-27T23:52:19.929992+0000","last_active":"2023-09-27T23:52:20.298776+0000","last_peered":"2023-09-27T23:52:20.298776+0000","last_clean":"2023-09-27T23:52:20.298776+0000","last_became_active":"2023-09-27T23:52:19.929686+0000","last_became_peered":"2023-09-27T23:52:19.929686+0000","last_unstale":"2023-09-27T23:52:20.298776+0000","last_undegraded":"2023-09-27T23:52:20.298776+0000","last_fullsized":"2023-09-27T23:52:20.298776+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:53:36.693830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.287956+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:20.287956+0000","last_peered":"2023-09-27T23:52:20.287956+0000","last_clean":"2023-09-27T23:52:20.287956+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:20.287956+0000","last_undegraded":"2023-09-27T23:52:20.287956+0000","last_fullsized":"2023-09-27T23:52:20.287956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.322208+0000","last_change":"2023-09-27T23:52:20.319566+0000","last_active":"2023-09-27T23:52:20.322208+0000","last_peered":"2023-09-27T23:52:20.322208+0000","last_clean":"2023-09-27T23:52:20.322208+0000","last_became_active":"2023-09-27T23:52:20.319281+0000","last_became_peered":"2023-09-27T23:52:20.319281+0000","last_unstale":"2023-09-27T23:52:20.322208+0000","last_undegraded":"2023-09-27T23:52:20.322208+0000","last_fullsized":"2023-09-27T23:52:20.322208+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:56:37.063490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.298372+0000","last_change":"2023-09-27T23:52:19.922542+0000","last_active":"2023-09-27T23:52:20.298372+0000","last_peered":"2023-09-27T23:52:20.298372+0000","last_clean":"2023-09-27T23:52:20.298372+0000","last_became_active":"2023-09-27T23:52:19.921473+0000","last_became_peered":"2023-09-27T23:52:19.921473+0000","last_unstale":"2023-09-27T23:52:20.298372+0000","last_undegraded":"2023-09-27T23:52:20.298372+0000","last_fullsized":"2023-09-27T23:52:20.298372+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:35:50.023345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":174,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.291799+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:20.291799+0000","last_peered":"2023-09-27T23:52:20.291799+0000","last_clean":"2023-09-27T23:52:20.291799+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:20.291799+0000","last_undegraded":"2023-09-27T23:52:20.291799+0000","last_fullsized":"2023-09-27T23:52:20.291799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302391+0000","last_change":"2023-09-27T23:52:20.298350+0000","last_active":"2023-09-27T23:52:20.302391+0000","last_peered":"2023-09-27T23:52:20.302391+0000","last_clean":"2023-09-27T23:52:20.302391+0000","last_became_active":"2023-09-27T23:52:20.297896+0000","last_became_peered":"2023-09-27T23:52:20.297896+0000","last_unstale":"2023-09-27T23:52:20.302391+0000","last_undegraded":"2023-09-27T23:52:20.302391+0000","last_fullsized":"2023-09-27T23:52:20.302391+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:35:23.663926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"188'1","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.303358+0000","last_change":"2023-09-27T23:52:19.930671+0000","last_active":"2023-09-27T23:52:20.303358+0000","last_peered":"2023-09-27T23:52:20.303358+0000","last_clean":"2023-09-27T23:52:20.303358+0000","last_became_active":"2023-09-27T23:52:19.930391+0000","last_became_peered":"2023-09-27T23:52:19.930391+0000","last_unstale":"2023-09-27T23:52:20.303358+0000","last_undegraded":"2023-09-27T23:52:20.303358+0000","last_fullsized":"2023-09-27T23:52:20.303358+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:27:32.553175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-09-27T23:52:19.905137+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:19.905137+0000","last_peered":"2023-09-27T23:52:19.905137+0000","last_clean":"2023-09-27T23:52:19.905137+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:19.905137+0000","last_undegraded":"2023-09-27T23:52:19.905137+0000","last_fullsized":"2023-09-27T23:52:19.905137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.297915+0000","last_change":"2023-09-27T23:52:20.295137+0000","last_active":"2023-09-27T23:52:20.297915+0000","last_peered":"2023-09-27T23:52:20.297915+0000","last_clean":"2023-09-27T23:52:20.297915+0000","last_became_active":"2023-09-27T23:52:20.294852+0000","last_became_peered":"2023-09-27T23:52:20.294852+0000","last_unstale":"2023-09-27T23:52:20.297915+0000","last_undegraded":"2023-09-27T23:52:20.297915+0000","last_fullsized":"2023-09-27T23:52:20.297915+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:02:58.786390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"188'4","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.304812+0000","last_change":"2023-09-27T23:52:19.930684+0000","last_active":"2023-09-27T23:52:20.304812+0000","last_peered":"2023-09-27T23:52:20.304812+0000","last_clean":"2023-09-27T23:52:20.304812+0000","last_became_active":"2023-09-27T23:52:19.930305+0000","last_became_peered":"2023-09-27T23:52:19.930305+0000","last_unstale":"2023-09-27T23:52:20.304812+0000","last_undegraded":"2023-09-27T23:52:20.304812+0000","last_fullsized":"2023-09-27T23:52:20.304812+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:34:29.973552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.301544+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:20.301544+0000","last_peered":"2023-09-27T23:52:20.301544+0000","last_clean":"2023-09-27T23:52:20.301544+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:20.301544+0000","last_undegraded":"2023-09-27T23:52:20.301544+0000","last_fullsized":"2023-09-27T23:52:20.301544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.313518+0000","last_change":"2023-09-27T23:52:20.309399+0000","last_active":"2023-09-27T23:52:20.313518+0000","last_peered":"2023-09-27T23:52:20.313518+0000","last_clean":"2023-09-27T23:52:20.313518+0000","last_became_active":"2023-09-27T23:52:20.309108+0000","last_became_peered":"2023-09-27T23:52:20.309108+0000","last_unstale":"2023-09-27T23:52:20.313518+0000","last_undegraded":"2023-09-27T23:52:20.313518+0000","last_fullsized":"2023-09-27T23:52:20.313518+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:34:30.986389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"188'2","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.303688+0000","last_change":"2023-09-27T23:52:19.922684+0000","last_active":"2023-09-27T23:52:20.303688+0000","last_peered":"2023-09-27T23:52:20.303688+0000","last_clean":"2023-09-27T23:52:20.303688+0000","last_became_active":"2023-09-27T23:52:19.922355+0000","last_became_peered":"2023-09-27T23:52:19.922355+0000","last_unstale":"2023-09-27T23:52:20.303688+0000","last_undegraded":"2023-09-27T23:52:20.303688+0000","last_fullsized":"2023-09-27T23:52:20.303688+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:12:00.388045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-09-27T23:52:19.908902+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:19.908902+0000","last_peered":"2023-09-27T23:52:19.908902+0000","last_clean":"2023-09-27T23:52:19.908902+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:19.908902+0000","last_undegraded":"2023-09-27T23:52:19.908902+0000","last_fullsized":"2023-09-27T23:52:19.908902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.314584+0000","last_change":"2023-09-27T23:52:20.310173+0000","last_active":"2023-09-27T23:52:20.314584+0000","last_peered":"2023-09-27T23:52:20.314584+0000","last_clean":"2023-09-27T23:52:20.314584+0000","last_became_active":"2023-09-27T23:52:20.309157+0000","last_became_peered":"2023-09-27T23:52:20.309157+0000","last_unstale":"2023-09-27T23:52:20.314584+0000","last_undegraded":"2023-09-27T23:52:20.314584+0000","last_fullsized":"2023-09-27T23:52:20.314584+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:19.885476+0000","last_clean_scrub_stamp":"2023-09-27T23:52:19.885476+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:56:28.166692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"188'2","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-09-27T23:52:20.302945+0000","last_change":"2023-09-27T23:52:19.923954+0000","last_active":"2023-09-27T23:52:20.302945+0000","last_peered":"2023-09-27T23:52:20.302945+0000","last_clean":"2023-09-27T23:52:20.302945+0000","last_became_active":"2023-09-27T23:52:19.923075+0000","last_became_peered":"2023-09-27T23:52:19.923075+0000","last_unstale":"2023-09-27T23:52:20.302945+0000","last_undegraded":"2023-09-27T23:52:20.302945+0000","last_fullsized":"2023-09-27T23:52:20.302945+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:18.872640+0000","last_clean_scrub_stamp":"2023-09-27T23:52:18.872640+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:45:19.709321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":152,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-09-27T23:52:19.918366+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:19.918366+0000","last_peered":"2023-09-27T23:52:19.918366+0000","last_clean":"2023-09-27T23:52:19.918366+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:19.918366+0000","last_undegraded":"2023-09-27T23:52:19.918366+0000","last_fullsized":"2023-09-27T23:52:19.918366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":33984,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":2961408,"data_stored":1834589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34799616},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.89600000000000002},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2250000000000001,"5min":1.569,"15min":1.569},"last":1.4810000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.94899999999999995,"15min":0.93400000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.466,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.78200000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.248,"5min":1.333,"15min":1.3360000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96799999999999997,"15min":0.97499999999999998},"min":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2929999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.92500000000000004},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.3069999999999999,"5min":1.341,"15min":1.341},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.93500000000000005},"min":{"1min":0.70799999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.202,"5min":1.359,"15min":1.359},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0760000000000001,"15min":1.079},"min":{"1min":0.77700000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.3879999999999999},"last":1.1850000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.216,"5min":1.821,"15min":1.821},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.03,"15min":1.0109999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.226,"5min":3.02,"15min":3.02},"last":1.619}]},{"osd":5,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0569999999999999,"15min":1.0620000000000001},"min":{"1min":0.75,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.70099999999999996},"max":{"1min":1.3759999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.4450000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.131,"15min":1.143},"min":{"1min":0.749,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.423,"5min":1.423,"15min":1.448},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.145,"15min":1.145},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.381,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.5149999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30268,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341572,"statfs":{"total":96636764160,"available":96605769728,"internally_reserved":0,"allocated":2347008,"data_stored":1237156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.97199999999999998},"min":{"1min":0.70599999999999996,"5min":0.60599999999999998,"15min":0.53400000000000003},"max":{"1min":1.214,"5min":1.792,"15min":1.792},"last":1.016},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.80400000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90200000000000002,"15min":0.90300000000000002},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67100000000000004,"5min":0.622,"15min":0.57099999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89500000000000002,"15min":0.90600000000000003},"min":{"1min":0.78100000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.147,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.018,"5min":0.98399999999999999,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.026,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.78000000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91000000000000003,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.6639999999999999,"5min":1.7,"15min":1.7},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.079,"5min":1.0289999999999999,"15min":1.0029999999999999},"min":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3089999999999999,"5min":1.3700000000000001,"15min":1.633},"last":0.876}]},{"osd":5,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.161,"15min":1.145},"min":{"1min":0.89500000000000002,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.075}]},{"osd":7,"last update":"Wed Sep 27 23:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.81100000000000005,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":1.637,"15min":1.679},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.155,"15min":1.1359999999999999},"min":{"1min":0.69199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":2359296,"data_stored":1244117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.95999999999999996,"15min":0.89700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.51900000000000002},"max":{"1min":10.058999999999999,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.657,"5min":1.0609999999999999,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.47899999999999998},"max":{"1min":13.757999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.96999999999999997,"15min":0.93200000000000005},"min":{"1min":0.70699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.1970000000000001,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.873}]},{"osd":2,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.95999999999999996,"15min":0.96099999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.629},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.97699999999999998,"15min":0.95899999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.0999999999999996,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.999,"15min":0.98999999999999999},"min":{"1min":0.84299999999999997,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":1.008},{"interface":"front","average":{"1min":1.256,"5min":1.075,"15min":1.042},"min":{"1min":0.79000000000000004,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.94999999999999996}]},{"osd":4,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91200000000000003,"15min":0.88900000000000001},"min":{"1min":0.72299999999999998,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.0469999999999999,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3640000000000001,"5min":1.45,"15min":1.4510000000000001},"last":1.1160000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":4.157,"5min":4.157,"15min":4.157},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.119,"15min":1.105},"min":{"1min":0.91900000000000004,"5min":0.71599999999999997,"15min":0.66600000000000004},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.196}]},{"osd":7,"last update":"Wed Sep 27 23:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.222,"15min":1.169},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.54700000000000004},"max":{"1min":6.3710000000000004,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.363,"15min":1.2629999999999999},"min":{"1min":1.0329999999999999,"5min":0.95499999999999996,"15min":0.71099999999999997},"max":{"1min":14.119,"5min":14.119,"15min":14.119},"last":1.52}]}]},{"osd":4,"up_from":15,"seq":64424509902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31064,"kb_used_data":2320,"kb_used_omap":0,"kb_used_meta":28736,"kb_avail":94340776,"statfs":{"total":96636764160,"available":96604954624,"internally_reserved":0,"allocated":2375680,"data_stored":1246207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29425664},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.45000000000000001},"max":{"1min":1.1240000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":1.3660000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.1180000000000001,"5min":1.893,"15min":2.2770000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.91500000000000004,"15min":0.90100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2010000000000001,"5min":1.746,"15min":1.746},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.53500000000000003},"max":{"1min":1.2390000000000001,"5min":1.758,"15min":2.1110000000000002},"last":0.95699999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94399999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2490000000000001,"5min":1.659,"15min":1.659},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.98299999999999998,"15min":0.96099999999999997},"min":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.53900000000000003},"max":{"1min":1.234,"5min":1.966,"15min":2.056},"last":1.2789999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.93400000000000005,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.998,"15min":2.032},"last":1.331},{"interface":"front","average":{"1min":1.016,"5min":1.079,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.58199999999999996},"max":{"1min":1.2709999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.012}]},{"osd":5,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.92400000000000004},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.45500000000000002},"max":{"1min":1.25,"5min":1.556,"15min":1.9930000000000001},"last":1.077},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.006,"15min":1.0149999999999999},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3129999999999999,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.2310000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0529999999999999,"15min":1.0509999999999999},"min":{"1min":0.81399999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.194},{"interface":"front","average":{"1min":1.232,"5min":1.1830000000000001,"15min":1.1679999999999999},"min":{"1min":0.92100000000000004,"5min":0.83799999999999997,"15min":0.495},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.173}]},{"osd":7,"last update":"Wed Sep 27 23:51:33 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.099,"15min":1.097},"min":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.66100000000000003},"max":{"1min":1.4350000000000001,"5min":2.069,"15min":2.069},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.153,"5min":1.161,"15min":1.1739999999999999},"min":{"1min":0.74399999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.329,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.306}]}]},{"osd":3,"up_from":15,"seq":64424509901,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30868,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":28544,"kb_avail":94340972,"statfs":{"total":96636764160,"available":96605155328,"internally_reserved":0,"allocated":2371584,"data_stored":1244311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29229056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.76000000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.0309999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.363},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":1.4099999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.569},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":0.86499999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.224},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.52}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.298}]}]},{"osd":2,"up_from":15,"seq":64424509902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30740,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":28416,"kb_avail":94341100,"statfs":{"total":96636764160,"available":96605286400,"internally_reserved":0,"allocated":2371584,"data_stored":1244783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29097984},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1339999999999999,"5min":1.345,"15min":1.4670000000000001},"last":0.74299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.84599999999999997},"min":{"1min":0.495,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.57099999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.3640000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94399999999999995},"min":{"1min":0.71299999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.262,"15min":1.4359999999999999},"last":0.91600000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0600000000000001,"15min":1.0800000000000001},"min":{"1min":0.79900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.415,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.91800000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3560000000000001,"5min":1.645,"15min":1.8009999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.3959999999999999,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1439999999999999,"15min":1.175},"min":{"1min":0.85199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5680000000000001,"5min":1.6459999999999999,"15min":1.8},"last":1.1140000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.96599999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.466,"5min":1.5780000000000001,"15min":1.718},"last":0.998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.196,"15min":1.1819999999999999},"min":{"1min":0.88600000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":1.6439999999999999,"15min":1.736},"last":0.97299999999999998}]},{"osd":7,"last update":"Wed Sep 27 23:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2350000000000001,"15min":1.23},"min":{"1min":1.121,"5min":0.873,"15min":0.873},"max":{"1min":1.5349999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.143},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.232},"min":{"1min":0.98399999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.607,"5min":1.6970000000000001,"15min":1.8129999999999999},"last":0.93600000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542608,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44936,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":42624,"kb_avail":94326904,"statfs":{"total":96636764160,"available":96590749696,"internally_reserved":0,"allocated":2359296,"data_stored":1238692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43646976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.84199999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0069999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.67900000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.075}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":1.103},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.2450000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.98799999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509903,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":29056,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":2957312,"data_stored":1834507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29753344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.113,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.47499999999999998},"max":{"1min":1.145,"5min":1.198,"15min":1.2470000000000001},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.502},"max":{"1min":1.169,"5min":1.169,"15min":1.2509999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.119,"5min":1.288,"15min":1.863},"last":1.125}]},{"osd":3,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88600000000000001,"15min":0.876},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2190000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.873},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.27,"15min":1.4390000000000001},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.042,"15min":1.052},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3160000000000001,"5min":1.573,"15min":1.7110000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.411,"5min":1.516,"15min":1.651},"last":1.1699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.1100000000000001},"min":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.73299999999999998},"max":{"1min":1.3380000000000001,"5min":1.444,"15min":1.512},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1519999999999999,"15min":1.175},"min":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4810000000000001,"5min":1.5629999999999999,"15min":1.847},"last":1.363}]},{"osd":6,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.167,"15min":1.1870000000000001},"min":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.76500000000000001},"max":{"1min":1.3500000000000001,"5min":1.6499999999999999,"15min":1.895},"last":1.288},{"interface":"front","average":{"1min":1.204,"5min":1.226,"15min":1.232},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.615,"5min":1.722,"15min":2.0510000000000002},"last":1.22}]},{"osd":7,"last update":"Wed Sep 27 23:51:45 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2589999999999999,"15min":1.2789999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.722,"5min":1.786,"15min":2.0030000000000001},"last":1.488},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.244,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4099999999999999,"5min":1.806,"15min":2.004},"last":1.706}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:28.068 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-27T23:52:28.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-27T23:52:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 Processor -- start 2023-09-27T23:52:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 -- start start 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e50155a90 0x7f6e500ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e50155e60 0x7f6e500ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e500c68a0 con 0x7f6e5006db70 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e500c6e60 con 0x7f6e50155a90 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e566d8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500ae4d0 con 0x7f6e50155e60 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.184+0000 7f6e556d6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e50155a90 0x7f6e500ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e50155a90 0x7f6e500ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55914/0 (socket says 172.21.15.64:55914) 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 -- 172.21.15.64:0/2317042268 learned_addr learned my addr 172.21.15.64:0/2317042268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e55ed7700 1 --2- 172.21.15.64:0/2317042268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e50155e60 0x7f6e500ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e47fff700 1 -- 172.21.15.64:0/2317042268 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013959346 0 0) 0x7f6e500c68a0 con 0x7f6e5006db70 2023-09-27T23:52:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e47fff700 1 -- 172.21.15.64:0/2317042268 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e3c003570 con 0x7f6e5006db70 2023-09-27T23:52:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 -- 172.21.15.64:0/2317042268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e50155e60 msgr2=0x7f6e500ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 --2- 172.21.15.64:0/2317042268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e50155e60 0x7f6e500ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 -- 172.21.15.64:0/2317042268 >> v1:172.21.15.64:6789/0 conn(0x7f6e5006db70 legacy=0x7f6e500ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 -- 172.21.15.64:0/2317042268 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500ae7b0 con 0x7f6e50155a90 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e556d6700 1 --2- 172.21.15.64:0/2317042268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155a90 0x7f6e500ad220 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0099b0 tx=0x7f6e4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbf10dbf2dde2ff server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e47fff700 1 -- 172.21.15.64:0/2317042268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00b680 con 0x7f6e50155a90 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e47fff700 1 -- 172.21.15.64:0/2317042268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c00b7e0 con 0x7f6e50155a90 2023-09-27T23:52:28.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e47fff700 1 -- 172.21.15.64:0/2317042268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00bb80 con 0x7f6e50155a90 2023-09-27T23:52:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 -- 172.21.15.64:0/2317042268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155a90 msgr2=0x7f6e500ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 --2- 172.21.15.64:0/2317042268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155a90 0x7f6e500ad220 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0099b0 tx=0x7f6e4c009250 comp rx=0 tx=0).stop 2023-09-27T23:52:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 -- 172.21.15.64:0/2317042268 shutdown_connections 2023-09-27T23:52:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 --2- 172.21.15.64:0/2317042268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e50155e60 0x7f6e500ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 --2- 172.21.15.64:0/2317042268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155a90 0x7f6e500ad220 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 -- 172.21.15.64:0/2317042268 >> 172.21.15.64:0/2317042268 conn(0x7f6e500c8cd0 msgr2=0x7f6e500c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:28.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 -- 172.21.15.64:0/2317042268 shutdown_connections 2023-09-27T23:52:28.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.188+0000 7f6e566d8700 1 -- 172.21.15.64:0/2317042268 wait complete. 2023-09-27T23:52:28.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 Processor -- start 2023-09-27T23:52:28.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 -- start start 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e5006db70 0x7f6e5006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 0x7f6e5006b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155e60 0x7f6e50146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500a7500 con 0x7f6e50155e60 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500a77e0 con 0x7f6e50155a90 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e566d8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500a7ac0 con 0x7f6e5006db70 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e55ed7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155e60 0x7f6e50146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 0x7f6e5006b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e556d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e5006db70 0x7f6e5006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 0x7f6e5006b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:56728/0 (socket says 172.21.15.64:56728) 2023-09-27T23:52:28.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e55ed7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155e60 0x7f6e50146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55922/0 (socket says 172.21.15.64:55922) 2023-09-27T23:52:28.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 -- 172.21.15.64:0/1923280570 learned_addr learned my addr 172.21.15.64:0/1923280570 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 -- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e5006db70 msgr2=0x7f6e5006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e5006db70 0x7f6e5006ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 -- 172.21.15.64:0/1923280570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155e60 msgr2=0x7f6e50146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155e60 0x7f6e50146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 -- 172.21.15.64:0/1923280570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0094f0 con 0x7f6e50155a90 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.192+0000 7f6e54ed5700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 0x7f6e5006b360 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c790 tx=0x7f6e40007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40010510 con 0x7f6e50155a90 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e500a7da0 con 0x7f6e50155a90 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40010670 con 0x7f6e50155a90 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40010a10 con 0x7f6e50155a90 2023-09-27T23:52:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e50147ce0 con 0x7f6e50155a90 2023-09-27T23:52:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e50147f60 con 0x7f6e50155a90 2023-09-27T23:52:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f6e40033bb0 con 0x7f6e50155a90 2023-09-27T23:52:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e457fa700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6e34060a70 0x7f6e34062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6e4007cac0 con 0x7f6e50155a90 2023-09-27T23:52:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.196+0000 7f6e556d6700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6e34060a70 0x7f6e34062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.200+0000 7f6e556d6700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6e34060a70 0x7f6e34062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0099b0 tx=0x7f6e4c017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:28.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.208+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6e4004ca50 con 0x7f6e50155a90 2023-09-27T23:52:28.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.440+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6e500a7920 con 0x7f6e50155a90 2023-09-27T23:52:28.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.452+0000 7f6e457fa700 1 -- 172.21.15.64:0/1923280570 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6e500a7920 con 0x7f6e50155a90 2023-09-27T23:52:28.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6e34060a70 msgr2=0x7f6e34062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6e34060a70 0x7f6e34062f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0099b0 tx=0x7f6e4c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:28.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 msgr2=0x7f6e5006b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 0x7f6e5006b360 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c790 tx=0x7f6e40007e10 comp rx=0 tx=0).stop 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 shutdown_connections 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6e34060a70 0x7f6e34062f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e50155e60 0x7f6e50146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50155a90 0x7f6e5006b360 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 --2- 172.21.15.64:0/1923280570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e5006db70 0x7f6e5006ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 >> 172.21.15.64:0/1923280570 conn(0x7f6e500c8cd0 msgr2=0x7f6e501544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 shutdown_connections 2023-09-27T23:52:28.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.456+0000 7f6e566d8700 1 -- 172.21.15.64:0/1923280570 wait complete. 2023-09-27T23:52:28.463 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-09-27T23:52:28.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:28.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.576+0000 7f744b4e6700 1 Processor -- start 2023-09-27T23:52:29.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.576+0000 7f744b4e6700 1 -- start start 2023-09-27T23:52:29.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74440680f0 0x7f7444068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744406c9f0 con 0x7f74440c2980 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7444068ab0 con 0x7f74440680f0 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74440c7f20 con 0x7f744413f890 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74440680f0 0x7f7444068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74440680f0 0x7f7444068570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55928/0 (socket says 172.21.15.64:55928) 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 -- 172.21.15.64:0/3585131811 learned_addr learned my addr 172.21.15.64:0/3585131811 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449282700 1 --2- 172.21.15.64:0/3585131811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 -- 172.21.15.64:0/3585131811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 msgr2=0x7f74440aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 --2- 172.21.15.64:0/3585131811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 -- 172.21.15.64:0/3585131811 >> v1:172.21.15.64:6789/0 conn(0x7f74440c2980 legacy=0x7f74440c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 -- 172.21.15.64:0/3585131811 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440c1c50 con 0x7f74440680f0 2023-09-27T23:52:29.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f7449a83700 1 --2- 172.21.15.64:0/3585131811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440680f0 0x7f7444068570 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f743c009f20 tx=0x7f743c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd02d4d255e2e53 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f743bfff700 1 -- 172.21.15.64:0/3585131811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00e070 con 0x7f74440680f0 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f743bfff700 1 -- 172.21.15.64:0/3585131811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c00b9d0 con 0x7f74440680f0 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f743bfff700 1 -- 172.21.15.64:0/3585131811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c0092e0 con 0x7f74440680f0 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- 172.21.15.64:0/3585131811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440680f0 msgr2=0x7f7444068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 --2- 172.21.15.64:0/3585131811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440680f0 0x7f7444068570 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f743c009f20 tx=0x7f743c009840 comp rx=0 tx=0).stop 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- 172.21.15.64:0/3585131811 shutdown_connections 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 --2- 172.21.15.64:0/3585131811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440680f0 0x7f7444068570 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 --2- 172.21.15.64:0/3585131811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- 172.21.15.64:0/3585131811 >> 172.21.15.64:0/3585131811 conn(0x7f74440c8850 msgr2=0x7f74440c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- 172.21.15.64:0/3585131811 shutdown_connections 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.580+0000 7f744b4e6700 1 -- 172.21.15.64:0/3585131811 wait complete. 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 Processor -- start 2023-09-27T23:52:29.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 -- start start 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74440680f0 0x7f74441684b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 0x7f744415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74440c3c60 con 0x7f74440c2980 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744415b580 con 0x7f74440680f0 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f744b4e6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74440c3370 con 0x7f744413f890 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f7448a81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 0x7f744415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f7448a81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 0x7f744415b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55936/0 (socket says 172.21.15.64:55936) 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.584+0000 7f7448a81700 1 -- 172.21.15.64:0/1923075908 learned_addr learned my addr 172.21.15.64:0/1923075908 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7449282700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74440680f0 0x7f74441684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7449a83700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7448a81700 1 -- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 msgr2=0x7f74440b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7448a81700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7448a81700 1 -- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74440680f0 msgr2=0x7f74441684b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7448a81700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74440680f0 0x7f74441684b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7448a81700 1 -- 172.21.15.64:0/1923075908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c009ae0 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f7448a81700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 0x7f744415b040 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f744000c5c0 tx=0x7f744000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440011590 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440011710 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74440c3610 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440011ad0 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7444167f00 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74440c3dd0 con 0x7f74440c2980 2023-09-27T23:52:29.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f744001b3b0 con 0x7f74440c2980 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f74397fa700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7428060aa0 0x7f7428062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.588+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f744007c5e0 con 0x7f74440c2980 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.592+0000 7f7449282700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7428060aa0 0x7f7428062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.592+0000 7f7449282700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7428060aa0 0x7f7428062f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7434009d10 tx=0x7f74340091a0 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.596+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f744004c610 con 0x7f74440c2980 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.792+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7444071120 con 0x7f74440c2980 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.792+0000 7f74397fa700 1 -- 172.21.15.64:0/1923075908 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f7440017090 con 0x7f74440c2980 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:29.246 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:52:20.268986+0000","modified":"2023-09-27T23:52:22.285887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10144":{"gid":10144,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/1929792685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1929792685},{"type":"v1","addr":"172.21.15.123:6835","nonce":1929792685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7428060aa0 msgr2=0x7f7428062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7428060aa0 0x7f7428062f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7434009d10 tx=0x7f74340091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 msgr2=0x7f744415b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 0x7f744415b040 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f744000c5c0 tx=0x7f744000c8d0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 shutdown_connections 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7428060aa0 0x7f7428062f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f744413f890 0x7f74440b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74440c2980 0x7f744415b040 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 --2- 172.21.15.64:0/1923075908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74440680f0 0x7f74441684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 >> 172.21.15.64:0/1923075908 conn(0x7f74440c8850 msgr2=0x7f74440af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 shutdown_connections 2023-09-27T23:52:29.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:28.796+0000 7f744b4e6700 1 -- 172.21.15.64:0/1923075908 wait complete. 2023-09-27T23:52:29.249 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 72 2023-09-27T23:52:29.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-27T23:52:20.268986+0000', 'modified': '2023-09-27T23:52:22.285887+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10144': {'gid': 10144, 'name': 'b', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6835/1929792685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 1929792685}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 1929792685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-27T23:52:29.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-27T23:52:29.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-27T23:52:29.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.764+0000 7f9ab9cdb700 1 Processor -- start 2023-09-27T23:52:29.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab9cdb700 1 -- start start 2023-09-27T23:52:29.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab9cdb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ab4158e20 0x7f9ab406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab9cdb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ab41591f0 0x7f9ab40ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab9cdb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab406b730 con 0x7f9ab415ca80 2023-09-27T23:52:29.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab9cdb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab406ab20 con 0x7f9ab41591f0 2023-09-27T23:52:29.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab9cdb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab406ae00 con 0x7f9ab4158e20 2023-09-27T23:52:29.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ab41591f0 0x7f9ab40ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab8cd9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ab4158e20 0x7f9ab406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ab41591f0 0x7f9ab40ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55940/0 (socket says 172.21.15.64:55940) 2023-09-27T23:52:29.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab94da700 1 --1- 172.21.15.64:0/2771642468 >> v1:172.21.15.123:6789/0 conn(0x7f9ab415ca80 0x7f9ab406c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.64:41016/0 (socket says 172.21.15.64:41016) 2023-09-27T23:52:29.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab3fff700 1 -- 172.21.15.64:0/2771642468 learned_addr learned my addr 172.21.15.64:0/2771642468 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/2771642468 >> v2:172.21.15.64:3300/0 conn(0x7f9ab41591f0 msgr2=0x7f9ab40ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/2771642468 >> v2:172.21.15.64:3300/0 conn(0x7f9ab41591f0 0x7f9ab40ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/2771642468 >> v1:172.21.15.123:6789/0 conn(0x7f9ab415ca80 legacy=0x7f9ab406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/2771642468 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab406b0e0 con 0x7f9ab4158e20 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/2771642468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab406a540 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9ab406e510 tx=0x7f9aa80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5da025380381df3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab37fe700 1 -- 172.21.15.64:0/2771642468 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8011630 con 0x7f9ab4158e20 2023-09-27T23:52:29.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab37fe700 1 -- 172.21.15.64:0/2771642468 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa8011790 con 0x7f9ab4158e20 2023-09-27T23:52:29.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.768+0000 7f9ab37fe700 1 -- 172.21.15.64:0/2771642468 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8011b30 con 0x7f9ab4158e20 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/2771642468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 msgr2=0x7f9ab406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/2771642468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab406a540 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9ab406e510 tx=0x7f9aa80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/2771642468 shutdown_connections 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/2771642468 >> v2:172.21.15.64:3300/0 conn(0x7f9ab41591f0 0x7f9ab40ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/2771642468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab406a540 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/2771642468 >> 172.21.15.64:0/2771642468 conn(0x7f9ab4067bb0 msgr2=0x7f9ab40cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/2771642468 shutdown_connections 2023-09-27T23:52:29.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.772+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/2771642468 wait complete. 2023-09-27T23:52:29.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 Processor -- start 2023-09-27T23:52:29.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 -- start start 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ab41591f0 0x7f9ab40c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 0x7f9ab4144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4159bb0 con 0x7f9ab415ca80 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4156330 con 0x7f9ab4158e20 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab9cdb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4156610 con 0x7f9ab41591f0 2023-09-27T23:52:29.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 0x7f9ab4144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab8cd9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 0x7f9ab4144440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55956/0 (socket says 172.21.15.64:55956) 2023-09-27T23:52:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 -- 172.21.15.64:0/996652668 learned_addr learned my addr 172.21.15.64:0/996652668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ab41591f0 0x7f9ab40c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 -- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ab41591f0 msgr2=0x7f9ab40c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ab41591f0 0x7f9ab40c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 -- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 msgr2=0x7f9ab4159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab4159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 -- 172.21.15.64:0/996652668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8009420 con 0x7f9ab415ca80 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab94da700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 0x7f9ab4144440 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00cc70 tx=0x7f9aac001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.776+0000 7f9ab0ff9700 1 -- 172.21.15.64:0/996652668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac005df0 con 0x7f9ab415ca80 2023-09-27T23:52:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab0ff9700 1 -- 172.21.15.64:0/996652668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac00a5b0 con 0x7f9ab415ca80 2023-09-27T23:52:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab41568f0 con 0x7f9ab415ca80 2023-09-27T23:52:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab0ff9700 1 -- 172.21.15.64:0/996652668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00a930 con 0x7f9ab415ca80 2023-09-27T23:52:29.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab4156eb0 con 0x7f9ab415ca80 2023-09-27T23:52:29.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab0ff9700 1 -- 172.21.15.64:0/996652668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9aac00ace0 con 0x7f9ab415ca80 2023-09-27T23:52:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab0ff9700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9a98060a70 0x7f9a98062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab0ff9700 1 -- 172.21.15.64:0/996652668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9aac07d070 con 0x7f9ab415ca80 2023-09-27T23:52:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9abd1e6740 1 Processor -- start 2023-09-27T23:52:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9abd1e6740 1 -- start start 2023-09-27T23:52:29.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.780+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9a98060a70 0x7f9a98062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190dae0 0x1b8ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b8b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1958790 0x1b8b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ab8f30 con 0x190d710 2023-09-27T23:52:29.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ab4ed0 con 0x1958790 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a7b0b0 con 0x190dae0 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b8b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b8b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55964/0 (socket says 172.21.15.64:55964) 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 -- 172.21.15.64:0/3424477709 learned_addr learned my addr 172.21.15.64:0/3424477709 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab94da700 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1958790 0x1b8b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9a98060a70 0x7f9a98062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80093f0 tx=0x7f9aa8017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190dae0 0x1b8ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 -- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190dae0 msgr2=0x1b8ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190dae0 0x1b8ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 -- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1958790 msgr2=0x1b8b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1958790 0x1b8b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 -- 172.21.15.64:0/3424477709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19599e0 con 0x190d710 2023-09-27T23:52:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab94da700 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1958790 0x1b8b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b8b390 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009d50 tx=0x7f9aa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374e59a994886386 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9a9dffb700 1 -- 172.21.15.64:0/3424477709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0009e50 con 0x190d710 2023-09-27T23:52:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9a9dffb700 1 -- 172.21.15.64:0/3424477709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa0011cf0 con 0x190d710 2023-09-27T23:52:29.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9a9dffb700 1 -- 172.21.15.64:0/3424477709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa001d6b0 con 0x190d710 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- 172.21.15.64:0/3424477709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 msgr2=0x1b8b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b8b390 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009d50 tx=0x7f9aa0009250 comp rx=0 tx=0).stop 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- 172.21.15.64:0/3424477709 shutdown_connections 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x1958790 0x1b8b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b8b390 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/3424477709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190dae0 0x1b8ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- 172.21.15.64:0/3424477709 >> 172.21.15.64:0/3424477709 conn(0x190deb0 msgr2=0x1b47880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- 172.21.15.64:0/3424477709 shutdown_connections 2023-09-27T23:52:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- 172.21.15.64:0/3424477709 wait complete. 2023-09-27T23:52:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 -- start start 2023-09-27T23:52:29.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.784+0000 7f9abd1e6740 1 Processor -- start 2023-09-27T23:52:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b47740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b71140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7ad50 con 0x190d710 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a53970 con 0x190dae0 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a6e430 con 0x1958790 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b47740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 10 client.0 _ll_register_callbacks cb 0x1afeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b47740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55972/0 (socket says 172.21.15.64:55972) 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/745167589 learned_addr learned my addr 172.21.15.64:0/745167589 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab94da700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b71140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 msgr2=0x1b71140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b71140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 msgr2=0x1b70c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b70c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/745167589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab4156eb0 con 0x190d710 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b70c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b47740 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009f50 tx=0x7f9aa8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176ab9c862173f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9a92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9a92ffd700 1 -- 172.21.15.64:0/745167589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa802f070 con 0x190d710 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9a92ffd700 1 -- 172.21.15.64:0/745167589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9aa8029640 con 0x190d710 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a53cc0 con 0x190d710 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9a92ffd700 1 -- 172.21.15.64:0/745167589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa8021070 con 0x190d710 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 10 client.10473 fetch_fsmap learned FSMap version 72 2023-09-27T23:52:29.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x7f9aa0009a00 con 0x190d710 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9a92ffd700 1 -- 172.21.15.64:0/745167589 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1a53cc0 con 0x190d710 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9a92ffd700 1 -- 172.21.15.64:0/745167589 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 72) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9aa8029a10 con 0x190d710 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 10 client.10473 fetch_fsmap finished waiting for FSMap version 72 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 10 client.10473 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 20 client.10473 populate_metadata read hostname 'smithi064' 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1b99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 4 client.10473 mds_command: new command op to 10144 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 --> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1a573e0 con 0x1b976b0 2023-09-27T23:52:29.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.788+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1b99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1b99a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fc04fb4f689c14 server_cookie=1931b715b90f89db in_seq=0 out_seq=0 2023-09-27T23:52:29.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9a92ffd700 10 client.10473 ms_handle_connect on v2:172.21.15.123:6834/1929792685 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9a92ffd700 1 -- 172.21.15.64:0/745167589 <== mds.0 v2:172.21.15.123:6834/1929792685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1a573e0 con 0x1b976b0 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9a92ffd700 10 client.10473 handle_command_reply: tid=0 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 client.10473 shutdown 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 msgr2=0x1b47740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b47740 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009f50 tx=0x7f9aa8021000 comp rx=0 tx=0).stop 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 shutdown_connections 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1b99a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b71140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9a92ffd700 0 client.10473 ms_handle_reset on v2:172.21.15.123:6834/1929792685 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b70c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/745167589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b47740 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 >> 172.21.15.64:0/745167589 conn(0x190deb0 msgr2=0x1b3fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 shutdown_connections 2023-09-27T23:52:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 1 -- 172.21.15.64:0/745167589 wait complete. 2023-09-27T23:52:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.792+0000 7f9abd1e6740 20 client.10473 trim_cache size 0 max 16384 2023-09-27T23:52:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 Processor -- start 2023-09-27T23:52:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 -- start start 2023-09-27T23:52:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190d710 0x1b99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b99fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 0x1b9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ab0b50 con 0x1958790 2023-09-27T23:52:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ab5770 con 0x190dae0 2023-09-27T23:52:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a7d920 con 0x190d710 2023-09-27T23:52:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9ab94da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 0x1b9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9ab94da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 0x1b9a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55988/0 (socket says 172.21.15.64:55988) 2023-09-27T23:52:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b99fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9ab94da700 1 -- 172.21.15.64:0/2905055564 learned_addr learned my addr 172.21.15.64:0/2905055564 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.812+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190d710 0x1b99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab3fff700 1 -- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190d710 msgr2=0x1b99a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190d710 0x1b99a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab3fff700 1 -- 172.21.15.64:0/2905055564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 msgr2=0x1b9a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 0x1b9a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab3fff700 1 -- 172.21.15.64:0/2905055564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4144ce0 con 0x190dae0 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab94da700 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 0x1b9a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b99fd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9aa001c000 tx=0x7f9aa0020040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f3af55432b3cd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9a9e7fc700 1 -- 172.21.15.64:0/2905055564 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0024040 con 0x190dae0 2023-09-27T23:52:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9a9e7fc700 1 -- 172.21.15.64:0/2905055564 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa0026040 con 0x190dae0 2023-09-27T23:52:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9a9e7fc700 1 -- 172.21.15.64:0/2905055564 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0011e80 con 0x190dae0 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 -- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 msgr2=0x1b99fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b99fd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9aa001c000 tx=0x7f9aa0020040 comp rx=0 tx=0).stop 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 -- 172.21.15.64:0/2905055564 shutdown_connections 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1958790 0x1b9a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b99fd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/2905055564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x190d710 0x1b99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 -- 172.21.15.64:0/2905055564 >> 172.21.15.64:0/2905055564 conn(0x190deb0 msgr2=0x1b6fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 -- 172.21.15.64:0/2905055564 shutdown_connections 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 -- 172.21.15.64:0/2905055564 wait complete. 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 -- start start 2023-09-27T23:52:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.816+0000 7f9abd1e6740 1 Processor -- start 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b6c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b6ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a8a550 con 0x190d710 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x19f1430 con 0x190dae0 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a8d910 con 0x1958790 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 10 client.0 _ll_register_callbacks cb 0x19d10e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b6c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b6c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55994/0 (socket says 172.21.15.64:55994) 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/1075226908 learned_addr learned my addr 172.21.15.64:0/1075226908 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b6ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab94da700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 msgr2=0x1b6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 msgr2=0x1b6ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b6ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 -- 172.21.15.64:0/1075226908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac020070 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab94da700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab8cd9700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b6c900 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8029eb0 tx=0x7f9aa802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c877c69c3d449b36 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9a92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9a92ffd700 1 -- 172.21.15.64:0/1075226908 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8021040 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9a92ffd700 1 -- 172.21.15.64:0/1075226908 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9aa8033410 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9aa0011cf0 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9a92ffd700 1 -- 172.21.15.64:0/1075226908 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa8033910 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 10 client.10479 fetch_fsmap learned FSMap version 72 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x19599e0 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9a92ffd700 1 -- 172.21.15.64:0/1075226908 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa0011cf0 con 0x190d710 2023-09-27T23:52:29.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9a92ffd700 1 -- 172.21.15.64:0/1075226908 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 72) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9aa8033a70 con 0x190d710 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 10 client.10479 fetch_fsmap finished waiting for FSMap version 72 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 10 client.10479 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 20 client.10479 populate_metadata read hostname 'smithi064' 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1becf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 4 client.10479 mds_command: new command op to 10144 tid=0[{"prefix": "status"}] 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 --> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1a89410 con 0x1b976b0 2023-09-27T23:52:29.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.820+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1becf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9a92ffd700 10 client.10479 ms_handle_connect on v2:172.21.15.123:6834/1929792685 2023-09-27T23:52:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab3fff700 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1becf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642e253092604086 server_cookie=67dac154762bd2b3 in_seq=0 out_seq=0 2023-09-27T23:52:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9a92ffd700 1 -- 172.21.15.64:0/1075226908 <== mds.0 v2:172.21.15.123:6834/1929792685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x1a89410 con 0x1b976b0 2023-09-27T23:52:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9a92ffd700 10 client.10479 handle_command_reply: tid=0 2023-09-27T23:52:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 client.10479 shutdown 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 msgr2=0x1b6c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b6c900 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8029eb0 tx=0x7f9aa802f000 comp rx=0 tx=0).stop 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 shutdown_connections 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:6834/1929792685,v1:172.21.15.123:6835/1929792685] conn(0x1b976b0 0x1becf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x1958790 0x1b6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x190dae0 0x1b6ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 --2- 172.21.15.64:0/1075226908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x190d710 0x1b6c900 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9a92ffd700 0 client.10479 ms_handle_reset on v2:172.21.15.123:6834/1929792685 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 >> 172.21.15.64:0/1075226908 conn(0x190deb0 msgr2=0x1b70da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 shutdown_connections 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 1 -- 172.21.15.64:0/1075226908 wait complete. 2023-09-27T23:52:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9abd1e6740 20 client.10479 trim_cache size 0 max 16384 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "9497c172-4ccf-4119-8910-f606cb59c9f0", 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 10144, 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 9.5539916290000004, 2023-09-27T23:52:29.832 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 72, 2023-09-27T23:52:29.833 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 189, 2023-09-27T23:52:29.833 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 189, 2023-09-27T23:52:29.833 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 14.151693077999999 2023-09-27T23:52:29.833 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:29.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9a98060a70 msgr2=0x7f9a98062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9a98060a70 0x7f9a98062f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80093f0 tx=0x7f9aa8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 msgr2=0x7f9ab4144440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 0x7f9ab4144440 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00cc70 tx=0x7f9aac001a70 comp rx=0 tx=0).stop 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 shutdown_connections 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9a98060a70 0x7f9a98062f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ab415ca80 0x7f9ab4144440 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ab41591f0 0x7f9ab40c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 --2- 172.21.15.64:0/996652668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab4158e20 0x7f9ab4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 >> 172.21.15.64:0/996652668 conn(0x7f9ab4067bb0 msgr2=0x7f9ab40b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 shutdown_connections 2023-09-27T23:52:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.824+0000 7f9ab9cdb700 1 -- 172.21.15.64:0/996652668 wait complete. 2023-09-27T23:52:29.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:52:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 Processor -- start 2023-09-27T23:52:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 -- start start 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4014155960 0x7f401415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40140c7f00 0x7f40140aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401406d990 con 0x7f401406de20 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401415a570 con 0x7f40140c7f00 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401adb0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401406cd80 con 0x7f4014155960 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401a5af700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40140c7f00 0x7f40140aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f40195ad700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f401406de20 0x7f40140c77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39334/0 (socket says 172.21.15.64:39334) 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f40195ad700 1 -- 172.21.15.64:0/2300221092 learned_addr learned my addr 172.21.15.64:0/2300221092 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f4019dae700 1 --2- 172.21.15.64:0/2300221092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4014155960 0x7f401415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401a5af700 1 -- 172.21.15.64:0/2300221092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4014155960 msgr2=0x7f401415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f401a5af700 1 --2- 172.21.15.64:0/2300221092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4014155960 0x7f401415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.952+0000 7f4018dac700 1 -- 172.21.15.64:0/2300221092 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973956997 0 0) 0x7f401406d990 con 0x7f401406de20 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401a5af700 1 -- 172.21.15.64:0/2300221092 >> v1:172.21.15.64:6789/0 conn(0x7f401406de20 legacy=0x7f40140c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401a5af700 1 -- 172.21.15.64:0/2300221092 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401406d060 con 0x7f40140c7f00 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401a5af700 1 --2- 172.21.15.64:0/2300221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40140c7f00 0x7f40140aca10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f400c0098e0 tx=0x7f400c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da29c8f4f1ece617 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f4018dac700 1 -- 172.21.15.64:0/2300221092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00b640 con 0x7f40140c7f00 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f4018dac700 1 -- 172.21.15.64:0/2300221092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c00b7a0 con 0x7f40140c7f00 2023-09-27T23:52:29.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f4018dac700 1 -- 172.21.15.64:0/2300221092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00bb40 con 0x7f40140c7f00 2023-09-27T23:52:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 -- 172.21.15.64:0/2300221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40140c7f00 msgr2=0x7f40140aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 --2- 172.21.15.64:0/2300221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40140c7f00 0x7f40140aca10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f400c0098e0 tx=0x7f400c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 -- 172.21.15.64:0/2300221092 shutdown_connections 2023-09-27T23:52:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 --2- 172.21.15.64:0/2300221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40140c7f00 0x7f40140aca10 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 --2- 172.21.15.64:0/2300221092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4014155960 0x7f401415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 -- 172.21.15.64:0/2300221092 >> 172.21.15.64:0/2300221092 conn(0x7f4014067bb0 msgr2=0x7f40140cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 -- 172.21.15.64:0/2300221092 shutdown_connections 2023-09-27T23:52:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.956+0000 7f401adb0700 1 -- 172.21.15.64:0/2300221092 wait complete. 2023-09-27T23:52:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 Processor -- start 2023-09-27T23:52:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 -- start start 2023-09-27T23:52:29.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f401406de20 0x7f40141661a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40140c7f00 0x7f401415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 0x7f401415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40140b0b90 con 0x7f4014155960 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4014153a20 con 0x7f40140c7f00 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4014153d00 con 0x7f401406de20 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 0x7f401415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 0x7f401415bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56020/0 (socket says 172.21.15.64:56020) 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 -- 172.21.15.64:0/1636375372 learned_addr learned my addr 172.21.15.64:0/1636375372 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:29.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f40195ad700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40140c7f00 0x7f401415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f4019dae700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f401406de20 0x7f40141661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 -- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f401406de20 msgr2=0x7f40141661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f401406de20 0x7f40141661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 -- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40140c7f00 msgr2=0x7f401415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40140c7f00 0x7f401415b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 -- 172.21.15.64:0/1636375372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c009420 con 0x7f4014155960 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f40195ad700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40140c7f00 0x7f401415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f4019dae700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f401406de20 0x7f40141661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401a5af700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 0x7f401415bc90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f400c013000 tx=0x7f400c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c011420 con 0x7f4014155960 2023-09-27T23:52:29.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c011580 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4014153fe0 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c011920 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.960+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40141545a0 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.964+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f400c011c90 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.964+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4014154790 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.964+0000 7f4009ffb700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4000060a70 0x7f4000062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.964+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f400c07e6a0 con 0x7f4014155960 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.964+0000 7f4019dae700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4000060a70 0x7f4000062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:29.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.964+0000 7f4019dae700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4000060a70 0x7f4000062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4004009ce0 tx=0x7f4004008040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:29.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:29.972+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f400c04e6d0 con 0x7f4014155960 2023-09-27T23:52:30.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.172+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f401404be40 con 0x7f4014155960 2023-09-27T23:52:30.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.172+0000 7f4009ffb700 1 -- 172.21.15.64:0/1636375372 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f400c051cf0 con 0x7f4014155960 2023-09-27T23:52:30.178 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4000060a70 msgr2=0x7f4000062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4000060a70 0x7f4000062f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4004009ce0 tx=0x7f4004008040 comp rx=0 tx=0).stop 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 msgr2=0x7f401415bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 0x7f401415bc90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f400c013000 tx=0x7f400c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 shutdown_connections 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4000060a70 0x7f4000062f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014155960 0x7f401415bc90 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40140c7f00 0x7f401415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 --2- 172.21.15.64:0/1636375372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f401406de20 0x7f40141661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 >> 172.21.15.64:0/1636375372 conn(0x7f4014067bb0 msgr2=0x7f40140c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 shutdown_connections 2023-09-27T23:52:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:30.176+0000 7f401adb0700 1 -- 172.21.15.64:0/1636375372 wait complete. 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:52:30.198 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:52:30.199 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:52:30.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:30.199 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:30.207 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 40234sec preferred_lft 40234sec 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:52:30.208 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:52:30.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:52:30.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.210 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:52:30.210 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:52:30.210 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:52:30.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:30.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:30.210 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:52:30.221 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:52:30.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:30.223 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:52:30.272 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:52:30.272 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:52:30.272 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:52:30.272 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:52:30.272 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:52:30.272 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:52:30.273 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:52:30.273 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:52:30.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:30.273 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:52:30.273 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:52:30.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:52:30.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:52:30.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:52:30.273 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:52:30.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:30.316 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:52:30.323 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:52:30.323 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:52:30.323 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:52:30.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:30.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:52:30.337 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:52:30.337 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:52:30.337 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:30.387 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:52:30.387 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:52:30.387 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:30.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:52:30.412 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:52:30.412 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:52:30.422 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:52:30.424 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:52:30.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:30.424 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:52:30.474 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:52:30.474 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:52:30.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:52:30.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:52:30.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:52:30.507 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:52:30.508 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:52:30.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:30.509 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:52:30.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:52:30.560+0000 7faabe199380 -1 init, newargv = 0x555e47ff9c40 newargc=15 2023-09-27T23:52:30.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[62498]: starting ceph client 2023-09-27T23:52:30.573 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[62498]: starting fuse 2023-09-27T23:52:31.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:52:31.681 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:52:31.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:52:31.695 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:52:31.696 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:52:31.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:31.697 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:52:31.703 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:52:31.704 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:52:31.704 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:52:31.705 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:52:31.706 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:52:31.706 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:52:31.706 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:31.706 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:52:31.706 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:52:31.788 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.62498.asok 2023-09-27T23:52:31.973 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62498.asok 2023-09-27T23:52:31.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:31.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62498.asok status 2023-09-27T23:52:32.115 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:32.115 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:52:32.115 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "62498", 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 10494, 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:52:32.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 10494 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2027673777 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:52:32.117 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2027673777 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.10494 192.168.0.1:0/2027673777", 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2027673777", 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 72, 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 189, 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:52:32.118 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:32.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:32.125 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:52:32.126 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:32.133 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:52:32.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:32.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:32.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:32.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-09-27T23:52:32.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 Processor -- start 2023-09-27T23:52:32.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 -- start start 2023-09-27T23:52:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f56600cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f566015bd00 0x7f566015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566006b2c0 con 0x7f566006b750 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566006e0f0 con 0x7f566015bd00 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665f37700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566006a6b0 con 0x7f56600cad10 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665736700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f566015bd00 0x7f566015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f565ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f566006b750 0x7f566006d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39340/0 (socket says 172.21.15.64:39340) 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f565ffff700 1 -- 172.21.15.64:0/657508030 learned_addr learned my addr 172.21.15.64:0/657508030 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5664f35700 1 --2- 172.21.15.64:0/657508030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f56600cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f565f7fe700 1 -- 172.21.15.64:0/657508030 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347686811 0 0) 0x7f566006b2c0 con 0x7f566006b750 2023-09-27T23:52:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f565f7fe700 1 -- 172.21.15.64:0/657508030 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f564c003570 con 0x7f566006b750 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665736700 1 -- 172.21.15.64:0/657508030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 msgr2=0x7f56600cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665736700 1 --2- 172.21.15.64:0/657508030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f56600cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665736700 1 -- 172.21.15.64:0/657508030 >> v1:172.21.15.64:6789/0 conn(0x7f566006b750 legacy=0x7f566006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f565f7fe700 1 -- 172.21.15.64:0/657508030 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836521282 0 0) 0x7f564c003570 con 0x7f566006b750 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665736700 1 -- 172.21.15.64:0/657508030 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566006a990 con 0x7f566015bd00 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f5665736700 1 --2- 172.21.15.64:0/657508030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f566015c180 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f5658009910 tx=0x7f56580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80678732181df9b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.256+0000 7f565f7fe700 1 -- 172.21.15.64:0/657508030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658011630 con 0x7f566015bd00 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f565f7fe700 1 -- 172.21.15.64:0/657508030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5658011790 con 0x7f566015bd00 2023-09-27T23:52:32.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f565f7fe700 1 -- 172.21.15.64:0/657508030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658011b10 con 0x7f566015bd00 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 -- 172.21.15.64:0/657508030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 msgr2=0x7f566015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 --2- 172.21.15.64:0/657508030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f566015c180 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f5658009910 tx=0x7f56580091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 -- 172.21.15.64:0/657508030 shutdown_connections 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 --2- 172.21.15.64:0/657508030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f566015c180 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 --2- 172.21.15.64:0/657508030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f56600cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 -- 172.21.15.64:0/657508030 >> 172.21.15.64:0/657508030 conn(0x7f5660067bb0 msgr2=0x7f56600cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 -- 172.21.15.64:0/657508030 shutdown_connections 2023-09-27T23:52:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 -- 172.21.15.64:0/657508030 wait complete. 2023-09-27T23:52:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 Processor -- start 2023-09-27T23:52:32.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.260+0000 7f5665f37700 1 -- start start 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b750 0x7f566016c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f566016c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f5660159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566016aa20 con 0x7f566015bd00 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660169e10 con 0x7f566006b750 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566016a0f0 con 0x7f56600cad10 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f5660159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f5660159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56044/0 (socket says 172.21.15.64:56044) 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 -- 172.21.15.64:0/687924139 learned_addr learned my addr 172.21.15.64:0/687924139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f565ffff700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f566016c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5664f35700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b750 0x7f566016c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 -- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 msgr2=0x7f566016c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f566016c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 -- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b750 msgr2=0x7f566016c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b750 0x7f566016c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 -- 172.21.15.64:0/687924139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009420 con 0x7f566015bd00 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665736700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f5660159db0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f5658009f00 tx=0x7f5658017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658011b10 con 0x7f566015bd00 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5658011e80 con 0x7f566015bd00 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566016a3d0 con 0x7f566015bd00 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565801a640 con 0x7f566015bd00 2023-09-27T23:52:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.264+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660146be0 con 0x7f566015bd00 2023-09-27T23:52:32.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.268+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f565801aa30 con 0x7f566015bd00 2023-09-27T23:52:32.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.268+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566016ab90 con 0x7f566015bd00 2023-09-27T23:52:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.268+0000 7f565cff9700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5640060aa0 0x7f5640062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.268+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f565807ed70 con 0x7f566015bd00 2023-09-27T23:52:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.268+0000 7f5664f35700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5640060aa0 0x7f5640062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.268+0000 7f5664f35700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5640060aa0 0x7f5640062f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f565000f570 tx=0x7f565000b040 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.276+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f565804eda0 con 0x7f566015bd00 2023-09-27T23:52:32.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.464+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f566004be40 con 0x7f5640060aa0 2023-09-27T23:52:32.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f565cff9700 1 -- 172.21.15.64:0/687924139 <== mgr.9940 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f566004be40 con 0x7f5640060aa0 2023-09-27T23:52:32.473 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-27T23:52:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-27T23:52:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephfs" 2023-09-27T23:52:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:32.473 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5640060aa0 msgr2=0x7f5640062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5640060aa0 0x7f5640062f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f565000f570 tx=0x7f565000b040 comp rx=0 tx=0).stop 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 msgr2=0x7f5660159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f5660159db0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f5658009f00 tx=0x7f5658017000 comp rx=0 tx=0).stop 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 shutdown_connections 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f5640060aa0 0x7f5640062f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566015bd00 0x7f5660159db0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f56600cad10 0x7f566016c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 --2- 172.21.15.64:0/687924139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b750 0x7f566016c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 >> 172.21.15.64:0/687924139 conn(0x7f5660067bb0 msgr2=0x7f56600b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 shutdown_connections 2023-09-27T23:52:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.472+0000 7f5665f37700 1 -- 172.21.15.64:0/687924139 wait complete. 2023-09-27T23:52:32.494 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-09-27T23:52:32.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.600+0000 7fd60c974700 1 Processor -- start 2023-09-27T23:52:32.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.600+0000 7fd60c974700 1 -- start start 2023-09-27T23:52:32.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd608158b70 0x7fd6080cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608158f40 0x7fd6080677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd608067d10 0x7fd608068170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080c86b0 con 0x7fd608158b70 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080c8380 con 0x7fd608067d10 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd608068880 con 0x7fd608158f40 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd608158b70 0x7fd6080cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd608158b70 0x7fd6080cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56046/0 (socket says 172.21.15.64:56046) 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd608067d10 0x7fd608068170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 -- 172.21.15.64:0/3398687237 learned_addr learned my addr 172.21.15.64:0/3398687237 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:32.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60659c700 1 --2- 172.21.15.64:0/3398687237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608158f40 0x7fd6080677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 -- 172.21.15.64:0/3398687237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608158f40 msgr2=0x7fd6080677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 --2- 172.21.15.64:0/3398687237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608158f40 0x7fd6080677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 -- 172.21.15.64:0/3398687237 >> v2:172.21.15.123:3300/0 conn(0x7fd608067d10 msgr2=0x7fd608068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 --2- 172.21.15.64:0/3398687237 >> v2:172.21.15.123:3300/0 conn(0x7fd608067d10 0x7fd608068170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 -- 172.21.15.64:0/3398687237 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608068b60 con 0x7fd608158b70 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd606d9d700 1 --2- 172.21.15.64:0/3398687237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158b70 0x7fd6080cb840 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fd608067290 tx=0x7fd5f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8247fd4a7f6ea837 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd605d9b700 1 -- 172.21.15.64:0/3398687237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800b6d0 con 0x7fd608158b70 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd605d9b700 1 -- 172.21.15.64:0/3398687237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f800b830 con 0x7fd608158b70 2023-09-27T23:52:32.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd605d9b700 1 -- 172.21.15.64:0/3398687237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800bbf0 con 0x7fd608158b70 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- 172.21.15.64:0/3398687237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158b70 msgr2=0x7fd6080cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- 172.21.15.64:0/3398687237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158b70 0x7fd6080cb840 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fd608067290 tx=0x7fd5f8009250 comp rx=0 tx=0).stop 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- 172.21.15.64:0/3398687237 shutdown_connections 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- 172.21.15.64:0/3398687237 >> v2:172.21.15.123:3300/0 conn(0x7fd608067d10 0x7fd608068170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- 172.21.15.64:0/3398687237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608158f40 0x7fd6080677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 --2- 172.21.15.64:0/3398687237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158b70 0x7fd6080cb840 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- 172.21.15.64:0/3398687237 >> 172.21.15.64:0/3398687237 conn(0x7fd6080cdc00 msgr2=0x7fd6080ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- 172.21.15.64:0/3398687237 shutdown_connections 2023-09-27T23:52:32.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.604+0000 7fd60c974700 1 -- 172.21.15.64:0/3398687237 wait complete. 2023-09-27T23:52:32.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 Processor -- start 2023-09-27T23:52:32.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 -- start start 2023-09-27T23:52:32.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608067d10 0x7fd6080c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608158b70 0x7fd608147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 0x7fd608155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd608169210 con 0x7fd608158f40 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b22a0 con 0x7fd608158b70 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080c7120 con 0x7fd608067d10 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 0x7fd608155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 0x7fd608155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56052/0 (socket says 172.21.15.64:56052) 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 -- 172.21.15.64:0/1045215278 learned_addr learned my addr 172.21.15.64:0/1045215278 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60659c700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608158b70 0x7fd608147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd606d9d700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608067d10 0x7fd6080c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 -- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608067d10 msgr2=0x7fd6080c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608067d10 0x7fd6080c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 -- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608158b70 msgr2=0x7fd608147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608158b70 0x7fd608147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 -- 172.21.15.64:0/1045215278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80094f0 con 0x7fd608158f40 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd606d9d700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608067d10 0x7fd6080c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60759e700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 0x7fd608155d90 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fd60000e8d0 tx=0x7fd6000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600013680 con 0x7fd608158f40 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6000137e0 con 0x7fd608158f40 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600013ba0 con 0x7fd608158f40 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.608+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6080c7400 con 0x7fd608158f40 2023-09-27T23:52:32.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.612+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6080c79c0 con 0x7fd608158f40 2023-09-27T23:52:32.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.612+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd608169380 con 0x7fd608158f40 2023-09-27T23:52:32.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.612+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 89) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd6000253d0 con 0x7fd608158f40 2023-09-27T23:52:32.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.612+0000 7fd5feffd700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd5e8060a70 0x7fd5e8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:32.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.612+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd60007d900 con 0x7fd608158f40 2023-09-27T23:52:32.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.612+0000 7fd606d9d700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd5e8060a70 0x7fd5e8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:32.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.616+0000 7fd606d9d700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd5e8060a70 0x7fd5e8062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd608067290 tx=0x7fd5f8017000 comp rx=0 tx=0).ready entity=mgr.9940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:32.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.620+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd60004d950 con 0x7fd608158f40 2023-09-27T23:52:32.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:32.824+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd608169350 con 0x7fd608158f40 2023-09-27T23:52:33.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.800+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 90) v1 ==== 80378+0+0 (secure 0 0 0) 0x7fd60004dd20 con 0x7fd608158f40 2023-09-27T23:52:33.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.804+0000 7fd5feffd700 1 -- 172.21.15.64:0/1045215278 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v90) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd6000513e0 con 0x7fd608158f40 2023-09-27T23:52:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd5e8060a70 msgr2=0x7fd5e8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd5e8060a70 0x7fd5e8062f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd608067290 tx=0x7fd5f8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 msgr2=0x7fd608155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 0x7fd608155d90 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fd60000e8d0 tx=0x7fd6000091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 shutdown_connections 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd5e8060a70 0x7fd5e8062f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd608158f40 0x7fd608155d90 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608158b70 0x7fd608147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 --2- 172.21.15.64:0/1045215278 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd608067d10 0x7fd6080c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 >> 172.21.15.64:0/1045215278 conn(0x7fd6080cdc00 msgr2=0x7fd60815c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 shutdown_connections 2023-09-27T23:52:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.808+0000 7fd60c974700 1 -- 172.21.15.64:0/1045215278 wait complete. 2023-09-27T23:52:33.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-09-27T23:52:33.927 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:33.923+0000 7f922913b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:52:33.949 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:33.944+0000 7f56397b8000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:52:33.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.944+0000 7f8caac03700 1 Processor -- start 2023-09-27T23:52:33.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.944+0000 7f8caac03700 1 -- start start 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ca406b7a0 0x7f8ca4161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca40cbee0 con 0x7f8ca40cae60 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca40cc370 con 0x7f8ca41514a0 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca40cb2d0 con 0x7f8ca406b7a0 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8ca40cae60 0x7f8ca406b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:39350/0 (socket says 172.21.15.64:39350) 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 -- 172.21.15.64:0/1333279896 learned_addr learned my addr 172.21.15.64:0/1333279896 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca91a0700 1 --2- 172.21.15.64:0/1333279896 >> v2:172.21.15.123:3300/0 conn(0x7f8ca406b7a0 0x7f8ca4161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799889644 0 0) 0x7f8ca40cbee0 con 0x7f8ca40cae60 2023-09-27T23:52:33.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c94003570 con 0x7f8ca40cae60 2023-09-27T23:52:33.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2856708256 0 0) 0x7f8c94003570 con 0x7f8ca40cae60 2023-09-27T23:52:33.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 >> v2:172.21.15.123:3300/0 conn(0x7f8ca406b7a0 msgr2=0x7f8ca4161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:33.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 --2- 172.21.15.64:0/1333279896 >> v2:172.21.15.123:3300/0 conn(0x7f8ca406b7a0 0x7f8ca4161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 >> v1:172.21.15.123:6789/0 conn(0x7f8ca41514a0 legacy=0x7f8ca4157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40cb5b0 con 0x7f8ca40cae60 2023-09-27T23:52:33.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f8c90001470 con 0x7f8ca40cae60 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 >> v1:172.21.15.64:6789/0 conn(0x7f8ca40cae60 legacy=0x7f8ca406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94003a60 0x7f8c94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c94004810 0x7f8c94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c940051c0 0x7f8c9400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40cb2d0 con 0x7f8c94004810 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9400c160 con 0x7f8c94003a60 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9400c320 con 0x7f8c940051c0 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c94004810 0x7f8c94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca899f700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94003a60 0x7f8c94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca91a0700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c940051c0 0x7f8c9400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 -- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c940051c0 msgr2=0x7f8c9400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c940051c0 0x7f8c9400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 -- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94003a60 msgr2=0x7f8c94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94003a60 0x7f8c94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 -- 172.21.15.64:0/1333279896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94003570 con 0x7f8c94004810 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca91a0700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c940051c0 0x7f8c9400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca91a0700 1 -- 172.21.15.64:0/1333279896 reap_dead start 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca3fff700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c94004810 0x7f8c94004c80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000c3c0 tx=0x7f8c90001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0ab95bcdf719c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000cc90 con 0x7f8c94004810 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000ce10 con 0x7f8c94004810 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8ca37fe700 1 -- 172.21.15.64:0/1333279896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90005b70 con 0x7f8c94004810 2023-09-27T23:52:33.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- 172.21.15.64:0/1333279896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c94004810 msgr2=0x7f8c94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c94004810 0x7f8c94004c80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000c3c0 tx=0x7f8c90001e50 comp rx=0 tx=0).stop 2023-09-27T23:52:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- 172.21.15.64:0/1333279896 shutdown_connections 2023-09-27T23:52:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 --2- 172.21.15.64:0/1333279896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c94004810 0x7f8c94004c80 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- 172.21.15.64:0/1333279896 >> 172.21.15.64:0/1333279896 conn(0x7f8ca40cc580 msgr2=0x7f8ca40cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- 172.21.15.64:0/1333279896 shutdown_connections 2023-09-27T23:52:33.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.948+0000 7f8caac03700 1 -- 172.21.15.64:0/1333279896 wait complete. 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 Processor -- start 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 -- start start 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94004810 0x7f8ca40ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ca416b400 0x7f8ca415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 0x7f8ca413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40ad410 con 0x7f8ca415bf20 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40ad6f0 con 0x7f8c94004810 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8caac03700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40ad9d0 con 0x7f8ca416b400 2023-09-27T23:52:33.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 0x7f8ca413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca899f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94004810 0x7f8ca40ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 0x7f8ca413cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56076/0 (socket says 172.21.15.64:56076) 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 -- 172.21.15.64:0/1797845769 learned_addr learned my addr 172.21.15.64:0/1797845769 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca3fff700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ca416b400 0x7f8ca415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 -- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ca416b400 msgr2=0x7f8ca415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ca416b400 0x7f8ca415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 -- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94004810 msgr2=0x7f8ca40ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94004810 0x7f8ca40ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 -- 172.21.15.64:0/1797845769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40adcb0 con 0x7f8ca415bf20 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.952+0000 7f8ca91a0700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 0x7f8ca413cee0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40ca130 tx=0x7f8c9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00fbd0 con 0x7f8ca415bf20 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c00fd30 con 0x7f8ca415bf20 2023-09-27T23:52:33.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca4153610 con 0x7f8ca415bf20 2023-09-27T23:52:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c016650 con 0x7f8ca415bf20 2023-09-27T23:52:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca4156340 con 0x7f8ca415bf20 2023-09-27T23:52:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca4071120 con 0x7f8ca415bf20 2023-09-27T23:52:33.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 90) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f8c9c016a00 con 0x7f8ca415bf20 2023-09-27T23:52:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca0ff9700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8c88060ac0 0x7f8c88062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca899f700 1 -- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8c88060ac0 msgr2=0x7f8c88062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:33.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8c9c07dd90 con 0x7f8ca415bf20 2023-09-27T23:52:33.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.956+0000 7f8ca899f700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8c88060ac0 0x7f8c88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:33.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:33.964+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8c9c051740 con 0x7f8ca415bf20 2023-09-27T23:52:34.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.124+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f8ca40cb890 con 0x7f8ca415bf20 2023-09-27T23:52:34.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8ca0ff9700 1 -- 172.21.15.64:0/1797845769 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8c9c00fea0 con 0x7f8ca415bf20 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8c88060ac0 msgr2=0x7f8c88062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8c88060ac0 0x7f8c88062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 msgr2=0x7f8ca413cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 0x7f8ca413cee0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40ca130 tx=0x7f8c9c00b040 comp rx=0 tx=0).stop 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 shutdown_connections 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8c88060ac0 0x7f8c88062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ca415bf20 0x7f8ca413cee0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ca416b400 0x7f8ca415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 --2- 172.21.15.64:0/1797845769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c94004810 0x7f8ca40ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 >> 172.21.15.64:0/1797845769 conn(0x7f8ca40cc580 msgr2=0x7f8ca4151870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:34.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 shutdown_connections 2023-09-27T23:52:34.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.132+0000 7f8caac03700 1 -- 172.21.15.64:0/1797845769 wait complete. 2023-09-27T23:52:34.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-09-27T23:52:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 Processor -- start 2023-09-27T23:52:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 -- start start 2023-09-27T23:52:34.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb80b08d0 0x7f2eb8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb815a3d0 con 0x7f2eb80ab870 2023-09-27T23:52:34.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb80b0e40 con 0x7f2eb80b08d0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebf8b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80b1120 con 0x7f2eb8159fb0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb80b08d0 0x7f2eb8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2eb80b08d0 0x7f2eb8159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56090/0 (socket says 172.21.15.64:56090) 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 -- 172.21.15.64:0/3101360268 learned_addr learned my addr 172.21.15.64:0/3101360268 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebde50700 1 --2- 172.21.15.64:0/3101360268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 -- 172.21.15.64:0/3101360268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 msgr2=0x7f2eb806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 --2- 172.21.15.64:0/3101360268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 -- 172.21.15.64:0/3101360268 >> v1:172.21.15.123:6789/0 conn(0x7f2eb80ab870 legacy=0x7f2eb80afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 -- 172.21.15.64:0/3101360268 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80b1400 con 0x7f2eb80b08d0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2ebce4e700 1 --2- 172.21.15.64:0/3101360268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb8159a70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80099b0 tx=0x7f2ea8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672a535e7ae726e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2eaffff700 1 -- 172.21.15.64:0/3101360268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800b860 con 0x7f2eb80b08d0 2023-09-27T23:52:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2eaffff700 1 -- 172.21.15.64:0/3101360268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800b9c0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.252+0000 7f2eaffff700 1 -- 172.21.15.64:0/3101360268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800bd60 con 0x7f2eb80b08d0 2023-09-27T23:52:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/3101360268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 msgr2=0x7f2eb8159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/3101360268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb8159a70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80099b0 tx=0x7f2ea8009250 comp rx=0 tx=0).stop 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/3101360268 shutdown_connections 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/3101360268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/3101360268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb8159a70 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/3101360268 >> 172.21.15.64:0/3101360268 conn(0x7f2eb80cc580 msgr2=0x7f2eb80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/3101360268 shutdown_connections 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/3101360268 wait complete. 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 Processor -- start 2023-09-27T23:52:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- start start 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80ab870 0x7f2eb80c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb80cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80ad0c0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80ac4b0 con 0x7f2eb80ab870 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80ac790 con 0x7f2eb8159fb0 2023-09-27T23:52:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb80cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb80cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56098/0 (socket says 172.21.15.64:56098) 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 -- 172.21.15.64:0/2123778393 learned_addr learned my addr 172.21.15.64:0/2123778393 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebd64f700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80ab870 0x7f2eb80c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebde50700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 -- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 msgr2=0x7f2eb806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 -- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80ab870 msgr2=0x7f2eb80c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80ab870 0x7f2eb80c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 -- 172.21.15.64:0/2123778393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80094f0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebce4e700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb80cb720 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8064900 tx=0x7f2ea8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8011e40 con 0x7f2eb80b08d0 2023-09-27T23:52:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8024440 con 0x7f2eb80b08d0 2023-09-27T23:52:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8024800 con 0x7f2eb80b08d0 2023-09-27T23:52:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb80aca70 con 0x7f2eb80b08d0 2023-09-27T23:52:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.256+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81514c0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.260+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb80ad230 con 0x7f2eb80b08d0 2023-09-27T23:52:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.260+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 90) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f2ea801a420 con 0x7f2eb80b08d0 2023-09-27T23:52:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.260+0000 7f2ead7fa700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e9c060ac0 0x7f2e9c062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.260+0000 7f2ebd64f700 1 -- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e9c060ac0 msgr2=0x7f2e9c062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.260+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2ea807efa0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.260+0000 7f2ebd64f700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e9c060ac0 0x7f2e9c062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:34.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.268+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2ea804efd0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.353 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.351+0000 7f922913b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:52:34.401 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.399+0000 7f922913b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:52:34.405 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:34.400+0000 7f56397b8000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:52:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.432+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f2eb8071120 con 0x7f2eb80b08d0 2023-09-27T23:52:34.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.444+0000 7f2ead7fa700 1 -- 172.21.15.64:0/2123778393 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2ea80525f0 con 0x7f2eb80b08d0 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e9c060ac0 msgr2=0x7f2e9c062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e9c060ac0 0x7f2e9c062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 msgr2=0x7f2eb80cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb80cb720 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8064900 tx=0x7f2ea8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 shutdown_connections 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2e9c060ac0 0x7f2e9c062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2eb8159fb0 0x7f2eb806d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb80b08d0 0x7f2eb80cb720 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 --2- 172.21.15.64:0/2123778393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb80ab870 0x7f2eb80c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 >> 172.21.15.64:0/2123778393 conn(0x7f2eb80cc580 msgr2=0x7f2eb80afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:34.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 shutdown_connections 2023-09-27T23:52:34.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.448+0000 7f2ebf8b3700 1 -- 172.21.15.64:0/2123778393 wait complete. 2023-09-27T23:52:34.458 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:34.452+0000 7f56397b8000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:52:34.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:34.465 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-09-27T23:52:34.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-09-27T23:52:34.510 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.507+0000 7f922913b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:34.594 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:34.588+0000 7f56397b8000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fbcbe4700 1 Processor -- start 2023-09-27T23:52:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fbcbe4700 1 -- start start 2023-09-27T23:52:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fbcbe4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9fb813f830 0x7f9fb80ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fbcbe4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb80cc200 con 0x7f9fb80aea30 2023-09-27T23:52:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fbcbe4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb80cb160 con 0x7f9fb80c7780 2023-09-27T23:52:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fbcbe4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb80cb440 con 0x7f9fb813f830 2023-09-27T23:52:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb5d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9fb80aea30 0x7f9fb80aeed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37770/0 (socket says 172.21.15.64:37770) 2023-09-27T23:52:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb5d9b700 1 -- 172.21.15.64:0/2181100734 learned_addr learned my addr 172.21.15.64:0/2181100734 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb659c700 1 --2- 172.21.15.64:0/2181100734 >> v2:172.21.15.123:3300/0 conn(0x7f9fb813f830 0x7f9fb80ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb559a700 1 -- 172.21.15.64:0/2181100734 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203687709 0 0) 0x7f9fb80cc200 con 0x7f9fb80aea30 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb559a700 1 -- 172.21.15.64:0/2181100734 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa0003570 con 0x7f9fb80aea30 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb659c700 1 -- 172.21.15.64:0/2181100734 >> v1:172.21.15.123:6789/0 conn(0x7f9fb80c7780 legacy=0x7f9fb80c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb659c700 1 -- 172.21.15.64:0/2181100734 >> v1:172.21.15.64:6789/0 conn(0x7f9fb80aea30 legacy=0x7f9fb80aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb659c700 1 -- 172.21.15.64:0/2181100734 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb80cb720 con 0x7f9fb813f830 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb659c700 1 --2- 172.21.15.64:0/2181100734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb813f830 0x7f9fb80ca7f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80098e0 tx=0x7f9fa80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a27828dc73a7176f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb559a700 1 -- 172.21.15.64:0/2181100734 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800b640 con 0x7f9fb813f830 2023-09-27T23:52:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb559a700 1 -- 172.21.15.64:0/2181100734 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa800b7a0 con 0x7f9fb813f830 2023-09-27T23:52:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.640+0000 7f9fb559a700 1 -- 172.21.15.64:0/2181100734 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800bb20 con 0x7f9fb813f830 2023-09-27T23:52:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/2181100734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb813f830 msgr2=0x7f9fb80ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/2181100734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb813f830 0x7f9fb80ca7f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80098e0 tx=0x7f9fa80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/2181100734 shutdown_connections 2023-09-27T23:52:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/2181100734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb813f830 0x7f9fb80ca7f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/2181100734 >> 172.21.15.64:0/2181100734 conn(0x7f9fb80ccc20 msgr2=0x7f9fb80cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/2181100734 shutdown_connections 2023-09-27T23:52:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/2181100734 wait complete. 2023-09-27T23:52:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 Processor -- start 2023-09-27T23:52:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 -- start start 2023-09-27T23:52:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fb80aea30 0x7f9fb8069a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.644+0000 7f9fbcbe4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb80c7780 0x7f9fb80670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fbcbe4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 0x7f9fb81693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fbcbe4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80b12f0 con 0x7f9fb813f830 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fbcbe4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb8069f40 con 0x7f9fb80c7780 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fbcbe4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80b0ce0 con 0x7f9fb80aea30 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 0x7f9fb81693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb659c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fb80aea30 0x7f9fb8069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 0x7f9fb81693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38108/0 (socket says 172.21.15.64:38108) 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 -- 172.21.15.64:0/1085123658 learned_addr learned my addr 172.21.15.64:0/1085123658 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb5d9b700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb80c7780 0x7f9fb80670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fb80aea30 msgr2=0x7f9fb8069a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fb80aea30 0x7f9fb8069a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb80c7780 msgr2=0x7f9fb80670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb80c7780 0x7f9fb80670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 -- 172.21.15.64:0/1085123658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa8009420 con 0x7f9fb813f830 2023-09-27T23:52:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fb6d9d700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 0x7f9fb81693b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c8e0 tx=0x7f9fb0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0012630 con 0x7f9fb813f830 2023-09-27T23:52:34.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb8169e70 con 0x7f9fb813f830 2023-09-27T23:52:34.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb80b0f80 con 0x7f9fb813f830 2023-09-27T23:52:34.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb0012790 con 0x7f9fb813f830 2023-09-27T23:52:34.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.648+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0012b50 con 0x7f9fb813f830 2023-09-27T23:52:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.652+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c0052b0 con 0x7f9fb813f830 2023-09-27T23:52:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.652+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 90) v1 ==== 80378+0+0 (secure 0 0 0) 0x7f9fb001a3a0 con 0x7f9fb813f830 2023-09-27T23:52:34.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.652+0000 7f9fa67fc700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:34.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.652+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9fb007d350 con 0x7f9fb813f830 2023-09-27T23:52:34.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.652+0000 7f9fb659c700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 msgr2=0x7f9f98062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:34.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.652+0000 7f9fb659c700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:34.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.660+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f9fb0050e10 con 0x7f9fb813f830 2023-09-27T23:52:34.707 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.703+0000 7f922913b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:52:34.757 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.755+0000 7f922913b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:34.822 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:34.816+0000 7f56397b8000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:52:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.856+0000 7f9fb659c700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 msgr2=0x7f9f98062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.856+0000 7f9fb659c700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:52:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:34.856+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f9c000bc0 con 0x7f9f98060ac0 2023-09-27T23:52:34.877 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:34.872+0000 7f56397b8000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:34.908 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.907+0000 7f922913b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:52:34.965 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:34.963+0000 7f922913b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:52:35.024 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.023+0000 7f922913b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:52:35.048 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:35.044+0000 7f56397b8000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:52:35.112 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:35.108+0000 7f56397b8000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:52:35.178 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:35.172+0000 7f56397b8000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:52:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:35.260+0000 7f9fb659c700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 msgr2=0x7f9f98062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:35.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:35.260+0000 7f9fb659c700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-27T23:52:35.322 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.319+0000 7f922913b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:52:35.434 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.431+0000 7f922913b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:52:35.515 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:35.508+0000 7f56397b8000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:52:35.635 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:35.628+0000 7f56397b8000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:52:35.716 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.715+0000 7f922913b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:52:35.877 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.875+0000 7f922913b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:35.931 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:35.924+0000 7f56397b8000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:52:35.944 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.943+0000 7f922913b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:52:35.997 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:35.995+0000 7f922913b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:52:36.053 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:36.051+0000 7f922913b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:52:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:36.064+0000 7f9fb659c700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 msgr2=0x7f9f98062f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:36.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:36.064+0000 7f9fb659c700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-27T23:52:36.102 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.096+0000 7f56397b8000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:36.169 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.164+0000 7f56397b8000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:52:36.224 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.220+0000 7f56397b8000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:52:36.226 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:36.223+0000 7f922913b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:36.281 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.276+0000 7f56397b8000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:52:36.281 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:36.279+0000 7f922913b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:52:36.348 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:36.343+0000 7f922913b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:52:36.459 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.452+0000 7f56397b8000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:36.498 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:36.495+0000 7f922913b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:52:36.516 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.512+0000 7f56397b8000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:52:36.559 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:36.555+0000 7f922913b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:52:36.582 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.576+0000 7f56397b8000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:52:36.737 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.732+0000 7f56397b8000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:52:36.800 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:36.796+0000 7f56397b8000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:52:37.152 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.151+0000 7f922913b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:52:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:37.184+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 91) v1 ==== 79987+0+0 (secure 0 0 0) 0x7f9fb004d750 con 0x7f9fb813f830 2023-09-27T23:52:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:37.184+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 msgr2=0x7f9f98062f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:37.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:37.184+0000 7f9fa67fc700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:37.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91e8ae4700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91e8ae4700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91e8ae4700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91e8ae4700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91e8ae4700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91ecaec700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.370 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91ecaec700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.370 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91ecaec700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.370 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91ecaec700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.370 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:37.367+0000 7f91ecaec700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:37.447 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:37.440+0000 7f56397b8000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:52:38.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.188+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 92) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f9fb007b590 con 0x7f9fb813f830 2023-09-27T23:52:38.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.188+0000 7f9fa67fc700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.188+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f9c000bc0 con 0x7f9f98060ac0 2023-09-27T23:52:38.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.192+0000 7f9fb659c700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.192+0000 7f9fb659c700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80093f0 tx=0x7f9fa8017000 comp rx=0 tx=0).ready entity=mgr.8936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:38.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.312+0000 7f9fa67fc700 1 -- 172.21.15.64:0/1085123658 <== mgr.8936 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9f9c000bc0 con 0x7f9f98060ac0 2023-09-27T23:52:38.316 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path snap_test_dir1 2023-09-27T23:52:38.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 msgr2=0x7f9f98062f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80093f0 tx=0x7f9fa8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:38.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 msgr2=0x7f9fb81693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 0x7f9fb81693b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c8e0 tx=0x7f9fb0001710 comp rx=0 tx=0).stop 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 shutdown_connections 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9f98060ac0 0x7f9f98062f70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb813f830 0x7f9fb81693b0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fb80c7780 0x7f9fb80670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 --2- 172.21.15.64:0/1085123658 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fb80aea30 0x7f9fb8069a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 >> 172.21.15.64:0/1085123658 conn(0x7f9fb80ccc20 msgr2=0x7f9fb80c3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 shutdown_connections 2023-09-27T23:52:38.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.316+0000 7f9fbcbe4700 1 -- 172.21.15.64:0/1085123658 wait complete. 2023-09-27T23:52:38.337 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-09-27T23:52:38.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-09-27T23:52:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda31613700 1 Processor -- start 2023-09-27T23:52:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda31613700 1 -- start start 2023-09-27T23:52:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda31613700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fda2c06dcf0 0x7fda2c159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda31613700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c06c850 con 0x7fda2c06aeb0 2023-09-27T23:52:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda31613700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0cb650 con 0x7fda2c158c90 2023-09-27T23:52:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda31613700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c06b470 con 0x7fda2c06dcf0 2023-09-27T23:52:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fda2c06aeb0 0x7fda2c159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37774/0 (socket says 172.21.15.64:37774) 2023-09-27T23:52:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2b7fe700 1 -- 172.21.15.64:0/1823046463 learned_addr learned my addr 172.21.15.64:0/1823046463 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda30e12700 1 --2- 172.21.15.64:0/1823046463 >> v2:172.21.15.123:3300/0 conn(0x7fda2c06dcf0 0x7fda2c159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43342628 0 0) 0x7fda2c06c850 con 0x7fda2c06aeb0 2023-09-27T23:52:38.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda14003570 con 0x7fda2c06aeb0 2023-09-27T23:52:38.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290932217 0 0) 0x7fda2c0cb650 con 0x7fda2c158c90 2023-09-27T23:52:38.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda2c06c850 con 0x7fda2c158c90 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727517988 0 0) 0x7fda14003570 con 0x7fda2c06aeb0 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 >> v2:172.21.15.123:3300/0 conn(0x7fda2c06dcf0 msgr2=0x7fda2c159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 --2- 172.21.15.64:0/1823046463 >> v2:172.21.15.123:3300/0 conn(0x7fda2c06dcf0 0x7fda2c159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 >> v1:172.21.15.123:6789/0 conn(0x7fda2c158c90 legacy=0x7fda2c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c06b750 con 0x7fda2c06aeb0 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7fda20001470 con 0x7fda2c06aeb0 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 >> v1:172.21.15.64:6789/0 conn(0x7fda2c06aeb0 legacy=0x7fda2c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14003c40 0x7fda14004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.460+0000 7fda2affd700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda14004810 0x7fda14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda14005210 0x7fda1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c06b470 con 0x7fda14004810 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1400c130 con 0x7fda14005210 2023-09-27T23:52:38.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1400c2c0 con 0x7fda14003c40 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2bfff700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14003c40 0x7fda14004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda14005210 0x7fda1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2b7fe700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda14004810 0x7fda14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 -- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14003c40 msgr2=0x7fda14004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14003c40 0x7fda14004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 -- 172.21.15.64:0/1823046463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda14004810 msgr2=0x7fda14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda14004810 0x7fda14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 -- 172.21.15.64:0/1823046463 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14003570 con 0x7fda14005210 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2bfff700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14003c40 0x7fda14004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 -- 172.21.15.64:0/1823046463 reap_dead start 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda30e12700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda14005210 0x7fda1400ba60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e210 tx=0x7fda2400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972a90547a2b5c87 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400eac0 con 0x7fda14005210 2023-09-27T23:52:38.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2400f040 con 0x7fda14005210 2023-09-27T23:52:38.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda2affd700 1 -- 172.21.15.64:0/1823046463 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400ee70 con 0x7fda14005210 2023-09-27T23:52:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 -- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda14005210 msgr2=0x7fda1400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda14005210 0x7fda1400ba60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e210 tx=0x7fda2400b040 comp rx=0 tx=0).stop 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 -- 172.21.15.64:0/1823046463 shutdown_connections 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 --2- 172.21.15.64:0/1823046463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda14005210 0x7fda1400ba60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 -- 172.21.15.64:0/1823046463 >> 172.21.15.64:0/1823046463 conn(0x7fda2c067bb0 msgr2=0x7fda2c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 -- 172.21.15.64:0/1823046463 shutdown_connections 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.464+0000 7fda31613700 1 -- 172.21.15.64:0/1823046463 wait complete. 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 Processor -- start 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 -- start start 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14005210 0x7fda2c153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 0x7fda2c1449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2c1581e0 0x7fda2c0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c144f00 con 0x7fda2c0c9be0 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0c5fc0 con 0x7fda2c1581e0 2023-09-27T23:52:38.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0c9750 con 0x7fda14005210 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 0x7fda2c1449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 0x7fda2c1449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38126/0 (socket says 172.21.15.64:38126) 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 -- 172.21.15.64:0/3736561808 learned_addr learned my addr 172.21.15.64:0/3736561808 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2bfff700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14005210 0x7fda2c153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda30e12700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2c1581e0 0x7fda2c0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 -- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14005210 msgr2=0x7fda2c153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14005210 0x7fda2c153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 -- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2c1581e0 msgr2=0x7fda2c0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2c1581e0 0x7fda2c0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 -- 172.21.15.64:0/3736561808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c0c8b40 con 0x7fda2c0c9be0 2023-09-27T23:52:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda2b7fe700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 0x7fda2c1449c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fda20001e30 tx=0x7fda20001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20011590 con 0x7fda2c0c9be0 2023-09-27T23:52:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20011710 con 0x7fda2c0c9be0 2023-09-27T23:52:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c0c8e20 con 0x7fda2c0c9be0 2023-09-27T23:52:38.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20011ad0 con 0x7fda2c0c9be0 2023-09-27T23:52:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.468+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c0c93e0 con 0x7fda2c0c9be0 2023-09-27T23:52:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.472+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c0c98c0 con 0x7fda2c0c9be0 2023-09-27T23:52:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.472+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 92) v1 ==== 80114+0+0 (secure 0 0 0) 0x7fda2001d460 con 0x7fda2c0c9be0 2023-09-27T23:52:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.472+0000 7fda13fff700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fda180607f0 0x7fda18062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:38.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.472+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fda2007ce80 con 0x7fda2c0c9be0 2023-09-27T23:52:38.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.472+0000 7fda2bfff700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fda180607f0 0x7fda18062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:38.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.472+0000 7fda2bfff700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fda180607f0 0x7fda18062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0014d0 tx=0x7fda1c007d30 comp rx=0 tx=0).ready entity=mgr.8936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:38.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.480+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fda2004d040 con 0x7fda2c0c9be0 2023-09-27T23:52:38.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.664+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fda2c04be40 con 0x7fda180607f0 2023-09-27T23:52:38.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.784+0000 7fda13fff700 1 -- 172.21.15.64:0/3736561808 <== mgr.8936 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fda2c04be40 con 0x7fda180607f0 2023-09-27T23:52:38.789 INFO:teuthology.orchestra.run.smithi064.stdout:Retention added to path snap_test_dir1 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fda180607f0 msgr2=0x7fda18062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fda180607f0 0x7fda18062ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0014d0 tx=0x7fda1c007d30 comp rx=0 tx=0).stop 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 msgr2=0x7fda2c1449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 0x7fda2c1449c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fda20001e30 tx=0x7fda20001d20 comp rx=0 tx=0).stop 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 shutdown_connections 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fda180607f0 0x7fda18062ca0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2c1581e0 0x7fda2c0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c0c9be0 0x7fda2c1449c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 --2- 172.21.15.64:0/3736561808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda14005210 0x7fda2c153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:38.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 >> 172.21.15.64:0/3736561808 conn(0x7fda2c067bb0 msgr2=0x7fda2c0cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:38.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 shutdown_connections 2023-09-27T23:52:38.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:38.788+0000 7fda31613700 1 -- 172.21.15.64:0/3736561808 wait complete. 2023-09-27T23:52:38.810 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-09-27T23:52:38.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-27T23:52:38.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2023-09-27T23:52:38.850 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2023-09-27T23:52:38.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.852 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 9497c172-4ccf-4119-8910-f606cb59c9f0 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.853 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2023-09-27T23:52:38.854 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2023-09-27T23:52:38.855 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 20 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 20 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 10 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2023-09-27T23:52:38.856 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:52:38.857 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: debug finisher = 20 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:52:38.858 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2023-09-27T23:52:38.859 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2023-09-27T23:52:38.860 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout: client snapdir = .snap 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-09-27T23:52:38.861 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-09-27T23:52:38.862 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-09-27T23:52:38.863 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-27T23:52:38.864 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-09-27T23:52:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.64,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 9497c172-4ccf-4119-8910-f606cb59c9f0 2023-09-27T23:52:38.866 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-09-27T23:52:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-09-27T23:52:38.868 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-09-27T23:52:38.869 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:52:38.870 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: debug finisher = 20 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-09-27T23:52:38.871 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-09-27T23:52:38.872 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-09-27T23:52:38.873 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout: client snapdir = .snap 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-09-27T23:52:38.874 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-09-27T23:52:38.875 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-09-27T23:52:38.876 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:52:38.876 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:52:38.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:38.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[62498]: fuse finished with error 0 and tester_r 0 2023-09-27T23:52:38.892 INFO:teuthology.orchestra.run:waiting for 900 2023-09-27T23:52:38.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:52:38.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:38.967 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:52:38.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-27T23:52:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 Processor -- start 2023-09-27T23:52:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 -- start start 2023-09-27T23:52:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b0cb0 0x7f4f7406a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f74069f50 0x7f4f740673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f740abd00 con 0x7f4f740b1080 2023-09-27T23:52:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740abb10 con 0x7f4f74069f50 2023-09-27T23:52:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f73fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740abfe0 con 0x7f4f740b0cb0 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f737fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f74069f50 0x7f4f740673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f737fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f74069f50 0x7f4f740673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38142/0 (socket says 172.21.15.64:38142) 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.080+0000 7f4f737fe700 1 -- 172.21.15.64:0/4118727712 learned_addr learned my addr 172.21.15.64:0/4118727712 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f72ffd700 1 --2- 172.21.15.64:0/4118727712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b0cb0 0x7f4f7406a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f737fe700 1 -- 172.21.15.64:0/4118727712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b0cb0 msgr2=0x7f4f7406a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f737fe700 1 --2- 172.21.15.64:0/4118727712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b0cb0 0x7f4f7406a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f737fe700 1 -- 172.21.15.64:0/4118727712 >> v1:172.21.15.64:6789/0 conn(0x7f4f740b1080 legacy=0x7f4f74069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f737fe700 1 -- 172.21.15.64:0/4118727712 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f740ac2c0 con 0x7f4f74069f50 2023-09-27T23:52:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f71ffb700 1 -- 172.21.15.64:0/4118727712 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965551367 0 0) 0x7f4f740abd00 con 0x7f4f740b1080 2023-09-27T23:52:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f737fe700 1 --2- 172.21.15.64:0/4118727712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f740673d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009f20 tx=0x7f4f6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279863cc6d1f2a87 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f71ffb700 1 -- 172.21.15.64:0/4118727712 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c00e070 con 0x7f4f74069f50 2023-09-27T23:52:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f71ffb700 1 -- 172.21.15.64:0/4118727712 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c00ba80 con 0x7f4f74069f50 2023-09-27T23:52:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f71ffb700 1 -- 172.21.15.64:0/4118727712 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0092e0 con 0x7f4f74069f50 2023-09-27T23:52:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 -- 172.21.15.64:0/4118727712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 msgr2=0x7f4f740673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 --2- 172.21.15.64:0/4118727712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f740673d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009f20 tx=0x7f4f6c009840 comp rx=0 tx=0).stop 2023-09-27T23:52:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 -- 172.21.15.64:0/4118727712 shutdown_connections 2023-09-27T23:52:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 --2- 172.21.15.64:0/4118727712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f740673d0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 --2- 172.21.15.64:0/4118727712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b0cb0 0x7f4f7406a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 -- 172.21.15.64:0/4118727712 >> 172.21.15.64:0/4118727712 conn(0x7f4f740cc530 msgr2=0x7f4f740c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 -- 172.21.15.64:0/4118727712 shutdown_connections 2023-09-27T23:52:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 -- 172.21.15.64:0/4118727712 wait complete. 2023-09-27T23:52:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 Processor -- start 2023-09-27T23:52:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.084+0000 7f4f73fff700 1 -- start start 2023-09-27T23:52:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f74166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f740b0cb0 0x7f4f74159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b1080 0x7f4f740bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:39.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7415b010 con 0x7f4f74069f50 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740a9cf0 con 0x7f4f740b0cb0 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740c0a20 con 0x7f4f740b1080 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f74166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f74166830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38154/0 (socket says 172.21.15.64:38154) 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 -- 172.21.15.64:0/3249242921 learned_addr learned my addr 172.21.15.64:0/3249242921 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f727fc700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f740b0cb0 0x7f4f74159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f737fe700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b1080 0x7f4f740bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 -- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b1080 msgr2=0x7f4f740bfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:39.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b1080 0x7f4f740bfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 -- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f740b0cb0 msgr2=0x7f4f74159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f740b0cb0 0x7f4f74159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 -- 172.21.15.64:0/3249242921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c009ae0 con 0x7f4f74069f50 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f72ffd700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f74166830 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d8c0 tx=0x7f4f68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680125f0 con 0x7f4f74069f50 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f68012750 con 0x7f4f74069f50 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f740c0d00 con 0x7f4f74069f50 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68012af0 con 0x7f4f74069f50 2023-09-27T23:52:39.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.088+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7415aca0 con 0x7f4f74069f50 2023-09-27T23:52:39.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.092+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7404be40 con 0x7f4f74069f50 2023-09-27T23:52:39.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.092+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 92) v1 ==== 80114+0+0 (secure 0 0 0) 0x7f4f6801e460 con 0x7f4f74069f50 2023-09-27T23:52:39.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.092+0000 7f4f5affd700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4f600607f0 0x7f4f60062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:39.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.092+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f4f6807d480 con 0x7f4f74069f50 2023-09-27T23:52:39.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.092+0000 7f4f727fc700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4f600607f0 0x7f4f60062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:39.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.096+0000 7f4f727fc700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4f600607f0 0x7f4f60062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c10f0 tx=0x7f4f64007ec0 comp rx=0 tx=0).ready entity=mgr.8936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:39.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.104+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4f6804d640 con 0x7f4f74069f50 2023-09-27T23:52:39.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.296+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f7406ac70 con 0x7f4f74069f50 2023-09-27T23:52:39.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.296+0000 7f4f5affd700 1 -- 172.21.15.64:0/3249242921 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f68050c60 con 0x7f4f74069f50 2023-09-27T23:52:39.302 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-27T23:52:39.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4f600607f0 msgr2=0x7f4f60062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:39.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4f600607f0 0x7f4f60062ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c10f0 tx=0x7f4f64007ec0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 msgr2=0x7f4f74166830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f74166830 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d8c0 tx=0x7f4f68008040 comp rx=0 tx=0).stop 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 shutdown_connections 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4f600607f0 0x7f4f60062ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f740b1080 0x7f4f740bfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f740b0cb0 0x7f4f74159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 --2- 172.21.15.64:0/3249242921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f74069f50 0x7f4f74166830 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 >> 172.21.15.64:0/3249242921 conn(0x7f4f740cc530 msgr2=0x7f4f740caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 shutdown_connections 2023-09-27T23:52:39.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:39.300+0000 7f4f73fff700 1 -- 172.21.15.64:0/3249242921 wait complete. 2023-09-27T23:52:39.317 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-27T23:52:39.317 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-27T23:52:39.318 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:52:39.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:39.318 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-27T23:52:39.329 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-27T23:52:39.329 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-09-27T23:52:39.330 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 40225sec preferred_lft 40225sec 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:83:64:c7:58 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-27T23:52:39.331 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::9cde:cdff:fee1:8279/64 scope link 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether e6:3d:a9:98:0f:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::e43d:a9ff:fe98:fe8/64 scope link 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 3a:07:48:5d:0d:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:52:39.332 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::3807:48ff:fe5d:d19/64 scope link 2023-09-27T23:52:39.333 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-27T23:52:39.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:39.333 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-27T23:52:39.388 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-27T23:52:39.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:39.390 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-27T23:52:39.398 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-27T23:52:39.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:39.399 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-27T23:52:39.400 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-27T23:52:39.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-27T23:52:39.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-27T23:52:39.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-27T23:52:39.400 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-27T23:52:39.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:39.442 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:52:39.498 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:52:39.499 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:52:39.499 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:52:39.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:39.499 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-27T23:52:39.515 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-27T23:52:39.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:52:39.515 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:39.569 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-27T23:52:39.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-27T23:52:39.569 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:39.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:52:39.637 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:52:39.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:52:39.653 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:52:39.654 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:52:39.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:39.655 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:52:39.662 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-27T23:52:39.662 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-27T23:52:39.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:52:39.722 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:52:39.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:52:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:52:39.733 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:52:39.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:39.734 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:52:39.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-09-27T23:52:39.788+0000 7f32b4ddc380 -1 init, newargv = 0x55ce44a29be0 newargc=15 2023-09-27T23:52:39.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[62839]: starting ceph client 2023-09-27T23:52:39.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[62839]: starting fuse 2023-09-27T23:52:40.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-09-27T23:52:40.899 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-27T23:52:40.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-27T23:52:40.913 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-27T23:52:40.914 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-27T23:52:40.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:40.915 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-09-27T23:52:40.961 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-09-27T23:52:40.962 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-09-27T23:52:40.962 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-27T23:52:40.963 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-09-27T23:52:40.964 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-09-27T23:52:40.964 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-09-27T23:52:40.964 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-27T23:52:40.964 DEBUG:teuthology.orchestra.run.smithi064:> 2023-09-27T23:52:40.964 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-09-27T23:52:40.964 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-09-27T23:52:41.048 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.62839.asok 2023-09-27T23:52:41.233 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62839.asok 2023-09-27T23:52:41.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:41.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62839.asok status 2023-09-27T23:52:41.325 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-09-27T23:52:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "471b6dfd2d990acf7f03e5d073b84a1b39d1ad28", 2023-09-27T23:52:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1161-g471b6dfd (471b6dfd2d990acf7f03e5d073b84a1b39d1ad28) quincy (stable)", 2023-09-27T23:52:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-09-27T23:52:41.325 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "62839", 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 10605, 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 10605 2023-09-27T23:52:41.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3769483182 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3769483182 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.10605 192.168.0.1:0/3769483182", 2023-09-27T23:52:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3769483182", 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 72, 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 190, 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-09-27T23:52:41.328 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:41.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:41.335 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-09-27T23:52:41.335 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:41.342 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-09-27T23:52:41.342 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:41.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:41.343 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:41.399 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-09-27-23_53_00 in ~22s... 2023-09-27T23:52:41.399 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-09-27T23:52:41.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-09-27T23:52:41.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.512+0000 7f7579100700 1 Processor -- start 2023-09-27T23:52:41.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.512+0000 7f7579100700 1 -- start start 2023-09-27T23:52:41.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7574159030 0x7f75740cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7574159400 0x7f7574068230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574068770 0x7f75740ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7574067da0 con 0x7f7574159030 2023-09-27T23:52:41.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7574067190 con 0x7f7574159400 2023-09-27T23:52:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7574067470 con 0x7f7574068770 2023-09-27T23:52:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7574159030 0x7f75740cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7574159030 0x7f75740cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38160/0 (socket says 172.21.15.64:38160) 2023-09-27T23:52:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 -- 172.21.15.64:0/1050979833 learned_addr learned my addr 172.21.15.64:0/1050979833 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f757359e700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574068770 0x7f75740ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f757259c700 1 --2- 172.21.15.64:0/1050979833 >> v2:172.21.15.123:3300/0 conn(0x7f7574159400 0x7f7574068230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 -- 172.21.15.64:0/1050979833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574068770 msgr2=0x7f75740ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574068770 0x7f75740ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 -- 172.21.15.64:0/1050979833 >> v2:172.21.15.123:3300/0 conn(0x7f7574159400 msgr2=0x7f7574068230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 --2- 172.21.15.64:0/1050979833 >> v2:172.21.15.123:3300/0 conn(0x7f7574159400 0x7f7574068230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 -- 172.21.15.64:0/1050979833 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574067750 con 0x7f7574159030 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f757359e700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574068770 0x7f75740ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f757259c700 1 --2- 172.21.15.64:0/1050979833 >> v2:172.21.15.123:3300/0 conn(0x7f7574159400 0x7f7574068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:41.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7572d9d700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159030 0x7f75740cb170 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f757406abe0 tx=0x7f7568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6197142004f9dd99 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7571d9b700 1 -- 172.21.15.64:0/1050979833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800b640 con 0x7f7574159030 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7571d9b700 1 -- 172.21.15.64:0/1050979833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800b7a0 con 0x7f7574159030 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7571d9b700 1 -- 172.21.15.64:0/1050979833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800bb60 con 0x7f7574159030 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- 172.21.15.64:0/1050979833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159030 msgr2=0x7f75740cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159030 0x7f75740cb170 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f757406abe0 tx=0x7f7568009250 comp rx=0 tx=0).stop 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- 172.21.15.64:0/1050979833 shutdown_connections 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574068770 0x7f75740ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- 172.21.15.64:0/1050979833 >> v2:172.21.15.123:3300/0 conn(0x7f7574159400 0x7f7574068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 --2- 172.21.15.64:0/1050979833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159030 0x7f75740cb170 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- 172.21.15.64:0/1050979833 >> 172.21.15.64:0/1050979833 conn(0x7f75740d0170 msgr2=0x7f75740d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- 172.21.15.64:0/1050979833 shutdown_connections 2023-09-27T23:52:41.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.516+0000 7f7579100700 1 -- 172.21.15.64:0/1050979833 wait complete. 2023-09-27T23:52:41.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 Processor -- start 2023-09-27T23:52:41.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 -- start start 2023-09-27T23:52:41.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7574068770 0x7f757415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574159030 0x7f757415f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 0x7f757414dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415fcb0 con 0x7f7574159400 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415f750 con 0x7f7574068770 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7579100700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757415f9d0 con 0x7f7574159030 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 0x7f757414dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574159030 0x7f757415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 0x7f757414dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38164/0 (socket says 172.21.15.64:38164) 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 -- 172.21.15.64:0/3390537153 learned_addr learned my addr 172.21.15.64:0/3390537153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:41.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f7572d9d700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7574068770 0x7f757415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 -- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574159030 msgr2=0x7f757415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574159030 0x7f757415f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 -- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7574068770 msgr2=0x7f757415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7574068770 0x7f757415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.520+0000 7f757359e700 1 -- 172.21.15.64:0/3390537153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009a20 con 0x7f7574159400 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f757359e700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 0x7f757414dcc0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f756c00ef90 tx=0x7f756c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00cbd0 con 0x7f7574159400 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00f040 con 0x7f7574159400 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7564009420 con 0x7f7574159400 2023-09-27T23:52:41.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c014480 con 0x7f7574159400 2023-09-27T23:52:41.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75680094f0 con 0x7f7574159400 2023-09-27T23:52:41.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75740708a0 con 0x7f7574159400 2023-09-27T23:52:41.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 93) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f756c0148b0 con 0x7f7574159400 2023-09-27T23:52:41.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.524+0000 7f755affd700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7560060b50 0x7f7560063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.528+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f756c07e390 con 0x7f7574159400 2023-09-27T23:52:41.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.528+0000 7f7572d9d700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7560060b50 0x7f7560063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.528+0000 7f7572d9d700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7560060b50 0x7f7560063000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f757406abe0 tx=0x7f7568017000 comp rx=0 tx=0).ready entity=mgr.8936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:41.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.536+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f756c04dfe0 con 0x7f7574159400 2023-09-27T23:52:41.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.720+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7574067a30 con 0x7f7560060b50 2023-09-27T23:52:41.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f755affd700 1 -- 172.21.15.64:0/3390537153 <== mgr.8936 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f7574067a30 con 0x7f7560060b50 2023-09-27T23:52:41.740 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:41.740 INFO:teuthology.orchestra.run.smithi064.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-09-27T23:52:41.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7560060b50 msgr2=0x7f7560063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7560060b50 0x7f7560063000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f757406abe0 tx=0x7f7568017000 comp rx=0 tx=0).stop 2023-09-27T23:52:41.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 msgr2=0x7f757414dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 0x7f757414dcc0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f756c00ef90 tx=0x7f756c00b040 comp rx=0 tx=0).stop 2023-09-27T23:52:41.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 shutdown_connections 2023-09-27T23:52:41.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7560060b50 0x7f7560063000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7574159400 0x7f757414dcc0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7574159030 0x7f757415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 --2- 172.21.15.64:0/3390537153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7574068770 0x7f757415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 >> 172.21.15.64:0/3390537153 conn(0x7f75740d0170 msgr2=0x7f75740b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:41.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 shutdown_connections 2023-09-27T23:52:41.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.736+0000 7f7579100700 1 -- 172.21.15.64:0/3390537153 wait complete. 2023-09-27T23:52:41.756 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-09-27T23:52:41.756 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-09-27T23:52:41.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-09-27T23:52:41.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 Processor -- start 2023-09-27T23:52:41.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 -- start start 2023-09-27T23:52:41.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e40ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4e406a720 0x7fa4e406abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e40cbfd0 con 0x7fa4e4156ba0 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e406b100 con 0x7fa4e406a720 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4eafec700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40cb1a0 con 0x7fa4e4068170 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e9fea700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa4e4156ba0 0x7fa4e40caa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37790/0 (socket says 172.21.15.64:37790) 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e9fea700 1 -- 172.21.15.64:0/1331409299 learned_addr learned my addr 172.21.15.64:0/1331409299 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e97e9700 1 --2- 172.21.15.64:0/1331409299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e40ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4ea7eb700 1 --2- 172.21.15.64:0/1331409299 >> v2:172.21.15.123:3300/0 conn(0x7fa4e406a720 0x7fa4e406abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e8fe8700 1 -- 172.21.15.64:0/1331409299 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796840262 0 0) 0x7fa4e40cbfd0 con 0x7fa4e4156ba0 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e8fe8700 1 -- 172.21.15.64:0/1331409299 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d0003570 con 0x7fa4e4156ba0 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e97e9700 1 -- 172.21.15.64:0/1331409299 >> v2:172.21.15.123:3300/0 conn(0x7fa4e406a720 msgr2=0x7fa4e406abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:41.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e97e9700 1 --2- 172.21.15.64:0/1331409299 >> v2:172.21.15.123:3300/0 conn(0x7fa4e406a720 0x7fa4e406abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e97e9700 1 -- 172.21.15.64:0/1331409299 >> v1:172.21.15.64:6789/0 conn(0x7fa4e4156ba0 legacy=0x7fa4e40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.872+0000 7fa4e97e9700 1 -- 172.21.15.64:0/1331409299 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40cb3c0 con 0x7fa4e4068170 2023-09-27T23:52:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 --2- 172.21.15.64:0/1331409299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e40ae9e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098e0 tx=0x7fa4d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3a11704bb6ff2b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e8fe8700 1 -- 172.21.15.64:0/1331409299 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4011630 con 0x7fa4e4068170 2023-09-27T23:52:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e8fe8700 1 -- 172.21.15.64:0/1331409299 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4011790 con 0x7fa4e4068170 2023-09-27T23:52:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e8fe8700 1 -- 172.21.15.64:0/1331409299 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4011b30 con 0x7fa4e4068170 2023-09-27T23:52:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- 172.21.15.64:0/1331409299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 msgr2=0x7fa4e40ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 --2- 172.21.15.64:0/1331409299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e40ae9e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098e0 tx=0x7fa4d40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- 172.21.15.64:0/1331409299 shutdown_connections 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 --2- 172.21.15.64:0/1331409299 >> v2:172.21.15.123:3300/0 conn(0x7fa4e406a720 0x7fa4e406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 --2- 172.21.15.64:0/1331409299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e40ae9e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- 172.21.15.64:0/1331409299 >> 172.21.15.64:0/1331409299 conn(0x7fa4e40ccdb0 msgr2=0x7fa4e406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- 172.21.15.64:0/1331409299 shutdown_connections 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- 172.21.15.64:0/1331409299 wait complete. 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 Processor -- start 2023-09-27T23:52:41.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- start start 2023-09-27T23:52:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e4152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 0x7fa4e415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e4156ba0 0x7fa4e415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40c52a0 con 0x7fa4e406a720 2023-09-27T23:52:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40c54b0 con 0x7fa4e4156ba0 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4eafec700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40c5790 con 0x7fa4e4068170 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 0x7fa4e415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 0x7fa4e415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38166/0 (socket says 172.21.15.64:38166) 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 -- 172.21.15.64:0/3792206641 learned_addr learned my addr 172.21.15.64:0/3792206641 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e9fea700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e4152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4ea7eb700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e4156ba0 0x7fa4e415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 -- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 msgr2=0x7fa4e4152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e4152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 -- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e4156ba0 msgr2=0x7fa4e415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.876+0000 7fa4e97e9700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e4156ba0 0x7fa4e415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4e97e9700 1 -- 172.21.15.64:0/3792206641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4009420 con 0x7fa4e406a720 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4e9fea700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e4152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4e97e9700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 0x7fa4e415c950 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009ed0 tx=0x7fa4d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4011900 con 0x7fa4e406a720 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4011a60 con 0x7fa4e406a720 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e40c5a70 con 0x7fa4e406a720 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4025640 con 0x7fa4e406a720 2023-09-27T23:52:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e40c6270 con 0x7fa4e406a720 2023-09-27T23:52:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e40c5f70 con 0x7fa4e406a720 2023-09-27T23:52:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 93) v1 ==== 80435+0+0 (secure 0 0 0) 0x7fa4d4025a30 con 0x7fa4e406a720 2023-09-27T23:52:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4d9ffb700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa4c8060b50 0x7fa4c8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.880+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa4d407ec60 con 0x7fa4e406a720 2023-09-27T23:52:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.884+0000 7fa4e9fea700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa4c8060b50 0x7fa4c8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:41.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.884+0000 7fa4e9fea700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa4c8060b50 0x7fa4c8063000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0007fd0 tx=0x7fa4e0009040 comp rx=0 tx=0).ready entity=mgr.8936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:41.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:41.892+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa4d404e9c0 con 0x7fa4e406a720 2023-09-27T23:52:42.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.072+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4e40c6490 con 0x7fa4c8060b50 2023-09-27T23:52:42.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.220+0000 7fa4d9ffb700 1 -- 172.21.15.64:0/3792206641 <== mgr.8936 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa4e40c6490 con 0x7fa4c8060b50 2023-09-27T23:52:42.225 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule removed for path snap_test_dir1 2023-09-27T23:52:42.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa4c8060b50 msgr2=0x7fa4c8063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa4c8060b50 0x7fa4c8063000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0007fd0 tx=0x7fa4e0009040 comp rx=0 tx=0).stop 2023-09-27T23:52:42.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 msgr2=0x7fa4e415c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 0x7fa4e415c950 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009ed0 tx=0x7fa4d4017000 comp rx=0 tx=0).stop 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 shutdown_connections 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa4c8060b50 0x7fa4c8063000 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e4156ba0 0x7fa4e415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4e406a720 0x7fa4e415c950 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 --2- 172.21.15.64:0/3792206641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4e4068170 0x7fa4e4152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 >> 172.21.15.64:0/3792206641 conn(0x7fa4e40ccdb0 msgr2=0x7fa4e4157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 shutdown_connections 2023-09-27T23:52:42.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.224+0000 7fa4eafec700 1 -- 172.21.15.64:0/3792206641 wait complete. 2023-09-27T23:52:42.245 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-09-27T23:52:42.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:42.246 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-09-27T23:52:42.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-09-27T23:52:42.265 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-09-27T23:52:42.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-09-27T23:52:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 Processor -- start 2023-09-27T23:52:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 -- start start 2023-09-27T23:52:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f04500c64a0 0x7f04500c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04500c70c0 0x7f04500aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04500c6db0 con 0x7f04500abfc0 2023-09-27T23:52:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0450067b10 con 0x7f04500c70c0 2023-09-27T23:52:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f04557f1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0450067df0 con 0x7f04500c64a0 2023-09-27T23:52:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04500c70c0 0x7f04500aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f04500c70c0 0x7f04500aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38178/0 (socket says 172.21.15.64:38178) 2023-09-27T23:52:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 -- 172.21.15.64:0/2610601448 learned_addr learned my addr 172.21.15.64:0/2610601448 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:42.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044ffff700 1 --2- 172.21.15.64:0/2610601448 >> v2:172.21.15.123:3300/0 conn(0x7f04500c64a0 0x7f04500c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 -- 172.21.15.64:0/2610601448 >> v2:172.21.15.123:3300/0 conn(0x7f04500c64a0 msgr2=0x7f04500c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 --2- 172.21.15.64:0/2610601448 >> v2:172.21.15.123:3300/0 conn(0x7f04500c64a0 0x7f04500c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 -- 172.21.15.64:0/2610601448 >> v1:172.21.15.123:6789/0 conn(0x7f04500abfc0 legacy=0x7f04500ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 -- 172.21.15.64:0/2610601448 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500680d0 con 0x7f04500c70c0 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044ffff700 1 --2- 172.21.15.64:0/2610601448 >> v2:172.21.15.123:3300/0 conn(0x7f04500c64a0 0x7f04500c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044f7fe700 1 --2- 172.21.15.64:0/2610601448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c70c0 0x7f04500aba80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f04500cb980 tx=0x7f0444009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fba40569ac9351 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044effd700 1 -- 172.21.15.64:0/2610601448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044400b860 con 0x7f04500c70c0 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044effd700 1 -- 172.21.15.64:0/2610601448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044400b9c0 con 0x7f04500c70c0 2023-09-27T23:52:42.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.440+0000 7f044effd700 1 -- 172.21.15.64:0/2610601448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044400bd60 con 0x7f04500c70c0 2023-09-27T23:52:42.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- 172.21.15.64:0/2610601448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c70c0 msgr2=0x7f04500aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 --2- 172.21.15.64:0/2610601448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c70c0 0x7f04500aba80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f04500cb980 tx=0x7f0444009250 comp rx=0 tx=0).stop 2023-09-27T23:52:42.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- 172.21.15.64:0/2610601448 shutdown_connections 2023-09-27T23:52:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 --2- 172.21.15.64:0/2610601448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c70c0 0x7f04500aba80 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 --2- 172.21.15.64:0/2610601448 >> v2:172.21.15.123:3300/0 conn(0x7f04500c64a0 0x7f04500c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- 172.21.15.64:0/2610601448 >> 172.21.15.64:0/2610601448 conn(0x7f04500cc530 msgr2=0x7f04500c8490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- 172.21.15.64:0/2610601448 shutdown_connections 2023-09-27T23:52:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- 172.21.15.64:0/2610601448 wait complete. 2023-09-27T23:52:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 Processor -- start 2023-09-27T23:52:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- start start 2023-09-27T23:52:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04500abfc0 0x7f045015f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 0x7f0450161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04500c70c0 0x7f0450150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045016d9b0 con 0x7f04500c64a0 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045016dc90 con 0x7f04500c70c0 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f04557f1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045016df70 con 0x7f04500abfc0 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f044f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 0x7f0450161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f044f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 0x7f0450161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38186/0 (socket says 172.21.15.64:38186) 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f044f7fe700 1 -- 172.21.15.64:0/4186783577 learned_addr learned my addr 172.21.15.64:0/4186783577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f0454ff0700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04500c70c0 0x7f0450150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.444+0000 7f044ffff700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04500abfc0 0x7f045015f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f044f7fe700 1 -- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04500abfc0 msgr2=0x7f045015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f044f7fe700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04500abfc0 0x7f045015f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f044f7fe700 1 -- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04500c70c0 msgr2=0x7f0450150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f044f7fe700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04500c70c0 0x7f0450150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f044f7fe700 1 -- 172.21.15.64:0/4186783577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04440094f0 con 0x7f04500c64a0 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f0454ff0700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04500c70c0 0x7f0450150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f044f7fe700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 0x7f0450161a10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0444009290 tx=0x7f0444017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444026480 con 0x7f04500c64a0 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045016e250 con 0x7f04500c64a0 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044401f070 con 0x7f04500c64a0 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444026850 con 0x7f04500c64a0 2023-09-27T23:52:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045015d3e0 con 0x7f04500c64a0 2023-09-27T23:52:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045016e430 con 0x7f04500c64a0 2023-09-27T23:52:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.448+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 93) v1 ==== 80435+0+0 (secure 0 0 0) 0x7f044401a430 con 0x7f04500c64a0 2023-09-27T23:52:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.452+0000 7f0437fff700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f043c060b50 0x7f043c063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.452+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f0444021070 con 0x7f04500c64a0 2023-09-27T23:52:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.452+0000 7f044ffff700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f043c060b50 0x7f043c063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.452+0000 7f044ffff700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f043c060b50 0x7f043c063000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0440006f00 tx=0x7f0440008040 comp rx=0 tx=0).ready entity=mgr.8936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:42.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.456+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f044404f2a0 con 0x7f04500c64a0 2023-09-27T23:52:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.652+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f045016daf0 con 0x7f04500c64a0 2023-09-27T23:52:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.768+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f044404f670 con 0x7f04500c64a0 2023-09-27T23:52:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.772+0000 7f0437fff700 1 -- 172.21.15.64:0/4186783577 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v94) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0444052d30 con 0x7f04500c64a0 2023-09-27T23:52:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f043c060b50 msgr2=0x7f043c063000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f043c060b50 0x7f043c063000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0440006f00 tx=0x7f0440008040 comp rx=0 tx=0).stop 2023-09-27T23:52:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 msgr2=0x7f0450161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 0x7f0450161a10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0444009290 tx=0x7f0444017000 comp rx=0 tx=0).stop 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 shutdown_connections 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f043c060b50 0x7f043c063000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04500c70c0 0x7f0450150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04500c64a0 0x7f0450161a10 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 --2- 172.21.15.64:0/4186783577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04500abfc0 0x7f045015f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 >> 172.21.15.64:0/4186783577 conn(0x7f04500cc530 msgr2=0x7f04500c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 shutdown_connections 2023-09-27T23:52:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:42.776+0000 7f04557f1700 1 -- 172.21.15.64:0/4186783577 wait complete. 2023-09-27T23:52:42.796 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-27T23:52:42.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:52:42.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-27T23:52:42.826 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-27T23:52:42.826 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-09-27T23:52:42.826 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-09-27T23:52:42.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-27T23:52:42.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[62839]: fuse finished with error 0 and tester_r 0 2023-09-27T23:52:42.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:52:42.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:42.840 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-27T23:52:42.903 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:42.898+0000 7fce3a603000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:52:42.907 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:42.900+0000 7ff448597000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-27T23:52:42.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:52:42.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:42.992 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-27T23:52:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-27T23:52:42.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:52:42.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:52:42.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:43.000 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-27T23:52:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-27T23:52:43.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:52:43.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:52:43.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:43.009 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-27T23:52:43.015 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-27T23:52:43.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:52:43.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.116+0000 7fa16759e700 1 Processor -- start 2023-09-27T23:52:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.116+0000 7fa16759e700 1 -- start start 2023-09-27T23:52:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.116+0000 7fa16759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa16806b510 0x7fa1681553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.116+0000 7fa16759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa168067d90 con 0x7fa168068220 2023-09-27T23:52:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16806c070 con 0x7fa168155010 2023-09-27T23:52:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16806bab0 con 0x7fa16806b510 2023-09-27T23:52:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa166d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa168068220 0x7fa1680b0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37800/0 (socket says 172.21.15.64:37800) 2023-09-27T23:52:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa166d9d700 1 -- 172.21.15.64:0/525978286 learned_addr learned my addr 172.21.15.64:0/525978286 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa165d9b700 1 --2- 172.21.15.64:0/525978286 >> v2:172.21.15.123:3300/0 conn(0x7fa16806b510 0x7fa1681553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500646039 0 0) 0x7fa168067d90 con 0x7fa168068220 2023-09-27T23:52:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa150003570 con 0x7fa168068220 2023-09-27T23:52:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580796328 0 0) 0x7fa16806c070 con 0x7fa168155010 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa168067d90 con 0x7fa168155010 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980575294 0 0) 0x7fa150003570 con 0x7fa168068220 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa165d9b700 1 -- 172.21.15.64:0/525978286 >> v1:172.21.15.123:6789/0 conn(0x7fa168155010 legacy=0x7fa1680afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa165d9b700 1 -- 172.21.15.64:0/525978286 >> v1:172.21.15.64:6789/0 conn(0x7fa168068220 legacy=0x7fa1680b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa165d9b700 1 -- 172.21.15.64:0/525978286 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16806bd90 con 0x7fa16806b510 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa165d9b700 1 --2- 172.21.15.64:0/525978286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa1681553e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0098e0 tx=0x7fa15c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489f887fa0f80472 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600017f0 con 0x7fa16806b510 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c011420 con 0x7fa16806b510 2023-09-27T23:52:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16559a700 1 -- 172.21.15.64:0/525978286 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0117a0 con 0x7fa16806b510 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- 172.21.15.64:0/525978286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 msgr2=0x7fa1681553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 --2- 172.21.15.64:0/525978286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa1681553e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0098e0 tx=0x7fa15c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- 172.21.15.64:0/525978286 shutdown_connections 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 --2- 172.21.15.64:0/525978286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa1681553e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- 172.21.15.64:0/525978286 >> 172.21.15.64:0/525978286 conn(0x7fa16806e160 msgr2=0x7fa1680adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- 172.21.15.64:0/525978286 shutdown_connections 2023-09-27T23:52:43.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.120+0000 7fa16759e700 1 -- 172.21.15.64:0/525978286 wait complete. 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 Processor -- start 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 -- start start 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 0x7fa16814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa168169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa168155010 0x7fa1680acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680a8fb0 con 0x7fa168068220 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680a91c0 con 0x7fa16806b510 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680a94a0 con 0x7fa168155010 2023-09-27T23:52:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 0x7fa16814ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 0x7fa16814ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38220/0 (socket says 172.21.15.64:38220) 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 -- 172.21.15.64:0/3087549228 learned_addr learned my addr 172.21.15.64:0/3087549228 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa165d9b700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa168169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa166d9d700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa168155010 0x7fa1680acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 -- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa168155010 msgr2=0x7fa1680acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa168155010 0x7fa1680acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 -- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 msgr2=0x7fa168169fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa168169fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 -- 172.21.15.64:0/3087549228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009420 con 0x7fa168068220 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16659c700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 0x7fa16814ac30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c530 tx=0x7fa158001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158011590 con 0x7fa168068220 2023-09-27T23:52:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1580116f0 con 0x7fa168068220 2023-09-27T23:52:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1680a9780 con 0x7fa168068220 2023-09-27T23:52:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158011a90 con 0x7fa168068220 2023-09-27T23:52:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.124+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1680a9d40 con 0x7fa168068220 2023-09-27T23:52:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.128+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16804be40 con 0x7fa168068220 2023-09-27T23:52:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.128+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fa15801b3b0 con 0x7fa168068220 2023-09-27T23:52:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.128+0000 7fa1567fc700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 0x7fa148062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.128+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa15807c750 con 0x7fa168068220 2023-09-27T23:52:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.128+0000 7fa165d9b700 1 -- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 msgr2=0x7fa148062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.128+0000 7fa165d9b700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 0x7fa148062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:43.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.132+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa15804c560 con 0x7fa168068220 2023-09-27T23:52:43.322 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.318+0000 7fce3a603000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:52:43.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.332+0000 7fa165d9b700 1 -- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 msgr2=0x7fa148062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:43.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.332+0000 7fa165d9b700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 0x7fa148062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:52:43.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.332+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa168067a20 con 0x7fa168068220 2023-09-27T23:52:43.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.332+0000 7fa1567fc700 1 -- 172.21.15.64:0/3087549228 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4610 (secure 0 0 0) 0x7fa158017090 con 0x7fa168068220 2023-09-27T23:52:43.339 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:43.340 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:52:20.268986+0000","modified":"2023-09-27T23:52:22.285887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10144":{"gid":10144,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/1929792685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1929792685},{"type":"v1","addr":"172.21.15.123:6835","nonce":1929792685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-09-27T23:52:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 msgr2=0x7fa148062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 0x7fa148062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 msgr2=0x7fa16814ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 0x7fa16814ac30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c530 tx=0x7fa158001030 comp rx=0 tx=0).stop 2023-09-27T23:52:43.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 shutdown_connections 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fa148060b00 0x7fa148062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa168155010 0x7fa1680acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa16806b510 0x7fa168169fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 --2- 172.21.15.64:0/3087549228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa168068220 0x7fa16814ac30 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 >> 172.21.15.64:0/3087549228 conn(0x7fa16806e160 msgr2=0x7fa1680c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 shutdown_connections 2023-09-27T23:52:43.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.336+0000 7fa16759e700 1 -- 172.21.15.64:0/3087549228 wait complete. 2023-09-27T23:52:43.345 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 72 2023-09-27T23:52:43.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:43.362 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:43.356+0000 7ff448597000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-27T23:52:43.370 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.366+0000 7fce3a603000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:52:43.416 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:43.412+0000 7ff448597000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-27T23:52:43.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 Processor -- start 2023-09-27T23:52:43.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 -- start start 2023-09-27T23:52:43.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4c8157e10 0x7ff4c80cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4c80cc930 0x7ff4c80b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c80b27b0 con 0x7ff4c80b2350 2023-09-27T23:52:43.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c80cb3b0 con 0x7ff4c8157e10 2023-09-27T23:52:43.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4cea46700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c80cb630 con 0x7ff4c80cc930 2023-09-27T23:52:43.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4c7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4c8157e10 0x7ff4c80cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4c7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4c8157e10 0x7ff4c80cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38224/0 (socket says 172.21.15.64:38224) 2023-09-27T23:52:43.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4c7fff700 1 -- 172.21.15.64:0/1318382249 learned_addr learned my addr 172.21.15.64:0/1318382249 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:43.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4c77fe700 1 --2- 172.21.15.64:0/1318382249 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80cc930 0x7ff4c80b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4c7fff700 1 -- 172.21.15.64:0/1318382249 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80cc930 msgr2=0x7ff4c80b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:43.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c7fff700 1 --2- 172.21.15.64:0/1318382249 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80cc930 0x7ff4c80b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.448+0000 7ff4c6ffd700 1 -- 172.21.15.64:0/1318382249 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479553933 0 0) 0x7ff4c0001490 con 0x7ff4c80b2350 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c7fff700 1 -- 172.21.15.64:0/1318382249 >> v1:172.21.15.64:6789/0 conn(0x7ff4c80b2350 legacy=0x7ff4c815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c7fff700 1 -- 172.21.15.64:0/1318382249 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80cb910 con 0x7ff4c8157e10 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c7fff700 1 --2- 172.21.15.64:0/1318382249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8157e10 0x7ff4c80cc3f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff4c806e5d0 tx=0x7ff4b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c3b5c5da9746a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c6ffd700 1 -- 172.21.15.64:0/1318382249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8009e70 con 0x7ff4c8157e10 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c6ffd700 1 -- 172.21.15.64:0/1318382249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b800b860 con 0x7ff4c8157e10 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4c6ffd700 1 -- 172.21.15.64:0/1318382249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800bc00 con 0x7ff4c8157e10 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- 172.21.15.64:0/1318382249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8157e10 msgr2=0x7ff4c80cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 --2- 172.21.15.64:0/1318382249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8157e10 0x7ff4c80cc3f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff4c806e5d0 tx=0x7ff4b8009250 comp rx=0 tx=0).stop 2023-09-27T23:52:43.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- 172.21.15.64:0/1318382249 shutdown_connections 2023-09-27T23:52:43.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 --2- 172.21.15.64:0/1318382249 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80cc930 0x7ff4c80b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 --2- 172.21.15.64:0/1318382249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c8157e10 0x7ff4c80cc3f0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- 172.21.15.64:0/1318382249 >> 172.21.15.64:0/1318382249 conn(0x7ff4c80ccd40 msgr2=0x7ff4c80cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:43.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- 172.21.15.64:0/1318382249 shutdown_connections 2023-09-27T23:52:43.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- 172.21.15.64:0/1318382249 wait complete. 2023-09-27T23:52:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 Processor -- start 2023-09-27T23:52:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- start start 2023-09-27T23:52:43.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 0x7ff4c8152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80cc930 0x7ff4c8151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c814acf0 0x7ff4c806c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.452+0000 7ff4cea46700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8154ab0 con 0x7ff4c80b2350 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4cea46700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8156f70 con 0x7ff4c80cc930 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4cea46700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8152a20 con 0x7ff4c814acf0 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 0x7ff4c8152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 0x7ff4c8152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38234/0 (socket says 172.21.15.64:38234) 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 -- 172.21.15.64:0/448799044 learned_addr learned my addr 172.21.15.64:0/448799044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:43.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4ccfe3700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c814acf0 0x7ff4c806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c77fe700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80cc930 0x7ff4c8151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 -- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c814acf0 msgr2=0x7ff4c806c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c814acf0 0x7ff4c806c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 -- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80cc930 msgr2=0x7ff4c8151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80cc930 0x7ff4c8151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 -- 172.21.15.64:0/448799044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80094f0 con 0x7ff4c80b2350 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4c7fff700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 0x7ff4c8152440 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80094c0 tx=0x7ff4b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800b9d0 con 0x7ff4c80b2350 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b800bb30 con 0x7ff4c80b2350 2023-09-27T23:52:43.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c8154180 con 0x7ff4c80b2350 2023-09-27T23:52:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8011670 con 0x7ff4c80b2350 2023-09-27T23:52:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c8154740 con 0x7ff4c80b2350 2023-09-27T23:52:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.456+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c8071110 con 0x7ff4c80b2350 2023-09-27T23:52:43.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.460+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7ff4b8011a60 con 0x7ff4c80b2350 2023-09-27T23:52:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.460+0000 7ff4affff700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff4b4060b00 0x7ff4b4062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.460+0000 7ff4c77fe700 1 -- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff4b4060b00 msgr2=0x7ff4b4062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:43.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.460+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7ff4b807f480 con 0x7ff4c80b2350 2023-09-27T23:52:43.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.460+0000 7ff4c77fe700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff4b4060b00 0x7ff4b4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.468+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff4b804f140 con 0x7ff4c80b2350 2023-09-27T23:52:43.476 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.474+0000 7fce3a603000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:43.528 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:43.524+0000 7ff448597000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:43.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.628+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4c80cbbf0 con 0x7ff4c80b2350 2023-09-27T23:52:43.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4affff700 1 -- 172.21.15.64:0/448799044 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17584 (secure 0 0 0) 0x7ff4b8052760 con 0x7ff4c80b2350 2023-09-27T23:52:43.637 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:43.637 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":190,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:37.175276+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:52:18.850130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-09-27T23:52:19.262770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3573447607":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6832/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6833/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/2413747055":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3774540904":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3310607164":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/4028139068":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1447825948":"2023-09-28T23:52:37.175204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff4b4060b00 msgr2=0x7ff4b4062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff4b4060b00 0x7ff4b4062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 msgr2=0x7ff4c8152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 0x7ff4c8152440 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80094c0 tx=0x7ff4b8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 shutdown_connections 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff4b4060b00 0x7ff4b4062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c814acf0 0x7ff4c806c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80cc930 0x7ff4c8151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 --2- 172.21.15.64:0/448799044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4c80b2350 0x7ff4c8152440 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 >> 172.21.15.64:0/448799044 conn(0x7ff4c80ccd40 msgr2=0x7ff4c814b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 shutdown_connections 2023-09-27T23:52:43.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.632+0000 7ff4cea46700 1 -- 172.21.15.64:0/448799044 wait complete. 2023-09-27T23:52:43.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-27T23:13:33.202942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-27T23:13:38.821082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-27T23:52:18.850130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-09-27T23:52:19.262770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-27T23:52:43.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-27T23:52:43.670 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.666+0000 7fce3a603000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:52:43.718 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.714+0000 7fce3a603000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:43.735 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:43.728+0000 7ff448597000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-27T23:52:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 Processor -- start 2023-09-27T23:52:43.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 -- start start 2023-09-27T23:52:43.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46100a88a0 0x7f461006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46100a8cf0 0x7f461006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4610067420 con 0x7f4610158d70 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46100b09a0 con 0x7f46100a8cf0 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f46157af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4610067700 con 0x7f46100a88a0 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46100a8cf0 0x7f461006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46100a8cf0 0x7f461006e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38242/0 (socket says 172.21.15.64:38242) 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 -- 172.21.15.64:0/1896805681 learned_addr learned my addr 172.21.15.64:0/1896805681 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f460f7fe700 1 --2- 172.21.15.64:0/1896805681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46100a88a0 0x7f461006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 -- 172.21.15.64:0/1896805681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46100a88a0 msgr2=0x7f461006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 --2- 172.21.15.64:0/1896805681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46100a88a0 0x7f461006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 -- 172.21.15.64:0/1896805681 >> v1:172.21.15.123:6789/0 conn(0x7f4610158d70 legacy=0x7f46100b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 -- 172.21.15.64:0/1896805681 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100679e0 con 0x7f46100a8cf0 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.756+0000 7f4614fae700 1 --2- 172.21.15.64:0/1896805681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f461006e5c0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4608009f20 tx=0x7f4608009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da70702cadf5268 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460effd700 1 -- 172.21.15.64:0/1896805681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460800e070 con 0x7f46100a8cf0 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460effd700 1 -- 172.21.15.64:0/1896805681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460800ba80 con 0x7f46100a8cf0 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460effd700 1 -- 172.21.15.64:0/1896805681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080092e0 con 0x7f46100a8cf0 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- 172.21.15.64:0/1896805681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 msgr2=0x7f461006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 --2- 172.21.15.64:0/1896805681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f461006e5c0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4608009f20 tx=0x7f4608009840 comp rx=0 tx=0).stop 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- 172.21.15.64:0/1896805681 shutdown_connections 2023-09-27T23:52:43.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 --2- 172.21.15.64:0/1896805681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f461006e5c0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 --2- 172.21.15.64:0/1896805681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46100a88a0 0x7f461006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- 172.21.15.64:0/1896805681 >> 172.21.15.64:0/1896805681 conn(0x7f461006b790 msgr2=0x7f461006bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- 172.21.15.64:0/1896805681 shutdown_connections 2023-09-27T23:52:43.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- 172.21.15.64:0/1896805681 wait complete. 2023-09-27T23:52:43.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 Processor -- start 2023-09-27T23:52:43.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- start start 2023-09-27T23:52:43.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46100a88a0 0x7f4610142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f46100ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158d70 0x7f461015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461015af60 con 0x7f46100a8cf0 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610155260 con 0x7f46100a88a0 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f46157af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46100ad990 con 0x7f4610158d70 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f46100ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f46100ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38256/0 (socket says 172.21.15.64:38256) 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 -- 172.21.15.64:0/428943723 learned_addr learned my addr 172.21.15.64:0/428943723 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460ffff700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46100a88a0 0x7f4610142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f4614fae700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158d70 0x7f461015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 -- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158d70 msgr2=0x7f461015db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158d70 0x7f461015db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 -- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46100a88a0 msgr2=0x7f4610142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46100a88a0 0x7f4610142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 -- 172.21.15.64:0/428943723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608009ae0 con 0x7f46100a8cf0 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.760+0000 7f460f7fe700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f46100ac480 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f460000d890 tx=0x7f4600008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46000125f0 con 0x7f46100a8cf0 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600012750 con 0x7f46100a8cf0 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600012af0 con 0x7f46100a8cf0 2023-09-27T23:52:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100adc70 con 0x7f46100a8cf0 2023-09-27T23:52:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46100ae230 con 0x7f46100a8cf0 2023-09-27T23:52:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461006c910 con 0x7f46100a8cf0 2023-09-27T23:52:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f460001e460 con 0x7f46100a8cf0 2023-09-27T23:52:43.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f4607fff700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f45f0060b00 0x7f45f0062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:43.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f460ffff700 1 -- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f45f0060b00 msgr2=0x7f45f0062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:43.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f460007d9d0 con 0x7f46100a8cf0 2023-09-27T23:52:43.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.764+0000 7f460ffff700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f45f0060b00 0x7f45f0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:43.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.768+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f461006c910 con 0x7f46100a8cf0 2023-09-27T23:52:43.788 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:43.784+0000 7ff448597000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:43.871 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.866+0000 7fce3a603000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:52:43.928 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.926+0000 7fce3a603000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:52:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.932+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f461006c910 con 0x7f46100a8cf0 2023-09-27T23:52:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.932+0000 7f4607fff700 1 -- 172.21.15.64:0/428943723 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7f461006c910 con 0x7f46100a8cf0 2023-09-27T23:52:43.939 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f45f0060b00 msgr2=0x7f45f0062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f45f0060b00 0x7f45f0062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 msgr2=0x7f46100ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f46100ac480 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f460000d890 tx=0x7f4600008040 comp rx=0 tx=0).stop 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 shutdown_connections 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f45f0060b00 0x7f45f0062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4610158d70 0x7f461015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100a8cf0 0x7f46100ac480 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 --2- 172.21.15.64:0/428943723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46100a88a0 0x7f4610142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 >> 172.21.15.64:0/428943723 conn(0x7f461006b790 msgr2=0x7f461006e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 shutdown_connections 2023-09-27T23:52:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:43.936+0000 7f46157af700 1 -- 172.21.15.64:0/428943723 wait complete. 2023-09-27T23:52:43.950 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:43.944+0000 7ff448597000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-27T23:52:43.953 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-27T23:52:43.988 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:43.986+0000 7fce3a603000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:52:44.009 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:44.004+0000 7ff448597000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-27T23:52:44.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 Processor -- start 2023-09-27T23:52:44.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 -- start start 2023-09-27T23:52:44.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34e0155ba0 0x7f34e0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e00c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34e0156e50 0x7f34e00c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e0067370 con 0x7f34e0155ba0 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e01569c0 con 0x7f34e0156e50 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34e646b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0068e00 con 0x7f34e0068840 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34dffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34e0155ba0 0x7f34e0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34dffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34e0155ba0 0x7f34e0155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38270/0 (socket says 172.21.15.64:38270) 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.056+0000 7f34dffff700 1 -- 172.21.15.64:0/868914111 learned_addr learned my addr 172.21.15.64:0/868914111 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34df7fe700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e00c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e4a08700 1 --2- 172.21.15.64:0/868914111 >> v2:172.21.15.123:3300/0 conn(0x7f34e0156e50 0x7f34e00c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34dffff700 1 -- 172.21.15.64:0/868914111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 msgr2=0x7f34e00c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34dffff700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e00c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34dffff700 1 -- 172.21.15.64:0/868914111 >> v2:172.21.15.123:3300/0 conn(0x7f34e0156e50 msgr2=0x7f34e00c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34dffff700 1 --2- 172.21.15.64:0/868914111 >> v2:172.21.15.123:3300/0 conn(0x7f34e0156e50 0x7f34e00c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34dffff700 1 -- 172.21.15.64:0/868914111 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00690e0 con 0x7f34e0155ba0 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34df7fe700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34dffff700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0155ba0 0x7f34e0155f70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f34e0156930 tx=0x7f34d000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a047aec7832b4dc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34deffd700 1 -- 172.21.15.64:0/868914111 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000dba0 con 0x7f34e0155ba0 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34deffd700 1 -- 172.21.15.64:0/868914111 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000dd00 con 0x7f34e0155ba0 2023-09-27T23:52:44.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34deffd700 1 -- 172.21.15.64:0/868914111 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d00196d0 con 0x7f34e0155ba0 2023-09-27T23:52:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 -- 172.21.15.64:0/868914111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0155ba0 msgr2=0x7f34e0155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0155ba0 0x7f34e0155f70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f34e0156930 tx=0x7f34d000b4f0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 -- 172.21.15.64:0/868914111 shutdown_connections 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 --2- 172.21.15.64:0/868914111 >> v2:172.21.15.123:3300/0 conn(0x7f34e0156e50 0x7f34e00c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 --2- 172.21.15.64:0/868914111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0155ba0 0x7f34e0155f70 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 -- 172.21.15.64:0/868914111 >> 172.21.15.64:0/868914111 conn(0x7f34e006a380 msgr2=0x7f34e006a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 -- 172.21.15.64:0/868914111 shutdown_connections 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 -- 172.21.15.64:0/868914111 wait complete. 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 Processor -- start 2023-09-27T23:52:44.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.060+0000 7f34e646b700 1 -- start start 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e0154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e0155ba0 0x7f34e00c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 0x7f34e00abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00c5aa0 con 0x7f34e0156e50 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0154ec0 con 0x7f34e0155ba0 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00af410 con 0x7f34e0068840 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 0x7f34e00abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 0x7f34e00abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38274/0 (socket says 172.21.15.64:38274) 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 -- 172.21.15.64:0/2627427679 learned_addr learned my addr 172.21.15.64:0/2627427679 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34df7fe700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e0155ba0 0x7f34e00c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34dffff700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e0154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 -- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 msgr2=0x7f34e0154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e0154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 -- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e0155ba0 msgr2=0x7f34e00c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e0155ba0 0x7f34e00c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 -- 172.21.15.64:0/2627427679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d000b790 con 0x7f34e0156e50 2023-09-27T23:52:44.071 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:44.064+0000 7ff448597000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-27T23:52:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e4a08700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 0x7f34e00abe40 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009e90 tx=0x7f34d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d800caa0 con 0x7f34e0156e50 2023-09-27T23:52:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d800f040 con 0x7f34e0156e50 2023-09-27T23:52:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e00af6f0 con 0x7f34e0156e50 2023-09-27T23:52:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d800ce50 con 0x7f34e0156e50 2023-09-27T23:52:44.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e00afcb0 con 0x7f34e0156e50 2023-09-27T23:52:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.064+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00afed0 con 0x7f34e0156e50 2023-09-27T23:52:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.068+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f34d8014690 con 0x7f34e0156e50 2023-09-27T23:52:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.068+0000 7f34c7fff700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f34cc060b00 0x7f34cc062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.068+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f34d807de00 con 0x7f34e0156e50 2023-09-27T23:52:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.068+0000 7f34dffff700 1 -- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f34cc060b00 msgr2=0x7f34cc062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:44.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.068+0000 7f34dffff700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f34cc060b00 0x7f34cc062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:44.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.076+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f34d80514e0 con 0x7f34e0156e50 2023-09-27T23:52:44.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34e0071110 con 0x7f34e0156e50 2023-09-27T23:52:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34c7fff700 1 -- 172.21.15.64:0/2627427679 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v190) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34d8018020 con 0x7f34e0156e50 2023-09-27T23:52:44.241 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 8 2023-09-27T23:52:44.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f34cc060b00 msgr2=0x7f34cc062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f34cc060b00 0x7f34cc062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 msgr2=0x7f34e00abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 0x7f34e00abe40 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009e90 tx=0x7f34d800b040 comp rx=0 tx=0).stop 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 shutdown_connections 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f34cc060b00 0x7f34cc062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34e0156e50 0x7f34e00abe40 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34e0155ba0 0x7f34e00c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 --2- 172.21.15.64:0/2627427679 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34e0068840 0x7f34e0154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 >> 172.21.15.64:0/2627427679 conn(0x7f34e006a380 msgr2=0x7f34e014f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 shutdown_connections 2023-09-27T23:52:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.236+0000 7f34e646b700 1 -- 172.21.15.64:0/2627427679 wait complete. 2023-09-27T23:52:44.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-27T23:52:44.284 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:44.278+0000 7fce3a603000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:52:44.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 Processor -- start 2023-09-27T23:52:44.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 -- start start 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc7c06e790 0x7fcc7c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc7c13cf80 0x7fcc7c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c0c8900 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c0cbe70 con 0x7fcc7c13cf80 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7bfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c0cb010 con 0x7fcc7c06e790 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc7c13cf80 0x7fcc7c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc7c13cf80 0x7fcc7c0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38278/0 (socket says 172.21.15.64:38278) 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7b7fe700 1 -- 172.21.15.64:0/515136466 learned_addr learned my addr 172.21.15.64:0/515136466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7a7fc700 1 --2- 172.21.15.64:0/515136466 >> v2:172.21.15.123:3300/0 conn(0x7fcc7c06e790 0x7fcc7c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.364+0000 7fcc7b7fe700 1 -- 172.21.15.64:0/515136466 >> v2:172.21.15.123:3300/0 conn(0x7fcc7c06e790 msgr2=0x7fcc7c0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7b7fe700 1 --2- 172.21.15.64:0/515136466 >> v2:172.21.15.123:3300/0 conn(0x7fcc7c06e790 0x7fcc7c0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7b7fe700 1 -- 172.21.15.64:0/515136466 >> v1:172.21.15.123:6789/0 conn(0x7fcc7c13cbb0 legacy=0x7fcc7c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7b7fe700 1 -- 172.21.15.64:0/515136466 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0cb260 con 0x7fcc7c13cf80 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7b7fe700 1 --2- 172.21.15.64:0/515136466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cf80 0x7fcc7c0caad0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06c4d0 tx=0x7fcc74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f467b877b965cfd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc79ffb700 1 -- 172.21.15.64:0/515136466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7400e070 con 0x7fcc7c13cf80 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc79ffb700 1 -- 172.21.15.64:0/515136466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7400ba20 con 0x7fcc7c13cf80 2023-09-27T23:52:44.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc79ffb700 1 -- 172.21.15.64:0/515136466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc740092e0 con 0x7fcc7c13cf80 2023-09-27T23:52:44.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 -- 172.21.15.64:0/515136466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cf80 msgr2=0x7fcc7c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/515136466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cf80 0x7fcc7c0caad0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06c4d0 tx=0x7fcc74009840 comp rx=0 tx=0).stop 2023-09-27T23:52:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 -- 172.21.15.64:0/515136466 shutdown_connections 2023-09-27T23:52:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/515136466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cf80 0x7fcc7c0caad0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/515136466 >> v2:172.21.15.123:3300/0 conn(0x7fcc7c06e790 0x7fcc7c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 -- 172.21.15.64:0/515136466 >> 172.21.15.64:0/515136466 conn(0x7fcc7c06b790 msgr2=0x7fcc7c06bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 -- 172.21.15.64:0/515136466 shutdown_connections 2023-09-27T23:52:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 -- 172.21.15.64:0/515136466 wait complete. 2023-09-27T23:52:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 Processor -- start 2023-09-27T23:52:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.368+0000 7fcc7bfff700 1 -- start start 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c06e790 0x7fcc7c1354e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 0x7fcc7c134e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7c13cf80 0x7fcc7c157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c15a1a0 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1577a0 con 0x7fcc7c06e790 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c159870 con 0x7fcc7c13cf80 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 0x7fcc7c134e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 0x7fcc7c134e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38282/0 (socket says 172.21.15.64:38282) 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 -- 172.21.15.64:0/1263395066 learned_addr learned my addr 172.21.15.64:0/1263395066 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7b7fe700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7c13cf80 0x7fcc7c157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7affd700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c06e790 0x7fcc7c1354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 -- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7c13cf80 msgr2=0x7fcc7c157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7c13cf80 0x7fcc7c157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 -- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c06e790 msgr2=0x7fcc7c1354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c06e790 0x7fcc7c1354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 -- 172.21.15.64:0/1263395066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74009ae0 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7a7fc700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 0x7fcc7c134e90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000ea70 tx=0x7fcc700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000eeb0 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7000c850 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c159b50 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000cc30 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c16ab50 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.372+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c159fd0 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.376+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fcc70013680 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.376+0000 7fcc62ffd700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcc68060b00 0x7fcc68062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.376+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fcc7007d660 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.376+0000 7fcc7affd700 1 -- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcc68060b00 msgr2=0x7fcc68062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:44.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.376+0000 7fcc7affd700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcc68060b00 0x7fcc68062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:44.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.380+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcc7004d3c0 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.387 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:44.380+0000 7ff448597000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-27T23:52:44.398 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:44.394+0000 7fce3a603000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:52:44.503 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:44.496+0000 7ff448597000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-27T23:52:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.536+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc7c04be40 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.536+0000 7fcc62ffd700 1 -- 172.21.15.64:0/1263395066 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc700509e0 con 0x7fcc7c13cbb0 2023-09-27T23:52:44.543 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-27T23:52:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcc68060b00 msgr2=0x7fcc68062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcc68060b00 0x7fcc68062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 msgr2=0x7fcc7c134e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 0x7fcc7c134e90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000ea70 tx=0x7fcc700091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 shutdown_connections 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcc68060b00 0x7fcc68062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc7c13cf80 0x7fcc7c157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc7c13cbb0 0x7fcc7c134e90 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 --2- 172.21.15.64:0/1263395066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c06e790 0x7fcc7c1354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 >> 172.21.15.64:0/1263395066 conn(0x7fcc7c06b790 msgr2=0x7fcc7c156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 shutdown_connections 2023-09-27T23:52:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.540+0000 7fcc7bfff700 1 -- 172.21.15.64:0/1263395066 wait complete. 2023-09-27T23:52:44.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-27T23:52:44.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021e75a700 1 Processor -- start 2023-09-27T23:52:44.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021e75a700 1 -- start start 2023-09-27T23:52:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021e75a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0218153a10 0x7f0218156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021e75a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02180c8e70 con 0x7f021806c900 2023-09-27T23:52:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021e75a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02180c8580 con 0x7f0218153640 2023-09-27T23:52:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021e75a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0218153e50 con 0x7f0218153a10 2023-09-27T23:52:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f02177fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f021806c900 0x7f0218156550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41472/0 (socket says 172.21.15.64:41472) 2023-09-27T23:52:44.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f02177fe700 1 -- 172.21.15.64:0/762793337 learned_addr learned my addr 172.21.15.64:0/762793337 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:44.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021ccf7700 1 --2- 172.21.15.64:0/762793337 >> v2:172.21.15.64:3300/0 conn(0x7f0218153a10 0x7f0218156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.660+0000 7f021ccf7700 1 -- 172.21.15.64:0/762793337 >> v1:172.21.15.123:6789/0 conn(0x7f0218153640 legacy=0x7f021806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021ccf7700 1 -- 172.21.15.64:0/762793337 >> v1:172.21.15.64:6789/0 conn(0x7f021806c900 legacy=0x7f0218156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021ccf7700 1 -- 172.21.15.64:0/762793337 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02180c8820 con 0x7f0218153a10 2023-09-27T23:52:44.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f0216ffd700 1 -- 172.21.15.64:0/762793337 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679833510 0 0) 0x7f02180c8e70 con 0x7f021806c900 2023-09-27T23:52:44.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021ccf7700 1 --2- 172.21.15.64:0/762793337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0218153a10 0x7f0218156c60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0210009e60 tx=0x7f0210009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dfa1776e4798bd server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f0216ffd700 1 -- 172.21.15.64:0/762793337 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000e070 con 0x7f0218153a10 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f0216ffd700 1 -- 172.21.15.64:0/762793337 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021000bc30 con 0x7f0218153a10 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f0216ffd700 1 -- 172.21.15.64:0/762793337 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210012430 con 0x7f0218153a10 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- 172.21.15.64:0/762793337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0218153a10 msgr2=0x7f0218156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 --2- 172.21.15.64:0/762793337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0218153a10 0x7f0218156c60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0210009e60 tx=0x7f0210009780 comp rx=0 tx=0).stop 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- 172.21.15.64:0/762793337 shutdown_connections 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 --2- 172.21.15.64:0/762793337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0218153a10 0x7f0218156c60 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- 172.21.15.64:0/762793337 >> 172.21.15.64:0/762793337 conn(0x7f02180c9790 msgr2=0x7f02180c9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- 172.21.15.64:0/762793337 shutdown_connections 2023-09-27T23:52:44.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- 172.21.15.64:0/762793337 wait complete. 2023-09-27T23:52:44.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 Processor -- start 2023-09-27T23:52:44.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- start start 2023-09-27T23:52:44.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 0x7f021814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0218153640 0x7f0218166b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0218153a10 0x7f0218167050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0218167590 con 0x7f021806c900 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02180c34a0 con 0x7f0218153a10 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.664+0000 7f021e75a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02181503e0 con 0x7f0218153640 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 0x7f021814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 0x7f021814fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35274/0 (socket says 172.21.15.64:35274) 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 -- 172.21.15.64:0/2897471648 learned_addr learned my addr 172.21.15.64:0/2897471648 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f02177fe700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0218153640 0x7f0218166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f021ccf7700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0218153a10 0x7f0218167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 -- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0218153640 msgr2=0x7f0218166b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0218153640 0x7f0218166b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 -- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0218153a10 msgr2=0x7f0218167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0218153a10 0x7f0218167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 -- 172.21.15.64:0/2897471648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0210009a20 con 0x7f021806c900 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f021ccf7700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0218153a10 0x7f0218167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f0217fff700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 0x7f021814fea0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f020800c590 tx=0x7f020800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208011590 con 0x7f021806c900 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0208011710 con 0x7f021806c900 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208011ad0 con 0x7f021806c900 2023-09-27T23:52:44.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0218158300 con 0x7f021806c900 2023-09-27T23:52:44.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02180c4c40 con 0x7f021806c900 2023-09-27T23:52:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02180c4f80 con 0x7f021806c900 2023-09-27T23:52:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f020801b3b0 con 0x7f021806c900 2023-09-27T23:52:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f01fffff700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0204060b00 0x7f0204062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f020807c940 con 0x7f021806c900 2023-09-27T23:52:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f02177fe700 1 -- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0204060b00 msgr2=0x7f0204062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.668+0000 7f02177fe700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0204060b00 0x7f0204062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:44.679 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:44.674+0000 7fce3a603000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:52:44.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.676+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f020804c6a0 con 0x7f021806c900 2023-09-27T23:52:44.824 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:44.820+0000 7ff448597000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-27T23:52:44.841 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:44.838+0000 7fce3a603000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:44.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0218071110 con 0x7f021806c900 2023-09-27T23:52:44.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f01fffff700 1 -- 172.21.15.64:0/2897471648 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02080170a0 con 0x7f021806c900 2023-09-27T23:52:44.845 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-27T23:52:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0204060b00 msgr2=0x7f0204062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0204060b00 0x7f0204062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 msgr2=0x7f021814fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 0x7f021814fea0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f020800c590 tx=0x7f020800c8a0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 shutdown_connections 2023-09-27T23:52:44.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0204060b00 0x7f0204062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0218153a10 0x7f0218167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0218153640 0x7f0218166b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 --2- 172.21.15.64:0/2897471648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f021806c900 0x7f021814fea0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 >> 172.21.15.64:0/2897471648 conn(0x7f02180c9790 msgr2=0x7f0218152720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 shutdown_connections 2023-09-27T23:52:44.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.840+0000 7f021e75a700 1 -- 172.21.15.64:0/2897471648 wait complete. 2023-09-27T23:52:44.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:44.905 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:44.902+0000 7fce3a603000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:52:44.957 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:44.954+0000 7fce3a603000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:52:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb192bc0700 1 Processor -- start 2023-09-27T23:52:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb192bc0700 1 -- start start 2023-09-27T23:52:44.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb192bc0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c06c7b0 0x7fb18c068800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb192bc0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c069ad0 con 0x7fb18c156890 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb192bc0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c068ec0 con 0x7fb18c156c60 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb192bc0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0691a0 con 0x7fb18c06c7b0 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb191bbe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb18c156890 0x7fb18c06c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41474/0 (socket says 172.21.15.64:41474) 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb191bbe700 1 -- 172.21.15.64:0/2589493837 learned_addr learned my addr 172.21.15.64:0/2589493837 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.952+0000 7fb1923bf700 1 --2- 172.21.15.64:0/2589493837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c06c7b0 0x7fb18c068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb190bbc700 1 -- 172.21.15.64:0/2589493837 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45070146 0 0) 0x7fb18c069ad0 con 0x7fb18c156890 2023-09-27T23:52:44.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb190bbc700 1 -- 172.21.15.64:0/2589493837 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb178003570 con 0x7fb18c156890 2023-09-27T23:52:44.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb190bbc700 1 -- 172.21.15.64:0/2589493837 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031762830 0 0) 0x7fb18c068ec0 con 0x7fb18c156c60 2023-09-27T23:52:44.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb1923bf700 1 -- 172.21.15.64:0/2589493837 >> v1:172.21.15.123:6789/0 conn(0x7fb18c156c60 legacy=0x7fb18c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb1923bf700 1 -- 172.21.15.64:0/2589493837 >> v1:172.21.15.64:6789/0 conn(0x7fb18c156890 legacy=0x7fb18c06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb1923bf700 1 -- 172.21.15.64:0/2589493837 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c069480 con 0x7fb18c06c7b0 2023-09-27T23:52:44.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb1923bf700 1 --2- 172.21.15.64:0/2589493837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c06c7b0 0x7fb18c068800 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb184009e90 tx=0x7fb184009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f9505f30375f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb190bbc700 1 -- 172.21.15.64:0/2589493837 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400e070 con 0x7fb18c06c7b0 2023-09-27T23:52:44.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb190bbc700 1 -- 172.21.15.64:0/2589493837 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400b920 con 0x7fb18c06c7b0 2023-09-27T23:52:44.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb190bbc700 1 -- 172.21.15.64:0/2589493837 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188001470 con 0x7fb18c06c7b0 2023-09-27T23:52:44.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 -- 172.21.15.64:0/2589493837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c06c7b0 msgr2=0x7fb18c068800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2589493837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c06c7b0 0x7fb18c068800 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb184009e90 tx=0x7fb184009780 comp rx=0 tx=0).stop 2023-09-27T23:52:44.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 -- 172.21.15.64:0/2589493837 shutdown_connections 2023-09-27T23:52:44.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2589493837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c06c7b0 0x7fb18c068800 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 -- 172.21.15.64:0/2589493837 >> 172.21.15.64:0/2589493837 conn(0x7fb18c0ccb50 msgr2=0x7fb18c0ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:44.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 -- 172.21.15.64:0/2589493837 shutdown_connections 2023-09-27T23:52:44.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 -- 172.21.15.64:0/2589493837 wait complete. 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.956+0000 7fb192bc0700 1 Processor -- start 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- start start 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 0x7fb18c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c156890 0x7fb18c0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18c156c60 0x7fb18c0a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0a8a20 con 0x7fb18c06c7b0 2023-09-27T23:52:44.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0a8d00 con 0x7fb18c156c60 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0a8fe0 con 0x7fb18c156890 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 0x7fb18c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 0x7fb18c155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35282/0 (socket says 172.21.15.64:35282) 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 -- 172.21.15.64:0/2064105434 learned_addr learned my addr 172.21.15.64:0/2064105434 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb1913bd700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c156890 0x7fb18c0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb1923bf700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18c156c60 0x7fb18c0a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 -- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c156890 msgr2=0x7fb18c0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c156890 0x7fb18c0b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 -- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18c156c60 msgr2=0x7fb18c0a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:44.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18c156c60 0x7fb18c0a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 -- 172.21.15.64:0/2064105434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184009a20 con 0x7fb18c06c7b0 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb1923bf700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18c156c60 0x7fb18c0a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb191bbe700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 0x7fb18c155a10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fb1880015a0 tx=0x7fb188001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188007360 con 0x7fb18c06c7b0 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1880074e0 con 0x7fb18c06c7b0 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c0a92c0 con 0x7fb18c06c7b0 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188004830 con 0x7fb18c06c7b0 2023-09-27T23:52:44.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c0aafe0 con 0x7fb18c06c7b0 2023-09-27T23:52:44.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.960+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c04be40 con 0x7fb18c06c7b0 2023-09-27T23:52:44.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.964+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fb188004bc0 con 0x7fb18c06c7b0 2023-09-27T23:52:44.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.964+0000 7fb181ffb700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 0x7fb170062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:44.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.964+0000 7fb1913bd700 1 -- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 msgr2=0x7fb170062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:44.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.964+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fb18807c910 con 0x7fb18c06c7b0 2023-09-27T23:52:44.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.964+0000 7fb1913bd700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 0x7fb170062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:44.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:44.968+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb18804c670 con 0x7fb18c06c7b0 2023-09-27T23:52:45.005 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.000+0000 7ff448597000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-27T23:52:45.014 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:45.010+0000 7fce3a603000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:52:45.072 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.068+0000 7ff448597000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-27T23:52:45.127 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.120+0000 7ff448597000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-27T23:52:45.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.164+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb18c063210 con 0x7fb18c06c7b0 2023-09-27T23:52:45.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.164+0000 7fb181ffb700 1 -- 172.21.15.64:0/2064105434 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4610 (secure 0 0 0) 0x7fb188018090 con 0x7fb18c06c7b0 2023-09-27T23:52:45.169 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:45.169 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:52:20.268986+0000","modified":"2023-09-27T23:52:22.285887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10144":{"gid":10144,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/1929792685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1929792685},{"type":"v1","addr":"172.21.15.123:6835","nonce":1929792685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-09-27T23:52:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb1913bd700 1 -- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 msgr2=0x7fb170062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:45.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb1913bd700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 0x7fb170062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 msgr2=0x7fb170062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 0x7fb170062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 msgr2=0x7fb18c155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 0x7fb18c155a10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fb1880015a0 tx=0x7fb188001bf0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 shutdown_connections 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb170060b00 0x7fb170062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18c156c60 0x7fb18c0a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb18c156890 0x7fb18c0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 --2- 172.21.15.64:0/2064105434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb18c06c7b0 0x7fb18c155a10 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 >> 172.21.15.64:0/2064105434 conn(0x7fb18c0ccb50 msgr2=0x7fb18c0c2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 shutdown_connections 2023-09-27T23:52:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.168+0000 7fb192bc0700 1 -- 172.21.15.64:0/2064105434 wait complete. 2023-09-27T23:52:45.174 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 72 2023-09-27T23:52:45.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:45.187 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:45.182+0000 7fce3a603000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:45.187 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.184+0000 7ff448597000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-27T23:52:45.243 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:45.238+0000 7fce3a603000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:52:45.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.272+0000 7fed350eb700 1 Processor -- start 2023-09-27T23:52:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.272+0000 7fed350eb700 1 -- start start 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed301547d0 0x7fed3015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed30154ba0 0x7fed30157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed3006bf30 0x7fed3006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed30158420 con 0x7fed3006bf30 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3015b990 con 0x7fed30154ba0 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3006c890 con 0x7fed301547d0 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed3006bf30 0x7fed3006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed3006bf30 0x7fed3006c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35292/0 (socket says 172.21.15.64:35292) 2023-09-27T23:52:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 -- 172.21.15.64:0/1294624934 learned_addr learned my addr 172.21.15.64:0/1294624934 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2ed9d700 1 --2- 172.21.15.64:0/1294624934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed301547d0 0x7fed3015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2e59c700 1 --2- 172.21.15.64:0/1294624934 >> v2:172.21.15.123:3300/0 conn(0x7fed30154ba0 0x7fed30157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 -- 172.21.15.64:0/1294624934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed301547d0 msgr2=0x7fed3015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 --2- 172.21.15.64:0/1294624934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed301547d0 0x7fed3015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 -- 172.21.15.64:0/1294624934 >> v2:172.21.15.123:3300/0 conn(0x7fed30154ba0 msgr2=0x7fed30157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 --2- 172.21.15.64:0/1294624934 >> v2:172.21.15.123:3300/0 conn(0x7fed30154ba0 0x7fed30157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 -- 172.21.15.64:0/1294624934 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3015b0f0 con 0x7fed3006bf30 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2e59c700 1 --2- 172.21.15.64:0/1294624934 >> v2:172.21.15.123:3300/0 conn(0x7fed30154ba0 0x7fed30157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2f59e700 1 --2- 172.21.15.64:0/1294624934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed3006bf30 0x7fed3006c350 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fed300cb8a0 tx=0x7fed28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a2987a3307d25e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2dd9b700 1 -- 172.21.15.64:0/1294624934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800e070 con 0x7fed3006bf30 2023-09-27T23:52:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2dd9b700 1 -- 172.21.15.64:0/1294624934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2800ba20 con 0x7fed3006bf30 2023-09-27T23:52:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed2dd9b700 1 -- 172.21.15.64:0/1294624934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280092e0 con 0x7fed3006bf30 2023-09-27T23:52:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- 172.21.15.64:0/1294624934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed3006bf30 msgr2=0x7fed3006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- 172.21.15.64:0/1294624934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed3006bf30 0x7fed3006c350 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fed300cb8a0 tx=0x7fed28009840 comp rx=0 tx=0).stop 2023-09-27T23:52:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- 172.21.15.64:0/1294624934 shutdown_connections 2023-09-27T23:52:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- 172.21.15.64:0/1294624934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed3006bf30 0x7fed3006c350 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- 172.21.15.64:0/1294624934 >> v2:172.21.15.123:3300/0 conn(0x7fed30154ba0 0x7fed30157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 --2- 172.21.15.64:0/1294624934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed301547d0 0x7fed3015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- 172.21.15.64:0/1294624934 >> 172.21.15.64:0/1294624934 conn(0x7fed300cde20 msgr2=0x7fed300ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- 172.21.15.64:0/1294624934 shutdown_connections 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.276+0000 7fed350eb700 1 -- 172.21.15.64:0/1294624934 wait complete. 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 Processor -- start 2023-09-27T23:52:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 -- start start 2023-09-27T23:52:45.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed3006bf30 0x7fed300b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 0x7fed300c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed30154ba0 0x7fed301527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300b0bc0 con 0x7fed301547d0 2023-09-27T23:52:45.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300b3950 con 0x7fed30154ba0 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed300b0e10 con 0x7fed3006bf30 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 0x7fed300c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 0x7fed300c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35306/0 (socket says 172.21.15.64:35306) 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 -- 172.21.15.64:0/1762526612 learned_addr learned my addr 172.21.15.64:0/1762526612 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2ed9d700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed3006bf30 0x7fed300b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2f59e700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed30154ba0 0x7fed301527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 -- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed3006bf30 msgr2=0x7fed300b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed3006bf30 0x7fed300b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 -- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed30154ba0 msgr2=0x7fed301527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed30154ba0 0x7fed301527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 -- 172.21.15.64:0/1762526612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009ae0 con 0x7fed301547d0 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2f59e700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed30154ba0 0x7fed301527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed2e59c700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 0x7fed300c7860 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fed2400d860 tx=0x7fed24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240125f0 con 0x7fed301547d0 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24012750 con 0x7fed301547d0 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed300b10f0 con 0x7fed301547d0 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24012af0 con 0x7fed301547d0 2023-09-27T23:52:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.280+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed30152d10 con 0x7fed301547d0 2023-09-27T23:52:45.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.284+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed30071110 con 0x7fed301547d0 2023-09-27T23:52:45.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.284+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7fed24013480 con 0x7fed301547d0 2023-09-27T23:52:45.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.284+0000 7fed16ffd700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fed1c060b00 0x7fed1c062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.284+0000 7fed2ed9d700 1 -- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fed1c060b00 msgr2=0x7fed1c062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:45.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.284+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fed2407d640 con 0x7fed301547d0 2023-09-27T23:52:45.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.284+0000 7fed2ed9d700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fed1c060b00 0x7fed1c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:45.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.296+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fed2404d3a0 con 0x7fed301547d0 2023-09-27T23:52:45.306 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:45.302+0000 7fce3a603000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:52:45.369 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.364+0000 7ff448597000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-27T23:52:45.427 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.420+0000 7ff448597000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-27T23:52:45.452 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:45.450+0000 7fce3a603000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:52:45.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.456+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed3015b3d0 con 0x7fed301547d0 2023-09-27T23:52:45.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.456+0000 7fed16ffd700 1 -- 172.21.15.64:0/1762526612 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17584 (secure 0 0 0) 0x7fed240509c0 con 0x7fed301547d0 2023-09-27T23:52:45.463 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:45.463 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":190,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:37.175276+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:52:18.850130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-09-27T23:52:19.262770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/3573447607":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6832/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6833/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/2413747055":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3774540904":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3310607164":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/4028139068":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1447825948":"2023-09-28T23:52:37.175204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fed1c060b00 msgr2=0x7fed1c062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fed1c060b00 0x7fed1c062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 msgr2=0x7fed300c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 0x7fed300c7860 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fed2400d860 tx=0x7fed24008040 comp rx=0 tx=0).stop 2023-09-27T23:52:45.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 shutdown_connections 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fed1c060b00 0x7fed1c062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed30154ba0 0x7fed301527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed301547d0 0x7fed300c7860 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 --2- 172.21.15.64:0/1762526612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed3006bf30 0x7fed300b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 >> 172.21.15.64:0/1762526612 conn(0x7fed300cde20 msgr2=0x7fed300ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 shutdown_connections 2023-09-27T23:52:45.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.460+0000 7fed350eb700 1 -- 172.21.15.64:0/1762526612 wait complete. 2023-09-27T23:52:45.477 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:52:45.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:45.494 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.488+0000 7ff448597000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-27T23:52:45.512 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:45.506+0000 7fce3a603000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:52:45.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 Processor -- start 2023-09-27T23:52:45.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 -- start start 2023-09-27T23:52:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 0x7f4b5415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415d900 0x7f4b540afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5415dd20 con 0x7f4b5415cff0 2023-09-27T23:52:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540b1fe0 con 0x7f4b540b1b60 2023-09-27T23:52:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.572+0000 7f4b5359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54067100 con 0x7f4b5415d900 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b52d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415d900 0x7f4b540afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b51d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 0x7f4b5415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b51d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 0x7f4b5415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:34962/0 (socket says 172.21.15.64:34962) 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b51d9b700 1 -- 172.21.15.64:0/2427666415 learned_addr learned my addr 172.21.15.64:0/2427666415 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2427666415 >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 msgr2=0x7f4b5415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b52d9d700 1 --2- 172.21.15.64:0/2427666415 >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 0x7f4b5415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2427666415 >> v1:172.21.15.123:6789/0 conn(0x7f4b5415cff0 legacy=0x7f4b5406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2427666415 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540673e0 con 0x7f4b5415d900 2023-09-27T23:52:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b51d9b700 1 --2- 172.21.15.64:0/2427666415 >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 0x7f4b5415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b52d9d700 1 --2- 172.21.15.64:0/2427666415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415d900 0x7f4b540afb40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009e60 tx=0x7f4b4c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57b5189f062ed6b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5159a700 1 -- 172.21.15.64:0/2427666415 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00e070 con 0x7f4b5415d900 2023-09-27T23:52:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5159a700 1 -- 172.21.15.64:0/2427666415 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00bb60 con 0x7f4b5415d900 2023-09-27T23:52:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5159a700 1 -- 172.21.15.64:0/2427666415 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c012430 con 0x7f4b5415d900 2023-09-27T23:52:45.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 -- 172.21.15.64:0/2427666415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415d900 msgr2=0x7f4b540afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2427666415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415d900 0x7f4b540afb40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009e60 tx=0x7f4b4c009780 comp rx=0 tx=0).stop 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 -- 172.21.15.64:0/2427666415 shutdown_connections 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2427666415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415d900 0x7f4b540afb40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2427666415 >> v2:172.21.15.123:3300/0 conn(0x7f4b540b1b60 0x7f4b5415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 -- 172.21.15.64:0/2427666415 >> 172.21.15.64:0/2427666415 conn(0x7f4b5406e160 msgr2=0x7f4b540adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 -- 172.21.15.64:0/2427666415 shutdown_connections 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 -- 172.21.15.64:0/2427666415 wait complete. 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 Processor -- start 2023-09-27T23:52:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.576+0000 7f4b5359e700 1 -- start start 2023-09-27T23:52:45.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b540b1b60 0x7f4b54157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415cff0 0x7f4b5416d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 0x7f4b5416d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54160cb0 con 0x7f4b5415d900 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54160f90 con 0x7f4b540b1b60 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415ffa0 con 0x7f4b5415cff0 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 0x7f4b5416d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 0x7f4b5416d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35314/0 (socket says 172.21.15.64:35314) 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2907814007 learned_addr learned my addr 172.21.15.64:0/2907814007 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5259c700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b540b1b60 0x7f4b54157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b51d9b700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415cff0 0x7f4b5416d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415cff0 msgr2=0x7f4b5416d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415cff0 0x7f4b5416d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b540b1b60 msgr2=0x7f4b54157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b540b1b60 0x7f4b54157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 -- 172.21.15.64:0/2907814007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b480094f0 con 0x7f4b5415d900 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5259c700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b540b1b60 0x7f4b54157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b52d9d700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 0x7f4b5416d590 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b6c0 tx=0x7f4b4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c012e90 con 0x7f4b5415d900 2023-09-27T23:52:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c027470 con 0x7f4b5415d900 2023-09-27T23:52:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c009a20 con 0x7f4b5415d900 2023-09-27T23:52:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c027830 con 0x7f4b5415d900 2023-09-27T23:52:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.580+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b541605e0 con 0x7f4b5415d900 2023-09-27T23:52:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.584+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f4b4c024070 con 0x7f4b5415d900 2023-09-27T23:52:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.584+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b54161120 con 0x7f4b5415d900 2023-09-27T23:52:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.584+0000 7f4b427fc700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4b34060b00 0x7f4b34062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.584+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f4b4c07fef0 con 0x7f4b5415d900 2023-09-27T23:52:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.584+0000 7f4b5259c700 1 -- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4b34060b00 msgr2=0x7f4b34062fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.584+0000 7f4b5259c700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4b34060b00 0x7f4b34062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:45.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.592+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f4b4c04fc50 con 0x7f4b5415d900 2023-09-27T23:52:45.654 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.648+0000 7ff448597000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-27T23:52:45.719 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:45.716+0000 7ff448597000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-27T23:52:45.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.780+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b5404be40 con 0x7f4b5415d900 2023-09-27T23:52:45.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.780+0000 7f4b427fc700 1 -- 172.21.15.64:0/2907814007 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f4b4c02c020 con 0x7f4b5415d900 2023-09-27T23:52:45.786 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:45.787 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:52:20.268986+0000","modified":"2023-09-27T23:52:22.285887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10144":{"gid":10144,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/1929792685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1929792685},{"type":"v1","addr":"172.21.15.123:6835","nonce":1929792685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4b34060b00 msgr2=0x7f4b34062fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4b34060b00 0x7f4b34062fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 msgr2=0x7f4b5416d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 0x7f4b5416d590 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b6c0 tx=0x7f4b4c019000 comp rx=0 tx=0).stop 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 shutdown_connections 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4b34060b00 0x7f4b34062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b5415d900 0x7f4b5416d590 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b5415cff0 0x7f4b5416d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 --2- 172.21.15.64:0/2907814007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b540b1b60 0x7f4b54157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 >> 172.21.15.64:0/2907814007 conn(0x7f4b5406e160 msgr2=0x7f4b540abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:45.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 shutdown_connections 2023-09-27T23:52:45.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.784+0000 7f4b5359e700 1 -- 172.21.15.64:0/2907814007 wait complete. 2023-09-27T23:52:45.790 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 72 2023-09-27T23:52:45.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:52:45.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aae02c700 1 Processor -- start 2023-09-27T23:52:45.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aae02c700 1 -- start start 2023-09-27T23:52:45.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aae02c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0aa80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aae02c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80cbab0 con 0x7f0aa80cc480 2023-09-27T23:52:45.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aae02c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80cc8c0 con 0x7f0aa8159010 2023-09-27T23:52:45.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aae02c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80caea0 con 0x7f0aa81593e0 2023-09-27T23:52:45.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa7fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0aa80cc480 0x7f0aa8067fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41488/0 (socket says 172.21.15.64:41488) 2023-09-27T23:52:45.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa7fff700 1 -- 172.21.15.64:0/1003407423 learned_addr learned my addr 172.21.15.64:0/1003407423 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:45.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa6ffd700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0aa80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230546982 0 0) 0x7f0aa80cbab0 con 0x7f0aa80cc480 2023-09-27T23:52:45.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a90003570 con 0x7f0aa80cc480 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531404229 0 0) 0x7f0a98001110 con 0x7f0aa8159010 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa80cbab0 con 0x7f0aa8159010 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1649777299 0 0) 0x7f0a90003570 con 0x7f0aa80cc480 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 msgr2=0x7f0aa80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0aa80cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 >> v1:172.21.15.123:6789/0 conn(0x7f0aa8159010 legacy=0x7f0aa80b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80cb180 con 0x7f0aa80cc480 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f0aa00017f0 con 0x7f0aa80cc480 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 >> v1:172.21.15.64:6789/0 conn(0x7f0aa80cc480 legacy=0x7f0aa8067fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0a90003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a90004180 0x7f0a90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a90004d50 0x7f0a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80caea0 con 0x7f0a90004d50 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000bba0 con 0x7f0a90004180 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000bd60 con 0x7f0aa81593e0 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa7fff700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a90004d50 0x7f0a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa77fe700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0a90003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.940+0000 7f0aa6ffd700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a90004180 0x7f0a90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 -- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 msgr2=0x7f0a90003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0a90003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 -- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a90004180 msgr2=0x7f0a90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a90004180 0x7f0a90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 -- 172.21.15.64:0/1003407423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a90003570 con 0x7f0a90004d50 2023-09-27T23:52:45.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa77fe700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa81593e0 0x7f0a90003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:45.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a90004d50 0x7f0a900051c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000a8d0 tx=0x7f0aa0002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b952b756e5d6710 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:45.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00070d0 con 0x7f0a90004d50 2023-09-27T23:52:45.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0007230 con 0x7f0a90004d50 2023-09-27T23:52:45.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa67fc700 1 -- 172.21.15.64:0/1003407423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00075f0 con 0x7f0a90004d50 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 -- 172.21.15.64:0/1003407423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a90004d50 msgr2=0x7f0a900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 --2- 172.21.15.64:0/1003407423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a90004d50 0x7f0a900051c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000a8d0 tx=0x7f0aa0002190 comp rx=0 tx=0).stop 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aa7fff700 1 -- 172.21.15.64:0/1003407423 reap_dead start 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 -- 172.21.15.64:0/1003407423 shutdown_connections 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 -- 172.21.15.64:0/1003407423 >> 172.21.15.64:0/1003407423 conn(0x7f0aa80d0ac0 msgr2=0x7f0aa80cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 -- 172.21.15.64:0/1003407423 shutdown_connections 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 -- 172.21.15.64:0/1003407423 wait complete. 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 Processor -- start 2023-09-27T23:52:45.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.944+0000 7f0aae02c700 1 -- start start 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 0x7f0aa80c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa80c4930 0x7f0aa815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa815ad80 0x7f0aa815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80c7b40 con 0x7f0aa8156210 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80c9a00 con 0x7f0aa815ad80 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80c9ca0 con 0x7f0aa80c4930 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 0x7f0aa80c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 0x7f0aa80c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35338/0 (socket says 172.21.15.64:35338) 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 -- 172.21.15.64:0/3372254746 learned_addr learned my addr 172.21.15.64:0/3372254746 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:45.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa6ffd700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa80c4930 0x7f0aa815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa7fff700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa815ad80 0x7f0aa815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 -- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa80c4930 msgr2=0x7f0aa815a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa80c4930 0x7f0aa815a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 -- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa815ad80 msgr2=0x7f0aa815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa815ad80 0x7f0aa815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 -- 172.21.15.64:0/3372254746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80c9f80 con 0x7f0aa8156210 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aa77fe700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 0x7f0aa80c7fd0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800fc00 tx=0x7f0a98007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98002e00 con 0x7f0aa8156210 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800d850 con 0x7f0aa8156210 2023-09-27T23:52:45.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa81565e0 con 0x7f0aa8156210 2023-09-27T23:52:45.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800dbf0 con 0x7f0aa8156210 2023-09-27T23:52:45.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa81571b0 con 0x7f0aa8156210 2023-09-27T23:52:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.948+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa8071110 con 0x7f0aa8156210 2023-09-27T23:52:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.952+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 94) v1 ==== 80418+0+0 (secure 0 0 0) 0x7f0a98006820 con 0x7f0aa8156210 2023-09-27T23:52:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.952+0000 7f0a8f7fe700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0a94060b30 0x7f0a94062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.952+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f0a9807d350 con 0x7f0aa8156210 2023-09-27T23:52:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.952+0000 7f0aa6ffd700 1 -- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0a94060b30 msgr2=0x7f0a94062fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6832/43188 2023-09-27T23:52:45.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.952+0000 7f0aa6ffd700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0a94060b30 0x7f0a94062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-27T23:52:45.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:45.956+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0a9804d010 con 0x7f0aa8156210 2023-09-27T23:52:46.108 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.102+0000 7fce3a603000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:52:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.140+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 95) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f0a9804d3e0 con 0x7f0aa8156210 2023-09-27T23:52:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.140+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0a94060b30 msgr2=0x7f0a94062fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-27T23:52:46.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.140+0000 7f0a8f7fe700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0a94060b30 0x7f0a94062fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aa80cc8c0 con 0x7f0aa8156210 2023-09-27T23:52:46.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0a8f7fe700 1 -- 172.21.15.64:0/3372254746 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f0a98069530 con 0x7f0aa8156210 2023-09-27T23:52:46.165 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:46.165 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-27T23:52:20.268986+0000","modified":"2023-09-27T23:52:22.285887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10144":{"gid":10144,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/1929792685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1929792685},{"type":"v1","addr":"172.21.15.123:6835","nonce":1929792685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 msgr2=0x7f0aa80c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 0x7f0aa80c7fd0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800fc00 tx=0x7f0a98007d30 comp rx=0 tx=0).stop 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 shutdown_connections 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f0a94060b30 0x7f0a94062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa815ad80 0x7f0aa815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0aa80c4930 0x7f0aa815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 --2- 172.21.15.64:0/3372254746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0aa8156210 0x7f0aa80c7fd0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 >> 172.21.15.64:0/3372254746 conn(0x7f0aa80d0ac0 msgr2=0x7f0aa80cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 shutdown_connections 2023-09-27T23:52:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.160+0000 7f0aae02c700 1 -- 172.21.15.64:0/3372254746 wait complete. 2023-09-27T23:52:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 72 2023-09-27T23:52:46.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:46.210 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce04040700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.710 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce04040700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.710 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce04040700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.710 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce04040700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.711 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce04040700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.712 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce07847700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.712 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce07847700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.712 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce07847700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.712 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce07847700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.712 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-09-27T23:52:46.206+0000 7fce07847700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.268+0000 7f14b759e700 1 Processor -- start 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.268+0000 7f14b759e700 1 -- start start 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14b806b510 0x7f14b81553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b8067d90 con 0x7f14b8155010 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b806c070 con 0x7f14b8067060 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b8068930 con 0x7f14b806b510 2023-09-27T23:52:46.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f14b8155010 0x7f14b80afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41498/0 (socket says 172.21.15.64:41498) 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b659c700 1 -- 172.21.15.64:0/1843317810 learned_addr learned my addr 172.21.15.64:0/1843317810 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b5d9b700 1 --2- 172.21.15.64:0/1843317810 >> v2:172.21.15.123:3300/0 conn(0x7f14b806b510 0x7f14b81553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839821599 0 0) 0x7f14b8067d90 con 0x7f14b8155010 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a0003570 con 0x7f14b8155010 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 816105261 0 0) 0x7f14a0003570 con 0x7f14b8155010 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 >> v2:172.21.15.123:3300/0 conn(0x7f14b806b510 msgr2=0x7f14b81553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 --2- 172.21.15.64:0/1843317810 >> v2:172.21.15.123:3300/0 conn(0x7f14b806b510 0x7f14b81553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 >> v1:172.21.15.123:6789/0 conn(0x7f14b8067060 legacy=0x7f14b8068220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b806bab0 con 0x7f14b8155010 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f14a80015f0 con 0x7f14b8155010 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 >> v1:172.21.15.64:6789/0 conn(0x7f14b8155010 legacy=0x7f14b80afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a0003a60 0x7f14a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a0004810 0x7f14a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a00051c0 0x7f14a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8068930 con 0x7f14a00051c0 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a000c0e0 con 0x7f14a0004810 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a000e4b0 con 0x7f14a0003a60 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a00051c0 0x7f14a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b659c700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a0003a60 0x7f14a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b5d9b700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a0004810 0x7f14a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 -- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a0003a60 msgr2=0x7f14a0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a0003a60 0x7f14a0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 -- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a0004810 msgr2=0x7f14a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:46.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a0004810 0x7f14a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 -- 172.21.15.64:0/1843317810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a0003570 con 0x7f14a00051c0 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b659c700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a0003a60 0x7f14a0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 -- 172.21.15.64:0/1843317810 reap_dead start 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b6d9d700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a00051c0 0x7f14a000ba60 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f14b00020a0 tx=0x7f14b00016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d291fe34134b7f83 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b00019e0 con 0x7f14a00051c0 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b000c8b0 con 0x7f14a00051c0 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b559a700 1 -- 172.21.15.64:0/1843317810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b000cc50 con 0x7f14a00051c0 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- 172.21.15.64:0/1843317810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a00051c0 msgr2=0x7f14a000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a00051c0 0x7f14a000ba60 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f14b00020a0 tx=0x7f14b00016d0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- 172.21.15.64:0/1843317810 shutdown_connections 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 --2- 172.21.15.64:0/1843317810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a00051c0 0x7f14a000ba60 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- 172.21.15.64:0/1843317810 >> 172.21.15.64:0/1843317810 conn(0x7f14b806e160 msgr2=0x7f14b80adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- 172.21.15.64:0/1843317810 shutdown_connections 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.272+0000 7f14b759e700 1 -- 172.21.15.64:0/1843317810 wait complete. 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 Processor -- start 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 -- start start 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a00051c0 0x7f14b814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 0x7f14b816cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14b816d190 0x7f14b80c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c6e20 con 0x7f14b816c800 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c6860 con 0x7f14b816d190 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c6b40 con 0x7f14a00051c0 2023-09-27T23:52:46.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 0x7f14b816cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 0x7f14b816cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35348/0 (socket says 172.21.15.64:35348) 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 -- 172.21.15.64:0/2383162271 learned_addr learned my addr 172.21.15.64:0/2383162271 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b659c700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a00051c0 0x7f14b814ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b6d9d700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14b816d190 0x7f14b80c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 -- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a00051c0 msgr2=0x7f14b814ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a00051c0 0x7f14b814ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 -- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14b816d190 msgr2=0x7f14b80c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14b816d190 0x7f14b80c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 -- 172.21.15.64:0/2383162271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8158a60 con 0x7f14b816c800 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b5d9b700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 0x7f14b816cc50 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0093f0 tx=0x7f14ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac015670 con 0x7f14b816c800 2023-09-27T23:52:46.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac0157d0 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8158d40 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.276+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac015b70 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.280+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b814b170 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.280+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 95) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f14ac013410 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.280+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f14ac07cee0 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.284+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b804be40 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.288+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f14ac04d160 con 0x7f14b816c800 2023-09-27T23:52:46.723 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-09-27T23:52:46.352+0000 7ff448597000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-27T23:52:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.452+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14b806bd90 con 0x7f14b816c800 2023-09-27T23:52:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14a67fc700 1 -- 172.21.15.64:0/2383162271 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f14ac022030 con 0x7f14b816c800 2023-09-27T23:52:46.724 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:46.724 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":191,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:46.131235+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-09-27T23:52:18.850130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-09-27T23:52:19.262770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2551836643":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3550075260":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3834368324":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3658378865":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/855337008":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/1582733572":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3573447607":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1334338128":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6832/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6833/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/2413747055":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3774540904":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3310607164":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/4028139068":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1447825948":"2023-09-28T23:52:37.175204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 msgr2=0x7f14b816cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 0x7f14b816cc50 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0093f0 tx=0x7f14ac008040 comp rx=0 tx=0).stop 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 shutdown_connections 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14b816d190 0x7f14b80c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14b816c800 0x7f14b816cc50 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 --2- 172.21.15.64:0/2383162271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14a00051c0 0x7f14b814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 >> 172.21.15.64:0/2383162271 conn(0x7f14b806e160 msgr2=0x7f14b80b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 shutdown_connections 2023-09-27T23:52:46.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.456+0000 7f14b759e700 1 -- 172.21.15.64:0/2383162271 wait complete. 2023-09-27T23:52:46.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-27T23:52:46.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a53d4a700 1 Processor -- start 2023-09-27T23:52:46.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a53d4a700 1 -- start start 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a53d4a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a4c0b2d90 0x7f6a4c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a53d4a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4c0cb710 con 0x7f6a4c159090 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a53d4a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4c0b3200 con 0x7f6a4c0b22f0 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a53d4a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4c0af490 con 0x7f6a4c0b2d90 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a51ae6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6a4c159090 0x7f6a4c0af620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41508/0 (socket says 172.21.15.64:41508) 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a51ae6700 1 -- 172.21.15.64:0/2943161428 learned_addr learned my addr 172.21.15.64:0/2943161428 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:46.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 --2- 172.21.15.64:0/2943161428 >> v2:172.21.15.123:3300/0 conn(0x7f6a4c0b2d90 0x7f6a4c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896515380 0 0) 0x7f6a4c0cb710 con 0x7f6a4c159090 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a38003570 con 0x7f6a4c159090 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2309296225 0 0) 0x7f6a38003570 con 0x7f6a4c159090 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 >> v2:172.21.15.123:3300/0 conn(0x7f6a4c0b2d90 msgr2=0x7f6a4c0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 --2- 172.21.15.64:0/2943161428 >> v2:172.21.15.123:3300/0 conn(0x7f6a4c0b2d90 0x7f6a4c0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 >> v1:172.21.15.123:6789/0 conn(0x7f6a4c0b22f0 legacy=0x7f6a4c0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c0cade0 con 0x7f6a4c159090 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f6a40001470 con 0x7f6a4c159090 2023-09-27T23:52:46.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 >> v1:172.21.15.64:6789/0 conn(0x7f6a4c159090 legacy=0x7f6a4c0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a38003a60 0x7f6a38003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a38004810 0x7f6a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a380051c0 0x7f6a3800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0af490 con 0x7f6a38004810 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3800c0d0 con 0x7f6a380051c0 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3800c290 con 0x7f6a38003a60 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a38004810 0x7f6a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a51ae6700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a38003a60 0x7f6a38003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a522e7700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a380051c0 0x7f6a3800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 -- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a38003a60 msgr2=0x7f6a38003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a38003a60 0x7f6a38003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 -- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a380051c0 msgr2=0x7f6a3800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a380051c0 0x7f6a3800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a522e7700 1 -- 172.21.15.64:0/2943161428 reap_dead start 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.844+0000 7f6a512e5700 1 -- 172.21.15.64:0/2943161428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38003570 con 0x7f6a38004810 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a512e5700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a38004810 0x7f6a38004c80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009bf0 tx=0x7f6a48009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fe532c0a144a6d server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48005a00 con 0x7f6a38004810 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48005b60 con 0x7f6a38004810 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a50ae4700 1 -- 172.21.15.64:0/2943161428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800e710 con 0x7f6a38004810 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- 172.21.15.64:0/2943161428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a38004810 msgr2=0x7f6a38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a38004810 0x7f6a38004c80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009bf0 tx=0x7f6a48009120 comp rx=0 tx=0).stop 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- 172.21.15.64:0/2943161428 shutdown_connections 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/2943161428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a38004810 0x7f6a38004c80 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- 172.21.15.64:0/2943161428 >> 172.21.15.64:0/2943161428 conn(0x7f6a4c0d0ab0 msgr2=0x7f6a4c0cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:46.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- 172.21.15.64:0/2943161428 shutdown_connections 2023-09-27T23:52:46.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- 172.21.15.64:0/2943161428 wait complete. 2023-09-27T23:52:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 Processor -- start 2023-09-27T23:52:46.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- start start 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a38004810 0x7f6a4c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a4c16c6d0 0x7f6a4c1695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 0x7f6a4c0b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c15a9f0 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c169ed0 con 0x7f6a38004810 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a53d4a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c16a1b0 con 0x7f6a4c16c6d0 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a522e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 0x7f6a4c0b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a522e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 0x7f6a4c0b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35368/0 (socket says 172.21.15.64:35368) 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a522e7700 1 -- 172.21.15.64:0/4265437109 learned_addr learned my addr 172.21.15.64:0/4265437109 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a512e5700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a4c16c6d0 0x7f6a4c1695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.848+0000 7f6a51ae6700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a38004810 0x7f6a4c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a522e7700 1 -- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a4c16c6d0 msgr2=0x7f6a4c1695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a522e7700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a4c16c6d0 0x7f6a4c1695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a522e7700 1 -- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a38004810 msgr2=0x7f6a4c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a522e7700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a38004810 0x7f6a4c0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a522e7700 1 -- 172.21.15.64:0/4265437109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c16a490 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a522e7700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 0x7f6a4c0b45a0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c088840 tx=0x7f6a44001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44006e80 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a44014430 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c15a680 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440147f0 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c16ac30 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c071110 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 95) v1 ==== 79970+0+0 (secure 0 0 0) 0x7f6a44014be0 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.852+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7023+0+0 (secure 0 0 0) 0x7f6a4407c4e0 con 0x7f6a4c16a7c0 2023-09-27T23:52:46.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:46.860+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6a4404abb0 con 0x7f6a4c16a7c0 2023-09-27T23:52:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.060+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6a4c0b13d0 con 0x7f6a4c16a7c0 2023-09-27T23:52:47.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.144+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 96) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f6a4404c210 con 0x7f6a4c16a7c0 2023-09-27T23:52:47.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.144+0000 7f6a3dffb700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6a30066d20 0x7f6a30069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.148+0000 7f6a51ae6700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6a30066d20 0x7f6a30069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.148+0000 7f6a51ae6700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6a30066d20 0x7f6a30069100 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6a40001ce0 tx=0x7f6a40001e50 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:47.798 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-09-27T23:52:47.796+0000 7f4c8dbe1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-27T23:52:47.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.808+0000 7f6a3dffb700 1 -- 172.21.15.64:0/4265437109 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6a44069e80 con 0x7f6a4c16a7c0 2023-09-27T23:52:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6a30066d20 msgr2=0x7f6a30069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6a30066d20 0x7f6a30069100 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6a40001ce0 tx=0x7f6a40001e50 comp rx=0 tx=0).stop 2023-09-27T23:52:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 msgr2=0x7f6a4c0b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 0x7f6a4c0b45a0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c088840 tx=0x7f6a44001640 comp rx=0 tx=0).stop 2023-09-27T23:52:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 shutdown_connections 2023-09-27T23:52:47.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f6a30066d20 0x7f6a30069100 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a4c16a7c0 0x7f6a4c0b45a0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a4c16c6d0 0x7f6a4c1695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 --2- 172.21.15.64:0/4265437109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a38004810 0x7f6a4c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 >> 172.21.15.64:0/4265437109 conn(0x7f6a4c0d0ab0 msgr2=0x7f6a4c0ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 shutdown_connections 2023-09-27T23:52:47.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.812+0000 7f6a53d4a700 1 -- 172.21.15.64:0/4265437109 wait complete. 2023-09-27T23:52:47.820 INFO:tasks.ceph.mds.b.smithi123.stderr: -3496> 2023-09-27T23:52:26.195+0000 7f9d13a2a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-27T23:52:47.821 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-27T23:52:47.835 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-27T23:52:47.851 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-09-27T23:52:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.952+0000 7f19893cb700 1 Processor -- start 2023-09-27T23:52:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.952+0000 7f19893cb700 1 -- start start 2023-09-27T23:52:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f1984156f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f198406c150 0x7f198406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198415a410 con 0x7f1984155e00 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198406cb10 con 0x7f198406c150 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198415a6f0 con 0x7f19841561d0 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f198406c150 0x7f198406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f198406c150 0x7f198406c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35370/0 (socket says 172.21.15.64:35370) 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 -- 172.21.15.64:0/1617168842 learned_addr learned my addr 172.21.15.64:0/1617168842 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19837fe700 1 --2- 172.21.15.64:0/1617168842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f1984156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 -- 172.21.15.64:0/1617168842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 msgr2=0x7f1984156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 --2- 172.21.15.64:0/1617168842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f1984156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 -- 172.21.15.64:0/1617168842 >> v1:172.21.15.123:6789/0 conn(0x7f1984155e00 legacy=0x7f19841599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:47.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 -- 172.21.15.64:0/1617168842 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1984156700 con 0x7f198406c150 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1988bca700 1 --2- 172.21.15.64:0/1617168842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198406c150 0x7f198406c5d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f197c009f20 tx=0x7f197c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fab73e9c3e197f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1982ffd700 1 -- 172.21.15.64:0/1617168842 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c00e070 con 0x7f198406c150 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1982ffd700 1 -- 172.21.15.64:0/1617168842 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c00b9d0 con 0x7f198406c150 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f1982ffd700 1 -- 172.21.15.64:0/1617168842 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0092e0 con 0x7f198406c150 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- 172.21.15.64:0/1617168842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198406c150 msgr2=0x7f198406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 --2- 172.21.15.64:0/1617168842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198406c150 0x7f198406c5d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f197c009f20 tx=0x7f197c009840 comp rx=0 tx=0).stop 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- 172.21.15.64:0/1617168842 shutdown_connections 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 --2- 172.21.15.64:0/1617168842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198406c150 0x7f198406c5d0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 --2- 172.21.15.64:0/1617168842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f1984156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- 172.21.15.64:0/1617168842 >> 172.21.15.64:0/1617168842 conn(0x7f19840ccb70 msgr2=0x7f19840ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:47.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- 172.21.15.64:0/1617168842 shutdown_connections 2023-09-27T23:52:47.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.956+0000 7f19893cb700 1 -- 172.21.15.64:0/1617168842 wait complete. 2023-09-27T23:52:47.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 Processor -- start 2023-09-27T23:52:47.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 -- start start 2023-09-27T23:52:47.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f198406c150 0x7f19840c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 0x7f19840c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f19840add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840c6ae0 con 0x7f1984155e00 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19841671d0 con 0x7f198406c150 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19840ae260 con 0x7f19841561d0 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 0x7f19840c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 0x7f19840c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35372/0 (socket says 172.21.15.64:35372) 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 -- 172.21.15.64:0/2844780764 learned_addr learned my addr 172.21.15.64:0/2844780764 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f1983fff700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f198406c150 0x7f19840c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f1988bca700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f19840add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 -- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 msgr2=0x7f19840add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:47.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f19840add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 -- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f198406c150 msgr2=0x7f19840c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f198406c150 0x7f19840c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:47.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 -- 172.21.15.64:0/2844780764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c009ae0 con 0x7f1984155e00 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19837fe700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 0x7f19840c6dc0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f197800e960 tx=0x7f19780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978013680 con 0x7f1984155e00 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19780137e0 con 0x7f1984155e00 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19840c4b20 con 0x7f1984155e00 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978013b80 con 0x7f1984155e00 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.960+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1984166e60 con 0x7f1984155e00 2023-09-27T23:52:47.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.964+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1984167380 con 0x7f1984155e00 2023-09-27T23:52:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.964+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 96) v1 ==== 80097+0+0 (secure 0 0 0) 0x7f19780123d0 con 0x7f1984155e00 2023-09-27T23:52:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.964+0000 7f196bfff700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f19700607a0 0x7f1970062c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.964+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7109+0+0 (secure 0 0 0) 0x7f197807cf90 con 0x7f1984155e00 2023-09-27T23:52:47.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.964+0000 7f1983fff700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f19700607a0 0x7f1970062c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:47.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.964+0000 7f1983fff700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f19700607a0 0x7f1970062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1974003a80 tx=0x7f1974007ec0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:47.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:47.972+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f19780acbb0 con 0x7f1984155e00 2023-09-27T23:52:48.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.176+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f198404be40 con 0x7f1984155e00 2023-09-27T23:52:48.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.816+0000 7f196bfff700 1 -- 172.21.15.64:0/2844780764 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1984167380 con 0x7f1984155e00 2023-09-27T23:52:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f19700607a0 msgr2=0x7f1970062c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f19700607a0 0x7f1970062c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1974003a80 tx=0x7f1974007ec0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 msgr2=0x7f19840c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 0x7f19840c6dc0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f197800e960 tx=0x7f19780091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 shutdown_connections 2023-09-27T23:52:48.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f19700607a0 0x7f1970062c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19841561d0 0x7f19840add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1984155e00 0x7f19840c6dc0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 --2- 172.21.15.64:0/2844780764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f198406c150 0x7f19840c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 >> 172.21.15.64:0/2844780764 conn(0x7f19840ccb70 msgr2=0x7f1984151820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 shutdown_connections 2023-09-27T23:52:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.820+0000 7f19893cb700 1 -- 172.21.15.64:0/2844780764 wait complete. 2023-09-27T23:52:48.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-27T23:52:48.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.956+0000 7fdbc57c4700 1 Processor -- start 2023-09-27T23:52:48.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.956+0000 7fdbc57c4700 1 -- start start 2023-09-27T23:52:48.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbc00abd30 0x7fdbc0068a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:48.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdbc00b1d60 0x7fdbc006c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:48.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc00afa90 con 0x7fdbc00ab960 2023-09-27T23:52:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc0069100 con 0x7fdbc00abd30 2023-09-27T23:52:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc00693e0 con 0x7fdbc00b1d60 2023-09-27T23:52:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbc00abd30 0x7fdbc0068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdbc00abd30 0x7fdbc0068a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35384/0 (socket says 172.21.15.64:35384) 2023-09-27T23:52:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 -- 172.21.15.64:0/3334886752 learned_addr learned my addr 172.21.15.64:0/3334886752 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:48.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc4fc3700 1 --2- 172.21.15.64:0/3334886752 >> v2:172.21.15.123:3300/0 conn(0x7fdbc00b1d60 0x7fdbc006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 -- 172.21.15.64:0/3334886752 >> v2:172.21.15.123:3300/0 conn(0x7fdbc00b1d60 msgr2=0x7fdbc006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 --2- 172.21.15.64:0/3334886752 >> v2:172.21.15.123:3300/0 conn(0x7fdbc00b1d60 0x7fdbc006c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 -- 172.21.15.64:0/3334886752 >> v1:172.21.15.123:6789/0 conn(0x7fdbc00ab960 legacy=0x7fdbc00ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 -- 172.21.15.64:0/3334886752 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc00696c0 con 0x7fdbc00abd30 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbb7fff700 1 --2- 172.21.15.64:0/3334886752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00abd30 0x7fdbc0068a00 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ccb60 tx=0x7fdba8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f3d84dac39a98 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbbf7fe700 1 -- 172.21.15.64:0/3334886752 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8009e70 con 0x7fdbc00abd30 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbbf7fe700 1 -- 172.21.15.64:0/3334886752 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800b860 con 0x7fdbc00abd30 2023-09-27T23:52:48.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbbf7fe700 1 -- 172.21.15.64:0/3334886752 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800bc00 con 0x7fdbc00abd30 2023-09-27T23:52:48.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- 172.21.15.64:0/3334886752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00abd30 msgr2=0x7fdbc0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:48.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/3334886752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00abd30 0x7fdbc0068a00 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ccb60 tx=0x7fdba8009250 comp rx=0 tx=0).stop 2023-09-27T23:52:48.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- 172.21.15.64:0/3334886752 shutdown_connections 2023-09-27T23:52:48.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/3334886752 >> v2:172.21.15.123:3300/0 conn(0x7fdbc00b1d60 0x7fdbc006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/3334886752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00abd30 0x7fdbc0068a00 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- 172.21.15.64:0/3334886752 >> 172.21.15.64:0/3334886752 conn(0x7fdbc00ccbc0 msgr2=0x7fdbc00ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:48.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- 172.21.15.64:0/3334886752 shutdown_connections 2023-09-27T23:52:48.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.960+0000 7fdbc57c4700 1 -- 172.21.15.64:0/3334886752 wait complete. 2023-09-27T23:52:48.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.964+0000 7fdbc57c4700 1 Processor -- start 2023-09-27T23:52:48.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.964+0000 7fdbc57c4700 1 -- start start 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.964+0000 7fdbc57c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 0x7fdbc00a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.964+0000 7fdbc57c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc00abd30 0x7fdbc0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.964+0000 7fdbc57c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbc00b1d60 0x7fdbc0167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.964+0000 7fdbc57c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc01592a0 con 0x7fdbc00ab960 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbc57c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0159580 con 0x7fdbc00abd30 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbc57c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0159860 con 0x7fdbc00b1d60 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 0x7fdbc00a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:48.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 0x7fdbc00a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35396/0 (socket says 172.21.15.64:35396) 2023-09-27T23:52:48.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 -- 172.21.15.64:0/323053192 learned_addr learned my addr 172.21.15.64:0/323053192 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:48.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbc4fc3700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbc00b1d60 0x7fdbc0167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:48.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbb7fff700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc00abd30 0x7fdbc0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 -- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbc00b1d60 msgr2=0x7fdbc0167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbc00b1d60 0x7fdbc0167ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 -- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc00abd30 msgr2=0x7fdbc0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc00abd30 0x7fdbc0155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 -- 172.21.15.64:0/323053192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80094f0 con 0x7fdbc00ab960 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbffff700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 0x7fdbc00a4650 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000c4e0 tx=0x7fdbb000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0011590 con 0x7fdbc00ab960 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb0011710 con 0x7fdbc00ab960 2023-09-27T23:52:48.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc0159b40 con 0x7fdbc00ab960 2023-09-27T23:52:48.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0011ad0 con 0x7fdbc00ab960 2023-09-27T23:52:48.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc00a4b90 con 0x7fdbc00ab960 2023-09-27T23:52:48.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.968+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc004be40 con 0x7fdbc00ab960 2023-09-27T23:52:48.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.972+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fdbb001b3b0 con 0x7fdbc00ab960 2023-09-27T23:52:48.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.972+0000 7fdbbcff9700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fdba0060a70 0x7fdba0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:48.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.972+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7109+0+0 (secure 0 0 0) 0x7fdbb007c810 con 0x7fdbc00ab960 2023-09-27T23:52:48.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.972+0000 7fdbb7fff700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fdba0060a70 0x7fdba0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:48.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.972+0000 7fdbb7fff700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fdba0060a70 0x7fdba0062f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ccb60 tx=0x7fdba8019000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:48.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:48.980+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fdbb004c4c0 con 0x7fdbc00ab960 2023-09-27T23:52:49.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:49.144+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdbc00cda60 con 0x7fdbc00ab960 2023-09-27T23:52:50.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.160+0000 7fdbbcff9700 1 -- 172.21.15.64:0/323053192 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v193) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdbb00170d0 con 0x7fdbc00ab960 2023-09-27T23:52:50.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fdba0060a70 msgr2=0x7fdba0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fdba0060a70 0x7fdba0062f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ccb60 tx=0x7fdba8019000 comp rx=0 tx=0).stop 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 msgr2=0x7fdbc00a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 0x7fdbc00a4650 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000c4e0 tx=0x7fdbb000c7f0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 shutdown_connections 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fdba0060a70 0x7fdba0062f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdbc00b1d60 0x7fdbc0167ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbc00abd30 0x7fdbc0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 --2- 172.21.15.64:0/323053192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbc00ab960 0x7fdbc00a4650 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 >> 172.21.15.64:0/323053192 conn(0x7fdbc00ccbc0 msgr2=0x7fdbc00b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 shutdown_connections 2023-09-27T23:52:50.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.168+0000 7fdbc57c4700 1 -- 172.21.15.64:0/323053192 wait complete. 2023-09-27T23:52:50.174 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-27T23:52:50.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-27T23:52:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 Processor -- start 2023-09-27T23:52:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 -- start start 2023-09-27T23:52:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f27b0156f90 0x7f27b0068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27b0157ef0 0x7f27b00b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b00b0640 0x7f27b00ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b0157360 con 0x7f27b0157ef0 2023-09-27T23:52:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b006b6b0 con 0x7f27b0156f90 2023-09-27T23:52:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6a0a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0067470 con 0x7f27b00b0640 2023-09-27T23:52:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27b0157ef0 0x7f27b00b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27b0157ef0 0x7f27b00b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35410/0 (socket says 172.21.15.64:35410) 2023-09-27T23:52:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 -- 172.21.15.64:0/680747475 learned_addr learned my addr 172.21.15.64:0/680747475 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5a08700 1 --2- 172.21.15.64:0/680747475 >> v2:172.21.15.123:3300/0 conn(0x7f27b0156f90 0x7f27b0068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b6209700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b00b0640 0x7f27b00ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 -- 172.21.15.64:0/680747475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b00b0640 msgr2=0x7f27b00ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b00b0640 0x7f27b00ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 -- 172.21.15.64:0/680747475 >> v2:172.21.15.123:3300/0 conn(0x7f27b0156f90 msgr2=0x7f27b0068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.292+0000 7f27b5207700 1 --2- 172.21.15.64:0/680747475 >> v2:172.21.15.123:3300/0 conn(0x7f27b0156f90 0x7f27b0068630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b5207700 1 -- 172.21.15.64:0/680747475 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b006b130 con 0x7f27b0157ef0 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6209700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b00b0640 0x7f27b00ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b5207700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b00b0100 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f27b006cb50 tx=0x7f27a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8125ecbfbd9a59 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b4a06700 1 -- 172.21.15.64:0/680747475 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0009e70 con 0x7f27b0157ef0 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b4a06700 1 -- 172.21.15.64:0/680747475 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a000b860 con 0x7f27b0157ef0 2023-09-27T23:52:50.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b4a06700 1 -- 172.21.15.64:0/680747475 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a000bc00 con 0x7f27b0157ef0 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- 172.21.15.64:0/680747475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 msgr2=0x7f27b00b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b00b0100 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f27b006cb50 tx=0x7f27a0009250 comp rx=0 tx=0).stop 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- 172.21.15.64:0/680747475 shutdown_connections 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b00b0640 0x7f27b00ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/680747475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b00b0100 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/680747475 >> v2:172.21.15.123:3300/0 conn(0x7f27b0156f90 0x7f27b0068630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- 172.21.15.64:0/680747475 >> 172.21.15.64:0/680747475 conn(0x7f27b00c8560 msgr2=0x7f27b00c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- 172.21.15.64:0/680747475 shutdown_connections 2023-09-27T23:52:50.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- 172.21.15.64:0/680747475 wait complete. 2023-09-27T23:52:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 Processor -- start 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- start start 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b00b0640 0x7f27b00ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b0156f90 0x7f27b014a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b014ac40 con 0x7f27b0157ef0 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b014d070 con 0x7f27b00b0640 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6a0a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00caa20 con 0x7f27b0156f90 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6209700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6209700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35422/0 (socket says 172.21.15.64:35422) 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.296+0000 7f27b6209700 1 -- 172.21.15.64:0/3060647394 learned_addr learned my addr 172.21.15.64:0/3060647394 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b5a08700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b00b0640 0x7f27b00ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b5207700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b0156f90 0x7f27b014a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6209700 1 -- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b0156f90 msgr2=0x7f27b014a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6209700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b0156f90 0x7f27b014a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6209700 1 -- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b00b0640 msgr2=0x7f27b00ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6209700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b00b0640 0x7f27b00ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6209700 1 -- 172.21.15.64:0/3060647394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a00094f0 con 0x7f27b0157ef0 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6209700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b0159da0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009fc0 tx=0x7f27a80099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800dbe0 con 0x7f27b0157ef0 2023-09-27T23:52:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a800f040 con 0x7f27b0157ef0 2023-09-27T23:52:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8015440 con 0x7f27b0157ef0 2023-09-27T23:52:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00cad00 con 0x7f27b0157ef0 2023-09-27T23:52:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b00cb2c0 con 0x7f27b0157ef0 2023-09-27T23:52:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.300+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b014d230 con 0x7f27b0157ef0 2023-09-27T23:52:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.304+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f27a801e3b0 con 0x7f27b0157ef0 2023-09-27T23:52:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.304+0000 7f27a5ffb700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2794060a70 0x7f2794062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.304+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6671+0+0 (secure 0 0 0) 0x7f27a807d850 con 0x7f27b0157ef0 2023-09-27T23:52:50.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.304+0000 7f27b5a08700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2794060a70 0x7f2794062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:50.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.304+0000 7f27b5a08700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2794060a70 0x7f2794062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009c80 tx=0x7f27ac0091a0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:50.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.312+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f27a804d5b0 con 0x7f27b0157ef0 2023-09-27T23:52:50.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:50.476+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f27b004be40 con 0x7f27b0157ef0 2023-09-27T23:52:51.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.168+0000 7f27a5ffb700 1 -- 172.21.15.64:0/3060647394 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v194) v1 ==== 169+0+0 (secure 0 0 0) 0x7f27a8050bd0 con 0x7f27b0157ef0 2023-09-27T23:52:51.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2794060a70 msgr2=0x7f2794062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2794060a70 0x7f2794062f20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009c80 tx=0x7f27ac0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 msgr2=0x7f27b0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b0159da0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009fc0 tx=0x7f27a80099b0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 shutdown_connections 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f2794060a70 0x7f2794062f20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b0157ef0 0x7f27b0159da0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27b0156f90 0x7f27b014a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 --2- 172.21.15.64:0/3060647394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b00b0640 0x7f27b00ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 >> 172.21.15.64:0/3060647394 conn(0x7f27b00c8560 msgr2=0x7f27b0154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 shutdown_connections 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.176+0000 7f27b6a0a700 1 -- 172.21.15.64:0/3060647394 wait complete. 2023-09-27T23:52:51.181 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-27T23:52:51.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-09-27T23:52:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 Processor -- start 2023-09-27T23:52:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 -- start start 2023-09-27T23:52:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efdbc1561d0 0x7efdbc156f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdbc06c150 0x7efdbc06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc06caf0 con 0x7efdbc155e00 2023-09-27T23:52:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc15a410 con 0x7efdbc06c150 2023-09-27T23:52:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbbfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc15a6f0 con 0x7efdbc1561d0 2023-09-27T23:52:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdbc06c150 0x7efdbc06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdbc06c150 0x7efdbc06c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35438/0 (socket says 172.21.15.64:35438) 2023-09-27T23:52:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdbb7fe700 1 -- 172.21.15.64:0/2837645733 learned_addr learned my addr 172.21.15.64:0/2837645733 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.300+0000 7efdba7fc700 1 --2- 172.21.15.64:0/2837645733 >> v2:172.21.15.123:3300/0 conn(0x7efdbc1561d0 0x7efdbc156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbb7fe700 1 -- 172.21.15.64:0/2837645733 >> v2:172.21.15.123:3300/0 conn(0x7efdbc1561d0 msgr2=0x7efdbc156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbb7fe700 1 --2- 172.21.15.64:0/2837645733 >> v2:172.21.15.123:3300/0 conn(0x7efdbc1561d0 0x7efdbc156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbb7fe700 1 -- 172.21.15.64:0/2837645733 >> v1:172.21.15.123:6789/0 conn(0x7efdbc155e00 legacy=0x7efdbc1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdb9ffb700 1 -- 172.21.15.64:0/2837645733 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612482377 0 0) 0x7efdac001110 con 0x7efdbc155e00 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbb7fe700 1 -- 172.21.15.64:0/2837645733 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc156700 con 0x7efdbc06c150 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbb7fe700 1 --2- 172.21.15.64:0/2837645733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06c150 0x7efdbc06c5b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7efdbc06e640 tx=0x7efdb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f807651d7af9243 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdb9ffb700 1 -- 172.21.15.64:0/2837645733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb400e070 con 0x7efdbc06c150 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdb9ffb700 1 -- 172.21.15.64:0/2837645733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb400ba20 con 0x7efdbc06c150 2023-09-27T23:52:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdb9ffb700 1 -- 172.21.15.64:0/2837645733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40092e0 con 0x7efdbc06c150 2023-09-27T23:52:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 -- 172.21.15.64:0/2837645733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06c150 msgr2=0x7efdbc06c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2837645733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06c150 0x7efdbc06c5b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7efdbc06e640 tx=0x7efdb4009840 comp rx=0 tx=0).stop 2023-09-27T23:52:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 -- 172.21.15.64:0/2837645733 shutdown_connections 2023-09-27T23:52:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2837645733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06c150 0x7efdbc06c5b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2837645733 >> v2:172.21.15.123:3300/0 conn(0x7efdbc1561d0 0x7efdbc156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 -- 172.21.15.64:0/2837645733 >> 172.21.15.64:0/2837645733 conn(0x7efdbc0ccb70 msgr2=0x7efdbc0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 -- 172.21.15.64:0/2837645733 shutdown_connections 2023-09-27T23:52:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.304+0000 7efdbbfff700 1 -- 172.21.15.64:0/2837645733 wait complete. 2023-09-27T23:52:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 Processor -- start 2023-09-27T23:52:51.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 -- start start 2023-09-27T23:52:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdbc06c150 0x7efdbc151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 0x7efdbc16a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdbc1561d0 0x7efdbc14bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc151dd0 con 0x7efdbc155e00 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0b0370 con 0x7efdbc06c150 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc16a770 con 0x7efdbc1561d0 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 0x7efdbc16a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 0x7efdbc16a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35444/0 (socket says 172.21.15.64:35444) 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 -- 172.21.15.64:0/2773609068 learned_addr learned my addr 172.21.15.64:0/2773609068 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbaffd700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdbc06c150 0x7efdbc151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbb7fe700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdbc1561d0 0x7efdbc14bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 -- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdbc1561d0 msgr2=0x7efdbc14bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdbc1561d0 0x7efdbc14bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 -- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdbc06c150 msgr2=0x7efdbc151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdbc06c150 0x7efdbc151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 -- 172.21.15.64:0/2773609068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009ae0 con 0x7efdbc155e00 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdba7fc700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 0x7efdbc16a230 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7efdb0010b70 tx=0x7efdb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ea60 con 0x7efdbc155e00 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb000ebc0 con 0x7efdbc155e00 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc0afa40 con 0x7efdbc155e00 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0011450 con 0x7efdbc155e00 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.308+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc0b0000 con 0x7efdbc155e00 2023-09-27T23:52:51.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.312+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc04be40 con 0x7efdbc155e00 2023-09-27T23:52:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.312+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7efdb0011800 con 0x7efdbc155e00 2023-09-27T23:52:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.312+0000 7efda2ffd700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efda8060a70 0x7efda8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.312+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7efdb007f6f0 con 0x7efdbc155e00 2023-09-27T23:52:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.312+0000 7efdbaffd700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efda8060a70 0x7efda8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.312+0000 7efdbaffd700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efda8060a70 0x7efda8062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdac001370 tx=0x7efdac007ba0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.320+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efdb004f6f0 con 0x7efdbc155e00 2023-09-27T23:52:51.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.480+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7efdbc156a40 con 0x7efdbc155e00 2023-09-27T23:52:51.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efda2ffd700 1 -- 172.21.15.64:0/2773609068 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v67) v1 ==== 115+0+0 (secure 0 0 0) 0x7efdb001e0b0 con 0x7efdbc155e00 2023-09-27T23:52:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efda8060a70 msgr2=0x7efda8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efda8060a70 0x7efda8062f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdac001370 tx=0x7efdac007ba0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 msgr2=0x7efdbc16a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 0x7efdbc16a230 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7efdb0010b70 tx=0x7efdb00091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 shutdown_connections 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efda8060a70 0x7efda8062f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efdbc1561d0 0x7efdbc14bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc155e00 0x7efdbc16a230 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 --2- 172.21.15.64:0/2773609068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdbc06c150 0x7efdbc151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 >> 172.21.15.64:0/2773609068 conn(0x7efdbc0ccb70 msgr2=0x7efdbc152610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 shutdown_connections 2023-09-27T23:52:51.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.488+0000 7efdbbfff700 1 -- 172.21.15.64:0/2773609068 wait complete. 2023-09-27T23:52:51.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-09-27T23:52:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.612+0000 7f58c91f5700 1 Processor -- start 2023-09-27T23:52:51.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.612+0000 7f58c91f5700 1 -- start start 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4151920 0x7f58c415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f58c4157820 0x7f58c4157c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c415b330 con 0x7f58c40ac820 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c4158830 con 0x7f58c4157820 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4151cf0 con 0x7f58c4151920 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c359e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f58c40ac820 0x7f58c40acc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41512/0 (socket says 172.21.15.64:41512) 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c359e700 1 -- 172.21.15.64:0/552566783 learned_addr learned my addr 172.21.15.64:0/552566783 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c2d9d700 1 --2- 172.21.15.64:0/552566783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4151920 0x7f58c415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c259c700 1 --2- 172.21.15.64:0/552566783 >> v2:172.21.15.123:3300/0 conn(0x7f58c4157820 0x7f58c4157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c1d9b700 1 -- 172.21.15.64:0/552566783 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475541315 0 0) 0x7f58c415b330 con 0x7f58c40ac820 2023-09-27T23:52:51.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c1d9b700 1 -- 172.21.15.64:0/552566783 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ac003570 con 0x7f58c40ac820 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c2d9d700 1 -- 172.21.15.64:0/552566783 >> v2:172.21.15.123:3300/0 conn(0x7f58c4157820 msgr2=0x7f58c4157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c2d9d700 1 --2- 172.21.15.64:0/552566783 >> v2:172.21.15.123:3300/0 conn(0x7f58c4157820 0x7f58c4157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c2d9d700 1 -- 172.21.15.64:0/552566783 >> v1:172.21.15.64:6789/0 conn(0x7f58c40ac820 legacy=0x7f58c40acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c2d9d700 1 -- 172.21.15.64:0/552566783 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4158270 con 0x7f58c4151920 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c2d9d700 1 --2- 172.21.15.64:0/552566783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4151920 0x7f58c415b7c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f58b40098e0 tx=0x7f58b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9c45e6f2f3d4ef server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c1d9b700 1 -- 172.21.15.64:0/552566783 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4011630 con 0x7f58c4151920 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c1d9b700 1 -- 172.21.15.64:0/552566783 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4011790 con 0x7f58c4151920 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c1d9b700 1 -- 172.21.15.64:0/552566783 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4011b10 con 0x7f58c4151920 2023-09-27T23:52:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- 172.21.15.64:0/552566783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4151920 msgr2=0x7f58c415b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 --2- 172.21.15.64:0/552566783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4151920 0x7f58c415b7c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f58b40098e0 tx=0x7f58b40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- 172.21.15.64:0/552566783 shutdown_connections 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 --2- 172.21.15.64:0/552566783 >> v2:172.21.15.123:3300/0 conn(0x7f58c4157820 0x7f58c4157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 --2- 172.21.15.64:0/552566783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4151920 0x7f58c415b7c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- 172.21.15.64:0/552566783 >> 172.21.15.64:0/552566783 conn(0x7f58c40ccf80 msgr2=0x7f58c40c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- 172.21.15.64:0/552566783 shutdown_connections 2023-09-27T23:52:51.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.616+0000 7f58c91f5700 1 -- 172.21.15.64:0/552566783 wait complete. 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 Processor -- start 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 -- start start 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 0x7f58c4152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c4151920 0x7f58c41673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4157820 0x7f58c4167910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41547a0 con 0x7f58c40ac820 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4168bf0 con 0x7f58c4151920 2023-09-27T23:52:51.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4153410 con 0x7f58c4157820 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 0x7f58c4152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 0x7f58c4152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35452/0 (socket says 172.21.15.64:35452) 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 -- 172.21.15.64:0/3370166520 learned_addr learned my addr 172.21.15.64:0/3370166520 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c259c700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c4151920 0x7f58c41673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c359e700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4157820 0x7f58c4167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 -- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4157820 msgr2=0x7f58c4167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4157820 0x7f58c4167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 -- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c4151920 msgr2=0x7f58c41673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c4151920 0x7f58c41673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 -- 172.21.15.64:0/3370166520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009420 con 0x7f58c40ac820 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c359e700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4157820 0x7f58c4167910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c2d9d700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 0x7f58c4152d40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f58c4151e30 tx=0x7f58b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4011b10 con 0x7f58c40ac820 2023-09-27T23:52:51.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4000c90 con 0x7f58c40ac820 2023-09-27T23:52:51.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c41536f0 con 0x7f58c40ac820 2023-09-27T23:52:51.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b401a660 con 0x7f58c40ac820 2023-09-27T23:52:51.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.620+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c4168880 con 0x7f58c40ac820 2023-09-27T23:52:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.624+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4071110 con 0x7f58c40ac820 2023-09-27T23:52:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.624+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f58b401aa50 con 0x7f58c40ac820 2023-09-27T23:52:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.624+0000 7f58aaffd700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58b0060a70 0x7f58b0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.624+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f58b407eec0 con 0x7f58c40ac820 2023-09-27T23:52:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.624+0000 7f58c259c700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58b0060a70 0x7f58b0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.624+0000 7f58c259c700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58b0060a70 0x7f58b0062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f58c40b01d0 tx=0x7f58b80091a0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.632+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f58b404eec0 con 0x7f58c40ac820 2023-09-27T23:52:51.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.792+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f58c4158550 con 0x7f58c40ac820 2023-09-27T23:52:51.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58aaffd700 1 -- 172.21.15.64:0/3370166520 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v68) v1 ==== 120+0+0 (secure 0 0 0) 0x7f58b40524e0 con 0x7f58c40ac820 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58b0060a70 msgr2=0x7f58b0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58b0060a70 0x7f58b0062f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f58c40b01d0 tx=0x7f58b80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 msgr2=0x7f58c4152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 0x7f58c4152d40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f58c4151e30 tx=0x7f58b4017000 comp rx=0 tx=0).stop 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 shutdown_connections 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f58b0060a70 0x7f58b0062f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58c4157820 0x7f58c4167910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58c4151920 0x7f58c41673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 --2- 172.21.15.64:0/3370166520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c40ac820 0x7f58c4152d40 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 >> 172.21.15.64:0/3370166520 conn(0x7f58c40ccf80 msgr2=0x7f58c4159800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 shutdown_connections 2023-09-27T23:52:51.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.804+0000 7f58c91f5700 1 -- 172.21.15.64:0/3370166520 wait complete. 2023-09-27T23:52:51.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-27T23:52:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e68d60700 1 Processor -- start 2023-09-27T23:52:51.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e68d60700 1 -- start start 2023-09-27T23:52:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e68d60700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1e640cc410 0x7f1e6415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e68d60700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e64068120 con 0x7f1e6415cc40 2023-09-27T23:52:51.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e68d60700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e640caf70 con 0x7f1e640b23d0 2023-09-27T23:52:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e68d60700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e64068350 con 0x7f1e640cc410 2023-09-27T23:52:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e637fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1e6415cc40 0x7f1e6415d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41528/0 (socket says 172.21.15.64:41528) 2023-09-27T23:52:51.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.932+0000 7f1e637fe700 1 -- 172.21.15.64:0/2448310524 learned_addr learned my addr 172.21.15.64:0/2448310524 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 --2- 172.21.15.64:0/2448310524 >> v2:172.21.15.123:3300/0 conn(0x7f1e640cc410 0x7f1e6415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219300540 0 0) 0x7f1e64068120 con 0x7f1e6415cc40 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e44003570 con 0x7f1e6415cc40 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843360424 0 0) 0x7f1e44003570 con 0x7f1e6415cc40 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 >> v2:172.21.15.123:3300/0 conn(0x7f1e640cc410 msgr2=0x7f1e6415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 --2- 172.21.15.64:0/2448310524 >> v2:172.21.15.123:3300/0 conn(0x7f1e640cc410 0x7f1e6415bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 >> v1:172.21.15.123:6789/0 conn(0x7f1e640b23d0 legacy=0x7f1e64067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64068630 con 0x7f1e6415cc40 2023-09-27T23:52:51.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f1e54001470 con 0x7f1e6415cc40 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 >> v1:172.21.15.64:6789/0 conn(0x7f1e6415cc40 legacy=0x7f1e6415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e44003a60 0x7f1e44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e44004810 0x7f1e44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e440051c0 0x7f1e4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64068350 con 0x7f1e440051c0 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4400c0d0 con 0x7f1e44004810 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4400c290 con 0x7f1e44003a60 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e440051c0 0x7f1e4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e5affd700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e44004810 0x7f1e44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 -- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e44003a60 msgr2=0x7f1e44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e44003a60 0x7f1e44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 -- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e44004810 msgr2=0x7f1e44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e44004810 0x7f1e44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 -- 172.21.15.64:0/2448310524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44003570 con 0x7f1e440051c0 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 -- 172.21.15.64:0/2448310524 reap_dead start 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e63fff700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e440051c0 0x7f1e4400ba60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1e640cc830 tx=0x7f1e5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dcb4a08c3dc4c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00ea90 con 0x7f1e440051c0 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00f040 con 0x7f1e440051c0 2023-09-27T23:52:51.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e62ffd700 1 -- 172.21.15.64:0/2448310524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00ee40 con 0x7f1e440051c0 2023-09-27T23:52:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 -- 172.21.15.64:0/2448310524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e440051c0 msgr2=0x7f1e4400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e440051c0 0x7f1e4400ba60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1e640cc830 tx=0x7f1e5c00b040 comp rx=0 tx=0).stop 2023-09-27T23:52:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 -- 172.21.15.64:0/2448310524 shutdown_connections 2023-09-27T23:52:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 --2- 172.21.15.64:0/2448310524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e440051c0 0x7f1e4400ba60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 -- 172.21.15.64:0/2448310524 >> 172.21.15.64:0/2448310524 conn(0x7f1e64069b90 msgr2=0x7f1e640cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 -- 172.21.15.64:0/2448310524 shutdown_connections 2023-09-27T23:52:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.936+0000 7f1e68d60700 1 -- 172.21.15.64:0/2448310524 wait complete. 2023-09-27T23:52:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 Processor -- start 2023-09-27T23:52:51.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 -- start start 2023-09-27T23:52:51.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e440051c0 0x7f1e64169090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e640cdb70 0x7f1e6416b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 0x7f1e6416be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6416c3d0 con 0x7f1e640cdfe0 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e640c8270 con 0x7f1e440051c0 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e640c7940 con 0x7f1e640cdb70 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 0x7f1e6416be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 0x7f1e6416be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35470/0 (socket says 172.21.15.64:35470) 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 -- 172.21.15.64:0/4265827665 learned_addr learned my addr 172.21.15.64:0/4265827665 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e5affd700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e640cdb70 0x7f1e6416b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e637fe700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e440051c0 0x7f1e64169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 -- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e640cdb70 msgr2=0x7f1e6416b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e640cdb70 0x7f1e6416b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 -- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e440051c0 msgr2=0x7f1e64169090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e440051c0 0x7f1e64169090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 -- 172.21.15.64:0/4265827665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e640c7c20 con 0x7f1e640cdfe0 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e63fff700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 0x7f1e6416be90 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00afd0 tx=0x7f1e5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c01d660 con 0x7f1e640cdfe0 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c022070 con 0x7f1e640cdfe0 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e640c7f00 con 0x7f1e640cdfe0 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.940+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c01da10 con 0x7f1e640cdfe0 2023-09-27T23:52:51.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.944+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e64157840 con 0x7f1e640cdfe0 2023-09-27T23:52:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.944+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e640ca310 con 0x7f1e640cdfe0 2023-09-27T23:52:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.944+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1e5c003020 con 0x7f1e640cdfe0 2023-09-27T23:52:51.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.944+0000 7f1e5bfff700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e48060a70 0x7f1e48062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:51.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.944+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f1e5c024070 con 0x7f1e640cdfe0 2023-09-27T23:52:51.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.944+0000 7f1e637fe700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e48060a70 0x7f1e48062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:51.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.948+0000 7f1e637fe700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e48060a70 0x7f1e48062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e54000c00 tx=0x7f1e54001a10 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:51.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:51.952+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f1e640ca310 con 0x7f1e640cdfe0 2023-09-27T23:52:52.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.112+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1e640ca310 con 0x7f1e640cdfe0 2023-09-27T23:52:52.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e5bfff700 1 -- 172.21.15.64:0/4265827665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e640ca310 con 0x7f1e640cdfe0 2023-09-27T23:52:52.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e48060a70 msgr2=0x7f1e48062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:52.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e48060a70 0x7f1e48062f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e54000c00 tx=0x7f1e54001a10 comp rx=0 tx=0).stop 2023-09-27T23:52:52.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 msgr2=0x7f1e6416be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:52.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 0x7f1e6416be90 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00afd0 tx=0x7f1e5c019000 comp rx=0 tx=0).stop 2023-09-27T23:52:52.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 shutdown_connections 2023-09-27T23:52:52.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1e48060a70 0x7f1e48062f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e640cdfe0 0x7f1e6416be90 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e640cdb70 0x7f1e6416b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 --2- 172.21.15.64:0/4265827665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e440051c0 0x7f1e64169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 >> 172.21.15.64:0/4265827665 conn(0x7f1e64069b90 msgr2=0x7f1e6415afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:52.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 shutdown_connections 2023-09-27T23:52:52.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.120+0000 7f1e68d60700 1 -- 172.21.15.64:0/4265827665 wait complete. 2023-09-27T23:52:52.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-09-27T23:52:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 Processor -- start 2023-09-27T23:52:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 -- start start 2023-09-27T23:52:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbc080ad980 0x7fbc080ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc080c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc08067da0 con 0x7fbc08155f20 2023-09-27T23:52:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc080c8260 con 0x7fbc080ad980 2023-09-27T23:52:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc0d448700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08068940 con 0x7fbc080c78e0 2023-09-27T23:52:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbc080ad980 0x7fbc080ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbc080ad980 0x7fbc080ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35474/0 (socket says 172.21.15.64:35474) 2023-09-27T23:52:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 -- 172.21.15.64:0/2729429395 learned_addr learned my addr 172.21.15.64:0/2729429395 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc077fe700 1 --2- 172.21.15.64:0/2729429395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc080c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 -- 172.21.15.64:0/2729429395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 msgr2=0x7fbc080c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 --2- 172.21.15.64:0/2729429395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc080c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 -- 172.21.15.64:0/2729429395 >> v1:172.21.15.123:6789/0 conn(0x7fbc08155f20 legacy=0x7fbc08068230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 -- 172.21.15.64:0/2729429395 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08067190 con 0x7fbc080ad980 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc067fc700 1 --2- 172.21.15.64:0/2729429395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc080ad980 0x7fbc080ade00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009f20 tx=0x7fbbf8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b529073f5c945ddf server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc05ffb700 1 -- 172.21.15.64:0/2729429395 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800e070 con 0x7fbc080ad980 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc05ffb700 1 -- 172.21.15.64:0/2729429395 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf800b9d0 con 0x7fbc080ad980 2023-09-27T23:52:52.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.244+0000 7fbc05ffb700 1 -- 172.21.15.64:0/2729429395 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80092e0 con 0x7fbc080ad980 2023-09-27T23:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 -- 172.21.15.64:0/2729429395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc080ad980 msgr2=0x7fbc080ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 --2- 172.21.15.64:0/2729429395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc080ad980 0x7fbc080ade00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009f20 tx=0x7fbbf8009840 comp rx=0 tx=0).stop 2023-09-27T23:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 -- 172.21.15.64:0/2729429395 shutdown_connections 2023-09-27T23:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 --2- 172.21.15.64:0/2729429395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc080c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 --2- 172.21.15.64:0/2729429395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc080ad980 0x7fbc080ade00 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 -- 172.21.15.64:0/2729429395 >> 172.21.15.64:0/2729429395 conn(0x7fbc080cc690 msgr2=0x7fbc080c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 -- 172.21.15.64:0/2729429395 shutdown_connections 2023-09-27T23:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 -- 172.21.15.64:0/2729429395 wait complete. 2023-09-27T23:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 Processor -- start 2023-09-27T23:52:52.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.248+0000 7fbc0d448700 1 -- start start 2023-09-27T23:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc080ad980 0x7fbc08169a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc0815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 0x7fbc080b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080c0250 con 0x7fbc08155f20 2023-09-27T23:52:52.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080bf640 con 0x7fbc080ad980 2023-09-27T23:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080bf920 con 0x7fbc080c78e0 2023-09-27T23:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 0x7fbc080b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc080ad980 0x7fbc08169a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc067fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc0815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 0x7fbc080b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35490/0 (socket says 172.21.15.64:35490) 2023-09-27T23:52:52.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 -- 172.21.15.64:0/1835714731 learned_addr learned my addr 172.21.15.64:0/1835714731 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 -- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 msgr2=0x7fbc0815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc0815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 -- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc080ad980 msgr2=0x7fbc08169a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:52.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc080ad980 0x7fbc08169a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 -- 172.21.15.64:0/1835714731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8009ae0 con 0x7fbc08155f20 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc06ffd700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc080ad980 0x7fbc08169a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc077fe700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 0x7fbc080b0c40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000ea50 tx=0x7fbc000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000ee90 con 0x7fbc08155f20 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0000c850 con 0x7fbc08155f20 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc080bfc00 con 0x7fbc08155f20 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000cc10 con 0x7fbc08155f20 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc08156c40 con 0x7fbc08155f20 2023-09-27T23:52:52.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.252+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc080c0940 con 0x7fbc08155f20 2023-09-27T23:52:52.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.256+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fbc00013470 con 0x7fbc08155f20 2023-09-27T23:52:52.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.256+0000 7fbbeeffd700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbbf4060aa0 0x7fbbf4062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.256+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fbc0007d210 con 0x7fbc08155f20 2023-09-27T23:52:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.256+0000 7fbc06ffd700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbbf4060aa0 0x7fbbf4062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:52.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.256+0000 7fbc06ffd700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbbf4060aa0 0x7fbbf4062f50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc000c00 tx=0x7fbbfc009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:52.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.260+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbc080c0940 con 0x7fbc08155f20 2023-09-27T23:52:52.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:52.480+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fbc080c0940 con 0x7fbc08155f20 2023-09-27T23:52:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.176+0000 7fbbeeffd700 1 -- 172.21.15.64:0/1835714731 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v1385) v1 ==== 155+0+0 (secure 0 0 0) 0x7fbc080c0940 con 0x7fbc08155f20 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbbf4060aa0 msgr2=0x7fbbf4062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbbf4060aa0 0x7fbbf4062f50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc000c00 tx=0x7fbbfc009040 comp rx=0 tx=0).stop 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 msgr2=0x7fbc080b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 0x7fbc080b0c40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000ea50 tx=0x7fbc000091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 shutdown_connections 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbbf4060aa0 0x7fbbf4062f50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc08155f20 0x7fbc080b0c40 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc080c78e0 0x7fbc0815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 --2- 172.21.15.64:0/1835714731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc080ad980 0x7fbc08169a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 >> 172.21.15.64:0/1835714731 conn(0x7fbc080cc690 msgr2=0x7fbc0806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:53.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 shutdown_connections 2023-09-27T23:52:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.184+0000 7fbc0d448700 1 -- 172.21.15.64:0/1835714731 wait complete. 2023-09-27T23:52:53.205 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-09-27T23:52:53.205 INFO:tasks.cephfs_test_runner: 2023-09-27T23:52:53.205 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-27T23:52:53.205 INFO:tasks.cephfs_test_runner:Ran 10 tests in 2305.776s 2023-09-27T23:52:53.205 INFO:tasks.cephfs_test_runner: 2023-09-27T23:52:53.205 INFO:tasks.cephfs_test_runner:OK 2023-09-27T23:52:53.205 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-09-27T23:52:53.214 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-27T23:52:53.221 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-27T23:52:53.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.222 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-27T23:52:53.229 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-27T23:52:53.229 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-27T23:52:53.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-27T23:52:53.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-27T23:52:53.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-09-27T23:52:53.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:52:53.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:52:53.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.453 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-09-27T23:52:53.461 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-09-27T23:52:53.461 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-27T23:52:53.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-27T23:52:53.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-27T23:52:53.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-27T23:52:53.555 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-09-27T23:52:53.697 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-27T23:52:53.706 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-09-27T23:52:53.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.816+0000 7fe4a57f3700 1 Processor -- start 2023-09-27T23:52:53.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.816+0000 7fe4a57f3700 1 -- start start 2023-09-27T23:52:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4a015c1f0 0x7fe4a00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:53.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006b3c0 0x7fe4a006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a01592e0 con 0x7fe4a015c5c0 2023-09-27T23:52:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a006bd60 con 0x7fe4a015c1f0 2023-09-27T23:52:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00c6e90 con 0x7fe4a006b3c0 2023-09-27T23:52:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4a015c1f0 0x7fe4a00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4a015c1f0 0x7fe4a00c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35502/0 (socket says 172.21.15.64:35502) 2023-09-27T23:52:53.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 -- 172.21.15.64:0/3937290550 learned_addr learned my addr 172.21.15.64:0/3937290550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a4ff2700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006b3c0 0x7fe4a006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 -- 172.21.15.64:0/3937290550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006b3c0 msgr2=0x7fe4a006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006b3c0 0x7fe4a006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 -- 172.21.15.64:0/3937290550 >> v1:172.21.15.123:6789/0 conn(0x7fe4a015c5c0 legacy=0x7fe4a0159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 -- 172.21.15.64:0/3937290550 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a00c7170 con 0x7fe4a015c1f0 2023-09-27T23:52:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a4ff2700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006b3c0 0x7fe4a006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49ffff700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00c6820 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe4940099b0 tx=0x7fe494009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3291d4703dddb131 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49effd700 1 -- 172.21.15.64:0/3937290550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400b860 con 0x7fe4a015c1f0 2023-09-27T23:52:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49effd700 1 -- 172.21.15.64:0/3937290550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49400b9c0 con 0x7fe4a015c1f0 2023-09-27T23:52:53.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe49effd700 1 -- 172.21.15.64:0/3937290550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400bd60 con 0x7fe4a015c1f0 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3937290550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 msgr2=0x7fe4a00c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00c6820 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe4940099b0 tx=0x7fe494009250 comp rx=0 tx=0).stop 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3937290550 shutdown_connections 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a006b3c0 0x7fe4a006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3937290550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00c6820 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3937290550 >> 172.21.15.64:0/3937290550 conn(0x7fe4a006a0d0 msgr2=0x7fe4a006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3937290550 shutdown_connections 2023-09-27T23:52:53.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.820+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3937290550 wait complete. 2023-09-27T23:52:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.824+0000 7fe4a57f3700 1 Processor -- start 2023-09-27T23:52:53.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.824+0000 7fe4a57f3700 1 -- start start 2023-09-27T23:52:53.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a006b3c0 0x7fe4a01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a00b1d90 0x7fe4a00adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00c0e40 con 0x7fe4a015c1f0 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00b13c0 con 0x7fe4a006b3c0 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00c0b60 con 0x7fe4a00b1d90 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35512/0 (socket says 172.21.15.64:35512) 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 -- 172.21.15.64:0/3357305080 learned_addr learned my addr 172.21.15.64:0/3357305080 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a4ff2700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a00b1d90 0x7fe4a00adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:53.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49ffff700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a006b3c0 0x7fe4a01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 -- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a00b1d90 msgr2=0x7fe4a00adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a00b1d90 0x7fe4a00adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 -- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a006b3c0 msgr2=0x7fe4a01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a006b3c0 0x7fe4a01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 -- 172.21.15.64:0/3357305080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940094f0 con 0x7fe4a015c1f0 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49ffff700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a006b3c0 0x7fe4a01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:53.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe49f7fe700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00b1850 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c4e0 tx=0x7fe49000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490011590 con 0x7fe4a015c1f0 2023-09-27T23:52:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490011710 con 0x7fe4a015c1f0 2023-09-27T23:52:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a00b0d70 con 0x7fe4a015c1f0 2023-09-27T23:52:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490011ad0 con 0x7fe4a015c1f0 2023-09-27T23:52:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a0158d00 con 0x7fe4a015c1f0 2023-09-27T23:52:53.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a006c520 con 0x7fe4a015c1f0 2023-09-27T23:52:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.828+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe49001b3b0 con 0x7fe4a015c1f0 2023-09-27T23:52:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.832+0000 7fe487fff700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe48c060aa0 0x7fe48c062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.832+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fe49007c340 con 0x7fe4a015c1f0 2023-09-27T23:52:53.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.832+0000 7fe49ffff700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe48c060aa0 0x7fe48c062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:53.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.832+0000 7fe49ffff700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe48c060aa0 0x7fe48c062f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4940099b0 tx=0x7fe494017000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:53.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:53.836+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe4a006c520 con 0x7fe4a015c1f0 2023-09-27T23:52:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.004+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4a006ed10 con 0x7fe4a015c1f0 2023-09-27T23:52:54.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.008+0000 7fe487fff700 1 -- 172.21.15.64:0/3357305080 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+14936 (secure 0 0 0) 0x7fe4a006ed10 con 0x7fe4a015c1f0 2023-09-27T23:52:54.012 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:54.012 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":194,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:51.153335+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2551836643":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3550075260":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6835/1929792685":"2023-09-28T23:52:47.067586+0000","172.21.15.123:0/3834368324":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3658378865":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/855337008":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6834/1929792685":"2023-09-28T23:52:47.067586+0000","172.21.15.123:0/1582733572":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3573447607":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1334338128":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6832/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6833/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/2413747055":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3774540904":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3310607164":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/4028139068":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1447825948":"2023-09-28T23:52:37.175204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe48c060aa0 msgr2=0x7fe48c062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe48c060aa0 0x7fe48c062f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4940099b0 tx=0x7fe494017000 comp rx=0 tx=0).stop 2023-09-27T23:52:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 msgr2=0x7fe4a00b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00b1850 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c4e0 tx=0x7fe49000c7f0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 shutdown_connections 2023-09-27T23:52:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe48c060aa0 0x7fe48c062f50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4a00b1d90 0x7fe4a00adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4a015c1f0 0x7fe4a00b1850 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 --2- 172.21.15.64:0/3357305080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a006b3c0 0x7fe4a01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 >> 172.21.15.64:0/3357305080 conn(0x7fe4a006a0d0 msgr2=0x7fe4a015a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 shutdown_connections 2023-09-27T23:52:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.012+0000 7fe4a57f3700 1 -- 172.21.15.64:0/3357305080 wait complete. 2023-09-27T23:52:54.031 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-27T23:52:54.031 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:54.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 Processor -- start 2023-09-27T23:52:54.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 -- start start 2023-09-27T23:52:54.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb34067060 0x7fcb34067430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb340c8e60 0x7fcb340ab920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb340c92b0 0x7fcb340abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3406b1a0 con 0x7fcb340c92b0 2023-09-27T23:52:54.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb340ae350 con 0x7fcb34067060 2023-09-27T23:52:54.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34067970 con 0x7fcb340c8e60 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb340c92b0 0x7fcb340abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb340c92b0 0x7fcb340abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35514/0 (socket says 172.21.15.64:35514) 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 -- 172.21.15.64:0/4257970968 learned_addr learned my addr 172.21.15.64:0/4257970968 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb32d9d700 1 --2- 172.21.15.64:0/4257970968 >> v2:172.21.15.123:3300/0 conn(0x7fcb34067060 0x7fcb34067430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3259c700 1 --2- 172.21.15.64:0/4257970968 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb340c8e60 0x7fcb340ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 -- 172.21.15.64:0/4257970968 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb340c8e60 msgr2=0x7fcb340ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 --2- 172.21.15.64:0/4257970968 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb340c8e60 0x7fcb340ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 -- 172.21.15.64:0/4257970968 >> v2:172.21.15.123:3300/0 conn(0x7fcb34067060 msgr2=0x7fcb34067430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:54.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 --2- 172.21.15.64:0/4257970968 >> v2:172.21.15.123:3300/0 conn(0x7fcb34067060 0x7fcb34067430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 -- 172.21.15.64:0/4257970968 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3406a900 con 0x7fcb340c92b0 2023-09-27T23:52:54.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3359e700 1 --2- 172.21.15.64:0/4257970968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb340abe60 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fcb34069500 tx=0x7fcb24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d77a30b7d05ccc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb31d9b700 1 -- 172.21.15.64:0/4257970968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24009e70 con 0x7fcb340c92b0 2023-09-27T23:52:54.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb31d9b700 1 -- 172.21.15.64:0/4257970968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2400b860 con 0x7fcb340c92b0 2023-09-27T23:52:54.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb31d9b700 1 -- 172.21.15.64:0/4257970968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400bc00 con 0x7fcb340c92b0 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 -- 172.21.15.64:0/4257970968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 msgr2=0x7fcb340abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4257970968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb340abe60 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fcb34069500 tx=0x7fcb24009250 comp rx=0 tx=0).stop 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 -- 172.21.15.64:0/4257970968 shutdown_connections 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4257970968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb340abe60 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4257970968 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb340c8e60 0x7fcb340ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.136+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4257970968 >> v2:172.21.15.123:3300/0 conn(0x7fcb34067060 0x7fcb34067430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.140+0000 7fcb3905c700 1 -- 172.21.15.64:0/4257970968 >> 172.21.15.64:0/4257970968 conn(0x7fcb340cc690 msgr2=0x7fcb340c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.140+0000 7fcb3905c700 1 -- 172.21.15.64:0/4257970968 shutdown_connections 2023-09-27T23:52:54.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.140+0000 7fcb3905c700 1 -- 172.21.15.64:0/4257970968 wait complete. 2023-09-27T23:52:54.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.140+0000 7fcb3905c700 1 Processor -- start 2023-09-27T23:52:54.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.140+0000 7fcb3905c700 1 -- start start 2023-09-27T23:52:54.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb34067060 0x7fcb34159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb340c8e60 0x7fcb34158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb34158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb341582d0 con 0x7fcb340c92b0 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340c3ba0 con 0x7fcb340c8e60 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340c4140 con 0x7fcb34067060 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb34158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb34158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35524/0 (socket says 172.21.15.64:35524) 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 -- 172.21.15.64:0/4022686412 learned_addr learned my addr 172.21.15.64:0/4022686412 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3259c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb340c8e60 0x7fcb34158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb32d9d700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb34067060 0x7fcb34159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 -- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb34067060 msgr2=0x7fcb34159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb34067060 0x7fcb34159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 -- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb340c8e60 msgr2=0x7fcb34158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb340c8e60 0x7fcb34158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 -- 172.21.15.64:0/4022686412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240094f0 con 0x7fcb340c92b0 2023-09-27T23:52:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3259c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb340c8e60 0x7fcb34158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb32d9d700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb34067060 0x7fcb34159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:54.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3359e700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb34158ab0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fcb3406bab0 tx=0x7fcb24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400baa0 con 0x7fcb340c92b0 2023-09-27T23:52:54.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24025420 con 0x7fcb340c92b0 2023-09-27T23:52:54.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240257e0 con 0x7fcb340c92b0 2023-09-27T23:52:54.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb341591a0 con 0x7fcb340c92b0 2023-09-27T23:52:54.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.144+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb340c3dd0 con 0x7fcb340c92b0 2023-09-27T23:52:54.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.148+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fcb24023070 con 0x7fcb340c92b0 2023-09-27T23:52:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.148+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34071110 con 0x7fcb340c92b0 2023-09-27T23:52:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.148+0000 7fcb2affd700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb14060aa0 0x7fcb14062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.148+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fcb2407e950 con 0x7fcb340c92b0 2023-09-27T23:52:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.148+0000 7fcb32d9d700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb14060aa0 0x7fcb14062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.148+0000 7fcb32d9d700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb14060aa0 0x7fcb14062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb340c5c60 tx=0x7fcb2c00b040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.156+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fcb2404e950 con 0x7fcb340c92b0 2023-09-27T23:52:54.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.308+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb340c4280 con 0x7fcb14060aa0 2023-09-27T23:52:54.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.316+0000 7fcb2affd700 1 -- 172.21.15.64:0/4022686412 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71124 (secure 0 0 0) 0x7fcb340c4280 con 0x7fcb14060aa0 2023-09-27T23:52:54.322 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.316+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb14060aa0 msgr2=0x7fcb14062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.316+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb14060aa0 0x7fcb14062f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb340c5c60 tx=0x7fcb2c00b040 comp rx=0 tx=0).stop 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.316+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 msgr2=0x7fcb34158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.316+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb34158ab0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fcb3406bab0 tx=0x7fcb24017000 comp rx=0 tx=0).stop 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 shutdown_connections 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fcb14060aa0 0x7fcb14062f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb340c92b0 0x7fcb34158ab0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb340c8e60 0x7fcb34158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 --2- 172.21.15.64:0/4022686412 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb34067060 0x7fcb34159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 >> 172.21.15.64:0/4022686412 conn(0x7fcb340cc690 msgr2=0x7fcb34153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 shutdown_connections 2023-09-27T23:52:54.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.320+0000 7fcb3905c700 1 -- 172.21.15.64:0/4022686412 wait complete. 2023-09-27T23:52:54.325 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:54.338 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-09-27T23:52:54.160974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262504,"kb_used_data":20072,"kb_used_omap":0,"kb_used_meta":242368,"kb_avail":754712216,"statfs":{"total":773094113280,"available":772825309184,"internally_reserved":0,"allocated":20553728,"data_stored":11397604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":248184832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-667624,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-285,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.996508"},"pg_stats":[{"pgid":"1.0","version":"191'348","reported_seq":1134,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501255+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:50.501255+0000","last_peered":"2023-09-27T23:52:50.501255+0000","last_clean":"2023-09-27T23:52:50.501255+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:50.501255+0000","last_undegraded":"2023-09-27T23:52:50.501255+0000","last_fullsized":"2023-09-27T23:52:50.501255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":348,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.162616+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:50.162616+0000","last_peered":"2023-09-27T23:52:50.162616+0000","last_clean":"2023-09-27T23:52:50.162616+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:50.162616+0000","last_undegraded":"2023-09-27T23:52:50.162616+0000","last_fullsized":"2023-09-27T23:52:50.162616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501132+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:50.501132+0000","last_peered":"2023-09-27T23:52:50.501132+0000","last_clean":"2023-09-27T23:52:50.501132+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:50.501132+0000","last_undegraded":"2023-09-27T23:52:50.501132+0000","last_fullsized":"2023-09-27T23:52:50.501132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.498720+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:50.498720+0000","last_peered":"2023-09-27T23:52:50.498720+0000","last_clean":"2023-09-27T23:52:50.498720+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:50.498720+0000","last_undegraded":"2023-09-27T23:52:50.498720+0000","last_fullsized":"2023-09-27T23:52:50.498720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.161617+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:50.161617+0000","last_peered":"2023-09-27T23:52:50.161617+0000","last_clean":"2023-09-27T23:52:50.161617+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:50.161617+0000","last_undegraded":"2023-09-27T23:52:50.161617+0000","last_fullsized":"2023-09-27T23:52:50.161617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-09-27T23:52:47.076912+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:47.076912+0000","last_peered":"2023-09-27T23:52:47.076912+0000","last_clean":"2023-09-27T23:52:47.076912+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:47.076912+0000","last_undegraded":"2023-09-27T23:52:47.076912+0000","last_fullsized":"2023-09-27T23:52:47.076912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.500922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:50.500922+0000","last_peered":"2023-09-27T23:52:50.500922+0000","last_clean":"2023-09-27T23:52:50.500922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:50.500922+0000","last_undegraded":"2023-09-27T23:52:50.500922+0000","last_fullsized":"2023-09-27T23:52:50.500922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-09-27T23:52:47.078866+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:47.078866+0000","last_peered":"2023-09-27T23:52:47.078866+0000","last_clean":"2023-09-27T23:52:47.078866+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:47.078866+0000","last_undegraded":"2023-09-27T23:52:47.078866+0000","last_fullsized":"2023-09-27T23:52:47.078866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.239142+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:50.239142+0000","last_peered":"2023-09-27T23:52:50.239142+0000","last_clean":"2023-09-27T23:52:50.239142+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:50.239142+0000","last_undegraded":"2023-09-27T23:52:50.239142+0000","last_fullsized":"2023-09-27T23:52:50.239142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":34432,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35258368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4810000000000001,"5min":1.569,"15min":1.569},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94999999999999996,"15min":0.93100000000000005},"min":{"1min":0.58199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.1020000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.159,"5min":1.3100000000000001,"15min":1.3360000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96899999999999997,"15min":0.97399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2949999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.66600000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4079999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97199999999999998,"15min":0.96899999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.1870000000000001,"5min":1.341,"15min":1.341},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2549999999999999,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.78100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2629999999999999,"5min":1.371,"15min":1.3879999999999999},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.3919999999999999,"5min":1.821,"15min":1.821},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.619,"5min":3.02,"15min":3.02},"last":1.171}]},{"osd":5,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0629999999999999,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3759999999999999,"5min":1.484,"15min":1.484},"last":1.137},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.103,"15min":1.113},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.028}]},{"osd":6,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.127,"15min":1.1419999999999999},"min":{"1min":0.80200000000000005,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.147,"15min":1.1479999999999999},"min":{"1min":0.81699999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5149999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.4319999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30332,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341508,"statfs":{"total":96636764160,"available":96605704192,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":1.3149999999999999,"5min":1.792,"15min":1.792},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.538,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.157,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.123}]},{"osd":2,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.629,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.494,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1919999999999999,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":1.2969999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.405,"5min":1.645,"15min":1.645},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.94999999999999996},"min":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.3340000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.872}]},{"osd":4,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.88500000000000001},"min":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.5720000000000001,"5min":1.7,"15min":1.7},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.6399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.95899999999999996,"15min":0.95999999999999996},"min":{"1min":0.73299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.401,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.242},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1599999999999999,"15min":1.1459999999999999},"min":{"1min":0.89100000000000001,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.71,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1699999999999999,"15min":1.1279999999999999},"min":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.74299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.5840000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":37888,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38797312},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.89700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.246,"5min":10.058999999999999,"15min":10.058999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0629999999999999,"15min":0.95899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":1.042}]},{"osd":1,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96099999999999997,"15min":0.93000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2929999999999999,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92600000000000005,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.159,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.077}]},{"osd":2,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"max":{"1min":1.2,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1200000000000001,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.85799999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":0.98999999999999999},"min":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.2470000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.997,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.747,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.1970000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.996}]},{"osd":4,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":0.88800000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.107,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.222,"5min":1.45,"15min":1.4510000000000001},"last":1.165}]},{"osd":6,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2529999999999999,"5min":4.157,"15min":4.157},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.341,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.194}]},{"osd":7,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.208,"15min":1.171},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.54700000000000004},"max":{"1min":1.4830000000000001,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.3420000000000001,"15min":1.2589999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.52,"5min":14.119,"15min":14.119},"last":1.232}]}]},{"osd":4,"up_from":15,"seq":64424509907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3660000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.2769999999999999,"5min":1.893,"15min":2.2770000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.3779999999999999,"5min":1.746,"15min":1.746},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95699999999999996,"15min":0.95599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.339,"5min":1.758,"15min":2.1110000000000002},"last":0.81499999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.95699999999999996,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2649999999999999,"5min":1.659,"15min":1.659},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98799999999999999,"15min":0.96199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.2789999999999999,"5min":1.966,"15min":2.056},"last":0.91600000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.331,"5min":1.998,"15min":2.032},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.079,"15min":1.0469999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58199999999999996},"max":{"1min":1.5,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.095}]},{"osd":5,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45500000000000002},"max":{"1min":1.4610000000000001,"5min":1.556,"15min":1.9930000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.419,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.046,"15min":1.0529999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1.052},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.181,"15min":1.163},"min":{"1min":0.749,"5min":0.749,"15min":0.495},"max":{"1min":1.554,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1299999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.091,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66100000000000003},"max":{"1min":1.6279999999999999,"5min":2.069,"15min":2.069},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.149,"15min":1.1719999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2170000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.337}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.2789999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.22},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.248},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.153}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.093},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1259999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30860,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.238,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.628},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54800000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0509999999999999,"5min":1.2250000000000001,"15min":1.345},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.014,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.25,"5min":1.383,"15min":1.383},"last":0.79400000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.89400000000000002,"15min":0.93300000000000005},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":1.0740000000000001},"min":{"1min":0.60899999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3280000000000001,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0740000000000001,"15min":1.069},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4330000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.046}]},{"osd":5,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.04,"15min":1.044},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.349,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":1.321},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1379999999999999,"15min":1.1659999999999999},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4399999999999999,"5min":1.5860000000000001,"15min":1.8},"last":1.1180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.151,"15min":1.157},"min":{"1min":0.85199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4810000000000001,"5min":1.5780000000000001,"15min":1.718},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.167,"15min":1.1779999999999999},"min":{"1min":0.82399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3879999999999999,"5min":1.6439999999999999,"15min":1.736},"last":1.1419999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.516,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3089999999999999,"5min":1.607,"15min":1.8129999999999999},"last":1.24}]}]},{"osd":1,"up_from":14,"seq":60129542613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28860,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":26496,"kb_avail":94342980,"statfs":{"total":96636764160,"available":96607211520,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27131904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.226},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.091}]}]},{"osd":0,"up_from":15,"seq":64424509908,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":29248,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":3006464,"data_stored":1864041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29949952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83699999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1950000000000001,"5min":1.198,"15min":1.2470000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0960000000000001,"5min":1.169,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.125,"5min":1.288,"15min":1.863},"last":1.0149999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89400000000000002,"15min":0.90100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.4390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.681},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0449999999999999,"15min":1.0569999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3360000000000001,"5min":1.516,"15min":1.5249999999999999},"last":1.3360000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.095,"15min":1.1080000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3700000000000001,"5min":1.444,"15min":1.512},"last":1.369},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5629999999999999,"15min":1.736},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.173,"15min":1.1850000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.746},"last":1.425},{"interface":"front","average":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5600000000000001,"5min":1.722,"15min":2.0510000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.248,"15min":1.27},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.544,"5min":1.786,"15min":2.0030000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2330000000000001,"15min":1.252},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.706,"5min":1.806,"15min":1.806},"last":1.4750000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:54.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:54.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5d76700 1 Processor -- start 2023-09-27T23:52:54.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5d76700 1 -- start start 2023-09-27T23:52:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5d76700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8de0155bc0 0x7f8de0159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5d76700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de00c9320 con 0x7f8de015a3c0 2023-09-27T23:52:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5d76700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de0067b10 con 0x7f8de00c8010 2023-09-27T23:52:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5d76700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de0067df0 con 0x7f8de0155bc0 2023-09-27T23:52:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5575700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8de015a3c0 0x7f8de00ad940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41534/0 (socket says 172.21.15.64:41534) 2023-09-27T23:52:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de5575700 1 -- 172.21.15.64:0/1450113525 learned_addr learned my addr 172.21.15.64:0/1450113525 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:54.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de4d74700 1 --2- 172.21.15.64:0/1450113525 >> v2:172.21.15.123:3300/0 conn(0x7f8de0155bc0 0x7f8de0159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272081584 0 0) 0x7f8de00c9320 con 0x7f8de015a3c0 2023-09-27T23:52:54.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc8003570 con 0x7f8de015a3c0 2023-09-27T23:52:54.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370627629 0 0) 0x7f8de0067b10 con 0x7f8de00c8010 2023-09-27T23:52:54.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de00c9320 con 0x7f8de00c8010 2023-09-27T23:52:54.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de4d74700 1 -- 172.21.15.64:0/1450113525 >> v1:172.21.15.123:6789/0 conn(0x7f8de00c8010 legacy=0x7f8de0159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de4d74700 1 -- 172.21.15.64:0/1450113525 >> v1:172.21.15.64:6789/0 conn(0x7f8de015a3c0 legacy=0x7f8de00ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.444+0000 7f8de4d74700 1 -- 172.21.15.64:0/1450113525 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00680d0 con 0x7f8de0155bc0 2023-09-27T23:52:54.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de4d74700 1 --2- 172.21.15.64:0/1450113525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0159770 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40098e0 tx=0x7f8dd40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=321677aff9e6e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4011630 con 0x7f8de0155bc0 2023-09-27T23:52:54.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd4011790 con 0x7f8de0155bc0 2023-09-27T23:52:54.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8ddf7fe700 1 -- 172.21.15.64:0/1450113525 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4011b30 con 0x7f8de0155bc0 2023-09-27T23:52:54.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- 172.21.15.64:0/1450113525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 msgr2=0x7f8de0159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 --2- 172.21.15.64:0/1450113525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0159770 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40098e0 tx=0x7f8dd40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- 172.21.15.64:0/1450113525 shutdown_connections 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 --2- 172.21.15.64:0/1450113525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0159770 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- 172.21.15.64:0/1450113525 >> 172.21.15.64:0/1450113525 conn(0x7f8de00cc530 msgr2=0x7f8de00c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- 172.21.15.64:0/1450113525 shutdown_connections 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- 172.21.15.64:0/1450113525 wait complete. 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 Processor -- start 2023-09-27T23:52:54.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- start start 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 0x7f8de00c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de015a3c0 0x7f8de00ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00c3b30 con 0x7f8de00c8010 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00a94e0 con 0x7f8de0155bc0 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de5d76700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00a97c0 con 0x7f8de015a3c0 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de4d74700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 0x7f8de00c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de4d74700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 0x7f8de00c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35536/0 (socket says 172.21.15.64:35536) 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8de4d74700 1 -- 172.21.15.64:0/2121205152 learned_addr learned my addr 172.21.15.64:0/2121205152 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.448+0000 7f8ddffff700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de5575700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de015a3c0 0x7f8de00ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de4d74700 1 -- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de015a3c0 msgr2=0x7f8de00ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de4d74700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de015a3c0 0x7f8de00ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de4d74700 1 -- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 msgr2=0x7f8de0149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de4d74700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de4d74700 1 -- 172.21.15.64:0/2121205152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4009420 con 0x7f8de00c8010 2023-09-27T23:52:54.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de5575700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de015a3c0 0x7f8de00ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de4d74700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 0x7f8de00c35f0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f8de006d380 tx=0x7f8dd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd401f070 con 0x7f8de00c8010 2023-09-27T23:52:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de00c2b10 con 0x7f8de00c8010 2023-09-27T23:52:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd4000bc0 con 0x7f8de00c8010 2023-09-27T23:52:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4025420 con 0x7f8de00c8010 2023-09-27T23:52:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de00c30d0 con 0x7f8de00c8010 2023-09-27T23:52:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de00c32f0 con 0x7f8de00c8010 2023-09-27T23:52:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8dd4023070 con 0x7f8de00c8010 2023-09-27T23:52:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8ddcff9700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8dcc060a70 0x7f8dcc062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.452+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f8dd407e4f0 con 0x7f8de00c8010 2023-09-27T23:52:54.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.456+0000 7f8ddffff700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8dcc060a70 0x7f8dcc062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.456+0000 7f8ddffff700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8dcc060a70 0x7f8dcc062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8de00cb5a0 tx=0x7f8dd0007f90 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.464+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8dd404e590 con 0x7f8de00c8010 2023-09-27T23:52:54.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.624+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8de00c2850 con 0x7f8dcc060a70 2023-09-27T23:52:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8ddcff9700 1 -- 172.21.15.64:0/2121205152 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71124 (secure 0 0 0) 0x7f8de00c2850 con 0x7f8dcc060a70 2023-09-27T23:52:54.637 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8dcc060a70 msgr2=0x7f8dcc062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8dcc060a70 0x7f8dcc062f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8de00cb5a0 tx=0x7f8dd0007f90 comp rx=0 tx=0).stop 2023-09-27T23:52:54.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 msgr2=0x7f8de00c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 0x7f8de00c35f0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f8de006d380 tx=0x7f8dd4017000 comp rx=0 tx=0).stop 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 shutdown_connections 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8dcc060a70 0x7f8dcc062f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de015a3c0 0x7f8de00ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de0155bc0 0x7f8de0149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 --2- 172.21.15.64:0/2121205152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8de00c8010 0x7f8de00c35f0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 >> 172.21.15.64:0/2121205152 conn(0x7f8de00cc530 msgr2=0x7f8de00ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 shutdown_connections 2023-09-27T23:52:54.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.636+0000 7f8de5d76700 1 -- 172.21.15.64:0/2121205152 wait complete. 2023-09-27T23:52:54.641 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:54.652 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-09-27T23:52:54.160974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262504,"kb_used_data":20072,"kb_used_omap":0,"kb_used_meta":242368,"kb_avail":754712216,"statfs":{"total":773094113280,"available":772825309184,"internally_reserved":0,"allocated":20553728,"data_stored":11397604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":248184832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-667624,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-285,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.996508"},"pg_stats":[{"pgid":"1.0","version":"191'348","reported_seq":1134,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501255+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:50.501255+0000","last_peered":"2023-09-27T23:52:50.501255+0000","last_clean":"2023-09-27T23:52:50.501255+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:50.501255+0000","last_undegraded":"2023-09-27T23:52:50.501255+0000","last_fullsized":"2023-09-27T23:52:50.501255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":348,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.162616+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:50.162616+0000","last_peered":"2023-09-27T23:52:50.162616+0000","last_clean":"2023-09-27T23:52:50.162616+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:50.162616+0000","last_undegraded":"2023-09-27T23:52:50.162616+0000","last_fullsized":"2023-09-27T23:52:50.162616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501132+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:50.501132+0000","last_peered":"2023-09-27T23:52:50.501132+0000","last_clean":"2023-09-27T23:52:50.501132+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:50.501132+0000","last_undegraded":"2023-09-27T23:52:50.501132+0000","last_fullsized":"2023-09-27T23:52:50.501132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.498720+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:50.498720+0000","last_peered":"2023-09-27T23:52:50.498720+0000","last_clean":"2023-09-27T23:52:50.498720+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:50.498720+0000","last_undegraded":"2023-09-27T23:52:50.498720+0000","last_fullsized":"2023-09-27T23:52:50.498720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.161617+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:50.161617+0000","last_peered":"2023-09-27T23:52:50.161617+0000","last_clean":"2023-09-27T23:52:50.161617+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:50.161617+0000","last_undegraded":"2023-09-27T23:52:50.161617+0000","last_fullsized":"2023-09-27T23:52:50.161617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-09-27T23:52:47.076912+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:47.076912+0000","last_peered":"2023-09-27T23:52:47.076912+0000","last_clean":"2023-09-27T23:52:47.076912+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:47.076912+0000","last_undegraded":"2023-09-27T23:52:47.076912+0000","last_fullsized":"2023-09-27T23:52:47.076912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.500922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:50.500922+0000","last_peered":"2023-09-27T23:52:50.500922+0000","last_clean":"2023-09-27T23:52:50.500922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:50.500922+0000","last_undegraded":"2023-09-27T23:52:50.500922+0000","last_fullsized":"2023-09-27T23:52:50.500922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-09-27T23:52:47.078866+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:47.078866+0000","last_peered":"2023-09-27T23:52:47.078866+0000","last_clean":"2023-09-27T23:52:47.078866+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:47.078866+0000","last_undegraded":"2023-09-27T23:52:47.078866+0000","last_fullsized":"2023-09-27T23:52:47.078866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.239142+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:50.239142+0000","last_peered":"2023-09-27T23:52:50.239142+0000","last_clean":"2023-09-27T23:52:50.239142+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:50.239142+0000","last_undegraded":"2023-09-27T23:52:50.239142+0000","last_fullsized":"2023-09-27T23:52:50.239142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":34432,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35258368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4810000000000001,"5min":1.569,"15min":1.569},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94999999999999996,"15min":0.93100000000000005},"min":{"1min":0.58199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.1020000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.159,"5min":1.3100000000000001,"15min":1.3360000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96899999999999997,"15min":0.97399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2949999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.66600000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4079999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97199999999999998,"15min":0.96899999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.1870000000000001,"5min":1.341,"15min":1.341},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2549999999999999,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.78100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2629999999999999,"5min":1.371,"15min":1.3879999999999999},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.3919999999999999,"5min":1.821,"15min":1.821},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.619,"5min":3.02,"15min":3.02},"last":1.171}]},{"osd":5,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0629999999999999,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3759999999999999,"5min":1.484,"15min":1.484},"last":1.137},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.103,"15min":1.113},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.028}]},{"osd":6,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.127,"15min":1.1419999999999999},"min":{"1min":0.80200000000000005,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.147,"15min":1.1479999999999999},"min":{"1min":0.81699999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5149999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.4319999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30332,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341508,"statfs":{"total":96636764160,"available":96605704192,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":1.3149999999999999,"5min":1.792,"15min":1.792},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.538,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.157,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.123}]},{"osd":2,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.629,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.494,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1919999999999999,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":1.2969999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.405,"5min":1.645,"15min":1.645},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.94999999999999996},"min":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.3340000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.872}]},{"osd":4,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.88500000000000001},"min":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.5720000000000001,"5min":1.7,"15min":1.7},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.6399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.95899999999999996,"15min":0.95999999999999996},"min":{"1min":0.73299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.401,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.242},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1599999999999999,"15min":1.1459999999999999},"min":{"1min":0.89100000000000001,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.71,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1699999999999999,"15min":1.1279999999999999},"min":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.74299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.5840000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":37888,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38797312},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.89700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.246,"5min":10.058999999999999,"15min":10.058999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0629999999999999,"15min":0.95899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":1.042}]},{"osd":1,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96099999999999997,"15min":0.93000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2929999999999999,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92600000000000005,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.159,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.077}]},{"osd":2,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"max":{"1min":1.2,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1200000000000001,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.85799999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":0.98999999999999999},"min":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.2470000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.997,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.747,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.1970000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.996}]},{"osd":4,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":0.88800000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.107,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.222,"5min":1.45,"15min":1.4510000000000001},"last":1.165}]},{"osd":6,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2529999999999999,"5min":4.157,"15min":4.157},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.341,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.194}]},{"osd":7,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.208,"15min":1.171},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.54700000000000004},"max":{"1min":1.4830000000000001,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.3420000000000001,"15min":1.2589999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.52,"5min":14.119,"15min":14.119},"last":1.232}]}]},{"osd":4,"up_from":15,"seq":64424509907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3660000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.2769999999999999,"5min":1.893,"15min":2.2770000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.3779999999999999,"5min":1.746,"15min":1.746},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95699999999999996,"15min":0.95599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.339,"5min":1.758,"15min":2.1110000000000002},"last":0.81499999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.95699999999999996,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2649999999999999,"5min":1.659,"15min":1.659},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98799999999999999,"15min":0.96199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.2789999999999999,"5min":1.966,"15min":2.056},"last":0.91600000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.331,"5min":1.998,"15min":2.032},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.079,"15min":1.0469999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58199999999999996},"max":{"1min":1.5,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.095}]},{"osd":5,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45500000000000002},"max":{"1min":1.4610000000000001,"5min":1.556,"15min":1.9930000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.419,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.046,"15min":1.0529999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1.052},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.181,"15min":1.163},"min":{"1min":0.749,"5min":0.749,"15min":0.495},"max":{"1min":1.554,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1299999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.091,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66100000000000003},"max":{"1min":1.6279999999999999,"5min":2.069,"15min":2.069},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.149,"15min":1.1719999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2170000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.337}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.2789999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.22},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.248},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.153}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.093},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1259999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30860,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.238,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.628},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54800000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0509999999999999,"5min":1.2250000000000001,"15min":1.345},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.014,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.25,"5min":1.383,"15min":1.383},"last":0.79400000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.89400000000000002,"15min":0.93300000000000005},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":1.0740000000000001},"min":{"1min":0.60899999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3280000000000001,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0740000000000001,"15min":1.069},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4330000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.046}]},{"osd":5,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.04,"15min":1.044},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.349,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":1.321},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1379999999999999,"15min":1.1659999999999999},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4399999999999999,"5min":1.5860000000000001,"15min":1.8},"last":1.1180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.151,"15min":1.157},"min":{"1min":0.85199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4810000000000001,"5min":1.5780000000000001,"15min":1.718},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.167,"15min":1.1779999999999999},"min":{"1min":0.82399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3879999999999999,"5min":1.6439999999999999,"15min":1.736},"last":1.1419999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.516,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3089999999999999,"5min":1.607,"15min":1.8129999999999999},"last":1.24}]}]},{"osd":1,"up_from":14,"seq":60129542613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28860,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":26496,"kb_avail":94342980,"statfs":{"total":96636764160,"available":96607211520,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27131904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.226},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.091}]}]},{"osd":0,"up_from":15,"seq":64424509908,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":29248,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":3006464,"data_stored":1864041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29949952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83699999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1950000000000001,"5min":1.198,"15min":1.2470000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0960000000000001,"5min":1.169,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.125,"5min":1.288,"15min":1.863},"last":1.0149999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89400000000000002,"15min":0.90100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.4390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.681},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0449999999999999,"15min":1.0569999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3360000000000001,"5min":1.516,"15min":1.5249999999999999},"last":1.3360000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.095,"15min":1.1080000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3700000000000001,"5min":1.444,"15min":1.512},"last":1.369},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5629999999999999,"15min":1.736},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.173,"15min":1.1850000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.746},"last":1.425},{"interface":"front","average":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5600000000000001,"5min":1.722,"15min":2.0510000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.248,"15min":1.27},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.544,"5min":1.786,"15min":2.0030000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2330000000000001,"15min":1.252},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.706,"5min":1.806,"15min":1.806},"last":1.4750000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:54.654 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-27T23:52:54.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:52:54.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 Processor -- start 2023-09-27T23:52:54.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 -- start start 2023-09-27T23:52:54.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f78142b00 0x7f8f780cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f780cf500 0x7f8f781594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f78067fc0 con 0x7f8f78142730 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f780673b0 con 0x7f8f78142b00 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7e40a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f78067690 con 0x7f8f780cf500 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f78142b00 0x7f8f780cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f78142b00 0x7f8f780cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53594/0 (socket says 172.21.15.64:53594) 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 -- 172.21.15.64:0/54496474 learned_addr learned my addr 172.21.15.64:0/54496474 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:54.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7dc09700 1 --2- 172.21.15.64:0/54496474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f780cf500 0x7f8f781594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 -- 172.21.15.64:0/54496474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f780cf500 msgr2=0x7f8f781594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 --2- 172.21.15.64:0/54496474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f780cf500 0x7f8f781594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 -- 172.21.15.64:0/54496474 >> v1:172.21.15.123:6789/0 conn(0x7f8f78142730 legacy=0x7f8f78068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f6bfff700 1 -- 172.21.15.64:0/54496474 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971564487 0 0) 0x7f8f78067fc0 con 0x7f8f78142730 2023-09-27T23:52:54.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.756+0000 7f8f7cc07700 1 -- 172.21.15.64:0/54496474 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78067970 con 0x7f8f78142b00 2023-09-27T23:52:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7cc07700 1 --2- 172.21.15.64:0/54496474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f78142b00 0x7f8f780cefc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0099b0 tx=0x7f8f6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede5e9076ab7f8d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f6bfff700 1 -- 172.21.15.64:0/54496474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c009ea0 con 0x7f8f78142b00 2023-09-27T23:52:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f6bfff700 1 -- 172.21.15.64:0/54496474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00b860 con 0x7f8f78142b00 2023-09-27T23:52:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f6bfff700 1 -- 172.21.15.64:0/54496474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00bc00 con 0x7f8f78142b00 2023-09-27T23:52:54.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 -- 172.21.15.64:0/54496474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f78142b00 msgr2=0x7f8f780cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/54496474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f78142b00 0x7f8f780cefc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0099b0 tx=0x7f8f6c009250 comp rx=0 tx=0).stop 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 -- 172.21.15.64:0/54496474 shutdown_connections 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/54496474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f780cf500 0x7f8f781594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/54496474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f78142b00 0x7f8f780cefc0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 -- 172.21.15.64:0/54496474 >> 172.21.15.64:0/54496474 conn(0x7f8f780cfea0 msgr2=0x7f8f780d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 -- 172.21.15.64:0/54496474 shutdown_connections 2023-09-27T23:52:54.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.760+0000 7f8f7e40a700 1 -- 172.21.15.64:0/54496474 wait complete. 2023-09-27T23:52:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 Processor -- start 2023-09-27T23:52:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 -- start start 2023-09-27T23:52:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 0x7f8f780b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f78142730 0x7f8f780b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f78158910 0x7f8f78140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c5c10 con 0x7f8f780cf500 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c6750 con 0x7f8f78142730 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7e40a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c5e20 con 0x7f8f78158910 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 0x7f8f780b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 0x7f8f780b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53606/0 (socket says 172.21.15.64:53606) 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 -- 172.21.15.64:0/85164760 learned_addr learned my addr 172.21.15.64:0/85164760 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7cc07700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f78142730 0x7f8f780b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7dc09700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f78158910 0x7f8f78140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 -- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f78158910 msgr2=0x7f8f78140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f78158910 0x7f8f78140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 -- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f78142730 msgr2=0x7f8f780b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f78142730 0x7f8f780b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 -- 172.21.15.64:0/85164760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0094f0 con 0x7f8f780cf500 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f7d408700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 0x7f8f780b1450 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400cd00 tx=0x7f8f74001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.764+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400ce60 con 0x7f8f780cf500 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7400aac0 con 0x7f8f780cf500 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f780c6100 con 0x7f8f780cf500 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400ae60 con 0x7f8f780cf500 2023-09-27T23:52:54.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f78158d80 con 0x7f8f780cf500 2023-09-27T23:52:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f78158fd0 con 0x7f8f780cf500 2023-09-27T23:52:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8f74012630 con 0x7f8f780cf500 2023-09-27T23:52:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.768+0000 7f8f697fa700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8f5c060a70 0x7f8f5c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.772+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f8f7407d740 con 0x7f8f780cf500 2023-09-27T23:52:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.772+0000 7f8f7cc07700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8f5c060a70 0x7f8f5c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.772+0000 7f8f7cc07700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8f5c060a70 0x7f8f5c062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0099b0 tx=0x7f8f6c017000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:54.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.776+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8f7404d7e0 con 0x7f8f780cf500 2023-09-27T23:52:54.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.940+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f780b1ae0 con 0x7f8f5c060a70 2023-09-27T23:52:54.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f697fa700 1 -- 172.21.15.64:0/85164760 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71124 (secure 0 0 0) 0x7f8f780b1ae0 con 0x7f8f5c060a70 2023-09-27T23:52:54.954 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:54.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8f5c060a70 msgr2=0x7f8f5c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8f5c060a70 0x7f8f5c062f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0099b0 tx=0x7f8f6c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 msgr2=0x7f8f780b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 0x7f8f780b1450 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400cd00 tx=0x7f8f74001b30 comp rx=0 tx=0).stop 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 shutdown_connections 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8f5c060a70 0x7f8f5c062f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f78158910 0x7f8f78140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f78142730 0x7f8f780b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 --2- 172.21.15.64:0/85164760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f780cf500 0x7f8f780b1450 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 >> 172.21.15.64:0/85164760 conn(0x7f8f780cfea0 msgr2=0x7f8f7815bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 shutdown_connections 2023-09-27T23:52:54.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:54.952+0000 7f8f7e40a700 1 -- 172.21.15.64:0/85164760 wait complete. 2023-09-27T23:52:54.957 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:52:54.969 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-09-27T23:52:54.160974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262504,"kb_used_data":20072,"kb_used_omap":0,"kb_used_meta":242368,"kb_avail":754712216,"statfs":{"total":773094113280,"available":772825309184,"internally_reserved":0,"allocated":20553728,"data_stored":11397604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":248184832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-667624,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-285,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.996508"},"pg_stats":[{"pgid":"1.0","version":"191'348","reported_seq":1134,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501255+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:50.501255+0000","last_peered":"2023-09-27T23:52:50.501255+0000","last_clean":"2023-09-27T23:52:50.501255+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:50.501255+0000","last_undegraded":"2023-09-27T23:52:50.501255+0000","last_fullsized":"2023-09-27T23:52:50.501255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":348,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.162616+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:50.162616+0000","last_peered":"2023-09-27T23:52:50.162616+0000","last_clean":"2023-09-27T23:52:50.162616+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:50.162616+0000","last_undegraded":"2023-09-27T23:52:50.162616+0000","last_fullsized":"2023-09-27T23:52:50.162616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501132+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:50.501132+0000","last_peered":"2023-09-27T23:52:50.501132+0000","last_clean":"2023-09-27T23:52:50.501132+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:50.501132+0000","last_undegraded":"2023-09-27T23:52:50.501132+0000","last_fullsized":"2023-09-27T23:52:50.501132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.498720+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:50.498720+0000","last_peered":"2023-09-27T23:52:50.498720+0000","last_clean":"2023-09-27T23:52:50.498720+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:50.498720+0000","last_undegraded":"2023-09-27T23:52:50.498720+0000","last_fullsized":"2023-09-27T23:52:50.498720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.161617+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:50.161617+0000","last_peered":"2023-09-27T23:52:50.161617+0000","last_clean":"2023-09-27T23:52:50.161617+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:50.161617+0000","last_undegraded":"2023-09-27T23:52:50.161617+0000","last_fullsized":"2023-09-27T23:52:50.161617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-09-27T23:52:47.076912+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:47.076912+0000","last_peered":"2023-09-27T23:52:47.076912+0000","last_clean":"2023-09-27T23:52:47.076912+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:47.076912+0000","last_undegraded":"2023-09-27T23:52:47.076912+0000","last_fullsized":"2023-09-27T23:52:47.076912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.500922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:50.500922+0000","last_peered":"2023-09-27T23:52:50.500922+0000","last_clean":"2023-09-27T23:52:50.500922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:50.500922+0000","last_undegraded":"2023-09-27T23:52:50.500922+0000","last_fullsized":"2023-09-27T23:52:50.500922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":170,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-09-27T23:52:47.078866+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:47.078866+0000","last_peered":"2023-09-27T23:52:47.078866+0000","last_clean":"2023-09-27T23:52:47.078866+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:47.078866+0000","last_undegraded":"2023-09-27T23:52:47.078866+0000","last_fullsized":"2023-09-27T23:52:47.078866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.239142+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:50.239142+0000","last_peered":"2023-09-27T23:52:50.239142+0000","last_clean":"2023-09-27T23:52:50.239142+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:50.239142+0000","last_undegraded":"2023-09-27T23:52:50.239142+0000","last_fullsized":"2023-09-27T23:52:50.239142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":34432,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35258368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4810000000000001,"5min":1.569,"15min":1.569},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94999999999999996,"15min":0.93100000000000005},"min":{"1min":0.58199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.1020000000000001}]},{"osd":1,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.159,"5min":1.3100000000000001,"15min":1.3360000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96899999999999997,"15min":0.97399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2949999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.66600000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4079999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97199999999999998,"15min":0.96899999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.1870000000000001,"5min":1.341,"15min":1.341},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2549999999999999,"5min":1.359,"15min":1.359},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.78100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2629999999999999,"5min":1.371,"15min":1.3879999999999999},"last":1.071}]},{"osd":4,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.3919999999999999,"5min":1.821,"15min":1.821},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.619,"5min":3.02,"15min":3.02},"last":1.171}]},{"osd":5,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0629999999999999,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3759999999999999,"5min":1.484,"15min":1.484},"last":1.137},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.103,"15min":1.113},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.028}]},{"osd":6,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.127,"15min":1.1419999999999999},"min":{"1min":0.80200000000000005,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.147,"15min":1.1479999999999999},"min":{"1min":0.81699999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5149999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.4319999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30332,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":27968,"kb_avail":94341508,"statfs":{"total":96636764160,"available":96605704192,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28639232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":1.3149999999999999,"5min":1.792,"15min":1.792},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.538,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.157,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.123}]},{"osd":2,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.629,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.494,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1919999999999999,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":1.2969999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.405,"5min":1.645,"15min":1.645},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.94999999999999996},"min":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.3340000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.872}]},{"osd":4,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.88500000000000001},"min":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.5720000000000001,"5min":1.7,"15min":1.7},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.6399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.95899999999999996,"15min":0.95999999999999996},"min":{"1min":0.73299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.401,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.242},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1599999999999999,"15min":1.1459999999999999},"min":{"1min":0.89100000000000001,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.71,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1699999999999999,"15min":1.1279999999999999},"min":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.74299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.5840000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":37888,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38797312},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.89700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.246,"5min":10.058999999999999,"15min":10.058999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0629999999999999,"15min":0.95899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":1.042}]},{"osd":1,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96099999999999997,"15min":0.93000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2929999999999999,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92600000000000005,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.159,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.077}]},{"osd":2,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"max":{"1min":1.2,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1200000000000001,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.85799999999999998}]},{"osd":3,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":0.98999999999999999},"min":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.2470000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.997,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.747,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.1970000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.996}]},{"osd":4,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":0.88800000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.107,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.222,"5min":1.45,"15min":1.4510000000000001},"last":1.165}]},{"osd":6,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2529999999999999,"5min":4.157,"15min":4.157},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.341,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.194}]},{"osd":7,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.208,"15min":1.171},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.54700000000000004},"max":{"1min":1.4830000000000001,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.3420000000000001,"15min":1.2589999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.52,"5min":14.119,"15min":14.119},"last":1.232}]}]},{"osd":4,"up_from":15,"seq":64424509907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3660000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.2769999999999999,"5min":1.893,"15min":2.2770000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.3779999999999999,"5min":1.746,"15min":1.746},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95699999999999996,"15min":0.95599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.339,"5min":1.758,"15min":2.1110000000000002},"last":0.81499999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.95699999999999996,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2649999999999999,"5min":1.659,"15min":1.659},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98799999999999999,"15min":0.96199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.2789999999999999,"5min":1.966,"15min":2.056},"last":0.91600000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.331,"5min":1.998,"15min":2.032},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.079,"15min":1.0469999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58199999999999996},"max":{"1min":1.5,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.095}]},{"osd":5,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45500000000000002},"max":{"1min":1.4610000000000001,"5min":1.556,"15min":1.9930000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.419,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.046,"15min":1.0529999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1.052},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.181,"15min":1.163},"min":{"1min":0.749,"5min":0.749,"15min":0.495},"max":{"1min":1.554,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1299999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.091,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66100000000000003},"max":{"1min":1.6279999999999999,"5min":2.069,"15min":2.069},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.149,"15min":1.1719999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2170000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.337}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.2789999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.22},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.248},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.153}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.093},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1259999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30860,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.238,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.628},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54800000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0509999999999999,"5min":1.2250000000000001,"15min":1.345},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.014,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.25,"5min":1.383,"15min":1.383},"last":0.79400000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.89400000000000002,"15min":0.93300000000000005},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":1.0740000000000001},"min":{"1min":0.60899999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3280000000000001,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0740000000000001,"15min":1.069},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4330000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.046}]},{"osd":5,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.04,"15min":1.044},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.349,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":1.321},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1379999999999999,"15min":1.1659999999999999},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4399999999999999,"5min":1.5860000000000001,"15min":1.8},"last":1.1180000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.151,"15min":1.157},"min":{"1min":0.85199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4810000000000001,"5min":1.5780000000000001,"15min":1.718},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.167,"15min":1.1779999999999999},"min":{"1min":0.82399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3879999999999999,"5min":1.6439999999999999,"15min":1.736},"last":1.1419999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.516,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3089999999999999,"5min":1.607,"15min":1.8129999999999999},"last":1.24}]}]},{"osd":1,"up_from":14,"seq":60129542613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28860,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":26496,"kb_avail":94342980,"statfs":{"total":96636764160,"available":96607211520,"internally_reserved":0,"allocated":2412544,"data_stored":1273673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27131904},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.52900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.024,"5min":1.3500000000000001,"15min":1.361},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.748},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.2509999999999999,"15min":1.383},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.69299999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.66600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2010000000000001,"5min":1.321,"15min":1.3720000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.98899999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.488,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.095,"15min":1.0980000000000001},"min":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.2889999999999999,"5min":1.831,"15min":1.831},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.1060000000000001},"min":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.55,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.226},"min":{"1min":0.94899999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.383,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.216},"min":{"1min":0.77600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.913,"15min":1.913},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:51:52 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.244,"15min":1.2549999999999999},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.528,"5min":2,"15min":2},"last":1.226},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.2290000000000001},"min":{"1min":0.99299999999999999,"5min":0.92100000000000004,"15min":0.82699999999999996},"max":{"1min":1.476,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.091}]}]},{"osd":0,"up_from":15,"seq":64424509908,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":29248,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":3006464,"data_stored":1864041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29949952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83699999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1950000000000001,"5min":1.198,"15min":1.2470000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0960000000000001,"5min":1.169,"15min":1.2509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.125,"5min":1.288,"15min":1.863},"last":1.0149999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89400000000000002,"15min":0.90100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.4390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.681},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0449999999999999,"15min":1.0569999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3360000000000001,"5min":1.516,"15min":1.5249999999999999},"last":1.3360000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.095,"15min":1.1080000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3700000000000001,"5min":1.444,"15min":1.512},"last":1.369},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5629999999999999,"15min":1.736},"last":1.288}]},{"osd":6,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.173,"15min":1.1850000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.746},"last":1.425},{"interface":"front","average":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5600000000000001,"5min":1.722,"15min":2.0510000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.248,"15min":1.27},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.544,"5min":1.786,"15min":2.0030000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2330000000000001,"15min":1.252},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.706,"5min":1.806,"15min":1.806},"last":1.4750000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:52:54.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-27T23:52:55.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48973fb700 1 Processor -- start 2023-09-27T23:52:55.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48973fb700 1 -- start start 2023-09-27T23:52:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48973fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 0x7f489006a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48973fb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900abd00 con 0x7f48900b1080 2023-09-27T23:52:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48973fb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900abfe0 con 0x7f4890069f50 2023-09-27T23:52:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48973fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48900abb10 con 0x7f48900b0cb0 2023-09-27T23:52:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f4895bf8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f48900b1080 0x7f4890069840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41790/0 (socket says 172.21.15.64:41790) 2023-09-27T23:52:55.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f4895bf8700 1 -- 172.21.15.64:0/4049847977 learned_addr learned my addr 172.21.15.64:0/4049847977 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48963f9700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 0x7f489006a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343210688 0 0) 0x7f48900abd00 con 0x7f48900b1080 2023-09-27T23:52:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.068+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c003570 con 0x7f48900b1080 2023-09-27T23:52:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437074942 0 0) 0x7f48900abfe0 con 0x7f4890069f50 2023-09-27T23:52:55.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48900abd00 con 0x7f4890069f50 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4112154139 0 0) 0x7f487c003570 con 0x7f48900b1080 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 msgr2=0x7f489006a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 0x7f489006a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 >> v1:172.21.15.123:6789/0 conn(0x7f4890069f50 legacy=0x7f48900673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900ac2c0 con 0x7f48900b1080 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2016035475 0 0) 0x7f488c001470 con 0x7f48900b1080 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 >> v1:172.21.15.64:6789/0 conn(0x7f48900b1080 legacy=0x7f4890069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 0x7f487c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487c004180 0x7f487c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487c004d50 0x7f487c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00bc20 con 0x7f487c004180 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00bd90 con 0x7f487c004d50 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00bf50 con 0x7f48900b0cb0 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487c004180 0x7f487c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48963f9700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 0x7f487c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4896bfa700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487c004d50 0x7f487c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 -- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 msgr2=0x7f487c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b0cb0 0x7f487c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 -- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487c004d50 msgr2=0x7f487c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487c004d50 0x7f487c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 -- 172.21.15.64:0/4049847977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c003570 con 0x7f487c004180 2023-09-27T23:52:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4895bf8700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487c004180 0x7f487c004810 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c2a0 tx=0x7f48900ab0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f714483ed0c3de server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00cb70 con 0x7f487c004180 2023-09-27T23:52:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00ccf0 con 0x7f487c004180 2023-09-27T23:52:55.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48953f7700 1 -- 172.21.15.64:0/4049847977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c005b70 con 0x7f487c004180 2023-09-27T23:52:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48973fb700 1 -- 172.21.15.64:0/4049847977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487c004180 msgr2=0x7f487c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48973fb700 1 --2- 172.21.15.64:0/4049847977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f487c004180 0x7f487c004810 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c2a0 tx=0x7f48900ab0e0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f4896bfa700 1 -- 172.21.15.64:0/4049847977 reap_dead start 2023-09-27T23:52:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48973fb700 1 -- 172.21.15.64:0/4049847977 shutdown_connections 2023-09-27T23:52:55.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48973fb700 1 -- 172.21.15.64:0/4049847977 >> 172.21.15.64:0/4049847977 conn(0x7f48900cc530 msgr2=0x7f48900c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:55.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48973fb700 1 -- 172.21.15.64:0/4049847977 shutdown_connections 2023-09-27T23:52:55.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.072+0000 7f48973fb700 1 -- 172.21.15.64:0/4049847977 wait complete. 2023-09-27T23:52:55.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 Processor -- start 2023-09-27T23:52:55.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 -- start start 2023-09-27T23:52:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489014dfe0 0x7f48900c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489014e3b0 0x7f48900ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 0x7f48900bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900c0e50 con 0x7f48900a4e80 2023-09-27T23:52:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48900c0840 con 0x7f489014e3b0 2023-09-27T23:52:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48973fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48900a5660 con 0x7f489014dfe0 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f4896bfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 0x7f48900bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f48963f9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489014dfe0 0x7f48900c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f4896bfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 0x7f48900bf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53622/0 (socket says 172.21.15.64:53622) 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f4896bfa700 1 -- 172.21.15.64:0/885414731 learned_addr learned my addr 172.21.15.64:0/885414731 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.076+0000 7f4895bf8700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489014e3b0 0x7f48900ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f4896bfa700 1 -- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489014dfe0 msgr2=0x7f48900c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f4896bfa700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489014dfe0 0x7f48900c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f4896bfa700 1 -- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489014e3b0 msgr2=0x7f48900ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f4896bfa700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489014e3b0 0x7f48900ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f4896bfa700 1 -- 172.21.15.64:0/885414731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900a5380 con 0x7f48900a4e80 2023-09-27T23:52:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f4896bfa700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 0x7f48900bf4a0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f4888001460 tx=0x7f4888009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880032e0 con 0x7f48900a4e80 2023-09-27T23:52:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888003460 con 0x7f48900a4e80 2023-09-27T23:52:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48900c0ae0 con 0x7f48900a4e80 2023-09-27T23:52:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888001930 con 0x7f48900a4e80 2023-09-27T23:52:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48900bf9e0 con 0x7f48900a4e80 2023-09-27T23:52:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.080+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48900c06a0 con 0x7f48900a4e80 2023-09-27T23:52:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.084+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4888001d20 con 0x7f48900a4e80 2023-09-27T23:52:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.084+0000 7f48867fc700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4874060a70 0x7f4874062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.084+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f488807ca90 con 0x7f48900a4e80 2023-09-27T23:52:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.084+0000 7f48963f9700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4874060a70 0x7f4874062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.084+0000 7f48963f9700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4874060a70 0x7f4874062f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48900cab70 tx=0x7f48800091a0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.092+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f48900c06a0 con 0x7f48900a4e80 2023-09-27T23:52:55.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.256+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48900c06a0 con 0x7f48900a4e80 2023-09-27T23:52:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.256+0000 7f48867fc700 1 -- 172.21.15.64:0/885414731 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+14936 (secure 0 0 0) 0x7f48900c06a0 con 0x7f48900a4e80 2023-09-27T23:52:55.263 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:52:55.263 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":194,"fsid":"9497c172-4ccf-4119-8910-f606cb59c9f0","created":"2023-09-27T23:13:20.992240+0000","modified":"2023-09-27T23:52:51.153335+0000","last_up_change":"2023-09-27T23:13:31.944551+0000","last_in_change":"2023-09-27T23:13:23.035794+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-27T23:13:33.202942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-27T23:13:38.821082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"915f831e-9c10-4ed9-8012-6e415f3f0fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33400},{"type":"v1","addr":"172.21.15.64:6801","nonce":33400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33400},{"type":"v1","addr":"172.21.15.64:6803","nonce":33400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33400},{"type":"v1","addr":"172.21.15.64:6807","nonce":33400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33400},{"type":"v1","addr":"172.21.15.64:6805","nonce":33400}]},"public_addr":"172.21.15.64:6801/33400","cluster_addr":"172.21.15.64:6803/33400","heartbeat_back_addr":"172.21.15.64:6807/33400","heartbeat_front_addr":"172.21.15.64:6805/33400","state":["exists","up"]},{"osd":1,"uuid":"019649f0-4ab6-44a5-afcd-ec3112182bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33405},{"type":"v1","addr":"172.21.15.64:6809","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33405},{"type":"v1","addr":"172.21.15.64:6811","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33405},{"type":"v1","addr":"172.21.15.64:6815","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33405},{"type":"v1","addr":"172.21.15.64:6813","nonce":33405}]},"public_addr":"172.21.15.64:6809/33405","cluster_addr":"172.21.15.64:6811/33405","heartbeat_back_addr":"172.21.15.64:6815/33405","heartbeat_front_addr":"172.21.15.64:6813/33405","state":["exists","up"]},{"osd":2,"uuid":"232c6eb7-ce53-44be-9ec1-f4bfba21c327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33402},{"type":"v1","addr":"172.21.15.64:6825","nonce":33402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33402},{"type":"v1","addr":"172.21.15.64:6827","nonce":33402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33402},{"type":"v1","addr":"172.21.15.64:6831","nonce":33402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33402},{"type":"v1","addr":"172.21.15.64:6829","nonce":33402}]},"public_addr":"172.21.15.64:6825/33402","cluster_addr":"172.21.15.64:6827/33402","heartbeat_back_addr":"172.21.15.64:6831/33402","heartbeat_front_addr":"172.21.15.64:6829/33402","state":["exists","up"]},{"osd":3,"uuid":"3f119991-cb63-4f2e-95b5-db5712f79f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33401},{"type":"v1","addr":"172.21.15.64:6817","nonce":33401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33401},{"type":"v1","addr":"172.21.15.64:6819","nonce":33401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33401},{"type":"v1","addr":"172.21.15.64:6823","nonce":33401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33401},{"type":"v1","addr":"172.21.15.64:6821","nonce":33401}]},"public_addr":"172.21.15.64:6817/33401","cluster_addr":"172.21.15.64:6819/33401","heartbeat_back_addr":"172.21.15.64:6823/33401","heartbeat_front_addr":"172.21.15.64:6821/33401","state":["exists","up"]},{"osd":4,"uuid":"7e01ef22-2cfe-4187-ac67-822e22a91142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":33568},{"type":"v1","addr":"172.21.15.123:6809","nonce":33568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":33568},{"type":"v1","addr":"172.21.15.123:6811","nonce":33568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":33568},{"type":"v1","addr":"172.21.15.123:6815","nonce":33568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":33568},{"type":"v1","addr":"172.21.15.123:6813","nonce":33568}]},"public_addr":"172.21.15.123:6809/33568","cluster_addr":"172.21.15.123:6811/33568","heartbeat_back_addr":"172.21.15.123:6815/33568","heartbeat_front_addr":"172.21.15.123:6813/33568","state":["exists","up"]},{"osd":5,"uuid":"07a4e7d9-c362-4b12-a607-6faf07adf98a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":33567},{"type":"v1","addr":"172.21.15.123:6820","nonce":33567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":33567},{"type":"v1","addr":"172.21.15.123:6824","nonce":33567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":33567},{"type":"v1","addr":"172.21.15.123:6831","nonce":33567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6827","nonce":33567},{"type":"v1","addr":"172.21.15.123:6829","nonce":33567}]},"public_addr":"172.21.15.123:6820/33567","cluster_addr":"172.21.15.123:6824/33567","heartbeat_back_addr":"172.21.15.123:6831/33567","heartbeat_front_addr":"172.21.15.123:6829/33567","state":["exists","up"]},{"osd":6,"uuid":"98c04d1b-babe-4851-82e7-a418c0c3e53b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":33584},{"type":"v1","addr":"172.21.15.123:6801","nonce":33584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":33584},{"type":"v1","addr":"172.21.15.123:6803","nonce":33584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":33584},{"type":"v1","addr":"172.21.15.123:6807","nonce":33584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":33584},{"type":"v1","addr":"172.21.15.123:6805","nonce":33584}]},"public_addr":"172.21.15.123:6801/33584","cluster_addr":"172.21.15.123:6803/33584","heartbeat_back_addr":"172.21.15.123:6807/33584","heartbeat_front_addr":"172.21.15.123:6805/33584","state":["exists","up"]},{"osd":7,"uuid":"d48f521b-c431-4a35-8069-a24a0648c612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":33569},{"type":"v1","addr":"172.21.15.123:6818","nonce":33569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":33569},{"type":"v1","addr":"172.21.15.123:6821","nonce":33569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":33569},{"type":"v1","addr":"172.21.15.123:6828","nonce":33569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6823","nonce":33569},{"type":"v1","addr":"172.21.15.123:6825","nonce":33569}]},"public_addr":"172.21.15.123:6818/33569","cluster_addr":"172.21.15.123:6821/33569","heartbeat_back_addr":"172.21.15.123:6828/33569","heartbeat_front_addr":"172.21.15.123:6825/33569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.374071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.075498+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.389501+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.493558+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.254186+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.500902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.363254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-27T23:13:28.369036+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:0/2551836643":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3550075260":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6835/1929792685":"2023-09-28T23:52:47.067586+0000","172.21.15.123:0/3834368324":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3658378865":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/855337008":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6834/1929792685":"2023-09-28T23:52:47.067586+0000","172.21.15.123:0/1582733572":"2023-09-28T23:52:46.131144+0000","172.21.15.123:0/3573447607":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1334338128":"2023-09-28T23:52:46.131144+0000","172.21.15.123:6832/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:6833/43188":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/2413747055":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3774540904":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/3310607164":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/4028139068":"2023-09-28T23:52:37.175204+0000","172.21.15.123:0/1447825948":"2023-09-28T23:52:37.175204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-27T23:52:55.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4874060a70 msgr2=0x7f4874062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4874060a70 0x7f4874062f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48900cab70 tx=0x7f48800091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 msgr2=0x7f48900bf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 0x7f48900bf4a0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f4888001460 tx=0x7f4888009040 comp rx=0 tx=0).stop 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 shutdown_connections 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4874060a70 0x7f4874062f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48900a4e80 0x7f48900bf4a0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f489014e3b0 0x7f48900ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 --2- 172.21.15.64:0/885414731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f489014dfe0 0x7f48900c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 >> 172.21.15.64:0/885414731 conn(0x7f48900cc530 msgr2=0x7f48900b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:55.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 shutdown_connections 2023-09-27T23:52:55.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:55.260+0000 7f48973fb700 1 -- 172.21.15.64:0/885414731 wait complete. 2023-09-27T23:52:56.279 INFO:tasks.ceph:Scrubbing osd.0 2023-09-27T23:52:56.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 Processor -- start 2023-09-27T23:52:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 -- start start 2023-09-27T23:52:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f155c156ba0 0x7f155c0ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f155c0caa90 0x7f155c06a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caee0 0x7f155c06ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c0cbfd0 con 0x7f155c0caa90 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c0cb3c0 con 0x7f155c156ba0 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15634c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0cb6a0 con 0x7f155c0caee0 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f155c0caa90 0x7f155c06a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f155c0caa90 0x7f155c06a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53628/0 (socket says 172.21.15.64:53628) 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 -- 172.21.15.64:0/145970315 learned_addr learned my addr 172.21.15.64:0/145970315 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1562cc7700 1 --2- 172.21.15.64:0/145970315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caee0 0x7f155c06ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f15624c6700 1 --2- 172.21.15.64:0/145970315 >> v2:172.21.15.123:3300/0 conn(0x7f155c156ba0 0x7f155c0ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 -- 172.21.15.64:0/145970315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caee0 msgr2=0x7f155c06ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 --2- 172.21.15.64:0/145970315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caee0 0x7f155c06ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 -- 172.21.15.64:0/145970315 >> v2:172.21.15.123:3300/0 conn(0x7f155c156ba0 msgr2=0x7f155c0ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 --2- 172.21.15.64:0/145970315 >> v2:172.21.15.123:3300/0 conn(0x7f155c156ba0 0x7f155c0ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 -- 172.21.15.64:0/145970315 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0cb980 con 0x7f155c0caa90 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.400+0000 7f1561cc5700 1 --2- 172.21.15.64:0/145970315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caa90 0x7f155c06a720 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f155c070020 tx=0x7f154c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84351823e2b577ea server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15614c4700 1 -- 172.21.15.64:0/145970315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00b640 con 0x7f155c0caa90 2023-09-27T23:52:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15614c4700 1 -- 172.21.15.64:0/145970315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c00b7a0 con 0x7f155c0caa90 2023-09-27T23:52:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15614c4700 1 -- 172.21.15.64:0/145970315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00bb60 con 0x7f155c0caa90 2023-09-27T23:52:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 -- 172.21.15.64:0/145970315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caa90 msgr2=0x7f155c06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 --2- 172.21.15.64:0/145970315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caa90 0x7f155c06a720 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f155c070020 tx=0x7f154c009250 comp rx=0 tx=0).stop 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 -- 172.21.15.64:0/145970315 shutdown_connections 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 --2- 172.21.15.64:0/145970315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caee0 0x7f155c06ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 --2- 172.21.15.64:0/145970315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caa90 0x7f155c06a720 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 --2- 172.21.15.64:0/145970315 >> v2:172.21.15.123:3300/0 conn(0x7f155c156ba0 0x7f155c0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 -- 172.21.15.64:0/145970315 >> 172.21.15.64:0/145970315 conn(0x7f155c0ccdb0 msgr2=0x7f155c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 -- 172.21.15.64:0/145970315 shutdown_connections 2023-09-27T23:52:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 -- 172.21.15.64:0/145970315 wait complete. 2023-09-27T23:52:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.404+0000 7f15634c8700 1 Processor -- start 2023-09-27T23:52:56.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- start start 2023-09-27T23:52:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caa90 0x7f155c0a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caee0 0x7f155c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c156ba0 0x7f155c0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0a6970 con 0x7f155c0caee0 2023-09-27T23:52:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0a73a0 con 0x7f155c156ba0 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0a7d70 con 0x7f155c0caa90 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caee0 0x7f155c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caee0 0x7f155c1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53630/0 (socket says 172.21.15.64:53630) 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 -- 172.21.15.64:0/2450021522 learned_addr learned my addr 172.21.15.64:0/2450021522 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15624c6700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caa90 0x7f155c0a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1562cc7700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c156ba0 0x7f155c0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 -- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caa90 msgr2=0x7f155c0a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caa90 0x7f155c0a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 -- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c156ba0 msgr2=0x7f155c0aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c156ba0 0x7f155c0aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 -- 172.21.15.64:0/2450021522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0094f0 con 0x7f155c0caee0 2023-09-27T23:52:56.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15624c6700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c0caa90 0x7f155c0a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1562cc7700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c156ba0 0x7f155c0aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f1561cc5700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caee0 0x7f155c1557e0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f154c009f70 tx=0x7f154c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c000d10 con 0x7f155c0caee0 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c00bb20 con 0x7f155c0caee0 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c155d20 con 0x7f155c0caee0 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c026680 con 0x7f155c0caee0 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c0a6d50 con 0x7f155c0caee0 2023-09-27T23:52:56.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.408+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f155c0a6ba0 con 0x7f155c0caee0 2023-09-27T23:52:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f154c011420 con 0x7f155c0caee0 2023-09-27T23:52:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15527fc700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1540060a70 0x7f1540062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f154c07ea80 con 0x7f155c0caee0 2023-09-27T23:52:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15527fc700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7f1540067830 0x7f1540069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 --> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f154006a330 con 0x7f1540067830 2023-09-27T23:52:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15624c6700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1540060a70 0x7f1540062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c080ea0 con 0x7f155c0caee0 2023-09-27T23:52:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f1562cc7700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7f1540067830 0x7f1540069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f1562cc7700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7f1540067830 0x7f1540069c60 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.412+0000 7f15624c6700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1540060a70 0x7f1540062f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ce0 tx=0x7f1558008040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.416+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== osd.0 v2:172.21.15.64:6800/33400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f154006a330 con 0x7f1540067830 2023-09-27T23:52:56.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 --> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f155c04be40 con 0x7f1540067830 2023-09-27T23:52:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15527fc700 1 -- 172.21.15.64:0/2450021522 <== osd.0 v2:172.21.15.64:6800/33400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f155c04be40 con 0x7f1540067830 2023-09-27T23:52:56.438 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:56.438 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:56.438 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:56.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7f1540067830 msgr2=0x7f1540069c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:6800/33400,v1:172.21.15.64:6801/33400] conn(0x7f1540067830 0x7f1540069c60 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1540060a70 msgr2=0x7f1540062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1540060a70 0x7f1540062f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ce0 tx=0x7f1558008040 comp rx=0 tx=0).stop 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caee0 msgr2=0x7f155c1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 --2- 172.21.15.64:0/2450021522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f155c0caee0 0x7f155c1557e0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f154c009f70 tx=0x7f154c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f1562cc7700 1 -- 172.21.15.64:0/2450021522 reap_dead start 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 shutdown_connections 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 >> 172.21.15.64:0/2450021522 conn(0x7f155c0ccdb0 msgr2=0x7f155c156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 shutdown_connections 2023-09-27T23:52:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.432+0000 7f15634c8700 1 -- 172.21.15.64:0/2450021522 wait complete. 2023-09-27T23:52:56.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-09-27T23:52:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 Processor -- start 2023-09-27T23:52:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 -- start start 2023-09-27T23:52:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab006a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ab00b1080 0x7f7ab0069840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00abd00 con 0x7f7ab00adf70 2023-09-27T23:52:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab00abb10 con 0x7f7ab00b1080 2023-09-27T23:52:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7ab4a0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00abfe0 con 0x7f7ab00b0cb0 2023-09-27T23:52:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7aae59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ab00b1080 0x7f7ab0069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.564+0000 7f7aae59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ab00b1080 0x7f7ab0069840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53632/0 (socket says 172.21.15.64:53632) 2023-09-27T23:52:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aaed9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab006a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aae59c700 1 -- 172.21.15.64:0/3684764748 learned_addr learned my addr 172.21.15.64:0/3684764748 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aae59c700 1 -- 172.21.15.64:0/3684764748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 msgr2=0x7f7ab006a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aae59c700 1 --2- 172.21.15.64:0/3684764748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab006a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aae59c700 1 -- 172.21.15.64:0/3684764748 >> v1:172.21.15.123:6789/0 conn(0x7f7ab00adf70 legacy=0x7f7ab015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aae59c700 1 -- 172.21.15.64:0/3684764748 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00ac2c0 con 0x7f7ab00b1080 2023-09-27T23:52:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aae59c700 1 --2- 172.21.15.64:0/3684764748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab0069840 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00099b0 tx=0x7f7aa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d51be790e1b046a server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aadd9b700 1 -- 172.21.15.64:0/3684764748 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0009ea0 con 0x7f7ab00b1080 2023-09-27T23:52:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aadd9b700 1 -- 172.21.15.64:0/3684764748 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa000b860 con 0x7f7ab00b1080 2023-09-27T23:52:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7aadd9b700 1 -- 172.21.15.64:0/3684764748 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000bc20 con 0x7f7ab00b1080 2023-09-27T23:52:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/3684764748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 msgr2=0x7f7ab0069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/3684764748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab0069840 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00099b0 tx=0x7f7aa0009250 comp rx=0 tx=0).stop 2023-09-27T23:52:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/3684764748 shutdown_connections 2023-09-27T23:52:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/3684764748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab0069840 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/3684764748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab006a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/3684764748 >> 172.21.15.64:0/3684764748 conn(0x7f7ab00cc530 msgr2=0x7f7ab00c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/3684764748 shutdown_connections 2023-09-27T23:52:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.568+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/3684764748 wait complete. 2023-09-27T23:52:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 Processor -- start 2023-09-27T23:52:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 -- start start 2023-09-27T23:52:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 0x7f7ab0155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab00c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab00a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00a89e0 con 0x7f7ab00b1080 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00a8cc0 con 0x7f7ab00adf70 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00a8fa0 con 0x7f7ab00b0cb0 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab00a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 0x7f7ab0155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aae59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab00c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab00a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53642/0 (socket says 172.21.15.64:53642) 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 0x7f7ab0155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.64:46688/0 (socket says 172.21.15.64:46688) 2023-09-27T23:52:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 -- 172.21.15.64:0/2296491515 learned_addr learned my addr 172.21.15.64:0/2296491515 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 -- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 msgr2=0x7f7ab00c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab00c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 -- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 msgr2=0x7f7ab0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 0x7f7ab0155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 -- 172.21.15.64:0/2296491515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa00094f0 con 0x7f7ab00b1080 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaed9d700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 0x7f7ab0155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7aaf59e700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab00a9a80 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c8e0 tx=0x7f7aa8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8012630 con 0x7f7ab00b1080 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa80127b0 con 0x7f7ab00b1080 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00a9280 con 0x7f7ab00b1080 2023-09-27T23:52:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8012b70 con 0x7f7ab00b1080 2023-09-27T23:52:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.572+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab00c6b80 con 0x7f7ab00b1080 2023-09-27T23:52:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.576+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab004be40 con 0x7f7ab00b1080 2023-09-27T23:52:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.576+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f7aa8003030 con 0x7f7ab00b1080 2023-09-27T23:52:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.576+0000 7f7a96ffd700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a9c060a70 0x7f7a9c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.576+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f7aa807cf20 con 0x7f7ab00b1080 2023-09-27T23:52:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.576+0000 7f7aaed9d700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a9c060a70 0x7f7a9c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.580+0000 7f7aaed9d700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a9c060a70 0x7f7a9c062f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4011480 tx=0x7f7aa40091a0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.584+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f7aa804cfc0 con 0x7f7ab00b1080 2023-09-27T23:52:56.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.748+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ab00a8e30 con 0x7f7a9c060a70 2023-09-27T23:52:56.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7a96ffd700 1 -- 172.21.15.64:0/2296491515 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7ab00a8e30 con 0x7f7a9c060a70 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a9c060a70 msgr2=0x7f7a9c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a9c060a70 0x7f7a9c062f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4011480 tx=0x7f7aa40091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 msgr2=0x7f7ab00a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab00a9a80 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c8e0 tx=0x7f7aa8001710 comp rx=0 tx=0).stop 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 shutdown_connections 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7a9c060a70 0x7f7a9c062f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab00b1080 0x7f7ab00a9a80 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab00b0cb0 0x7f7ab00c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 --2- 172.21.15.64:0/2296491515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab00adf70 0x7f7ab0155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 >> 172.21.15.64:0/2296491515 conn(0x7f7ab00cc530 msgr2=0x7f7ab006d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 shutdown_connections 2023-09-27T23:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.752+0000 7f7ab4a0f700 1 -- 172.21.15.64:0/2296491515 wait complete. 2023-09-27T23:52:56.760 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2023-09-27T23:52:56.770 INFO:tasks.ceph:Scrubbing osd.1 2023-09-27T23:52:56.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:56.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.880+0000 7fd9f9847700 1 Processor -- start 2023-09-27T23:52:56.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.880+0000 7fd9f9847700 1 -- start start 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f9847700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd9f40aeda0 0x7fd9f40b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f9847700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f40b1450 con 0x7fd9f40ca370 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f9847700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f40b0b20 con 0x7fd9f40b18e0 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f9847700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f40b0e00 con 0x7fd9f40aeda0 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f3fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd9f40ca370 0x7fd9f40af170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41796/0 (socket says 172.21.15.64:41796) 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f3fff700 1 -- 172.21.15.64:0/1371165907 learned_addr learned my addr 172.21.15.64:0/1371165907 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f8845700 1 --2- 172.21.15.64:0/1371165907 >> v2:172.21.15.64:3300/0 conn(0x7fd9f40aeda0 0x7fd9f40b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f37fe700 1 -- 172.21.15.64:0/1371165907 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731590184 0 0) 0x7fd9f40b1450 con 0x7fd9f40ca370 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f37fe700 1 -- 172.21.15.64:0/1371165907 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9dc003570 con 0x7fd9f40ca370 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f37fe700 1 -- 172.21.15.64:0/1371165907 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863114868 0 0) 0x7fd9f40b0b20 con 0x7fd9f40b18e0 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f8845700 1 -- 172.21.15.64:0/1371165907 >> v1:172.21.15.123:6789/0 conn(0x7fd9f40b18e0 legacy=0x7fd9f40b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f8845700 1 -- 172.21.15.64:0/1371165907 >> v1:172.21.15.64:6789/0 conn(0x7fd9f40ca370 legacy=0x7fd9f40af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f8845700 1 -- 172.21.15.64:0/1371165907 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40b10e0 con 0x7fd9f40aeda0 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f8845700 1 --2- 172.21.15.64:0/1371165907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f40b3770 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80098e0 tx=0x7fd9e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38208d50e1ba0949 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f37fe700 1 -- 172.21.15.64:0/1371165907 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800b640 con 0x7fd9f40aeda0 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f37fe700 1 -- 172.21.15.64:0/1371165907 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e800b7a0 con 0x7fd9f40aeda0 2023-09-27T23:52:56.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f37fe700 1 -- 172.21.15.64:0/1371165907 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800bb40 con 0x7fd9f40aeda0 2023-09-27T23:52:56.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.884+0000 7fd9f9847700 1 -- 172.21.15.64:0/1371165907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 msgr2=0x7fd9f40b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 --2- 172.21.15.64:0/1371165907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f40b3770 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80098e0 tx=0x7fd9e80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 -- 172.21.15.64:0/1371165907 shutdown_connections 2023-09-27T23:52:56.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 --2- 172.21.15.64:0/1371165907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f40b3770 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 -- 172.21.15.64:0/1371165907 >> 172.21.15.64:0/1371165907 conn(0x7fd9f40ce790 msgr2=0x7fd9f40ad680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:56.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 -- 172.21.15.64:0/1371165907 shutdown_connections 2023-09-27T23:52:56.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 -- 172.21.15.64:0/1371165907 wait complete. 2023-09-27T23:52:56.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 Processor -- start 2023-09-27T23:52:56.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.888+0000 7fd9f9847700 1 -- start start 2023-09-27T23:52:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f40b18e0 0x7fd9f4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9f40ca370 0x7fd9f40c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40c6b10 con 0x7fd9f40aeda0 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40c6ca0 con 0x7fd9f40b18e0 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f415a450 con 0x7fd9f40ca370 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f4159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53648/0 (socket says 172.21.15.64:53648) 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 -- 172.21.15.64:0/2927246677 learned_addr learned my addr 172.21.15.64:0/2927246677 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9046700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9f40ca370 0x7fd9f40c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f3fff700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f40b18e0 0x7fd9f4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 -- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9f40ca370 msgr2=0x7fd9f40c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9f40ca370 0x7fd9f40c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 -- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f40b18e0 msgr2=0x7fd9f4153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f40b18e0 0x7fd9f4153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 -- 172.21.15.64:0/2927246677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8009420 con 0x7fd9f40aeda0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f8845700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f4159f10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80097b0 tx=0x7fd9e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e801f070 con 0x7fd9f40aeda0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e8002f70 con 0x7fd9f40aeda0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f4158270 con 0x7fd9f40aeda0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800b910 con 0x7fd9f40aeda0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f4158830 con 0x7fd9f40aeda0 2023-09-27T23:52:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.892+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9f404be40 con 0x7fd9f40aeda0 2023-09-27T23:52:56.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fd9e8023070 con 0x7fd9f40aeda0 2023-09-27T23:52:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd9e0060a70 0x7fd9e0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fd9e807e3a0 con 0x7fd9f40aeda0 2023-09-27T23:52:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7fd9e00679a0 0x7fd9e0069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 --> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9e006a4a0 con 0x7fd9e00679a0 2023-09-27T23:52:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9e8080900 con 0x7fd9f40aeda0 2023-09-27T23:52:56.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f3fff700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd9e0060a70 0x7fd9e0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f9046700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7fd9e00679a0 0x7fd9e0069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:56.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f9046700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7fd9e00679a0 0x7fd9e0069dd0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f3fff700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd9e0060a70 0x7fd9e0062f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40c45b0 tx=0x7fd9e4009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:56.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.896+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== osd.1 v2:172.21.15.64:6808/33405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd9e006a4a0 con 0x7fd9e00679a0 2023-09-27T23:52:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 --> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd9f40ca7c0 con 0x7fd9e00679a0 2023-09-27T23:52:56.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f0ff9700 1 -- 172.21.15.64:0/2927246677 <== osd.1 v2:172.21.15.64:6808/33405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd9f40ca7c0 con 0x7fd9e00679a0 2023-09-27T23:52:56.921 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:56.921 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:56.922 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:56.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7fd9e00679a0 msgr2=0x7fd9e0069dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:6808/33405,v1:172.21.15.64:6809/33405] conn(0x7fd9e00679a0 0x7fd9e0069dd0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd9e0060a70 msgr2=0x7fd9e0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fd9e0060a70 0x7fd9e0062f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40c45b0 tx=0x7fd9e4009040 comp rx=0 tx=0).stop 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 msgr2=0x7fd9f4159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 --2- 172.21.15.64:0/2927246677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9f40aeda0 0x7fd9f4159f10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80097b0 tx=0x7fd9e8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9046700 1 -- 172.21.15.64:0/2927246677 reap_dead start 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 shutdown_connections 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 >> 172.21.15.64:0/2927246677 conn(0x7fd9f40ce790 msgr2=0x7fd9f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 shutdown_connections 2023-09-27T23:52:56.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:56.916+0000 7fd9f9847700 1 -- 172.21.15.64:0/2927246677 wait complete. 2023-09-27T23:52:56.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-09-27T23:52:57.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.044+0000 7f61c875f700 1 Processor -- start 2023-09-27T23:52:57.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c875f700 1 -- start start 2023-09-27T23:52:57.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c875f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61c0159090 0x7f61c00b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c875f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c00b24c0 0x7f61c00afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c875f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00cb710 con 0x7f61c00af620 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c875f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c00af490 con 0x7f61c0159090 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c875f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00cade0 con 0x7f61c00b24c0 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c64fb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61c0159090 0x7f61c00b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c64fb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f61c0159090 0x7f61c00b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53660/0 (socket says 172.21.15.64:53660) 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c64fb700 1 -- 172.21.15.64:0/3667371103 learned_addr learned my addr 172.21.15.64:0/3667371103 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:57.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c6cfc700 1 --2- 172.21.15.64:0/3667371103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c00b24c0 0x7f61c00afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c54f9700 1 -- 172.21.15.64:0/3667371103 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109017012 0 0) 0x7f61c00cb710 con 0x7f61c00af620 2023-09-27T23:52:57.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c54f9700 1 -- 172.21.15.64:0/3667371103 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ac003570 con 0x7f61c00af620 2023-09-27T23:52:57.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c6cfc700 1 -- 172.21.15.64:0/3667371103 >> v2:172.21.15.64:3300/0 conn(0x7f61c0159090 msgr2=0x7f61c00b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c6cfc700 1 --2- 172.21.15.64:0/3667371103 >> v2:172.21.15.64:3300/0 conn(0x7f61c0159090 0x7f61c00b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c6cfc700 1 -- 172.21.15.64:0/3667371103 >> v1:172.21.15.123:6789/0 conn(0x7f61c00af620 legacy=0x7f61c00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c6cfc700 1 -- 172.21.15.64:0/3667371103 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c00cb0c0 con 0x7f61c00b24c0 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c6cfc700 1 --2- 172.21.15.64:0/3667371103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c00b24c0 0x7f61c00afa90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009e60 tx=0x7f61b8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba94eda124a921c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c54f9700 1 -- 172.21.15.64:0/3667371103 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800e070 con 0x7f61c00b24c0 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c54f9700 1 -- 172.21.15.64:0/3667371103 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b800bab0 con 0x7f61c00b24c0 2023-09-27T23:52:57.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.048+0000 7f61c54f9700 1 -- 172.21.15.64:0/3667371103 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800be70 con 0x7f61c00b24c0 2023-09-27T23:52:57.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 -- 172.21.15.64:0/3667371103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c00b24c0 msgr2=0x7f61c00afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 --2- 172.21.15.64:0/3667371103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c00b24c0 0x7f61c00afa90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009e60 tx=0x7f61b8009780 comp rx=0 tx=0).stop 2023-09-27T23:52:57.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 -- 172.21.15.64:0/3667371103 shutdown_connections 2023-09-27T23:52:57.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 --2- 172.21.15.64:0/3667371103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c00b24c0 0x7f61c00afa90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 --2- 172.21.15.64:0/3667371103 >> v2:172.21.15.64:3300/0 conn(0x7f61c0159090 0x7f61c00b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 -- 172.21.15.64:0/3667371103 >> 172.21.15.64:0/3667371103 conn(0x7f61c00d0ab0 msgr2=0x7f61c00cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 -- 172.21.15.64:0/3667371103 shutdown_connections 2023-09-27T23:52:57.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 -- 172.21.15.64:0/3667371103 wait complete. 2023-09-27T23:52:57.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 Processor -- start 2023-09-27T23:52:57.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.052+0000 7f61c875f700 1 -- start start 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 0x7f61c00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00b24c0 0x7f61c00a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c0159090 0x7f61c0169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c016b450 con 0x7f61c00af620 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c016a840 con 0x7f61c00b24c0 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c016ab20 con 0x7f61c0159090 2023-09-27T23:52:57.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 0x7f61c00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 0x7f61c00ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53676/0 (socket says 172.21.15.64:53676) 2023-09-27T23:52:57.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 -- 172.21.15.64:0/2103376389 learned_addr learned my addr 172.21.15.64:0/2103376389 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:57.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c6cfc700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c0159090 0x7f61c0169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c5cfa700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00b24c0 0x7f61c00a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 -- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c0159090 msgr2=0x7f61c0169b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c0159090 0x7f61c0169b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 -- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00b24c0 msgr2=0x7f61c00a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00b24c0 0x7f61c00a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 -- 172.21.15.64:0/2103376389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8009a20 con 0x7f61c00af620 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c6cfc700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c0159090 0x7f61c0169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c5cfa700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00b24c0 0x7f61c00a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c64fb700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 0x7f61c00ade30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e980 tx=0x7f61b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0013680 con 0x7f61c00af620 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00137e0 con 0x7f61c00af620 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c016ae00 con 0x7f61c00af620 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0013b60 con 0x7f61c00af620 2023-09-27T23:52:57.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c00ae370 con 0x7f61c00af620 2023-09-27T23:52:57.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.056+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c0071110 con 0x7f61c00af620 2023-09-27T23:52:57.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.060+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f61b000c400 con 0x7f61c00af620 2023-09-27T23:52:57.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.060+0000 7f61b67fc700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f61a4060a70 0x7f61a4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.060+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f61b007d1f0 con 0x7f61c00af620 2023-09-27T23:52:57.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.060+0000 7f61c5cfa700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f61a4060a70 0x7f61a4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.060+0000 7f61c5cfa700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f61a4060a70 0x7f61a4062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f61c015bbd0 tx=0x7f61bc007d40 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.068+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f61b0050c80 con 0x7f61c00af620 2023-09-27T23:52:57.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f61c016b590 con 0x7f61a4060a70 2023-09-27T23:52:57.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61b67fc700 1 -- 172.21.15.64:0/2103376389 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f61bc00fa70 con 0x7f61a4060a70 2023-09-27T23:52:57.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f61a4060a70 msgr2=0x7f61a4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f61a4060a70 0x7f61a4062f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f61c015bbd0 tx=0x7f61bc007d40 comp rx=0 tx=0).stop 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 msgr2=0x7f61c00ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 0x7f61c00ade30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e980 tx=0x7f61b0009250 comp rx=0 tx=0).stop 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 shutdown_connections 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f61a4060a70 0x7f61a4062f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61c0159090 0x7f61c0169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c00b24c0 0x7f61c00a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 --2- 172.21.15.64:0/2103376389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61c00af620 0x7f61c00ade30 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.236+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 >> 172.21.15.64:0/2103376389 conn(0x7f61c00d0ab0 msgr2=0x7f61c00ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.240+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 shutdown_connections 2023-09-27T23:52:57.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.240+0000 7f61c875f700 1 -- 172.21.15.64:0/2103376389 wait complete. 2023-09-27T23:52:57.245 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2023-09-27T23:52:57.255 INFO:tasks.ceph:Scrubbing osd.2 2023-09-27T23:52:57.255 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:57.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.364+0000 7fb098059700 1 Processor -- start 2023-09-27T23:52:57.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.364+0000 7fb098059700 1 -- start start 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.364+0000 7fb098059700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb090068230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.364+0000 7fb098059700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb090067da0 con 0x7fb090068770 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.364+0000 7fb098059700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb090067190 con 0x7fb090159030 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.364+0000 7fb098059700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900cb880 con 0x7fb090159400 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0965f6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb090068770 0x7fb0900ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41804/0 (socket says 172.21.15.64:41804) 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0965f6700 1 -- 172.21.15.64:0/2222000581 learned_addr learned my addr 172.21.15.64:0/2222000581 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:57.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0955f4700 1 --2- 172.21.15.64:0/2222000581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb090068230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227217526 0 0) 0x7fb090067da0 con 0x7fb090068770 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c003570 con 0x7fb090068770 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254314799 0 0) 0x7fb090067190 con 0x7fb090159030 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb090067da0 con 0x7fb090159030 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0955f4700 1 -- 172.21.15.64:0/2222000581 >> v1:172.21.15.123:6789/0 conn(0x7fb090159030 legacy=0x7fb0900cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0955f4700 1 -- 172.21.15.64:0/2222000581 >> v1:172.21.15.64:6789/0 conn(0x7fb090068770 legacy=0x7fb0900ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1729889376 0 0) 0x7fb07c003570 con 0x7fb090068770 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0955f4700 1 -- 172.21.15.64:0/2222000581 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090067470 con 0x7fb090159400 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb0955f4700 1 --2- 172.21.15.64:0/2222000581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb090068230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009910 tx=0x7fb08c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f9e3416f879332 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0880017f0 con 0x7fb090159400 2023-09-27T23:52:57.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c011420 con 0x7fb090159400 2023-09-27T23:52:57.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb094df3700 1 -- 172.21.15.64:0/2222000581 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0117c0 con 0x7fb090159400 2023-09-27T23:52:57.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 -- 172.21.15.64:0/2222000581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 msgr2=0x7fb090068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 --2- 172.21.15.64:0/2222000581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb090068230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009910 tx=0x7fb08c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 -- 172.21.15.64:0/2222000581 shutdown_connections 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 --2- 172.21.15.64:0/2222000581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb090068230 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 -- 172.21.15.64:0/2222000581 >> 172.21.15.64:0/2222000581 conn(0x7fb0900d0170 msgr2=0x7fb0900d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 -- 172.21.15.64:0/2222000581 shutdown_connections 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.368+0000 7fb098059700 1 -- 172.21.15.64:0/2222000581 wait complete. 2023-09-27T23:52:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 Processor -- start 2023-09-27T23:52:57.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 -- start start 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb090068770 0x7fb0901409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb090159030 0x7fb0900c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb0900c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900c9b00 con 0x7fb090159030 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0901518d0 con 0x7fb090068770 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb090153500 con 0x7fb090159400 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb090159030 0x7fb0900c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb090159030 0x7fb0900c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53682/0 (socket says 172.21.15.64:53682) 2023-09-27T23:52:57.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 -- 172.21.15.64:0/2151929961 learned_addr learned my addr 172.21.15.64:0/2151929961 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb095df5700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb090068770 0x7fb0901409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 -- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 msgr2=0x7fb0900c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0965f6700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb0900c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb0900c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 -- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb090068770 msgr2=0x7fb0901409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb090068770 0x7fb0901409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 -- 172.21.15.64:0/2151929961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c009420 con 0x7fb090159030 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb095df5700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb090068770 0x7fb0901409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0965f6700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb090159400 0x7fb0900c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb0955f4700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb090159030 0x7fb0900c8760 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0030c0 tx=0x7fb08c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c01cc50 con 0x7fb090159030 2023-09-27T23:52:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c01cdb0 con 0x7fb090159030 2023-09-27T23:52:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0900c91e0 con 0x7fb090159030 2023-09-27T23:52:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c01a6a0 con 0x7fb090159030 2023-09-27T23:52:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.372+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0900c94b0 con 0x7fb090159030 2023-09-27T23:52:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0900708a0 con 0x7fb090159030 2023-09-27T23:52:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fb08c01aa90 con 0x7fb090159030 2023-09-27T23:52:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb085ffb700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb074060a70 0x7fb074062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fb08c07efc0 con 0x7fb090159030 2023-09-27T23:52:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb095df5700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb074060a70 0x7fb074062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb085ffb700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7fb0740679a0 0x7fb074069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 --> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb07406a4a0 con 0x7fb0740679a0 2023-09-27T23:52:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb08c07f390 con 0x7fb090159030 2023-09-27T23:52:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.376+0000 7fb0965f6700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7fb0740679a0 0x7fb074069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.380+0000 7fb095df5700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb074060a70 0x7fb074062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb080007fd0 tx=0x7fb080009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.380+0000 7fb0965f6700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7fb0740679a0 0x7fb074069dd0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.380+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== osd.2 v2:172.21.15.64:6824/33402 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb07406a4a0 con 0x7fb0740679a0 2023-09-27T23:52:57.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.396+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 --> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb090067750 con 0x7fb0740679a0 2023-09-27T23:52:57.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb085ffb700 1 -- 172.21.15.64:0/2151929961 <== osd.2 v2:172.21.15.64:6824/33402 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb090067750 con 0x7fb0740679a0 2023-09-27T23:52:57.404 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:57.404 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:57.404 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7fb0740679a0 msgr2=0x7fb074069dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:6824/33402,v1:172.21.15.64:6825/33402] conn(0x7fb0740679a0 0x7fb074069dd0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb074060a70 msgr2=0x7fb074062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fb074060a70 0x7fb074062f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb080007fd0 tx=0x7fb080009040 comp rx=0 tx=0).stop 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb090159030 msgr2=0x7fb0900c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 --2- 172.21.15.64:0/2151929961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb090159030 0x7fb0900c8760 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0030c0 tx=0x7fb08c016000 comp rx=0 tx=0).stop 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb0965f6700 1 -- 172.21.15.64:0/2151929961 reap_dead start 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 shutdown_connections 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 >> 172.21.15.64:0/2151929961 conn(0x7fb0900d0170 msgr2=0x7fb0900b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 shutdown_connections 2023-09-27T23:52:57.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.400+0000 7fb098059700 1 -- 172.21.15.64:0/2151929961 wait complete. 2023-09-27T23:52:57.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-09-27T23:52:57.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8162d33700 1 Processor -- start 2023-09-27T23:52:57.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8162d33700 1 -- start start 2023-09-27T23:52:57.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8162d33700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8162d33700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c0cb3e0 con 0x7f815c158e40 2023-09-27T23:52:57.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8162d33700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c068960 con 0x7f815c0b1be0 2023-09-27T23:52:57.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8162d33700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0caab0 con 0x7f815c159210 2023-09-27T23:52:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8161d31700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f815c158e40 0x7f815c067f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41814/0 (socket says 172.21.15.64:41814) 2023-09-27T23:52:57.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8161d31700 1 -- 172.21.15.64:0/456273156 learned_addr learned my addr 172.21.15.64:0/456273156 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.520+0000 7f8161530700 1 --2- 172.21.15.64:0/456273156 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073929266 0 0) 0x7f815c0cb3e0 con 0x7f815c158e40 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8148003570 con 0x7f815c158e40 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964335827 0 0) 0x7f815c068960 con 0x7f815c0b1be0 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815c0cb3e0 con 0x7f815c0b1be0 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8161530700 1 -- 172.21.15.64:0/456273156 >> v1:172.21.15.123:6789/0 conn(0x7f815c0b1be0 legacy=0x7f815c0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8161530700 1 -- 172.21.15.64:0/456273156 >> v1:172.21.15.64:6789/0 conn(0x7f815c158e40 legacy=0x7f815c067f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8161530700 1 -- 172.21.15.64:0/456273156 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0cad90 con 0x7f815c159210 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8161530700 1 --2- 172.21.15.64:0/456273156 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0b2c00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8150009910 tx=0x7f81500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3781fa038ed080 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150011630 con 0x7f815c159210 2023-09-27T23:52:57.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150011790 con 0x7f815c159210 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8160d2f700 1 -- 172.21.15.64:0/456273156 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150011b30 con 0x7f815c159210 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 -- 172.21.15.64:0/456273156 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 msgr2=0x7f815c0b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 --2- 172.21.15.64:0/456273156 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0b2c00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8150009910 tx=0x7f81500091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 -- 172.21.15.64:0/456273156 shutdown_connections 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 --2- 172.21.15.64:0/456273156 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0b2c00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 -- 172.21.15.64:0/456273156 >> 172.21.15.64:0/456273156 conn(0x7f815c0d0900 msgr2=0x7f815c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 -- 172.21.15.64:0/456273156 shutdown_connections 2023-09-27T23:52:57.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.524+0000 7f8162d33700 1 -- 172.21.15.64:0/456273156 wait complete. 2023-09-27T23:52:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 Processor -- start 2023-09-27T23:52:57.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 -- start start 2023-09-27T23:52:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c0b1be0 0x7f815c150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 0x7f815c0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c8d10 con 0x7f815c158e40 2023-09-27T23:52:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c15b3e0 con 0x7f815c0b1be0 2023-09-27T23:52:57.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c16db70 con 0x7f815c159210 2023-09-27T23:52:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 0x7f815c0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 0x7f815c0c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53692/0 (socket says 172.21.15.64:53692) 2023-09-27T23:52:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 -- 172.21.15.64:0/3204628517 learned_addr learned my addr 172.21.15.64:0/3204628517 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161d31700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c0b1be0 0x7f815c150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162532700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 -- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 msgr2=0x7f815c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 -- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c0b1be0 msgr2=0x7f815c150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c0b1be0 0x7f815c150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 -- 172.21.15.64:0/3204628517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150009420 con 0x7f815c158e40 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8161530700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 0x7f815c0c81a0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f8150009f00 tx=0x7f8150017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815001f070 con 0x7f815c158e40 2023-09-27T23:52:57.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150002f70 con 0x7f815c158e40 2023-09-27T23:52:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c16cdb0 con 0x7f815c158e40 2023-09-27T23:52:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150011900 con 0x7f815c158e40 2023-09-27T23:52:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.528+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c16d370 con 0x7f815c158e40 2023-09-27T23:52:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.532+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c04be40 con 0x7f815c158e40 2023-09-27T23:52:57.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.532+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8150023070 con 0x7f815c158e40 2023-09-27T23:52:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.532+0000 7f814dffb700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8140060a70 0x7f8140062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.532+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f815007e2a0 con 0x7f815c158e40 2023-09-27T23:52:57.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.532+0000 7f8161d31700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8140060a70 0x7f8140062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.536+0000 7f8161d31700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8140060a70 0x7f8140062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8158007fd0 tx=0x7f8158009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.540+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f815004e340 con 0x7f815c158e40 2023-09-27T23:52:57.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.700+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f815c0cb070 con 0x7f8140060a70 2023-09-27T23:52:57.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f814dffb700 1 -- 172.21.15.64:0/3204628517 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f815c0cb070 con 0x7f8140060a70 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8140060a70 msgr2=0x7f8140062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8140060a70 0x7f8140062f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8158007fd0 tx=0x7f8158009040 comp rx=0 tx=0).stop 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 msgr2=0x7f815c0c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 0x7f815c0c81a0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f8150009f00 tx=0x7f8150017000 comp rx=0 tx=0).stop 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 shutdown_connections 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8140060a70 0x7f8140062f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f815c159210 0x7f815c0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f815c158e40 0x7f815c0c81a0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 --2- 172.21.15.64:0/3204628517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f815c0b1be0 0x7f815c150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 >> 172.21.15.64:0/3204628517 conn(0x7f815c0d0900 msgr2=0x7f815c0cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 shutdown_connections 2023-09-27T23:52:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.704+0000 7f8162d33700 1 -- 172.21.15.64:0/3204628517 wait complete. 2023-09-27T23:52:57.712 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2023-09-27T23:52:57.723 INFO:tasks.ceph:Scrubbing osd.3 2023-09-27T23:52:57.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:57.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.828+0000 7f7840f2a700 1 Processor -- start 2023-09-27T23:52:57.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.828+0000 7f7840f2a700 1 -- start start 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f783c15cee0 0x7f783c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0b0b40 con 0x7f783c158b20 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0cea80 con 0x7f783c0cb3a0 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c1596b0 con 0x7f783c15cee0 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f783c158b20 0x7f783c158fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41826/0 (socket says 172.21.15.64:41826) 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f783c15cee0 0x7f783c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783bfff700 1 -- 172.21.15.64:0/48495775 learned_addr learned my addr 172.21.15.64:0/48495775 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:57.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783affd700 1 -- 172.21.15.64:0/48495775 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462989481 0 0) 0x7f783c0b0b40 con 0x7f783c158b20 2023-09-27T23:52:57.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783affd700 1 -- 172.21.15.64:0/48495775 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003570 con 0x7f783c158b20 2023-09-27T23:52:57.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783b7fe700 1 -- 172.21.15.64:0/48495775 >> v1:172.21.15.123:6789/0 conn(0x7f783c0cb3a0 legacy=0x7f783c0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783b7fe700 1 -- 172.21.15.64:0/48495775 >> v1:172.21.15.64:6789/0 conn(0x7f783c158b20 legacy=0x7f783c158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783b7fe700 1 -- 172.21.15.64:0/48495775 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c0ced60 con 0x7f783c15cee0 2023-09-27T23:52:57.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783b7fe700 1 --2- 172.21.15.64:0/48495775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c15cee0 0x7f783c15d2b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f78240098e0 tx=0x7f78240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74c73c75b5bfb80 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783affd700 1 -- 172.21.15.64:0/48495775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824011630 con 0x7f783c15cee0 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783affd700 1 -- 172.21.15.64:0/48495775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824011790 con 0x7f783c15cee0 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f783affd700 1 -- 172.21.15.64:0/48495775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824011b30 con 0x7f783c15cee0 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- 172.21.15.64:0/48495775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c15cee0 msgr2=0x7f783c15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 --2- 172.21.15.64:0/48495775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c15cee0 0x7f783c15d2b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f78240098e0 tx=0x7f78240091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- 172.21.15.64:0/48495775 shutdown_connections 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 --2- 172.21.15.64:0/48495775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c15cee0 0x7f783c15d2b0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- 172.21.15.64:0/48495775 >> 172.21.15.64:0/48495775 conn(0x7f783c0cff20 msgr2=0x7f783c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- 172.21.15.64:0/48495775 shutdown_connections 2023-09-27T23:52:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.832+0000 7f7840f2a700 1 -- 172.21.15.64:0/48495775 wait complete. 2023-09-27T23:52:57.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 Processor -- start 2023-09-27T23:52:57.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 -- start start 2023-09-27T23:52:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb3a0 0x7f783c0c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c158b20 0x7f783c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c15cee0 0x7f783c1697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0ca130 con 0x7f783c158b20 2023-09-27T23:52:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0c94b0 con 0x7f783c0cb3a0 2023-09-27T23:52:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7840f2a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0c9790 con 0x7f783c15cee0 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c158b20 0x7f783c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c158b20 0x7f783c0c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53704/0 (socket says 172.21.15.64:53704) 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 -- 172.21.15.64:0/1169241290 learned_addr learned my addr 172.21.15.64:0/1169241290 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f783b7fe700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb3a0 0x7f783c0c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f783bfff700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c15cee0 0x7f783c1697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 -- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c15cee0 msgr2=0x7f783c1697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f783c15cee0 0x7f783c1697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 -- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb3a0 msgr2=0x7f783c0c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f783c0cb3a0 0x7f783c0c6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.836+0000 7f7833fff700 1 -- 172.21.15.64:0/1169241290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824009420 con 0x7f783c158b20 2023-09-27T23:52:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f7833fff700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c158b20 0x7f783c0c7510 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c590 tx=0x7f782c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c011590 con 0x7f783c158b20 2023-09-27T23:52:57.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c011710 con 0x7f783c158b20 2023-09-27T23:52:57.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c0c9a70 con 0x7f783c158b20 2023-09-27T23:52:57.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c011ad0 con 0x7f783c158b20 2023-09-27T23:52:57.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c1563c0 con 0x7f783c158b20 2023-09-27T23:52:57.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f783c04be40 con 0x7f783c158b20 2023-09-27T23:52:57.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f782c01b3b0 con 0x7f783c158b20 2023-09-27T23:52:57.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f78337fe700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7820060a70 0x7f7820062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.840+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f782c07cc70 con 0x7f783c158b20 2023-09-27T23:52:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f783b7fe700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7820060a70 0x7f7820062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f78337fe700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7f7820067920 0x7f7820069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 --> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f782006a420 con 0x7f7820067920 2023-09-27T23:52:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f782c07cf60 con 0x7f783c158b20 2023-09-27T23:52:57.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f783bfff700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7f7820067920 0x7f7820069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:57.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f783b7fe700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7820060a70 0x7f7820062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78240091e0 tx=0x7f7824016000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f783bfff700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7f7820067920 0x7f7820069d50 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:57.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.844+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== osd.3 v2:172.21.15.64:6816/33401 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f782006a420 con 0x7f7820067920 2023-09-27T23:52:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.860+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 --> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f783c0cf040 con 0x7f7820067920 2023-09-27T23:52:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.860+0000 7f78337fe700 1 -- 172.21.15.64:0/1169241290 <== osd.3 v2:172.21.15.64:6816/33401 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f783c0cf040 con 0x7f7820067920 2023-09-27T23:52:57.868 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:57.868 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:57.868 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7f7820067920 msgr2=0x7f7820069d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:6816/33401,v1:172.21.15.64:6817/33401] conn(0x7f7820067920 0x7f7820069d50 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7820060a70 msgr2=0x7f7820062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f7820060a70 0x7f7820062f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78240091e0 tx=0x7f7824016000 comp rx=0 tx=0).stop 2023-09-27T23:52:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c158b20 msgr2=0x7f783c0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 --2- 172.21.15.64:0/1169241290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f783c158b20 0x7f783c0c7510 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c590 tx=0x7f782c007f10 comp rx=0 tx=0).stop 2023-09-27T23:52:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f783bfff700 1 -- 172.21.15.64:0/1169241290 reap_dead start 2023-09-27T23:52:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 shutdown_connections 2023-09-27T23:52:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 >> 172.21.15.64:0/1169241290 conn(0x7f783c0cff20 msgr2=0x7f783c158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 shutdown_connections 2023-09-27T23:52:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.864+0000 7f7840f2a700 1 -- 172.21.15.64:0/1169241290 wait complete. 2023-09-27T23:52:57.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-09-27T23:52:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.988+0000 7ff2b2195700 1 Processor -- start 2023-09-27T23:52:57.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b2195700 1 -- start start 2023-09-27T23:52:57.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b2195700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:57.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b2195700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac068670 con 0x7ff2ac069f50 2023-09-27T23:52:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b2195700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac0c6690 con 0x7ff2ac159040 2023-09-27T23:52:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b2195700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0c6970 con 0x7ff2ac1586f0 2023-09-27T23:52:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1994700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff2ac069f50 0x7ff2ac0ac140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41828/0 (socket says 172.21.15.64:41828) 2023-09-27T23:52:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1994700 1 -- 172.21.15.64:0/3705261407 learned_addr learned my addr 172.21.15.64:0/3705261407 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:57.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1193700 1 --2- 172.21.15.64:0/3705261407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389955000 0 0) 0x7ff2ac068670 con 0x7ff2ac069f50 2023-09-27T23:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff298003570 con 0x7ff2ac069f50 2023-09-27T23:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244258274 0 0) 0x7ff2ac0c6690 con 0x7ff2ac159040 2023-09-27T23:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ac068670 con 0x7ff2ac159040 2023-09-27T23:52:58.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1193700 1 -- 172.21.15.64:0/3705261407 >> v1:172.21.15.123:6789/0 conn(0x7ff2ac159040 legacy=0x7ff2ac069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1193700 1 -- 172.21.15.64:0/3705261407 >> v1:172.21.15.64:6789/0 conn(0x7ff2ac069f50 legacy=0x7ff2ac0ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1193700 1 -- 172.21.15.64:0/3705261407 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0c6c50 con 0x7ff2ac1586f0 2023-09-27T23:52:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.992+0000 7ff2b1193700 1 --2- 172.21.15.64:0/3705261407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0c6000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009910 tx=0x7ff2a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a86e62812da94e server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8011630 con 0x7ff2ac1586f0 2023-09-27T23:52:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a8011790 con 0x7ff2ac1586f0 2023-09-27T23:52:58.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff29ffff700 1 -- 172.21.15.64:0/3705261407 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8011b10 con 0x7ff2ac1586f0 2023-09-27T23:52:58.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 -- 172.21.15.64:0/3705261407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 msgr2=0x7ff2ac0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 --2- 172.21.15.64:0/3705261407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0c6000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009910 tx=0x7ff2a80091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 -- 172.21.15.64:0/3705261407 shutdown_connections 2023-09-27T23:52:58.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 --2- 172.21.15.64:0/3705261407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0c6000 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 -- 172.21.15.64:0/3705261407 >> 172.21.15.64:0/3705261407 conn(0x7ff2ac0cc530 msgr2=0x7ff2ac0c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 -- 172.21.15.64:0/3705261407 shutdown_connections 2023-09-27T23:52:58.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:57.996+0000 7ff2b2195700 1 -- 172.21.15.64:0/3705261407 wait complete. 2023-09-27T23:52:58.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 Processor -- start 2023-09-27T23:52:58.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 -- start start 2023-09-27T23:52:58.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 0x7ff2ac0a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2ac159040 0x7ff2ac168c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0c24a0 con 0x7ff2ac069f50 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0af6a0 con 0x7ff2ac159040 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b2195700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0c5380 con 0x7ff2ac1586f0 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 0x7ff2ac0a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 0x7ff2ac0a44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53716/0 (socket says 172.21.15.64:53716) 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 -- 172.21.15.64:0/398988032 learned_addr learned my addr 172.21.15.64:0/398988032 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b0992700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1994700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2ac159040 0x7ff2ac168c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 -- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 msgr2=0x7ff2ac0a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 -- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2ac159040 msgr2=0x7ff2ac168c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2ac159040 0x7ff2ac168c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1193700 1 -- 172.21.15.64:0/398988032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a8009420 con 0x7ff2ac069f50 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.000+0000 7ff2b1994700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2ac159040 0x7ff2ac168c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff2b1193700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 0x7ff2ac0a44a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009860 tx=0x7ff2a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80119d0 con 0x7ff2ac069f50 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a8027420 con 0x7ff2ac069f50 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac0c5660 con 0x7ff2ac069f50 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80277e0 con 0x7ff2ac069f50 2023-09-27T23:52:58.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0af330 con 0x7ff2ac069f50 2023-09-27T23:52:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac04be40 con 0x7ff2ac069f50 2023-09-27T23:52:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7ff2a8025070 con 0x7ff2ac069f50 2023-09-27T23:52:58.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff29d7fa700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff290060a70 0x7ff290062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.004+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7ff2a80807c0 con 0x7ff2ac069f50 2023-09-27T23:52:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.008+0000 7ff2b0992700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff290060a70 0x7ff290062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.008+0000 7ff2b0992700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff290060a70 0x7ff290062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0c26a0 tx=0x7ff2a0007d50 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.012+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7ff2a8050860 con 0x7ff2ac069f50 2023-09-27T23:52:58.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.176+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2ac0af7e0 con 0x7ff290060a70 2023-09-27T23:52:58.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.176+0000 7ff29d7fa700 1 -- 172.21.15.64:0/398988032 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff2a000f6b0 con 0x7ff290060a70 2023-09-27T23:52:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff290060a70 msgr2=0x7ff290062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff290060a70 0x7ff290062f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0c26a0 tx=0x7ff2a0007d50 comp rx=0 tx=0).stop 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 msgr2=0x7ff2ac0a44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 0x7ff2ac0a44a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009860 tx=0x7ff2a8017000 comp rx=0 tx=0).stop 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 shutdown_connections 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7ff290060a70 0x7ff290062f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2ac159040 0x7ff2ac168c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2ac1586f0 0x7ff2ac0a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 --2- 172.21.15.64:0/398988032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2ac069f50 0x7ff2ac0a44a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 >> 172.21.15.64:0/398988032 conn(0x7ff2ac0cc530 msgr2=0x7ff2ac157de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 shutdown_connections 2023-09-27T23:52:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.180+0000 7ff2b2195700 1 -- 172.21.15.64:0/398988032 wait complete. 2023-09-27T23:52:58.187 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2023-09-27T23:52:58.197 INFO:tasks.ceph:Scrubbing osd.4 2023-09-27T23:52:58.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.304+0000 7f8ebbfff700 1 Processor -- start 2023-09-27T23:52:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.304+0000 7f8ebbfff700 1 -- start start 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ebc15cee0 0x7f8ebc15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ebc0cb3a0 0x7f8ebc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ebc0cea80 con 0x7f8ebc158b20 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ebc1596b0 con 0x7f8ebc15cee0 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ebc0b0b40 con 0x7f8ebc0cb3a0 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ebc15cee0 0x7f8ebc15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ebc15cee0 0x7f8ebc15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53726/0 (socket says 172.21.15.64:53726) 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 -- 172.21.15.64:0/2523277360 learned_addr learned my addr 172.21.15.64:0/2523277360 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8eba7fc700 1 --2- 172.21.15.64:0/2523277360 >> v2:172.21.15.123:3300/0 conn(0x7f8ebc0cb3a0 0x7f8ebc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 -- 172.21.15.64:0/2523277360 >> v2:172.21.15.123:3300/0 conn(0x7f8ebc0cb3a0 msgr2=0x7f8ebc0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 --2- 172.21.15.64:0/2523277360 >> v2:172.21.15.123:3300/0 conn(0x7f8ebc0cb3a0 0x7f8ebc0ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 -- 172.21.15.64:0/2523277360 >> v1:172.21.15.123:6789/0 conn(0x7f8ebc158b20 legacy=0x7f8ebc158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 -- 172.21.15.64:0/2523277360 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0ced60 con 0x7f8ebc15cee0 2023-09-27T23:52:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebaffd700 1 --2- 172.21.15.64:0/2523277360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc15d2b0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc088440 tx=0x7f8eac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f70d9bc763d9ec server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8eb9ffb700 1 -- 172.21.15.64:0/2523277360 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00e070 con 0x7f8ebc15cee0 2023-09-27T23:52:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8eb9ffb700 1 -- 172.21.15.64:0/2523277360 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac00ba10 con 0x7f8ebc15cee0 2023-09-27T23:52:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8eb9ffb700 1 -- 172.21.15.64:0/2523277360 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0092e0 con 0x7f8ebc15cee0 2023-09-27T23:52:58.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- 172.21.15.64:0/2523277360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc15cee0 msgr2=0x7f8ebc15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 --2- 172.21.15.64:0/2523277360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc15d2b0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc088440 tx=0x7f8eac009840 comp rx=0 tx=0).stop 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- 172.21.15.64:0/2523277360 shutdown_connections 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 --2- 172.21.15.64:0/2523277360 >> v2:172.21.15.123:3300/0 conn(0x7f8ebc0cb3a0 0x7f8ebc0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 --2- 172.21.15.64:0/2523277360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc15d2b0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- 172.21.15.64:0/2523277360 >> 172.21.15.64:0/2523277360 conn(0x7f8ebc0cff20 msgr2=0x7f8ebc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- 172.21.15.64:0/2523277360 shutdown_connections 2023-09-27T23:52:58.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.308+0000 7f8ebbfff700 1 -- 172.21.15.64:0/2523277360 wait complete. 2023-09-27T23:52:58.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.312+0000 7f8ebbfff700 1 Processor -- start 2023-09-27T23:52:58.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.312+0000 7f8ebbfff700 1 -- start start 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ebc0cb3a0 0x7f8ebc15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc158b20 0x7f8ebc14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0c7c80 con 0x7f8ebc158b20 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0c7070 con 0x7f8ebc15cee0 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0c7350 con 0x7f8ebc0cb3a0 2023-09-27T23:52:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc158b20 0x7f8ebc14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc158b20 0x7f8ebc14a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53730/0 (socket says 172.21.15.64:53730) 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 -- 172.21.15.64:0/550084421 learned_addr learned my addr 172.21.15.64:0/550084421 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebb7fe700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebaffd700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ebc0cb3a0 0x7f8ebc15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 -- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ebc0cb3a0 msgr2=0x7f8ebc15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ebc0cb3a0 0x7f8ebc15a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 -- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ebc15cee0 msgr2=0x7f8ebc14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 -- 172.21.15.64:0/550084421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009ae0 con 0x7f8ebc158b20 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebb7fe700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ebc15cee0 0x7f8ebc14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:58.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8eba7fc700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc158b20 0x7f8ebc14a7f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006b00 tx=0x7f8eb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00125f0 con 0x7f8ebc158b20 2023-09-27T23:52:58.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0012770 con 0x7f8ebc158b20 2023-09-27T23:52:58.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc0c7630 con 0x7f8ebc158b20 2023-09-27T23:52:58.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0012b30 con 0x7f8ebc158b20 2023-09-27T23:52:58.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc15a8e0 con 0x7f8ebc158b20 2023-09-27T23:52:58.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.316+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ebc04be40 con 0x7f8ebc158b20 2023-09-27T23:52:58.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8eb001e460 con 0x7f8ebc158b20 2023-09-27T23:52:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ea2ffd700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8ea8060a70 0x7f8ea8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f8eb007d750 con 0x7f8ebc158b20 2023-09-27T23:52:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ea2ffd700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f8ea80679a0 0x7f8ea8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 --> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ea806a4a0 con 0x7f8ea80679a0 2023-09-27T23:52:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb007fcb0 con 0x7f8ebc158b20 2023-09-27T23:52:58.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ebaffd700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8ea8060a70 0x7f8ea8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ebb7fe700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f8ea80679a0 0x7f8ea8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ebaffd700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8ea8060a70 0x7f8ea8062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009880 tx=0x7f8eac017000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.320+0000 7f8ebb7fe700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f8ea80679a0 0x7f8ea8069dd0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.324+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== osd.4 v2:172.21.15.123:6808/33568 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8ea806a4a0 con 0x7f8ea80679a0 2023-09-27T23:52:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.340+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 --> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8ebc0cf040 con 0x7f8ea80679a0 2023-09-27T23:52:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ea2ffd700 1 -- 172.21.15.64:0/550084421 <== osd.4 v2:172.21.15.123:6808/33568 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8ebc0cf040 con 0x7f8ea80679a0 2023-09-27T23:52:58.350 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:58.350 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:58.351 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f8ea80679a0 msgr2=0x7f8ea8069dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6808/33568,v1:172.21.15.123:6809/33568] conn(0x7f8ea80679a0 0x7f8ea8069dd0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8ea8060a70 msgr2=0x7f8ea8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f8ea8060a70 0x7f8ea8062f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009880 tx=0x7f8eac017000 comp rx=0 tx=0).stop 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc158b20 msgr2=0x7f8ebc14a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 --2- 172.21.15.64:0/550084421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ebc158b20 0x7f8ebc14a7f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006b00 tx=0x7f8eb0008040 comp rx=0 tx=0).stop 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebb7fe700 1 -- 172.21.15.64:0/550084421 reap_dead start 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 shutdown_connections 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 >> 172.21.15.64:0/550084421 conn(0x7f8ebc0cff20 msgr2=0x7f8ebc0d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 shutdown_connections 2023-09-27T23:52:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.344+0000 7f8ebbfff700 1 -- 172.21.15.64:0/550084421 wait complete. 2023-09-27T23:52:58.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-09-27T23:52:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 Processor -- start 2023-09-27T23:52:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 -- start start 2023-09-27T23:52:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f996006bc20 0x7f99600cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99600ca700 0x7f99600af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99601570b0 con 0x7f9960156ce0 2023-09-27T23:52:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99600cabe0 con 0x7f99600ca700 2023-09-27T23:52:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966932700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99600caec0 con 0x7f996006bc20 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99600ca700 0x7f99600af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99600ca700 0x7f99600af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53740/0 (socket says 172.21.15.64:53740) 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 -- 172.21.15.64:0/297772784 learned_addr learned my addr 172.21.15.64:0/297772784 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f996512f700 1 --2- 172.21.15.64:0/297772784 >> v2:172.21.15.123:3300/0 conn(0x7f996006bc20 0x7f99600cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 -- 172.21.15.64:0/297772784 >> v2:172.21.15.123:3300/0 conn(0x7f996006bc20 msgr2=0x7f99600cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 --2- 172.21.15.64:0/297772784 >> v2:172.21.15.123:3300/0 conn(0x7f996006bc20 0x7f99600cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 -- 172.21.15.64:0/297772784 >> v1:172.21.15.123:6789/0 conn(0x7f9960156ce0 legacy=0x7f996006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.472+0000 7f9966131700 1 -- 172.21.15.64:0/297772784 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600cb1a0 con 0x7f99600ca700 2023-09-27T23:52:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966131700 1 --2- 172.21.15.64:0/297772784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99600ca700 0x7f99600af0a0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f996006e110 tx=0x7f9958009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754662546aab6a77 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f996492e700 1 -- 172.21.15.64:0/297772784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995800e070 con 0x7f99600ca700 2023-09-27T23:52:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f996492e700 1 -- 172.21.15.64:0/297772784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995800ba20 con 0x7f99600ca700 2023-09-27T23:52:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f996492e700 1 -- 172.21.15.64:0/297772784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99580092e0 con 0x7f99600ca700 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 -- 172.21.15.64:0/297772784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99600ca700 msgr2=0x7f99600af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 --2- 172.21.15.64:0/297772784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99600ca700 0x7f99600af0a0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f996006e110 tx=0x7f9958009840 comp rx=0 tx=0).stop 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 -- 172.21.15.64:0/297772784 shutdown_connections 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 --2- 172.21.15.64:0/297772784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99600ca700 0x7f99600af0a0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 --2- 172.21.15.64:0/297772784 >> v2:172.21.15.123:3300/0 conn(0x7f996006bc20 0x7f99600cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 -- 172.21.15.64:0/297772784 >> 172.21.15.64:0/297772784 conn(0x7f996006e160 msgr2=0x7f99600adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 -- 172.21.15.64:0/297772784 shutdown_connections 2023-09-27T23:52:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 -- 172.21.15.64:0/297772784 wait complete. 2023-09-27T23:52:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 Processor -- start 2023-09-27T23:52:58.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.476+0000 7f9966932700 1 -- start start 2023-09-27T23:52:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f996006bc20 0x7f9960141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99600ca700 0x7f996014ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 0x7f99600b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996014d9f0 con 0x7f9960156ce0 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996014e570 con 0x7f996006bc20 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9960142380 con 0x7f99600ca700 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 0x7f99600b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 0x7f99600b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53746/0 (socket says 172.21.15.64:53746) 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 -- 172.21.15.64:0/275771420 learned_addr learned my addr 172.21.15.64:0/275771420 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f996512f700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99600ca700 0x7f996014ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9965930700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f996006bc20 0x7f9960141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 -- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99600ca700 msgr2=0x7f996014ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99600ca700 0x7f996014ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 -- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f996006bc20 msgr2=0x7f9960141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f996006bc20 0x7f9960141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 -- 172.21.15.64:0/275771420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958009ae0 con 0x7f9960156ce0 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966131700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 0x7f99600b1860 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9958012000 tx=0x7f9958018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99580089b0 con 0x7f9960156ce0 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996014dc40 con 0x7f9960156ce0 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9958008b10 con 0x7f9960156ce0 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996014e200 con 0x7f9960156ce0 2023-09-27T23:52:58.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.480+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958026460 con 0x7f9960156ce0 2023-09-27T23:52:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.484+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996014e700 con 0x7f9960156ce0 2023-09-27T23:52:58.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.484+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f9958003040 con 0x7f9960156ce0 2023-09-27T23:52:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.484+0000 7f9951ffb700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9944060aa0 0x7f9944062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.484+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f995807eff0 con 0x7f9960156ce0 2023-09-27T23:52:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.484+0000 7f9965930700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9944060aa0 0x7f9944062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.484+0000 7f9965930700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9944060aa0 0x7f9944062f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f996006df60 tx=0x7f995c007f10 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.488+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f995804ef80 con 0x7f9960156ce0 2023-09-27T23:52:58.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.648+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f996004be40 con 0x7f9944060aa0 2023-09-27T23:52:58.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.648+0000 7f9951ffb700 1 -- 172.21.15.64:0/275771420 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f996004be40 con 0x7f9944060aa0 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9944060aa0 msgr2=0x7f9944062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9944060aa0 0x7f9944062f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f996006df60 tx=0x7f995c007f10 comp rx=0 tx=0).stop 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 msgr2=0x7f99600b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 0x7f99600b1860 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9958012000 tx=0x7f9958018000 comp rx=0 tx=0).stop 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 shutdown_connections 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f9944060aa0 0x7f9944062f50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9960156ce0 0x7f99600b1860 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99600ca700 0x7f996014ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 --2- 172.21.15.64:0/275771420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f996006bc20 0x7f9960141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 >> 172.21.15.64:0/275771420 conn(0x7f996006e160 msgr2=0x7f996014cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 shutdown_connections 2023-09-27T23:52:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.652+0000 7f9966932700 1 -- 172.21.15.64:0/275771420 wait complete. 2023-09-27T23:52:58.659 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2023-09-27T23:52:58.669 INFO:tasks.ceph:Scrubbing osd.5 2023-09-27T23:52:58.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:58.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.780+0000 7f1f1ffff700 1 Processor -- start 2023-09-27T23:52:58.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.780+0000 7f1f1ffff700 1 -- start start 2023-09-27T23:52:58.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f20068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f20159700 0x7f1f20159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200680e0 con 0x7f1f200cc4d0 2023-09-27T23:52:58.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1ffff700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2015a740 con 0x7f1f20159700 2023-09-27T23:52:58.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2015a460 con 0x7f1f2006c860 2023-09-27T23:52:58.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1f200cc4d0 0x7f1f2006c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41836/0 (socket says 172.21.15.64:41836) 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1effd700 1 -- 172.21.15.64:0/973646322 learned_addr learned my addr 172.21.15.64:0/973646322 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1e7fc700 1 --2- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f20068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1f7fe700 1 --2- 172.21.15.64:0/973646322 >> v2:172.21.15.123:3300/0 conn(0x7f1f20159700 0x7f1f20159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1dffb700 1 -- 172.21.15.64:0/973646322 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089843965 0 0) 0x7f1f200680e0 con 0x7f1f200cc4d0 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1dffb700 1 -- 172.21.15.64:0/973646322 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f08003570 con 0x7f1f200cc4d0 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1f7fe700 1 -- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 msgr2=0x7f1f20068570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1f7fe700 1 --2- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f20068570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1f7fe700 1 -- 172.21.15.64:0/973646322 >> v1:172.21.15.64:6789/0 conn(0x7f1f200cc4d0 legacy=0x7f1f2006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1f7fe700 1 -- 172.21.15.64:0/973646322 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20067a90 con 0x7f1f20159700 2023-09-27T23:52:58.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1f7fe700 1 --2- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 0x7f1f20159b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1f140098e0 tx=0x7f1f140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4a0aac06ddd9d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1dffb700 1 -- 172.21.15.64:0/973646322 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14011630 con 0x7f1f20159700 2023-09-27T23:52:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1dffb700 1 -- 172.21.15.64:0/973646322 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14011790 con 0x7f1f20159700 2023-09-27T23:52:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.784+0000 7f1f1dffb700 1 -- 172.21.15.64:0/973646322 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14011b30 con 0x7f1f20159700 2023-09-27T23:52:58.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 -- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 msgr2=0x7f1f20159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 --2- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 0x7f1f20159b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1f140098e0 tx=0x7f1f140091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 -- 172.21.15.64:0/973646322 shutdown_connections 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 --2- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 0x7f1f20159b60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 --2- 172.21.15.64:0/973646322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f20068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 -- 172.21.15.64:0/973646322 >> 172.21.15.64:0/973646322 conn(0x7f1f200ccbc0 msgr2=0x7f1f200ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 -- 172.21.15.64:0/973646322 shutdown_connections 2023-09-27T23:52:58.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 -- 172.21.15.64:0/973646322 wait complete. 2023-09-27T23:52:58.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 Processor -- start 2023-09-27T23:52:58.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.788+0000 7f1f1ffff700 1 -- start start 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f201525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200cc4d0 0x7f1f20153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 0x7f1f2015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20151430 con 0x7f1f200cc4d0 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f201515c0 con 0x7f1f20159700 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20152140 con 0x7f1f2006c860 2023-09-27T23:52:58.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200cc4d0 0x7f1f20153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200cc4d0 0x7f1f20153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53756/0 (socket says 172.21.15.64:53756) 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 -- 172.21.15.64:0/607235040 learned_addr learned my addr 172.21.15.64:0/607235040 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1effd700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f201525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1f7fe700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 0x7f1f2015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 -- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 msgr2=0x7f1f201525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f2006c860 0x7f1f201525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 -- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 msgr2=0x7f1f2015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f20159700 0x7f1f2015d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 -- 172.21.15.64:0/607235040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14009420 con 0x7f1f200cc4d0 2023-09-27T23:52:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1e7fc700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200cc4d0 0x7f1f20153fa0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e930 tx=0x7f1f100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10013680 con 0x7f1f200cc4d0 2023-09-27T23:52:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f100137e0 con 0x7f1f200cc4d0 2023-09-27T23:52:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f20151810 con 0x7f1f200cc4d0 2023-09-27T23:52:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10013b80 con 0x7f1f200cc4d0 2023-09-27T23:52:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f20151dd0 con 0x7f1f200cc4d0 2023-09-27T23:52:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.792+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f2004be40 con 0x7f1f200cc4d0 2023-09-27T23:52:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f1f1000c3e0 con 0x7f1f200cc4d0 2023-09-27T23:52:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f0c060a70 0x7f1f0c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f1f1007d120 con 0x7f1f200cc4d0 2023-09-27T23:52:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7f1f0c0679a0 0x7f1f0c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 --> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f0c06a4a0 con 0x7f1f0c0679a0 2023-09-27T23:52:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1007f760 con 0x7f1f200cc4d0 2023-09-27T23:52:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f1effd700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f0c060a70 0x7f1f0c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f1f7fe700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7f1f0c0679a0 0x7f1f0c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f1effd700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f0c060a70 0x7f1f0c062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005c50 tx=0x7f1f18009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f1f7fe700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7f1f0c0679a0 0x7f1f0c069dd0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.796+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== osd.5 v2:172.21.15.123:6817/33567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1f0c06a4a0 con 0x7f1f0c0679a0 2023-09-27T23:52:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.812+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 --> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1f200cda60 con 0x7f1f0c0679a0 2023-09-27T23:52:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f06ffd700 1 -- 172.21.15.64:0/607235040 <== osd.5 v2:172.21.15.123:6817/33567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1f200cda60 con 0x7f1f0c0679a0 2023-09-27T23:52:58.820 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:58.820 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:58.820 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7f1f0c0679a0 msgr2=0x7f1f0c069dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6817/33567,v1:172.21.15.123:6820/33567] conn(0x7f1f0c0679a0 0x7f1f0c069dd0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f0c060a70 msgr2=0x7f1f0c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f1f0c060a70 0x7f1f0c062f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005c50 tx=0x7f1f18009040 comp rx=0 tx=0).stop 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200cc4d0 msgr2=0x7f1f20153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 --2- 172.21.15.64:0/607235040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f200cc4d0 0x7f1f20153fa0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e930 tx=0x7f1f100091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1f7fe700 1 -- 172.21.15.64:0/607235040 reap_dead start 2023-09-27T23:52:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 shutdown_connections 2023-09-27T23:52:58.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 >> 172.21.15.64:0/607235040 conn(0x7f1f200ccbc0 msgr2=0x7f1f20068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 shutdown_connections 2023-09-27T23:52:58.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.816+0000 7f1f1ffff700 1 -- 172.21.15.64:0/607235040 wait complete. 2023-09-27T23:52:58.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-09-27T23:52:58.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff497a4700 1 Processor -- start 2023-09-27T23:52:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff497a4700 1 -- start start 2023-09-27T23:52:58.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff497a4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff440a9960 0x7eff440ab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff497a4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff440a8cb0 con 0x7eff44158eb0 2023-09-27T23:52:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff497a4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4406c7e0 con 0x7eff440a88a0 2023-09-27T23:52:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff497a4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff44067190 con 0x7eff440a9960 2023-09-27T23:52:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff43fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7eff44158eb0 0x7eff440a9250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41838/0 (socket says 172.21.15.64:41838) 2023-09-27T23:52:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff43fff700 1 -- 172.21.15.64:0/2720618085 learned_addr learned my addr 172.21.15.64:0/2720618085 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:58.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff437fe700 1 --2- 172.21.15.64:0/2720618085 >> v2:172.21.15.64:3300/0 conn(0x7eff440a9960 0x7eff440ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff42ffd700 1 -- 172.21.15.64:0/2720618085 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516819088 0 0) 0x7eff440a8cb0 con 0x7eff44158eb0 2023-09-27T23:52:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff42ffd700 1 -- 172.21.15.64:0/2720618085 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff2c003570 con 0x7eff44158eb0 2023-09-27T23:52:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff42ffd700 1 -- 172.21.15.64:0/2720618085 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137348426 0 0) 0x7eff2c003570 con 0x7eff44158eb0 2023-09-27T23:52:58.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff437fe700 1 -- 172.21.15.64:0/2720618085 >> v1:172.21.15.123:6789/0 conn(0x7eff440a88a0 legacy=0x7eff44068230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff437fe700 1 -- 172.21.15.64:0/2720618085 >> v1:172.21.15.64:6789/0 conn(0x7eff44158eb0 legacy=0x7eff440a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.936+0000 7eff437fe700 1 -- 172.21.15.64:0/2720618085 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44067470 con 0x7eff440a9960 2023-09-27T23:52:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff437fe700 1 --2- 172.21.15.64:0/2720618085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff440a9960 0x7eff440ab260 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7eff380098e0 tx=0x7eff380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312623c1b9531025 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff42ffd700 1 -- 172.21.15.64:0/2720618085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff340015f0 con 0x7eff440a9960 2023-09-27T23:52:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff42ffd700 1 -- 172.21.15.64:0/2720618085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff38011420 con 0x7eff440a9960 2023-09-27T23:52:58.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff42ffd700 1 -- 172.21.15.64:0/2720618085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380117a0 con 0x7eff440a9960 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 -- 172.21.15.64:0/2720618085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff440a9960 msgr2=0x7eff440ab260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 --2- 172.21.15.64:0/2720618085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff440a9960 0x7eff440ab260 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7eff380098e0 tx=0x7eff380091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 -- 172.21.15.64:0/2720618085 shutdown_connections 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 --2- 172.21.15.64:0/2720618085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff440a9960 0x7eff440ab260 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 -- 172.21.15.64:0/2720618085 >> 172.21.15.64:0/2720618085 conn(0x7eff4406e160 msgr2=0x7eff440adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 -- 172.21.15.64:0/2720618085 shutdown_connections 2023-09-27T23:52:58.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 -- 172.21.15.64:0/2720618085 wait complete. 2023-09-27T23:52:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 Processor -- start 2023-09-27T23:52:58.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.940+0000 7eff497a4700 1 -- start start 2023-09-27T23:52:58.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440a88a0 0x7eff440ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff440a9960 0x7eff44153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 0x7eff4414be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4414e9e0 con 0x7eff44158eb0 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff44150c70 con 0x7eff440a88a0 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4414e420 con 0x7eff440a9960 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 0x7eff4414be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 0x7eff4414be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53780/0 (socket says 172.21.15.64:53780) 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 -- 172.21.15.64:0/2903353793 learned_addr learned my addr 172.21.15.64:0/2903353793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff43fff700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440a88a0 0x7eff440ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff437fe700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff440a9960 0x7eff44153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 -- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff440a9960 msgr2=0x7eff44153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff440a9960 0x7eff44153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 -- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440a88a0 msgr2=0x7eff440ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440a88a0 0x7eff440ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 -- 172.21.15.64:0/2903353793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff38009420 con 0x7eff44158eb0 2023-09-27T23:52:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff437fe700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff440a9960 0x7eff44153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:58.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff48fa3700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 0x7eff4414be50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c8e0 tx=0x7eff3c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c012630 con 0x7eff44158eb0 2023-09-27T23:52:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c012790 con 0x7eff44158eb0 2023-09-27T23:52:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4414e700 con 0x7eff44158eb0 2023-09-27T23:52:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c012b50 con 0x7eff44158eb0 2023-09-27T23:52:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff44151160 con 0x7eff44158eb0 2023-09-27T23:52:58.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.944+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff44151350 con 0x7eff44158eb0 2023-09-27T23:52:58.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.948+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7eff3c00a3b0 con 0x7eff44158eb0 2023-09-27T23:52:58.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.948+0000 7eff2bfff700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff30060aa0 0x7eff30062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:58.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.948+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7eff3c07d2f0 con 0x7eff44158eb0 2023-09-27T23:52:58.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.948+0000 7eff43fff700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff30060aa0 0x7eff30062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:58.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.948+0000 7eff43fff700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff30060aa0 0x7eff30062f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff34000c00 tx=0x7eff34001640 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:58.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:58.952+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7eff3c050e20 con 0x7eff44158eb0 2023-09-27T23:52:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7eff4404be40 con 0x7eff30060aa0 2023-09-27T23:52:59.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff2bfff700 1 -- 172.21.15.64:0/2903353793 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7eff4404be40 con 0x7eff30060aa0 2023-09-27T23:52:59.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff30060aa0 msgr2=0x7eff30062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff30060aa0 0x7eff30062f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff34000c00 tx=0x7eff34001640 comp rx=0 tx=0).stop 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 msgr2=0x7eff4414be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 0x7eff4414be50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c8e0 tx=0x7eff3c001710 comp rx=0 tx=0).stop 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 shutdown_connections 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7eff30060aa0 0x7eff30062f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff44158eb0 0x7eff4414be50 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff440a9960 0x7eff44153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 --2- 172.21.15.64:0/2903353793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff440a88a0 0x7eff440ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.120+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 >> 172.21.15.64:0/2903353793 conn(0x7eff4406e160 msgr2=0x7eff440af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.124+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 shutdown_connections 2023-09-27T23:52:59.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.124+0000 7eff497a4700 1 -- 172.21.15.64:0/2903353793 wait complete. 2023-09-27T23:52:59.129 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2023-09-27T23:52:59.139 INFO:tasks.ceph:Scrubbing osd.6 2023-09-27T23:52:59.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:59.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 Processor -- start 2023-09-27T23:52:59.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 -- start start 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ef00cb3a0 0x7f3ef015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef0158b20 0x7f3ef0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00cea80 con 0x7f3ef015cee0 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00b0b40 con 0x7f3ef00cb3a0 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef5422700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00ced60 con 0x7f3ef0158b20 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ef00cb3a0 0x7f3ef015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ef00cb3a0 0x7f3ef015d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53796/0 (socket says 172.21.15.64:53796) 2023-09-27T23:52:59.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 -- 172.21.15.64:0/1314272287 learned_addr learned my addr 172.21.15.64:0/1314272287 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3ef4c21700 1 --2- 172.21.15.64:0/1314272287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef0158b20 0x7f3ef0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 -- 172.21.15.64:0/1314272287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef0158b20 msgr2=0x7f3ef0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 --2- 172.21.15.64:0/1314272287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef0158b20 0x7f3ef0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 -- 172.21.15.64:0/1314272287 >> v1:172.21.15.123:6789/0 conn(0x7f3ef015cee0 legacy=0x7f3ef00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 -- 172.21.15.64:0/1314272287 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00cf040 con 0x7f3ef00cb3a0 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eef7fe700 1 --2- 172.21.15.64:0/1314272287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef00cb3a0 0x7f3ef015d2b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00099b0 tx=0x7f3ee0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8356908bc2ff7d16 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eeeffd700 1 -- 172.21.15.64:0/1314272287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0009ea0 con 0x7f3ef00cb3a0 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eeeffd700 1 -- 172.21.15.64:0/1314272287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee000b860 con 0x7f3ef00cb3a0 2023-09-27T23:52:59.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.248+0000 7f3eeeffd700 1 -- 172.21.15.64:0/1314272287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000bc00 con 0x7f3ef00cb3a0 2023-09-27T23:52:59.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 -- 172.21.15.64:0/1314272287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef00cb3a0 msgr2=0x7f3ef015d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 --2- 172.21.15.64:0/1314272287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef00cb3a0 0x7f3ef015d2b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00099b0 tx=0x7f3ee0009250 comp rx=0 tx=0).stop 2023-09-27T23:52:59.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 -- 172.21.15.64:0/1314272287 shutdown_connections 2023-09-27T23:52:59.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 --2- 172.21.15.64:0/1314272287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef0158b20 0x7f3ef0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 --2- 172.21.15.64:0/1314272287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef00cb3a0 0x7f3ef015d2b0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 -- 172.21.15.64:0/1314272287 >> 172.21.15.64:0/1314272287 conn(0x7f3ef00cff20 msgr2=0x7f3ef00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 -- 172.21.15.64:0/1314272287 shutdown_connections 2023-09-27T23:52:59.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 -- 172.21.15.64:0/1314272287 wait complete. 2023-09-27T23:52:59.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 Processor -- start 2023-09-27T23:52:59.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.252+0000 7f3ef5422700 1 -- start start 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef00cb3a0 0x7f3ef01510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef0158b20 0x7f3ef00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef015cee0 0x7f3ef00c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef01692c0 con 0x7f3ef015cee0 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0151aa0 con 0x7f3ef0158b20 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0151d80 con 0x7f3ef00cb3a0 2023-09-27T23:52:59.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef015cee0 0x7f3ef00c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef015cee0 0x7f3ef00c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53810/0 (socket says 172.21.15.64:53810) 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 -- 172.21.15.64:0/247031190 learned_addr learned my addr 172.21.15.64:0/247031190 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3eef7fe700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef0158b20 0x7f3ef00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3eeffff700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef00cb3a0 0x7f3ef01510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 -- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef00cb3a0 msgr2=0x7f3ef01510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ef00cb3a0 0x7f3ef01510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 -- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef0158b20 msgr2=0x7f3ef00b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef0158b20 0x7f3ef00b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 -- 172.21.15.64:0/247031190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee00094f0 con 0x7f3ef015cee0 2023-09-27T23:52:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef4c21700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef015cee0 0x7f3ef00c9150 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009e90 tx=0x7f3ee800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800cc20 con 0x7f3ef015cee0 2023-09-27T23:52:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee800f040 con 0x7f3ef015cee0 2023-09-27T23:52:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef0152060 con 0x7f3ef015cee0 2023-09-27T23:52:59.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8014480 con 0x7f3ef015cee0 2023-09-27T23:52:59.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.256+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef01516c0 con 0x7f3ef015cee0 2023-09-27T23:52:59.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ef004be40 con 0x7f3ef015cee0 2023-09-27T23:52:59.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f3ee8014830 con 0x7f3ef015cee0 2023-09-27T23:52:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ee7fff700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3ed0060aa0 0x7f3ed0062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f3ee807e390 con 0x7f3ef015cee0 2023-09-27T23:52:59.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3eeffff700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3ed0060aa0 0x7f3ed0062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ee7fff700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7f3ed00679d0 0x7f3ed0069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 --> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ed006a4d0 con 0x7f3ed00679d0 2023-09-27T23:52:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.260+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee807e680 con 0x7f3ef015cee0 2023-09-27T23:52:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.264+0000 7f3eef7fe700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7f3ed00679d0 0x7f3ed0069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.264+0000 7f3eeffff700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3ed0060aa0 0x7f3ed0062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8000c00 tx=0x7f3ed8009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.264+0000 7f3eef7fe700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7f3ed00679d0 0x7f3ed0069e00 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.264+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== osd.6 v2:172.21.15.123:6800/33584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3ed006a4d0 con 0x7f3ed00679d0 2023-09-27T23:52:59.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 --> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3ef00cf320 con 0x7f3ed00679d0 2023-09-27T23:52:59.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ee7fff700 1 -- 172.21.15.64:0/247031190 <== osd.6 v2:172.21.15.123:6800/33584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3ef00cf320 con 0x7f3ed00679d0 2023-09-27T23:52:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:59.290 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:59.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7f3ed00679d0 msgr2=0x7f3ed0069e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6800/33584,v1:172.21.15.123:6801/33584] conn(0x7f3ed00679d0 0x7f3ed0069e00 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3ed0060aa0 msgr2=0x7f3ed0062f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f3ed0060aa0 0x7f3ed0062f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8000c00 tx=0x7f3ed8009040 comp rx=0 tx=0).stop 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef015cee0 msgr2=0x7f3ef00c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 --2- 172.21.15.64:0/247031190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ef015cee0 0x7f3ef00c9150 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009e90 tx=0x7f3ee800b040 comp rx=0 tx=0).stop 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef4c21700 1 -- 172.21.15.64:0/247031190 reap_dead start 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 shutdown_connections 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 >> 172.21.15.64:0/247031190 conn(0x7f3ef00cff20 msgr2=0x7f3ef00c4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 shutdown_connections 2023-09-27T23:52:59.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.284+0000 7f3ef5422700 1 -- 172.21.15.64:0/247031190 wait complete. 2023-09-27T23:52:59.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-09-27T23:52:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a1fda700 1 Processor -- start 2023-09-27T23:52:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a1fda700 1 -- start start 2023-09-27T23:52:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a1fda700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f989c159210 0x7f989c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a1fda700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0cb3e0 con 0x7f989c158e40 2023-09-27T23:52:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a1fda700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c068960 con 0x7f989c0b1be0 2023-09-27T23:52:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a1fda700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c0caab0 con 0x7f989c159210 2023-09-27T23:52:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f989c159210 0x7f989c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a0fd8700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f989c158e40 0x7f989c067f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41842/0 (socket says 172.21.15.64:41842) 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f989c159210 0x7f989c0b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53822/0 (socket says 172.21.15.64:53822) 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f98a0fd8700 1 -- 172.21.15.64:0/2666033031 learned_addr learned my addr 172.21.15.64:0/2666033031 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989bfff700 1 -- 172.21.15.64:0/2666033031 >> v1:172.21.15.123:6789/0 conn(0x7f989c0b1be0 legacy=0x7f989c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989bfff700 1 -- 172.21.15.64:0/2666033031 >> v1:172.21.15.64:6789/0 conn(0x7f989c158e40 legacy=0x7f989c067f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989bfff700 1 -- 172.21.15.64:0/2666033031 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0cad90 con 0x7f989c159210 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989bfff700 1 --2- 172.21.15.64:0/2666033031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c159210 0x7f989c0b2c00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098e0 tx=0x7f988c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e2bf2fdb623292 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989b7fe700 1 -- 172.21.15.64:0/2666033031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c009e80 con 0x7f989c159210 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989b7fe700 1 -- 172.21.15.64:0/2666033031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00b860 con 0x7f989c159210 2023-09-27T23:52:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.412+0000 7f989b7fe700 1 -- 172.21.15.64:0/2666033031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00bbe0 con 0x7f989c159210 2023-09-27T23:52:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- 172.21.15.64:0/2666033031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c159210 msgr2=0x7f989c0b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 --2- 172.21.15.64:0/2666033031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c159210 0x7f989c0b2c00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098e0 tx=0x7f988c0091a0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- 172.21.15.64:0/2666033031 shutdown_connections 2023-09-27T23:52:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 --2- 172.21.15.64:0/2666033031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c159210 0x7f989c0b2c00 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- 172.21.15.64:0/2666033031 >> 172.21.15.64:0/2666033031 conn(0x7f989c0d0900 msgr2=0x7f989c0cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- 172.21.15.64:0/2666033031 shutdown_connections 2023-09-27T23:52:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- 172.21.15.64:0/2666033031 wait complete. 2023-09-27T23:52:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 Processor -- start 2023-09-27T23:52:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- start start 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c0b1be0 0x7f989c143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 0x7f989c0c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c159210 0x7f989c0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c140e40 con 0x7f989c158e40 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0c7ea0 con 0x7f989c159210 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f98a1fda700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0c8740 con 0x7f989c0b1be0 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f989bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 0x7f989c0c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f989bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 0x7f989c0c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53828/0 (socket says 172.21.15.64:53828) 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.416+0000 7f989bfff700 1 -- 172.21.15.64:0/128199332 learned_addr learned my addr 172.21.15.64:0/128199332 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f98a0fd8700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c0b1be0 0x7f989c143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f98a17d9700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c159210 0x7f989c0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f989bfff700 1 -- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c0b1be0 msgr2=0x7f989c143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f989bfff700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c0b1be0 0x7f989c143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f989bfff700 1 -- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c159210 msgr2=0x7f989c0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f989bfff700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c159210 0x7f989c0c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f989bfff700 1 -- 172.21.15.64:0/128199332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009420 con 0x7f989c158e40 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f989bfff700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 0x7f989c0c3890 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b3c0 tx=0x7f988c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00b9d0 con 0x7f989c158e40 2023-09-27T23:52:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00bb30 con 0x7f989c158e40 2023-09-27T23:52:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c15c3e0 con 0x7f989c158e40 2023-09-27T23:52:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c025650 con 0x7f989c158e40 2023-09-27T23:52:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c0c83d0 con 0x7f989c158e40 2023-09-27T23:52:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c0c88d0 con 0x7f989c158e40 2023-09-27T23:52:59.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.420+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f988c023070 con 0x7f989c158e40 2023-09-27T23:52:59.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.424+0000 7f9898ff9700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f987c060a70 0x7f987c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.424+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f988c07e770 con 0x7f989c158e40 2023-09-27T23:52:59.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.424+0000 7f98a0fd8700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f987c060a70 0x7f987c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.424+0000 7f98a0fd8700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f987c060a70 0x7f987c062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9884007130 tx=0x7f9884007eb0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.424+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f988c04e810 con 0x7f989c158e40 2023-09-27T23:52:59.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f989c04be40 con 0x7f987c060a70 2023-09-27T23:52:59.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f9898ff9700 1 -- 172.21.15.64:0/128199332 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f989c04be40 con 0x7f987c060a70 2023-09-27T23:52:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f987c060a70 msgr2=0x7f987c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f987c060a70 0x7f987c062f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9884007130 tx=0x7f9884007eb0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 msgr2=0x7f989c0c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 0x7f989c0c3890 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b3c0 tx=0x7f988c017000 comp rx=0 tx=0).stop 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 shutdown_connections 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f987c060a70 0x7f987c062f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c159210 0x7f989c0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989c158e40 0x7f989c0c3890 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 --2- 172.21.15.64:0/128199332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c0b1be0 0x7f989c143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 >> 172.21.15.64:0/128199332 conn(0x7f989c0d0900 msgr2=0x7f989c0b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 shutdown_connections 2023-09-27T23:52:59.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.584+0000 7f98a1fda700 1 -- 172.21.15.64:0/128199332 wait complete. 2023-09-27T23:52:59.593 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2023-09-27T23:52:59.603 INFO:tasks.ceph:Scrubbing osd.7 2023-09-27T23:52:59.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-09-27T23:52:59.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.708+0000 7f4bf3184700 1 Processor -- start 2023-09-27T23:52:59.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.708+0000 7f4bf3184700 1 -- start start 2023-09-27T23:52:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4bec0cb3a0 0x7f4bec15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4bec158b20 0x7f4bec158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec0cea80 con 0x7f4bec15cee0 2023-09-27T23:52:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0b0b40 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0ced60 con 0x7f4bec158b20 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4bec0cb3a0 0x7f4bec15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4bec0cb3a0 0x7f4bec15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53844/0 (socket says 172.21.15.64:53844) 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 -- 172.21.15.64:0/3010428522 learned_addr learned my addr 172.21.15.64:0/3010428522 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf2983700 1 --2- 172.21.15.64:0/3010428522 >> v2:172.21.15.123:3300/0 conn(0x7f4bec158b20 0x7f4bec158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 -- 172.21.15.64:0/3010428522 >> v2:172.21.15.123:3300/0 conn(0x7f4bec158b20 msgr2=0x7f4bec158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 --2- 172.21.15.64:0/3010428522 >> v2:172.21.15.123:3300/0 conn(0x7f4bec158b20 0x7f4bec158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 -- 172.21.15.64:0/3010428522 >> v1:172.21.15.123:6789/0 conn(0x7f4bec15cee0 legacy=0x7f4bec0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 -- 172.21.15.64:0/3010428522 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0cf040 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1981700 1 --2- 172.21.15.64:0/3010428522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec15d2b0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4bec068b60 tx=0x7f4be8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8fb339604295d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1180700 1 -- 172.21.15.64:0/3010428522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800b820 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1180700 1 -- 172.21.15.64:0/3010428522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800b980 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf1180700 1 -- 172.21.15.64:0/3010428522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800bd20 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- 172.21.15.64:0/3010428522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 msgr2=0x7f4bec15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 --2- 172.21.15.64:0/3010428522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec15d2b0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4bec068b60 tx=0x7f4be8009250 comp rx=0 tx=0).stop 2023-09-27T23:52:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- 172.21.15.64:0/3010428522 shutdown_connections 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 --2- 172.21.15.64:0/3010428522 >> v2:172.21.15.123:3300/0 conn(0x7f4bec158b20 0x7f4bec158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 --2- 172.21.15.64:0/3010428522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec15d2b0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- 172.21.15.64:0/3010428522 >> 172.21.15.64:0/3010428522 conn(0x7f4bec0cff20 msgr2=0x7f4bec0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- 172.21.15.64:0/3010428522 shutdown_connections 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.712+0000 7f4bf3184700 1 -- 172.21.15.64:0/3010428522 wait complete. 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 Processor -- start 2023-09-27T23:52:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 -- start start 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bec158b20 0x7f4bec0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bec15cee0 0x7f4bec14a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1591d0 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c8a90 con 0x7f4bec158b20 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c81f0 con 0x7f4bec15cee0 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53854/0 (socket says 172.21.15.64:53854) 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 -- 172.21.15.64:0/3368250564 learned_addr learned my addr 172.21.15.64:0/3368250564 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf1981700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bec158b20 0x7f4bec0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2983700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bec15cee0 0x7f4bec14a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 -- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bec15cee0 msgr2=0x7f4bec14a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bec15cee0 0x7f4bec14a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 -- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bec158b20 msgr2=0x7f4bec0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bec158b20 0x7f4bec0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 -- 172.21.15.64:0/3368250564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80094f0 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf1981700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bec158b20 0x7f4bec0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf2182700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec0c7990 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b5e0 tx=0x7f4bdc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00f510 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00f690 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0c84d0 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00fa50 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.716+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec0c7ed0 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bec04be40 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f4bdc024430 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4be27fc700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4bd0060a70 0x7f4bd0062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f4bdc07d5d0 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4be27fc700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7f4bd0067920 0x7f4bd0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 --> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bd006a420 con 0x7f4bd0067920 2023-09-27T23:52:59.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bdc07d9a0 con 0x7f4bec0cb3a0 2023-09-27T23:52:59.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4bf1981700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4bd0060a70 0x7f4bd0062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4bf2983700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7f4bd0067920 0x7f4bd0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4bf1981700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4bd0060a70 0x7f4bd0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bec068b60 tx=0x7f4be8016000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.720+0000 7f4bf2983700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7f4bd0067920 0x7f4bd0069d50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.724+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== osd.7 v2:172.21.15.123:6816/33569 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4bd006a420 con 0x7f4bd0067920 2023-09-27T23:52:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 --> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4bec0cf320 con 0x7f4bd0067920 2023-09-27T23:52:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4be27fc700 1 -- 172.21.15.64:0/3368250564 <== osd.7 v2:172.21.15.123:6816/33569 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4bec0cf320 con 0x7f4bd0067920 2023-09-27T23:52:59.750 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-27T23:52:59.750 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-27T23:52:59.750 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-27T23:52:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7f4bd0067920 msgr2=0x7f4bd0069d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6816/33569,v1:172.21.15.123:6818/33569] conn(0x7f4bd0067920 0x7f4bd0069d50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4bd0060a70 msgr2=0x7f4bd0062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f4bd0060a70 0x7f4bd0062f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bec068b60 tx=0x7f4be8016000 comp rx=0 tx=0).stop 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 msgr2=0x7f4bec0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 --2- 172.21.15.64:0/3368250564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bec0cb3a0 0x7f4bec0c7990 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b5e0 tx=0x7f4bdc009120 comp rx=0 tx=0).stop 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf2983700 1 -- 172.21.15.64:0/3368250564 reap_dead start 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 shutdown_connections 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 >> 172.21.15.64:0/3368250564 conn(0x7f4bec0cff20 msgr2=0x7f4bec0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 shutdown_connections 2023-09-27T23:52:59.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.744+0000 7f4bf3184700 1 -- 172.21.15.64:0/3368250564 wait complete. 2023-09-27T23:52:59.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-09-27T23:52:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 Processor -- start 2023-09-27T23:52:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 -- start start 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd44159210 0x7fbd440b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbd440b1be0 0x7fbd440b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd440cb3e0 con 0x7fbd44158e40 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd44068960 con 0x7fbd44159210 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.872+0000 7fbd496d0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd440caab0 con 0x7fbd440b1be0 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd44159210 0x7fbd440b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd44159210 0x7fbd440b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53868/0 (socket says 172.21.15.64:53868) 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 -- 172.21.15.64:0/3975581461 learned_addr learned my addr 172.21.15.64:0/3975581461 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd48ecf700 1 --2- 172.21.15.64:0/3975581461 >> v2:172.21.15.123:3300/0 conn(0x7fbd440b1be0 0x7fbd440b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 -- 172.21.15.64:0/3975581461 >> v2:172.21.15.123:3300/0 conn(0x7fbd440b1be0 msgr2=0x7fbd440b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 --2- 172.21.15.64:0/3975581461 >> v2:172.21.15.123:3300/0 conn(0x7fbd440b1be0 0x7fbd440b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 -- 172.21.15.64:0/3975581461 >> v1:172.21.15.123:6789/0 conn(0x7fbd44158e40 legacy=0x7fbd44067f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 -- 172.21.15.64:0/3975581461 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440cad90 con 0x7fbd44159210 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd437fe700 1 --2- 172.21.15.64:0/3975581461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd440b2c00 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fbd440890e0 tx=0x7fbd34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57af3dbe8541b51b server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd42ffd700 1 -- 172.21.15.64:0/3975581461 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34009e70 con 0x7fbd44159210 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd42ffd700 1 -- 172.21.15.64:0/3975581461 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400b860 con 0x7fbd44159210 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd42ffd700 1 -- 172.21.15.64:0/3975581461 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400bc00 con 0x7fbd44159210 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 -- 172.21.15.64:0/3975581461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 msgr2=0x7fbd440b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3975581461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd440b2c00 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fbd440890e0 tx=0x7fbd34009250 comp rx=0 tx=0).stop 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 -- 172.21.15.64:0/3975581461 shutdown_connections 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3975581461 >> v2:172.21.15.123:3300/0 conn(0x7fbd440b1be0 0x7fbd440b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3975581461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd440b2c00 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 -- 172.21.15.64:0/3975581461 >> 172.21.15.64:0/3975581461 conn(0x7fbd440d0900 msgr2=0x7fbd440cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 -- 172.21.15.64:0/3975581461 shutdown_connections 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.876+0000 7fbd496d0700 1 -- 172.21.15.64:0/3975581461 wait complete. 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 Processor -- start 2023-09-27T23:52:59.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 -- start start 2023-09-27T23:52:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd440b1be0 0x7fbd44169d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd44158e40 0x7fbd44152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd44159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4415b0d0 con 0x7fbd44159210 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4415bc50 con 0x7fbd440b1be0 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44153080 con 0x7fbd44158e40 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd48ecf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd44159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd48ecf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd44159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53884/0 (socket says 172.21.15.64:53884) 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd48ecf700 1 -- 172.21.15.64:0/3349494184 learned_addr learned my addr 172.21.15.64:0/3349494184 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd440b1be0 0x7fbd44169d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd437fe700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd44158e40 0x7fbd44152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 -- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd44158e40 msgr2=0x7fbd44152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd44158e40 0x7fbd44152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 -- 172.21.15.64:0/3349494184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 msgr2=0x7fbd44159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:52:59.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd44159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 -- 172.21.15.64:0/3349494184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340094f0 con 0x7fbd440b1be0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd48ecf700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd44159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd43fff700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd440b1be0 0x7fbd44169d20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800c590 tx=0x7fbd38007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011590 con 0x7fbd440b1be0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd38011710 con 0x7fbd440b1be0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4415a510 con 0x7fbd440b1be0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011ad0 con 0x7fbd440b1be0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.880+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4415b600 con 0x7fbd440b1be0 2023-09-27T23:52:59.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.884+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4404be40 con 0x7fbd440b1be0 2023-09-27T23:52:59.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.884+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fbd38033bb0 con 0x7fbd440b1be0 2023-09-27T23:52:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.884+0000 7fbd2bfff700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbd30060a70 0x7fbd30062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:52:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.884+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fbd3807c520 con 0x7fbd440b1be0 2023-09-27T23:52:59.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.884+0000 7fbd437fe700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbd30060a70 0x7fbd30062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:52:59.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.884+0000 7fbd437fe700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbd30060a70 0x7fbd30062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd440890e0 tx=0x7fbd34017000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:52:59.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:52:59.896+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbd3804c520 con 0x7fbd440b1be0 2023-09-27T23:53:00.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.056+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd4415bd90 con 0x7fbd30060a70 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd2bfff700 1 -- 172.21.15.64:0/3349494184 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbd3400b9d0 con 0x7fbd30060a70 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbd30060a70 msgr2=0x7fbd30062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbd30060a70 0x7fbd30062f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd440890e0 tx=0x7fbd34017000 comp rx=0 tx=0).stop 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd440b1be0 msgr2=0x7fbd44169d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd440b1be0 0x7fbd44169d20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800c590 tx=0x7fbd38007ec0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 shutdown_connections 2023-09-27T23:53:00.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fbd30060a70 0x7fbd30062f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd44159210 0x7fbd44159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd44158e40 0x7fbd44152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 --2- 172.21.15.64:0/3349494184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd440b1be0 0x7fbd44169d20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 >> 172.21.15.64:0/3349494184 conn(0x7fbd440d0900 msgr2=0x7fbd440b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 shutdown_connections 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.060+0000 7fbd496d0700 1 -- 172.21.15.64:0/3349494184 wait complete. 2023-09-27T23:53:00.095 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2023-09-27T23:53:00.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:53:00.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 Processor -- start 2023-09-27T23:53:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 -- start start 2023-09-27T23:53:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4c80b0cb0 0x7fe4c806a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b1080 0x7fe4c8069840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c80abd00 con 0x7fe4c80adf70 2023-09-27T23:53:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80abb10 con 0x7fe4c80b0cb0 2023-09-27T23:53:00.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4cdc5e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80abfe0 con 0x7fe4c80b1080 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.200+0000 7fe4ccc5c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4c80b0cb0 0x7fe4c806a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe4c80b0cb0 0x7fe4c806a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53898/0 (socket says 172.21.15.64:53898) 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/3423138888 learned_addr learned my addr 172.21.15.64:0/3423138888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4c7fff700 1 --2- 172.21.15.64:0/3423138888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b1080 0x7fe4c8069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/3423138888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b1080 msgr2=0x7fe4c8069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 --2- 172.21.15.64:0/3423138888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b1080 0x7fe4c8069840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/3423138888 >> v1:172.21.15.123:6789/0 conn(0x7fe4c80adf70 legacy=0x7fe4c815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/3423138888 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80ac2c0 con 0x7fe4c80b0cb0 2023-09-27T23:53:00.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4ccc5c700 1 --2- 172.21.15.64:0/3423138888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80b0cb0 0x7fe4c806a730 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0099b0 tx=0x7fe4bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1faaccff90a28f server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:00.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4c77fe700 1 -- 172.21.15.64:0/3423138888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc009ea0 con 0x7fe4c80b0cb0 2023-09-27T23:53:00.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4c77fe700 1 -- 172.21.15.64:0/3423138888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc00b860 con 0x7fe4c80b0cb0 2023-09-27T23:53:00.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4c77fe700 1 -- 172.21.15.64:0/3423138888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00bc20 con 0x7fe4c80b0cb0 2023-09-27T23:53:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/3423138888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80b0cb0 msgr2=0x7fe4c806a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/3423138888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80b0cb0 0x7fe4c806a730 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0099b0 tx=0x7fe4bc009250 comp rx=0 tx=0).stop 2023-09-27T23:53:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/3423138888 shutdown_connections 2023-09-27T23:53:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/3423138888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b1080 0x7fe4c8069840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/3423138888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80b0cb0 0x7fe4c806a730 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/3423138888 >> 172.21.15.64:0/3423138888 conn(0x7fe4c80cc530 msgr2=0x7fe4c80c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/3423138888 shutdown_connections 2023-09-27T23:53:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/3423138888 wait complete. 2023-09-27T23:53:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 Processor -- start 2023-09-27T23:53:00.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.204+0000 7fe4cdc5e700 1 -- start start 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 0x7fe4c8150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b0cb0 0x7fe4c8155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c80b1080 0x7fe4c8156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8158de0 con 0x7fe4c80adf70 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80c66e0 con 0x7fe4c80b1080 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8150560 con 0x7fe4c80b0cb0 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 0x7fe4c8150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 0x7fe4c8150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53908/0 (socket says 172.21.15.64:53908) 2023-09-27T23:53:00.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/2402544143 learned_addr learned my addr 172.21.15.64:0/2402544143 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4c7fff700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b0cb0 0x7fe4c8155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cd45d700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c80b1080 0x7fe4c8156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b0cb0 msgr2=0x7fe4c8155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b0cb0 0x7fe4c8155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c80b1080 msgr2=0x7fe4c8156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c80b1080 0x7fe4c8156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 -- 172.21.15.64:0/2402544143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc0094f0 con 0x7fe4c80adf70 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4ccc5c700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 0x7fe4c8150020 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006fd0 tx=0x7fe4bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc01f070 con 0x7fe4c80adf70 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc00b9d0 con 0x7fe4c80adf70 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c8158f70 con 0x7fe4c80adf70 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc011420 con 0x7fe4c80adf70 2023-09-27T23:53:00.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c80a9900 con 0x7fe4c80adf70 2023-09-27T23:53:00.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.208+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c804be40 con 0x7fe4c80adf70 2023-09-27T23:53:00.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.212+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7fe4bc011790 con 0x7fe4c80adf70 2023-09-27T23:53:00.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.212+0000 7fe4c4ff9700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe4b4060a70 0x7fe4b4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:00.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.212+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fe4bc07e4b0 con 0x7fe4c80adf70 2023-09-27T23:53:00.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.212+0000 7fe4c7fff700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe4b4060a70 0x7fe4b4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:00.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.212+0000 7fe4c7fff700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe4b4060a70 0x7fe4b4062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4c815b1a0 tx=0x7fe4b80091a0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:00.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.220+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe4bc04e570 con 0x7fe4c80adf70 2023-09-27T23:53:00.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.372+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4c806ac70 con 0x7fe4b4060a70 2023-09-27T23:53:00.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.380+0000 7fe4c4ff9700 1 -- 172.21.15.64:0/2402544143 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71197 (secure 0 0 0) 0x7fe4c806ac70 con 0x7fe4b4060a70 2023-09-27T23:53:00.387 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:53:00.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe4b4060a70 msgr2=0x7fe4b4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe4b4060a70 0x7fe4b4062f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4c815b1a0 tx=0x7fe4b80091a0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 msgr2=0x7fe4c8150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 0x7fe4c8150020 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006fd0 tx=0x7fe4bc017000 comp rx=0 tx=0).stop 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 shutdown_connections 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7fe4b4060a70 0x7fe4b4062f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c80b1080 0x7fe4c8156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4c80b0cb0 0x7fe4c8155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 --2- 172.21.15.64:0/2402544143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4c80adf70 0x7fe4c8150020 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 >> 172.21.15.64:0/2402544143 conn(0x7fe4c80cc530 msgr2=0x7fe4c8157790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 shutdown_connections 2023-09-27T23:53:00.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:00.384+0000 7fe4cdc5e700 1 -- 172.21.15.64:0/2402544143 wait complete. 2023-09-27T23:53:00.391 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:53:00.405 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-09-27T23:53:00.162845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262760,"kb_used_data":20136,"kb_used_omap":0,"kb_used_meta":242560,"kb_avail":754711960,"statfs":{"total":773094113280,"available":772825047040,"internally_reserved":0,"allocated":20619264,"data_stored":11425088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":248381440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003181"},"pg_stats":[{"pgid":"1.0","version":"191'348","reported_seq":1134,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501255+0000","last_change":"2023-09-27T23:13:34.481939+0000","last_active":"2023-09-27T23:52:50.501255+0000","last_peered":"2023-09-27T23:52:50.501255+0000","last_clean":"2023-09-27T23:52:50.501255+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:52:50.501255+0000","last_undegraded":"2023-09-27T23:52:50.501255+0000","last_fullsized":"2023-09-27T23:52:50.501255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:33.291501+0000","last_clean_scrub_stamp":"2023-09-27T23:13:33.291501+0000","objects_scrubbed":0,"log_size":348,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:02:29.066064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.162616+0000","last_change":"2023-09-27T23:13:40.364293+0000","last_active":"2023-09-27T23:52:50.162616+0000","last_peered":"2023-09-27T23:52:50.162616+0000","last_clean":"2023-09-27T23:52:50.162616+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:52:50.162616+0000","last_undegraded":"2023-09-27T23:52:50.162616+0000","last_fullsized":"2023-09-27T23:52:50.162616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.501132+0000","last_change":"2023-09-27T23:13:40.368908+0000","last_active":"2023-09-27T23:52:50.501132+0000","last_peered":"2023-09-27T23:52:50.501132+0000","last_clean":"2023-09-27T23:52:50.501132+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:52:50.501132+0000","last_undegraded":"2023-09-27T23:52:50.501132+0000","last_fullsized":"2023-09-27T23:52:50.501132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:04:37.415242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.498720+0000","last_change":"2023-09-27T23:13:40.447355+0000","last_active":"2023-09-27T23:52:50.498720+0000","last_peered":"2023-09-27T23:52:50.498720+0000","last_clean":"2023-09-27T23:52:50.498720+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:50.498720+0000","last_undegraded":"2023-09-27T23:52:50.498720+0000","last_fullsized":"2023-09-27T23:52:50.498720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:01:58.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":179,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.161617+0000","last_change":"2023-09-27T23:13:40.363743+0000","last_active":"2023-09-27T23:52:50.161617+0000","last_peered":"2023-09-27T23:52:50.161617+0000","last_clean":"2023-09-27T23:52:50.161617+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:52:50.161617+0000","last_undegraded":"2023-09-27T23:52:50.161617+0000","last_fullsized":"2023-09-27T23:52:50.161617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:24:17.282792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.165712+0000","last_change":"2023-09-27T23:13:40.428444+0000","last_active":"2023-09-27T23:52:50.165712+0000","last_peered":"2023-09-27T23:52:50.165712+0000","last_clean":"2023-09-27T23:52:50.165712+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:50.165712+0000","last_undegraded":"2023-09-27T23:52:50.165712+0000","last_fullsized":"2023-09-27T23:52:50.165712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:31:21.428862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.500922+0000","last_change":"2023-09-27T23:13:40.427822+0000","last_active":"2023-09-27T23:52:50.500922+0000","last_peered":"2023-09-27T23:52:50.500922+0000","last_clean":"2023-09-27T23:52:50.500922+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:52:50.500922+0000","last_undegraded":"2023-09-27T23:52:50.500922+0000","last_fullsized":"2023-09-27T23:52:50.500922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T00:55:39.649624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.166245+0000","last_change":"2023-09-27T23:13:40.367847+0000","last_active":"2023-09-27T23:52:50.166245+0000","last_peered":"2023-09-27T23:52:50.166245+0000","last_clean":"2023-09-27T23:52:50.166245+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:50.166245+0000","last_undegraded":"2023-09-27T23:52:50.166245+0000","last_fullsized":"2023-09-27T23:52:50.166245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:38:15.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-09-27T23:52:50.239142+0000","last_change":"2023-09-27T23:13:40.367825+0000","last_active":"2023-09-27T23:52:50.239142+0000","last_peered":"2023-09-27T23:52:50.239142+0000","last_clean":"2023-09-27T23:52:50.239142+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:52:50.239142+0000","last_undegraded":"2023-09-27T23:52:50.239142+0000","last_fullsized":"2023-09-27T23:52:50.239142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:13:39.337878+0000","last_clean_scrub_stamp":"2023-09-27T23:13:39.337878+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:24:53.685726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":34432,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35258368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4810000000000001,"5min":1.569,"15min":1.569},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94999999999999996,"15min":0.93100000000000005},"min":{"1min":0.58199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.042}]},{"osd":1,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.159,"5min":1.3100000000000001,"15min":1.3360000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96899999999999997,"15min":0.97399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2949999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.135}]},{"osd":2,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.66600000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4079999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97199999999999998,"15min":0.96899999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.1870000000000001,"5min":1.341,"15min":1.341},"last":0.63700000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2549999999999999,"5min":1.359,"15min":1.359},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.78100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2629999999999999,"5min":1.371,"15min":1.3879999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.3919999999999999,"5min":1.821,"15min":1.821},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.619,"5min":3.02,"15min":3.02},"last":0.93300000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0629999999999999,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3759999999999999,"5min":1.484,"15min":1.484},"last":1.071},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.103,"15min":1.113},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.173}]},{"osd":6,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.127,"15min":1.1419999999999999},"min":{"1min":0.80200000000000005,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.147,"15min":1.1479999999999999},"min":{"1min":0.81699999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5149999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.86599999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30412,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28032,"kb_avail":94341428,"statfs":{"total":96636764160,"available":96605622272,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28704768},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":1.3149999999999999,"5min":1.792,"15min":1.792},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.538,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.157,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.123}]},{"osd":2,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.629,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.494,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1919999999999999,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":1.2969999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.405,"5min":1.645,"15min":1.645},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.94999999999999996},"min":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.3340000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.872}]},{"osd":4,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.88500000000000001},"min":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.5720000000000001,"5min":1.7,"15min":1.7},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.6399999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.95899999999999996,"15min":0.95999999999999996},"min":{"1min":0.73299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.401,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.242},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1599999999999999,"15min":1.1459999999999999},"min":{"1min":0.89100000000000001,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.71,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1699999999999999,"15min":1.1279999999999999},"min":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.74299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.5840000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":37888,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38797312},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.89700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.246,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0629999999999999,"15min":0.95899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":0.93899999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96099999999999997,"15min":0.93000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2929999999999999,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92600000000000005,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.159,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.755}]},{"osd":2,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"max":{"1min":1.2,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1200000000000001,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.875}]},{"osd":3,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":0.98999999999999999},"min":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.2470000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.997,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.747,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.1970000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.99299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":0.88800000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.107,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.02},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.222,"5min":1.45,"15min":1.4510000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2529999999999999,"5min":4.157,"15min":4.157},"last":1.097},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.341,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.077}]},{"osd":7,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.208,"15min":1.171},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.54700000000000004},"max":{"1min":1.4830000000000001,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.3420000000000001,"15min":1.2589999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.52,"5min":14.119,"15min":14.119},"last":1.155}]}]},{"osd":4,"up_from":15,"seq":64424509908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3660000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.2769999999999999,"5min":1.893,"15min":2.2770000000000001},"last":0.73299999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.3779999999999999,"5min":1.746,"15min":1.746},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95699999999999996,"15min":0.95599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.339,"5min":1.758,"15min":2.1110000000000002},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.95699999999999996,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2649999999999999,"5min":1.659,"15min":1.659},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98799999999999999,"15min":0.96199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.2789999999999999,"5min":1.966,"15min":2.056},"last":0.94399999999999995}]},{"osd":3,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.331,"5min":1.998,"15min":2.032},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.03,"5min":1.079,"15min":1.0469999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58199999999999996},"max":{"1min":1.5,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.2749999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45500000000000002},"max":{"1min":1.4610000000000001,"5min":1.556,"15min":1.9930000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.419,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.046,"15min":1.0529999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.181,"15min":1.163},"min":{"1min":0.749,"5min":0.749,"15min":0.495},"max":{"1min":1.554,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.153}]},{"osd":7,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.091,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66100000000000003},"max":{"1min":1.6279999999999999,"5min":2.069,"15min":2.069},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.149,"15min":1.1719999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.1990000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.294,"15min":1.377},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.76600000000000001,"5min":0.61799999999999999,"15min":0.58999999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.97399999999999998}]},{"osd":2,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.872,"15min":0.86899999999999999},"min":{"1min":0.68200000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2410000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.71699999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.093}]},{"osd":4,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98799999999999999,"15min":0.98699999999999999},"min":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.621},"max":{"1min":1.464,"5min":1.464,"15min":1.599},"last":1.018},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.089,"15min":1.069},"min":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.68700000000000006},"max":{"1min":1.4770000000000001,"5min":1.587,"15min":1.587},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.117,"15min":1.103},"min":{"1min":0.82499999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.627,"5min":1.627,"15min":1.6990000000000001},"last":1.361},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1950000000000001,"15min":1.181},"min":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.72699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6499999999999999,"15min":1.659},"last":1.2789999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.135,"15min":1.1279999999999999},"min":{"1min":0.90800000000000003,"5min":0.873,"15min":0.77800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.659},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2370000000000001,"15min":1.228},"min":{"1min":0.98699999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5780000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.2589999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2709999999999999,"15min":1.25},"min":{"1min":1.0209999999999999,"5min":0.93500000000000005,"15min":0.85299999999999998},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.024,"5min":0.84599999999999997,"15min":0.73599999999999999},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.3919999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30860,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.238,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54800000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0509999999999999,"5min":1.2250000000000001,"15min":1.345},"last":1.1299999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.014,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.25,"5min":1.383,"15min":1.383},"last":0.98399999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.89400000000000002,"15min":0.93300000000000005},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.288}]},{"osd":4,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":1.0740000000000001},"min":{"1min":0.60899999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3280000000000001,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0740000000000001,"15min":1.069},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4330000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.256}]},{"osd":5,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.04,"15min":1.044},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.349,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":1.048},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1379999999999999,"15min":1.1659999999999999},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4399999999999999,"5min":1.5860000000000001,"15min":1.8},"last":1.1930000000000001}]},{"osd":6,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.151,"15min":1.157},"min":{"1min":0.85199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4810000000000001,"5min":1.5780000000000001,"15min":1.718},"last":1.79},{"interface":"front","average":{"1min":1.101,"5min":1.167,"15min":1.1779999999999999},"min":{"1min":0.82399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3879999999999999,"5min":1.6439999999999999,"15min":1.736},"last":1.228}]},{"osd":7,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.516,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3089999999999999,"5min":1.607,"15min":1.8129999999999999},"last":1.3280000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542614,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28940,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94342900,"statfs":{"total":96636764160,"available":96607129600,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99299999999999999,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0229999999999999,"5min":1.3500000000000001,"15min":1.361},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.71199999999999997,"15min":0.73899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.046,"5min":1.2509999999999999,"15min":1.383},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1679999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93899999999999995,"15min":0.94499999999999995},"min":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.83399999999999996},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.377},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.73899999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.96899999999999997,"15min":0.97999999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.109,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.135,"15min":1.133},"min":{"1min":0.78500000000000003,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.454,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.008}]},{"osd":5,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.089,"15min":1.1000000000000001},"min":{"1min":0.83799999999999997,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.7110000000000001,"5min":1.831,"15min":1.831},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.0940000000000001,"15min":1.101},"min":{"1min":0.71899999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2569999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0509999999999999}]},{"osd":6,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.232,"15min":1.2270000000000001},"min":{"1min":0.96899999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.544,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.246},{"interface":"front","average":{"1min":1.198,"5min":1.206,"15min":1.2170000000000001},"min":{"1min":0.90600000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.8160000000000001,"5min":1.913,"15min":1.913},"last":1.2789999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.228,"15min":1.25},"min":{"1min":0.82399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.4950000000000001,"5min":2,"15min":2},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.2150000000000001,"15min":1.2210000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.82699999999999996},"max":{"1min":1.304,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.3089999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509909,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":29312,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30015488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83699999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1950000000000001,"5min":1.198,"15min":1.2470000000000001},"last":1.038}]},{"osd":2,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0960000000000001,"5min":1.169,"15min":1.2509999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.125,"5min":1.288,"15min":1.863},"last":0.88500000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89400000000000002,"15min":0.90100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.4390000000000001},"last":0.53000000000000003}]},{"osd":4,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.681},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.0449999999999999,"15min":1.0569999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3360000000000001,"5min":1.516,"15min":1.5249999999999999},"last":0.91300000000000003}]},{"osd":5,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.095,"15min":1.1080000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3700000000000001,"5min":1.444,"15min":1.512},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5629999999999999,"15min":1.736},"last":0.85399999999999998}]},{"osd":6,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.173,"15min":1.1850000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.746},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5600000000000001,"5min":1.722,"15min":2.0510000000000002},"last":0.94499999999999995}]},{"osd":7,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.248,"15min":1.27},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.544,"5min":1.786,"15min":2.0030000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2330000000000001,"15min":1.252},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.706,"5min":1.806,"15min":1.806},"last":1.016}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:53:00.409 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-27T23:13:33.291501+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=33, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.409 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.409 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.409 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.410 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.410 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.410 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.410 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.410 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-27T23:13:39.337878+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=13, tm_sec=39, tm_wday=2, tm_yday=270, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=27, tm_hour=23, tm_min=52, tm_sec=55, tm_wday=2, tm_yday=270, tm_isdst=0) 2023-09-27T23:53:00.410 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-27T23:53:20.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-27T23:53:20.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.519+0000 7efd88677700 1 Processor -- start 2023-09-27T23:53:20.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.519+0000 7efd88677700 1 -- start start 2023-09-27T23:53:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd88677700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd800cc2f0 0x7efd800af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd88677700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd800b1dd0 0x7efd800b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd88677700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd800afc30 con 0x7efd8015cbd0 2023-09-27T23:53:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd88677700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800b1310 con 0x7efd800b1dd0 2023-09-27T23:53:20.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd88677700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd80156fe0 con 0x7efd800cc2f0 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd800b1dd0 0x7efd800b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd800b1dd0 0x7efd800b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48018/0 (socket says 172.21.15.64:48018) 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 -- 172.21.15.64:0/97338614 learned_addr learned my addr 172.21.15.64:0/97338614 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd86413700 1 --2- 172.21.15.64:0/97338614 >> v2:172.21.15.123:3300/0 conn(0x7efd800cc2f0 0x7efd800af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 -- 172.21.15.64:0/97338614 >> v2:172.21.15.123:3300/0 conn(0x7efd800cc2f0 msgr2=0x7efd800af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 --2- 172.21.15.64:0/97338614 >> v2:172.21.15.123:3300/0 conn(0x7efd800cc2f0 0x7efd800af6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 -- 172.21.15.64:0/97338614 >> v1:172.21.15.123:6789/0 conn(0x7efd8015cbd0 legacy=0x7efd8015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 -- 172.21.15.64:0/97338614 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80067060 con 0x7efd800b1dd0 2023-09-27T23:53:20.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85c12700 1 --2- 172.21.15.64:0/97338614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd800b2250 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7efd800cad50 tx=0x7efd7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccdcf6c24a4e740 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:20.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85411700 1 -- 172.21.15.64:0/97338614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c009e70 con 0x7efd800b1dd0 2023-09-27T23:53:20.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85411700 1 -- 172.21.15.64:0/97338614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c00b860 con 0x7efd800b1dd0 2023-09-27T23:53:20.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.523+0000 7efd85411700 1 -- 172.21.15.64:0/97338614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00bc00 con 0x7efd800b1dd0 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 -- 172.21.15.64:0/97338614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 msgr2=0x7efd800b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 --2- 172.21.15.64:0/97338614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd800b2250 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7efd800cad50 tx=0x7efd7c009250 comp rx=0 tx=0).stop 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 -- 172.21.15.64:0/97338614 shutdown_connections 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 --2- 172.21.15.64:0/97338614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd800b2250 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 --2- 172.21.15.64:0/97338614 >> v2:172.21.15.123:3300/0 conn(0x7efd800cc2f0 0x7efd800af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 -- 172.21.15.64:0/97338614 >> 172.21.15.64:0/97338614 conn(0x7efd800d0a70 msgr2=0x7efd800cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 -- 172.21.15.64:0/97338614 shutdown_connections 2023-09-27T23:53:20.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 -- 172.21.15.64:0/97338614 wait complete. 2023-09-27T23:53:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 Processor -- start 2023-09-27T23:53:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.527+0000 7efd88677700 1 -- start start 2023-09-27T23:53:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd80156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd800cc2f0 0x7efd80169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd8015cbd0 0x7efd8015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd80169fb0 con 0x7efd800b1dd0 2023-09-27T23:53:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8015e930 con 0x7efd8015cbd0 2023-09-27T23:53:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8015d180 con 0x7efd800cc2f0 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd80156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd80156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48030/0 (socket says 172.21.15.64:48030) 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 -- 172.21.15.64:0/509887245 learned_addr learned my addr 172.21.15.64:0/509887245 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86c14700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd8015cbd0 0x7efd8015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd85c12700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd800cc2f0 0x7efd80169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 -- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd800cc2f0 msgr2=0x7efd80169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd800cc2f0 0x7efd80169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 -- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd8015cbd0 msgr2=0x7efd8015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd8015cbd0 0x7efd8015db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 -- 172.21.15.64:0/509887245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0094f0 con 0x7efd800b1dd0 2023-09-27T23:53:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd86413700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd80156580 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7efd7000ec80 tx=0x7efd700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000c850 con 0x7efd800b1dd0 2023-09-27T23:53:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7000c9d0 con 0x7efd800b1dd0 2023-09-27T23:53:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8015f7d0 con 0x7efd800b1dd0 2023-09-27T23:53:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000cd90 con 0x7efd800b1dd0 2023-09-27T23:53:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8015d700 con 0x7efd800b1dd0 2023-09-27T23:53:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd80071110 con 0x7efd800b1dd0 2023-09-27T23:53:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.531+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7efd70013680 con 0x7efd800b1dd0 2023-09-27T23:53:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.535+0000 7efd767fc700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd64060a70 0x7efd64062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.535+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7efd7007d300 con 0x7efd800b1dd0 2023-09-27T23:53:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.535+0000 7efd85c12700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd64060a70 0x7efd64062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.535+0000 7efd85c12700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd64060a70 0x7efd64062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd800cad50 tx=0x7efd7c017000 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.539+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efd7004d3a0 con 0x7efd800b1dd0 2023-09-27T23:53:20.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.699+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 --> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd800afe70 con 0x7efd64060a70 2023-09-27T23:53:20.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.707+0000 7efd767fc700 1 -- 172.21.15.64:0/509887245 <== mgr.9026 v2:172.21.15.123:6832/43188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71149 (secure 0 0 0) 0x7efd800afe70 con 0x7efd64060a70 2023-09-27T23:53:20.711 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:53:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd64060a70 msgr2=0x7efd64062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd64060a70 0x7efd64062f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd800cad50 tx=0x7efd7c017000 comp rx=0 tx=0).stop 2023-09-27T23:53:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 msgr2=0x7efd80156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd80156580 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7efd7000ec80 tx=0x7efd700091a0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 shutdown_connections 2023-09-27T23:53:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7efd64060a70 0x7efd64062f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd8015cbd0 0x7efd8015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd800cc2f0 0x7efd80169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 --2- 172.21.15.64:0/509887245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd800b1dd0 0x7efd80156580 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 >> 172.21.15.64:0/509887245 conn(0x7efd800d0a70 msgr2=0x7efd800cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 shutdown_connections 2023-09-27T23:53:20.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.711+0000 7efd88677700 1 -- 172.21.15.64:0/509887245 wait complete. 2023-09-27T23:53:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-27T23:53:20.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-27T23:53:20.168664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262824,"kb_used_data":20136,"kb_used_omap":0,"kb_used_meta":242624,"kb_avail":754711896,"statfs":{"total":773094113280,"available":772824981504,"internally_reserved":0,"allocated":20619264,"data_stored":11425088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":248446976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003550"},"pg_stats":[{"pgid":"1.0","version":"194'349","reported_seq":1146,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:53:05.722964+0000","last_change":"2023-09-27T23:53:05.722887+0000","last_active":"2023-09-27T23:53:05.722964+0000","last_peered":"2023-09-27T23:53:05.722964+0000","last_clean":"2023-09-27T23:53:05.722964+0000","last_became_active":"2023-09-27T23:13:34.479331+0000","last_became_peered":"2023-09-27T23:13:34.479331+0000","last_unstale":"2023-09-27T23:53:05.722964+0000","last_undegraded":"2023-09-27T23:53:05.722964+0000","last_fullsized":"2023-09-27T23:53:05.722964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'349","last_scrub_stamp":"2023-09-27T23:53:05.722674+0000","last_deep_scrub":"194'349","last_deep_scrub_stamp":"2023-09-27T23:53:05.722674+0000","last_clean_scrub_stamp":"2023-09-27T23:53:05.722674+0000","objects_scrubbed":2,"log_size":349,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T09:34:27.093017+0000","scrub_duration":0.012150457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":190,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:53:01.012051+0000","last_change":"2023-09-27T23:53:01.011976+0000","last_active":"2023-09-27T23:53:01.012051+0000","last_peered":"2023-09-27T23:53:01.012051+0000","last_clean":"2023-09-27T23:53:01.012051+0000","last_became_active":"2023-09-27T23:13:40.364066+0000","last_became_peered":"2023-09-27T23:13:40.364066+0000","last_unstale":"2023-09-27T23:53:01.012051+0000","last_undegraded":"2023-09-27T23:53:01.012051+0000","last_fullsized":"2023-09-27T23:53:01.012051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:53:01.011732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:53:01.011732+0000","last_clean_scrub_stamp":"2023-09-27T23:53:01.011732+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T05:14:02.404108+0000","scrub_duration":0.0067414049999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:53:09.682533+0000","last_change":"2023-09-27T23:53:09.682418+0000","last_active":"2023-09-27T23:53:09.682533+0000","last_peered":"2023-09-27T23:53:09.682533+0000","last_clean":"2023-09-27T23:53:09.682533+0000","last_became_active":"2023-09-27T23:13:40.368560+0000","last_became_peered":"2023-09-27T23:13:40.368560+0000","last_unstale":"2023-09-27T23:53:09.682533+0000","last_undegraded":"2023-09-27T23:53:09.682533+0000","last_fullsized":"2023-09-27T23:53:09.682533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:53:09.682132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:53:09.682132+0000","last_clean_scrub_stamp":"2023-09-27T23:53:09.682132+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T03:40:51.577880+0000","scrub_duration":0.0056655280000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":172,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:52:58.779535+0000","last_change":"2023-09-27T23:52:58.779508+0000","last_active":"2023-09-27T23:52:58.779535+0000","last_peered":"2023-09-27T23:52:58.779535+0000","last_clean":"2023-09-27T23:52:58.779535+0000","last_became_active":"2023-09-27T23:13:40.446757+0000","last_became_peered":"2023-09-27T23:13:40.446757+0000","last_unstale":"2023-09-27T23:52:58.779535+0000","last_undegraded":"2023-09-27T23:52:58.779535+0000","last_fullsized":"2023-09-27T23:52:58.779535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:58.779419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:58.779419+0000","last_clean_scrub_stamp":"2023-09-27T23:52:58.779419+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T07:44:28.465516+0000","scrub_duration":0.0038994070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":190,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:53:02.047131+0000","last_change":"2023-09-27T23:53:02.047013+0000","last_active":"2023-09-27T23:53:02.047131+0000","last_peered":"2023-09-27T23:53:02.047131+0000","last_clean":"2023-09-27T23:53:02.047131+0000","last_became_active":"2023-09-27T23:13:40.363239+0000","last_became_peered":"2023-09-27T23:13:40.363239+0000","last_unstale":"2023-09-27T23:53:02.047131+0000","last_undegraded":"2023-09-27T23:53:02.047131+0000","last_fullsized":"2023-09-27T23:53:02.047131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:53:02.046713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:53:02.046713+0000","last_clean_scrub_stamp":"2023-09-27T23:53:02.046713+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T04:44:38.113117+0000","scrub_duration":0.0058890990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:52:59.004232+0000","last_change":"2023-09-27T23:52:59.004153+0000","last_active":"2023-09-27T23:52:59.004232+0000","last_peered":"2023-09-27T23:52:59.004232+0000","last_clean":"2023-09-27T23:52:59.004232+0000","last_became_active":"2023-09-27T23:13:40.427887+0000","last_became_peered":"2023-09-27T23:13:40.427887+0000","last_unstale":"2023-09-27T23:52:59.004232+0000","last_undegraded":"2023-09-27T23:52:59.004232+0000","last_fullsized":"2023-09-27T23:52:59.004232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:59.003872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:59.003872+0000","last_clean_scrub_stamp":"2023-09-27T23:52:59.003872+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T08:36:21.593042+0000","scrub_duration":0.0058983530000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:53:08.730204+0000","last_change":"2023-09-27T23:53:08.730097+0000","last_active":"2023-09-27T23:53:08.730204+0000","last_peered":"2023-09-27T23:53:08.730204+0000","last_clean":"2023-09-27T23:53:08.730204+0000","last_became_active":"2023-09-27T23:13:40.427388+0000","last_became_peered":"2023-09-27T23:13:40.427388+0000","last_unstale":"2023-09-27T23:53:08.730204+0000","last_undegraded":"2023-09-27T23:53:08.730204+0000","last_fullsized":"2023-09-27T23:53:08.730204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:53:08.729829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:53:08.729829+0000","last_clean_scrub_stamp":"2023-09-27T23:53:08.729829+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T11:38:21.865096+0000","scrub_duration":0.0065871419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":182,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:52:58.003772+0000","last_change":"2023-09-27T23:52:58.003727+0000","last_active":"2023-09-27T23:52:58.003772+0000","last_peered":"2023-09-27T23:52:58.003772+0000","last_clean":"2023-09-27T23:52:58.003772+0000","last_became_active":"2023-09-27T23:13:40.367127+0000","last_became_peered":"2023-09-27T23:13:40.367127+0000","last_unstale":"2023-09-27T23:52:58.003772+0000","last_undegraded":"2023-09-27T23:52:58.003772+0000","last_fullsized":"2023-09-27T23:52:58.003772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:52:58.003618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:52:58.003618+0000","last_clean_scrub_stamp":"2023-09-27T23:52:58.003618+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T02:21:35.284685+0000","scrub_duration":0.0049894290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":169,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-09-27T23:53:07.307906+0000","last_change":"2023-09-27T23:53:07.307842+0000","last_active":"2023-09-27T23:53:07.307906+0000","last_peered":"2023-09-27T23:53:07.307906+0000","last_clean":"2023-09-27T23:53:07.307906+0000","last_became_active":"2023-09-27T23:13:40.367374+0000","last_became_peered":"2023-09-27T23:13:40.367374+0000","last_unstale":"2023-09-27T23:53:07.307906+0000","last_undegraded":"2023-09-27T23:53:07.307906+0000","last_fullsized":"2023-09-27T23:53:07.307906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-27T23:53:07.307604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-27T23:53:07.307604+0000","last_clean_scrub_stamp":"2023-09-27T23:53:07.307604+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-29T06:45:19.660370+0000","scrub_duration":0.0065713630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":34432,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35258368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4810000000000001,"5min":1.569,"15min":1.569},"last":0.995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94999999999999996,"15min":0.93100000000000005},"min":{"1min":0.58199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.159,"5min":1.3100000000000001,"15min":1.3360000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96899999999999997,"15min":0.97399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2949999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.66600000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4079999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97199999999999998,"15min":0.96899999999999997},"min":{"1min":0.70899999999999996,"5min":0.68100000000000005,"15min":0.628},"max":{"1min":1.1870000000000001,"5min":1.341,"15min":1.341},"last":0.871}]},{"osd":3,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2549999999999999,"5min":1.359,"15min":1.359},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.78100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.2629999999999999,"5min":1.371,"15min":1.3879999999999999},"last":1.0820000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.53300000000000003},"max":{"1min":1.3919999999999999,"5min":1.821,"15min":1.821},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.622},"max":{"1min":1.619,"5min":3.02,"15min":3.02},"last":1.206}]},{"osd":5,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0629999999999999,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3759999999999999,"5min":1.484,"15min":1.484},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.103,"15min":1.113},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Wed Sep 27 23:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.127,"15min":1.1419999999999999},"min":{"1min":0.80200000000000005,"5min":0.748,"15min":0.72199999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.151},{"interface":"front","average":{"1min":1.145,"5min":1.147,"15min":1.1479999999999999},"min":{"1min":0.81699999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5149999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.127}]}]},{"osd":6,"up_from":15,"seq":64424509911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30412,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28032,"kb_avail":94341428,"statfs":{"total":96636764160,"available":96605622272,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28704768},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":1.3149999999999999,"5min":1.792,"15min":1.792},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.538,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.89800000000000002,"15min":0.89300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.157,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.82199999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.629,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.494,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.96399999999999997,"15min":0.94999999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1919999999999999,"5min":1.3660000000000001,"15min":1.3720000000000001},"last":1.1659999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.405,"5min":1.645,"15min":1.645},"last":0.874},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.94999999999999996},"min":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"max":{"1min":1.3340000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91100000000000003,"15min":0.88500000000000001},"min":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.41599999999999998},"max":{"1min":1.5720000000000001,"5min":1.7,"15min":1.7},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.95899999999999996,"15min":0.95999999999999996},"min":{"1min":0.73299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.401,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1599999999999999,"15min":1.1459999999999999},"min":{"1min":0.89100000000000001,"5min":0.70399999999999996,"15min":0.53100000000000003},"max":{"1min":1.71,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed Sep 27 23:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1699999999999999,"15min":1.1279999999999999},"min":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.627},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.1599999999999999,"15min":1.137},"min":{"1min":0.74299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.02}]}]},{"osd":5,"up_from":15,"seq":64424509911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40268,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":37888,"kb_avail":94331572,"statfs":{"total":96636764160,"available":96595529728,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38797312},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.94499999999999995,"15min":0.89700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.246,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0629999999999999,"15min":0.95899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":13.757999999999999,"15min":13.757999999999999},"last":1.0169999999999999}]},{"osd":1,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96099999999999997,"15min":0.93000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2929999999999999,"5min":4.1970000000000001,"15min":4.1970000000000001},"last":1.153},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92600000000000005,"15min":0.91400000000000003},"min":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58299999999999996},"max":{"1min":1.159,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.88300000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"max":{"1min":1.2,"5min":1.2709999999999999,"15min":1.4690000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1200000000000001,"5min":4.0999999999999996,"15min":4.0999999999999996},"last":0.91600000000000004}]},{"osd":3,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":0.98999999999999999},"min":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.58899999999999997},"max":{"1min":1.2470000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.755},{"interface":"front","average":{"1min":0.997,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.747,"5min":0.68799999999999994,"15min":0.65300000000000002},"max":{"1min":1.1970000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.2050000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":0.88800000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.107,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.222,"5min":1.45,"15min":1.4510000000000001},"last":1.127}]},{"osd":6,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2529999999999999,"5min":4.157,"15min":4.157},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.341,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.208,"15min":1.171},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.54700000000000004},"max":{"1min":1.4830000000000001,"5min":6.3710000000000004,"15min":6.3710000000000004},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.3420000000000001,"15min":1.2589999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.52,"5min":14.119,"15min":14.119},"last":1.252}]}]},{"osd":4,"up_from":15,"seq":64424509912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3660000000000001,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":1.2769999999999999,"5min":1.893,"15min":2.2770000000000001},"last":0.94799999999999995}]},{"osd":1,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.3779999999999999,"5min":1.746,"15min":1.746},"last":1.161},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95699999999999996,"15min":0.95599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.339,"5min":1.758,"15min":2.1110000000000002},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.95699999999999996,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2649999999999999,"5min":1.659,"15min":1.659},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98799999999999999,"15min":0.96199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.2789999999999999,"5min":1.966,"15min":2.056},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.331,"5min":1.998,"15min":2.032},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.03,"5min":1.079,"15min":1.0469999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.58199999999999996},"max":{"1min":1.5,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.2150000000000001}]},{"osd":5,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45500000000000002},"max":{"1min":1.4610000000000001,"5min":1.556,"15min":1.9930000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.0129999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.419,"5min":1.9339999999999999,"15min":2.0910000000000002},"last":1.002}]},{"osd":6,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.046,"15min":1.0529999999999999},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1.26},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.181,"15min":1.163},"min":{"1min":0.749,"5min":0.749,"15min":0.495},"max":{"1min":1.554,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.107}]},{"osd":7,"last update":"Wed Sep 27 23:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.091,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66100000000000003},"max":{"1min":1.6279999999999999,"5min":2.069,"15min":2.069},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.149,"15min":1.1719999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.3100000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31308,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28928,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29622272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98099999999999998,"5min":1.2509999999999999,"15min":1.3560000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93799999999999994,"5min":1.294,"15min":1.377},"last":0.79200000000000004}]},{"osd":1,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.004,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.89600000000000002,"15min":0.89300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.337,"5min":1.72,"15min":1.72},"last":1.0760000000000001}]},{"osd":2,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85499999999999998,"15min":0.86599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.95599999999999996,"15min":0.94499999999999995},"min":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2789999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.91700000000000004}]},{"osd":4,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":0.98499999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.363,"5min":1.464,"15min":1.599},"last":1.018},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0740000000000001,"15min":1.0640000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.464,"5min":1.587,"15min":1.587},"last":0.95399999999999996}]},{"osd":5,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.113,"15min":1.101},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.569,"5min":1.627,"15min":1.6990000000000001},"last":1.151},{"interface":"front","average":{"1min":1.119,"5min":1.1739999999999999,"15min":1.179},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4259999999999999,"5min":1.6499999999999999,"15min":1.659},"last":1.232}]},{"osd":6,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.111,"15min":1.1200000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.522,"5min":1.6299999999999999,"15min":1.659},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.214,"15min":1.228},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.591,"5min":1.694,"15min":1.8100000000000001},"last":1.4039999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2470000000000001,"15min":1.2490000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7250000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.2130000000000001,"15min":1.2230000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.73599999999999999},"max":{"1min":1.595,"5min":1.849,"15min":1.849},"last":1.3200000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30860,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":28480,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29163520},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48699999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.238,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54800000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0509999999999999,"5min":1.2250000000000001,"15min":1.345},"last":0.752}]},{"osd":1,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.014,"5min":1.1899999999999999,"15min":1.3959999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84199999999999997,"15min":0.84399999999999997},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.25,"5min":1.383,"15min":1.383},"last":1.0109999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.89400000000000002,"15min":0.93300000000000005},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.048,"15min":1.0740000000000001},"min":{"1min":0.60899999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3280000000000001,"5min":1.4350000000000001,"15min":1.5349999999999999},"last":1.091},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0740000000000001,"15min":1.069},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4330000000000001,"5min":1.645,"15min":1.8009999999999999},"last":1.125}]},{"osd":5,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.04,"15min":1.044},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.349,"5min":1.6200000000000001,"15min":1.6779999999999999},"last":1.204},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1379999999999999,"15min":1.1659999999999999},"min":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4399999999999999,"5min":1.5860000000000001,"15min":1.8},"last":1.274}]},{"osd":6,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.151,"15min":1.157},"min":{"1min":0.85199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4810000000000001,"5min":1.5780000000000001,"15min":1.718},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.167,"15min":1.1779999999999999},"min":{"1min":0.82399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3879999999999999,"5min":1.6439999999999999,"15min":1.736},"last":1.052}]},{"osd":7,"last update":"Wed Sep 27 23:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.516,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.329},{"interface":"front","average":{"1min":1.083,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3089999999999999,"5min":1.607,"15min":1.8129999999999999},"last":1.302}]}]},{"osd":1,"up_from":14,"seq":60129542618,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28940,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":26560,"kb_avail":94342900,"statfs":{"total":96636764160,"available":96607129600,"internally_reserved":0,"allocated":2428928,"data_stored":1280544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27197440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99299999999999999,"5min":1.1659999999999999,"15min":1.2070000000000001},"last":1.117},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0229999999999999,"5min":1.3500000000000001,"15min":1.361},"last":1.0209999999999999}]},{"osd":2,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.71199999999999997,"15min":0.73899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.046,"5min":1.2509999999999999,"15min":1.383},"last":0.749},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1679999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93899999999999995,"15min":0.94499999999999995},"min":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.83399999999999996},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.377},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.96899999999999997,"15min":0.97999999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.109,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.135,"15min":1.133},"min":{"1min":0.78500000000000003,"5min":0.752,"15min":0.71999999999999997},"max":{"1min":1.454,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.98699999999999999}]},{"osd":5,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.089,"15min":1.1000000000000001},"min":{"1min":0.83799999999999997,"5min":0.68300000000000005,"15min":0.621},"max":{"1min":1.7110000000000001,"5min":1.831,"15min":1.831},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.04,"5min":1.0940000000000001,"15min":1.101},"min":{"1min":0.71899999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2569999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.155}]},{"osd":6,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.232,"15min":1.2270000000000001},"min":{"1min":0.96899999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.544,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.22},{"interface":"front","average":{"1min":1.198,"5min":1.206,"15min":1.2170000000000001},"min":{"1min":0.90600000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.8160000000000001,"5min":1.913,"15min":1.913},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Sep 27 23:52:53 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.228,"15min":1.25},"min":{"1min":0.82399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.4950000000000001,"5min":2,"15min":2},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.2150000000000001,"15min":1.2210000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.82699999999999996},"max":{"1min":1.304,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.278}]}]},{"osd":0,"up_from":15,"seq":64424509913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":29376,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":3022848,"data_stored":1870912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":30081024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83699999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1950000000000001,"5min":1.198,"15min":1.2470000000000001},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0960000000000001,"5min":1.169,"15min":1.2509999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.125,"5min":1.288,"15min":1.863},"last":1.0649999999999999}]},{"osd":3,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.86899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89400000000000002,"15min":0.90100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.4390000000000001},"last":0.99299999999999999}]},{"osd":4,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0269999999999999,"15min":1.0409999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.681},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0449999999999999,"15min":1.0569999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3360000000000001,"5min":1.516,"15min":1.5249999999999999},"last":0.92800000000000005}]},{"osd":5,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.095,"15min":1.1080000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3700000000000001,"5min":1.444,"15min":1.512},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.363,"5min":1.5629999999999999,"15min":1.736},"last":1.278}]},{"osd":6,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.173,"15min":1.1850000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.587,"5min":1.6499999999999999,"15min":1.746},"last":1.218},{"interface":"front","average":{"1min":1.181,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5600000000000001,"5min":1.722,"15min":2.0510000000000002},"last":1.3160000000000001}]},{"osd":7,"last update":"Wed Sep 27 23:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.248,"15min":1.27},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.544,"5min":1.786,"15min":2.0030000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2330000000000001,"15min":1.252},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.706,"5min":1.806,"15min":1.806},"last":1.365}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-27T23:53:20.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-09-27T23:53:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 Processor -- start 2023-09-27T23:53:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 -- start start 2023-09-27T23:53:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83200b23d0 0x7f8320067a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f832015bca0 0x7f832015c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200caf70 con 0x7f832015cc40 2023-09-27T23:53:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8320068350 con 0x7f832015bca0 2023-09-27T23:53:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83251fb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8320068630 con 0x7f83200b23d0 2023-09-27T23:53:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f832015cc40 0x7f832015d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:41618/0 (socket says 172.21.15.64:41618) 2023-09-27T23:53:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831ffff700 1 -- 172.21.15.64:0/2418319861 learned_addr learned my addr 172.21.15.64:0/2418319861 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-09-27T23:53:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83249fa700 1 --2- 172.21.15.64:0/2418319861 >> v2:172.21.15.64:3300/0 conn(0x7f832015bca0 0x7f832015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831effd700 1 -- 172.21.15.64:0/2418319861 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795262679 0 0) 0x7f83200caf70 con 0x7f832015cc40 2023-09-27T23:53:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831effd700 1 -- 172.21.15.64:0/2418319861 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8308003570 con 0x7f832015cc40 2023-09-27T23:53:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831f7fe700 1 --2- 172.21.15.64:0/2418319861 >> v2:172.21.15.123:3300/0 conn(0x7f83200b23d0 0x7f8320067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83249fa700 1 -- 172.21.15.64:0/2418319861 >> v2:172.21.15.123:3300/0 conn(0x7f83200b23d0 msgr2=0x7f8320067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:53:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83249fa700 1 --2- 172.21.15.64:0/2418319861 >> v2:172.21.15.123:3300/0 conn(0x7f83200b23d0 0x7f8320067a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83249fa700 1 -- 172.21.15.64:0/2418319861 >> v1:172.21.15.64:6789/0 conn(0x7f832015cc40 legacy=0x7f832015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83249fa700 1 -- 172.21.15.64:0/2418319861 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320068910 con 0x7f832015bca0 2023-09-27T23:53:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f83249fa700 1 --2- 172.21.15.64:0/2418319861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015bca0 0x7f832015c100 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f83200ca340 tx=0x7f8318009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8536cb498c062c server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831effd700 1 -- 172.21.15.64:0/2418319861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800e070 con 0x7f832015bca0 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831effd700 1 -- 172.21.15.64:0/2418319861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831800bb60 con 0x7f832015bca0 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.843+0000 7f831effd700 1 -- 172.21.15.64:0/2418319861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318012430 con 0x7f832015bca0 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 -- 172.21.15.64:0/2418319861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015bca0 msgr2=0x7f832015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 --2- 172.21.15.64:0/2418319861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015bca0 0x7f832015c100 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f83200ca340 tx=0x7f8318009780 comp rx=0 tx=0).stop 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 -- 172.21.15.64:0/2418319861 shutdown_connections 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 --2- 172.21.15.64:0/2418319861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015bca0 0x7f832015c100 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 --2- 172.21.15.64:0/2418319861 >> v2:172.21.15.123:3300/0 conn(0x7f83200b23d0 0x7f8320067a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 -- 172.21.15.64:0/2418319861 >> 172.21.15.64:0/2418319861 conn(0x7f8320069b90 msgr2=0x7f83200cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 -- 172.21.15.64:0/2418319861 shutdown_connections 2023-09-27T23:53:20.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 -- 172.21.15.64:0/2418319861 wait complete. 2023-09-27T23:53:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 Processor -- start 2023-09-27T23:53:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.847+0000 7f83251fb700 1 -- start start 2023-09-27T23:53:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200b23d0 0x7f832014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f832015bca0 0x7f83200c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 0x7f832015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200c34f0 con 0x7f832015cc40 2023-09-27T23:53:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200c37d0 con 0x7f83200b23d0 2023-09-27T23:53:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83200c3ab0 con 0x7f832015bca0 2023-09-27T23:53:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 0x7f832015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 0x7f832015af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48044/0 (socket says 172.21.15.64:48044) 2023-09-27T23:53:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 -- 172.21.15.64:0/2850457372 learned_addr learned my addr 172.21.15.64:0/2850457372 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f831ffff700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200b23d0 0x7f832014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f831f7fe700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f832015bca0 0x7f83200c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 -- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f832015bca0 msgr2=0x7f83200c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f832015bca0 0x7f83200c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 -- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200b23d0 msgr2=0x7f832014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200b23d0 0x7f832014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 -- 172.21.15.64:0/2850457372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318009a20 con 0x7f832015cc40 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f831ffff700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200b23d0 0x7f832014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83249fa700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 0x7f832015af30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f8318009fd0 tx=0x7f8318019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318012e80 con 0x7f832015cc40 2023-09-27T23:53:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8318021070 con 0x7f832015cc40 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83200c3d90 con 0x7f832015cc40 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318026660 con 0x7f832015cc40 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83200c4350 con 0x7f832015cc40 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83200c4570 con 0x7f832015cc40 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.851+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f8318023030 con 0x7f832015cc40 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.855+0000 7f8307fff700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f830c060a70 0x7f830c062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:20.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.855+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f831807f200 con 0x7f832015cc40 2023-09-27T23:53:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.855+0000 7f831ffff700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f830c060a70 0x7f830c062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.855+0000 7f831ffff700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f830c060a70 0x7f830c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83100058d0 tx=0x7f8310009040 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:20.859+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f831804f350 con 0x7f832015cc40 2023-09-27T23:53:21.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.023+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f832004be40 con 0x7f832015cc40 2023-09-27T23:53:21.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.031+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f831804f350 con 0x7f832015cc40 2023-09-27T23:53:21.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.031+0000 7f8307fff700 1 -- 172.21.15.64:0/2850457372 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v70)=0 v70) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8318052970 con 0x7f832015cc40 2023-09-27T23:53:21.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f830c060a70 msgr2=0x7f830c062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f830c060a70 0x7f830c062f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83100058d0 tx=0x7f8310009040 comp rx=0 tx=0).stop 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 msgr2=0x7f832015af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 0x7f832015af30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f8318009fd0 tx=0x7f8318019000 comp rx=0 tx=0).stop 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 shutdown_connections 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f830c060a70 0x7f830c062f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f832015cc40 0x7f832015af30 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f832015bca0 0x7f83200c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 --2- 172.21.15.64:0/2850457372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200b23d0 0x7f832014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 >> 172.21.15.64:0/2850457372 conn(0x7f8320069b90 msgr2=0x7f832015a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:21.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 shutdown_connections 2023-09-27T23:53:21.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.035+0000 7f83251fb700 1 -- 172.21.15.64:0/2850457372 wait complete. 2023-09-27T23:53:21.052 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-27T23:53:21.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-27T23:53:21.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 Processor -- start 2023-09-27T23:53:21.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 -- start start 2023-09-27T23:53:21.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c80c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53c8159040 0x7f53c80c7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53c8158ac0 0x7f53c8069840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c8068670 con 0x7f53c8158ac0 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80c6690 con 0x7f53c8159040 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53cce0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c6970 con 0x7f53c81586f0 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53c7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53c8158ac0 0x7f53c8069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53c7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53c8158ac0 0x7f53c8069840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48056/0 (socket says 172.21.15.64:48056) 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53c7fff700 1 -- 172.21.15.64:0/2463488233 learned_addr learned my addr 172.21.15.64:0/2463488233 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53c6ffd700 1 --2- 172.21.15.64:0/2463488233 >> v2:172.21.15.123:3300/0 conn(0x7f53c8159040 0x7f53c80c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.159+0000 7f53c77fe700 1 --2- 172.21.15.64:0/2463488233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c80c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c7fff700 1 -- 172.21.15.64:0/2463488233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 msgr2=0x7f53c80c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:53:21.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c7fff700 1 --2- 172.21.15.64:0/2463488233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c80c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c7fff700 1 -- 172.21.15.64:0/2463488233 >> v2:172.21.15.123:3300/0 conn(0x7f53c8159040 msgr2=0x7f53c80c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-27T23:53:21.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c7fff700 1 --2- 172.21.15.64:0/2463488233 >> v2:172.21.15.123:3300/0 conn(0x7f53c8159040 0x7f53c80c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c7fff700 1 -- 172.21.15.64:0/2463488233 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80c6c50 con 0x7f53c8158ac0 2023-09-27T23:53:21.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c7fff700 1 --2- 172.21.15.64:0/2463488233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8158ac0 0x7f53c8069840 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f53c806cdd0 tx=0x7f53c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2d056c2ced278 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:21.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c67fc700 1 -- 172.21.15.64:0/2463488233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000b640 con 0x7f53c8158ac0 2023-09-27T23:53:21.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c67fc700 1 -- 172.21.15.64:0/2463488233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f53c000b7a0 con 0x7f53c8158ac0 2023-09-27T23:53:21.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53c67fc700 1 -- 172.21.15.64:0/2463488233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000bb80 con 0x7f53c8158ac0 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 -- 172.21.15.64:0/2463488233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8158ac0 msgr2=0x7f53c8069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 --2- 172.21.15.64:0/2463488233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8158ac0 0x7f53c8069840 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f53c806cdd0 tx=0x7f53c0009250 comp rx=0 tx=0).stop 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 -- 172.21.15.64:0/2463488233 shutdown_connections 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 --2- 172.21.15.64:0/2463488233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8158ac0 0x7f53c8069840 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 --2- 172.21.15.64:0/2463488233 >> v2:172.21.15.123:3300/0 conn(0x7f53c8159040 0x7f53c80c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 --2- 172.21.15.64:0/2463488233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c80c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 -- 172.21.15.64:0/2463488233 >> 172.21.15.64:0/2463488233 conn(0x7f53c80cc530 msgr2=0x7f53c80c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 -- 172.21.15.64:0/2463488233 shutdown_connections 2023-09-27T23:53:21.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.163+0000 7f53cce0f700 1 -- 172.21.15.64:0/2463488233 wait complete. 2023-09-27T23:53:21.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 Processor -- start 2023-09-27T23:53:21.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 -- start start 2023-09-27T23:53:21.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c8140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53c8158ac0 0x7f53c80ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 0x7f53c80aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c3020 con 0x7f53c8159040 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80af2b0 con 0x7f53c8158ac0 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53cce0f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8141040 con 0x7f53c81586f0 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 0x7f53c80aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 0x7f53c80aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48070/0 (socket says 172.21.15.64:48070) 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53c7fff700 1 -- 172.21.15.64:0/1472225152 learned_addr learned my addr 172.21.15.64:0/1472225152 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53c77fe700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c8140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.167+0000 7f53c6ffd700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53c8158ac0 0x7f53c80ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53c7fff700 1 -- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 msgr2=0x7f53c8140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53c7fff700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c8140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53c7fff700 1 -- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53c8158ac0 msgr2=0x7f53c80ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53c7fff700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53c8158ac0 0x7f53c80ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53c7fff700 1 -- 172.21.15.64:0/1472225152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00094f0 con 0x7f53c8159040 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53c7fff700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 0x7f53c80aed70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f53c00094c0 tx=0x7f53c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000b640 con 0x7f53c8159040 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f53c001f070 con 0x7f53c8159040 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00266f0 con 0x7f53c8159040 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c80ca510 con 0x7f53c8159040 2023-09-27T23:53:21.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80caad0 con 0x7f53c8159040 2023-09-27T23:53:21.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c80c3160 con 0x7f53c8159040 2023-09-27T23:53:21.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.171+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 97) v1 ==== 80361+0+0 (secure 0 0 0) 0x7f53c001a430 con 0x7f53c8159040 2023-09-27T23:53:21.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.175+0000 7f53af7fe700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f53b4060a70 0x7f53b4062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-27T23:53:21.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.175+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f53c0021070 con 0x7f53c8159040 2023-09-27T23:53:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.175+0000 7f53c77fe700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f53b4060a70 0x7f53b4062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-27T23:53:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.175+0000 7f53c77fe700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f53b4060a70 0x7f53b4062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009d10 tx=0x7f53bc0091a0 comp rx=0 tx=0).ready entity=mgr.9026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-27T23:53:21.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.183+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f53c80c3160 con 0x7f53c8159040 2023-09-27T23:53:21.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.375+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53c80c3160 con 0x7f53c8159040 2023-09-27T23:53:21.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53af7fe700 1 -- 172.21.15.64:0/1472225152 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f53c0015de0 con 0x7f53c8159040 2023-09-27T23:53:21.380 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-27T23:53:21.381 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2975099738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2975099738},{"type":"v1","addr":"172.21.15.64:6833","nonce":2975099738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1711277765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1711277765},{"type":"v1","addr":"172.21.15.64:6835","nonce":1711277765}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2895760357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2895760357},{"type":"v1","addr":"172.21.15.123:6837","nonce":2895760357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":9125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2056892613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2056892613},{"type":"v1","addr":"172.21.15.123:6835","nonce":2056892613}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[]} 2023-09-27T23:53:21.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f53b4060a70 msgr2=0x7f53b4062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f53b4060a70 0x7f53b4062f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009d10 tx=0x7f53bc0091a0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 msgr2=0x7f53c80aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-27T23:53:21.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 0x7f53c80aed70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f53c00094c0 tx=0x7f53c0009250 comp rx=0 tx=0).stop 2023-09-27T23:53:21.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 shutdown_connections 2023-09-27T23:53:21.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:6832/43188,v1:172.21.15.123:6833/43188] conn(0x7f53b4060a70 0x7f53b4062f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53c8159040 0x7f53c80aed70 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53c8158ac0 0x7f53c80ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 --2- 172.21.15.64:0/1472225152 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53c81586f0 0x7f53c8140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-27T23:53:21.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 >> 172.21.15.64:0/1472225152 conn(0x7f53c80cc530 msgr2=0x7f53c8154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-27T23:53:21.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 shutdown_connections 2023-09-27T23:53:21.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-27T23:53:21.379+0000 7f53cce0f700 1 -- 172.21.15.64:0/1472225152 wait complete. 2023-09-27T23:53:21.385 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 74 2023-09-27T23:53:21.398 DEBUG:tasks.cephfs.filesystem:already dead... 2023-09-27T23:53:21.398 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-27T23:53:21.398 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-27T23:53:21.399 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.411 INFO:tasks.ceph.mds.a:Stopped 2023-09-27T23:53:21.412 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-27T23:53:21.412 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.429 INFO:tasks.ceph.mds.c:Stopped 2023-09-27T23:53:21.430 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-27T23:53:21.430 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.445 INFO:tasks.ceph.mds.b:Stopped 2023-09-27T23:53:21.445 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-27T23:53:21.445 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.507 INFO:tasks.ceph.mds.d:Stopped 2023-09-27T23:53:21.507 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-27T23:53:21.507 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-27T23:53:21.507 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.564 INFO:tasks.ceph.osd.0:Stopped 2023-09-27T23:53:21.564 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-27T23:53:21.564 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.642 INFO:tasks.ceph.osd.1:Stopped 2023-09-27T23:53:21.642 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-27T23:53:21.642 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.713 INFO:tasks.ceph.osd.2:Stopped 2023-09-27T23:53:21.713 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-27T23:53:21.713 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.790 INFO:tasks.ceph.osd.3:Stopped 2023-09-27T23:53:21.791 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-09-27T23:53:21.791 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.877 INFO:tasks.ceph.osd.4:Stopped 2023-09-27T23:53:21.878 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-09-27T23:53:21.878 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:21.956 INFO:tasks.ceph.osd.5:Stopped 2023-09-27T23:53:21.956 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-09-27T23:53:21.956 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.030 INFO:tasks.ceph.osd.6:Stopped 2023-09-27T23:53:22.030 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-09-27T23:53:22.030 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.100 INFO:tasks.ceph.osd.7:Stopped 2023-09-27T23:53:22.100 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-27T23:53:22.101 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-27T23:53:22.101 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.131 INFO:tasks.ceph.mgr.y:Stopped 2023-09-27T23:53:22.131 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-27T23:53:22.131 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.164 INFO:tasks.ceph.mgr.x:Stopped 2023-09-27T23:53:22.164 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-27T23:53:22.165 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-27T23:53:22.165 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.189 INFO:tasks.ceph.mon.a:Stopped 2023-09-27T23:53:22.189 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-27T23:53:22.189 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.209 INFO:tasks.ceph.mon.b:Stopped 2023-09-27T23:53:22.209 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-27T23:53:22.209 INFO:teuthology.orchestra.run:waiting for 300 2023-09-27T23:53:22.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-09-27T23:53:22.278 INFO:tasks.ceph.mon.c:Stopped 2023-09-27T23:53:22.278 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-27T23:53:22.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-09-27T23:53:22.302 INFO:teuthology.orchestra.run.smithi064.stdout:1695856562.9658217 mon.a (mon.0) 372 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-27T23:53:22.302 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-09-27T23:53:22.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-09-27T23:53:22.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-09-27T23:53:22.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-09-27T23:53:22.390 INFO:teuthology.orchestra.run.smithi064.stdout:1695856562.9658217 mon.a (mon.0) 372 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-27T23:53:22.391 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:53:22.392 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-27T23:53:22.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:53:22.696 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-27T23:53:22.944 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:53:22.944 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-27T23:53:23.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:53:23.070 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-27T23:53:23.142 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:53:23.142 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-09-27T23:53:23.415 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:53:23.415 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-09-27T23:53:23.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:53:23.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:53:23.682 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:53:23.683 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-09-27T23:53:23.894 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:53:23.894 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-09-27T23:53:24.067 INFO:tasks.ceph:Archiving mon data... 2023-09-27T23:53:24.068 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/data/mon.a.tgz 2023-09-27T23:53:24.068 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-09-27T23:53:24.075 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.XndLpjf4Fc 2023-09-27T23:53:24.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.XndLpjf4Fc 2023-09-27T23:53:24.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.XndLpjf4Fc 2023-09-27T23:53:24.519 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.XndLpjf4Fc is 2MB 2023-09-27T23:53:24.619 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.XndLpjf4Fc 2023-09-27T23:53:24.626 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/data/mon.b.tgz 2023-09-27T23:53:24.626 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-09-27T23:53:24.633 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.vBwKlEnU1r 2023-09-27T23:53:24.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vBwKlEnU1r 2023-09-27T23:53:25.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.vBwKlEnU1r 2023-09-27T23:53:25.045 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.vBwKlEnU1r is 3MB 2023-09-27T23:53:25.140 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.vBwKlEnU1r 2023-09-27T23:53:25.148 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/data/mon.c.tgz 2023-09-27T23:53:25.148 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-09-27T23:53:25.153 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.oezMmCJfuB 2023-09-27T23:53:25.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.oezMmCJfuB 2023-09-27T23:53:25.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.oezMmCJfuB 2023-09-27T23:53:25.613 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.oezMmCJfuB is 3MB 2023-09-27T23:53:25.707 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.oezMmCJfuB 2023-09-27T23:53:25.714 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-27T23:53:25.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-27T23:53:25.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-27T23:53:25.731 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-27T23:53:25.731 DEBUG:teuthology.orchestra.run.smithi064:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-27T23:53:25.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-27T23:53:25.800 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-27T23:53:25.820 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-27T23:53:25.827 INFO:tasks.ceph:Archiving crash dumps... 2023-09-27T23:53:25.829 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/remote/smithi064/crash 2023-09-27T23:53:25.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-27T23:53:25.847 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/remote/smithi123/crash 2023-09-27T23:53:25.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-27T23:53:25.862 INFO:tasks.ceph:Shutting down logrotate 2023-09-27T23:53:25.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:53:25.865 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-27T23:53:25.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-27T23:53:25.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-27T23:53:25.979 INFO:tasks.ceph:Compressing logs... 2023-09-27T23:53:25.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-27T23:53:25.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-27T23:53:27.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-09-27T23:53:27.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-09-27T23:53:27.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-09-27T23:53:27.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-09-27T23:53:27.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-09-27T23:53:27.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-09-27T23:53:27.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-09-27T23:53:43.553 INFO:tasks.ceph:Archiving logs... 2023-09-27T23:53:43.553 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/remote/smithi064/log 2023-09-27T23:53:43.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-27T23:53:46.314 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/remote/smithi123/log 2023-09-27T23:53:46.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-27T23:53:48.123 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-27T23:53:48.132 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-27T23:53:48.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-27T23:53:48.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-27T23:53:48.207 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-27T23:53:48.216 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-27T23:53:48.217 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-27T23:53:48.219 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-27T23:53:48.242 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2023-09-27T23:53:48.242 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2023-09-27T23:53:48.242 INFO:teuthology.orchestra.run.smithi064.stdout:*hv01.front.sepi 44.4.53.2 2 u 127 128 377 0.129 1.822 0.411 2023-09-27T23:53:48.242 INFO:teuthology.orchestra.run.smithi064.stdout:+hv02.front.sepi 63.231.80.2 3 u 117 128 377 0.111 1.911 0.113 2023-09-27T23:53:48.242 INFO:teuthology.orchestra.run.smithi064.stdout:+hv03.front.sepi 74.6.168.72 3 u 104 128 377 0.114 -1.595 0.269 2023-09-27T23:53:48.242 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2023-09-27T23:53:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: remote refid st t when poll reach delay offset jitter 2023-09-27T23:53:48.243 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================== 2023-09-27T23:53:48.243 INFO:teuthology.orchestra.run.smithi123.stdout:*hv01.front.sepi 44.4.53.2 2 u 102 128 377 0.123 -0.348 0.125 2023-09-27T23:53:48.243 INFO:teuthology.orchestra.run.smithi123.stdout:+hv02.front.sepi 63.231.80.2 3 u 49 128 377 0.104 0.040 0.145 2023-09-27T23:53:48.243 INFO:teuthology.orchestra.run.smithi123.stdout:+hv03.front.sepi 74.6.168.72 3 u 45 128 377 0.118 -3.688 0.152 2023-09-27T23:53:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2023-09-27T23:53:48.244 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-27T23:53:48.253 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-27T23:53:48.254 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-27T23:53:48.262 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-27T23:53:48.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-27T23:53:48.280 INFO:teuthology.task.internal:Duration was 3033.225533 seconds 2023-09-27T23:53:48.280 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-27T23:53:48.289 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-27T23:53:48.290 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-27T23:53:48.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-27T23:53:48.346 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-27T23:53:48.347 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-09-27T23:53:48.347 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-27T23:53:48.358 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2023-09-27T23:53:48.358 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-27T23:53:48.370 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-27T23:53:48.370 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-27T23:53:48.372 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-27T23:53:48.410 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-27T23:53:48.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-27T23:53:48.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-27T23:53:48.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-27T23:53:48.553 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-27T23:53:48.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-27T23:53:48.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-27T23:53:48.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-27T23:53:48.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-27T23:53:48.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-27T23:53:48.609 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2023-09-27T23:53:48.611 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-09-27T23:53:48.628 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-27T23:53:48.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:53:48.677 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-27T23:53:48.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-27T23:53:48.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-27T23:53:48.693 INFO:teuthology.task.internal:Transferring archived files... 2023-09-27T23:53:48.694 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/remote/smithi064 2023-09-27T23:53:48.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-27T23:53:48.720 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461/remote/smithi123 2023-09-27T23:53:48.720 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-27T23:53:48.746 INFO:teuthology.task.internal:Removing archive directory... 2023-09-27T23:53:48.746 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-27T23:53:48.761 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-27T23:53:48.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-27T23:53:48.779 INFO:teuthology.task.internal:Not uploading archives. 2023-09-27T23:53:48.779 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-27T23:53:48.789 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-27T23:53:48.789 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-27T23:53:48.791 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-27T23:53:48.797 INFO:teuthology.orchestra.run.smithi064.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 27 23:53 /home/ubuntu/cephtest 2023-09-27T23:53:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 27 23:53 /home/ubuntu/cephtest 2023-09-27T23:53:48.821 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-27T23:53:48.832 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-27T23:53:48.857 INFO:teuthology.nuke:Checking targets against current locks 2023-09-27T23:53:48.883 DEBUG:teuthology.nuke:shortname: smithi064 2023-09-27T23:53:48.884 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-27T23:53:48.910 DEBUG:teuthology.nuke:shortname: smithi123 2023-09-27T23:53:48.910 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-27T23:53:48.915 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-27 22:53:49.707654', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-27T23:53:48.937 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:23:53-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7403461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-27 22:53:49.708687', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-27T23:53:48.970 INFO:teuthology.orchestra.console.smithi064:Power off 2023-09-27T23:53:48.971 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-27T23:53:48.993 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2023-09-27T23:53:48.993 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:53:49.006 INFO:teuthology.orchestra.console.smithi123:Power off 2023-09-27T23:53:49.006 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-27T23:53:49.017 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-09-27T23:53:49.026 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-09-27T23:53:49.026 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:53:49.047 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-27T23:53:53.018 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:53:53.048 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:53:53.131 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-09-27T23:53:53.158 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-27T23:53:57.136 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:53:57.159 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:53:57.247 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-09-27T23:53:57.267 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-27T23:54:01.250 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:54:01.269 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:54:01.360 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2023-09-27T23:54:01.378 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-09-27T23:54:01.461 INFO:teuthology.orchestra.console.smithi064:Power off completed 2023-09-27T23:54:01.479 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-09-27T23:54:01.583 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 3033.2255330085754 failure_reason: '"1695856562.9658217 mon.a (mon.0) 372 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-09-27T23:54:01.583 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:54:01.667 INFO:teuthology.run:FAIL